#juju-dev 2012-08-20
<fwereade_> davecheney, if you are bored and listless at any time, you might enjoy taking a look at https://codereview.appspot.com/6460110
<fwereade_> davecheney, which has a real live uniter inside :)
<davecheney> fwereade_: will do
<mramm> fwereade_: impressive
<Aram> morning.
<davecheney> howdy
<Aram> so
<Aram> when is the release meeting?
<Aram> in 70 minutes?
<davechen1y> Aram: tomorrow at 9pm AEST I thought
<davechen1y> mramm: is that correct ?
<TheMue> Aram, davechen1y: tomorrow after the regular weekly
<Aram> I see.
<TheMue> Aram: Just accept the mail invitation and your calendar will show it.
<Aram> too much trouble to regenerate a new password or two factor auth or whatever we use now.
 * Aram hates this crap
<TheMue> Aram: It's a simple google account. Once setup and ok. I like it all networked, so I always see all our appointments and my private one in my desktop appl as well as on my phone.
<Aram> I've setup it once, but for some reason it doesn't remember my password so I have to enter it all the time.
<Aram> and because it's a random password I don't remember it.
<TheMue> Aram: That's why I don't have random passwords.
<Aram> yeah, but for some reason I can only use the canonical provided password, couldn't change it.
<TheMue> Aram: The calendar is a google accout and it can be changed, once you switched to this account.
<TheMue> Aram: And random pwds are always hard to remember and don't interest brute force attacs. So I choose long ones with stupid sentences like TheQuickBrownFoxJumpsOverTheLazyDog.
<TheMue> Aram: But only more stupid. ;)
<TheMue> thats strange: when NOT closing a watcher channel, the according test for the closed watcher fails, so far ok. but when I add the closing a test for unexpected changes fails, that's executed BEFORE the watcher is stopped (and the channel closed). *dunno*
<fwereade_> TheMue, sounds like a bug to me :)
<TheMue> fwereade_: yeah, but what kind of bug? it's in the SRW. your added tests like assertNoChange(), which mostly work fine.
<TheMue> fwereade_: funnily the fail happens after the w.Stop(), but the test is befor.
<fwereade_> TheMue, ServiceRelationsWatcher?
<TheMue> oh, have to break for a moment, the technician for our heating is there.
<TheMue> fwereade_: yes
<fwereade_> TheMue, I can take a look if yu give me a pointer to the branch
<fwereade_> TheMue, but I guess what it *sounds* like is some actual error is killing the watcher at an unhelpful time... is that possible
<TheMue> fwereade_: I changed almost nothing in your test, only regarding Kill() and Die(). but the watcher now works differently. i'll paste it.
<TheMue> fwereade_: http://pastebin.ubuntu.com/1157107/ line 56 fails
<TheMue> fwereade_: without closing the change channel line 69 fails
<fwereade_> TheMue, ok, so it's line 56 that gets an unexpectedly closed channel; does that not surely indicate that something has suffered some sort of error caused by the relation removal?
<fwereade_> TheMue, try heavy logging in the watcher to figure out why it dies?
<TheMue> fwereade_: line 56 get's no closed channel, it receives an unexpected change of "nothing"
<fwereade_> TheMue, if the watcher isn't dying I think there's definitely something weird going on
<fwereade_> TheMue, then that just sounds like the watchers is sending something it shouldn't
<TheMue> fwereade_: i already logged around the sending (there's only one point), but it isn't explicity called
<fwereade_> TheMue, ok, for us to receive, either we are sending or we are closing, right?
<fwereade_> TheMue, if you have verified we are not sending, surely we are closing, which implies that something is killing the watcher
<TheMue> fwereade_: sorry, don't understand
<TheMue> fwereade_: ah, now it gets more clear
<fwereade_> TheMue, ok, the problem is that the test receievs an unwanted event on the changes chan, yeah?
<TheMue> fwereade_: then i already have an idea, one moment
<fwereade_> TheMue, defer a log of w.tomb.Err just before you start the loop maybe?
<fwereade_> TheMue, fwiw I suspect something is manipulating a relation after it's removed
<TheMue> fwereade_: one moment please, test is running
<TheMue> fwereade_: hehe, it passes
<TheMue> fwereade_: it has indeed been a wrong handling of a dying relation life watcher. i signaled the SRWs tomb to kill itself
<TheMue> fwereade_: instead only doing so in case of an error
<fwereade_> TheMue, ha, bad luck
<TheMue> fwereade_: so i can propose the new SRW now
<TheMue> fwereade_: it behaves a bit different than you wanted it to do
 * fwereade_ raises an eyebrow
<fwereade_> TheMue, would you hit the high points please?
<TheMue> fwereade_: as i get the events of alive/dying/dead by individual life watchers you don't get them bundled
<fwereade_> TheMue, as long as it's the same event, but it just happens to have a single change at a time, that is just fine, so long as we still get a bundle for the first event
<TheMue> fwereade_: the SRW can't know when the last event will be received, so it just sent what it gets.
<fwereade_> TheMue, I don't mind individual item events in any except the first event
<fwereade_> TheMue, is the first event getting the proper state?
<TheMue> fwereade_: the life watcher sends a first proper event, but individual
<TheMue> fwereade_: so if the watcher starts and 5 relations are active you'll get 5 'alive' events
<fwereade_> TheMue, so when I do <-s.WatchRelations().Changes(), I get something different to when I call s.Relations()?
<fwereade_> TheMue, that can't work, sorry
<TheMue> fwereade_: see line 74ff
<fwereade_> TheMue, I will think about it a bit, but offhand I'm not sure how I can make that work
<fwereade_> TheMue, I would strongly favour something that worked like presence.ChildrenWatcher
<TheMue> fwereade_: the problem would have been the same in the SRW. i would have to store the startup state, ignore the alive events for all those of the startup state and then contine sending those which are not part of the startup
<fwereade_> TheMue, that sounds sensible to me, what's the issue with it?
<TheMue> fwereade_: the SRW is so beautiful clean now, it would make it more complex. :D
<fwereade_> TheMue, it doesn't matter how simple it is if it doesn't do what we need ;p
<TheMue> fwereade_: just define your requirement new *lol*
<fwereade_> TheMue, I will think on how I can work with those events but I'm pretty strongly -1 on implementing just this watcher completely differently from all the others
<TheMue> fwereade_: sorry, it ISN'T completely different, absolutely not
<fwereade_> TheMue, initial event?
<fwereade_> TheMue, that seems like a pretty vast semantic change to me
<TheMue> fwereade_: a change yes, but the word completely is too strong
<fwereade_> TheMue, well, all other watchers send an initial event that represents the state, and then send diffs
<fwereade_> TheMue, this sends a sequence of initial events representing the state, and at someunknowable point starts sending diffs
<TheMue> fwereade_: This one also sends an initial even, only sadly per relation
<fwereade_> TheMue, and how is a client meant to know when they've consumed that initial state?
<TheMue> fwereade_: but i understand your reason and will change it
 * fwereade_ is relieved :)
<fwereade_> TheMue, if you look at RelationUnitsWatcher you should see a similar model in place
<fwereade_> TheMue, rather than what you described earlier, I would suggest: ignore initial state except to start a watcher for each; consume the initial events on each of those watchers, and collate those into the initial event; send the watches over to their own goroutines for further processing
<fwereade_> TheMue, sorry, s/send the watches over to their own goroutines/start a goroutine for each watch/
<fwereade_> TheMue, sane?
<TheMue> fwereade_: that's already running, but the first part sound's good
<TheMue> fwereade_: consuming the initial events.
<fwereade_> TheMue, have a look at RUW -- it's undeniably more complex but I think it still works out pretty clean
<mramm> well, I went back to my room for a nap at 5pm after the pycon sprints before dinner
<mramm> and just woke up with a fever, sore throat, sinus problems, etc
<mramm> I thought I was just tired from the jetlag
 * Aram is too hungry for the time of the day.
<niemeyer> Morning!
<fwereade_> niemeyer, heyhey
<fwereade_> mramm, ouch :(
<mramm> niemeyer: morning
<mramm> fwereade_: yea I hate being sick while traveling
<fwereade_> mramm, it sucks :(
<niemeyer> I personally hate being sick. Period. :-)
<TheMue> niemeyer: morning
<TheMue> fwereade_: another question: think of service s1 having relations r1 and r2, both are alive
<TheMue> fwereade_: i get the SRW of s1, and now wait for the first events of r1 and r2. i get r1 as alive and now wait for r2. but during this time i get a r1 as dying
<TheMue> fwereade_: so after reciving alive from r2 i send r1 as dying and r2 as alive, ok?
<fwereade_> TheMue, if you get a second event on one of the it should be happening on its own goroutine
<TheMue> fwereade_: yes, it will, and the srw collects it
<fwereade_> TheMue, surely it shouldn't collect it until it has finished handling the initial state?
<TheMue> fwereade_: otherwise i would send you the dying as a single event before the inital event is out (while waiting for the last life watcher)
<TheMue> fwereade_: please rephrase
<fwereade_> TheMue, surely you should use an intrnal channel to marshal the subgoroutine events onto the main goroutine
<TheMue> fwereade_: i do so, yes
<fwereade_> TheMue, surely therefore you would not even pick them up for sending until you'd finished the initial event creation on the main goroutine
<TheMue> fwereade_: and here i talk about getting 2 or more events of r1 before r2 (which is also part of the initial state) sent its first life
<fwereade_> TheMue, why is it relevant that you got more than one event? maybe you did, and the sub-goroutine is trying to send it to the main one, but the main one won't pick it up until it hits its main select again
<TheMue> fwereade_: i am in the main select loop
<fwereade_> TheMue, have you looked at what RUW does?
<fwereade_> TheMue, most of what I say is attempts to helpfully translate that into english :)
<fwereade_> TheMue, but, ok, you've just got an initial event that indicates that N relations exist
<TheMue> fwereade_: may i describe how it works now (before i turn everything around)?
<fwereade_> TheMue, surely you go through each of those relations, consuming event and starting goroutine, and send the collected initial state, all before you leave the current case?
<fwereade_> TheMue, please do :)
<fwereade_> TheMue, your knowledge will be better than my guesses :)
<TheMue> fwereade_: when the srw starts it watches the topology, as before, and retrieves the relations of the service. for each relation a lifewatcher in an own goroutine is started.
<fwereade_> TheMue, ah, OK, to get what we need I think you need to start all the watchers on the main goroutine to consume the initial events
<TheMue> fwereade_: the all write their events back to the srw in one channel, where the srw collects then and sends the changes to the receiver of Changes()
<fwereade_> TheMue, only once you have consumed the first event do you start a new goroutine to handle the subsequent ones
<TheMue> fwereade_: yes, that my question, please wait with writing
<fwereade_> TheMue, sorry
<TheMue> fwereade_: now, as you wrote above, i want to consume the first events of those life watchers before sending it combined as the initial state to you
<TheMue> fwereade_: the question is now, that it may happen, that the state of r1 changes before r2 is received.
<fwereade_> TheMue, yes, that can happen if you don't start the life watches on the main goroutine
<TheMue> fwereade_: but because i'm in the same loop i receive it.
<fwereade_> TheMue, right, so... don't do it that way :)
<TheMue> fwereade_: ah, ok, ic, while setting the life watchers i directly wait blocking on their first event
<fwereade_> TheMue, yeah, sorry unclear
<TheMue> fwereade_: eh, sorry, i wanted to say that, when setting up the life watcher for a relation I immediatelly wait for its first event. so after setting up the first set i've got all first life states.
<TheMue> fwereade_: that will be sent then and afterwards the single events of the life watchers
<niemeyer> fwereade_, TheMue: I was following but didn't want to interrupt, but seems like we've reached agreement now
<niemeyer> fwereade_, TheMue: I'm curious about what's the case there, given we'll be doing things slightly differently soon
<fwereade_> niemeyer, sorry, I'm not sure what you're asking
<TheMue> niemeyer: we talked about the service relation watcher, which will handles the lifecycles of realations
<niemeyer> fwereade_: I'm wondering what the underlying issue being debated above is about
<TheMue> niemeyer: and william and i discussed about how to get a propper initial state and then only diffs
<niemeyer> fwereade_: Trying to load mind state since we have to do that stuff in mstate
<fwereade_> niemeyer, just "should a watcher always send initial state as a single event" -- I feel that is part of the definition of a watcher
<TheMue> niemeyer: now it's more clear to me, thx to williams feedback
<niemeyer> fwereade_: Okay
<niemeyer> fwereade_: That's easy, thanks :)
<fwereade_> niemeyer, cool
<TheMue> niemeyer, fwereade_ +1
<TheMue> ;)
<fwereade_> TheMue, cheers :)
<fwereade_> niemeyer, btw, I don't know if you saw, there's a Uniter in review
<niemeyer> fwereade_: Yeah, brilliant
<niemeyer> fwereade_: How're things going there, btw?  Are you happy with it?
<niemeyer> fwereade_: Or with progress, in general
<fwereade_> niemeyer, it's basically the whole unit agent state machine, excluding relations, because basically everything interacts with everything else and looks very different if you try to implement bits without other bits
<fwereade_> niemeyer, honestly I think it's awesome, it does an awful lot in very few lines of code
<niemeyer> fwereade_: Woot!
<fwereade_> niemeyer, and the tests are pretty cool too
<fwereade_> niemeyer, I fully expect to have missed something, but I was feeling pretty damn happy when I proposed it :)
<fwereade_> niemeyer, uniter and modes are each under 250 lines
<niemeyer> fwereade_: Sweeeet
<fwereade_> niemeyer, I'm sad that I didn't et it proposed on friday, but happy that I think relation support will be easy to add
<niemeyer> fwereade_: I get all excited just thinking that we're about to run our first charm
<fwereade_> niemeyer, yeah :D
<fwereade_> niemeyer, and, heh, the name Uniter feels like a surprisingly good joke, it really does pull almost everything together in one place
<niemeyer> fwereade_: Hah.. hadn't thought of it from that perspective
<TheMue> fwereade_: Yes, Sir, Mr Reade, Sir. Initial state is up and running. Thx again. ;)
<fwereade_> TheMue, awesome, tyvm
<niemeyer> Aram: ping
<Aram> pong
<niemeyer> Aram: Yo
<Aram> hi
<niemeyer> Aram: I've just sent you a review on confignode
<niemeyer> Aram: It's definitely awesome
<Aram> looking
<niemeyer> Aram: There are a couple of high-levels we should understand, just in terms of how we'll be porting these concepts
<niemeyer> Aram: The logic itself is very cool
<Aram> 1 min to answer IN the CL.
<niemeyer> Aram: Okay, although I'd like to follow up here if that's fine with you
<Aram> done
<niemeyer> Aram: Okay, given those comments, LGTM on the branch going in as-is
<niemeyer> Aram: We can tweak it further in follow ups
<Aram> thanks :).
<niemeyer> Aram: Re. the path, I'm wondering if we should be using context-specific collections
<Aram> that might be a good kidea.
<Aram> idea
<niemeyer> Aram: On the version, once I finish the txn package (which is almost there, btw), this should become automatic, so it's cool to live as-is for now
<niemeyer> s/live/leave
<Aram> cool.
<niemeyer> Aram: Spent a good while polishing it further over the weekend.. I haven't seen a single error with the current logic so far.. last night I've finally started on the handling of inserts/removes, and I *think* the approach is sane and easy to get right..
<Aram> yeah, I've seen your G+ post. so basically, after insert and delete it should be good to go?
<niemeyer> Aram: It's not yet functional, but I suspect I can do that in a couple of hours tonight
<niemeyer> Aram: Yep
<niemeyer> Hmm.. I have to do some errands today, like banking and expense-reporting
 * Aram needs to buy new backpack, laptop, and some small laptop carying thingy.
<niemeyer> Aram: What's the plan for the new laptop? Anything interesting these days?
<niemeyer> fwereade_: Nice one on the downloader refactoring
<fwereade_> niemeyer, cheers, was rather pleasing
<Aram> meh, I wanted to buy the new ThinkPad X1 carbon, it's a great machine, 14'' screen in 13'' chassis and weight, 1600x900 matte screen etc.
<niemeyer> fwereade_: Isn't it awesome playing with lego when you have the right blocks?  :-)
<Aram> but at home I want to use two external monitors and this new thinkpad only has a stupid windows onlu usb (?!?) dock.
<Aram> no drivers for linux at all.
<fwereade_> niemeyer, yeah, it really is -- this last week or so has been a salutary lesson in the benefits of building bottom-up
<fwereade_> niemeyer, not sure we could have done it like that starting from scratch, but that's maybe just me :)
<Aram> If I didn't need to use two external monitors, I'd buy the new ThinkPad X1 carbon.
<Aram> mainly because of screen.
<niemeyer> Aram: Ouch
<niemeyer> fwereade_: Yeah, we should definitely sit down for a couple of hours over the next sprint evaluating what we did
<fwereade_> niemeyer, postmortems++
<niemeyer> fwereade_: I tend to agree that to do what we're doing right now, we need crystal-clear vision of where we're going
<fwereade_> niemeyer, ...but when we *do*, it's awesome :)
<niemeyer> fwereade_: +1000 x 0.001
<niemeyer> :-)
<niemeyer> fwereade_:  You have a LGTM! That branch is great
<fwereade_> niemeyer, haha :)
<fwereade_> niemeyer, sweet
<fwereade_> niemeyer, thank you :D
<niemeyer> fwereade_: Another review on uniter-support-tweaks
<niemeyer> fwereade_: I'm afraid this one isn't so positive.. please let me know when you want to talk about it
<fwereade_> niemeyer, you're probably right on most of it given a quick scan, and it's definitely worth a chat -- but are you about to have lunch?
<fwereade_> niemeyer, because I am rather hungry myself, but will be back shortly
<niemeyer> fwereade_: Indeed I am.. if there's anything quick you wanna talk before that, I'd be happy to unblock you
<niemeyer> fwereade_: Otherwise I'll be back soon after lunch + bank
<fwereade_> niemeyer, nah, I'm good, just need to work up the mental resolve to stop coding and go get some fod ;)
<niemeyer> fwereade_: Superb, sounds like a plan :)
<niemeyer> robbiew: Sorry, I know you just mentioned it in the meeting, but I'm not sure: do we have the consistency call today?
<robbiew> lol
<robbiew> niemeyer: are you trolling me
<robbiew> no call
<niemeyer> robbiew: Well, that's getting pretty consistent I guess :)
<robbiew> indeed
<fwereade_> niemeyer, ping, I have a bit of pushback on HookContext.CmdGetter in the review
<fwereade_> niemeyer, although honestly I could live without any or all of the things I propose -- they just make a few bits of the Uniter arguably neater
<niemeyer> fwereade_: Agreed, I can also live without any code, although it wouldn't be so fun I believe :-)
<niemeyer> fwereade_: That's not justification to get things in blindly, though.. CmdGetter feels like a pretty awkward piece at first sight
<niemeyer> fwereade_: I'm happy to talk, though.. have been wrong a non-trivial number of times
<fwereade_> niemeyer, I think there's an underlying point we haven't discussed:
<fwereade_> niemeyer, IMO it's simplest right now to start a jujuc server for each hook and stop it when the hook's done
<fwereade_> niemeyer, I know that this approach will probably not live forever, but I don't think it's better *now* to implement a persistent server which chooses contexts, without having any contexts to choose, than it is to defer implementing that until we know exactly what we want
<fwereade_> niemeyer, assuming you're ok with this, then we can debate CmdGetter, although ...actually, I'm happy to concede it's unnecessary
<fwereade_> niemeyer, if you're not ok with the above, we should probably talk more :)
<niemeyer> fwereade_: I don't have a strong feeling either way, but I'd appreciate understanding what it means in practice
<niemeyer> fwereade_: My view is that it has a context to choose, and we just haven't implemented the bit that makes it choose different contexts
<niemeyer> fwereade_: If we already have that neatly in place, I'm curious about why we'd go backwards
<niemeyer> fwereade_: Although, maybe we have the model entirely wrong
<fwereade_> niemeyer, ok, but the actual mechanics of context choice become more complicated if there's ever more than one possibility: that is, if I have a server running all the time, I need to keep track of the current hook context in the Uniter, and to worry about what happens if someone happens to call into it when I'm not expecting it
<niemeyer> fwereade_: Well, that's the very reason why we have a context id, right?
<fwereade_> niemeyer, I'm not saying they're intractable, just that they're not something we need yet when we can just run single-context servers
<niemeyer> fwereade_: Yeah, this is interesting.. let's explore that for a bit
<fwereade_> niemeyer, sure, the concept of a context id has been flowed through from the start because we're confident we'll want to do something interesting with it one day
<niemeyer> fwereade_: We had that conversation in the sprint where we agreed to do something entirely different from what we originally envisioned
<niemeyer> fwereade_: Which can actually affect this in a good way
<fwereade_> niemeyer, cool
<niemeyer> fwereade_: We had that thing with.. how was it.. "juju-context foo.sh" or something?
<fwereade_> niemeyer, that was the suggestion, I think, yes
<niemeyer> fwereade_: Okay.. and IIRC we agreed that it was going to be serialized too
<niemeyer> fwereade_: Or am I on crack?
<fwereade_> niemeyer, although I'm thinking juju-run might actually be nicer
<niemeyer> fwereade_: Yeah, that sounds better
<fwereade_> niemeyer, no, I am entirely comfortable with this idea
<niemeyer> fwereade_: Okay, so.. hmm
<niemeyer> fwereade_: If it's all serialized, you're on the right track.. why do we have multiple contexts
<niemeyer> fwereade_: There's still a role for the context id as originally incepted, though, I think
<fwereade_> niemeyer, validation, primarily, I think -- commands will act slightly differently in a relation context, for example
<niemeyer> fwereade_: Bingo!
<niemeyer> fwereade_: What if we replaced it by a nonce?
<fwereade_> niemeyer, expand please, not sure how that's superior
<niemeyer> fwereade_: What we want is to prevent stuff from randomly accessing the juju server
<niemeyer> fwereade_: When they've reportedly left the context in which they were running
<niemeyer> fwereade_: If I juju-run my-stuff.sh, nothing from my-stuff.sh should be touching the server after juju-run returns
<fwereade_> niemeyer, yes, exactly
<niemeyer> fwereade_: A nonce is a good way to implement the mechanics for that, without separating into multiple contexts on the server side
<niemeyer> fwereade_: When we start to run something, we provide a nonce to the environment, when we stop running it, we reset the nonce
<niemeyer> fwereade_: Any requests that arrive with the wrong nonce are denied
<fwereade_> niemeyer, ok, so, just as we have been doing, but I like this perspective, it doesn't have to be tied to the context
<niemeyer> fwereade_: yeah, pretty much as we've been doing, except we lose the concept of the context id, because the client can't "select the context"
<niemeyer> fwereade_: It's just able or unable to talk to "the one context"
<fwereade_> niemeyer, we can keep the same context forever, and just take a little bit of care with concurrency
<niemeyer> fwereade_: Right
<fwereade_> niemeyer, ok, that sounds awesome
<fwereade_> niemeyer, I have a plan, then: I drop evenything except the ExpandTo change, and rework the tests as you suggest
<fwereade_> niemeyer, then you can take a look at the real-live-uniter branch and decide whether the command running is horrifying enough to demand that fix as a prereq, or whether you can live with it a a followup
<fwereade_> niemeyer, sane?
<niemeyer> fwereade_: +1
<niemeyer> fwereade_: Is CmdGetter there?
<niemeyer> fwereade_: Or, will it remain there?
<fwereade_> niemeyer, nah, I'll write the same tiny little closure inline, unless you're willing to put up with it as a temporary measure
<niemeyer> fwereade_: I think it could go away entirely
<niemeyer> fwereade_: but certainly willing to review for more context
<niemeyer> fwereade_: This closure is doing if a != b { error }
<fwereade_> niemeyer, yeah
<fwereade_> niemeyer, oh, yes, we never did talk properly about charm.Status
<fwereade_> niemeyer, from the questions you ask, I think I maybe got the name wrong
<fwereade_> niemeyer, in a sense there are only 3 statuses -- missing, changing, installed -- but there are multiple reasons to be changing the charm, and I think the Uniter wants to keep track of them somehow
<fwereade_> niemeyer, it seemed sensible to store them all in one type, for the convenience of the package's only client, which does care about those distinctions
<niemeyer> fwereade_: I'm just wondering why
<fwereade_> niemeyer, why the distinction between the two upgrade types?
<fwereade_> niemeyer, because forced ones don't want to run hooks while already in a potential error state
<niemeyer> fwereade_: Why the forced status is necessary
<niemeyer> fwereade_: In the charm upgrading directory
<niemeyer> fwereade_: If we have a charm directory upgrade in progress, it is in progress and we must complete it.. it doesn't matter how we got there
<niemeyer> fwereade_: retry is unrelated to that.. force means "upgrade even if in error"
<fwereade_> niemeyer, hum, I was not involved in the original implementation there, but AIUI it was precisely so that users *could* upgrade hte charm while in the middle of a hook error without otherwise perturbing the unit's state
<fwereade_> niemeyer, ok, and given that the process could die N times in the course of the upgrade, when we come up in the middle of an operation, I think it is useful to know exactly what sort of operation it is
<niemeyer> fwereade_: It's an upgrade operation., ?
<fwereade_> niemeyer, right, but it wants to finish it idfferently depending on whether or not it's forced
<fwereade_> niemeyer, python elevated the forcedness as the most important thing -- ie a forced upgrade would never run hooks
<niemeyer> fwereade_: Exactly.. so why do we need both an "upgradeing" and an "upgrading forced" status in a charm directory?
<fwereade_> niemeyer, because how the upgrade operation is completed differers, depending on that status
<niemeyer> fwereade_: Why?
<fwereade_> niemeyer, I was not privy to the original reasoning, but I would not be surprised if the word "twisted" didn't feature in the answer somewhere
<niemeyer> fwereade_: Okay, so can we drop that status?
<fwereade_> niemeyer, as long as you're happy to make a forced upgrade in started state act exactly like a normal upgrade
<fwereade_> niemeyer, I would be very happy with that
<niemeyer> fwereade_: I don't understand how it's any different, to be honest.. a "forced" upgrade indicates whether or not we're willing to start the upgrade even if in an error status
<fwereade_> niemeyer, and then any upgrade operation will, when it completes, run hooks or not based purely on existing hook error
<niemeyer> fwereade_: After that decision, we're running the upgrade.. it's already in progress and we can't stop it, no matter how we got to start running it
<fwereade_> niemeyer, ok, sure, but when we finish I think we still do not want to run the upgrade-charm hook if another hook has an unresolved error
<niemeyer> fwereade_: Once the upgrade completes, there's a decision to be made regarding running hooks or not, and that's not related to whether we said "force" or not, or is it?
<fwereade_> niemeyer, the only reason it is related is because it is implemented like that in python
<fwereade_> niemeyer, and I doubt anyone is depending heavily on the ability to upgrade charms without hooks in started status
<niemeyer> fwereade_: Okay, I can't see the reasoning, but maybe we'll find out while trying to drop it.. if we don't, even better
<fwereade_> niemeyer, so if you're ok with the behaviour change then I am ++happy to drop it
<niemeyer> fwereade_: Okay, just to be clear, what's the behavior change again?
<fwereade_> niemeyer, whether or not to run an upgrade-charm hook at the end of a forced charm upgrade in started status
<fwereade_> niemeyer, I think the change is a good one
<fwereade_> niemeyer, ie that forcedness should be entirely ignored in started mode
<niemeyer> fwereade_: Huh.. that's what I expected it meant, so I guess the change from whatever was being done is a reasonable one
<fwereade_> niemeyer, this also lets me -wip that branch before you take a look, the install/upgrade will get *much* nicer as a result
<niemeyer> fwereade_: It's like "rm -f foo".. if foo would be removed without -f.. well.. that's fine :)
<fwereade_> niemeyer, yeah, the meaning of force should not have been overloaded in the first place :)
<fwereade_> niemeyer, ok, I think I'm going to rework my branches
<fwereade_> niemeyer, the one we've been discussing will lose UpgradingForced, will get the ExpandTo test fixes, and will also get a reworking of the state unit/service charm API that I ended up needing in the followup
<niemeyer> fwereade_: Cool
<fwereade_> niemeyer, when that's ready I'll repropose and gtw on real-live-uniter with nicer charm changes
<niemeyer> fwereade_: Superb, cheers
<fwereade_> niemeyer, ok, now I'm looking at the state charm stuff on its own, I'm thinking some more
 * niemeyer listens
<fwereade_> niemeyer, do we have any reason to have CharmURL or SetCharmURL at all? I don't think we really do
<niemeyer> fwereade_: Go on?
<fwereade_> well, they're barely used, and they allow for inconsistency that setting and getting actual *state.Charms do not
<niemeyer> fwereade_: I'm not yet following
<niemeyer> fwereade_: I don't want to state the obvious, but they're the way we download charms
<niemeyer> fwereade_: What am I missing?
<fwereade_> niemeyer, ok, SetCharmURL(something-made-up) is possible; SetCharm(some-actual-charm) is not
<fwereade_> niemeyer, ok, that made no sense
<fwereade_> niemeyer, I'll try again: it should not be possible to set a unit's, or a service's, charm URL to a charm that is not in state
<fwereade_> niemeyer, the two ways of accomplishing this are (1) check against state whenever we set a charm url or (2) allow setting of charm urls purely by setting actual *state.Charm values, which should be guaranteed to represent real charms
<fwereade_> niemeyer, I favour (2), it feels cleaner
<niemeyer> fwereade_: I don't understand how that's possible
<niemeyer> fwereade_: Charm URLs are signed so that we can access a private bucket
<fwereade_> niemeyer, no, charm urls -- *charm.URL -- not bundle urls
<fwereade_> niemeyer, units and services identify their charm by charm URL
<niemeyer> fwereade_: Oh, oka
<niemeyer> y
<fwereade_> niemeyer, when we come to download it, we get the bundle url from the state charm itself -- that's fine, I think it's immutable
<niemeyer> fwereade_: So what's the thing on the left-hand side of SetCharmURL again? :-)
 * niemeyer looks at the code
<fwereade_> niemeyer, unit/service
<niemeyer> fwereade_: Okay, sorry, I think I get it now
<niemeyer> fwereade_: Are you suggesting we replace SetCharmURL and CharmURL by SetCharm and Charm?
<fwereade_> niemeyer, yes
<niemeyer> fwereade_: +1
<fwereade_> niemeyer, sweet -- and, while I'm there, I kinda want to change state.Unit.serviceName to state.unit.service -- I think we've always got a service available when we create a unit, and it then means that we can actually get to all the relevant state given just a *Unit
<fwereade_> niemeyer, rather than having to store units and services together in a bunch of places
<fwereade_> niemeyer, well, 2 places, anyway
<fwereade_> niemeyer, (the point is that we can then have (*Unit)Service() *Service
<fwereade_> )
<niemeyer> fwereade_: If it's cleaner, certainly +1
<niemeyer> fwereade_: Well, hmm
<niemeyer> fwereade_: I guess that sounds sane.. I'm just wondering about how that will impact performance and the caching stuff
<fwereade_> niemeyer, only positively, I *hope*
<niemeyer> fwereade_: Yeah, probably
<niemeyer> fwereade_: These changes should take place on mstate at the same time
<niemeyer> fwereade_: I think we're already covering that part of the API there
<niemeyer> Aram?
<fwereade_> niemeyer, I think the Unit methods are missing, but that's just less code to delete :)
<niemeyer> fwereade_: Yeah, and it'll probably be easier to add to mstate than state even, and it'll also give you a nice idea of how it maps
<fwereade_> niemeyer, yeah, I have yet toget myhands dirty in there
<fwereade_> niemeyer, sorry, more things to get agreement on ahead of time, actually
<fwereade_> niemeyer, (1) units should start out without charms set
<fwereade_> niemeyer, (2) the charm state file will actually want a charm URL *and* a status, because I won't be able to trust the charm URL I set in state (what if I'm restarted on a different machine?)
<fwereade_> niemeyer, I think that's it
<niemeyer> fwereade_: Hmm
<niemeyer> fwereade_: (1) seems rather curious
<fwereade_> niemeyer, well, given (2), we could ignore (1), but I guess the question is what exactly a unit's charm URL *means*
<niemeyer> fwereade_: Is there any doubts about that?
<niemeyer> fwereade_: It's the charm the unit is supposed to run?
<fwereade_> niemeyer, if it's the charm the unit *should* have, it puts us in a position of potentially writing 10k nodes in order to upgrade the charm for a big service
<fwereade_> niemeyer, if it's the charm the unit *does* have, IMO it should absolutely start out empty and only be set once the charm is installed
<niemeyer> fwereade_: Yes, assuming we want to upgrade all at once
<fwereade_> niemeyer, well, I guess, we do write those 10k *anyway* in order to actually tell them to upgrade
<niemeyer> fwereade_: Yeah
<fwereade_> niemeyer, if we're telling them what to upgrade to that's only twice as much work :)
<niemeyer> fwereade_: It's also the easy part of the job, assuming we're upgrading 10k units :)
<fwereade_> niemeyer, haha
<niemeyer> fwereade_: Although.. I can easily see us doing better in that area
<niemeyer> fwereade_: The thinking around all of that is to enable partial upgrades
<niemeyer> fwereade_: Which I'm assuming *will* happen.. nobody will be upgrading 10k nodes at once to a new version if they have a real production system
<niemeyer> fwereade_: Well, I should fix that.. nobody *should* do that.. they probably will :)
<fwereade_> niemeyer, ha
<fwereade_> niemeyer, well, ok, this is again a behaviour change, about which I'm a little less certain, because I don't know whether I will be implementing upgrade-charm, and this all needs to tie in nicely there
<fwereade_> niemeyer, ATM python depends on (1) the unit's upgrade flag and (2) the service's charm
<fwereade_> niemeyer, and it uses the unit's charm in state to determine that it doesn't need an upgrade
<niemeyer> fwereade_: I quite like what we did in the agent stuff
<niemeyer> fwereade_: We could mimic it
<fwereade_> niemeyer, that last part is crack anyway, really, it should be using local state to know what's already installed
<fwereade_> niemeyer, that has crossed my mind
<niemeyer> fwereade_: We have proposed and current
<niemeyer> fwereade_: Which means we support both ends
<niemeyer> fwereade_: That doesn't change what you just said, though
<niemeyer> fwereade_: Current is set via local state
<niemeyer> fwereade_: This also means no behavioral change is necessary
<fwereade_> niemeyer, ok, yep, it's not an external behaviour change, wrong term -- what I mean is that it's a change that has ramifications beyond my current narrow focus
<fwereade_> niemeyer, ie in the implementation of juju upgrade-charm itself
<fwereade_> niemeyer, which I will capture in the bug, so np there :)
<fwereade_> niemeyer, so: service just has Charm/SetCharm, and the service's charm controls the proposed-charm that new units are created with
<fwereade_> niemeyer, unit has proposed-charm and current-charm, which starts out empty
<fwereade_> niemeyer, whenever the UA finishes unpacking a charm to disk, it sets current-charm
<niemeyer> fwereade_: +1
<niemeyer> fwereade_: Well, I assume proposed-charm should start out filled
<fwereade_> niemeyer, so unit gets Charm/SetCharm, and ProposedCharm, and maybe something else
<fwereade_> niemeyer, SetProposedCharm and SetNeedsUpgrade sound like they could profitably be combined
<niemeyer> fwereade_: As it doesn't make sense to fire a unit if we don't know what it is
<niemeyer> fwereade_: Agreed
<fwereade_> niemeyer, ok, I'm starting to feel that the ripple effects may be pretty significant, because a diff between current and proposed should be enough information to determine that an upgrade should happen
<niemeyer> fwereade_: Exactly
<niemeyer> fwereade_: Alternatively, we may dump that half-baked idea
<niemeyer> fwereade_: Since we're paying cost for nothing so far
<fwereade_> niemeyer, I therefore feel like we should stick with the existing upgrade flag approach, because that does contain one valuable bit (force) along with the less valuable bit (upgrade)
<niemeyer> fwereade_: And say that proposed-charm lives in the service, while current-charm lives in the unit
<fwereade_> niemeyer, hey, nice
<fwereade_> niemeyer, and that then does mean that the unit starts out empty
<niemeyer> fwereade_: Right
<fwereade_> niemeyer, then we keep the upgrade flag exactly as is
<fwereade_> niemeyer, and then in fact we just keep the existing API, but use Charm instead of CharmURL
<fwereade_> niemeyer, because really unit.CurrentCharm/service.ProposedCharm don't make a huge amount of sense, while .Charm/.Charm can be sensibly documented
<niemeyer> fwereade_: I think NeedsUpgrade can go
<niemeyer> fwereade_: Agreed
<fwereade_> niemeyer, just the API method? yeah, nobody needs to read that without watching, I don;t think
<niemeyer> fwereade_: The whole idea of the flag in the unit, I mean
<niemeyer> fwereade_: needing upgrade == service.Charm != unit.Charm
<fwereade_> niemeyer, hmm... just a moment ago you convinced me that it would be a good idea to keep the possibility of upgrading parts of a service
<fwereade_> niemeyer, and we still need somewhere for the force flag to live -- I guess it could be the service, but see above, partial upgrades
<niemeyer> fwereade_: Agreed, but I've been changing my mind since then.. we're putting brand new logic in place that doesn't scale, for a benefit we don't yet have
<niemeyer> fwereade_: DOuble loss
<fwereade_> niemeyer, well, ok, cool, but also ouch, more infrastructure rework :)
<niemeyer> fwereade_: Well, you mean the method in Unit? That sounds trivial.. I'd be worry if we had logic in the uniter already
<niemeyer> worried
<fwereade_> niemeyer, I'm talking about the watcher really
<fwereade_> niemeyer, I expect it will be trivial, it's just that for some reason it flips my perception of my blockers from "meh, I'll probably finish it tonight" to "omg this will take forever"
<fwereade_> niemeyer, which is self-evidently insane, I do not need to be told ;)
<niemeyer> fwereade_: Ah, yeah, you mean we need a watcher for the service which I suppose we don't yet have
<fwereade_> niemeyer, yeah, that's all really
<niemeyer> fwereade_: A lot of blocks indeed
<fwereade_> niemeyer, but still, I'll see what I can do :)
<niemeyer> fwereade_: I wonder about the force upgrade flag SetCharm(charm, force)?
<niemeyer> fwereade_: service.Charm() (charm, force)?
<niemeyer> Well
<niemeyer> fwereade_: service.Charm() (charm, force, error)?
<fwereade_> niemeyer, yeah, I *think* so, but I'm still following it through
<fwereade_> niemeyer, hmmm, how do we know when the force flag can be cleared without scanning all units?
<niemeyer> fwereade_: I don't think we have to clear it
<niemeyer> fwereade_: (!)
<fwereade_> niemeyer, ah, yeah, next upgrade we just set a new charm and force it or not
<niemeyer> fwereade_: Yep
<niemeyer> fwereade_: It's awesome that the right path generally shows itself obviously good by the number of things we *don't* have to do
<fwereade_> niemeyer, hmm, is it meaningful to unforce the upgrade to a particular charm?
<fwereade_> niemeyer, IIRC we have some logic that says you can't unforce an upgrade, but I forget why
<niemeyer> fwereade_: I actually commented in the review that this was weird, but accepted not to argue until later
<fwereade_> niemeyer, ha :)
<niemeyer> fwereade_: The upgrade operation as "I want to use this charm, with forcing enabled/disabled" feels as good as it gets
<fwereade_> niemeyer, sgtm, can we explore the idea a bit more in the context of compatibility?
<fwereade_> niemeyer, Service.SetCharm gives us a great point at which to enforce interface compatibility... maybe that's all we need
<fwereade_> niemeyer, trying to figure out how much damage a zombie unit can do
<fwereade_> niemeyer, I guess the answer is "none" as long as it's operating well enough that it kept its relations alive
<niemeyer> fwereade_: Hm
<fwereade_> niemeyer, because those relations, had they had that unit as a member, would not have dies, and would therefore themselves have blocked the upgrade in the first place
<niemeyer> fwereade_: I'm afraid I'm a bit out of context
<niemeyer> fwereade_: I agree re. SetCharm being a nice point
<fwereade_> niemeyer, sorry, it feels like it's connected to, er, 1 sec
<niemeyer> fwereade_: It's actually brilliant, in the sense that we can have a single point efficiently checking for all units
<niemeyer> fwereade_: I don't know what a zombie unit is, though :)
<fwereade_> niemeyer, I mean a unit that comes back up after a long break, which is using a hugely outdated charm, and may even be dying
<niemeyer> fwereade_: Ok, so what's the issue with it again?
<fwereade_> niemeyer, I'm just feeling like we should consider https://bugs.launchpad.net/juju-core/+bug/1032557 in this context before we go too far
<fwereade_> niemeyer, but on a reread I don;t think it's relevant
<fwereade_> niemeyer, it's just another chunk of ugliness for the uniter, wwhich I'll hopefully be able to keep under control :)
<niemeyer> fwereade_: The bug sounds a bit awkward in its wording
<niemeyer> fwereade_: Oh, wait
<niemeyer> fwereade_: upgrade-charm as in "juju upgrade-charm"?
<niemeyer> fwereade_: I had the hook name in my mind for whatever reason
<fwereade_> niemeyer, ah, sorry :)
<niemeyer> fwereade_: No, it's my bad
<fwereade_> niemeyer, but anyway, I don't think it actually does hurt us here
<niemeyer> fwereade_: I think we should start by just being conservative.. no changes in established relations
<niemeyer> fwereade_: We can improve from there, always staying on the safe side
<fwereade_> niemeyer, ok, sure, there's still a chunk of ugliness to deal with in the uniter
<niemeyer> fwereade_: How so?
<fwereade_> niemeyer, toactually stay on the safe side we need to make sure we don't join relations that don't exist in the current version of the charm
<fwereade_> niemeyer, otherwise we'll get a bunch of hooks "executed" despite their not actually existing
<niemeyer> fwereade_: Wow.. my mind just spinned for a second.. how's that again? :-)
<fwereade_> niemeyer, unit has cs:series/dummy-0, service has cs:series/dummy-1, upgrade is unforced, uniter is in error state
<fwereade_> niemeyer, relation is added using an interface that exists in -1, but not in -0
<niemeyer> fwereade_: Great, I'm with you now
<niemeyer> fwereade_: Aha, thanks, gotcha
<fwereade_> niemeyer, when the uniter goes into the started state, it... well, really, it just needs to check the upgrade flag before it starts messing with relations
<fwereade_> niemeyer, ok, it's easier than I thought :)
<fwereade_> niemeyer, yay!
<niemeyer> fwereade_: Just when I was starting to get worried!? LOL
<fwereade_> niemeyer, haha, sorry :)
<niemeyer> fwereade_: The fact we disallow "correct multi-versioning" actually seems like a great simplification there
<fwereade_> niemeyer, yes indeed
<niemeyer> fwereade_: The unit should either follow along, or be broken
<niemeyer> fwereade_: Which is not only easy to implement, but also easy to understand
<fwereade_> niemeyer, yep, and as soon as it's unbroken it catches up
<fwereade_> niemeyer, hum, I have a new scenario
<niemeyer> fwereade_: Uh oh
<niemeyer> :-)
<fwereade_> niemeyer, unit with v-0 is running a hook which takes a long time; someone upgrades the service and adds a new v-1 relation, such that both watches are ready when the unit comes out
<fwereade_> niemeyer, again it's easy, do a quick check of the upgrade flag before going back into the main select
<niemeyer> fwereade_: Welllll
<fwereade_> niemeyer, but the fiddliness factor is growing
<niemeyer> fwereade_: I think we can be more deterministic than that
<niemeyer> fwereade_: We know what charm the unit is running.. we know which relations it takes
<fwereade_> niemeyer, hey, wait, it's even easier
<fwereade_> niemeyer, just *totally ignore* all relations that don;t make sense yet
<fwereade_> niemeyer, we'll start new relation watches when pick ourselves up from the upgrade
<fwereade_> niemeyer, and it will all shake out then
<fwereade_> niemeyer, niiice
<niemeyer> fwereade_: It doesn't feel so straightforward
<fwereade_> niemeyer, oh :(
<niemeyer> fwereade_: I mean, it is, but you're trying to order two events that happen independently and chaotically
<niemeyer> fwereade_: Doing: 1) do I have upgrades; 2) do I have new relations
<niemeyer> fwereade_: Isn't safe
<niemeyer> fwereade_: Because between 1 and 2, I can very quickly upgrade the charm and add a new relation
<fwereade_> niemeyer, agreed, which is why the simpler solution is also *better* -- the only question I need to ask is "does this relation look sane?" and if it doesn't I can just ignore it
<fwereade_> niemeyer, trusting that the rest of the system has enough integrity that sooner or later I'll pick up the upgrade that will make that relation make sense
<niemeyer> fwereade_: Right, and by sane I suppose you mean verifying against the charm that the unit is running ATM
<fwereade_> niemeyer, yeah, exactly
<niemeyer> fwereade_: Cool
<niemeyer> fwereade_: The only detail then is that we have to "re-watch" after upgrading
<niemeyer> fwereade_: Since we may have lost events that were responsible for the creation of such relations
<fwereade_> niemeyer, yeah, the idea is that upgrading is a spearate mode from starting, which is what actually watches for relations
<fwereade_> niemeyer, s/starting/started/
<niemeyer> fwereade_: Cool, I haven't yet run through the uniter in detail, so I'll just trust you on that.. it definitely sounds great :-)
<fwereade_> niemeyer, once we're in started, we start a whole bunch of watches that only last for the duration of that mode, and relations is one of them
<niemeyer> fwereade_: Neat
<fwereade_> niemeyer, that said, I should -wip it, and I haven't yet
<fwereade_> niemeyer, these underlying changes are only going to have good effects at the uniter level :)
<niemeyer> fwereade_: Yeah, I have a great feeling about those conversations
<niemeyer> fwereade_: It sounds like we're actually understanding the problems we've been dealing with, and are making good progress towards a fully working system
<fwereade_> niemeyer, I am approaching a state of mystical oneness with the Uniter :)
<fwereade_> niemeyer, I guess I should go write a spot more code, though, not sure how much the branches will need to be split up
<niemeyer> fwereade_: That sounds so great.. LOL
<niemeyer> fwereade_: Yeah, I felt tempted to suggest splitting things up in a few occasions too
<fwereade_> niemeyer, I *think* it'll come out surprisingly clean
<niemeyer> fwereade_: +1
<fwereade_> niemeyer, fixing up the bundle stuff is trivial, though, I'll getthat one out of the way in the first branch before anything else
<niemeyer> fwereade_: Sounds good.. happy to review in-promptu as well
<fwereade_> niemeyer, cool, I ballsed up my sleep patterns over the weekend so I'll be around for a bit, I'll shout if I'm unsure of anything
<niemeyer> fwereade_: I know how that goes.. :-)   I'll have to step out for a moment soon, but I'm also going to be working on the evening, so we can stay in sync
<fwereade_> niemeyer, great
<niemeyer> Okay, biab
<fwereade_> niemeyer, ping
<fwereade_> niemeyer, I was wondering about the motivation behind the (1) continue-on-error and (2) write-revision-last behaviours of the current charm.ExpandTo
<niemeyer> fwereade_: Hmm, let me check
<niemeyer> fwereade_: Can't tell why the heck it continues on error
<niemeyer> fwereade_: revision-last is an artifact of the revision bumping
<fwereade_> niemeyer, ahhh, ofc -- and we actually use that on bundles?
<fwereade_> niemeyer, it doesn't look like we do
<niemeyer> fwereade_: Probably not.. it was part of the Charm interface at some point, which is how it ended up there
<fwereade_> niemeyer, and it feels like there's somehow something odd about being allowed to, actually
<fwereade_> niemeyer, can I drop it please?
<niemeyer> fwereade_: Sure
<fwereade_> niemeyer, sweet, thanks
<fwereade_> niemeyer, also, just to check, IMO files and symlinks should fail to replace directories; sane?
<niemeyer> fwereade_: Why?
<niemeyer> fwereade_: It really depends on the intended semantics
<fwereade_> niemeyer, hmm, just because it feel a bit drastic to my personal gut
<fwereade_> niemeyer, it's equally perfectly fine by me to say Bundle.ExpandTo will overwrite with prejudice
<niemeyer> fwereade_: I think we have to pick something sane from the upgrading perspective
<niemeyer> fwereade_: If that's what we're doing
<niemeyer> fwereade_: You know what we could do..
<fwereade_> niemeyer, well, as long as people don't store data in dirs that then get turned into files/symlinks by future versions of the charm, we're fine
<niemeyer> fwereade_: We could use bzr
<fwereade_> niemeyer, how so?
<niemeyer> fwereade_: Well, "as long as people don't do something that feels ok doing" is somewhat of a dead end :)
<niemeyer> fwereade_: bzr has all the logic for merging state against working trees
<niemeyer> fwereade_: We'll spend weeks trying to get close to it, and won't
<fwereade_> niemeyer, I'm not sure it feels like an entirely ok thing to do to me, honestly
<niemeyer> fwereade_: You're thinking too rationally about it, IMO
<fwereade_> niemeyer, haha
<niemeyer> fwereade_: People will delete the directory.. hack some more.. write the file.. deploy
<niemeyer> fwereade_: This isn't particularly bad looking
<fwereade_> niemeyer, is it really rational of them to think that they can have two entirely different things coexisting with the same name, though?
<niemeyer> fwereade_: Again, they won't sit in a terminal thinking "I'm replacing a directory by a file!"
<niemeyer> fwereade_: It will simply happen, and it will blow up after the fact
<fwereade_> niemeyer, and how will bzr deal with that situation? just that it'll tell us there's a problem?
<fwereade_> niemeyer, in that case I favour just falling over directly when we try to delete a dir
<fwereade_> niemeyer, do the charm tests have an upgrade-from-previous-version bit?
<niemeyer> fwereade_: http://pastebin.ubuntu.com/1157995/
<fwereade_> niemeyer, http://pastebin.ubuntu.com/1158000/
<fwereade_> niemeyer, I think that is the situation we have to worry about
<fwereade_> niemeyer, with the added bonus that there is nothing that stops people from creating their state directories only after the charm has been deployed
<fwereade_> davecheney, heyhey
<niemeyer> fwereade_: Right, and bzr seems to have done the right thing in both cases
<fwereade_> niemeyer, hmm, ok, it would also have some other nice advantages like being able to delete files as well
<niemeyer> fwereade_: Right, and only if they changed
<niemeyer> erm
<niemeyer> fwereade_: Right, and only if they changed NOT
<niemeyer> :-)
<niemeyer> fwereade_: bzr import -h, btw
<niemeyer> fwereade_: We'd manage the importing into a side repo, and then pull onto the production side
<niemeyer> fwereade_: Will all the nice benefits of that (don't screw up unless it all works, etc)
<niemeyer> fwereade_: Rollbacks (!)
<niemeyer> fwereade_: For the moment, though, bzr import + bzr pull sounds like an awesome low-hanging fruit
<fwereade_> niemeyer, well, goodness me: so we just create a little standalone tree with the first expanded content and branch that into the charm dir; and subsequently import direct from the bundle into that side repo, then pull from that into the charm dir?
<niemeyer> fwereade_: Exactly
<niemeyer> fwereade_: We can share the repo between them, for added goodness
<fwereade_> niemeyer, sorry, between who?
<niemeyer> fwereade_: The two working trees
<niemeyer> fwereade_: Although I'm not sure if bzr import could make our day
 * niemeyer tries
<fwereade_> fwereade suspects this is the point at which he should actually figure out bzr rather than just using it in a spirit of optimism ;)
<niemeyer> LOL
<niemeyer> BOOOM
<niemeyer> [niemeyer@gopher ~/foo]% bzr import ../foo3.zip
<niemeyer> bzr: ERROR: bzrlib.errors.MalformedTransform: Tree transform is malformed [('overwrite', 'new-3', u'dir')]
<niemeyer> fwereade_: Yeah, we need the side-repo :-)
<fwereade_> niemeyer, aww
<niemeyer> fwereade_: Well, side branch
<niemeyer> fwereade_: So a side branch where we import stuff
<niemeyer> fwereade_: bzr branch import-branch final-charm-location
<fwereade_> niemeyer, ok, yep, and the actual charm dir is just a branch of that
<niemeyer> fwereade_: run unit on final-charm-location
<fwereade_> niemeyer, ok, great
<fwereade_> niemeyer, we'll also want a final pass to fix up file modes, I think, which is a bit annoying
<niemeyer> fwereade_: On upgrade, 1) bzr import charm.bundle import-branch (I think that works out of the box);   2) bzr commit final-charm-location -m "Preparing for upgrade.";  3) bzr merge import-branch final-charm-location; 4) bzr commit final-charm-location
<niemeyer> fwereade_: Hmm, why?
<fwereade_> niemeyer, there are tests for modes like 750 that apparently can't be expressed by bzr
<niemeyer> fwereade_: I'd wait until that is a real problem
<niemeyer> fwereade_: Getting sane upgrades would already be outstanding
<fwereade_> niemeyer, ok, then I'll just drop the unwanted features of ExpandTo and implement this upgrade logic separately
<niemeyer> fwereade_: May I suggest ignoring ExpandTo for the moment?
<fwereade_> niemeyer, yeah, why not :)
<niemeyer> fwereade_: Well, unless you're already half-way through it
<niemeyer> fwereade_: Otherwise, it feels like it won't help or get in the way
<fwereade_> niemeyer, yeah, it's totally orthogonal really, I'll leave the branch around and finish it off in my Copious Free Time
 * fwereade_ has a *new* plan forming
<niemeyer> fwereade_: Awesome
<fwereade_> niemeyer, now embarrassingly trivial: https://codereview.appspot.com/6454169
<niemeyer> fwereade_: Looking
<niemeyer> fwereade_: Love embarrassingly trivial branches.. LGTM
<fwereade_> niemeyer, cheers
<niemeyer> fwereade_: Please just tweak the message before submitting
<fwereade_> niemeyer, thanks for the reminder :)
<niemeyer> fwereade_: np :)
<fwereade_> niemeyer, ok, more ramifications trickle into my mind: this now means we have another potential "error" state which is not hook-related but which is susceptible to human intervention
<fwereade_> niemeyer, so this is good, it even fits sanely with --retry (thank goodness we didn't call it retry-hooks ;))
<fwereade_> niemeyer, it would be possible for us to revert and continue, but I don't think it's actually helpful to do so
<niemeyer> fwereade_: Hmm.. I'm on the fence on it
<niemeyer> fwereade_: Either way, it feels like a plus we don't have to worry about for now
<fwereade_> niemeyer, ok, I'll go with a pause then; retry can revert and try again with (hopefully, presumably) the new version of the charm they've set on the service, which doesn't have this problem -- while no-retry implies that they have completed the change themselves
<fwereade_> niemeyer, if we auto-revert they don't get much of an opportunity to fix it themselves unless we also add a pause-unit feature or something
<niemeyer> fwereade_: Good point
<niemeyer> fwereade_: Hmm.. what does "completing themselves" mean?
<fwereade_> niemeyer, fixed the conflicts
<niemeyer> fwereade_: It's not clear --retry makes sense for upgrading, besides for the hook itself
<fwereade_> niemeyer, I think it offers a nice path back to a working unit by settings a new charm that's ok -- or even the old one again
<niemeyer> fwereade_: After "juju resolved", it seems like we could run "bzr resolved --all', and in case that succeeds, commit the merge
<fwereade_> niemeyer, yeah, that would be what do do on non-retry
<niemeyer> fwereade_: How's that different from yes-retry?
<fwereade_> niemeyer, yes-retry, I think, means that the user has set a different charm on the service and wants us to try an upgrade from scratch
<niemeyer> fwereade_: We don't have to ask the user whether he wants that or not
<niemeyer> fwereade_: We just need metadata to tell us what is in the directory
<niemeyer> fwereade_: Same trick we use with the agent
<niemeyer> fwereade_: if proposed != current { upgrade }
<fwereade_> niemeyer, hmm... ok, so we also store what charm we were trying to upgrade to; and if that is different from the current one on the service, we revert and try again with the new charm?
<fwereade_> niemeyer, or if the service's new charm is the same as the unit's old one, I guess we just revert :)
#juju-dev 2012-08-21
<niemeyer> fwereade_: The charm being upgraded to and the old charm are both in the same location
<niemeyer> fwereade_: When we resolve, we read it to tell what's in there
<niemeyer> fwereade_: If the user moved backwards, the metadata will have moved backwards with it
<niemeyer> fwereade_: and juju will automatically put it back to work, and then upgrade it again if necessary
<fwereade_> niemeyer, ok, yes, that makes sense... I'm going to have a lie down and a think about this for a bit, and if I'm lucky I won;t come back until tomorrow :)
<niemeyer> fwereade_: I'll be wishing for the best :-)
<niemeyer> fwereade_: Have a good night
<niemeyer> davecheney: Heya
<davecheney> niemeyer: howdy!
<niemeyer> davecheney: Sorry for being a bit behind on your reviews.. I know you have some good stuff in there
<davecheney> np, i know you have your hands full
<davecheney> i've got pleanty of bugs (most that I have registered :) to keep me busy
<davecheney> niemeyer: how is your transaction stuff going ?
<niemeyer> davecheney: Pretty good.. working on it right now, finishing insert/remove
<niemeyer> davecheney: The rest seems solid
<davecheney> awesome
<niemeyer> davecheney: What are you up to?
<davecheney> niemeyer: working on the final part of https://bugs.launchpad.net/juju-core/+bug/1025656
<niemeyer> davecheney: Neat
<niemeyer> davecheney: Do you have plans for what to do after that?
<davecheney> apart from fixing bugs, i'm looking to you and mark to provide direction
<niemeyer> davecheney: Cool, I was wondering if you might appreciate working on being a bit of an integration champion for the next couple of months
<davecheney> niemeyer: i would love that
<niemeyer> davecheney: I appreciated a lot the stuff you've done in the sprint and after that to make it work for real
<davecheney> aww shucks
<niemeyer> davecheney: Having you actively looking for the missing pieces of the puzzle, and chasing down broken parts either directly or by poking people to fix stuff, will be invaluable
<davecheney> speaking of that, i tried to run juju on arm yesterday
<davecheney> with mixed results
<davecheney> niemeyer: i heard you have a juju intergratino testing tool
<davecheney> you were talking about that in the sprint
<davecheney> would you like to hand that over ?
<niemeyer> davecheney: I have a half baked functional test runner that I think we should revive, but it's nothing to sweat at
<niemeyer> davecheney: lp:~juju/juju/ftests
<niemeyer> Internal compiler error.. boom
<niemeyer> Silly one, though
<rogpeppe> mornin' all
<TheMue> morning
<rogpeppe> TheMue: hiya
<TheMue> rogpeppe: hi, how has your free day been?
<rogpeppe> TheMue: good thanks. it was our first wedding anniversary - we went for a nice walk and for a meal at a nice restaurant.
<TheMue> rogpeppe: oh, congratulations
<TheMue> rogpeppe: we'll have our silver anniversery in two years ;)
<rogpeppe> TheMue: i see the release meeting was postponed to today, so i didn't miss anything, right?
<rogpeppe> TheMue: wow!
<TheMue> rogpeppe: yep, yesterday no meeting, and mark felt ill
<davecheney> rogpeppe: he had been at PyCon
<rogpeppe> davecheney: ah, *that* sort of ill :-)
<davecheney> and you remember that story he told us at that Bar across the road from the Pirate Cafe in Lisbon ?
<rogpeppe> davecheney: mornin' BTW!
<davecheney> that might have been related
<davecheney> rogpeppe: yes hello
<davecheney> i'm just about to sign off for a bit, gotta get the dinner on
<rogpeppe> davecheney: k
<rogpeppe> davecheney: enjoy - see you for the meeting in a little bit
<Aram> moin.
<TheMue> Aram: moin
<TheMue> so, merged trunk into my three open branches, now start a new one
<TheMue> Aram: regarding life, when calling State.RemoveMachine() you set the state of the machine to Dying. how do you remove a machine? IMHO one should Kill() and later Die() a machine before RemoveMachine() is checking that it's dead and then removes the machine from the state.
<Aram> I don't understand the question, yes, you should call Kill + Die (or just die, alive -> die is valid now) before calling RemoveMachine.
<Aram> is this not so?
<TheMue> Aram: not in trunk
<TheMue> Aram: here RemoveMachine() in mstate searches the machine with life = Alive and sets it to Dying
<Aram> ?!?
<Aram> let me take a look
<Aram> aaah
<Aram> yes
<Aram> removemachine
<TheMue> state line 71ff
<Aram> for some stupid reason I though you were talking about RemoveRelation.
<Aram> yes, lifecycle for relation is not done yet.
<Aram> erm
<TheMue> Aram: hehe, no
<Aram> for machine
<Aram> only relations have full lifecycle in mstate ATM.
<TheMue> Aram: ah, ok, so i'm still right with our Kill() -> Die() agreement
<Aram> yes.
<TheMue> *pheeew*
<TheMue> +1
<Aram> in fact, lifecycle for machines is what I should do right now, after I push confignodes.
<TheMue> Aram: i only stumbled about it when doing a cross-check. sate has lifecycle now for relations as well as a life watcher and a service relation watcher (all in the review queue) and today i'm starting with units
<Aram> why units before machines?
<Aram> or are you talking about the unit watcher?
<TheMue> Aram: i'm going bottom up, as discussed with niemeyer. relations, units, services, machines
<TheMue> Aram: before you free a machine everything else has to be done
<TheMue> Aram: another question. Relations() of Service returns only alive relations in mstate. i thought we always return all, individual life state can then be retrieved with Relation.Life()
<Aram> you're right, I haven't done that yet because I didn't want to modify the tests.
<Aram> (yet).
<TheMue> Aram: ok
<Aram> btw, confignodes were beautiful.
<Aram> the implementation is really nice.
<Aram> it takes advantage of the dynamic, schemaless nature of mongodb.
<TheMue> Aram: i'm pretty sure that it will be more elegant than the zk counterpart
<Aram> meeting in 4 minutes?
<davecheney> yy
<Aram> I'll be late ~10 minutes, start without me.
<davecheney> mate, we'll all be late
<davecheney> ain't no mramm yet
<Aram> hhe.
<davecheney> nor niemeyer
<TheMue> davecheney: not yet awakened
<niemeyer> Goooood morning!
<davecheney> heeeelo
<TheMue> niemeyer: heeeeeeya
<niemeyer> Is it party time?
<TheMue> niemeyer: indeed, but mramm is not yet here
<niemeyer> Okay, I'm sending invites then
<rogpeppe> niemeyer: seems to be.
<niemeyer> rogpeppe: Hey!
<niemeyer> rogpeppe: Welcome back
<rogpeppe>  fwereade__: ping
<fwereade__> rogpeppe, pong
<rogpeppe> fwereade__: meeting time...
<fwereade__> rogpeppe, oh, so it is :)
<davecheney> mramm is in my time zone
<davecheney> should I call him ?
<rogpeppe> davecheney: good plan
<davecheney> imma skype him
<rogpeppe> davecheney: youa doa thata
<niemeyer> Aram: ping?
<Aram> sorry for being late guys
<Aram> had an emergency
<davecheney> are we all also going to UDS ?
<davecheney> I only have an invite for the pre UDS kegger
<davecheney> and that was all anyone wrote
<mramm> https://bugs.launchpad.net/juju-core/+bug/1028437
<mramm> https://launchpad.net/juju-core/+milestone/1.3
<mramm> thanks again everybody!
<mramm> I know that putting those two meetings together made it very long
<rogpeppe> anyone fancy having a chat about the design of the upgrade command?
<rogpeppe> niemeyer, fwereade__, davecheney: ^ ?
<fwereade__> rogpeppe, a little focused elsewhere atm but I'll try to keep half an eye on one if it happens :)
<rogpeppe> fwereade__: ta
<rogpeppe> niemeyer: does this LGTY, or had you just not finished the complete review yet? https://codereview.appspot.com/6463057/
<niemeyer> rogpeppe: I'm just grabbing some chimarrÃ£o..
<rogpeppe> niemeyer: nop
<rogpeppe> np
<davecheney> night lads, pumpkin time
<mramm> night all
<niemeyer> rogpeppe: I haven't reviewed it after you fixed.. will check it out again now
<rogpeppe> niemeyer: thanks
<niemeyer> rogpeppe: Well, and you fixed it 2h ago.. so that's not too surprising :-)
<rogpeppe> niemeyer: np. it just seemed like there might have been some more substantial comments waiting in the wings. i'm glad that there aren't!
<niemeyer> rogpeppe: Ah, cool, that was mainly it really
<niemeyer> rogpeppe: LGTM, thanks!
<rogpeppe> niemeyer: lovely, thanks a lot
<rogpeppe> niemeyer: any chance we could have a brief discussion about the juju upgrade command?
<rogpeppe> niemeyer: because that's what i'm doing next, i think.
<niemeyer> rogpeppe: Of course
<niemeyer> rogpeppe: Let's go
<rogpeppe> niemeyer: i've been wondering about two commands
<rogpeppe> niemeyer: juju upload-tools and juju upgrade
<rogpeppe> niemeyer: and i've been wondering how the flags to upgrade might work (if we decide to provide some capability to upgrade at finer granularity than "everything all at once")
<niemeyer> rogpeppe: We already have an --upload-tools flag in bootstrap.. what about upgrade --upload-tools?
<rogpeppe> niemeyer: the thing is that upload-tools is useful independently of upgrade
<rogpeppe> niemeyer: (for instance if you want to make tools for several different platforms available)
<niemeyer> rogpeppe: The only moment I can foresee upload-tools being used independently is to build the public bucket
<niemeyer> rogpeppe: Otherwise, there's nothing wrong with doing upgrade --upload-tools on the different platform too
<rogpeppe> niemeyer: because nobody will ever do cross-platform stuff with a development version?
<niemeyer> rogpeppe: I'm sure Dave will :-)
<niemeyer> rogpeppe: upgrade --upload-tools should work
<rogpeppe> niemeyer: hmm, i suppose it doesn't really matter that we'll get half the upgrade done first, then half later.
<niemeyer> rogpeppe: Right
<rogpeppe> niemeyer: but i was thinking about building the public bucket too
<niemeyer> rogpeppe: Yeah, for that it may be useful, but the semantics may be different.. we don't need an environment running for that
<rogpeppe> niemeyer: true.
<niemeyer> rogpeppe: So I suggest getting --upload-tools working first, since that gets us going development wise..
<niemeyer> rogpeppe: The command should probably be called "upgrade-juju", btw
<rogpeppe> niemeyer: ok. what about --version? do we allow the specification of a desired version or  not?
<rogpeppe> niemeyer: +1
<niemeyer> rogpeppe: To avoid the ambiguity with upgrading charms
<niemeyer> rogpeppe: Not for now, at least
<rogpeppe> niemeyer: and another thing: it would be nice to support this workflow: change source; juju-upgrade; test; change source; juju-upgrade; test
<TheMue> niemeyer: if i scanned the code right the ServiceUnitWatcher is not been used today. where will it be used later?
<rogpeppe> niemeyer: but we won't be able to do that unless the version number changes each time.
<rogpeppe> niemeyer: so i was wondering about --bump-version or something like that.
<niemeyer> rogpeppe: Yeah, sounds nice to have something like this.. I wonder about the details, though
<rogpeppe> niemeyer: which would force to version to be one more than the current max version available in the private storage.
<niemeyer> rogpeppe: Ah, ok, without touching the local version
<rogpeppe> niemeyer: yeah
<niemeyer> rogpeppe: Sounds sensible
<niemeyer> TheMue: There were a bunch of things that were going to use it.. watching upgrade flags, watching agent version, ...
<niemeyer> TheMue: Funny enough, they both were moved in different directions
<TheMue> niemeyer: ic
<rogpeppe> niemeyer: i had some thoughts about major-version upgrades too, but i think they can wait
<niemeyer> rogpeppe: +1
<TheMue> niemeyer: just ask because of lifecycle
<niemeyer> fwereade__: Do we have any need for the unit watcher nowadays in the uniter?
<TheMue> niemeyer: the watcher has to be changed to deliver alive, dying and dead units
<rogpeppe> niemeyer: so for now, here's the plan: juju upgrade-juju [--upload-tools] [--bump-version]
<niemeyer> fwereade__: Or anything you can foresee?
<niemeyer> rogpeppe: +1!
<rogpeppe> niemeyer: cool, will do
<fwereade__> niemeyer, what TheMue said, I think
<niemeyer> rogpeppe: So neat
<fwereade__> niemeyer, it will want to know when it's Dying/Dead
<niemeyer> fwereade__: Ah, of course
<niemeyer> TheMue: ^
<TheMue> fwereade__: yes
<TheMue> fwereade__: so if we'll have a user of the watcher once it's worse change it (like the service relation watcher)
<TheMue> fwereade__: but today it's unused ;)
<fwereade__> TheMue, +1
<rogpeppe> niemeyer: one other little twist occurs to me: the upgrade-juju command will need to know the series and architecture of the machine that each agent is running on. i'm *thinking* that it will find this out by looking at the agent's proposed tools (which should be set before the agent is assigned to a machine). does that sound reasonable?
<niemeyer> TheMue: If we do the mstate migration fast enough we don't have to worry about that in state
<niemeyer> rogpeppe: Yeah, sounds like a reasonable way forward
<niemeyer> rogpeppe: We'll need constraint support in the near future, but we're not there yet, and it's not clear it'd be any better either way
<TheMue> niemeyer: ok, so i'll stop here now and continue with mstate after lunch ;)
<rogpeppe> niemeyer: yeah, i think that when we *do* have constraints, the proposed agent tools will still encapsulate the solved constraint in exactly the way that the tools require, so won't need changing.
<TheMue> lunchtime (today a bit later)
<niemeyer> TheMue: Please synchronize carefully with Aram
<niemeyer> TheMue: It's easy to parallelize, and it's also very easy to step on each other's toes
<niemeyer> rogpeppe: Agreed
<niemeyer> TheMue: Enjoy
<rogpeppe> niemeyer: great.
<fwereade__> niemeyer, https://codereview.appspot.com/6441169
<niemeyer> fwereade__: looking
<fwereade__> niemeyer, when you're done, I think I also need to talk about upgrades a bit more, I'll go make a sandwich and try to marshal my thoughts
<niemeyer> fwereade__: Done
<niemeyer> fwereade__: It looks good, but my first impression is that it could be a lot slimmer with a config node and no new ServiceCharm type
<niemeyer> fwereade__: Pretty much: 1) read config node; 2) return value a value b
<fwereade__> niemeyer, hm, maybe -- but the watcher needs a single type to return
<fwereade__> niemeyer, and, honestly, the fiddling around with types seems to cost as many lines as just doing it with a serviceNode does
<niemeyer> fwereade__: ServiceCharmChange..
<fwereade__> niemeyer, yeah, fair enough
<niemeyer> fwereade__: I don't get your comment regarding fiddling with types
<niemeyer> fwereade__: There's a lot of logic there that can go away entirely
<fwereade__> niemeyer, casting stuff to strings/bools, dealing with what happens if they're not right
<niemeyer> fwereade__: Well, just don't.. that's what you're doing now
<fwereade__> niemeyer, you mean the retryChange on the serviceNode? that's only there because I thought you'd prefer it in case we added more fields
<fwereade__> niemeyer, what, just panic if I get rubbish in the node?
<niemeyer> fwereade__: Using such a type is no different than doing "v, _ := i.(bool)"
<fwereade__> niemeyer, yeah, I guess, it just seemed like ConfigNode was overkill for two totally predictable fields
<fwereade__> niemeyer, anyway, happy to do it as you suggest
<niemeyer> fwereade__: I'd agree if we weren't doing non-fun stuff by hand as an alternative
<fwereade__> niemeyer, heh, the confignode seemed to me like the unfun way there
<fwereade__> niemeyer, anyway sgtm
 * fwereade__ remembers why he'd put the toaster away in the cupboard in the first place, you need to watch the damn thing like a hawk :/
<niemeyer> fwereade__: Unfun as in significantly more logic, in exchange for ... ?
<niemeyer> fwereade__: I may be on crack
<niemeyer> fwereade__: Let's see how it looks like
<fwereade__> niemeyer, it'll probably actually trun out better, your instincts are usually solid :0
<niemeyer> fwereade__: Cheers! :)
<fwereade__> niemeyer, ok, should be ready to propose again in a mo, but I think I need another chat about charm upgrade errors
<fwereade__> niemeyer, so, when we get an error, it's because there was a conflict trying to pull into the charm tree from the charms repo
<niemeyer> fwereade__: Right
<fwereade__> niemeyer, but I'm still not sure about how we handle it -- what does juju resolved imply here?
<fwereade__> niemeyer, ISTM that it just means the user has done *something*, and they might themselves have updated the charm to the desired version and they might have reverted and made some more changes that should allow the upgrade to go ahead
<niemeyer> fwereade__: We could make it do "bzr resolved --all + bzr commit -m ..."
<fwereade__> niemeyer, and then we just try to pull again, which might do nothing or might upgrade successfully or might cause a new error
<fwereade__> ?
<fwereade__> niemeyer, and from that operation we determine what the new state of the charm really is?
<niemeyer> fwereade__: I think we can just allow the charm to run again if the operation succeeds
<niemeyer> fwereade__: Oh, wait.. we can't
<niemeyer> fwereade__: Because we need to take a decision regarding whether to run the hook or not
<fwereade__> niemeyer, ok, and the success is what determines what the version is? oh, bother
<fwereade__> niemeyer, ah ok, yeah, that should be easy though
<niemeyer> fwereade__: Yeah, I was just thinking that we could put the charm back in normal working mode and allow it to re-run the upgrade, but we can't.. we really have to take a decision right there if the upgrade is happening or not
<fwereade__> niemeyer, yeah, I think that once we've started upgrading we shouldn't stop until we succeed
<niemeyer> fwereade__: Allowing for a rollback also seems fine.. if diskVersion == oldVersion { abort upgrade and restart normally }
<fwereade__> niemeyer, sorry, I'm not following
<fwereade__> niemeyer, what causes us to do that?
<niemeyer> fwereade__: We're saving the charm version within the bzr branch we use to prepare the upgrade, right?
<fwereade__> niemeyer, perhaps; but where exactly?
<fwereade__> niemeyer, and can we depend on that if it's something the user has access to?
<niemeyer> fwereade__: I assumed it would be part of the charm directory itself.. we talked about unit.SetCharm(charm) when the unit is running, to report the version actually in use, right?
<fwereade__> niemeyer, the second part, yes
<niemeyer> fwereade__: Well, the user has access to the kernel of the machine too :-)
<fwereade__> niemeyer, sure, but we aren't telling the user "hey, go and fix the kernel"
<niemeyer> fwereade__: This is metadata that should be in a dot file of some sort within the charm directory.. if the user hacks that manually, good luck :)
<niemeyer> fwereade__: Are we not? If the kernel breaks, we definitely say that
<fwereade__> niemeyer, my instinct says that we should be storing it somewhere outside the charm directory and updating it based on our knowledge of the success of change operations, just as I expected we would the installing/upgrading/installed state of the charm
<niemeyer> fwereade__: I don't get the contention in this bit.. you're saving the whole charm and snapshotting it.. how come it feels bad to save the version being snapshotted with it?
<fwereade__> niemeyer, I'm not entirely sure it does, I just was expecting to leave the charm directory as the user's playground as much as possible
<fwereade__> niemeyer, and to keep the state we depend upon separate
<fwereade__> niemeyer, but I'm pretty comfortable either way
<niemeyer> fwereade__: I think it's fine to have the state, as in, what's going on with the charm, somewhere else
<niemeyer> fwereade__: But being able to tell what charm version is that thing we're looking at feels very convenient
<niemeyer> fwereade__: It's much more likely that a version being snapshotted with the content will be right than something we keep ourselves independently
<niemeyer> fwereade__: Otherwise, "bzr revert".. boom.. state is wrong
<fwereade__> niemeyer, ok, that seems good then
<fwereade__> niemeyer, so external state controls how we interpret internal state
<niemeyer> fwereade__: Right.. external state tells what we've been doing about it.. internal state simply says what it is ATM
<niemeyer> fwereade__: Well.. version really.. let's avoid one more "state" :-)
<fwereade__> niemeyer, ok, I have some lurking uncertainty but I won't know what is is or whether it's real until I've implemented a bit more :)
<niemeyer> fwereade__: I can sympathize
<fwereade__> niemeyer, anyway, reproposed https://codereview.appspot.com/6441169 :)
<niemeyer> fwereade__: How did it feel?
<fwereade__> niemeyer, just as good -- not convinced it's really better but I don;t think it's any worse :)
<fwereade__> niemeyer, it probably did get a little smaller though, I didn't count
<niemeyer> fwereade__: Yeah, a few types/functions/closures less, thanks
<niemeyer> fwereade__: Reviewed
<niemeyer> fwereade__: Glad to see 100 lines going away
<fwereade__> niemeyer, yeah, always nice :)
<fwereade__> niemeyer, thanks
<fwereade__> niemeyer, all looks sound
<niemeyer> Lunch
<rogpeppe> fwereade_: ping
<niemeyer> Somewhat of a quiet day..
<rogpeppe> niemeyer: i hope to remedy that tomorrow, but i'm off now....
<rogpeppe> niemeyer: see you tomorrow
<niemeyer> rogpeppe: A calm day is good every once in a while..
<niemeyer> rogpeppe: Have a good time!
#juju-dev 2012-08-22
<TheMue> morning
<rogpeppe> TheMue: morning
<TheMue> rogpeppe: hiya
<TheMue> fwereade: heya
<fwereade> TheMue, morning
<fwereade> TheMue, how's it going?
<TheMue> fwereade: fine, just waiting for Aram to discuss a mstate idea with him
<fwereade> TheMue, cool
<TheMue> fwereade: and how are you?
<fwereade> TheMue, so-so :)
<TheMue> fwereade: doesn't sound so really cool
<fwereade> TheMue, still trying to figure out how niemeyer's upgrade plans will *actually* fit into the uniter
<fwereade> TheMue, so finishing it feels far away agin
<TheMue> fwereade: will be like driving on cobblestone for a while but it will fit
<fwereade> TheMue, yeah, we'll get there :)
<fwereade> TheMue, cheers :)
<TheMue> fwereade: the hardest parts will be the semantic changes with lifecycle and caching/refreshing, where state using code has to be changed
<fwereade> TheMue, I'm hoping actually the lifecycle will turn out ok, I sketched that out the other day and I *think* its impact is pretty localised from my pov
<fwereade> TheMue, that's not to say it's easy on you ofc ;)
<TheMue> fwereade: so far it's not too hard, only effort. but every (m)state user will have to take care that the api is changing
<Aram> yo.
<davecheney> sup!
<TheMue> Aram: hi
<TheMue> davecheney: oh, hi too ;
<TheMue> Aram: need your help
<Aram> I'll try my best.
<TheMue> Aram: when building mstate I get funnily a "type *mgo.Collection has no field or method UpdateId" even with mgo v2 (with UpdateId) installed
<TheMue> Aram: did you had the same
<TheMue> ?
<Aram> no.
<davecheney> TheMue: go get -u .../mgo
<Aram> perhaps you should reupdate mgo?
<davecheney> err, go get -u ...mgo
<TheMue> Aram, davecheney: will try
<TheMue> ooops, getting a brz error now during go get
<TheMue> oh, only temporary, now it updated
<TheMue> and fixed, thx Aram and davecheney
<Aram> great
<TheMue> Aram: take a look at http://pastebin.ubuntu.com/1160449/ and http://pastebin.ubuntu.com/1160447/. a way to reuse ensureLife()
<Aram> yeah, the only problem is that machines have integer ids
<Aram> I wish they had string ids
<Aram> relations also have integer ids...
<Aram> I wish everything had string ids
<Aram> we could use interface{} for as the id function parrameter for ensureLife in the meantime
<Aram> mgo uses reflection and it should work.
<TheMue> Aram: it's worth a try, yes
<TheMue> Aram: i'll give it a try here for units
<TheMue> Aram: in State.RemoveRelation() you check if the relation is Dead but you don't refresh before. so your life may be Dying (and so the function returns an error) while a concurrent operation already may have marked it Dead.
<TheMue> Aram: this is no error
<TheMue> Aram: i only think about if a refresh would make sense here
<TheMue> Aram: at least your Dead-check in the bson selector is the most important part
<Aram> no, it's fine the way it is.
<Aram> you should not remove a relation before you are sure it is dead (e.g. you called r.Die() or r.Refresh())
<rogpeppe> Aram: yeah, machine ids being integers is problematic in other places too
<Aram> that forces the client to always call r.Die() if he wants to remove a machine, and that's fine.
<Aram> it can be called any number of times.
<TheMue> Aram: to me it's a bit harsh to panic on a maybe already dead entity
<Aram> if you want to remove a relation, call r.Die() and you'll never panic :).
<TheMue> Aram: so it's only to discipline the state user
<Aram> yes.
<TheMue> Aram: *lol*
<Aram> perhaps we could add a DieAndRemove() function if it's such a common operation.
<Aram> no
<Aram> we should not
<Aram> yes, I remember now
<Aram> DieAndRemove() is harmful, usually the watcher that receives the dead event calls remove.
<Aram> and that guy gets a machine with a dead cache and it's fine to call remove.
<Aram> DieAndRemove will mask the dead event, and that may or may not be a bug.
<Aram> if you really want to remove it independent of any watchers, call r.Die() and r.Remove().
<Aram> that makes it very explicit.
<TheMue> Aram: ic, sounds reasonable
<Aram> TheMue: many tests use regular nodes (e.g. machine nodes) as confignodes to inject corrupt data or to inject data without calling the real setters. that's... clever. do we want to preserve this behavior?
<Aram> because I had a plan to make tests in mstate_test pure, not to do any trickery outside the public API.
<Aram> and move the tests that can't be pure into internal tests.
<Aram> but many tests use this trick.
<Aram> plus I'd need to modifcy confignodes, as now they are in a different collection, so a machine node can never be a config  node.
<TheMue> Aram: we have to check them individually. maybe the concrete reason is only in using ZK
<TheMue> Aram: i like the fact that they#ve got an own collection
<Aram> TheMue: yeah, I did a check and the config node trick is only used in 4 tests.
<TheMue> Aram: great, i think we'll find a way to get around
<TheMue> Aram: btw, i like the speed of the mstate tests. there still several missing, but i hope it will stay so fast.
<Aram> I hope that as well.
<Aram> watchers will slow it somewhat, but I hope it will still e fast
<TheMue> Aram: i just proposed https://codereview.appspot.com/6476044/
<TheMue> Aram: we could move all lifecycle stuff in an own file and give it an own test suite
<Aram> will take a look after lunch
<TheMue> Aram: ok
<Aram> TheMue: you have a review.
<TheMue> Aram: thx
<TheMue> Aram: oh, removed the service check, shâ¦ ;)
<TheMue> Aram: the rest of RemvoveUnit() is like relation and i wanted to keey it common
<Aram> that should be changed too.
<TheMue> Aram: then it's ok that i change it back, will do so
<Aram> thanks
<TheMue> Aram: what problem do you have with "if err := foo(); err != nil { â¦ }"?
<TheMue> Aram: IMHO is a common idiom
<Aram> I find it hard to read, personally, I know it's common.
<Aram> v, ok := m[something]
<Aram> that's fine to me because it makes ok have a very local scope
<Aram> which is great
<Aram> but for errors I don't like it that much
<TheMue> Aram: ok, even if i like it i will change it here ;)
<Aram> if you feel strongly about it, keep it, it boils down to personal preference.
<TheMue> Aram: already changed :P
<TheMue> Aram: you've seen, i added one simple test of the state tests
<TheMue> Aram: there it's the final one before all watcher tests
<Aram> yes, seen it
<Aram> I'm surprised that it works though
<TheMue> Aram: maybe RemoveService() currently doesn't care for lifecycles?
<Aram> ah, it marks associated units as dying
<TheMue> Aram: *lol* ok, works by accident
<TheMue> Aram: do you have a table of which state tests are already ported to mstate?
<Aram> no.
<Aram> I just picked and chosen as I saw fit
<Aram> when I implemented a feature I searched for tests that use that feature
<Aram> and ported those
<Aram> TheMue: another review
<TheMue> thx
<TheMue> Aram: ah, thx for the explanation of the "hard death" :)
<TheMue> Aram: i would like to move the table driven test to a lifecycle suite. because if all use the same ensureLife() there's no need for testing it multiple times
<Aram> yes, I think it's good if we move the Life type and ensureLife to life.go, and stateChanges and the other tests to life_test.go, in a different suite.
<TheMue> Aram: shall I do it?
<Aram> TheMue: please do, and also convert relations to the new function.
<TheMue> Aram: ok
<niemeyer> Yo!
<Aram> hi
<TheMue> niemeyer: heya
<niemeyer> Aram: Shall be ready for us to poke on it
<Aram> niemeyer: what, txn?
<niemeyer> Aram: Watches would need a small extension to have mtime on the transaction, but that's trivial.. we can convert meanwhile
<niemeyer> Aram: Yeah
<Aram> great.
<Aram> show me what you got, anytime
 * TheMue listens too
<niemeyer> Aram: Cool, I'll just move to the main repo, as I've been mistakingly pushing it in a branch in a subdir
<niemeyer> Aram: done
<niemeyer> Aram: It's not tagged, so you'll need to bzr pull it
<Aram> I can't use go.pkgdoc with it? :(
 * Aram runs godoc
<niemeyer> Aram: Not yet..
<niemeyer> Aram: I'll announce it when I have a moment
<niemeyer> Aram: and then tag it properly
<Aram> no issue, godoc is great
 * Aram is reading now
<niemeyer> Aram: I'll also document it better, and write a blog post describing details
<niemeyer> Aram: But for the moment it feels wise to just get out of your way :)
<Aram> ok, I read the docs.
<Aram> now I shall read the tests to get a feel for it.
<niemeyer> Aram: sim_test.go is where the meat is
<niemeyer> Aram: txn_test.go can see improvements
<Aram> yes, reading it already
<niemeyer> Aram: The API is almost boring, thankfully
<Aram> great, thanks.
<Aram> I'm going to play with it.
<rogpeppe> niemeyer: ping
<niemeyer> rogpeppe: Yo
<rogpeppe> niemeyer: i was wondering how much the upgrade logic needs to be aware of the dev/release (odd/even) version numbers
<rogpeppe> niemeyer: i'm thinking it perhaps doesn't, but i'm probably wrong there
<niemeyer> Aram: I think we should start by porting the simple stuff to it
<niemeyer> Aram: Making sure that the changes ("$set", etc) are happening in transactions
<niemeyer> rogpeppe: IIRC, in our last conversation (in Lisbon?) we agreed it doesn't
<Aram> yes, I'll do it after I'm done with the things I'm working on ATM.
<rogpeppe> niemeyer: cool, i'd forgotten about that conversation.
<niemeyer> rogpeppe: We agreed, IIRC, that all we needed was in the upgrade-juju command itself
<niemeyer> rogpeppe: If the decision to use it has been made, then why should the agent care
<niemeyer> rogpeppe: IIRC
<rogpeppe> niemeyer: ah, sorry, i was talking about the logic in the upgrade-juju command itself
<rogpeppe> niemeyer: perhaps all it needs to do is make sure it never upgrades from a release version to a dev version.
<rogpeppe> (and vice versa?)
<niemeyer> rogpeppe: Unless --dev is used
<niemeyer> rogpeppe: Vice versa sounds fine
<rogpeppe> niemeyer: i think i'll add a dev flag to environs.BestTools
<niemeyer> rogpeppe: Sounds sane
<rogpeppe> niemeyer: cool
<rogpeppe> niemeyer: i'd appreciate a review of this when you have a moment BTW, as i'm stacking stuff on top of it. https://codereview.appspot.com/6485044/
<rogpeppe> niemeyer: (it's fairly trivial stuff)
<niemeyer> rogpeppe: Looking
<niemeyer> rogpeppe: LGTM, with a small side-tracking peeve about the hackish storage test type
<rogpeppe> niemeyer: yeah, i thought you might not be keen on that. i wasn't entirely sure of the best way to do it. perhaps pushing into the dummy env might be better.
<niemeyer> rogpeppe: Why? It's just a type with methods
<niemeyer> rogpeppe: What's wrong with just having that?
<rogpeppe> niemeyer: quite a few methods that need to be changed every time the Environ type changes.
<rogpeppe> niemeyer: hence i thought best to inherit them all so that piece of code is only sensitive to the methods it cares about.
<rogpeppe> niemeyer: i agree it feels hackish tho'
<rogpeppe> niemeyer: hmm, actually i can avoid the *most* hackish bits more easily
<niemeyer> rogpeppe: Having that in the dummy environment sounds sane to
<niemeyer> too
<niemeyer> rogpeppe: A dummy.Storage we can make use of
<niemeyer> rogpeppe: So we can do something less awkward once
<rogpeppe> niemeyer: the dummy Storage already works actually
<niemeyer> rogpeppe: Well, so do we have a problem? :)
<rogpeppe> niemeyer: probably not. the FindTools tests do it already. i was doing too much of a naive translation of the tests, i think.
<TheMue> Yeah!
<TheMue> Aram: found a nice way to do the table driven lifecycle test for all entities in life_test.go
<Aram> how?
<TheMue> Aram: let me just add unit (relation is already integrated, and i'll show you).
<Aram> please
<TheMue> oops, bracket set too late. ;)
<TheMue> Aram: it's a kind of "for each living do the same as aram has done for relation before".
<TheMue> Aram: Great! Easier than thought. Will propose it now.
<niemeyer> TheMue: Hm
<niemeyer> TheMue: I'm not sure if I see what you mean, but FWIW, what Aram has done in relations shouldn't be *copied*
<niemeyer> TheMue: The idea is to refactor it so that we have a single implementation that is used for all types
<TheMue> niemeyer: yes, we are working on it. first part has been the only function ensureLife()
<TheMue> niemeyer: but the goal shall be an embeddable type
<niemeyer> TheMue: Superb, I imagined you'd already going in that direction, but decided to mention as it's cheaper than fixing later :-)
<Aram> pro tip: if you are using chrome and you have pinned tabs with e-mail etc, never ever open more then one window. you'll close them in wrong order and it will forget your precious saved tabs.
<TheMue> niemeyer: indeed :D
<TheMue> Aram: take a look at https://codereview.appspot.com/6481045/
<TheMue> niemeyer: btw, /me likes state on mongo. pretty clean (and fast), simply elegant
<niemeyer> TheMue: That's great to hear!
<Aram> TheMue: you have a review
<TheMue> Aram: thx *click*
<TheMue> Aram: yes, your idea sounds reasonable. will move it.
<Aram> great
<TheMue> Aram: you've seen the idea? when e.g. service is migrated we'll just add it to the slice of envs. ;)
<TheMue> Aram: thanks to your idea with relation it's quite simple
<Aram> yes, it's great
<TheMue> 4 secs for the whole beast, pretty fast. sadly it will slow down when all watcher tests are added.
<niemeyer> TheMue: Let's keep an eye on those
<niemeyer> TheMue: It should be possible to keep the impact pretty low
<TheMue> niemeyer: right time, i just wanted to ask if there's already a first proposal or impl for watchers in mstate
<niemeyer> TheMue: No, as we were waiting to see if the txn idea would flop or not
<niemeyer> TheMue: It seems to work, so we can now start working on it
<niemeyer> TheMue: (in that direction, rather than the old ones)
<TheMue> niemeyer: hey, c'mon, it's by you, so it can't flop *lol*
<niemeyer> TheMue: Oh man, I wish that was true
<TheMue> niemeyer: i know have to leave the keyboard for a short time. can you msg me the uri to pull it?
<Aram> I just did a bzr pull inside the mgo dir and it was there afterwards
<niemeyer> TheMue: ^
<Aram> niemeyer: multiple txn runners can use the same collection, right?
<niemeyer> Aram: Yeah, they must if they are sharing affected documents
<Aram> ok.
<niemeyer> Aram: We only need a single tc
<Aram> do we need more than one runner?
<Aram> niemeyer: ^
<niemeyer> Aram: No, everybody can share the same one
<Aram> great.
<Aram> thanks
<niemeyer> Aram: np
<fwereade_> taking a break, back later
<Aram> niemeyer: how can I do an UpdateAll with your txn?
<Aram> is it possible?
<niemeyer> Aram: It's not
<Aram> :(
<niemeyer> Aram: Yeah, this is a change to consider
<niemeyer> Aram: But we can live without it for now I suppose
<Aram> well, we do use UpdateAll
<Aram> to update all units of a service
<niemeyer> Aram: Sure.. where are we using it ATM?
<Aram> in RemoveService
<niemeyer> Aram: To begin with, we can put them all in a single transaction and run it
<niemeyer> Aram: Won't scale to extremes, but we can address that down the road
<Aram> but we don't know the ids
<niemeyer> Aram: Hmm.. interesting.. let me think for a moment how we can solve this cheaply
<niemeyer> Aram: What's the change document?
<Aram> change = bson.D{{"$set", bson.D{{"life", Dying}}}}
<Aram> more precisely
<Aram> 	sel = bson.D{{"service", svc.doc.Name}}
<Aram> 	change = bson.D{{"$set", bson.D{{"life", Dying}}}}
<Aram> 	_, err = s.units.UpdateAll(sel, change)
<niemeyer> Aram: Hmm
<niemeyer> Aram: I think we can make the units themselves responsible for setting their dying state
<niemeyer> Aram: And we can ref-count live units on the service itself
<niemeyer> Aram: So we know when the service can transition to dead
<niemeyer> Aram: The units then have to monitor the service to see when they're supposed to shut down
<Aram> so each unit watches for its service, and sets itself to dying when the service is set to dying?
<niemeyer> Aram: Yeah
<Aram> that's potentially a lot of watches :).
<Aram> ok.
<niemeyer> Aram: In the same transaction that dec's the alive refcounter
<Aram> I've put a // BUG in the code so I remember to do this after I implement watcher
<Aram> (obviously I can't do it before).
<niemeyer> Aram: Well, not really, sorry
<niemeyer> Aram: Erm.. I'm correcting myself, sorry
<niemeyer> Aram: The ref count should only be dec'd when the unit goes Dead
<niemeyer> Aram: Regarding being a lot of watchers, not really
<niemeyer> Aram: We already have to monitor the service from the unit for other reasons
<niemeyer> Aram: and again, the watching logic should be implemented *client side*
<niemeyer> Aram: There shouldn't be a significant delta in monitoring the service or not
<niemeyer> Aram: Which code?
<Aram> niemeyer: ?
<niemeyer> Aram: What's the // BUG?
<Aram> in RemoveService
<Aram> 	// BUG(aram): units should monitor their service and set themselves
<Aram> 	// to dead. waiting for watchers.
<niemeyer> Aram: Why RemoveService cares?
<Aram> niemeyer: RemoveService doesn't care, I care so i remember to do it after I implement watchers.
<niemeyer> Aram: Okay, but we can get rid of UpdateAll already, right?
<Aram> no?
<niemeyer> Aram: Okay, so that's what I'm trying to talk about :)
<niemeyer> Aram: What has to change for us to fix RemoveService in both state and mstate now?
<niemeyer> Aram: Well, or maybe just mstate
<Aram> first we need to implement lifecycle for services.
<Aram> niemeyer: if I have an Assert:     txn.DocMissing, the Run() will fail if I call it twice, right?
<Aram> (with the same ops)
<Aram> (and if I insert something in that ops).
<niemeyer> Aram: That's right
<niemeyer> Aram: It'll return ErrAborted when the assertion is false
<niemeyer> Aram: (without touching anything)
<Aram> niemeyer: I think I found a bug in txn.
<niemeyer> Aram: Woot
<niemeyer> Aram: What's up?
<Aram> basically you can add documents that have the same no-id field identical, and that field is Unique: true
<Aram> so you can add {"_id": 0, "foo": x} and {"_id": 1, "foo": y} even if "foo" should be unique.
<niemeyer> Aram: Well, I'm not sure I understand the problem.. you certainly can't add the document because MongoDB itself will prevent that from happening
<Aram> maybe it doesn't end up in the database, but Run doesn't complain.
<niemeyer> Aram: Please read the documentation of Operation in the Insert and Assert fields
<Aram> I did, not sure your point.
<Aram> my assert is txn.DocMissing whoch is not strong enough.
<niemeyer> Aram: My point is that what you're complaining about is precisely described there
<niemeyer> Aram: Of course.. DocMissing means the id isn't there.. it certainly won't dig in the database to see what are all the unique fields that the document may have and which ones are satisfied or not
<niemeyer> Aram: The way to stop the transaction is via Assert
<Aram> what should I write in the Assert in this scenario?
<Aram> stuff in Assert can only refer to that on object with DocId
<Aram> s/on/one/
<niemeyer> Aram: Right
<niemeyer> Aram: There's basically no way to prevent non-existence of something else that could potentially conflict with the document in any unique field
<niemeyer> Aram: We may somewhat easily workaround it, though
<Aram> how?
<niemeyer> Aram: Can you describe the case in more detail?
<Aram> niemeyer: this is called many times with a different id: http://paste.ubuntu.com/1161111/ it should fail if the key is unique, but it doesn't. concrete case is in relations, they also have an index on key, not only on _id.
<niemeyer> Aram: By "case" I mean the real problem we have.. I understand the basic issue you're talking about, and we can easily workaround it
<niemeyer> Aram: With relations, we could use key as _id
<Aram> that's the real problem we have. we can't fail adding same relation twice.
<niemeyer> Aram: Which sounds realistic
<niemeyer> Aram: And have an auxiliar field for the numeric id
<Aram> yes, we can use key as _id
<Aram> that was the original design :)).
<niemeyer> Aram: Sorry, my time machine had a bug on it and I couldn't use it
<niemeyer> Aram: I'll step out for lunch.. happy to continue in a bit
<Aram> enjoy
<niemeyer> Cheers
<niemeyer> For the record, I'll take the chance we're low on reviews right now and I'm taking the  the afternoon off to relax a bit after the intense day+night+no weekend of the last few weeks. I'll be around for a moment still to finish the blog post, so feel free to poke
#juju-dev 2012-08-23
<twobottux`> aujuju: How can I manage multiple administrators with juju? <http://askubuntu.com/questions/179230/how-can-i-manage-multiple-administrators-with-juju>
<rogpeppe> fwereade_: mornin'
<fwereade_> rogpeppe, heyhey
<rogpeppe> fwereade_: how's the uniter trauma going?
<fwereade_> rogpeppe, I *think* it's starting to feel sane again :)
<fwereade_> rogpeppe, I'll see what I can pull together this morning :)
<rogpeppe> fwereade_: cool. i haven't got what the suggested change is. is it to do with using bzr for the upgrading?
<fwereade_> rogpeppe, that's actually quite a small part of it -- the stuff I've been having difficulty with is the relatively innocuous suggestion that there shouldn't be any distinction between forced and unforced upgrade operations, and uh, something else I forget
<rogpeppe> interesting
<rogpeppe> fwereade_: BTW i just discovered a piece of cmd/juju testing which is fairly crackful - all the DeploySuite tests are executed 4 times each!
<fwereade_> rogpeppe, golly
<fwereade_> rogpeppe, how did that happen?
<rogpeppe> fwereade_: it's 'cos other command test suites are embedding DeploySuite
<rogpeppe> fwereade_: not quite sure how that got past my radar
<fwereade_> rogpeppe, gaaaaah
<fwereade_> rogpeppe, well, hey, faster tests :)
<fwereade_> rogpeppe, nice catch
<rogpeppe> fwereade_: now i just have to work out how much of DeploySuite can be replaced by JujuConnSuite
 * fwereade_ hopes lots
<rogpeppe> fwereade_: lots but... maybe not all; i think perhaps the JUJU_REPOSITORY setting needs to remain
<fwereade_> rogpeppe, still :)
<rogpeppe> yeah
<TheMue> morning
<fwereade_> TheMue, heyhey
<TheMue> fwereade_: heya
 * niemeyer waves
<fwereade_> niemeyer, heyhey
<fwereade_> niemeyer, nice response :)
<fwereade_> niemeyer, (on the list)
<niemeyer> fwereade_: Morning!
<niemeyer> fwereade_: Cheers.. tricky too
<fwereade_> getting coffee + sunlight, bbs
<niemeyer> Nice plan
<niemeyer> No sun here yet, but I could get some coffee
<TheMue> niemeyer, Aram: you should get a link to a document via mail. so far e don't use very much watchers in trunk.
<Aram> cheers.
<niemeyer> TheMue: Sure, but the question is which watchers are needed/used
<niemeyer> TheMue: You can talk to William to see details of the unit agent, and the Python code base for evaluation
<Aram> niemeyer: I tried yesterday an experiment. I saw that you defined a local M type in txn and used that instead of the cluncky bson.M. I tried the same with bson.D in mstate. I did a local type D []bson.DocElem and then s/bson\.D/D/g but it won't work, any idea why.
<TheMue> niemeyer: yes, i think especially in fwereade_s queue are several usages
<niemeyer> Aram: Yeah, bson.D is handled specially internally
<Aram> interesting.
<niemeyer> Aram: I guess we could make it check for the element type instead
<niemeyer> Aram: Will have a look as I'm preparing an update
<niemeyer> (mainly to get a few details of txn polished)
<fwereade_> TheMue, offhand, the watchers I do/will need for the UA are: Service.WatchRelations; Service.WatchCharm; Unit.WatchResolved; Unit.WatchLife;  RelationUnit.Watch
<fwereade_> Aram, ^^
<TheMue> fwereade_: hehe, thx, just scanning your proposals
<fwereade_> TheMue, Aram: of those, only the last one should be tricky
<Aram> fwereade_: thank you kind sir
<Aram> yep
<fwereade_> TheMue, Aram: if I find I've forgotten one I'll let you know ;)
<TheMue> fwereade_: great, thx again
<TheMue> Aram: I'll add it to the file
 * fwereade_ sighs with relief -- the Uniter has been massaged into a new shape, which passes (near-enough-exactly) the original tests :)
<fwereade_> (excluding actual upgrades, not everything in place there yet, but I feel vindicated all the same :))
 * niemeyer sighs with fwereade_
<fwereade_> niemeyer, ok, I has a plan
<fwereade_> niemeyer, I want a damn uniter, so I intend to crudely hack out everything charm-related that I don't need, just to do a single dumb install without sufficiently smart recovery
<fwereade_> niemeyer, happily your suggestions made it a hell of a lot easier to do that, much easier to decouple charmy modes from hooky modes
<niemeyer> fwereade_: +1!
<niemeyer> fwereade_: Glad to hear the refactoring went well
<fwereade_> niemeyer, rough order, then: basic-uniter; charm-smart-upgrades; uniter-upgrade-charm; uniter-with-relations
<fwereade_> niemeyer, in which uuc depends on csu depends on bu; and uwr hopefully just depends on bu
<TheMue> fwereade_: Found a NeedsUpgradeWatcher in uniter/modes.go, but not yet in trunk. is it in one of your proposals too?
<fwereade_> TheMue, I killed that the other day
<fwereade_> TheMue, replaced by Service.WatchCharm
<TheMue> fwereade_: ah, fine, will kill it too ;)
<fwereade_> TheMue, cheers :)
<niemeyer> fwereade_: Neat
<fwereade_> niemeyer, yeah, it did -- it took a depressingly long time to find something that didn't acively hurt to look at
<fwereade_> niemeyer, in fact, I'd better try to document the proposed charm.Manager (sorry :( ) API so you can point out all the ways it's crackful before I go too far ;)
<niemeyer> fwereade_: I'd be happy to have a look
<fwereade_> niemeyer, actually, huh, it should obviously be charm.Directory :/
<niemeyer> fwereade_: I'm not sure.. we have charm.Dir already :)
<fwereade_> niemeyer, hmm :)
<fwereade_> niemeyer, I'll stick with manager for now, then, Directory isn't actually quite right because it needs to keep other stuff in a separate state dir
<fwereade_> niemeyer, http://paste.ubuntu.com/1162551/
 * fwereade_ tries not to look anxious
<niemeyer> fwereade_: Looks quite nice :-)
<fwereade_> niemeyer, cool :)
<niemeyer> fwereade_: Slightly surprised to see state.Charm there, for some reason
<fwereade_> niemeyer, I *think* it's the right type to use in general; maybe crackfulness will become apparent when you see it used :)
<niemeyer> fwereade_: func (*Manager) ReadStatus() (Status, *state.Charm, error)
<niemeyer> fwereade_: How can the manager know the *state.Charm* in use?
<fwereade_> niemeyer, (man, it did take forever to figure out the right bits -- for a while I had a separate Operation type which really seemed like a good idea)
<niemeyer> fwereade_: I'd expect it to know the URL instead
<fwereade_> niemeyer, it's passed a *state.State on create (whoops, forgot that bit)
<fwereade_> niemeyer, entirely so that it can give back a *state.Charm when asked
<niemeyer> fwereade_: Yeah, I'm arguing that this is a bit awkward
<niemeyer> fwereade_: It should really give back the URL, IMO
<fwereade_> niemeyer, I *think* it works out better like this than messing around creating them in the Uniter
<niemeyer> fwereade_: I think it breaks down a level of encapsulation for little benefit
<niemeyer> fwereade_: state.Charm(url)
<fwereade_> niemeyer, but, well, feedback like this is why I showed you -- thanks :)
<niemeyer> fwereade_: That's all it takes
<niemeyer> fwereade_: Thanks :)
<fwereade_> niemeyer, I was on the fence, and I think it saved a couple of lines of code, but you're right -- cleaner API > cleaner client code
<fwereade_> brb
<TheMue> lunchtime
<niemeyer> Aram: Curious
<niemeyer> Aram: Turns out I was wrong
<Aram> yes
<niemeyer> Aram: bson.D isn't special
<niemeyer> Aram: It's already looking at the element type it seems
<Aram> so what's the issue?
<Aram> hmm
<niemeyer> Aram: Ah, hmm
<niemeyer> Aram: No, I'm wrong again
<niemeyer> Aram: It's used explicitly through different logic..
<Aram> ok.
<Aram> niemeyer: this should work as an Assert in txn, right?
<Aram> bson.D{{"$or", []bson.D{
<Aram> 		bson.D{{"needsupgrade", nil}},
<Aram> 		bson.D{{"needsupgrade", NeedsUpgrade{Upgrade: true, Force: force}}},
<Aram> 	}},
<Aram> }
<Aram> just an $or query...
<Aram> I have a mysterious issue here.
<niemeyer> Aram: You're probably getting caught into the difference of non-existence vs. nil
<niemeyer> Aram: That said, this stuff has changed in state in the past couple of days
<niemeyer> Aram: It's a lot more friendly to mstate now
<niemeyer> Aram: It's probably easier to migrate it
<Aram> ok, I'll do that, but I'm still interested in the issue.
<Aram> it fails if needsupgrade was already set to that value.
<Aram> but why would it, the assertion should hold true in that case, right?
<Aram> I have another $or assert like that, also one of the options is nil, and that works fine.
<niemeyer> Aram: Yeah, it's worth understanding why that is happening before moving on
<niemeyer> Aram: Ah
<niemeyer> Aram: []bson.D isn't right
<niemeyer> Aram: Is that even working?
<niemeyer> Aram: I mean., compiling
<Aram> sure.
<Aram> all my $or queries are []bson.D
<niemeyer> Aram: Hmm, sorry, I'm on crack.. it's right
<niemeyer> Aram: What is the value like in the database?"
<niemeyer> Aram: At the moment the assertion fails
<Aram> NeedsUpgrade{Upgrade: true, Force: force}
<Aram> exactly what the assertion says
<Aram> I add it once in the begining, it matches nil and it puts it in the db, then I try to add it again which should also work
<Aram> but I get txn.ErrAborted
<niemeyer> Aram: Ah, hmm
<niemeyer> Aram: I know what's wrong
<niemeyer> Aram: Let me confirm it
<niemeyer> Aram: Yeah
<Aram> so what is?
<niemeyer> Aram: txn is using "$or" too
<niemeyer> Aram: I'll add a test and fix it
<niemeyer> Aram: Thanks for persisting on it
<Aram> so it's a txn bug?
<Aram> heh.
<Aram> great.
<niemeyer> Aram: Yeah.. I was (ab)using $or internally to make it cheaper to build the query
<niemeyer> Aram: I have to mix the queries instead
<niemeyer> Aram: So it ended up like {"$or": [{"$or": ...}]}
<hazmat> will juju-core use mongodb to distribute charm files instead of provider storage?
<Aram> no.
<Aram> (or at least not yet?)
<niemeyer> Aram: Curious.. i can't reproduce it in a test
<niemeyer> hazmat, Aram: It will, it doesn't yet
<niemeyer> Aram: Ah, got it
<niemeyer> Aram: I can't reproduce it
<niemeyer> Aram: Can you please paste the snippet of code that is failing?
<hazmat> niemeyer, with the move to that, the only use of provider storage will be the client map to api/db servers? or is the plan to ditch provider object storage entirely?
<niemeyer> hazmat: We may be able to ditch it entirely, but that's a bit off still
<niemeyer> Aram: It looks like the nesting actually works
<Aram> niemeyer: I will try to produce a self contained example
<niemeyer> Aram: Can you please just paste the snippet first?
<Aram> sure
<hazmat> niemeyer, cool, thats fine, i'm just talking future speak to an interested provider.
<niemeyer> hazmat: I see, I do hope we're able to handle without the provider storage
<niemeyer> Aram: I've addressed the custom []DocElem, btw
<niemeyer> Aram: Will be out, probably as soon as we nail down why you're having trouble
<Aram> niemeyer: http://paste.ubuntu.com/1162690/
<Aram> I've marked the failed assert
<Aram> with THIS FAILS
<niemeyer> Aram: Thanks
<niemeyer> Aram: This test passes, btw: http://paste.ubuntu.com/1162673/
<Aram> niemeyer: wrote a self reproducing test
<niemeyer> Aram: Oh, brilliant!
<Aram> niemeyer: works with go run
<Aram> http://paste.ubuntu.com/1162701/
<niemeyer> Aram: I'll try moving that onto a test case
<niemeyer> Aram: Uh oh
<niemeyer> Aram: This is bogus
<Aram> niemeyer: wait, something is worng
<niemeyer> Aram: Yeah
<niemeyer> Aram: It's this bit: {{"a", nil}, {"a", "a"}}
<niemeyer> Aram: This isn't a list of documents
<niemeyer> Aram: It's a single document with two repeated fields
<niemeyer> Aram: But that's not what you have in the real code
<Aram> it isn't?
<Aram> 	sel := bson.D{{"$or", []bson.D{
<Aram> 			bson.D{{"needsupgrade", nil}},
<Aram> 			bson.D{{"needsupgrade", nu}},
<Aram> 		}},
<Aram> 	}
<Aram> that's the real code
<niemeyer> Aram: Right.. that's not what the self-contained example has
<niemeyer> Aram: The example is wrong, the real code is right
<Aram> right
<Aram> well, let me fix that
<niemeyer> Aram: Btw, the THIS FAILS seems misplaced
<niemeyer> Aram: It's pointing to a get rather than a set
<niemeyer> Aram: Is the the set right above it the proper place?
<Aram> niemeyer: yeah, sorry.
<Aram> the set above
<Aram> niemeyer: fixed my self contained example: http://paste.ubuntu.com/1162716/
<Aram> but I can't reproduce it either
<Aram> it works there
<Aram> and it's a good test, if I change "a" to b "b" in the assert, it fails the second change, which is correct.
<Aram> so nothing wrong on the txn side.
<niemeyer> Aram: and the only way to get ErrAborted is really for the assertion to fail
<niemeyer> Aram: Nothing else returns it.. so it's a bit puzzling
<Aram> yeah, what's difference between the self contained example and the real code... the assertion is the same.
<Aram> the only difference is that one is a *string and one is a *T
<niemeyer> Aram: Can you please enable mgo.SetDebug(true) right before executing the failing statement, so we can see what's running?
<niemeyer> Aram: Ah, and mgo.SetLogger(c)
<niemeyer> Aram: Hmmmmmm
<niemeyer> Aram: Let me check something
<Aram> niemeyer: debug log: http://paste.ubuntu.com/1162725/
<niemeyer> Aram: I bet it's the reordering of the fields
<niemeyer> Aram: Yeaahhhh
<Aram> interesting.
<niemeyer> Aram: http://pastebin.ubuntu.com/1162730/
<Aram> hmm...
<niemeyer> Aram: txn is using a map to inject some data in the inserted document (such as its id, revno, etc)
<niemeyer> Aram: Will have to change it to a bson.D to avoid his
<niemeyer> this
<Aram> so it was a real bug, after all
<niemeyer> Aram: Yep, can't reorder fields since people can trust on it
<niemeyer> Aram: Thanks for the care on this, btw
<Aram> welcme
<fwereade_> niemeyer, also, 3rd time's the charm... maybe... https://codereview.appspot.com/6482053
<fwereade_> niemeyer, and on that bombshell, I am *exhausted*, and am going to knock off a little early today -- I'll almost certainly be back on later with my heart in my mouth ;)
<fwereade_> later all
<Aram> cheers
<fwereade_> (and if anyone else is kinda lounging around, not sure what to do with themselves, please do take a look...)
<fwereade_> ;p
<niemeyer> fwereade_: Get some good rest!
<niemeyer>   * installation (not really resumable, except by luck)
<niemeyer> LOL
<TheMue> Aram: you got the list of the test comparison, now i'll focus on adding the missing ones
<niemeyer> Lunch
<TheMue> niemeyer: enjoy
 * niemeyer waves
<rogpeppe> i'm off now. struggling with test stuff all day, hopefully make more progress tomorrow.
<rogpeppe> see yous tomorrow
<niemeyer> rogpeppe: Have a good time
<niemeyer> fwereade_: I hope the review on the first branch makes sense
<niemeyer> Doc appointment.. back in 30-60mins
<niemeyer> fwereade_: Uniter <3
<fwereade_> niemeyer, w00t!
<niemeyer> fwereade_: I was actually looking at the modes rather than the Uniter itself.. that stuff is superb
<niemeyer> fwereade_: Can almost read it to a child :)
<fwereade_> niemeyer, I was hoping you'd like it :D
<fwereade_> niemeyer, it will get a touch heavier when we add the various other watches, but actually much less so than one might fear
<niemeyer> fwereade_: Of course, but the mechanism itself is straightforward
<niemeyer> fwereade_: Easy to navigate mentally between the states
<fwereade_> niemeyer, yep -- you remember I originally wanted to make them types? I though the setup would be significant enough to hurt, but actually it really isn't
<fwereade_> niemeyer, and yeah, just 2 more modes, upgrading and conflicted
<fwereade_> niemeyer, I have a sketch with a mildly interesting helper type for ModeStarted, but it doesn't become relevant until upgrades *and* relations are in play... and even then it's a touch marginal, will be interesting to see if it's worthwhile when I'm watching lifecycles as well
<niemeyer> fwereade_: Yeah
<fwereade_> niemeyer, good comments on the preceding review, btw, I think I will refrain from addressing them until I'm a bit soberer ;)
<niemeyer> fwereade_: LOL
<niemeyer> fwereade_: Sounds like a plan :)
<niemeyer> davechen1y: Heya
<davechen1y> niemeyer: hey
<niemeyer> davecheney: Regarding the env config
<davecheney> yes
<niemeyer> davecheney: I think what you did is sane, and I was thinking of something that is not an issue..
<davecheney> ok, phew
<niemeyer> davecheney: My main concern is that we shouldn't have in state something that is an invalid environ.Config
<niemeyer> erm
<niemeyer> environs/config.Config
<davecheney> niemeyer: right, i don't think that is a problem with this proposal
<niemeyer> davecheney: Otherwise the whole thing will break down when we try to make state.EnvironConfig() return a first class type
<niemeyer> davecheney: yeah.. the fields being removed are fine
<niemeyer> davecheney: They're specific to the environ
<davecheney> yup
#juju-dev 2012-08-24
<niemeyer> Baking another mgo release
<davecheney> lp is really giving me the shits today
<davecheney> lucky(~/src/launchpad.net/juju-core) % lbox propose
<davecheney> error: Failed to load data for member "dave-cheney": Get https://api.launchpad.net/devel/~dave-cheney: unexpected EOF
<niemeyer> davecheney: Aw
<niemeyer> davecheney: Hmm.. they might be doing a release
<niemeyer> It used to be thursday-evening/your morning
<davecheney> nope, just needs persistence
<niemeyer> Ouch :)
<davecheney> niemeyer: i have a suspicion that these EOF problems we see, are possibly related to the high concurency failures that others are finding in the http package
<niemeyer> davecheney: Hmm.. perhaps
<niemeyer> Okay, that's bed time for me
<niemeyer> Hopefully will go through the full night today
<niemeyer> davecheney: Have a good day man
<davecheney> you too mate
<Aram> heyheyhey
<TheMue> morning
<Aram> yo
<Aram> TheMue: good news
<Aram> we have watchers
<Aram> machine,service,unit, and relation watchers
<TheMue> Aram: cheers, great! +1
<TheMue> Aram: integration of life and moving it into the own file also go on
<TheMue> Aram: did you see the list of missing tests?
<wrtp> Aram, TheMue: mornin'
<TheMue> wrtp: morning, today as wrtp?
<rogpeppe> :-)
<TheMue> :P
 * TheMue watches the creeps on his arm and once again things that starting the day with listening loud to Porcupine Tree is a good idea
<Aram> TheMue: yes, saw the list of missing tests, great.
<Aram> rogpeppe: yo.
<Aram> fwereade: hey
<fwereade> hey Aram, hey everyone
<Aram> TheMue: there will be some pain merging our branches.
<Aram> TheMue: I want to do it today as monday I won't be here.
<Aram> taking a day off.
<TheMue> fwereade: heya
<fwereade> TheMue, heyhey
<TheMue> Aram: ok, my next step is pushing the unit life extension in and then continue with moving life into an own file with own tests as you've seen. got reviews here.
<TheMue> Aram: will your txn stuff go in today?
<Aram> I hope.
<Aram> but if it doesn't you can pull it and use it as -req
<TheMue> Aram: ok
<Aram> doez bzr have something like: git merge --strategy=ours?
<Aram> that makes the VCS acknowledge the merge, but doesn't do any change.
<Aram> (git also has theirs instead of ours, btw).
<Aram> bzr resolve --take-this
<Aram> worked :).
<Aram> error: Failed to update merge proposal log: EOF
<Aram> die
<TheMue> Aram: i pulled your branches, how can i set them as prerequisite of my already existing life branch?
<Aram> you need to create a new branch where you merge them.
<Aram> well
<Aram> now I'll push some approved branches to trunk
<TheMue> Aram: ic
<TheMue> Aram: hoped to have the txn stuff in for life.go
<Aram> TheMue: I think it would be very counterproductive to be working on anything lifecycle related until the txn stuff is not in. I'm already fighting merging wars ATM :).
<Aram> I'll do my best to have everything ready by the end of the day, so you can pick it up on monday. In fact it may be ready earlier in the day.
<Aram> Perhaps you could look over state/presence in the meantime? I only glanced over that.
<TheMue> Aram: right now i handle the lifecycle review (w/o txn), mostly improving the tests for readbility
<TheMue> Aram: i've already outlined txn in lifecycle depending on your branch and the first look has been easy
<Aram> TheMue: btw, with txn and watchers, tests already take 18 seconds :(.
<TheMue> Aram: but that'll be a second step after the merge w/o txn
<TheMue> Aram: hehe, as expected, but still fast
<TheMue> Aram: for presence we'll have to see how to do it, i'll take a look. which is your watcher branch?
<Aram> lp:~aramh/juju-core/58-mstate-watchers-basic, but pull lp:~aramh/juju-core/59-mstate-bson-D-trivial because it's better
<Aram> btw, I'm updating those as we speak so caveat emptor.
<TheMue> Aram: thx
<TheMue> Aram: interesting that you prefix them with a number, dave imho does so too while others are doing a go-mstate-â¦ or go-worker-...
<Aram> it's easier to see them and the relationship between them this way
<TheMue> Aram: watchPollFreq = 1000 * time.Millisecond, why not time.Second?
<Aram> I change it all the time, and usually make it much less
<TheMue> Aram: ok, and I've seen that you impl stuff like State.WatchMachines() in watcher.go. in state the watchers are in watcher.go but their usage is in their type file, here state.go
<Aram> I know, it was a conscious decision.
<Aram> we talked about it with rogpeppe at lisbon.
<TheMue> i'm not happy with it, watcher.go reads very confusing due to the type jumping. every few lines a context change
<TheMue> i had been so happy about williams work to order state regarding the tests
<Aram> I am not happy with the old way :).
<TheMue> so that stuff that belongs to each other is close
<TheMue> for me due to maintenance reasos it's hard to have code for one type so distributed
<TheMue> reasons
<TheMue> it may be ok just in the moment of development, because the developer knows where the stuff is
<TheMue> but it's hard for anyone who is new to the code
<Aram> it's the oposite to me. if I want to look at the watchers, I want to see the function that creates them close to the watchers.
<Aram> it's very annoying to me every time I modify something not to find the function that creates the thing that I modified right away so I can modify that too.
<TheMue> and if you want to look at the service you don't expect its code in 5 different files
<Aram> I need to grep since I have no idea what file it's in
<TheMue> so you can turn it around depending from which directions you ome
<TheMue> oderwise we could have a file where all naming stuff is handled, one for all txn stuff, one for all resolved stuff, one for all ports stuff
<TheMue> and so on
<TheMue> my question is: are you interested in technological details or in the business object/entity
<Aram> if I modify a thing, say a data model for a type, it's very likely that I need to also modify the thing that creates it. so I want the function that creates the watchers to be close with the watchers, the function that creates the service to be close with the service (e.g. in service.go, not state.go as it is now) etc.)
<TheMue> i've often seen that later you're interested in the entities, because the biusiness needs are changing
<TheMue> Aram: why isn't Service.AddUnit() in unit.go?
<Aram> it should be
<Aram> it isn't because I just emulated state
<TheMue> Aram: ok, so today it's inconsistent, what makes it even more difficult. history in state is in the py implementation. i would ask you to talk to niemeyer for a clear direction
<TheMue> Aram: i can live with both, so that we can harmonize it
<Aram> yes, it's bad that it's inconsistent.
<Aram> it should be consistent.
<TheMue> +1
<TheMue> lunchtime
<Aram> TheMue: you might want to pull again, significant changes, I merged your first branch that you pushed to trunk. I had to change ensureLife signature.
<Aram> one new *State param plus another int return.
<TheMue> ok
<TheMue> Aram: what do you thing about a signature change to ensureLife(State, Collection, Id, Life, Descr) (rev, error)?
<TheMue> Aram: i think it would be more logical this way
<Aram> TheMue: I'll do it right away in this branch.
<TheMue> cheers
<Aram> TheMue: done, pull lp:~aramh/juju-core/60-mstate-ensureLife-sigchange
<TheMue> Aram: done too, great, tyvm
<Aram> TheMue: I updated the test list document, as now we have all confignode tests and some watcher tests
<TheMue> Aram: great
<TheMue> Aram: how do i select something in mgo if i have the collection as string and the id?
<TheMue> collection name
<Aram> DB("").C(colname).FindId(id)
<Aram> session.DB("").C(colname).FindId(id)
<TheMue> tyvm
<fwereade_> Aram, TheMue, rogpeppe: anyone free for a potentially bikesheddy conversation?
<rogpeppe> fwereade_: sure
<TheMue> fwereade_: go ahaed
<fwereade_> rogpeppe, TheMue: ok, this is about state persistence for the uniter
<rogpeppe> fwereade_: by which you don't mean "State", presumably... :-)
<fwereade_> rogpeppe, TheMue: sorry, local state; anyway for the purposes of this discussion, we care about hook state and charm state
<rogpeppe> fwereade_: ok
<fwereade_> rogpeppe, charm state is Installing, Deployed, Upgrading, Conflicted
<TheMue> ic
<rogpeppe> fwereade_: ok
<fwereade_> rogpeppe, hook state is Running, Commiting, Complete
<rogpeppe> committing :-)
<fwereade_> rogpeppe, when we have installed the charm itself, we always want to run the install hook
<fwereade_> rogpeppe, yes indeed :)
<rogpeppe> fwereade_: ok
<fwereade_> rogpeppe, what I do at the moment is set the charm state to Installing, install the charm, and leave that state around until I've written Running for the followup hook
<TheMue> so far reasonable :)
<fwereade_> rogpeppe, TheMue: before running the actual hook, I set charm status to Deployed
<fwereade_> rogpeppe, and so the states overlap neatly and I can always come back up to roughly where I was is the process goes down
<TheMue> fwereade_: so 1st hook.Running, then charm.Deployed, then hook.Run()
<rogpeppe> fwereade_: ok
<fwereade_> rogpeppe, TheMue: *but* it is a bit weird to be messing around setting charm states as part of the runHook method
<fwereade_> TheMue, exactly
<fwereade_> rogpeppe, TheMue: now, most of the time, the Mode funcs return the result of a func called hookStateMode
<rogpeppe> fwereade_: it kinda seems reasonable for the install hook to be doing that.
<fwereade_> rogpeppe, well, indeed, it's not *bad* but it gave niemeyer pause
<rogpeppe> fwereade_: uh huh
<fwereade_> rogpeppe, so anyway there is a conventional way we choose the next mode to go to
<fwereade_> rogpeppe, I was wondering whether adding a Queued state to hook might be neater
<rogpeppe> fwereade_: what would Queued imply?
<fwereade_> rogpeppe, "next time you call hookStateMode, it will run this hook for you"
<rogpeppe> fwereade_: ah, queued to run but not actually running?
<fwereade_> rogpeppe, yeah
<fwereade_> rogpeppe, TheMue: so charm.Installing -> hook.Queued -> charm.Deployed
<rogpeppe> fwereade_: so then it would be hook.Queued, charm.Deployed,, hook.Running
<fwereade_> rogpeppe, yep
<rogpeppe> fwereade_: that seems reasonable to me.
<rogpeppe> fwereade_: would hooks *always* go through a Queued state?
<fwereade_> rogpeppe, TheMue: the issue for me is that the transition to running hides away in hookStateMode, which is... well it is an obvious place to find it, I guess
<fwereade_> rogpeppe, I don't think they have to -- but I can't offhand think of a reason they shouldn't
<rogpeppe> fwereade_: what's hookStateMode again? is it that func type?
<fwereade_> rogpeppe, yeah, it does a few things
<fwereade_> rogpeppe, it goes straight to ModeHookError if a hook is apparently Running
<fwereade_> rogpeppe, if it's Committing, it recommits
<fwereade_> rogpeppe, if it's Complete, the next mode is determined by what the hook was
<rogpeppe> fwereade_: isn't it better to think of it as an action that does whatever is appropriate and returns the next action?
<TheMue> fwereade_: and the func is selected based on the state?
<fwereade_> rogpeppe, yeah, I think I've convinced myself, and I think I probably need a better name then hookStateMode
<rogpeppe> fwereade_: so then it's evident that *of course* that's where to find the next mode, because that's how all mode transitions work.
<rogpeppe> s/the next mode/the transition to running/
<rogpeppe> fwereade_: i think hookStateMode is fine.
<fwereade_> rogpeppe, if it's running hooks as well itself that doesn't feel quite right
<fwereade_> TheMue, yes, the next mode func will be Starting if the last hook was Install, and Started in all other cases
<rogpeppe> fwereade_: perhaps "uniterMode" might be more appropriate
<fwereade_> TheMue, (so far, anyway, unit lifecycle will change that a bit I think)
<rogpeppe> fwereade_: or just "mode" :-)
<fwereade_> rogpeppe, ha, maybe
<rogpeppe> fwereade_: 'cos it actually represents the global state of the uniter, right?
<fwereade_> rogpeppe, well, in a sense, yeah
<TheMue> fwereade_: to me transit sounds more natural, coming from a state, going to a state (maybe the same)
<rogpeppe> fwereade_: i saw that top level loop... that's all it does :-)
<fwereade_> TheMue, that's a wrinkle, if we're returning to the same state we're doing it wrong
<rogpeppe> TheMue: i think "mode" represents the "mode of operation" between states quite well.
<fwereade_> TheMue, no sense shutting down a bunch of watches only to restart them and handle the initial events all over again
<rogpeppe> fwereade_: we return to the same state when we crash :-)
<fwereade_> TheMue, rogpeppe: and what this then means is that hookStateMode is not the *only* thing that causes a state transition -- it's just the usual thing to do so
<rogpeppe> fwereade_: sorry, i missed something. what else causes a state transition?
<fwereade_> rogpeppe, if we're in ModeStarted, for example, we run the hook and directly return ModeHookError if it fails
<fwereade_> rogpeppe, we don't want to leave ModeStarted
<rogpeppe> fwereade_: isn't it ModeStarted the thing that runs the hook?
<rogpeppe> s/it //
<fwereade_> rogpeppe, in that case, yes
<rogpeppe> fwereade_: isn't ModeStarted a hookStateMode?
<fwereade_> rogpeppe, sorry, no
<rogpeppe> fwereade_: ah, why not?
<fwereade_> rogpeppe, hookStateMode is a single func that returns a mode func, based on hook state
<rogpeppe> fwereade_: seems odd that it's not, but i'm probably missing something
<rogpeppe> ah!
<rogpeppe> i thought that each mode returned the next mode to transition to.
<fwereade_> rogpeppe, yes it does
<fwereade_> rogpeppe, sorry, I am clearly communicating something really badly
 * rogpeppe goes to the source
<fwereade_> rogpeppe, as it happens, though, hookStateMode itself *does* have the right signature to be itself a mode
<rogpeppe> fwereade_: where's hookStateMode defined and used?
<fwereade_> rogpeppe, modes.go, 2nd func, used throughout
<fwereade_> rogpeppe, sorry it's still called nextMode in that proposal
<rogpeppe> fwereade_: ah!
 * rogpeppe finds the if ... { return } elses distracting
<rogpeppe> fwereade_: i think nextMode is perhaps a better name. or hookNextMode.
<fwereade_> rogpeppe, I'm thinking the signature is telling me it should be ModeSomething
<rogpeppe> fwereade_: perhaps "deduceNextMode"
<rogpeppe> :-)
<fwereade_> rogpeppe, I'm starting to actually feel fond of ModeTransition
<rogpeppe> fwereade_: another way of slicing it would be that if a mode returns nil, the main loop would call nextMode
<rogpeppe> fwereade_: 'cos it seems to me that the modes are only returning nextMode because they don't really know what's going to happen next.
<fwereade_> rogpeppe, I experimented with that, felt a bit off in practice
<fwereade_> rogpeppe, well, not really, they're returning it because while the logic is simple in the various individual places it makes it easier to think about if, where possible, we use the same logic whether we're recovering from a crash or just switching normally
<TheMue> Aram: see the "new" life_test.go at https://codereview.appspot.com/6481045/diff/9001/mstate/life_test.go, that's why i needed the generic selection
<rogpeppe> fwereade_: how is recovering from a crash different?
<Aram> hey niemeyer.
<Aram> niemeyer: we have watches.
<Aram> machine, unit, service, and relation watches.
<niemeyer> Heya!
<rogpeppe> niemeyer: yo!
<niemeyer> Aram: Wow
<niemeyer> rogpeppe: Heya
<TheMue> niemeyer: hi
<niemeyer> Aram: I've been thinking about how to implement them, but I guess you've already moved forward with something
<niemeyer> Aram: What's the plan there?
<niemeyer> TheMue: Heya
<Aram> niemeyer: just the lisbon plan, but using transactions in a way to avoid races.
<niemeyer> Aram: What does that actually mean? :-)
<niemeyer> TheMue: Do you have that list of watches we need?
<Aram> documents have a rev field, watchers remember last rev and poll perdiodically, rev field is incremented in transactions along with a global rev field.
<TheMue> niemeyer: you've got it in your mail, as well as the missing tests
<niemeyer> Aram: Okay, can we have a quick call to save ourselves time?
<niemeyer> Aram: I mean, just for more bandwidth
<Aram> niemeyer: sure
<TheMue> niemeyer: what we have to look for is the presence watcher
<niemeyer> Aram: So we can sync up
<niemeyer> TheMue: Can you join us too?
<TheMue> niemeyer: yep
<niemeyer> TheMue: Yeah, that's an interesting case.. but let's nail down the others first
<niemeyer> (since we're already on them)
 * Aram is ready anytime
<niemeyer> Starting
<Aram> niemeyer: how is that field called, txn_rev?
<niemeyer> fwereade_: "Not really so keen on "pending", that implies to me that it hasn't started."
<niemeyer> fwereade_: Funny enough, that's exactly the case.. :-)
<niemeyer> fwereade_: When we save that hook state, the hook hasn't started
<fwereade_> niemeyer, well, it is the case at the time we write ie
<fwereade_> s/ie/it/
<niemeyer> Aram: txn-revno
<Aram> thanks
<niemeyer> Aram: np
<fwereade_> niemeyer, what do yu think of running/committing/complete?
<niemeyer> fwereade_: "running" has the same issue of "started"
<fwereade_> niemeyer, yeah, and I can convince myself if means completion-pending, as it were
<niemeyer> fwereade_: It's awkward to find a hook in a running state to mean it's in an error state and in fact not running
<fwereade_> niemeyer, agreed
<fwereade_> niemeyer, now, about setting charm state in runHook -- I'm just adding a Queued hook state so that changeCharm can set that before marking the charm as Deployed before it returns
<fwereade_> niemeyer, seems quite neat; sound sane?
<niemeyer> fwereade_: Definitely, cheers
<fwereade_> niemeyer, so: queued, pending, comitting, complete?
 * fwereade_ just cannot spell that word today :/
<niemeyer> fwereade_: +1 :)
<fwereade_> niemeyer, cheers
<fwereade_> niemeyer, just reproposed https://codereview.appspot.com/6482053
<niemeyer> fwereade_: Super, thanks
<niemeyer> Aram: Hmmm.. I thought we had agreed to sort out the "path" issue before integrating the confignode stuff?
<Aram> niemeyer: is renaming to key fine?
<niemeyer> Aram: LOL.. no :)
<niemeyer> Aram: I'm still happy to push these branches forward for the moment, but this is not correct.. we should fix it next week
<niemeyer> Aram:         environConfigKey = "environ"
<niemeyer>         return readConfigNode(s, environConfigKey)
<niemeyer> config, err = readConfigNode(s.st, s.Name())
<niemeyer> Aram: Guess what happens if we have a service named "environ"
<Aram> yeah, that's not great.
<niemeyer> Aram: For the moment, I suggest namespacing stuff with slashes
<niemeyer> Aram: "unit/<name>"
<niemeyer> Aram: "service/<name>"
<niemeyer> Aram: Well, hmm
<niemeyer> Aram: That's not right
<niemeyer> Aram: We don't need config nodes for unit or service.. we already have their own nodes
<niemeyer> Aram: Where do we need config nodes?
<niemeyer> Aram: EnvironConfig.. Relation..
<Aram> niemeyer: no idea, state has it: http://go.pkgdoc.org/launchpad.net/juju-core/state#Service.Config
<Aram> so I did the same
<Aram> ditto for units
<niemeyer> Aram: Ah, service config, yes.. as in the config machinery
<niemeyer> Aram: Hmm.. units don't
<niemeyer> Aram: The Service.Config stuff is what we hook into for the "juju set" and "config-get" commands
<Aram> they don't seem to have now, they did when I implemeted them though.
<niemeyer> Aram: Which means it's indeed generic
<Aram> somebody deleted them in the meantime :).
<niemeyer> Aram: (as in, we have no control over which keys exist)
<niemeyer> Aram: I don't think that's true
<niemeyer> Aram: Units don't have a configuration of their own in that fashion
<niemeyer> Aram: The unit configuration is the determined by service configuration
<niemeyer> Aram: That's always been the case
<Aram> niemeyer: ah, yes. but my branch doesn't have confignodes for units though either
<Aram> only for service and environment config
<niemeyer> Aram: I know.. I was exploring the field with you so we know what we're going to do, rather than blaming or anything
<Aram> ah, yes
<niemeyer> Aram: So, we need namespacing.. how do we call that collection again? /me looks
<Aram> cfgnodes
<niemeyer> Aram: Hmmm.. we don't have nodes in Mongo.. we have documents.. Can we call the collection "settings"?  I think that's how we've been referring to the concept generically speaking
<Aram> settings it is
<niemeyer> Aram: So.. we need them for.. the service config.. for relations, both scoped and global, ...
<niemeyer> Aram: Anything else?
<Aram> environment
<niemeyer> Aram: +1
<niemeyer> Aram: We should also rename ConfigNode to Settings.. but let's wait until we delete the current state package so we don't have to worry about it
<Aram> yes
<niemeyer> Aram: Awesome, looks like that's all
<niemeyer> Aram: So, we have a couple of choices.. the first one is splitting collections, the other is namespacing
<niemeyer> Aram: I think we have to namespace either way, because we can't determine the number of relation scopes ahead of time
<Aram> namespacing sounds better
<niemeyer> Aram: So that's probably the better choice for everything
<niemeyer> Aram: Cool, +1
 * TheMue likes the Settings idea and namespaces, so +1
<niemeyer> Aram: "e", "s/<name>", "r/<scope, etc>"
<Aram> sounds fine
<niemeyer> Aram: No point in using a large prefix and forcing a huge number of comparisons on indexes and whatnot with common prefixes
<TheMue> niemeyer: as our model is relative fixed it's ok. only the second s or r would break it.
 * TheMue so far finds nothing that doesn't fit into the rest of 22 letters
<niemeyer> Aram: Review sent
<Aram> thanks
<niemeyer> Aram: LGTM with these in, actually
<Aram> niemeyer: txn will never insert phantom documents in the "real" collection, right?
<niemeyer> Aram: That's rihgt
<niemeyer> Aram: The only way the real collection and documents is ever changed is in the "txn-revno" and "txn-queue" fields of real documents
<niemeyer> Aram: I very much didn't want to deal with crack in the real content
<niemeyer> Aram: The "There's one race" aspect mentioned in the post is a side effect of that
 * Aram tries to understand why DeepEquals fails since the debug print of what we got from the watcher and what we expect is the same.
<fwereade_> niemeyer, http://paste.ubuntu.com/1164568/
<niemeyer> Aram: int vs. int64 is a common issue
<niemeyer> fwereade_: wtf
<niemeyer> fwereade_: "unrevisioned executability"!?
<fwereade_> niemeyer, yeah, I know :/
<Aram> niemeyer: I mistakenly assumed I could use a type newDoc struct { Doc; something } else in a query, and it would fill the fields of the embedded document, but I was wrong.
<niemeyer> Aram: You're half-wrong only, though
<niemeyer> Aram: ",inline"
<Aram> hah, thanks!
<niemeyer> Aram: np
<Aram> niemeyer: dumping the database, I see that these two documents have the same txn-revno: http://paste.ubuntu.com/1164608/
<Aram> how can that be?
<Aram> they were added in different Run() calls.
<niemeyer> Aram: revno is per document
<Aram> :(
<niemeyer> Aram: What's the issue?
<niemeyer> Aram: The "txn-" prefix is there for namespacing since the package is injecting that into alien documents
<niemeyer> Aram: Rather than meaning "the revision number of the transaction executed"
<Aram> well, if it's per document I don't see how I can use it in watchers. perhaps I'm missing something.
<niemeyer> Aram: Why?
<niemeyer> Aram: Can you run me through the procedure you had in mind?
<Aram> the watcher remembers the last used revno, and when it polls it does a query to see documents with a revno greater then it remembered.
<Aram> since new documents start with a fresh revno, it won't find them this way.
<Aram> did you had in mind a different procedure?
<niemeyer> Aram: Hmm
<niemeyer> Aram: No, I just hadn't considered the possibility of doing that kind of grouping query
<niemeyer> Aram: The idea was the same, but individually per-docuemnt
<niemeyer> Aram: But, let me ponder for a moment. You may be onto something
<niemeyer> Aram: I think all the logic in the txn package would work equally well if it used max(revno)+1, which seems to solve the issue
<niemeyer> Aram: Let me go to the drawing board for a second
<niemeyer> Aram: Nope, doesn't work
<Aram> indeed.
<niemeyer> Aram: Obviously, transactions that affect independent documents can't guarantee to have a monotonically increasing revno
<niemeyer> Aram: So, back to the question: what was the logic you were implementing?
<niemeyer> Aram: I'm still curious because I think I'm missing something
<niemeyer> Aram: If we have 100k documents, we're likely only interested in a selection of them.. why do we care about a global revno?
<Aram> how do we detect new documents?
<niemeyer> Aram: I guess it depends on the scenario..
<niemeyer> Aram: Let's pick one to evaluate
<Aram> machines
<niemeyer> Aram: Cool
<niemeyer> Aram: Thinking about the best approach, just a sec
<niemeyer> Aram: Okay, so here are a few constraints:
<niemeyer> Aram: 1) We don't want to scan everything regularly
<niemeyer> Aram: 2) We don't want a gigantic document with all the ids
<niemeyer> Aram: I think we can solve both of them by having a control document that is touched every time the selection of machines changes
<Aram> niemeyer: sorry, I have to go. I have to pack and I have been here for 12 hours already.
<Aram> so this will have to wait for next week
<niemeyer> Aram: Of course, that's not a problem
<niemeyer> Aram: Have fun, and thanks for the hard work
<Aram> thanks
<Aram> cheer, and have a nice weekend
<niemeyer> Aram: Thanks
<niemeyer> Lunch here
<rogpeppe> fwereade_: ping
<fwereade_> rogpeppe, pong
<rogpeppe> fwereade_: should there be an entry in the State for the bootstrap Machine?
<fwereade_> rogpeppe, yes, I think so
<rogpeppe> fwereade_: currently i don't think there is, though i may be missing something
<fwereade_> rogpeppe, hmm, I thought initzk was going to do that, maybe it doesn't yet
<rogpeppe> fwereade_: hmm, maybe it does, let me check
<fwereade_> rogpeppe, that's what the --instance-id param is for
<rogpeppe> fwereade_: oh yes, so it does. thanks.
<fwereade_> rogpeppe, cool
<fwereade_> niemeyer, hmm, when you get back... if the charm dir's a repo, can you think offhand of any reason *not* to version everything and commit it at the end of each hook?
<fwereade_> rogpeppe, TheMue, ^^
<rogpeppe> fwereade_: and do a revert when starting?
<fwereade_> rogpeppe, offhand, that would maybe seem to make sense
 * fwereade_ thinks through
<rogpeppe> fwereade_: it seems to me there might be some advantage in using the same kind of mechanism for storing charm state as for storing hook state
<fwereade_> rogpeppe, the mechanisms seem pretty similar to me
<rogpeppe> fwereade_: that way we're not reliant on the VCS except for upgrades
<fwereade_> rogpeppe, ohhh right sorry misunderstood
<fwereade_> rogpeppe, well if there's any doubt of the vcs's ability to handle it I can see your point
<rogpeppe> fwereade_: istm that by going that route we'd be conflating two different kinds of thing for a relatively small convenience.
<fwereade_> rogpeppe, really? comprehensive logging of charm state feels like it would be a pretty big convenience to some people
<rogpeppe> fwereade_: i mean, if you *do* go that route, why not do it for all hook state too?
<fwereade_> rogpeppe, well, the important stuff there comes for free, doesn't it?
<rogpeppe> fwereade_: ?
<fwereade_> rogpeppe, if we did commit the charm dir at every significant point, with a message derived from the hook.Info, you get AFAICT a comprehensive history of the unit for a ridiculously small amount of effort
<rogpeppe> fwereade_: shouldn't we be logging all of that anyway?
<rogpeppe> fwereade_: (i'm presuming you're talking about committing the {Installing, Deployed, Upgrading, Conflicted} states)
<fwereade_> rogpeppe, wouldn't we be?
<fwereade_> rogpeppe, no, I'm talking about committing the *charm dir* itself
<rogpeppe> fwereade_: ah
<rogpeppe> fwereade_: sorry, i think that's a good idea, yeah
<fwereade_> rogpeppe, sweet :)
<rogpeppe> fwereade_: two different kinds of charm state!
<fwereade_> rogpeppe, and *way* too many kinds of state in general
<fwereade_> rogpeppe, I just realized state is actually a bad name, way too generic :)
<rogpeppe> fwereade_: the only down side might be that storage space grows indefinitely, and if you accidentally upgrade to a charm with a big blob in it, you can never remove it.
<fwereade_> rogpeppe, it only gets twice as bad; I'm storing the bundles locally too
<fwereade_> ;p
<rogpeppe> fwereade_: at least the bundles are easily GC'd if need be
<fwereade_> rogpeppe, actually, anyway, that's a price we've already kinda agreed to pay by putting it under VC in the first place
<fwereade_> rogpeppe, extra commits aren't going to hurt unless someone's storing frequently-changing big blobs
<rogpeppe> fwereade_: i guess. 'cos we're already committing?
<fwereade_> rogpeppe, yeah, niemeyer thought of it the other night
<rogpeppe> fwereade_: it's a nice idea
<rogpeppe> fwereade_: what bzr primitive do you use?
<fwereade_> rogpeppe, well... actually bzr has an embarrassing crash in the specific situation that prompted the idea
<fwereade_> rogpeppe, http://paste.ubuntu.com/1164568/
<fwereade_> rogpeppe, so I'm actually doing it in git atm :)
<rogpeppe> fwereade_: oops
<niemeyer> fwereade_: The idea of committing seems sound
<fwereade_> niemeyer, cool
<niemeyer> fwereade_: git has a nice -A option to add that will be helpful, btw
<niemeyer> fwereade_: It replaces the "import" command nicely
<niemeyer> fwereade_: On the unfortunate side, we'll need to tweak ExpandTo so it puts a ".empty" file on empty dirs
<niemeyer> fwereade_: To avoid loosing them
<fwereade_> niemeyer, oh yes ofc
<fwereade_> niemeyer, oh, yeah: ISTM that what we need to do is actually to write ExpandTo so that it overwrites everything with extreme prejudice
<fwereade_> niemeyer, ...except hmm, it should probably actually delete the stuff that isn't there beforehand
<fwereade_> niemeyer, *actually*, we could just always expand into a fresh dir and copy .git into it
<fwereade_> niemeyer, that seems neatest actually
<niemeyer> Just note that the .git you copy must be the one from the branch that has imports only
<niemeyer> Not the one from the live charm dir
<niemeyer> (that has changes from hooks, etc)
<fwereade_> niemeyer, indeed so :)
<rogpeppe> niemeyer, fwereade_: i'm off now.. it's a public holiday here on Mon, so i'll see y'all Tues...  have a great weekend!
<fwereade_> rogpeppe, cool, have fun
<niemeyer> rogpeppe: Have a great weekend
<niemeyer> fwereade__: ping
<fwereade__> niemeyer, pong, if you're there
#juju-dev 2012-08-26
<davecheney> mramm: howdy!
#juju-dev 2013-08-19
<axw> ffs. one of these days I'll learn to do mp's right
<thumper> :)
<thumper> I've had a very broken day
<thumper> and it looks like that'll continue
 * thumper sighs
<axw> thumper: heya. broken day?
<thumper> axw: bitsy, where I have other things breaking up the day into small parts
<thumper> like kids sports
<thumper> and errands
<axw> ah
<axw> I thought something bad had happened
<axw> :)
<axw> thumper: are there any component diagrams of the juju internals around?
<axw> I'm slowly understanding the relationships between things (worker, api, etc.), but a diagram would probably help
<thumper> no, nothing bad, just a little frustrating
<thumper> axw: not a diagram that would be any good
<thumper> :)
<axw> no worries
<thumper> axw: I could probably describe the world reasonably succinctly in a hangout
<thumper> I have a glass of wine now
<thumper> it is after wine O'clock
<axw> hehe - just, I'll just get a glass of water
<axw> it is nowhere near wine o'clock here
<thumper> :)
<thumper> I'm sure there are some ways around the rules
<thumper> like if you are talking to someone where it is past the yard arm
<axw> thumper:  https://plus.google.com/hangouts/_/5d3396bb74d23c8b35414d4c66c7aa4e1c4b7bad?hl=en-GB
<axw> not sure if that's the right URL ...
<TheMue> morning guys, back again
<noodles775> Are changes to goose done just via a Launchpad MP? If so, could someone take a look at https://code.launchpad.net/~michael.nelson/goose/test-instance-state/+merge/180788 pls?
 * noodles775 checks if lbox works with goose also.
<noodles775> So also at https://codereview.appspot.com/12897044/
<noodles775> dimitern: Morning! no rush, or pressure, but (one implementation of) the goose change we talked about on Friday: https://codereview.appspot.com/12897044/
<dimitern> noodles775: hey! will take a look shortly
<dimitern> noodles775: reviewed
<noodles775> Thanks dimitern. With the first change (new status), did you mean you just want s/HP //, or actually truncate the comment?
<noodles775> dimitern: nm, I hadn't expanded your comment.
<dimitern> noodles775: my point was to mention HP Cloud, not just HP
<noodles775> dimitern: done - the second change was slightly different than suggested. Should I lbox submit it now (ie. one LGTM, or still wait for a second?)
<dimitern> noodles775: according to the mail today, now only one LGTM is needed (a month's experiment, let's hope it doesn't go wrong)
<dimitern> noodles775: but lbox submit won't work, because goose also uses tarmac
<jam> dimitern, noodles775: https://codereview.appspot.com/12897044/ looks fine to land, I can mark it as such if you want.
<dimitern> noodles775: so you'll need to do it the same way as for juju-core - set commit message on the MP and mark it approved
<jam> dimitern: I'm not sure if noodles775 is in the right group to approve his own MPs.
<jam> but if he is, more power to him :)
<dimitern> jam: ah, right, that might be the case
<noodles775> Nope, I'm not.
<jam> noodles775: I just did it
<noodles775> Thanks jam
<axw> jam: "This is the email I want to send to juju-dev" - sent to juju-dev? :)
<jam> axw: I had run a preview copy of it by others, forgot to remove that line :)
<axw> heh ok
<dimitern> jam: :) busted!
<dimitern> fwereade: hey
<fwereade> dimitern, heyhey
<dimitern> fwereade: I have a review for you if you have 10m ?
<jam> fwereade: heyheyheyhey
<fwereade> jam, let's not get geometric here
<fwereade> dimitern, sure
<dimitern> fwereade: roger and I discussed it on friday, he raised some concerns, but in the end I think we decided it's not that bad to land
<dimitern> fwereade: https://codereview.appspot.com/13055043/
<dimitern> fwereade: his concerns were about properly deleting settings keys and not doing a mongo read+write for every write, possibly having a RelationUnit.WriteSettings call that bypasses the state.Settings caching magic altogether
<TheMue> *phew* many mails, many code changes - and that in only two weeks ;)
<dimitern> TheMue: hey! welcome back :)
<fwereade> dimitern, sorry, offhand, what's a params.Settings?
<rogpeppe>   mornin' all
<TheMue> dimitern: heya
<dimitern> fwereade: map[string]interface{}
<TheMue> rogpeppe: morning
<dimitern> rogpeppe: morning
<rogpeppe> fwereade: i wondered about that too
<fwereade> dimitern, rogpeppe: needs to be map[string]string for relation settings, doesn't it?
<rogpeppe> fwereade: i suggested just using a charm.Settings
<dimitern> rogpeppe: could you send you draft comments if you have them?
<fwereade> dimitern, rogpeppe: whoa, what's that for? charm settings can hold non-strings, can't they?
<rogpeppe> fwereade: that actually seems like a better idea
<fwereade> dimitern, rogpeppe: a possible reason to use m[s]i is to allow nil for delete though
<dimitern> fwereade: so coerce everything to a string on the settings when sending?
<rogpeppe> fwereade: alternatively just treat empty string as delete
<dimitern> fwereade: yeah, rogpeppe suggested that
<rogpeppe> fwereade: as, presumably, the uniter does anyway?
<fwereade> dimitern, they all have to be strings anyway
<rogpeppe> yeah, there's no settings schema
<fwereade> rogpeppe, dimitern: empty string == delete is probably sensible, it's effectively what we do anyway
<TheMue> rogpeppe: empty string as delete reminds me of options. but then empty strings never can be used as valid value. is this the case here?
<dimitern> fwereade: rogpeppe was also concerned that some charms might send set huge settings once and change smaller ones often, and with the API implementation we send all changes on Write(), so it might incurr more bandwidth
<rogpeppe> TheMue: config options are different because there's a schema
<TheMue> rogpeppe: yep, only question is, if an empty string could be a valid value here too?
<rogpeppe> fwereade: yeah, i thought that we could apply the same intelligence client-side, so we could write only the settings that have changed
<rogpeppe> TheMue: it can, yes
<dimitern> fwereade: maybe after you review it we can gather for a quick g+ with rogpeppe?
<rogpeppe> TheMue: but as a charm, you can't tell the difference between empty string and deleted
<fwereade> dimitern, rogpeppe: I don't quite understand that one, we should definitely do that, but I don't see how that's relevant to this CL
<rogpeppe> fwereade: it isn't
<fwereade> dimitern, rogpeppe: isn't that purely about the client-side implementation?
<rogpeppe> fwereade: well, it depends
<rogpeppe> fwereade: on the semantics we give to the write operation
<dimitern> fwereade: well, it the s-side endpoint is just WriteSettings, there's no incremental Update
<rogpeppe> fwereade: dimitern's original thought was to have WriteSettings delete all unmentioned attrs
<fwereade> well, yeah, I'd rather call it update if that's what it does ;p
<fwereade> rogpeppe, -1000
<rogpeppe> fwereade: agreed
<dimitern> fwereade: so we need WriteSettings which overwrites and UpdateSettings that deletes empty keys and only updates the rest?
<fwereade> dimitern, that breaks really hard as soon as there's more than one source of relation unit settings events
<fwereade> dimitern, and addressability stuff is very likely to cause that
<rogpeppe> fwereade: hmm, is that possible?
<rogpeppe> fwereade: ah, interesting point
<fwereade> dimitern, I don't think there's any reason for Write when Update exists
<dimitern> fwereade: I don't quite get that one
<rogpeppe> fwereade: +1
<dimitern> fwereade: still g+?
<fwereade> dimitern, when machine addressses change, we'll need to write them into unit settings
<rogpeppe> dimitern: i'm probably happier on IRC tbh as my network connectivity isn't great here
<dimitern> fwereade: ah
<dimitern> fwereade: so you're saying rename WriteSettings to UpdateSettings
<dimitern> fwereade: also: treat empty keys as deletes and still update the rest
<rogpeppe> i think WriteSettings is probably still ok
<fwereade> rogpeppe, well, I'd kinda prefer Update
 * noodles775 wonders if it's normal for tarmac to take 20mins to pick up an approved branch?
<rogpeppe> noodles775: have you set the commit msg?
<fwereade> noodles775, not unheard of especially if others are approved, but what rogpeppe said :)
<rogpeppe> fwereade: fair enough
<dimitern> noodles775: it could happen at times if there are a lot queued up
<dimitern> noodles775: but it doesn't seem to be the case
<rogpeppe> fwereade: i'm fine with UpdateSettings too
<fwereade> rogpeppe, cool
<dimitern> fwereade: please leave comments on what needs to change, it'll be easier for me that way
<fwereade> dimitern, sure, just doing that
<dimitern> fwereade: cheers
<fwereade> dimitern, needed to discuss first though :)
<dimitern> fwereade: sure :)
<jam> noodles775: so by your "10s timeout" comment. Does that mean the machine still hasn't showed up after 10s? Or is this that HP treats the machine as BUILDING(something) which we should probably just treat as BUILD?
<dimitern> fwereade: also, are you ok with having a mongo read for every write as it is now? Should I add a TODO to optimize this if we find it inefficient?
<jam> noodles775: It normally takes about 15min to run the test suite.
<noodles775> rogpeppe, fwereade: Yep, commit msg set, (it's a goose branch) https://code.launchpad.net/~michael.nelson/goose/test-instance-state/+merge/180788
<dimitern> rogpeppe: yeah, we really should land that change on goamz that reduces environs/ec2 tests from 3m to 6s
<rogpeppe> dimitern: it's landed
<fwereade> dimitern, my gut says "meh, fix it when it's a problem"
<dimitern> rogpeppe: cool! haven't tried yet
<rogpeppe> dimitern: but i haven't update juju-core to use it yet
<dimitern> fwereade: ok, I'd still like to add a TODO just as a reminder lest we forget
<dimitern> rogpeppe: ahh
<fwereade> dimitern, fair enough
<jam> noodles775: ah, for goose it should be faster, but if it got queued behind a juju-core branch. I'll check what the bot is up to.
<noodles775> jam: It's a goose branch (local tests run in a few seconds). RE the HP bug, I still need to update that juju-core branch once the goose branch lands (with the dependency), but yes, the machine turns up as BUILD(spawning) after 8-9 seconds and status continues to success (once mongod is up etc.).
<noodles775> :)
<rogpeppe> dimitern: i agree. there's no point in reading a Settings object when we know exactly what updates we need to perform without reading anything
<dimitern> rogpeppe: but not for now
<noodles775> jam: I'll run a bootstrap 10 times or so, and if it's always around the 8-9 second mark, I'll update shortAttempt to 15s?
<fwereade> dimitern, rogpeppe, my only problem is with half-fixing it, because IIRC Settings is still kinda crack
<rogpeppe> dimitern: yeah, as we agreed
<jam> noodles775: so the machine doesn't show up at all before 8-9s ?
<rogpeppe> fwereade: we'd bypass Settings entirely
<rogpeppe> fwereade: and have a RelationUnit.WriteSettings, or something
<dimitern> fwereade: it's one of the crackiest for sure :)
<noodles775> jam: it does - shows up as BUILD(networking), but if you continue from that point we never connect to mongod.
 * noodles775 finds paste.
<rogpeppe> fwereade: RelationUnit.UpdateSettings
<dimitern> jam: HP just uses more fine-grained BUILD status codes
<fwereade> rogpeppe, hmm, something's bugging me about that
<noodles775> jam: previous conversation about that (with the link to the paste): http://irclogs.ubuntu.com/2013/08/16/%23juju-dev.html#t07:24
<rogpeppe> fwereade: oh, go on
<fwereade> rogpeppe, I'm trying to figure out what it is
<jam> noodles775: your patch was merged, but didn't get pushed up properly, fixing now.
<dimitern> noodles775: nice! I didn't know we have such nice logs :)
<rogpeppe> fwereade: BTW i did that audit of all the charms i could find in the charm store for relation names
<rogpeppe> fwereade: nothing funky
<fwereade> rogpeppe, oh yes?
 * fwereade cheers
<rogpeppe> fwereade: all quite normal
<rogpeppe> fwereade: in the process i made a little command for downloading all charms:
<rogpeppe> fwereade: launchpad.net/juju-utils/cmd/getallcharms
 * fwereade dances, sings, was fully expecting someone to be insane
<fwereade> rogpeppe, isn't there a `charm getall` or something already?
<rogpeppe> fwereade: and an even smaller command for iterating over all the metadata of all charms in a directory
<fwereade> rogpeppe, (not in juju)
<dimitern> fwereade: yep, it seems we need something like [a-z][a-z0-9]*(-[a-z0-9]+)*
<fwereade> rogpeppe, that sounds nice
<rogpeppe> fwereade: oh, yeah, there is a getall :-)
<rogpeppe> fwereade: launchpad.net/juju-utils/cmd/charmmeta
<fwereade> rogpeppe, that's cool, thanks
<rogpeppe> fwereade: i hadn't seen "charm getall", but my getallcharms will be much faster, i think
<rogpeppe> fwereade: as it downloads bundles directly from the charm store
<fwereade> rogpeppe, nice
<rogpeppe> fwereade: it actually doesn't take long to download them all - try it
<rogpeppe> !
<fwereade> rogpeppe, dimitern: RelationUnitPair is interesting
<fwereade> rogpeppe, dimitern: the LocalUnit echoes all the others, but should not really be required
<rogpeppe> fwereade: i think i suggested that
<fwereade> rogpeppe, dimitern: whether or not RemoteUnit is accessible is a permissions question
<fwereade> rogpeppe, dimitern: *but* if the local unit is inferred internally in that call, we should probably be doing so in all the others, and that feels out of whack itself
<dimitern> fwereade: exactly!!
<dimitern> fwereade: we shouldn't do that
<rogpeppe> fwereade: personally i think that would be fine, but i realise you like those bulk calls
<dimitern> fwereade: otherwise we might get to "why we need bulk ops at all when we infer what to return by the auth'ed entity alone?"
<fwereade> rogpeppe, dimitern: however I *think* there's a disticntion in this case
<rogpeppe> dimitern: ooh, don't ask that question! that way lies madness.
<fwereade> rogpeppe, dimitern: it's only required in this case because of how state is implemented
<fwereade> rogpeppe, ;p
<dimitern> fwereade: so you're saying let's open the door there
<dimitern> :)
<fwereade> rogpeppe, dimitern: it's a side effect of a weird underlying implementation
<rogpeppe> fwereade: the fact that we don't know what entity the calls are being executed on behalf of?
<rogpeppe> fwereade: (in state, that is)
<dimitern> rogpeppe: of couse we know - it's still the same unit, just there's a relation involved as well
<fwereade> rogpeppe, dimitern: I think the symetry makes it ok, but we need to be sure to translate permissions errors from ReadSettings into api permissions errors
<dimitern> fwereade: sure
<rogpeppe> fwereade: you mean not-found errors, presumably?
<fwereade> rogpeppe: I think I mean permission
<rogpeppe> fwereade: i'm not sure we'd need to do that, would we?
<rogpeppe> fwereade: oh... one mo
<rogpeppe> fwereade: i'm not sure what you mean.
<fwereade> rogpeppe, ha, it's that ReadSettings bug we already discussed
<rogpeppe> fwereade: isn't ReadSettings itself an API call, so it can just return an ErrPerm
<fwereade> rogpeppe, so NotFound is one case
<fwereade> rogpeppe, that should indeed be translated
<rogpeppe> fwereade: i'm not sure actually
<fwereade> rogpeppe, but ReadSettings should itself barf if you try to ask for a unit on the wrong side of the relation
<rogpeppe> fwereade: i'm a bit wary of this blanket "translate not-found in to perm-denied" thing
<rogpeppe> fwereade: how can a unit ask for its own settings then?
<fwereade> rogpeppe, it mustn't!
<fwereade> rogpeppe, ah
<fwereade> hmm
<fwereade> rogpeppe, its own I guess is ok, hadn't thought through the shape of the API
<fwereade> rogpeppe, but it dfinitely shouldn't ask for its siblings
<rogpeppe> fwereade: yeah, i thought its own is ok, but not any siblings, yeah
<fwereade> rogpeppe, *at the moment* ReadSettings is never called for itself
<fwereade> rogpeppe, dimitern: I'm at least a little inclined to suggest that maybe reading one's own settings and reading one's relatives' are different enough operations to want to be separate calls
<fwereade> rogpeppe, dimitern: tell me why that's crazy
<rogpeppe> fwereade: i thought that they both read the settings of a unit and have exactly the same params
<rogpeppe> fwereade: so why not have them as the same call
<fwereade> rogpeppe, because it complicates the permissions to a maybe unhealthy degree
<fwereade> rogpeppe, and because RemoteUnit doesn't really fit in the self case
<rogpeppe> fwereade: if we omit LocalUnit (as i think we've now agreed) it can just be "Unit"
<fwereade> rogpeppe, I didn't think we'd done that
<rogpeppe> fwereade: ah, i misunderstood then
<fwereade> rogpeppe, sorry
<rogpeppe> fwereade: when you said "the symmetry makes it ok" i was assuming the opposite to what you meant, i think
<fwereade> rogpeppe, dimitern: how would you real about the equivalent of ReadSettings(rel, unit) and ReadRelatedSettings(rel, unit, remote)
<fwereade> ?
<fwereade> s/real/feel/
 * fwereade wonders how his brain generated that typo
<fwereade> TheMue, not sure I said hi, welcome back
<fwereade> mgz, morning
<rogpeppe> fwereade: you meant "the symmetry makes it ok to have a bulk call for ReadSettings, i'm now presuming?
<fwereade> rogpeppe, sorry, yeah
<mgz> fwereade: hey!
<rogpeppe> fwereade: ah. oh well :-|
<fwereade> rogpeppe, remember that we're open to just dropping those params across the board one day and nothing'll break ;p
<rogpeppe> fwereade: so... let's just drop 'em now!
<fwereade> rogpeppe, but if we're going to do that I would like to do so consistently
<rogpeppe> fwereade: well, we can't really drop the params across the board without being left with calls that still take slice of params, but aren't actually bulk calls.
<fwereade> rogpeppe, and there are definite implementation advantages to including the caller at the moment, in terms of shared passthrough implementations
<rogpeppe> fwereade: but i suppose historical artifacts like that are par for the course in aging APIs
<TheMue> fwereade: not yet, but heya back from me too :D
<rogpeppe> fwereade: i agree to an extent about the shared impl thing (it works well for Life for example)
<fwereade> rogpeppe, dimitern: so, how about two calls, including the remote unit only when asking for a remote unit?
<fwereade> rogpeppe, dimitern: they're used in rather different contexts
<rogpeppe> fwereade: i'm not quite sure what you're suggesting.
<fwereade> rogpeppe, ReadSettings(relation, unit); ReadRelatedSettings(relation, unit, remote)
<dimitern> fwereade, rogpeppe: sorry, got a little side-tracked, catching up with the backlog
<rogpeppe> fwereade: i still think it's weird having two calls there where we could have one, but there y'go
<dimitern> fwereade, rogpeppe: RS() and RRS() sgtm - I had originally done them like that
<rogpeppe> dimitern: yeah, i'm sorry for suggesting you do it differently
<dimitern> rogpeppe: no worries, it was for different reasons iirc
<dimitern> fwereade: so how about permissions in the "own settings" case?
<rogpeppe> dimitern: similar reasons really - i didn't see why we had two calls when we could use one
<fwereade> dimitern, not sure what you're asking re permissions? you only get to RS your own, you only get to RRS others'
<dimitern> fwereade: so no notfound->errPerm translation for your own settings, assuming you're auth'ed for that unit-tag ?
<fwereade> dimitern, I hadn't really been thinking about your own
<fwereade> dimitern, for remote settings, it would be ideal if we could figure out the permissions errors before even needing to read them (although NotFound->ErrPerm for supposedly-accessible ones that don't exist also sounds sensible)
<dimitern> fwereade: not sure how we can know which units are supposed to be related to your unit
<fwereade> dimitern, is it a member of the other service in the relation? then it's ok
<dimitern> fwereade: I mean, from the relation you can get all units somehow, but how to know which ones you are supposed to be able to see or not
<fwereade> dimitern, visibility is I think purely service-level here
<dimitern> fwereade: so 1) we get the RU from the given rel-id + our unit (the auth'ed), 2) get the remote unit, its service and get the related endpoints for it from the relation
<dimitern> fwereade: if we have any related endpoints, we're fine
<fwereade> dimitern, if you want to be pedantic, you should beable to read the settings of any unit that's a member of a service in the relation, so long as (1) the service's role is related to your own and (2) the unit is not the one asking
<fwereade> sorry (2) was not expressed well; was meaning clear anyway?
<dimitern> fwereade: mine 2 or your 2 ? :)
<fwereade> dimitern, mine, I'm trying to see if I can simplify yours a bit
<fwereade> dimitern, we could ofc just translate notfounds at the api level
<dimitern> fwereade: so let me rephrase it as I understood it
<fwereade> dimitern, and separately fix ReadSettings itself to inject fake NotFounds if you're trying to read sibling data
<dimitern> fwereade: 1) the service's role is related: that's what RelatedEndoints(serviceName) returns anyway
<fwereade> dimitern, ah! bingo, yeah, I'd forgotten that method
<dimitern> fwereade: so any results there - we can proceed
<jam> noodles775: sorry about the delay (lunch). I read over the context a bit, does the IP address change after BUILD(networking) into BUILD(spawning) ?
<dimitern> fwereade: we can pretty much filter 2) out before trying 1)
<fwereade> dimitern, sounds sane I think
<dimitern> fwereade: e.g. GetAuthTag() != thisEntityWeReChecking.Tag
<fwereade> dimitern, yeah
<dimitern> fwereade: ok, sgtm, thanks
<dimitern> fwereade: did you notice my sleazy InScope hack?
<fwereade> dimitern, that seemed fine to me, if anything it should probably have gone further -- would potentially simplify a bunch of scope tests in state, I think
<dimitern> fwereade: :) wasn't sure you'd like it, but I couldn't make the tests pass with a RelationScopeWatcher through the API and it seemed too much hassle anyway
<noodles775> jam: I've not looked into why it fails to connect if we continue from BUILD(networking). I can check though.
<fwereade> dimitern, one day I think I'd like a Relations() method returning all RUs for which the unit is in scope
<fwereade> dimitern, that'd fix the local-relation-state problem in uniter
<dimitern> fwereade: ah, that sounds nice
<jam> axw: bug #1213832 I don't know of  an 'upgrade-tools'. Do you mean 'juju bootstrap --upload-tools" or "juju sync-tools" or ?
<_mup_> Bug #1213832: Intermittent errors during upgrade-juju with local provider <juju-core:New> <https://launchpad.net/bugs/1213832>
<jam> axw: (I'm guesing --upload). I'll also just check that you're using go 1.1
<axw> yes go 1.1, and sorry that was a typo
<axw> the real command is in the one liner
<axw> I'll remove the bit about me isolating it, because that's bunk
<dimitern> fwereade: so my plan for today is, to land this CL, then add a skeleton of the client-side API with all files, no tests and all TODOs written in them. If I manage, I'd like to do the unit calls as well, but might not manage
<dimitern> fwereade: I moved to the new place and need to sort out some stuff this afternoon
<fwereade> dimitern, <3, I'll just do a quick skip through the CL for anything I've missed
<dimitern> fwereade: thanks
<dimitern> fwereade: and also, the car really helped with moving all the stuff, thanks again!
<fwereade> dimitern, cool, glad it came in handy :)
<fwereade> dimitern, oh, just a thought, when will you be back again?
<dimitern> fwereade: on the 30th
<noodles775> jam: Yes, it gets a second private address after the initial BUILD(networking) but before BUILD(spawning), and it's that address which is used to connect to mongod: http://paste.ubuntu.com/6002242/
<dimitern> fwereade: updated the team calendar
<fwereade> dimitern, sweet, cath will need the keys again the following day sometime )
<jam> noodles775: second "private" which is actually the public one.
<jam> I see BUILD(scheduling) right away
<jam> and then BUILD(networking), etc.
<jam> mgz: ^^ would it make sense to included re-querying HP at a logical time during 'first status' ?
<dimitern> fwereade: well, technically on the 30th I'm still off, but I'll drop by to give you the keys; or you could come and see the new flat :)
<noodles775> jam: yep, the full set are listed here: http://docs.hpcloud.com/api/compute#ServerStates
<fwereade> dimitern, sounds good, ping me when you're back and I'll come round
<dimitern> fwereade: sure thing
<noodles775> jam: just in case, but pls don't update the status of that MP yet, I'm still adding a test.
<jam> noodles775: so.... I'd like us to handle more BUILD states, and I'd really like us to handle address changes, but that is certainly out of scope for what you are doing.
<noodles775> yep.
<mgz> jam: yes, hp does have its own little world of statuses
<jam> mgz: so we can handle treating BUILD* as BUILD. but the question is how will the addressing changes poke at stuff like this.
<jam> noodles775: I've even seen BUILD(networking) that for 1s had only private address, and then in another second had a public address.
<mgz> well, it's good to know that it doesn't go from having no addresses to having all in one step
<jam> Presumably it would have to have the public address at the point we get to spawning.
<mgz> as the obvious poll logic is to stop when a macine has anything
<jam> mgz: BUILD(scheduling) no addresses, BUILD(networking) 10.* address, BUILD(networking) 10.* + 15.*, BUILD(spawning) 10.*+15.*
<jam> that is from doing "watch -n1 nova list" as I do a bootstrap.
<noodles775> Similar here with a log.Infof: http://paste.ubuntu.com/6002242/
<jam> mgz: well we know that on HP we must end up with 2? Could we hack that in?
<mgz> well, the likely adptation is to force one more address check at a later boot stage, either from machine status, or when the machine is up and the machine agent reports in
<dimitern> jam, noodles775: I believe an update to goose on the bot is in order due to that change?
<jam> maybe something in DNSName that knows to wait until we have a public address?
<jam> dimitern: goose is updated on the bot from noodles landing the patch.
<dimitern> jam: perhaps even a juju-dev email
<mgz> we're not guarenteed to get a public address on boot, is one thing there
<jam> dimitern: well I don't think noodles patch depending on the goose change has landed.
<noodles775> dimitern: we can email if/when the juju-core branch lands right?
<jam> mgz: not guaranteed by who?
<jam> openstack?
<jam> or by HP
<jam> ?
<dimitern> noodles775: right
<fwereade> dimitern, reviewed -- but, one thing, you shouldn't send the settings up in EnterScope
<mgz> clouds in general. hp and aws do give you a public address
<fwereade> dimitern, we can keep that behind the api
<dimitern> fwereade: you mean always call EnterScope with nil?
<fwereade> dimitern, I mean get the private address inside the API version of EnterScope
<fwereade> dimitern, we still need to put it in, but the uniter doesn't need to worry about it any more I think
<dimitern> fwereade: not sure I get you
<fwereade> dimitern, inside the API EnterScope method, you have the unit available
<fwereade> dimitern, construct the settings map there
<fwereade> dimitern, don't require that the uniter send it up
<dimitern> fwereade: and drop the settings field from the args?
<fwereade> dimitern, yeah, please
<dimitern> fwereade: how should the map look like when constructed?
<jam> mgz: "clouds in general" sure, but we could just require it, and have people do stuff like use-floating-ip:true or sshuttle. I suppose if our polling said "10.* doesn't count as routable" but then you are using sshuttle, then we aren't playing nicely
<dimitern> fwereade: "private-address": unit.PrivateAddress() ?
<jam> mgz: note that elmo specifically feels that we should be using floating ips by default on canonistack now.
<fwereade> dimitern, {"private-address": <unit's private address>}
<jam> as they got (2?) /24 blocks for it.
<dimitern> fwereade: ok
<fwereade> jam, mgz: isn't that a matter of using an existing config setting?
<dimitern> the use-floating-ips is false by default
<jam> fwereade: use-floating-ip is, the main discussion about "how can we poll appropriately to be sure we are actually getting a routable address for 'juju status'" is the other question.
<jam> fwereade: and the root discussion there
<jam> fwereade: is that HP will show us the machine before it has finished building
<mgz> I guess my point is it's simple enough to do the right thing for hp without stopping juju from working on more basic setups
<fwereade> jam, got you, just confiirming I remembered use-floating-ips accurately :)
<jam> and will give it a private IP address that we don't want to treat as canonical location
<jam> until we get the public address.
<jam> mgz: which is don't treat a machine as alive until it gets to BUILD(spawning) ?
<mgz> jam: for addresses? it'd just be check again later as well.
<jam> noodles775: yeah, the only state here http://docs.hpcloud.com/api/compute#ServerStates I haven't seen is "BUILD(block_device_mapping)" but since we don't mount devices, we won't really see that one :)
<jam> mgz: when ?
<mgz> at some point after BUILD
<jam> mgz: so looking at the openstack/provider.go code, we just always select the last of the list of private IPs. Which happens to work on HP?
<jam> SelectPublicAddress seems to try to find an address labeled public, otherwise it overwrites "mostpublic" with each entry in the list until it gets to the end.
<mgz> jam: yup, it's a cute hp-influenced hack
<mgz> there's not a slightly fancier way of doing that, which I've not yet committed
<jam> mgz: sort with 10.* and 127.* treated as more local?
<jam> (less public)
<mgz> auto-detect private addresses as private, yeah
<jam> noodles775: so after diving into this, treating BUILD* as BUILD would be ~ok, but as a short-term just treating BUILD(spawning) as BUILD would be a great step forward.
<noodles775> jam: k - I'll finish adding my test and re-propose. Thanks.
 * fwereade coffee, bbiab
 * TheMue => lunchtime
<noodles775> dimitern, jam: I've updated the description with a bit of info (in particular, one thing which I couldn't find a way to test which I would like to test - let me know if you've ideas): https://codereview.appspot.com/12795045/
 * noodles775 -> lunch
<rogpeppe> fwereade, dimitern, TheMue, mgz: huge but trivial: https://codereview.appspot.com/12940044
<dimitern> rogpeppe: great! will have a look after the standup
<fwereade> rogpeppe, if it's really just the gc fix, LGTM sight unseen
<rogpeppe> fwereade: it is, yeah
<rogpeppe> fwereade: i hacked up some code to make the change automatically
<fwereade> rogpeppe, nice
<TheMue> fwereade: trust as LGTM, nce :D
<TheMue> rogpeppe: automated change?
<rogpeppe> TheMue: yeah
<TheMue> rogpeppe: ok
<jam> TheMue: https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471 standup?
<mgz> rog: try the lowest bandwidth mode if you've not yet
<rogpeppe> mgz: i am using that
<rogpeppe> mgz: (audio only)
<rogpeppe> mgz: it all *started* quite well!
<TheMue> jam: oh, shit, yes
<rogpeppe> jam, fwereade: apologies. it looks like i can't do this.
<jam> rogpeppe: np
<fwereade> rogpeppe, np
<jam> you can just text what you'reup to on IRC
<jam> if there's anything
<rogpeppe> jam: i'm getting 10-of-a-second bursts of audio
<rogpeppe> 10th-of-a-second
<rogpeppe> ah, i hear dimitern!
<rogpeppe> mgz: was there some problem with me submitting that enormous branch?
<mgz> rogpeppe: no, I was just joking that I wanted to land my branches first, rather than have to deal with conflicts :)
<rogpeppe> mgz: too late, sorry!
<mgz> gah, and I get kicked
<dimitern> jam, TheMue, rogpeppe, mgz, natefinch: fwereade just texted me he has some issues with this connection and will try to resolve them, but he'll be doing "some damn programming" :) there's also a bug assigned to him which he'll work on
<mgz> :)
<rogpeppe1> gah! my laptop just crashed
<dimitern> it's a bad case of mondays
<rogpeppe1> dimitern, mgz: more conflict fuel:  https://codereview.appspot.com/13105043
<natefinch> man.. I don't know if it's my wifi or my internet connection, but I've been getting like 500kbps upload speed, when my connection is supposed to be 25 megs up :/
<rogpeppe1> natefinch: that's as good as i ever get! :-)
<natefinch> rogpeppe1: wow, I'm sorry :)
<dimitern> rogpeppe1: LGTMed
<rogpeppe1> dimitern: ta
<dimitern> rogpeppe1: how is the last batch coming along?
<rogpeppe1> dimitern: i was just waiting for 2nd batch to merge :-)
<rogpeppe1> dimitern: will propose last batch now
<dimitern> rogpeppe1: great! I wanted to wait for all of them to land before I branch off
<dimitern> rogpeppe1: probably the last bit of massive automatic changes like that is to make sure all imports everywhere are grouped
<rogpeppe1> dimitern: just resolving conflicts in relationunit_test.go
<dimitern> rogpeppe1: yep, sorry about that - they are mostly just additions
<rogpeppe1> dimitern: yeah, but additions i need to change all of, and there are quite a few! (that's fine though, par for the course)
<dimitern> rogpeppe1: you're running all the tests before proposing, right? saves a bit of time from having to go back and fix stuff the bot bounced
<rogpeppe1> dimitern: tbh i just made sure everything compiled
<rogpeppe1> dimitern: although i'm running the tests where i've manually fixed conflicts
<dimitern> rogpeppe1: ok
<rogpeppe1> dimitern: last one: https://codereview.appspot.com/13105044
<dimitern> rogpeppe1: LGTM
<rogpeppe1> dimitern: ta
<rogpeppe1> lunch
<noodles775> This is ready for Status->Approved if someone has a moment: https://code.launchpad.net/~michael.nelson/juju-core/1208504-post-bootstrap-hp-no-instances-found-try2/+merge/179899
<dimitern> noodles775: please always include a link to the CL on rietveld when setting the commit message
<dimitern> noodles775: also in this specific case I think a bit more is needed in the message, although surely not the full CL description
<noodles775> dimitern: Yeah - I found it strange that `lbox submit` (which I've only continued to use because it does everything up until changing the status) copies the MP description to the commit msg (which was way to big), so I manually updated the MP. Let me update the commit msg again.
<dimitern> noodles775: instead of lbox submit, you can just use lbox propose to do all the work and the change the commit message on the MP
<noodles775> dimitern: k, will do. Updated - let me know if you still want further changes (or you can probably update the commit msg yourself?).
<dimitern> noodles775: thanks, i'll just quickly go through it
<dimitern> noodles775: a couple of comments
<rogpeppe1> noodles775: in general, i really like having the whole codereview description in the commit msg
<rogpeppe1> noodles775: it means you can look at the commit log and see all the relevant trunk changes and links to their code review
<noodles775> rogpeppe1: oh - OK, I'll do that then (makes it easier - I just thought people would think it noise).
<dimitern> noodles775: otherwise lgtm, but you already have one, so it's ok
<sidnei> rogpeppe1: dimitern: any clues on this one: https://pastebin.canonical.com/96004/
<noodles775> heh, thanks dimitern. I'll update after the call and ping you for the status update.
<noodles775> s/the/a/
<rogpeppe1> sidnei: looking
<rogpeppe1> sidnei: hmm, is that happening repeatedly?
<sidnei> rogpeppe1: as in every x secs? let me ask pedronis. it happened with local provider after rebooting the machine.
<rogpeppe1> sidnei: ah, that makes sense perhaps
<rogpeppe1> sidnei: i guess the socket has survived the machine rebooting (do unix domain sockets do that?)
<sidnei> maaaybe?
<sidnei> which would explain a different issue im having with a charmed service that uses unix sockets (supervisord)
<rogpeppe1> sidnei: i'm slightly surprised actually, if that's what's happening
<sidnei> rogpeppe1: confirmed with pedronis it's printed repeatedly
<rogpeppe1> sidnei: hmm, he could try removing it
<rogpeppe1> sidnei: that's perhaps what the code should be doing, becasue EADDRINUSE should never happen in normal circumstances for a uniter
<rogpeppe1> fwereade: what do you think?
 * fwereade reads back a mo
<fwereade> rogpeppe1, sidnei: I think davecheney hit that the other day, there's a bug, proposed fix looks sane... just a mo...
<fwereade> rogpeppe1, sidnei: https://bugs.launchpad.net/juju-core/+bug/1212916
<_mup_> Bug #1212916: uniter/worker: unit agent MUST remove stale unix socket <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1212916>
<rogpeppe1> more bikeshed colour suggestions anyone? https://codereview.appspot.com/13053043/
<fwereade> rogpeppe1, it should obviously be puce
<rogpeppe1> fwereade: using the @ namespace sounds sane (insofar as unix sockets are sane at all, which is a difficult call)
 * rogpeppe1 shall try to avoid using names starting with '@' in the future (WTF?!)
<rogpeppe1> fwereade: it is now indeed a delicate shade of puce
<noodles775> dimitern: Updated with your changes. Pls approve when you've time. Thanks!
<noodles775> dimitern: btw, do you or someone from juju-core want to send out the email re updating goose, otherwise I can send it.
<dimitern> noodles775: thanks; approved
<dimitern> noodles775: I'd appreciate if you send the mail please
<dimitern> noodles775: i'm elbow-deep in the API now :)
<noodles775> dimitern: sure thing.
<noodles775> dimitern: sorry, did you possibly have the Launchpad MP already open when you approved it? (It's complaining that there are new revs since you approved). Can you pls refresh and re-approve?
<dimitern> noodles775: I did! sorry, will do
<dimitern> noodles775: approved
<mgz> noodles775: the approve that matters for that error is the mark on the proposal, not the vote, so setting back to needs review then approved again is all that's needed
<mgz> (or are you not in ~juju yet?)
<noodles775> mgz: I'm not - hence the ping :)
<mgz> well, we should fix that :)
 * noodles775 hears dimitern breathing a sigh of relief.
<dimitern> noodles775: uh? sorry missed that
<dimitern> noodles775: :)
<noodles775> dimitern: heh, I just meant I won't need to ping you to update the status :)
<dimitern> noodles775: ah, ok :) no trouble at all
<mgz> niemeyer: can you add ~michael.nelson to ~juju please? :)
<fwereade> dammit, my mgo-munging mental structures have atrophied
<fwereade> or possibly they've matured, and have allowed me to realise something's harder than I thought
<fwereade> grar
<mgz> you have not taken your medicine enough recently
 * fwereade ponders the wisom of afternoon caffeine
<niemeyer> mgz: Sure
<rogpeppe1> my network connectivity is likely to be dodgy for the next couple of hours
<geme_> I've just deployed the tomcat6 charm on a private openstack instance and port 8080 hasn't been added to the security group rules after the service has been exposed. Is this a known problem ?
<noodles775> Do I need another LGTM after re-merging trunk and updating to fix some test failures? (failures due to moved code/new namespaces in trunk) Rev 1662 at https://code.launchpad.net/~michael.nelson/juju-core/1208504-post-bootstrap-hp-no-instances-found-try2/+merge/179899
<mgz> geme_: seems the charm doesn't call open-port at all
<mgz> geme_: maybe try the branch linked in bug 1015202 instead?
<_mup_> Bug #1015202: new-charm Tomcat <Juju Charms Collection:New for negronjl> <https://launchpad.net/bugs/1015202>
<mgz> that actually calls open-port
<noodles775> mgz: care to re-Approve my MP ^^ (seems I'm not yet part of ~juju)
<mgz> noodles775: done
<mgz> niemeyer: ^ poke? thanks.
<geme_> mgz: ok
<geme_> mgz: do you have a deploy url for that ? The link reporting page not found
<mgz> geme_: ? `bzr branch` the url listed in the bug report, and deploy from local
<dimitern> rogpeppe1, fwereade, jam: for consideration https://codereview.appspot.com/13107043
<rogpeppe1> dimitern: looking
<dimitern> rogpeppe1: it's not final, some things will probably still change along the way, but I think it's good to have a roadmap-in-code like this
<rogpeppe1> dimitern: erm, *would* look, if i had more than itty bitty connectivity
<rogpeppe1> dimitern: i think i know what the CL is likely to be, and i approve :-)
<dimitern> rogpeppe1: cheers :)
<dimitern> rogpeppe1: it's mostly stuf like this http://paste.ubuntu.com/6003214/
<dimitern> rogpeppe1: does that open?
<rogpeppe1> dimitern: yup, i can see that, thanks
<rogpeppe1> dimitern: i thought as much
<geme_> mgz: Could you repeat the tomcat6 bug report again pls. Chat session drop out
<dimitern> rogpeppe1: can you at least send the LGTM? or rietveld is not opening for you
<mgz> geme_: bug 1015202
<_mup_> Bug #1015202: new-charm Tomcat <Juju Charms Collection:New for negronjl> <https://launchpad.net/bugs/1015202>
<rogpeppe1> dimitern: i'd like to review the API proposal properly before LGTMing
<dimitern> rogpeppe1: sure
<dimitern> rogpeppe1: no rush
 * dimitern gtg, back here in a couple of hours
<rogpeppe1> dimitern: i'll do that when i get back home in a couple of hours
<niemeyer> mgz: Done
<niemeyer> mgz: Btw, William is an admin as well
<fwereade> dimitern, LGTM, just a few extra notes that'd be worth adding
<natefinch> I don't understand... how can c.Check(foo, <checker>, bar) AND c.Check(foo, gc.Not(<checker>), bar) BOTH fail?  Shouldn't that be impossible?
<mgz> niemeyer: thanks!
<mgz> natefinch: I can think of various cases where that might be the case, not sure which apply in go (funny float values, for instance)
<natefinch> mgz: this is just some pretty boring equals stuff, like length of two slices...
<natefinch> mgz: I wrote a new checker to compare the unordered contents of slices, and the first few checks in the function are just stuff like "are the two slices the same length" ... the code in the function is returning false at the right spot...  but when I wrap it in a gc.Not(), it's still saying the check failed.... with the message I gave from the function
<mgz> natefinch: my guess is then that your checker is borked somehow :)
<natefinch> but... regardless of the code in the function.... it always returns true or false.... and gc.not should flip that, and it's  not somehow
<mgz> geme_: are you having any luck?
<natefinch> mgz: seems like if I put anything in the error string, gc.Not has no effect and it's treated as a failure regardless of what the result is
<natefinch> tested by just putting return false, "something"  at the top of the check function
<mgz> hm.
<natefinch> (same thing with return true, "something"
<natefinch> mgz: I think it's just me misunderstanding the Checker interface... which could use slightly better documentation imo.  I think the error string is for errors in the test that just don't make sense, like nil pointers or comparing different types.  Whereas I was using error as a way to explain why the test failed
<natefinch> mgz: yeah, the code in c.Check is clear, it treats any non-empty error string as a failure
<natefinch> and gc.Not() just passes up the error string from the thing it wraps
<mgz> that sounds somewhat sensible
<natefinch> mgz: docmentation would make it clearer.  I still wish I could control the error msg
<geme_> mgz: I've been using robert ayres tomcat charm previously with the python version. Just setting up these charms.
<fwereade> natefinch, I'm not quite sure, but are you looking for http://godoc.org/launchpad.net/gocheck#Commentf ?
<geme_> mgz: Robert Ayres tomcat charms look good. Security group rules added
<mgz> geme_: ace. maybe you could comment on that bug as well?
<natefinch> fwereade: that looks like something you write at the test site, not inside the test function.
<fwereade> natefinch, ah, sorry, I misunderstood what you were saying
<natefinch> fwereade: I was looking for a way to pass back information from the test as to why it failed, rather than just saying "failed" and printing out the two values, forcing the dev to figure out why those values failed
 * rogpeppe1 is back in the world of wi-fi
<geme_> mgz: sure
<rogpeppe1> done for the day.
<rogpeppe1> g'night all
<thumper> morning folks
<thumper> hi fwereade
<thumper> fwereade: are you perhaps around?
<fwereade> thumper, heyhey
<thumper> hi fwereade
<thumper> fwereade: got time for a quick hangout?
<fwereade> thumper, sure
 * thumper getting pinged all over
<fwereade> thumper, I think I'll be around for a bit if you'd prefer
<thumper> that's fine, I'll start one
<arosales> is there a juju command to create juju tools locally?
<arosales> http://juju-dist.s3.amazonaws.com/
<thumper> arosales: not really
<thumper> arosales: although the local provider does it
<arosales> thumper, ok so I assume your core guys do some magjc to generate tools for aws and hp?
<thumper> if by magic you mean run a script, then yes
<thumper> arosales: although dave probably knows most about that
<arosales> thumper, ok
<arosales> bigjools, around?
<wallyworld_> thumper: z'up
<thumper> hi wallyworld_
<thumper> wallyworld_: how are you doing today?
<thumper> I half thought you wouldn't be around
<wallyworld_> ok. not too sore. a nice scar on the side of my head
<wallyworld_> nah, stuff to do
<wallyworld_> you working on addressability stuff?
<thumper> no, working of refactoring agent.Conf
<thumper> although wondering how deep to hit this
<thumper> several things depend on it
<wallyworld_> i'm not too familiar with that bit of the code base
<wallyworld_> if the change driving towards HA?
<thumper> diving towards a consistent manner of handling "pseudo environment variables", and a more readable and less duplicated agent config file
<thumper> with parts that will be writable
<thumper> like addresses of endpoints
<thumper> and other parts that shouldn't be changed
<thumper> like agent tag
<wallyworld_> sounds nice
<thumper> so will be used for HA bits too
<thumper> and some of the pending work I have
<thumper> with logging
<wallyworld_> +1 for logging :-)
<arosales> bigjools, ping me when you return on juju provider tools
<arosales> https://bugs.launchpad.net/juju-core/+bug/1214181
<_mup_> Bug #1214181: Azure Provider always uploading 1.12 tools <juju-core:New> <https://launchpad.net/bugs/1214181>
#juju-dev 2013-08-20
 * davecheney waves at thumper
<bigjools> arosales: hi
<bigjools> what governs which tools get picked?
<bigjools> that's not provider-specific is it?
<davecheney> bigjools: it is and it isn't
<bigjools> helpful :)
<davecheney> the version is not provider specific, it is determined by the version rules we have
<davecheney> ie, if x.y.z, y % 2 == 0 then we'll look for stable tools with an exact match
<davecheney> if y % 2 != 0 then it's a devel version and we take the latest .z version
<davecheney> however there is a bigger problem
<davecheney> for each provider, there are two locations to look for tools
<bigjools> http://paste.ubuntu.com/6004647/
<davecheney> the public (shared) bucket
<davecheney> and private (control) bucket
<davecheney> if no public bucket exists, or does not match the exct version for released tools
<davecheney> i *think* the current behavior is to invoke the juju sync-tools function
<davecheney> copy all the tools from the master bucket in s3
<davecheney> into your private control bucket
<davecheney> this control bucket will be deleted when you destroy the environment
<bigjools> not sure I really understand tbh
<bigjools> my knowledge of the juju internals is pretty limited
<davecheney> bigjools: two secs, gotta hang out the washing
<davecheney> then I will return and regail you with tails of woe
<bigjools> hanging out the washing ... love it
<bigjools> we've not heavily tested (read: no testing) of using public tools with azure yet
<davecheney> bigjools: +1 for testing
<davecheney> bigjools: i'm probably wrong
<davecheney> but does azure have the concept of a public bucket ?
<bigjools> you can have public containers, if that's the same thing
<bigjools> but I don't think ec2's concepts map directly
<bigjools> nobody uploaded public tools to azure yet I guess
<davecheney> bigjools: nope, if
<davecheney> 1. we do that
<davecheney> 2. provide some sort of link to then in environments.yaml
<davecheney> this will solve the problem
<bigjools> there is config for it
<davecheney> bigjools: so I think what happens is
<davecheney> you manually download the tools from the s3 bucket
<davecheney> then manually upload them to the azure public bucket
<davecheney> you might be able to arrange you environments.yaml to be able to use sync-tools to automate this
<davecheney> wally/jam/gz could probably explain how to do this will less pain
<bigjools> there is no public Canonical storage account in Azure AFAIK
 * bigjools starting to understand
<bigjools> there's config for public-storage-account-name and public-storage-container-name once tools are uploaded
<bigjools> who generally uploads new tools?
<davecheney> bigjools: unknown
<davecheney> it's handled very ad hockly
<bigjools> so when there's a new release, how does it work?
<davecheney> there is a vague requirement that 'tools should be uploaded as part of the release'
<davecheney> but that assumes I/we/someone knows where all the places tools sohuld be uploaded
<davecheney> at the moment it's just ec2 and hp cloud providers
<davecheney> if you're not one of those
<davecheney> you're probably on your own
<bigjools> \o/
<davecheney> this is why wallyworld_ and I want to stop doing the public bucket
<davecheney> and distribute the tools as part of the client
<bigjools> well put it like this, we're no longer doing azure, so I don't think it's us who are on our own
<wallyworld_> the simplestreams stuff i am working on will solve a lot of this
<bigjools> hey wallyworld_ how's the boat race?
<wallyworld_> not too bad. still fucking ugly
<wallyworld_> modern medicine isn't that advanced yet
<bigjools> the surgeon didn't give you chiselled good looks then
<wallyworld_> nah, he said something about silk purse, sow's ear
 * davecheney hands out 'hoff masks
<bigjools> wallyworld_, davecheney: I won't be around for the first day of the sprint, BTW. I have to drive to Maleny.
<davecheney> bigjools: ok, i don't know where maleny is
<davecheney> but it sounds like a long way
<bigjools> about 100km north
<wallyworld_> it
<wallyworld_> 's where he gets his weed from
<davecheney> bigjools: good, that means you won't be around to tell wallyworld_ and I to not drink wine
<bigjools> *cough*
<bigjools> I am sure you guys will cope
<davecheney> bigjools: it would be better if you could bring the choof on the first day
<davecheney> but we'll manage somehow
<wallyworld_> i'll have to leave the hotel early most days to get my kid from school
<bigjools> he'll miss out on the evening festivities then
<wallyworld_> i also have soccer in the evenings :-( except for tuesday
 * davecheney thinks this sprint is a wonderful use of our travel budget
<bigjools> Just send them to the Candy Club
<axw> probably should've just left my induction till October :)
<davecheney> axw: it's not your fault
<davecheney> and waiting to october would be lame
<bigjools> yeah it's not nice to wait that long
 * bigjools not bitter
<axw> wallyworld_: not sure if you've seen this yet: https://codereview.appspot.com/12926045/
<wallyworld_> i haven't will look
<axw> this fixes the all-machines.log spewing bug, but I don't quite understand why :)
<axw> thanks
<davecheney> axw: thanks, looking
<axw> I understand what that line does, but not why its omission would lead to repetition
<axw> davecheney: thanks
<wallyworld_> axw: does this replace the other work that was done in the previous merge proposal?
<axw> wallyworld_: yes, that was all unnecessary. sorry, I totally misdiagnosed it
<axw> I scrapped that MP
<davecheney> if there are bugs that have been fixed, but not marked as 1.13.2
<davecheney> https://launchpad.net/juju-core/+milestone/1.13.2
<davecheney> could you please do so
<wallyworld_> axw: i can't see why adding that break stops the repetition either, since the next rule doesn't match and there's so subsequent contents in the conf file. but if it works, why not i guess
<davecheney> wallyworld_: lets just hulk smash this fucker
<wallyworld_> yeah, it seems it fixes the problem, and doesn't do any obvious harm
<axw> wallyworld_: ok. weird. perhaps an rsyslog bug
<wallyworld_> maybe, who knows. or maybe i'm missing something obvious, which is more likely
<davecheney> why does juju wait til so late in the piece to tell me the env is already boostrapped
<davecheney> http://paste.ubuntu.com/6004960/
<davecheney> especially after I have taken destructive action in the bucket
<sidnei> file a bug?
<davecheney> sidnei: there is already one one
<davecheney> open
<sidnei> cool, maybe noodles775 can take a look in the morning
<arosales> bigjools, back
<bigjools> arosales: hey
<bigjools> I commented on your bug
<bigjools> you need to use --upload-tools for now
<arosales> bigjools, davecheney I see you guys have had the juju release and tools discussion
<arosales> davecheney, I thought you volunteered to upload tools as part of the release process ;-)
<arosales> bigjools, so I can create a public container in Azure, no problem
<davecheney> arosales: yes sir i did
<davecheney> however you'll remember the caviet 'to all known clouds'
<bigjools> we don't have a tool to do that for azure yet
<davecheney> there is a potentially large set of unknown clouds
 * bigjools waits for the Donald Rumsfeld-like speech
<arosales> davecheney, ah I was confused by:
<arosales> <bigjools> who generally uploads new tools?
<arosales> <davecheney> bigjools: unknown
<davecheney> bigjools: you are indeeed correct
<arosales> well lets just star with the clouds we know to work with juju atm
<bigjools> \o/
<davecheney> arosales: the short answer is, i'll upload tools for clouds we know about
<davecheney> which now includes azure
<arosales> lets not solve all the worlds problems, just yet
<davecheney> (note, no credentials to do so btw)
<arosales> davecheney, ack on Azure, we just go the green light to go ahead an test
<bigjools> we need to create a storage account and container on Azure, for public juju tools
<davecheney> wallyworld_: is there any chance simple streams will save us from this pain ?
<davecheney> in the near term ?
<arosales> davecheney, I do think simple streams for juju tools solves the public bucket in all clouds issues
<davecheney> arosales: in what timeframe ?
<wallyworld_> davecheney: i hope to have something done this week, maybe not for release. but CPCs should still have their own tools container
<arosales> at least just need to uploads tools to one place, and have juju look there
<arosales> time frame is dependent on juju-core and wallyworld_
<wallyworld_> arosales: and IS
 * davecheney ages
<wallyworld_> they are setting up the repository
<arosales> wallyworld_, have you filed an RT?
<wallyworld_> arosales: yes, you were cc'ed on it :-)
<arosales> wallyworld_, cool I'll take you word for it.
<wallyworld_> you didn't see any email?
<arosales> wallyworld_, CPCs need public buckets for faster boots?
<davecheney> arosales: they need it for any boot at all
<arosales> wallyworld_, I am  just behind on my mail.
<arosales> davecheney, aside from why tools are needed
<arosales> why they need a local bucket
<wallyworld_> arosales: yes, a CPC can mirror juju.canonical.com/tools for faster boot
<arosales> wallyworld_, so in theory they could go to juju.canonical.com
<wallyworld_> once this stuff is done, there will be no need for a public bucket
<arosales> but that would require out of cloud access
<davecheney> arosales: not following your last comment
<wallyworld_> arosales: yes, it requires out of cloud access. hence clouds should be able to mirror the tools
<wallyworld_> if they don't want that
<arosales> wallyworld_, so we will still need a public bucket once simple streams is all done
<arosales> but that public bucket would just be a mirror of juju.canonical.com/tools
<arosales> davecheney, sorry I am just trying to get public bucket and simple streams straight in my head.
<arosales> bigjools, for azure in the near term (before simple streams is done)
<davecheney> arosales: i can't help with that :)
<davecheney> i don't have it straight in mine
<bigjools> arosales: --upload-tools is your friend for now
<arosales> bigjools, I can create an public container, but it seem uploading tools to a blob requires some azure api foo
<bigjools> arosales: you can use the web ui as well
<arosales> bigjools, really?
<bigjools> we have a command line tool but I am not sure of its state at the moment
<arosales> bigjools, I didn't find any way to upload a blob via the web, do you have a pointer?
<bigjools> oh wait you're right, you can only download .... gah
<bigjools> I have a tool to upload
<bigjools> as assuming you set up the container in the web ui, we can do it
<arosales> I got a container set up via the azure web ui
<arosales> bigjools, storage name = jujutools
<arosales> container name = juju-tools
<arosales> blobs endpoint http://jujutools.blob.core.windows.net/
 * arosales not sure how to give access to write to it though . . .
<bigjools> if you want me to upload something there for you, I need the storage key
<bigjools> you can get it from the management ui
<bigjools> PM it :)
<arosales> bigjools, ack
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1214209
<_mup_> Bug #1214209: cmd/juju: add-machine should take a -n param <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1214209>
<bigjools> bugs with the would "should" in the title need to be instantly marked invalid
<bigjools> word*
<bigjools> can't type today
<davecheney> bigjools: i think you meant
<davecheney> bugs witht he word "should" in the title MUST been instantly marked invalid
<bigjools> you are possibly not wrong
<davecheney> bigjools: i can edit the titile if required
<wallyworld_> arosales: sorry, was otp. each cpc would have a repository to mirror the tools, but it won't need to be configured as a public bucket in juju
<davecheney> arosales: public buckets in juju are really weird
<davecheney> they are a bastardised version of the private (control) bucket
<wallyworld_> so no more of that public-bucket-url stuff needing to be configured in each client
<davecheney> so can't be shared across providers
<arosales> wallyworld_, and for cpc we should know what the public bucket is
<davecheney> arosales: why ? how would that information be used ?
<davecheney> axw: w1/0:config-changed %
<davecheney> ^ thanks
<davecheney> works well
<wallyworld_> arosales: there will be a url used to access the tools, but it will be in the simple streams mirrors definitin, not in juju
<wallyworld_> so users don't need to care
<axw> davecheney: cool.
<arosales> wallyworld_, so are you saying that juju tools simple stream definition will also list the known cpc public bucket/juju tool mirror locations?
<wallyworld_> arosales: yes
<arosales> wallyworld_, ok, I think the light is slowly staring to turn on
<arosales> wallyworld_, so you still need info on where a particular cloud is housing the juju-tools, correct?
<wallyworld_> you = juju folks maintaining the official tools repo
<wallyworld_> not users
<arosales> correct
<davecheney> arosales: and the code to retrieve the tools from a bucket indicated by simple streams data is provider specific
<wallyworld_> arosales: so, juju will look locally within a cloud first. failing that it will look at juju.canonical.com/tools and it can always get tools from there. but, if a mirror has been set up, the tools will be fetched locally
<arosales> wallyworld_, yes, sorry for not clarifying
<arosales> davecheney, agreed
<wallyworld_> tgere will also be user config where a dev/user can set up their own tools
<wallyworld_> and also their own image metadata
<wallyworld_> so they can experiment/test etc
<arosales> wallyworld_, ack on the user config
<arosales> wallyworld_, when you say, " juju will look locally within a cloud first."
<arosales> we need to tell juju where to look locally first
<arosales> and that is specific for each cloud
<arosales> and I think we still need to set that up, correct
<wallyworld_> arosales: for openstack, the cloud admin has the option of setting up keystone catalog items for the tools/image metadata
<wallyworld_> so juju will look there
<wallyworld_> but the end user doesn't know or care
<wallyworld_> arosales: the keystone thing is how we make canonistack work for example
<wallyworld_> it's not a cpc, but it just works regardless for users
<arosales> wallyworld_, given keystone is configured correct, and given it is an openstack cloud
<arosales> wallyworld_, but how would you handle say a joyent or azure cloud
<arosales> I am guessing we need to still tell juju where the local juju tools mirror is
<wallyworld_> arosales: i'm not sure what facilities those have for a central config equivalent. but we will be introducing the idea of central, provider based config which will fill the gap there
<wallyworld_> that last bit is still beig discussed
<wallyworld_> as to how it will work
<wallyworld_> arosales: but yes, in the short term, a url may well still be required for defining where local tools are
<wallyworld_> the tools will always be available centrally
<wallyworld_> from juju.canonical.com/tools
<arosales> wallyworld_, agreed
<wallyworld_> for CPCs, no url though :-)
<arosales> I was just trying to tease out how you are handing non-openstack clouds that _need_ a local copy of the juju tools
<arosales> and how you make that just work for the user.
<wallyworld_> sure, understood :-)
<arosales> sounds like that is still be in solved?
<arosales> wallyworld_, from what I can decipher I think canonical will still need to host a mirror of the juju tools for non-openstack based clouds.
<arosales> and then tell juju what that hosting location is, so juju can look there first before going to juju.canonical.com/tools
<wallyworld_> arosales: yes, that's what juju.canonical.com/tools will be
<bigjools> ok where can I download the current tools
<arosales> bigjools, http://juju-dist.s3.amazonaws.com/
<wallyworld_> http://juju-dist.s3.amazonaws.com/tools
<arosales> wallyworld_, I think we went full circle
<arosales> wallyworld_,  :-)
<wallyworld_> lol, yeah
<arosales> wallyworld_, so ack on juju.c.c/tools being a central _out_ of cloud location
<wallyworld_> yep
<wallyworld_> out of cloud is the key. also, it will be master copy
<arosales> but, for Azure, canonical still needs to host a local mirror of the juju tools somplace
<arosales> local == in cloud
<wallyworld_> within azure?
<arosales> correct
<wallyworld_> yes, you are right. we do that for HP and AWS
<arosales> and when simple streams is all done and tidy
<wallyworld_> that's where the mirror def will point to
<arosales> we will still need to host the juju tools mirrors in these locations
<wallyworld_> yes
<arosales> and then tell juju what these locations are, correct?
<wallyworld_> so, for hp for example, juju looks at juju.c.com/tools, see there's a local mirror on hp, and gets the tools from there
<wallyworld_> it's the same public bucket as now
<wallyworld_> but it's transparent
<arosales> gotcha
<arosales> but we specifically told juju what that public bucket is
<wallyworld_> yes, in the mirror metadata
<wallyworld_> but not in user env.yaml :-)
<arosales> agreed
<wallyworld_> so we need to know, not the user \o/
<arosales> so canonical still needs to setup and host a juju tools in Azure
<wallyworld_> yeah
<arosales> and once we know what that is, the juju tools mirror metadata needs to know the location
<wallyworld_> and allow for public access without credentials
<wallyworld_> yes, we can update the mirror files
<arosales> wallyworld_, well any writing will needs creds
<wallyworld_> yes
<wallyworld_> sorry, i meant read access
<arosales> ack
<arosales> wallyworld_, so a lot of these buckets are a little ad hoc . . .
<arosales> wallyworld_, I would suggest that with the juju.c.c/tools RT
<arosales> we open up the conversation to IS maintaining these local  _in cloud_ juju tools mirrors
<arosales> IS maintains the CPC mirrors, so perhaps also having a bucket/container in that mirror also host the juju tools mirror may be wise .  . .
<wallyworld_> agreed. the exact process still needs to be massaged. but yeah
<arosales> wallyworld_, also it sounds like we can change the _in cloud_ endpoint and it will be transparent to users
<wallyworld_> yes indeed
<wallyworld_> we still need to sort out signing though
<wallyworld_> did i copy you on my email reply to (scott i think) on that topic?
<arosales> wallyworld_, you may have . .  . .
<wallyworld_> i basically think that signing right now is adhoc and needs to be centralised
<wallyworld_> managed via a tool chain and not individually on laptops etc
<arosales> ah I think that was part of the "juju work" thread
<wallyworld_> maybe, was more than a day ago so my fifo brain has wiped it :-)
<arosales> wallyworld_, btw I also see your RT 63925, for juju.c.c/tools
<wallyworld_> cool
<arosales> wallyworld_, I just forgot :-/
<wallyworld_> np. happens to me all the time :-)
<arosales> wallyworld_, exactly on my fifo brain
<arosales> very limited queue
<wallyworld_> lol
<wallyworld_> yeah, me too
<arosales> wallyworld_, so I am going to work with bigjools on setting up a "public bucket" for Azure
<wallyworld_> excellent
<wallyworld_> bigjools *loves* Azure :-D
<arosales> I don't think it will be the final one, but I think it will be the one we use until we get an IS official solution.
<bigjools> fuck off
<arosales> lol
<bigjools> and when you get there, fuck off again :)
<wallyworld_> arosales: right now, the mirrors work is still in review, and i need to do tools to generate the files etc. so there's work to do
<arosales> wallyworld_, once we have what the "public bucket" is we will let you know so you can tool juju tools metadata
<wallyworld_> ok
<wallyworld_> bigjools: ok, i've done that, twice. now what?
<arosales> wallyworld_, thanks for the explanation on juju-tools mirroring. I'll work to ensure CPC accounts for some sort of juju-tools _in cloud_ miroring of juju tools and let you know about it.
<wallyworld_> arosales: np. i'm still in the middle of doing the code for it all. expect a better explanation when it's all done. it's very raw and a wip at the moment
<arosales> wallyworld_, no worries. I appreciate your time. I just want to make sure we have all the bits for Azure as it comes online
<wallyworld_> understood. will be great to have it all working
<arosales> davecheney, I'll follow up with you on uploads once we know what that is for Azure
<wallyworld_> bigjools says he want to help maintain it to, so keep him in the loop
<arosales> davecheney, did I miss any other of your questions?
<bigjools> wallyworld_ said he wanted a full run-down next week so he could understand everything to do with azure
<wallyworld_> nooooooooo
<bigjools> it will be my duty to make sure he's fully up to speed
 * wallyworld_ just puffed on an exploding cigar
<bigjools> is that a euphemism?
<wallyworld_> you wish
 * thumper stabs the convuluted use of shit loads of structures in the juju codebase in the face and goes to make coffee
<arosales> bigjools, 2nd failure on " no reachable servers" with, perhaps the 3rd time is the charm
<bigjools> wallyworld_: is your coffee machine fixed?
<wallyworld_> bigjools: no :-(
<bigjools> wallyworld_: then I won;t tell you how nice this one is that I just made
<wallyworld_> i can't have hot liquids for a few days anyway
<wallyworld_> fo
<bigjools> can you have body temperature liquids?
<thumper> bigjools: you can tell me how good your coffee is and wallyworld_ will be forced to listen
<wallyworld_> bigjools: you volunteering?
<bigjools> thumper: it is delicious.  I was even thinking of wallyworld_ when I did a latte art penis
<thumper> I'm not very good at latte art
<bigjools> it's not hard to do a cock and balls, just ask wallyworld_
 * thumper snorts
<thumper> it is bloody hard to unpick the use of exported structures all over the show
<wallyworld_> it's hard for bigjools to get them small enough if he wants to do a scale model of hos own
<thumper> I'm replacing agent.Conf with an interface
<thumper> oh, smack talk
<wallyworld_> thumper: you have my pity. i wish we had used interfaces right from the word Go ha ha ha
<thumper> heh
<thumper> funny man
<wallyworld_> instead of structs everywhere :-(
<thumper> we'd be in a better state if we did
<thumper> no pun intended there
<bigjools> wallyworld_: http://tinyurl.com/n2soctz
 * wallyworld_ nods sadly
<thumper> as I think the "state" name and package is bollocks
<thumper> it is like having a package called "db"
<thumper> or "ui"
<wallyworld_> lol. couldn't agree more
<thumper> big +1 though to the new "providers" package
<thumper> now if we could rename the interface
<wallyworld_> except the name clashes
<wallyworld_> with what providers will be in the future
<thumper> small steps
<wallyworld_> sure, but we could have chosen a better name
<wallyworld_> only do the rename once
<bigjools> wallyworld_: well I was thinking of taking you out for a coffee tomorrow but if you can;t have hot liquids....
<wallyworld_> bigjools: i'm sure i could manage a warm one
<bigjools> you could have breakfast with me and jen at the plum?
<wallyworld_> what would doctors know anyway
<wallyworld_> \o/
<bigjools> heading there after school drop off
<bigjools> before I head into the cbd
<wallyworld_> most excellent :-D
<wallyworld_> ah yes, good luck with that
<bigjools> will get told how buggered I am
<wallyworld_> hopefully no too much
<bigjools> hopefully
<bigjools> think I have made a connection - but ETOPIC
<bigjools> hey jam are you around yet?
<jam> bigjools: /wave
<jam> wallyworld_: good to see you around, I didn't think you'd be fully coherent today.
<wallyworld_> jam: lots to do :-)
<bigjools> how would we tell the difference?
<wallyworld_> http://tinyurl.com/n2soctz
<bigjools> ahahahaha
<bigjools> jam: hey man, I want to catch up on the tags work you did if you can spare ten minutes?
<jam> bigjools: yeah no problem
<jam> (just reading backlog right now)
<bigjools> jam: it's more maas specific so I shall switch to #maas
<jam> np
<axw> any particular reason why this isn't closed?  https://bugs.launchpad.net/juju-core/+bug/806241
<_mup_> Bug #806241: It should be possible to deploy multiple units  to a machine (unit placement) <production> <juju:Confirmed> <juju-core:Triaged> <https://launchpad.net/bugs/806241>
<axw> isn't this just --to?
<axw> ah sorry, it's --to & -n
<axw> .. no it's not, the summary is just confusing
<arosales> bigjools, I added https://bugs.launchpad.net/juju-core/+bug/1214181/comments/4
<_mup_> Bug #1214181: Azure Provider always uploading 1.12 tools <juju-core:New> <https://launchpad.net/bugs/1214181>
<arosales> not sure why that is occuring. . .
<arosales> not sure if 1.12 tools are needed for some reason.
<arosales> but be interesting to see if you hit that by using public-storage-*
 * arosales is going to put up for the night
<bigjools> arosales: ok
<bigjools> I'll need to punt to an expert
<arosales> bigjools, I had you pegged as the expert :-)
<bigjools> arosales: well I know about azure, but juju not so much :)
<arosales> ah
<arosales> bigjools, would you mind checking if you can reproduce what I am seeing?
<arosales> bigjools, I'll check the bug comments in the morning. Thanks for your help.
<arosales> bigjools, davecheney, wallyworld_ have a good rest of your day and thanks for the help.
<wallyworld_> anytime, you too
<jam> thumper: are you still around?
<bigjools> arosales: sure
<axw> does anyone know if there's a reason why we don't use "write_files" in our cloud-init config? rather than a script that echos the contents... improperly interpreting escape sequences
<bigjools> I can suggest one but it's rude
<bigjools> jam: do all the providers take mem= in MB?
<davecheney> axw: probably no good reason
<davecheney> i think we should use cloud init whereever possible
<davecheney> scott moser has already solved these problem
<davecheney> we shouldn't reimplment his wheel
<bigjools> yeah it's quite amazing the number of formats cloud-init can handle
<bigjools> however you are tied to the oldest version in old releases
 * davecheney sobs
<davecheney> 18 months ago everyone was like 'precise is the best release ever'
<davecheney> now they are like' urgh, precise, soooo oooold'
<davecheney> axw: https://code.launchpad.net/~axwalk/juju-core/juju-add-machine/+merge/179840
<davecheney> you have a conflict in your merge proposal
<axw> davecheney: I'm not surprised. I was hoping to get some comments before I actually attempted to merge. Are you attempting to test it?
<davecheney> nah, got as far as reading the diff and saw it was dirty
<davecheney> just merge it to trunk
<davecheney> then lbox propose agian
<axw> yeah ok
<davecheney> s/to/from
<jam> axw: write_files (IIRC) was not available in cloud-init on the versions we wanted to use
<jam> I could be wrong
<jam> I think I remember digging into that befgor
<jam> before
<axw> jam: okey dokey. I'll confirm, and add a comment if that's the case
<axw> thanks
<jam> bigjools: I'm not very familiar with pyjuju, but I'm pretty sure that in go-juju you specify mem directly "juju deploy --constraints=mem=512M
<bigjools> oh that makes more sense
<davecheney> jam: correct
<davecheney> --constraints="arch=amd64 mem=2G" etc
<axw> jam: yeah, seems write_files was not in cloud-init 0.6.3 :(
<axw> oh wlel
<jam> davecheney, axw: I would think that newer cloud-init might end up in the minor updates (12.10.2, etc). But I don't have any proof of that, we'd have to ask smoser
<axw> jam: do we stop supporting .1 when .2 is available?
<davecheney> jam: i'll not hold my breath
<jam> axw: we look in simplestreams to find what "precise" is. I imagine it always points to the latest.
<jam> There is still userland and custom images, etc.
<axw> yeah.. I guess you'd want to give people some warning about doing something like that
<jam> axw: the other problem is that we can't really "just detect" what a given image has available, so it is probably most reasonable to stick with least-common-denominator
<axw> yep
<axw> I'll add a comment in the code for the next person :)
<axw> davecheney: merged and reproposed
<bigjools> hey, does a mongo binary need to exist with the public tools even on saucy?
<jam> bigjools: we use the ppa even on precise
<jam> bigjools: we don't use it from the bucket for a while now.
<bigjools> I ask because I am testing on azure and it gets stuck when trying to connect to mongo
<bigjools> conn refused
<jam> bigjools: I think that is because we aren't initializing properly on the machine, and we only detect that because mongo didn't get started.
<jam> bigjools: the jujud binary writes a custom upstart
<davecheney> bigjools: no, it does not need to exist for any series
<davecheney> it is redundant
<bigjools> k
<bigjools> do we expect it to take a while to get started?
<davecheney> bigjools: it comes up pretty much instantly
<jam> bigjools: not very long vs the machine being up and running. I think on HP it is < a minute
<bigjools> ok, something's buggered then
<davecheney> but we can't predict when that instant occurs
<jam> bigjools: can you ssh in to look at log files?
<bigjools> I *could* if I had not already destroyed my env :)
<davecheney> jam: yup < 90 seconds for HP
<davecheney> bigjools: tail -f /var/log/cloud-init-output.log // the goods
<bigjools> yup
<bigjools> ta
<bigjools> waiting for a bootstrap to finish is awkward from a UI PoV, are there any plans to improve this experience?
<bigjools> 9 minutes later, still waiting...!
<bigjools> ah there it is
<davecheney> bigjools: there is a discuission on the juju-dev ML i belive
<bigjools> I remember one from a while ago :)
<davecheney> bigjools: oh, it's still ongoing
<rogpeppe1> mornin' all
<bigjools> wallyworld_: so see you at the Plum any time after 08:45 then?
<bigjools> hello rogpeppe1
<rogpeppe> bigjools: hiya
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: - | Bugs: 7 Critical, 92 High - https://bugs.launchpad.net/juju-core/
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: rogpeppe | Bugs: 7 Critical, 92 High - https://bugs.launchpad.net/juju-core/
<axw> davecheney: 1121914 was released in 1.13.1
<wallyworld_> bigjools: ok, sounds good
<dimitern> morning
<rogpeppe> fwereade: i have just come to the conclusion that the collect logic in entityWatcher is entirely redundant - state/watcher can never deliver more than one event for the same document in the same cycle AFAICS.
<rogpeppe> fwereade: (the same is not true of lifeCycleWatcher, of course)
<davecheney> axw: did we call it out in the release notes ?
<axw> davecheney: yes
<axw> under "Resolved issues"
<davecheney> ok, removed
<davecheney> thanks for checking
<axw> nps
<davecheney> night y;all
<fwereade> rogpeppe, my worry is that the state code doesn't get to control the watcher cycles
<fwereade> rogpeppe, (and I'd rather keep the watcher implementations in line as far as possible... it will make life easier if we ever get as far as generating some of this code)
<rogpeppe> fwereade: the issue is that it's impossible for me to write a test that will fail
<rogpeppe> davecheney: g'night dave
<fwereade> gn davecheney
<fwereade> rogpeppe, a test that *would* fail, as determined by our best attempt at analysis, is all I'm really looking for here
<fwereade> rogpeppe, (hmm, if we *were* to force a collect on the initial event that would resolve the double-event ickiness that was causing intermittent failures in the first place... they definitely can fail ;p)
<rogpeppe> fwereade: as i do usually, i try to write a test that actually fails (at least once) when the code is wrong.
<fwereade> rogpeppe, (but it wouldn't hit the steady state path anyway, so, meh)
<TheMue> fwereade: ping
<fwereade> rogpeppe, that is indeed a good thing to do in general and I applaud the approach, but I think it's also occasionally legitimate to write assumption tests that don't fail at the time of writing
<fwereade> rogpeppe, and entityWatcher is an internal detail anyway -- it's not the only NotifyWatcher out there, and their various implementations do differ
<fwereade> TheMue, pong
<TheMue> fwereade: after syncing I'm now continuing with  https://bugs.launchpad.net/juju-core/+bug/1194945
<_mup_> Bug #1194945: juju set is overloaded <juju-core:In Progress by themue> <https://launchpad.net/bugs/1194945>
<TheMue> fwereade: the first approach following the "set --default" way showed, that this is not optimal in the sense of usability
<fwereade> TheMue, cool, I wasn't that keen  myself -- what's your preference now?
<rogpeppe> fwereade: the only two watchers that call collect are entityWatcher and lifecycleWatcher
<TheMue> fwereade: it leads to mixed situations like "set foo=bar --default baz" etc
<rogpeppe> TheMue: i don't think there's any need to allow mixing
<TheMue> fwereade: I would like an own command for it, like "juju unset ..." or "juju reset ..."
<fwereade> TheMue, sweet, so would I :)
<TheMue> rogpeppe: yeah, now allowance, but people may call it that way
<rogpeppe> TheMue: if they do that, they'd get an error
<TheMue> rogpeppe: dimiter and jam mentioned it too, because it is not really clear to the user
<rogpeppe> TheMue: fair enough. i think i suggested it too originally, but it didn't really seem to justify a whole new command
<TheMue> rogpeppe: that's what I've done so far, but it doesn't feel good. any reasons against unset or reset as own commands?
<fwereade> TheMue, I think I favour unset, because reset seems to be a bit more overloaded
<TheMue> fwereade: yep, I'm feeling better with unset too
<rogpeppe> TheMue, fwereade: i'd be ok with juju unset
<rogpeppe> TheMue: the help text for set should mention the existence of unset
<TheMue> fwereade, rogpeppe: ok, doing that, makes me feeling better too ;)
<TheMue> rogpeppe: oh, yes, very good idea
<fwereade> rogpeppe, my position is that the watchers that don't collect() probably should in general, but that Lax is appropriate for those cases in the meantime; but making every notify checker lax is not a good idea, because it means we can't write tests for alternative notifywatchers that do need to collect
<fwereade> rogpeppe, the code was written to support being built on in predictable ways
<fwereade> rogpeppe, ...but, look, from my perspective you made something worse and are fussing about undoing that change because it was less than perfect to begin with
<fwereade> rogpeppe, can we please just reinstate what we had before?
<rogpeppe> fwereade: i really don't think Lax helps much
<rogpeppe> fwereade: it's smoke-and-mirrors
<fwereade> rogpeppe, you turned *everything* into a Lax one
<rogpeppe> fwereade: oh, sorry, i forgot the terminology
<rogpeppe> fwereade: if we have some functionality that needs to be tested, we should have a test for that functionality
<rogpeppe> fwereade: rather than pretending that a few nanoseconds of scheduler decisions might give us a test failure more than once a century
<fwereade> rogpeppe, it doesn't help *much*, I know
<fwereade> rogpeppe, I added it because it helped a *bit*
<fwereade> rogpeppe, and you removed it with a clear statement of "I don't understand what it does, but meh"
<rogpeppe> fwereade: i didn't understand what it did, because it didn't do what it was intended to do
<fwereade> rogpeppe, what do you believe it was intended to do?
<rogpeppe> fwereade: i can easily add a test for aggregation
<rogpeppe> fwereade: i believe it was intended to make some tests fail if aggregation was not working
<rogpeppe> fwereade: but it's not clear which tests are intended to fail in that case
<fwereade> rogpeppe, and your problem is that those tests do not *always* fail? yes, this is less than ideal -- but you seem to be saying things are better with that behaviour untested?
<rogpeppe> fwereade: my problem is that in all likelihood those tests will never fail
<rogpeppe> fwereade: and i'm not sure which tests they are
<jam> axw: do you have any idea of why cloud-init stuff was working for a while but seems to have changed?
<rogpeppe> fwereade: because the "non-lax" code was sprayed around everywhere, as if everywhere was trying to test this issue
<axw> jam: no, but I never saw it in the working state
<jam> axw: k. Is there a reason why 'shquote' doesn't grow the escape '\' code?
<fwereade> rogpeppe, it was written to test against the NotifyWatcher interface without regard for the underlying implementation
<rogpeppe> fwereade: out of interest, which tests *should* (potentially) fail if the aggregation isn't working?
<fwereade> rogpeppe, CleanupWatcher at least IIRC -- and the point is that the tests should be somewhat useful canaries against non-obvious underlying implementation changes
<fwereade> rogpeppe, this is how a NotifyWatcher should act regardless
<axw> jam: not technically, but I think whomever wrote it intended it to be like http://www.daemon-systems.org/man/shquote.3.html
<axw> or something similar
<fwereade> rogpeppe, the fact that the implementation of one particular one is such that it can't fail is just a detail, and not a justification for using less strict testing infrastructure
<axw> jam: IOW, I didn't want to surprise anyone
<rogpeppe> fwereade: how many tests for impossible happenstances should we write?
<TheMue> rogpeppe: thx
<TheMue> rogpeppe: "happenstance" is a new word for me, like it
<fwereade> rogpeppe, quite a lot, really, because if you use inspection of the implementation as justification not to test we can drop a whole load of tests... but then they *don't fail when someone changes the implementation*
<rogpeppe> fwereade:i did think it should be possible to make a test fail by changing the code it's testing. if that's no longer the case, then i really see no bound on the amount of test code we need to write.
<fwereade> rogpeppe, then in that case use the Lax watcher explicitly in those cases, because I cannot be bothered to argue that
<rogpeppe> fwereade: the implementation can change in so many possible ways - there are zillions of ways that things could fail if the underlying implementation is free to change completely. i really do find this a difficult issue to deal with.
<fwereade> rogpeppe, but that is *still* using asusmptions about the implementation in tests for the interface
<rogpeppe> fwereade: yes, i believe it's ok to assume the implementation. otherwise we don't have any idea of how to test the code well.
<fwereade> rogpeppe, the existence of Lax is a sad acknowledgement that we cannot entriely dodge the issue, but Lax is meant to be the one that's eventually dropped, not the other way round
<rogpeppe> fwereade: Lax==Sync, right?
<fwereade> rogpeppe, yeah
<rogpeppe> fwereade: StartSync is a total hack
<rogpeppe> fwereade: it shouldn't exist at all really
<rogpeppe> fwereade: (and it didn't, originally)
<fwereade> rogpeppe, Sync is a total hack, StartSync is the only one that reasonably approximates real conditions
<axw> jam: now that I read the docstring for utils.ShQuote properly, tho, it seems fair to put it there. I'll update it.
<fwereade> rogpeppe, sync only helps when you've only got one level do deal with
<rogpeppe> fwereade: StartSync gives you no guarantees of anything happening at all
<fwereade> rogpeppe, exactly so
<fwereade> rogpeppe, sync sometimes does and sometimes doesn't
<rogpeppe> fwereade: it at least guarantees that the watcher has received the sync message...
<dimitern> rogpeppe: https://codereview.appspot.com/13113045 - two more uniter API calls needed I forgot about
<rogpeppe> fwereade: perhaps that's a better way to fix this
<fwereade> rogpeppe, it guarantees that the message has travelled *some* distance through the plumbing, but not how far it has actually travelled
<fwereade> rogpeppe, *frequently* it is good enough, but that just encourages faulty assumptions
<rogpeppe> fwereade: it doesn't guarantee, unfortunately, that the sync message has been acted on at all
<rogpeppe> fwereade: oh, Sync, yes
<rogpeppe> fwereade: how about this:
<fwereade> rogpeppe, StartSync is better because it is reliably unreliable, as opposed to unreliably reliable
<axw> jam: actually, sorry, last word on this ;)   utils.ShQuote single quotes its input, so '\' characters shouldn't be interpreted
<rogpeppe> fwereade: i change the watcher code so that StartSync actually works properly, and use it everywhere we can.
<rogpeppe> fwereade: in fact, rename StartSync to Sync
<dimitern> mgz: hey
<fwereade> rogpeppe, define "works properly", because I'm not following
<dimitern> jam: ?
<rogpeppe> fwereade: by "works properly" i mean "when you've called (Start)Sync, you know that the underlying watcher implementation is synchronising, and won't act on random other timer events before that"
<fwereade> rogpeppe, (also, StartSync cannot be blocked by a faulty watcher implementation, and Sync can)
<rogpeppe> fwereade: that is true
<dimitern> fwereade, rogpeppe: sorry to interrupt, but I'd like to bug one of you for a review
<rogpeppe> dimitern: looking
<fwereade> rogpeppe, seeking common ground: if we were to call it SyncSoon instead of StartSync, would you agree that was more accurate?
<dimitern> rogpeppe: ta
<rogpeppe> fwereade: i'd just call it "Sync"
<rogpeppe> fwereade: and make it work
<rogpeppe> fwereade: yeah, SyncSoon is a better description of the current behaviour
<rogpeppe> fwereade: but it isn't very useful behaviour
<rogpeppe> fwereade: because it is inherently difficult to predict its results, which makes it hard to write good tests using it
<fwereade> rogpeppe, it's hard, and inherently timing-dependent, and that sucks -- but to write useful tests by those lights, we can't depend on anything internal to state/watcher *anyway*, because we're affected by the layers of plumbing between that package and the actual test
<fwereade> rogpeppe, Sync is useful iff there is a single layer
<rogpeppe> fwereade: yeah, i'm suggesting deleting the current Sync implementation (which simplifies state/watcher a bit too)
<rogpeppe> fwereade: and renaming StartSync to Sync, but making it more deterministic
<rogpeppe> fwereade: my objections to Sync when it was originally written were similar to yours
<fwereade> rogpeppe, that is to me the problem -- being a bit more deterministic just encourages poor assumptions
<rogpeppe> fwereade: i think it's reasonable to assume that the underlying implementation is using state/watcher
<rogpeppe> fwereade: (that's what (Start)Sync is talking to, after all
<rogpeppe> )
<jam> dimitern: I'm around, but I'm not sure what you are asking for
<fwereade> rogpeppe, I think it's impossible to make state/watcher *clients* more deterministic by tweaking state/watcher -- we're always vulnerable to the client implementations
<dimitern> jam: sorry :) was just a ping
<dimitern> jam: you wanted to discuss the api?
<fwereade> dimitern, what's Resolved() for?
<dimitern> fwereade: what do you mean?
<fwereade> dimitern, what calls it?
<dimitern> fwereade: let me check, just a sec
<rogpeppe> fwereade: if we assume that the clients function as a deterministic result of events received from state/watcher, i think that by tweaking state/watcher, we *can* make the clients more deterministic too.
<dimitern> fwereade: filter.go:432:	if resolved := f.unit.Resolved(); resolved != f.resolved {
<rogpeppe> dimitern: why does Unit.Resolved need to make an API call? can't we get the resolved mode with the rest of the unit params?
<rogpeppe> s/params/fields/
<dimitern> rogpeppe: how?
<fwereade> dimitern, forgive my brainfart
<rogpeppe> dimitern: Resolved is in unitDoc, right?
<dimitern> rogpeppe: since LifeGetter is now common, there's no way to inject stuff that get's refreshed when you call Refresh()
<rogpeppe> dimitern: then surely your problem is that you're using LifeGetter for getting all the fields in a unit?
<dimitern> rogpeppe: what?
<jam> axw: did you dig into cloud-init at all to see what it might be unescaping?
<dimitern> rogpeppe: LifeGetter reports back the life only
<fwereade> rogpeppe, life and resolution are essentially orthogonal -- it's an artifact of the current implementation that we get them at the same time
<jam> axw: for example: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-run-cmds.txt
<fwereade> rogpeppe, this gives us the opportunity to move towards less monolithic event filtering at some point in the future
<rogpeppe> fwereade: ah, i thought it's actually quite nice that we get more than one attribute of an object when we ask for the object
<rogpeppe> fwereade: because the overhead is much smaller than making n different API calls
<rogpeppe> fwereade: but if that's no longer considered a good approach, fair enough
<fwereade> rogpeppe, filter is a beast *because* all these different pieces of information are glommed together
<jam> there are no '\' characters there, but it seems to imply that cloud-init won't do anything with '\' but whether you pass a list or a string makes a difference.
<dimitern> rogpeppe: we changed everything else, so that's a minor thing :)
<axw> jam: I started to, but didn't get to the bottom of it. I noted that the YAML produced shouldn't be escaped. I didn't want to spend a lot of time on it, but if you prefer I can dig further.
<rogpeppe> dimitern: yeah, i'm just wondering about how many extra round-trips we'll be having
<rogpeppe> dimitern: but since we aren't measuring anything, who knows?
<fwereade> rogpeppe, I'd like it most of all if we only even bothered to check/watch resolved status when the uniter knew it had to
<fwereade> rogpeppe, we can't get there in just one step though
<dimitern> rogpeppe: once we do it, we can measure easily
<jam> axw: I mostly want us to understand what is going on rather than just saying "oh, add another \" because that may not be appropriate always, as it might get interpreted as literal "\" + "n"
<jam> which would then fail for a different reason, etc.
<dimitern> rogpeppe: and optimize as needed
<axw> jam: yep fair enough. I'll keep digging
<jam> mgz: poke about maas constraints and py-juju
<rogpeppe> dimitern: optimizing will be difficult if we can't change the API (which is the whole rationale for the bulk call thing AFAIR)
<jam> axw: for example we also write agent.conf via this mechanism, and some other files, it may be that we should always escape, or never, or write it differently, etc.
<rogpeppe> dimitern: but i agree anyway
<jam> axw: what I'm really concerned with is if in precise we need to do it one way, and in saucy it will be different.
<dimitern> rogpeppe: we're all smart guys, I'm sure we'll find a way when comes to that :)
<jam> axw: I think runcmds goes through "shellify" which will escape things if passed in as a list, but *not* escape them if passed in as a string
<jam> I don't know for sure what cloud-init path we are doing atm
<jam> axw: though if I'm reading it correctly, AddFile uses AddScripts which uses RunCmd
<rogpeppe> axw: i think i was probably responsible for shquote
<rogpeppe> axw: what's the issue?
<axw> jam: understood. it's a string and not a list, so it will go through the shell
<jam> axw: and it looks like we are doing a single string for the whole script to run
<axw> rogpeppe: nothing specific to shquote. the rsyslog conf written out has its \n interpreted
<rogpeppe> axw: ah, so you can't have '\n'
<dimitern> axw, jam, rogpeppe: AddFile and AddScripts were recently changed by sidnei IIRC
<jam> axw: and we *could* pass it in as "args" rather than "literal" which might be a whole lot more obvious through all the transformation steps
<axw> jam: sorry I don't follow that last statement
<jam> axw: going back to this example: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-run-cmds.txt
<jam> cloud-config takes a yaml description
<jam> in that
<jam> you can have a list of things to run
<jam> each item in that list
<jam> can either be a literal string
<jam> "ls -l /root"
<jam> or a list
<jam> "[ ls, -l, root ]"
<jam> if you pass in a literal string
<jam> then cloud-init just passes it directly to sh to interpret
<jam> if you pass in a list of strings
<jam> cloud-init will escape them for you before passing it to sh
<jam> axw: it *looks* like our internal code has a "command" object in cloudinit.go
<jam> which can take either a "literal string" or a "args []string"
<jam> but all of our AddFile code uses the "literal string" approach.
<jam> And I'm wondering if instead of playing games with escaping on our side
<jam> we might just go with
<jam> the args []string form
<axw> jam: indeed, I missed that. I'll look into using that
<axw> thanks
<jam> axw, dimitern, sidnei: it looks like we switched from doing "echo content > filename; chmod MODE filename" to "install -m MODE /dev/null filename; echo CONTENT > filename" any idea why?
<jam> (I don't think it changes the failure mode here, but I'm trying to understand the logic behind the change)
<fwereade> rogpeppe, anyway: *WatcherC targets the ideal *Watcher behaviour; some of them suck, and need Lax*WatcherCs, but those should be dropped one day; thank you for tracking down the case where non-lax was not good enough, but please reinstate the stricter versions and use them where they don't cause failures
<axw> no idea
<jam> the commit comment just says "move addFile from environments/cloudinit.go to cloudinit/cloudinit.go"
<rogpeppe> fwereade: how are you with my plan to do away with Sync entirely? (and fix StartSync so it works reliably for us - i've nearly done it, it's not a hard change)
<jam> I guess the overall commit says "create the file with the correct mode first, so that we slightly increase security". Which is fine, as long as that doesn't actually break this :)
<dimitern> jam: https://codereview.appspot.com/12352043/ - that's the change
<axw> jam: using a list isn't really viable here, since it wants to redirect output
<fwereade> rogpeppe, I still consider it philosophically suspect -- I don't see how guaranteeing that a sync has actually started helps anyone, because what we really care about is that *all* consequences have completed, and we can't affect that at the state/watcher level
<rogpeppe> fwereade: the issue it solves is this:
<rogpeppe> fwereade: a test makes some changes, then wants to see what happens
<rogpeppe> fwereade: it calls StartSync, but the regular timer event happens first, at the wrong time
<rogpeppe> fwereade: if we make StartSync actually guarantee that the watcher is doing a sync *right now*
<dimitern> rogpeppe: how's the review looking? :)
<rogpeppe> fwereade: then we avoid that possibility
<rogpeppe> dimitern: apologies, getting back on it
<fwereade> rogpeppe, keep explaining, I'm not seeing how that situation causes problems
<fwereade> dimitern, can't we determine service name unambiguously from knowing the unit name?
<dimitern> fwereade: hmm
<dimitern> fwereade: yes, actually you're right, but we can't be sure we can get it from state perhaps?
<fwereade> dimitern, the Unit surely knows its own name, though
<dimitern> fwereade: yes it does
<dimitern> fwereade: but unit.Service() -> *Service, error
<dimitern> fwereade: should we always return nil there in the api?
<fwereade> dimitern, I'm confused, the method I'm looking at is called ServiceName
<rogpeppe> fwereade: there's a good example of the problem in https://codereview.appspot.com/12352044/
<dimitern> fwereade: i mean, for the error, otherwise we construct and return a proxy uniter.Service object ofc
<dimitern> fwereade: ah, that one - yeah, you're right there - I should just extract the service name from the unit tag and return it
<dimitern> fwereade: it's a good candidate for such a call in names
<fwereade> dimitern, +1
<rogpeppe> fwereade: another possibliity, i suppose, would be to raise the watcher period to forever
<rogpeppe> fwereade: in tests
<dimitern> fwereade: but we still need GetService at the server-side, right?
<rogpeppe> fwereade: so events we get are exactly and always as a result of StartSync calls
<fwereade> dimitern, yeah, think so
<dimitern> fwereade: for the other call: func (u *Unit) Service() (*Service, error)
<dimitern> fwereade: ok
<jam> axw: so looking at cloud-init-output.log I see the echo line, but I don't see *any* single quotes
<fwereade> rogpeppe, that is an interesting thought
<jam> are they getting interpreted in the yaml maybe?
<fwereade> rogpeppe, because just guaranteeing that a sync is in progress doesn't say anything about whether or not the last event landed before the sync started
<rogpeppe> fwereade: (if we have any tests that are actually relying on the regular polling, they're broken anyway)
<axw> jam: just recreating my env, I destroyed it prematurely. sounds feasible though
<fwereade> rogpeppe, agreed
<jam> axw: I'm trying to figure out if cloud-init writes out the script it is going to be running somewhere to more easily inspect iti
<jam> or I can try querying the metadata server
<axw> jam: I think it's in /var/lib/cloud/seed/instances or something like that
<fwereade> rogpeppe, but I'm pretty comfortable as it is with the idea that an initial event may reflect the past, not so much because it's nice behaviour but because it's an insidious sort of assumption that it's hard to spot
<fwereade> rogpeppe, and I'm concerned about the impact of changing it
<fwereade> rogpeppe, because I think *that* will also be subtle
<rogpeppe> fwereade: i don't *think* i was suggesting that, was i?
<rogpeppe> fwereade: (it's a pretty fundamental part of the watcher implementation)
<jam> axw: /var/lib/cloud/instance/user-data.txt.i
<axw> jam: there's a runcmd file too
<axw> jam: I just munted my cloudinit output tho, so need to run it again...
<rogpeppe> fwereade: the random interference of timer events is actually perhaps the underlying problem here, not the StartSync behaviour. still thinking it over.
<fwereade> rogpeppe, yeah, I'm a bit uncertain myself
<fwereade> rogpeppe, might go eat something and think it over a bit
<rogpeppe> fwereade: okeydokey
<jam> axw: so if I look at instance/scripts/runcmd it uses #!/bin/sh and calls "echo '....'" and that has a %\n" in it.
<jam> however my /etc/rsyslog.d/25-juju.conf doesn't have the \nd
<jam> I'm noticing we *aren't* using "echo -e" soecho shouldn't be interpreting it.
<jam> but it might be
<jam> or #!/bin/sh might be
<jam> axw: ugh
<jam> seems that dash and bash interpret 'echo' differently
<axw> jam: doh
<dimitern> rogpeppe: cheers for the review
<jam> axw: do 'man dash' and then /echo
<jam> it always interprets '\'
<jam> while *bash* echo
<jam> only interprets '\' characters if you specify "echo -e"
<rogpeppe> jam: there are many different versions of echo
<dimitern> rogpeppe: but the comment should've been about ServiceName(), not Service(), right?
<jam> (and you can force echo -E to disable it)
<rogpeppe> jam: even bash has several built in
<jam> dash's echo -E just prints "-E"
<mgz> jam: hey
<rogpeppe> jam: that's the original echo (it just interprets -n)
<jam> axw: so... if we run on a machine where "/bin/sh" is dash, then we need to escape, if we run on a machine with "bash" then we *shouldn't* escape
<jam> axw: fun times :)
<axw> jam: how about I just pipe it through base64 ;)
<jam> axw: hence my "lets understand what is going on"
<axw> jam: good call.
<rogpeppe> jam: one possibility is to call /bin/echo
<rogpeppe> jam: but that can vary too, of course, sysv vs bsd etc
<jam> axw: /bin/echo seems to be sane here
<axw> are we ever going to support anything that uses non-GNU?
 * rogpeppe has no idea what portability range we're aiming for
<jam> axw: I know we have plans to support centos, but I don't know of any immediate plans for BSD (also maybe windows, but that will be entirely different anyway)
<axw> CentOS will be fine. We can always change it later anyway
<axw> /bin/echo it is
<jam> axw: I can confirm that write_files is not available on the precise image I'm looking at after bootstrapping.
<axw> jam: thanks. I also checked in the cloud-init source from 12.04, and confirmed it didn't have the module
 * rogpeppe can't work out how to reliably print a string with /bin/echo either
<axw> rogpeppe: ?
<axw> you mean different implementations?
<rogpeppe> axw: what if the string is "-E"?
<rogpeppe> axw: no, i mean with gnu echo
<jam> rogpeppe: -- ?
<rogpeppe> jam: nope, doesn't work
<axw> heheh
<rogpeppe> frickin' magnificent
<jam> rogpeppe: so it doesn't matter in this case, but yes, trying to echo things starting with "-" is not very well supported
<rogpeppe> it's ok if we don't mind an extra space at the start
<rogpeppe> /bin/echo '' $x
<rogpeppe> /bin/echo '' $x | sed 's/ //'
<rogpeppe> :-)
<jam> rogpeppe: in our particular case we actually start the string with a blank newline, so it really doesn't matter :)(
<axw> | cut -c2-
<axw> but I'm not going to go there
<rogpeppe> oh G N U we love U
<jam> axw: my current vote is for /bin/echo for AddFile
<axw> jam: yep that's what I'll do
<axw> with some comments...
<rogpeppe> axw: printf "%s\n" $var ?
<rogpeppe> axw: avoids the whole echo minefield
<axw> oh, I suppose so :)
<axw> printf "%s\n" '-E'
<axw> yay
<jam> rogpeppe: "printf is a shell builtin" are we sure dash and bash won't interpret it differently?
<jam> axw: http://pubs.opengroup.org/onlinepubs/009696799/utilities/echo.html
<jam> seems to state "echo is not portable across POSIX so use printf"
<jam> so lets use printf :)
<axw> just updating tests now
<rogpeppe> dimitern: my comment *was* actually about GetService
<rogpeppe> dimitern: it just returns the service name currently
<dimitern> rogpeppe: it returns the service tag, and that'll be used in Service()
<rogpeppe> dimitern: but we know the service tag given the unit name
<dimitern> rogpeppe: but we need to get it from state, right?
<rogpeppe> dimitern: why?
<dimitern> rogpeppe: it might be missing or something
<dimitern> rogpeppe: why is there an error return on state.Unit.Service() otherwise?
<rogpeppe> dimitern: can a service be deleted when it's still got units?
<rogpeppe> dimitern: because state.Unit.Service actually gets other info too
<rogpeppe> dimitern: if we're planning to get that other info as part of the GetService call, then fine
<dimitern> rogpeppe: i stil think there should be a server-side GetService call
<rogpeppe> dimitern: but it doesn't look like it
<rogpeppe> dimitern: what value does it add?
<dimitern> rogpeppe: I could change it to get the life as well
<dimitern> rogpeppe: but we already have Life that works on services
<rogpeppe> dimitern: so is the plan that we don't hold any state locally at all for an entity?
<rogpeppe> dimitern: so we won't have a Refresh call
<axw> jam: I'll finish this later, going to get my daughter ready for bed
<dimitern> rogpeppe: I'll propose a follow-up that adds the mentioned ServiceFromUnitTag to names, and can remove GetService then
<dimitern> rogpeppe: no, we hold the life
<rogpeppe> dimitern: ah, in which case GetService needs to get the life too, right?
<dimitern> rogpeppe: no, Life already works on service tags
<rogpeppe> dimitern: oh, of course.
<dimitern> rogpeppe: so perhaps uniter.Unit.Service() should really do a Life call underneath, like uniter.Unit does
<rogpeppe> dimitern: yup, that seems about right.
<rogpeppe> dimitern: it should probably just call Service.Refresh to avoid making assumptions about the fields in a service
<dimitern> rogpeppe: who should?
<dimitern> rogpeppe: Life already gets the service afresh
<rogpeppe> dimitern: the client side uniter.Unit.Service should create a local instance of Service, then call Refresh on it, rather than making the Life call itself.
<rogpeppe> dimitern: the Service.Refresh call will make the Life call itself
<dimitern> rogpeppe: that works as well, yes
<rogpeppe> dimitern: that way if we add more fields to Service, we just need to change that one place (Refresh)
<dimitern> rogpeppe: yep
<dimitern> rogpeppe: there it is https://codereview.appspot.com/13112045/
 * TheMue => lunchtime
<dimitern> rogpeppe: included that what talked about above
<rogpeppe> dimitern: i'm halfway through :-)
<dimitern> rogpeppe: cool
<fwereade> dimitern, couple of comments
<dimitern> fwereade: ta
<dimitern> fwereade: not sure what you mean by the second comment - are you suggesting to drop .ServiceName() ?
<fwereade> dimitern, if it's not used elsewhere, having a names method for converting unit tag -> service tag seems most helpful here
<dimitern> fwereade: if what's not used elsewhere?
<rogpeppe> dimitern: reviewed
<rogpeppe> fwereade: i've just suggested a slightly different approach
<dimitern> rogpeppe: thanks!
<rogpeppe> fwereade: keep the conversion on names only.
<fwereade> rogpeppe, sgtm
<rogpeppe> fwereade: that way it's only the name-related functions that panic on unexpected things
<dimitern> rogpeppe: sgtm as well
<fwereade> rogpeppe, yeah, definitely better, thanks
<rogpeppe> fwereade: cool, ta
<jamespage> mgz or hazmat: any chance either of you can take a look at these test failures
<jamespage> http://paste.ubuntu.com/6006051/
<jamespage> juju-0.7 on saucy - going to re-introduce so as to not kill folks with existing py-juju deployments
<dimitern> rogpeppe: quick glance before I submit it? https://codereview.appspot.com/13112045/
<rogpeppe> dimitern: glancing
<mgz> jamespage: I think I filed some bugs for test failures that I got only when run as part of the package build process, not outside it
<mgz> hm, not those failures though
<hazmat> jamespage, sure
<jamespage> hazmat, thanks
<natefinch> man, testing the azure environ is... less than fun
<rogpeppe> dimitern: one thought: we should implement Unit.String
<dimitern> rogpeppe: it's easy enough to do if we need it
<rogpeppe> dimitern: (lots of things call printf %q with units and services
<dimitern> rogpeppe: it also applies to the other proxy objects, like service, etc.
<dimitern> rogpeppe: only for units and services?
<rogpeppe> dimitern: any types which have String defined on them in state, i guess
<dimitern> rogpeppe: and they should return names, not tags
<rogpeppe> dimitern: i think so
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: will add them next then, along with other stuff
<dimitern> rogpeppe: ta
<jam> dimitern, rogpeppe: https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471 standup?
<jamespage> hazmat, thanks for the licensing update for jujuclient btw - I just uploaded that and juju-deployer + websocket-client to saucy
<hazmat> jamespage, awesome, thanks
<dimitern> anybody seen this debug hooks test failure? http://paste.ubuntu.com/6006151/
<jam> dimitern: I haven't seen it before, though it does look like one of those "it should take 100ms but real-world variability is higher than that". At the least, we need a better message because that is *really* hard to read.
<dimitern> jam: +1
<hazmat> looks like its got a skew of 1s
<jam> wallyworld_: so the bot seems to say "parse-tools-simplestreams" has already been merged
<jam> which is why it isn't merging it again, I'll poke a bit more.
<wallyworld_> jam: i got a singlr LGTM, so i laned it today
<jam> wallyworld_: k, looks like LP didn't notice, and I'm not sure why that is.
<dimitern> rogpeppe: the String()s are here https://codereview.appspot.com/13079044
<wallyworld_> jam: sorry, just got off the call, is lp sorted out with the merge?
<smoser> jam, axw there is no plan to put write_files into 12.04. features don't really qualify for SRU, but we could make that happen if it was terribly useful.
<axw> smoser: IMHO it's not worth the trouble - it's not a big deal to continue doing what we're doing.
<axw> (not worth the trouble to update 12.04's cloud-init)
<smoser> what is it that you'ore doing ?
<axw> just writing out rsyslog config files
<axw> there was a bug in our script that was writing the files; they were using echo, which happened to be the shell's builtin
<axw> and it was interpreting \n
<axw> that shell was dash, evidently. whereas bash's builtin, and /bin/echo don't interpret by default
<dimitern> rogpeppe: ping
<jam> wallyworld_: I think I sorted LP out.
<wallyworld_> great
<geme> I'm moving over to juju-core - are there any docs / blogs that explain the key changes ?
<TheMue> geme: the repository contains a doc dir with some very helpful files
<andrew_w_deane> rogpeppe: hi. I'm testing against the api using a local provider and I'm getting "unknown provider type local" errors. I was under the impression that the local provider was supported now.
<geme> TheMue: Thanks
<natefinch> allenap: got a minute for a juju - azure environ question?
<rogpeppe> andrew_w_deane, dimitern: sorry, was at lunch
<allenap> natefinch: Sure, go for it.
<andrew_w_deane> rogpeppe: no worries.
<rogpeppe> andrew_w_deane: what are you trying to do, exactly?
<rogpeppe> dimitern: pong
<natefinch> allenap: I'm trying to test a new method I added to the azure environ to return the addresses (hostname and ips) of instances
<natefinch> allenap: testing the azure stuff seems to use  gwacl.PatchManagementAPIResponses  to mock out the responses from the server, but I'm having difficulty getting it to do exactly what I want
<allenap> natefinch: Yeah, it's rather involved.
<natefinch> allenap: let me whip up a pastebin so you can see what I'm doing
<allenap> natefinch: Cool.
<dimitern> rogpeppe: did you have a chance to look at it?
<rogpeppe> dimitern: only briefly, will do soon
<dimitern> rogpeppe: ok, thanks
 * dimitern bbiab
<natefinch> allenap: http://pastebin.ubuntu.com/6006401/
<natefinch> allenap: the problem is that the test isn't getting the IP address returned from the call to Addresses.... even though I'm creating the deployment with the RoleInstanceList with a RoleInstance that has the IP Address set
<natefinch> allenap: sorry, that Addresses function was half modified... here's the one that actually compiles ;)   http://pastebin.ubuntu.com/6006409/
<smoser> axw, i dont think cloud-init was getting in your way, was it ?
<smoser> the message https://code.launchpad.net/~axwalk/juju-core/lp1212148-cloudinit-escapes/+merge/180988 implies cloud-init
<allenap> natefinch: I'm going to patch this in and fire it up.
<rogpeppe> smoser: will cloudinit interpret backslashes inside single quotes ?
<natefinch> allenap: I haven't actually tested the code against a real azure environment... that was going to be my next question :)
<natefinch> allenap: and thanks :)
<smoser> well, cloud-init reads yaml (through python-yaml).
<rogpeppe> smoser: hmm, sorry, yes, was thinking of upstart
<smoser> afaik, cloud-init correctly escapes commands in 'runcmd' or bootcmd when it writes the run script.
<smoser> rogpeppe, whenever i have to deal with fighting multiple levels of escaping (which i dont think cloud-init is actually giving you pain itself here), i often just base64 encode stuff.
<rogpeppe> smoser: yeah, that's always an option
<smoser> echo BASE64_ENCODED_STUFF | base64 --decode > output
<rogpeppe> smoser: assuming the base64 command is provided by default
<axw> smoser: yeah I mistook it to be the culprit, it's not a cloud-init problem at all
<smoser> $ dpkg -S `which base64`
<smoser> coreutils: /usr/bin/base64
<axw> sorry, didn't mean to blemish your good name :)
<smoser> $ apt-cache show coreutils | grep Ess
<smoser> Essential: yes
<smoser> what that means is your'e extremely unlikely to find a debian or ubuntu system without coreutils.
<rogpeppe> smoser: ok, that's good to know, thanks
<smoser> and iirc busybox even provides 'base64'
<axw> I did consider base64'ing it
<axw> no good reason not to I suppose, just didn't want to obfuscate the data and make debugging *slightly* more onerous
<rogpeppe> smoser: i think the problem was actually in the echo command
<rogpeppe> smoser: which is, as we know of old, problematic
<rogpeppe> smoser: so we're going to use printf(1)
<smoser> yeah, i was going to suggest that too.
<smoser> printf "%s" "whatever-wont-get-interpreted-here"
<rogpeppe> smoser: sad really, but yes, that's the preferred solution
<rogpeppe> smoser: let's blame SysV
<smoser> it not sad.
<natefinch> allenap: I actually have to head out for probably an hour to an hour and a half.  If you need to do something more pressing, please feel free, but any help would be much appreciated.  I'm sure I'm just doing something wrong in setting up the test (whether or not the actual function is correct is another matter).
<rogpeppe> smoser: it's a bit sad that echo is not fit for purpose. "you only had one job" and all that.
<allenap> natefinch: I'll look at in the meantime. It's good actually; you're not blocked while you're out :)
<natefinch> allenap: thanks, I appreciate it :)
<smoser> rogpeppe, fair enough :)
<smoser> but printf just has one job too.
<smoser> and it does what you want.
<axw> long live the mp, the mp is dead
<allenap> natefinch-afk: http://paste.ubuntu.com/6006470/ is my solution. Addresses() calls GetDeployment(), which only expects a single deployment to be returned.
<allenap> natefinch-afk: Fwiw, where you do `if len(d.RoleInstanceList) > 0 {` I suggest you instead loop. There *should* only be one instance, but my spidey senses think it'll be easier to debug if someone does start another instance in the deployment (by hand, for example). I'm not sure that Azure promises ordering of instance lists.
<mgz> yeah, I agree with that
<jamespage> mgz, fwereade: hey - I just uploaded golang 1.1.2 to saucy - I'm guessing that you guys want that in the Juju golang PPA right?
<mgz> jamespage: that would be ace
<fwereade> jamespage, yes please :)
<jamespage> mgz, also fixes up some problems with os/user
<jamespage> fwereade, mgz: OK _ as soon as it lands in the release pocket I'll do the backports
<mramm> mgz: jamespage: what do you guys think about a short network addressability sprint in your hometown?
<fwereade> dimitern, hey, mea culpa for lazy reviewing, but an error on .Count() does not mean "ehh pretend not in scope" it means "something went wrong, report it"
<fwereade> dimitern, (in InScope)
<jamespage> mramm, depends when
<mramm> jamespage: I know you have a "life event" coming
<mramm> jamespage: but I don't remember when (sorry)
<mgz> these events have a habit of not turning up quite when expected as well :)
<mgz> sounds like a neat idea if james can fit it in though
<mramm> mgz: jamespage: yea, and I think it would be worth doing even if james can't come
<jamespage> mramm, probably this week :-)
<mramm> jamespage: cool.   How much family leave time are you thinking you will take?
<jamespage> mramm, 2 weeks
<mramm> cool
<mramm> so I was thinking about mid september for a network addressability sprint
<mramm> well, actually jam suggested it
<dimitern> fwereade: hmm, then how come all tests pass? :)
<fwereade> dimitern, because you implemented it to ignore errors
<dimitern> fwereade: :)
<fwereade> dimitern, we aren't encountering errors under test conditions
<dimitern> fwereade: ok, I see your point, will change it to return an error and a bool
<fwereade> dimitern, thanks :)
<mramm> natefinch-afk: ping me when you get back RE: windows client
 * axw snores
<axw> night folks
<natefinch> allenap, mramm: I'm back
<allenap> natefinch: I replied soon after you afk'ed. Ping again if you didn't get it.
<natefinch> allenap: saw it.
<allenap> Cool.
<natefinch> allenap: not sure how we'd handle that... should we return all addresses of all instances?
<natefinch> allenap: if not, how do we detect which one is "ours"?
<mgz> natefinch: in practice, there should only be one instance
<mgz> but we at least want some handling of len()!=1 in case something is weird
<natefinch> mgz: right now I handle zero and > 0  :)  for > 0 I just pick the first one.   If there's a better way to handle 2+, I'm all for it
<allenap> natefinch: What mgz said. I think it would be better to return all the addresses instead of the first of an unordered list. Maybe better yet would be to return an error.
<natefinch> allenap: an error sounds good to me. Indicates someone mucked with our environment
<fwereade> rogpeppe, jam, I'd appreciate your thoughts on https://codereview.appspot.com/12841044 -- particularly the question in the description
<allenap> natefinch: Yeah, good point, I agree.
<fwereade> rogpeppe, jam: it's a step towards https://bugs.launchpad.net/juju-core/+bug/1192433
<_mup_> Bug #1192433: relation-list reporting dying units <jujud> <relations> <juju-core:Triaged by fwereade> <https://launchpad.net/bugs/1192433>
<natefinch> allenap, mgz: sorry, brb, gotta help with emergency diaper duty
<allenap> natefinch: Heh, I know what that's like. Go! :)
<fwereade> rogpeppe, jam: I'm not quite sure whether this is a good point at which to introduce a provider/requirer asymetry -- can you think of potential issues there?
<rogpeppe> fwereade: that was the first thought that crossed my mind too
<fwereade> rogpeppe, my heart is saying you should PrepareLeaveScope if you're a peer or a provider, but not if you're a requirer
<rogpeppe> fwereade: i'm not sure
<rogpeppe> fwereade: but it's a difficult call
<natefinch> mgz, allenap: back, crisis averted ;)
<fwereade> rogpeppe, my problem is that I'm not seeing a great deal of difficulty in the call, but I feel like I might be missing something ;)
<rogpeppe> fwereade: i can't think of any argument that applies to providers that doesn't apply to requirers too
<rogpeppe> fwereade: a requirer could very easily set up creds for a provider, for example
<fwereade> rogpeppe, it *could* but that would be very annoying ;p
<fwereade> rogpeppe, I guess that's the problem with having left it wide open thus far
<rogpeppe> fwereade: how do you mean?
<rogpeppe> fwereade: annoying for us as developers?
<fwereade> rogpeppe, annoying for anyone who takes the meanings of provide/require at face value really
<rogpeppe> fwereade: i do find the provider-requirer thing a bit odd (you can have a requirer with a relation to several providers, right?)
<rogpeppe> fwereade: but i find it particularly difficult trying to second guess how people might be using our primitives
<rogpeppe> fwereade: maybe we should do this for peer relations only
<rogpeppe> fwereade: the other question is: why does querying a dying unit fail?
<rogpeppe> fwereade: if we made that work, would your CL be necessary?
<fwereade> rogpeppe, I read that to mean "the dying unit is busy shutting down replication, and an app-level attempt to connect fails"
<rogpeppe> fwereade: ah, i see, yes
<fwereade> rogpeppe, so I don't think that's something we can affect
<rogpeppe> fwereade: another possiblity would be to add a flag to relation-list to allow it to ask for only alive units
<rogpeppe> fwereade: that way a charm could exclude dying units at its own discretion
<rogpeppe> fwereade: and we wouldn't need to break symmetry
<fwereade> rogpeppe, I think that'd be somewhat challenging
<fwereade> rogpeppe, from the charm perspective, it introduces a whole new concept, and I'd need to think pretty hard before I was sure that didn't introduce its own sources of confusion
<rogpeppe> fwereade: i think i'd be inclined to make this happen for peer relations only, addressing the specific use case the bug talks about
<rogpeppe> fwereade: you're assuming that requirer-provider implies client-server, but that's by no means the only possibility
<fwereade> rogpeppe, I *think* that all I'm assuming is that it means requirer-provider, but I'm probably being dense; expand?
<rogpeppe> fwereade: well, AFAICS, the situation you're concerned about it this:
<mgz> allenap: any ideas on bug 1214451?
<_mup_> Bug #1214451: Unable to bootstrap maas environment <juju-core:New> <https://launchpad.net/bugs/1214451>
<rogpeppe> fwereade: provider sees requirer go away, so shuts off access, but requirer is still shutting down and tries to access the provider to flush any final information,  but it can't
<rogpeppe> fwereade: because access is shut off
<rogpeppe> fwereade: is that an accurate representation?
<fwereade> rogpeppe, yeah
<rogpeppe> fwereade: so, i don't really see how that logic doesn't work the other way around too (with "requirer" swapped with "provider")
<fwereade> rogpeppe, well, it does indeed all hinge on the naming, that a "provider" is expected to "provide" something that a "requirer" "requires" ;)
<dimitern> fwereade: quick review? https://codereview.appspot.com/12782045
<fwereade> rogpeppe, and that it's not a problem if something not required is not present, but the converse does not hold
<allenap> mgz: My guess is that it's trying to create an empty provider-state, but for some reason MAAS forbids an empty file. Sigh. It's probably our fault, but you never know because Django gets flustered at the difference between "" and None. I'll investigate...
<rogpeppe> fwereade: i think that's perhaps a step too far - it's that a resource that we knew about is going away, and we want to be able to shut down in an orderly manner
<rogpeppe> fwereade: and that seems to apply for both requirers and providers
<mgz> allenap: a django version difference also sounds like a good bet
<mgz> natefinch: you have a moment for a quick catchup in a sec? we can do kanbany things while we're at it
<fwereade> rogpeppe, it still STM that providing something not required is fine; but requiring something not provided is a problem
<fwereade> rogpeppe, this discussion is I guess orthogonal to the practical question of "how many relations are implemented backwards"
<fwereade> dimitern, LGTM btw
<rogpeppe> fwereade: ... or "how many charms shut off all access on relation-departed?"
<dimitern> fwereade: ta
<fwereade> rogpeppe, well, those that do will I think be helped by this change, so long as they're defined the natural way round; those that don't won't care, AFAICS
<natefinch> mgz: sure
<rogpeppe> fwereade: i was thinking of applying this change across the board
<mgz> natefinch: same hangout as this morning?
<rogpeppe> fwereade: and wondering how many things would break
<natefinch> sure
<fwereade> rogpeppe, to consciously and deliberately remove a requirer's, er, requirements, seems a little perverse though
<fwereade> rogpeppe, even if nothing broke it'd cause the relationship to make less sense
<rogpeppe> fwereade: well, the requirements are going away in a moment anyway
<allenap> mgz: I want Django to die in a lake of burning acid, but that doesn't help here. RaphaÃ«l landed a fix in r1540 to allow empty files. jamespage, can you use ppa:maas-maintainers/dailybuilds instead?
<jamespage> allenap, OK - let me try
<fwereade> rogpeppe, yeah, the other perspective is "this is the same as an independent breakage of some unit, and we should WONTFIX it, because charms have to handle that anyway"
<rogpeppe> fwereade: the question is whether it's important that we get told that they've gone away *after* they've entirely shut down, or before they start to shut down
<fwereade> rogpeppe, but I think it's unhelpful to enforce a model that obscures that (IMO) real distinction
<rogpeppe> fwereade: the difficulty is that there's no right answer
<rogpeppe> fwereade: both answers to the question "what are the active units in this relation" are equally valid
<mgz> hm, nate still isn't in the list of users that I canassign cards to on the kanban board?
<mgz> fwereade: I don't suppose you have the right kanban access to fix that?
<fwereade> mgz, I don't *think* so but I'll try to poke around
<fwereade> mgz, if I can I can't figure out how
<fwereade> mgz, *but* he seems to be on the list -- just pushed off the bottom of the quick-access submenu
<mgz> that worked, thanks!
<fwereade> rogpeppe, well, we do already define the reality we present to the charm, and what we show doesn't necessarily bear much relation to what's really going on
<fwereade> rogpeppe, units can be available for a long time before a related unit knows about them, for example
<rogpeppe> fwereade: yeah.
<rogpeppe> fwereade: i think it's worth a post to the mailing list
<rogpeppe> fwereade: interestingly my changes to the watcher logic exposed a bug in cleanupWatcher
<rogpeppe> fwereade: it didn't coalesce events
<fwereade> rogpeppe, ha :) nicely done then
<fwereade> rogpeppe, ah, hmm, looking at that code I have something knocking at the back of my mind
<rogpeppe> fwereade: the watcher code?
<fwereade> rogpeppe, the cleanup watcher
<rogpeppe> fwereade: oh yes?
<fwereade> rogpeppe, all I can remember is that it was implemented as simply as possible for some specific reason
<rogpeppe> fwereade: hmm
<rogpeppe> fwereade: perhaps we don't really care if we receive two cleanup events
<fwereade> rogpeppe, that's basically it -- a Cleanup() with nothing to clean up is cheap enough that we don't really care
<fwereade> rogpeppe, but, that said, there's no reason *not* to coalesce
<rogpeppe> fwereade: hmm, i could just fix the test
<fwereade> rogpeppe, I'm pretty sure it predated collect anyway, and the introduction of collect was not done across the board
<rogpeppe> fwereade: (although it was no hassle to make it call collect)
<fwereade> rogpeppe, I think a collect is a good idea tbh
<fwereade> rogpeppe, ah wait
<rogpeppe> fwereade: (the only awkward bit was that collect assumed string keys, but that wasn't a problem to change)
<fwereade> rogpeppe, let's go with it
<TheMue> Ah, found my silly mistake. *phew*
<natefinch> rogpeppe: easy one, if you have time - https://codereview.appspot.com/13126043
<natefinch> mgz: back
<rogpeppe> natefinch: looking
<mgz> natefinch, rogpeppe: also an easy one https://codereview.appspot.com/12741048
<natefinch> rogpeppe: checker was needed for my next couple changesets
<rogpeppe> natefinch: isn't it O(nÂ³) in fact ?
<rogpeppe> natefinch: (not that it matters much, probably)
<natefinch> rogpeppe: yeah with the reslicing, yeah, you're right
<natefinch> rogpeppe: moral of the story, keep n under 10 and you won't notice
<rogpeppe> natefinch: i think you can keep the amount of code about the same and still be O(n*log(n))
<natefinch> rogpeppe: you going to make me sort the slices?
<rogpeppe> natefinch: nope (you can't do that easily)
<natefinch> rogpeppe: right, that's why I didn't :)  Sure, lay it on me
<rogpeppe> natefinch: just sketching it
<mgz> natefinch: I think I win my bet :)
<natefinch> mgz: haha yep
<natefinch> rogpeppe: just saying... you're optimizing something that is only used in tests and likely only ever used on n < ~5
<mgz> but it's fun!
<rogpeppe> natefinch: it should make the code more obviously correct too
<natefinch> rogpeppe: I'm all for more obviously correct. I know it's not the prettiest right now, but the algorithm is pretty simple
<rogpeppe> natefinch: something like this? http://paste.ubuntu.com/6007096/
<rogpeppe> natefinch: i *think* that should be sufficient, but i might be missing something
<rogpeppe> natefinch: i have to go in exactly 3 minutes
<natefinch> rogpeppe: ok no big deal
<natefinch> rogpeppe: I'll test that out
<rogpeppe> natefinch: does that paste make sense?
<natefinch> rogpeppe: yep.
<rogpeppe> natefinch: essentially we just need to check that each slice has the same number of each item
<natefinch> rogpeppe: yep
<rogpeppe> mgz: i think the simpler code is worth it, even despite the performance difference, which i don't really care about.
<mgz> rogpeppe: I do agree really, also fun is worth it :)
<rogpeppe> mgz: :-)
<mgz> rogpeppe: go now :)
<natefinch> rogpeppe: thanks
<rogpeppe> mgz: yes, my wife has a bottle of cold champagne up my back - it's our anniversary and i really must
<rogpeppe> g'night all
<natefinch> rogpeppe:  enjoy!
<hazmat> jamespage, try again re builldd on 0.7 rel branch, i was able to get through a ppa build of the same branch. there is some vagary based on buildd performance and running the test suite reliably. unfortunately its impossible to trigger outside of the buildd envs.
<mgz> hazmat: that's pretty much what I found previously ;_;
<mgz> gah!
<natefinch> mgz: btw, I got azure working with some help from allenap... he had posted a fix that I missed earlier
<mgz> ace
<thumper> morning
<thumper> mramm: ping?
<sidnei> thumper: around?
<sidnei> ah, nm. found https://code.launchpad.net/~themue/juju-core/037-empty-strings-in-charm-config/+merge/178318
<thumper> hi sidnei
<thumper> was putting on slow cook asian pork for dinner in the slow cooker
<sidnei> got bitten by the bug above
<sidnei> with juju-deployer and setting ''
<sidnei> changed to ' ', which did the trick, but /o\
<fwereade> sidnei, it's been languishing a bit, TheMue was on holiday for a couple of weeks, but he's back and actively on it AIUI
<sidnei> fwereade: thanks!
<thumper> fwereade: you wouldn't believe how long it takes to unpick the dependencies on structs across packages
<mramm> thumper: sorry, lost track of time
<fwereade> thumper, oh, I would, that's why I tend to give up before I start :(
<mramm> fwereade: hahaha
#juju-dev 2013-08-21
<davecheney> bigjools: sup
<davecheney> thumper: do you have any contacts on u1 ?
<davecheney> sidnei: ping
<sidnei> davecheney: pong
<davecheney> sidnei: nate is looking for someone that can help him with windows installer tools
<wallyworld_> thumper: did you write the DiskManager stuff for tools?
<davecheney> does u1 use anything for producing it's windows installer that would be of use ?
<sidnei> davecheney: uhm, brian curtin was our windows guy, i think mike mccraken is in charge now. let me see if i can figure out what they are using.
<davecheney> sidnei: what timezone are they in
<davecheney> i wanna try to cross connect nate and those two guys
<sidnei> mike is in cali
<davecheney> kk
<davecheney> that'll work
<sidnei> i think brian left recently
<davecheney> oh snap
<davecheney> ok, i'll write to mike
<sidnei> davecheney: seems to be bitrock: http://bitrock.com/
<sidnei> http://bazaar.launchpad.net/~ubuntuone-control-tower/ubuntuone-windows-installer/trunk/view/head:/scripts/build_installer.py fwiw
<davecheney> sidnei: thanks
 * sidnei disappears again
<thumper> wallyworld_: no
<wallyworld_> yeah, i saw john did it
<thumper> davecheney: yes
<wallyworld_> but nothing seems to use it apart from tests
<davecheney> thumper: yes what
 * thumper shrugs
<davecheney> i think I forgot the question
<thumper> davecheney: yes I have contacts on u1
<thumper> davecheney: beuno
<davecheney> thumper: ahh, this is a question from Nate
<davecheney> who wants to know what we use for win32 installers
<thumper> ah
<davecheney> sidnei: said talk to Miek McCracken
<thumper> they are in very similar time zones
<davecheney> thumper: anyone else who delivers one win32 in the company I can point him too ?
<thumper> sorry, don't really know
<davecheney> thumper: s'okj
<davecheney> it's an unusual request
<thumper> although jamesh used to work on u1
<thumper> he may know
 * thumper pings him
<davecheney> axw: do you get much chance to talk with Nate ?
<davecheney> he's in a pretty shitty timezone for you
<davecheney> WA <> Central time is not awesome
<axw> davecheney: haven't spoken to him since I've started
<axw> yeah it's like, the opposite
<axw> :)
<thumper> they must be ~12 hours apart
<axw> yep
<davecheney> axw: it is the fall that enslaves us all
<axw> "most isolated city in the world", even when you're working remotely
<davecheney> axw: you have your own currency over there, right ?
<axw> yeah, the WA bux
<axw> actually we just trade in iron ore
<davecheney> with a picture of the little creatures brewery on the $20 bill
<thumper> davecheney: <jamesh> The win32 code is at https://launchpad.net/ubuntuone-windows-installer
<davecheney> ha! raw coke iron
<davecheney> thumper: ta
<thumper> heh, might be using py2exe
<davecheney> axw: do the ATMs dispence iron ore and uranium ?
<thumper> but should still use a .msi somehwere
<axw> looks the same as bzr's
<axw> davecheney: what is an ATM? I keep my money under my bed
<axw> anyway... ;)
<thumper> money... or opals and ore?
<axw> yep, not a very comfortable bed
<axw> I am shattered
<axw> my daughter woke up twice last night, my son three times
<axw> :(
<thumper> :(
<thumper> I remember those days
<thumper> axw: don't stress, and sleep if you need to
<thumper> the world won't fall apart if you miss a day :)
<axw> thumper: thanks, I'm kinda used to it, just a bit worse than usual
 * thumper nods
<axw> thumper: have you ever tried upgrading a local provider? :)
<axw> thumper: the bootstrap machine agent restarts repeatedly
<axw> I'll make sure I can reproduce it again, and log a bug..
<thumper> axw: we decided that we didn't support upgrading with the local provider
<thumper> axw: however, if it is easy to fix
<thumper> then we could do it I suppose
<thumper> I just wasn't going to waste cycles on it
<axw> thumper: ah ok
<thumper> we should either document the limitation or fix it I guess
<axw> do we want a bug in LP anyway? at least then we can decide to tell "upgrade-juju" to not attempt
<axw> Bug #1214676  -- logged anyway, at least then there's a record for the next unsuspecting user
<_mup_> Bug #1214676: upgrade-juju in local environment causes bootstrap machine agent to restart continuously <juju-core:New> <https://launchpad.net/bugs/1214676>
<davecheney> jamespage: thanks for keeping https://blueprints.launchpad.net/ubuntu/+spec/servercloud-s-juju-2-delivery up to date
<davecheney> i'll try to find out when we're going to deliver 1.14 stable as soon as I can
<bigjools> o/
<davecheney> axw: thanks for 12742045/
<davecheney> how come none of the tests have to change ?
<davecheney> does that mean we don't have a test that checks that the listener file exists on disk ?
<axw> davecheney: because you don't need to manually delete unix sockets in Go
<axw> Close() unlinks them
<davecheney> axw: you can still leave socket turds around
<axw> if the process terminates, yes
<davecheney> service jujud-unit-agent stop
<davecheney> will leave a turd around
<davecheney> that was how I ran across this
<axw> the existing tests all specify the path as non-abstract
<axw> i.e. they don't go through that codepath that specifies them as abstract
<davecheney> hmm, ok
<davecheney> i'm not going to complain
<davecheney> but maybe someone will
<axw> ideally we'd have a test for it, but it seemed like an awful lot of work for very little gain
<davecheney> axw: yup
<bigjools> davecheney: so, you are prepared for an Azure onslaught?
<bigjools> and axw :)
<davecheney> bigjools: yes, bring it on
<bigjools> because bugs are getting filed already (not sure if they are real bugs yet...)
<davecheney> bigjools: yeah, we'll manage
<bigjools> I'll help as much as I can of course
<bigjools> will take you through it next week
<davecheney> ok
<bigjools> will prob make Monday after all
<davecheney> \o/
<davecheney> your weed delivery came through early ?
<bigjools> nah sending the wife to get it
<axw> bigjools: I know nothing about Azure, so ... probably not prepared. But I'll look forward to learning
<bigjools> axw: there is indeed much to learn
<bigjools> on the bright side, the weather here is marvellous
<axw> here too - spring is slightly early
<jam> has anyone been trying canonistack with 'use-floating-ip: true' ? I saw the machine come up, and I can ssh to the private address, but the public address tells me invalid public key
<axw> I haven't, but I can give it a try
<jam> axw: if you would, to give me a point of comparison
<axw> jam: it works for me if I specify the identity file (~/.canonistack/axw_lcy02.key) and user as ubuntu
<axw> i.e. what's in my .ssh/config for canonistack, for the private IP range
<jam> axw: ah, user ubuntu, probably my fault
<jam> different machine, I don't have my normal setup
<jam> axw: thanks for the reminder
<axw> jam: no worries
<jam> default key was fine, didn't have the config for ubuntu@ for 172.* addresses.
<jam> axw: interestingly, username isn't part of the 'ssh -v' output.
<axw> so it would seem - helpful!
<bigjools> arosales: hopeful ping
<arosales> bigjools, hello
<bigjools> arosales: wow you're up!
<bigjools> arosales: I updated your config doc with my version of things, let me know how it looks
<arosales> for a little bit longer
<arosales> bigjools, thanks
 * arosales is waiting for agent-state to go to started in Azure
<arosales> using --upload-tols
<arosales> bigjools, I saw your and gavin's reply so I was mistaken there. I was unaware the --upload-tools built juju-tools locally to match the version being used
<bigjools> arosales: no worries
<bigjools> I'll see if I can re-create your deployment bug
<arosales> https://bugs.launchpad.net/juju-core/+bug/1214636 may be due to mismatch juju-tools
<bigjools> I highly suspect so yes
<arosales> I am trying with --uploads-tools now
<arosales> https://bugs.launchpad.net/juju-core/+bug/1214178 I think we can make better by having the user specify the settings file
<bigjools> I am just bootstrapping then I'll try to deploy
<bigjools> arosales: this is the first time I ever saw a settings file :)
<arosales> https://bugs.launchpad.net/juju-core/+bug/1214181 ~should~ be resolved once we get the correct tools in Azure
<_mup_> Bug #1214181: Azure Provider always uploading 1.12 tools <juju-core:Incomplete> <https://launchpad.net/bugs/1214181>
<arosales> bigjools, I take it you also saw Ben's tool in the google set up doc
<bigjools> I did
<bigjools> not looked at the code
<arosales> ok
<bigjools> we just saw the stuff on the management UI about creating and uploading a certificate
<arosales> Azure complained my pem didn't have the right structure when I openssl command
<arosales> I too went that route initially
<bigjools> I love the way their ui insists your file must be a .cer, like that means anything
<arosales> but  I think the settings file makes for a better user experience.
<arosales> bigjools, I feel your guys' pain
<bigjools> arosales: all of Red used the openssl tool to generate a certificate fine
<arosales> to a _very_ small degree
<bigjools> ha :)
<bigjools> arosales: the settings file can only be downloaded once you upload a certificate, right?
<davecheney> arosales:   openssl req -config /usr/share/ssl-cert/ssleay.cnf -x509 -nodes \
<davecheney>     -days 3650 -newkey rsa:2048 -keyout azure.pem -out azure.pem
<davecheney>    openssl x509 -inform pem -in azure.pem -outform der -out azure.cer
<arosales> bigjools, that may be my pilot error on the pem gen .  .
<bigjools> davecheney: exactly
<arosales> I thought I was following Azure instructions, but the settings file helps simplify that.
<bigjools> is the settings file documented anywhere?
<arosales> hm . . . I'll have to check with utlemming on that
<bigjools> I can't see anything on manage.windowsazure.com about it
<davecheney> what is the settnigs file ?
<arosales> Azure hands down the pems for a subscription in a xml based file
<bigjools> AFAICT it's some xml that contains a subscription ID and the certificate
<bigjools> but given that you have to generate and upload a certificate I question its usefulness for juju
<arosales> bigjools, aiui this settings file generates the need certificate.
<arosales> so I didn't have to upload a cert to have juju work.
<arosales> just needed to parse the settings file and put the pem in the correct path
<bigjools> arosales: ok, would love to see any docs on that
<bigjools> how did you find out about it?
<arosales> bigjools, ok I'll follow up with utlemming on it.
<bigjools> thanks
<arosales> utlemming also says this helps solve the China Azure endpoint problem too
<bigjools> arosales: what problem?
<arosales> utlemming told me about it when I was having some initial bootstrap issues
<arosales> managing multiple certificates for different Azure end points
<arosales> bigjools, I'll start a thread with utlemming so when he gets up the morning he can shed some more light on it.
<bigjools> arosales: I honestly can't work out what the problem is with that, if Azure needs a separate cert then we just config it in a separate juju env
<bigjools> ok cool
<arosales> ubg state still in pending with --upload-tools
<arosales> bigjools, http://pastebin.ubuntu.com/6009219/
<arosales> I seem to not be able to get out of pending
<arosales> what version of juju should I be using?
<arosales> and is there any other special juju set up I need?
 * arosales using 1.13.2 (compiled yesterday)
<bigjools> arosales: I am using tip of trunk
<bigjools> I just did a deployment, so let's see how it goes
<arosales> bigjools, ok thanks
<bigjools> arosales: you can try ssh-ing into machine 1
<bigjools> check the agent log
 * arosales just destroyed
<arosales> will redeploy again
<bigjools> I just deployed deploy cs:precise/juju-gui on saucy
<arosales> bigjools, juju didn't complain about the series mismatch?
<bigjools> it does unless you force the series
<arosales> I was just deploying wordpress out of a local saucy repo
<bigjools> ok
<arosales> bigjools, I guess you just set you "default-series" to "precise" in your env.yaml, correct?
<bigjools> arosales: no, that's saucy still.  I literally just did "juju deploy cs:precise/juju-gui"
<bigjools> and off it goes
<arosales> huh ok
<arosales> bigjools, have you had success with a local deploy?
<bigjools> I haven't tried
<arosales> I just downloaded the latest wordpress charm and did
<arosales> --repository=/home/arosales/devel/local-charms/   local:saucy/wordpress
 * arosales stating the obvious
<bigjools> I'm getting public key error trying to ssh into machine 1
<bigjools> this means cloud-init is probably hosed
<bigjools> either that or juju's user data went wrong - and given the bootstrap worked I suspect the latter
<arosales> ugh
<bigjools> I've definitely deployed before so something has recently broken
<arosales> there was a recent fix for cloud-init ssh access on azure . .  .
<bigjools> sadly no way of finding out since I can't ssh in ...
<bigjools> oh do you have a reference?
<bigjools> the fix might not be in the image
<arosales> https://bugs.launchpad.net/cloud-init/+bug/1212723
<_mup_> Bug #1212723: cloud-init fails to set user password on Windows Azure <amd64> <apport-bug> <cloud-images> <saucy> <cloud-init:Fix Committed by smoser> <cloud-init (Ubuntu):Fix Released by smoser> <https://launchpad.net/bugs/1212723>
<arosales> I had thought that had gone into the Monday's daily . . .
<bigjools> arosales: ah that's not relevant here
<bigjools> we don't use passwords
<bigjools> so I can ssh into machine 0 but not 1
<arosales> ah ok
<bigjools> something catastrophic has gone wrong in cloud-init if it hasn't picked up the ssh key
<bigjools> I'll write this up on the bug arosales
<arosales> ya 0 has been able to go to a started state for me, its just subsequent services that get stuck in pending
<arosales> bigjools, if you can point to something in cloud-init I can pick this back up with utlemming and smoser in the morning (us time)
<bigjools> it could be cloud-init or juju's fault
<bigjools> so I'll write up as much as I can and then you can get smoser to take a look I guess
<bigjools> we could do with a debug setting to put a password on the account
<arosales> bigjools, which bug are you documenting in?
<bigjools> arosales: https://bugs.launchpad.net/juju-core/+bug/1214636
<_mup_> Bug #1214636: Azure Provider: Deployed service never goes to started <juju-core:New> <https://launchpad.net/bugs/1214636>
<arosales> ok
<arosales> bigjools, did that fail with the charm store gui deploy and the local charm or just the local?
<bigjools> arosales: with the cs one
<arosales> ok
<bigjools> not sure it matters
<bigjools> it's not getting that far
<bigjools> arosales: I'm going to try with a different (older) image that I used successfully before and see if that helps
<arosales> bigjools, ack, thanks for looking into and the help
<bigjools> arosales: not a problem
<arosales> geesh deploys are taking about 9 minutes
<arosales> and that just waiting for the service to connect to the state server
<bigjools> arosales: azure is slow :/
<arosales> bigjools, is there a particular API call that takes long
<arosales> I would like to bring this up with msft on why deploys take so long
<bigjools> arosales: most of them return quickly, it's just the provisioning process
<bigjools> waiting for a machine to come up and then boot and then provision.
<bigjools> a certain amount of time is wasted if the image is not new enough as the apt-get update/upgrade takes a while
<arosales> we should be hitting a local mirror for apt-get updates on the order of less than a minute, especially for a daily image.
<arosales> It seems the console provisions vms faster than 5 minutes, but I haven't done one recently
<arosales> would be good comparison
<bigjools> arosales: the api calls that are slow are mostly deletion of stuff
<bigjools> but there's plenty of bugs filed about that already
<arosales> ok, but that shouldn't affect the deploy times
<arosales> and there are fixing delete
<arosales> I think you saw the latest API for that
<bigjools> yep
<axw> jam: I can't make the standup. should I just send comments to the list?
<axw> (for shared review)
<noodles775> allenap: Hi! I saw your branch with improvements to the Makefile... Another thought I had the other day is adding -y to the apt-add-repository and apt-get installs (or at least allowing -y via env or similar?). What do you think?
<arosales> bigjools, on a quick test from the console gallery it takes about 4 minutes to create a virtual machine
<bigjools> arosales: sounds about right
<bigjools> arosales: huh, my known-good image no longer works
<bigjools> davecheney: why would mongo be coming up on port 27017 and juju wanting to connect on 37017?
<arosales> :-(
<bigjools> arosales: you should probably call it a night! If you can grab Scott tomorrow maybe he can help debug based on my bug comments
<arosales> bigjools, will do
<arosales> ok good night fellas
<bigjools> nn arosales
<arosales> bigjools, I'll touch base with smoser tomorrow morning
<bigjools> ok
<bigjools> thanks
 * bigjools eats
<noodles775> 5
<jam> axw: sounds good. I think we're going to try and have Tim run a shared review conversation sometime in the AU-friendly timezones. But I haven't seen him to coordinate it.
<axw> jam: okey dokey, thanks
<rogpeppe> mornin' all
<fwereade> rogpeppe, heyhey
<rogpeppe> fwereade: yo!
<TheMue> rogpeppe: heya
<rogpeppe> TheMue: hiya
<fwereade> TheMue, ping
<allenap> noodles775: I think that's a fair point. I'll add the -ys.
<fwereade> TheMue, actually I'll just mark https://codereview.appspot.com/12347043/ WIP, please reject it yourself if you're doing a fresh branch for unset
<noodles775> allenap: thanks (it wasn't really a point about your branch, just something I needed/wanted yesterday, and since you were there... :) ).
<TheMue> Gna, update just made me reboot.
<TheMue> fwereade: Seen that you pinged me?
<bigjools> hey, anyone know why would mongo be coming up on port 27017 and juju wanting to connect on 37017?
<jam> bigjools: "apt-get install mongodb" brings it up on 27017
<jam> bigjools: but jujud coming up sets an upstart config that puts a different one on 37017
<jam> with TLS enabled, etc.
<bigjools> jam: this was after I did a bootstrap
<jam> b
<bigjools> bootstrap node didn't finish coming up
<jam> bigjools: that sounds like cloud-init successfully installed mongodb-server, but jujud did not successfully finish bringing itself and mongodb up
<bigjools> and ssh-ing in showed this
<bigjools> yeah
<bigjools> in cloud-init-output.log it just shows lots of failed connections
<bigjools> no other errors obvious
<jam> can you tar up and post the cloud-init-output ?
<jam> I can take a look at it
<bigjools> I an do better and put your public key on the machine if you want?
<jam> bigjools: ssh-import-id jameinel should wokr
<jam> work
<bigjools> one sec
<bigjools> ssh ubuntu@juju-azure-9nz3pmvw7e.cloudapp.net
<jam> success
<bigjools> jam: I'll come clean on something - this is using an old-ish saucy image but I needed to work out why I can't deploy on the latest daily.  This old image used to work.
<bigjools> the latest daily shows a publickey error, so can't ssh in and see
<jam> bigjools: root@default:~# mongod --version
<jam> db version v2.0.4, pdfile version 4.5
<jam> Wed Aug 21 09:05:32 git version: nogitversion
<jam> that looks like a really old db version
<jam> 2.0.4
<jam> vs 2.4+ should be in new saucy
<bigjools> weird - I have deployed with this image before
<bigjools> why would this stop working?
<jam> ||/ Name                     Version                  Description
<jam> +++-========================-========================-===================================================
<jam> ii  mongodb-server           1:2.0.4-1ubuntu2.1       object/document-oriented database (server package)
<jam> bigjools: /etc/lsb-release says this is a Precise image, not saucy
<bigjools> yeah was about to sya
<bigjools> but should still work nonetheless
<bigjools> oh wait ... haha
<jam> bigjools: if we know it is precise, we add-apt-repository ppa:juju/stable  to get newer mongodbx
<bigjools> I know what;s up, I think my default series is wrong :)
<bigjools> right :)
<bigjools> thanks for helping me see the end of my nose!
<jam> bigjools: so your machine thinks it is deploying saucy, but it is actually precise
<davechen1y> jam: bigjools you need to install juju:ppa/stable
<davechen1y> to pick up the mongodb dep
<bigjools> davechen1y: jam figured it out, I had a dodgy config
<davechen1y> bigjools: also
<davechen1y> i wasn't thinking
<davechen1y> we always insert that ppa via cloud-innit on the remote bootstrap machine
<davechen1y> if required
<bigjools> understandable
 * bigjools heads towards TV to watch Aussies lose at cricket again
<davechen1y> bigjools: we're going to get on like a house on fire next week
<davechen1y> i'm super confident of that fact
<davechen1y> jam: yolanda has a questoin about bootstrapping on canonistack
<bigjools> davechen1y: :D
<yolanda> hi, i just upgraded to 1.12 version of juju, trying to deploy on canonistack, but deployment is stuck on INFO juju open.go:69 state: opening state; mongo addresses: ["10.55.60.49:37017"]; entity ""
<davechen1y> she's seeing it trying to contact the bootstrap node on the private 10
<davechen1y> ip
<yolanda> gives timeout
<davechen1y> jam: mgz any ideas ?
<davechen1y> how do you setup the port forwarding for canonistack ?
<rogpeppe1> fwereade, jam, mgz, TheMue: https://codereview.appspot.com/13089045
<rogpeppe1> fwereade: i hope this addresses your concerns about the lax stuff.
<jam> davechen1y: you can either use 'sshuttle' or you can just set "use-floating-ips: true'
<jam> davechen1y: canonistack got 2 new /24 for public IPs, so we should have enough now.
<TheMue> rogpeppe1: *click*
<jam> davechen1y: but the port forwarding I use is: apt-get install sshuttle; sshuttle -r ubuntu@$BOOTSTRAP 10.55.0.0/16
<yolanda> jam, so i'll try it
<jam> yolanda: k, if you have any more questions, feel free to ask
<yolanda> jam, that worked!
<davechen1y> woot
<davechen1y> yolanda: i the default image type on openstack machines is 1core 1gb ram
<davechen1y> jam: and mgz wrote this
<davechen1y> they would be able to tell you
<rogpeppe1> i just realised that i reviewed (comprehensively) entirely the wrong file
<jam> rogpeppe1: we'll still appreciate that review :)
<yolanda> tried a juju debug-log, having that error: http://paste.ubuntu.com/6009664/
<rogpeppe1> jam: i reviewed constraints.go
<rogpeppe1> jam: (a small file, but with primitives used in quite a few places)
<jam> rogpeppe1: I can make that the assignment for next week :) I was going to do relations.go but constraints is worthwhile for the group, too.
 * davechen1y looks
<yolanda> that url that it complains about, works nice in a browser
<davechen1y> yolanda: could be sporadic
<davechen1y> can you try again
<jam> yolanda: you can try a) swift list and b) just try juju debug-log again
<yolanda> jam, davechen1y, different answer now
<davechen1y> jujy deploy --constraints="cores=4 mem=4G" $SERVICE
<davechen1y> something like this should ask for a different instance
<davechen1y> i forget if constraints are best fit or absolute match
<TheMue> rogpeppe1: you've got a review
<rogpeppe1> TheMue: thanks!
<davechen1y> yolanda: if you need more disk i *beleve* you get it by matching an instance type that has more cores and more ram
<jam> davechen1y: they should be a "minimum" so if there is only a 4-core 8G service we'll give it to you
<davechen1y> jam: /me can't remember the instance types available on canonistack
<rogpeppe1> TheMue: as I said in the description, the plan is to rename Sync to StartSync - i didn't want to confuse matters by doing it in this CL
<jam> yolanda, davechen1y: sidnei was putting together a patch to add a "root-disk" constraint to specify how much disk space the OS gets. But that isn't in a released version. (It might have landed and will be in 1.13.2, though)
<rogpeppe1> jam, fwereade: i would very much appreciate your feedback on this CL https://codereview.appspot.com/13089045
<davechen1y> jam: yolanda do all canonistack instances get the same root disk partitoin ?
<davechen1y> i thought it was larger if you asked for more cores/ram
<yolanda> davechen1y, i normally got more disk by setting a larger instance-type, for example m1.small instead of m1.tiny
<jam> davechen1y: can you try running your mongodb with --no-unix-socket? We have a really hard time in the test suite if we try to start mongodb with a flag it doesn't recognize. So if it is ~roughly sane, we can go with it, but if there is a chance it will be hard to debug, I don't think it is worth it.
<jam> axw: ^^
<jam> davechen1y, yolanda: 'nova flavor-list' shows about 1300 options, so it isn't like you can remember them all :)
<jam> sorry 300
<jam> they all start with a 1
<axw> jam: --nounixsocket works (I tested it before across all the code), --no-unix-socket isn't a thing
<jam> axw: I realize it works on your machine, I'm concerned about it running on all developers and platforms we want to run the test suite on.
<axw> right sorry, misunderstood
<jam> We ran into trouble in the --no-ssl switch (where having an old mongo just hangs the test suite for 600s before the test times out, with poor information about why it is failing)
<axw> fwiw I verified it's on the PPA version
<axw> so it'll be no worse than lacking SSL
<TheMue> rogpeppe1: rename Sync to StartSync? that's what IMHO would be wrong.
<rogpeppe1> TheMue: no, rename StartSync to Sync.
<yolanda> jam, i normally use a instance-type=m1.small and that's all, or flavor 2, that works for canonistack
<rogpeppe1> TheMue: there should be no occurrences of Sync left (other than calls to presence Sync)
<jam> axw: I can confirm that with the 2.2.4 that we produced in the tarball on S3, --nounixsocket works
<axw> jam: thanks
<axw> gtg to dinner, adios
<TheMue> rogpeppe1: oh, you written the other direction above. so the first step is the new StartSync() and rename all Sync()s to StartSync() and then later to Sync()?
<jam> axw: have a good evening
<axw> cheers, you too jam
<rogpeppe1> TheMue: yes
<rogpeppe1> axw: g'night
<TheMue> rogpeppe1: ah, then absolute +1, my fault
<rogpeppe1> TheMue: np
<jam> rogpeppe1: "the watcher loop will not do anything else until it finishes sync" is that appropriate? (It sounds like we are blocking when we should be doing something in a background thread)
<davechen1y> axw: ship it
<rogpeppe1> jam: the watcher would never do anything else while syncing
<davechen1y> lucky(~/src/launchpad.net/juju-core/provider) % mongod --whogivesafuck
<davechen1y> error command line: unknown option whogivesafuck
<davechen1y> use --help for help
<davechen1y> lucky(~/src/launchpad.net/juju-core/provider) % mongod --nounixsocket
<davechen1y> Wed Aug 21 20:05:53 [initandlisten] MongoDB starting : pid=23438 port=27017 dbpath=/data/db/ 64-bit host=lucky
<davechen1y> Wed Aug 21 20:05:53 [initandlisten] db version v2.2.4, pdfile version 4.5
<rogpeppe1> jam: the only change now is that there's no time interval between asking for a sync and it actually starting one
<davechen1y> works for me
<jam> davechen1y: thanks for confirming, thats what I see here as well.
<davechen1y> jam: jolly good
<davechen1y> was just a bit gun shy after last time
<jam> davechen1y: mongodb's start flags are surprisingly hard to expose and respond to.
 * davechen1y sobs
<yolanda> davechen1y, i also tried a juju destroy, debug showed unit is dying, but service doesn't die, and it has been a long time, there should be some problem?
<davechen1y> fffuuuu web 2.0
<davechen1y> yolanda: weird, pastebinit ?
<yolanda> davechen1y http://paste.ubuntu.com/6009736/
<yolanda> just got that "unit is dying" and no more notice
<yolanda> but service is still there
<davechen1y> yolanda: did you do remove-unit or destroy-service ?
<yolanda> destroy-service
<yolanda> in status it just shows "dying"
<davechen1y> yolanda: juju status $YOURSERVICE should show
<davechen1y> one service
<davechen1y> with no units
<davechen1y> rogpeppe1: +1 for nuking Sync
<rogpeppe1> davechen1y: cool
<davechen1y> for no other reason than we always use StartSync
<davechen1y> so it's unneeded
<rogpeppe1> davechen1y: well, we *did* use Sync in quite a few places
<rogpeppe1> davechen1y: but it's always unnecessary
<davechen1y> exactly
<davechen1y> and one way to do things is better than two
<rogpeppe1> davechen1y: and i think my changes give StartSync the same amount of useful guarantee that we had from Sync previously
<rogpeppe1> davechen1y: (also, i plan to rename StartSync to Sync once the dust has settled)
<yolanda> davechen1y, i destroyed environment and redeployed again, as charm was in a failed state, but no error in the debug-log
<davechen1y> yolanda: im sorry it's not working on canonistack
<davechen1y> i do not test on canonistack
<davechen1y> this is my faliing
<davechen1y> failing
<yolanda> davechen1y, you mean the debug-log? or which issue?
<jam> mgz: natefinch: given you are adding Addresses to lots of providers, are we missing a LiveTest across all implementations that it is available and gives a "sane" result?
<jam> or is the idea that you have to build it up first and then implement the conformance test?
<jam> (I'd rather see a patch that adds a conformance test and stubs out the ones that don't implement it until we fix them, since that clearly records the current state)
<natefinch> greetings all
<TheMue> rogpeppe1, fwereade, natefinch: https://codereview.appspot.com/12752044
<TheMue> natefinch: good morning
<rogpeppe1> natefinch: hiya
<natefinch> jam: Yes, I believe we're missing a live test to make sure they all work... though I think they all work, since we're explicitly implementing them to get at information we know exists
<fwereade> rogpeppe1, (btw I cast a quick eye over it, and I think I like, but I've got a lot of reviews to churn through today)
<rogpeppe1> fwereade: should i hold on for your review?
<jam> fwereade: fwiw I've done quite a few of them for you already :)
<davechen1y> yolanda: not working well on canonistack in general
<fwereade> jam, <3
<yolanda> davechen1y, noticed some problems, yes
 * TheMue => lunchtime
<jam> fwereade: of course, the only one that I didn't dig into is yours :)
<fwereade> rogpeppe1, go ahead and merge it, I'll keep a tab open and throw a fit after the fact if I spot something awful, but I don;t expect to -- I did sneak a look and it seems solid to me
<rogpeppe1> fwereade: cool, thanks
<fwereade> jam, haha
<fwereade> jam, I'm a little conflicted about it anyway, I don't think it's our top priority
<jam> fwereade: unfortunately when you comment via Reitveld, LP doesn't move the branch from "Requested reviews" to "Reviews I am doing".
<jam> :(
<rogpeppe1> jam: you ok with the StartSync changes going in? (from your question earlier, i presume you've at least had a glance)
<jam> fwereade: can you land or reject: https://code.launchpad.net/~fwereade/juju-core/errors-cleanup/+merge/168928
<jam> rogpeppe1: I haven't looked at the patch yet.
<rogpeppe1> jam: np
<jam> I've been reading IRC to follow along with the discussion, though.
<fwereade> jam, hell, sorry, hat was rotted a month ago :( rejecting
<rogpeppe1> jam: i can hold on for you if you'd like
<jam> rogpeppe1: first thing I saw was the "string => interface" change. Is the collect logic actually correct for things that aren't strings?
<jam> (
<rogpeppe1> jam: i believe so, as long as they can be used as map keys
<fwereade> allenap, ping
<jam> rogpeppe1: I'm slightly uncomfortable about the change to Lifecycle watcher being bundled with changing the semantics of Sync.
<rogpeppe1> jam: in the case of Cleanup, the ids are of type ObjectIdHex
<allenap> fwereade: pong
<fwereade> allenap, do you recall, how does maas count cores? does hyperthreading count?
<rogpeppe1> jam: it seemed like a fairly trivial change, but i could split the collect change out into another CL if you like
<rogpeppe1> jam: (it's only 4 lines)
<allenap> fwereade: Let me see....
<jam> rogpeppe1: just conceptually it is something I need to think about, and seems very orthogonal to what is going on
<rogpeppe1> jam: i needed to use collect to fix the Cleanup watcher, and it had the wrong type
 * fwereade hopes it's counting logical cores, not physical, because that seems to fit best with the ec2/openstack situation
<rogpeppe1> jam: so the lifecycleWatcher needed to change for that, so it's not entirely orthogonal, but i can propose that change independently as a prereq if that's your preference
<fwereade> and actually... jam, mgz: do you know offhand how openstack counts a flavor's cores?
<allenap> fwereade: I think it counts cores. It evaluates the xpath count(//node[@id='core']/node[@class='processor'][not(@disabled)]) against an lshw XML dump.
<jam> fwereade: well they are all virtual there, right?
<jam> I don't know how it maps hyperthreading into available cores.
<fwereade> jam, indeed, and overcommit is a whole new can of worms
<bigjools> whatever the kernel reports
<rogpeppe1> jam: essentially it's just moving a dynamic type cast out of collect and into the watcher-specific logic
<jam> rogpeppe1: so given that "StartSync" actually just puts the "reqSync" into the queue, and when req := <-w.request sees it, it calls handle() and *then* flush() before we loop around to 'if w.needSync', doesn't that mean your statement about "it syncs and waits for it to complete" isn't actually true?
<natefinch> fwereade: nproc on my hyperthreaded 4 core processor returns 8, btw.
<rogpeppe1> jam: i don't *think* so
<rogpeppe1> jam: because handle of a reqSync will not actually add anything to be flushed
<natefinch> fwereade: I would hope we somehow discount hyperthreaded cores, since they don't even come close to actually doubling processing power
<fwereade> natefinch, indeed, I'm really just fretting about whether consistency is even possible given some of our providers
<jam> rogpeppe1: in the time since we start handling the reqSync, more might come into the channel, which will be flushed before we finish handling the reqSync.
<jam> that time window is very small
<jam> since we only really need to set the bool
<jam> but it does exist, doesn't it?
<rogpeppe1> jam: how could that happen? if there are no events to be flushed, then flush will never read on the channel
<jam> natefinch: when I was testing it 7 years ago, 2+2 hyperthreaded cores were easily equivalent to 3 cores (as in, enabling hyperthreading allowed my threaded code to run 50% faster on 2 physical cored
<jam> cores)
<rogpeppe1> jam: and so i'm fairly sure there are no places where new events can arrive between handling a request and calling sync()
<natefinch> jam: last I remember it was like a +25% and then only for jobs that have a lot of thread switching.... but regardless, we shouldn't treat them as fully powered cores
<jam> natefinch: I think the precision of this stuff is such that people basically need to test it and figure out what works for them. :) I don't think MaaS or Juju need to grow all that aware of what the actual benchmark results for a given workload map into "cores=X".
<jam> We just want to provide a way for people to give their input.
<jam> Ignoring hyperthreading completely doesn't seem quite correct, though neither is considering them 100%. But I think doing "something" is reasonable, as long as it is consistent.
<jam> natefinch: in the case of MaaS most likely people who *really* cared would use hardware-based tags to essentially define their own flavors, and deploy based on that.
<natefinch> jam: yep, totally makes sense.
<TheMue> standup
<jam> rogpeppe1: fwereade: https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471
<jam> mgz: ^^
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: - | Bugs: 7 Critical, 92 High - https://bugs.launchpad.net/juju-core/
<smoser> bigjools, or arosales i just tested a saucy daily from today and it seems functional. you could have been bit yesterday by bug 1214541
<_mup_> Bug #1214541: hostname setting is erroring out <amd64> <apport-bug> <cloud-images> <saucy> <cloud-init (Ubuntu):Fix Released> <https://launchpad.net/bugs/1214541>
<smoser> tested == tested from cloud-init's perspective (it ran user-data, provisioned user ... )
<rogpeppe1> jam: "
<rogpeppe1> LifecycleWatcher coalescing its changes is a good thing, and a pretty notable
<rogpeppe1> change to have it silently added.
<rogpeppe1> "
<rogpeppe1> jam: Lifecycle watcher *was* previously coalescing its changes
<rogpeppe1> jam: my CL doesn't change that
<rogpeppe1> jam: it's quite possible the testing for that wasn't great though
<jam> rogpeppe1: so is your change just making it use a common helper, or ? Either way it is a modest change that should probably make it into the "and I changed LifecycleWatcher to XXXX". It helps frame an understanding of what is actually changing. Given that I clearly got it wrong 2 times now :)
<jam> "make it into the *summary*"
<rogpeppe1> jam: lifecycleWatcher is still calling the same helper it always called
<rogpeppe1> jam: it's just that i needed to change the signature of that helper so i could use it with the CleanupWatcher
<rogpeppe1> jam: i could put "I changed the collect helper function to use interface{} keys" into the summary if you think that's worth it
<jam> rogpeppe1: so I guess some of it is diff context: looking here: https://codereview.appspot.com/13089045/patch/5001/6009
<jam> it certainly looks like lifecycle watcher is changing
<jam> but I realize that last diff block could be CleanupWatcher
<jam> which I'm *pretty* sure means CleanupWatcher is now collecting when it didn't before
<rogpeppe1> jam: hmm, i don't see that in my diff
<jam> rogpeppe1: https://codereview.appspot.com/13089045/diff/5001/state/watcher.go is a different way to look at it.
<jam> but collect() is now getting called where it wasn't before (unless I'm *completely* misreading this)
<rogpeppe1> jam: i think you're misreading the unified diff
<rogpeppe1> jam: the collect code is being added to cleanupWatcher
<rogpeppe1> jam: occupational hazard with unified diffs, i fear
<jam> rogpeppe1: so I misread that it was lifecycle, but the code is added to cleanupWatcther which is the same thing I'm mentioning (adding collection to something that wasn't collecting before)
<jam> rogpeppe1: so s/Lifecycle/Cleanup/ and my comment still applies, I think.
<rogpeppe1> jam: you're right. sorry, i was thrown off by the name
<jam> rogpeppe1: we seem to be missing a test for CleanupWatcher now, given that if you revert that change no tests will break, right?
<jam> so we don't *know* that cleanup is collecting
<rogpeppe1> jam: no, i made the change because tests broke
<rogpeppe1> jam: in particular, TestWatchCleanup failed
<rogpeppe1> jam: i'll retry to make sure of that
<rogpeppe1> jam: hmm, it doesn't seem to fail any more!
<jam> fwereade: just a gentle nudge to remind you to submit a couple bugs about container.go
 * jam is off for the evening, though I'll probably respond at some point later
<fwereade> jam, still talking to ian, have a doc open ready to convert :)
<rogpeppe1> jam: i'll add a specific test for cleanup event coalescence
<smoser> rvba, are you able to reproduce https://bugs.launchpad.net/juju-core/+bug/1214636 ?
<_mup_> Bug #1214636: Azure Provider: Deployed service never goes to started <cloud-init:New> <juju-core:Confirmed> <https://launchpad.net/bugs/1214636>
<rogpeppe1> i'm going to be offline for an hour or so, then i should have sporadic network access for 5 hours after that
<rogpeppe1> fwereade: ping
<natefinch> anyone here familiar with lxc?  I used juju to deploy locally, and I'd like to open up the service to computers on my network, but I don't really know how to do it. Right now the services just have local IP addresses like 10.0.3.187... .how do I expose that to my local network?
<mgz> there's not a trivial way of doing that
<natefinch> huh ok
<mgz> you can use iptables or similar to manually route traffic on a port in, for instance
<mgz> and we've briefly discussed making juju expose do something like that
<natefinch> mgz: yeah, manually routing the traffic is pretty much what I was thinking
<mgz> I'd give you the iptables command you need, but you can probably google it as easily as me :0
<natefinch> mgz:  haha yeah, that's what I was just doing, no worries
<arosales> fwereade, or rogpeppe1 any core folks interested/have time to join our weekly charm sync.
<arosales> fwereade, rogpeppe1 wed at 16:00
<arosales> utc, that is.
<fwereade> arosales, yes please, sign me up
<arosales> fwereade, thanks I'll add you to the invite. Please feel free to delegate and/or let me know if I should any other folks.
<fwereade> arosales, I might end up doing so occasionally, but I'm very interested personally
<arosales> fwereade, be great to have some core folks there, thanks! :-)
<arosales> fwereade, invite sent.
<fwereade> hey, it's after 6; I'm tired, see you all later :)
<arosales> got a juju ssh question if any folks are around
<natefinch> arosales: I'm around, but you probably know more than I do.  However, might as well ask :)
<arosales> on juju core I am sure you have me beat :-)
<natefinch> arosales: We'll see :)  I can at least search the codebase pretty easily for answers that can be answered that way
 * arosales getting
<arosales> 2013-08-21 17:43:25 ERROR juju supercommand.go:282 command failed: required environment variable not set for credentials attribute: User
<arosales> error: required environment variable not set for credentials attribute: User
<arosales> but I am using keys values not user
<arosales> also odd it is opening up my hp environment when I state an aws one . . .
<arosales> juju --debug ssh 1 -e aws-go
<arosales> 2013-08-21 17:43:25 INFO juju provider.go:121 environs/openstack: opening environment "hp-go"
<arosales> hmm, that may be the problem . . .
<natefinch> heh
<arosales> thats odd
<natefinch> the code looks like it'll say "user" even with keys
<natefinch> which looks like a copy and paste error
<natefinch> (or at the very least, the error message should be made clearer)
<natefinch> nah, itlooks like copy and paste.... looks like the section on username/password got copied for keys and then didn't change the error message
<natefinch> But, it sounds like you may have figured it out anyway?
<arosales> juju seems to be picking up my bash env setting when trying to ssh instead of my command line option, and just for ssh
<arosales> juju --debug stat -e aws-go
<arosales> 2013-08-21 17:52:08 INFO juju ec2.go:137 environs/ec2: opening environment "aws-go"
<arosales> but ssh tries my "hp-go" environment even though I state "-e aws-go"
<natefinch> arosales: sounds like a bug, though it's weird that it would happen only in one subcommand, I'd expect that to be shared logic... but I can see if I can find where that's set
<arosales> i'll open a bug and see if I can work around this by taking out my env  setting in my bashrc file
<natefinch> certainly if you say -e and it doesn't use that environment, that's a bug :)
<arosales> https://bugs.launchpad.net/juju-core/+bug/1215052
<_mup_> Bug #1215052: juju ssh ignores the command line "-e"  and instead uses JUJU_ENV in my .bashrc <juju-core:New> <https://launchpad.net/bugs/1215052>
<arosales> bug filed
<arosales> odd issue
<natefinch> yeah, pretty weird
<smoser> arosales, are you sure it wouldn't work with
<smoser> juju -e aws-go --debug ssh 1
<arosales> smoser, yup
<arosales> that the exact command I ran
<smoser> well, its not what you typed above
<smoser> its not completely unreasonable if juju stopped looking for flags to it after it saw 'ssh', and instead passed those forward to something else.
<arosales> <arosales> juju --debug ssh 1 -e aws-go
<smoser> ie
<smoser> <smoser> juju -e aws-go --debug ssh 1
<arosales> smoser, so you are saying order of arguments matter
<smoser> order maybe important
<smoser> yes. its not terribly uncommon.
<arosales> good point
<smoser> especially if juju was going to pass other options on to ssh
<smoser> ie:
<smoser> juju --debug ssh 1 run this command
<arosales> smoser, I'll try your order here
<smoser> arosales, also, icame here wondering if you tested aws and saucy
<smoser> (which i suspect you were trying :)
<arosales> smoser, btw I was working on trying to confirm precise ssh is still working
<arosales> per the Azure bug <arosales> juju --debug ssh 1 -e aws-go
<natefinch> smoser: good point, ssh may be different because it expects to be passed arbitrary arguments to be run
<arosales> https://bugs.launchpad.net/juju-core/+bug/1214636
<_mup_> Bug #1214636: Azure Provider: Deployed service never goes to started <cloud-init:New> <juju-core:Confirmed> <https://launchpad.net/bugs/1214636>
<arosales> lol
<arosales> juju -e aws-go --debug ssh 1
<arosales> error: flag provided but not defined: -e
<smoser> ?
<arosales> but, the following works
<smoser> odd.
<arosales> juju --debug stat -e aws-go
<arosales> 2013-08-21 18:13:05 INFO juju ec2.go:137 environs/ec2: opening environment "aws-go"
<arosales> smoser, if I put -e towards the front of the command, juju doesn't recognize it
<smoser> put it after ssh and before '1'
<smoser> maybe. just try that.
<arosales> smoser, so I can't run your command example, 'juju -e aws-go --debug ssh 1"
<smoser> arosales, try
<smoser> juju --debug ssh -e aws-go 1
<smoser> that would not be terribly unreasonable if the '-e' was not a juju global flag, but was a flag to the 'ssh' sub command.
<smoser> (and just happened to be a flag to many subcommands)
<arosales> smoser, that does work
<smoser> alright.
<smoser> well thats at least moderately sane
<smoser> (i'd even argue "perfectly fine")
<arosales> but from a mere mortal like me totally unreasonable
<smoser> so did you verify that juju works with saucy ?
<arosales> smoser, so precise works on aws
<arosales> I can ssh
<natefinch> yeah... seems like juju -e should work, even if the command you're running doesn't care about the environment
<smoser> arosales, bigjools reported precise (with his custom image) worked.
<arosales> smoser, I went back to precse on aws as I was getting the above error
<smoser> i wanted to see if saucy worked on aws. to rule out general saucy error.
<smoser> as it really does seem to me that cloud-init is functioning correctly.
<arosales> smoser, trying that now
<smoser> arosales, can you ssh to your precise aws ?
<arosales> smoser, yes
<smoser> i was going to ask you to give me output of 'ec2metadata --user-data'
<arosales> bootstrapping with saucy now on aws
<smoser> in a secure chanel
<arosales> ah, just destroyed
<arosales> let me see if saucy works and I can get you user data off precise
<arosales> smoser, saucy ssh with juju works on aws
<arosales> smoser, so sounds like the issue is Azure specific . .  . ?
<smoser> arosales, can you point me at the doc you have so far ?
<smoser> and i'll try to reproduce it?
<smoser> marcoceppi, http://marcoceppi.com/2013/07/compiling-juju-and-the-local-provider/
<smoser> export GOPATH="~/.juju/"
<smoser> is wrong
<smoser> quoting the '~' explicitly creates a directory called '~'
<marcoceppi> smoser: that's...odd.
<smoser> its expected :)
<marcoceppi> It's odd that I put that quoted in the post
<marcoceppi> I'll update it!
<smoser> 2 other things
<smoser> hm..
<smoser> you need bzr for 'go get'
<marcoceppi> smoser: ack, that was already installed on my system
<marcoceppi> I didn't run these against a "clean" machine
<marcoceppi> Will update the post
<smoser> marcoceppi, becaus eyour'e the expert (i'm follooing your blog)
<smoser> do you know
<smoser> http://paste.ubuntu.com/6011504/
<smoser> anyone else maybe ?
<marcoceppi> I've not recieved that error before. What version of ubuntu? I can try to replicate
<natefinch> smoser: almost certainly because you're running go 1.02 and juju uses 1.1 now
<natefinch> unfortunately apt-get still installs 1.0.2
<marcoceppi> natefinch: ah, was that a recent switch? I've not tried to compile since 1.11.4
<natefinch> marcoceppi: yeah in the last month or so, I think
<marcoceppi> natefinch: ah, sorry smoser, I'll update my blog to reflect the golang ppa too
<marcoceppi> natefinch: actually, I thought there was a ~gophers ppa, but I don't see 1.1 in there
<natefinch> marcoceppi: https://groups.google.com/forum/#!topic/golang-nuts/iJFhI8K5a2Y
<marcoceppi> natefinch: bummer :\
<smoser> yeah, there is no raring either in golang ppa
<smoser> saucy!
<natefinch> marcoceppi: when I started I think I had to use the tarball from golang.org
<sidnei> smoser, marcoceppi: https://launchpad.net/~james-page/+archive/golang-backports
<marcoceppi> sidnei: awesome, thanks@
<marcoceppi> sidnei smoser updated the blog post
<smoser> how do you normally tell juju about ssh public keys?
<smoser> for maas it seems to have explicitly (in config)
<smoser>     authorized-keys-path: ~/.ssh/authorized_keys # or any file you want.
<smoser>     # Or:
<smoser>     # authorized-keys: ssh-rsa keymaterialhere
<smoser> but that is provider specific ?
<marcoceppi> smoser: it normally just uses whatever is in ~/.ssh/id_rsa.pub
<marcoceppi> You can add additional ones using either the two keys above
<natefinch> marcoceppi: I deployed discourse using juju and it looks great except that in the registration confirmation emails, it's using the internal hostname of the EC2 instance, instead of the external hostname.... any thoughts on how to fix that?
<marcoceppi> natefinch: Yeah, you'll need to edit the config/database.yml file
<marcoceppi> I need to add "external hostname" configuration option to the charm
<marcoceppi> to automatically add that information in
<natefinch> marcoceppi: I'd be happy to contribute to the charm :)
<marcoceppi> natefinch: please do!
 * marcoceppi syncs his local version to charm branch
<natefinch> marcoceppi: I didn't think you'd say no ;)
<natefinch> marcoceppi: I am completely new to the charms, so... it'll take me some ramp up time. But especially for this particular charm, I want to use it for personal reasons. So I have skin in the game that it works well :)\
<marcoceppi> natefinch: I've got several fixes to the auto-thin configuration that I need to land, so I'm not sure how far behind my cs:~marcoceppi/discourse version actually is
<marcoceppi> but the github branch is always a little ahead
<marcoceppi> and then I sync when the charm is "stable" again to the charmstore branch
 * marcoceppi enjoys convoluted processes
<natefinch> marcoceppi: lol fair enough
<marcoceppi> but yeah, that part could use a bit of work (ie, update database.yml file and the nginx config with the hostname)
<marcoceppi> I tried to keep the charm pretty simple, as I wanted to use it as an example charm, but it's kind of grown a bit beyond that, so if you have any questions as to why I've backed bits of crack in to the charm, don't hesitate to ask
<marcoceppi> baked*
<natefinch> haha
<natefinch> ok
<smoser> any ideas?
<smoser> http://paste.ubuntu.com/6011776/
<sidnei> whoa
<sidnei> looks like it choked on the cert?
<smoser> http://paste.ubuntu.com/6011792/
<smoser> it sure does look like that. i agree
<smoser> that above is what i've done so far to get here.
<bigjools> morning
<bigjools> hi arosales
<arosales> bigjools, morning
<arosales> bigjools, smoser was unable to reproduce what you and I found on saucy
<bigjools> arosales: because he wasn't using juju
<arosales> bigjools, actually he was
<bigjools> I only saw the azure command line tool being used
<arosales> compiled the latest even
 * arosales not sure if he updated with his latest
<bigjools> arosales: the last comment on https://bugs.launchpad.net/bugs/1214636 doesn't show juju
<_mup_> Bug #1214636: Azure Provider: Deployed service never goes to started <cloud-init:New> <juju-core:Confirmed> <https://launchpad.net/bugs/1214636>
<arosales> bigjools, smoser had to run so he may not have updated his bug with the latest
<bigjools> righto
<thumper> hi folks
<thumper> this branch is giving me the shits
<thumper> I thought it would be a small, simple branch
<thumper> but OH, NO, no it isn't at all
<thumper> just replacing agent.Conf with an interface, and unexporting the structure
<thumper> FFS it is tedious
<thumper> I seem to have all tests passing except for agent/agent_test
<thumper> which is because there are shed loads of explicit struct tests
<thumper> so I left it for last
<thumper> trying to do the bare minimum to get this landed
<thumper> aargghh!
<thumper> that and no ubuntu edge to make me feel better
<fwereade> thumper, ha, I only just got round to thinking "bah, I'd better get one in case a mystery benefactor kicks in 20M at the last moment", but paypal can go fuck itself, so meh
<thumper> :)
<thumper> so no comment on the rest of the rant then :)
<thumper> fwereade: I may throw it at you for review
<fwereade> thumper, just reading backwards
 * thumper looks at the size
<fwereade> thumper, I won't say it'd be a *pleasure*, but I won't complain too much ;p
 * thumper sucks wind
<thumper> 1600 lines ATM
<fwereade> ouch
<thumper>  17 files changed, 441 insertions(+), 446 deletions(-)
<thumper> and this is the simplest thing
<thumper> that made sense
<fwereade> fucking structs
<thumper> long live interfaces
 * fwereade girds his loins in preparation for the morrow then
<sidnei> thumper: im trying to figure out why local provider is not setting the hostname of the containers it creates, and if this is an lxc bug or not
<thumper> sidnei: it normally does
<thumper> sidnei: using clone?
<sidnei> thumper: nope, trunk without my changes
<sidnei> thumper: but im using the daily lxc ppa
<sidnei> so might be a change there
<thumper> sidnei: it should, and is a template param
<thumper> --hostid
<thumper> line 154 of container/lxc/lxc.go
<wallyworld> thumper: i'm going to move the Broker interface from worker/provisioner to (somewhere, probs instance), and make Environ be composed from Broker and remove the duplicated Environ methods
<wallyworld> as part of some refactoring
<thumper> ok
<wallyworld> since part of it is extracting common start nstance code
<wallyworld> i am sharing some of your refactoring pain :-)
<wallyworld> i still don't fully understand why the fcuk we used structs and not interfaces
<wallyworld> maybe whoever did it did not read software enginerring 101
<thumper> I believe it was because "we don't need it yet" was the rationale
<thumper> however, the retrofitting of interfaces is a royal PITA
<wallyworld> there is no such thing as "we don't need it yet" with interfaces
<wallyworld> they are needed for all sorts of things from day one, not the least of which is for tests
<wallyworld> and extensible, refactorable code
<wallyworld> and Go's design almost mandates their use unless you want gobs of cut and paste boilerplate everywhere
<thumper> wallyworld: I agree
 * wallyworld sighs
<thumper> wallyworld: but then again, you and I often do
<wallyworld> often but not always :-)
<wallyworld> like with rugby
<thumper> no, that would be boring
<thumper> so, think you have a shot this weekend?
<wallyworld> nope :-(
<thumper> heh
 * thumper heads to the gym
<wallyworld> we're fooked
#juju-dev 2013-08-22
<davecheney> unsubtle reminder, https://docs.google.com/a/canonical.com/document/d/1eeHzbtyt_4dlKQMof-vRfplMWMrClBx32k6BFI-77MI/edit#heading=h.reab82nqtett
<arosales> bigjools, thanks again for the help on the azure bits
<bigjools> arosales: no prob
<bigjools> wallyworld gets the baton next week and he told me he's really looking forward to it
<wallyworld> no he's not and no he isn't
<bigjools> davecheney: I want to move the gwacl code on to the juju landing bot. Where do I start?
<davecheney> bigjools: i'm sorry I don't know how to do that
<davecheney> john owns the bot
<bigjools> ok HAL
<bigjools> jam, when you get in.... ta :)
<smoser> bigjools, so saucy + azure + juju core "works for me"
<bigjools> smoser: yeah I just got it working too
<bigjools> NFI why it didn't before
<smoser> http://paste.ubuntu.com/6012428/
<smoser> bigjools, i think you just were'nt being patient.
<bigjools> smoser: 30 minutes ought to be enough patience.
<bigjools> and actually I left it several hours
<bigjools> although I have a sneaking suspiscion that it was booting an image without the new cloud-init
<smoser> not on saucy.
<smoser> Cloud-init v. 0.7.3 finished at Wed, 21 Aug 2013 21:33:21 +0000. Datasource DataSourceAzureNet [seed=/dev/sr0].  Up 534.78 seconds
<smoser> thats from bootstrap node.
<smoser> i certainly dont think you could have been running saucy without cloud-init.  there was a fix in a few days ago
<smoser> Cloud-init v. 0.7.3 finished at Wed, 21 Aug 2013 21:48:55 +0000. Datasource DataSourceAzureNet [seed=/dev/sr0].  Up 520.95 seconds
<smoser> oops
<smoser> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1214541
<_mup_> Bug #1214541: hostname setting is erroring out <amd64> <apport-bug> <cloud-images> <saucy> <cloud-init (Ubuntu):Fix Released> <https://launchpad.net/bugs/1214541>
<smoser> but that didn't really stop boot.
<smoser> bigjools, one thing to note, it sure appears to me that you're sending in the linuxprovisioningxml
<smoser> <HostName>default</HostName>
<bigjools> smoser: not sure we even set that
<smoser> oh. maybe not.
<smoser> it probably makes sense to send the hostname as the hostname :)
<thumper> axw: I don't suppose you can work your test speed ups on the ec2 tests?
<axw> thumper: heh :) I was planning to go across the codebase
<thumper> \o/
<axw> just responding to fwereade's comments on manual provisioning now
<thumper> cool
 * thumper is trying to finish off the refactoring branch of doom
<axw> thumper: which is that? agent config changes?
<thumper> axw: yeah, replacing agent.Conf with an interface
<thumper> getting kinda epic
<axw> :)
<thumper> even for the minimum case
<axw> thumper: https://codereview.appspot.com/12831043/patch/8002/9024  -- second last comment. fwereade suggested I ask you about whether the upstart job's name ought to go into agent config
<axw> what do you reckon?
<thumper> hmm...
 * thumper looks and considers
<thumper> axw: so my answer is yes, but not now
<axw> okey dokey
<thumper> axw: I want to add an environment type thing to agent config
<thumper> but that is coming after the mega branch
<axw> environment type thing?
<thumper> so the agent config can have key/value string pairs
<axw> ah right
<thumper> for arbitrary extra info
<axw> *nods*
<axw> I will add a TODO here then
<thumper> kk
<wallyworld> thumper: axw: you guys run the ec2 tests on trunk recently? i have a failure in TestAddresses
<thumper> wallyworld: have you updated goamz?
<wallyworld> thumper: axw: also, reading back scroll, rodger is working on ec2 test speedups
<wallyworld> maybe not
<thumper> my trunk from the other day works
<wallyworld> ok, i was out of date, thanks. will try again in a bit
<axw> wallyworld: I think gz changed something. and yeah I saw the other day he sped them up significantly - not sure where he's at with it now. I'll ping him later
<wallyworld> it may have been rodger not martin
<axw> sorry, I conflated two comments
<axw> yes, rog
<thumper> OOPS: 1 passed, 3 FAILED, 1 PANICKED, 4 MISSED
<thumper> but at least they now freaking compile
<thumper> that is the agent tests
 * thumper digs
<thumper> 2 passed
<thumper> 3 passed
<thumper> 4 passed
<thumper> kid run
 * thumper stabs useless tests  in the face
<thumper> testing something that never happens and doesn't make sense
<thumper> omg might be there...
<thumper> running make check
<axw> woo :)
<thumper> heh
<thumper> I got more strict in what I expect
<thumper> now I have a few more failures to fix
<thumper> although should be trivial
<thumper> not as trivial as I had hoped...
<thumper> 22 files changed, 781 insertions(+), 866 deletions(-)
<thumper> totalling around 2.5k changes
<thumper> at least william will do it :)
<thumper> finally, all tests pass
<thumper> for the interested: https://codereview.appspot.com/13164043 (and fwereade)
 * thumper goes to take kid to sport
<thumper> back for the meeting
<axw> adios
<jam> bigjools: I'm around, where do you need gwacl moved to?
<jam> also, wallyworld, dimitern and some other people should know how to do it in case I'm not available
<bigjools> jam: I currently have a landing bot on my home server, and I want it moved so the juju bot lands it please
<jam> bigjools: ah, just moved to the bot. Sure. Is it just tarmac?
<bigjools> the project is part of juju
<bigjools> yes, thanks
<bigjools> I can give you the tarmac conf later if you need it - my home server decided to crash and burn so I am rebuilding it
<jam> bigjools: so the "standard" config I have is this: http://paste.ubuntu.com/6012846/
<jam> I think we need a little bit of change for your code, because we need to put it in a different GOPATH so you can land stuff that would otherwise break juju-core until you have the patch that matches juju-core again.
<jam> bigjools: but otherwise, whatever verify_command, etc you want , just let me know (make check, etc)
<jam> bigjools: maybe it was only with older tools but "-gocheck.v" wasn't compatible with "./..." when I've tested it in the past.
<jam> I guess it doesn't matter terribly, but I'm guessing your bot wasn't testing "dedent" and "logging" on each run.
<jam> "make check | grep dedent" doesn't show anything for me
<bigjools> jam: make check is all you need
<jam> bigjools: but it doesn't run the tests in the subdirs
<jam> (as I mentioned just now)
<jam> it is a bug in "go test" but if you pass special flags like "-gocheck.v" it doesn't notice "./..."
<bigjools> make check works fine locally, what's up with the bot?
<jam> bigjools: test it yourself "make check | grep dedent"
<jam> empty output for me
<jam> on my personal machine
<jam> go 1.1.1
<bigjools> why are you grepping for dedent?
<jam> bigjools: because that is part of your test suite "dedent/dedent_test.go"
<bigjools> oh////
<bigjools> meh
<bigjools> sorry I am slow todya
<jam> bigjools: so your "make check" doesn't run tests in subdirs
<bigjools> jam: ok so the regular go test ./...
<bigjools> and
<bigjools> go build example/*/run.go
<bigjools> we can add something for formatting later
<bigjools> alternatively we can add a new make target for the bot
<davecheney> afternoon all, see in yo in a few hours for the meeting
<natefinch> jam: is the hangout link different from the morning standup link?
<axw> natefinch: https://plus.google.com/hangouts/_/bf3f4cfe715017bf60521d59b0628e5873f2a1d3?authuser=1
<natefinch> axw: thanks
<jam> natefinch: you should be on the calendar entry, which is the g+ hangout we use. Just checking that it is working for you.
<mramm> hey, for some reason I can't get back in the hangout
<nate-finch> jam: Yep, I have the calendar entry - I just missed the link in the event... I'm too used to seeing a long ugly URL, not a small "Join Video Call" link :)
<wallyworld> fwereade: if you want a (slightly) smaller branch ( 2392 lines (+425/-580) 36 files modified) as an entree to thumper's: https://codereview.appspot.com/13081044/   plus I updated https://codereview.appspot.com/13047043/
<fwereade> wallyworld, aww, thanks ;p
<wallyworld> i'm here to please :-P
<wallyworld> it's a net deletion of code which i a good thing, a lot of duplicated stuff gone :-)
<fwereade> wallyworld, cool, thanks
<axw> jam: just looking at the upgrade bug again...
<axw> what happened was, I did an upgrade-juju --upload-tools in local
<axw> the bootstrap machine agent got the tools it needed to upgrade, and restarted
<axw> then upgrade-juju proceeded to attempt to upload additional tools, and failed
<axw> *additionally* the machine agent just restarts in a loop
<fwereade> axw, huh, don't we upload all the tools before we set the agent version?
 * axw has a look
<axw> I have no idea :)
<fwereade> axw, (and, sorry, which machine agent? 0?)
<axw> yes, 0
<fwereade> axw, don't suppose the log said anything useful about why?
<axw> there are some errors about connecting to a websocket? but I haven't had time to dig
 * fwereade sighs
<fwereade> thanks axw
<fwereade> I think it's probably clear how to fix it
<fwereade> we just need to wait to make connections rather than bouncing every time things go wrong
<axw> fwereade: yes it does appear that the version is set last. Perhaps I already had machine-0 in a bad state when I attempted to upgrade again
<axw> but the simple upgrade-juju is still a problem
<axw> i.e. the steps in #1214676 are repeatable
<_mup_> Bug #1214676: upgrade-juju in local environment causes bootstrap machine agent to restart continuously <juju-core:Triaged> <https://launchpad.net/bugs/1214676>
<fwereade> axw, sure, I'm not trying to claim otherwise -- thanks :)
 * fwereade breakfast
<jam>  axw: it sounds like we should be uploading all the tools before we ask things to upgrade themselves.
<axw> jam: possibly we are. I'm no longer sure about the interaction of this bug and the one I link to the description
<axw> I'll have to try and reproduce the original one again
<axw> anywho, gotta get dinner on - I'll be back for a bit later
<jam> np
<jam> have a good evening
<fwereade> wallyworld, remind me: are we now requiring that we bootstrap with matching version rather than picking latest? or is that really really imminent?
<wallyworld> fwereade: soon. i justed wanted to get the core refactoring done first. there will be a lot of tests etc to fix when we change the boot strap tools search
<wallyworld> and the branch was already huge
<fwereade> wallyworld, sure, understood :)
<fwereade> wallyworld, btw, why the change from tools-metadata-url to tools-url? the first feels more correct to me
<fwereade> mgz, you have quite a lot of branches up that I think are already LGTMed, are there any I should be looking at?
<mgz> fwereade: nope, though last check I think a couple of the trivial ones weren't lgmted, I'll look again
<yolanda> hi, i'm using juju-core with canonistack. I destroyed a service but now it keeps on status "dying" for all the time. Tried to terminate the machine, removing the unit, without success, and now i cannot redeploy that again although i destroy all the environment. Is there some way where i can manually destroy that buggy service?
<mgz> nope, all ready to land, I'll queue 'em up
<mgz> yolanda: you may be able to resolve the unit, then destroy it, depending on exactly what state things got confused in. otherwise, it's destroy-environment to wipe everything
<yolanda> mgz, machine was stuck with out of memory errors, but charm state looked as started
<yolanda> i tried everything without success, now i manually removed the machine with nova delete, but no progress
<mgz> yolanda: what's the current state of the unit? can you pastebin status?
<mgz> ah, probably not easily cleanupable any more then
<mgz> destroy-environment for the wipe :)
<yolanda> http://paste.ubuntu.com/6013589/
<yolanda> but that's after all the things i've done
<yolanda> i was trying to setup a jenkins unit, but complained about memory
<yolanda> ok, i'll destroy everything and try again
<mgz> you probably need a constraint on mem=somethingbig for the jenkins service
 * TheMue => lunchtime
<yolanda> mgz, will try with 2G now
<jam> fwereade: https://plus.google.com/hangouts/_/6c44984654c782bf948ffabe55b69414a3a38aea ?
<arosales> jam, thanks for the ideas on azure
<nate-finch> I was up at 4am, and I'm in the standup....
<jam> mgz: wallyworld: https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471 standup ?
<nate-finch> mgz: standup
<mgz> ta
<jam> mgz: poke if you could ask jamespage whether they will be getting go-1.1.x into precise's backports
<yolanda> mgz, problems with jenkins again, now nova show machine is on error state, and juju shows it as pending, is there any way to solve it without destroying?
<yolanda> i got a nasty error:
<yolanda> http://paste.ubuntu.com/6013816/
<jamespage> jam: you could ask me yourself :-)
<jam> jamespage: I thought given it was Thurs he might be sitting right next to you, and could have the high-bandwidth conversation
<jam> the question also originated on our phone call, but he lagged out for a bit.
<jam> but yes, jamespage, what is the current plan for getting juju into precise-backports and its associated build dependencies
<jam> ?
<fwereade> TheMue, ping
<fwereade> TheMue, I'm in the standup hangout when you're ready
<TheMue> fwereade: I wanted to to it here, but hangout is fine too.
<TheMue> fwereade: still there?
<jam> jamespage: that's why I don't ask you, you don't answer :)
<jamespage> jam: urgh - sorry
<jamespage> I have my head in a nasty openvswitch datapath compatibility issue
<jamespage> and was mid cherry pick
<jam> np
<jam> it isn't urgent
<jamespage> jam: right now backporting is on hold
<jamespage> jam: two reasons - 1) we now have a stable PPA and 2) 12.04 cloud archive is going to get a cloud-tools pocket which would have golang + other stuff in
 * fwereade needs to be off a bit early today, see you all soon
<mgz_> later fwereade!
<TheRealMue> cu
<thumper> morning
 * thumper stretches
<weblife> morning thumper
<thumper> hi weblife
<thumper> fwereade: hey
<thumper> fwereade: you really around?
 * thumper sighs
<thumper> lots of test failures after merging trunk
 * thumper does some manual testing
<thumper> bah humbug
 * thumper has broken the local provider... locally at least
 * thumper enfixorates
<fwereade> thumper, heyhey
<thumper> fwereade: hey
<thumper> fwereade: I was going to ask about the breaking bootstrap change
<thumper> and now my local provider is unauthorized in bootstrap
 * thumper needs to figure out how...
<fwereade> thumper, it's just that juju currently bootstraps with the latest available tools, regardless of client version -- but is it possible it's finding *older* installed tools and trying to bootstrap with those?
<fwereade> thumper, wallyworld is going to be changing it to demand matching tools on bootstrap, which I think will help
<thumper> I remember change some behaviour around the initial password
<thumper> fwereade: but I didn't change that, right?
<fwereade> thumper, ehhh, I felt a tiny twinge around that but read it quite carefully and *thought* it was solid
<thumper> yeah, I thought it was solid too
<thumper> but not enough to commit it to trunk without trying it out :)
<thumper> ec2 is working
<thumper> hmm... "bzr ssh 0 " doesn't do what I mean
<thumper> I mean "juju ssh 0"
<thumper> my computer needs more mind reading
<fwereade> thumper, well, I am inclined to prejudge it as all being about the upload/sync confusion in the local provider, but that says more about me than about juju I think
<fwereade> thumper, ha, yeah, I confuse the two quite a lot
<thumper> can't wait until we can change the certs to be stored in base64
<thumper> kinda crazy looking at the config files now
<thumper> fwereade: no, I don't think it is that...
<thumper> fwereade: more likely to be assumptions made during the initial config writing and bootstrapping
<fwereade> thumper, ha, yes, tis true those bits don't quite match the non-local case either, right?
<thumper> no, it is a bit hand wavey
<thumper> and "this is not the config you are looking for"
<fwereade> thumper, haha
<thumper> there is a very slight change in behaviour
<thumper> prior, when writing out the new password
<thumper> it would not keep the old one
<thumper> but replace it with ""
<thumper> now I keep the old one around
<thumper> however I don't think that is the current problem
<thumper> there is an assumed two-phase commit with the passwords
<thumper> which isn't quite there
<thumper> save the password, then change it
<thumper> and hope all is god
<thumper> good
<thumper> however since it is possible for the rename of the file to fail
<thumper> there is a possibility, however small, that we get stuffed up
<fwereade> heh, damn, I missed that; I'd prefer not to lose that just because it's a possible screwup
<fwereade> I doubt it's what you're seeing though
<thumper> it is no different to before
<thumper> just I realised it is there
<thumper> the behaviour is the same
<thumper> no, this isn't what I'm seeing
<thumper> it is just an unauth on the initial create log db part in mongo
<fwereade> ah, that's good then, I *thought* I had verified it still was, but was feeling ready to assume hallucinations on my part
<thumper> ah fuck
<thumper> I think I know what it is
<thumper> fuckity fuck fuck
<thumper> it is the magic "localhost"
<thumper> which I noticed while doing the local provier
<thumper> provider
 * thumper double checks
<thumper> yep...
<thumper> at least that fixed
<thumper> it
 * thumper wonders if he'll conflict with wallyworld
<wallyworld> hope not
 * thumper just checks
<thumper> wallyworld: how are you doing?
<wallyworld> ok, landing a bunch of stuff
<wallyworld> thumper: i was hoping to get in first with my stuff. now i will have lots of conflicts i'm sure :-(
<thumper> wallyworld: well, mine is in the queue too :)
<bigjools> good day
<thumper> the freaking epic add an interface branch
<thumper> hi bigjools
<wallyworld> thumper: yours in landing now
<thumper> \o/
<bigjools> wallyworld: where did you buy your microserver?
<wallyworld> bigjools: some online place, i searched on staticice.com.au
<bigjools> ok ta
<bigjools> hp's site is useless
<wallyworld> yep
<wallyworld> thumper: only one conflict :-)
 * bigjools hugs hot coffee
 * wallyworld still has no coffee machine :-(
<thumper> wallyworld: still?
<thumper> geez
<thumper> go get a rental
 * thumper needs another coffee
<thumper> wallyworld: I have some errands in town
<thumper> so will be back later
<bigjools> wallyworld: you can always work here
<thumper> happy to have my epic branch landed though
<wallyworld> thumper: i'm about to go to the police station myself
<thumper> wallyworld: is you tools branch the one that fixes bootstrap?
<thumper> so don't try to bootstrap with old tools?
<thumper> wallyworld: ack
<wallyworld> thumper: not quite. next branch
<thumper> cool
<thumper> I'll be doing more reviews this afternoon
<wallyworld> mine does a bunch of other pre -req stuff
<thumper> around axw's work on manual provisioning
<wallyworld> yay
 * thumper nods
<thumper> I want to add a new branch for adding string key/value pairs into agent config
<thumper> which would lead to the logging branch working
<thumper> and tweaking the maas stuff for containers
<thumper> bigjools: I have a fix for the maas provider which is less smash of the network conf
<thumper> bigjools: using "source" as an include mechanism
<thumper> bigjools: so when maas gets better at network handling
<bigjools> thumper: splendid.
<thumper> it won't kill it
<bigjools> when you say better ...
<thumper> :)
<bigjools> we're adding more constraint types next week
<thumper> what does that mean for me?
<bigjools> so the core had better hurry up and support it :)
<bigjools> custom tags as per the pyjuju
<bigjools> --to in the new world IIRC
#juju-dev 2013-08-23
<thumper> wallyworld: WTF?
<thumper> wallyworld: I'm just frustrated that we're playing musical chairs with tools
<thumper> code went from environs/tools -> tools -> agent -> agent/tools -> and back again
<thumper> wallyworld: reading the description though it kinda makes sense, just crazy talking
<thumper> wallyworld: I always throught that we should have reasonable top level packages
<axw> thumper: what's the issue with provisioners tearing down machines without an environment instance? (re manual provisioning)
<axw> i.e. why exactly do we need the provider type to tell the provisioner not to touch it?
<thumper> axw: hangout?
<axw> thumper: sure, just gotta disconnect from the monitor
<axw> one min
<thumper> ?!
<thumper> wat?
<axw> webcam is on my laptop screen only :)
<axw> ... which was facing the wall
<axw> thumper: https://plus.google.com/hangouts/_/dc98b01f03a4224c9e7e30ce8f28aa33933dd6b1?authuser=1&hl=en-GB
<wallyworld> thumper: with the tools, i needed to separate out the stuff that fetched the tools from the provider, vs the stuff that managed the tools on disk. and then due to circular imports, i needed a core tools package to hold the key interfaces
 * thumper nods
<thumper> I kinda understand
<thumper> I was frustrated when my top level tools got merged into agetn
<thumper> which I then had to extract again
<thumper> just to see some bits move back into tools
 * thumper shrugs
<thumper> I thought it was a good idea before, it still is
<wallyworld> thumper: i do like that now there's a clear semantic separation between locating and downloading tools vs unpacking and using them. the former interacts with providers and metadata and urls and storage; the later needs to know none of that, just how to unpack into a lib somewhere
 * thumper nods
<wallyworld> on a separate note, i got home from the cops and my internet is down :-( the whole street is out. lucky i still have data left on my phone
<bigjools> wallyworld: I have coffee and freshly-baked bread
<wallyworld> bigjools: i'll be there in 15, ok?
<bigjools> np :)
<bigjools> you might hear me swearing a lot
 * wallyworld goes to the bigjools coffee shop
<axw> thumper: re "I think if part of the bash script that gets executed to install the
<axw> machine agent first checks to see if one is there, and bails if it is,
<axw> that should be reasonable protection."
<axw> that's happening now
<thumper> ok, cool
<axw> I just thought it a bit sucky that the machine ID gets allocated and can't be reused
<thumper> yeah, it is a bit sucky
<thumper> this is because we key things on ID
<thumper> which we shouldn't
<thumper> but instead against a guid for the machine
<thumper> but we suck
<axw> :)
<axw> also, for the Remove... mind if I make it Remove/RemoveAndStop, rather than Remove/RemoveOnly?
<thumper> there is a long standing agreement that we should refactor at somestage in the future to move to linking with guids
<thumper> I originally had RemoveAndStop
<axw> ok
<thumper> however
<thumper> all the existing use cases were to be remove and stop
<thumper> so was going for minimul changes
<thumper> I'd prefer "StopAndRemove"
<thumper> over RemoveAndStop
<axw> yes sorry, that is the order isn't it
 * thumper nods
<axw> I prefer this over RemoveOnly, because "Remove" doesn't tell me what else it's doing
<thumper> fair enough
<thumper> I'm happy with that
<axw> cool
 * axw gets a little hung up on naming sometimes
<thumper> me too, and this is a good thing
<thumper> naming things is very important
<thumper> one of the two hardest problems with computing
<thumper> the others are: cache invalidation, and off by one errors
<thumper> bigjools: how long for wallyworld to get to you?
<wallyworld> bigjools: where's my !@@!!$$!@ coffee
<thumper> hi wallyworld
 * bigjools prepares enema for wallyworld
 * wallyworld spreads
<thumper> wallyworld, axw: when shall we schedule a package review time?
<wallyworld> what do you mean by when?
<thumper> wallyworld: well, we should look at what to look at in advance, then schedule a time to talk about it
<wallyworld> ok, i wasn't sure if you were talking about setting up a rescurring time each week, or just the next one off
<axw> thumper: it's all the same to me... maybe Wednesday, same as the others (assuming that's regular)?
<thumper> wallyworld: we should have a recurring one
<thumper> just at a friendly time for us
<thumper> than midnight
<thumper> or dinner time in perth
<wallyworld> maybe 1pm AEST?
<thumper> that is 3pm here
<thumper> school pickup time
<wallyworld> i was thinking of summer time
<thumper> 3:15 should be ok
<wallyworld> ok
<thumper> it isn't summer yet
<wallyworld> maybe 1:30 to be safe
<thumper> not for us here yet
<thumper> kk
 * thumper puts it in the calendar
<axw> that is fine with me
<wallyworld> thumper: i'm trying the local provider for the first time. i assume status will show pending while the image is downloaded in the background?
<thumper> yep
<wallyworld> cool. just smoke testing my great big env refactoring
<thumper> :)
<wallyworld> i notice bootstrap node is raring, while node 1 is precise
<thumper> ack
<thumper> well the machine you are running is obviously raring
<thumper> the default-series is still precise though
<wallyworld> thumper: btw, this isn't the previously landed tools work, this is new code i haven't landed yet :-)
<wallyworld> ok, so bootstrap matches local machine
 * wallyworld taps fingers and waits, and waits
<thumper> ack
<wallyworld_> thumper: if i get a 'hook failed: "config-changed"' error when deploying wordpress and mysql, i should be ok to assume staring instances went ok, right?
<thumper> yes, they would have started
<wallyworld_> the instances appear running, but the charm deployment went badly for some reason
<wallyworld_> my changes were around insteace start and bootstrap
<wallyworld_> and juju status shows stuff running, but agent state has issues
<wallyworld_> but then again, my machine died 1/2 way through starting first instance, i removed it and tried again
<wallyworld_> thumper: hmmm. i did a destroy-environment, but it left behind the provider-state file and other artifacts :-(
<wallyworld_> so bootstrap fails the next time
<wallyworld_> i need to delete those manually
<thumper> hmm...
<thumper> wallyworld_: it doesn't normally
<wallyworld_> i'll try again
<wallyworld_> how long does agent-state normally say pending
<wallyworld_> instance-id is set after deploying mysql, but agent still pending
<wallyworld_> agent finally started. took ages
<thumper> :)
<wallyworld_> so it all looks ok, time to land i think
<thumper> wallyworld_: I don't think agent state is fully running until after the install hook has run
<wallyworld_> ok
<wallyworld_> thumper: last time, i didn't sudo to destroy env and missed the error message
<wallyworld_> so all good
<thumper> kk
<davecheney> 1.13.2 has been tagged as revno 1703
<davecheney> # launchpad.net/juju-core/provider/ec2
<davecheney> src/launchpad.net/juju-core/provider/ec2/ec2.go:131: inst.Instance.IPAddress undefined (type *ec2.Instance has no field or method IPAddress)
<davecheney> src/launchpad.net/juju-core/provider/ec2/ec2.go:136: inst.Instance.PrivateIPAddress undefined (type *ec2.Instance has no field or method
<davecheney> any ideas ?
<axw> davecheney: there was an email the other day to update goamz
<axw> probably that
<davecheney> axw: weird, goamz is always pulled from source by the build
<davecheney> nest goamz lp:~gophers/goamz/trunk src/launchpad.net/goamz
<davecheney> has the location of the goamz trunk moved ?
<axw> ah sorry, I was thinking of the bot.. hmm
<axw> I doubt it ...
<axw> I'll update my local copy, one moment
<axw> davecheney: trunk's ec2.Instance definitely has a field called IPAddress
<davecheney> ok, thanks
<davecheney> axw: what revno do you have for juju-core trunk ?
<axw> davecheney: 1690 (I'm not up to date tho)
 * davecheney throws a chair at bzr revnos
 * davecheney fiddles with build recipes
<rogpeppe1> mornin' all
<axw> morning rogpeppe1
<rogpeppe1> axw: hiya
<rogpeppe1> hmm, not sure i'll be able to do any G+ hangouts today. download & upload speed both 0.05Mbps
<axw> wow :(
<axw> rogpeppe1: what do you normally get, out of interest?
<rogpeppe1> axw: ~3-4 down and 0.5-1 up
<rogpeppe1> axw: but i'm working from somewhere else today
<axw> slightly better than me
<axw> ah
<rogpeppe1> axw: it seems that internet bandwidth isn't the highest priority in stately homes :-)
<axw> heh :)
<TheMue> ah, today is a good day, 32 in 7 out. but that's not usual.
<TheMue> morning btw
<axw> morning TheMue
<TheMue> axw: heya
<rogpeppe1> TheMue: yo
<rogpeppe1> !
<TheMue> rogpeppe1: how has your holiday yesterday been?
<rogpeppe1> TheMue: really good, thanks
<rogpeppe1> TheMue: i went up high into the Cairngorm mountains
<TheMue> rogpeppe1: btw, do british people say holiday or vacation?
<rogpeppe1> TheMue: holiday, usually
<TheMue> rogpeppe1: thx
<TheMue> rogpeppe1: Cairngorm? google maps click
<TheMue> rogpeppe1: ah, Scotland already?
<rogpeppe1> TheMue: yes
<TheMue> rogpeppe1: so it matches my Amy concert, hehe
<rogpeppe1> TheMue: your Amy concert?
<TheMue> rogpeppe1: yep, Amy Macdonald on Wednesday
<rogpeppe1> TheMue: here's a photo from yesterday: https://docs.google.com/file/d/0B_ViUkJwG88Cc1JrUUZLUVprZVE/edit?usp=sharing
<TheMue> *sigh* don't make me jealous
<rogpeppe2> fwereade: this speeds up provider/ec2 tests considerably: https://codereview.appspot.com/12787050
<rogpeppe2> mgz, jam, TheMue, nate-finch, axw: review appreciated
<axw> rogpeppe2: looking
<rogpeppe2> axw: thanks
<axw> 80->3s=awesome
<fwereade> rogpeppe2, that looks sexy as hell, but I'm concerned I'm not reading it properly because it's a major context switch right now
<rogpeppe2> fwereade: np
<fwereade> rogpeppe2, I'd prefer it if someone with more recent env-testing experience did the actual LGTM
<rogpeppe2> fwereade: yeah. mgz?
<rogpeppe2> fwereade: or perhaps rvba or jtv2 ?
<fwereade> rogpeppe2, mgz/jam spring to mind, given the openstack/ec2 similarities
<jtv2> Huh what?
<rogpeppe2> fwereade: yeah
<jtv2> rogpeppe2: hey!  You fixed my goamz bug?  Great!
<rogpeppe2> jtv2: which bug was that?
<jtv2> That was a 3Ã speedup of the overall test suite on my machine.
 * jtv2 digs
<jtv2> rogpeppe2: https://bugs.launchpad.net/goamz/+bug/1209480
<_mup_> Bug #1209480: s3's attempt strategy is very long for tests <goamz:New> <https://launchpad.net/bugs/1209480>
<rogpeppe2> jtv2: ha, cool, i didn't know it had a bug
<jtv2> I was a bit conservative in the numbers there...  Actually I saw about a 150Ã speedup in the EC2 provider's tests.
<jtv2> Anyway, did you need a review?
<rogpeppe2> jtv2: a review is always good!
<rogpeppe2> jtv2: the MP is actually mostly about refactoring the ec2 local provider tests
<rogpeppe2> jtv2: they'd got pretty crufty
<jtv2> Ah, that's good too...
<jtv2> BTW it's usually a good habit to go through the changes in the description.  Helps reviewers get their bearings, but also, has been found to correlate with much lower defect rates.
<jtv2> How does the change in localNonUSEastSuite work?  I notice that it's no longer getting its attributes set in registerLocalTests.
<jtv2> Just not needed?
<TheMue> rogpeppe2: not yet through to understand everything, but so far a nice one, indeed
<rogpeppe2> jtv2: yes, just not needed
<axw> jtv2: it's set in the suite's setup
<axw> ?
<rogpeppe2> axw: yeah
<rogpeppe2> jtv2: actually, i think that's a good point - we could initialise the test config inside SetUpSuite inside both localServerSuite and localLive suite, and that might read more nicely
<rogpeppe2> jtv2: the main change in localNonUSEastSuite is that it doesn't embed jujutest.Tests - it wasn't necessary and I think it made matters more oblique and confusing.
<axw> reviewed rogpeppe2
<rogpeppe2> axw: thanks!
<jtv2> Good stuff.
<axw> nps, thank you for speeding them up :)
<axw> fwereade: in case you missed it, I fixed the tools issue in manual provisioning since my last batch of replies
<axw> I think I have anyway :)
<fwereade> axw, cool, thanks, I'll be back to reviewing later today
<axw> thanks
<fwereade> axw, it sounded like you understood the issues, I expect it's fine :)
<jtv2> rogpeppe2: for extra karma, perhaps you should assign bug 1209480 to yourself and close it.  :)
<_mup_> Bug #1209480: s3's attempt strategy is very long for tests <goamz:New> <https://launchpad.net/bugs/1209480>
<axw> I did understand the problem, not so much how to solve it until I looked properly
<axw> I'm off now, have a nice weekend all. See you on Monday wallyworld_ and bigjools
<rogpeppe2> axw: have a great weekend!
<rogpeppe2> afk for 10 minutes or so
<jtv2> rogpeppe2: done.  Forgot the magic word originally, so review notes and vote are in separate comments.
<rogpeppe2> jtv2: thanks
<rogpeppe2> fwereade: here's (finally!) the branch that introduces EnvironProvider.Prepare: https://codereview.appspot.com/13187043/
<fwereade> rogpeppe2, awesome!
<rogpeppe2> fwereade: i'm toying with a slightly different way of doing things in the dummy provider, but i wanted to get it out and looked at - it's not too bad as is, i think
<fwereade> rogpeppe2, I will try to get on that soon, but I'm still in somewhat heavy email mode -- mgz, since you're OCR, can I ask you to put that one near the top of your queue please? we have a few things blocked on it :)
<mgz> sure
<nate-finch> I'm trying to deploy stuff to amazon using juju, but the amazon instances that get deployed only have 8 gigs of space in /  ... what's up with that?  is there a way to fix that?  an m1.small is supposed to have 160 gigs
<mgz> hm, wonder if that's fallout from the rootdrive change?
<mgz> what happens if you pass that as a constraint with a bigger value?
<nate-finch> lemme give it a try
<fwereade> nate-finch, mgz: it's always been the case
<fwereade> nate-finch, mgz: I extracted a promise from sidnei that he'd add handling to the ec2 provider to interpret the constraint, but it had to come behind some other things
<mgz> right, didn't think he'd landed a change to the ec2 params yet
<fwereade> nate-finch, mgz: the instance storage still exists, so you probably do have gigs and gigs available
<fwereade> nate-finch, mgz: but not on the root drive
<nate-finch> fwereade: right
<mgz> just the 8G values in the hardcoded dict
<fwereade> mgz, yeah, that really is just documentation of the existing situation
<nate-finch> fwereade: right, so what's the fix? Is it possible to pass in a constraint from the command line?
<mgz> nate-finch: not currently
<nate-finch> mgz: wow, that is um... broken
<mgz> you probably don't actually need lots of root space though? what's breaking for you?
<nate-finch> I'm trying to deploy discourse, which deploys to /home/discourse/discourse   it also stores uploaded images and all that there.
<mgz> you can probbly fix the charm then...
<mgz> charms doen't currently have proper interfaces for interacting with storage, which does make some of this a little painful
<nate-finch> Yeah, I was looking at the charm
<nate-finch> Well, so... the thing is, it looks like there is no other space... there's the temporary disk space under /mnt ... but I think the real EBS space is simply unused
<mgz> but you could potentially mount that home dir on the block storage or something
<nate-finch> right
<nate-finch> I guess my thought was, if I get 160 gigs with the instance regardless, why not just create root with 160 gigs?
<mgz> it might actually cost more, I can't remember how it works
<nate-finch> it looks like if you're using on-demand instances, that it's just a flat per hour fee... but I might be misreading that.... amazon is like 8 pages of tables that all probably interrelate somehow
<rogpeppe2> nate-finch: standup?
<nate-finch> oops thanks
<rogpeppe2> mramm: standup?
<fwereade> rogpeppe2, ok, google hates me
<fwereade> rogpeppe2, or virgin
<fwereade> rogpeppe2, or someone
<mgz> god?
<fwereade> mgz, I knew *that*
<mgz> :)
<fwereade> doesn't look like it wants me back, I'm going back to my emails :)
 * fwereade lunches
<TheMue> fwereade: the branch is in again for review
 * TheMue is at lunch too, but for a longer time. bbl.
<rogpeppe2> mgz: responded to your review: https://codereview.appspot.com/13187043/
<rogpeppe2> mgz: does the 'bot know about dependencies.tsv yet?
<mgz> rogpeppe2: not yet, but it's one of the changes I'm still wroking on
<rogpeppe2> mgz: ok, cool. just realised that i should change it in one of my recent branches
<sidnei> nate-finch: yes, the 8G is as before, we're not handling the root-disk constraint in ec2 just yet. need to expose passing a block device mapping at start-instance in  goamz
<nate-finch> sidnei: is that something I can help implement? I have some interest in seeing it work :)
<sidnei> nate-finch: definitely!
 * rogpeppe2 goes in search of lunch
<nate-finch> mgz: the addressability changes... are they actually used anywhere? I was trying to figure out where in the live code they're used, but it looked like maybe they're not actually hooked up to anything yet?  Am I wrong?
<mgz> the compat loop via dnsname is used
<mgz> otherwise, it's not
<nate-finch> mgz: I don't know what the compat loop is :)
<mgz> we changed the dnsname implementations to use Addresses
<mgz> and DNSName is sill used
<nate-finch> mgz: ahh I see
<mgz> but PublicAddress and PrivateAddress methods don't yet look up things from state
<sidnei> natefinch: sorry, forgot to follow up. here's the bug: https://bugs.launchpad.net/juju-core/+bug/1212688
<_mup_> Bug #1212688: ec2: pass root-disk constraint via block device mapping <juju-core:Triaged by sidnei> <https://launchpad.net/bugs/1212688>
<sidnei> natefinch: can i assign it to you?
<natefinch> sidnei: Let me see if I can find time to work on it over the weekend. I have stuff due next week that would pre-empt that, and I wouldn't want someone else to *not* work on the bug because it was assigned to me.  If you know what I mean.
<sidnei> sure.
<sidnei> someone else would probably be me :)
<natefinch> haha
<natefinch> basically I just don't want it to be officially on my plate, so my boss doesn't get mad at me for working on something other than my assigned work. :)
<natefinch> I'll do it my spare time as soon as I can, and if someone else needs/wants to fix it before then, that's cool.
<natefinch> is there a way to migrate from one provider to another?   Like if you start off on a small MaaS and then want to migrate to amazon or something to scale out further
<sidnei> natefinch: migrate or just have an environment with things into multiple providers?
<natefinch> sidnei: migrate
<sidnei> natefinch: not as in live migration i guess?
<fwereade> natefinch, migration is not in the roadmap; bursting to another cloud is, although some way down the line: the manual provisioning work is one of the steps we're taking in that direction
<natefinch> sorry, had to step out for a bit. fwereade, sidnei: thanks.  just curious (friends and family keep asking me these questions about juju that I don't know the answers to)
<fwereade> natefinch, np, it's a pleasure
<teknico> wedgwood: about those charm-helpers lint errors, I went a little overboard: :-) https://code.launchpad.net/~teknico/charm-helpers/lint-fixes/+merge/181859
<mgz> rogpeppe3: whoops, didn't hit send on my review reply. I think it's landable when the prereq is in.
<mgz> (and conflicts resolved, joy joy)
<rogpeppe3> mgz: cool, thanks
<rogpeppe3> mgz: the problem with withState is that it makes the logic more complex in places where we've got multiple return values and several return statements. we can do it, of course, with named return parameters, but the benefit isn't quite so clear then.
<mgz> hm, yeah
<TheMue> have a nice weekend everyone
<rogpeppe3> off for a bit
<rogpeppe> i'm getting lots of "port in use" error when testing. what's going on?
<rogpeppe> i wonder if the port selection logic has changed
 * rogpeppe has reached eod.
<rogpeppe> g'night all
<natefinch> rogpeppe: g'night
#juju-dev 2013-08-25
<Soroush731> join #wirelessarmy
<davecheney> arosales: ping
<davecheney> arosales: bigjools says that there is something called a 'enterprise account
<davecheney> for azure that I should get you to add my credentials too ?
#juju-dev 2014-08-18
<menn0> davecheney: i've just pushed more wrench changes again. can you have another quick look?
<menn0> if the scanner returns an error it's logged now
<menn0> and various other things you raised have been addressed (except where I don't think they should be :-p)
<davecheney> menn0: yup, LGTM, thanks
<menn0> davecheney: cheers
<davecheney> argh
<davecheney> thumper:
<davecheney> all this canWatch("") stuff is only so the tests can check the error path
<davecheney> in the real code they are hard coded to return true
<davecheney> but in the tests we set them to return false just so we can cause an error which we can then check ...
<thumper> hmm...
<thumper> seems suspect
<davecheney>         if _, ok := <-watch.Changes(); ok {
<davecheney>                 result.NotifyWatcherId = p.resources.Register(watch)
<davecheney> what the code wants to do is cause this line to fail
<davecheney> but it can't, instead it causese the line above
<davecheney>         if !canWatch("machine-0") {
<davecheney>                 return result, common.ErrPerm
<davecheney> to fail
<davecheney> so it gets it's error
<davecheney> thumper: so i can do two things
<davecheney> 1. supply a dummy, but valid tag
<davecheney> 2. remove the logic and the test
<davecheney> as it can never be try in production
<davecheney> 1. solves todays problem by pushing it off onto someone else later on
<thumper> where is the code that always returns true?
<davecheney> state/apiserver/provisioner:NewProvsionerApi
<davecheney> for example
<davecheney> the tests's don't use that function, the construct a provisioner by hand
<thumper> all I'd want to see a test for is that the provisioner api end point can watch the environment
<thumper> any tests for failures there is a waste IMO
<thumper> the auth func tests the tag
<davecheney> ok, let me prepare a proposal
<wallyworld> axw: morning, you have a minute?
<axw> wallyworld: heya, yep
<thumper> although it does seem that there should be a check around whether or not the machine can read the secrets
<thumper> davecheney: as that is dependent on the state server job
<davecheney> thumper: yes, that is different
<thumper> ok
<wallyworld> axw: standup hangout?
<davecheney> thumper: https://github.com/juju/juju/pull/533
<davecheney> i'd appreciate your thoughts
<thumper> ok
<davecheney> this is option (2)
<davecheney> which is more wholesome
<davecheney> but more difficult to stoumach
<davecheney> thumper: thanks for the review
<thumper> np
<davecheney> this resolves 80% of the outstanding problems
<davecheney> 20% coming real soon now
<davecheney> then this is DONE
<davecheney> and we can change common.AuthFunc
<davecheney> thumper: https://github.com/juju/juju/pull/534
<davecheney> lucky(~/src/github.com/juju/juju) % juju ssh 0
<davecheney> ERROR upgrade in progress - Juju functionality is limited
<davecheney> grrr
<davecheney> menn0: http://paste.ubuntu.com/8076514/
<davecheney> why did this happen, i thought it was fixed
<menn0> davecheney: I believe when you do --upload-tools the initial version in the machine agent's conf file is different from version.Current
<menn0> so upgrades still happen
<menn0> "upgrade mode" should be pretty short lived
<davecheney> that sounds odd
<menn0> have you got the machine agent log handy
<menn0> ?
<davecheney> --upload-tools derives the versino of the tools from the local juju that just did --upload-tools
<menn0> the logs will show the previous and next version, and also when the upgrade steps worker finished
<davecheney> menn0: http://paste.ubuntu.com/8076553/
 * menn0 looks
<davecheney> menn0: there should be no version difference
<davecheney> there was no previous environment
<davecheney> i bootstrapped it from the version of juju i just built
<menn0> here's the clue: "starting upgrade from 1.21-alpha1-precise-amd64 to 1.21-alpha1.1-precise-amd64"
<davecheney> menn0: this is going to generate issues from the juju deployer folks if they use --upload-tools
<menn0> the agent started at 02:57:34 and upgrades were done at 02:58:14
<davecheney> sure
<davecheney> it is slow
<menn0> so "upgrade mode" was 45s long
<davecheney> but i was able to trigger it in 3 out of 3 cases
<davecheney> juju bootstrap --upload-tools && juju deploy cs:mysql
<davecheney> will fail
<menn0> if there's an actual upgrade to perform, we need to limit the API (aka "upgrade mode")
<davecheney> right, but why is there an upgrade
<davecheney> ther eis nothing to upgrade from
<menn0> no idea
<menn0> I didn't write that
<menn0> but I'm happy to have a look
<davecheney> thanks
<davecheney> i just know its going to generate more bugs from the juju installer folks
<davecheney> will log issue
<menn0> I suspect it's to do with some functionality that keeps giving you new patch version every time you run upgrade-juju --upload-tools
<menn0> assign it to me
<menn0> it would be nice if we could avoid this following bootstrap
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1358078
<mup> Bug #1358078: cmd/juju: juju bootstrap --upload-tools on a fresh environment triggers upgrade mode <juju-core:New> <https://launchpad.net/bugs/1358078>
<davecheney> menn0: i know CTS always script their juju deployments and they assume if a command returned 0 then it is safe to continue with the next command
<menn0> davecheney: although that's ok with most commands I wonder if you be 100% that the machine agent is ready to go immediately after the bootstrap client command returns
<menn0> it's certainly worse now with the restricted API
<menn0> so I'll try to fix that
<davecheney> menn0: its not the machine agent
<davecheney> it's being able to create entires in the state database
<davecheney> maybe that is what you were asking
<menn0> yeah but the bootstrap machine agent runs the API server
<davecheney> did you mean machine agent == provisioner
<menn0> sorry, I should have said state server, not machine agent
<davecheney> iff you can connect to the api, the expectaion was it would accept commands
<menn0> agreed.
<menn0> I guess because the client retries API requests it does just tend to work
<menn0> the API server doesn't come up immediately when the bootstrap machine agent comes up
<menn0> but the client side retries mask that from the user's persepective
<menn0> anyway... let me have a look at this problem
<davecheney> thanks
<wallyworld> menn0: does the above problem affect 1.20?
<wallyworld> i would argue that we don't do upgrade steps if just the build number is different
 * menn0 checks 1.20 tag
<wallyworld> sorry, i could have checked too, thought you may have known ottofh
<wallyworld> ottoyh even
<davecheney> wallyworld: i think it only affects using --upload tools
<wallyworld> davecheney: sure, but they use that in 1.20 :-)
<menn0> wallyworld: yes it does affect 1.20
<wallyworld> :-(
<wallyworld> ok, i'll assign the bug to 1.20 also
<davecheney> wallyworld: it only afects you if you start with an empty enviromment
<davecheney> then bootstrap --upload-tools
<wallyworld> which is done for local provider
<davecheney> BALLS!
<menn0> and it's worse there because my recent work to avoid upgrade mode to if it's not required isn't in 1.20
<wallyworld> ooops
<menn0> or at least it's not in 1.20.2
<wallyworld> we are about to release 1.20.5
<wallyworld> tomorrow
<menn0> this problem has been there all through 1.20
<wallyworld> but this new issue will need to be fixed for 1.20.6
<wallyworld> there was a bug raised actually
<wallyworld> but we closed it
<wallyworld> i think it does need some love however
<wallyworld> menn0: i fixed the milestones on the bug - generally juju-core is assigned to a dev milestone; you mark a bug as affecting 1.20 series and then assign to a 1.20.x milestone for work done on that series
<thumper> waigani: a lot of problems?
<waigani> thumper: just had to redo everything
<menn0> wallyworld: ok thanks. I'm a launchpad newbie
<thumper> waigani: why?
<wallyworld> menn0: np at all, just an fyi :-)
<davecheney> is anyone paying attention to how many times the tests fail in CI and only pass because we retry them with less load ?
<wallyworld> davecheney: yes
<davecheney> wallyworld: good
<wallyworld> there will be an email this week
<davecheney> wallyworld: right
<davecheney> thanks, glad to know you're on top of it
<davecheney> i dunno what you are thinking, but I'm thinking 'remove the retry'
<davecheney> we can use humans for this
<wallyworld> been busy, would have liked to have initiated something sooner
<wallyworld> davecheney: the plan is to have people who wrote the tests be responsible for fixing them as a matter of priority; and yes the retry will go as soon as practical; the expectation will be that tests will pass first time, not the other way around
<wallyworld> ie we should be surprised when they fail, not when they pass
<wallyworld> QA have offered to do a report on the failing tests
<wallyworld> will make it easier to see what fails and how often
<waigani> thumper: okay it's up
<davecheney> wallyworld: sgtm
<waigani> thumper: shall I jump back in hanyout?
<menn0> wallyworld, davecheney: I've targeted bug 1350111 to 1.20 too because we need something there too. It'll be a somewhat different patch however as the code has changed a lot since 1.20
<mup> Bug #1350111: machine agent enters "upgrade mode" unnecessarily <juju-core:Fix Committed by menno.smits> <juju-core 1.20:In Progress by menno.smits> <https://launchpad.net/bugs/1350111>
<thumper> waigani: yeah
<wallyworld> menn0: thanks, will be good to get that sorted in 1.20 also
<davecheney> wallyworld: did you think we'll have a 1.22 stable release for U ?
<davecheney> supporting 1.20 for the whole of U and backported to T would be unpleasant
<wallyworld> davecheney: i am hoping so
<wallyworld> but there are so many bugs still to fix :-(
<davecheney> ï½¡ï½¥ï¾ï¾ï½¥(>Ð´<)ï½¥ï¾ï¾ï½¥ï½¡
<menn0> davecheney: there there
<menn0> wallyworld: do you have time for a quick hangout?
<wallyworld> sure
<menn0> wallyworld: https://plus.google.com/hangouts/_/canonical.com/onyx-standup ?
<wallyworld> menn0: waiting in hangout
 * thumper sends off an email and calls it a day
<thumper> laters
<menn0> davecheney, wallyworld: I have a fix for bug 1350111 (for trunk anyway). Will propose shortly.
<mup> Bug #1350111: machine agent enters "upgrade mode" unnecessarily <juju-core:Fix Committed by menno.smits> <juju-core 1.20:In Progress by menno.smits> <https://launchpad.net/bugs/1350111>
<wallyworld> great
<menn0> sorry, wrong number. I meant bug 1358078.
<mup> Bug #1358078: cmd/juju: juju bootstrap --upload-tools on a fresh environment triggers upgrade mode <juju-core:In Progress by menno.smits> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1358078>
<davecheney> menn0: sweet
<menn0> davecheney, wallyworld: https://github.com/juju/juju/pull/535
<wallyworld> looking
<menn0> wallyworld: thanks for the review
<wallyworld> thank you for the fix
<menn0> i've tested local already and am doing EC2 now
<wallyworld> \o/
<menn0> I have to deal with kids for a bit but will be back later
<axw> wallyworld: fyi, I've gone down a bit of a rabbit hole - there's lots of badness in our maas provider code. going to try and fix it a bit while I'm there
<wallyworld> ok
<bigjools> \o/
<wallyworld> /o\
<axw> we don't convert juju architectures to maas ones, so our constraints are just broken
<wallyworld> \o/
<axw> we don't return hardware characteristics
<axw> we choose arbitrary tools after acquring a node of an arbitrary arch
<axw> :|
<bigjools> and nobody noticed?  Is any of this tested? :)
<axw> bigjools: there's a warning and a bug, I'm surprised nobody is kicking up more of a stink. I think we're just lucky because we default to amd64, which is what most people would be using I assume.
<bigjools> yeah agreed
<dimitern> morning all
<axw> morning dimitern
<wallyworld> bigjools: it hasn't been fully tested because we have been begging for maas hardware for 9 months
<bigjools> I offered use of our CI lab as well
<axw> wallyworld: bigjools: actually turns out we were just not passing the subarch, and we happen to have the same arch identifiers. false alarm on that specific bit...
<bigjools> I was gonna say, that would have been spectacular
<wallyworld> phew
<bigjools> not surprised about subarch
<bigjools> you don't *need* it
<wallyworld> axw: how complete is the bit of code to pass the tools to target nodes via cloud init?
<axw> wallyworld: 100% functional, just fixing bits around the edge and will have a bunch of tests to update.
<axw> wallyworld: I have tested local and ec2, I haven't tested manual yet but that shouldn't be an issue
<axw> wallyworld: sorry, I guess not quite 100% because we still need to update metadata...
<voidspac_> morning all
<wallyworld> axw: i'm thinking of cherry picking just that bit for use in 1.20, because the tools url hacking i need for local to use a mounted dir is messy, plus it won't work for kvm
<wallyworld> o/
<axw> wallyworld: it's not something that can easily be cherrypicked. provider/maas needs updates to return the arch name, for one thing
<axw> morning voidspac_
<wallyworld> axw: this would just be for local provider
<wallyworld> to eliminate the need to get tools from http server
<axw> wallyworld: it's much more intrusive than that... I've had to move all the "EnsureTools" stuff *outside* the providers
<axw> so it's all or none
<wallyworld> rightio
<axw> menn0: I'm not sure that change is right
<axw> menn0: we always bootstrap the same version as the CLI
<axw> then upgrade toe the desired version
<axw> to*
<davecheney> axw: is this the --upload-tools bug ?
<axw> davecheney: https://github.com/juju/juju/pull/535
<davecheney> ok
<axw> so yes, looks like it
<davecheney> but whern I do --upload-tools, why is there a disagreemnt between the tools I uploaded, and the tools that the env thinks it is running
<axw> dunno, that sounds broken
<axw> davecheney: ah, probably because upload increments the build number on the tools
<davecheney> axw: i think we have some logic that fudges the tools version uplaoded to not match any existing tools
<davecheney> ie, the .1 that gets stuffed in there
<axw> yeh
<axw> p
<davecheney> that's the bug
<axw> gtg get my daughter from school, bbl
<jam> morning dimitern
<dimitern> morning jam! welcome back :)
<jam> thanks
<jam> dimitern: you just hung, but it might be me, I'll try reconnecting
<dimitern> jam, i've reconnected as well
<dimitern> jam, you seem hung in the g+
<dimitern> jam1, i've rejoined several times, each time it says you're in the room, but then once I'm in it says waiting for people to join
<dimitern> jam1, wanna try juju-sapphire g+ instead?
<jam1> dimitern: my internet just died for 10 sec
<jam1> but I should be able to connect now
<menn0> axw: ping?
<axw> menn0: pong
<menn0> axw: so this proposed fix... I think it's ok but you have concerns
<menn0> as far as I can tell, when --upload-tools is used
<menn0> the wrong version gets written to agent.conf
<menn0> so that when the machine agent comes up it think it needs to run upgrade steps
<axw> menn0: ignore --upload-tools for a moment
<menn0> ok
<axw> menn0: when we bootstrap, we look for the most recent tools that matches major.minor of the CLI's tools
<axw> menn0: we bootstrap with the exact same tools the CLI is running, but set agent-version=most recent
<axw> the effect of this is that the machine agent comes up and immediately upgrades to agent-version
<axw> that's what we *want* to happen
<axw> and it does happen like that now
<axw> menn0: IIANM, your change makes it so that the machine agent thinks it's running agent-version already, and so it doesn't run the upgrade steps
<menn0> ok
<axw> it'll still replace the binary, it just won't run the upgrade steps
<axw> menn0: the issue with --upload-tools is that it increments the Build number in the tools
<menn0> yep, I understand the --upload-tools case
<axw> so what's deployed is never the same as the CLI
<menn0> I didn't know about the standard bootstrap case
<menn0> I will have to rethink then
<menn0> what we want is:
<menn0> - normal bootstraps to work as you just described
<menn0> - juju upgrade-juju --upload-tools which increments just the build number to still trigger upgrade steps (for developers)
<menn0> - juju bootstrap --upload-tools to NOT trigger the upgrade steps
<axw> yup
<axw> menn0: I think it's as simple as not incrementing the build number on bootstrap
<axw> I don't see why we'd ever want to do that
<menn0> yep
<menn0> that makes sense
 * menn0 looks at code
 * axw should really document bootstrap some time
<menn0> axw: I think I know why we still increment the version on bootstrap when --upload-tools is passed
<menn0> if the tools storage is shared between envs
<menn0> axw: is it?
<axw> menn0: I don't think you can do that without running into problems
<axw> it's going to change real soon now anyway
<menn0> ok then it should be fine
<axw> we're getting rid of provider storage
<menn0> axw: I've been hunting through revision history
<menn0> do you think the incrementing the build number is done to ensure that the uploaded tools are used in preference to any tools in the public streams?
<menn0> axw: I'm wondering about another way to skin this cat:
<axw> menn0: ugh, that may be a problem, yeah.
<axw> menn0: TBH, it might be worth waiting till I'm done with my changes... this may be fixed incidentally
<menn0> if bootstrap is given --upload-tools, we get that version into the agent.conf, if not use version.Current
<axw> except that won't fix 1.20
<axw> menn0: can do, but sounds like it might be messy
<axw> how will you convey that information?
<menn0> via a field in MachineConfig perhaps?
<menn0> axw: so is the Tools field in cloudinit.MachineConfig definitely the target agent version, not the initial one?
<axw> menn0: until my changes go in, each provider's Bootstrap creates its own MachineConfig
<axw> hmm
<axw> menn0: it is the bootstrap tools, but hmmmm
<axw> menn0: I think I may have misunderstood the change
<axw> I'll take another look...
<menn0> axw: np
<axw> menn0: sorry, your original change was right :)
<menn0> phew!
<axw> it'll bootstrap with those tools
<axw> then it'll see agent-version is different and upgrade
<axw> sorry about that
<menn0> no problems
<menn0> I'd like to test the non-uploading case but that's kinda hard without having these changes in a official release
<menn0> is there another way? (custom streams or something)
<menn0> axw: ^^
<axw> urhm
<axw> you could use sync-tools to generate the metadata
<axw> menn0: simplest way would probably be to tweak version.go, sync-tools, then revert version.go
<menn0> axw: cool. I will look in to that tomorrow.
<menn0> I also need to read simplestreams-metadata.txt
<menn0> thanks for your help and for being concerned about my change
<axw> no worries :)
<menn0> I'd rather have that than bad code getting in
<menn0> I'm EOD
<TheMue> morning
<dimitern> morning TheMue, voidspac_
<TheMue> dimitern: I pushed my change to my repo on Friday, only the tests are missing. but I needed an additional API call to see if a machine is manually provisioned. looks good so far.
<dimitern> TheMue, sweet! I'm looking forward to seeing it
<TheMue> dimitern: the current changes are at https://github.com/TheMue/juju/compare/capability-detection-for-networker
<voidspac_> TheMue: dimitern: morning
<TheMue> voidspac_: hello
<dimitern> TheMue, looking
<dimitern> TheMue, looks nice, although for the IsManual API call, I'd implement it a bit differently
<dimitern> TheMue, like getting the IsManual flag as part of getting the machine's live value
<TheMue> dimitern: itâs able for bulk calls
<dimitern> TheMue, i.e. caching it, so you can return it directly without an extra call
<dimitern> TheMue, yeah, LifeGetter also supports bulk calls
<TheMue> dimitern: will take a look
<dimitern> TheMue, cheers
<jam1> TheMue: I'm just finishing up lunch, I'll be a little bit late.
<TheMue> jam1:  ok, just ping
<mattyw> fwereade_, ping?
<jam1> TheMue: I'm in the hangout
<voidspac_> given a package name, how do I tell what files / binaries it provides?
<dimitern> voidspac_, you can run a godoc server for that package locally
<voidspac_> dimitern: by package I mean ubuntu package, sorry
<dimitern> voidspac_, ah :)
<voidspac_> dimitern: go doesn't have packages, does it?
<voidspac_> I mean, it doesn't use that term
<voidspac_> it has "dependencies", which can be anything pretty much
<dimitern> voidspac_, they are packages
<dimitern> :)
<voidspac_> well, they're not
<dimitern> voidspac_, for debs: $ apt-cache showsrc juju-core
<voidspac_> they're a hodge-podge of files
<dimitern> voidspac_, :)
<voidspac_> dimitern: thanks
<voidspac_> dimitern: hmmm... not quite it, I want to know what files it will put where
<voidspac_> dimitern: is there a determinstic way of knowing that?
<voidspac_> I guess now as the install scripts execute code
<voidspac_> I mean, I guess not
<dimitern> voidspac_, well, these 3 files are the only ones in the deb archive actually
<voidspac_> dimitern: the tarballs?
<dimitern> voidspac_, if you want to see the source itself, try apt-get source <package>
<voidspac_> dimitern: so download the package and inspect it
<voidspac_> fair enough
<dimitern> voidspac_, yeah, and take a look at the debian tarball for hooks I guess
<dimitern> voidspac_, and rules (which is as readable as any generated Makefile :)
<voidspac_> dimitern: thanks :-)
<wallyworld> fwereade_: morning, you up for a chat about health checks sometime?
<fwereade_> wallyworld, heyhey
<wallyworld> let me know when you have time and we can do a hangout
<fwereade_> wallyworld, what time is it for you?
<wallyworld> 20:30
<fwereade_> wallyworld, hmm, and how early do you like to get up?
<wallyworld> i'm up around 6 but need to take the kid to school, back around 7
<dimitern> jam1, standup?
 * fwereade_ restarting then hopefully with wallyworld
<voidspac_> perrito666: ping
<voidspac_> Command failed: mongodump --dbpath /var/lib/juju/db
<voidspac_> Error: bash: line 9: mongodump: command not found
<voidspac_> perrito666: do you recognise that? No mongodump on my state server.
<ericsnow> voidspac_: /var/lib/juju/mongodump is installed as part of the tools in a local juju environment
<voidspac_> ericsnow: this isn't local this is an openstack environment
<voidspac_> ericsnow: and that error message was the output from "juju backup", that location is where the backup script was looking
<voidspac_> ericsnow: but thanks :-)
<ericsnow> voidspac_: (yeah, drop the "local" part)
<ericsnow> voidspac_: weird
<voidspac_> ericsnow: I just destroyed the environment and will try again
<voidspac_> ericsnow: I think I started the backup too early
<ericsnow> voidspac_: FYI horacio is out today
<voidspac_> ericsnow: hmmm, installed as part of the tools?
<ericsnow> voidspac_: ah, that makes sense
<ericsnow> voidspac_: right
<voidspac_> I wonder if that works when you do --upload-tools
<voidspac_> ericsnow: I don't need upload-tools anyway, as restore is run on the client
<voidspac_> although that might be weird...
<voidspac_> we'll see
<voidspac_> WARNING no prepackaged tools available
<voidspac_> uploading tools
<voidspac_> so I get no choice in the matter :-)
<ericsnow> voidspac_: while restore runs on the client, most of the work is happening on the remote host
<voidspac_> ericsnow: yeah, but it's mostly batch scripts uploaded from the client :-)
<voidspac_> ericsnow: the part I'm *changing* runs on the client
<voidspac_> so that's the bit I want to check works
<ericsnow> voidspac_: disclaimer--most of what I know of restore is from working on backup :)
<voidspac_> hah
<voidspac_> ericsnow: I'm working on hacking a fix into the old plugin anyway
<voidspac_> ericsnow: not touching the shiny new stuff you did :-)
<ericsnow> voidspac_: right
<ericsnow> voidspac_: still polishing :)
<voidspac_> :-)
<voidspac_> ericsnow: so when I bootstrap an openstack environment from dev I don't see a mongodump in the uploaded tools
<voidspac_> I might have to blow away dev and try with 1.18/1.20
<voidspac_> it might be a curiousity of upload-tools
<ericsnow> voidspac_: not sure then
<ericsnow> voidspac_: I never got around to figuring out for myself where mongodump, etc. came from
<ericsnow> voidspac_: it should be in the same dir as mongod
<voidspac_> ah right
<voidspac_> mongod isn't in the uploaded tools directory either, must be elsewhere
<voidspac_> I'm trying with 1.18 now
<ericsnow> voidspac_: yeah, not the uploaded tools dir
<ericsnow> voidspac_: pretty sure /var/lib/juju holds just the juju-built mongo binaries
<voidspac_> ericsnow: ok, will check when this environment has bootstrapped
<voidspac_> thanks
<voidspac_> (again)
<voidspac_> :-)
<ericsnow> :)
<ericsnow> voidspac_: yeah!  I was helpful to someone :)
<voidspac_> ericsnow: /usr/lib/juju/bin
<voidspac_> not /var/lib/juju
<ericsnow> voidspac_: ah
<voidspac_> ericsnow: ok, interesting
<voidspac_> ericsnow: so now with a freshly bootstrapped machine from dev
<voidspac_> I *do* have mongodump in /usr/lib/juju/bin
<voidspac_> but it's not in path
<ericsnow> voidspac_: right
<voidspac_> I'm giving it a chance for bootstrap to complete before I try mongodump
<voidspac_> looks like just a path issue then
<ericsnow> voidspac_: if I recall correctly it tries /usr/lib/juju/bin explicitly first and then falls back to $PATH
<voidspac_> ericsnow: backup and restore CI test is passing, so it's not broken - it just may be hard / impossible to manually test without recreating some of the CI infastructure
<voidspac_> (they avoid using --upload-tools in tests for other reasons)
<ericsnow> voidspac_: makes sense
<voidspac_> brb
<voidspac_> right
<voidspac_> goodnight
<katco> i need some help with facades: i see a FacadeCall("ContainerConfig",...), but i can't figure out where "ContainerConfig" is registered as a Facade...
<ericsnow> katco: registration happens with a call to state/apiserver/common.RegisterStandardFacade() (or similar)
<ericsnow> katco: e.g. state/apiserver/client/client.go
<katco> ericsnow: yeah, i gathered that, but a quick grep shows no registrations for "ContainerConfig"... indeed the only _reference_ i can find to that facade is the call...
<ericsnow> katco: I don't see one under apiserver for ContainerConfig
<ericsnow> katco: is it a facade or something on a facade?
<ericsnow> katco: perhaps it's dead code?
<katco> ericsnow: perhaps... i'm trying to wind my way through a failing test
<katco> ericsnow: it's in api/provisioner/provision.go::CreateConfig()
<ericsnow> katco: looks like FacadeCall is just a wrapper around a facade for calling a method on that facade
<ericsnow> katco: so ContainerConfig is a method on some facade rather than a facade itself
<katco> ericsnow: ah ok... so i'm trying to trace down the facade that's passed in; it's probably a method on there?
<ericsnow> katco: yep
<katco> ericsnow: ok we'll see where this yarn ends then :)
<katco> ericsnow: thank you
<ericsnow> katco: apiserver/provisioner/provisioner.go defines a ContainerConfig method
<katco> ericsnow: yeah i had been looking at that, but it looks correct. i'm probably missing something eyeballing it
<katco> ericsnow: arrrrrrg, it was a test mocking out that call.
<ericsnow> katco: :(
<sinzui> katco, ericsnow: have another version increment for stable because we are releases 1.20.5 today. https://github.com/juju/juju/pull/537
<katco> sinzui: thanks, sinzui. very exciting :) lgtm too
<sinzui> oh, for anyone looking at recent history of CI, Hp and AWS had soem bad hours. The QA team cleanup some machines and restarted the tests. master was *never* broken
<thumper> mramm: morning, we hanging out this morning?
<mramm> yep
<mramm> sorry I am late
<mramm> thumper: you still around?
<thumper> mramm: aye
 * thumper jumps into hangout
<katco> wallyworld_: no worries. emacs just crashed when you sent that >.<
<alexisb> thumper, on and ready when you are
<thumper> alexisb: ack, coming
<alexisb> no rush
<alexisb> I just have a hard stop at the top of the hour
<menn0> thumper, davecheney, waigani_ : regarding that bootstrap version behaviour that you didn't like the sound of
<menn0> looking at the code
<menn0> it seems like it might be ok
<thumper> good
<menn0> it's asking the cloud instance itself for the tools metadata it has
<thumper> right...
<menn0> and therefore it's only checking through tools that are available /on that cloud/
<thumper> yeah...
<thumper> but does it make sense for it it automatically upgrade?  Just wondering
<thumper> and if so, to what version?
<menn0> so it's not going to ask the new environment to upgrade to tools that it doesn't have
<thumper> if I bootstrap 1.18
<thumper> what happens?
<thumper> assuming I have a 1.18.0 client
<menn0> from what I have gathered from the code
<menn0> it will bootstrap 1.18.0
<menn0> but set the env agent-version to the latest 1.18.X available on the cloud
<menn0> so when the bootstrap machine agent comes up it will immediately upgraded to 1.18.X
<menn0> if there is no 1.18 available on the cloud
<menn0> it'll upload the local tools and the env will be running 1.18.0 (no upgrade)
<menn0> does that sound reasonable?
<menn0> if not, talk to Andrew
<menn0> (I'm just the messenger!)
<thumper> hmm...
<thumper> but the initial version is the same as the client?
<menn0> yes, but only for a short time
<davecheney> waigani: if you want to be on call reviewer for today, that would be good, we only have brits
<menn0> the upgrade (i.e. restart into the new version) happens almost immediately
<thumper> and it is only the patch number that is allowed to change?
<thumper> so major.minor have to be the same?
<menn0> yes
<waigani> davecheney: okay
<menn0> it filters on major.minor
<thumper> I suppose that is ok
<menn0> my guess for the reasoning is to ensure the the env is running with the most bugs fixed
<menn0> thumper, davecheney: one problem that just comes to mind is that it has the potential the break the "juju bootstrap && juju deploy foo" use case.
<menn0> if the bootstrap machine agent comes up and then restarts soon after
<menn0> and then goes in to "upgrade mode" soon after
<thumper> well... hopefully the user has an up to date client :)
#juju-dev 2014-08-19
<davecheney> thumper: grrr
<davecheney> this canWatch("") thing is really pernicious
<davecheney> gonna take some more time
 * thumper looks in a dictionary
<davecheney> keep finding more test cases that fail whenever I make AuthAlways demand a valid tag
<thumper> heh
<thumper> it will be worthwhile
<thumper> you're doing a good job
<thumper> keep it up :)
<menn0> now that's what I call Managing :-p
<menn0> and Github is broken...
<menn0> Storage server temporarily offline. See http://status.github.com for GitHub system status.
<menn0> the graphs on that page show a sudden spike in Exceptions
<waigani> I'm getting random prs going offline e.g: https://github.com/juju/juju/pull/499
<waigani> and https://github.com/juju/juju/pull/517
<waigani> can others see those prs?
<waigani> oh, haha menn0 I just read your messages above
<waigani> that would explain it
<menn0> haha
<menn0> status.github.com is now broken too :)
<menn0> "Service Unavailable"
<menn0> it's back
<menn0> and now they have a note that they know something is wrong with some repos
<menn0> awesome
<menn0> thumper: my merge passed its test run but CI was unable to perform the merge because the repo on Github is borked ("Repository offline")
<menn0> thumper: once Github is back will a manual merge be possible?
<thumper> hmm...
<thumper> I'd say go with CI doing it again
<thumper> github say some repos are borked?
<menn0> thumper: yep. See the email I just forwards to juju-dev.
<menn0> thumper: the web stuff seems to be fine but interacting with the repo isn't working.
<thumper> :(
<menn0> thumper: it's back again
<menn0> can someone have a quick look over this? It's the 1.20 backport of the tools version change that's going in to trunk. https://github.com/juju/juju/pull/538
<menn0> thumper: the bot won't pick up the PR that it's already tested
<menn0> thumper: any chance you can click the merge button (I thought team leads could do this)
<menn0> thumper: the tests passed here: http://juju-ci.vapour.ws:8080/job/github-merge-juju/328/
<menn0> thumper: the PR is here: https://github.com/juju/juju/pull/535
<thumper> wallyworld_: do you know how to kick this bot?
<wallyworld_> wot
<wallyworld_> bot is kicked
<thumper> wallyworld_: ta
<thumper> nice
<wallyworld_> you just need a "Build Failed: blah" comment
<wallyworld_> to tell the bot that a $$ $$ is valid
<wallyworld_> a second $$ $$ i should say
<wallyworld_> thumper: could i get you to look at this pr. it's a 1.20 only fix to address an lxc startup issue https://github.com/juju/juju/pull/539
<thumper> ack
<wallyworld_> ta
<davecheney> thumper: menn0 waigani https://github.com/juju/juju/pull/540
<davecheney> ^ quite a nice solution i think
 * thumper afk for bit from 2pm
<wallyworld_> axw: once you finish your maas branch changes, i just wanted to have a quick chat about copying tools
<axw> wallyworld_: sure
<wallyworld_> ta, just ping me
 * bigjools cracks joke about copying wallyworld_
 * wallyworld_ tells bigjools to FO
<axw> wallyworld_: changes uploaded, ready to chat when you are
<wallyworld_> axw: i'll just lgtm first, one sec
<wallyworld_> axw: i'm still not sure why we need to pass in the series to the tools match, when we get the series from the tools we are matching on. the only reson i can see ottomh is so that we get the error if there is more than one series
<axw> wallyworld_: ah, right
<axw> I don't know why I did that now
<axw> wallyworld_: I've taken it out, that was nonsense AFAICT
<wallyworld_> ok, i've marked it as lgtm
<axw> thanks
<wallyworld_> join you in stand up
<menn0> axw or wallyworld_: can you do a 10s look at this please? https://github.com/juju/juju/pull/538
<wallyworld_> sure
<wallyworld_> 10, 9, 8, ....
<menn0> it's the 1.20 backport of the agent.conf tools version change that just went in
<menn0> axw: thanks
<axw> np
<axw> jcastro: https://plus.google.com/102738380796586573408/posts/BD1bYop5x7z
<axw> (the post I got you to review)
<axw> wallyworld_: ^^ finally published
<wallyworld_> axw: \o/ - i was meaning to ask you about that but kept forgetting
<axw> wallyworld_: was waiting for a new 1.20 release
<wallyworld_> ah, yes
<davecheney> axw: appears to be truncated
<davecheney> Work on llgo  has been progressing, mostly due to Pe...ï»¿
<axw> oh ffs
<axw> the link dropped
<axw> davecheney: thanks, fixing
<axw> jcastro: wallyworld_: davecheney: https://plus.google.com/+AndrewWilkinsTheFirst/posts/EfTDNXS4SET
<menn0> davecheney, wallyworld_: bug 1358078 sorted for trunk and 1.20 \o/
<mup> Bug #1358078: cmd/juju: juju bootstrap --upload-tools on a fresh environment triggers upgrade mode <juju-core:Fix Committed by menno.smits> <juju-core 1.20:Fix Committed by menno.smits> <https://launchpad.net/bugs/1358078>
<wallyworld_> great :-D
<wallyworld_> thank you
<davecheney> nnnice
<menn0> wallyworld_: just for you :)  https://github.com/juju/juju/pull/541
<wallyworld_> oh goody
<davecheney> thumper: if you have a momemnt, i've replied to your comments, https://github.com/juju/juju/pull/540
<thumper> davecheney: ack
<menn0> wallyworld_: thanks for the review. merging now.
<wallyworld_> np, thaks for fixing
<davecheney> thumper: thanks for the review
<davecheney> one more coming up real soon now
<thumper> kk
<davecheney> it's very similar
<davecheney> what I think happened is we started out with the common.AuthFunc
<davecheney> which says "this authoriser" can read/write/view/etc this entity (tag)
<davecheney> but common.Environ{,Machines}Watcher are really "global"
<davecheney> things
<davecheney> this is also indicatated that Authtorizer.AuthEnvionWatcher is not derived from the entity of the authoriser
<davecheney> it's actually some special permission bit stored in the database
<davecheney> menn0: nice cleanup of the upgrade stuff
<davecheney> +1
<menn0> davecheney: cheers
<thumper> waigani: can you email matty and say you'll take over the EnvUser work?
<thumper> axw, wallyworld_: do you know if fwereade is working this week? I normally leave before he would start
<wallyworld_> yes he is
<thumper> ok..
<waigani> thumper: this branch: https://github.com/juju/juju/pull/432
<thumper> cheers
<thumper> waigani: yes
<waigani> thumper: okay, I was on call reviewer today. I'll make a start on it tomorrow.
<thumper> waigani: can I get you to email him before EOD today, so he'll see it?
<waigani> thumper: doing it now
<thumper> cheers
<thumper> waigani, davecheney, menn0: fyi - I have friday off
<waigani> okay, thanks for the heads up
 * thumper just realised that I also have a meeting friday morning...
<thumper> hmm...
<menn0> waigani, davecheney: party time!
 * thumper emails rick_h__ et al
<menn0> wallyworld_: aaaaaaand here's the 1.20 port: https://github.com/juju/juju/pull/542
<wallyworld_> thanks menn0
<menn0> wallyworld_: thanks. I'll leave you alone for the rest of the day :)
<wallyworld_> promise,promises
<davecheney> thumper: https://github.com/juju/juju/pull/543
<thumper> davecheney: done, and now I'm off to guardians of the galaxy
<thumper> \o/
<davecheney> TheMue: danka
<davecheney> enjoy
<davecheney> i really hope this is the last little fucker
<menn0> davecheney: thumper was gone when you replied so autocomplete got the wrong person :)
<davecheney> oopas
<davecheney> TheMue: sorry mate
<davecheney> wallyworld_: Fetched 8,775 kB in 5s (1,593 kB/s)
<davecheney> Reading package lists...
<davecheney> + sudo apt-get upgrade -y
<davecheney> is there any way to stop the bot
<davecheney> doing apt-get update/upgrade for every CI run ?
<davecheney> that isn't necessary
<davecheney> https://github.com/juju/juju/pull/545
<davecheney> anyone for a little one ?
<davecheney> it's pretty uncontravesial
<wallyworld_> davecheney: apt updates are necessary to pull in all the deps required
<davecheney> wallyworld_: nah
<davecheney> why is it updating the kernel and building an initrd
<wallyworld_> the instances started to run the test aren't necessarily up to date
<davecheney> that is a waste of time
<wallyworld_> sure, but besides the kernel it also pulls in other dependencies that are needed AFAIK
<dimitern> morning all
<axw> wallyworld_: my memory is hazy, why do we need auto-upload on bootstrap? why not just require people to --upload-tools?
<wallyworld_> axw: from memory, it was "too hard" and the out of the box experience should be that it Just Works if there are no matching prepackaged tools available
<axw> wallyworld_: mk. I'm not sure it holds its weight given that it only applies to dev builds
<axw> it's a bit of a PITA to try to get it to work the same way in my new branch
<wallyworld_> i'm not personally wedding to the idea of it being automatic - it was done because of user requests
<wallyworld_> wedded
<wallyworld_> we can look to get your stuff landed and see if it can be tastefully added back in
<fwereade> wallyworld_, axw: IMO --upload-tools is pretty fundamentally broken
<wallyworld_> i wish we could drop it
<wallyworld_> but we've been told we can't
<fwereade> wallyworld_, axw: but the magical fall back to locally available tools is a good thing
<axw> fwereade: remind me why --upload-tools is fundamentally broken?
<TheMue> Morning
<TheMue> dimitern: I'm at a funeral this morning. So I'll start after lunch. :(
<dimitern> TheMue, oh :( alright then
<TheMue> dimitern: I would like to discuss your idea then. IsManual is special to machines, so I would place it at machiner.Machine, not agent.Entity.
<TheMue> dimitern: Will ping you then.
<dimitern> TheMue, ok, we'll discuss it
<TheMue> dimitern: Thanks
<Beret> wallyworld_, anyway we can get whatever fix you guys deem appropriate for https://github.com/juju/utils/pull/22 in 1.20.x ?
<wallyworld_> Beret: i think so. by the looks of the comment from kapil, the approach needs to be re-thought. we can try and get it done for 1.20.6
<Beret> thanks
<wallyworld_> i'm hoping that 1.20.6 might be able to be released this time next week
<Beret> +1
<Beret> we're trying 1.20.5 but from the looks of the bug fix list, it should be pretty solid
<Beret> hoping so
<wallyworld_> we're still struggling a bit to inderstand fully the lxc issues being seen
<Beret> that's the only thing really outstanding still right? (other than this apt retry thing)
<wallyworld_> there's a few other smaller bugs
<wallyworld_> Beret: if there's any extra info that can be provided for bug 1354027 that would be great eg contents of /var/lib/juju/containers when it fails
<mup> Bug #1354027: LXC was not created, no errors, no logs -> pending state. <cloud-installer> <landscape> <juju-core:Triaged> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1354027>
<Beret> wallyworld_, ok, we'll see what we can do
<wallyworld_> thanks :-)
<fwereade> axw, sorry, I missed your question
<axw> fwereade: hey, it was
<axw> fwereade: remind me why --upload-tools is fundamentally broken?
<fwereade> axw, the biggest issue with upload-tools is that it's the core reason tools distribution was always so awful
<fwereade> axw, it was a separate code path written *really* early on
<fwereade> axw, and the developers were *always* using upload-tools
<fwereade> axw, and so the real code paths... well, they barely even existed, and certainly weren't mature, when we first released
<fwereade> axw, then users found out about upload-tools
<fwereade> axw, which obscures really important info, like *which damn build is running*
<fwereade> axw, but let them actually use juju
<fwereade> axw, so it got kinda promoted
<fwereade> axw, and hangs around our necks to this day
<axw> ok
<fwereade> axw, finding local tools of matching version and using those automatically, though? that's fine IMO
<axw> fwereade: with the auto upload stuff, I'm trying to avoid having providers call back into common bootstrap code to "ensure tools"
<axw> I think I can do it, it's just a bit messy
<axw> in my new branch, environs/bootstrap finds all the available tools, and presents them to provider's Bootstrap
<axw> that chooses an arch & series and reports that back
<fwereade> axw, it's always a tough call but I could probably get behind a bit of extra duplication across the providers if we actually simplify what's happening
<axw> common code then uploads if necessary
<fwereade> axw, that sounds good
<fwereade> axw, slightly better fit with what we do for normal machines
<voidspace> speaking of upload-tools, I think backup is broken if you use it (at least I'm seeing a path issue for mongodump that only seems to happen with upload-tools)
<voidspace> and that makes it a real pain to test changes to restore...
<voidspace> jam: TheMue: dimitern: standup?
<dimitern> sorry, brt
<jam> voidspace: dimitern, brt,
<voidspace> jam: I don't think you succeeded in pushing your changes
<voidspace> jam: github thinks that branch is identical to master
<jam> void
<voidspace> https://github.com/jameinel/juju/compare/test-local-backup
<jam> voidspace: master is, but 'test-local-backup' has the interesting stuff
<voidspace> jam: according to github compare "juju:master and jameinel:test-local-backup are identical."
<voidspace> ooh, it looks like restore might be working against an amazon juju
<jam> voidspace: committing helps
<voidspace> backup failed until I linked in the commands, but restore has not failed in the same way that restore failed against canonistack
<voidspace> jam: :-)
<voidspace> local will definitely be faster
<voidspace> jam: I was booted out
<voidspace> jam: https://github.com/voidspace/juju/compare/backup-ssh
<voidspace> ericsnow: ping
<voidspace> ericsnow: have you worked on the old backup plugin?
<TheMue> back again (together with a heavy thunderstorm)
<jam> TheMue: welcome back, we missed you. How's it going?
<TheMue> jam: itâs ok, has been the father of a good friend I know for more than 30 years now
<jam> TheMue: do you have a bug number for the LXC IPv6 issues?
<jam> I want to raise it in the networking discussion.
<TheMue> jam: yep, https://bugs.launchpad.net/juju-core/+bug/1358376
<mup> Bug #1358376: IPv6 connectivity of LXC containers <ipv6> <lxc> <network> <juju-core:New> <https://launchpad.net/bugs/1358376>
<ericsnow> natefinch, wwitzel3: standup?
<wwitzel3> ericsnow: yep
<natefinch> ericsnow: coming
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1358768
<sinzui> natefinch, I just reported bug 1358768. Can you find an engineer to fix the regressions
<mup> Bug #1358768: No tools found in i386 and ppc64el unit tests <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1358768>
<natefinch> sinzui: will do
<ericsnow> natefinch: 1-on-1?
<natefinch> ericsnow: oh yeah, ok
<natefinch> dammit, I'm getting 500s from github trying to look at commits on juju/juju
<wwitzel3> natefinch: I was getting that earlier through the browser
<wwitzel3> natefinch: though everything through the git command line seemed to be working ok
<katco> natefinch: wwitzel3: i know they had some storage issues last night
<wwitzel3> doesn't help for looking at PRs
<wwitzel3> katco: ahh, probably some residual issues from that then
<katco> wwitzel3: https://status.github.com/
<katco> wwitzel3: natefinch: 0:28 UTC: We're investigating problems accessing a small percentage of repositories.
<natefinch> katco, wwitzel3 : thanks
<natefinch> rick_h__: have some time today to talk about charm feature flags?  We need to implement support for them, probably this week.
<rick_h__> natefinch: sure thing
<rick_h__> natefinch: free from now for the next couple of hours
<natefinch> rick_h__: half hour?  trying to unblock CI
<rick_h__> natefinch: rgr, shoot me a link when you're ready
<natefinch> sinzui: fix applied.... required revert of https://github.com/juju/juju/pull/536
<natefinch> sinzui: I didn't look too deeply into the cause of the bug, possibly something simple... looks like we're restricting the list of returned tools based on the current architecture... but figured Andrew can fix it when he's on and re-submit.
<sinzui> thank you natefinch
<natefinch> sinzui: where do the CI machines keep godeps?  I wrote a test that runs godeps and ensures dependencies.tsv matches what godeps outputs.... but the test fails in CI because it can't find godeps.
<sinzui> natefinch, godeps is not available. It is installed to create the tarball, then it is removed because Ubuntu doesn't want it packaged
<sinzui> natefinch, we need to add godeps to the makefile maybe to ensure it is available for tests
<sinzui> natefinch, or we tell ubuntu that godeps is now a requirement
<natefinch> sinzui: how can we run the tests if we don't run godeps first?   Sorry, maybe CI was the wrong word... wherever the unit tests get run
<natefinch> sinzui: I presume there's a machine where we pull down the code, run godeps to set the correct revisions on all the repos, and then run go test.  That's the machine I need to be compatible with my test
<natefinch> sinzui: see here: https://github.com/juju/juju/pull/495
<sinzui> natefinch, we test the release, not the code. we test that tarball made in a previous step.
<sinzui> natefinch, run-unit-test runs make install-dependencies after unpacking the tarball under test to get the deps
<sinzui> natefinch, Since you name have a test that requires godeps, I think I can justify that the tarball include it.
<natefinch> sinzui: ok, thanks. Please dump it in $PATH or $GOPATH/bin - those are the only two logical places where developers would have it.
<sinzui> oh, natefinch, the tarball cannot contain /bin because then the tarball is not a source tarball.
<sinzui> natefinch, I think we need to change the test rules to build...
<sinzui> nm, we already call build
<sinzui> natefinch, I will make changes in a few hours to include godeps in the tarball.
<sinzui> I will ping you when it is ready
<natefinch> sinzui:  thanks.  This is just the tarball we dump on the test machine for testing... we wouldn't be including godeps in the package, right?
<sinzui> NO
<sinzui> natefinch, this is the real tarball
<sinzui> since golang cannot be trusted to always build the same tarball (Ubuntu's opinion) we make it once, we test it as the release, and if it passes, I can release it
<sinzui> We cannot tamper with the tarball's contents once we make it
<natefinch> sinzui: ok, sure.  If ubuntu's rules require that we include a whole bunch of crap that the actual application doesn't even use, just because it's used to test the application, so be it.  I'm tired of trying to fight that fight.
<sinzui> natefinch, that is indeed the reason they wanted it removed
<sinzui> natefinch, why doesn't the juju Makefile ensure godeps is installed?
<sinzui> natefinch, I can make that change too
<natefinch> sinzui: it looks like on july 7th, it was changed so that it wouldn't install it unless a certain env variable was set
<sinzui> natefinch, the Makefile or make-package-with-tarball.bash, or run-unit-tests?
<natefinch> sinzui: https://github.com/juju/juju/blob/master/Makefile#L36
<sinzui> natefinch, sorry. I was looking at 1.20.
<sinzui> natefinch, okay, I will get this change into 1.20 and change run-unit-tests to call it
<natefinch> sinzui: the godeps build target I think will fail if you don't have the actual code.... possibly it'll just go download all the code for all the dependent repos.
<natefinch> sinzui: I think the latter
<sinzui> natefinch, I see it doing go get launchpad.net/godeps
<sinzui> natefinch, and I can add a step to install it if need be in run-unit-test.
<natefinch> sinzui: there's two steps, go get launchpad.net/godeps and then godeps -u dependencies.tsv
<sinzui> I see both in the current Makefile
<natefinch> sinzui: yes, the second one, godeps -u, will go get all the source code for Juju and set the repos up to be at the correct revision
<natefinch> (well, it'll get all the source code for juju that isn't under github.com/juju/juju)
<natefinch> sinzui: *I* don't care that it'll do that.... just warning you that it will, in case that will either a.) fail, or b.) be a problem for some other reason
<sinzui> natefinch. The making of the tarball 15 minutes before is more likely to catch the problem
<sinzui> natefinch, when thumper drops a tab, the tarball doesn't build, testing ends immediately because there is nothing to release
<sinzui> natefinch, So i think think this test will always pass in ci
<sinzui> natefinch, and the git-merge-juju will do the same since the tarball is made before the unit tests are run from it
<natefinch> sinzui: This test will *currently* always pass CI, I believe.... but it could fail if we stop running godeps -u first, which is a nice thing to test.  Mainly it's for developers who may forget to run godeps -u
<natefinch> rick_h__: do you have time now?
<rick_h__> natefinch: sure thing
<natefinch> rick_h__: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1
<thumper> fwereade: around?
 * thumper is off to see the physio at 10
<waigani> thumper: AdminUser is defined as a const string in state. Do we want to ditch that and make AdminUser a names.UserTag("admin") ?
<waigani> thumper: or do we want to build the tag everywhere AdminUser is used?
<waigani> i.e. names.UserTag(state.AdminUser)
<waigani> doh just saw thumper is at physio
<thumper> waigani: back
<thumper> waigani: no, we don't want to do that
<thumper> waigani: I actually have a directive to change the admin user to be the username of the user bootstrapping
<waigani> thumper: okay, I've just built the usertag where I needed
<waigani> thumper: I commented another question on the PR for you
<thumper> which PR?
<waigani> thumper: 432
<waigani> thumper: https://github.com/juju/juju/pull/432/files
<thumper> kk, otp with hazmat
<waigani> thumper: basically, are we ready to append env to usernames i.e. bob@local and should I look at doing that in this PR?
<thumper> waigani: not yet
<thumper> followed by "it depends"
<waigani> thumper: I'm good with "not yet"
#juju-dev 2014-08-20
<davecheney> waigani_: menn0 thumper https://github.com/juju/juju/pull/548
<davecheney> here is an easy one to warm up with this morning
<hazmat> anyone know instance distributor provider code?
<hazmat> just curious how it knows service name
<menn0> thumper: got a sec?
<thumper> menn0: aye
<menn0> thumper: hangout?
<thumper> sure
<menn0> thumper: calling you now
<menn0> thumper: https://plus.google.com/hangouts/_/g4pl2c7bq3jqz2jy3rsjdtvakea?hl=en-GB
<menn0> thumper: grrr
<menn0> let's use the standup hangout
<thumper> ok
<davecheney> wow, my branch from a month ago merged mostly cleanly with master
<davecheney> not bad
<thumper> \o/
<axw> can I please get a review on https://github.com/juju/juju/pull/552 to unblock CI?
<axw> wallyworld_: ^^  very small one
<wallyworld_> sure
<davecheney> OOPS: 34 passed, 6 FAILED
<davecheney> --- FAIL: Test (18.44s)
<davecheney> FAIL
<davecheney> exit status 1
<davecheney> FAIL    github.com/juju/juju/state/apiserver/provisioner        18.541s
<davecheney> sigh
<davecheney> well, at least it's only 3 of the apiserver packages
<davecheney> which is better than before
<davecheney> but still a lot of tests to fix
<thumper> waigani: I'm about to drop maia down at hockey
<thumper> waigani: how about we hangout when I get back and talk about the testing ?
<waigani> thumper: that would be great, thanks
<davecheney> ... obtained params.ErrorResults = params.ErrorResults{Results:[]params.ErrorResult{params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:&params.Error{"not found", "machine 42 not found"}}, params.ErrorResult{Error:&params.Error{"not found", "unit \"foo/0\" not found"}}, params.ErrorResult{Error:&params.Error{"not found", "service \"bar\" not found"}}}}
<davecheney> ... expected params.ErrorResults = params.ErrorResults{Results:[]params.ErrorResult{params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:<nil>}, params.ErrorResult{Error:&params.Error{"not found", "machine 42 not found"}}, params.ErrorResult{Error:&params.Error{"unauthorized access", "permission denied"}}, params.ErrorResult{Error:&params.Error{"unauthorized access", "permission denied"}}}}
<davecheney> welcome to my day
<davecheney> looking at long likes of quasi json and spotting the n differences
<thumper> waigani: back
<waigani> thumper: https://github.com/juju/juju/pull/553
<thumper> waigani: looking now, tests work ok?
<wwitzel3> waigani: I added the comments for the public methods on https://github.com/juju/juju/pull/511
<waigani> thumper: all tests passed except state/envuser_test.go which I fixed and tested individually
<waigani> thumper: I'll do one more make test to be sure
<waigani> wwitzel3: great thanks :)
<waigani> wwitzel3, davecheney is my review mentor so you'll have to poke him for an LGTM
<wwitzel3> thanks :)
<davecheney> two secs
<thumper> waigani: you are missing the new files...
<thumper> waigani: state/envuser.go + test are not there
<thumper> wwitzel3: o/
<waigani> thumper: fuk
<wwitzel3> thumper: heya
<waigani> thumper: state/envuser.go is already in master?
<thumper> waigani: is it?
 * thumper looks
<waigani> thumper: is for me?
<thumper> waigani: umm... no. don't think so
<waigani> thumper: okay my local master is stuffed up
<waigani> thumper: let me clean this up and try again
<thumper> ok
<menn0> thumper: that change you helped me with: https://github.com/juju/juju/pull/554/
<menn0> thumper: do you have time to review?
<thumper> sure
<menn0> thumper: i've already made sure this solves the problem of downgrades in local/manual envs
<menn0> so once this is in
<menn0> the other change can be proposed straight away
<waigani> thumper: cleaned up, I'll address your initial comments before pushing again.
<waigani> thumper: user take does not have a Name() method?
<thumper> yes it does
<thumper> are your deps up to date?
<waigani> thumper: I just ran the godeps tool
<thumper> but your branch probably isn't up to date with trunk
<thumper> menn0: only one real comment
<menn0> thumper: I don't think what you've suggested is necessary
<thumper> why?
<menn0> the test intentionally has no tools in env storage
<menn0> so if a download was attempted it would fail
<menn0> this work was done test first, and it failed because of this
<waigani> thumper: dad duty just started, I'll get it back up later tonight.
<thumper> waigani: ack
<menn0> thumper: does that make sense?
<thumper> menn0: in which case, please add a comment :-) as it wasn't obvious to me
<menn0> thumper: will do
<menn0> thumper: added comment and rearranged the test a little to make things clearer
<menn0> how's this? https://github.com/juju/juju/pull/554/files
<thumper> menn0: cheers
<davecheney> wwitzel3: you've got a LGTM on that change
<davecheney> there is nothing to be gained from another round of reviews
<davecheney> do what you think is reasonable then submit that sukka before we get locked down again
<thumper> menn0: a slightly premature merge :)
<menn0> thumper: sorry. I wasn't sure if you were going to say anything else
<thumper> menn0: that's fine :)
<menn0> thumper: is there a problem
<menn0> ?
<thumper> no
<menn0> just saw your LGTM :)
<wwitzel3> davecheney: thanks
<menn0> potentially dumb question: what does this do? go test -p 2 ./...
<menn0> and why does the build do it?
<menn0> wallyworld_'s last merge got aborted because a test took too long and then the tests were run again as above and passed
<wallyworld_> menn0: it runs the tests with reduced parallelism
<wallyworld_> because our tests sucks
<menn0> oh right
<wallyworld_> and often fail spuriously
<menn0> I didn't realise we were running tests in parallel at all!
<wallyworld_> but -p 2 works more often than not but is slower
<wallyworld_> menn0: that's what Go does out of the box
<menn0> right
<menn0> well I have learned something today
<wallyworld_> \o/
<menn0> time to put my feet up
<menn0> :)
<wallyworld_> enjoy
<davecheney> menn0: you and andrew gerrand
<davecheney> he didn't know -p $NUMCPU was the default either
<menn0> davecheney: well I don't feel so bad then :)
<davecheney> possibly he doesn't spend as long as you and I do waiting for tests to ru
 * thumper goes to make dinner
<wwitzel3> I'm off, be back in the morning o/
<davecheney> wallyworld_: ping
<wallyworld_> davecheney: hey
<davecheney> wallyworld_: i have a question about the keymanager
<davecheney> when we pass user: juju-system-key
<davecheney> to the keymanager
<davecheney> is that a tag ?
<wallyworld_> hmmm
<wallyworld_> i didn't add that bit, but let me look at the code
<davecheney> i've had to put a horrible hack in to make juju accept that as a tag
<davecheney> ok, ifyou didnd't writ it
<davecheney> that is ok
<davecheney> you have TODOs all over it
<wallyworld_> i don't think i did - i wrote the key manager itself
<wallyworld_> but
<davecheney> i have exactly one test to fix
<davecheney> then this bloody branch is done
<wallyworld_> i think someone came along and added support for a system-key
<davecheney> well, they cocked it up
<wallyworld_> let me look briefly
<wallyworld_> i could have been me, but i can't recall
<wallyworld_> davecheney: looks like it was me. a key can be for a user eg fred, or is the juju system key
<davecheney> right, so they aren't tags
<wallyworld_> i guess that was done back before tags
<davecheney> yeah
<davecheney> which makes it a bit of a hack to use common.AuthFunc
<davecheney> anyway
<davecheney> i've done a gross hack
<davecheney> it'll hold
<wallyworld_> yuk, sorry
<davecheney> meh
<davecheney> we'll live
<davecheney> it can't be fixed now because of BACKWARDS COMPATABILITY !!1!
<wallyworld_> yep :-(
<davecheney> wallyworld_: https://github.com/juju/juju/pull/556
<davecheney> DONE
<davecheney> it's only taken a month
<davecheney> i'm off to the pub
<wallyworld_> looking
<wallyworld_> have a drink for me :-)
<voidspace> perrito666: let me know when you arrive :-)
<jam> wallyworld_: I'm pretty sure you'll be having a drink for yourself, too :)
<TheMue> morning
<wallyworld_> jam: yes i will :-)
<jam> morning TheMue
<TheMue> dimitern: ping
<dimitern> TheMue, hey
<TheMue> dimitern: I changed my approach a bit, but I still hesitate to use the Entity approach.
<TheMue> dimitern: see https://github.com/TheMue/juju/compare/capability-detection-for-networker
<dimitern> TheMue, looking
<TheMue> dimitern: Entity is really generic, for all entities with a tag. But IsManual is a flag for machines.
<TheMue> dimitern: I now read it only once when creating a local Machine instance, so it is cached and will never bee refreshed, because it wonât change.
<jam> TheMue: AIUI Jobs is also specific to Machines, but the "thing" is Entity which is more generic
<dimitern> TheMue, no, please let's not make 2 calls for every life request
<TheMue> jam: Entity doesnât know about jobs
<dimitern> TheMue, yes jam's right - there's already a precedent with Jobs, adding another flag is not a big deal I think
<jam> dimitern: themue seems to think I've got the wrong object, I could be wrong there.
<jam> But I thought I remembered seeing other stuff that was machine specific on the generic class
<TheMue> Aaargh, I see, weâre talking about AgentGetEntitiesResult
<dimitern> yes! :)
<TheMue> But Iâm still not happy with overloading that struct more and more
<dimitern> TheMue, that's what versioning is for, no?
<TheMue> dimitern: I donât talk about differences, I talk about common and specific fields, separation of concerns
<dimitern> TheMue, the agents need this data to do their jobs (efficiently), and it's also an internal api
<TheMue> dimitern: So I would be more happy to let the machine fetch a doc especially for a machine
<TheMue> dimitern: still not clean and optimal for maintainability, you see how easy one can move into the wrong direction
<dimitern> TheMue, I don't think this is us moving in the wrong direction
<TheMue> dimitern: IMHO the Machiner would be responsible for Machine operations and the Machine in the should know what it has to know about machines
<dimitern> TheMue, fwiw, we probably need to have 2 separate unit and machine agent facades, rather than pretending we can treat both and their entities the same
<dimitern> TheMue, and if you look at AgentGetEntitiesResults, only Life is common between machines and units, ContainerType and Jobs only apply to machines
<TheMue> dimitern: yeah, maybe. so to come to a quick solution I will do the AgentGetEntitiesResult change. and Iâll file an issue for refactoring with low priority
<dimitern> TheMue, that sounds good to me!
<voidspace> So "juju restore" bootstraps again (which it needs to)
<voidspace>  but if it fails it leaves the instance created - but with no running agent.
<voidspace> So "juju status" (etc) fail and you have to manually destroy the instance (or accidentally leave it running because you don't realise)...
<voidspace> jam: your hacked-up-backup script (for local provider)
<voidspace> jam: it uses $JUJU_ENV
<voidspace> this isn't set when I'm running the script directly, so I have to set it myself I guess
<voidspace> or would you expect it to be set?
<jam> voidspace: I believe it is set if you run "juju restore" but I just use "JUJU_ENV=local ./juju-restore"
<voidspace> jam: right, cool
<jam> voidspace: if you just run "juju-backup" it fails too, right?
<jam> My understanding is that the plugin infrastructure sets that variable
<voidspace> jam: it fails for publiic-key
<voidspace> jam: right
<voidspace> jam: that's what I *thought*, but I as just checking my system wasn't screwed in new and exciting ways
<jam>  voidspace: did you do any more hacking on it, or shall I see if I can get it through to completion?
<voidspace> jam: I was trying (and failing) to get amazon working
<voidspace> jam: I've now switched to the local script
<voidspace> backup seems to work fine :-)
<dimitern> jam, got a minute?
<jam> voidspace: well local backup "works" in that it generates a tarball, but all the paths match exactly the local paths, which means they *don't* match the "official" paths.
<dimitern> jam, I'd like to pass an idea by you
<jam> well, what *I* would consider official, at least (/var/lib vs /home/$USER/.juju/etc)
<voidspace> jam: right
<voidspace> yep
<jam> dimitern: G+ or just in IRC?
<dimitern> jam, just IRC should be fine
<jam> voidspace: I think the thing to do there would actually be to rewrite them, but I don't expect backup of Amazon to be able to restore into local
<voidspace> sure!
<voidspace> just local to local will be a good start
<voidspace> local to amazon might be useful :-)
<dimitern> jam, so after experimenting with the net package's Interfaces() and InterfaceAddrs() on precise, trusty and utopic daily image, I realized we can get all the information we need about the network interfaces *without* ever bothering to read /etc/network/interfaces or /interfaces.d/*.cfg
<voidspace> create your system locally, then "restore" to your production systems
<jam> dimitern: what is it that we wanted but don't have access to?
<dimitern> jam, like what?
<jam> "we can't get all the information we need" such as?
<jam> looking here: http://golang.org/pkg/net/#Interface
<jam> it looks like we don't have anything about bridging, etc.
<dimitern> jam, we can discover all interfaces, bridges included, their flags (up, down, broadcast, loopback, multicast), and their index and names, also any addresses assigned
<dimitern> jam, I'm proposing to just rewrite the interfaces config as needed for ifup/down to work at boot, and as the networker starts it will read all configs and check if there are any changes between what we have in state and what's configured on the machine, and rewrite the configs as needed
<jam> dimitern: ok, I read what you said wrong. I read it as "we cannot get all the information", if we *can* that's great, though I have my doubts. I'm also wondering where "net" gets its information from, is it just the live state, or is it the state that will be started on restart
<dimitern> jam, the net package gets the info using syscalls and some obscure netlink packets
<jam> dimitern: k, so it would be the "live" state, vs the saved state
<dimitern> jam, it is more or less equivalent to what we can see using the "ip" command (i.e. real-time)
<jam> dimitern: right, it is a kernel sort of interface
<jam> "ifup" and /etc/network/interfaces are a debian/ubuntu-ism
<dimitern> jam, yes, and it works just as well in precise as in utopic
<jam> IIRC, Fedora/Redhat use a different saved state
<dimitern> jam, well, there is support only for linux to discover the interfaces like that, but I've yet to try centos and fedora
<dimitern> jam, that's true, we'll have to render different stored config for other distros, but let's worry about this as we get there :)
<dimitern> jam, another up-side of using the net package is we don't need to rely on /etc/network/interfaces being there to decide whether to start the networker or not (more so now, as we have the "safe mode" to disable it selectively)
<jam> dimitern: sorry about the delay, emergency take-the-dog-out action.
<jam> dimitern: so I like the idea a bit (no safe mode would be very nice)
<jam> *but*
<jam> I wonder if we really don't want to save our configuration, which means writing to /etc/network/*
<dimitern> jam, no, no, I'm not saying that :)
<jam> dimitern: perhaps we could only rewrite things when we've actually decided we need to change things
<dimitern> jam, we do write the network config in /etc, but we don't read it - we use the net package to discover what nics are there
<jam> dimitern: but we shouldn't write it unless we think there is a change from what's written there, should we?
<jam> in which case, we have to read what is there to write anythnig
<dimitern> jam, we need to write the config, otherwise without a working juju machine agent + networker, the networking won't be setup correctly after system boot
<dimitern> jam, yes, we get all NICs, read what's there and render and write "managed" config for each nic that we need to setup, but only if it's pre-existing config on disk is different from what the networker would generate
<dimitern> jam, am I making more sense? :)
<jam> dimitern: so I can understand that we may not need to parse them, I'm not 100% sure that our method of comparison as byte-strings is quite what we want (what about comments in the files, etc).
<dimitern> jam, ok, so let's backtrack just a bit
<dimitern> jam, the user trusts (or we hope they do) juju to do the right thing with the machines juju manages, right? the same should be true wrt the networker taking control over the machine's networking config and managing it, without expecting the user to interfere?
<jam> dimitern: the whole point of "Safe" is because we don't trust us on Manual or Local
<dimitern> jam, we make this obvious by using the comment header "# Managed by Juju, please do not change." in each config file the networker generates
<dimitern> jam, right, so for "safe mode", don't write changes to the config or run any ifup/down commands, effectively just sitting there and logging what we discovered
<dimitern> jam, I'm hoping we can get rid of safe mode for manual machines once we have a way for the user to explicitly tell us how to setup networking on them, or we're able to discover it somehow, smartly
<jam> dimitern: TheMue: voidspace: standup?
<voidspace> jam: omw
<voidspace> perrito666: ping
<jam> voidspace: are you still there?
<jam> marcoceppi: poke about charm tools: https://bugs.launchpad.net/charm-tools/+bug/1359170
<mup> Bug #1359170: juju charm create fails in an unhelpful fashion with juju-1.21 in PATH <Juju Charm Tools:New> <https://launchpad.net/bugs/1359170>
<jam> Also, when I look at: https://juju.ubuntu.com/docs/authors-charm-writing.html
<jam> It has the lines:
<jam> juju charm create vanilla
<jam> but in my browser, those are wrapped onto 2 lines
<jam> so it looks like 2 commands
<jam> Ctrl Refresh fixed it for me.... very weird
<perrito666> morning voidspace
<voidspace> perrito666: morning
<voidspace> perrito666: solved my horrendous backup / restore problems
<voidspace> perrito666: well, more or less
<perrito666> voidspace: heh, all backup and restore problems are horrendous
<voidspace> perrito666: backup wasn't working *at all*
<perrito666> voidspace: odd, since when?
<voidspace> perrito666: but jam finally sussed it - the juju 1.18 juju-backup is in the path and the dev one isn't
<voidspace> perrito666: only for me on trunk
<voidspace> perrito666: so I have to run the backup script directly to try it
<perrito666> sweet, is there a PR already?
<voidspace> perrito666: not a PR yet
<voidspace> perrito666: I'll show you what I have (complete now I think)
<jam> perrito666: the issue is that the 1.18 "juju-backup" shell script didn't know about /usr/lib/juju/bin/mongodump which you fixed for 1.20, but 1.18 is the one in /usr/bin/juju
<voidspace> perrito666: although I need to test it, hence the problem with backup not working
<voidspace> perrito666: https://github.com/voidspace/juju/compare/backup-ssh
<jam> and just running "juju backup" searches $PATH for "juju--backup" so even though juju itself is 1.21, "juju backup" was running the 1.18 version.
<voidspace> jam: so backup and restore from trunk worked fine for me - which is a first
<voidspace> now to try it with my code...
<perrito666> ah very true, my usual test uses absolute path
<jam> perrito666: you mean /home/user/path/to/juju/cmd/plugins/juju-backup/juju-backup ?
<perrito666> jam: yup, since I use the automated test from qa I added my paths to make sure it alwas runs the same
<jam> perrito666: and then presumably you just set JUJU_ENV to test the right environment?
<jam> TheMue: didn't you land a patch to allow "@" syntax for "juju set" ?
<perrito666> jam: the CI test requires it as a param actually (test_recovery.py)
<jam> perrito666: I don't see a test_recovery.py in lp:juju-ci-tools
<jam> test_assess_recovery.py ?
<perrito666> jam: it was test_recovery.py until friday
 * perrito666 pulls
<jam> perrito666: perhaps it is a different source code, because test_assess_recovery.py imports "test_recovery" at the top.
<perrito666> RM  test_recovery.py => assess_recovery.py
<jam> perrito666: looks like test_assess_recovery.py didn't get fixe
<jam> fixed
<jam> perrito666: and it looks like CI does, indeed, run "juju-backup" and not "juju backup" which might be something we should think carefully about, since *users* are going to be doing the latter.
<jam> but we are testing the former
<perrito666> I must have local changes to ensure that, ah I see, those tests ship their own version of backup
 * perrito666 wonders why
<voidspace> interesting
<voidspace> yeah, seems odd
<perrito666> jam: good point Ill nag sinzui about it when he shows
<voidspace> anyway, about to test my restore
<voidspace> just killed the state server
<voidspace> jam: looks like restore does fetch the tools from the correct storage location when used in conjunction with upload-tools (just btw)
<jam> voidspace: good to hear. I expected as much, since we are using the control-bucket as defined in the .jenv
<jam> I almost wonder if backup should be saving the .jenv as well.
<jam> since you really need both (I believe)
<voidspace> yep
<voidspace> jam: so my restore failed
<jam> voidspace: did stock juju core succeed?
<voidspace> jam: it looks like it opened the apiState fine, but then the Client failed to get an address for "machine-1" (my mysql unit - so the call to client.PublicAddress("machine-1") failed
<voidspace> jam: yes
<voidspace> jam: heh, although to be fair I didnt try with a unit with trunk as I wanted a quick test
<voidspace> but this seems like a plausible failure
<voidspace> I'm going to make coffee and then dig in
<voidspace> no public address found: unknown unit or machine "machine-1
<jam> voidspace: the nice thing about backup and restore is that once you have the backup, you don't have to backup again, just try restore
<voidspace> yep :-)
<jam> voidspace: that sounds like a "tag" vs an "id" issue
<jam> as in, it might be expecting "1" for "machine-1"
<voidspace> ah right, just "1" maybe
<voidspace> will try that
<voidspace> even before coffee :-)
<voidspace> although if restore fails you have to manually kill the ec2 instance
<voidspace> as a failed restore leaves the instance it created running
<TheMue> jam: sorry, just seen your question. have been to lunch.
<voidspace> ok, restore underway again
<voidspace> coffee time
<TheMue> jam: have to see what it has been exactly, but yes, IMHO it has been the @ to lead from files
<TheMue> jam: https://bugs.launchpad.net/juju-core/+bug/1216967
<mup> Bug #1216967: Missing @ syntax for reading config setting from file content <compat> <docs> <hours> <improvement> <juju-core:Fix Committed by themue> <https://launchpad.net/bugs/1216967>
<jam> TheMue: k, I thought it would show up in "juju help set" but I didn't see it there
<TheMue> jam: interesting, afair it should
<TheMue> jam: the set doc has a hint. three paragraphs, the second one. which version do you use?
<jam> TheMue: http://paste.ubuntu.com/8097539/
<jam> is all I see for "juju help set
<TheMue> jam: see line 28 in juju/cmd/set.go, itâs more. but itâs not yet released
<TheMue> jam: so if you donât use your own build you wonât see it.
<jam> TheMue: $ ~
<jam> ~/dev/go/bin/juju set -h
<jam> I'm using trunk
<jam> ~/dev/go/bin/juju --version
<jam> 1.21-alpha1-trusty-amd64
<jam> TheMue: you have "setDoc" set to the correct thing
<jam> but line 48
<jam> has "Doc: "set one or more ..."
<jam> not Doc: setDoc
<jam> TheMue: care to submit a bug and a patch for it?
<TheMue> jam: aaargh, I found setDoc before and expanded it. never seen that it isnât used.
<TheMue> jam: yep, will quickly do it.
<voidspace> jam: perrito666: looks to me like restore worked fine!
<perrito666> voidspace: sweet
<voidspace> jam: perrito666: will double check the code and create PR
<voidspace> and then *re-PR* the database shutoff
<jam> voidspace: greate. why didn't it work the first time? (did you fix something?) was it just the "machine-1" vs "1" issue?
<voidspace> jam: yep
<voidspace> jam: switching to id instead of tag fixed it
<jam> voidspace: I wish our code was more consistent here, but that is what API versioning could get us to :)
<jam> voidspace: can you file a bug tagged "API" about the inconsistency?
<jam> and tech-debt
<voidspace> jam: yep, and I'm adding a note to the PublicAddress docstring that "target" is an id not a tag.
<perrito666> aghh the convoluted settings of networkmanager/dnsmasq in 14.04 drive me crazy
<jam> perrito666: it looks like we broke plugins in 1.21 as well, which would have been caught if we were doing "juju restore" instead of "juju-restore", not that we should have used that to actually catch the bug
<perrito666> jam: thats huge, dont we have a test to make sure plugins actually works?
<jam> perrito666: apparently not https://bugs.launchpad.net/juju-core/+bug/1359170
<mup> Bug #1359170: juju charm create fails in an unhelpful fashion with juju-1.21 in PATH <Juju Charm Tools:New> <juju-core:Triaged> <https://launchpad.net/bugs/1359170>
<jam> I tried to run the charm-tools plugin
<jam> and apparently we stopped passing arguments to plugins in 1.21
<jam> I'm testing 1.20 now
<jam> it seems broken there, too
<jam> bugger
<jam> curtis won't be happy
<jam> wallyworld_: can I point you to bug #1359170 when you're around?
<mup> Bug #1359170: arguments no longer passed to plugins <Juju Charm Tools:Invalid> <juju-core:Triaged> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1359170>
<jam> I'll try to pick it up since I discovered it
<jam> CI isn't failing, so I'm not sure if it should be "CI regression" and block trunk, but it is a *serious* f-up on our part.
<jam> I can't quite see why it is failing yet, though.
<jam> sinzui: you'll certainly be happy to hear that I just opened a critical bug against 1.20...
<sinzui> :)
<jam> sinzui: bug #1359170
<mup> Bug #1359170: arguments no longer passed to plugins <regression> <Juju Charm Tools:Invalid> <juju-core:Triaged by jameinel> <juju-core 1.20:Triaged by jameinel> <https://launchpad.net/bugs/1359170>
<sinzui> jam, That is nicer that something breaking the entire packaging and publication step. I think you know what is wrong in you bug. at this moment CI is dead and I don't know why
<TheMue> Hmm, itâs more intelligent to push branches when planing to create a PR.
<sinzui> abentley, mgz: Looks like by stress from yesterday was just waiting for me to return to the job this morning. My top priority is to backport some ubuntu packaging rules into our source package branch. We want 1.20.5 accepted into utopic by feature freeze this Friday
<abentley> sinzui: Hoo boy.  Have fun.
<sinzui> Looks like one of the packaging changes broke CI. I will revert, but I think we need to rerun 1.20 each time I want to verify my changes to the source package branch
<TheMue> jam: https://github.com/juju/juju/pull/559, extreme trivial :D
<abentley> sinzui: Is "/home/ubuntu/juju-build/*.deb" right?  Should it be in the jenkins home dir?
<jam> TheMue: I'd like to have a Launchpad bug to reference for this fix, also, you can add a test case in test_main.go which asserts the output of "juju set -h" matches setDoc.
<jam> TheMue: otherwise LGTM
<sinzui> abentley, no, because the script was written to work on a remote instance.
<TheMue> jam: OK, the bug already exists, will put it in the description. And will add a test. Do we test the doc output in other commands too? Otherwise we should add tests there too.
<sinzui> abentley, I added a hack to the script to run on local host to use the jenkin's workspace...but we haven't dismantled publish-revision to build on separate machines.
<jam> TheMue: I've written tests in "main_test.go" before in order to assert that things were actually registered
<jam> TheMue: for example "juju sync-tools --help" is in there.
<jam> TheMue: *I* did it, because it was the most obvious place to check the lines in main.go where we have "r.Register(&Command)"
<jam> TheMue: (test that the command exists because we get proper help text for it, rather than trying to say run the command and have unwanted side effects)
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1359170
<TheMue> jam: fine, so should look if this is done for all commands (but in an extra change)
<jam> ah ffs, things work *if* you pass an environment name in "-e"....
<sinzui> jam: I would like your advise. backup restore doesn't work unless mongodb-clients is not installed on the client machine. I think this is a " Depends" relationship in packaging. It was suggested to me that I use "recommends" because it is not needed by everyone.
<jam> sinzui: if having something installed/not installed on the *client* changes whether backup + restore works then we just have a bug.
<jam> we should not depend on local stuff in order for us to run stuff on the server.
<jam> sinzui: it sounds like we are doing some sort of "does X exist" check on the client, when it needs to be run on the server.
<perrito666> sinzui: mongodb-clients isrequired on the server not the client
<sinzui> perrito666, oh!
<perrito666> sinzui: and it is only required for restore
<perrito666> and restore installs it if its not there
<sinzui> perrito666, fab. Thank you very much perrito666 , so I am going to mark the bug as fixed in 1.20.x by you
<perrito666> sinzui: indeed, the bug you just marked as fixed has the wrong diagnose, but it has been fixed in 1.20
<sinzui> perrito666, Yeah. I elected to show it fixed in the latest to encourage people to seek it out
<perrito666> sinzui: jam talked about it earlier, its because backup in 1.18 does not do a proper lookup for the tools which are in a juju folder
<jam> dimitern: you're OCR today, so https://github.com/juju/juju/pull/560 if you want to give it a look. It needs a test that we operate properly under the circumstances I described in the bug.
<jam> dimitern: I'd also like to run it past thumper, since I'm pretty sure he wrote the environmentCommandWrapper stuff.
 * sinzui restarts CI from the publish-revision step for master
<dimitern> jam, sure, will have a look in a bit
<jam> perrito666: so it turns out we wouldn't have discovered my bug by running "juju restore", because it only breaks Args when you don't have an Environment, and "juju-restore" must have an environment set anyway.
<perrito666> jam: you do make an interesting case for a test anyway
<jam> perrito666: so we might have a test, but it either has a default environment, or what it is testing requires one.
<jam> perrito666: I may be unique in how I do my environment management, and if I'm unique enough, maybe we should just deprecate it and not actually support it
<jam> either that, or we need to be doing more testing that my way is actually supported.
<jam> bash autocompletion also was (is?) buggy for me because it assumed you'd have an environment set
<perrito666> jam: well, if we support it, we need to test it
<alexisb> natefinch, wwitzel3: TOSCA call?
<natefinch> alexisb: coming
<wwitzel3> alexisb: I thought I was already in it ..
<wwitzel3> alexisb: restarting the hangout :)
<dimitern> jam, reviewed; just one comment - sorry it took so long :/
<ericsnow> dimitern: you have a minute to take a look at a PR? https://github.com/juju/juju/pull/530
<alexisb> TheMue, on my way, but will be a few minutes late
<TheMue> alexisb: ok
<ericsnow> perrito666, natefinch, wwitzel3: standup?
<perrito666> ericsnow: going, just fighting with my hangut
<natefinch> email I just got: "This is to notify you that the sum of 800,000 Britain Currency has been Donated to you by Colin Chris Weir Donation Scheme. for claims provide the below details"
<gsamfira> ahh, they toned it down a bit. Used to be 10X that :P
<natefinch> I just though "britain currency" was cute
<natefinch> Also, funny they call it a "donation scheme"
<natefinch> fwereade: at some point I'd like to pick your brain about charm feature flags.  I was hoping to work on it this week, but I'm less certain now, after talking to rick_h__ about it.
<natefinch> sinzui: did you make progress on getting godeps in the tarball?  No big deal if it's not in yet, just checking.
<fwereade> natefinch, dammit, I wanted to talk to you today, but the babysitter's just arrived and I'm trying to finish an email
<fwereade> natefinch, what time is it for you now?
<natefinch> fwereade: 12:30
<fwereade> natefinch, cool, I hope I can get back to you in the next 5.5h
<natefinch> fwereade: cool, ping me when you have time.  I appreciate you making time for it in your evening.
<sinzui> natefinch, I failed. I cannot make it run run in unittests. But this morning I realised I want to run that as a precheck before the tarball is created. I hope to have that sorted out this week. Sorry, it wasn't an easy change, and I need to focus on putting 1.20.5 into utopicbefore Friday
<alexisb> jam, you doing the cloudsigma call today
<natefinch> sinzui: no problem.  It's not pressing.  It's a nice to have.  What I can do is make the test skip itself for now if godeps doesn't exist, that way it can still help developers (who should have godeps).
<sinzui> natefinch, thank you that works for me because godeps does exist when we make the tarball.
<rogpeppe1> natefinch: did you consider making that godeps test part of CI rather than an actual test in the suite?
<natefinch> rogpeppe1: it's at least as important for developers as CI..... since innumerable times I've seen people ask why something was failing, only to figure out later that their dependencies were out of date
<natefinch> (which has happened to me, as well)
<natefinch> sinzui: I would like to make it mandatory at some point.  If you're developing on juju, you *need* godeps... I don't want someone to start hacking on juju and wonder why it's broken, just because they haven't run godeps.
<sinzui> ericsnow, I am indeed very busy today I will reply before I EOD. Hp is my preferred could this week, and the the juju-core-slave machine might be ideal for setting up review board there. I need to think about your questions in more details though
<ericsnow> sinzui: no worries :)
<rogpeppe1> natefinch: ok. BTW, in your test, why not just use diff?
<natefinch> rogpeppe1: what diff?
<rogpeppe1> natefinch: diff(1)
<natefinch> rogpeppe1: windows?
<rogpeppe1> natefinch: no diff on windows?
<natefinch> rogpeppe1: lol, no :)
<gsamfira> git diff works :P
<gsamfira> or were you referring to the manual? :P
<gsamfira> in which case...no :D
<rogpeppe1> gsamfira: i was referring to a command-line diff tool. i guess git has access to one, but perhaps it's not available for general use
<gsamfira> it is available
<rogpeppe1> natefinch: ^
<gsamfira> if you have Git installed its in C:\Program Files (x86)\Git\bin\diff.exe
<gsamfira> along with less, grep, etc
<gsamfira> a small slice of home when having to work on windows
<rogpeppe1> gsamfira: hmm, but i guess that won't be in the path, so you'd need some custom logic to work out how to find it (perhaps it's not in C:)
<gsamfira> setx PATH "%PATH%;C:\Program Files (x86)\Git\bin"
<gsamfira> and there ya go
<natefinch> the last thing I want is yet another external dependency
<gsamfira> if you set up your own system
<gsamfira> you know where it is :)
<rogpeppe1> natefinch: out of interest, what proportion of our tests pass under windows?
<gsamfira> well, yes...an external dependency sucks :)
<rogpeppe1> natefinch: you're probably right. but you could reduce your "monstrosity" by factoring out a "diff" function (diff(s, t string) []string, or something
<natefinch> rogpeppe1:  a bunch, and that's next on our list of things to do - get tests passing on windows (except those we deem we can skip on windows because we just can't support them, like containers)
<natefinch> rogpeppe1: yep
<gsamfira> most tests on Windows fail because of the same reasons: unsanitized paths, short paths vs long paths, directory names containing illegal characters on windows (like ":")
<gsamfira> there will be a few pull requests this week to fix some of them
<gsamfira> the new github.com/juju/juju/wrench package also breaks tests on windows
<natefinch> wwitzel3: want to do our 1:1?
<wwitzel3> natefinch: yep
<natefinch> heh, I had to fix dependencies.tsv on my PR that tests dependencies.tsv, because it was wrong.
<wwitzel3> :)
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1359170, 1359333
<sinzui> natefinch, can you find an engineer to look into bug 1359333
<mup> Bug #1359333: win client thinks the cloud-init log is on the c drive <ci> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1359333>
<natefinch> sinzui: crud. Yep.
<natefinch> fwereade: forgot about the all team meeting tonight, I'm going to have to run to get some stuff done before then. Hopefully we can talk in the morning.
<thumper> (â¯Â°â¡Â°)â¯ï¸µ â»ââ»
<wwitzel3> yep
<thumper> menn0: still need that review?
<thumper> can do it while all tests running
<menn0> thumper, yes pleased
<menn0> please even
<menn0> (urgh... I have had just over 2 hours sleep - sick kids suck)
<thumper> menn0: trade you https://github.com/juju/juju/pull/563
<menn0> thumper, done
<thumper> menn0: that was quick, did you read it ?
 * thumper pokes
<menn0> thumper, no I meant "yes, that trade sounds fair"
<thumper> oh
<thumper> heh
<menn0> tiiiiiired
<ericsnow> menn0: when you get a chance could you take a look at https://github.com/juju/juju/pull/530
<menn0> the fun thing about me reviewing your change is that you then have to review my review :)
<thumper> heh
<alexisb> menn0, heh
<thumper> hmm
<menn0> thumper, maybe we ask dave or someone else to do that for this situation
<menn0> ericsnow, will do
<ericsnow> menn0: thanks!
<menn0> let me get this review done and another small PR up first
<menn0> thumper, regarding your PR... is it safe to assume that people won't/can't upgrade directly from pre 1.18 to post 1.20?
<menn0> thumper, they have to go through an intermediate 1.18 release first?
<thumper> menn0: yes
<menn0> actually 1.18 and 1.20
<thumper> that has always been the approach
<thumper> the idea with the upgrade steps is that now it is *possible* to jump
<thumper> but we don't support it yet
<menn0> thumper, is that enforced?
<menn0> thumper, found the bit that enforces it: validate() in upgradejuju.go
<thumper> not explicitly I think
<menn0> shame that it's enforced on the client side
<thumper> oh, good :)
<menn0> thumper, with the new release number scheme it's not quite right either...
<menn0> thumper, the logic tries to find the next stable version by bumping minor by 1 if the current version is a dev release and by 2 otherwise
<menn0> thumper, that's no longer right is it?
<thumper> no
<thumper> please file a bug
<menn0> thumper, already doing it :)
<menn0> thumper, when does the new scheme start? 1.20? i.e. will the next stable release be 1.21?
<thumper> menn0: yes
<thumper> 1.21 will be a stable release
<waigani> thumper:  talking of reviews: https://github.com/juju/juju/pull/553
 * thumper nods
<menn0> thumper, please check the milestones and priorities for bug 1359435
<mup> Bug #1359435: Next version selection for upgrades is no longer correct <juju-core:New> <juju-core 1.20:New> <https://launchpad.net/bugs/1359435>
<menn0> thumper, review done.
 * thumper does a bootstrap 
<menn0> thumper, so the stateservinginfo doc was only being created via the functionality for dealing with legacy envs? nice.
<thumper> yeah
<thumper> found that by failing tests when I removed it
<thumper> local bootstraps fine
<menn0> thumper, cool
<menn0> thumper, I've replied to your review comments
 * thumper nods
<thumper> menn0: that comment makes no sense..
<thumper> This is capturing the upgradedToVersion field from the agent's config as the agent starts up - i.e. the version we've just upgraded from
<thumper> I read that as:
<thumper> 'upgradedToVersion' is the version we have just upgraded from
<thumper> which seems arse backwards
<menn0> it is
<menn0> upgradeToVersion is the version for which upgrades have successfully run
<menn0> it gets updated to version.Current once upgrade steps have successfully run
<menn0> but when a machine agent starts up, before the upgrade steps have run
<menn0> it'll be set to the previous agent version
<thumper> ok, that kinda makes sense
<menn0> I didn't name this stuff
<menn0> it's before my time
<menn0> thumper, I'm so used to this part of the code now that it doesn't even seem strange to me anymore
<menn0> but I do remember finding it confusing initially
 * thumper nods
<thumper> wallyworld: I took a very quick look at bug 1359333 (blocking landing)
<mup> Bug #1359333: win client thinks the cloud-init log is on the c drive <ci> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1359333>
<thumper> but I can't see how the diffs cause the failure
<wallyworld> thumper: ok, i'm about to go into a meeting, but can look after that
 * thumper has standup now too
<davecheney> wallyworld: https://github.com/juju/juju/commit/3a8f263f8c2977241fc39450e9fb47b4d56fc975
<davecheney> this was the commit that broke CI
<davecheney> please revert it
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1359333
<mup> Bug #1359333: win client thinks the cloud-init log is on the c drive <ci> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1359333>
<wallyworld> davecheney: hi, just finished a call, looking now
<davecheney> wallyworld: the bug is it uses paths.DataDir which uses version.Current
<davecheney> and version.Current, when you're using windows tools, will using windows tools
<davecheney> this is reason number 9001 why version.Current is lethal
<wallyworld> davecheney: do these guys doing windows develppment event test their stuff on windows before landing?
<davecheney> wallyworld: that's between them and the person that reviewed their code
<wallyworld> well i can understand how stuff gets through a review, but not to test on the platform you are developing for....
<thumper> holy cow: 44 changed files with 1,954 additions and 360 deletions.
<thumper> wallyworld: I don't think they are really testing it manually
<thumper> that should really be raised as a concern
<wallyworld> YES
<thumper> alexisb: ^^
<wallyworld> thumper: we'll discuss tonight at team leads' meeting
<thumper> ack
<wallyworld> thumper: davecheney: so are we 100% sure that https://github.com/juju/juju/pull/189 is the bad one? i'll just hit the Revert button if so
<davecheney> wallyworld: not 100% sure
<davecheney> what's the worse that can happen ?
<thumper> wallyworld: can't say 100%, but indications are high
<davecheney> we just re apply iy
<wallyworld> ok
<davecheney> man version.Current is a timb bomb
<alexisb> thumper, wallyworld, davecheney: sorry I have to run, but I added a note about the windows workload reviews to the leads agenda so we wont forget
<wallyworld> +1, and not just reviews, actual proper testing and due dilligence prior to landing
<wallyworld> davecheney: thumper: reverted, let's now wait to see if CI is happy again
<thumper> davecheney: did you mark it as fixes the bug so ci is unblocked?
<thumper> sorry davecheney
<thumper> wallyworld: ??^^
<davecheney> thumper: thats ok thumper
<davecheney> wallyworld: how long do we have to wait ?
<wallyworld> thumper: i didn't because the revert didn;t go through ci
<wallyworld> but i can mark the bug as resolved
<wallyworld> pending CI running it's tests
<wallyworld> we can reopen if needed
 * thumper wonders if it should have...
<davecheney> \o/, one down, one to go
<wallyworld> thumper: maybe it should have, but i'll pull trunk now and run the tests
<wallyworld> WTF
<wallyworld> FAIL: dependencies_test.go:42: dependenciesTest.TestGodepsIsRight
<wallyworld> dependencies_test.go:77:
<wallyworld>     ...
<wallyworld> dependencies_test.go:70:
<wallyworld>     c.Fatal(string(out))
<wallyworld> ... Error: godeps: no version control system found for "/usr/lib/go/src/pkg/bufio"
<wallyworld> godeps: no version control system found for "/usr/lib/go/src/pkg/bytes"
<wallyworld> thumper: davecheney: you guys seen ^^^^ as a result of PR 495?
<thumper> huh? nope
<wallyworld> sigh
<thumper> I do think that a test for godeps isn't right
<thumper> and the make rules I added were better
<thumper> but ..
 * thumper shrugs
 * wallyworld will reply to list
#juju-dev 2014-08-21
<davecheney> wallyworld: i don't see the value of pr 495
<davecheney> i would not have approved it
<davecheney> CI already tests godeps works
<wallyworld> me either
<davecheney> we don't need to run it locally
<davecheney> +1 for reverting it for more review
<wallyworld> well, i'll talk to nate first
<rick_h__> davecheney: wallyworld I think that was casuing CI some issues. They had issues with it today
<davecheney> wallyworld: that's +2 for revcerting it
<davecheney> that is a carrying margin
<davecheney> nate can always repropose it
<davecheney> nothing is lost
<wallyworld> rick_h__: what issues do you know?
<rick_h__> wallyworld: it was down and not functional for a bit. I know sinzui and he were chasing down things and trying to get something with deps right
<rick_h__> wallyworld: sorry, only idly watched irc as I wasn't directly effected
<wallyworld> np
<rick_h__> wallyworld: I can check my logs for more info
<rick_h__> wallyworld: but then this is logged right?
<wallyworld> what is? CI output?
<rick_h__> wallyworld: IRC
<wallyworld> yes
<wallyworld> i can search
<wallyworld> davecheney: let's discuss real soon in juju meeting, then we can revert if needed
 * thumper sighs
<thumper> I wish our code wasn't so shit
 * thumper thinks hard about hooking this up
<menn0> thumper, I think I see a race in PR 555 (the downgrade one). Fixing it now. This also makes the code clearer around the bit that confused you.
<thumper> ok, cool
<wallyworld> natefinch: what happens if you run "ls -l /usr/lib/go/src/pkg/archive/zip"   is there a .hg in there?
<wallyworld> menn0: are you going to fix bug 1359435 ?
<mup> Bug #1359435: Next version selection for upgrades is no longer correct <juju-core:Triaged> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1359435>
<menn0> wallyworld, I just noticed that today incidentally while reviewing a PR from thumper
<menn0> wallyworld, I had no intention of looking at it today
<wallyworld> ok, ta
<menn0> wallyworld, I'm unlikely to get to it this week and I'm off next week.
<menn0> happy to look at it when I get back though
<menn0> (if it's not fixed by then)
<wallyworld> np, we'll get it sorted, i want to be able to release 1.20.6 early next week
<thumper> coffee time...
<thumper> wallyworld: how will we know if ci is happy with the windows build?
<thumper> wallyworld: and when can we land the pending stuff?
<wallyworld> thumper: i *think* it detects that the bug is fix committed
<wallyworld> hence i marked it as such
<wallyworld> but i'm not sure
<wallyworld> regardless, there's still one critical bug open
<wallyworld> one other
<thumper> what is the other?
<wallyworld> https://bugs.launchpad.net/bugs/1359170
<mup> Bug #1359170: arguments no longer passed to plugins if you don't have an environment set <regression> <Juju Charm Tools:Invalid> <juju-core:Triaged by jameinel> <juju-core 1.20:Triaged by jameinel> <https://launchpad.net/bugs/1359170>
 * thumper digs
<thumper> I had a look at john's fix for that
<thumper> but disagreed with it
 * thumper looks again
<thumper> wallyworld: this isn't a critical regression though
<thumper> wallyworld: this has been this way for quite some time
<thumper> at least since someone change the env command to be the wrapper as it is now
<wallyworld> thumper: i didn't mark it as a regression. i guess we can unmark it
 * thumper removes the tag
 * thumper marks it high, but not regression
<thumper> and works on it
<thumper> wallyworld: I'm getting the godeps errors now I've pull trunk too
<wallyworld> \o/
<davecheney> wallyworld: thumper see my reply to the mail thread
<davecheney> you already know my solution ...
<thumper> seems reasonable to me :-)
<wallyworld> me too :-)
<axw> wallyworld: thanks for the review. will make those doc updates and test again after rebasing
<axw> lots of changes from the windows PR...
<wallyworld> sure, np
<wallyworld> axw: i reverted it
<axw> wallyworld: oh?
<wallyworld> one of the windows ones
<axw> the userdata one?
<wallyworld> yep
<wallyworld> it caused a regression
<axw> bugger :(
<axw> gabriel will be sad
<wallyworld> my comment is in the revert
<axw> I'll take a look, thanks
<wallyworld> axw: well, he MUST learn to test his stuff
<wallyworld> i mean, you develop for windows, you test your stuff on windows before landuing surely?
<davecheney> minutes for handout in 10 mins are https://docs.google.com/a/canonical.com/document/d/1eeHzbtyt_4dlKQMof-vRfplMWMrClBx32k6BFI-77MI/edit
<davecheney> please add items to discuss
<axw> wallyworld: I'm asking ben howard for access to Azure China for testing - is there anyone else who should have access?
<wallyworld> axw: we're all in the juju core meeting :-)
<axw> oops
 * davecheney insert paddling
<davecheney> https://github.com/juju/juju/pull/567
<axw> wallyworld: Internet is scheduled for connection today, btw
<axw> could be up to 48h before it's active tho
<wallyworld> axw: let's hope they do it on time  :-)
<perrito666> does anyone know if its possible to ssh from a state server to an agent? (meaning, do we have a private key in place that is allowed on the agents?)
<menn0> axw or wallyworld : any chance you can have a look at this? https://github.com/juju/juju/pull/566
<wallyworld> sure
<wallyworld> perrito666: yes, i believe so, that's how juju run works, right thumper ?
 * thumper looks up
<thumper> wat?
<menn0> perrito666, yep, there's an identity file that you can use
<thumper> right
<thumper> yes
<thumper> I think it is in /var/lib/juju/identity
<thumper> use with "ssh -i ..."
<thumper> anyone? https://github.com/juju/juju/pull/568
<menn0> wallyworld, thanks
<wallyworld> np
<wallyworld> looking
<thumper> wallyworld: inlining now...
<wallyworld> ta
<davecheney> wallyworld: can you kill http://juju-ci.vapour.ws:8080/job/github-merge-juju/364/console
<davecheney> it's not going to pass
<wallyworld> sure
<davecheney> ta
<menn0> look at that merge queue... it's a thing of beauty :)
<thumper> where is the merge queue?
<thumper> menn0: ?
<menn0> thumper, I mean all the jobs queued up here: http://juju-ci.vapour.ws:8080/job/github-merge-juju/
<menn0> it was longer before
<menn0> wallyworld, the 1.20 backport of what you just reviewed for me: https://github.com/juju/juju/pull/569
<menn0> can you pls have a look?
<wallyworld> \o/
<wallyworld> yep
<wallyworld> menn0: 1.20 is a lot different isn't it
<menn0> wallyworld, yeah the upgrade code has changed a lot so the backports are non-trivial
<menn0> wallyworld, I don't see much more upgrade stuff being backported to 1.20 now though
<menn0> this PR might be the last one (wishful thinking anyway)
<wallyworld> menn0: my only concern is that maybe there is a test or two missing that may be needed in the backport?
<menn0> wallyworld, the other tests that were in trunk don't make sense in 1.20
<menn0> b/c upgradeWorkerContext doesn't exist in 1.20
<wallyworld> rightio
<wallyworld> done, thanks for fixing \o/
<menn0> the higher level test that I did include does cover the change though
<menn0> wallyworld, well I made the problem too so don't thank me too much :)
<wallyworld> ok, thanks for nothing then :-)
<menn0> menn0, that's more like it :)
<menn0> wallyworld, even.... tiiiirrred
<wallyworld> lol
<wallyworld> menn0: the first sign of insanity is talking to yourself
<wallyworld> wallyworld tells wallyworld that all the time
<menn0> LOL
<davecheney> menn0: i thought the definition of insanity was trying to submit a pull request over and over again and expecting differnt results
<menn0> davecheney, if that' the case then we're all a little bit insane
 * thumper is done for the day (until meetings tonight)
 * thumper afk until 10pm local
<davecheney> wallyworld: thumper-afk axw https://github.com/juju/juju/pull/571
<davecheney> ^ this PR tries to peal off the most dangerous parts of 556
<davecheney> i'd like to submit tihs today and see if CI burps overnight
<davecheney> then push 556 tomorrow
<axw> davecheney: commented
<davecheney> axw: thanks, good suggestion
<davecheney> give me a few minutes
<axw> sure, ping me and I'll take another look
<davecheney> axw: PTAL
<axw> davecheney: LGTM
<davecheney> axw: thanks, lets see if this asplodes ci overnight
<dimitern> morning all
<wallyworld> axw: can you take a peek at this one for me? https://github.com/juju/juju/pull/574
<axw> sure
<wallyworld> thanks axw
<axw> wallyworld: hmm, should we be throwing away non-public addresses if they have the same IP address component?
<axw> wallyworld: I think we may only want to throw away if it's public...
<axw> otherwise that changes the internal address selection
<wallyworld> hmmmm
<wallyworld> when could there be two addresses, same IP, but one marked public, one marked private?
<axw> wallyworld: if the floating IP returns the same as a private address range?
<axw> un moment
<axw> wallyworld: https://bugs.launchpad.net/juju-core/+bug/1348287/comments/4
<wallyworld> it won't do that - floating ip addresses are public
<mup> Bug #1348287: Juju status returns private IP in 'public-ip' field. <cloud-installer> <landscape> <openstack-provider> <juju-core:In Progress by wallyworld> <juju-core 1.20:In Progress by wallyworld> <https://launchpad.net/bugs/1348287>
<wallyworld> the problem was that their public and private ip address ranges were both 10.x.x.x
<wallyworld> so juju couldn't guess which one was public
<wallyworld> but there will never be 2 addresses with the same IP, that doesn't make sense
<axw> well the auto-detection logic will classify it as private
<axw> because it's in the 10. range
<axw> but...
<axw> I think the floating IPs will always come from a separate pool
<wallyworld> yes, which is why we mark it as public
<axw> so that's ok
<wallyworld> ie the provider knows that a 10.a.b.c address is public if it is the floating ip address which was assigned
<axw> wallyworld: the point I'm trying to make is that we shouldn't disqualify that address from use in internal communications
<wallyworld> sure, but juju will guess that it is private when it is not
<wallyworld> i do think that if it is known to be a floating ip address, it should be marked as public and nothing else
<axw> public/private are relative. they may be the same thing depending on where you are. anyway, it doesn't matter because they'll be separate address ranges still
<wallyworld> what worries me is that would could have in juju status both public and private be the same address
<wallyworld> because the floating one was guessed as private and so was picked
<wallyworld> and hences masks the true private address
<wallyworld> axw: do you agree with my statement above?
<axw> wallyworld: sounds sane
<wallyworld> righto
<wallyworld> thanks
<wallyworld> axw: i killed your job because it was going to fail
<wallyworld> trusty vs precise test failure
<axw> wallyworld: yeah, I noticed, but doesn't killing it orphan the ec2 instance?
<wallyworld> hmmm, yeah maybe
<wallyworld> i'll ping curtis later to clean up
<wallyworld> gotta run to soccer, back later for meetings \o/
<axw> wallyworld: actually looks like it may have run the terminate script
<axw> adios
<voidspace> wallyworld: o/
<voidspace> axw: morning
<axw> voidspace: howdy
<voidspace> any news on where the october sprint will be?
<voidspace> I'm in India until October 5th
<voidspace> sprint starts on the 6th...
<axw> voidspace: I have not had an official answer
<davecheney> voidspace: i heard brusssles
<voidspace> axw: I haven't heard anything either
<davecheney> but that was not confirmed
<voidspace> davecheney: that would be cool
<voidspace> davecheney: I mean, literally cool
<voidspace> Brussells in October
<voidspace> hmm, neither of us can spell Brussels it seems
<voidspace> I'm still not convinced I've got it right
<axw> Bruxelles
<axw> :p
<davecheney> yet we can still communicate our intentions perfectly
<voidspace> :-)
<voidspace> if only computers worked like that...
<davecheney> \o/ evolution
<free> hello, could anyone have a look at #1325946 for Landscape? it should be a no-brainer backport to 1.20.x
<mup> Bug #1325946: Can't start named MAAS instances if environment is named kvm or lxc <add-machine> <kvm> <landscape> <lxc> <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1325946>
<voidspace> free: I will have some spare cycles shortly and will see if I can get to it
<voidspace> free: unless someone else beats me to it...
<free> voidspace: thanks!
<free> voidspace: is there already a target date for the next 1.20.x release?
<voidspace> free: that I don't know, it would depend on sinzui
<voidspace> free: but we seem to do them fairly regularly, he's pretty efficient
<free> voidspace: alright, would be great to have this little backport in the next one
<voidspace> free: understood
<voidspace> free: I can see the motivation from the bug report :-)
<free> voidspace: he :)
<voidspace> free: for the record, we're actively working on "multi-tenant state servers"
<voidspace> free: which means that api calls won't automatically be scoped against a specific environment
<free> voidspace: gotcha
<free> voidspace: we're fine passing the UUID (as opposed to the name)
<voidspace> free: yep, understood - and we need to be aware of that constraint
<TheMue> morning
<voidspace> TheMue: morning
<jam> TheMue: so the issue is that if you back out your fix, the test doesn't fail, does it?
<jam> because we are essentially testing that set.Help() == set.Help()
<jam> and *not* testing that the contents of setDoc are the contents of set.Help()
<jam> So for checking "is set registered" it works fine, but for testing whether the actual content is sane, it doesn't quite cover it.
<voidspace> jam: morning o/
<voidspace> coffee
<voidspace> brb
<jam> morning voidspace
<TheMue> jam: hmm, than I took to test of the deploy help wrong
<jam> TheMue: so there is 2 things that are good to test, (a) that "juju set" exists as a command, and (b) that the content of "juju set --help" contains what we want it to.
<jam> Some people like to have tests like: c.Assert(helpText(), gc.Equals, `The content of the help text`) others feel like you shouldn't repeat yourself, as it means you have to update too many tests when something changes.
<jam> TheMue: However, the test you added *wouldn't* have caught that changing setDoc doesn't change the output of "juju set --help"
<jam> because it is effectively testing that
<jam> c.Assert(set.helpText(), gc.Equals, registeredHelpText("set"))
<jam> The "deploy" help has the same problem
<TheMue> jam: IC
<jam> as does sync-tools help
<TheMue> jam: you said you added a good test. could you point me to it?
<jam> TheMue: I can understand that people feel "the help text is com
<jam> combined from 3 different places, and it is clumsy to just report it
<jam> "
<jam> TheMue: I believe I added the sync-tools one, but it suffers from what I just observed.
<jam> I had written that test to check for (a)
<jam> but I realize now that (b) would be useful to have
<jam> and we don't
<jam> TheMue: actually, when I wrote the test, I think I actually did: c.Assert(registeredHelpText("sync-tools"), gc.Equals, syncToolsDocString)
<jam> TheMue: but now that we have stuff like EnvironCommandWrapper
<jam> the *actual* help text is a bit obscured from just the raw string we have somewhere.
<jam> So I'm wondering if we would benefit from splitting the test we have into actual concrete comparisons
<TheMue> jam: OK, so a good test would be, for each command, to retrieve the help by executing the command and compare it to a variable where the help text is stored (like setDec)
<jam> TheMue: or whether people would complain too much about having to update a test when they tweak the help content. (I happen to like the safety-net of tests)
<jam> TheMue: actually, my preferred is to compare it to a local string
<jam> TheMue: so in the table, "out: " would contain the actual help text string
<TheMue> jam: *iiirks*
<voidspace> jam: TheMue: for similar tests in the past I've just done a "contains test"
<TheMue> jam: but then we would have to do it for args and purpose too, just to be consistent
<voidspace> jam: TheMue: i.e. not duplicating the full text in the test, but asserting that it contains "some relevant part that we require it to contain"
 * TheMue is reminded of UI tests in unit tests
<voidspace> which is less clumsy to maintain and still tests (somewhat) that the output is being generated correctly
<jam> TheMue: voidspace: so some of it is "the output of juju set --help is something that I'd actually like to read"
<jam> and having it all in one place rather than just "contains" helps with that.
<TheMue> voidspace: that wouldnât work if we for example add some content
<voidspace> TheMue: well, it wouldn't *fail*
<jam> TheMue: voidspace: I don't think people today (core developers for example) are actually running "juju help $COMMAND" and reading all of it to actually know if it makes sense.
<voidspace> jam: yes but it's still horrible
<TheMue> voidspace: if you donât change the test it wonât fail
<voidspace> TheMue: but is "adding some text" something you want to test
<voidspace> TheMue: and if it is it's easy to test
<voidspace> TheMue: you can have several contains tests if there are several important parts
<jam> voidspace: it might be good to know that you *didn't* add "and you're all a bunch of a$$hats" to every command :)
<voidspace> jam: and copying and pasting output into a test doesn't actually test readability at all
<TheMue> jam: inside the tests we have access to setDoc and co. so I would like the convention that doc never is set directly, but always via a variable and we compare the output to that variable
<jam> TheMue: I'm reasonably ok with that as a compromise.
<jam> TheMue: I think juju core could benefit from someone actually spending time reading the various help texts and how they reference eachother, etc. and actually make that a nice experience for users.
<jam> that doesn't have to be done via the test suite, though.
<TheMue> jam: so I could add a TestAllHelps and change the code to fit this convention. itâs good for a table driven test. :)
 * voidspace really goes to get coffee
<TheMue> jam: when refactoring the code this way I would spend some time in reviewing the help texts
<TheMue> jam: the quality canât be tested, but at least itâs a good opportunity to do so
<jam> TheMue: my idea in having the full content there (after all generation steps) is that when auditing a change, you can see what the actual final outputs are. Though I realize that then when "-e" gets slightly different wording, it ends up changing 50 different commands' help content.
<jam> though that is, honestly, just a search and replace, right?
<TheMue> jam: for later changes Iâm with you, itâs no problem. I only have my troubles with the duplication of all output
<TheMue> jam: but hey, itâs not double in the binary, itâs only in the test
<TheMue> jam: so itâs  ok
<jam> TheMue: there is the concept of Dont Repeat Yourself
<jam> and I want to be tasteful here
<jam> and people certainly have different opinions in this.
<jam> I'm just putting forth a thought.
<jam> That maybe nobody actually knows what all the help texts say because we hide it in all of our layers and wrappers, etc.
<TheMue> Yeah, thereâs a lot of indirection.
<jam> TheMue: and we have stuff where "global" help settings don't show up in the default "juju set --help"
<TheMue> From a quality point of view it definitely would be better to ensure a correct output (but not the quality of the output).
<jam> TheMue: "juju help global-options" shows you options that are available but aren't shown in "juju help" or "juju help set"
<TheMue> But having all texts repeated in one file would make it easier to look for a consistent help and wording
<jam> and "juju help set" doesn't *tell* you about "juju help global-options" you have to find it from "juju help topics"
<TheMue> We hide features, not good. :/
<jam> TheMue: I'm pretty sure thumper did it intentionally to avoid cluttering the short help, but I *think* we at least want a one-line reference so it can be discovered.
<TheMue> So a change like this would have to goals: 1. ensure that the shown help is the correct one but even more important 2. that it has a good quality, is consistent and complete
<rogpeppe1> wallyworld: ping
<rogpeppe1> or anyone that has installed Go from a PPA - i'd like to try find out why godeps is failing in that case
<jam> rogpeppe1: mine is just from the archive, I think, and it is failing
<jam> rogpeppe1: https://bugs.launchpad.net/bugs/1359573
<mup> Bug #1359573: API inconsistency: machine tag vs id <api> <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1359573>
<jam> sorry
<jam> ii  golang-go           2:1.2.1-2ubunt amd64          Go programming language compiler
<jam> ii  golang-go           2:1.2.1-2ubunt amd64          Go programming language compiler
<jam> $ godeps -t ./...
<jam> godeps: no version control system found for "/usr/lib/go/src/pkg/go/build"
<jam> godeps: no version control system found for "/usr/lib/go/src/pkg/bufio"
<rogpeppe1> jam: what does this program print for you? http://paste.ubuntu.com/8104597/
<rogpeppe1> jam: and what's the exact output you get from "godeps -t ./..." ?
<jam> rogpeppe1: http://paste.ubuntu.com/8104605/ for the latter
<jam> rogpeppe1: $ ./goroot
<jam> /usr/lib/go
<rogpeppe1> jam: and does godeps actually produce a non-zero exit code?
<jam> rogpeppe1: $ godeps -t ./... 2>/dev/null ; echo $?
<jam> 1
<jam> yeah
<jam> exit 1
<rogpeppe1> jam: i think i know what's going on now, thanks. the fix should hopefully not be hard.
<jam> rogpeppe1: I'm around if you need me to poke at stuff
<rogpeppe1> jam: thanks - i may well ask you to try an updated version.
<Beret> wallyworld, around?
<Beret> wallyworld, https://code.launchpad.net/~lutostag/gomaasapi/fix_nonce_generation/+merge/231638
<Beret> that seems like a hack to me and not a proper fix
<Beret> aren't we just changing the odds that we'll hit that bug again rather than properly preventing it?
<Beret> wallyworld, nevermind
<voidspace> who is OCR today?
<voidspace> and the title is incorrect as neither of those two bugs are critical and open
<voidspace> doesn't look like I'm allowed to change the title though
<TheMue> voidspace: it can be changed via mup
<voidspace> TheMue: ah, ok
<TheMue> voidspace: but have to remember the syntax myself :)
<voidspace> TheMue: do you know the magic incantation?
<voidspace> heh
<TheMue> voidspace: and btw, even without being ocr I just reviewed your PR
<voidspace> TheMue: ah, cool - thanks
<TheMue> voidspace: today Nate and Menno are OCRs
<voidspace> TheMue: thanks
<voidspace> I was just looking for the doc
<voidspace> not sure if I have it starred
<voidspace> I should do
<TheMue> yeah, simplifies the quick access
<voidspace> it's slow to calculate
<voidspace> I have it starred now
<voidspace> TheMue: the one I really need reviewing is this one: https://github.com/juju/juju/pull/561
<voidspace> TheMue: the other one was reviewed to death already ;-)
<TheMue> voidspace: *lol*
<TheMue> voidspace: OK, will take a look
<voidspace> TheMue: I don't understand your comment "why two asserts?"
<voidspace> TheMue: I don't see two
<voidspace> TheMue: maybe I'm being dense
<voidspace> TheMue: and I have a philosophical objection to testing that we don't do things
<voidspace> even where it is something we used to do
<voidspace> because such tests build up and don't really test anything useful anymore
<TheMue> voidspace: provider/openstack/live_test.go lines 183 and 184
<voidspace> TheMue: ah, 184 isn't shown in the snippet
<voidspace> I'll look
<voidspace> TheMue: the answer maybe, "you'll have to ask the person who originally wrote the test" - but I'll look :-D
<TheMue> voidspace: yeah, I wonât test it too, but in this case you do a change. something that has been open before is now closed. the test would document it.
<voidspace> TheMue: the test would document something that doesn't happen
<voidspace> TheMue: there are an infinity of things that don't happen we could document
<TheMue> voidspace: I know it hasnât been done by you, I only seen it
<voidspace> it just becomes a historical note
<voidspace> TheMue: sure I'll look
<voidspace> TheMue: the test makes sense at the time you write it, but not after that
<TheMue> voidspace: yes, a documentation of the change, not of what doesnât happen
<voidspace> TheMue: as I said, I object...
<voidspace> I don't think our tests should be a historical archive
<voidspace> they should document what we do, not what we used to do
<voidspace> TheMue: oh, it does look like that assert is just duplicated
<voidspace> TheMue: duplicate assert removed, thanks
<voidspace> TheMue: the question I would ask is "if we had never opened the state port would we have a test that it is not opened"?
<voidspace> TheMue: i.e. is the test useful in it's own right?
<TheMue> voidspace: I would like a test that only those ports we want are open and no others
<voidspace> TheMue: pretty sure we have that actually
<voidspace> TheMue: some of those changes are me *removing* StatePort from the list of expected ports
<TheMue> voidspace: yep, Iâve seen
<voidspace> TheMue: I'm pretty sure if you run the modified tests against unmodified trunk you'll see failures because StatePort is open when the tests don't expect it to be
<dimitern> jam, meeting?
<voidspace> TheMue: I do agree that that is a useful thing to test
<voidspace> TheMue: I'm pretty sure it's covered though
<TheMue> voidspace: Iâm not talking directly about your changes, more about a general way. e.g. having one test doing it for each provider the same way
<voidspace> TheMue: yeah, unfortunately that code is provider specific
<voidspace> TheMue: we choose which ports to open inside each provider
<voidspace> which is a bit sucky
<voidspace> so we could still screw it up for a provider and not have a test fail
<TheMue> thatâs what I meant, yes
<jam> dimitern: made it
<voidspace> TheMue: I would think that only a live test could usefully do that
<voidspace> TheMue: or we move the code that chooses ports to open out of the providers, which would be a better fix
<rogpeppe1> jam: could you "go get -u launchpad.net/godeps" and try it again, please - the issue should be fixed now
<jam> rogpeppe1: looks good to me
<rogpeppe1> jam: cool
<jam> rogpeppe1: at least, no errors
<TheMue> voidspace: a live test could at least have a problem with the local provider
<TheMue> voidspace: it simply is a difficult topic. :D
<voidspace> TheMue: yeah :-/
<voidspace> TheMue: are we switching existing code to errors.Annotate as we see them?
<voidspace> TheMue: I know I have to Errorf in my branch - one existing and one new
<voidspace> I can switch both, no problem
<voidspace> but just asking as a principle
<TheMue> voidspace: as I understood jam weâre doing
<jam> TheMue: voidspace: sorry we're not available for the standup today, we have a call with Mark S
<jam> dimitern and I
<voidspace> jam: :-(
<voidspace> ok
<voidspace> jam: dimitern: enjoy, don't promise too many things...]
<jam> voidspace: you can probably chat with dimitern after we're done, but then I have the Team Lead call
<voidspace> jam: ok
<voidspace> my next immediate task (after fixing things from TheMue review) is backporting a bugfix to 1.20
<voidspace> then I need a day long task
<jam> voidspace: so the db access stuff is sorted out?
<voidspace> jam: yep, basically
<voidspace> jam: two PRs and two reviews
<jam> voidspace: have you read the IPv6 and charms doc?
<jam> voidspace: https://docs.google.com/a/canonical.com/document/d/1PZ0RipScWNmhpRuyVCZfpSLbH0UGekyApIzZU2_ca_Q/edit
<voidspace> jam: I've read dimiter's networking model - the recent revision
<voidspace> jam: not sure about the charms one
<voidspace> jam: will make sure I digest it
<voidspace> thanks, starring
<voidspace> charms code I am not very familiar with - so will be fun to work on (maybe "fun")
<jam> voidspace: so I think we may want to start with "container addressibility" https://docs.google.com/a/canonical.com/document/d/1XZN2Wnqlag9je73mGqk-Qs9tx1gvOH0-wxMwrlDrQU4/edit#heading=h.h1grzzgqa6st (sorry for the giant doc)
<voidspace> ah, the vegas doc
<jam> the shorter (older) doc was https://docs.google.com/a/canonical.com/document/d/1Gu422BMAJDohIXqm6Vq4WTrtBV8hoFTTdXvXDQCs0Gs/edit, it isn't very complete
<voidspace> jam: container addressability is what TheMue was working on, right
<voidspace> experimenting with
<jam> dammit, why is "clear the screen" in Pidgin the same ^L as "go to the address bar" in Firefox
<jam> https://docs.google.com/a/canonical.com/document/d/1UzJosV7M3hjRaro3ot7iPXFF9jGe2Rym4lJkeO90-Uo/edit#heading=h.a92u8jdqcrto is another one to look at
<voidspace> "Networking for containers" is terse
<TheMue> voidspace: stopped it to do other tasks. would like to see you setting up a test environment too, maybe itâs simply an error by me
<voidspace> it doesn't explain why we want containers - we have the future use case of "more dense deployments"
<voidspace> TheMue: yep
<voidspace> TheMue: I followed your progress
<jam> voidspace: TheMue was working IPv6 and containers which isn't quite the same
<voidspace> TheMue: I started to get nested containers setup
<voidspace> TheMue: but didn't get much further than that
<voidspace> jam: right - ipv6 addressability
<jam> this is more about asking EC2 to give us another address, and then getting that address routed into one of our containers
<voidspace> jam: do we have a more immediate use case for container addressability
<jam> so that LXC-1 on machine-1 is visible to LXC-N on machine-2
<voidspace> right
<TheMue> jam: oh, that sounds nice too
<jam> voidspace: right now the only environment that can deploy into LXC and get the service out is MaaS because it assigns addresses from DHCP
<voidspace> jam: but why do we need that?
<voidspace> jam: I mean, I know why it would be nice. But why is it a priority?
<jam> voidspace: "juju deploy mysql --to lxc:0"
<jam> voidspace: density
<voidspace> that's why it would be nice
<voidspace> jam: ok, so that's shifted up the priority stack
<voidspace> cool, cool
<jam> voidspace: its part of the general "networking model"
<voidspace> after a while my vm saturates both cores on this laptop and the machine becomes unseable until I restart parallels
<voidspace> "a while" is a couple of days
<jam> voidspace: ouch
<voidspace> but still
<jam> voidspace: also, we need to change how we do container addresses in MaaS starting with MaaS 1.6
<jam> as they want us to do the same thing ther
<jam> ask MaaS for another address, and then we can use it for the container
<voidspace> jam: right
<jam> rather than going via DHCP
<voidspace> so if we need to do it right for MaaS we might as well solve the general problem
<dimitern> voidspace, TheMue, guys, you're still standuping ? :)
<dimitern> doesn't look so
<dimitern> ok, I need a short ciggie break
<mattyw> thumper, thanks for pointing out this problems in metrics. I'll put a card on the board to remind me to do them
<thumper> mattyw: np
<perrito666> ill ask this again in case there is someone new
<perrito666> does anyone know if its possible to ssh from a state server to an agent? (meaning, do we have a private key in place that is allowed on the agents?)
<wallyworld> katco: just finishing a meeting
<katco> wallyworld: ok
<gsamfira> perrito666: the code suggests that it is possible. I can bootstrap an environment in 5 minutes on MaaS and let you know if you'd like.
<perrito666> gsamfira: I tried under a certain cirsumstance and failed, Ill do it myself and take a look
<gsamfira> if not, you can simply enable ssh agent forwarding on your machine
<gsamfira> and that will allow you to use your local key even if you hop through another machine
<perrito666> gsamfira: I need to do something that does this automatically
<gsamfira> gotcha
<katco> good morning, team. how is everyone?
<TheMue> katco: fine, and you?
<katco> TheMue: getting over a cold, but looking forward to landing some code :) it's the perfect remedy :)
<TheMue> katco: hehe, is it available on recipe?
<katco> TheMue: what do you mean by recipe?
<perrito666> bbl
<TheMue> katco: the remedy (itâs listed here in my dict also as medicine)
<katco> TheMue: oh, sorry... thought maybe that was a technical reference :) haha, it's in the Programmer's book of home remedies ;)
<TheMue> katco: a good book ;)
<katco> TheMue: it only has 3 pages: one on caffeine, one on the asymptotic time it takes to get better, and then the third page just says, "Code."
<katco> TheMue: apparently i applied a O(1) remedy ;)
<TheMue> katco: so will follow chapter 3 now, currently adding a new version of an API call
<katco> TheMue: :)
<katco> wallyworld: actually, are you still around?
<wallyworld> yup
<katco> wallyworld: just looking at how i'm going to set this DisablePackageCommands... and i'm a little fuzzy on the precedent. what happens if the config settings conflict with this?
<wallyworld> let me look into it a bit
<katco> i'm looking at state/apiserver/client/client.go
<katco> wallyworld: seems like maybe it should be an error if the two settings conflict on a local machine
<wallyworld> katco: there's nowhere in the Go codebase that sets that bool; it must be used via a python client. but if the user has specified it, that's what they want so it should override any config settings
<katco> wallyworld: but if they also set the new config values, shouldn't we alert them to the fact that their preferences are conflicting?
<wallyworld> hmmm, i just trying to remember the use case
<wallyworld> it's used in manual prvisioning to set up a host5
<wallyworld> that's within juju, not sure about external usage
<katco> wallyworld: host5?
<wallyworld> bah, typo
<wallyworld> i think a warning printed on the console is sufficient
<katco> wallyworld: and then prefer the disablepackagecommands?
<wallyworld> yep. but the check doesn't happen client side, unless the enc config is fetched
<wallyworld> env
<katco> wallyworld: i also put in a comment that that setting was deprecated as of 1.20.6... is that safe?
<wallyworld> a code comment?
<katco> wallyworld: yes
<wallyworld> that's fine
<wallyworld> it will remain deprecated until juju 2.0
<katco> wallyworld: gotcha
<wallyworld> actually, i think you need to add support for the new bools in the command
<katco> juju cmd?
<wallyworld> the ProvisioningScript api call
<katco> hmm ok
<wallyworld> sorry, wrong terminology
<wallyworld> because if we say DisablePackageCommands is deprecated, we need to prvide the new alternative
<katco> right
<wallyworld> i think you can just add the extra bools to the end of the struct
<katco> gosh, also... if the defaults for these 2 new settings are true, basically anytime someone uses the DisablePackageCommands setting, they're going to get these warnings.
<wallyworld> but hmmm
<katco> they wouldn't be able to set these 2 new settings in a < 1.20.6 client
<wallyworld> that's not such a good idea, because we won't know which variation they'e using
<katco> or w/e this is going in
<wallyworld> let's leave it for now
<wallyworld> we may just need to wait till the old bool is removed and replace it with the 2 new ones
<katco> wallyworld: sorry, so what now? "leave it"?
<wallyworld> support the original DisablePackageCommands bool but not worry about the new ones yet
<katco> wallyworld: ah, so maybe use the original to drive the new ones? that way the finer-grained control is all in place, but the clients just can't use it that way yet?
<wallyworld> yup
<wallyworld> as the DisablePackageCommands is pulled off the wire, map it to the 2 new ones
<katco> wallyworld: ok. i _think_ that should still provide performance benefits when DisablePackageCommands = true.
<wallyworld> yes it will
<wallyworld> cause neither upgrade nor update wil be run
<katco> wallyworld: yeah, and then we're all set up to provide the finer-grained control
<wallyworld> yep
<katco> wallyworld: ok thanks :)
<wallyworld> np, i think it's a decent solution. we can always add the finer grained stuff to the command if really needed
<wallyworld> s/command/api
<katco> wallyworld: yeah, people might not even need it
<wallyworld> exactly
<sinzui> hi jam, wallyworld, natefinch, There is a request to backport a change to 1.20. The proposed change is certainly backportable, but I need your agreement that it will solve the problem, see bug 1325946
<mup> Bug #1325946: Can't start named MAAS instances if environment is named kvm or lxc <add-machine> <kvm> <landscape> <lxc> <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1325946>
<katco> wallyworld: kind of begs the question, why wasn't this doing what we wanted earlier?
<wallyworld> katco: it's only for a very specific api call
<katco> wallyworld: ah that's right, it's not in the environ config
<wallyworld> sinzui: it may do, that bug looks kinda ugly
<voidspace> sinzui: wallyworld: the backport is a workaround for the bug, right?
<wallyworld> that's what i was thinking too
<voidspace> sinzui: wallyworld: and does solve the problem (as far as I can tell anyway)
<wallyworld> i think the bug should stay open
<wallyworld> but we can do the backport
<sinzui> wallyworld, thank you. voidspace , thank you
<wallyworld> sinzui: i am hoping we can do a 1.20.6 release early next week, say tuesday?
<voidspace> free: ^^^
<free> voidspace: awesome
<wallyworld> i have 2 or 3 small fixes to do for bugs, plus a backport or 2
<free> voidspace: would the fix we spoke about be in?
<voidspace> free: yep
<free> voidspace: great
<wallyworld> which one?
<voidspace> wallyworld: that's the one we just discussed :-)
<sinzui> wallyworld, possibly...I expect to be hiding from the beach. If I have network I can guarantee it. The QA team can follow the template/script, but I think they might have some anxiety
<wallyworld> ah
<voidspace> bug 1325946
<mup> Bug #1325946: Can't start named MAAS instances if environment is named kvm or lxc <add-machine> <kvm> <landscape> <lxc> <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1325946>
<wallyworld> sinzui: when you back?
<sinzui> wallyworld, 1.20.5 is in utopic today
<sinzui> wallyworld, the following week
<free> voidspace: if it's not too big of a deal the last thing we'd need for Landscape is #1359714. But that we could workaround because we can sniff the machine ID by looking at the current directory name in the hook
<mup> Bug #1359714: Add JUJU_MACHINE_ID to the hooks environment <charms> <landscape> <juju-core:Triaged> <https://launchpad.net/bugs/1359714>
<wallyworld> sinzui: i think they should do the release, can't rely on one person, if we do that, something is wrong with the process
<sinzui> wallyworld, I really am hiding from the beach. I intend to hack on something
<wallyworld> ok :-)
<wallyworld> sinzui: the reason for wanting to release is there's some fixes in there many people are interested in
<voidspace> free: I'm not sure how suitable for backporting it would be once we fix it (new feature and all that)
<sinzui> free, do you need that behaviour in 1.20.x?
<voidspace> free: but we can look at it
<free> sinzui: we can workaround it, but would be nice
<sinzui> thank you free
<katco> wallyworld: is this the common boilerplate you were referring to? juju/juju/environs/boilerplate_config.go
<sinzui> I am going to lower all the criticals affecting 1.21-alpha1 that have a  1.20.x to High because we do not need to take further action with them
<wallyworld> katco: yes, but i see that it is not really suitable to add the config stuff to - it may need to be done individually for all providers
 * sinzui wants the bug listing to clearly show what we want to fix now
<katco> wallyworld: i thought it most applicable to point out in lxc and manual. it is still available in the others, but maybe not as important to call out. thoughts?
<wallyworld> sinzui: there are no open criticals for 1.21 though, right?
<wallyworld> katco: sounds reasonable, but i think people were asking about it for ec2 etc as well
<katco> wallyworld: ah ok. in it goes!
<sinzui> wallyworld, the topic says so, but you cannot easily check the bug listing to see if they were fixed while you slept :(
<wallyworld> katco: that's what he said
<katco> wallyworld: LOL
<katco> wallyworld: in a time when the world is crazy, it's good to see some humor is multi-national :)
<wallyworld> :-D
<sinzui> wallyworld, actually, all claim to be fixed. CI lost an ec2 instance 9 hours ago and stalled. I have restarted the building and publication of the last revision added to 1.20
<wallyworld> sinzui: actually, you reminded me - i killed a couple of jenkins landing jobs that were going to fail - that may have left orphaned instances?
<sinzui> wallyworld, ah. Indeed. I see them from time to time I assume any machine 12 hours old is safe to remove
<wallyworld> remove away
<wallyworld> sinzui: so have a think about 1.20.6 release - would be nice for early next week, and you can see how well you can rely on your team :-)
<sinzui> +1
<wallyworld> i'll email when we are ready, but i expect it to be tuesday at latest unless something comes up
<jam> sinzui: so https://github.com/juju/juju/commit/ee7cfef912eea184822cb3a536aff04b56bf14e4 looks like it would promote compatibility, which seems ok to me
<jam> though the error message there means the patch isn't quite complete
<jam> return nil, fmt.Errorf("invalid environment name %q", p.Placement.Scope)
<jam> doesn't quite make sense when the Scope is a UUID
<jam> sinzui: their actual issue is several lines earlier which is:
<jam> containerType, err := instance.ParseContainerType(p.Placement.Scope)
<jam> though why we are passing a plain "scope"
<jam> that isn't properly tagged is beyond me
<jam> ah, ffs, this code looks all sorts of buggy and broken :(
<sinzui> hey everyone. I got successful runs of unit tests in lxc yesterday. I am going to propose some changes the devel and stable makefile that will make unittests more reliable in the many envs we try them in
<jam> sinzui: so the thing they are asking for is a bad fix to their original bug, but we're possibly ok backporting it. The main problem is that I really don't think we should be changing the CLI unless we know the API server is updated.
<jam> so maybe 1.21 is already broken against 1.20 and we just don't know it
<sinzui> jam, thank you!
<mattyw> fwereade, ping?
<zirpu> in the beginning was the pong. and it moved back and forth.
<perrito666> hey natefinch wwitzel3 ericsnow I am here but I am getting the party is over on hangout
<natefinch> perrito666: works for me.  Try restarting your browser
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1w, 1359170 1354027
<jcw4> I've noticed that go 1.3* uncovers a few testing defects for me; I assume because of the forced random iteration of maps
<jcw4> are we running tests using latest go in CI anywhere?
<jcw4> should I just file defects for the tests that seem to fail only on go 1.3?
<natefinch> jcw4: yes file bugs, or just fix them.  We shouldn't ever be relying on map order
<jcw4> natefinch: +1
<hazmat> anyone know the instance distributor code?
<natefinch> hazmat: I don't even know what the instance distributor is
<hazmat> natefinch, its zone spread
<natefinch> hazmat: ahh, ok
<hazmat> but it seems to be operating at a instance level, trying to figure out how it ties back to the service level
<natefinch> hazmat: no idea, that sounds like something that the Australian/New Zealand guys would have written
<hazmat> natefinch, yeah.. i know who write it..
<hazmat> nevermind .. i  figured it out.. it populates a distribution group with the extant instances that have common services
<marcoceppi> Why doesn't this work?
<marcoceppi> JUJU_RELATION_ID=mongos:3 JUJU_REMOTE_UNIT=shard1/0 relation-get
<marcoceppi> I get an error that no relation id is specified
<rick_h__> mongos?
<marcoceppi> I'm running this on mongodb charm
<rick_h__> aside from that no idea how that stuff owrks
<marcoceppi> if i set those as -r  and give it a remote unit int he command line it work
<marcoceppi> sbut setting the environment variable before the command does not
<marcoceppi> is there an additional environment variable in hook that's telling it it's not a realtion?
<marcoceppi> natefinch: also, it looks like there's already a JUJU_HOOK_NAME in the environment, could we just use that instead of argv[1] ?
<natefinch> marcoceppi: yes
<marcoceppi> natefinch: then that has my +1 since it's already there in core
<marcoceppi> natefinch: any idea on my above query?
<marcoceppi> only thing I can think is somehow relation-get checks CONTEXT_ID and knows it's not a relation
<natefinch> marcoceppi: I don't really know that area of the code, so I can't really help.
<marcoceppi> natefinch: who can I bother :)
<jrwren_> marcoceppi: relation-get requires a parameter
<marcoceppi> jrwren_: I'm not even getting that far
<marcoceppi> error: no relation id specified
<marcoceppi> even with key, still same error. I've added it to environment via export and by setting it preceeding the command
<jrwren_> marcoceppi: requires the -r if you are outside a relation hook. Is that what you are trying to fake by setting those env var?
<marcoceppi> jrwren_: yes, how does it know? context_id ?
<marcoceppi> I figured it would check env, then check -r
<marcoceppi> but it's not even looking for the envs
<jrwren_> I don't know. I'd love to find out.
<ericsnow> marcoceppi: see newRelationIdValue in worker/uniter/jujuc/context.go
<natefinch> man our code is hard to navigate
<marcoceppi> haha
<marcoceppi> so, ctx.HookRelation looks promising, I see it's checking for relation context
<marcoceppi> that's probably being pulled in from JUJU_CONTEXT_ID
<marcoceppi> really wish I could fake that
<katco> is lxc-clone no longer defaulted to true?
<katco> in the code, it's specified with schema.Omit, but i'm looking at Ian's comment on 7/31 here (https://bugs.launchpad.net/juju-core/+bug/1350493), and it says Juju 1.20 ships with it set to true
<mup> Bug #1350493: 1.20.x local provider not running apt-get update <charms> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1350493>
<katco> https://github.com/juju/juju/blob/master/environs/config/config.go#L872
<sinzui> smoser, wallyworld, Lauren Mayberry sings "I communicate in simple streams" on  the  Chvrches - Night Sky song
<katco> i thought i understood this, but i'm missing something. in environments.yaml, how do you set a default config value, and then detect if it's using the default or a user-specified value?
<natefinch> katco: not sure
<katco> natefinch: doh :(
<katco> i mean i can think of a way to do it, but i'm not sure if that's the canonical way
<natefinch> katco: likely the only way to know for sure is if we have some explicit code in there for detecting an unset value and using a default instead... like a nil *string or whatever
<katco> natefinch: well, there's schema.Omit, and if i specify that, i can return a value in the setting's wrapper-method if it's not found
<katco> natefinch: ahh... it looks like the magic-combo might be to specify the setting in alwaysOptional as schema.Omit, but then give it a default in allDefaults
<katco> natefinch: doh, no that's not right either =|
<natefinch> If a field is not present in the map and defaults to Omit, the missing field will be ommitted from the coerced map as well.
<natefinch> That looks like English...
<natefinch> but I have no idea what it means :)
<katco> natefinch: so i am no expert, but basically when defining the config variable, you can give it a default value
<katco> natefinch: if you specify schema.Omit, it will not add it to a validated config object's map
<katco> natefinch: which is exactly what i want, but i also want a default value if it's not specified. and apparently that's the tricky bit (or at least the part i haven't figured out). i'm going with a work-around and someone can correct me lol
 * perrito666 bashes a bit his head on the keyboard
<katco> perrito666: may i join you, sir?
<perrito666> katco: certainly, you will have to provide your own keyboard though
<katco> alas, i have already broken mine.
 * perrito666 thinks katco might remember the effect he has on keyboards
<katco> rofl
<katco> yeah you can just replace yours!
<perrito666> katco: well I have bulky thinkpad now so it is even easier and there is not violence involved
<katco> just need to finish this test, rebase, test again and then finall submit this stupid PR
<katco> perrito666: oh did you get a new laptop already?
<perrito666> katco: I swapped my mac with my wifes thinkpad
<katco> perrito666: ah nice. how do you like it?
<perrito666> katco: I always like thinkpads, this is a t420, the only downside of it is to carry it
<perrito666> and I have plenty of ram, which allows me to do a lot
<katco> perrito666: i know what you mean. i went from a... 11"? macbook air to a 15" s57
<katco> perrito666: i thought about a smaller one, but at sprints i want a screen i can work off of
<perrito666> well the asus was a nifty piece of hardware, but the soldered 4G of ram where a pain
<perrito666> I mean, nice screen, nice kb (after replacing), decent battery, decent processor but really, 4G?
<katco> perrito666: yeah bleck
<sinzui> perrito666, do You have a few minutes to review https://github.com/juju/juju/pull/582/files
<perrito666> sinzui: I do
<perrito666> sinzui: you should know that as per the new review mentor plan my reviews lack power unless upgraded with a reviewer combo :p (holds true for all newcommers)
<sinzui> perrito666, thank you for the explanation.I wasn't sure how new is new.
<perrito666> sinzui: las vegas new
<sinzui> perrito666, you started before Las Vegas :)
<perrito666> sinzui: mm, lets put it this way,anyone who's first sprint was vegas or after :p
<perrito666> sinzui: your pr messages dont compile in english
<alexisb> thumper, ping
<menn0> alexisb, I believe thumper has the day off today
<urulama> thumper: you owe us a beer next time ;)
<alexisb> menn0, ug
<alexisb> he accepted the invite and forced urulama to stay up until midnight :)
<menn0> alexisb, ok, then maybe he will be around?
<alexisb> I always forget that you guys are a day ahead
<menn0> alexisb, do you want me to SMS him?
<alexisb> I think he just didnt think about the timing giving it is a reoccurring meeting
<alexisb> menn0, nah
<alexisb> were good
<alexisb> he just owes urulama a beer ;)
<urulama> alexisb: tbh, i'm always up at this time, but, still a belgium beer it is :D
<menn0> urulama, alexisb: I think you should ask for more than one :)
<perrito666> so, how is this mechanism to extort beers from thumper?
<urulama> alexisb: or maybe i can get by for not reviewing his errgo/errors branch :S
<alexisb> perrito666, you have to invite thumper to a meeting he cannot attend and trick him into accepting the invite...
<alexisb> then make sure it is at midnight your time and ensure everyone feels bad for you staying up
<perrito666> alexisb: ah that should be easy, people tend to accept invites without looking
<alexisb> then make the request on IRC
<alexisb> we should write this process in a google doc and send it out to juju-dev ;)
<urulama> :D
<menn0> davecheney, waigani: email standup today?
<waigani> menn0: sounds good
<wallyworld> sinzui: i merged your makefile changes directly
<davecheney> menn0: sure
<davecheney> sinzui: how are the builds looking at the momnet
<davecheney> i see there are no blockers
<davecheney> i'm going to try to land my large api refactor branch today
<davecheney> and I'd like to not land it on top of other breakage
<waigani> davecheney: I don't know how I missed that, I have the conflict locally now thanks
<davecheney> waigani: you are a winner
<davecheney> http://www.teddideppner.com/wp-content/uploads/2013/08/korben-dallas-winner-500x278.jpg
<waigani> whoop whoop
<perrito666> anyone very expert in api calls?
<davecheney> perrito666: proably me or jam
#juju-dev 2014-08-22
<xavierbick> Hello everyone, I saw this bug on launchpad (https://bugs.launchpad.net/juju-core/+bug/1336473) and wanted to submit a fix to github, but I'm not quite sure what the normal procedure is for contributing a bug fix to juju.  Can I just assign myself to the bug and mark it as in progress?
<mup> Bug #1336473: Support new t2 instance types on AWS <ec2-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1336473>
<davecheney> xavierbick: sure
<xavierbick> great, thanks davecheney
<perrito666> davecheney: I need to make the api server able to restart jujud with a given result from a FacadeCall, much like what happens for workers but in this case the call would answer to the client forwarding the result and then restart, yet I am not very sure where to add this
<davecheney> perrito666: i think someone else asked this question recently
<davecheney> menn0-afk: did you ask this recently
<perrito666> davecheney: I send a similar question to the mailing list some time ago
<perrito666> before understanding the actual problem fully
<davecheney> ahh
<perrito666> davecheney: that means you dont have an answerfor me right?
<perrito666> :p
<davecheney> perrito666: i'm not even sure I understand the question
<perrito666> I most likely suck at explaining it
<davecheney> perrito666: you want an api method to restart the apiserver, but not just exit the process ?
<perrito666> Ill try to explain better, bear with me Its late :)
<perrito666> I created a Client API method for Restore, which basically... restores :)
<perrito666> now restore its a rather destructive thing which shuffles things around so, after finishing it I need to restart jujud
<perrito666> so It was suggested by william that I should handle the return value of said call and restart jujud on a given result
<perrito666> now, even if restarting jujud, I need to answer to the client first so it acknowledges that the command finished
<davecheney> perrito666: i dunno
<davecheney> i'd just do exit(0)
<perrito666> davecheney: what I am not sure is where to :) there is a blurry area between state/api/client and state/apiserver/client :p which is not all that clear to me, even though is properly documented
<perrito666> anyway, 31C at 11PM is too much for my brain I am going to sleep under the AC until the head clears off
<perrito666> jam: if you read the abovewhen you arrive and have an answer please drop it here and Ill read it in the morning
<davecheney> 31c, fuk that noise
<perrito666> noise?
<perrito666> 31Â°C <-- temp, fun fact, we are in the middle of winter
<davecheney> bzzt
<wallyworld> davecheney: that status bug has been fixed in trunk, i just backported to 1.20
<wallyworld> you did the fix for trunk a little while back
<davecheney> it's sort of a fix
<davecheney> it's still shit
<davecheney> we should just stop trying to carry on if api.GetStatus screws up
<wallyworld> sure, but at least the nil pointer is fixed and 1.20 is consistent with master
<wallyworld> axw: just otp, there soon
<axw> wallyworld: nps
<wallyworld> axw: ready now
<katco> wallyworld: i need to take care of some things, but i think i addressed all the issues your brought up in https://github.com/juju/juju/pull/584
<wallyworld> katco: awesome, thank you, will look
<katco> wallyworld: no, thank you :)
 * katco away for a bit, maybe for the night.
<axw> wallyworld: just putting my desk back together, will bbs to review
<wallyworld> sure
<jam> perrito666: state/api/client is the actual user/client/not-the-server side of the code, and state/apiserver/client is the stuff running inside the API server. "client" in this case is that it is the set of "client facing" APIs (Status, Deploy, Upgrade, etc), not Agent facing APIs.
<jam> perrito666: essentially everything in state/apiserver should be mirrored as state/api, the former being the one producing the API, and the latter being the one consuming it.
<jam> perrito666: as for restarting gracefully, the worker/Upgrader code does this by raising UpgradeReadyError
<jam> which signals up the worker/runner stack that we want to restart our process.
<jam> However, that is being run in a *worker* not during an API call.
<jam> I don't have an immediate "this is clearly how to do it" but we should probably take this to an on list discussion, so that fwereade et al can get in on the conversation
<wallyworld> axw: manual provisioning question - in manual/provisioner.go, there's a ProvisionMachine() method; know the one i mean?
<axw> wallyworld: yup?
<wallyworld> in that method, we call args.Client.ProvisioningScript
 * axw nods
<wallyworld> and DisablePackageCommands will be false
<katco> wallyworld: i have to go to bed, but i just pushed up more changes. thanks for the review.
<wallyworld> katco: np, still going through it :-)
<axw> wallyworld: orly? /me looks
<wallyworld> katco: thanks for the effort :-)
 * katco hands wallyworld a fine-toothed comb
<wallyworld> lol
<wallyworld> i don't have nits :-P
<katco> wallyworld: yeah well, this has drug out long enough. i wanted to take advantage of you being online ;)
<axw> wallyworld: DisablePackageCommands isn't set, so yes, it'll be false
<wallyworld> axw: so that means we will run upgrade/update on manual machines, no?
<katco> ok, sorry, off to bed. thanks again wallyworld, axw.
<axw> wallyworld: ProvisioningSCript calls cloudcfg.SetAptUpgrade(false)
<wallyworld> katco: night
<axw> katco: night night
<axw> wallyworld: the idea was that manual should only install what it needs, and not muck about with the machine otherwise
<wallyworld> axw: oh, so it does, so it ignores the DisablePackageCommands
<wallyworld> whereas now, we want to default to the current behaviour, but allow it to be overridden
<axw> wallyworld: IIRC, DisablePackageCommands is set on the MachineConfig in the API server code
<axw> wallyworld: probably do that in the API server code then, I think.
<axw> apiserver/client
<wallyworld> axw: it's not set all for the ProvisioningScript call
<wallyworld> it's only set on machineconfig by the lxc container manager
<axw> wallyworld: it is. look at state/apiserver/client/client.go
<axw> there's a Client.ProvisioningScript method that sets up the MachienConfig, then calls manual.ProvisioningSCript
<wallyworld> yes, i meant that nothing sets its value to true
<axw> wallyworld: kapil's external code does :)
<wallyworld> right
<wallyworld> nothing in juju
<wallyworld> so instead in juju we hard code disableaptupgrade to false
<wallyworld> so now in the new branch, that is configurable
<axw> wallyworld: upgrade is *always* false for manual, regardless of what you do. you can *also* disable update&install by setting DisablePackageCommands
<axw> ok
<axw> mmm
<axw> yeah, that sounds fine
<wallyworld> ie it defaults to disabled like now
<wallyworld> but the user can choose to have their box upgraded if they want
<axw> as long as it doesn't override DisablePackageCommands
<wallyworld> that's the hard bit i'm pondering about - how to stay backwards compatible
<wallyworld> since we don't know if DisablePackageCommands was explicitly set or not
<wallyworld> if think if it is false, we use the new config settings
<wallyworld> if true, we disable everything regardless of the settings
<axw> wallyworld: +1
<axw> wallyworld: the client side doesn't need to change at all
<wallyworld> it's not "perfect", bu i think the best we can do
<axw> just state/apiserver/client/client.go
<wallyworld> yep
<axw> manual.ProvisioningScript shouldn't force upgrade=false anymore, that needs to move into apiserver
<axw> which has access to config
<wallyworld> yep
<menn0> wallyworld, thanks for dealing with bug 1359435
<mup> Bug #1359435: Next version selection for upgrades is no longer correct <juju-core:Triaged> <juju-core 1.20:In Progress by wallyworld> <https://launchpad.net/bugs/1359435>
<wallyworld> menn0: no problem, thanks for noticing the issue
<wallyworld> axw: a small one https://github.com/juju/juju/pull/591
<axw> looking
<wallyworld> axw: thanks
<davecheney> wallyworld: could you please kill http://juju-ci.vapour.ws:8080/job/github-merge-juju/390/console
<davecheney> it's not going to pass and there is a large backlog behind it
<wallyworld> davecheney: it will pass, why won't it?
<wallyworld> that uniter failure happens all the time
<wallyworld> passes 2nd time through
<davecheney> wallyworld: ok,
<wallyworld> davecheney: did i mention i hate our tests?
<davecheney> wallyworld: to tell
<wallyworld> of late, most runs require the fallback -p 2 to pass
<wallyworld> i will be cataloging the failures and trying to get people to fix them
<davecheney> wallyworld: +1 to turning off the retry
<davecheney> tests pass or GTFO
<wallyworld> i think so, we'll try and get an itemised list of failures first
<davecheney> you'll have no shortage of canidates then
<davecheney> wallyworld: i'll close with this graph, http://juju-ci.vapour.ws:8080/job/github-merge-juju/buildTimeTrend
<davecheney> things clearly went wrong, around this commit waigani endpoint-on-bootstrap-take3
<davecheney> not saying it was this commit
<wallyworld> davecheney: yeah, early after the cutover, we were building in 18 or so minutes
<wallyworld> we had hoped to get it down to 10 or 11 with the use of lxc or nail up instances
<mattyw> morning folks
<TheMue> morning
<voidspace> TheMue: morning
<voidspace> TheMue: I updated this PR to use errors.Annotate everywhere
<voidspace> TheMue: https://github.com/juju/juju/pull/561
<TheMue> voidspace: great, will take a look
<voidspace> TheMue: I would appreciate a quick look for the sake of sanity when you have a moment
<voidspace> thanks!
<voidspace> TheMue: as I was touching the file I did all of the ones that were appropriate to change
<voidspace> (in some cases there isn't an existing error to annotate of course, so those remain Errorf)
<TheMue> voidspace: Iâm still fighting with changing an existing API part into a new version. itâs a rattail :/
<voidspace> fun
<TheMue> voidspace: the errors package IMHO has useful funcs there too
<voidspace> TheMue: ok
<TheMue> voidspace: youâve got a LGTM
<voidspace> TheMue: awesome, thanks
<voidspace> TheMue: and are you happy(ish) for me to merge the other PR without an additional test?
<voidspace> TheMue: given that we concluded that we *do* test (in provider specific ways) that only the requested ports are open
<TheMue> voidspace: hereâs the errors package http://godoc.org/github.com/juju/errors
<TheMue> voidspace: have to take a deeper look myself
<voidspace> TheMue: thanks
<voidspace> I will peruse
<voidspace> I wonder how good the documentation is
<TheMue> voidspace: and regarding the other PR I can live w/o another test, yes. it only have been more general thoughts about how to ensure that only the wanted ports are open. but that would be a huge change :)
<voidspace> TheMue: and we have to open some ports in a provider specific way - as we have to specify some ports when we create the machine for some providers I believe
<voidspace> as we need some ports immediately
<voidspace> so a *general change* is maybe not even possible
<voidspace> although we don't need the state port immediately just to bootstrap
<TheMue> voidspace: yep, youâre right
<Beret> wallyworld, nice update, thanks
<Beret> wallyworld, we'll do all we can to get you more information when we hit those LXC issues
<wallyworld> Beret: no problem and thank you
<perrito666> morning all
<voidspace> perrito666: morning
<voidspace> dimitern: TheMue: standup?
<TheMue> dimitern, voidspace: standup?
<TheMue> voidspace: oh, just typed the same
<voidspace> hah
<TheMue> ;)
<dimitern> voidspace, TheMue, sorry, brt
<dimitern> voidspace, TheMue, others? networker refactoring branch - https://github.com/juju/juju/pull/593 , I'd appreciate a review when you have some time
<wwitzel3> dimitern: I will take a look, I'm one of the on-call reviewers today :)
<TheMue> dimitern: will look
<dimitern> wwitzel3, TheMue, cheers guys!
<perrito666> natefinch: 1:1?
<perrito666> lol just saw the email
<wwitzel3> man, trying to use our existing providers as a baseline for reviewing a new provider is turning out to be confusing.
<wwitzel3> I feel like all our existing providers implement the same behavior as each other, just in a slightly different way.
<wwitzel3> which provider should be considered the "gold standard"?
<katco> wwitzel3: +1
<katco> wwitzel3: i just did some work on providers, and i can't answer that.
<mgz> wwitzel3: well, ah
<mgz> they're all bad in different ways :)
<katco> wwitzel3: if i had to guess, i'd say ec2 since it's probably been around the longest?
<TheMue> wwitzel3: always the next one, which will make it all better
<katco> TheMue: lol!
<wwitzel3> I can distill these answers to "Yes, good luck"
<wwitzel3> :D
<TheMue> wwitzel3: +1
<katco> wwitzel3: haha... well what are you trying to review? there are probably some concepts that in their pure form remain the same
<wwitzel3> katco: the CloudSigma provider set of PRs
<katco> wwitzel3: link to one?
<wwitzel3> katco: https://github.com/juju/juju/pull/173
<wwitzel3> katco: it is also the fact that it is just 2,000+ new lines. So there is no reference built in to the diff.
<katco> wwitzel3: yeah, i see that! tough review...
<wwitzel3> katco: that's why I'm trying to look to our existing providers as a reference / guide
<katco> wwitzel3: yeah, i think i'd be doing what you are. and probably bouncing between ec2 which is probably the oldest, and maybe local which is one of the newer ones. or azure.
 * katco dusts hands off. glad i could provide absolutely no value for you. ;)
<wwitzel3> sanity checks are always helpful
<perrito666> katco: well if you manage to make a career out of that you can become an argentinian real stateagent
<wwitzel3> hah
<katco> perrito666: i don't have it in me, unfortunately. it would drive me crazy.
<perrito666> katco: there is good money in it and you can code while ignoring your customers
<perrito666> fwereade: are you around?
<fwereade> perrito666, yeah
<katco> perrito666: lol
<fwereade> perrito666, meant to ping yu this morning
<fwereade> perrito666, you were asking last night about restarting the state server from within an api call?
<perrito666> fwereade: heh yeah, our discussion the other day end up including more people and opinions and I am not entirely sure how to implement that :p
<fwereade> perrito666, so, I have been 100% focused on leader-election things for the last day or two, I am probably missing context
<perrito666> fwereade: Well I triedto implement that according to what we spoke, and suddenly I noticed that there was a gray area on state where I am not that familiar
<fwereade> perrito666, are there concerns about the special-error approach?
<fwereade> perrito666, ah, go on
<fwereade> perrito666, I'd expect state to be out of the loop on that sort of question
<perrito666> fwereade: the deal is, I am not entirely sure where is that I should be catching the error to ensure the client gets the response and trigger the restart (I might not be as good explaining myself as I think I am on this particular matter)
<fwereade> perrito666, well -- somewhere after the function returns, and before it gets packaged up and sent back over the API
<fwereade> perrito666, how many choices do you have there?
<fwereade> perrito666, presumably it needs to be a layer that has, or can easily and sanely be given, access to the apiserver worker's tomb
<fwereade> perrito666, so I would guess that the apiserver worker itself would be the right one to actually handle sending on the error to its runner
<fwereade> perrito666, but I can well believe that we need another layer in there somewhere
<fwereade> perrito666, I don't immediately recall what's actually in play there
<perrito666> fwereade: as I get it th call propagates like this: apiclient->rpc layer->(X)->apiserver
<perrito666> and I believe I should be standing in (X)
<fwereade> perrito666, hmm, that sounds wrong
<fwereade> perrito666, the apiserver needs to know that it should stop
<fwereade> perrito666, and I think it has to finish the call, passing a "everything's good, restarting now!" response down via rpc, while arranging its own tidy suicide separately
<perrito666> fwereade: I see, makes sense
<fwereade> perrito666, possible approach: return result *and* error in that situation
<fwereade> perrito666, in apiserver, define the error (type) that indicates "return these results and then terminate the api server with ErrStateRestored(?)"
<fwereade> perrito666, except
<fwereade> perrito666, can you really actually *do* the restore while everything's running?
<fwereade> perrito666, what's the actual sequence of events?
<perrito666> fwereade: yes, writing
<natefinch> perrito666: standup?
<perrito666> natefinch: getting there, I was looking for a webcam
<perrito666> :p
<TheMue> dimitern: great work, but I would like a second to take a look too
<perrito666> fwereade: so this is how it works: [Bootstrap in safe mode][Upload File]<stop mongo><restore mongo><restore conf files for juju> <start mongo><restart jujud?>
<fwereade> perrito666, then I don't quite understand where the API server comes in
<fwereade> perrito666, ah: so immediately after you've bootstrapped, you tell the API server to do all that stuff?
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1354027 1360286
<perrito666> yup
<perrito666> fwereade: so once I have a working api server I ask for it to perform the restore
<fwereade> perrito666, and is the API server still responding to requests while this is going on?
<perrito666> fwereade: mm I see where you are going, I could make API server deny requests, much like what upgrade does
<perrito666> update
<sinzui> natefinch, jam: We have a new regression. Can you find an engineer to fix or revert the problem? https://bugs.launchpad.net/juju-core/+bug/1360286
<mup> Bug #1360286: ppc64el and i386 unittests cannot find tools <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1360286>
<fwereade> perrito666, I'm really rather nervous about having the api server running at all while this is going on
<fwereade> perrito666, in particular I am nervous about what firewaller, networker, etc might do while running against incorrect state
<fwereade> perrito666, the provisioner might take down machines so that's the obvious risk
<perrito666> fwereade: actually the state is down (and API will not connect to it until restarted)
<fwereade> perrito666, how do you ensure it's down? starting jujud will start all sorts of workers, surely, and you won't know to shut them down until you get this api request
<perrito666> fwereade: hold, I think we could be not on the same page here, let me finish this standup and then I can give you my full attention (I would really love tho have a white board between us but that is too much asking)
<fwereade> perrito666, a whiteboard in one's webcam's FOV is a useful thing, I never got round to setting it up though
<fwereade> perrito666, could be
<fwereade> perrito666, natefinch: brb, but will join yu in your standup in 5
<perrito666> fwereade: we are no longer there, just ping me when you are back
 * perrito666 suddenly hits his head on an open window... brb
<fwereade> perrito666, natefinch: will be in the hangout when you're ready
<perrito666> back
<ericsnow> fwereade, perrito666: mind if I join?
<fwereade> ericsnow, ofc not
<perrito666> ericsnow: you are welcome to join
<ericsnow> fwereade, perrito666: you guys are the best! :)
<perrito666> aghh my ears are aching
<natefinch> anyone have thoughts on why a machine after upgrading from 1.18 to 1.20 would says "missing API Hosts"?
<wwitzel3> it's lonely?
<wwitzel3> </nothelpful>
<perrito666> natefinch: agent.conf corrupted?
<natefinch> perrito666: interesting you should say that, he did say he had to update the agent.conf by hand to add apiaddresses
<natefinch> or at least, he thought that was the problem, but adding it didn't help
<natefinch> can someone else give me a hand? I may have to leave soonish.  On #juju in canonical irc, guy's nick is ivoks, he's onsite with a customer right now.
 * fwereade bbs
<perrito666> natefinch: sorry was buying foood
<katco> how do i got about upgrading an environment to my dev version? "juju upgrade-juju --dry-run --version 1.21-alpha1" gives me "ERROR no matching tools available"
<natefinch> --upload-tools should work I think
<natefinch> can anyone else help ivoks in canonical irc #juju?  I have to go
<natefinch> sinzui, alexisb, mgz, wwitzel3 ^
<natefinch> perrito666 was looking at iot but had to go too
<katco> natefinch: ah i see... it doesn't like --upload-tools with --dry-run
<katco> natefinch: that's what was tripping me up
<sinzui> katco, interesting. tip is definitely broken for ppc64el and i386, what arch you testing?
<sinzui> see bug 1360286
<mup> Bug #1360286: ppc64el and i386 unittests cannot find tools <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1360286>
<katco> sinzui: amd64, i think it was just the combination of --dry-run
<katco> sinzui: if i do --upload-tools alone, it gives me a list of available tools, and best version
<sinzui> katco, oh, we have one or two related bugs where --upload-tools doesn't in other combinations
 * sinzui looks
<sinzui> katco, These are the upload-tools bugs https://bugs.launchpad.net/juju-core/+bugs/?field.tag=upload-tools and bug 1307643 looks like the issue
<mup> Bug #1307643: juju upgrade-juju --upload-tools does not honor the arch <upgrade-juju> <upload-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1307643>
<katco> sinzui: interesting, thank you so much. it looks like it might? have worked?
<katco> sinzui: i'm noticing it incrementing the agent-version
<sinzui> katco, status shows the new version?
<katco> sinzui: yeah: 1.21-alpha1.x
<katco> sinzui: where x is incrementing...
<sinzui> katco, upload-tool will always do that
<sinzui> katco, that is to distinguish a version taken from the client instead of the net
<katco> sinzui: so how do i know that it's actually updated? state is started
<sinzui> katco, I trust the log 99% of the time. When I doubt, I ssh to the machine and run "jujud version"
<sinzui> katco, I have never seen it differ from status. Once, did this and found th reason for a failed upgrade was bad symlink. There were to jujuds on the machine. I deleted the symlink and juju correctly upgraded itself a minute later
<katco> sinzui: lol ok, so rule of thumb: trust agent-status
<perrito666> ericsnow: you are a cruel man, such lengthy email on a friday afternoon
<perrito666> ericsnow: why would backup allow upload?
<perrito666> ericsnow: I have not yet looked at the metadata but I do have a request
<perrito666> ericsnow: lemme know when we can have a call
<perrito666> ericsnow: ?
<ericsnow> perrito666: hey just got back
<perrito666> hey, read above :)
<ericsnow> we should hang out
<ericsnow> moonstone?
<perrito666> ericsnow: sure
<katco> rick_h__: you still around?
#juju-dev 2014-08-24
<thumper> morning folks
<thumper> hi waigani
<thumper> wallyworld: morning
<wallyworld> hey
<waigani> thumper: hello
<thumper> wallyworld: https://bugs.launchpad.net/juju-core/+bug/1360286
<mup> Bug #1360286: ppc64el and i386 unittests cannot find tools <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1360286>
<thumper> wallyworld: my guess is axw's bootstrap refactoring
<thumper> wallyworld: do you think we should revert or fix?
<thumper> wallyworld: blocking CI
<wallyworld> could be, i'll look if my network connection satys up long enough :-(
<wallyworld> let me look first
<waigani> thumper: checking the user exists I can only see an unexported func state.checkUserExists
<thumper> waigani: what is this for?
<waigani> thumper: AddEnvironmentUser
<waigani> thumper: checking if user exists before adding it
<thumper> waigani: ok, there are two main possibilities
<thumper> waigani: 1) check at the apiserver level using state.User(name) to load the user
<thumper> 2) look inside the actual state method to add an environment user
<thumper> I'd probably prefer 2.
<waigani> thumper: okay, also how do we tell if they are local?
<thumper> waigani: the tag Provider method is "local"
<waigani> thumper: thanks
<wallyworld> thumper: the failing test can be removed since the logic has changed - i'll do a branch with it skipped to unblock ci and confirm with andrew and then remove it
<thumper> wallyworld: cool
<thumper> wallyworld: http://juju-ci.vapour.ws:8080/job/github-merge-juju/buildTimeTrend
<thumper> wallyworld: FYI looks like Build #374 (21-Aug-2014 07:52:39) axw environ-bootstrap-upload-tools is the one that started big time changes
<wallyworld> thumper: our build times have also been going up because tests are really unreliable and we now fail back to -p 2 most times
<thumper> ah
 * thumper suggests that we only check once
<thumper> fail faster if we are going to fail
<wallyworld> trouble is the 2nd run normally works
<wallyworld> but is much slower than with full parallelism
<wallyworld> we just need to fix the freaking tests
<thumper> wallyworld: agreed
<thumper> the replicaset test seems to be overly fragile
<thumper> and long
<thumper> can we not force that out of our normal test run into something more suitable?
<thumper> and write a robust test?
<wallyworld> you would think so
<wallyworld> also, jujud tests are horrible
<wallyworld> and there's a failing uniter test
#juju-dev 2015-08-17
<davecheney> thumper: http://paste.ubuntu.com/12069959/
<davecheney> this is from this morning's run
<davecheney> summary: juju is now as stable on ppc64 as any other platform
<thumper> hazaah
<menn0> thumper: here's the final allenvwatcher pr: http://reviews.vapour.ws/r/2374/
<menn0> thumper: any chance you can look at the above? ^^
 * menn0 is keen to be done with this 
 * thumper is looking now
<menn0> thumper: thanks
<thumper> menn0: done
<thumper> one (strong) suggestion
<menn0> thumper: ok...
<menn0> thumper: no problems. I had something like that at one point.
<menn0> thumper: to be fair those funcs aren't really for truly public use (hence the comment)
<thumper> well... if they have capital letters, they're public
<dimitern> TheMue, morning
<dimitern> TheMue, I see you've assigned the constraints card to yourself, but you're yet to land your other branch, so I've reassigned it to myself
<TheMue> dimitern: morning, the landing card is just waiting, that CI is unblocked
<dimitern> TheMue, cool
<TheMue> dimitern: so which card can I start with now?
<dimitern> TheMue, hmm.. let me have a look
<dimitern> TheMue, you have a follow-up to do, don't you? :)
<dimitern> also it seems I've assigned the wrong card to myself - I'm actually working on the other constraints card
<TheMue> dimitern: hehe
<TheMue> dimitern: you mean "Test isNetworkingEnvironment() in ..."? we haven't moved it from backlog to iteration backlog, so I thought it's not as important as the spaces cards
<dimitern> TheMue, yeah, we forgot about this one
<dimitern> TheMue, the net-cli stuff is the most important once the previous tasks are done :)
<TheMue> dimitern: ok, so I'll move it
<dimitern> TheMue, cheers, and feel free to estimate it as well
<TheMue> let's move it, move it, let's move it, move it *sing*
<TheMue> dimitern: ok
<voidspace> dimitern: ping
<dimitern> voidspace, pong
<voidspace> dimitern: did you write ListSubnets?
<voidspace> dimitern: unlike all the other API methods it tages a *names.SpaceTag rather than a names.SpaceTag and I'm wondering why
<voidspace> I changed it, as it was marginally inconvenient in one place, and a few things broke
<voidspace> so I probably have to put it back
<dimitern> voidspace, I did - it's because the space is optional for filtering
<dimitern> voidspace, it was supposed to make calling the client-side method more convenient :)
<voidspace> dimitern: and you need to be able to distinguish between not used (nil) and an explicit empty space ("")
<voidspace> dimitern: so, changing API methods to take tags instead of strings means changing a bunch of CLI calls too
<voidspace> dimitern: they need to use the tags and do the validation
<voidspace> so I'm now down to only 23 test failures...
<voidspace> (the CLI methods were already validating of course - but not always storing a tag. And it's *mostly* tests that need to change.)
<dimitern> voidspace, :)
<dimitern> voidspace, sounds good
<dimitern> TheMue, voidspace, dooferlad, guys, I'm suggesting to cancel the 1:1 meetings today - happy to chat about anything, if you want of course
<dimitern> but in the face of the feature freeze I'd rather concentrate on that with priority
<TheMue> dimitern: frome my side nothing special, so sounds good
<dimitern> TheMue, cheers
<voidspace> dimitern: sure
<voidspace> dimitern: thanks
<dooferlad> dimitern: +1
<dimitern> thanks guys!
<voidspace> dimitern: TheMue: dooferlad: ready in a minute or two - coffee!!!
<voidspace> dimitern: TheMue: dooferlad: not sure if my last message got through, my internet went down
<voidspace> sorry
<voidspace> I'm back now but the hangout was empty
<TheMue> voidspace: you've been frozen and then the connection dropped
<TheMue> voidspace: so the question is, what has been the last message? ;)
<voidspace> that was my last message...
<voidspace> "my internet went down"
<voidspace> :-)
<dimitern> voidspace, oh - I think I heard only "own" :)
<TheMue> hehe
<voidspace> dimitern: own?
<voidspace> dimitern: out of "down"?
<dimitern> voidspace, I think so yeah
<dimitern> (well it sounded like "on")
<voidspace> heh
<voidspace> sorry
<voidspace> :w
<voidspace> wrong window...
<dimitern> voidspace, so it seemed we should chat about subnet add/create vs tagging?
<voidspace> dimitern: well, I thought we had talked about it a few times already!
<voidspace> dimitern: that's why I was so surprised
<voidspace> dimitern: previously you seemed to agree that Create was easier and we didn't need to touch tagging for the MVP
<dimitern> voidspace, yeah, I might have misinformed you unintentionally
<voidspace> dimitern: if we *have* to do Add, then so-be-it, but I think it's *more work*
<dimitern> voidspace, we'll implement create, but not include in the MVP
<voidspace> sure, I'm talking about what is the least work we can do which gets something useful
<dimitern> voidspace, also add doesn't need to respect pre-existing tags on a subnet in ec2
<dimitern> for the MVP
<voidspace> so what does it do?
<dimitern> voidspace, it adds it to state, validating it exists on the provider
<voidspace> just assumes the subnet exists and adds it to state
<voidspace> so it does go out to the provider
<voidspace> (the current implementation doesn't...)
<dimitern> voidspace, yes, as we have already a Subnets call we can call - it shouldn't be too bad to add this check
<voidspace> ok
<voidspace> so if we ignore tagging then Add is easier :-)
<dimitern> voidspace, indeed - we'll add tagging (discovery and updates) later if we manage - it's strictly an improvement in UX and observability, not so much in functionality
<voidspace> dimitern: fair enough I guess
<voidspace> dimitern: like I said, based on our previous conversations I was just surprised
<voidspace> dimitern: updated PR http://reviews.vapour.ws/r/2360/
<dimitern> voidspace, cheers, looking
<dimitern> dooferlad, others: http://reviews.vapour.ws/r/2376/ spaces constraints
<dooferlad> dimitern: already reading :-)
<dimitern> dooferlad, ta!
<dimitern> voidspace, didn't we talk about ListSubnets returning something like []params.SubnetInfo?
<dimitern> voidspace, I mean the client-side api method
<voidspace> dimitern: I don't recall a conversation about that, let me look and see what it does return
<voidspace> it's been a while since I wrote that code
<voidspace> dimitern: hmm... it returns a list of params.Subnet
<voidspace> dimitern: there is no params.SubnetInfo is there?
<dimitern> voidspace, sorry, I meant params.Subnet
<voidspace> dimitern: well, it has a result type that holds a list of params.Subnet
<dimitern> voidspace, well, looking at your branch it returns ListSubnetsResults and an error
<voidspace> dimitern: do you mean removing the ListSubnetsResults part from the returned value
<dimitern> voidspace, that's pretty unusual for client-side apis
<dimitern> voidspace, just return the Results part of it and the error
<voidspace> dimitern: ah
<voidspace> dimitern: ListSpaces does the same
<voidspace> dimitern: ah, and the CLI expects just a list of subnets
<voidspace> dimitern: so that needs fixing
<voidspace> probably the same for ListSpaces too
<voidspace> dimitern: thanks
<voidspace> dimitern: just return response.Results or can we get rid of the response type completely and just return a slice of Subnet ?
<voidspace> (I mean can the apiserver return []params.Subnet or do we need a Result type to hold it)
<voidspace> I'm just changing the api List* methods
<dimitern> voidspace, nope, the apiserver-side still needs to use the result type like this
<voidspace> yeah
<dimitern> voidspace, you have a review
<voidspace> dimitern: thanks
<voidspace> dimitern: why do we need to verify that the command members are set correctly on error?
<voidspace> dimitern: given that we report the error to the user and exist
<dimitern> voidspace, I'd appreciate a second look on my review as well - esp. if any of the TODOs reminds you of something I might've missed
<voidspace> dimitern: ok
<dimitern> voidspace, it's not a big deal to leave this in place
<dimitern> voidspace, is it?
<voidspace> dimitern: well, as we're now using SubnetTag I can't construct a SubnetTag for an invalid CIDR
<voidspace> dimitern: so in the case of error we can't check that
<voidspace> dimitern: and if there's no *need* for the others then it's better not to test them
<dimitern> voidspace, ah
<voidspace> we should only test behaviour we actually need
<dimitern> voidspace, sure, then it's fine
<voidspace> thanks
<dimitern> voidspace, unlike APIs, those CLI methods are not usable from outside the package
<voidspace> yep
<voidspace> that was my thinking
<voidspace> dimitern: what's the value in changing providerId to network.Id?
<voidspace> dimitern: there's no validation, so it doesn't really offer any benefit like SubnetTag and SpaceTag do
<voidspace> it can take arbitrary strings
<voidspace> I believe
<voidspace> and it probably means finding and fixing another crap-load of tests :-p
<dimitern> voidspace, fwereade reminded me on more than one occasion that we should use typed strings for foreign ids, like the provider ones
<voidspace> ok
<voidspace> fair enough
<voidspace> I'll bite the bullet
<dimitern> voidspace, yeah, no validation for now, having a type makes adding this easier
<dimitern> if/when needed
<dimitern> voidspace, thanks
<voidspace> so for our imaginery future use case then
<voidspace> ;-)
<dimitern> well :)
<voidspace> dimitern: network.Id is used for too many *different* things for us to ever be able to add meaningful validation
<voidspace> we'd have to create a *specific* type
<dimitern> good design and architecture includes some need for precognition and such
<dimitern> voidspace, it's used for provider-specific ids
<voidspace> dimitern: many *different* provider-specific ids with different specifications
<dimitern> voidspace, at least having them marked as such will simplify changing them to separate types if needed IMO
<dimitern> voidspace, I hope you don't mind too much for a string cast here and there :)
<voidspace> dimitern: I'm not totally convinced about the *semantic* value of using types (that offer no tangible other benefit) to represent "things" that actually *are* strings, and having to case to and from everywhere they're used
<voidspace> dimitern: I'll do it, I'm just not convinced it adds value
<dimitern> voidspace, I'll wait for your and mine branches to land and then create the net-cli-mvp branch
<dimitern> voidspace, right
<voidspace> dimitern: cool
<voidspace> dimitern: I'll take lunch shortly
<voidspace> I'll try and get the branch landable first
<dimitern> voidspace, ta!
<voidspace> dimitern: right, made those changes
<voidspace> dimitern: switching to network.Id touched 6 files, so you might want to take another brief look
<voidspace> :-D
<voidspace> maybe "want" is the wrong word
<dimitern> voidspace, sure, looking
<dooferlad> dimitern: https://github.com/juju/juju/compare/net-cli...dooferlad:net-cli-add-deployment-support-for-spaces?expand=1 seems to be all https://canonical.leankit.com/Boards/View/101652562/116603811 needs.
<dooferlad> just let me know about printing/logging the deprecation warning and I will tidy up that comment and get it in for review
<dimitern> dooferlad, ok, I'll have a look shortly
<dooferlad> dimitern: thanks.
<dimitern> voidspace, get some lunch first
<dimitern> voidspace, you're eating tests otherwise :)
<voidspace> :-)
<voidspace> dimitern: well, I want to meditate and it will be a little bit of time before I can persuade my wife to clear out of the room I need :-D
<dimitern> voidspace, I see ok :)
<dimitern> voidspace, dooferlad, only now I realize ListSpaces API doesn't conform to the usual (results, error) behavior - it returns results and an error, but only the first error when fetching a space's subnets
<voidspace> dimitern: you mean it should collect all errors
<dimitern> it needs to use a Results and Error fields of the results for such errors
<voidspace> dimitern: as the only possible error is connection with state failing (so all should fail or none)
<voidspace> dimitern: I don't think it's a big issue
<dimitern> voidspace, yeah, only return an error if it can't return anything
<voidspace> I mean, we should fix it - but add a card for after mvp maybe
<dimitern> voidspace, well, just because we're catching this now means it lacks bulk call coverage
<voidspace> "bulk call coverage"?
<voidspace> it's a bulk response not a bulk call...
<voidspace> it's a single call with a bulk result
<dimitern> voidspace, yeah - a call to it with e.g. one valid, one invalid, and another valid - checking 2 results and 1 error is returned
<dimitern> voidspace, the apiserver method I mean
<voidspace> right
<dimitern> voidspace, it's trivial to fix and we should do it for the mvp by all means, as changing the api later is worse
<voidspace> dimitern: ok
<dimitern> voidspace, I'd add a card for it, but please add a TODO about it in your PR - apiserver/spaces
<voidspace> dimitern: TODO added
<voidspace> well, pushing now
<dimitern> voidspace, thanks!
<voidspace> done
<voidspace> right, room free
<voidspace> off to meditate
<dimitern> dooferlad, hey
<dimitern> dooferlad, the help docs need to mention the spaces constraints but not --networks btw
<dimitern> dooferlad, but there are also - tests we do ignore --networks and dropping those that don't (or at least adding TODOs/skipping them)
<dimitern> dooferlad, hmm sorry, I might have described it badly in the card
<dimitern> dooferlad, the idea is not only to give a warning for it, but also not to pass it through the layers as well, so it can't interfere with the other work around spaces for deployments
<dimitern> dooferlad, does that make sense?
<dimitern> does anyone else get build errors for gwacl on master after updating and running godeps ?
<dimitern> axw, ping
<dooferlad> dimitern: yea, that isn't what I got from the card. I read it as we would still accept --networks, but we should mark it as deprecated to discourage its use before removing it in a later version.
<dimitern> dooferlad, we would've done that if currently it was at least remotely useful
<dimitern> dooferlad, and it will soon go away to be replaced by deploy time bindings of the right kind, with spaces (once we figure out how'd they look like)
<dimitern> anyone up for a trivial review? http://reviews.vapour.ws/r/2377/
<dimitern> TheMue, voidspace, dooferlad ^^
<dooferlad> dimitern: *click*
<dooferlad> dimitern: you said that space contraints are handled elsewhere in the card I am looking at. Where is that?
<dooferlad> dimitern: also, with network constraints, do you want me to just remove them or just not wire them up in the CLI?
<dooferlad> or just leave as-is I guess.
<dimitern> dooferlad, they need to go, but that's a separate card, which will require a lot of prerequisites (mostly provider and provisioner related) landed first
<dimitern> dooferlad, not sure what do you mean by "handled" - my PR implements parsing and tests for them, and since they're constraints that makes them usable everywhere
<dooferlad> dimitern: doh! Haven't merged that in :-)
<dimitern> whether we do anything with them is another thing
<dimitern> dooferlad, did that answer your questions btw? :)
<dooferlad> dimitern: I am leaving the constraints code as it is, so it will accept networks. I have removed the code path that --networks used for juju deploy.
<dooferlad> dimitern: I have had a look at add-machine and add-unit and both seem to have no explicit --network support, so they are just constraints code again
<dooferlad> dimitern: if that is right, yes.
<dimitern> dooferlad, indeed - I just double checked
<dimitern> dooferlad, are there a lot of tests affected by --networks being ignored ?
<dooferlad> dimitern: I just removed the two that started to fail (correctly) once support for the --network flag was removed.
<dooferlad> dimitern: I didn't look at any already skipped ones.
<dimitern> dooferlad, by "removed" do you mean it returns an error like "flag provided but not defined: --networks" or it accepts it, prints a warning and continues?
<dooferlad> prints a warning and continues as if it isn't there
<dimitern> dooferlad, great!
<dooferlad> dimitern: http://reviews.vapour.ws/r/2382/
<dooferlad> dimitern: I haven't added any support, so much as removed other support.
<dimitern> dooferlad, that's a lot better - less fallout than my original attempt which did the former
<dooferlad> so the title is rubbish
<dimitern> dooferlad, it's fine :)
<dimitern> dooferlad, almost done with the review btw
<dimitern> dooferlad, was the removal of that status test case necessary?
<dooferlad> dimitern: yes, it failed.
<dimitern> dooferlad, that's because we're not stopping tests from calling deploy APIs with networks
<dimitern> dooferlad, rather than removing it, can you comment it out for now with a TODO about fixing it once the API and state layers also ignore (requested)networks (which --networks sets effectively) ?
<dimitern> dooferlad, I'll add a comment to the review
<dimitern> dooferlad, reviewed
<voidspace> dimitern: ping
<dimitern> voidspace, pong
<voidspace> dimitern: so, I add a *Error field to params.Space so we can return partial results
<voidspace> dimitern: the top level params.ListSpaceResult has an Error field (in case fetching the spaces fails - the new error is for an error fetching subnets for an individual space)
<voidspace> dimitern: and ListSpaces also returns an error
<voidspace> dimitern: so if we get an error fetching the subnets for a single space obviously we set the Error field on Space
<dimitern> voidspace, you did all this in your subnets api branch?
<voidspace> dimitern: do we set an error on ListSpaceResult and do we return an error too?
<dimitern> voidspace, or in a follow-up?
<voidspace> dimitern: no, new branch - looking at it now
<dimitern> voidspace, ah, great
<voidspace> dimitern: not sure if I can get it done before I go as I go to the gym in 40minutes
<voidspace> dimitern: that branch landed on net-cli by the way
<dimitern> voidspace, did the other one land?
<voidspace> yep
<dimitern> voidspace, awesome!
<dimitern> voidspace, so I added a card for the ListSpace/Subnet errors
<voidspace> I tried looking for other examples in the apiserver but failed to find any and thought it would be quicker to ask
<voidspace> dimitern: it should just be ListSpace I believe
<voidspace> dimitern: ListSubnets doesn't make extra calls per subnet, so either it all works or it all fails
<dimitern> voidspace, so usually bulk calls that return many results for many entities
<voidspace> (I believe)
<voidspace> dimitern: with ListSpaces we can fetch the spaces and then fail to fetch the subnets for some of the spaces
<voidspace> so partial results make sense in theory
<dimitern> voidspace, ..only return errors if it's not possible to do its job
<voidspace> I say in theory, because in practise the only error is that the connection to state goes
<voidspace> so again, in reality it will be all or nothing
<voidspace> but it's better to stick to the pattern
<voidspace> dimitern: right, so we don't return a top level error
<voidspace> dimitern: what about setting an error on ListSpacesResult?
<dimitern> voidspace, let me have a look at the code now for a moment
<voidspace> dimitern: or should we rely on the client checking the Error field of each Space
<dimitern> voidspace, for ListSpaces you're correct - the backing error should be returned as top-level, others can only be per-space (fetching subnets failed)
<cmars> wallyworld, http://reviews.vapour.ws/r/2387/ please
<voidspace> dimitern: I'm sorry I don't understand
<voidspace> dimitern: what is "the backing error"?
<voidspace> dimitern: if we get an error fetching the subnets for a space obviously we set the Error on the params.Space
<dimitern> voidspace, coming from the state (backing)
<voidspace> dimitern: do we set an error anywhere else?
<voidspace> dimitern: so we also set an error on ListSpaceResults? or are you saying we also return the error at the very top level?
<dimitern> voidspace, sorry, let me explain better
<dimitern> voidspace, https://github.com/juju/juju/pull/2980/files#diff-ec0483c931b7d3d2d378df66d465deb1R100 < this is a top-level error (with empty results)
<voidspace> dimitern: yes
<dimitern> voidspace, https://github.com/juju/juju/pull/2980/files#diff-ec0483c931b7d3d2d378df66d465deb1L109 < this is a per result error
<voidspace> yes
<voidspace> dimitern: that's what we do currently
<dimitern> voidspace, in the latter case, the results != empty, err = nil
<voidspace> right
<voidspace> and we set the error on the individual params.Space we create
<voidspace> (we should I mean)
<voidspace> I'm asking do we *also* do anything else with it?
<dimitern> voidspace, there's no individual params.Space
<dimitern> voidspace, ListSpaces takes no args at apiserver side
<voidspace> the object there "result" is a params.Space
<voidspace> so instead of "return results, err" it would be
<voidspace> result.Error = common.ServerError(err); result.Results = append(result.Results, result);continure
<voidspace> *continue
<dimitern> voidspace, ah, got you
<voidspace> dimitern: that's the change you have said we ought to make
<dimitern> voidspace, yes
<voidspace> I'm asking if we should *also* set results.Error
<dimitern> voidspace, and I've just realized ListSubnets apiserver side is not yet done, so it's fine
<voidspace> and we've agreed that the returned err should still be nil
<voidspace> ListSubnets won't have the same problem
<voidspace> I'm asking if we should *also* set results.Error
<voidspace> and we've agreed that the returned err should still be nil
<dimitern> voidspace, yes, if you can fetch spaces ok, but not all subnets -> return results(with Error set as needed), nil
<dimitern> voidspace, if you can't fetch spaces - as it is now
<voidspace> so I should set result.Error for each Space where fetching subnets fail, and set a results.Error to indicate that there are only partial results
<dimitern> voidspace, sorry, I'm confused - what do you mean by result.Error for each space?
<voidspace> dimitern: every space we fetch is represented by a params.Space
<dimitern> voidspace, yes, yes
<voidspace> dimitern: that will grow an Error field
<dimitern> voidspace, sorry
<voidspace> dimitern: that is settled
<dimitern> voidspace, I need to get in a call
<dimitern> voidspace, it is, yes
<voidspace> dimitern: I am asking, do we *also* set an Error on ListSpaceResults
<dimitern> voidspace, where is that ListSpaceResults defined?
<voidspace> sorry params.ListSpacesResults
<voidspace> at the moment it doesn't have an Error field because the method would either return all results or an error
<voidspace> so it didn't need one there were no partial results
<voidspace> we're now adding partial results
<voidspace> so I wonder if ListSpacesResults (apiserver/params/network.go I believe)
<voidspace> I wonder if it now also needs an Error field
<voidspace> used to indicate partial results
<dimitern> voidspace, I believe we need to make api.ListSpaces() ([]params.Space, error)
<voidspace> dimitern: earlier today you said that apiserver couldn't return []params.Space and ListSpacesResults was still needed :-)
<dimitern> voidspace, then also change ListSpacesResults' Results field to be of type ListSpacesResult (or whatever), which has a Space and Error *Error fields
<voidspace> instead of adding an Error to Space...
<voidspace> add a new intermediate type
<dimitern> voidspace, it's just confusing because of the api/apiserver transition
<voidspace> I think you're still slightly misundestanding my question/suggestion :-(
<voidspace> I think you have too much in your head
<voidspace> we don't need a new intermediate result type, we can just add an Error field to Space
<dimitern> voidspace, ok, I see how params.Space can be used as what I'm describing as ListSpacesResult (singular)
<voidspace> I am asking if the *plural*, the container results type, ListSpacesResults also needs an Error field
<voidspace> because if we get partial results (an error occurred somewhere) don't we want a top level Error field to indicate that the results are partial?
<voidspace> maybe not, and just having the error printed inline with the space description is fine
<dimitern> voidspace, it's not common to have ErrPartialResults from API server methods
<dimitern> voidspace, and in this case I don't think it's even useful
<voidspace> ok
<voidspace> so we won't
<voidspace> thanks
<dimitern> voidspace, thanks for bearing with me :)
<voidspace> :-)
<voidspace> I go soon
<voidspace> so I'll be leaving you to it
<natefinch> ericsnow: if we always flush immediately after set, we should be able to remove the mergeProcMaps
<dimitern> voidspace, sure
<ericsnow> natefinch: right
<voidspace> dimitern: cmd/juju/spaces/list.go - ListSpaces command fetches the subnets using api.AllSubnets()
<voidspace> dimitern: it doesn't use the subnets on the Space!!
<voidspace> I guess space.Subnets() is newer than the ListSpaces command...
<voidspace> I don't think I can fix that in the time I have
<voidspace> it will make the code simpler though
<voidspace> I'll add it to the card
<dimitern> voidspace, ack, otp - thanks
<voidspace> dimitern: done, card updated - including the code to set the error correctly
<voidspace> dimitern: have a good week
<voidspace> TheMue: dooferlad: EOW
<voidspace> I begin my journey home
<voidspace> good luck with the MVP
<TheMue> voidspace: have a safe trip
<voidspace> thanks
<natefinch> ericsnow: updated the code according to your suggestions, would like a re-review so we can get it done
<TheMue> dimitern: maybe I'm missing something and you can point me to the right way. I've got my test, I've got my mock, but the way to inject is seems difficult
<ericsnow> natefinch: will do
<natefinch> ericsnow: realized I didn't publish my responses to your earlier comments.  Just did so.
<ericsnow> natefinch: thanks
<natefinch> (only one real comment)
<dimitern> TheMue, can I look at the code you're having issues with?
<dooferlad> dimitern: do you have an opinion about what card I should do next?
<dimitern> dooferlad, I don't see a card for apiserver subnets - create and lest
<dimitern> list
<TheMue> dimitern: it's not a concrete issue, more a missing idea.
<dooferlad> dimitern: you meen the apiserver side of what voidspace has done?
<TheMue> dimitern: in the test in http://paste.ubuntu.com/12109009/ I try to inject the mock
<dimitern> dooferlad, yeah
<dimitern> dooferlad, we seem to have missed this - can you add a card and start on Add first perhaps?
<dimitern> TheMue, I'll have a look shortly
<dooferlad> dimitern: isn't it in the archive? https://canonical.leankit.com/Boards/View/101652562/115620070
<dooferlad> dimitern: also https://canonical.leankit.com/Boards/View/101652562/115620186
<dimitern> dooferlad, both of these are for spaces
<dooferlad> dimitern: doh!
<dooferlad> dimitern: yes, you are right
<dimitern> dooferlad, I know it's confusing - it gets me every time :)
<dimitern> dooferlad, and I'm wrong - apiserver has AddSubnets, but no list yet
<dimitern> dooferlad, we don't need CreateSubnets for the MVP, so we can just do Add and List
<dimitern> dooferlad, there's a card I added re ListSpacesResults and ListSubnetsResults - if you prefer
<dimitern> dooferlad, also voidspace has a branch with some progress on it
<dooferlad> dimitern: https://github.com/voidspace/juju/commits/spaces-subnets-api ?
<dooferlad> dimitern: that is merged
<dimitern> dooferlad, I think this is it - https://github.com/voidspace/juju/tree/spaces-list-errors
<dooferlad> dimitern: that is about spaces, not subnets
<dooferlad> but if both need list, then I can probably finish that and add subnets
<dimitern> TheMue, it's about ListSpaces returning proper results, which will apply to ListSubnets as well even more (apiserver side)
<dimitern> dooferlad, ^^
<dimitern> TheMue, sorry, I meant to tell you to have a look at TestManageEnvironDoesNotRunFirewallerWhenModeIsNone
<TheMue> dimitern: ah, thx. what a java-like name
<dimitern> TheMue, :)
<dimitern> TheMue, the next one after it is another example
<dimitern> TheMue, since this is a pre-existing test suite, no need to go wild with mocks and things when they're not already in place
<dimitern> dooferlad, does that make sense?
<dooferlad> dimitern: yes
<dimitern> dooferlad, cheers, let's discuss it tomorrow if needed
<dooferlad> dimitern: it is EOD for me, so will make some notes in that card about what is needed and start tomorrow, at which point I will probably have questions :-)
<TheMue> dimitern: my test so far is pretty similar, but environs.SupportsNetworking() tests against the environment. that's why I want to mock it here
<dimitern> dooferlad, awesome!
<dimitern> TheMue, you don't need to mock it with the dummy provider
<TheMue> dimitern: when using dummy environs.SupportsNetworking() always returns true
<dimitern> TheMue, ah, yeah - it wasn't calling SupportsAddressAllocation
<dimitern> TheMue, then, the mocks are unfortunately needed
<dimitern> TheMue, *or*
<dimitern> TheMue, I guess you can make it easier to write tests  for all future cases like this
<dimitern> TheMue, by making the dummy provider *not* support networking when a given environ config setting is there
<TheMue> dimitern: yeah, the better approach, as we already discussed about, is mnot to use type assertion
<dimitern> TheMue, yeah - it will make writing tests a lot easier
<TheMue> dimitern: definitely
<TheMue> dimitern: I change the card description with my ideas. could you take a look please?
<dimitern> TheMue, looks good, only the name SupportsNetworking() on netEnv needs to be better I think
<dimitern> TheMue, but I need to think about it - I'd suggest to go ahead and do it as you suggest, we can rename the NetworkingEnviron method name later as you propose it
<TheMue> dimitern: exactly, renaming it en bloc is simple
<dimitern> TheMue, +1
<TheMue> dimitern: while doing it, how about ProvidesAddressAllocation or AllowsAddressAllocation for Allocate... and Release...
<TheMue> dimitern: ?
<dimitern> TheMue, we have this already - SupportsAddressAllocation - true for both Release and Allocate
<dimitern> anyway, EOD for me
<TheMue> dimitern: enjoy
<dimitern> TheMue, thanks, same to you :)
<TheMue> dimitern: ta
<katco> natefinch: hey are you still blocked on feature tests for juju status?
<natefinch> katco: I haven't spent time on it, but in theory, no.
<natefinch> katco: I can unblock the card
<katco> natefinch: k just checking. probably makes sense to stay on the 3-pointer?
<natefinch> katco: yeah
<katco> natefinch: kk
<katco> wwitzel3: were you working on the rename cards?
<wwitzel3> katco: haven't started them yet, just about have the feature test buttoned up, then that was my next item to start
<katco> wwitzel3: cool
<wwitzel3> katco: thanks to ericsnow's hand holding, I am just about done with the test
<katco> lol
<natefinch> ericsnow: does parseID return name, id or id, name? https://github.com/juju/juju/blob/feature-proc-mgmt/process/info.go#L42
<natefinch> ericsnow: nvm, usage indicates name, id
<ericsnow> natefinch: k
<mup> Bug #1485781 opened: Juju is unreliable on Joyent <joyent-provider> <reliability> <repeatability> <juju-core:Triaged> <https://launchpad.net/bugs/1485781>
<mup> Bug #1485784 opened: Error creating container juju-trusty-lxc-template; Failed to parse config <oil> <juju-core:New> <lxc:New> <https://launchpad.net/bugs/1485784>
<davecheney> thumper: free now ?
<thumper> davecheney: just munching down some food, 5 min?
<davecheney> i'll see you in the hangout
<davecheney> take your time
#juju-dev 2015-08-18
<huwshimi> thumper: Afternoon. Are you using any features from nose in jujulib?
<thumper> no, just running tests
<thumper> huwshimi: as long as "make check" works, I'm happy
<huwshimi> thumper: Ah cool, I have some comments on my PR about just using pytest, so I'll fix that up if that works for you?
<thumper> kk
<huwshimi> thumper: Cool, cheers mate.
<mup> Bug #1484993 changed: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Fix Released by wallyworld> <https://launchpad.net/bugs/1484993>
<dimitern> fwereade, ping
<fwereade> dimitern, pong
<dimitern> fwereade, I'd like to ask you about a quirk with our environ constraints implementation that bit me badly
<dimitern> fwereade, we have Tags *[]string ",omitempty" and the doc comment says if they're nil they're not set, but if they are an empty slice, they are replaced
<fwereade> dimitern, ok
<dimitern> fwereade, this is a lie - esp. for env constraints, as they use a single doc, so there's *no way* to set tags= once there's a value on it
<dimitern> fwereade, the same applies to Networks and Spaces (i'm now adding)
<dimitern> fwereade, so my question is - is it a doc issue or by design you can't unset a list-style constraint once set on an environment
<fwereade> dimitern, that just sounds like a mistake honestly -- although, I admit, I forget the details
<fwereade> dimitern, what happens if you just do set-constraints mem=8G, on anything -- does it not overwrite the whole set of constraints?
<fwereade> dimitern, at that level?
<dimitern> fwereade, so first, this only happens if you're using the same constraints document - i.e. environ constraints is an obvious case (not sure if you can do it on a machine)
<fwereade> dimitern, ahhh -- so it's a mgo level thing? we're not doing $unset when we should?
<fwereade> dimitern, would also apply to services, but, yeah, not machines
<dimitern> fwereade, then, it's pretty easy to try it out - juju environment set-constraints mem=8G tags=foo,bar; then again with "mem=4G" -> you get "mem=4G tags=foo,bar"; trying with "tags=" -> same result
<fwereade> dimitern, ...unless we get cleverer about retry-provissioning-with0different-constraints one day
<dimitern> fwereade, it is a mongo thing indeed
<fwereade> dimitern, right, yeah, I see -- we just $set and don't $unset everything else
<fwereade> dimitern, dammit
<dimitern> fwereade, so what I did to fix the failing tests for the time being - added ResetEnvironConstraints which does removeOp+createOp in export_test
<dimitern> fwereade, we just $set
<dimitern> fwereade, but this needs fixing I think - it's pretty surprising
<fwereade> dimitern, I would suggest that those tests are failing because they should, and ResetEnvironConstraints is a Bad Thing
<fwereade> dimitern, but, good catch
<dimitern> fwereade, if the behavior is "don't touch anything I don't explicitly set" - that's great, but there needs to be a way to reset them
<fwereade> dimitern, I'm not sure how great it is actually
 * fwereade thinks
<dimitern> fwereade, they only started failing because so far there were only 2 cases where we do "set a list-constraint to empty", then "set it to not empty" - last 2 cases, any other case added after it reveals the bug
<dimitern> fwereade, tags were totally untested in this set-env-check-inherited-repeat
 * fwereade glowers around in an undirected sort of way and reminds everybody that tests are critically important
<dimitern> fwereade, I think this deserves a bug report and I'll file one
<fwereade> dimitern, at the very least, `an empty value always means "not constrained"` and we're failing there
<fwereade> dimitern, https://jujucharms.com/docs/devel/charms-constraints
<fwereade> dimitern, yes please
<dimitern> fwereade, it can be fixed easily, but the expected behavior needs to be well defined - do we unset what you don't provide implicitly? if not, how can you tell juju to do unset+set when you want?
<fwereade> dimitern, I think we should indeed unset everything not provided
<dimitern> fwereade, and provide a "merge existing" command?
<dimitern> fwereade, we don't really specify that the merge-by-default is what happens, so I guess we could safely do replace-by-default, at least for environ constraints
<fwereade> dimitern, yeah, I *think* the merging behaviour can be reasonably considered a bug
<fwereade> dimitern, I don't think there's much reason for a merge-existing?
<fwereade> dimitern, the inheritance is confusing enough
<fwereade> dimitern, effectively having an extra level of it is IMO worse than assuming people know what they want to set
<dimitern> fwereade, it's also that it only really matters for lists, as other fields usually have a non-ambiguous "empty" value
<dimitern> fwereade, fun fact: "cpu-cores=" != "" != "cpu-cores=0"
<fwereade> dimitern, yeah, I think we definitely want to overwrite lists rather than merge
<dimitern> fwereade, ok, then I'll make this part of my fix
<fwereade> dimitern, yeah, `cpu-cores=` means "fall back" where `cpu-cores=0` means "any number of cores is fine" -- right?
<dimitern> fwereade, not quite :) - all of the cases match omitempty
<dimitern> fwereade, but only when stored in state
<fwereade> dimitern, wait, omitempty ignores the difference between `nil` and `new(int)`?
<dimitern> fwereade, but e.g. (*int)(value-with-0) and (*int)(nil) are treated the same
 * fwereade resists the urge to render #juju-dev NSFW forever
<dimitern> fwereade, :D
 * fwereade is going for a quiet rage-filled cigarette
<fwereade> dimitern, thanks for many good catches then :)
<fwereade> bbs
<dimitern> fwereade, cheers :)
<dimitern> fwereade, env:"mem=8G", svc:"mem="; effective: "mem=" which is the same as "mem=0"
<dimitern> so "X=" is never treated as "", but as "X=<empty-value>" instead
<TheMue> morning, lots of traffic on CI *lol*
<TheMue> large packages w/o sub-packages lead to story-telling test function names
<dimitern> TheMue, dooferlad, guys, will be a couple of minutes late for standup
<TheMue> dimitern: ok
 * TheMue looks left
 * TheMue looks right
<TheMue> I'm a poor, lonesome coder *sing*
<dooferlad> TheMue: https://plus.google.com/hangouts/_/canonical.com/juju-sapphire?authuser=0 ?
<TheMue> dooferlad: I'm here https://plus.google.com/hangouts/_/canonical.com/juju-sapphire too. the one that has been in the reminder mail of the calendar.
<dooferlad> TheMue: we are in there too
<TheMue> dooferlad: will reconnect
<fwereade> TheMue, dimitern: why are we pulling environ config down to the machine agent to figure out if we support networking?
<fwereade> TheMue, dimitern: shouldn't this be a job?
<dimitern> fwereade, because we need an environs.Environ to type-assert on
<fwereade> dimitern, I don't think there's any justification for creating an environ in an agent
<fwereade> dimitern, we can figure that out at machine-creation time, can't we? when we've already got the state policy available?
<dimitern> fwereade, it a different sort of job - address (de)allocation and manage networking
<fwereade> dimitern, yeah, I'm not saying its the same as JMN -- but wouldn't it be JobAllocateAddresses or something?
<dimitern> fwereade, yeah, I agree the best thing is to add a job
<dimitern> fwereade, however, as a incremental improvement, we can make our tests better around checking if the environment supports address (de)allocation (which we don't have full coverage of now)
<dimitern> fwereade, and then build on top of this by introducing a job
<dimitern> fwereade, sane?
<fwereade> dimitern, what I'm worried about is that this is further degradation of machine agent sanity -- it's bad enough that there are some bits of code that switch on provider type, but I'm really bothered by *new* code that actually pulls down a whole env config and tries to construct it and hopes that the foul overwrite-secrets hack doesn't cause problems
<fwereade> dimitern, imo the machine agent shouldn't even know what sort of provider it's running on
<dimitern> fwereade, fwiw this only happens once per environment
<fwereade> dimitern, how does that make env-construction in the machine agent any less evil?
<dimitern> fwereade, and we did drop one (the only remaining?) provider type checks in the MA using JobManageNetworking
<TheMue> fwereade: how do you compare address allocation support with firewaller modes? the depend on the providers capabilities too. could a similar approach be feasable?
<dimitern> fwereade, why is it evil for the JobManageEnviron agent?
<dimitern> fwereade, it has full access to it anyway
<fwereade> dimitern, the *agent* shouldn't know anything about the environ
<fwereade> dimitern, it might run some job that knows about it
<fwereade> dimitern, but that's a completely distinct responsibility
<fwereade> TheMue, I'd say that firewaller modes should not be using environ config either
<dimitern> fwereade, by "some job that knows about it" you mean a worker that pulls the environ config on start?
<fwereade> dimitern, yeah
<dimitern> fwereade, even if it doesn't need it except to check whether it needs to do anything at all
<fwereade> dimitern, via a facade, and for a worker, that's allowed to know about it
<fwereade> dimitern, do we know why we added Environment() to the Agent facade in the first place?
<dimitern> fwereade, not really
<dimitern> fwereade, I assume because it was there
<fwereade> dimitern, oh, feck, it's not even that
<dimitern> TheMue, +1 about the firewaller
<fwereade> dimitern, someone just thought "lol  I'll use the wrong facade in the agent"
<dimitern> fwereade, :)
<dimitern> fwereade, ok, can we agree to add a job later?
<dimitern> fwereade, I promise we won't forget about it
<TheMue> dimitern: my though is that we have more and more providers with different capabilities, so how to only start those workers (in their right modes) for the current environment?
<fwereade> dimitern, TheMue: this all seems like code that's been added in the last couple of days though
<fwereade> dimitern, TheMue: and should frankly have never passed review
<dimitern> fwereade, the change is strictly an improvement over what we had before
<TheMue> "should" is a wonderful subjunctive
<fwereade> dimitern, TheMue: at the *very least*, why is this stuff not handled inside the addresser worker?
<dimitern> fwereade, this check is simply for a better UX - not to spam the logs with "addresses: not starting.."
<TheMue> fwereade: yep, agree to dimitern, the old approach always started the worker, got the events but done nothing
<dimitern> fwereade, it used to be, but TheMue suggested it should be done at the apiserver level with a single call
<dimitern> fwereade, and I liked the idea of not violating layers unnecessarily
<fwereade> dimitern, TheMue: I don't see how gunking up the already-terrible agent code is an improvement over making the addresser exit quietly with a single log message when it finds it's in an inappropriate env
<dimitern> fwereade, it is, after all, a job that requires both access to state and the provider - apiserver is the obvious place for that, isn't it?
<TheMue> dimitern: the dont-start-not-needed-worker came later and is a different one. imho that's wht we currently talk about
<dimitern> fwereade, it exits quietly ever 3s as it gets restarted by the runner
<fwereade> dimitern, TheMue: wait, I hope it doesn't use state directly, does it?
<TheMue> fwereade: nope, that has been the original idea of the change
<dimitern> fwereade, not anymore, that was the whole point of the exercise - it uses its own apiserver facade
<TheMue> fwereade: no more state usage
<fwereade> ah cool :)
<TheMue> :)
<fwereade> (although... why would we *ever* have used state directly?)
<fwereade> (we did *so much fucking work* to cut out direct state access)
<dimitern> fwereade, because its on the JobManageEnviron MA
<fwereade> dimitern, that dooesn't mean iit gets to see mongo
<dimitern> fwereade, this card along with a bunch of others (some of which already landed) is our effort to cut state access for JME workers
<fwereade> dimitern, re the 3s restart -- why don't you just *return nil*?
<fwereade> dimitern, meaning "I have done all the work I can, don't bother me agaib"
<dimitern> fwereade, if the runner supported a way to tell it "don't restart me anymore"
<dimitern> fwereade, it would've been simpler
<dimitern> fwereade, IIRC that leads to the same thing
<fwereade> runner.go:205
<fwereade> runner_test.go:55
<dimitern> fwereade, hmm, ok fair enough
<TheMue> fwereade: nice
<dimitern> fwereade, so it does not restart if it return nil from the loop?
<fwereade> dimitern, yeah
<dimitern> fwereade, that's ok, but not good enough I'm afraid
<fwereade> dimitern, go on
<dimitern> fwereade, we need a way not to start it in the first place
<fwereade> dimitern, why?
<TheMue> fwereade: dimitern: so addresses starts, first time talks to API, get's a not supported error back, and quits nicely
<dimitern> fwereade, i.e. a way to return something != nil from New() that tells the runner not to run it
<TheMue> addresser
<fwereade> dimitern, TheMue: why can't the worker just detect that it can't do anything and stopp itself?
<fwereade> dimitern, TheMue: it's not an error -- it has examined the world and determined that it doesn't need to do anything ever
<dimitern> fwereade, hmm.. I guess the apiserver facade's WatchIPAddresses can return a NotSupportedf if it doesn't make sense to watch or run the worker?
<TheMue> fwereade: it needs the environment to check this capability, so we move the question from machine.go into the addresser worker itself. that's what you meant?
<fwereade> dimitern, TheMue: I'd prefer a sepaarate api call, rather than overloading that one
<dimitern> fwereade, that's fine - it can be separate
<TheMue> sounds good
<dimitern> fwereade, what's important is will it work before the loop starts
<fwereade> TheMue, I *would* probably like it most if we represented it as a job, but yeah, it's definitely better to have it scoped to the addresser worker
<fwereade> TheMue, dimitern: although actually, yeah, maybe it *is* just part of JME
<fwereade> TheMue, dimitern: this may be relevant: in the unit agent, now, the manifolds themselves return dependency.ErrMissing when started in an inappropriate context
<fwereade> TheMue, dimitern: this means we can have nice declarative conditional-free definitions of various sets of workers and their dependencies
<fwereade> TheMue, dimitern: relevant/sane?
<TheMue> fwereade: have to read tha last sentence twice
<fwereade> TheMue, dimitern: see cmd/jujud/agent/unit/manifolds.go
<TheMue> fwereade: thx for the hint
<fwereade> TheMue, dimitern: and then the feature-flag checkinng inside worker/rsyslog and worker/logsender
<fwereade> TheMue, dimitern: in practice we'll only run one of those at a time I think
<dimitern> fwereade, I like this a lot
<fwereade> TheMue, dimitern: but they can make that determination individually
<dimitern> fwereade, it will also make testing this a *lot* easier
<fwereade> dimitern, :D
<dimitern> fwereade, one of each *per environ*
<fwereade> dimitern, yeah, agreed
<fwereade> dimitern, TheMue: so, to reiterate
 * TheMue listens
<dimitern> fwereade, yeah, let's summarize
<fwereade> dimitern, TheMue: I don't think we do need a separate job there, sorry for the noise on that front
<dimitern> fwereade, in practice, no we don't
<fwereade> dimitern, TheMue: but if we can just make the addresser (and firewaller?) check for their own validity, log once, and return no error
<fwereade> dimitern, TheMue: then we get a slightly cleaner agent
<fwereade> dimitern, TheMue: for relatively little in the way of code changes
<TheMue> fwereade: dimitern: oh yes +100
<fwereade> dimitern, TheMue: and it *also* makes my job much easier when I come to use manifolds in the machine agent
<fwereade> dimitern, TheMue: which I am very keen to do :)
<TheMue> fwereade: so I'm changing addresser again and you the firewaller *lol*
<fwereade> TheMue, if you can bear to hit both of them I would be most grateful :)
<dimitern> TheMue, fwereade, let's not do all at once though - first the addresses, then the firewaller
<fwereade> dimitern, yeah, that works for me
<TheMue> +1
<dimitern> addresser even
<fwereade> dimitern, TheMue: hopefully having done one will make the second one trivial anyeay
<dimitern> fwereade, it should, yeah
<fwereade> dimitern, TheMue: cool, thank you very much
<fwereade> dimitern, TheMue: and then RB2390 becomes ~redundant, in favour of another unit test or two in each of the relevant workers
<dimitern> fwereade, so, if I'm getting it right, you suggest to have a method in the workers' apiserver facade like CanRun()
<TheMue> fwereade: 2390 will be dropped
<dimitern> fwereade, which is called at the start of Handle() ? Or in the constructor returning the worker?
<dimitern> fwereade, I'd really like it if we can avoid starting the loop if we can - i.e. constructor is better
<fwereade> dimitern, sadly I'm not sure how we handle failing to start a worker... so it might perhaps be best to return a degenerate, already-stoppped worker from NewWorker?
<fwereade> dimitern, little bit hackish
<dimitern> fwereade, this assumes we have a special error like ErrCannotStart, which startWorker understands
<fwereade> dimitern, but easily replaceable, I think, when I use manifolds
<fwereade> dimitern, yeah, if you want to add ErrWorkerNotNeeded handling to worker.Runner, that could work
<fwereade> dimitern, but Runner *is* a bit hairy
<dimitern> fwereade, yeah, very much so - ok, then constructing a "degenerate" worker in this case might be better until it can be converted to use manifolds
<fwereade> dimitern, TheMue: http://paste.ubuntu.com/12117329/
<fwereade> dimitern, TheMue: and then for the manifolds we don't even need that
<TheMue> dimitern: so this degenerated one has a Handle(), but that only returns I'mNotNeeded
<fwereade> dimitern, TheMue: just return dependency.ErrMissing and it won't try again until its dependencies channge
<fwereade> dimitern, TheMue: which is quite likely to be "never"
<fwereade> dimitern, TheMue: so addresser.NewWorker would either return a StringsWorker or a FinishedWorker
<TheMue> fwereade: a little misusage of ErrMissing, eh?
<fwereade> TheMue, sort of, yeah :)
<fwereade> TheMue, although it *does* seem appropriate to try again when, eg, the api conn bounces
<fwereade> TheMue, you might get a different answer next time ;)
<dimitern> fwereade, TheMue, ok, in NewWorker we log about it and return FinishedWorker
<TheMue> fwereade: ah, how I love the benefit of working with interfaces as arguments or return values. does not mean the error, only the return of worker.Worker
<fwereade> dimitern, sgtm
<fwereade> TheMue, yeah, indeed
<dimitern> fwereade, where does dependency.ErrMissing come into play is what I don't get yet
<fwereade> dimitern, it's only relevant when I get around to using manifolds
<dimitern> fwereade, I see, great!
<fwereade> dimitern, fingers crossed I'll make a start on that this week...
<fwereade> dimitern, TheMue: thanks guys, gtg
<TheMue> dimitern: so, master of the saphire cards, how we wonna split it?
<dooferlad> TheMue/dimitern: could I get a quick review? http://reviews.vapour.ws/r/2392/
<dooferlad> dimitern: and when you have a moment, can we discuss my next card.
<TheMue> dimitern: todays test card can get the addresser startup change card
<TheMue> dimitern: the firewaller one has should be a new one for backlog
<dimitern> TheMue, so, it sounds like the following:
<dimitern> dooferlad, sure, will have a look shortly
<dimitern> TheMue, a new CanStart() error on the apiserver, returning nil if address allocation with network.AnySubnet returns true, nil
<dimitern> TheMue, we can have such apiserver methods I think, if not, then (params.BoolResult, error) and the error is not used or result.Error is not - I don't mind
<TheMue> dimitern: would prefer SupportsAddressAllocation(), CanStart() sounds too generic
<dimitern> TheMue, ok, but that will cause confusion
<dimitern> TheMue, let's not call it the same way as the Networking method
<dimitern> TheMue, CanDeallocateAddresses() is better IMO
<TheMue> dimitern: thats fine with me, better than asking an API "do you can start?" ;)
<TheMue> dimitern: and meaning "can I start?"
<dimitern> TheMue, then, in the NewWorker if it returns != nil, log it and return FinishedWorker, nil
<dimitern> TheMue, yeah
<dimitern> TheMue, then the test in the MA will be simply - patch newAddresser to return FinishedWorker, check the next one starts and you're done
<TheMue> dimitern: btw, I prefer bool instead of misuse nil as "I can"
<TheMue> dimitern: yep
<dimitern> TheMue, because testing that CanDeallocateAddresses() works with or without the feature flag is an apiserver test, not an agent or api test
<TheMue> dimitern: exactly
<dimitern> TheMue, ok, that's easy - the client-side api returns (bool, error) and you use params.BoolResults (no ",error") at the apiserver side
<TheMue> dimitern: top
<dimitern> TheMue, testing the FinishedWorker logic is only relevant to the agent, so test it there only
<dimitern> TheMue, on the worker side, check it stops and logs when the feature flag is off and that's all I think
<TheMue> dimitern: sounds pretty complete, yes
<dimitern> TheMue, awesome! can I ask you to add these steps to the card so it capture what we decided?
<dimitern> TheMue, I really need to get some branches landed :)
<TheMue> dimitern: that's what I wonna do and also add a dummy firewaller card to the backlog
<dimitern> TheMue, cheers!
<dimitern> dooferlad, back to your review btw
<dimitern> dooferlad, doesn't the client-side api need to change as well?
<dooferlad> dimitern: probably!
<dimitern> dooferlad, it does, now it returns response.Results, err
<dooferlad> dimitern: OK, will fix. Sorry, not a great start to the day :-|
<dooferlad> dimitern: did you want me to look at spaces list after this?
<dimitern> dooferlad, no worries
<dimitern> dooferlad, yes, I'm almost done with the review
<dooferlad> dimitern: just wanted to know what to queue up next.
<dimitern> dooferlad, do you mean subnets list?
<dooferlad> dimitern: https://canonical.leankit.com/Boards/View/101652562/116678115
 * dimitern looks
<dimitern> dooferlad, just let me finish the review first, bbiab
<dooferlad> dimitern: sure
 * dooferlad seems to have got confused. Perhaps coffee is required.
<dimitern> dooferlad, reviewed
<dimitern> dooferlad, let me know what you think
<dimitern> dooferlad, I forgot to mention the needed client-side api change as well.. adding it now
<dimitern> dooferlad, hmm, actually returning all results might be better for the CLI - it can display spaces fetched successfully and display errors for others (like juju status does) in the Status field of the output
<perrito666> morning
<dimitern> perrito666, morning
 * perrito666 waits for the coffee before hitting the review queue
<dimitern> dooferlad, the good thing about this is we don't have to display the error (i.e. change the CLI) right away
<ericsnow> fwereade, cmars: it would be nice if the new ValueWorker found its way into master sooner rather than later :)
<ericsnow> (https://github.com/juju/juju/pull/3006)
<fwereade> ericsnow, I'm a little bit distracted right now, but if you'd like to pull it across I will gladly LGTM it, I think the diff should be clean
<mup> Bug #1486074 opened: juju bootstrap fails to work due to incorrect Identities restriction. <juju-core:New> <juju-core 1.24:New for cox-katherine-e> <juju-core 1.26:New> <https://launchpad.net/bugs/1486074>
<ericsnow> fwereade: k, thanks
<fwereade> ericsnow, thank you, you're doing the tedious bit ;)
<ericsnow> fwereade: http://reviews.vapour.ws/r/2397/
<ericsnow> fwereade, cmars: keep me posted if you do anything to help with nesting dependency engines :)
<fwereade> ericsnow, ship it :)
<ericsnow> fwereade: thanks
<wwitzel3> ericsnow: ping
<dooferlad> dimitern/TheMue: when you have a moment: http://reviews.vapour.ws/r/2392/
<TheMue> dooferlad: yep
<dimitern> dooferlad, will have a look soon
<fwereade> axw, wallyworld: a bit clunky, perhaps, but does http://paste.ubuntu.com/12118757/ strike you as a useful primitive?
<benji> k
<benji> #wrongwindow
<dimitern> dooferlad, reviewed
<mup> Bug #1486106 opened: TestNetworkInterfaces fails on vivid/wily <ci> <ec2-provider> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1486106>
<mup> Bug #1486106 changed: TestNetworkInterfaces fails on vivid/wily <ci> <ec2-provider> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1486106>
<mup> Bug #1486106 opened: TestNetworkInterfaces fails on vivid/wily <ci> <ec2-provider> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1486106>
<mup> Bug #1461890 changed: leadership unreliable in HA <canonical-bootstack> <ha> <leadership> <juju-core:Fix Released by fwereade> <juju-core 1.24:Fix Released by fwereade> <https://launchpad.net/bugs/1461890>
<mup> Bug #1478024 changed: Looping config-changed hooks in fresh juju-core 1.24.3 Openstack deployment <canonical-bootstack> <leadership> <upgrade-juju> <juju-core:Fix Released by fwereade> <juju-core 1.24:Fix Released by fwereade> <https://launchpad.net/bugs/1478024>
<natefinch> damn, just hit one more reason I don't like putting test code in _test packages - if you have tests you need to put in the actual package, you can't access the helper code that is in the _test package.  le sigh.
<sinzui> katco: can you ask someone to look into bug 1486166. I cannot tell were the regression lies.
<mup> Bug #1486166: JES deploy fails <ci> <jes> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1486166>
<katco> sinzui: tal
<katco> sinzui: dimitern's team is on bug squad this iteration. can it wait until they come back online?
<sinzui> katco, sure, I am not going to block trunk for the failure
<katco> sinzui: ok ty, i'll send an email to dimitern
<dimitern> katco, sinzui, I've looked at this one and it seems to me it's badly written - assuming it will only run on environments where the jes feature flag is on
<mup> Bug #1486166 opened: JES deploy fails <ci> <jes> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1486166>
<perrito666> need to leave for a sec, bbl
<alexisb> katco, sinzui I think that is a bug for thumpers team
<alexisb> I can ping him when he is online
<wwitzel3> ericsnow: ping, how's it going?
<ericsnow> wwitzel3: good
 * katco grabs popcorn for this fascinating conversation
<katco> perrito666: any chance you could do a mini-review for a critical 1.24 bug?
<katco> perrito666: http://reviews.vapour.ws/r/2402/
<perrito666> katco: going
<perrito666> katco: ship it
<katco> perrito666: ty sir
<perrito666> katco: please update the re-broken bug
<katco> perrito666: yep, ty for the reminder
<perrito666> katco: btw, this might be sympthom of a different problem
<katco> perrito666: yes, i agree. but this is a good start as it may be affecting multiple things, and the bug it fixed was much lower priority
<perrito666> if we pass a set of explicit ident files and expect one of those to be the correct one and it isn t then there is a bigger issue there
<katco> perrito666: yep.
<katco> perrito666: more trying to address this: https://bugs.launchpad.net/juju-core/+bug/1486074
<mup> Bug #1486074: juju bootstrap fails to work due to incorrect Identities restriction. <juju-core:Triaged> <juju-core 1.24:In Progress by cox-katherine-e> <juju-core 1.26:Triaged> <https://launchpad.net/bugs/1486074>
<katco> perrito666: err, oops
<katco> perrito666: https://bugs.launchpad.net/juju-core/+bug/1472632
<mup> Bug #1472632: regression: juju ssh dies with (publickey) <regression> <ssh> <juju-core:Triaged> <juju-core 1.24:Triaged by cox-katherine-e> <https://launchpad.net/bugs/1472632>
<perrito666> yup, I read the others, the fix does more harm than good :)
<katco> perrito666: meaning the fix i backed out? or the fix to fix the fix? :)
 * perrito666 is very tempted to answer in french and step this into a whole new realm of lexical complexity
<katco> haha
<mup> Bug #1486254 opened: Raw Go errors reported to users <juju-core:New> <https://launchpad.net/bugs/1486254>
<mup> Bug #1486254 changed: Raw Go errors reported to users <juju-core:New> <https://launchpad.net/bugs/1486254>
<mup> Bug #1486254 opened: Raw Go errors reported to users <juju-core:New> <https://launchpad.net/bugs/1486254>
<alexisb> thumper, ping
<thumper> alexisb: otp
<alexisb> thumper, let me know when you are free
<perrito666> alexisb: oh, just check his refcount
 * perrito666 hits himself for that joke
<alexisb> :)
<menn0> perrito666: no, that was good :)
<perrito666> menn0: oh, you are too kind.... what do you need?
<menn0> perrito666: nothing :)
<thumper> alexisb: free now
<thumper> alexisb: free now
<alexisb> thumper, can your team take a look at <https://launchpad.net/bugs/1486166>
<mup> Bug #1486166: JES deploy fails <ci> <jes> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1486166>
<thumper> yes, I even have that bug open already
<thumper> alexisb: was it just that?
<alexisb> yep
<alexisb> unless you have things you want to discuss :)
<thumper> heh... well nothing I've not said before...
<thumper> sinzui: the jes deploy test logs are useless due to only having INFO level logging, can you run it with debug please?
<thumper> menn0: the jes deploy test is failing as machine 1 and machine 2 cannot download the tools from the state server during cloud-init
<thumper> investigating more
#juju-dev 2015-08-19
<sinzui> thumper: yes, I will jigger it now
<mwhudson_> davechen1y: https://launchpad.net/ubuntu/+source/golang/2:1.5~rc1-0ubuntu1 \o/
<sinzui> thumper: http://reports.vapour.ws/releases/2985/job/joyent-deploy-jes-trusty-amd64/attempt/117 used --debug
<davechen1y> mwhudson_: /me pops cork
<sinzui> thumper: menn0: joyent's networking non-sense is often a cause of not downloading agents from the state-server. That is the main reason for the reties on joyent jobs. Even when agents are started, the units fail. I think you can see thie in the 117 ^ attempt.
<thumper> sinzui: the unit agent failure is almost certainly a problem with JES and the new debug log writer
<thumper> menn0: ^^
<thumper> sinzui: the failing to download tools in the machine agents cloud init is something else
<thumper> menn0: I'm guessing the new manifold bollocks for the log sender is a problem
<thumper> menn0: the people making defining the dependencies almost certainly got it wrong
<sinzui> thumper: yeah, joyent has 72.* and 165.* addresses. We often see the private nets cannot see each other when joyent gives us differing public addresses
<thumper> ah...
<thumper> sinzui: what is our solution there?
<menn0> thumper: sorry, playing catch up. where can I see the unit agent failure?
<sinzui> thumper: retry, or deploy lots of machines to hold the 72.* addresses, release the 165.* addresses
<thumper> menn0: http://data.vapour.ws/juju-ci/products/version-2985/joyent-deploy-jes-trusty-amd64/build-117/unit-dummy-source-0.log
<sinzui> thumper: we tried prefer-ipv6, but we still see agents downloads using ip4
<menn0> sinzui, thumper: i made a joyent specific routing fix a long time ago which allowed the various private nets they allocate to talk to each other
<thumper> sinzui: hmm...
<menn0> sinzui, thumper: is that not working any more?
 * thumper shrugs
<sinzui> menn0: I don't think it is that reliable, but maybe the firewall issue i describe in another bug is the root cause. I haven't escallated the other issue because I am still gathering evidence
<menn0> sinzui: or maybe joyent has changed the way their network works
<sinzui> menn0: they changed firewal rules recently :)
<menn0> sinzui: ok right
<menn0> thumper: should I take a look at this panic?
<thumper> menn0: if you have the bandwidth, yes please
<menn0> thumper: well there's also this bootstrap issue and the State pool work. tell me my priorites and I'll work to that :)
<thumper> ugh
<thumper> menn0: I'll poke the panic, you continue with bootstrap issue
<menn0> thumper: k
 * thumper sighs
<thumper> at least the panic is entirely reproducable with the local provider
 * thumper goes to make coffee
<menn0> thumper: that's good to know. should be easy enough to track down then.
<thumper> well, you'd think that
<menn0> :)
 * thumper pokes more
<menn0> thumper: here's the bootstrap fix: http://reviews.vapour.ws/r/2407/
<thumper> menn0: the connection is null in sendLogRecord
 * menn0 pulls up current master
<thumper> bah humbug
 * thumper takes a breath
<thumper> menn0: if you have time
<thumper> would love a hangout to talk this through as I read the code
<menn0> thumper: sure
 * menn0 close the office door
<thumper> menn0: I'm just testing, but I *think* that the one line error fix will make things work
<thumper> just not as cleanly as we'd like
<menn0> thumper: b/c it retries
<thumper> ack
<menn0> thumper: and at some point the api info will be correct....
<menn0> nasty
<thumper> ack
<thumper> yeah
<thumper> I'd still like to represent the dependencies correctly, but this will get us over the hump
 * thumper is deploying now
 * menn0 nods
<menn0> it would definitely be nice to do it right
<thumper> yep, that works
<huwshimi> thumper: Hey. I've got a branch ready to land for jujulib. Is there a process for landing branches yet?
<thumper> nope, but I can land it for you
<thumper> if it is reviewed
<huwshimi> thumper: It's this one. Has a couple of plus ones. https://github.com/markramm/jujulib/pull/3
 * thumper takes a look at the actual diff too
<thumper> huwshimi: merged
<thumper> huwshimi: thanks, looks real good
<huwshimi> thumper: Brilliant, thanks!
<thumper> now that I've dealt with the bug I was looking at, I'm going to fix my review comments that didn't get addressed before mramm2 merged my branch
<thumper> huwshimi: hmm...
<thumper> huwshimi: found a bug in the makefile :)
<huwshimi> thumper: oh...
<thumper> calling make check when there is no .env setup fails
 * thumper will fix
<thumper> huwshimi: "rm .file" fails if .file doesn't exist
<thumper> "rm -f .file" does not fail if it doesn't exist
<huwshimi> thumper: Ah, nicely spotted. Thanks
<thumper> now I get a lot of lint
 * thumper sighs
<huwshimi> thumper: Yes, yes you do.
<huwshimi> thumper: and an 'assert False' :)
<thumper> huwshimi: that was from mramm
 * thumper passes the buck
<huwshimi> haha
 * thumper is done
<thumper> laters
<dimitern> morning
<dimitern> fwereade, hey
<fwereade> dimitern, o/
<dimitern> fwereade, when you have a moment, I'd like you to review this http://reviews.vapour.ws/r/2406/ please
<dimitern> fwereade, it should do everything we discussed about setting/getting constraints
<fwereade> dimitern, cool, I will try to get to that, I see menn0 has covered a few things already?
<dimitern> fwereade, yeah, and I really appreciate that, but since it's changes core functions a second look will be nice :)
<Mmike> Hi, lads. What is juju using for leader election, which algo/protocl?
<urulama> just to verify: can openstack provider deal with user tokens from Keystone instead of usernames and passwords? In case it does, how is refreshing done and where are they stored?
<dimitern> dooferlad, hey, so my constraints branch is landing now
<dimitern> dooferlad, after it lands it's a good time to sync up net-cli with master
<mup> Bug #1486553 opened: i/o timeout errors can cause non-atomic service deploys <juju-core:New> <https://launchpad.net/bugs/1486553>
<mattyw> fwereade, http://reviews.vapour.ws/r/2415/
<jogarret6204> hi all,  I have an upgrade stuck, causing many errors such as: ..."blocked because upgrade in progress".  Any ideas how to unstick it?
<frankban> hi ocr, could you please take a look at https://github.com/juju/juju/pull/3035 ? (this is a MP against a feature branch for the GUI embedded story). thank you!
<natefinch> jogarret6204: upgrading what version to what version?
<frankban> katco: ^^^
<jogarret6204> natefinch:i think i am 1.24.3 now...  not sure how to tell what it is targeting..
<jogarret6204> agent-version: 1.24.3.1
<natefinch> jogarret6204: likely going to 1.24.5, if you started the upgrade today or yesterday.
<jogarret6204> any way to kick it along?  seeing other issues now
<jogarret6204> message: agent is lost, sorry! See 'juju status-history all-in-one/34'
<jogarret6204> but can't check that, that command returns: ERROR upgrade in progress - Juju functionality is limited
<natefinch> jogarret6204: is it just one machine or a bunch of machines?
<jogarret6204> bunch.  I have maas on baremetal, it uses a juju VM on same box for state server.  then about 10 of these machings in issue right now
<jogarret6204> I opened a juju-deployer bug - this may actually be the cause of that
<jogarret6204> http://bit.ly/1KvMybk
<natefinch> jogarret6204: interesting.  I don't really know the deployer code, so can't comment on what it does or does not do.  But certainly the unit number should not be used as the count of units.
<natefinch> frankban: katco is going to be in late this morning, btw
<frankban> natefinch: ok thanks, there is no rush
<jogarret6204> natefinch:  i'm thinking that it may not be a bug if I get this upgrade issue fixed.
<wwitzel3> ericsnow, natefinch: can we delay standup maybe 10 minutes? I lost track of time and have bacon on the stove
<wwitzel3> well, actually, in the oven, but same thing
<natefinch> wwitzel3: I can wait for bacon
<ericsnow> fwereade, cmars: how do I "uninstall" a worker from an engine?  (Runner has StopWorker...)
<fwereade> ericsnow, you can't, yet; was waiting for a direct need to do so
<fwereade> ericsnow, what's your use case?
<ericsnow> fwereade: I have per-workload-process workers with a definite lifetime
<ericsnow> fwereade: when Juju stops tracking such a workload process then the worker must be stopped and forgotten
<ericsnow> fwereade: for now I am resorting to using runners but would rather use dependency engine
<fwereade> ericsnow, that sounds sane for the short term
<ericsnow> fwereade: yeah, I figured we'd sort it out later :)
<fwereade> ericsnow, sorry, I wasn't expecting to need it until I got relatively deeply stuck into the machine agent
<ericsnow> fwereade: no worries :)
<fwereade> ericsnow, (out of interest, what are the dependencies of your process workers?)
<fwereade> ericsnow, (and their responsibilities?)
<ericsnow> fwereade: deps - mostly API client; responsibilities - e.g. periodically update status from the underlying technology (e.g. docker)
<fwereade> ericsnow, are the workers expected to, e.g., restart the processes if they fail?
<fwereade> ericsnow, or are they just observers?
<ericsnow> fwereade: not yet
<ericsnow> fwereade: for now just observing
<ericsnow> fwereade: later potentially starting and stopping them
<fwereade> ericsnow, cool, thanks
<ericsnow> fwereade: np
<fwereade> ericsnow, a thought re responsibilities, not sure if it's good
<fwereade> ericsnow, how hard would it be for one such worker to know when it was finished, itself, and return something like ErrUninstallMePlease?
<ericsnow> fwereade: I'll think about it (OTP)
<fwereade> ericsnow, I'm not sure that even covers all the machine-agent use cases tbh, it might just be a bad idea, but let me know if you think of anything
<ericsnow> fwereade: will do
<mup> Bug #1486297 opened: Action doesn't correctly translate unit name into tag if hyphen present <juju-core:New> <python-jujuclient:New> <https://launchpad.net/bugs/1486297>
<mup> Bug #1486297 changed: Action doesn't correctly translate unit name into tag if hyphen present <juju-core:New> <python-jujuclient:New> <https://launchpad.net/bugs/1486297>
<mup> Bug #1486297 opened: Action doesn't correctly translate unit name into tag if hyphen present <juju-core:New> <python-jujuclient:New> <https://launchpad.net/bugs/1486297>
<marcoceppi> alexisb: health checks, are those on the roadmap?
<alexisb> heh marcoceppi we were just chatting about that
 * marcoceppi mind melds
<mup> Bug #1486297 changed: Action doesn't correctly translate unit name into tag if hyphen present <juju-core:New> <python-jujuclient:New> <https://launchpad.net/bugs/1486297>
<fwereade> dimitern, I'm feeling dense, would you explain: I think we need some way to distinguish between the "fallback to env spaces constraint" and "explicitly clear spaces constraint" cases
<fwereade> dimitern, do we not need it; or do we do it but I just don't see it?
<dimitern> fwereade, I'll try at least :)
<marcoceppi> How do I tell juju what agent to bootstrap with?
<marcoceppi> I have 1.24.4 isntalled but it bootstraps 1.24.5 trying to validate a regression
<dimitern> fwereade, so explicitly empty values always override matching fallbacks (i.e. "mem=" overrides "" and "mem=4G"), but only when doing resolution from deployment to provisioning constraints
<dimitern> fwereade, that's what now happens after my changes
<dimitern> (soon to be available in master as well, when we merge net-cli)
<wwitzel3> ericsnow, natefinch: https://github.com/juju/charm/pull/143
<ericsnow> wwitzel3: ship-it
<natefinch> officially used loggo's per-package logging adjustments for the first time in 2 years: juju set-env logging-config="juju.worker.leadership=WARNING"
<natefinch> fwereade: any chance we were going to de-spam the leadership logging sometime soon?
<fwereade> natefinch, huh, I thought we had
<fwereade> natefinch, I thought iit was mostly at trace level
<fwereade> natefinch, hmm, maybe we didn't do tracker?
<natefinch> fwereade: yeah, the log lines I see are all from tracker
<fwereade> natefinch, damn, sorry
<natefinch> fwereade: not the end of the world.  Fixable via loggo (as long as you don't need to see anything under warning from leadership)
<fwereade> dimitern, ah, ok, and if we had "spaces=foo" and replaced it with "mem=4G" we'd get the fallback spaces; but "mem=4G spaces=" would ensure no spaces constraints? or have I completely confused myself?
<dimitern> fwereade, exactly right
<fwereade> dimitern, cool
<fwereade> dimitern, ok, then, in state -- how do we store the distinction between those two cases?
<fwereade> dimitern, we seem to have lost the pointers in that struct
<dimitern> fwereade, FWIW resolution was broken in a few places, e.g. adding a machine does resolution, SetConstraints on it before deployment doesn't and takes whatever you give it
<fwereade> dimitern, ahh, machine constraints weren't including env fallbacks?
<dimitern> fwereade, when set, but when added it worked as expected
<dimitern> fwereade, I *think* it's only important to store non-empty values (after doing resolution)
<fwereade> dimitern, sorry, lost again, how can you set constraints on a machine?
<dimitern> fwereade, before provisioning
<dimitern> fwereade, m.SetConstraints()
<fwereade> dimitern, can users do that?
<fwereade> dimitern, (other than when adding?)
<fwereade> dimitern, I think those values should just be coming from the resolved env+service constraints for the unit whose addition triggered machine addition
<dimitern> fwereade, I don't think so
<dimitern> fwereade, but it's still a bug
<fwereade> dimitern, but either way, when we're storing service constraints in state we shouldn't resolve them
<dimitern> fwereade, I agree, and I made it so it's definitely like this in both cases
<dimitern> fwereade, we're not
<dimitern> fwereade, we only resolve unit constraints when asked
<fwereade> dimitern, how can we do that correctly if we're throwing away the distinction between "fallback" and "clear" in the service constraints we store in mongo?
<dimitern> fwereade, let me look at the code
<fwereade> dimitern, np, sorry it's taken me so long to start looking
<dimitern> fwereade, ok, that's a good catch sir
<fwereade> dimitern, yay, my brain still works :)
<dimitern> fwereade, so we should store them when empty, at least for the services
<fwereade> dimitern, I think so, yeah
<dimitern> fwereade, good, it should be easy to fix
<fwereade> dimitern, and probably across the board, even if the distinction is academic once resolved
<fwereade> dimitern, cool
<TheMue> dimitern: just before riding home by bike, http://reviews.vapour.ws/r/2419/ contains the changes we talked about yesterday. hints regarding the testing of the finishedWorker are welcome. I'll take a look when I'm at home
<wwitzel3> cmars: ping
<mup> Bug #1486640 opened: Typos in help  <juju-core:New> <https://launchpad.net/bugs/1486640>
<dimitern> TheMue, sure, will have a look in a bit
<jogarret6204> just checking back in and seeing a team call here.. am I in wrong group for "general noob help"?  Sorry if so.  where is general help?
<lazyPower> jogarret6204: general noob help is in #juju :)
<lazyPower> #juju-dev is primarly for those hacking on juju core
<lazyPower> you're more than welcome to hang in both places though, we welcome all feedback
<jogarret6204> don't want to slow you down here, so I'll hit the other.  thanks LazyPower and natefinch.
<natefinch> jog: sorry, yeah, #juju is more the general help channel :)
<natefinch> jog: sorry, obv not meant for you
<sinzui> katco: dimitern : can either of you arrange fix for bug 1486675. I think the test needs more smarts, juju is fine
<mup> Bug #1486675: supportedSeriesWindowsSuite.TestSupportedSeries fails <blocker> <ci> <regression> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1486675>
 * perrito666 is connected though his phone and loving his isp
<perrito666> well creating a simple stream over 1/2 3g really makes me save on heating, the phone is enough for the whole office
<natefinch> heh
<natefinch> wow, this is a dumb error: ERROR environment destruction failed: destroying environment: container "nate-local-machine-1" is not yet created
<perrito666> well, that was actually possible
<perrito666> I think I recall davechen1y or thumper talking about a test that tried to reproduce that by being a race condition
<mup> Bug #1486166 changed: JES deploy fails <ci> <jes> <regression> <juju-core:Fix Released by thumper> <https://launchpad.net/bugs/1486166>
<mup> Bug #1486675 opened: supportedSeriesWindowsSuite.TestSupportedSeries fails <blocker> <ci> <regression> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1486675>
<katco> sinzui: dimitern's team is on bug-squad, so i'll leave it to him
<natefinch> perrito666: my point was rather, if I am destroying the environment, I don't care that there's a container that isn't created yet, that's just one less thing to tear down.
<dimitern> TheMue, you have a review
<dimitern> sinzui, katco, sure, let me have a look
<dimitern> sinzui, this looks like a fallout of a recent change I saw
<sinzui> dimitern: yes
<dimitern> sinzui, this one most likely https://github.com/juju/juju/pull/2981
<dimitern> bogdanteleaga, are you around?
<sinzui> yep, that is what I saw
<bogdanteleaga> dimitern, looking into it
<dimitern> sinzui, katco, so in these case what - is a revert in order?
<dimitern> bogdanteleaga, thanks!
<sinzui> dimitern: a revert will unblock. otherwise race to land a fix
<bogdanteleaga> dimitern, sinzui https://github.com/juju/juju/pull/3040
<bogdanteleaga> dimitern, sinzui I g2g now, merge it once it gets reviewed please
<sinzui> thank you bogdanteleaga
<dimitern> bogdanteleaga, awesome, ta!
<dimitern> sinzui, setting it to merge
<sinzui> you rock dimitern
 * dimitern wishes all bugs where like this :)
<natefinch> OMG.... just realized what the problem I've been fighting with all day..... printing a value out with %v was causing a panic from inside fmt somewhere
<perrito666> aghh gce is saying me I am not authenticated... only for one particular operation wth
<dimitern> sinzui, btw I'm not sure if you're monitoring feature branches for trends in failures like for the main branches, but if there is some data about "net-cli", which we're planning to merge tomorrow, will be awesome
<sinzui> dimitern: you cannot merge it because it has never passed http://reports.vapour.ws/releases#net-cli
<sinzui> dimitern: merge tip, When CI blesses it, we can merge it into master
<dimitern> sinzui, hmm that's useful to know
<dimitern> sinzui, we just did that today
<sinzui> dimitern: I shall try to force ci to retest net-cli.
<dimitern> sinzui, it's currently only 4 commits behind
 * sinzui is trying ton retest 1.24 today as well
<dimitern> sinzui, great, thanks! it will be nice to have some early feedback
<sinzui> dimitern: maybe this issues are fixed already https://bugs.launchpad.net/juju-core/net-cli
<dimitern> sinzui, I hope so, however the windows one is a bit worrying
<dimitern> sinzui, or you mean because it's gone from master?
<sinzui> dimitern: I hope the issue was really in master, and your merge fixed it
<dimitern> sinzui, I'll give it a try now as I'm changing that list command
<sinzui> alexisb: I think we have enough evidence to say Joyent's firewall changes did hurt Juju, and that deleting them when Juju destroys the environment fixes the issue: I want bug 1485781 fixes in 1.25 and 1.24 (maybe 1.22 if we ever plan a release)
<mup> Bug #1485781: Juju is unreliable on Joyent <joyent-provider> <reliability> <repeatability> <juju-core:Triaged> <juju-core 1.22:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1485781>
<katco> ericsnow: natefinch-afk: wwitzel3: sorry i missed the stand-up this morning. anything i can help with?
<ericsnow> katco: review http://reviews.vapour.ws/r/2405/ ?
<ericsnow> katco: (and don't sweat missing standup :)
<katco> ericsnow: tal
<alexisb> sinzui, thank you for getting the info on joyent and opening the bug
<alexisb> that is good stuff
<mup> Bug #1486712 opened: Race on uniter-hook-execution, prevents to resolve unit. <sts> <juju-core:New> <https://launchpad.net/bugs/1486712>
<katco> ericsnow: reviewed
<ericsnow> katco: thanks!
<mup> Bug #1474885 changed: juju deploy fails with ERROR EOF <local-provider> <precise> <juju-core:Fix Released> <https://launchpad.net/bugs/1474885>
<mup> Bug #1486749 opened: juju backups create should fail earlier for hosted environments <juju-core:In Progress by cherylj> <https://launchpad.net/bugs/1486749>
<mup> Bug #1486749 changed: juju backups create should fail earlier for hosted environments <juju-core:In Progress by cherylj> <https://launchpad.net/bugs/1486749>
<mup> Bug #1486749 opened: juju backups create should fail earlier for hosted environments <juju-core:In Progress by cherylj> <https://launchpad.net/bugs/1486749>
<davecheney> alexisb: ping
<menn0> waigani: would you mind taking a look at http://reviews.vapour.ws/r/2425/ pls?
<perrito666> ericsnow: still here?
<ericsnow> perrito666: barely
 * perrito666 sees ericsnow fading
<perrito666> ericsnow: go use gce with the fields instead of the json file, is it enough to just copy the values of the json?
<ericsnow> perrito666: pretty much
<perrito666> ericsnow: I believe the issue I told you about earlier might be because storage provisioner tries to do stuff in a machine and finds itself lacking these creds
<ericsnow> perrito666: the PK might not copy-and-paste quite right so you have to watch that
<ericsnow> perrito666: does it do more than make calls on the provider?
<perrito666> ericsnow: calls that require auth
<ericsnow> perrito666: the provider has all the auth it needs
<perrito666> the provider?
<ericsnow> perrito666: provider/gce/...
<perrito666> ericsnow: well I am getting auth errors from one of the machines
<perrito666> so :) something is wrong
<ericsnow> :)
<ericsnow> perrito666: you're adding new methods to the gceConnection interface (in environ.go), right?
<perrito666> ericsnow: yep
<ericsnow> perrito666: then I would definitely not expect auth issues :/
<perrito666> well it is only happening with one machine i think
<perrito666> that I added with add-machine
<ericsnow> perrito666: do you have to enable some extra permissions in the GCE developer console?
<perrito666> so I am looking it up
<ericsnow> (manually)
<perrito666> ericsnow: where is that stored on the server?
<ericsnow> perrito666: where is what stored?
<perrito666> ericsnow: the oauth token
#juju-dev 2015-08-20
<ericsnow> perrito666: it's buried away in the connection code in provider/gce/google
<perrito666> k tx :) ill dig it up
<ericsnow> perrito666: k
<ericsnow> perrito666: gotta run; catch me in the morning if you have more questions
<perrito666> tx a lot ericsnow
<ericsnow> perrito666: np, good luck :)
<marcoceppi> ericsnow: ping on https://github.com/juju/juju/pull/3044
<perrito666> marcoceppi: eric left a bit over an hour ago
<menn0> master is unblocked again people
<anastasiamac> menn0: tyvm :D
<mup> Bug #1486675 changed: supportedSeriesWindowsSuite.TestSupportedSeries fails <blocker> <ci> <regression> <unit-tests> <windows> <juju-core:Fix Released by bteleaga> <https://launchpad.net/bugs/1486675>
<mup> Bug #1486675 opened: supportedSeriesWindowsSuite.TestSupportedSeries fails <blocker> <ci> <regression> <unit-tests> <windows> <juju-core:Fix Released by bteleaga> <https://launchpad.net/bugs/1486675>
<mup> Bug #1486675 changed: supportedSeriesWindowsSuite.TestSupportedSeries fails <blocker> <ci> <regression> <unit-tests> <windows> <juju-core:Fix Released by bteleaga> <https://launchpad.net/bugs/1486675>
<dooferlad> dimitern / TheMue: please take a look: http://reviews.vapour.ws/r/2423/
<dimitern> dooferlad, sure, looking
<dimitern> dooferlad, so you went with returning an error with --networks given and these are the only tests that failed?
<dooferlad> dimitern: yeo
<dooferlad> yep
<dimitern> dooferlad, good!
 * dooferlad tries testing juju with go 1.5 for fun
<dimitern> dooferlad, reviewed
<dimitern> dooferlad, btw do you have a windows machine around that you can try running juju unit tests on?
<dooferlad> dimitern: not really. I have my gaming box, but I haven't got it set up for development
<dooferlad> dimitern: https://aws.amazon.com/windows/ ?
<dimitern> dooferlad, it's fine then, np
<dimitern> dooferlad, after the yesterday's sync up with master net-cli *almost* got blessed :)
<dimitern> dooferlad, a couple of tests remaining, one of which is windows specific and "native" to net-cli - bug 1469807
<mup> Bug #1469807: ListSuite.TestOutputFormats fails on Windows <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Incomplete> <juju-core net-cli:Triaged> <https://launchpad.net/bugs/1469807>
<dimitern> dooferlad, we need to sort this out before we can merge net-cli into master
<mattyw> cmars, tasdomas wallyworld https://github.com/wallyworld/juju/pull/29
<cmars> mattyw, tasdomas fwereade wallyworld PTAL, http://reviews.vapour.ws/r/2432/
<dooferlad> dimitern: are you sure you want me to remove the SupportsNetworking check in deploy.go?
<dooferlad> dimitern: do we make sure that networking is supported somewhere else?
<dimitern> dooferlad, since we'll be using constraints, it's fine to drop that
<dimitern> dooferlad, because constraints are not guaranteed to be supported and are verified at provisioning time anyway
<dooferlad> dimitern: thanks. Pushing changes now and will $$merge$$
<TheMue> dimitern: I yesterday fixed the comments for my latest branch, only go logical trouble with the test of the FinishedWorker. the fact, that the runner stops a worker after Wait() returned nil is tested elsewhere, so due to the simpleness of the FW we need only to assure that it implements worker (simple static assignment) and if Wait() returns nil
<TheMue> dimitern: so I would like to drop the one I added and create a new real finishedworker_test.go
 * dooferlad thinks that localServerSuite.TestNetworkInterfaces has a race
 * TheMue fetches his chequered flag
<dooferlad> dimitern: have you seen this? http://pastebin.ubuntu.com/12134188/
<dimitern> TheMue, let's discuss it during standup?
<dimitern> dooferlad, looking
<TheMue> dimitern: sure
<dimitern> dooferlad, yes, but it's hard to reproduce on my machine
<dooferlad> dimitern: I think I have seen the problem, but I wanted to check I wasn't duplicating work
<dimitern> dooferlad, there's a bug about it
<TheMue> dimitern: could you point me again to the test you mentioned?
<dimitern> TheMue, have a look how newWorkerStarter() is used
<TheMue> dimitern: do you mean newTestWorkerStarter()?
<dimitern> TheMue, yes, that's the one
<TheMue> dimitern: yeah, it works with the newTestWorkerStarter, which works with the testWorker which really has a goroutine in the backend. we don't have that.
<TheMue> dimitern: IMHO TestOneWorkerFinish() tests that any worker finishes if Wait() returns nil
<TheMue> dimitern: opposite to TestOneWorkerRestart()
<TheMue> dimitern: but here this is achieved by telling the goroutine to end with err == nil
<dimitern> TheMue, ok, then no need to test the finishedworker in runner_test?
<TheMue> dimitern: that's what I meant, yes. only one that says "yes, I'm a Worker" and "yes, I return nil on Wait()".
<dimitern> TheMue, just check newAddresser in cmd/jujud/agent/machine.go returns FinishedWorker{} when disabled?
<TheMue> dimitern: that's in the worker_test right now, but can see how to do it in machine_test too
<dimitern> TheMue, in machine_test, I'd suggest saving the original agent.NewAddresser value (from export_test), then patching it with another which calls it internally and asserts the result is finishedworker
<TheMue> dimitern: ok
<TheMue> all: core team meeting?
<dimitern> I'll skip it this time I'm afraid
<mattyw> wallyworld, axw is this basically what you wanted? https://github.com/wallyworld/juju/pull/31/files#diff-abee59b3dd781e907de49c2651b6a332R32
<perrito666> hey people I need to take my wife to work I cant make it to the meeting sorry
<dooferlad> dimitern: goamz has changed a lot :-| at least I found a real bug in juju (TestNetworkInterfaces relied on map ordering)
<dooferlad> dimitern: ah, go-amz, not goamz
<dimitern> dooferlad, yeah
<dimitern> :)
<dimitern> dooferlad, what's the ordering bug about?
<mup> Bug #1486965 opened: Destroying the leader sets the service status to terminated <juju-core:New> <https://launchpad.net/bugs/1486965>
<mup> Bug #1486965 changed: Destroying the leader sets the service status to terminated <juju-core:New> <https://launchpad.net/bugs/1486965>
<mup> Bug #1486965 opened: Destroying the leader sets the service status to terminated <juju-core:New> <https://launchpad.net/bugs/1486965>
<dooferlad> dimitern: that is https://github.com/juju/juju/pull/3052
<dooferlad> dimitern: no idea why I didn't create a pull request before lunch
<dooferlad> it was a map ordering problem
<dooferlad> https://github.com/go-amz/amz/pull/61 is about thread safety
<dooferlad> and now, windows!
<dimitern> dooferlad, awesome! looking
<dimitern> dooferlad, both approved and the goamz one merged
<dooferlad> dimitern: thanks!
<fwereade> tasdomas, cmars: I think we should talk about uniter availability again, I don't quite get how that last branch solves the problem
<tasdomas> cmars, fwereade sure
<wallyworld> fwereade: https://github.com/wallyworld/juju/pull/32
<wallyworld> axw: https://github.com/wallyworld/juju/pull/32
<fwereade> dooferlad, TheMue: dimitern is battered by storms and without power
<dooferlad> fwereade: thanks for the update. Yikes!
<perrito666> yet he manages to communicate with the guy in an island :p
<fwereade> perrito666, and whose own power is out again, the house is being rewired :)
<perrito666> lol
<mattyw> wallyworld, could you take a look at https://github.com/wallyworld/juju/pull/31 and let me know if it seems reasonable?
<mattyw> fwereade, you might be interested as well ^^
<frankban> ericsnow: when you ahev time, could you please take a look at https://github.com/juju/juju/pull/3035 ? it's a mp for a long-running branch that we are using to implement the embedded gui story. thanks!
<bac> hi i've got a juju-core pull request on github and reviewboard.  with a 'shipit' vote from davecheney.  does it require a second? when ready to merge, do you just press the 'shipit' button on reviewboard?
<abentley> sinzui: Want to talk about weekly test candidates?
<sinzui> abentley: indeed. have you been listening to my thoughts this morning?
<wwitzel3> ericsnow: ping
<abentley> sinzui: Err, not intentionally?
<sinzui> dooferlad: or anyone working on net-cli, ping who when bug 1469807 has a fix merged. I want to enure we test it
<mup> Bug #1469807: ListSuite.TestOutputFormats fails on Windows <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Incomplete> <juju-core net-cli:Triaged> <https://launchpad.net/bugs/1469807>
<dooferlad> sinzui: I am looking at it now. Will let you know
<dooferlad> TheMue: Could you take a look at https://github.com/juju/cmd/commit/4a5e8abce36462f89d92d28bd69b54c7845db818 please?
<dooferlad> TheMue: I pushed to the wrong branch, but I have tested that this works :-(
<TheMue> dooferlad: yep
<TheMue> dooferlad: a direct usage of target with the defer of it's closing isn't possible?
<dooferlad> TheMue: no, because the type of target doesn't have a Close
<dooferlad> but File does
<TheMue> dooferlad: ah, and the assigning is a kind of downcast, yes
<TheMue> dooferlad: ok
<katco> natefinch: standup?
<natefinch> katco: oops, coming
<ericsnow> frankban: yeah, I'll take a look
<frankban> ericsnow: ty!
<TheMue> Ah, my last strange tests are now working too, phew.
<mattyw> wallyworld, another one https://github.com/wallyworld/juju/pull/34
<TheMue> Grmpf, still an issue. Those two tests running together by pattern work, also isolated, but inside the whole suite they don't work.
<TheMue> Error is "connection is shut down" when using the api in the MachineSuite of jujud/agent/machine_test.go
<dooferlad> TheMue: http://reviews.vapour.ws/r/2437/
<TheMue> dooferlad: *click*
<dooferlad> Just a rev bump to bring in the change I just made to cmd/juju
<TheMue> dooferlad: done
<dooferlad> TheMue: thanks!
<TheMue> dooferlad: yw
<TheMue> dooferlad: I'm still looking for my strange test failures. normally one test shouldn't effect another one. my both work when called single or together, but with the whole suite one fails
<dooferlad> TheMue: timeouts?
<dooferlad> TheMue: races?
<TheMue> dooferlad: connection of the API
<TheMue> dooferlad: here I have an idea
<TheMue> dooferlad: I exchange a function variable, so this has to be done atomically
<ericsnow> cmars: you have a patch up to run the feature tests only upon request?
<cmars> ericsnow, correct. though, that will be set by default in jenkins
<cmars> ericsnow, so it is on request in your dev env
<ericsnow> cmars: a big +1 from me, but why the change from opt-out?
<ericsnow> cmars: BTW, this might have helped :/  http://reviews.vapour.ws/r/1647/
<cmars> ericsnow, unit tests are appropriate for development, integration tests are appropriate for CI
<ericsnow> cmars: totally agree; perhaps I'm assuming too much to think there was a rationale for opt-out originally
<ericsnow> cmars: PTAL @ my review on http://reviews.vapour.ws/r/2432/
<TheMue> dooferlad: seen the mail? do you know the branches?
<dooferlad> TheMue: just replied
<dooferlad> TheMue: no, there is nothing on his github
<TheMue> dooferlad: just seen your reply
<TheMue> ;)
<dooferlad> TheMue: I assume he is working on https://canonical.leankit.com/Boards/View/101652562/116729781 and maybe https://canonical.leankit.com/Boards/View/101652562/116652760
<TheMue> dooferlad: yeah, and I don't see branches for it in his account
<TheMue> dooferlad: but at least I found my failure, a misunderstanding of the helpers of the test suite
<marcoceppi> ericsnow: ping
<ericsnow> marcoceppi: hey
<marcoceppi> ericsnow: I had some questions on https://github.com/juju/juju/pull/3044
<ericsnow> marcoceppi: yep
<marcoceppi> ericsnow: > for clarification does this mean that if I have a script called `juju-foo` in my path `juju foo` will no longer work?
<katco> marcoceppi: no different concept of plugins
<marcoceppi> cool, thanks
<katco> marcoceppi: this has nothing to do with juju plug-ins
<marcoceppi> just making sure
 * marcoceppi puts all the red flags away
<dimitern> dooferlad, TheMue, o/
<ericsnow> marcoceppi: yeah, what katco said :)
<dooferlad> dimitern: o/
<natefinch> marcoceppi: we are disabling bundles and the config-changed hook, though
<dimitern> dooferlad, so I need to merge your changes from net-cli into my branch and push it
<ericsnow> marcoceppi: I should have made it more clear that this relates to the workload processes feature
<dooferlad> dimitern: ok, I will look out for a review request
<dooferlad> dimitern: will let you get on with it for now. Ping me if there is anything you want. I am idling/hangout out with the daughter (first steps!) until given somethine else to do.
<dimitern> dooferlad, cheers, I'll try to be quick
<dooferlad> (since it is EOD)
<dimitern> dooferlad, :) ok, sounds good
<dooferlad> I will see what I can do this evening once she is in bed.
<dooferlad> (if anything is needed)
<marcoceppi> natefinch: wait...what
<natefinch> marcoceppi: lol
<natefinch> marcoceppi: just messing with you
<natefinch> ericsnow: you around?
<ericsnow> natefinch: yep
<natefinch> ericsnow: have you tried the process removal code?  I call it and returns no errors and... then the process still exists.  Maybe I'm missing something somewhere
<ericsnow> natefinch: there are tests, but we certainly may have missed something between the layers
<natefinch> cannot for the life of me get log level = trace to work
<TheMue> dimitern: ah, back online. I'm idling here too for any task. latest branch is ready for 2nd review (after fighting with testing troubles this afternoon)
<dimitern> TheMue, cool, I'll have a look then
<wwitzel3> cmars: ping
<dimitern> I'm ready with mine as well - all tests pass, now need to merge
<TheMue> top
<dimitern> TheMue, reviewed
<TheMue> dimitern: thx, will take a look in a few moments
<natefinch> gah, rolling over a 401k is such an outdated process... mailing paperwork and then mailing paper checks... what is this, 1970?
 * natefinch is going to receive a check in the mail worth approximately half the cost of his house :/
<natefinch> old brokerage: "But we're sending it priority mail, so we can track it."  Fantastic.
<dimitern> dooferlad, TheMue, if any of you are still around I'd appreciate a quick look on this; it's quite big, but it's also well tested, including a feature test which confirms CLI works end-to-end to state for spaces list|create
<dooferlad> dimitern: can do
<dimitern> dooferlad, TheMue, tyvm, http://reviews.vapour.ws/r/2440/
<TheMue> *click*
<dooferlad> dimitern: I almost had several comments, but it looks good!
<dimitern> dooferlad, awesome!
<dimitern> dooferlad, I have a similar one for subnets CLI, but it's already very late
<dooferlad> dimitern: up to you. I can come back after I have eaten
<dimitern> dooferlad, well, if I'm not done within the next half hour with it, it won't happen, but there are a few tests still left
<TheMue> dimitern: reviewd
<dimitern> TheMue, thanks!
<katco> ericsnow: hey going to be just a little late
<ericsnow> katco: no worries
<mup> Bug #1487191 opened: TestAddOkay mismatch <ci> <regression> <unit-tests> <juju-core:Incomplete> <juju-core feature-proc-mgmt:Triaged> <https://launchpad.net/bugs/1487191>
<mup> Bug #1487191 changed: TestAddOkay mismatch <ci> <regression> <unit-tests> <juju-core:Incomplete> <juju-core feature-proc-mgmt:Triaged> <https://launchpad.net/bugs/1487191>
<bogdanteleaga> did the procedure for using development tools change? I can't seem to be able to bootstrap anymore, it complains about tools not found
<bogdanteleaga> http://paste.ubuntu.com/12137669/
<mup> Bug #1487191 opened: TestAddOkay mismatch <ci> <regression> <unit-tests> <juju-core:Incomplete> <juju-core feature-proc-mgmt:Triaged> <https://launchpad.net/bugs/1487191>
<cherylj> bogdanteleaga: have you tried using --upload-tools?
<bogdanteleaga> nope, I have them uploaded somewhere and I'm using agent-metadata-url
<bogdanteleaga> it was a permissions issue though I fixed it now
<cherylj> bogdanteleaga: ah, cool
<bogdanteleaga> though, it still seems like it defaults to devel even though I tried specifying releases
<cherylj> bogdanteleaga: iirc, wallyworld was doing some work around that, but I'm not sure where things landed.
<bogdanteleaga> cherylj, I found this https://github.com/juju/juju/pull/2985
<bogdanteleaga> I'll try again soon, it may have tried to read devel anyway for no reason and fail like that
<cherylj> anyone else having problems with reviewboard not creating review requests for PRs?
<waigani> cherylj: I just put one up, worked for me
<thumper> abentley: call?
<abentley> waigani, thumper: Sure.  I'm in https://plus.google.com/hangouts/_/canonical.com/jes-feature?authuser=1
<thumper> abentley: we are in the one defined in the calendar
<thumper> jes-testing
#juju-dev 2015-08-21
<menn0> thumper, waigani: I just did a "juju destroy-environment local --yes  --force" to 2 env JES both of which had new LXC containers coming up and the containers were left behind
<menn0> known limitation until the new env descruction stuff happens?
<waigani> hmmm...
<thumper> menn0: known...
<waigani> menn0: thanks for the heads up, I'll add it to my things to manually test
<thumper> menn0: I can explain...
<thumper> not surprising
<thumper> which is why we want to tell people to use system destroy
<thumper> and environment destroy
<thumper> not destroy-environment
<menn0> cool
<menn0> just making sure it was a known thing
<thumper> I'll explain when I'm not shoveling food in my mouth
<menn0> I can pretty much guess where the problem is so no rush
<menn0> thumper: i thought I just finished the state pool stuff but it seems i've made the state server panicky
<menn0> thumper: I think I know why though
<thumper> menn0: did you want to hop on a call to hear why destroy-environment --force is evil?
<menn0> thumper: sure
<thumper> menn0, waigani: https://plus.google.com/hangouts/_/canonical.com/evil
<waigani> thumper: http://reviews.vapour.ws/r/2250/, cheers
<sinzui> thumper: "destroy-environment --force" is  eveil  because it circumvents blocks, leaves resources behind, and users report bugs that juju did exactly as they told it to do?
<thumper> for JES it is evil because it doesn't contact the api server, and just destroys the system environment, leaving all hosted environments running but headless
 * thumper goes to make a coffee urgently
<sinzui> thumper: ouch, but --force also circumvents the api server, which is why my blocks can be ignored
<thumper> sinzui: yes, it doesn't even try
<thumper> sinzui: the new system destroy and environment destroy commands do try
<sinzui> thumper: hurray for sanity
<menn0> thumper: if you have time: http://reviews.vapour.ws/r/2445/
<menn0> thumper: i'm really happy with how this worked out
 * thumper looks
<thumper> menn0: mostly good, just concerned about reordering the defer calls
<menn0> thumper: hmmm I wasn't intending on reordering them
 * menn0 checks
<thumper> you moved them all into a single function that is deferred
<thumper> but then still call defer on each of the internals
<thumper> and you changed the order
<thumper> effectively reorering
<menn0> but defers run inside out
<thumper> either reorder and don't defer
<thumper> or defer
<thumper> yes
 * menn0 does a quick check on play
<thumper> happy to talk through
<menn0> thumper: I think I did the right thing: http://play.golang.org/p/fzFmAHa86q
<menn0> sorry
<menn0> hang on
<menn0> thumper: this one: http://play.golang.org/p/S4ehts8JxY
<thumper> menn0: but you did version three in http://play.golang.org/p/2d9vMgldxc
<menn0> thumper: I see what I screwed up
<menn0> all the defers in the new func shouldn't be there
<menn0> i'm an idiot
<thumper> right
<menn0> thumper: my eyes were not seeing the inner defers
<menn0> man I was trying to make that code clearer and unintentionally made it much worse
<thumper> :)
<menn0> thumper: i saw the bit about env.Life ! state.Alive too
<menn0> i'll fix that in a followup PR
<menn0> it's not really related to this change
<menn0> I think the condition should still be there, but as: else if env.Life() == state.Dead
<thumper> hmmm...
<thumper> nope
<thumper> because we may want to get logs from a dead environment
<thumper> prior to everything being deleted some time later
<menn0> thumper: hmm ok
<thumper> the dying / dead aspects should stop us adding things to the environment
<thumper> but we need to be able to view / introspect it while it is dying
<thumper> and after it is dead
 * thumper looks in the morgue
<menn0> thumper: but remmeber we've removed most of the assertions on env life
<menn0> thumper: I guess the critical places, like adding a machine still check
 * menn0 wishes you could edit past IRC comments :)
 * menn0 pushes the StatePool PR
<thumper> right
<thumper> dead / dying environments can't get new machines or services
<thumper> everything else needs a machine or service
<menn0> can't get storage either i'm pretty sure
<thumper> although in the coming future we should also check environment level network spaces and storage
 * thumper nods
 * thumper has to head into town briefly
<thumper> bbl to review waigani's branch
 * menn0 has to swap his UK driver's license for a NZ one
<thumper> gee menn0
<thumper> jumping the merge gun
<thumper> :)
 * thumper does a post hoc shipit
<menn0> thumper: sorry... missed there wasn't one there already
<menn0> thumper: if you have time: http://reviews.vapour.ws/r/2446/
<dimitern> dooferlad, TheMue, morning
<dooferlad> dimitern: hello!
<dimitern> dooferlad, TheMue, can you have a look at http://reviews.vapour.ws/r/2448/ please?
<dooferlad> dimitern: on it
<dimitern> dooferlad, cheers
<dooferlad> dimitern: we really need to get better at splitting cosmetic changes from functional changes :-|
<dooferlad> dimitern: or get a better diff viewer!
<dimitern> dooferlad, yeah, when there's no deadline to catch :)
<dooferlad> dimitern: :-)
<dimitern> dooferlad, also, if you can give it a spin on EC2 - bootstrap, space list, space create, subnet add, subnet list (with juju subnet -- debug .. if needed), subnet add with CIDR and/or ProviderId
<dimitern> dooferlad, --constraints spaces= does not cause errors in my tests, but obviously doesn't yet respect them
<dooferlad> dimitern, TheMue: hangout
<dimitern> dooferlad, omw, getting coffee
<TheMue> dooferlad: omw, got coffee
<dooferlad> dimitern: do you have a moment for EC2 testing?
<dimitern> dooferlad, sure
<dooferlad> dimitern: https://ide.c9.io/dooferlad/juju
<dooferlad> TheMue: ^^ if you want to join in
<TheMue> dooferlad: having phone, come in a few moments
<dimitern> dooferlad, omw
<TheMue> dimitern: btw, would you like a feature flag for the spaces support? like for address allocation? that way we could turn it on and off in tests with dummy.
<dimitern> TheMue, nope, we need another way to turn it on and off - e.g. having a "broken" method is one way
<TheMue> dimitern: ok
<dimitern> TheMue, or a flag settable by calling dummy.SetSupportsSpaces(bool)
<TheMue> dimitern: a direct method needs that you have direct access to the provider. often it is hidden deeply in our embedded tests. so I prefer the "broken" approach
<dimitern> TheMue, I'm fine with either - it's the same effort I think
<TheMue> dimitern: in first implementation yes, in tests I really liked the indirect approach - here with feature flag, but broken is fine too - during the tests
<mattyw> axw, https://github.com/juju/juju/pull/3072/files#diff-aa7f5128113461cdc1ee5c2335a09d6cR52
<frankban> ericsnow: I replied to your requests at http://reviews.vapour.ws/r/2416/ could you please take another look?
<sinzui> dimitern: master wasn't merged into net-cli. your branch failed. Master now has singnigicant divergence and we don't know if it is good.
<sinzui> dimitern: I am not sure what to recommend. test master and hope for a pass, then test the updated net-cli to get a pass?
<natefinch> finally got my code working... now to remove 100 printfs
<dimitern> sinzui, we merged master this morning
<dimitern> sinzui, and I have a PR which hopefully solves pingerSuite.TestAgentConnectionDelaysShutdownWithPing and MachineSuite.TestManageEnvironRunsInstancePoller
<sinzui> dimitern: excellent. master just started testing. I will make net-cli the most important branch to test next. I am setting CI to wait 8 hours if there is a failure to give people time to investigate and retry failures cause by subtrates
<dimitern> sinzui, awesome
<dimitern> dooferlad, TheMue, please have a look http://reviews.vapour.ws/r/2454/
<TheMue> dimitern: yep
<dooferlad> dimitern: on it
<dimitern> cheers guys!
<wwitzel3> ericsnow, natefinch, katco: hangout working for you guys?
<katco> was in one half an hour ago
<ericsnow> wwitzel3: yep
<katco> ericsnow: moonstone working fine
<natefinch> wwitzel3: make sure it's having you join with your canonical account
<TheMue> dimitern: reviewed
<dimitern> TheMue, ta!
<dooferlad> dimitern: done
<ericsnow> cmars: is this what I think it is?  http://reviews.vapour.ws/r/2453
<cmars> ericsnow, yep
<ericsnow> cmars: cool!
<cmars> :)
<dimitern> dooferlad, thanks!
<ericsnow> cmars: at least it got reviewed <wink>
<cmars> ericsnow, we're going to re-review it in greater detail next week. important to get it landed for now to get CI testing, but it won't be glossed over. all other PRs have been reviewed carefully
<ericsnow> cmars: oh, I don't doubt that :)
<ericsnow> cmars: thanks for getting it done!
<cmars> ericsnow, wasn't me, really. axw & wallyworld are the heroes
<ericsnow> cmars: ah, right; commit history FTW :)
<dimitern> TheMue, I'm not sure what's your last comment about? adding a comment in the patched function checking the result of newWorker?
<TheMue> dimitern: only adding comments I missed when creating this test. that FinishedWorker means the Addresser doesn't run while not FinishedWorker means it's the Addresser (sadly not direct testable)
<TheMue> dimitern: but it's only maintenance cosmetic, for later
<TheMue> dimitern: maybe we should add a worker.HasFinished(w worker.Worker) bool for it soon, when those tests get more. will keep it in mind for the Firewaller
<dimitern> TheMue, I see, ok I'll add comments about the meaning of a FinishedWorker as a result
<dimitern> dooferlad, about your comment
<dimitern> dooferlad, I don't believe the issue is related to clock time
<dooferlad> dimitern: oh, I thought you were increasing the timeout
<dimitern> dooferlad, yeah, but your comment caused me to think of a different approach
<dimitern> dooferlad, using attempt loops we don't have to sleep explicitly
<ericsnow> frankban: I'm reading through your responses to my review right now :)
<frankban> ericsnow: ty!
<niemeyer_> Hey all
<natefinch> howdy niemeyer_
<niemeyer_> Do we have any envs with MongoDB deployed on IPv6?
<natefinch> there was talk about that on the mailing list.  I think the answer is that it needs work, i.e. not really working currently, though I don't know the details.
<mgz> cmars: hey, are you around
<ericsnow> frankban: thanks for the thoughtful responses to my review comments
<ericsnow> frankban: I see where you're coming from now
<frankban> ericsnow: cool, thank you, reading your review
<ericsnow> frankban: BTW, good call on making the JSON field tag "yaml" :)
<ericsnow> frankban: my only remaining concern is about adding the "guiserver" methods to the "public" API facade
<frankban> ericsnow: yeah I was looking at that
<frankban> ericsnow: the GUI is really just a user client, and I am not sure about having a separate facade for the GUI because the GUI basically uses all the Client methods already (including the mega-watcher)
<frankban> ericsnow: actually I think the GUI is the most used juju API client after juju itself ;-)
<ericsnow> frankban: :)
<ericsnow> frankban: I think it would be good to at least have methods like GetBundleChanges on a separate facade
<frankban> ericsnow: so the GUI is JS connecting the ws, it needs the user to authenticate and all the endpoint it uses are in the client facade (excpet for the initial admin.login one)
<ericsnow> frankban: ah
<ericsnow> frankban: that's fine
<frankban> ericsnow: so basically like this the change is transparent, the GUI connects and sends requests over the ws, and GetBundleChanges is one request like many others (ServiceDeploy etc.)
<ericsnow> frankban: so one goal is to minimize changes to the GUI client code?
<frankban> ericsnow: yes, and we can adjust things later if the amount of GUI specific endpoints increases, but I don;t believe so
<ericsnow> frankban: using multiple facades on the same WS should be fine, no?
<ericsnow> frankban: k
<frankban> ericsnow: thanks
<ericsnow> frankban: I still think there should be a bit more discussion (on juju-dev) about where methods like GetBundleChanges should live in the API
<ericsnow> frankban: but moving the code to a more appropriate place later (if needed) wouldn't be hard
<frankban> ericsnow: I agree, but consider this is merging into a feature branch, I think we will be able to introduce these changes gradually, and rick_h_ is already working on this
<ericsnow> frankban: how urgent is it that GetBundleChanges land?  Could it wait a few days?
<ericsnow> frankban: oh, duh
<frankban> ericsnow: this is not landing in trunk
<ericsnow> frankban: feature branch :)
<ericsnow> frankban: I'll give you a ship-it but encourage you to at least ask jam and fwereade about it :)
<ericsnow> frankban: oh, and there are some missing tests
<frankban> ericsnow: thanks! yes this will be a long running branch and with rick_h_ we are already working on getting people aware of our goals and the changes to be made for the embedded gui story
<frankban> ericsnow: which ones? back to reading the review
<rick_h_> ericsnow: there's a spec with this work out to wallyworld and jam atm
<rick_h_> ericsnow: we're waiting on feedback but with holidays and such we're moving forward with the feature branch with the caveat that things will be tweaked based on feedback
<ericsnow> rick_h_: cool!
<rick_h_> ericsnow: as you mention, moving the endpoint isn't really going to make/break anything here.
<rick_h_> ericsnow: https://docs.google.com/document/d/1zZY0a-dby-VnrBXaoFARIN8Mj6pdXLeZzTXmH0mRUik/edit in case it helps at all or you're curious
<ericsnow> rick_h_: yep :)
<rick_h_> ericsnow: but we're definitely eager for core folks to know/udnersteand what's up and where things are headed as we want to make sure we're doing thigns by the book, that your folks feel comfy with fixes that might be required later/etc.
<rick_h_> ericsnow: so please feel free to push back on anything you would to another core team
<ericsnow> rick_h_: sounds good; I'll be sure to read through the spec
<rick_h_> ericsnow: ty, any feedback is welcome. I ran thumper through it last night
<ericsnow> rick_h_: you can count on me to speak up :)
<rick_h_> ericsnow: and we look forward to the other when they're back on duty
<rick_h_> :)
<frankban> ericsnow: GetBundleChanges is tested in apiserver/client/bundles_test.go
<ericsnow> frankban: hmm, we're trying to move away from full-stack tests like that
 * ericsnow looks at tests for other API methods (on server and client)
<frankban> ericsnow: thanks
<ericsnow> frankban: blech, our existing tests aren't very good examples :/
<frankban> :-/
<ericsnow> frankban: for an example of unit-ish tests see apiserver/backups and api/backups
<ericsnow> frankban: "feature" tests (effectively integration tests) live in the top-level featuretests package
<ericsnow> frankban: we're making an effort to improve our approach to testing in core which is why I'm bringing all this up
<frankban> ericsnow: sure, I made a note and I'll take a look on Monday
<ericsnow> frankban: coolio
<frankban> ericsnow: thanks and have a nice weekend
<ericsnow> frankban: you too and thanks for working on this!
<natefinch> ericsnow: this id/name thing is killing me
<ericsnow> natefinch: how so?
<natefinch> ericsnow: just the fact that sometimes we munge the name/id and sometimes we don't.... and so I have to know exactly when to munge the name/id to match what's stored in whichever map
<ericsnow> natefinch: we should always be identifying them by the full ID
<ericsnow> natefinch: where in particular are we not?
<natefinch> ericsnow: but what is the full ID and where do I get it from?  This is the problem I have with untrack.    So the current problem is getting the test infrastructure to do the right thing when I tell it to untrack.... which is actually kind of a problem in itself that we have logic in the test infrastructure
<natefinch> ericsnow: like stubAPIClient
<natefinch> ericsnow: https://github.com/natefinch/juju/blob/untrack-cmd/process/context/base_test.go#L287
<ericsnow> natefinch: the user should have passed in the full ID (or the name, by which we look up the full ID)
<natefinch> ericsnow: the user doesn't know what the full ID is, since that's a composed thing which we create, isn't it?  They know the name... I don't think they can possibly be expected to know anything else
<ericsnow> natefinch: that's fine; we just have to do the lookup for them
<natefinch> ericsnow: my point is... the fact that we're doing all this logic stuff in stubAPIClient is bad.  We're duplicating production code logic, and relying on this test harness to have the same implementation.
<ericsnow> natefinch: the API client shouldn't be doing any of that lookup
<ericsnow> natefinch: I thought we agreed that we would hold off on that part
<ericsnow> natefinch: i.e. the API client should always expect full IDs
<natefinch> ericsnow: I'm talking about this - https://github.com/natefinch/juju/blob/untrack-cmd/process/context/base_test.go#L201
<natefinch> ericsnow: that's logic that is duplicating what the real apiclient would be doing
<ericsnow> natefinch: setNew is just a convenience function to inject values into the API
<ericsnow> natefinch: it's not part of the stubbed API implementation
<natefinch> ericsnow: Ahh, I was missing a flush in there... I didn't expect that to make a difference, but it does.  Sorry.
<ericsnow> natefinch: np :)
<natefinch> ok, everything works... now to merge wwitzel3's rename commit :/
<natefinch> oh this is gonna suck
<natefinch> evidently git is not smart enough to merge <file renamed> with <file modified>   .... how is that not something it can just do automatically?
<natefinch> "oh hey, the old file just got moved, so I'll merge the changes into the new location" :/
<ericsnow> natefinch: yeah, I just went through that pain yesterday
<natefinch> google/stack overflow to the rescue....  git rebase feature-proc-mgmt -X rename-threshold=5%
<natefinch> ....mostly
<katco> can i get a rubber-stamp on http://reviews.vapour.ws/r/2455/? this exact patch has already been landed in 1.24
<perrito666> katco: done
<katco> perrito666: ty sir
<perrito666> since I reviewed the other one I believe you :)
<katco> haha
<perrito666> I must say, buying an electric toaster is a life changing decision, I no longer forget and burn my toasts
<perrito666> (which mostly happens while reading this channel or reading code of juju)
#juju-dev 2015-08-22
<mup> Bug #1487727 opened: TestAddresserWorkerStopsWhenAddressDeallocationNotSupported fails on precise and windows <blocker> <ci> <precise> <regression> <unit-tests> <windows> <juju-core:Incomplete> <juju-core net-cli:Triaged> <https://launchpad.net/bugs/1487727>
<mup> Bug #1487727 changed: TestAddresserWorkerStopsWhenAddressDeallocationNotSupported fails on precise and windows <blocker> <ci> <precise> <regression> <unit-tests> <windows> <juju-core:Incomplete> <juju-core net-cli:Triaged> <https://launchpad.net/bugs/1487727>
<mup> Bug #1487727 opened: TestAddresserWorkerStopsWhenAddressDeallocationNotSupported fails on precise and windows <blocker> <ci> <precise> <regression> <unit-tests> <windows> <juju-core:Incomplete> <juju-core net-cli:Triaged> <https://launchpad.net/bugs/1487727>
#juju-dev 2016-08-22
<menn0> veebers: i'm able to replicate what you're seeing... still digging
<menn0> veebers: ah ... now I see why
<menn0> veebers: users connected to the controller using "juju register" have a macroon, not a password
<anastasiamac> axw: wallyworld: this is on 1.25 with old azure... is may have troubles deploying to it and could become critical... is it easily fixed?  https://bugs.launchpad.net/juju-core/+bug/1459148
<mup> Bug #1459148: azure: juju can't create compute/network optimized instances <add-machine> <azure-provider> <canonical-is> <constraints> <feature> <juju-core:Triaged> <https://launchpad.net/bugs/1459148>
<menn0> veebers: the migration infrastructure doesn't support that
<menn0> damn...
<anastasiamac> axw: my pR has not landed. will consider a proper fix rather than a badage solution :D
<veebers> menn0: hmm, so should I be doing something different or are we attempting to test something that isn't completely there?
<menn0> veebers: you have uncovered a gaping hole in functionality :-/
<veebers> ah ok, that's a result at least :-)
<menn0> veebers: the way users are handled has changed a lot and migrations needs to be updated to work with it
<menn0> veebers: better that we find this out now rather than later :)
<veebers> heh yeah true that :-)
<menn0> veebers: that'll be me for the rest of the day then
<veebers> menn0: ok, let me know how you get on (or have a build you would like me to test.) I'll clean up this test and get it ready for review
<anastasiamac> axw: the problem was me - reverted tests. should be all good now \o/
<axw> anastasiamac: "The bug is reference in PR title. All you need to do is look it up."  <- I know it's there, and I know I can look it up. The usual pattern is to have it in the description, so I didn't see it to start with. Please do what's standard so it makes a reviewer's life easier.
<axw> anastasiamac: if you wanted to go the extra mile, it would be even better to use a clickable link. not everyone does that, but it's more helpful than just a bug number
<anastasiamac> axw: sure. however, each reviewer works differently. m happy to add bug link in pr - no need to stress on Monday :D
<axw> anastasiamac: thank you
<anastasiamac> axw: a rare pleasure. i'd rather have u smiling :)
<anastasiamac> or at least content \o/
<axw> anastasiamac wallyworld: assuming I can get chrome and my webcam to play nice, happy to chat whenever. what was it about? tools/image arch constraints?
<blahdeblah_> wallyworld: anastasiamac tells me you are awesome for fixing bugs quickly. So thanks. :-)
<anastasiamac> axw: wallyworld: yes and m not really stressing about but m happy to chat :)
<wallyworld> blahdeblah_: i didn't know it was a bug, the need for it came up in another context
<anastasiamac> wallyworld: u r too humble \o/
<anastasiamac> wallyworld: u've fixed it and now must accept the title of "awesome" ;)
<wallyworld> well if a 50 line bug fix is all it takes...
<axw> wallyworld: it's a trap, they're trying to get you to stop doing feature work
<wallyworld> i know right
<blahdeblah_> features schmeatures
<axw> menn0: sorry missed yor message before
<menn0> axw: no worries
<axw> menn0: well we have model status, shouldn't we be setting it when we're doing a migration?
<menn0> axw: is this in the output of the show-model command?
<axw> menn0: yes
<menn0> axw: yep, we should show something there
<menn0> axw: there will be a "migration" field under "status" with an info string
<menn0> axw: but the "current" field should reflect the status too
<menn0> axw: i'll be changing show-model soon so I'll make that change then as well
<axw> menn0: cool, sounds good
<menn0> axw: thanks, I just wanted to clarify that it you were talking about show-model and not something else
<axw> wallyworld: FYI I have a fix that works for speeding up add-machine in azure, basically by not synchronising on the provisioning state for VMs
<wallyworld> great ok
<axw> wallyworld: doing this means we'll regress on a recent change though, you won't be able to see why provisioning fails when it does
<wallyworld> yeah
<anastasiamac> axw: m looking at bug 1567747 as I am in this area right now. do u have more info? what provider? what commands, especially flags and options were run? pastebin is no longer accessbile...
<mup> Bug #1567747: "juju metadata generate-image" does not validate architectures <observability> <simplestreams> <ui> <juju-core:Triaged> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1567747>
<axw> wallyworld: so I'm looking at whether we can easily get that info into the instance status
<wallyworld> i think we'd be able to hopefully
<axw> wallyworld: yeah just not sure about how much work is involved, whether we should defer till next beta
<wallyworld> IMO having a fast deploy is more important
<axw> anastasiamac: sorry, I don't know - pretty sure that pastebin came from the user
<axw> anastasiamac: IIRC you just pass "-a x86_64" ?
<axw> anastasiamac: which should fail, because x86_64 is not an arch we understand
<anastasiamac> axw: k. tyvm
<thumper> axw: do you have any examples of charms / deployment args I could use on lxd to test migration of storage constraints?
<axw> thumper: here's a metadata.yaml that I use: http://paste.ubuntu.com/23077118/. create a charm dir with that, then use "juju deploy <path/to/charm> --storage filesystem=tmpfs,1G"
<thumper> axw: awesome, ta
<mup> Bug #1219582 changed: bootstrap lookups tools multiple times <canonistack> <performance> <simplestreams> <juju-core:Invalid> <https://launchpad.net/bugs/1219582>
<mup> Bug #1237378 changed: "juju bootstrap --upload-tools" fails sometimes for ssl-hostname-verification: false <bootstrap> <simplestreams> <upload-tools> <juju-core:Invalid> <https://launchpad.net/bugs/1237378>
<mup> Bug #1219582 opened: bootstrap lookups tools multiple times <canonistack> <performance> <simplestreams> <juju-core:Invalid> <https://launchpad.net/bugs/1219582>
<mup> Bug #1237378 opened: "juju bootstrap --upload-tools" fails sometimes for ssl-hostname-verification: false <bootstrap> <simplestreams> <upload-tools> <juju-core:Invalid> <https://launchpad.net/bugs/1237378>
<mup> Bug # changed: 1219582, 1237378, 1317680, 1345567, 1349691
<axw> wallyworld: it looks like it's going to be a PITA to get the error message, so I'm thinking I'll propose as is and then address after. sound ok?
<wallyworld> yup
<axw> wallyworld: there's another approach we can take to provisioning as well, which will speed everything up further - and then the error messages can be got at
<wallyworld> sgtm
<mup> Bug #1317680 opened: can't juju add machine when multiple images with same arch/release exist <amd64> <apport-bug> <cloud-installer> <kvm> <simplestreams> <streams> <trusty> <juju-core:Fix Released> <juju-core (Ubuntu):Invalid> <https://launchpad.net/bugs/1317680>
<mup> Bug #1345567 opened: problems bootstrapping aws due to s3 write errors when uploading tools <ec2-provider> <simplestreams> <upload-tools> <juju-core:Invalid> <https://launchpad.net/bugs/1345567>
<mup> Bug #1349691 opened: support aws govcloud <ec2-provider> <simplestreams> <streams> <juju-core:Fix Released> <https://launchpad.net/bugs/1349691>
<mup> Bug #1317680 changed: can't juju add machine when multiple images with same arch/release exist <amd64> <apport-bug> <cloud-installer> <kvm> <simplestreams> <streams> <trusty> <juju-core:Fix Released> <juju-core (Ubuntu):Invalid> <https://launchpad.net/bugs/1317680>
<mup> Bug #1345567 changed: problems bootstrapping aws due to s3 write errors when uploading tools <ec2-provider> <simplestreams> <upload-tools> <juju-core:Invalid> <https://launchpad.net/bugs/1345567>
<mup> Bug #1349691 changed: support aws govcloud <ec2-provider> <simplestreams> <streams> <juju-core:Fix Released> <https://launchpad.net/bugs/1349691>
<thumper> car's ready, relocating back home
<thumper> bbs
<mup> Bug #1210482 changed: use simplestreams info for endpoint labels instead of specifying endpoints <simplestreams> <juju-core:Fix Released> <https://launchpad.net/bugs/1210482>
<mup> Bug #1212173 changed: juju bootstrap searches for tools 2 times <bootstrap> <performance> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1212173>
<mup> Bug #1230367 changed: environs/simplestreams: could do with splitting <simplestreams> <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1230367>
<anastasiamac> wallyworld: plz update bug 1262175
<mup> Bug #1262175: debate drop --upload-tools flag <jujuqa> <simplestreams> <upload-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1262175>
<wallyworld> wow, that's an old one
<wallyworld> sadly the bug is still valid
<wallyworld> upload tools still exists and is used, it's just now implicit
<wallyworld> so the complaint in the bug report is still there
<wallyworld> and because people rely on it, and we now need it for snaps, it's probably here to stay
<wallyworld> will have to discuss a bit before marking as invalid
<anastasiamac> wallyworld: m not asking u to mark as invali. m asking u to update the bug with current view (like the work that you have done in the area last couple of weeks)
<wallyworld> i think we'll just mark it as invalid, i just need to check will william. the work that's been done doesn't change anything that the bug says is "evil", all the functionality is still present in juju; the only thing different is that upload-tools flag is gone
<wallyworld> i could update the bug title
<wallyworld> done, that is now more accurate
<veebers> where can I find docs on the --show-log argument?
<wallyworld> not sure that there is any
<veebers> oh ok, so --show-log puts all the logs to stdout? Nothing goes to stdin?
<anastasiamac> axw: wallyworld: in ur recent work with credentials revamp, did u encounter/fixed scenarios with # in pwds?
<anastasiamac> https://bugs.launchpad.net/juju-core/+bug/1567607
<mup> Bug #1567607: vsphere bootstrap fails when environments.yaml has a password with the # character <bootstrap> <cdo-qa> <vsphere> <juju-core:Triaged> <https://launchpad.net/bugs/1567607>
<axw> anastasiamac: no
<wallyworld> ino from me too
<wallyworld> didn't realise we had that issue
<anastasiamac> axw: wallyworld: k. thnx
<wallyworld> veebers: --show-log pritns to stdout what would normally not be seen because it goes to a log sink rather than stdou
<thumper> menn0: back, and in hangout
<menn0> thumper: ok, hang on
<veebers> wallyworld: with --show-log used are errors still spat out to stderr or is everything now out to stdout via the logs (which include normal and error messages etc.)
<wallyworld> errors should still go to stderr
<veebers> (I of course meant stderr not stdin beforehand :-\)
<wallyworld> the only change IIANM is that we show additonal output
<mup> Bug #1400971 changed: all agents except machine agent logging "INFO juju.worker.upgrader upgrader.go:121 desired tool version: 1.18.4.3" every 15 minutes following
<mup> "juju upgrade-juju" <prodstack> <simplestreams> <upgrade-juju> <upload-tools> <juju-core:Invalid> <juju-core 1.18:Won't Fix> <https://launchpad.net/bugs/1400971>
<mup> Bug #1457068 changed: bootstrap failed, no tools available <simplestreams> <ui> <juju-core:Won't Fix> <https://launchpad.net/bugs/1457068>
<veebers> wallyworld: cool thanks
<veebers> wallyworld: seems --show-log outputs the logs to stderr, is this expected behaviour? (i.e. juju --show-log status --help 2>>out.err 1>>out.std && cat out.err)
<wallyworld> yes
<wallyworld> you don't want to sully stdout with it
<veebers> ack, fair enough
<wallyworld> people might need to parse stdout yaml or whatever
<veebers> yeah that's a very good ponint
<mup> Bug #1242476 changed: Bootstrap fails with --upload-tools on private openstack cloud <bootstrap> <canonical-is> <canonical-webops> <openstack-provider> <simplestreams> <upload-tools> <juju-core:Fix Released> <https://launchpad.net/bugs/1242476>
<mup> Bug #1242476 opened: Bootstrap fails with --upload-tools on private openstack cloud <bootstrap> <canonical-is> <canonical-webops> <openstack-provider> <simplestreams> <upload-tools> <juju-core:Fix Released> <https://launchpad.net/bugs/1242476>
<thumper> veebers: logs generally go to stderr, except when you use the command debug-log, when they go to stdout
<veebers> thumper: ack thanks.
<mup> Bug #1242476 changed: Bootstrap fails with --upload-tools on private openstack cloud <bootstrap> <canonical-is> <canonical-webops> <openstack-provider> <simplestreams> <upload-tools> <juju-core:Fix Released> <https://launchpad.net/bugs/1242476>
<wallyworld> thumper: i got 2 PRs up, you able to do your OCR duties :-)
<thumper> yep
<wallyworld> \o/
<thumper> FFS
<thumper> grr
<thumper> $ juju dump-model
<thumper> ERROR facade "ModelManager" not supported for model API connection (not supported)
<thumper> someone broke dump-model
<thumper> why limit it?
<thumper> FFS
<axw> thumper: can you please review http://reviews.vapour.ws/r/5492/ if you haven't already
<axw> if you aren't*
<thumper> ack
<wallyworld> thumper: thanks for reviews
<wallyworld> thumper: rog made that change, they need to separate controller vs model facades for jaas
<wallyworld> it was explained in an email last week
<wallyworld> or the week before maybe
<wallyworld> ironically, it also broke their gui and metrics
<thumper> I recall it coming up
<thumper> but just frustrated
<veebers> thumper: if you file a bug for the dump-model issue you just saw can you tag it EDA and link me to the bug please?
<thumper> EDA?
<veebers> thumper: Escaped Defect Analysis. Means we'll (qa) have a meeting discussing bugs/defects/issues that could have or should have been caught by testing
<veebers> i.e why did this get through the net
<mup> Bug #1219123 changed: Azure provider: Juju unable to locate any image when "image-stream: released" is set (doesn't validate content of image-stream) <azure-provider> <papercut> <juju-core:Invalid> <https://launchpad.net/bugs/1219123>
<anastasiamac> thumper: m not sure why this failed to land... https://github.com/juju/juju/pull/5289 shall i try again?
<thumper> anastasiamac: no, it needs a tweak
<thumper> and I need to ask sinzui what the tweak is
<anastasiamac> thumper: \o/
<anastasiamac> thumper: there are also 2 Jesse's MADE PRs.. they seem to want a rebase :)
<thumper> yes, I know
<mup> Bug #1588897 changed: Unable to kill or destroy the lxd controller <destroy-controller> <lxd-provider> <juju-core:Expired> <https://launchpad.net/bugs/1588897>
<mup> Bug #1591939 changed: juju failed unmarshal the /server/{server_id} api  response body <juju-core:Expired> <https://launchpad.net/bugs/1591939>
<mup> Bug #1593303 changed: Google Compute Engine provider often reports wrong IP address as the public address <juju-core:Expired> <https://launchpad.net/bugs/1593303>
<voidspace> has anyone managed to go back to lxd 2.0.0 on Wily after 2.1 landed in the stable PPA?
<babbageclunk> Is anyone working on bug 1614559? I've got a fix for it if not, but I haven't proposed it because I still can't bootstrap.
<mup> Bug #1614559: Juju rejects lxd 2.1 <bootstrap> <ci> <jujuqa> <lxd> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614559>
<babbageclunk> (So I guess not totally a fix)
<babbageclunk> I get "ERROR failed to bootstrap model: cannot start bootstrap instance: unable to get LXD image for ubuntu-xenial: The requested image couldn't be found."
<babbageclunk> Which is weird because I can lxc launch a xenial instance fine.
<babbageclunk> dimitern: ^^
<dimitern> babbageclunk: I'm looking at the code to see where's that version check
<dimitern> babbageclunk: do you know how to get lxd 2.1 ?
<babbageclunk> dimitern: It was just on this loaner laptop I'm using.
<babbageclunk> I think you can just update?
<dimitern> babbageclunk: is it running yakkety?
<dimitern> babbageclunk: nope, on xenial at least I see these options: launch a xenial instance fine.
<dimitern> <babbageclunk> dimitern: ^^
<dimitern> [#juju-dev] oops not that
<dimitern> babbageclunk: http://paste.ubuntu.com/23077711/
<babbageclunk> dimitern: no, it's xenial
<dimitern> babbageclunk: ok I did `sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable` and now I see lxd 2.1 as available
<babbageclunk> dimitern: I think this is using the lxd ppa
<babbageclunk> dimitern: yeah, that would do it.
<babbageclunk> dimitern: https://github.com/juju/juju/compare/master...babbageclunk:lxd-version?expand=1
<babbageclunk> dimitern: That's where it does the version check, but I don't think my error's around there.
<dimitern> babbageclunk: I'll try that, but I don't think this is the real fix
<dimitern> babbageclunk: we should be checking the lxd api version to be 1.0, not the lxc cli version
<babbageclunk> dimitern: Well, it fixes the bad logic in the version checking.
<babbageclunk> dimitern: oh, sure.
<babbageclunk> dimitern: really I just hacked something up so I can bootstrap.
<babbageclunk> dimitern: (but I can't).
<dimitern> babbageclunk: sure :)
<babbageclunk> dimitern, fwereade: could you have a look at this? https://github.com/juju/juju/pull/6042
<dimitern> babbageclunk: to just fix bootstrapping, why not try to comment the if !isSupportedLxdVersion() check in tools/lxdclient/client.go ?
<babbageclunk> dimitern: I did better than that - I made the check right for 2.1. (Although checking the API version is better still.) But then something else still fails.
<axw> dimitern: hey, don't suppose you had any time to look at the neutron APIs?
<dimitern> axw: not yet :/ when do we need to know/reply about that?
<axw> dimitern: (completely unrelated) in general, do you know if cloud-init should be writing /etc/network/interfaces.d/50-cloud-init.cfg with all NICs of a cloud VM? I've created a VM in Azure with 2 NICs, and it's only got a stanza for eth0
<axw> dimitern: ASAP, but if you don't have time I can have a stab
<dimitern> axw: we're taking the defaults for any cloud apart from maas - there we disable cloud-init's /e/n/i generation explicitly, so we don't get the fallback 50-cloud-init.cfg to mess up our statically configured /e/n/i
<dimitern> axw: If you can, I'd appreciate it! I'll can have a look if something else could be needed for spaces support
<axw> dimitern: yeah, just wondering if you know what I should expect from cloud-init's default behaviour. can't find any docs on this bit
<axw> dimitern: ok, I was specifically asking about what's needed for spaces though :)
<axw> dimitern: I'll have a look tomorrow
<dimitern> axw: the default behaviour is to get a fallback dhcp eth0 from cl-in
<axw> dimitern: only eth0, even if there's multiple NICs?
<dimitern> axw: for spaces we need basically 2 things at minimum - listing subnets of a network, and deploying to a subnet
<dimitern> axw: yeah - it's "designed" to be the fallback case when we don't known anything else
<babbageclunk> dimitern, fwereade: also this? http://reviews.vapour.ws/r/5495/
<dimitern> babbageclunk: I managed to bootstrap with lxd 2.1 and this quick-n-dirty patch: http://paste.ubuntu.com/23077748/
<dimitern> babbageclunk: looking
<fwereade> babbageclunk, former LGTM
<natefinch> jam: good morning
<voidspace> anyway to get reviewboard to notice PRs created when it was down?
<voidspace> short of re-creating the PR
<jam> hi natefinch. brt
<jam> natefinch: https://hangouts.google.com/hangouts/_/canonical.com/john-meinel-nat
<natefinch> jam: cool.  https://hangouts.google.com/hangouts/_/canonical.com/moonstone?authuser=1
<natefinch> ha
<natefinch> I'll go to yours
<babbageclunk> fwereade: thanks!
<jam> tag
<dimitern> babbageclunk: LGTM on your ealier PR as well
<jam> ok. I'm in mine. I'll wait a bit longer.
<babbageclunk> dimitern: Also thanks! Hmm - I wonder what's going on with my lxd then?
<dimitern> babbageclunk: what do you get as errors?
<voidspace> lxd being screwed is a real downer
<dimitern> babbageclunk: ok, sorry - my bootstrap appears to be still going and logging errors in c-i-output.log
<babbageclunk> dimitern: I get "ERROR failed to bootstrap model: cannot start bootstrap instance: unable to get LXD image for ubuntu-xenial: The requested image couldn't be found."
<voidspace> and in trying to revert to lxd 2.0 in xenial I've managed to screw lxd totally it seems
<babbageclunk> voidspace: oh no
<dimitern> babbageclunk: ah, that's different - try lxc image copy ubuntu:16.04 --alias ubuntu-xenial local:
<voidspace> babbageclunk: heh, yeah
<voidspace> babbageclunk: trying a purge and re-install of 2.1 to at least get back to a "just juju and lxd is screwed" situation
<babbageclunk> dimitern: ahh, thanks. Why haven't I needed to do that before?
<voidspace> restarting
<voidspace> o/
<fwereade> babbageclunk, I'm a bit twitchy about AllMachineRemovals being []model => []machine (rather than [][]machine -- I think of them as an array of requests). did my brain completely skate over that in the apiserver review?
<dimitern> babbageclunk: not sure - but I've seen this months ago and that's how I fixed it (it was needed for trusty as well)
<babbageclunk> fwereade: I think it did, yeah. :( I can see why you say that now though.
<dimitern> babbageclunk: the errors preventing my lxd bootstrap were due to "no space left on device" for my zfs pool
<mup> Bug #1615552 opened: Juju 2: Cannot add new key to default model config. <juju-core:New> <https://launchpad.net/bugs/1615552>
<babbageclunk> dimitern: ha, yeah I've seen that before.
<babbageclunk> dimitern: thanks, seems to be working now.
<voidspace> yay, back to "normally screwed" instead of termially screwed
<babbageclunk> fwereade: By which I mean I don't think you mentioned that (I'll check now). I can change it though.
<fwereade> babbageclunk, sorry :(
<dimitern> katco: are you working on bug 1614559 ?
<mup> Bug #1614559: Juju rejects lxd 2.1 <bootstrap> <ci> <jujuqa> <lxd> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1614559>
<dimitern> katco: the card's assigned to you, but the bug to rick_h_ ..
<dimitern> katco: figured out how to fix it, and I'll take it over, if you don't mind
<voidspace> dimitern: katco talked about it at standup on Friday - not sure if she's started it though
<voidspace> dimitern: ah, hah
<voidspace> dimitern: I doubt she'll mind you doing it
<dimitern> ;) ok
<voidspace> dimitern: you might even finish before I succeed in deploying a bundle to aws!! :-) (slooooow)
<dimitern> basically we need to bump the lxd shared library version we're using to stable-2.0 and check the APIVersion field of server status, not its binary version
<dimitern> voidspace: ;) let's see
<voidspace> dimitern: yep
<voidspace> dimitern: rick_h_ said he wanted an option (like a force option) to attempt to use a more recent version of the api than the one we expect
<voidspace> dimitern: current api version is 1.0 I believe
<dimitern> voidspace: that would've been ok, if we actually checked the api version :)
<voidspace> dimitern: we need to work with lxd 2.0 still though as well as 2.1
<voidspace> sure
<mup> Bug #1615552 changed: Juju 2: Cannot add new key to default model config. <juju-core:New> <https://launchpad.net/bugs/1615552>
<mup> Bug #1615552 opened: Juju 2: Cannot add new key to default model config. <juju-core:New> <https://launchpad.net/bugs/1615552>
<babbageclunk> dimitern: now when I bootstrap jujud won't start in the container - it says unable to connect to: 10.82.88.1:8443 - any ideas?
<dimitern> babbageclunk: here's my fix - I'd appreciate a review and a QA on it - see if it fixes the issue?
<dimitern> voidspace: ^^
<dimitern> http://reviews.vapour.ws/r/5496/
<voidspace> dimitern: looking
<babbageclunk> dimitern: also looking
<dimitern> thanks guys!
<voidspace> dimitern: hmmm... I can't checkout your branch
<voidspace> dimitern: I can checkout your master but not lp-1614559-lxd-api-version
<voidspace> weird
<dimitern> voidspace: do you have my remote added?
<voidspace> dimitern: yep :-)
<dimitern> voidspace: try `git remote -v update`
<dimitern> voidspace: and then checkout
<voidspace> dimitern: I think I just need a fetch --all
<voidspace> yep
<voidspace> doh
<voidspace> sorry
<dimitern> voidspace: ;) np
<voidspace> dimitern: works for me
<frobware> dimitern: I commented on your PR. the changes break packaging.
<voidspace> dimitern: just trying again as I didn't specify build-agent, however that doesn't matter as it works
<frobware> dimitern: I know this as I had to revert my changes that used the shared.Devices form of Init()
<dimitern> voidspace: cheers!
<dimitern> frobware: can you clarify please?
<voidspace> someone with lxd 2.0 should check it still works with this as well
<frobware> dimitern: HO?
<dimitern> frobware: ok, joining standup now
<frobware> dimitern: http://pastebin.ubuntu.com/23078034/
<babbageclunk> dimitern: That works for me but then I get the same error - unable to connect to 10.82.88.1:8443
<voidspace> frobware: have you created a bot to add "in-progress" tags to PRs?
<frobware> voidspace: nope
<frobware> voidspace: were you expecting me to?
<voidspace> frobware: https://github.com/juju/juju/pull/6055
<voidspace> frobware: an in progress label appeared within seconds of me creating the PR!
<mup> Bug #1615580 opened: logging emits inappropriate ANSI escape codes on non-ANSI terminal <juju-core:New for thumper> <https://launchpad.net/bugs/1615580>
<babbageclunk> fwereade: so, I'll make AllMachineRemovals return a params.EntitiesResults, which has Results which is []params.Entities?
<babbageclunk> fwereade: does that mean that WatchMachineRemovals should also return a NotifyWatchResults (instead of a NotifyWatchResult as it does now) for the same reason?
<babbageclunk> fwereade: Hmm, I guess that doesn't actually need to - it can be one watcher which notifies when any of the things change, right?
<dimitern> babbageclunk: are you sure lxd is listening on that address? lxc config show ?
<babbageclunk> dimitern: That just shows "core.https_address: '[::]'"
<babbageclunk> dimitern: does that mean it's only listening on ipv6?
<dimitern> babbageclunk: and what's your lxd-bridge config? /etc/default/lxd-bridge
<dimitern> babbageclunk: no, it should listen on both
<menn0> frobware: what's with the "in progress" label?
<frobware> menn0: why is everybody asking me this?
<frobware> menn0, voidspace: there's a disconnect here. I don't know anything about this label.
<babbageclunk> frobware: Why'd you tag my PR in-progress?
<frobware> babbageclunk: ^^
<babbageclunk> dimitern: http://paste.ubuntu.com/23078076/
<menn0> frobware: I just created a new PR and within seconds you apparently added a label to the PR. looks like something automated that's using your credentials or something.
<babbageclunk> frobware: :)
<babbageclunk> frobware's been haxxored.
<menn0> frobware: see here: https://github.com/juju/juju/pull/6056
<frobware> menn0: ah, crap. I have an idea.
 * menn0 hears a distant penny dropping
<frobware> menn0, voidspace, babbageclunk: apologies. I was playing with waffle.io this morning, took the dog for a walk, then forget I had integrated that into my github account. oops.
 * babbageclunk can't work out why all these brooms keep bringing more water in!
<dimitern> babbageclunk: for good measure, I'd re-run sudo dpkg-reconfigure -p medium lxd
<dimitern> babbageclunk: and just take the current settings
<menn0> frobware: that'll be it then :)
<frobware> menn0: yep, sorry for the noise. :(
<dimitern> babbageclunk: that seems to help sometimes with partially screwed up lxd bridge config
<menn0> frobware: no harm done
<frobware> menn0: I revoked access to juju/juju on my account for waffle.io
<babbageclunk> dimitern: Aha, that's something - at the end of the reconfigure there's an error restarting lxd
<dimitern> babbageclunk: there you go :) what's the error?
<babbageclunk> dimitern: http://paste.ubuntu.com/23078092/
<babbageclunk> dimitern: might be time for a reboot
<dimitern> babbageclunk: do you have srw-rw----  1 root lxd            0 Aug 22 10:15 unix.socket in /var/lib/lxd ?
<babbageclunk> dimitern: yup
<dimitern> babbageclunk: lxc info ?
<fwereade> babbageclunk, hell sorry
<babbageclunk> dimitern: connection refused - presumably that's because it failed to restart after the reconfigure.
<fwereade> babbageclunk, I think both really ought to be multiple, shouldn't they?
<dimitern> babbageclunk: anything in /var/log/syslog re lxd-bridge ?
<babbageclunk> fwereade: I could see it either way in terms of the watcher.
<fwereade> babbageclunk, I would follow the common.AgentEntityWatcher model, I think
<babbageclunk> dimitern: lots of "lxd-bridge.service: Start request repeated too quickly." - scrolling back to see if there's a different error at the start.
<fwereade> babbageclunk, at least NotifyWatchResults already exists
<babbageclunk> fwereade: ok - makes sense. It all ends up the same on the client side, for now.
<babbageclunk> fwereade: True. :)
<fwereade> babbageclunk, cheers
<babbageclunk> dimitern: Lots of this: http://paste.ubuntu.com/23078109/
<dimitern> babbageclunk: I see - network manager gets in the way
<dimitern> babbageclunk: or maybe not .. still looking
<dimitern> babbageclunk: what's your `sudo iptables-save` content?
<dimitern> babbageclunk: it looks like iptables rules lxd-bridge tries to add are duplicated or clash somehow
<babbageclunk> dimitern: http://paste.ubuntu.com/23078114/
<dimitern> babbageclunk: ufw it is then!
<babbageclunk> dimitern: huh>
<babbageclunk> ?
 * babbageclunk is lost.
<dimitern> babbageclunk: ufw is the so-called "uncomplicated firewall"
<dimitern> babbageclunk: with seems to be eager to take over most of your iptables with custom rules for chains that are missing (e.g. ufw-before-forward)
<babbageclunk> dimitern: So it's got some config that's stopping lxd-bridge from listening on port 8443?
<dimitern> babbageclunk: try this - set ENABLED=no in /etc/ufw/ufw.conf and then systemctl restart ufw.service
<dimitern> babbageclunk: it's trying to firewall off most things, including lxd-bridge
<dimitern> babbageclunk: you can either configure ufw to allow specific traffic, or (better IMO) just disable ufw
<dimitern> babbageclunk: sorry, so the ufw custom chains are there, but the default policy seems to be DROP, not ACCEPT - hence the issues, disabling ufw should clean up the output of iptables-save and let you lxd-bridge to start ok
<babbageclunk> dimitern: ok, ufw is now "active (exited)"
<dimitern> babbageclunk: good - mine shows the same
<babbageclunk> dimitern: but restarting lxd-containers still isn't working. Hang on, checking syslog
<dimitern> babbageclunk: restart lxd-bridge.service instead
<dimitern> babbageclunk: lxd-containers seems to rely on the former
<babbageclunk> dimitern: ah, ok
<babbageclunk> dimitern: No, still see lots of "Bad rule (does a matching rule exist in that chain?)." in `systemctl status lxd-bridge.service`
<babbageclunk> dimitern: to be precise: http://paste.ubuntu.com/23078151/
<dimitern> babbageclunk: reviewed your second PR btw
<babbageclunk> dimitern: Thanks
<dimitern> babbageclunk: I'd try reconfiguring lxd again, but picking another subnet, e.g. 10.0.40.0/24 - it might be getting confused by the IP range you used
<babbageclunk> dimitern: Is it possible that the rules ufw set up are still lurking in iptables?
<babbageclunk> dimitern: ok, will try that.
<dimitern> babbageclunk: grep sudo iptables-save for ufw
<babbageclunk> dimitern: yeah, still lots of them
<babbageclunk> dimitern: reconfiguring didn't help
<dimitern> babbageclunk: so it ufw didn't remove its rules :/ I guess a reboot should fix it
<babbageclunk> dimitern: w00t!
<babbageclunk> dimitern: see you in a bit. Thanks for all the help!
<dimitern> np, hope it helps ;)
 * dimitern steps out for ~30m
 * babbageclunk goes to grab some lunch and post some shorts.
<natefinch> fwereade: an idea I had on friday when it seemed likely we were compiling with the wrong version of go in some places in CI: http://reviews.vapour.ws/r/5499/diff/#
<fwereade> natefinch, nice! LGTM
<natefinch> fwereade: :D
 * dimitern is back
<dimitern> mgz: ping
<rick_h_> dimitern: yea, the bug is meant to be grabbed from me to the dev working on it
<rick_h_> dimitern: that just indicates I've claimed the bug for my team vs alexis's
<dimitern> rick_h_: I see - good to know, ok :)
<dimitern> rick_h_: it was easy to fix, but frobware has some concerns mgz raised earlier re packaging
<rick_h_> dimitern: yea, you'll see a bunch of bugs assigned to me, and they line up with the cards in the todo lane
<rick_h_> dimitern: how so? is this in the backlog?
<mup> Bug #1615601 opened: 2.0 beta 14 + openstack: generated hostnames exceed 255 byte limit <juju-core:New> <https://launchpad.net/bugs/1615601>
<dimitern> rick_h_: no, from a chat they had in london, as frobware changed something similar to my fix
<rick_h_> dimitern: oh interesting. Ok, let me know if we want to talk it through or something then
<frobware> rick_h_, dimitern: I don't believe we should land the change until we have chatted with mgz
<dimitern> rick_h_: let's raise it up at standup I guess
<rick_h_> frobware: rgr
<rick_h_> mgz: ping around?
<rick_h_> frobware: dimitern you two have time to chat on networking/Y?
<frobware> rick_h_: yep, would appreciate doing so now so I can go grab some lunch....
<rick_h_> frobware: k, meet you in the standup room
<dimitern> rick_h_: omw as well
<mup> Bug #1615552 changed: Juju 2: Cannot add new key to default model config. <juju-core:Invalid> <https://launchpad.net/bugs/1615552>
<frobware> rick_h_, dimitern: for reference https://bugs.launchpad.net/juju-core/+bug/1597342 regarding LXD deps bump
<mup> Bug #1597342: Juju 2 is using a LXD API that is not in the released version of LXD 2.0 <juju-core:Fix Released by frobware> <https://launchpad.net/bugs/1597342>
<frobware> rick_h_, dimitern: and the PR - https://github.com/juju/juju/pull/5769
<dimitern> frobware: looking
<babbageclunk> No alexisb today?
<babbageclunk> I thought she was back from "horsing around".
<anastasiamac> babbageclunk: it may be still a bit early for her :)
<babbageclunk> anastasiamac: That's probably why she normally leaves her camera off.
<anastasiamac> babbageclunk: maybe :D
<voidspace> fwereade: ping
<voidspace> fwereade: will you have some bandwidth tomorrow to talk about bug 1534103
<mup> Bug #1534103: "unknown operation kind run-action" (1.26alpha3) <2.0-count> <actions> <sts> <juju-core:Triaged by rharding> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1534103>
<voidspace> fwereade: IRC will be fine if you're still badwidth-ly challenged
<anastasiamac> fwereade: did wallyworld ping u for bug 1262175? do we have any further comments?
<mup> Bug #1262175: debate drop tools upload capability <jujuqa> <simplestreams> <upload-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1262175>
<rick_h_> dimitern: fwereade macgreagoir ping for standup
<voidspace> macgreagoir: you have a review by the way
<natefinch> sinzui: where did we land with the 1.25 landing bot using go 1.6?  THe makefile seems to install the "golang" package, but I have no idea what that installs on various systems.  I wish we could be more explicit in our dependency on a specific version of Go.
<natefinch> cc mgz ^^
<sinzui> natefinch: I repeat the bot ues GO 1.6 and has since April 10
<sinzui> natefinch: If juju's makefile is correct, it will also use go 1.6
<natefinch> sinzui: http://juju-ci.vapour.ws:8080/job/github-merge-juju/8853/artifact/artifacts/trusty-out.log    output from go version: go version go1.2.1 linux/amd64
<sinzui> natefinch: Unit tests are runing according to how the branch states they run. The essential comands are: make install-dependencies; make build; make check
<sinzui> natefinch: again, the 1.25 branch's makefile needs updating to match the engineering intent.
<sinzui> natefinch: update you branch, and then your work and everyone elses will test with the right version of go
<fwereade> voidspace, sorry! I would be happy to talk about it now in fact
<voidspace> fwereade: I'm nearly EOD and going to the gym
<fwereade> voidspace, ack, np, tomorrow is fine then :)
<voidspace> fwereade: if you're around later I'll be back online
<voidspace> kk
<fwereade> voidspace, might be, but with family, so relatively unlikely
<voidspace> fwereade: ok, cool
<fwereade> anastasiamac, I don't think so
<natefinch> sinzui: are people going to have a heart attack if I just explicitly curl the right version of go from golang.org?  Because that's really the only way we can ensure things don't change out from underneath us.
<sinzui> natefinch: you cannot curl because that wont work on canonical machines
<fwereade> anastasiamac, I am generally in favour -- but it all depends on the experience of self-publishing them, right?
<sinzui> natefinch: go1.6 a deb packge ready to be installed
<sinzui> natefinch: follow the example of the master Marefile
<natefinch> sinzui: ahh, yeah, ok.
<sinzui> natefinch: master is doing this: https://pastebin.canonical.com/163656/
<voidspace> katco: http://reviews.vapour.ws/r/5497/
<voidspace> katco: it's little :-)
<katco> voidspace: yeah already looking at it :)
<voidspace> great
<katco> voidspace: you mentioned that there are existing tests, but how do we qa that the bug is actually fixed?
<sinzui> natefinch: though I think the gccgo clause cannot ever be reeasched
<natefinch> sinzui: I sure hope not, because it won't work
<voidspace> katco: that's a question we haven't been able to answer for any of the places we've fixed this bug
<katco> voidspace: ah bc it's based on load?
<voidspace> katco: we can't deterministically trigger the mongo behaviour
<voidspace> katco: yep
<katco> gotcha
<katco> fair enough
<voidspace> mongo turns a bson.D into a bson.M (I think that's the right way round) and our assert becomes order dependent
<katco> yeah that whole thing =|
<katco> voidspace: ok, lgtm. ship it
<katco> voidspace: sorry i didn't find this fri.
<natefinch> sinzui: should I add bzip2 to the deps?  master has that for some reason
<sinzui> natefinch: that wont hurt
<voidspace> katco: thanks
<sinzui> natefinch: in vaguely recall there is a  missing package for race testing in master's makefile. I am not sure how to handle that case if the package is not the same in all series
<anastasiamac> fwereade: awesome! tyvm :D
<natefinch> sinzui: should we update our list of supported architectures in 1.25 then also?
<natefinch> sinzui: like, just the list in the makefile
<sinzui> natefinch: absolutely. It's driven by the golang goodness
<natefinch> ok
<sinzui> natefinch: we officially don't support s390x with 1.x, but is does build and appear to work
<natefinch> sinzui: mind if I change the no-gopath check to an error?  Seems like if you're ever running the makefile, you really must have a GOPATH
<sinzui> natefinch: +1
<frobware> voidspace: when the nodes don't commission using my scripts does it eventually error with "error: maas19-node7 did not reach 'shut off' state after 240s"?
<natefinch> sinzui: http://reviews.vapour.ws/r/5503/
<voidspace> frobware: didn't try with maas 1.9  - don't recall any error like that
<voidspace> frobware: I'm just off now, can try later if you like
<frobware> voidspace: don't worry
<sinzui> natefinch: LGTM
<natefinch> sinzui: thanks
<dimitern> rick_h_, frobware: I've updated http://reviews.vapour.ws/r/5496/ to allow unsupported versions, but log a warning as suggested
<mup> Bug #1615013 opened: Autopilot: Nagios uses the wrong subnet IP to reach one host <landscape> <juju-core:New> <Landscape Server:Invalid> <MAAS:New> <nagios (Juju Charms Collection):New> <https://launchpad.net/bugs/1615013>
 * natefinch adds files that aren't even compiled.... CI fails :/
<natefinch> (not because of my files, just flaky tests on windows)
<natefinch> rick_h_: oh, btw, talked to john, he ok'd the use of the go-jsschema package I was using, as long as we also remove the use of the gojsonschema package we've been using (which is fine, it's only used in a couple files, and should be less than a dozen lines of code to change, all told).
<natefinch> sinzui, mgz: how hard is it to set up https://github.com/juju/jsonschema with the landing bot?  All it really needs to do is run go get github.com/juju/jsonschema && go test github.com/juju/jsonschema
<mgz> natefinch: it's pretty easy
<rick_h_> natefinch: <3
<natefinch> mgz: when you get some time, please and thank yuo.
<mgz> natefinch: sure thing
<mgz> natefinch: branch currently just has a licence?
<natefinch> mgz: I can't land the PR with the code without a bot to land it :)
<mgz> natefinch: okay :)
<dimitern> mgz: hey there ;)
<dimitern> mgz: did you see my ping earlier?
<mgz> ah, I see, it already has bots over hackers in ther perms
<mgz> dimitern: no, sorry, only just around now
<mgz> dimitern: how can I help
<mgz> hm, interesting dep version issue of some kind?
<mgz> lets have a look at the lxd code
<mgz> dimitern: what did you need the bump for?
<sra__> can anyone provide me the better requirements for deploying openStack on a single VM using JUJU OpenStack-base bundle
<dimitern> mgz: well, frobware and you had some chat about packaging issue wrt lxd and juju
<dimitern> mgz: see http://reviews.vapour.ws/r/5496/
<mgz> dimitern: our main issue is that because lxd is in the archive, even api compatible code changes to lxd can cause issues, if the 'abi' has changed,
<mgz> so, for example,
<mgz> we update a dep to a new version that has an extra field in a struct - which is normally fine
<mgz> however, when we backport that juju code, it is then building against the older version of the dep without that field, and breaks
<mgz> so, the fix would be sru the dep at the same time,
<dimitern> ok, but that seems unsolvable
<dimitern> catch 22
<mgz> but, if *anything else at all* uses that dep, we risk then breaking the other way, as those projects could expect something different at compile time
<dimitern> juju-core does not depend on lxd AFAICS (packaging-wise)
<mgz> we build against it
 * dimitern is still confused what's the right call here
<mgz> anyway, the specifics matter here, we may be okay
<mgz> dimitern: I don't see how we do this other than by taking the change, and as needed, dropping our build dep on the archive
<mgz> it's what we had to do before
<mgz> it means we build with a smaller and smaller set of packaged deps as things update
<mgz> but snaps sort of avoid all this anyway
<mgz> and that's where we're going
<mgz> dimitern: I'll comment on the mp
<dimitern> mgz: cheers!
<mgz> dimitern: from the diff, it's not totally clear to me, did you need the lxdDevices arg? or did you update just because?
<dimitern> mgz: it was needed after bumping lxc/lxd's revision, Init() now takes devices
<mgz> dimitern: when I last looked that change wasn't on stable-2.0 branch
<dimitern> mgz: it still isn't
<frobware> dimitern: and did we need to bump as far as we did for this change?
<mgz> is the change you actually need on that branch?
<dimitern> frobware: this is the commit that is needed: https://github.com/lxc/lxd/commit/0d172e3080f768fd419f0c97f5246983797db243
<frobware> dimitern: oh, 10 days ago... :(
<mgz> so, we flat out depend on lxd 2.1 then...
<dimitern> yeah, so if it's so recent I thought bumping to tip of lxc/lxd master shouldn't be a big deal
<dimitern> mgz: if we need to cut it exactly, 2.0.4 will work
<dimitern> mgz: https://github.com/lxc/lxd/commit/1a6adb0332e8f1da761cf899a92f9b8d8af53268
<frobware> mgz, dimitern: wouldn't 2.0.4 be better here
<mgz> yeah, I think while we can
<mgz> we should stick on that branch
<frobware> agreed
<frobware> 2.1 would invalidate all the testing we've done
<babbageclunk> fwereade: Take another look at http://reviews.vapour.ws/r/5495/? The types should be more to your liking now.
<fwereade> babbageclunk, ta
<mgz> this isn't a panacea for the basic problem of 'abi' compat, but does keep what we test and expect to work more consitent at least
<dimitern> mgz, frobware: updated the PR to use 2.0.4's rev# and just pushed the change
<mgz> dimitern: thanks! does that build? reviewboard only shows me the depedencies.tsv change for the interdiff
<frobware> dimitern: how did Init(...with... devices) end up in 2.0.4?
<frobware> dimitern: isn't that a breaking API change on 2.0?
<dimitern> mgz: it builds and works - redid my live bootstrap
<dimitern> frobware: I don't know, but it's there
<dimitern> frobware: it's a breaking ABI change more than an API change :)
<dimitern> well, both I guess
<dimitern> one of those oops moments - sh*t we forgot to report the API version!
<mgz> yeah, the terms don't quite map to go
<redir> morning
<katco> morning redir
<redir> :)
<mup> Bug #1615719 opened: [juju-2.0-beta15] during the bootstrap stage the no-proxy config is ignored <openstack-provider> <juju-core:New> <https://launchpad.net/bugs/1615719>
<fwereade> babbageclunk, reviewed
 * fwereade out for a while, back for thumper meeting this evening
<babbageclunk> fwereade: thanks!
<dimitern> rick_h_, frobware, mgz: I'll leave http://reviews.vapour.ws/r/5496/ and we should decide tomorrow for sure whether to land it or not
<frobware> dimitern: ack
<redir> brb system wants reboot
 * rick_h_ goes to get the boy from school, biab
<menn0> veebers: I have the changes ready for review which add macaroon support for migrations. once that lands you should be able to proceed with the authentication tests
<menn0> thumper: http://reviews.vapour.ws/r/5498/
<thumper> dev?
<thumper> in bootstrap
<thumper> lxd?
<menn0> thumper: after a helpful chat with rogpeppe last night, i'm going to improve the way macaroons are used with migrations but this PR is good enough for now and lets veebers press on with CI tests
<thumper> ok
<thumper> menn0: on the release call just now
<veebers> menn0: cool, cheers :-)
<rogpeppe> thumper: did you see this bug BTW? https://bugs.launchpad.net/juju-core/+bug/1615580
<mup> Bug #1615580: logging emits inappropriate ANSI escape codes on non-ANSI terminal <juju-core:New for thumper> <https://launchpad.net/bugs/1615580>
<menn0> thumper: yep, dev is my lxd cloud with all the right options set
<rogpeppe> thumper: (hi, BTW!)
<thumper> o/ rogpeppe
<thumper> rogpeppe: I'll make term = dumb work
<rogpeppe> thumper: thanks
 * menn0 needs to update the wiki with the latest config advice
<rogpeppe> thumper: really i think it should be anything except term=ansi, but i guess not many people use non-ansi ttys any more :)
<rogpeppe> thumper: strictly speaking you should be parsing termcap/terminfo
<thumper> $ env | grep TERM
<thumper> TERM=xterm-256color
<thumper> rogpeppe: it makes a syscall to determin whether a terminal or not
<thumper> and it is someone else's package
<rogpeppe> thumper: yeah, but that says nothing about what escape codes it supports
<thumper> but where we are using it, we can check for particulars
<rogpeppe> thumper: tbh i'm not sure it's great to be adding this stuff as a dep to juju
<rogpeppe> thumper: couldn't we just colourize post-hoc if desired?
<thumper> some people desire color
<thumper> I can understand that some don't like it, but they are definitely a minority
<rogpeppe> thumper: i just see all the trivial deps we're adding to anything that imports juju/cmd
<thumper> it has been specifically asked for by sabdfl
<rogpeppe> thumper: fair enough. doesn't necessarily mean it needs to built in at quite such a low level though, i think.
<mup> Bug #1597601 opened: ERROR cannot deploy bundle: cannot deploy application: i/o timeout <2.0> <bundles> <deploy> <oil> <oil-2.0> <repeatability> <retry> <juju-core:New for thumper> <https://launchpad.net/bugs/1597601>
 * rick_h_ runs for the day
<redir> Have a good evening rick_h_
<mup> Bug #1615839 opened: Manual-provider claims s390x is not supported <ci> <manual-provider> <regression> <s390x> <juju-core:Triaged> <https://launchpad.net/bugs/1615839>
<mup> Bug # changed: 918386, 1227942, 1358219, 1453096, 1470820, 1473197, 1473466, 1476060, 1477357, 1479194, 1483672, 1486965, 1488139, 1490480, 1491353, 1492237, 1493458, 1493598, 1493600, 1493601, 1493604, 1493606, 1494661, 1495978, 1496159, 1497351, 1498235, 1499900, 1500298, 1503990, 1510689,
<mup> 1513466, 1516669, 1516676, 1520373, 1524171, 1532670
<mup> Bug # changed: 1534923, 1536461, 1536480, 1539216, 1547665, 1552815, 1557918, 1566545, 1568160, 1569106, 1574076, 1574773, 1575764, 1581878, 1582105, 1590143, 1592887, 1598291, 1598292, 1602508, 1610397, 1613516, 1613823, 1613864
<axw> wallyworld: cam is still playing up, will join soon
<wallyworld> ok
<wallyworld> thumper: alexisb standup?
<mup> Bug # opened: 1534923, 1536461, 1536480, 1539216, 1547665, 1552815, 1557918, 1566545, 1568160, 1569106, 1574076, 1574773, 1575764, 1581878, 1582105, 1590143, 1592887, 1598291, 1598292, 1602508, 1610397, 1613516, 1613823, 1613864
<perrito666> wallyworld: you dropped
<mup> Bug # changed: 802117, 1386425, 1445174, 1454059, 1490789, 1501709, 1510133, 1534923, 1536461, 1536480, 1539190, 1539216, 1547665, 1552815, 1554120, 1557918, 1563705, 1566545, 1568160, 1569106, 1570269, 1570285, 1574076, 1574773, 1575764, 1576700, 1577598, 1581878, 1582105, 1590143, 1590598,
<mup> 1590821, 1590961, 1592887, 1598291, 1598292, 1602508, 1609643, 1610397, 1613516, 1613823, 1613864, 1614072, 1615013
<alexisb> thumper, can you rejoin please
<thumper> anastasiamac: hello on call reviewer
<thumper> anastasiamac: http://pastebin.ubuntu.com/23077491/
<anastasiamac> thumper: want me to review a pastebin? :D
 * thumper sighs
<thumper> anastasiamac: http://reviews.vapour.ws/r/5505/
<thumper> wrong thing in the buffer
<anastasiamac> thumper: looking
<alexisb> wallyworld, free when you are
<wallyworld> ok, joining standup
<menn0> thumper: http://reviews.vapour.ws/r/5498/ if you have a chance
<thumper> menn0: ok
#juju-dev 2016-08-23
<mup> Bug # changed: 1167616, 1292116, 1346597, 1349949, 1383453, 1425808, 1426728, 1453805, 1455627, 1465317, 1466514, 1473069, 1475212, 1480616, 1485781, 1498232, 1501475, 1504658, 1506044, 1509635, 1510688, 1510787, 1513667, 1516150, 1517632, 1518480, 1518482, 1521453, 1530840, 1531444, 1532130,
<mup> 1532266, 1534103, 1535838, 1536447, 1538583, 1540301, 1546794, 1548078, 1552274, 1555475, 1555727, 1555808, 1557726, 1557914, 1558185, 1560107, 1560888, 1563015, 1563611, 1563933, 1564026, 1566014, 1566426, 1566583, 1566639, 1566684, 1566791, 1567607, 1567747, 1567934, 1568183, 1568666, 1569802,
<mup> 1570216, 1570660, 1571982, 1572116, 1572695, 1572700, 1573407, 1573742, 1576534, 1577524, 1577567, 1577587, 1577589, 1577590, 1577593, 1577638, 1577798, 1578898, 1579887, 1579976, 1580231, 1580423, 1580485, 1580717, 1581894, 1582021, 1582065, 1582463, 1582801, 1584193, 1585878, 1586089, 1586890,
<mup> 1587345, 1587503, 1587734, 1588041, 1589641, 1590362, 1591488, 1592613, 1592811, 1593185, 1593350, 1594332, 1594663, 1595600, 1596619, 1596687, 1596688, 1596842, 1596858, 1596888, 1596890, 1596893, 1596906, 1597481, 1597490, 1597601, 1597830, 1598206, 1598290, 1598707, 1599272, 1599570, 1599886,
<mup> 1600063, 1600221, 1600453, 1600515, 1600523, 1600546, 1602032, 1602237, 1602416, 1602749, 1602840, 1602885, 1602952, 1603060, 1603176, 1603228, 1603473, 1603640, 1603841, 1604120, 1604817, 1604883, 1604915, 1604961, 1604965, 1604988, 1605241, 1605335, 1605449, 1605653, 1605714, 1605986, 1606282,
<mup> 1606308, 1606354, 1606487, 1606506, 1606617, 1606684, 1606917, 1606922, 1607161, 1607170, 1607347, 1607599, 1607601, 1607608, 1607611, 1607855, 1607895, 1607964, 1607971, 1608494, 1608529, 1608597, 1608677, 1608956, 1608959, 1609893, 1610880, 1611067, 1611093, 1611159, 1611391, 1611404, 1611427,
<mup> 1611453, 1611514, 1611799, 1611981, 1611990, 1612046, 1612105, 1612110, 1612417, 1612500, 1612624, 1612717, 1612836, 1613150, 1613200, 1613262, 1613300, 1613429, 1613444, 1613459, 1613491, 1613688, 1613782, 1613838, 1613839, 1613855, 1613858, 1613866, 1613892, 1613960, 1613992, 1614010, 1614161,
<mup> 1614345, 1614364, 1614559, 1614635, 1614689, 1614948, 1615051, 1615839
<perrito666> chill out bot, you will get something
<redir> bbiab ~730PDT
<anastasiamac> thumper-dogwalk: RB strikes again :D PR behind this RB has been merged but RB is still open.. could u plz close it? http://reviews.vapour.ws/r/5472/
<mup> Bug # changed: 1205113, 1334482, 1397201, 1403722, 1444066, 1445066, 1455625, 1468188, 1492530, 1494939, 1536448, 1539156, 1542488, 1545712, 1554116, 1555801, 1556176, 1564168, 1566023, 1569632, 1570651, 1571831, 1571855, 1572350, 1573099, 1574798, 1575245, 1575283, 1575405, 1576184, 1576295,
<mup> 1576301, 1576342, 1576359, 1576851, 1579833, 1579836, 1581138, 1581211, 1581612, 1581879, 1582018, 1585289, 1590237, 1591379, 1591499, 1592031, 1592837, 1594440, 1595330, 1596476, 1596593, 1596603, 1596605, 1596607, 1596612, 1596615, 1596616, 1596853, 1604106, 1607303, 1609463, 1609879, 1610007,
<mup> 1610254, 1611111, 1611267, 1611269, 1611271, 1611273, 1611275, 1612043, 1612048, 1612478, 1612645, 1612658, 1612687, 1612775, 1612793, 1613537, 1613672, 1614065, 1614230, 1614239, 1614329, 1614330, 1614571, 1614633, 1614732, 1614749, 1614809, 1615108, 1615601
<mup> Bug #1615868 opened: interactive bootstrap fails with tools mismatch <juju-core:Triaged> <https://launchpad.net/bugs/1615868>
<mup> Bug #1615868 changed: interactive bootstrap fails with tools mismatch <juju-core:Triaged> <https://launchpad.net/bugs/1615868>
<mup> Bug #1615868 opened: interactive bootstrap fails with tools mismatch <juju-core:Triaged> <https://launchpad.net/bugs/1615868>
<mup> Bug #1615868 changed: interactive bootstrap fails with tools mismatch <juju:Triaged> <https://launchpad.net/bugs/1615868>
<wallyworld> thumper-dogwalk: can you ping me, or see last comment in bug 1615051
<mup> Bug #1615051: Dubious hook failures deploying trivial charm <ci> <deploy> <hooks> <jujuqa> <regression> <juju:Triaged by thumper> <https://launchpad.net/bugs/1615051>
<thumper> wallyworld: seems reasonable
<wallyworld> thumper: so you will fix :-)
<thumper> wallyworld: you got to blame it on me
<thumper> um...
<wallyworld> \o/
<thumper> they need to fix their tests
<wallyworld> but the charm is broken
<wallyworld> we have broken the production charm
<thumper> so they need to fix their charm
<wallyworld> for mediawiki and others
<wallyworld> a lot of stuff will just break now
<thumper> wallyworld: marcoceppi seemed to think that most charms use charmtools and wouldn't be impacted by this change
<wallyworld> does charm tools do the right thing? what does it do?
<thumper> it always says --format json
<wallyworld> so maybe we just got unlucky
<marcoceppi> charmhelpers*
<thumper> marcoceppi: sorry
<wallyworld> the dummy test charm and the mediawiki one the CI tests use
<thumper> marcoceppi: would you expect mediawiki to work?
<marcoceppi> it's probalby bash
<marcoceppi> because it's garbage
<wallyworld> no, ot's python
<wallyworld> https://api.jujucharms.com/charmstore/v5/mediawiki/archive/hooks/cache-relation-changed
<thumper> wallyworld: it uses --format json
<wallyworld> really?
<wallyworld> rl = subprocess.Popen("relation-list",stdout=subprocess.PIPE)
<wallyworld> no python there that i can see - that's the bit that is causing the failure
<wallyworld> unless i am mistaken
<marcoceppi> wallyworld: it's not using charmhelpers, it's not a good charm
<wallyworld> agreed
<thumper> p = subprocess.Popen(["relation-get", "--format", "json", "-", memcached_unit.strip()],
<wallyworld> thumper: right but that's not the issue
<thumper> ugh
<thumper> sabdfl said "fix the charms"
<wallyworld> the issue is that the memcahced_unit value has changed
<wallyworld> thumper: so for now, i guess we ask CI to set the feature flag
<thumper> wallyworld: that would certainly be the easiest solution
<thumper> wallyworld: I'll email appropriate people
<wallyworld> thumper: can you be the one then to mark the bug as "invalid" or "won't fix" for core :-)
<wallyworld> ta
<wallyworld> thumper: also cc rick who was the one who the bug passed through
<thumper> ack
<redir> woah, hot bug action.
<katco> thumper: hey, i responded to your review of http://reviews.vapour.ws/r/5485/ can you give me a ship it (or not)?
<thumper> sure, I'll look again shortly
<katco> thumper: no rush at all. as long as i wake up in 10h or so and something's there :)
<thumper> :)
<katco> thumper: ta
 * katco disappears
<redir> poof
<natefinch> so I guess juju on launchpad is now what we're using for 2.0 bugs?
<natefinch> as opposed to juju-core
<mup> Bug #1615095 changed: storage: volumes not supported <landscape> <juju:Triaged> <https://launchpad.net/bugs/1615095>
<mup> Bug #1615580 changed: logging emits inappropriate ANSI escape codes on non-ANSI terminal <juju:Triaged> <https://launchpad.net/bugs/1615580>
<anastasiamac> natefinch: yes. this si the attempt to make sure that we important 2.0 bugs do not disappear in the heap of old 15K bugs.. when I say old, i mean like 2012,2013...:D
<mup> Bug #1615580 opened: logging emits inappropriate ANSI escape codes on non-ANSI terminal <juju:Triaged by thumper> <https://launchpad.net/bugs/1615580>
<natefinch> anastasiamac: cool.  Hopefully someone will send out an email soon, so we don't add new bugs to the wrong place?
<anastasiamac> natefinch: Rcik sent an email couple of weeks ago.
<natefinch> anastasiamac: oh. huh.
<anastasiamac> Rick* even :D
<natefinch> anastasiamac: ahh, I see it.  It's buried at the bottom of a long-ish email about bug cleanup :)
<anastasiamac> natefinch: i like the idea of following up to sya "we are done" but i think a redirect from juju-core to juju (or some kind of note that 2.0 bugs do not belong there) would be nice..
<natefinch> anastasiamac: so it's on 2.0 bugs that'll get moved over?
<natefinch> s/on/only
<anastasiamac> natefinch: yes, 2.+
<anastasiamac> \o/
<natefinch> Oh, I know why I missed it, rick sent it to cloud
<anastasiamac> natefinch: yes :D
<natefinch> I don't ever look at cloud, because it's so spammy
<anastasiamac> natefinch: sorry. want to file a bug? :D
<natefinch> haha
<mup> Bug #1615580 changed: logging emits inappropriate ANSI escape codes on non-ANSI terminal <juju:Triaged> <https://launchpad.net/bugs/1615580>
<mup> Bug #1615719 changed: [juju-2.0-beta15] during the bootstrap stage the no-proxy config is ignored <openstack-provider> <juju:Triaged> <https://launchpad.net/bugs/1615719>
<natefinch> just wondering if I should add juju-dev to my re-forwarding of the email
<natefinch> seems like it would be good for people outside canonical to know, too.  but maybe that was on purpose
<wallyworld> thumper: probs not - did we have anything like dave's process info endpoint in 1.25?
<thumper> yes
<thumper> but it was less easy to access
<wallyworld> thumper: oh awesome, i didn't think it was for 1.25, can you point me to docs?
<thumper> it landed in 1.25 I think
<thumper> https://github.com/juju/juju/wiki/pprof-facility
<thumper> 1.25.4
<wallyworld> awesome thanks
<mup> Bug # changed: 1456659, 1457092, 1457148, 1459300, 1493058, 1507359, 1510651, 1510944, 1512782, 1536230, 1536333, 1554088, 1559299, 1560331, 1568186, 1568189, 1568715, 1578327, 1580221, 1582268, 1588147, 1596559, 1605593, 1606300, 1615121
<mup> Bug # changed: 1614599, 1614622, 1614835, 1614992, 1615008
<wallyworld> can has small review? http://reviews.vapour.ws/r/5506/
<mup> Bug # opened: 1614599, 1614622, 1614835, 1614992, 1615008
<redir> wallyworld: looking
<wallyworld> \o/
<wallyworld> redir: need any help with the config stuff - just let me know
<redir> yeah, I have a couple ? in a minute, wallyworld
<wallyworld> wow, this is kinda awesome, just saw it in warthogs http://www.boredpanda.com/ive-just-painted-all-25-ubuntu-animals/
<wallyworld> axw: it seems that we have Schema() methods on various environ providers but don't call this method anywhere but from tests, and I can't see an interface that declares it either - it's not on the EnvironProvider interface. am i missing something?
<menn0> veebers: the big macaroon changes for migrations have landed
<axw> wallyworld: I don't think that work was ever completed
<axw> wallyworld: I think it'll be obsoleted by natefinch's work
<wallyworld> ok, but i need it now :-)
<wallyworld> for proper default config management
<wallyworld> this week
<wallyworld> first in wins :-)
<axw> wallyworld: what do you need to do?
<wallyworld> when bootstrapping and setting up config defaults, we currently ignore provider specific config - we only handle what's in config/config.go. So juju model-defaults will not show use-floating-ip for example on openstack
<mup> Bug # changed: 1242783, 1514555, 1515016, 1517076, 1519147, 1519176, 1519189, 1521020, 1521699, 1528070, 1528572, 1531719, 1534626, 1536336, 1536340, 1536345, 1542988, 1543839, 1544838, 1544855, 1545713, 1547891, 1548921, 1554136, 1554797, 1556238, 1559701, 1559708, 1559789, 1560152, 1560618,
<mup> 1561526, 1561959, 1563335, 1563373, 1563380, 1563400, 1565461, 1565462, 1567396, 1569529, 1571065, 1571687, 1572102, 1573681, 1574637, 1575229, 1578237, 1580714, 1582264, 1583304, 1585300, 1589628, 1603221, 1604959, 1605767, 1605769, 1605776, 1606256, 1606265, 1606302, 1606303, 1606310, 1606313,
<mup> 1606566, 1606568, 1606571, 1606711, 1606939, 1606991, 1607044, 1607362, 1607365, 1607368, 1607794, 1607858, 1607859, 1608528, 1608533, 1608938, 1608942, 1608947, 1609405, 1609818, 1609896, 1609980, 1610238, 1610255, 1610993, 1611764, 1611766, 1612026, 1612744, 1612745, 1612747, 1613311, 1613824,
<mup> 1613829, 1613882, 1613888, 1614256, 1614599, 1614622, 1614835, 1614992, 1615008, 1615118
<veebers> menn0: awesome, I'll build and fire off a run shortly :-)
<wallyworld> and when a user sets or resets a value, they get a bullshit warning because those provider config items are not in the vocab
<wallyworld> so i can easily solve the issue by processing the provider schema at bootstrap time
<axw> wallyworld: ok, sounds like adding Schema to EnvironProvider is reasonable. please let natefinch know your requirements so he can work that into what he's doing
<wallyworld> will do
<wallyworld> he may even be lurking here now :-)
<mup> Bug # opened: 1242783, 1514555, 1515016, 1517076, 1519147, 1519176, 1519189, 1521020, 1521699, 1528070, 1528572, 1531719, 1534626, 1536336, 1536340, 1536345, 1542988, 1543839, 1544838, 1544855, 1545713, 1547891, 1548921, 1554136, 1554797, 1556238, 1559701, 1559708, 1559789, 1560152, 1560618,
<mup> 1561526, 1561959, 1563335, 1563373, 1563380, 1563400, 1565461, 1565462, 1567396, 1569529, 1571065, 1571687, 1572102, 1573681, 1574637, 1575229, 1578237, 1580714, 1582264, 1583304, 1585300, 1589628, 1603221, 1604959, 1605767, 1605769, 1605776, 1606256, 1606265, 1606302, 1606303, 1606310, 1606313,
<mup> 1606566, 1606568, 1606571, 1606711, 1606939, 1606991, 1607044, 1607362, 1607365, 1607368, 1607794, 1607858, 1607859, 1608528, 1608533, 1608938, 1608942, 1608947, 1609405,
<mup> 1609818, 1609896, 1609980, 1610238, 1610255, 1610993, 1611764, 1611766, 1612026, 1612744, 1612745, 1612747, 1613311, 1613824, 1613829, 1613882, 1613888, 1614256, 1615118
<mup> Bug # changed: 1242783, 1514555, 1515016, 1517076, 1519147, 1519176, 1519189, 1521020, 1521699, 1528070, 1528572, 1531719, 1534626, 1536336, 1536340, 1536345, 1542988, 1543839, 1544838, 1544855, 1545713, 1547891, 1548921, 1554136, 1554797, 1556238, 1559701, 1559708, 1559789, 1560152, 1560593,
<mup> 1560618, 1561526, 1561959, 1563335, 1563373, 1563380, 1563400, 1563888, 1565461, 1565462, 1567396, 1569529, 1571065, 1571687, 1572102, 1573681, 1574637, 1575229, 1575400, 1578237, 1580714, 1582264, 1583304, 1585300, 1585361, 1589628, 1589890, 1590239, 1596071, 1603221, 1604959, 1605767, 1605769,
<mup> 1605776, 1606256, 1606265, 1606302, 1606303, 1606310, 1606313, 1606566, 1606568, 1606571, 1606711, 1606939, 1606991, 1607044, 1607362, 1607365, 1607368, 1607794, 1607858, 1607859, 1608528, 1608533, 1608938, 1608942, 1608947, 1609405, 1609818, 1609896, 1609980, 1610238, 1610255, 1610993, 1611764,
<mup> 1611766, 1612026, 1612744, 1612745, 1612747, 1613311, 1613824, 1613829, 1613882, 1613888, 1614256, 1615118
<redir> wallyworld: sooo for passing params to the ComposeNewModelConfig from apiserver/modelmanager were you thinking params that live in apiserver/params/... or somewhere else?
<wallyworld> redir: let me look at the code
<redir> wallyworld: I'll put it there and make a PR, so I wrap and have feedback for tomorrow, whence I can also get the modeldefaults done and move on to get/set
<wallyworld> redir: ok. with regards to ComposeNewModelConfig, you are talking about weaving in the region defaults right?
<wallyworld> if so, there's nothing to pass across the wire for that, so nothing from params
<redir> wallyworld: yes, currently just passing cloudSpec
<redir> but only need cloud and region
<redir> wallyworld: right nothing for the wire.
<redir> internal?
<wallyworld> no, stuff in those dirs is just for on the wire data
<redir> wallyworld: in internal is only for wire data?
<wallyworld> yes, internal api calls
<wallyworld> from workes to state etc
<redir> OK.
<wallyworld> everything in apiserver should be about pulling stuff of the wire for remote calls and composing into business objects for use elsewhere, but sadly we have put a lot of business logic in apiserver
<wallyworld> which should really be elsewhere
<wallyworld> there needs to be a separate business service layer
<wallyworld> but we have conflated apiserver for that
<axw> wallyworld: I'm going to pause on azure stuff and look at removing the remaining model config from the lxd provider
<wallyworld> awesome^100
<wallyworld> i'd love to get all this sorted this week
<menn0> thumper: blocking of user API requests to a model while it is being imported http://reviews.vapour.ws/r/5508/
<axw> wallyworld: I have a PR up for azure that reduces the number of API calls made. I didn't get as far as using templates, but that can be done later
<axw> no rush to review, just FYI
<wallyworld> ok, ta
<mup> Bug # changed: 1190293, 1258051, 1287658, 1372543, 1421650, 1433161, 1488777, 1512481, 1516975, 1524823, 1528073, 1531995, 1534511, 1535891, 1536819, 1545563, 1546142, 1554802, 1556167, 1559402, 1560527, 1561375, 1562088, 1564452, 1564880, 1566044, 1566271, 1566893, 1567712, 1569109, 1569963,
<mup> 1571923, 1572585, 1573668, 1574677, 1575310, 1575895, 1576324, 1578383, 1579592, 1579593, 1579849, 1580626, 1581069, 1581537, 1581600, 1582214, 1582818, 1582948, 1583787, 1584170, 1584212, 1584896, 1585674, 1585847, 1588092, 1588224, 1589351, 1589581, 1590468, 1590520, 1592583, 1592872, 1593492,
<mup> 1593708, 1594578, 1594883, 1595314, 1595720, 1595937, 1596038, 1596039, 1596496, 1596850, 1598319, 1599269, 1599612, 1600061, 1600296, 1600559, 1602838, 1603202, 1603888, 1604542, 1604551, 1604586, 1605976, 1606337, 1606569, 1606611, 1606706, 1606709, 1607109, 1607766, 1608422, 1608723, 1609910,
<mup> 1610450, 1611076, 1611110, 1612099, 1612112, 1612163, 1613842
<mup> Bug # changed: 1545568, 1548809, 1558657, 1558668, 1558924, 1560428, 1560920, 1561339, 1569949, 1572703, 1573665, 1577606, 1580400, 1581337, 1584832, 1584897, 1584899, 1584900, 1584901, 1584903, 1584905, 1584906, 1588970, 1589339, 1589350, 1590699, 1590732, 1596960
<mup> Bug # opened: 1545568, 1548809, 1558657, 1558668, 1558924, 1560428, 1560920, 1561339, 1569949, 1572703, 1573665, 1577606, 1580400, 1581337, 1584832, 1584897, 1584899, 1584900, 1584901, 1584903, 1584905, 1584906, 1588970, 1589339, 1589350, 1590699, 1590732, 1596960
<mup> Bug # changed: 1463608, 1536885, 1543404, 1543408, 1545568, 1548809, 1558657, 1558668, 1558924, 1560428, 1560920, 1561339, 1564157, 1564500, 1566268, 1567952, 1569949, 1572703, 1573665, 1577606, 1580400, 1581337, 1584832, 1584897, 1584899, 1584900, 1584901, 1584903, 1584905, 1584906, 1588970,
<mup> 1589339, 1589350, 1590699, 1590732, 1593274, 1596960, 1598127, 1600600
<redir> wallyworld: http://reviews.vapour.ws/r/5510/ at your leisure. I'm off for the night. If that is close I should be able to move on to the defaults output and then the get/set bits
<redir> night juju-dev
<wallyworld> thaks reed
<wallyworld> ttyl
<mup> Bug # opened: 1463608, 1536885, 1543404, 1543408, 1564157, 1564500, 1566268, 1567952, 1593274, 1598127, 1600600
<mup> Bug # changed: 1463608, 1536885, 1543404, 1543408, 1557540, 1563117, 1564157, 1564500, 1564677, 1566268, 1567952, 1593274, 1598127, 1600600
<mup> Bug #1557540 opened: Missing help for payloads <docs> <payloads> <juju:Triaged> <https://launchpad.net/bugs/1557540>
<mup> Bug #1563117 opened: api: data race adding local charm. <race-condition> <unit-tests> <juju:Triaged> <https://launchpad.net/bugs/1563117>
<mup> Bug #1564677 opened: Create one binary for juju and all plugins <osx> <packaging> <juju:Triaged> <https://launchpad.net/bugs/1564677>
<mup> Bug # changed: 1512569, 1557540, 1563117, 1564677
<axw> wallyworld: can you please see my comments on http://reviews.vapour.ws/r/5502/, make sure I'm not leading anastasiamac up the garden path again
<wallyworld> ok
<wallyworld> axw: could you take look at my small pr that reed looked at? just to give sign off?
<axw> wallyworld: sure, looking
<wallyworld> axw: fmt.Fprintf(ctx.Stderr) vs logger.Warning - i was hoping to avoid the logger extras and keep the output clean. was your issue that we should be showing a WARNING text?
<axw> wallyworld: I was thinking that yeah
<axw> wallyworld: not too concerned
<wallyworld> axw: ok, i'll add warning to the text
<wallyworld> axw: axctually, i meant that you look at this one http://reviews.vapour.ws/r/5506/
<axw> wallyworld: heh whoops
<wallyworld> the other one may need to have other changes added
<axw> looking :)
<wallyworld> ta
<axw> wallyworld: LGTM
<wallyworld> ta
<anastasiamac> axw: wallyworld: re: defaults for image metadata, want to discuss tomorrow after standup? i think we are all pretty much in agreement but i;d rather minimise the churn on PR \o/
<axw> anastasiamac: sure
<anastasiamac> axw: \o/
<axw> anastasiamac: I have a meeting at 8am, so may have to be short or after that
<anastasiamac> axw: that's k. as long as it's sometime tomorrow :) i have a meeting at 11am my time =9am urs? :) so maybe early afternoon :D
<axw> anastasiamac: and I plan to be out between 9:30 and don't-know-when looking at cars (again)
<anastasiamac> axw: that's exciting :D know what u want?
<axw> anastasiamac: there's a couple that are ok. I'm not too bothered, as long as I don't get ripped off
<anastasiamac> axw: well, actually, i'll remove the method and will ensure that data is populated at the right spots so it may be just the matter of re-viewing changes ;)
<axw> anastasiamac: ok, sounds good
<anastasiamac> axw: i love shopping for cars :)
<mup> Bug #1615958 opened: Switch to model name does't work anymore, admin should not be allowed to switch to users model <juju-core:New> <https://launchpad.net/bugs/1615958>
<mup> Bug #1615960 opened: remove-machine and remove-application don't work on google cloud <juju-core:New> <https://launchpad.net/bugs/1615960>
<voidspace> fwereade: ping
<voidspace> dimitern: land your lxd fix and let mgz handle the packaging mess when he returns ;-)
 * voidspace wants lxd working again
<dimitern> voidspace: :)
<voidspace> I might try one more time to go back to 2.0
<dimitern> voidspace: yeah, and I'm not convinced there is any packaging issue anyway
<voidspace> I burned at least two hours on it yesterday
<voidspace> dimitern: it seems odd
<voidspace> dimitern: however I'm sure frobware wouldn't invent it - so there must be some issue
<voidspace> dimitern: maybe a trusty thing?
<voidspace> but still can't see how
<voidspace> I'm sure mgz will explain :-(
<dimitern> voidspace: it was something about building .debs that includes building the lxd source and running into conflicts
<dimitern> but I don't believe this is avoidable at all
<dimitern> it'll break if lxd changes binary compatibility, like adding an extra arg to Init()
<babbageclunk> Hey fwereade, could you take another look at http://reviews.vapour.ws/r/5495/? I really think I've got it this time! :)
<fwereade> voidspace, babbageclunk: heyhey, I'm off today but you're in luck, everybody else is still asleep for the moment
<voidspace> fwereade: hehe
<voidspace> fwereade: I'm making some progress code spelunking on my own
<fwereade> voidspace, excellent
<voidspace> fwereade: but it's unfamiliar code so I'm sure you can help with some quick pointers
<voidspace> fwereade: IRC or hangout?
<fwereade> voidspace, irc probably better
<babbageclunk> fwereade: Yay thanks! Hope your day off gets better once voidspace stops bothering you. So rude!
 * babbageclunk does a big wink.
<voidspace> babbageclunk: always
<voidspace> babbageclunk: and stop winking
<babbageclunk> voidspace: oh, that's involuntary - I think it's my meds.
<voidspace> babbageclunk: :-)
<fwereade> babbageclunk, hahaha
<thumper> o/
<thumper> fwiw I'm fixing my fubar
<thumper> will need some reviewers soon
<dimitern> mgz, frobware, voidspace, babbageclunk: I'm about to push https://github.com/juju/juju/pull/6054 for landing, unless anyone is strongly against it.
<dimitern> \o thumper
<voidspace> dimitern: sounds good to me
<voidspace> thumper: o/
<thumper> o/
<babbageclunk> thumper: o/
<thumper> babbageclunk: how's the laptop?
<babbageclunk> thumper: :( I'm using a loaner at the moment.
<babbageclunk> thumper: But it's super chunky - 8 cores & 32 Gb!
<babbageclunk> thumper: (the loaner, I mean)
<thumper> babbageclunk: so fucked then?
<babbageclunk> thumper: And it was a pretty good test of my ansible setup scripts.
<babbageclunk> thumper: yup - just the drive, though
<babbageclunk> thumper: need to order a new ssd
 * thumper nods
<thumper> lose much?
<thumper> here is the first branch: http://reviews.vapour.ws/r/5511/
<babbageclunk> thumper: a couple days' work - although lots of that was working out time rather than typing time, so it wasn't too bad. Definitely pushing a lot more frequently now though!
<thumper> :)
<macgreagoir> dimitern: I want to retry a merge, but I can wait if you're about to queue-up the LXD fix.
<dimitern> macgreagoir: I've just sent the merge comment on mine
<macgreagoir> Cool
 * thumper runs all tests
<thumper> prelim review anyone? http://reviews.vapour.ws/r/5512/
<thumper> I'm running all the tests now to make sure I didn't miss something
<dimitern> thumper: looking at both, in order
<thumper> dimitern: first one has landed already
<thumper> :)
<thumper> it is a juju/cmd change
<thumper> which the second needs
<dimitern> oh
<dimitern> ok
<thumper> so far, so good
<thumper> up to featuretests
<thumper> worker/uniter was the other big failure
<macgreagoir> You just jinxed it! :-)
<thumper> perhaps
<thumper> oh poop
<thumper> anyone ? http://reviews.vapour.ws/r/5513/
<thumper> real fix waiting now for the above cmd fix
<thumper> babbageclunk ? ^^
<babbageclunk> thumper: I started looking but it was longer than my attention span! I'll take another run at it.
<thumper> the one above is a missed behaviour change in juju/cmd
<thumper> I have an update for the big branch once that has landed
<babbageclunk> thumper: Oh, right - looking at that now
<thumper> babbageclunk: thanks
<babbageclunk> thumper: LGTM
<babbageclunk> thumper: How are you verifying these?
<babbageclunk> thumper: (I mean against the old one.)
<thumper> I have been looking at the test changes I did on my friday branch
<thumper> and making the code like that again
<thumper> then fixing other things that need fixing
<babbageclunk> gotch
<macgreagoir> thumper: 5512 lgtm, fwiw
<babbageclunk> a
<babbageclunk> wow macgreagoir, how'd you do that? sharp packets?
<macgreagoir> babbageclunk: gotch...a ? I'd been reviewing 5512 much longer than that :-)
<thumper> babbageclunk, macgreagoir: 5512 just got a fresh push
 * thumper runs another complete test run just to be sure
<babbageclunk> frobware: Have you got your hacked trusty image around? Need to recreate my kvm-maases.
 * thumper is still hoping for a +1 from babbageclunk
 * babbageclunk sighs.
<babbageclunk> ok looking for reals now
<thumper> babbageclunk: thanks, I appreciate it
<thumper> babbageclunk: FWIW, it isn't very exciting
<babbageclunk> thumper: Ok, done - LGTM!
<babbageclunk> thumper: It wasn't very exciting though. :)
<thumper> thanks
<thumper> tests all good this side
<thumper> I'm submitting and signing off
<thumper> laters
<macgreagoir> \o
<rick_h_> macgreagoir: adding one more item for that list for the card you're working on please, controller version
<macgreagoir> rick_h_: ack
<rick_h_> ty macgreagoir
<rick_h_> dimitern: around to chat?
<dimitern> rick_h_: yeah, sorry - joining our 1:1 now
 * babbageclunk has to reboot to enable on virtualisation!
<rick_h_> mgz: ping
<babbageclunk> frobware: ping?
<babbageclunk> voidspace: Are you around? Do you have frobware's hacked xenial image with the fixed cloud-init?
<babbageclunk> voidspace: I'm sorry about those mean things I said this morning.
<babbageclunk> voidspace: That was before I realised I needed something from you.
<voidspace> babbageclunk: haha
<voidspace> babbageclunk: I don't - I thought cloud-init was fixed in the standard images now
<voidspace> babbageclunk: that's all I've been using
<babbageclunk> voidspace: hmm - my xenial instance won't start
<babbageclunk> voidspace: just hangs during startup. although at a different point than the cloud-init problem did, now that I look harder at it.
<voidspace> babbageclunk: oh, weird
<voidspace> babbageclunk: maas?
<voidspace> pretty sure I've had it working fine
<babbageclunk> voidspace: Well, I haven't got to the point of installing maas yet - this is just starting the controller.
<babbageclunk> voidspace: I'm using the daily build - maybe I should use something else?
<frobware> babbageclunk: http://178.62.20.154/~aim/xenial-server-cloudimg-amd64-disk1.img
<babbageclunk> frobware: Thanks, I'll try that instead.
 * frobware is unexpedtedly out of office today dealing with leaking pipes :(
<babbageclunk> frobware: good luck!
<babbageclunk> frobware: That worked, thanks.
<mup> Bug #1616059 opened: Unit test failure in github.com/juju/juju/cmd/juju/application	 <juju-core:Triaged> <https://launchpad.net/bugs/1616059>
<mgz> rick_h_: late pong
<mup> Bug #1511822 opened: imports github.com/juju/juju/workload/api/internal/client: use of internal package not allowed <blocker> <ci> <regression> <wily> <juju-core:New> <juju-core 1.25:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1511822>
<mup> Bug #1511822 changed: imports github.com/juju/juju/workload/api/internal/client: use of internal package not allowed <blocker> <ci> <regression> <wily> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.25:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1511822>
<mup> Bug # opened: 1425808, 1457575, 1474607, 1510651, 1513165, 1534643, 1538583, 1573136, 1580221, 1584616, 1587644, 1592887, 1596597, 1597830, 1599503, 1604988, 1610880, 1612645, 1613992, 1614364, 1614633
<mup> Bug # changed: 1425808, 1457575, 1474607, 1510651, 1513165, 1534643, 1538583, 1573136, 1580221, 1584616, 1587644, 1592887, 1596597, 1597830, 1599503, 1604988, 1610880, 1612645, 1613992, 1614364, 1614633
<mup> Bug #1511822 opened: imports github.com/juju/juju/workload/api/internal/client: use of internal package not allowed <blocker> <ci> <regression> <wily> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.25:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1511822>
<mup> Bug #1511822 changed: imports github.com/juju/juju/workload/api/internal/client: use of internal package not allowed <blocker> <ci> <regression> <wily> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.25:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1511822>
<mup> Bug # opened: 1425808, 1457575, 1474607, 1493058, 1510651, 1513165, 1534643, 1538583, 1554088, 1573136, 1584616, 1587644, 1592887, 1596597, 1597830, 1599503, 1604988, 1610880, 1612645, 1613992, 1614364, 1614633, 1615118
<babbageclunk> voidspace: Did you have any trouble commissioning nodes in maas19?
<natefinch> rick_h_: btw, that email you sent about moving bugs from juju-core to juju was only sent to cloud@canonical - should it have been sent to juju and juju-dev@ubuntu also?  Seems like it's pretty important for everyone to see it.
<rick_h_> natefinch: yea, I guess I assumed folks are on the wider distribution lists
<rick_h_> natefinch: we're going to send a new email today focusing on how to file bugs and to confirm the move and such
<voidspace> babbageclunk: I didn't
<babbageclunk> voidspace: No worries, worked it out - missed a step in DHCP setup.
<voidspace> ah
<natefinch> rick_h_: cool.  Yeah, I honestly rarely look at the cloud list, because it's fairly spammy, and obviously people outside canonical aren't on that list
<rick_h_> katco: dimitern ping for standup
<mup> Bug #1616098 opened: Juju 2.0 uses random IP for 'PUBLIC-ADDRESS' with MAAS 2.0 <cpec> <juju-core:New> <https://launchpad.net/bugs/1616098>
<mgz> rick_h_: mini thing, should I add cards to the board for the random tasks I end up picking up? (like changing the release tools behaviour with patches and such)
<rick_h_> mgz: yes please
<rick_h_> mgz: anything that's more than a couple of hours to jfdi
<mgz> okay, thanks
<mgz> that one was actually pretty short, but I'll make sure to add anything meatier
<babbageclunk> Oh, of course trying to bootstrap to two maases at once gets all confused.
<natefinch> rick_h_: so, should I continue working on the add-cloud stuff or... ?
<rick_h_> natefinch: no, please grab a critical
<natefinch> rick_h_: will do
<mup> Bug #1616098 changed: Juju 2.0 uses random IP for 'PUBLIC-ADDRESS' with MAAS 2.0 <cpec> <juju-core:New> <https://launchpad.net/bugs/1616098>
<babbageclunk> dimitern: around?
<dimitern> babbageclunk: yeah
<babbageclunk> dimitern: hey! I'm just doing some manual testing of my machine undertaker worker (it works yay), but I want to try it with multi-nic containers.
<babbageclunk> dimitern: Is that just a matter of setting up the hosts with multiple nics and those will show up bridged in the containers?
<rick_h_> katco: ping, got a sec?
<katco> rick_h_: yep
<rick_h_> can you jump in https://hangouts.google.com/hangouts/_/canonical.com/developer?authuser=1 please?
<dimitern> babbageclunk: you need to also use maas
<natefinch> https://hangouts.google.com/hangouts/_/canonical.com/sorry-you're-fired?authuser=1
<babbageclunk> dimitern: yeah, sorry, should have said that - that's what I'm testing on.
<dimitern> babbageclunk: and all top-level nics that have children you want to bridge need addresses\
<dimitern> babbageclunk: but having that should give you multi-nic containers
<babbageclunk> dimitern: Awesome. And giving them addresses is just a matter of setting the nics to auto-assign in maas, right? Does it matter if they're all on the same subnet?
<dimitern> babbageclunk: auto assign or static assign should work - dhcp or unconfigured won't (the latter set on a parent NIC will lead to br-parent not getting created)
<babbageclunk> dimitern: Cool, it totally just works!
<babbageclunk> dimitern: This juju thing's pretty neat.
<dimitern> babbageclunk: awesome! :) when it works..
<mup> Bug #1536477 changed: utils/debugstatus: test failure <juju-core:Fix Released> <https://launchpad.net/bugs/1536477>
<mup> Bug #1538583 changed: manual provider add-machine fails immediately after bootstrap <add-machine> <ci> <manual-provider> <regression> <juju:Triaged> <juju-core:Invalid> <juju-core 1.25:Invalid> <https://launchpad.net/bugs/1538583>
<mup> Bug #1590161 changed: apiserver/client: panic: Session already closed <juju-core:Fix Released> <https://launchpad.net/bugs/1590161>
<natefinch> rick_h_: did you get the logs for this bug? https://bugs.launchpad.net/juju/+bug/1610880
<mup> Bug #1610880: Downloading container templates fails in manual environment <juju:Triaged by rharding> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1610880>
<rick_h_> natefinch: yes, sec. Let me find that email
<siv> Hi. I want to develop a juju charm which will go and  change the Openstack cinder service configuration file. created charm will perform this action post Openstack juju deployment. can I create a charm like this?
<rick_h_> siv: yes, look into https://jujucharms.com/docs/2.0/authors-subordinate-services to help do something like that
<siv> rich_h: Hi. I have an Ubuntu OpenStack autopilot setup. After the regular OpenStack install using AutoPilot, my created juju charm used to modify cinder driver settings. This is my intension.
<siv> Can I modify the cinder driver settings post autopilot openstack deployment using my created charm?
<mup> Bug #1616149 opened: Incompatible protocol between older client and candidate server <ci> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1616149>
<siv> How do a newly created juju charm get certified? what is the procedure for that? how many days that it will take to get certify the newly created juju charm?
<rick_h_> siv: check out https://jujucharms.com/docs/stable/authors-charm-store and https://jujucharms.com/docs/stable/charm-review-process
<siv> rich_h : thank you
<mup> Bug #1615958 changed: Switch to model name does't work anymore, admin should not be allowed to switch to users model <juju:New> <https://launchpad.net/bugs/1615958>
<mup> Bug #1615960 changed: remove-machine and remove-application don't work on google cloud <juju:New> <https://launchpad.net/bugs/1615960>
<mup> Bug #1616059 changed: Unit test failure in github.com/juju/juju/cmd/juju/application	 <juju:Triaged> <https://launchpad.net/bugs/1616059>
<mup> Bug #1615986 opened: Agents failing, blocked by upgrade but no upgrade performed <canonical-is> <juju-core:New> <https://launchpad.net/bugs/1615986>
<bdx> postgresql-peeps: say I have an application that needs to connect to 2 separate postgresql applications
<bdx> postgresql-peeps: is there a way to react to the states of the same service under a different name? Is this done by providing service sensitive interface names?
<bdx> charmers:^
<bdx> eeeh, s/postgresql applications/postgresql db instances/
<marcoceppi> bdx: hey
<marcoceppi> bdx: might be better to ask in #juju
<marcoceppi> bdx: but you can define two different relations with the same interface
<marcoceppi> and that way have a scope of reacting to each relation name
<bdx> my bad, thought I was on #juju ... grr
<marcoceppi> bdx: no worries! does my reply make sense?
<bdx> yes, can you give more context in #juju though?
<natefinch> sigh... the instructions for bootstrapping to manual don't actually work :/ https://jujucharms.com/docs/stable/clouds-manual
<mup> Bug #1474607 changed: worker/uniter/relation: HookQueueSuite.TestAliveHookQueue failure <ci> <go1.5> <go1.6> <regression> <windows> <juju:Fix Released by axwalk> <juju-core:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1474607>
<mup> Bug #1616197 opened: juju restore-backup error <juju-core:New> <https://launchpad.net/bugs/1616197>
<bdx> marcoceppi: i see now
<bdx> marcoceppi: thx
<bdx> exactly what I was thinking
<alexisb_> thumper, ping
<thumper> alexisb_: hey
<alexisb_> thumper, can you take a look at htis bug: https://bugs.launchpad.net/juju/+bug/1616158
<mup> Bug #1616158: Int overflow during build for Windows <blocker> <ci> <regression> <juju:Triaged by alexis-bruemmer> <https://launchpad.net/bugs/1616158>
 * thumper looks
<alexisb_> https://github.com/juju/juju/commit/5ac0e259560382ea0c0e2a4120e028f2613ed857
<alexisb_> ^^ this is the suspect commit
<natefinch> we don't build x86 anymore, do we?
<thumper> alexisb_: poo
<thumper> alexisb_: mgo is assuming a 64 bit int
<thumper> seems that windows still specifies 32?
<natefinch> thumper: only if we're specifying a 32 bit architecture
<thumper> natefinch: thoughts on the bug https://bugs.launchpad.net/juju/+bug/1616158
<mup> Bug #1616158: Int overflow during build for Windows <blocker> <ci> <regression> <juju:Triaged by alexis-bruemmer> <https://launchpad.net/bugs/1616158>
<thumper> mgo.v2/bson/json.go:320: constant 9007199254740992 overflows int
<natefinch> thumper: yeah, I was just looking at it.
<thumper> int is a signed integer type that is at least 32 bits in size
<thumper> ^^ language spec
<thumper> how can we find out how many bits the int uses?
<natefinch> thumper: if you build with GOOS=windows GOARCH=amd64 it works fine  if you build with GOARCH=386 regardless of OS (windows or linux) it fails
<natefinch> so... let's not build with GOARCH=386
<natefinch> there's something tickling inside my head that says we were doing that on purpose for some reason... but I can't remember why now
<natefinch> sinzui: do you know why we're building juju for windows as 32 bit?
<mup> Bug #1616149 changed: Incompatible protocol between older client and candidate server <ci> <regression> <test-failure> <juju-core:Won't Fix> <https://launchpad.net/bugs/1616149>
<thumper> natefinch: can you try replacing line 320 with this:
<thumper> if int64(n) <= 1<<53 {
<thumper> and then test the 386
<natefinch> sure.. you can do it too (just sayin').  cross compile in go 1.5 and up is just "GOOS=windows GOARCH=386 go build ./..."
<thumper> hmm...
<thumper> ok
<thumper> ok that works
<thumper> alexisb_: we have two choices
<thumper> alexisb_: stop building the windows client with GOARCH=386
<thumper> alexisb_: or patch mgo
<natefinch> either way, we should file a bug against mgo... I'm sure it was not intentional to break 386
<sinzui> natefinch: yes, *you* explained we should because Windows 64 bit is not common. We never chnged the rules you wrote.
<thumper> heh
<sinzui> natefinch: If you believe 64bit clients are good for windows, we can change the build script in a few hours
<thumper> apparently 32 bit versions stopped with windows 7 and vista
<thumper> oh
<thumper> no
<thumper> 8.1 can still be in 32 bit
<thumper> we should support 32 bit still
<thumper> so... patch mgo
<thumper> and file a bug etc etc
<natefinch> I guess so.  honestly, I doubt anyone actually runs 32 bit windows anymore, but if 8.1 supports it, we probably should too
<alexisb_> we dont support 32 bit linux
<natefinch> and yes, 32 bit applications run on both 32 bit and 64 bit windows... the reverse is not true.
<alexisb_> why would we do that for windows?
<sinzui> thumper: natefinch : I the client is needs about 5 lines of change to switch to 64 bit when you are ready. We just match the rules of the win agent
<natefinch> alexisb_: previously, presumably because it was free
<perrito666> alexisb_: I believe that is because ubuntu no longer builds for 32
<natefinch> alexisb_: i.e. no extra work
<thumper> well... how many people likely to be using juju will be using 32 bit windows?
<thumper> perhaps we just make it 64 bit and see who screams?
<thumper> alexisb_: who makes this call?
<alexisb_> sinzui, do we have stats on 32 bit windows client downloads
<sinzui> alexisb_:  to be clear, agents cannot be 32 bit in clouds. and Ubuntu doesn't support 32 bit server anywhere. Clients are in a grey area. Ubuntu makes 32 bit clients and we don't say no.
<sinzui> alexisb_: I can get them from the milestone pages
<natefinch> alexisb_:  the problem is, we can't tell if people downloading have 32 or 64 bit windows, since the 32 bit download is all we offer.
<natefinch> pretty sure you haven't been able to buy a machine with 32 bit windows in several years.... so the only people with 32 bit would be those who upgraded an old machine
<thumper> Given today's hardware and defaults, I think we'd be fine saying 64 bit only windows...
<thumper> I'll file a bug on mgo though
<natefinch> gotta run to dinner, back in a bit
<thumper> https://github.com/go-mgo/mgo/issues/325
<thumper> simple review for someone: http://reviews.vapour.ws/r/5516/
<perrito666> thumper: shipit, btw, you used the wrong dialect of markup
<thumper> :)
<thumper> w00t, current master lets me do this: `juju debug-log --replay --color --location | less -R`
<thumper> for show me the log in color, no tail
<thumper> marcoceppi, jcastro: ^^
<mup> Bug #1513165 changed: Containers registered with MAAS use wrong name <cdo-qa-blocker> <sts> <sts-needs-review> <juju:Fix Released by thumper> <juju-core:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1513165>
<mup> Bug #1421270 changed: juju deploy failure WebSocketConnection closed <api> <oil> <oil-bug-1372407> <juju-core:Won't Fix> <juju-deployer:Invalid> <https://launchpad.net/bugs/1421270>
<wallyworld> redir: did you want to chat?
<redir> y
<redir> brt
<marcoceppi> thumper: --color isn't default ;)
<thumper> marcoceppi: yes it is
<marcoceppi> oh, well I saw it explicitly called out
<thumper> marcoceppi: but just like ls, if it isn't a terminal, it isn't there by default
<marcoceppi> oh, because of the detection stuff, nvm
<thumper> :)
<perrito666> meh, squid never comes ok out of a reboot
<thumper> perrito666: oh, is that what does it?
<thumper> hadn't worked that out
<perrito666> thumper: at least for me, needs to restart squid-deb-proxy
<thumper> do you clear out the cache as well?
<thumper> I had always assumed that was necessary
<perrito666> thumper: I find it empty, I assume it empties on reboot?
 * thumper shrugs
<perrito666> nah, that is not the cache, my bad
<perrito666> was looking at wrong dir
<perrito666> I dont clean it
<wallyworld> alexisb_: standup? we miss you
<babbageclunk> Hey thumper! Long time no see.
<thumper> o/
<babbageclunk> Could you review this for me? https://github.com/juju/gomaasapi/pull/56
<babbageclunk> thumper: ^^
 * thumper looks
<thumper> babbageclunk: LGTM
<babbageclunk> thumper: Awse. Thanks!
<perrito666> is it me or we all sound like daft punk songs in this call?
#juju-dev 2016-08-24
<katco> tvansteenburgh: ping, you on?
<tvansteenburgh> katco: yup
<katco> tvansteenburgh: heya
<katco> tvansteenburgh: re. bug 1611514, what does the API do now when you try and add a charm and then deploy it via "local:"?
<mup> Bug #1611514: "local" charm schema should allow deploying previously deployed local charms <feature> <juju:In Progress by cox-katherine-e> <https://launchpad.net/bugs/1611514>
<tvansteenburgh> katco: it returns a message that it can't find the charm
<katco> tvansteenburgh: what API calls are you using?
<tvansteenburgh> katco: i really should've kept that test script :/
<katco> tvansteenburgh: haha
<tvansteenburgh> katco: i can add steps to repro but not til tomorrow, i'm well past eod
<katco> tvansteenburgh: same here, but that would help. i'm coming at this from the juju command side of things. i'll probably cover your case as well, but want to make sure
<katco> tvansteenburgh: (i.e. tomorrow is just fine)
<tvansteenburgh> katco: cool
<katco> tvansteenburgh: thanks :) looking forward to getting this taken care of for you all
<alexisb_> redir, are you actually working this bug: https://bugs.launchpad.net/juju/+bug/1614571
<mup> Bug #1614571: Data race: TestLogOutput <ci> <jujuqa> <race-condition> <unit-tests> <juju:In Progress by reedobrien> <https://launchpad.net/bugs/1614571>
<redir> alexisb_: while waiting for test runs, yes.
<alexisb_> ok, I am putting it in the inprogress lane than, htanks
<redir> but if there's someone else to work on it then take it away!
<alexisb_> nope
<redir> OK
<redir> bbiab couple hours or so.
<katco> wallyworld: ping, got a sec to chat?
<wallyworld> sure
<wallyworld> katco: 1:1?
<katco> wallyworld: yep
<axw> wallyworld: FYI going out soon, maybe be out for an hour or two
<wallyworld> axw: no worries, let's chat briefly when you get back
<thumper> wallyworld: http://reviews.vapour.ws/r/5518/
<thumper> +3 â1,688
<thumper> one of the biggest removals I've done in a while
<thumper> wallyworld, axw, anastasiamac, perrito666: anyone know of a charm that says it has payloads?
<anastasiamac> thumper: no, maybe best to ask charmers? stuart might have an idea :)
<menn0> veebers: the change to block access to importing models has landed btw
<veebers> menn0: cool, so we shouldn't see any "mongo not in the proper state" or whatever error? (I also added a check on test end to ensure the charm/deploy had fully finished
<menn0> veebers: yep, that error shouldn't happen now. the only concern is, what will the test do with the error that gets returned while the model is importing?
<veebers> menn0: hmm, I'm pretty sure the minor change I added should over that. we'll see soon enough if there is any further additions needed
<menn0> cool
<natefinch> thumper: lol, payloads. Sigh.
<thumper> they exist, so they need to be migrated :-|
<natefinch> at least it's a pretty trivial amount of information
<menn0> thumper: review done
<thumper> ta
<thumper> natefinch: yeah
<redir> wallyworld: yt?
<wallyworld> yt?
<wallyworld> young transvestite?
<thumper> natefinch: do you recall the hook tool to register a payload?
<natefinch> thumper: buh
<thumper> nm
<thumper> I was busy looking in worker/uniter/runner/jujuc
<thumper> where all the other hook tools were
<natefinch> why would you do a silly thing like that?
<redir> wallyworld: you there... Do we still want nil here https://goo.gl/y8F48q
<wallyworld> let me look
<mup> Bug #1616298 opened: DebugMetricsCommandSuite.TearDownTest fails due to "no reachable servers." <juju-core:New> <https://launchpad.net/bugs/1616298>
<wallyworld>  we pass in nil for controller config for that test func, so no harm in also not setting inherited region config either
<natefinch> good lord that needs an args struct
<natefinch> heh.... st, err := state.Initialize(state.InitializeParams{
<wallyworld> redir: bit you'll need to add a test in UpdateModelConfig to ensure that removing a config key correctly picks up a region default
<natefinch> well at least the production code uses an args struct
<wallyworld> because that's the place that uses the controllerInheritedConfig func that was incorrect and we didn;t have test coverage
<veebers> menn0: seems that attempting to call status while model is in migration causes a failure: http://juju-ci.vapour.ws:8080/job/functional-model-migration/118/console
<veebers> the failure is because the call to 'juju status' errors
<mup> Bug #1616298 changed: DebugMetricsCommandSuite.TearDownTest fails due to "no reachable servers." <juju-core:New> <https://launchpad.net/bugs/1616298>
<veebers> menn0: should status for the model error or just provide status of 'migrating' or similar?
<mup> Bug #1616298 opened: DebugMetricsCommandSuite.TearDownTest fails due to "no reachable servers." <juju-core:New> <https://launchpad.net/bugs/1616298>
<natefinch> hmmm.... does the manual provider do something special with mongo?  seems like I may have broken manual when I updated TLS ciphers in 1.25
<natefinch> that or I'm just hitting a weird similar-looking bug.  I bootstrap manual on 1.25 and it can't connect to mongo
<natefinch> or rather, it can't connect to itself.... different symptom than what I was thinking of, I think.... still
<natefinch> 2016-08-23 20:29:01 WARNING juju.replicaset replicaset.go:98 Initiate: fetching replication status failed: cannot get replica set status: can't get local.system.replset config from self or any seed (EMPTYCONFIG)
<natefinch> Anyone have any ideas?
<anastasiamac> axw: wallyworld: PTAL http://reviews.vapour.ws/r/5502/ :D
<wallyworld> soon, need to get some stuff finished
<anastasiamac> wallyworld: nps. i prefer to wait for axw anyway
<natefinch> menn0, wallyworld - thoughts on my mongo problem above?
<wallyworld> nfi
<wallyworld> could be anything, literally
<wallyworld> network, config
<wallyworld> you need to drill down to find the root cause
<menn0> natefinch: that happens sometimes. it's been a problem for a long time. there's a ticket for it.
<menn0> natefinch: I've tried to figure it out before but was never able to replicate it. lots of ppl have seen it though.
<menn0> natefinch: is this during a test or a bootstrap?
<natefinch> menn0: hmm.... what I've seen before is that the machine thinks it has some hostname, but then the networking is screwed up so it can't actually connect to that hostname for itself.  But not sure if that's it here.  I'll try again.
<natefinch> menn0: bootstrap
<menn0> natefinch: are there any logs you can grab?
<natefinch> menn0: probably.  I'll try logging into the mongo console and see what the replicaset collection says
<menn0> natefinch: that would be good. given that it's hard to replicate it would be good if you could grab everything you can.
<menn0> natefinch: in the past it's been people outside the team who have seen it and they either didn't collect logs or the logs weren't at DEBUG
<menn0> natefinch: so this is a good opportunity to learn more
<natefinch> menn0: this is my second failure in a row like this
<menn0> natefinch: even better :)
<menn0> natefinch: this is the closest open ticket: https://bugs.launchpad.net/juju/+bug/1346597
<mup> Bug #1346597: cannot get replset config: not authorized for query on local.system.replset <cloud-installer> <landscape> <oil> <juju:Incomplete> <juju-core:Won't Fix> <juju-core 1.24:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1346597>
<menn0> natefinch: the title has a slightly different error but there's discussion about the same problem you're seeing
<natefinch> menn0: I'll take a look
<menn0> natefinch: pls attach any bootstrap and agent logs,  the mongodb logs from /var/log/syslog and what you find out from looking at the replicaset status in the shell
<axw> anastasiamac: sorry took longer than expected, just having lunch then will review
<anastasiamac> axw: awesome \o/ most importantly - di u get a car?
<anastasiamac> did*
<axw> anastasiamac: yes, bought a qashqai
<anastasiamac> \o/
<anastasiamac> axw: found an issue anyway while waiting - I can deploy fine to "controller" model (image is found!) but on hosted model like "default", m still seeing  "image not found"
<anastasiamac> axw: so, i'd like to land what i have and address multi-model as a separate PR... (in addition to separate PR for the bug that I have identified with simplestreams)...
<anastasiamac> axw: and Nissans are awesome :D well done!
<natefinch> menn0: looks like a networking problem. I can telnet to localhost:37017, but can't telnet to <myIP>:37017, and that's what's in the replicaset config
<natefinch> (from that machine, obviously)
<menn0> natefinch: ok, that's super useful information
<menn0> natefinch: is it because the wrong IP is being used in the replicaset config? (i.e. the machine has multiple ips and Juju is choosing the wrong one?)
<menn0> natefinch: (pls update the ticket with these findings)
<natefinch> menn0: it's a GCE machine, it's putting its public IP, not the internal IP
<natefinch> O
<natefinch> but for manual, that should be correct
<menn0> natefinch: firewall rules?
<natefinch> except that maybe we're not fixing the firewall
<natefinch> menn0: $ sudo iptables -S
<natefinch> -P INPUT ACCEPT
<natefinch> -P FORWARD ACCEPT
<natefinch> -P OUTPUT ACCEPT
<natefinch> that's it
<menn0> natefinch: GCE will have it's own firewall external to the hosts
<natefinch> right
<menn0> controlled via the API
<axw> anastasiamac: reviewed
<axw> anastasiamac: I agree that the other issue should be solved separately
<anastasiamac> axw: \o/ brilliant! made my day :D
<axw> wallyworld: you wanted to chat?
<menn0> thumper: review done
<natefinch> welp, I have to bed
<natefinch> menn0: added some comments on the bug - https://bugs.launchpad.net/juju/+bug/1346597
<mup> Bug #1346597: cannot get replset config: not authorized for query on local.system.replset <cloud-installer> <landscape> <oil> <juju:Incomplete> <juju-core:Won't Fix> <juju-core 1.24:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1346597>
<natefinch> menn0: might be worth making a new bug for the different error message, just to make it easier to find
<menn0> natefinch: good idea (might be worth checking if there's already one that's been closed before)
<natefinch> menn0: https://bugs.launchpad.net/juju-core/+bug/1412621
<mup> Bug #1412621: replica set EMPTYCONFIG MAAS bootstrap <adoption> <bootstrap> <bug-squad> <charmers> <cpec> <cpp> <maas-provider> <mongodb> <oil> <juju-core:Fix Released by frobware> <juju-core 1.24:Won't Fix> <juju-core 1.25:Fix Released by frobware> <https://launchpad.net/bugs/1412621>
<natefinch> last comment is about someone using manual on EC2 and needing to open the ec2 firewall :P
<menn0> natefinch: so it was only fixed for EC2?
<natefinch> menn0: I don't think it's exactly fixable when using the manual provider
<menn0> try
<menn0> true
<menn0> I guess with manual people are on their own
<natefinch> but... juju should test for this case and give a better error message
<menn0> exactly what I was just about to say :)
<natefinch> :)
<natefinch> I'll make a bug for that
<redir> ot valid"
<redir> ignore that paste error
 * redir goes eod
<wallyworld> redir: sorry, had to go out at short notice
<natefinch> menn0: https://bugs.launchpad.net/juju/+bug/1616310
<mup> Bug #1616310: Test replicaset address before replSetInitiate <mongodb> <juju:New> <juju-core:New> <https://launchpad.net/bugs/1616310>
<natefinch> and now I am EOD as well
<menn0> natefinch: nice bug report
<menn0> natefinch: thanks
<natefinch> menn0: welcome :)
<wallyworld> axw: yeah, if you have a moment, we can talk
<axw> wallyworld: sure, 1:1?
<wallyworld> ok
<mup> Bug #1591488 opened: Unable to bootstrap with --metadata-source <cpe-sa> <eda> <juju:In Progress by anastasia-macmood> <juju-core:Fix Committed> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1591488>
<mup> Bug #1591488 changed: Unable to bootstrap with --metadata-source <cpe-sa> <eda> <juju:In Progress by anastasia-macmood> <juju-core:Fix Committed> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1591488>
<anastasiamac> wallyworld: do u recall if resources are model-specific or controller-?
<wallyworld> they are specific to a charm
<wallyworld> stored in the blob store
<anastasiamac> wallyworld: sure, but once we put reference to them in mongo, are they model-bound or available to all models for this controller?
<wallyworld> i can't recall how the path is set up
<anastasiamac> wallyworld: we definitely have "resources" collection in db
<wallyworld> i'd have to double check
<wallyworld> is that in state or blobstore
<wallyworld> if in state, it would justr be the references
<anastasiamac> wallyworld: m looking in state
<wallyworld> if it's marked a global collection, it is for all models
<anastasiamac> well, it's not.. and m wondering if it should be..
<wallyworld> don't think so, i think the design was to tie resources to models
<anastasiamac> k
<wallyworld> the data in blob store is deduped
<wallyworld> so even if the same reosurce is assigned to two models, it will only be stored once
<mup> Bug #1591488 opened: Unable to bootstrap with --metadata-source <cpe-sa> <eda> <juju:Fix Committed by anastasia-macmood> <juju-core:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1591488>
<mup> Bug #1591488 changed: Unable to bootstrap with --metadata-source <cpe-sa> <eda> <juju:In Progress by anastasia-macmood> <juju-core:Fix Committed> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1591488>
<anastasiamac> axw: wallyworld: PTAL https://github.com/juju/juju/pull/6077
<anastasiamac> dunno why RB is not picking it up
<anastasiamac> it's tiny but oh, so wonderful :D
<axw> anastasiamac: I don't think this is the correct solution. different models will want different image metadata
<anastasiamac> axw: right now, how will they get it?
<axw> anastasiamac: what is "they"?
<anastasiamac> axw: at this stage, not being able to deploy to any other model but "controller" is going to b poor experience for cloud
<anastasiamac> they = other models
<axw> anastasiamac: I don't disagree, but that doesn't mean we should break the data model
<anastasiamac> axw: when i say "cloud", i mean private cloud or anyone using metadata-source, for example
<axw> anastasiamac: perhaps what we should do is combine sources, like I did a while ago for tools
<axw> anastasiamac: look first in the model, then in the controller's model
<anastasiamac> axw: I think that we can do this for now, and have a wishlist item for bigger and shinier solution when we get a chance to tackle it, say for 2.1
<axw> and in the case of image metadata, only look in the controller's if they're the same cloud
<anastasiamac> axw: sure, but right now, there is no way to add to model image collection
<axw> anastasiamac: if we do this quick hack, at least the image metadata worker needs to be made singular
<axw> hm, but probably not
<axw> because multiple regions
<anastasiamac> i think what i propose is good for a while until we iterate... it may even be tackled along with metadta in config.. :D
<axw> anastasiamac: I can live with this, but please remove the model-uuid field from cloud image metadata
<axw> anastasiamac: and please live test in a cloud with multiple regions
<anastasiamac> axw: i've pmed u.. m live testing against aws. or do u want me to test deploying/adding model to a different region than bootstrap?
<axw> wallyworld: LXD improvements: https://github.com/juju/juju/pull/6078  -- RB has crapped itself
<wallyworld> axw: awesome, otp will look soon
<anastasiamac> axw: PR is ready to go: removed model-uui and tested live with multi-regions \o/
<axw> anastasiamac: reviewed
<anastasiamac> wallyworld: PTAL https://github.com/juju/juju/pull/6077
<wallyworld> still otp :-(
<anastasiamac> :(
<hoenir> the .fail is looking good. http://juju.fail/
<wallyworld> axw: if you have a moment before eod, would like to land this for beta16 https://github.com/juju/juju/pull/6079
<axw> wallyworld: looking
<wallyworld> ta
<axw> wallyworld: reviewed
<wallyworld> ta
<wallyworld> axw: with the embedding, i was going to wait and see how nate's stuff played out
<wallyworld> i expect it will change somewhat
<wallyworld> i can add a todo i guess
<axw> wallyworld: pretty sure each provider is going to have to have a schema either way, it's just a different type of schema
<wallyworld> yeah, fair enough
<wallyworld> everything else fixed
<babbageclunk> frobware: No dimitern?
<frobware> babbageclunk: might have been up late fixing an issue
<babbageclunk> frobware: Ahh.
<babbageclunk> frobware: (Not to make you feel like second choice but) could you take a look at this? http://reviews.vapour.ws/r/5521/
<frobware> babbageclunk: in a second... :)
<frobware> babbageclunk: otp
<frobware> babbageclunk: going to have to come back to this today - need to urgently fix/verify bug #1616098
<mup> Bug #1616098: Juju 2.0 uses random IP for 'PUBLIC-ADDRESS' with MAAS 2.0 <cpec> <juju:Triaged by dimitern> <https://launchpad.net/bugs/1616098>
<babbageclunk> frobware: No hurry!
<mup> Bug #1615552 opened: Juju 2: Cannot add new key to default model config. <juju-core:Fix Committed by wallyworld> <https://launchpad.net/bugs/1615552>
<perrito666> wallyworld: still around?
<wallyworld> no
<wallyworld> maybe
<wallyworld> depends who wants to know
<perrito666> wallyworld: irs
<perrito666> nah, I just pushed the changed we discussed last night
<wallyworld> ok
 * wallyworld looks soon
 * perrito666 makes a joke about looking old instead of soon
<siv> ,m'
<babbageclunk> fwereade: Are you working today?
<fwereade> babbageclunk, I am, but I have to go out on an extended errand very shortly -- how can I help?
<babbageclunk> fwereade: I'm working on the unit tests for the machine undertaker, but I've put a WIP version up for review just in case there are any things tha t jump out at you.
<babbageclunk> fwereade: Want to get any nasty surprises early! ;)
<babbageclunk> fwereade: Annoyingly I seem to have formatted my PR text in a way that makes the RB bot think it's already got a review.
<fwereade> haha
<babbageclunk> fwereade: recreating now
<fwereade> babbageclunk, ah, ok, I was just peering at /pulls and wondering what I was missing ;p
 * babbageclunk shrugs
<babbageclunk> fwereade: Oh well, here it is on github: https://github.com/juju/juju/pull/6082
<fwereade> babbageclunk, looks eminently sane, thanks, only the most trivial notes at first glance
<babbageclunk> fwereade: Ooh, I hadn't noticed you'd reviewed my other one, thanks!
<babbageclunk> fwereade: Great!
<fwereade> :D
<fwereade> later all
<fwereade> I might well not make standup but I'll be around later
<babbageclunk> o/
 * babbageclunk is off to lunch.
<wallyworld> perrito666: why don't you think a test is needed?
<perrito666> wallyworld: I think the test is there
<wallyworld> sure? why did CI fail then?
<perrito666> wallyworld: I added the test as part of that PR
<wallyworld> adding code without changing a test doesn't make sense
<wallyworld> ok, let me check the test
<perrito666> http://reviews.vapour.ws/r/5517/diff/2/#1
<wallyworld> perrito666: you are right, i thought remove user deleted rather than marking as inactive
<wallyworld> ignore me
<perrito666> I usually do :p
<ram___> Hi all. For writing a JUJU which language is preferable with community? Actually I want to develop in Shell script.
<ram___> for writing a juju charm
<ram___> Please anyone suggest me
<perrito666> ram___: I believe that the ideal platform is python, if you ask in #juju you might get better answers
<ram___> perrito666 : OK. thank you.
<SimonKLB> hey guys, im getting a panic on aws when deploying the openstack bundle and i think it might be due to lxc being removed from the supported container types
<SimonKLB> 2016-08-24 11:32:11 INFO juju.provisioner container_initialisation.go:98 initial container setup with ids: [6/lxc/0 6/lxc/1 6/lxc/2]
<SimonKLB> 2016-08-24 11:32:11 INFO juju.worker runner.go:262 stopped "6-container-watcher", err: worker "6-container-watcher" exited: panic resulted in: runtime error: invalid memory address or nil pointer dereference
<SimonKLB> it could be caused by https://github.com/juju/juju/blob/master/worker/provisioner/container_initialisation.go#L102
<katco> SimonKLB: hi, sorry you're experiencing some issues. can you please file a bug @ https://bugs.launchpad.net/juju/+filebug ?
<katco> SimonKLB: and please be sure to attach logs, include information on what version you're using
<SimonKLB> yea sure, but might be good to verify asap that it isn't as critical as it looks, probably more people than me that are using the openstack bundle on aws
<katco> SimonKLB: sure. what version of Juju are you running?
<SimonKLB> 2.0 beta15
<katco> SimonKLB: let me just check and see if this has already been reported/fixed
<katco> SimonKLB: i don't see anything being tracked
<SimonKLB> alright, ill file a bug then
<SimonKLB> might just be something on my end
<katco> SimonKLB: ta. i'm also checking with our internal openstack team to see if they've seen anything similar.
<SimonKLB> katco: great thanks
<katco> SimonKLB: also, <standard disclaimer about betas and such> ;)
<SimonKLB> katco: haha yea no worries!
<katco> SimonKLB: are you just deploying openstack-base?
<SimonKLB> katco: i was deploying the bundle that included telemetry
<katco> frobware: fwereade: mgz: standup time
<SimonKLB> i can try the base only
<katco> SimonKLB: no worries, just include that info in the bug so we can repro
<mgz> katco: omw
<dimitern> frobware, fwereade: standup?
<dimitern> babbageclunk: ? if you like ^
<babbageclunk> dimitern: Sure!
<dimitern> jam: ping
<mattyw> katco, ping?
<katco> mattyw: sec, in standup
<mattyw> katco, no problem, it can wait
<katco> mattyw: you can ask async if you like :)
<mattyw> katco, I'm making some changes to the deploy command, my understanding is your subjecting it to a much needed refactoring, just wanted to make sure we don't step on each other, or avoid it as much as possible :)
<natefinch> macgreagoir: btw, the phone home stuff is in github.com/juju/juju/charmstore/latest.go
<natefinch> macgreagoir: jujuMetadataHTTPHeader
<macgreagoir> Cheers
<katco> mattyw: the change i'm making to it now won't refactor anything, so we should be ok. out of curiosity, what are you working on?
 * dimitern figured out what's wrong with his lxd on maas setup !
<macgreagoir> natefinch: Oh... I have that open too :-)
<dimitern> lxd on maas is 2.0.3
<dimitern> mgz: so this is a possible issue we were talking about wrt packaging I guess
<jamespage> SimonKLB, hey  - so the telemetry and base bundles are designed to deploy on hardware - am I correct in reading you're using the bundle on AWS?
<mgz> dimitern: well, we have potential confusions
<mgz> in theory, if everything using lxd on one machine is using the api
<mgz> it shouldn't matter if they were compiled with different minor versions of the go library
<dimitern> mgz: yeah, but *if* it happens to be <2.0.4, juju will be unable to deploy lxd with it
<mgz> oh, with trunk? why is that?
<dimitern> mgz: and that's really lxd's fault - they changed the 1.0 api across not even a minor version release
<SimonKLB> jamespage: yes, is that not possible you say?
<mgz> er....
<jamespage> SimonKLB, its tricky
<mgz> that is surely a regression they just need to fix?
<dimitern> mgz: as it happens 2.0.3 lxd is the most recent version in the archive, so it might become an issue for people not using ppa:ubuntu-lxd/lxd-stable
<jamespage> SimonKLB, the main issue right now is that LXD containers are not fully networked on AWS; that's relatively easy to work-around by updating the bundle to be 'sparse' i.e. an instance per service, rather than using containers at all
<jamespage> I understand that this willchange once container networking is fully implemented across providers.
<redir> morning
<katco> redir: morning
<SimonKLB> jamespage: i see, would local lxd containers even be an option or is maas the only provider that works well at the moment?
<jamespage> SimonKLB, the other problem areas are a) AWS does not support nested KVM (so you have to use the 'lxd' or 'qemu' virt-type option on the nova-compute charm) and b) I think they block alot of overlay network types and c) you can get a raw l2 port for north-south traffic routing
<jamespage> SimonKLB, https://github.com/openstack-charmers/openstack-on-lxd would let you do a whole stack in a single machine
<SimonKLB> jamespage: yea i mean this is only while developing, running a cloud inside aws wouldnt make much sense hah :)
<jamespage> such as your laptop
<SimonKLB> jamespage: that would be neat!
<jamespage> SimonKLB, co-incidentally I was just migrating the docs on those bundles into our OpenStack charm dev guide
<SimonKLB> unfortunately the charm im working on is running docker containers which is not supported by LXD on juj atm
<SimonKLB> juju*
<jamespage> but the content is much the same
<lazyPower> SimonKLB - thats mostly true
<lazyPower> SimonKLB - you can manually apply the docker profile to your lxd containers, and if you're using the docker.io package from apt, it will work inside lxd
<SimonKLB> as soon as profiles are implemented it should be good to go i guess?
<SimonKLB> lazyPower: aah, nice
<lazyPower> limited, but it will work. I think you'll be missing some of the cgroups management bits of the docker daemon due to how that profile functions.
<lazyPower> we're still actively working through some of those limitations ourselves wrt k8s. and yes, the missing profile apply from juju is a bummer at the moment but we remain hopeful ti will land in the next cycle after 2.0 gets pressed as release
<SimonKLB> cool!
<SimonKLB> katco: marcoceppi i tried deploying using the commandline client this time and it used lxd containers instead, could there be something that converts the yaml configuration from lxc to lxd in the commandline client and not when deploying through juju-gui?
<katco> SimonKLB: it looks like for bundles we do convert LXC placements to LXD
<SimonKLB> katco: yes but only in the commandline client and not in juju-gui?
<katco> SimonKLB: ahh that i don't know. urulama ?
<urulama> SimonKLB: which version of GUI are you using?
<SimonKLB> urulama: how do i see that, i started it after i bootstrapped aws
<urulama> SimonKLB: juju upgrade-gui --list
<SimonKLB> $ juju upgrade-gui --list
<SimonKLB> 2.1.8
<SimonKLB> 2.1.2
<urulama> ok, so it's 2.1.8
<urulama> that should be covered
<urulama> SimonKLB: and, because of all the beta changes, which juju?
<SimonKLB> beta15
<urulama> SimonKLB: ok, please download tarball here https://github.com/juju/juju-gui/releases/tag/2.1.10 and run "juju upgrade-gui /path/to/tarball"
<urulama> it'll get into official streams soon
<frankban> urulama, hatch, SimonKLB: AFAICT in the GUI we do not make the conversion LXC -> LCD when getting bundle changes for Juju
<frankban> hatch: could you please create a card for that? the logic is quite easy: if containerType == 'lxc' and juju version >= 2 then containerType = 'lxd'
<hatch> sorry I missed the first part of this conversation, is the bundle that's being imported have machine placement directives with lxc with Juju 2?
<frankban> hatch: I guess so
<katco> tvansteenburgh: hey when you get around to creating that script, can you just attach it to the bug?
<katco> tvansteenburgh: (and ping me)
<tvansteenburgh> katco: i did
<katco> tvansteenburgh: ah ok, ty!
<SimonKLB> urulama: i dont seem to be able to search the search the store with 2.1.10
<SimonKLB> search the store*
<urulama> hm, that's new
<urulama> frankban: ^
<hatch> frankban: I have confirmed you're correct, we do not munge the machine placement based on the Juju version
<hatch> I'm not entirely sure we should
<SimonKLB> combo?app/components/search-results/search-results-select-filter-min.js&app/components/search-resulâ¦:5 Search request failed.
<tvansteenburgh> katco: np, lemme know if you have any questions
<hatch> SimonKLB: can you open up the network tab and inspect the request
<hatch> did it 404?
<frankban> hatch: we should, but let's talk on daily
<urulama> uiteam: call
<SimonKLB> urulama: haha turns out it was the "privacy badger" addon :D
<SimonKLB> thought you guys were tracking me
<urulama> LOL
<urulama> not (yet) :D
<SimonKLB> ;D
<hatch> :)
<SimonKLB> alright lets see if it deploys this time
<SimonKLB> urulama: nope, lxc again
<hatch> SimonKLB: can you re-post the error? Sorry I logged in after
<SimonKLB> 2016-08-24 11:32:11 INFO juju.provisioner container_initialisation.go:98 initial container setup with ids: [6/lxc/0 6/lxc/1 6/lxc/2]
<SimonKLB> 2016-08-24 11:32:11 INFO juju.worker runner.go:262 stopped "6-container-watcher", err: worker "6-container-watcher" exited: panic resulted in: runtime error: invalid memory address or nil pointer dereference
<SimonKLB> when deploying the openstack bundle through the commandline client i seems to convert lxc to lxd in the yaml config
<SimonKLB> but not when deploying through the gui
<hatch> ahh
<hatch> ok yeah the gui definitely doesn't do that conversion
<hatch> at least not yet :)
<SimonKLB> :)
<hatch> SimonKLB: a simple workaround should be to just s/lxc/lxd
<hatch> at least to get you going right now
<SimonKLB> hatch: np, i can use the cli client
<SimonKLB> it wasnt super obvious to begin with though, i just happened to use the gui because i was lazy :)
<katco> tvansteenburgh: great comment, ta
<tvansteenburgh> katco: you're welcome!
<urulama> SimonKLB: we missed this in GUI when migrating from LXC to LXD ... it'll be a quick fix. thanks for letting us know
<SimonKLB> urulama: no worries!
<hatch> SimonKLB: we'll be fixing it shortly
<natefinch> fwereade: you around?
<ram____> [21:24] <ram_____> Hi. I followed https://jujucharms.com/docs/stable/getting-started. I deployed wiki charm. It was giving error. pasted error log : http://paste.openstack.org/show/563091/. please provide me the solution.
<dimitern> ram____: the lack of addresses shouldn't be an error in the log, the cause for containers getting stuck in pending is different I think
<dimitern> ram____: can you successfully run `lxc launch ubuntu:16.04 xenial-test` and then `lxc exec xenial-test -- 'ping google.com'` ?
<ram____> dimitern: Yes.
<dimitern> ram____: ok, so lxd is working without juju - always good to check that first :)
<dimitern> ram____: are you using any proxy in your network?
<jcastro> so do all new bugs go into lp.net/juju now?
<natefinch> jcastro: for 2.0 yes
<mgz> it's going to take some adjusting to :)
<fwereade> natefinch, hey, that errand was much more extended than expected, but I'm here
<ram____> dimitern: No proxy.
<natefinch> fwereade: see https://bugs.launchpad.net/juju/+bug/1616523
<mup> Bug #1616523: one badly formed credential makes bootstrap fail <bootstrap> <juju:New> <https://launchpad.net/bugs/1616523>
<natefinch> fwereade: wondering if we should try to soldier on if part of the yaml is not what we expect
<dimitern> ram____:  can you see any other errors? do any of the containers come up at all?
<fwereade> natefinch, yeah, definitely
<fwereade> natefinch, for historical interest, that's one thing we did do right in environments.yaml
<fwereade> natefinch, not try to interpret a given environment until someone needed it
<ram____> dimitern: No containers come up. paste output of # juju status --format yaml    http://paste.openstack.org/show/563096/
<ram____> dimitern: "Failed to get device attributes: no such file or directory"
<dimitern> ram____: ok, can you try juju kill-controller lxd-test -y and then juju bootstrap like before but add --debug and --config logging-config='<root>=TRACE' to it, then paste the output
<ram____> dimitern: $sudo juju bootstrap lxd-test localhost --debug --config  logging-config='<root>=TRACE'  output  http://paste.openstack.org/show/563099/.
<dimitern> ram____: no need for sudo btw
<dimitern> ram____: thanks, looking at the log
<katco> tvansteenburgh: ping
<dimitern> ram____: ok, so far so good - try `juju switch lxd-test:controller` then `juju add-machine -n 2` and then monitor it: `watch 'juju status --format=yaml'` until you get the error, then `juju ssh 0 -- sudo cat /var/log/juju/machine-0.log'` and paste the output?
<ram____> dimitern: juju machines created and running without any error. http://paste.openstack.org/show/563102/
<dimitern> ram____: nice!
<dimitern> ram____: so your issue is gone then?
<tvansteenburgh> katco: hey
<katco> tvansteenburgh: hey... i'm able to deploy a previous deploy charm as a new application without any changes to jujud
<katco> tvansteenburgh: so something is either wrong with your script or python-jujuclient
<katco> tvansteenburgh: i'm trying to figure out what
<tvansteenburgh> katco: eh, deploying a previously deployed charm as a new app wasn't the problem
<tvansteenburgh> katco: basically i need to know how to deploy a local charm with the api under juju2. the steps that worked for juju1 (the script) no longer work
<katco> tvansteenburgh: oops, that's what this bug is regarding. is it just deploying local charms?
<katco> tvansteenburgh: ah i see. check this out: https://github.com/juju/juju/blob/master/cmd/juju/application/deploy.go#L681-L725
<katco> tvansteenburgh: you *might* be missing a call to GetCharmInfo in between add_local_charm and deploy
<katco> tvansteenburgh: i.e. follow this logic: https://github.com/juju/juju/blob/master/cmd/juju/application/deploy.go#L401
 * katco goes to grab some lunch
<ram____> dimitern : Thanks.
<dimitern> ram____: you're very welcome :) I'm glad it worked!
<balloons> are there any PR's folks are waiting on? I'm trying to test the new machine for the lander, but nothing has come up. Are folks waiting on PR's?
<natefinch> thumper: morning
<thumper> morning
<natefinch> thumper: looks like someone is having another sort of these problems: https://bugs.launchpad.net/juju-core/+bug/1485784
<mup> Bug #1485784: Error creating container juju-trusty-lxc-template; Failed to parse config <lxc> <oil> <juju-core:Fix Released by cherylj> <juju-core 1.25:Fix Released by cherylj> <https://launchpad.net/bugs/1485784>
<natefinch> except it's here now: https://bugs.launchpad.net/juju/+bug/1610880
<mup> Bug #1610880: Downloading container templates fails in manual environment <juju:Triaged by rharding> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1610880>
<thumper> ugh
<natefinch> I repro'd with 1.25 manual deployed to a GCE instance
<natefinch> thumper: anything I should look for in particular?
<thumper> oh... is this the missing lxc-templates package?
<natefinch> I don't know?
<thumper> no
<natefinch> thumper: yeah, just checked, it's installed
<thumper> -- https://10.2.0.186:17070/environment/80234a11-2d53-436e-855c-da998c76d6ca/images/lxc/trusty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz
<thumper> Connecting to 10.2.0.186:17070... connected.
<thumper>     ERROR: certificate common name '*' doesn't match requested host name '10.2.0.186'.
<thumper> To connect to 10.2.0.186 insecurely, use `--no-check-certificate'.
<thumper> from the in after
<thumper> line
<thumper> I wonder who is creating the cert, and why it doesn't like it
<natefinch> I dont have that error in my logs... but otherwise the same problem
<natefinch> might be a red herring
<natefinch> ls
<thumper> natefinch: that is from trace output of the lxc package
<thumper> module golxc.run
<thumper> in the description of the bug
<natefinch> oh yeah.. I missed that it wasn't a juju log
<natefinch> where does lxc log?  I don't see anything in /var/log/lxc
<thumper> the golxc package uses loggo
<thumper> so it is in the juju log
<thumper> you just need to set the logging config appropriately
<natefinch> oh ok
<natefinch> thumper: 2016-08-24 20:50:21 TRACE golxc.run.lxc-create golxc.go:448 run: lxc-create [-n juju-trusty-lxc-template -t ubuntu-cloud -f /var/lib/juju/containers/juju-trusty-lxc-template/lxc.conf -- --debug --userdata /var/lib/juju/containers/juju-trusty-lxc-template/cloud-init --hostid juju-trusty-lxc-template -r trusty -T https://10.142.0.2:17070/environment/3d787fce-8f2a-49ea-8239-bc5ecda353c1/images/lxc/trusty/amd64/ubuntu
<natefinch> -14.04-server-cloudimg-amd64-root.tar.gz]
<natefinch> 2016-08-24 20:50:21 TRACE golxc.run.lxc-create golxc.go:458 run failed output: + '[' amd64 = i686 ']'
<thumper> ha
<thumper> interesting
<thumper> I missed that bit
<natefinch> that was in my log
<natefinch> after I turned on trace and ran another deploy
<thumper> it is in the bug description too
<thumper> where does that script come from?
<natefinch> no idea
<natefinch> if you want I can give you access to my repro machine.  I have to run for dinner time.  Will be back in about 4 hours.
<natefinch> thumper: ssh ubuntu@104.196.3.75   should work
<alexisb_> menn0, did you want to meet
<menn0> alexisb_: gah... sorry forgot
<menn0> alexisb_: I don't have anything specific to discuss
<alexisb_> menn0, ok, have you 30 mins back then :)
<menn0> alexisb_: ok :)
<thumper> wallyworld: if you have time after the release call, I'd love to chat about the tools look up stuff
<thumper> looking at manual s390x failures
<veebers> perrito666: I just tried a fresh juju and the updated grant/revoke test and get a failure, seems that the removed users are still in list-shares output. Digging further now
<wallyworld> ok
<wallyworld> menn0: trivial 4 or 5 line review if you have time http://reviews.vapour.ws/r/5524/
<menn0> wallyworld: looking
<menn0> wallyworld: ship it
<wallyworld> menn0: tyvm
<thumper> menn0: fixing because code landed before I did before http://reviews.vapour.ws/r/5525/
<menn0> thumper: looking
<menn0> thumper: ship it
<veebers> perrito666: yeah I see users that were removed appearing in list-shares
<wallyworld> thumper: did you still need to talk?
<thumper> wallyworld: let me finish this review for katco, then can we chat?
<wallyworld> ok
<wallyworld> alexisb_: i need to talk to tum, i'll ping when i'm free
<katco> thumper: not sure if you saw my message earlier; i pushed up new version. i had some dead-code-comments in there
<thumper> kk, I'll hit refresh
<alexisb_> k
<alexisb_> who is tum?
<katco> alexisb_: when wallyworld is hungry, that's who he talks to
<alexisb_> lol
 * wallyworld was ripping of the NZ accent
<wallyworld> *off
<thumper> katco: shipit
<thumper> wallyworld: chat?
<wallyworld> sure
<wallyworld> 1:1 i guess
<katco> thumper: ta for the quick turn-around
<alexisb_> wallyworld, o I see how it is
<alexisb_> perrito666, since it is obvious that wallyworld is going to stand me up would you like to meet early?
<wallyworld> alexisb_: i'll be done in 3 minutes if horatio is not available
<wallyworld> alexisb_: ready now
<mup> Bug #1580501 opened: cloudimg-base-url parameters not in Juju2 anymore <4010> <cpe-sa> <orangebox> <sts> <juju-core:Triaged> <https://launchpad.net/bugs/1580501>
<mup> Bug #1580501 changed: cloudimg-base-url parameters not in Juju2 anymore <4010> <cpe-sa> <orangebox> <sts> <juju-core:Triaged> <https://launchpad.net/bugs/1580501>
<perrito666> alexisb_: sorry was away, am there now
<perrito666> but now you change me for wallyworld
<perrito666> veebers: its because the outstanding pr hasnt landed
<mup> Bug #1580501 opened: cloudimg-base-url parameters not in Juju2 anymore <4010> <cpe-sa> <orangebox> <sts> <juju-core:Triaged> <https://launchpad.net/bugs/1580501>
<veebers> perrito666: oh I could have sworn I saw that it had :-\
<veebers> perrito666: sorry false alarm though, I screwed up in expecting the fix to be in
<perrito666> veebers: this is the GH PR https://github.com/juju/juju/pull/6074
<perrito666> should merge RSN unless something blocks it
<veebers> perrito666: sweet, I see it's queued
<perrito666> hey, I wont make it to the standup I have been fighting with 1616167 the whole day and am just now fixing the tests to PR a fix.
<perrito666> alexisb_: wallyworld ^^
<alexisb_> perrito666, that is fine
<menn0> thumper: RB seems to have missed this one. https://github.com/juju/juju/pull/6088
 * thumper looks
<menn0> thumper: standup?
<wallyworld> axw: tghere's also removing the restricted config stuff etc too right?
<axw> wallyworld: that's what I said I had a PR up for :)
<axw> wallyworld: https://github.com/juju/juju/pull/6083
<axw> wallyworld: RB didn't pick it up
<wallyworld> ah sorry, was dstracted :-)
<mup> Bug #1615552 changed: Juju 2: Cannot add new key to default model config. <juju-core:Fix Committed by wallyworld> <https://launchpad.net/bugs/1615552>
<mup> Bug #1615552 opened: Juju 2: Cannot add new key to default model config. <juju-core:Fix Committed by wallyworld> <https://launchpad.net/bugs/1615552>
<mup> Bug #1615552 changed: Juju 2: Cannot add new key to default model config. <juju-core:Fix Committed by wallyworld> <https://launchpad.net/bugs/1615552>
<anastasiamac> wallyworld: alexisb_: re: machine count, this is the bug I was referring to in standup https://bugs.launchpad.net/juju/+bug/1602032
<mup> Bug #1602032: Add machine and core count to 'models' output <2.0> <usability> <juju:Triaged> <https://launchpad.net/bugs/1602032>
#juju-dev 2016-08-25
<anastasiamac> axw: menn0:thumper:wallyworld: an awesome, small review plz :D http://reviews.vapour.ws/r/5523/
<thumper> alexisb_: ping
<wallyworld> otp with reed can look soon
<thumper> alexisb_: we should have a quick word
<menn0> anastasiamac: looking
<anastasiamac> menn0: \o/
<menn0> anastasiamac: well that was easy. ship it
<anastasiamac> menn0: cmars: tyvm :) m loving to see this beta's bug count going down or fixes count going up - glass half full :D
<thumper> dog walk time
<mup> Bug #1471770 changed: TestPrunes fails occasionally still <ci> <intermittent-failure> <test-failure> <juju:Triaged by menno.smits> <https://launchpad.net/bugs/1471770>
<mup> Bug #1580501 changed: cloudimg-base-url parameters not in Juju2 anymore <4010> <cpe-sa> <orangebox> <sts> <juju:Triaged> <https://launchpad.net/bugs/1580501>
<axw> anastasiamac: I think you might've broken something: ERROR failed to bootstrap model: model "controller" of type manual does not support instances running on "amd64"
<axw> (I'm looking into it)
<anastasiamac> axw: well, the only thing i can think of is that manual constraints validator does not have arches, and if there are no images for it, then nothing will be merged...
<axw> anastasiamac: yeah, same thought
<anastasiamac> axw: i think I'll need to put arches in in there
<axw> anastasiamac: yup
<anastasiamac> k. i'll do it now :)
<axw> anastasiamac: I can probably fix it in my branch
<anastasiamac> axw: ooh even better ;)
<axw> should be a pretty small change
<anastasiamac> axw: i wonder if other providers will need to have similar thing.. i think  manual slipped through finger because maybe it never had arches vocab defined..?
<axw> anastasiamac: yeah, it doesn't
<anastasiamac> axw: \o/ tyvm. let me know if there is something i can do to assist
<axw> wallyworld: have you noticed that "model-config" says FROM=model for logging-config and resource-tags in a fresh model?
<wallyworld> axw: loggin-config i'd expect because juju itself sets the value via the api. resource-tags i'd need to look at, but i suspect the issue is the schema coeecsion to a map from a string
<wallyworld> bah, string to map i mean
<axw> wallyworld: yeah, almost certainly. I was thinking we should just set the default logging-config though?
<axw> to the same as what the agent runs
<axw> and configure the client differently if needed
<wallyworld> axw: it starts out as one thing and juju sets to another (debug->info)
<wallyworld> or something like that
<wallyworld> but yeah, maybe we could do better, i recall at the time it made sense how it is
<axw> wallyworld: *shrug* it seems odd to me that OOTB the config says it's not default
<wallyworld> agreed. i can't recall the specifics off hand, but juju messes with it
<wallyworld> we can clean up next week
<axw> anastasiamac: would you please review https://github.com/juju/juju/pull/6083/commits/32fdee6ef69e1355480ed9dbd208ca69c97fdd0f
<anastasiamac> axw: looks awesome :) did u get a chance to test live too?
<axw> anastasiamac: yup, bootstraps fine after this change
<anastasiamac> axw: \o/ LGTM for this commit :D
<axw> anastasiamac: ta
<mup> Bug #1449210 changed: cloudsigma index file has no data for cloud <bootstrap> <cloudsigma-provider> <simplestreams> <tech-debt> <juju:Triaged> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1449210>
<mup> Bug #1616197 changed: juju restore-backup error <20160826> <backup-restore> <juju:Triaged> <https://launchpad.net/bugs/1616197>
<mup> Bug #1616298 changed: DebugMetricsCommandSuite.TearDownTest fails due to "no reachable servers." <intermittent-failure> <mongodb> <unit-tests> <juju:Triaged> <https://launchpad.net/bugs/1616298>
<natefinch> is canonical IRC down for anyone else?
<anastasiamac> natefinch: yes
<miken> Just back now
<natefinch> oh good, then I haven't been fired yet.
<anastasiamac> miken: not for me... still down \o/
<anastasiamac> natefinch: unless we've all been :)
<miken> Oh - I'm connecting from an internal IP address, and it just reconnected to irc.c.c 2mins ago.
<wallyworld> thumper: you'll need to resubmit your race fix pr i think, after adding "Build failed:" comment to trick the bot
<thumper> wallyworld: ack
<thumper> axw: got some time to talk manual providers?
<axw> thumper: sure
<thumper> axw: https://hangouts.google.com/hangouts/_/canonical.com/manual?authuser=0
 * anastasiamac about to lose electricity - going afk for lunch and fun
<natefinch> thumper: you guys talking about that bug I was looking at?
<anastasiamac> natefinch: this one? https://bugs.launchpad.net/juju-core/1.25/+bug/1610880
<mup> Bug #1610880: Downloading container templates fails in manual environment <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1610880>
<natefinch> anastasiamac: yeah
<natefinch> anastasiamac: was going to ask you if you had any thoughts about that one
<anastasiamac> natefinch: looks like the fix needs to go into 1.25 not master
<anastasiamac> natefinch: no thoughts whatsoever - hence, m happy to go with advice from wallyworld to mark it as Invalid. I do wish there was a bit more explanations as to why it is Invalid...
<natefinch> anastasiamac: well, the customer who is experiencing it is on 1.25, yeah. I don't know if it happens in 2.0, honestly
<wallyworld> it has to be invalid as it's lxc only
<anastasiamac> natefinch: k :) do u have enough context to fix?
<natefinch> oh yeah, dug
<wallyworld> 2.0 uses a totlly different mechanism
<natefinch> duh
<natefinch> it's not invalid for 1.25
<natefinch> but it is invalid for 2.0
<wallyworld> correct
<anastasiamac> yes.
<wallyworld> all it did was remove the targetted juju project
<wallyworld> left it targetted to juju-core
<mup> Bug #1610880 changed: Downloading container templates fails in manual environment <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1610880>
<wallyworld> you can tell it's only 1.25 from reading the logs
<natefinch> yes, of course.  I hadn't really thought that part through :)
<anastasiamac> i've removed juju-core and left for 1.25
<natefinch> I added a note as to why it's not applicable to 2.0 :)
<mwenning> Hi, is there a way to tear down a model from the gui?
<anastasiamac> generally if the bug is in 1.25, we also keep it to be fixed in 2.0
<redir> wallyworld: updates in http://reviews.vapour.ws/r/5510/
<anastasiamac> however, in this case, lxc is not in 2.0; so bug is invalid
<wallyworld> looking
<mwenning> from command line I can go 'juju destroy-model blah'
<anastasiamac> natefinch: thnx :D
<wallyworld> anastasiamac: correct
<anastasiamac> wallyworld: m glad that u r agreeing \o/
<wallyworld> that it is 1.25 only? yes :-)
<axw> thumper: it's possible that the "should be terminated" is coming from the pkill issued by DestroyController, and the SIGABRT stack trace is due to the killall run by the CI script
<anastasiamac> wallyworld: and to a new world order, no? :D
<thumper> axw: hmm... I'll poke around
<axw> thumper: so maybe it's just a case of the agent not shutting down fast enough
<wallyworld> that too, peace in the middle east and all that
<thumper> axw: actually... you may well be right
<thumper> perhaps we need to make the destroy controller method on the manual provider wait until the process has been removed
<thumper> axw: I think this is the most likely case, and what I'll try for first
<axw> thumper: maybe, could be that the agent is wedged though? we wouldn't normally care, because the controller machine gets destroyed in cloud environments
<thumper> the agent will hang around for a while to answer current api calls
<thumper> but no, don't think it is fully wedged.
<thumper> but I'll look too
<thumper> still think it is worthwhile waiting
<axw> thumper: hmmk. seems reasonable to wait, yeah
<wallyworld> redir: really close, got a fix it then ship it, let me know if anything is unclear
<redir> wallyworld: k. tc
<redir> tx even
<thumper> wallyworld: hmm...
<wallyworld> things that make you go
<thumper> wallyworld: trying to bootstrap a manual provider in lxd
<thumper> ERROR failed to bootstrap model: model "controller" of type manual does not support instances running on "amd64"
<thumper> wat?
<wallyworld> thumper: damn, that's fallut from anatasia's changes for fixing simplestreams issues, you running from tip?
<thumper> yep
<wallyworld> ok, will need to be fixed
<thumper> trying to look at the manual provider leaving stuff behind
<thumper> but can't bootstrap
<thumper> I don't mind fixing if I can be told what needs fixing
<wallyworld> you can comment out the error
<thumper> huh?
<wallyworld> it's a bit complicated, image metadata been reworked, so different rules to figure out what can be bootstrapped
<wallyworld> comment out the error return
<wallyworld> ie don't do the check - that will get you going so you can do your fix
<thumper> where is that?
<wallyworld> don't commit that change of course
<wallyworld> validateUploadAllowed
<wallyworld> environs/bootstrap/tools.go
<thumper> ack
<thumper> wallyworld: that'll be the s390x manual bootstrap bug too
<wallyworld> ah, yes, could be
<wallyworld> i'll talk to her when she's back online
<redir> So Posgres 9.6 tl;dr notes: parallel query scans, joins, and aggregates, inceremental vacuum freeze, sycronous replication with  multiple standbys, 10 will start a new version scheme (thing firefox/chrome)
<redir> other than that mostly lots of talk about the uber paper.
<thumper> redir: oh?
<redir> uber: PostgreSQL -> Some schemaless object store thing on MySQL
<redir> Appearntly theis has caused some hubbbub https://eng.uber.com/mysql-migration/
<redir> thumper: live from the East Bay Postgres meetup at Pandora...
<redir> Time to go be social at the pub, getting kicked soon.
<redir> wallyworld: http://reviews.vapour.ws/r/5530/ when you have a chance later. I haven't yet updated what we talked about earlier.
<redir> wallyworld: also ignore the bits from the other PR,this is stacked on that so it has those issues too.
<redir> later juju-dev
<wallyworld> ty
<thumper> axw: in my local lxd testing, it took two to three seconds from the time kill-controller had exited to the time the jujud agent stopped running
<axw> thumper: sounds about right. in the CI failure it's still running 10 minutes later...
<thumper> 10 minutes?
<thumper> I thought it was much sooner than tha
 * thumper double checks
<axw> thumper: terminationworker says to terminate at 3:41, then the SIGABRT stack trace comes at about 3:50
<axw> 3:51 actually, I guess the CI script is waiting 10 minutes
<wallyworld> anastasiamac: did you see there's fallout with the arch / image stuff and manual provider with lxd?
<anastasiamac> wallyworld: the one that axw and i discussed and he has fixed (and i lgtm-ed) on his branch?
<axw> wallyworld: I've got a fix for manual in my branch, can't land because master is blocked
<wallyworld> axw: i'd say make it a blocker and use $$fixes$$
<anastasiamac> axw: i think u can land ur branch
<axw> wallyworld: is there a bug #?
<anastasiamac> axw: if u do not have a bug, jfdi
<wallyworld> anastasiamac: thanks, didn't see there's a fix, thumper ran into it before
<thumper> axw: no... wasn't 10 minutes
<axw> ok
<thumper> was almost immediate
<thumper> AFAICT
<axw> thumper: http://paste.ubuntu.com/23087267/
<axw> from http://reports.vapour.ws/releases/4301/job/manual-deploy-precise-amd64/attempt/4018
<thumper> on attempt 4017 it was more immediate
<thumper> axw: that timing doesn't match the log outputs at all from 4018
<thumper> kill was here: 02:34:05
<axw> thumper: I think the logs are appended to
<axw> thumper: I'm probably looking at something old
<thumper> that test log output is now in local time
<thumper> but still
<axw> thumper: I was looking from the top of the log, it looks like there's multiple test runs in the same log file
<axw> searching from the back, I concur that it's immediate
<thumper> ok... good :)
<axw> thumper: though there *is* a very slow one at the top of the log, so it's not consistent
<thumper> one problem at a time :)
<axw> wallyworld: in your lxd PR, there's another target var lower down. I didn't realise that it exited early if the alias exists - maybe the check is still needed? does CopyImage return immediately if the image is already there?
<wallyworld> axw: in my testing, i deleted all lxd images. bootstrap the first time downloaded the image (slowly, with progress shown). then another bootstrap did not
<wallyworld> and lxc image list shows the one image
<axw> wallyworld: but that might be because there's still another call to GetAlias
<wallyworld> the instance started immediately though
<wallyworld> so it's using whatever it cached the first time
<wallyworld> i can't see any obvious difference in behaviour
<axw> wallyworld: no, I'm just saying there's still another call to GetAlias that looks like it should be removed. but I'm not sure of the impact.
<wallyworld> oh, i miss understood you. i saw that call too but didn't follow what it did so left it
<anastasiamac> i've seen a few cpu/mem spike related bugs... if i were a memory leak in juju, where would i be? :D
<thumper> axw, menn0: http://reviews.vapour.ws/r/5532/
<axw> thumper: LGTM, thanks
<menn0> thumper: double ship it :)
<wallyworld> axw: i've got a few very small reviews up if you get a chance later. one is the lxd one which seems ok to me given it behaves as expected when testing
<axw> wallyworld: sure, just finishing up QA for my add-model changes
<wallyworld> no worries
<axw> wallyworld: add-model changes: http://reviews.vapour.ws/r/5534/
<wallyworld> looking
<axw> wallyworld: I'm QAing your lxd branch, and bootstrap is fetching images that I have again. possibly due to that code removal
<wallyworld> hmmm, it didn't fetch mine again
<wallyworld> but i started from a clean slate
<wallyworld> what are your aliases?
<wallyworld> ubuntu-xenial etc?
<axw> wallyworld: yep
<axw> I have ubuntu-xenial
<wallyworld> hmmm, ok, i'll bootstrap again and see what it does
<wallyworld> nfi why it doesn't work for you
<axw> wallyworld: yep, I put the code back in and it doesn't do it now
<axw> wallyworld: possibly once it has the image again, it wouldn't copy again
<wallyworld> yeah, that's what i was thinking
<wallyworld> there might me some implicit alias or something
<wallyworld> which i think is ok behaviour - so long as it only fetches once. stephane was adament we should be doing it this way or else auto update would not work
<axw> wallyworld: it's probably fine, just doing one last test to satisfy myself
<wallyworld> sounds good, best to be sure
<wallyworld> i'm testing again too
<wallyworld> but download is sloooooooooow
<axw> wallyworld: so, I think the issue is that the local alias I had did not match the image that was in the source
<axw> wallyworld: so it replaced it
<wallyworld> yeah, whereas before maybe we were setting the alias name
<axw> wallyworld: if you were to put that GetAlias code back in, people could continue using their existing images... but I guess they wouldn't auto-update
<wallyworld> that's my understanding
<wallyworld> and we want auto update
<axw> wallyworld: ok. seems fine, maybe just add a release note that it will force an image refresh on everyone?
<wallyworld> sure
<wallyworld> axw: in the add-model / cloud branch - i've just srted looking - do we reject add-model cloud where the controller doesn't support the cloud asked for?
<axw> wallyworld: it will complain that "foo" is not a cloud or a region
<axw> wallyworld: because you can't add clouds to a controller, the only cloud it'll find is the one that was bootstrapped
<axw> wallyworld: I did test that actually, just didn't add in the QA steps
<axw> will do that now
<wallyworld> ta, that would be good as i was wondering
<axw> wallyworld: updated steps under LXD
<wallyworld> great ty
<wallyworld> axw: and you can +1 the lxd pr?
<axw> wallyworld: sorry yes
<axw> done
<wallyworld> not sure if i should land before beta
<wallyworld> might be good to get auto update fixed
<wallyworld> axw: "is neither a cloud nor a region". i don't like that message because aws is a cloud. it's just not supported by the current controller. so people will get confused by the message i think?
<axw> wallyworld: well, it's not a cloud so far as the controller is concerned
<axw> wallyworld: I agree it's a sucky message
<axw> wallyworld: I guess we could look in the client's list of clouds first?
<wallyworld> could we rephrase to say that this controller doesn't support models on cloud "aws", only clouds "lxd" are supported
<wallyworld> yeah, look at client clouds, and if it is a valid cloud name, be smart about the message
<wallyworld> "... are supported by this controller"
<wallyworld> or something
<axw> wallyworld: we don't have an API to list clouds yet. I suppose I could add it
<wallyworld> axw: that's one of the things martin asked for
<wallyworld> so it won't go to waste
<wallyworld> and we have the cloud facade
<axw> wallyworld: yeah, was trying to keep this minimal. shouldn't take too long tho
<wallyworld> understood
<wallyworld> but the message sucks :-)
<wallyworld> axw: gotta duck out to do school pickup, but one last question - on the apiserver side where an unsupported cloud is passed in - it returns an annotated not found error but i think we can d better with the error message there also
<wallyworld> "such and such cloud is not supported, try one of these instead" type thing
<axw> wallyworld: where's that?
<axw> wallyworld: "getting cloud definition" ?
<wallyworld> yeah
<wallyworld> didn;t matter before but now that we are allowing people to specify the cloud themselves
<wallyworld> need to tighten it up IMO
<axw> wallyworld: isn't that redundant if we have the client query the supported clouds?
<wallyworld> that's in add-model, what abot via the api
<wallyworld> python juju client, controller proxy etc
<axw> hmm I guess so
<wallyworld> auto pilot, conjure up etc - they all use the api
<wallyworld> and in conjure up, someone could easily specify an unsupported cloud
<wallyworld> gotta run, bbiab, got to update release notes at some point
<frobware> axw: ping - larry shared his vsphere setup which i think you've used recently. Did you have any problems bootstrapping? For me it doesn't complete using beta15.
<axw> frobware: hey. I didn't get past authentication. I think the issue I was seeing was that the client downloads the cloud image and then uploads it to vsphere. I'm quite far away, so that was so slow it timed out
<axw> frobware: sorry I mean, it authenticated but didn't get any further (functionally) than that
<frobware> axw: I get as far as... https://pastebin.canonical.com/163942/
<frobware> axw: lines 75 & 76 repeat until timeout
<frobware> axw: I have never bootstrapped on vsphere before so could be operator error too
<axw> frobware: ah, well you got further than me :p   sorry, I don't know what's up with it. I've never used vsphere before that one time
<axw> and I was just verifying that my auth changes were good
<frobware> axw: the only addition I made to the cloud definition was adding to clouds.yaml:  vsphere: regions: dc0 {}
<frobware> axw: which was largely done based on a bug comment I think you made... somewhere... :)
<axw> frobware: if the issue was with clouds.yaml, it would have failed much earlier
<axw> I don't think it's user error
<axw> more likely the provider or vsphere is broken
<frobware> axw: which it did. could not bootstrap because 'datacenter' was undef
<frobware> axw: I'll try going back to beta8 as that's where the bug was reported, but largely to see if bootstrap has regressed since.
<axw> frobware: oh I see what you mean. yeah, larry's original clouds.yaml was broken
<frobware> oh
<axw> frobware: this is what I've got: https://pastebin.canonical.com/163945/
<frobware> axw: you mean it was broken and needed the regions bit?
<axw> yep
<axw> frobware: well, and he was trying to use non-standard keys. that one I linked is in the valid format
<frobware> axw: this is what I'm currently using: https://pastebin.canonical.com/163946/
<axw> frobware: yep that's fine
<axw> auth-types is unnecessary but won't cause a problem
<axw> wallyworld: how's this? https://pastebin.canonical.com/163947/
<wallyworld> looking
<wallyworld> axw: yay, much nicer, thank you
<axw> wallyworld: cool. just gotta write some tests, and improve error messages on the server side now
<wallyworld> sgtm
<wallyworld> axw: you could potentially make the add-model cmd dumb and not do any checks and allow them all to be done on the server side
<wallyworld> since you need to make an api call to list clouds anyway
<wallyworld> you could avoid that call
<wallyworld> and just make the create model call
<axw> wallyworld: thought about it, but that makes the cloud/region unstructured which I'm not too keen on
<wallyworld> you could still split on /
<axw> wallyworld: this way we may also support auto-upload of cloud definition, if we want to do that
<axw> wallyworld: sure, but you still don't know if it's cloud or region if there is no /
<wallyworld> true
<wallyworld> ok, ignore me, just thinking out loud
<thumper> wallyworld: hey
<thumper> wallyworld: still hanging around?
<wallyworld> maybe
<wallyworld> thumper: what's up?
<thumper> pretty sure bug 1615839 is that bit you got me to comment out
<mup> Bug #1615839: Manual-provider claims s390x is not supported <ci> <manual-provider> <regression> <s390x> <juju:In Progress by thumper> <https://launchpad.net/bugs/1615839>
<thumper> is anastasiamac on that?
<thumper> or shall I take a look?
<thumper> might take me longer
<wallyworld> thumper: i think axw landed a driveby
<wallyworld> ot has one in train
<wallyworld> or
<thumper> but I could muddle through it
<wallyworld> all good, we broke it, we fix it
<wallyworld> thanks for offering
<thumper> who shall I assign the card and bug to?
<wallyworld> check that axw is/has done it, otherwise to anastasia
<thumper> axw: have you fixed it?
<redir> is hudson back?
<wallyworld> redir: late for you, go to bed :-)
<redir> yeah just got home and eating something, then bed
<redir> who knew postgres folks were such talkers:)
<axw> thumper: sorry was on school run, it should be fixed by my latest merge, have marked Fixed Committed
<thumper> axw: ok, cool
<thumper> what was the fix by the way?
<axw> wallyworld: updated my PR, PTAL
<wallyworld> looking
<wallyworld> axw: looks great, ty
<wallyworld> axw: when it lands, let urulama and mhilton know as they've started to need the Clouds() API and are assuming a return of []string whereas we are offering a map of cloud details
<axw> wallyworld: sure
<axw> wallyworld: gonna have to get a second review, this is >500
<axw> I'll point martin at it, maybe he'll be willing :)
<wallyworld> hmmm
<wallyworld> stupid rule
<anastasiamac> wallyworld: well, the rule would not bite if the PR are manageable
<anastasiamac> >500 is not manageable for any reviewer
<wallyworld> disagree
<wallyworld> depends on the type of change
<anastasiamac> of course u do
<wallyworld> and who's reviewing
<wallyworld> we had a much larger limit in launchpad
<wallyworld> 800
<wallyworld> 500 is too small
<anastasiamac> no, usually only dev knows what they wrote for PR >500
<wallyworld> not just dev
<wallyworld> i know what's in that pr and i didn;t write it
<anastasiamac> u r very specail
<anastasiamac> special*
<wallyworld> axw: off to make dinner, updated the pr, thanks for reviewing
<axw> wallyworld: will look in a sec. I'm reviewing your show-user one now
 * frobware is back in ~1 hour
<mhilton> wallyworld, axw: what have you done to my API deisgn!
<wallyworld> mhilton: we needed more than just the cloud names :-) you get the names as the map keys
<wallyworld> and also you have allowable regions etc which are really useful for the gui
<mhilton> wallyworld: It's fine, I'm curous. We were getting the regions from the Cloud() endpoint. but doing it all in one go is probably better.
<wallyworld> mhilton: yeah, we think so, one call to get all the info you need
<wallyworld> rogpeppe1: thanks for review, did you just want to check my answer to you question in the review http://reviews.vapour.ws/r/5533/
<rogpeppe1> wallyworld: i've just published a review of http://reviews.vapour.ws/r/5533/
<wallyworld> ta
<rogpeppe1> wallyworld: weird, i didn't think i'd published anything until now...
<rogpeppe1> wallyworld: i think you were maybe talking about axw's question
<wallyworld> oh dear, i was
<rogpeppe1> wallyworld: i was wondering about external user access too, although i forgot to mention it in my review
<wallyworld> this review isn't about any of that
<rogpeppe1> wallyworld: i think if we left the access field out entirely, things would become more obvious
<wallyworld> it's all already been done
<wallyworld> the access field is also pre existing so i don't realy want to move it
<wallyworld> this review is just about proerly looking up external user access
<rogpeppe1> wallyworld: the problem is that it *looks* as if the access field tells you what access rights a given user has
<wallyworld> it does
<rogpeppe1> wallyworld: but actually it doesn't tell you that
<wallyworld> why?
<wallyworld> it tell you what access that user has to the controller
<rogpeppe1> wallyworld: because if access has been granted to everyone, everyone will have at least those rights
<rogpeppe1> wallyworld: and when we implement general group checking, that issue will become still worse
<wallyworld> oh, i see what you're saying. yes right now it just tells you what that user has specifically been granted
<wallyworld> groups will require a whole lot of change
<rogpeppe1> wallyworld: yup. i think that's misleading, and we'd be better off fixing things now.
<rogpeppe1> wallyworld: i.e. remove the Access field
<rogpeppe1> wallyworld: because that's the only problematic part
<rogpeppe1> wallyworld: otherwise it's just about looking up information about a specific user
<wallyworld> why can;t we look up the access transatively and just fill in the access bit
<wallyworld> we need to give the access value back to the caller
<rogpeppe1> wallyworld: why does it need to be in the same API call?
<wallyworld> why not? for distributed systems you aim to minimse the api calls
<wallyworld> fewer bulk calls is the design goal
<rogpeppe1> wallyworld: that's an optimisation - i generally prefer to start by being as clear as possible and optimise later
<wallyworld> we disgaree there, i remember this discussion when juju's api was first being designed
<rogpeppe1> wallyworld: so how many bulk calls are actually being used as bulk calls now? :)
<axw> mhilton: heh sorry :p  but yeah, one query lets us get all the stuff we want in one go, rather than getting names and then calling Cloud a bunch of times
<rogpeppe1> wallyworld: POitRoAE... still!
<wallyworld> i can't answer that - i don't know what api clients people have written
<rogpeppe1> wallyworld: juju is the only api client for all the agent stuff
<wallyworld> i can imagine landcape etc would use bulk calls
<wallyworld> and the gui certainly *should*
<rogpeppe1> wallyworld: and bulk calls are used approximately zero times
<wallyworld> then they're doing it wrong if that's the case
<rogpeppe1> wallyworld: no. mostly you do only have one thing to do at a time
<rogpeppe1> wallyworld: and this isn't HTTP
<wallyworld> more's the pity
<wallyworld> it should be restful
<wallyworld> but that's another discussion
<rogpeppe1> wallyworld: HTTP1 is bad because the calls are expensive and cannot return replies out of order
<rogpeppe1> wallyworld: the RPC API doesn't have that limitation
<rogpeppe1> wallyworld: the overhead of a call is small
<wallyworld> sure, but HTTP1 is so last century
<rogpeppe1> wallyworld: we still only use HTTP1
<wallyworld> why is that out of curiouslty?
<wallyworld> besides that Go is stuck in the 70s :-)
<rogpeppe1> wallyworld: for all the bulk calls we have, making several calls concurrently is faster than actually using the bulk call as a bulk call
<rogpeppe1> wallyworld: because we make our own http transport
<rogpeppe1> wallyworld: otherwise we'd get HTTP2 out of the box
<wallyworld> hmmm, i'd like to see the numbers. may be fast for some things, but chatty api calls are evil for distributed system
<wallyworld> that's why we should just stick to standards instead of rolling out own
<rogpeppe1> wallyworld: tell that to google, amazon, heroku, etc etc
<rogpeppe1> wallyworld: they all use RPC systems
<wallyworld> i've seen way too many distributed system fall over due to inefficient apis
<rogpeppe1> wallyworld: and none of them have a "bulk calls only" policy
<rogpeppe1> wallyworld: HTTP APIs, right?
<wallyworld> rpc
<rogpeppe1> wallyworld: if a system is inefficient, optimise it
<rogpeppe1> wallyworld: it's not a hard thing to do
<wallyworld> easier said than done once the apis are set
<rogpeppe1> wallyworld: that's why we have versioning
<wallyworld> you can use a bulk call singularly but not the other way around
<wallyworld> and versioning is horrible for us to try and us
<wallyworld> each time we rev a facade verison it introuces a world of hurt
<wallyworld> anyway, i need to change the access look up to take account of the everyone group
<wallyworld> regardless of the api design
<rogpeppe1> wallyworld: you can make many singular calls concurrently
<rogpeppe1> wallyworld: there really is very little overhead in doing so
<wallyworld> at the cost of many network reosurces
<rogpeppe1> wallyworld: no
<rogpeppe1> wallyworld: at the cost of *some* extra bandwidth
<rogpeppe1> wallyworld: but much less than you'd think
<frobware> do we really have bandwidth issues?
<rogpeppe1> frobware: not AFAIK
<wallyworld> bandwidth is a finite resource
<rogpeppe1> wallyworld: so are developer resources
<wallyworld> try living in australia
<wallyworld> the point being?
<wallyworld> where there's latency, bulk calls are much better
<rogpeppe1> wallyworld: the point being that we've expended 1000s of hours of extra effort making every call "bulk" and we never use that capability
<rogpeppe1> wallyworld: actually no
<wallyworld> what extra effort?
<wallyworld> we designed the api once
<wallyworld> and we do use it? how do you know we dont't?
<wallyworld> have you audited every external juju api client?
<rogpeppe1> wallyworld: because most of the entry points in the api package don't even expose the bulk functionality
<rogpeppe1> wallyworld: i'm talking about the agent API here
<rogpeppe1> wallyworld: because that's easily checked
<wallyworld> the juju api layer only exposes singular calls, but pythin juju client, conjure up, etc etc don't use that
<rogpeppe1> wallyworld: testing and implementing a bulk API call is probably 5 times more effort than a single one
<wallyworld> wot?
<wallyworld> i don't agree with that
<wallyworld> i don't find it any different
<rogpeppe1> wallyworld: there are lots more edge cases to be tested
<rogpeppe1> wallyworld: zero, one, many, all error, some errors, etc
<wallyworld> that is true, but once you get standard patterns in place, it falls out pretty easily
<rogpeppe1> wallyworld: BTW when there's latency, concurrent calls are better because you can get replies out of order and start dealing with them sooner, rather than waiting for all replies at once
<wallyworld> i wonder why there's such stark difference of opinion here
<rogpeppe1> wallyworld: you mean you copy and paste
<wallyworld> no, we have helper structs also
<wallyworld> like our errors
<rogpeppe1> % find api apiserver -name '*.go' | xargs cat | wc
<rogpeppe1>  125883  386115 3838814
<rogpeppe1> wallyworld: in a very few cases, you can have helper structs. but if you're returning actual results, you can't use 'em
<rogpeppe1> wallyworld: and you still need to have all those test cases
<wallyworld> sure
<wallyworld> not hard though
<rogpeppe1> wallyworld: it all adds up
<rogpeppe1> wallyworld: our api code is *huge*
<rogpeppe1> wallyworld: and it's mostly noise
<wallyworld> lol
<wallyworld> Go code is mostly noise :-P
<wallyworld> so much boilerplate and copy and paste
<rogpeppe1> wallyworld: you're writing it wrong then
<wallyworld> due to not having generics etc etc
<wallyworld> so all those sort functions are wrong
<wallyworld> different ones for int vs string etc
<rogpeppe1> wallyworld: very little of what we're doing in the API could be made better with generics
<wallyworld> no, i was making apoint about the fact that you critised our aoi for cut and paste when the language itself is just as bad :-)
<rogpeppe1> wallyworld: i honestly don't see that much copy and paste in decent Go code
<wallyworld> we have so many cut and paste functions for "is this string in this slice"
<wallyworld> etc
<rogpeppe1> wallyworld: even implementing sort only involves copying and pasting two lines
<wallyworld> and each time i have to do it i die a little inside
<wallyworld> no other language ive used makes you do that
<rogpeppe1> wallyworld: i guess you've never used C then
<wallyworld> not for years
<wallyworld> luckily
<wallyworld> bbiab, SIGWIFE
<mup> Bug #1616832 opened: manual environment juju-db timeout <juju-core:New> <https://launchpad.net/bugs/1616832>
 * fwereade bbl
<frobware> dimitern: ping - I tried your patch but it didn't work for me.
<dimitern> frobware: oh, what was wrong?
<frobware> dimitern: just trying to repro again to ensure it's all true...
<dimitern> frobware: ok
<frobware> dimitern: but it was eseentially the same problem that ivoks ran into initially
<dimitern> frobware: DNS hostname (resolved) != PUBLIC ADDRESS in status?
<frobware> dimitern: double-checking. have too many pots on the go.
<mup> Bug #1616832 changed: manual environment juju-db timeout <juju:New> <juju-core 1.25:New> <https://launchpad.net/bugs/1616832>
<rogpeppe1> juju server certs need unique serial numbers, it seems. this PR adds them. small PR, review appreciated :) https://github.com/juju/juju/pull/6100
<rogpeppe1> please could someone give this small PR a review? (i need review from someone in -core), as it's blocking us right now: http://reviews.vapour.ws/r/5538/
<rogpeppe1> dimitern, frobware: ^
<frobware> rogpeppe1: looking
<rogpeppe1> frobware: thanks
<frobware> rogpeppe1: can you please list some QA steps
<rogpeppe1> frobware: ah, ok, sure
<rogpeppe1> frobware: done
<frobware> rogpeppe1: without your change you cannot connect?
<rogpeppe1> frobware: no, there's no externally visible behaviour change
<rogpeppe1> frobware: except, i guess, that if you use a browser to connect, it can do so
<rogpeppe1> frobware: hmm, maybe the QA steps could specify that i guess
<frobware> rogpeppe1: right - I wanted to look at before and after
<rogpeppe1> frobware: let me check how they've been doing it - it involves creating a new CA key, adding its cert to your browser, and making a websocket connection to the API
<rogpeppe1> frobware: actually, even that won't quite check it. i think you need to do that with two controllers.
<rogpeppe1> frobware: oh yes, the controllers need to be bootstrapped using the new CA key
<fwereade> frobware, http://reviews.vapour.ws/r/5539/ is up if you have a moment :)
<fwereade> and now I'm going to have some lunch, ping me if you need me and I'll catch up when I'm back
<frobware> fwereade: ack
<sinzui> anastasiamac: when you deleted the juju-core task, but left a 1.25 task on bug 1616832, you removed the bug from search. All bugs that affed to a project series must also affect the project.
<mup> Bug #1616832: manual environment juju-db timeout <juju:New> <juju-core:New> <juju-core 1.25:New> <https://launchpad.net/bugs/1616832>
<mup> Bug #1616832 opened: manual environment juju-db timeout <juju:New> <juju-core:New> <juju-core 1.25:New> <https://launchpad.net/bugs/1616832>
<rogpeppe1> frobware: i replied to your question
<frobware> rogpeppe1: ok, will take a look. looking at the other review atm
<rogpeppe1> frobware: as we're the ones affected, perhaps we should do the QA
<frobware> rogpeppe1: I think so.
<rogpeppe1> frobware: ok, cool. in that case your LGTM would be much appreciated.
<frobware> rogpeppe1: I dropped another question on the review
<rogpeppe1> frobware: i don't get which line 170 you're talking about
<rogpeppe1> the only new(big.Int) we're now using is on cert.go:133
<rogpeppe1> frobware: is that the one you're referring to?
<rogpeppe1> frobware: otherwise i'm seeing cert_test.go:170 is 	expiry, err := time.Parse("2006-01-02 15:04:05.999999999 -0700 MST", "2012-11-28 15:53:57 +0100 CET")
<rogpeppe1> frobware: and cert.go:170 is 	if !ok {
<frobware> rogpeppe1: ah, sorry. line 170 was in the old file.
<rogpeppe1> frobware: so we're not using new(big.Int) there any more
<frobware> rogpeppe1: dropped
<rogpeppe1> frobware: ok, ta
<babbageclunk> dimitern, fwereade: Could you look at this? http://reviews.vapour.ws/r/5540/
<dimitern> babbageclunk: looking
<frobware> babbageclunk: looking (I'm OCR)
<babbageclunk> It's the machine undertaker worker with tests. (Also I worked out how to get it onto RB, since the bot wasn't helping!)
<babbageclunk> dimitern, frobware: Thanks!
<frobware> babbageclunk: one quick question, why did the pattern of s.waitRemoved() & s.waitForRemovalMark() calls appear to now be the other way around?
<babbageclunk> frobware: I've changed the provisioner not to remove machines anymore, so the tests can't wait for the machine to be removed.
<babbageclunk> frobware: Instead they wait for it to be marked for removal.
<babbageclunk> frobware: I'm not sure if that was quite what you were asking.
<frobware> babbageclunk: yes
<frobware> babbageclunk: thx
<babbageclunk> frobware: cool cool. I saw something weird while testing manually, just seeing if I can reproduce it.
<babbageclunk> frobware, dimitern: MAAS was giving the error "node with this hostname already exists" if I tried to create containers on two hosts at the same time.
<frobware> babbageclunk: but goes away if done in series?
<babbageclunk> frobware: That was what it seemed like - trying to reproduce it now.
<babbageclunk> frobware: (Most importantly, trying to reproduce it on upstream/master)
<katco> dimitern: frobware: standup time
<frobware> katco: omw
<babbageclunk> frobware: Hmm, can't reproduce it on master or my branch now.
<dimitern> omw
<ram____> Hi. I have a question. While selecting components from landscape UI for autopilot openstack deployment , can we give set external configuration parameters for a particular component?
<natefinch> ram____: you'd do better to ask in #juju
<katco> frobware: sorry, didn't mean to overlap
<frobware> katco: not a problem
<ram____> natefinch: Ok. tahnk you.
<babbageclunk> mgz: ping?
<mgz> heya
<babbageclunk> I'm trying to investigate this: https://bugs.launchpad.net/juju/+bug/1606308
<mup> Bug #1606308: Restore cannot initiate replica set <backup-restore> <ci> <mongodb> <regression> <juju:Triaged by alexis-bruemmer> <https://launchpad.net/bugs/1606308>
<babbageclunk> I can't really remember how to go about running the CI tests.
<mgz> babbageclunk: so following the links through to a recent failure gives you the rough outline
<mgz> this is a test we run on aws, so it's pretty easy
<mgz> want to do a ho or something quickly to go over?
<babbageclunk> yeah, that would be brilliant
<mgz> babbageclunk: okay, I am in the meeting named core
<natefinch> fwereade: I'm looking at this: https://bugs.launchpad.net/juju-core/+bug/1485784
<mup> Bug #1485784: Error creating container juju-trusty-lxc-template; Failed to parse config <lxc> <oil> <juju-core:Fix Released by cherylj> <juju-core 1.25:Fix Released by cherylj> <https://launchpad.net/bugs/1485784>
<natefinch> fwereade: actually, srory, wrong link, this one: https://bugs.launchpad.net/juju-core/1.25/+bug/1610880
<mup> Bug #1610880: Downloading container templates fails in manual environment <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1610880>
<natefinch> though they're similar
<natefinch> fwereade: we're running lxc-create and trying to download the lxc image from the server - lxc-create [-n juju-trusty-lxc-template -t ubuntu-cloud -f /var/lib/juju/containers/juju-trusty-lxc-template/lxc.conf -- --debug --userdata /var/lib/juju/containers/juju-trusty-lxc-template/cloud-init --hostid juju-trusty-lxc-template -r trusty -T https://10.2.0.186:17070/environment/80234a11-2d53-436e-855c-da998c76d6ca/images/lxc/tru
<natefinch> sty/amd64/ubuntu-14.04-server-cloudimg-amd64-root.tar.gz]
<natefinch> That's getting a cert error
<natefinch> I notice that I get a similar error if I just try to curl that URL
<frobware> babbageclunk: sorry, been sidetracked. I wanted to try your changes
<fwereade> natefinch, I don't have any immediate insight I'm afraid :(
<natefinch> fwereade: np
<babbageclunk> frobware: no worries
<babbageclunk> mgz: oops, I guess firefox gave up? I think that's enough to go on with, thanks heaps!
<mgz> babbageclunk: yup, think it's pretty hung
<mgz> babbageclunk: no problems, yell if you need more
<babbageclunk> mgz: I almost certainly will!
<babbageclunk> frobware: If there are some issues that you want me to look at in the meantime you can put up a partial review while you're testing?
<frobware> babbageclunk: haven't got that far. :( still trying to get vsphere to boot whilst larry is about
<babbageclunk> frobware: ok, fair enough :)
<babbageclunk> frobware: I'm not blocked, so no stress
<ram____> Hi. For testing purpose I developed a simple charm using the shell script  to modify the cinder configuration file for the post-deployment of OpenStack. cinder configuration modified. But I am saw some error in charm log. pasted information of my issuehttp://paste.openstack.org/show/563408/
<ram____> Please anyone provide me some solution.
<babbageclunk> mgz: I'm getting a "no such file or directory" when it tries to call euca-terminate-instances - where should I get that from?
<perrito666> anyone that has knowledge of migrations on this time zone?
<babbageclunk> mgz, oh looks like euca2ools
<mgz> babbageclunk: hm, that really should be part of the deps
<mgz> but it should also be switched to boto....
<alexisb> perrito666, voidspace but I dont see him online
<katco> alexisb: he is travelling
<alexisb> katco, thanks
<alexisb> katco, there is a mail I need him to respond to as well
<alexisb> I will loop you in for tracking purposes
<katco> alexisb: he mentioned he might check-in. if so, i'll let him know
<rogpeppe1> natefinch: do you know how we've ended up with 5 packages under github.com/juju/juju/resource with no tests at all?
<rogpeppe1> natefinch: it makes life a bit awkward when refactoring in that area
<natefinch> rogpeppe1: yeah.
<natefinch> rogpeppe1: I think the only one with any substantial amount of code is the resourceadapters directory
<natefinch> rogpeppe1: I don't have a good answer for that.
<ram____> Hi. I tried to deploy "cinder-xtremio " charm in our local Juju openstack environment like $juju deploy cinder-xtremio. I was facing errors. pasted error log : http://paste.openstack.org/show/563432/. Please anyone provide me some solution for this.
<ram____> Hi. I want to develop a cinder-storagedriver  charm. And i want to integrate it with Ubuntu-autopilot . SO can I give input parameters like san IP, san user and san password from landscape autopilot UI. Otherwise everything we have to hardcode into the charm. And different users for the same storage array have different credentials.
<natefinch> ram____: you might get a better response if you send an email to the juju mailing list
<natefinch> ram____: most of us here work on the core code for juju itself, and we don't know much about the ubuntu autopilot code, or the openstack charms in general
<ram____> natefinch : Ok. thank you
<natefinch> tych0: you around?
<katco> tvansteenburgh: hey, have you made any progress on bug 1616574 ? still stuck?
<mup> Bug #1616574: Can't deploy local charm via API <juju:Triaged> <https://launchpad.net/bugs/1616574>
<tvansteenburgh> katco: i haven't made progress
<katco> tvansteenburgh: ok; did you have a look at the go code?
<cmars> if i'm bootstrapping and I see "2.0-beta16-xenial-amd64.tar.gz", does that mean i'm downloading the agent from simplestreams?
<cmars> sorry "Fetching agent 2.0-beta16-xenial-amd64.tar.gz" i mean
<natefinch> cmars: should be, yeah
<cmars> natefinch, is there a way to get juju to pick up a jujud binary i've already built?
<cmars> natefinch, i can do --build-agent, but if i've already built it..
<katco> cmars: it just has to be in your $PATH i believe. see wallyworld's email a week or so ago
<natefinch> cmars: if you're bootstrapping with a built juju, it's supposed to automagically figure it out and do upload--tools
<cmars> i've got jujud in my path, but it's not getting picked up
<cmars> so open a bug?
<wallyworld> it needs to be in the same dir as juju
<wallyworld> $GOPATH/bin usually
<cmars> ah
<cmars> yeah, i think it was
<cmars> but, i'll try again
<wallyworld> and it needs to match juju exactly in terms of version
<wallyworld> --show-log will have more info
<cmars> wallyworld, ok, thanks!
<wallyworld> it should all work, let me know if now and we can debug
<wallyworld> *not
<tvansteenburgh> katco: i did, yes
<katco> tvansteenburgh: i'm looking through and comparing the 2 now; did anything pop out?
<cmars> it's not critical, but it would shave a minute or two off our CI to not build twice
<katco> tvansteenburgh: the logic i'm using is: if the juju binary can do this, there's no reason python-jujuclient shouldn't be able to as well. i.e. i don't think it's a fix on our end?
<katco> tvansteenburgh: is there a flaw in that reasoning?
<tvansteenburgh> katco: yes :)
<katco> tvansteenburgh: haha
<katco> what am i missing?
<tvansteenburgh> katco: the logic i'm using is, this works with juju1 but not juju2
<tvansteenburgh> katco: maybe it works with juju2 but there's another step or something, i dunno
<katco> tvansteenburgh: there have been many many breaking api changes between the 2 versions
<tvansteenburgh> katco: no, that's not the problem
<katco> tvansteenburgh: but that's my point: i'm going to help you figure out what's wrong, but i don't know why this is targeted against the juju project and not python-jujuclient?
<tvansteenburgh> katco: here's the thing. customer comes and says "how do i deploy a local charm using the juju2 api". i can't fix python-jujuclient until i know the answer to that.
<tvansteenburgh> katco: so far no one has been able to tell me how to do it
<tvansteenburgh> that's what the bug is for
<katco> tvansteenburgh: ok, well let's get this figured out. the go code i pointed you at is how we do it, so we just have to figure out what the difference is
<tvansteenburgh> katco: is it possible that the local charm should be uploaded to the controller and not the model?
<tvansteenburgh> no, that didn't work either
<natefinch> tvansteenburgh: if a customer came and asked how to deploy a local charm using the juju2 api, I'd say "don't"
<natefinch> our API is not designed to be used directly by third parties.  It's too granular and requires too much knowledge of the internal workings of juju.
<katco> natefinch: well, that's why we supply libs to wrap that, which is what tvansteenburgh is trying to fix
<cmars> wallyworld, ah, i figured out how to force bootstrap to use jujud out of $PATH. i set the agent & image metadata url to localhost and streams to "nope"
<natefinch> katco: yes, I get that
<cmars> wallyworld, that fails over to the "Preparing local Juju agent binary" case
<cmars> wallyworld, do you think that's expected behavior?
<cmars> (actually, i'm not sure i should have messed with image... i have no idea what i'm doing!)
<katco> tvansteenburgh: i see your placement args are empty. placement.Scope must be the model UUID i think
<wallyworld> cmars: it will only use a local juju if it can't find any binaries in streams. setting the url like that will cause that search to fail
<wallyworld> cmars: we gave beta16 binaries now, i bet your master source code still says beta16
<katco> wallyworld: cmars: maybe the issue is that your client is reporting beta16
<tvansteenburgh> katco: thanks i'll try that
<cmars> wallyworld, ah! i bet that's it
<katco> cmars: it has to report higher than that
<cmars> wallyworld, i think i'll keep it like this.. i want to test exactly what i've built
<cmars> wallyworld, kind of a hacked-up --agent-binary feature
<wallyworld> that will happen 99.999% of the time
<wallyworld> it's just we have an hour windows just after a release
<wallyworld> where the source code is not yet updated to say beta+1
 * wallyworld needs coffee
<anastasiamac> sinzui: k. thnx
<perrito666> wallyworld: awake again?
<wallyworld> almost
<tvansteenburgh> katco: no luck with that http://pastebin.ubuntu.com/23090486/
<tvansteenburgh> katco: if i call the CharmInfo with the charm-url i also get a "charm not found" error back
<tvansteenburgh> CharmInfo api i mean
<katco> tvansteenburgh: here is the entire client-side call-chain serialized out, freshmen in cs101 failing miserably style: http://pastebin.ubuntu.com/23090503/
<katco> tvansteenburgh: let me ponder your CharmInfo comment a moment
<katco> tvansteenburgh: substitutions in that pastebin can be searched by triple "/" (e.g. ///)
<tvansteenburgh> katco: cool, looking
<alexisb> thumper, ping
<thumper> morning
<katco> heya thumper
<alexisb> morning thumper
<alexisb> ping
 * thumper is munching on breakfast, been a busy morning
<thumper> kids off to ski trip
<alexisb> nice
<alexisb> thumper, do you mind joining a HO?
<thumper> sure
<thumper> I'll just be mute
<alexisb> if you join now no need fo ryou to be on release call
<alexisb> https://hangouts.google.com/hangouts/_/canonical.com/bug-scrub
<alexisb> thumper, ^^^
 * redir lunches
<katco> tvansteenburgh: it looks like environment.py::add_local_charm is missing schema & revision queryargs
<katco> tvansteenburgh: L65-67 in pastebin
<tvansteenburgh> katco: i noticed that but figured i'd get an error back if they were needed. i'll try adding them though
 * katco doesn't know, is just pedantically going through the diff
<tvansteenburgh> katco: after uploading my charm, and getting a url back, a call to the Charms.List api does not list my charm
<tvansteenburgh> i'll try the extra args now
<katco> tvansteenburgh: ok, so we're narrowing it down here at least ^.^
<katco> tvansteenburgh: fyi i have a call in 7m which EOD me
<katco> which will EOD me
<tvansteenburgh> katco: ack
<tvansteenburgh> katco: extra args didn't seem to make any difference
<katco> tvansteenburgh: hm, can you verify that the lib's Charms.List call works at all? just so we know the scope of this problem?
<tvansteenburgh> katco: yeah i have output from it
<tvansteenburgh> it just doesn't include the charm i uploaded
<tvansteenburgh> it does include other charms in the model thought
<tvansteenburgh> though
<katco> tvansteenburgh: k... so it has to be something with add_local_charm_dir down right?
<tvansteenburgh> well i get a charm-url back from that, as if the upload was successful
<tvansteenburgh> but then when i list via the api, it's not there
<katco> can you manually verify it's in the environment? i.e. look in mongo?
<katco> or use juju bin?
<tvansteenburgh> katco: can the juju cli list apps that haven't been deployed?
<katco> tvansteenburgh: it depends on if they're just "pending" or uploaded by not placed... i don't know which adding a charm does =/
<katco> tvansteenburgh: i would just look in mongo to be sure
<tvansteenburgh> katco: never done that, are there instructions somewhere? :D
<katco> tvansteenburgh: yeah sec
<katco> tvansteenburgh: https://lists.ubuntu.com/archives/juju-dev/2016-July/005772.html
<tvansteenburgh> coool
 * tvansteenburgh tries
 * tvansteenburgh waits for mongo client to install
<tvansteenburgh> katco: http://pastebin.ubuntu.com/23090623/
<katco> tvansteenburgh: http://pastebin.ubuntu.com/23090638/
<katco> tvansteenburgh: try this version
<tvansteenburgh> katco: same :(
<tvansteenburgh> is my mongo shell version ok?
<katco> tvansteenburgh: oh, no... you need >= 3.2
<katco> tvansteenburgh: sorry, didn't catch that
<tvansteenburgh> ok
<tvansteenburgh> katco: ok i'm connected
<katco> tvansteenburgh: `use juju`
<menn0> thumper: target prechecks infrastructure: http://reviews.vapour.ws/r/5543/
<katco> tvansteenburgh: `db.charms.find()`
<tvansteenburgh> katco: i see my local ubuntu charms
<katco> tvansteenburgh: the ones you've been uploading?
<tvansteenburgh> yeah
<thumper> menn0: looking
<tvansteenburgh> i'll pastebin one
<thumper> veebers: can we hangout?
<tvansteenburgh> katco: http://pastebin.ubuntu.com/23090657/
<tvansteenburgh> katco: now i notice that model uuid is not the one i passed to Placement.Scope
<katco> tvansteenburgh: ah. wortha  try
<tvansteenburgh> katco: no luck
<veebers> thumper: sure thing, what about?
<thumper> veebers: running a ci test locally
<veebers> thumper: sure, where you want to meet?
<thumper> veebers: https://hangouts.google.com/hangouts/_/canonical.com/friday?v=1471633360&clid=9319256218B181C1&authuser=0
<katco> tvansteenburgh: ok, we'll have to pick this up tomorrow. at least we've narrowed the scope
<tvansteenburgh> katco: sounds good, thanks
<katco> tvansteenburgh: np
<cmars> is master blocked?
<alexisb> aaah, anastasiamac we should remove blocking tags
<anastasiamac> yes. i will now \o/
<anastasiamac> alexisb: there are no blocking bugs i see
<menn0> anastasiamac: https://bugs.launchpad.net/juju/+bugs?field.tag=blocker
<menn0> anastasiamac: I see 4 blockers
<mup> Bug #1615986 changed: Agents failing, blocked by upgrade but no upgrade performed <canonical-is> <juju:Invalid> <juju-core:Invalid> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1615986>
<anastasiamac> menn0: looking. i'll remove.. nothing is blocking according to juju.fail :D
<menn0> anyone know who runs juju.fail? I suspect it needs to be updated to look at "juju" instead of "juju-core" on launchpad
<katco> menn0: i think it's marcoceppi
<anastasiamac> ah... it's becasue juju fail may looks at launchpad juju-core not juju project
<anastasiamac> sideeffect of the move :)
<anastasiamac> \o/
<menn0> that's what I said :)
<menn0> haha
<anastasiamac> :D
 * menn0 emails marcoceppi 
<cmars> thanks all
<tvansteenburgh> katco: i see what's happening. the uploads are being tagged with the controller uuid instead of the default model uuid. i'm not sure how to fix it though
<marcoceppi> menn0 katco I'll update it, and add a link to bugs
<menn0> marcoceppi: thank you
<marcoceppi> actually, there is a link, at the bottom of the page
<marcoceppi> that says I made it
<marcoceppi> ;)
<menn0> marcoceppi: the awesomeness of the rest of the page must have blinded us to that part ;-)
<marcoceppi> the citools from QA has changed, I have to go patch the scripts
<marcoceppi> menn0: this may seem odd, but apparently there are no blockers?
<alexisb> anastasiamac, standup
<menn0> marcoceppi: that's correct... there were an hour or so ago, but not now
<alexisb> perrito666, standup
<marcoceppi> menn0: ah, I see, we'll it's switched over now
<marcoceppi> menn0: the next time you ahve a blocking bug double check to make sure it works
<menn0> marcoceppi: will do!
<tvansteenburgh> katco: i figured it out. bug updated with the details. TL;DR "I'm sorry"
#juju-dev 2016-08-26
<menn0> thumper: poke regarding http://reviews.vapour.ws/r/5543/ :)
<thumper> menn0: sorry, got distracted
<menn0> thumper: no worries
<thumper> menn0: do you recall how to get the engine as a dependency?
<menn0> thumper: only vaguely
 * menn0 checks
<menn0> thumper: is worker/dependency.SelfManifold what you want?
<thumper> probably
<thumper> I'm halfway through your review
<thumper> wallyworld: with you shortly, just want to finish this review
<menn0> thumper: looks like it's currently only used in some tests
<wallyworld> sure
 * wallyworld is doing a review too
<menn0> thumper: regarding version.Number over the API, that's how it's done elsewhere and it's fine. version.Number has custom JSON marshalling defined which turns it into a string.
<thumper> menn0: if we are already doing it then fine
<thumper> drop em
<tych0> natefinch: hi, i am now
<axw> wallyworld: can you please take a look at http://reviews.vapour.ws/r/5534/diff/3-4/, made some changes in response to mhilton's review
<wallyworld> sure
<natefinch> tych0: hey.  I'm having difficulty figuring out this bug: https://bugs.launchpad.net/juju-core/1.25/+bug/1610880
<mup> Bug #1610880: Downloading container templates fails in manual environment <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1610880>
<natefinch> tych0: the crux seems to be that lcx-create is trying a wget for the image ... and for some reason on a manual juju environment it fails and on a normal juju environment it works
<natefinch> tych0: I know this is only tangentially related to anything you've worked on, but I was wondering if you had any ideas
<thumper> why is the lxd provider downloading a new ubuntu-xenial image?
<thumper> does it auto update?
<natefinch> new as in - it already has one and it's getting another?
<natefinch> and the answer is.. AFAIK, LXD maintains its own list of images, what it does with those is mysterious magic.  I don't *think* we tell it to auto-update, but I could be wrong.
<thumper> logging added, ci test running again, time to get food
<thumper> ugh
<thumper> ffs
<thumper> rerunning
<natefinch> oh interesting.... so, in a manual environment, we're not adding the cloud-local address for the machine to the cert
<menn0> wallyworld: I like the new bootstrap output. looks much better
<menn0> wallyworld: shouldn't these 2 things be the other way around though:
<menn0> Installing curl, cpu-checker, bridge-utils, cloud-utils, tmux
<menn0> Running apt-get update
<axw> pjdc: do you have an env that's currently exhibiting CPU/mem spikes as in https://bugs.launchpad.net/juju/+bug/1587644? I'm after a CPU profile in addition to what's been provided already
<mup> Bug #1587644: jujud and mongo cpu/ram usage spike <canonical-bootstack> <canonical-is> <juju:Fix Released> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1587644>
<pjdc> axw: not right now. but if you can tell us how to capture what you'd like captured, i'll add it to our ticket
<axw> pjdc: you should just use the same command as for the heap profile, but use /debug/pprof/profile instead of /debug/pprof/heap
<pjdc> axw: righto - will update the ticket
<axw> pjdc: which is described here for 1.25: https://github.com/juju/juju/wiki/pprof-facility
<axw> pjdc: thanks
<pjdc> axw: just testing it here; does this look right? https://pastebin.canonical.com/164042/plain/
<axw> pjdc: hrm, nope, that doesn't look right. should be much bigger. odd, it worked for me just now
<axw> pjdc: looks like the right command line invocation though ...
<pjdc> the single quote character seems pretty odd
<wallyworld> menn0: sorry, was at lunch. you could be right, i'll have to check
<anastasiamac> thumper: since u r in the manul area, ci also seem to have observed long bootstrap (>45min)... if u have any thoughts on this would be awesome to have them in this bug
<anastasiamac> https://bugs.launchpad.net/juju/+bug/1617137
<mup> Bug #1617137: Timing out fetching agent (xenial/s390x)  <bootstrap> <ci> <regression> <juju:Triaged> <https://launchpad.net/bugs/1617137>
<wallyworld> anastasiamac: that's not related, that's due to closed network
<wallyworld> curtis needs to reimport the images
<anastasiamac> wallyworld: if u can comment in the bug, m sure QA and veebers will apprecaite
<wallyworld> anastasiamac: curits already knows - he and i disucssed
<wallyworld> he has more detial than i do
<wallyworld> as to exactly what needs to be done
<wallyworld> as he did it originally
<veebers> wallyworld, anastasiamac ah ok, seems I missfiled that then. I'll get that fixed
<anastasiamac> wallyworld: sure, veebers filed the bug and this info is useful for anyone who was not in on ur discussion :D
<anastasiamac> awesome! tyvm :D
<wallyworld> but i don't have all the detail
<wallyworld> i only know generalities, i'd rather get the person who knows to comment
<wallyworld> sounds likes the QA folks need to talk to each other more :-)
<veebers> anastasiamac, wallyworld: If we can keep the bug for now (maybe not marked critical) I've updated the rule/issue to indicate it's a ci infra issue and we'll get those in the know to comment/remove/etc. thebug
<wallyworld> no worries
<wallyworld> having a bug means it can be tracked
<anastasiamac> veebers: feel free to adjust priority on it :D
<veebers> anastasiamac: sweet, have done. also added affects ci with a (vague) comment about network
<anastasiamac> veebers: wallyworld \o/
<wallyworld> the bug should be retargetted to CI, as curtis confirm it's not juju
<veebers> wallyworld: it has been
<wallyworld> awesome, tyvm
<veebers> wallyworld: oh no wait, I didn't remove juju, just added juju-ci
<veebers> perhaps I should have lied just now and said that I had removed juju too :-)
<wallyworld> we can add juju back if needed, but for now the best info is that it's a CI issue - "someone" needs to import LXD images
<menn0> veebers: with the changes landed today, there a bunch more prechecks in place
<veebers> wallyworld: removed now
<wallyworld> so that LXD uses those importd instead of calling out to cloud-images
<wallyworld> \o/
<veebers> menn0: oh neat :-) Does it break any assumptions the test makes currently?
<menn0> veebers: the main one you'll be interested in is that it isn't possible to upgrade if the target controller tools version is less than the model tools version
<menn0> veebers: no it shouldn't break the existing tests
<menn0> veebers: the tests to ensure the source and target aren't controller are done too (mostly landed)
<menn0> veebers: i'm implementing the prechecks to ensure that the source controller, model and target controller machines are healthy now
<veebers> menn0: nice. Might look at getting a test for between versions going next week
<menn0> veebers: sounds good.
<menn0> veebers: just keeping you informed :)
<veebers> menn0: :-)
<natefinch> wallyworld: is it me, or does this seem dangerous? https://github.com/juju/juju/blob/master/worker/certupdater/certupdater.go#L119  We're updating the saved addresses before knowing if we've actually successfully updated the cert.
<wallyworld> yeah, seems suboptimal
<natefinch> wallyworld: I'm looking at this bug about lxc containers on manual provider, and it seems like we're not adding the cloud-local address to the cert for some reason
<wallyworld> we use c.addresses to short circuit any future updates, i guess if things fail, that means we'll never process those addresses again
<natefinch> wallyworld: I don't think that's the actual problem
<natefinch> wallyworld: it just looks suspicious
<wallyworld> i can't recall enough about the code to know why cloud local addresses are not arriving at the cert updater
<wallyworld> they must be filtered out upstream somewhere
<wallyworld> the machine address setting code is a bit gnarly
<natefinch> wallyworld: yeah... we call state.APIHostPorts() when the cert updater starts, and add any local-cloud addresses... on manual, it gets no addresses, on gce, it gets the correct local-cloud address
<wallyworld> oh, manual
<wallyworld> we won't get any
<natefinch> why not?
<wallyworld> the cloud local addresses come from the instance poller from memory
<wallyworld> and there's no such thing as an instance poller for manual IIANM
<natefinch> ug
<wallyworld> this is a bit hand wavy
<wallyworld> i could be wrong
<wallyworld> but generally the instance poller is a major source of our knoweledge of machine addresses
<natefinch> you're at least slightly wrong... I see juju.state address.go:137 setting API hostPorts: [[104.196.3.75:17070 10.142.0.2:17070 127.0.0.1:17070 [::1]:17070]] on manual
<wallyworld> with manual, we wil get machine local addresses
<wallyworld> right, but it depends on how those are classified
<natefinch> that 10.142 address is what lxc-create is trying to wget the image from
<wallyworld> by our address heuristics
<natefinch> hmm
<wallyworld> we label addresses as machine local, cloud local, public etc
<wallyworld> what is the machine foe which those host ports are being set above>
<wallyworld> ?
<wallyworld> a controller or a worker machine?
<natefinch> controller
<wallyworld> anyways, 127.0.0.1 looks wrong
<wallyworld> cause if that is handed out as a controller address, it can't work
<natefinch> works from that machine ;)
<wallyworld> might not be the same issue as your seeing, but looks wrong to me
<natefinch> it gets set that way on my gce controller as well, so probably not the problem
<wallyworld> so from memory, set addreses happens and we filter somehow and then hand out to cert updater, but i can't really recall exactly
<wallyworld> maybe the filtering takes care of localhost
<wallyworld> axw: i've updated http://reviews.vapour.ws/r/5533/ if you could PTAL, a bit of code was moved around
<axw> okey dokey
<wallyworld> axw: added a fix and a test for the tranitive permission etc
<thumper> well, with menn0's help, debugging is progressing
<thumper> it is in the proxy updater where it is trying to set the lxd proxies
<thumper> it is just blocking forever
<thumper> holding workers up
<thumper> anyway
<thumper> more debugging for monday
 * thumper is done for now
<thumper> laters folks
<axw> wallyworld: found an existing bug, but otherwise LGTM
<wallyworld> ta, looking
<wallyworld> axw: yeah, that was existing behaviour before I started this PR. it does seem wrong doesn't it
<wallyworld> actually
<wallyworld> it is as per the all users case above, but it makes more sense there
<wallyworld> i'll change to errperm
<wallyworld> axw: ah, the existing client code will bail out with an error if *any* of the requested users results in an error
<axw> wallyworld: :/
<wallyworld> so that's why the server side was skipping over unpermitted users
<axw> wallyworld: we should do that filtering on the client, not on the server
<wallyworld> agreed
<wallyworld> axw: for now, how about i just modify the api caller to skip err perms
<wallyworld> but error for other things
<axw> wallyworld: sounds OK I guess
<wallyworld> or the other things is we could print the errors for tabular output
<wallyworld> as well as the users it can find
<wallyworld> we don't have such a good pattern for this
<wallyworld> bah, too much churn, i'll add a todo
<wallyworld> and the CLI only passes one at a time anyway
<axw_> mhilton: thanks for the review
<mhilton> axw_: np
<tych0> natefinch: replied on the bug
<tych0> but it looks like wget is refusing to download something because the certs don't match?
<lazyPower> interesting new format of juju debug-log in beta16, it appears it rolled over to json formatting by default?
<babbageclunk> mgz: around?
<mgz> babbageclunk: yo
<babbageclunk> Trying to cleanup instances from running CI tests with --keep-env
<babbageclunk> but I can't find my AWS credentials - I may have lost them when my machine died.
<mgz> so, what I do is JUJU_DATA=~/cloud-city/jes-homes/gz-test-env-name $GOPATH/bin/juju destroy-controller
<babbageclunk> oh, actually, I have the user name and password, just don't have the URL for the canonical console
<mgz> depends if you have the env on disk still? you don't need seperate creds unless you wiped the env from disk
<mgz> which creds were you using to test?
<babbageclunk> I don't think that'll work - the api isn't up because the restore failed.
<mgz> the shared dev ones, or CI's?
<babbageclunk> Hmm - I guess the CI's ones.
<mgz> babbageclunk: then fall back to kill-controller - which again should get the aws details out of JUJU_DATA
<babbageclunk> opk
<babbageclunk> ok
<mgz> try that, yell if you have problems
<mgz> the aws console details for ci are in the consoles.txt file
<mgz> be careful poking around there, if you do use
<babbageclunk> ok, thanks mgz
<ram___> andrey-mp: Hi. w.r.t your information yesterday, I created "cinder-storagedriver"  charm which will pass config data to relation to modify the configuration. I have taken your scaleio-openstack as reference. How do you certify scaleio-openstack charm? Please provide me the juu charm certification requirements.
<katco> natefinch: standup time
<natefinch> katco: oops, thanks
<admcleod> looks like juju has overwritten authorized_keys rather than appending to it, is that normal? (1.25)
<mgz> admcleod: yes
<admcleod> mgz: oh.
<admcleod> mgz: thanks
<mgz> admcleod: juju expects to manage the file, so it hass list/add/delete/import
<mgz> to manage
<admcleod> right fair enough
<mgz> that is a surprise if you've edited it manually on a machine yourself perhaps :)
<mgz> babbageclunk: yell if you have any questions about my ssh key branch
<katco> frobware: dimitern: thank you for all your hard work for ivoks
<dimitern> katco: :) cheers
<babbageclunk> mgz: LGTM!
<mgz> babbageclunk: merci
<katco> dimitern: lmk when you have that hotfix ready so i can pass it on
<dimitern> katco: will do
<katco> dimitern: ta
<mattyw> is someone able to talk to me about the new upload-tools logic?
<natefinch> mattyw: it just works, as long as jujud is in the same directory as the juju you're running and they have the exact same version number
<mattyw> natefinch, that's the thing - for me it didn't just work
<mattyw> natefinch, so I'm trying to work out why
<natefinch> mattyw: is it uploading when it's not supposed to, or vice versa?
<babbageclunk> perrito666: ping?
<perrito666> babbageclunk: pong
<dimitern> katco: I've sent this patch to ivoks - tested locally and seems to work, waiting on feedback later: http://paste.ubuntu.com/23093530/
<babbageclunk> perrito666: I'm trying to understand backup and restore! :)
<katco> dimitern: yay! ty! hope the feedback is good
<babbageclunk> perrito666: various people said you were the one to talk to.
<perrito666> babbageclunk: sadly I am
<alexisb> dimitern, frobware you guys are rockstars thank you!
<katco> perrito666: don't you wish you had a... backup?
<katco> perrito666: it would... restore your sanity
 * babbageclunk lols
 * katco groans
<perrito666> katco: you should do that for a living
<dimitern> :) let's see if it'll be any good on site
<alexisb> babbageclunk, perrito666 I actually wanted to talk to both of you about the bugs youa reworking
<alexisb> do you guys have time for a quik HO?
<perrito666> alexisb: sure
<babbageclunk> alexisb: sure!
<alexisb> sorry got distracted by some other fires
<alexisb> perrito666, babbageclunk https://hangouts.google.com/hangouts/_/canonical.com/a-team-standup
<perrito666> its a trap
<perrito666> could anyone http://reviews.vapour.ws/r/5546/ ?
<perrito666> its a quick short one
<natefinch> I can look
<perrito666> natefinch: just added QA steps
<natefinch> ahh crap, forgot about QA steps
<natefinch> well that quintuples the amount of time I have to spend reviewing it :/
<marcoceppi> why does the Juju client still hard code support for OS?
<marcoceppi> https://bugs.launchpad.net/juju/+bug/1616531
<mup> Bug #1616531: "panic: unknown OS for series" when running client on Fedora <juju:Triaged> <https://launchpad.net/bugs/1616531>
<natefinch> lol fedora
<perrito666> natefinch: oh, that is not nice
<perrito666> fedora is a fine distro
<natefinch> I am sure it is.
<natefinch> also, why in the everloving hell does the client give a crap what OS it's running on?
<perrito666> I thought we had fixed that bug ages ago
<natefinch> no
<natefinch> we talked about it in december and never did it
<marcoceppi> :sadpanda:
<alexisb> perrito666, is this still an issue: https://bugs.launchpad.net/juju/+bug/1530840
<mup> Bug #1530840: juju status-history too noisy with update-status <landscape> <juju:In Progress by alexis-bruemmer> <https://launchpad.net/bugs/1530840>
<perrito666> alexisb: priv
 * redir lunches
<redir> so quiet on fridays:)
<natefinch> yep
<natefinch> ok, I give up
<natefinch> see you all next week
<redir> I didn' miss anyone at the stand-up did I?
<redir> I mean I just didn't join this week because there hasn't been anyone there since we rearranged
<redir> ok
 * redir goes EoW soon-ish
<redir> see you next week juju
#juju-dev 2016-08-27
<gennadiy> hi everybody
<gennadiy> i have a question: do i need to build juju agent for each platform separately? or i can use the same jujud for ubuntu and centos?
<gennadiy> because i have found a set of agents here - https://streams.canonical.com/juju/tools/agent/2.0-beta16/ but haven't found any information about it on the github
<mup> Bug #1617526 opened: cmd/juju: no help available for common flags <juju-core:New> <https://launchpad.net/bugs/1617526>
<admcleod> getting a 404 on lxc container creation for this:
<admcleod> 2016-08-27 11:27:30 DEBUG juju.container image.go:89 lxc image for xenial (s390x) is https://cloud-images.ubuntu.com/server/releases/xenial/release-20160826/xenial-server-cloudimg-s390x.tar.gz
<mup> Bug #1617602 opened: juju status <service-name> stucking  <juju-core:New> <juju-utils:New> <https://launchpad.net/bugs/1617602>
#juju-dev 2016-08-28
<thumper> menn0: found the issue, but just running car in to get fixed
<thumper> would love to chat when I'm back
<menn0> thumper: ok sounds good
<thumper> back
<thumper> 1:1?
<thumper> menn0: ^^
<menn0> thumper: coming
<mup> Bug #1617820 opened: JUJU fails in bootstrapping used with openstack <juju-core:New> <https://launchpad.net/bugs/1617820>
<mup> Bug #1617820 changed: JUJU fails in bootstrapping used with openstack <juju-core:New> <https://launchpad.net/bugs/1617820>
<mup> Bug #1617820 opened: JUJU fails in bootstrapping used with openstack <juju-core:New> <https://launchpad.net/bugs/1617820>
<thumper> menn0: ffs
<thumper> menn0: recreated manual lxd images
<thumper> not it is working
<thumper> without me changing anything
<thumper> :-|
<thumper> tych0: hello there
<menn0> thumper: the client side cert is still stored somewhere?
<menn0> thumper: isn't the cert for all comms to the daemon, not per container?
<thumper> when I ran lxc list on one of the newly created machines, it said "generating cert" again
<menn0> ah ok
<menn0> thumper: I was just guessing
<thumper> but after it worked...
<thumper> ffs
<menn0> thumper: did you try the client before you tried juju? (i.e. did the client generate the cert before you tried out the juju repro again?)
<thumper> nope
<thumper> I did a clean recreate
<thumper> then tried the juju ci tst
<thumper> and was surprised when it worked first time
<thumper> hadn't rebuilt juju or anything
<menn0> nfi
<thumper> so much for knowning what was going on
<menn0> thumper: better get a hold of a lxd guru
<thumper> yeah, will email
<thumper> veebers: heading to the gym for lunch, lets chat after that
<veebers> thumper: hah, was going to say the same thing to you :-)
<veebers> sounds good
#juju-dev 2017-08-21
<menn0> thumper: could you pls take a look at this: https://github.com/juju/juju/pull/7762
<menn0> no rush
<menn0> thumper: i'm particularly interested to see what you think about embedded Model in IAASModel
<thumper> menn0: ok, will look a bit later
<thumper> I have a guy here doing wiring
<thumper> getting three unifi APs put up
<veebers> thumper: presumably he knows more than I do and has an easier time than I did at mine :-P
<thumper> :)
<thumper> I'm just leaving him to it
<veebers> heh, smart move
<thumper> well, just come down from the ceiling
<veebers> I can't help but feel that's missing some context thumper
<thumper> not through the ceiling
<menn0> thumper: 1:1?
<thumper> coming
 * babbageclunk goes for a run
<babbageclunk> axw: some sanity checking - in upgrading agents I had some code that was updating the API password with the password the upgrade tool was using to connect to the target controller. But that's wrong, isn't it? I should just be leaving the passwords in the config alone, because they'll be correct once the migrated model is imported in the target controller, right?
<axw> babbageclunk: yes, the password hashes should be sent across in the migration description
<babbageclunk> axw: cool thanks
<Guest20465> jam, menn0: https://github.com/juju/juju/pull/7766
<thumper> wow...
<thumper> wonder why that happened
<jam> thumper a guest in your own house :)
<blahdeblah> What's the best way to get juju debug-log to show me just charm output, and get rid of all the agent messages?  https://pastebin.canonical.com/196296/ <-- only about 5 lines of that is from the actual charm
<blahdeblah> The agents have become *much* more chatty under 2.2.2+
<blahdeblah> Also, is there a way to specify applications or unit wildcards?  If I have 8 units of a given application, I don't want to have to list all of them as --includes
#juju-dev 2017-08-22
<thumper> babbageclunk: could I get you to take a look at https://github.com/juju/juju/pull/7766 please?
 * thumper goes to walk the dog
<babbageclunk> thumper: sure
<thumper> thanks
<menn0> thumper: easy review pls: https://github.com/juju/description/pull/19
 * thumper looks
<thumper> hmm...
<thumper> I too have a branch that bumps the model type to 4
<thumper> model vesion
<thumper> perhaps we should get them merged close to gether
<menn0> makes sense
<menn0> thumper: is yours ready?
<thumper> um...
<thumper> kinda
<menn0> thumper: well if I merge what I have now, and you're not far off, that should be fine
<menn0> thumper: would you be able to hit merge on https://github.com/juju/description/pull/19 pls?
<thumper> yep
<thumper> menn0: done
<menn0> thumper: thanks
<babbageclunk> axw: thanks for the review!
<menn0> thumper: review pls: https://github.com/juju/juju/pull/7770
<babbageclunk> thumper: Have you profiled the FullStatus call? Would be interesting to see what hotspots that highlighted.
<thumper> babbageclunk: I haven't yet, but I'd like to
<axw> anastasiamac jam: to update the models, list-controllers --refresh could just call the RefreshModels method. we can/should make RefreshModels replace the entire set of models stored in the cache with what's retrieved
<axw> anastasiamac jam: still need to get model status to set the machine count though
<axw> or we could update the ListModels method to include that in the result
 * axw wishes machine count wasn't cached
<anastasiamac> axw: yes, the idea is similar ecxcept maybe a bit more efficient than RefreshModels. I was going to improve RefreshModels - right now, we make a call to a store to update each model sequentially which also acquires alock for each call..
<anastasiamac> axw: i was going to change store to accept a collection of models to update (more efficient when we know that some controllers have a *few* models)
<anastasiamac> axw: that call would not be great for 'controllers' because it's controller specific, i.e. we'd end up with an additional api call for each controller in the list.. I think it's best to do what we've agreed on: api call will return desired models info. this will only have 1 api call instead of 1+nXcontrollers
<anastasiamac> axw: i'll propose later on today- would appreacite an opinion :D
#juju-dev 2017-08-23
<axw> anastasiamac: if we can avoid it, I'd rather not change the API to be bulk, but instead, have a transaction API. Begin, <make a series of updates>, End
<anastasiamac> axw: why not? with current indiivudal update, u r taking  lock every single time... if u have 100s models to update, u'll b getting this lock 100s times... feels yuck
<anastasiamac> axw: there is no problems having an UpdateModel and UpdateModels
<axw> anastasiamac: any additional code is not free, so I reject the statement that there are "no problems". what I meant by a transactional API is one that takes a lock at the beginning of the transaction, and releases it at the end, and only does a single write at the end
<axw> anastasiamac: that will be useful for doing things other than just writing a bunch of models at once. e.g. writing controller, account, and model details all at once
<anastasiamac> axw: u misread what i said... right now jujuclient api has 2 official/our implementations: file and mem. mem is used for testing. file is the real deal/production we ship. file one takes a lock in implementation (i.e. once u r in a call to UpdateModel)...
<anastasiamac> axw: right now, RefreshModels call store to UpdateModel per each model and each of these calls takes a lock.
<axw> anastasiamac: I understand all that, so not following how I've misread...
<anastasiamac> axw: if u r updating a collection, u have n muber of models and n of locks
<anastasiamac> axw: store API should support updating bulk
<anastasiamac> axw: don;t get thr resisitance... how did we add accounts, cookie jar, etc to jujuclient api...
<axw> anastasiamac: yep, at the moment. again: I'm proposing we introduce a pair of methods, Begin & End (say), which will make the locking happen in the Begin and End calls rather than each UpdateModel call
<axw> anastasiamac: maybe this is better on a HO, I don't think I'm getting my point across very well here
<anastasiamac> axw: sure, but this are internal to jujuclint implementation. why expose Begin and End, especially i interface?
<anastasiamac> axw: k. m ahppty to HO... gimme 10mins to put a face on :D
<axw> anastasiamac: because it would be useful for reasons other than updating a bunch of models at once...
<anastasiamac> axw: sure, but would b irrelevant to other implementation of client.. like mem...
<anastasiamac> let's talk ina 10-15.. i'll ping
<axw> anastasiamac: not really. they can use an (in-memory) lock too. they're just for testing atm though. talk soon...
<babbageclunk> axw: ping?
<axw> babbageclunk: pong!
<babbageclunk> axw: Hi! Can you explain the maas agent name thing to me?
<babbageclunk> axw: What's it updating it from/to?
<axw> babbageclunk: sure. do you know what agent_name is?
<axw> I mean, conceptually, not the value
<babbageclunk> axw: nope
<axw> babbageclunk: ok. when you acquire a node from MAAS, you can specify a string (agent_name value) to associate with that node. you can then use that for filtering node queries later on
<axw> babbageclunk: we use it to identify nodes for a model
<axw> babbageclunk: in 1.25, we used to generate a new UUID for the agent_name. in juju 2.x, we just use the model UUID
<babbageclunk> axw: So is it destructive? After the update could the old environment still run?
<babbageclunk> axw: Thanks though, that makes sense.
<axw> babbageclunk: the old environment won't find those nodes any more, unless you were to change it back again
<babbageclunk> axw: I guess in 1.25 the uuid is stored somewhere in config - could we update it there so it still matched up?
<axw> rollback for that can't really be automated, because you need to get onto the region controller
<axw> babbageclunk: yes it is, and yes we could
<thumper> oh ffs
<babbageclunk> axw: Ok - I'll do that, if you can point me to the right place?
<thumper> I'm looking at merging 2.2 into devel
 * babbageclunk ducks
<thumper> there are so many conflicts
<thumper> I thought that wallyworld fixed those
<axw> babbageclunk: wait... the command already does that :)
<babbageclunk> thumper: I think they're probably caused by the rearranging of API facades?
<axw> babbageclunk: at the end, it calls UpdateEnvironConfig with the new maas_agent
<babbageclunk> axw: ha ha, that was clever of you!
<thumper> babbageclunk: some of it
<babbageclunk> thumper: Might be simpler if we make the same change in 2.2 branch.
<wallyworld> each time you merge into develop now you'll get conflicts
 * thumper enfixorates
<thumper> wallyworld: really? that blows
<thumper> can't it figure this shit out?
<babbageclunk> axw: Ok cool, so it sounds like this isn't really a destructive thing.
<wallyworld> not that i've seen
<axw> babbageclunk: yeah sorry, I forgot I did that bit
<axw> babbageclunk: if you were to revert to an older version of state, it would be busted
<axw> in that case you'd need to update maas_agent in the MAAS pg database
<wallyworld> thumper: the issue from what i recall is the big facade moves, it gets confused with those if you make facade changes
<babbageclunk> axw: You mean an older version in the juju1 tree?
<babbageclunk> thumper: That's why I think it might be easier if we do the same rearranging in the 2.2 branch.
<axw> babbageclunk: no I mean, if you took a mongo backup before running the update-maas-agentname command, and reverted to that
<babbageclunk> Ah, right.
<babbageclunk> thumper: (although I'm not sure how much work the rearranging was - at least a lot of import fixing, maybe more than that.)
<thumper> I have a feeling that some of these methods that 2.2 is trying to add to the client backend interface have moved
 * thumper sighs
 * thumper takes a stab
<thumper> oh ffs
<thumper> axw: do you have a few minutes?
<thumper> perhaps jump in tech board early?
<axw> thumper: sure, brt
<thumper> looking at 2.2 merge into develop
<babbageclunk> thumper: hey do you have a moment?
#juju-dev 2017-08-24
<menn0> thumper, wallyworld, axw or babbageclunk: removed remaining ForModel calls in apiserver: https://github.com/juju/juju/pull/7781
<axw> menn0: awesome, will take a look after standup
<axw> menn0: or now... too easy. thanks!
<menn0> axw: cheers. couldn't resist given how easy these were to remove.
<thumper> wallyworld: ick... looks like it may be an internal database race
<wallyworld> oh joy
 * babbageclunk goes for a run
<wallyworld> babbageclunk: if you feel inclined when you get back, here's a pretty much mechanical PR to refactor relations watcher for subsequent use in a followup pr https://github.com/juju/juju/pull/7783
<babbageclunk> wallyworld: looking now
<wallyworld> babbageclunk: damn, i'll need to do a juju/description  pr as well to add status
<babbageclunk> wallyworld: ah stink
<wallyworld> trivial, so no biggie
<wallyworld> just a dep change in the main pr
<thumper> oh ffs
<thumper> finally got all conflicts resolved and tests passing
<thumper> and now develop has moved to create more conflicts
 * thumper shakes fist at sky
<anastasiamac> sky is not on irc today, thumper :D
<wallyworld> babbageclunk: thats for review, muchly appreciated
<wallyworld> *thanks
<babbageclunk> wallyworld: nw - sorry, should have pinged you here too
<wallyworld> babbageclunk: no worries, i'll quickly whip up the juju/description change to add status
<wallyworld> axw: remind me - if there's an extra field in an exported model entity, will an older import complain about the extra field? If so, I'll need to bump the version of the entity. The field is given a default so importing from earlier exports is no problem
<anastasiamac> axw: update PR. forgot to deal with current_model potential change. PTAL when u get a chance? I'd like to get jam to review as well once u r happy :D
<axw> wallyworld: entity? are you talking about the all watcher? I'm not familiar with the compatibility there
<axw> anastasiamac: thanks, looking
<wallyworld> axw: no model import/export; juju/description
<wallyworld> adding status to relation
<axw> wallyworld: ok. I don't know, but I think it'd be best to bump the version
<wallyworld> ok
<wallyworld> axw: no rush, here's that small juju/description pr https://github.com/juju/description/pull/20
<wallyworld> jam: did you have 5 to touch base about that network get pr?
<jam> wallyworld: sure. I also didn't realize that witold's patch had not landed yet, as he addressed some of those things as well
<jam> https://github.com/juju/juju/pull/7707
<wallyworld> jam: ok, let me look briefly at that pr first. i need to cook dinner etc soon, so just wanted to catch up so i could not be blocked later
<jam> wallyworld: no problems, let me know when you're around
<wallyworld> jam: ok, got the context, got 5 now?
<wallyworld> maybe techboard ho
<thumper> axw: https://github.com/juju/juju/pull/7786 if you could just check the model deletion stuff
<axw> thumper: ok
<thumper> thanks
<axw> thumper: thank *you*
<axw> thumper: LGTM
<thumper> sweet
<redir> s
<rick_h> redir: how's the little one?!
<redir> heh
<redir> good
<redir> thanks
<redir> Didn't realize I was chatting
<redir> surprised that wasn't :1
<redir> surprised that wasn't :q even
<redir> how's your little one?
<redir> Chloe is starting to bat at toys hanging from her activity mat.
<redir> Amazing how simple it is but how much of an advance to see watching her for the last 11 weeks
<rick_h> redir: <3 good to hear. Dinner here so going to run but woot
<alexisb> redir, congrats!  Chloe is a great name :)
<babbageclunk> yay, congrats redir!
<redir> thank you all
<wallyworld> redir: hey! nice to see you online, hope all is going well
<wallyworld> hml: i left a few comments on the pr, see if they make sense, pretty close now
<hml> wallyworld: ty! looking shortly
<wallyworld> rick_h: i'd love to chat about cmr address things, to fill you in on plans. maybe we can schedule something for next week
<rick_h> wallyworld: k, before our regular sync?
<rick_h> wallyworld: ah, we don't have it next week it looks like. Have a preference?
<wallyworld> rick_h: we were supposed to have one this morning but i guess we missed it. maybe we can aim for early in the week?
<wallyworld> just wanted to keep you in the loop
<rick_h> wallyworld: rgr, will add something
<wallyworld> rick_h: i was trying to avoid charm changes but there will need to be. your juju show example will need telegraph to be tweaked
<rick_h> wallyworld: rgr
<wallyworld> and/or prometheus
#juju-dev 2017-08-25
<axw> thumper: I'm back
<axw> wallyworld: there appears to be an intermittent failure in TestRemoteRelationRequirerRoleConsumingSide, timing out. see http://ci.jujucharms.com/blue/organizations/jenkins/github-check-merge-juju-pl/detail/PR-7779/1/pipeline
<wallyworld> thanks, will look
<thumper> axw: got 5 minutes?
<axw> thumper: yup
<thumper> axw: 1:1?
<axw> ok
<axw> menn0: you've got compile errors in cmd/juju/application/bundle_test.go, in your annotations branch
<menn0> axw: ok thanks
<wallyworld> axw: i've pushed changes to https://github.com/juju/juju/pull/7760 if you get a chance whenever
<axw> wallyworld: done
<wallyworld> axw: thanks, i'll adjust the versioning. belinda is feeling quite ill again so i'll likely be afk for a bit
<axw> wallyworld: :( ok
<wallyworld> axw: yeah, back to the hospital, showing same signs as last time. ttyl
<babbageclunk> wallyworld: bugger. Hope she's doing ok.
#juju-dev 2017-08-27
<wallyworld> babbageclunk: you around?
<babbageclunk> wallyworld: sorry, omw
#juju-dev 2018-08-21
<mup> Bug #1788095 opened: aws-gov can't read cloud streams file <juju-core:New> <https://launchpad.net/bugs/1788095>
<mup> Bug #1788095 changed: aws-gov can't read cloud streams file <cloud-images:New> <juju:Incomplete> <https://launchpad.net/bugs/1788095>
