#juju-dev 2012-05-21
<TheMue> morning
 * davecheney waves
<davecheney> TheMue: https://codereview.appspot.com/6203083
<davecheney> coming along
<davecheney> http://paste.ubuntu.com/998421/
<rogpeppe> TheMue, davecheney: yo!
<TheMue> rogpeppe: Yo'  man.
<rogpeppe> davecheney: looking good. one little question: is it possible to start a machine that's already been created?
<rogpeppe> davecheney: i mean, will we ever want to actually do that?
<davecheney> rogpeppe: good question, curretly the start machine and create machine.loop are closely tied
<davecheney> similarly when a stop is successful, the machine will regestier itself so i can't accept more messages
<davecheney> *deregister
<rogpeppe> davecheney: if we *don't*', then i'm not sure there's a need for the "start" message - we can just start when the machine is created.
<davecheney> rogpeppe: true, it's probably overengineered as it stands now
<davecheney> i'm not doing any more work on it this evening
<rogpeppe> davecheney: and the loop becomes a single channel receive...
<davecheney> i'm tyring to fix the tests on https://code.launchpad.net/~dave-cheney/juju/go-watch-machines-topology/+merge/106294
<rogpeppe> davecheney: cool.
<davecheney> rogpeppe: I also have to implement so kind of backoff logic in there
<rogpeppe> davecheney: yeah.
<rogpeppe> davecheney: that should be straightforward though - the goroutines mean no state machine is necessary, which should make the logic nice.
<davecheney> <- time.After(5 *time.Seconds) would do in a pinch
<rogpeppe> davecheney: yeah. maybe add a touch of randomness too.
<davecheney> rogpeppe: i'm thinking a sort of incresing backoff
<davecheney> that you kind of hint by saying .Success() or .Failed()
<davecheney> which increases or decreasese the delay
<davecheney> so if things start to crap out, the delay between attempts grows
<rogpeppe> davecheney: if you want to factor out the backoff strategy, you might want to have a look at attemptStrategy in environs/ec2, as a possible API example
<davecheney> rogpeppe: that sounds like exactly what I want
<rogpeppe> davecheney: it doesn't do exponential backoff
<rogpeppe> davecheney: but the API works nicely and was much discussed at the time
<davecheney> all the better to reuse it then
<rogpeppe> davecheney: mind you, perhaps you never want to give up, in which case perhaps the API is overkill.
<rogpeppe> davecheney: BTW did you see, someone discovered my shell? :-) http://debu.gs/entries/inferno-part-1-shell
<davecheney> \o/ hacker news
 * rogpeppe never looks at hacker news. one stream too many.
<TheMue> rogpeppe: Any usage statistics for your shell? Download numbers?
<rogpeppe> TheMue: absolutely no idea. probably in the low 20s :-)
<davecheney> rogpeppe: i was going to say, it's probably not allowed to drink
<rogpeppe> TheMue: noone uses it
<TheMue> rogpeppe: Hehe, not even you?
<davecheney> rogpeppe: do you know a guy called Chris Collins
<rogpeppe> davecheney: occasionally.
<rogpeppe> drink?
<rogpeppe> davecheney: not that i can remember, why?
<davecheney> he's a plan 9 fan here in AU
<davecheney> he's also a hardware hoarder like you
<rogpeppe> davecheney: problem is inferno lives in a walled environment. it's great for dealing with stuff inside that environment, but not great if you want to interact with the host system
<rogpeppe> davecheney: i am *not* a h/w hoarder :-)
<rogpeppe> davecheney: i just have a NeXT!
<davecheney> rogpeppe: can the watcher.ContentWatcher return "" ?
<davecheney> ie, return spuriously
<rogpeppe> davecheney: it could return "" if the contents were ""
<davecheney> what if ther were "" before
<rogpeppe> davecheney: and i think if there's an error, the channel will be closed
<davecheney> the problem i'm having atm is the watcher fires once, but the contents don't change
<davecheney> so it gets out of phase with what the test expects
<rogpeppe> davecheney: if the contents don't change, the watcher shouldn't fire
<rogpeppe> davecheney: is that what you want?
<davecheney> let me check the channel isn't closing
<davecheney> watcher channel closed
<davecheney> watcher_test.go:275:
<davecheney> sonovabitch
<davecheney> ok, let me find out why
<davecheney> btw, /usr/share/zookeeper/bin/zkCli.sh
<davecheney> ^ i'm sure you guys know about his
<davecheney> this
<rogpeppe> davecheney: it's really nasty. i wrote my own little cli for it.
<davecheney> watcher channel closed: watcher: can't get content of node "/machines": zookeeper: existsw "/machines": zookeeper is closing
<davecheney> urg
<davecheney> rogpeppe: it is really nasty, you can't set a key that contains whitespace
<rogpeppe> davecheney: nice informative error message though :-)
<rogpeppe> davecheney: *really*!?
<rogpeppe> davecheney: omg that's not nice
<davecheney> rogpeppe: really.
<davecheney> rogpeppe: i ended up adding code into the provisionng agent to populate zk for me
<davecheney> as a test
<rogpeppe> davecheney: oh, you mean the cli shell script
<rogpeppe> davecheney: i thought you meant zk proper
<davecheney> rogpeppe: nah, just zkCli
<rogpeppe> davecheney: if you look at it, it's got so much abstraction in it it's not true. not worth using.
<rogpeppe> davecheney: definitely written by a jobsworth.
<davecheney> rogpeppe: right, problem fixed, was watching the wrong key
<davecheney> but now I have another problem
<davecheney> ... obtained *state.MachinesChange = &state.MachinesChange{Added:[]*state.Machine{(*state.Machine)(0xf8400c9360)}, Deleted:[]*state.Machine(nil)}
<davecheney> ... expected state.MachinesChange = state.MachinesChange{Added:[]*state.Machine{(*state.Machine)(0xf8400daf00)}, Deleted:[]*state.Machine(nil)}
<rogpeppe> davecheney: that's an easy one to fix
<davecheney> *Machines don't define equility
<rogpeppe> davecheney: just expect the pointer type
<rogpeppe> davecheney: it's doing DeepEquals, right?
<davecheney> mm
 * davecheney reads the godoc for gocheck
 * TheMue moved his office to the veranda due to the beautiful weather today.
<rogpeppe> TheMue: the weather *should* be beautiful today, but the haar has come in. dull again.
<TheMue> rogpeppe: We are to far away from coast for haar (about 50 to 80 km, depending on if the Jadebusen, a bay, is seen as coast too). So now it's sunny and warm.
<TheMue> s/to far/too far/
<rogpeppe> TheMue: we had the first decent sunshine in ages yesterday.
<TheMue> rogpeppe: Hehe, that's England. ;) But we're also not very spoiled by the sun here.
<rogpeppe> TheMue: it has been *particularly* bad in the last month.
<TheMue> rogpeppe: We had a short sunny interrupt in March, but in total the spring is too late this year.
<rogpeppe> TheMue: same here. must be the same across most of northern europe, i'd guess
<TheMue> rogpeppe: Yep
<niemeyer> Hello jujuers!
<TheMue> niemeyer: Heya
<Aram> evening.
<niemeyer> Aram: Heya
<andrewsmedina> niemeyer: morning
<rogpeppe> niemeyer: hiya
<niemeyer> Hey guys
<andrewsmedina> niemeyer: I sent the code for review
<niemeyer> andrewsmedina: That's great, thank you
<andrewsmedina> niemeyer: :D
<niemeyer> rogpeppe: ping
<rogpeppe> niemeyer: pung
<niemeyer> rogpeppe: Yo
<niemeyer> rogpeppe: Just wondering what's the status of your branches
<rogpeppe> niemeyer: how's tricks?
<niemeyer> rogpeppe: You've mentioned you had a pre-req that was being merged in
<rogpeppe> niemeyer: yeah, you LGTM'd that and i submitted on friday.
<niemeyer> rogpeppe: Is the branch named trunk (!?) ready for review?
<niemeyer> rogpeppe: Right, but did you repush the follow up?
<niemeyer> s/repush/re-proposed/
<rogpeppe> niemeyer: ahem, forgot to branch -m that one
<rogpeppe> niemeyer: i *think* that was an orthogonal branch. let me check.
<niemeyer> rogpeppe: Ok, just want to make sure I'm not reviewing something you don't want me to
<rogpeppe> niemeyer: yes, i'd love a review of that on if poss.
<niemeyer> rogpeppe: Sounds good, starting right away
<rogpeppe> niemeyer: if i was able, i'd rename the branch "go-environs-simplify-provider-interface"
<niemeyer> rogpeppe: Cool, no worries
<rogpeppe> niemeyer: i think you'll like it BTW
<niemeyer> rogpeppe: Sweet
<niemeyer> rogpeppe: As an unrelated note that I just reminded, I pushed a goamz branch over the weekend to improve and fix s3 a bit
<niemeyer> rogpeppe: I gave a try at using it for dreamhost's Objects service.. works!
<rogpeppe> niemeyer: yeah, i'm intending to look at it. sorry about the breakage BTW!
<niemeyer> rogpeppe: I mean, with the branch..
<niemeyer> rogpeppe: No worries.. I think we all just have to be more diligent at running tests pre-submit
<rogpeppe> niemeyer: maybe lbox submit's auto-merge functionality is really unnecessary, 'cos you should merge and test before submitting anyway
<niemeyer> rogpeppe: Turns out it's quite boring to do that by hand all the time
<rogpeppe> niemeyer: yeah, i guess.
<niemeyer> rogpeppe: I know for sure.. been there :)
<niemeyer> (for years!)
<rogpeppe> niemeyer: but it certainly makes it easier to submit broken code...
<niemeyer> rogpeppe: It makes it easier to submit any code.. broken or not
<niemeyer> rogpeppe: Making that process more painful isn't a great way to solve the issue
<rogpeppe> niemeyer: case: i merge, test, someone else submits a change to an API i'm using, i submit, lbox auto-merges => broken build.
<TheMue> niemeyer: Any chance to take a look at https://codereview.appspot.com/6200044/ ?
<rogpeppe> niemeyer: the gap between me testing and me submitting can be arbitrarily short
<niemeyer> TheMue: There are great chances, yeah :)
<niemeyer> TheMue: I've been reviewing the queue regularly
<niemeyer> TheMue: So if it's in the active branches list, it will get reviewed
<TheMue> niemeyer: Just wondered, because you already reviewed a follow-up.
<niemeyer> rogpeppe: Sure, that's a problem.. let's fix it. We can do that without making our lives more painful.
<TheMue> niemeyer: Right now I've finished Addâ¦Relation() with tests as a next branch.
<niemeyer> TheMue: Yeah, as I said, I've been reviewing regularly
<niemeyer> TheMue: Every day
<rogpeppe> niemeyer: is there a way to fix it without getting lbox to run the tests itself?
<niemeyer> TheMue: I've also been ordering reviews by branch size
<niemeyer> TheMue: If you want faster turn arounds, smaller branches will win
<TheMue> niemeyer: OK
<rogpeppe> i was thinking that branch size should be judged by bzr diff | grep '^>' | wc -l
<niemeyer> rogpeppe: Branch sizes is ordered by diff size
<rogpeppe> otherwise we get penalised for many small changes, and also for deleting code
<niemeyer> rogpeppe: https://code.launchpad.net/juju/+activereviews shows the number of lines
<niemeyer> rogpeppe: Eventually we can have that.. we'll have to write logic that checks out branches, analyzes them, and builds a list in our preferred fashion
<niemeyer> rogpeppe: Meanwhile, Launchpad has that nice column :-)
<niemeyer> rogpeppe: Very nice indeed
<rogpeppe> niemeyer: cool, thanks
<niemeyer> rogpeppe: Some trivial suggestions only
<rogpeppe> niemeyer: am just currently doing the awkward merge between that branch and the one submitted on friday
<niemeyer> TheMue: Ah, I've already reviewed most of that branch on Friday, btw
<rogpeppe> niemeyer: i'm a bit surprised that goyaml conventions for naming are different from json and xml
<niemeyer> TheMue: I've purposefully not submitted the review because I wanted to look through it again after the weekend
<niemeyer> TheMue: You've (silently?) refactored significantly the way Python stored information about the endpoints, and I want to see that again with a fresh mind today
<rogpeppe> niemeyer: that case changing stuff was deliberately removed from them to make the mapping clearer, i think.
<rogpeppe> niemeyer: i'll remove the tags anyway though
<TheMue> niemeyer: Yes, that's a result of the discussion between William and me during UDS.
<niemeyer> TheMue: Ok, this is a major change, though
<niemeyer> TheMue: It shouldn't be submitted silently, without even a note in the description
<niemeyer> TheMue: I'm still thinking through it. At this point just saying I'd appreciate a note next time.
<TheMue> niemeyer: OK, will handle it more carefully next time.
<niemeyer> TheMue: Thanks
<niemeyer> rogpeppe: xml had stuff that was a *lot* more magical
<niemeyer> rogpeppe: json too, IIRC
<rogpeppe> niemeyer: still, i do like the "no magic at all" approach
<rogpeppe> niemeyer: it makes the code more transparent.
<rogpeppe> IMHO
<niemeyer> rogpeppe: There's no magic in goyaml.. it's the lowercased name.
<niemeyer> rogpeppe: If it doesn't match, it doesn't
<rogpeppe> niemeyer: does that mean you can't marshal to an Uppercased name ?
<niemeyer> rogpeppe: That no
<niemeyer> rogpeppe: Erm
<niemeyer> rogpeppe: No
<rogpeppe> niemeyer: so it applies in one direction but not the other?
<niemeyer> rogpeppe: If you want that, you need to use a tag
<niemeyer> rogpeppe: It makes sense (to me, anyway) because most yaml is lowercase
<rogpeppe> niemeyer: i'll remove the tags, but let me lodge an objection for the future. it's nice if marshalling packages are consistent like this.
<niemeyer> rogpeppe: Same thing with mgo
<niemeyer> rogpeppe: MongoDB is massively lowercase
<niemeyer> rogpeppe: So goyaml and bson are consistent ;)
<rogpeppe> niemeyer: i think having a non-bijective transformation is asking for trouble
<niemeyer> rogpeppe: Not sure about what you mean. I've just said it's bijective.
<niemeyer> rogpeppe: The marshaling name is the lowercased field name.
<rogpeppe> niemeyer: ah. so i *can't* marshal to an upper-cased name?
<niemeyer> rogpeppe: You have to use a tag name.
<niemeyer> Erm.. s/name//
<rogpeppe> niemeyer: i see.
<rogpeppe> niemeyer: sorry, i misunderstood
<rogpeppe> niemeyer: that's not too bad, i guess.
<niemeyer> rogpeppe: Yeah, that's how I feel too.
<niemeyer> Anyway, lunch time
<niemeyer> Back in a bit
<niemeyer> TheMue: Will cover your branch first thing in the afternoon
<TheMue> niemeyer: Thx a lot
<rogpeppe> niemeyer: not sure about "LoadConfig" as a name. how about NewConfig ?
<niemeyer> rogpeppe: +1
<rogpeppe> niemeyer: cool, done already :-)
<rogpeppe> niemeyer: you might want to have a brief once-over of environs/interface.go before i submit, as i didn't quite use the comment text you suggested.
<rogpeppe> lunch
<niemeyer> rogpeppe: Looking again.
<niemeyer> rogpeppe: Replied
<rogpeppe> niemeyer: thanks. actually i think it can be simpler still: // Every provider must accept the "name" and "type" attributes, the environment name and provider type.
<niemeyer> rogpeppe: That's misleading. It's not A and B, C and D.
<niemeyer> rogpeppe: If you're happy with the suggestion, can we please go with it?
<rogpeppe> niemeyer: i'm not keen on the "with" in your suggestion. am trying to think of a better phrasing.
<niemeyer> rogpeppe: Alright, please use whatever you fancy then
<niemeyer> rogpeppe: If "with" is unacceptable I can't really improve on it
<rogpeppe> niemeyer: it reads oddly, sorry, i can't quite say why.
<rogpeppe> niemeyer: how about "Every provider must accept the name of the environment ("name") and the environment type ("type")."
<niemeyer> rogpeppe: Sucks
<rogpeppe> hrm
<rogpeppe> niemeyer: what about s/with/holding/
<rogpeppe> ?
<niemeyer> rogpeppe: "provide the foo argument with the key" is extremely usual
<niemeyer> rogpeppe: holding is equally usual
<niemeyer> rogpeppe: (which means, I'm fine with it too)
<rogpeppe> niemeyer: cool, i'll go with that, and ponder why my "awkward-english" alarm went off for "with" :-)
<niemeyer> rogpeppe: Please let me know if you find out
<niemeyer> rogpeppe: and thanks
<TheMue> So, off for the moment. Next branch is in.
<rogpeppe> niemeyer: something to do with the fact there are two keys, i think.
<rogpeppe> niemeyer: but can't pin it down more than that...
<niemeyer> rogpeppe: Doesn't parse as related in my head
<niemeyer> or, as Kevin would say, in my "heeeed"
<rogpeppe> :-)
<rogpeppe> niemeyer: in the end, i'm not really sure how "holding" is better than "giving", but i'm going with it anyway.
<niemeyer> rogpeppe: It doesn't *give* anything to anyone.. it contains, and is.
<rogpeppe> niemeyer: one use of the word "give" is "to impart or communicate"
<rogpeppe> niemeyer: which is how i was using it.
<niemeyer> rogpeppe: Try to put "communicates" or "imparts" there..
<niemeyer> rogpeppe: It's bad too.
<rogpeppe> / Every provider must accept the "name" and "type" attributes, communicating the environment name and provider type respectively.
<rogpeppe> seems to work ok to me.
<rogpeppe> apart from the length of "communicate" :-)
<niemeyer> rogpeppe: Still sounds bad to me. But we don't have to agree on that.
<rogpeppe> niemeyer: it's fine. i'll try to avoid it in the future...
<niemeyer> rogpeppe: Thakns
<rogpeppe> niemeyer: submitted. thanks a lot for the review.
<niemeyer> rogpeppe: np
<niemeyer> robbiew: Do you know if we'll have that meeting in 7 mins?
<robbiew> no clue...I can't make it due to a conflict though
<robbiew> niemeyer: it looks like a lot of folks declined, so perhaps not
<niemeyer> robbiew: There are 20 people in that call.. it seems likely that lots of folks will decline :)
<robbiew> lol
<robbiew> good point
<rogpeppe> niemeyer: micro branch for review: https://codereview.appspot.com/6219054
<niemeyer> rogpeppe: Neat, LGTM
<rogpeppe> niemeyer: thanks
<rogpeppe> niemeyer: yay, fastest turnaround ever!
<niemeyer> rogpeppe: Small branches FTW! :-)
<rogpeppe> niemeyer: a larger branch for review: https://codereview.appspot.com/6198064
<niemeyer> rogpeppe: Cheers
<rogpeppe> niemeyer: though it's not nearly as large as it appears.
<rogpeppe> niemeyer: i'm off for the day. see you tomorrow!
<niemeyer> rogpeppe: Have a good one
<niemeyer> Stepping out for a while
<andrewsmedina> gt
#juju-dev 2012-05-22
<niemeyer> andrewsmedina: Heya
<niemeyer> andrewsmedina: Just delivered a review on your branch
<niemeyer> andrewsmedina: Seems pretty close..
<niemeyer> andrewsmedina: just trivial stuff
<niemeyer> Achievement unlocked!
<niemeyer> Nothing to review on the Go front!
<niemeyer> Phew
<andrewsmedina> niemeyer: thanks for review! I will work to resolve the things that you said
<andrewsmedina> anybody here?
 * davecheney waves
<davecheney> andrewsmedina: are you doing a full port for the local provider to juju/go ?
<davecheney> that would be awesome
<rogpeppe> davecheney: hiya
<davecheney> aoy
<rogpeppe> davecheney: i'm a big fan of sharing-memory-by-communicating in general, but sometimes a mutex gets the job done more cleanly IMHO. http://paste.ubuntu.com/1000362/
<rogpeppe> davecheney: (which also has the advantage that we don't serialise all requests)
<davecheney> rogpeppe: yup, that will work also
<davecheney> there is no reason why we couldn't also do
<davecheney> case cmd := <- p.cmd ; go cmd(p.env)
<davecheney> i'm not sure gustavo is sold on the idea of a proxy yet
<davecheney> but I do admit your idea is simpler
<rogpeppe> davecheney: invoking go doesn't quite have the same semantics (something may be operating on the environment while it changes) but perhaps that doesn't matter.
<davecheney> rogpeppe: it's fine to operate on a stale copy
<davecheney> because that is inevitable
<rogpeppe> davecheney: if i'm passing a function into a goroutine just to get mutual exclusion, then i start to think i'm doing something a bit wrong.
<davecheney> rogpeppe: i was following the way the topology worked
 * rogpeppe obviously hasn't looked too closely at that code :-)
<rogpeppe> davecheney: if you don't care whether things can run on the same environment when it changes, then things can be even simpler: http://paste.ubuntu.com/1000370/
<davecheney> nice
<davecheney> i'll take that, thank you
<rogpeppe> davecheney: cool, glad to be of help.
<davecheney> i had some reason why it needed to be the original way
<davecheney> but I can't remember them now
<davecheney> so until I do
<davecheney> lets go for simplicity
<rogpeppe> davecheney: one thing: do we really need to block in NewProxyEnviron? can't we just let the first operation block until there's a valid environ?
<davecheney> nope, no need to block
<rogpeppe> davecheney: that would simplify things even more: http://paste.ubuntu.com/1000379/
<davecheney> that was a personal chioce
<rogpeppe> davecheney: down from 167 lines to 86 :-)
<davecheney> excellent
<davecheney> i won't mention that it won't compile anymore
<rogpeppe> davecheney: minor details
<rogpeppe> :-)
<davecheney> ahh, that is why we block til a valid environ
<davecheney> so we don't need to check for nil
<rogpeppe> davecheney: ah, that's easy to fix too.
<davecheney> when NewProxyEnviron returns, there is something that proports to be an envion there
<davecheney> so no worrying about NPE
<rogpeppe> davecheney: something like this, perhaps? http://paste.ubuntu.com/1000381/
<davecheney> no thanks
<davecheney> locked isn't protected by a mutex, and the locks and unlocks are unbalanced
<rogpeppe> davecheney: locked doesn't need to be protected by a mutex (it's local). we ensure that the lock is held on entry to loop. but yeah, it's not so pretty.
<rogpeppe> davecheney: i had the locked logic wrong BTW: http://paste.ubuntu.com/1000384/
<rogpeppe> davecheney: a slightly less unsavoury option: http://paste.ubuntu.com/1000388/
<rogpeppe> davecheney: but maybe it's best just to do the wait explicitly in NewProxyEnviron as before.
<davecheney> lets wait for the request that it doesn't block during startup
<davecheney> i don't see it as a issue, as nothing can act on the Environ til it's valid anyway
<davecheney> thanks rogpeppe that made the code a lot simpler
<davecheney> tests still don't pass, but whatever
<davecheney> the main goal is to convince gustavo why a proxy is a good way of managing the unstable environment
<rogpeppe> davecheney: i think that the simpler it is the more likely we are of that
<rogpeppe> davecheney: BTW i'm not convinced this deserves its own package. it's quite specific to the provisioning agent, isn't it?
<davecheney> i hope that it makes it easier to test
<davecheney> ie, we test we can do all the dummy tests against this Environ
<davecheney> rather than trying to test it via the provisioning agent
<rogpeppe> davecheney: there's nothing stopping us having internal tests for the provisioning agent code
<davecheney> but your point is well made that in theory the provisinng agent is the only one talking to the Environ
<rogpeppe> davecheney: i've done that quite a bit in the ec2 provider
<rogpeppe> davecheney: also, as a general mechanism it's not quite there
<rogpeppe> davecheney: because if a client keeps a Storage around, that won't work well - you'd have to proxy that too.
<davecheney> rogpeppe: yup - i saw that, maybe that can be solved via convention
<davecheney> on error, get a new storage
<rogpeppe> davecheney: yeah, but will the provisioning agent actually be writing to storage ever?
<rogpeppe> davecheney: i.e. do we really care?
<davecheney> rogpeppe: that, i do not know
<rogpeppe> davecheney: we could have a Storage method that just paniced.
<rogpeppe> davecheney: (or returned a Storage that always returned an error, i suppose)
<rogpeppe> davecheney: oh, i'm talking bollocks, of course the provisioning agent needs to talk to the storage.
<davecheney> ok, i'll make a proxy storage
<rogpeppe> davecheney: yeah. it could always call the relevant storage method in proxy before invoking its method.
<rogpeppe> davecheney: it's slightly easier if you assume that the provisioning agent will never need to *write* to the storage (which i think is the case) http://paste.ubuntu.com/1000412/
<rogpeppe> davecheney: and i'm thinking that it's probably best to have NewProxyEnviron block until the environment is valid, and let the caller deal with that.
<rogpeppe> davecheney: rather than putting an error test in every method.
<rogpeppe> davecheney: anyway, i should get back to my *own* damn code!
<davecheney> rogpeppe: thanks for your help
<davecheney> out
<Aram> morning.
<rogpeppe> Aram: hiya
<rogpeppe> gotta love ec2 request reliability
<TheMue> rogpeppe: Why?
<TheMue> rogpeppe: Heya btw.
<rogpeppe> TheMue: (hiya too) i've run the environs/ec2 amazon tests four times in a row and they've failed each time for different reasons.
<rogpeppe> TheMue: we really need some retry logic at the goamz/ec2 level - that will help a lot.
<rogpeppe> TheMue: it's great when a test fails 3/4s of the way through, reading a file that it already read successfully, saying "file not found"
<TheMue> rogpeppe: Yes, transparent for the user. Only real hard errors we know should raise.
<rogpeppe> TheMue: yeah. you can't deal with the eventual consistency stuff automatically sadly. but the "premature EOF" and "handshake failure" errors are entirely possible.
<TheMue> rogpeppe: Is EC2 in general so unreliable? Or is it only a special part of their API?
<rogpeppe> TheMue: i have a feeling it's all that unreliable
<rogpeppe> TheMue: but i only have the one data point
<TheMue> rogpeppe: Uuh, not good.
<rogpeppe> TheMue: it applies to S3 too
<rogpeppe> TheMue: yay, put in a couple of retry loops and it all works now
<TheMue> rogpeppe: Are there any hints by Amazon on an optimal retry strategy (how often, which delay between requests, â¦)?
<rogpeppe> TheMue: no. they don't even talk about eventual consistency issues. it all looks hunky-dory in their docs.
<TheMue> rogpeppe: I've got to admit that I expected this answer. I only had a little hope left.
<rogpeppe> TheMue: building on shifting sands...
<TheMue> rogpeppe: So take a house boat, able to move but tied in place.
<rogpeppe> TheMue: right, time for some lunch
<TheMue> rogpeppe: Enjoy, I just had.
<niemeyer> Hullah!
<TheMue> niemeyer: Moin. ;)
<niemeyer> TheMue: Heya
<TheMue> niemeyer: I'm just thinking about the "endpointInfo" in topology.go. It's indeed no beatiful solution and tries to wrap what "get_relation_service" and "get_relations_for_service" return in the Python version.
<TheMue> niemeyer: Its really very similar to "RelationEndpoint2, only that there are names while "endpointInfo" deals with keys.
<niemeyer> TheMue: Right
<TheMue> niemeyer: I could add the keys to "RelationEndpoint" as private fields, that would safe us one type
<niemeyer> TheMue: Why do we need the keys there?
<niemeyer> TheMue: See the Python code
<TheMue> niemeyer: Have to look, one moment. It's taken from the Python code.
<niemeyer> TheMue: I don't think it is
<niemeyer> TheMue: This branch diverges wildly from Python all around
<TheMue> niemeyer: The information that the two functions using the "endpointInfo" returns is the same like the two Python functions named above.
<TheMue> niemeyer: But there dynamically lists and dictionaries are taken.
<niemeyer> TheMue: That's not what we were talking about..
<niemeyer> <TheMue> niemeyer: I could add the keys to "RelationEndpoint" as private fields, that would safe us one type
<niemeyer> <niemeyer> TheMue: Why do we need the keys there?
<niemeyer> <niemeyer> TheMue: See the Python code
<rogpeppe> gorgeous weather out there for a change.
<rogpeppe> niemeyer: mornin', BTW
<TheMue> niemeyer: "get_relations_for_service" stores at least the "relation_id", I don't yet know where it is used. Regarding the "ServiceKey" William and I discussed that it's better than the name, which can be retrieved via "ServiceName(key)".
<rogpeppe> niemeyer: thanks a lot for the review last night. you might've seen two tiny branches proposed this morning: https://codereview.appspot.com/6220065/ and https://codereview.appspot.com/6221058/
<niemeyer> rogpeppe: Morning
<niemeyer> TheMue: Sorry, I don't understand what you mean.. the service key is a service key, the service name is a service name. Neither is better than the other without context.
<niemeyer> rogpeppe: No, haven't run through reviews yet
<rogpeppe> niemeyer: that's fine. they're both independent anyway,
<niemeyer> rogpeppe: That's great, thanks
<TheMue> niemeyer: If I have one of it (service name or key) I can retrieve the other (only that from key to name is a bit faster). For relations I need the key to retrieve the name.
<niemeyer> TheMue: Yes. How does that change the picture in the scenario we were talking about earlier?
<niemeyer> TheMue: You can use RelationEndpoint to work with relations in the same way that Python deos.
<niemeyer> does
<niemeyer> TheMue: You don't need to store any private keys in RelationEndpoint.
<TheMue> niemeyer: That's only a first observation for me to get a better understanding
<niemeyer> TheMue: RelationEndpoint needs a RelationName
<niemeyer> TheMue: All of that has to be fixed
<niemeyer> TheMue: And all of it are divergences from what Python does
<TheMue> niemeyer: It has now a name.
<niemeyer> TheMue: Simplifying is cool, but it needs a lot of attention to what is being done if we are to diverge from what is being done in the Python code. I get quite nervous when I see the implementation diverging wildly both in terms of implementation and in terms of *semantics*.
<TheMue> niemeyer: What I'm looking for is how to optimally return informations that get_relations_for_service and get_relation_service return.
<niemeyer> TheMue: Ok, let me look at that
<TheMue> niemeyer: There they are e.g. dynamic lists with the interface as first field and service topology as second field.
<niemeyer> TheMue: What is "service topology"?
<TheMue> niemeyer: For the rest I already adopted your suggestions.
<niemeyer> TheMue: I suppose get_relations_for_service should return []relation?
<TheMue> niemeyer: Eh, wrong wording, see return (relation_data["interface"], relation_data["services"][service_id])
<TheMue> niemeyer: Good idea, would only have to add the key here (currently it doesn't has).
<niemeyer> TheMue: Yeah, that sounds fine.. RelationKey
<niemeyer> Well, not really
<niemeyer> TheMue: relation{ Key: ... }
<TheMue> niemeyer: It would be persisted.
<niemeyer> TheMue: Not necessarily.. omitempty
<TheMue> niemeyer: Ah, yes, I remember.
<niemeyer> TheMue:                     service=services[service_id]))
<niemeyer> TheMue: What is that data in the service key?
<niemeyer> {"role": r, "name": n}, I suppose..?
<TheMue> niemeyer: The function only looks for one service. So our Services map would contain the role and the key.
<niemeyer> TheMue: Why does it do that? I'm not sure we should mimic it
<TheMue> niemeyer: Let me look for a consumer of this function, moment.
<niemeyer> TheMue: If we get a relation back, it should be the actual relation, with its actual.. it's not clear to me why we should be stripping it off arbitrarily
<niemeyer> s/its actual/its actual data/
<niemeyer> TheMue: I was looking at that too.. still not clear.. feels like a poor interface
<TheMue> niemeyer: So far I don't see any reason too.
<niemeyer> TheMue: Ok.. let's just return a proper relation with its real data then
<niemeyer> TheMue: If we find out why we can talk again
<niemeyer> TheMue: But I suspect this should do quite nicely
<TheMue> niemeyer: Think so too.
<TheMue> niemeyer: At least it "feels" better than my current solution.
<niemeyer> rogpeppe: You got a major LGTM, and a major OMG WHAT ARE YOU DOING?
<rogpeppe> niemeyer: interesting
<rogpeppe> niemeyer: the way to fix that OMG is to implement signed URLs in s3, which i haven't got around to yet.
<rogpeppe> niemeyer: if i put in a TODO, would that be ok?
<niemeyer> rogpeppe: No.. we're not making anything public and swiping that under the carpet
<niemeyer> s/anything/everything/
<niemeyer> rogpeppe: Been there, doesn't end up well
<rogpeppe> niemeyer: ok. i'll do the s3 thing first then.
<rogpeppe> niemeyer: only problem is, i looked for how to do the signed URL thing before and didn't come up with anything
<niemeyer> rogpeppe: I can do that if necessary
<niemeyer> rogpeppe: Actually, let me do that
<niemeyer> rogpeppe: Are you happy with the former s3 branch?
<rogpeppe> niemeyer: just a pointer to some docs would be fine. or, if you wanna do that, that would be lovely too!
<rogpeppe> niemeyer: oh shit, forgot about that. one mo.
<niemeyer> rogpeppe: Cheers
<rogpeppe> niemeyer: you've got a review
<niemeyer> rogpeppe: "could we give a nicer error here when we panic?"
<niemeyer> rogpeppe: When would it panic?
<rogpeppe> niemeyer: if the host in the headers didn't have two dots in it?
<niemeyer> rogpeppe: This code can't execute if that's the case
<rogpeppe> niemeyer: ah yes.
<niemeyer> rogpeppe: Will fix the rest
<niemeyer> rogpeppe: Thanks for the review
<rogpeppe> niemeyer: doh, sorry for that
<niemeyer> rogpeppe: np
<niemeyer> rogpeppe: I'll add the "" on a line on its own.. it's not using field names purposefully in this case
<rogpeppe> niemeyer: that's fine
<niemeyer> rogpeppe: It should fail if we add a field and forget to update one of the structs
<rogpeppe> niemeyer: sure.
<niemeyer> rogpeppe: ${bucket} would probably be a more sensible replacement var
<rogpeppe> niemeyer: yeah, that would be fine. although if there's ambiguity between $name and the rest, there's gonna be ambiguity between the bucket name and the rest...
<niemeyer> rogpeppe: Hm?
<rogpeppe> niemeyer: isn't that why you're preferring ${bucket} ?
<rogpeppe> niemeyer: because it's unambiguously replaceable?
<rogpeppe> i suppose the host name might have a $ in.
<rogpeppe> if that's allowed
<niemeyer> rogpeppe: $namesis shouldn't get replaced
<rogpeppe> niemeyer: yeah, true. it's a URL really and URLs can contain $s
<rogpeppe> niemeyer: ${bucket} is just fine.
<rogpeppe> niemeyer: lots better than %s :-)
<niemeyer> rogpeppe: Indeed
<niemeyer> "@ylastic #AWS #EC2 API calls returning intermittent errors "Unavailable: Unable to process request, please retry shortly""
<niemeyer> Things are getting more and more "eventual"
<niemeyer> rogpeppe: Branch re-proposed
<rogpeppe> niemeyer: it's wonderful. i tried the amazon test 4 times in a row this morning and it failed each time with a different error. we need to get that retry logic in!
<rogpeppe> niemeyer: LGTM
<niemeyer> rogpeppe: Thanks
<andrewsmedina> rogpeppe, niemeyer morning :D
<rogpeppe> andrewsmedina: hiya!
<andrewsmedina> rogpeppe, niemeyer thanks for review
<rogpeppe> andrewsmedina: np. thanks for bearing with us!
<andrewsmedina> rogpeppe: when you have a time, I wanted to talk with you about the reivew
<rogpeppe> andrewsmedina: i've got a meeting in 5 minutes, but in an hour or so, no problem.
<andrewsmedina> rogpeppe: ok
<rogpeppe> andrewsmedina: ping
<andrewsmedina> rogpeppe: pong
<andrewsmedina> rogpeppe: about the %d
<rogpeppe> andrewsmedina: about the %d
<andrewsmedina> rogpeppe: virsh use the %d to create a autoincrement number for the net/bridge name
<andrewsmedina> rogpeppe: the juju python version uses it
<rogpeppe> andrewsmedina: is that needed, given that the name needs to unique anyway?
<rogpeppe> s/to /to be /
<rogpeppe> andrewsmedina: fair enough. *is* it actually documented anywhere, or is it just a bit of black magic known to those who've read the source?
<andrewsmedina> rogpeppe: http://bazaar.launchpad.net/~juju/juju/trunk/view/head:/juju/providers/local/network.py#L13
<andrewsmedina> rogpeppe: I dont know if this is a oficial feature =/
<rogpeppe> andrewsmedina: ok. i see that, thanks. but i'd like to know why we're doing that. or at any rate write a comment saying what it's there for.
<rogpeppe> niemeyer, hazmat: do you know, by any chance, why we need an autoincrement number in the lxc network bridge name?
<andrewsmedina> rogpeppe: I will do a comment about it
<rogpeppe> andrewsmedina: thanks a lot
<andrewsmedina> rogpeppe: about the split
<hazmat> rogpeppe, we do? we just use the default libvirt name
<hazmat> s/name/network
<andrewsmedina> rogpeppe: virsh always returns the same string pattern
<rogpeppe> hazmat: see link above
<andrewsmedina> I need to check anyway?
<andrewsmedina> hazmat: http://bazaar.launchpad.net/~juju/juju/trunk/view/head:/juju/providers/local/network.py#L13
<rogpeppe> andrewsmedina: yes, but virsh isn't part of the juju program. if someone changes the virsh output, we want a "unexpected virsh output" error or similar, rather than an array bounds panic.
<hazmat> andrewsmedina, we don't specify that value
<hazmat> andrewsmedina, its escaped because libvirt uses it
<hazmat> %%d
<rogpeppe> hazmat: why do we need it?
<hazmat> rogpeppe, we don't.. libvirt does
<rogpeppe> hazmat: isn't the bridge name unique enough, given that it includes the network name?
<hazmat> i'd suggest not being slavish about copying that aspect.. i've got a separate branch local-cloud-img that gets rid of libvirt entirely
<andrewsmedina> launch time :(
<andrewsmedina> sorry
<hazmat> rogpeppe, its not
<rogpeppe> hazmat: ok.
<rogpeppe> hazmat: when you say "getting rid of libvirt", you mean "not using virsh" ?
<rogpeppe> hazmat: i haven't looked too deeply into how all the lxc stuff is set up. i should!
<hazmat> rogpeppe, no i mean getting rid of libvirt
<hazmat> rogpeppe, the name comes out virbr
<hazmat> again this has nothing to do with juju
<hazmat> this is libvirt's configuration
<hazmat> and it needs that string insertion for its templates %d
<rogpeppe> hazmat: i didn't see any mention of the %d in the libvirt docs. but i'm probably looking in the wrong place.
<rogpeppe> hazmat: (i looked in
<rogpeppe> http://libvirt.org/formatnetwork.html and
<rogpeppe> http://libvirt.org/sources/virshcmdref/html/sect-net-define.html
<rogpeppe> )
<rogpeppe> hazmat: so your branch eschews the portability layer and talks to lxc directly?
<hazmat> rogpeppe, what portability layer?
<hazmat> rogpeppe, i think your misunderstanding something fundamental here
<rogpeppe> hazmat: libvirt looks like a portability layer to me, but i may be misinterpreting
<rogpeppe> hazmat: almost certainly!
<hazmat> rogpeppe, we don't use libvirt outside of setting up the network
<hazmat> rogpeppe, its a rather broken portability layer when it comes to lxc
<hazmat> rogpeppe, it has many additional functionalities that have no meaning on the context of lxc
<rogpeppe> hazmat: sure. but that is the role it's trying to play, no? or i'm i fundamentally misunderstanding?
<niemeyer> It's lunch time
<hazmat> rogpeppe, we're just piggybacking on its packinging of a default network
<hazmat> in the lxc in precise, this is also in the lxc package
<hazmat> moreover local provider doesn't use cloud-init or cloud images at the moment, the branch i mentioned local-cloud-img, rips out libvirt, uses cloud images for local provider, and uses cloud-init to configure
<hazmat> rogpeppe, yes libvirt is an abstraction layer over different virt providers
<rogpeppe> hazmat: ok, cool. i'm not too far off then!
<hazmat> rogpeppe, but its not particularly good for several of them
<rogpeppe> hazmat: that's in the nature of portability layers i guess. lowest common denominator.
<hazmat> rogpeppe, for example openstack uses libvirt to address several different tech, but there are also standalone providers for some of the same ones that libvirt provides an abstraction for
<hazmat> within openstack
<hazmat> rogpeppe, its actually quite rich as a common api, but in the context of lxc specifically its typically out-of-date, and overkill
<hazmat> rogpeppe, we had a discussion about trying to use libvirt directly a while ago instead of lxc directly
<rogpeppe> hazmat: given that we're directly reliant on lxc in other aspects, i'd be inclined to agree, although i haven't looked at how hard it is to configure networks in lxc directly.
<rogpeppe> hazmat: that's the other possibility of course
<hazmat> rogpeppe, its pretty trivial, just needs the packaging effort around creating the bridge, etc
<hazmat> trivial to configure that is
 * hazmat preps for his next meeting
<rogpeppe> andrewsmedina: anyway, with a comment, i'm happy with that code.
<rogpeppe> TheMue: i see sporadic test failures testing state: http://paste.ubuntu.com/1001083/
<TheMue> TheMue: I've done no changes at watcher for a longer time, but I've just merged the trunk into my branch and get a watcher failure too.
<rogpeppe> TheMue: i don't *think* it's my fault :-)
 * niemeyer waves
<niemeyer> TheMue: Perhaps a race?
<rogpeppe> niemeyer, TheMue: i suspect a race, or an inadequate timeout.
<rogpeppe> niemeyer: BTW if i take out that PublicRead change, does that "make amazon tests work" branch LGTY?
<niemeyer> rogpeppe: I've stopped checking it out there
<TheMue> So, back, had to do an emergency repair here (jalousie broke down).
<rogpeppe> niemeyer: ok. 'cos i need that branch in an upcoming branch that already has a dependency...
<niemeyer> rogpeppe: Ok, what's the plan?
<rogpeppe> niemeyer: i've pushed that branch without that change. the amazon tests no longer pass of course, so i'm changing it temporarily locally so i can test. but the other changes are less trivial and also necessary.
<rogpeppe> niemeyer: so when s3 implements signed URLs we'll use them and the amazon tests will pass
<niemeyer> rogpeppe: Ok, I mean what's the plan as far as the branch goes
<rogpeppe> niemeyer: the "fix amazon tests" branch?
<niemeyer> rogpeppe: Whatever branch you mean.. yous said you need that branch.. what's your plan
<TheMue> rogpeppe: I few moments ago I had your watcher error too, but now everything runs. Will investigate it later.
<rogpeppe> niemeyer: i'll hold off proposing again until the "fix amazon tests" branch is in.
<niemeyer> rogpeppe: Ok.. so I'm' not sure about what you meant
<niemeyer> <rogpeppe> niemeyer: BTW if i take out that PublicRead change, does that "make amazon tests work" branch LGTY?
<niemeyer> <rogpeppe> niemeyer: ok. 'cos i need that branch in an upcoming branch that already has a dependency...
<rogpeppe> niemeyer: it would be nice to have a review of the "fix tests" branch so i can propose that upcoming branch
<niemeyer> rogpeppe: You already have a review.. there's a change that can't go in
<rogpeppe> niemeyer: it no longer has that PublicRead change
<rogpeppe> niemeyer: i've re-proposed it
<niemeyer> rogpeppe: Ok.. so you've reproposed and you want another review?
<rogpeppe> niemeyer: yes please
<rogpeppe> niemeyer: you should have got a mail saying "please take a look" :-)
<niemeyer> rogpeppe: Heh
<niemeyer> rogpeppe: That's a pretty difficult way to ask for a review
<niemeyer> rogpeppe: Will have a look
<rogpeppe> niemeyer: isn't that implied? if not, what's the point of the mail?
<niemeyer> rogpeppe: You see the history above?
<niemeyer> rogpeppe: "Please have a look at http://... again" is fine
<rogpeppe> niemeyer: fair enough
<rogpeppe> niemeyer: (but i thought that was the point of the "Please take a look" email!)
<niemeyer> rogpeppe: Nevermind!
<rogpeppe> yay! go tools now working on the server again...
<rogpeppe> niemeyer: BTW while (if?) you're on little reviews, this is trivial, assuming you agree with the premise.  https://codereview.appspot.com/6188068/
<niemeyer> rogpeppe: What's the point of these loops?
<niemeyer> for i := 0; i < 5; i++ {
<niemeyer> ?
<andrewsmedina> niemeyer: ping
<niemeyer> andrewsmedina: hi
<andrewsmedina> niemeyer: thanks for your review
<niemeyer> andrewsmedina: np
<andrewsmedina> niemeyer: I think it is okay
<andrewsmedina> now
<rogpeppe> niemeyer: same as for other eventual consistency issues
<rogpeppe> niemeyer: i copied the retry code from elsewhere. we could use a different retry time or count
<niemeyer> rogpeppe: Ok, can you please reduce the delay on each retry then, to maybe 1e8, increase the limit to 30 or so, and add a comment explaining?
<rogpeppe> niemeyer: ok, will do, and in the other place too.
<niemeyer> rogpeppe: No need to be too extensive in the comment, just a reminder
<andrewsmedina> niemeyer: I will work on local provider interface now: p
<niemeyer> rogpeppe: Thanks
<niemeyer> rogpeppe: "LGTM assuming reduced timeouts and increased N as discussed online."
<rogpeppe> niemeyer: thanks a lot
<niemeyer> rogpeppe: No problem.. that was easy :)
<niemeyer> rogpeppe: Hopefully we can get a fix to the signature issue in very soon
<niemeyer> rogpeppe: I'll try to address that today still
<rogpeppe> *splash*
<rogpeppe> (the sound of a branch landing)
<andrewsmedina> niemeyer: It needed something more to my proposal be accepted?
<rogpeppe> right, gotta go. have fun all. see you tomorrow.
<niemeyer> rogpeppe: See ya
<niemeyer> andrewsmedina: I haven't re-reviewed your branch yet
<niemeyer> andrewsmedina: have you addressed the latest comments by rog/
<niemeyer> ?
<twobottux> aujuju: What is the best way to use the mysql charm in Juju with dynamic database credientials? <http://askubuntu.com/questions/140818/what-is-the-best-way-to-use-the-mysql-charm-in-juju-with-dynamic-database-credie>
<andrewsmedina> niemeyer: yes, about %d rog said it is ok
<andrewsmedina> niemeyer: about split, virsh always returns the same pattern
<niemeyer> andrewsmedina: Cool, need to step out for a doc appointment.. back soon, though
<andrewsmedina> niemeyer: I'm using the same strategy used in juju python code
<andrewsmedina> niemeyer: ok
<SpamapS> Multiple IPs!!
 * SpamapS does a dance
<SpamapS> niemeyer: great news!
<SpamapS> the multiple IPs, not the doc appointment ;)
<andrewsmedina> SpamapS: :-p
<niemeyer> SpamapS: Indeed!!
<niemeyer> Ouch.. that's the sleepiest time of the day
<niemeyer> Alright, not working.. I'll do it Windows style and take a short nap to reboot
<Aram> sleep is when the garbage collector runs.
<niemeyer> davecheney: Morning!
<niemeyer> Aram: Indeed
<davecheney> mornign!
<davecheney> niemeyer: lets talk about the proxy environ
<niemeyer> davecheney: Yeah, was planning on that too
<niemeyer> davecheney: Good we're overlapping today
<davecheney> how can I convince you of it's utility
<niemeyer> davecheney: Hehe :-)
<davecheney> niemeyer: yes, sorry about that, i had to go intot he city to finish some of the employment paper work that was dropped when veronika moved on
<niemeyer> davecheney: Ah, no worries at all
<niemeyer> davecheney: The concept of having such a wrapper feels pretty icky to me
<davecheney> niemeyer: ok, maybe the name is a problem
<niemeyer> davecheney: It's a bit like replacing the car every time the battery goes bad
<niemeyer> davecheney: Not really
<niemeyer> davecheney: It's the concept itself
<davecheney> ok, let me put it like this
<davecheney> we dont' wrap one Environ in another
<davecheney> the proxy wraps a *ConfigNode in an Environ
<niemeyer> davecheney: We wrap it so that it is magically replaced every time its configuration changes
<davecheney> yes
<niemeyer> davecheney: That foundational idea is that seems misleading
<davecheney> ok
<niemeyer> davecheney: If the configuration of an environment changes, we should replace its configuration at an appropriate well known time, not the entire thing behind the scenes at an arbitrary moment
<davecheney> niemeyer: hmm, this is a bit of a change of heart frmo UDS
<davecheney> where we talked about making the environment a *ConfigNide
<davecheney> Node
<niemeyer> davecheney: We're talking about different things
<davecheney> then the provisioning agent would start to watch that node, until it got a valid configuration
<davecheney> ok
<niemeyer> davecheney: I'm talking about the thing that offers an environs.Environ interface, not the content of the /environment node
<davecheney> ok
<niemeyer> davecheney: Whenever the configuration of an environment changes, there's a window of time that the value in memory we have that represents the environment (implements environs.Environ) will have a wrong configuration.
<davecheney> yup
<davecheney> i see that as inevitable
<niemeyer> davecheney: This is a problem inherent to the distributed context that we can't solve..
<niemeyer> davecheney: Understanding that gives us some freedom too, interestingly. It means we can *choose* the best moment to make the configuration active, since we have to deal with it being wrong no matter what.
<davecheney> i don't think that matters
<davecheney> the config changing is not the thing that broke the environment
<davecheney> take the case of AWS keys chanign
<niemeyer> davecheney: It matters, in the sense that we don't have to rush to replace the configuration of an environment.
<davecheney> ok, so i see your concern as proxy.loop() replacing the configuration at wil
<niemeyer> davecheney: Not the configuration, it is replacing *the environment*
<niemeyer> davecheney: That environs.Environ, is not just a configuration
<davecheney> how are those not the same thing
<niemeyer> davecheney: It's the environment state we have in memory.
<davecheney> the environ.Environ is formed from materialising the configuration stored in zk
<niemeyer> davecheney: Is httpd and httpd.conf the same thing?
<davecheney> that is not a good example
<niemeyer> :-)
<davecheney> but I take your point
<davecheney> to offer a counter example, port 80 and httpd.conf are the same thing
<niemeyer> davecheney: Wait, waht?
<niemeyer> davecheney: *port 80* and *httpd.conf* are the same thing!?
<davecheney> as a counter example
<davecheney> not being too literal
<niemeyer> davecheney: I can't process or even argue about that.. it seems so wrong in so many levels
<davecheney> sure, lets get back to the point
<davecheney> working from the point of view of the PA
<davecheney> it needs to see changes to the topology relating to machines
<niemeyer> davecheney: RIght
<davecheney> and try to make reality match that
<davecheney> so, it needs an Environ
<davecheney> this we know
<niemeyer> davecheney: Right, I'm with you
<davecheney> however the PA doen't ahve environments.yaml, only the data stored in zk
<davecheney> again, no supprise
<niemeyer> davecheney: Right, all good
<davecheney> so we've added support to state and environs to be able to construct an Envrion interface from zk data
<niemeyer> Yep, sensible
<niemeyer> (now comes the interesting leap)
<davecheney> so, what happens when the underlying data in zk changes
<davecheney> there are number of possible choices, and it looks liek I've chosen the wrong logic
 * davecheney re-reads the preceeding discussion before continuing
<davecheney> ah, yes, you previously said, "If the configuration of an environment changes, we should replace its configuration at an  appropriate well known time,
<davecheney> "
<davecheney> i offer that the proxy environ does that
<davecheney> without having to teach the real environs how to do that
<davecheney> and it is not papering over their limitations
<davecheney> but a good seperation of concerns
<davecheney> hey, what did you miss ?
<niemeyer> LOL, perfect timing for the disconnection..
<niemeyer> <niemeyer> davecheney: Right, all good
<niemeyer> <davecheney> so we've added support to state and environs to be able to construct an Envrion interface from zk data
<niemeyer> <niemeyer> Yep, sensible
<niemeyer> <niemeyer> (now comes the interesting leap)
<davecheney> 08:18 < davecheney> so, what happens when the underlying data in zk changes
<davecheney> 08:19 < davecheney> there are number of possible choices, and it looks liek I've chosen the wrong logic
<davecheney> 08:20  * davecheney re-reads the preceeding discussion before continuing
<davecheney> 08:20 < davecheney> ah, yes, you previously said, "If the configuration of an environment changes, we should replace  its configuration at an  appropriate well known time,
<davecheney> 08:20 < davecheney> "
<davecheney> 08:21 < davecheney> i offer that the proxy environ does that
<davecheney> 08:21 < davecheney> without having to teach the real environs how to do that
<davecheney> 08:21 < davecheney> and it is not papering over their limitations
<davecheney> 08:21 < davecheney> but a good seperation of concerns
<niemeyer> davecheney: and that was the interesting leap
<niemeyer> davecheney: THe proxy doesn't replace the configuration.. it replaces the whole environment
<davecheney> i accept that
<niemeyer> davecheney: and has to wrap its whole interface so that whoever has it can pick the latest replacement
<davecheney> it does so because there is no Environ.ReloadConfiguration() facility
<niemeyer> davecheney: Exactly
<niemeyer> davecheney: That's what we need.. well, something like it
<davecheney> how would such a faciliy with given that Environ knows nothing of state
<niemeyer> davecheney: SetConfig, potentially
<niemeyer> davecheney: func (e) SetConfig(config EnvironConfig) error, more precisely
<davecheney> ok
<davecheney> i think that pushes a lot of logic down to the environ
<davecheney> it may potentially hold locks while it's config is reloading
<davecheney> i think using a proxy acomplishes the same thing with less work
<niemeyer> davecheney: The logic is in the environment anyway, if you see it from the perspective that the wrapper is actually an environment that deals just with that one factor
<davecheney> sounds like good separation of concerns
<niemeyer> davecheney: Not really, from the perspective that two sequential calls to the same environment interface are actually dealing with different environments
<niemeyer> davecheney: This is really not that great
<davecheney> ok, now I am understanding your concern
<davecheney> i see that as inevitable, and something we need to code for anyway
<davecheney> as the environ can't hold any state about it's remote provider
<niemeyer> davecheney: Why is it inevitable?
<niemeyer> davecheney: Why?
<davecheney> well it can hold state, but it must validate that state before using it
<davecheney> but that is a digression
<niemeyer> davecheney: Yes
<davecheney> so, we're always talking about the case of AWS creds failing
<davecheney> s/failing/changing
<niemeyer> davecheney: Yes.. or failing.. both have to be dealt with
<davecheney> so, take the case of calling .Instances() to get the instance data, then using StopInstances to stop some of those instances
<davecheney> during the two calls the environ is replaced
<davecheney> i dont' see the problem
<davecheney> the data from Instances() comes from the real AWS somewhere, and is passed back to the real AWS in the second call
<davecheney> ah, let me approach it another way
<niemeyer> davecheney: I don't want to be using a value that is changing behind my feet.. I don't want to be programming with that mindset. The problems we handle are complex enough by themselves.. we don't have to spoil our own environments.
<davecheney> so you are happen for the configuration inside an Environ to change, but not the reference to the Environ itself ?
<niemeyer> davecheney: Yes, I'm happy to see the configuration within the environment changing when the configuration of the environment changes
<niemeyer> (!)
<davecheney> maybe i can approach this another way
<davecheney> with a different example
<davecheney> two machines are added to the topology, the PA starts the first, then is restarted, then starts the second
<davecheney> the reason i ask this is I am trying to understand what is so important inside the specific instance of the real Environ that you want to preserve (that isn't its configuration)
<niemeyer> davecheney: func MyFunc(e environs.Environ) { ... <code> ... }.. is it fine to replace what e *is* in the middle of MyFunc?
<davecheney> niemeyer: if e is an interface, yes
<davecheney> wait, let me reread that again
<davecheney> yes, if e is an interface, yes
<niemeyer> davecheney: Oh, ok.. so it doesn't matter how e is implemented, or what <code> does? It's always ok, necessarily?
<davecheney> in the situation of juju, we have to code to taht
<davecheney> for the specific case that e represents a set of calls to an external provider of resources
<davecheney> here is a better example
<davecheney> if e was an SQL connection
<niemeyer> davecheney: Not at all.. we don't have to code with a wild environment where we have no idea about what our own code is doing
<davecheney> it is well established to give back a proxy e (sql connection) that goes an gets a real sql connection when invoked
<niemeyer> davecheney: You've lost me
<davecheney> let me back up
<niemeyer> davecheney: I have no idea about what you mean there
<davecheney> lets not talk about environs for a moment
<davecheney> you know how db connection pools work
<niemeyer> davecheney: Yep
<davecheney> you get a proxy connection from the pool, then when you do work on it, it checks out a real connection
<niemeyer> davecheney: No
<davecheney> ok, i'll cease that line of argument
<davecheney> ok, to summarise, i am aruging that using a proxy environ achieves the same as an (unwritten) Envrion.SetConfig()
<davecheney> but I can't offer any argument that says it is superior (apart from we don't ahve to write SetConfig)
<davecheney> so I will drop this branch and go an do SetConfig
<niemeyer> davecheney: I understand your argument, and I've been pointing out that it doesn't.. an Environ is an interface.. the implementation of that interface can do anything it pleases, cache anything it wants, start any goroutines it wants..
<davecheney> niemeyer: ok, yes, and that would be sensible
<davecheney> howeve the consumer of the interface can't assume any of that
<niemeyer> davecheney: Replacing the credentials that an environment uses should not destroy all of that.
<niemeyer> davecheney: Nor should it cause code that takes an Environ to blindly talk to two different values in an arbitrary moment.
<davecheney> ok, you've won me over
<davecheney> i would point out that the python version does rebuild a new environ when the config changes
<davecheney> but i won't take long to do SetConfig
<niemeyer> davecheney: We're talking about *credentials*.. why are we killing the value representing the whole environment to have its credentials changed
<davecheney> i have a slight concern that at some point in the future configuration may include more than credentials
<davecheney> but for right now, and what we need for this cycle
<davecheney> i can't see that happened
<niemeyer> davecheney: The Python provisioning agent isn't something I'd put in a frame, for a few different reasons.. I'm sure you can come up with something we'll be a lot more proud of
<davecheney> flattery will get you everywhere :)
<niemeyer> davecheney: I'm being honest!
<davecheney> ok, i'll add SetConfig
<davecheney> but I would point out my concern that SetConfig will have to drop (potentially) any caches or other internal state
<davecheney> but having just writtne that
<davecheney> it's not going to be that hard to recognise those things
<niemeyer> davecheney: Agreed, and it's also conscious.. the exact moment in time when the agent replaces the environment configuration is under our control.
<davecheney> ah that raises an intersting point
<davecheney> when you say under our control, i imagine you saying 'when nothing else is using it'
<davecheney> is that fair to say ?
<niemeyer> davecheney: You know it's not :)
<davecheney> good
<davecheney> because that would make things harder
<niemeyer> davecheney: But the provisioning agent itself should not be spinning and replacing the configuration in the background.. there's no reason to, I believe
<davecheney> so, currently in the proposed PA, the actual stop/start signal is delegated to a goroutine to handle any potential retry logic
<davecheney> the aim of that was, the signal of topology change is sent one time
<davecheney> on the change, you can't observe it again
<davecheney> so we have to store it until it's actioned
<niemeyer> davecheney: We've debated at length the idea of how the agents should be organized, and our tentative design direction was to have a main loop that is responsible for dispatching tasks
<niemeyer> davecheney: I imagine the configuration replacement being done as one of the potential tasks in that main loop
<davecheney> ok my recollection was the opposite, but i'm happy to be corrected
<niemeyer> davecheney: That's all theoretical, but it gives you the feeling we have about the problem, at least
<niemeyer> davecheney: I recall we talked about one goroutine per machine.. is that what you are alluding to?
<davecheney> yes
<niemeyer> davecheney: Yeah, this is a recent idea
<davecheney> so, either the main loop servicing the watcher maintains a list of machines to be added and delted
<niemeyer> davecheney: I'm not entirely sure about it yet..
<davecheney> then services that list when there is nothing else to do
<niemeyer> davecheney: I'm not in a good position to say either way since it depends a lot on how the details are done
<davecheney> or do something like this
<davecheney> http://codereview.appspot.com/6220063/
<niemeyer> davecheney: I suspect having goroutines per machine should make the problem simpler, at least in terms of moving things concurrently and focusing on a single task at once
<davecheney> i think so too
<davecheney> however it makes the problem of reloading the environment configuration a bit trickier
<niemeyer> davecheney: Still, there's some maintenance task which is unrelated to an individual machine, such as configuration changing, which seems to make sense to be done by a main loop
<davecheney> as, from the point of view of those workers, it can happen at any time
<niemeyer> davecheney: That orchestrates the whole thing, if you will
<davecheney> yup, so it's selecting on the watcher for machines and the watcher for confgiuration
<davecheney> machine changes get delegated
<davecheney> configuration change call environ.SetConfig
<niemeyer> davecheney: Right, but even those changes need some taking care of
<niemeyer> davecheney: E.g. a controlled reboot needs to take into account those tasks
<davecheney> right, you're talking about limiting work in progress and the number of concurrent connections in progress
<niemeyer> davecheney: No, not just that
<davecheney> can I ask a semi related question ?
<niemeyer> davecheney: I'm saying that it's not just a background task that we fire and forget
<davecheney> do machines, that is to say their topology name, ever get reused
<niemeyer> davecheney: It needs to be managed
<niemeyer> davecheney: If the provisioning agent has to reboot (e.g. upgrades) it should be done in a controlled fashion
<davecheney> can you expand a bit on that last sentance pelase
<niemeyer> davecheney: Pretty much all background activity must be accounted for, and properly terminated synchronously when necessary
<niemeyer> davecheney: Watchers, for example, have Stop methods that request the termination of the background task
<niemeyer> davecheney: Those methods are synchronous.. they ask for the background activity to die, and wait until they do so
<davecheney> can you expand on this bit "If the provisioning agent has to reboot (e.g. upgrades) it should be done in a controlled  fashion
<niemeyer> davecheney: It's critical that things that start background activity, have hooks onto the termination of those activities, and that we can wait and verify that the given activity was ok
<davecheney> specifically about rebooting the PA
<niemeyer> davecheney: We need to be able to upgrade, from day zero
<niemeyer> davecheney: Upgrade == get new version, unpack, reboot
<niemeyer> davecheney: s/reboot/restart/, maybe that's clear
<davecheney> ok, i'm not sure i understand the significance of that
<niemeyer> davecheney: process stops and starts?
<davecheney> i'm assuming this code will run on EC2 therefore stands a strong chance
<davecheney> of being rebooted at any time
<niemeyer> davecheney: Yes, why does that matter?
<davecheney> i'm writing everything to assume no local state
<davecheney> everything is sensed from the zk state
<davecheney> and acts uppon it anew every time
<niemeyer> davecheney: Yep, that's nice.. but you have local state no matter how much you pray not to.. in memory
<niemeyer> davecheney: Code running, variables flowing, error conditions
<davecheney> sure, but lets not get pedantic
<niemeyer> davecheney: No no, I'm serious
<niemeyer> davecheney: It's not ok to say 1) start machine; 2) reboot
<davecheney> the only assumption i'm building on is that the watcher will always tell me about anything I have missed on the first cycle
<niemeyer> davecheney: Yes, that's fine
<davecheney> tf the PA can start at any point and learn about the complete topology to that point without having to maintain it's own journal
<niemeyer> davecheney: I'm talking about background activity.. it's not ok to put a goroutine to do something and then ignore it
<niemeyer> davecheney: Watchers have Stop methods
<niemeyer> davecheney: We can stop them, and get the error condition
<niemeyer> davecheney: That's not what I'm saying.. I'm saying that activities should be controllable.. it's not ok to run stuff in the background and be unable to ask them to stop or to tell if they worked or not
<niemeyer> davecheney: That's hard to test, hard to debug, hard to make work correctly
<davecheney> ok, i understand your concerns
<davecheney> i will work towards addressing them today
<niemeyer> davecheney: Python code base is largely unmanaged.. watchers are very hard to deal with
<niemeyer> davecheney: We have code hacks in place for managing the watcher logic that was obtained from trial and error
<niemeyer> davecheney: "Oh, hey, yeah.. maybe it's still running"
<niemeyer> davecheney: That sucks.. we know better now.. let's keep track of stuff running so we have a handle on them
<davecheney> right, because watchers drive twisted callbacks, which might still be firing
<niemeyer> davecheney: Exactly
<davecheney> so, to conclude, because this has taken a lot of your time
<davecheney> you are happy for me to continue to itterat on this
<davecheney> i do worry that the PA is becoming a blocker to having _something_ to get us a golang juju
<niemeyer> davecheney: I am very happy with that, and I find that conversation very healthy
<niemeyer> davecheney: It is a blocker indeed, but you had to learn about the problems we face
<niemeyer> davecheney: and we all have to learn about how to design it properly
<niemeyer> davecheney: I suggest small branches, that we can quickly merge/refactor/reject/whatever
<andrewsmedina> Im back
<niemeyer> davecheney: So that as a side effect of these conversations we move forward steadily, and get unblocked
 * davecheney applauds
<niemeyer> Alright.. I have to go help my wife for a moment.. back later
<niemeyer> andrewsmedina: Heya
<davecheney> nw
<davecheney> thanks for the talk
<niemeyer> andrewsmedina: Will still be back today, in case you wanna talk
<andrewsmedina> niemeyer: ok
<andrewsmedina> niemeyer: juju client on centos working fine :D
<niemeyer> andrewsmedina: Woohay
<jimbaker`> andrewsmedina, is this the python version of the juju client?
<niemeyer> I suppose.. the Go version isn't runnable yet
<andrewsmedina> jimbaker`: in this python version
<andrewsmedina> niemeyer: we will work now to do vms create by juju works fine with centos :D
<niemeyer> andrewsmedina: Sweeet
<niemeyer> andrewsmedina: How's the PaaS side of things going?
<andrewsmedina> niemeyer: the PaaS have a cli and a webserver (rest api) both written in go
<davecheney> andrewsmedina: cool
<davecheney> and that runs ok on RHEL5 ?
<davecheney> i ask because technically 2.6.18 isn't a supported kernel
<andrewsmedina> davecheney: it's running on ubuntu at the moment
<davecheney> ah, ok
<andrewsmedina> davecheney: we will use centos 6.
<andrewsmedina> 6.2
<davecheney> no problems then
<davecheney> 2.6.32 is fine
<davecheney> go could be made to run on 2.6.18 easily
<andrewsmedina> to create machines for projects (django, php, rails) and services (mysql, mongo, elastic search) the webserver calls juju in the backend
<andrewsmedina> and with juju we can use openstack or ec2 for IaaS
<andrewsmedina> we are using openstack essex
<andrewsmedina> this project is opensource https://github.com/timeredbull/tsuru
<davecheney> niemeyer: are you still there
<davecheney> niemeyer: nm, will be back in 30 mins (breakfast)
<niemeyer> davecheney: Here
<davecheney> was just looking at the latest changes to environs/interface.go
<niemeyer> Trying to get that S3 signing stuff working
<niemeyer> davecheney: Ok?
<davecheney> what is the signature that Environs.SetConfig() should use
<davecheney> should it take an EnvironConfig ?
<davecheney> or just a plain map[string]interface{}
<niemeyer> davecheney: Yeah, EnvironConfig
<niemeyer> davecheney: So that we force the raw data to go through NewConfig
<niemeyer> davecheney: Which validates and processes it into something the environment is happy with
<davecheney> ok, might have to do some type checking on the way in
<niemeyer> davecheney: I think it's fine to assume it's the correct one
<niemeyer> davecheney: It will panic if it's not, and that seems fine given it would be a major coding mistake
<davecheney> yup
<davecheney> ok, afk for real this time
<niemeyer> davecheney: See ya later
#juju-dev 2012-05-23
<davecheney> niemeyer: http://paste.ubuntu.com/1002200/
<davecheney> as an example API
<davecheney> i will have to add mutexs around all of the variables touched by SetConfig to ensure their visibility
<niemeyer> davecheney: Might make sense to have one or more internal helper methods.. e.g.: func (e) credentials() (accessKey, secretKey string)
<niemeyer> davecheney: So the locking is constrained
<davecheney> yeah, each providor is different
<niemeyer> davecheney: yeah, that's for internal use only
<davecheney> in ec2, config is used to construct an ec2 and s3 at open time
<davecheney> so those will need to be reconstructed
<niemeyer> davecheney: Same idea.. rather than credentials, func (e) s3() *S3 then..
<davecheney> yup
<niemeyer> davecheney: So locking is only necessary within the method itself
<davecheney> i will try to make it as fine grained as possible
<niemeyer> davecheney: I was more worried about convenience than granularity in this case
<niemeyer> davecheney: THis is a very rare event
<davecheney> yup
<niemeyer> ... value *net.OpError = &net.OpError{Op:"remote error", Net:"", Addr:net.Addr(nil), Err:0x28} ("remote error: handshake failure")
<niemeyer> Com'on Amazon
<niemeyer> Woohay
<niemeyer> Signed URL works
<niemeyer> and proposed
<davecheney> niemeyer: just go though a first cut of Environ.SetConfig
<niemeyer> davecheney: Nice
<niemeyer> davecheney: How does it feel?
<davecheney> not too bad
<davecheney> had to use _variables
<davecheney> which feel icky
<davecheney> but adding accessors for the configuration items wasn't too bad
<davecheney> just running off lbox propose
<niemeyer> davecheney: Feels icky indeed
<davecheney> https://code.launchpad.net/~dave-cheney/juju/go-environs-setconfig/+merge/106930
<niemeyer> davecheney: Will let you know if I can think of something else tomorrow
<niemeyer> davecheney: Right now it's past bed time here
<davecheney> niemeyer: at the risk of reactivating the discussion from earlier
<davecheney> the proxy environ was cleaner
<davecheney> as in the case of ec2, we replace everything when we call SetConfig
<davecheney> the end result is very similar
<davecheney> but we can continue that discussion tomorrow
<niemeyer> davecheney: It's actually not, for the reasons I explained.. doing e.Foo(); e.Bar() and being completely unable to tell whether e is a completely different value is a no no.
<davecheney> fair enough, i'm happy the discussino is closed
<niemeyer> davecheney: I'm happy to discuss other possibilities, but something that has that property won't fly
<davecheney> niemeyer: understood
<niemeyer> davecheney: The locking you did, even if it feels like overhead, is putting you in control
<davecheney> why does lp treat me so badly ?
<davecheney> % lbox propose -wip -edit
<davecheney> error: Failed to load data for project "juju": Get https://api.launchpad.net/devel/juju: unexpected EOF
<davecheney> it's totally random
<niemeyer> davecheney: Good question.. I haven't seen that level of weirdness
<niemeyer> davecheney: and I'm not specially close to it either
<niemeyer> Anyway, bed for reals now :)
<niemeyer> davecheney: Have a good EOD
<davecheney> kk
<davecheney> rogpeppe: around today ?
<TheMue> davecheney: Heya
<davecheney> hey frank
<davecheney> how are you going ?
<TheMue> davecheney: Fine, pretty wither today here in Germany. And you?
<davecheney> pretty cold in sydney
<davecheney> still grinding on the provisining agent
<davecheney> still rewriting the environs interface :)\
<TheMue> davecheney: I'm still working on the relations, currently three depending branches in the queue.
<rogpeppe> davecheney: just up
<rogpeppe> davecheney, TheMue: morning!
<TheMue> rogpeppe: Heya
<Aram> morning.
<rogpeppe> Aram: hiya
<TheMue> Aram: Moin
<rogpeppe> fwereade: yo!
<fwereade> rogpeppe, heyhey
<fwereade> rogpeppe, I just saw yu had anothe CL in that same sort of area perhaps?
<fwereade> rogpeppe, if my comments are redundant, sorry
<rogpeppe> fwereade: i'm moving it all forward bit by bit...
<fwereade> rogpeppe, cool -- I'm just not sure about fixing the name without fixing the type ;p
<rogpeppe> fwereade: you think that InstanceSpec is ill-conceived? what should we have there?
<fwereade> rogpeppe, the type itself is not so bad, but half the fields are misleading/useless
<TheMue> fwereade: Moin
<rogpeppe> fwereade: "a type that is moderately ill-conceived in the first place." :-)
<fwereade> rogpeppe, we have no interest in daily, desktop, or instance-store images
<rogpeppe> fwereade: ok, i'll take out that param
<fwereade> rogpeppe, when fully 50% of a type's fields are irrelevant, I think a little of the ill-conceivedness leaks out into the type itself :p
<rogpeppe> fwereade: naah
<fwereade> rogpeppe, ok, "obscures any essential rightness the type may be in posession of"
<rogpeppe> fwereade: it's a place-holder for things to come IMHO
<rogpeppe> fwereade: the important change is that the i want the type to represent the constraints on an instance, not just an machine image.
<fwereade> rogpeppe, sure, a type can be incomplete, but it's carrying baggage that's just unhelpful, and that's all I'm complaining about
<rogpeppe> s/the i/i/
<fwereade> TheMue, heyhey
<rogpeppe> fwereade: sure. maybe this is a good CL to change some of those in.
<rogpeppe> fwereade: or perhaps i could do that in another CL
<rogpeppe> fwereade: i'm trying hard to keep CL sizes down.
<rogpeppe> fwereade: i.e. move things forward in small increments.
<fwereade> rogpeppe, I'm easy
<rogpeppe> fwereade: my next branch is too big and i'm going to need to split it up, which is a right pain
<fwereade> rogpeppe, yeah, I seem to spend rather too much of my time doing that sort of thing
<rogpeppe> fwereade: i was thinking last night that i want some way of saying "factor *these* changes into a new branch" - in about one line of shell.
<Aram> so you want git? :D.
 * Aram hides.
 * rogpeppe has never used git
<Aram> it does that thing.
<Aram> probably the only thing it does well.
<fwereade> that was kinda my first thought too :)
<rogpeppe> Aram: out of interest, what git command would you use to do that?
<Aram> if you want to do it at commit time, you just git add your changes (my understanding is that you can git add parts of the file as well, not whole files).
<Aram> basically in git you don't commit everything by default, but have to chose what to commit every time.
<Aram> there's also cherry picking.
<Aram> that makes it easy to move only some changes between branches, but not every change.
<Aram> cherry picking is literally picking commits from one place and applies them somewhere else, so you don't have to do any manual merging step or whatever. really handy.
<Aram> as much as I dislike git and I complain about it all the time, it does some things well.
<rogpeppe> fwereade: hope you've had a good few days off, BTW
<rogpeppe> fwereade: for PersistentStorage/Daily/Desktop, are you saying that we will *never* want to use anything but PersistentStorage=true, Daily=false, Desktop=false ?
<rogpeppe> fwereade: because that logic is in the python version, so i copied it across
<rogpeppe> fwereade: but if we know the settings for all time, i'm always happy to delete code!
<fwereade> rogpeppe, yeah: all that stuff is gone from the python
<rogpeppe> fwereade: sigh
<fwereade> rogpeppe, and niemeyer appears to be -1 on handling instance-store stuff
<rogpeppe> fwereade: ok, i'll remove the settings in an upcoming branch
<fwereade> rogpeppe, which I can get behind tbh, even though it would be trivial to implement
<fwereade> rogpeppe, cool, thanks
<fwereade> updates, restarting
<rogpeppe> test
<TheMue> rogpeppe: success
<rogpeppe> TheMue: thanks. my web browser seems to be playing up.
<TheMue> fwereade: Thx for review. Requirer as noun exists? Sounds strange. But ok. ;)
<rogpeppe> cool google logo today
<fwereade> TheMue, it's not one I'd use in conversation, but it's definitely closer to legitimate english than, say, "ReadCloser" :p
<TheMue> rogpeppe: Already played a song?
<TheMue> fwereade: Hehe, yeah.
<TheMue> fwereade: So, name change done. No merge it again into the follow-up.
<fwereade> TheMue, cool, thanks
<fwereade> rogpeppe: in testing charm repos, would you agree it makes most sense to run them against an actual store.Server?
<rogpeppe> fwereade: it depends how much interaction with the server it has, i guess
<rogpeppe> fwereade: if it's just a simple GET, then maybe there's no point.
<rogpeppe> fwereade: we don't necessarily want to fire up mongodb for every test.
<rogpeppe> fwereade: but that's a *totally* off-the-cuff response!
<fwereade> rogpeppe, that's the thing... it does feel a little bit heavyweight
<fwereade> rogpeppe, but a canned-response tet server feels a little lightweight :/
<rogpeppe> fwereade: does it actually do anything other than a simple GET?
<fwereade> rogpeppe, I'll think on it some more; early lunch today, maybe somewhat extended; will be back for the rest of my "morning" in a bit
<fwereade> rogpeppe, (taking half-days this week)
<rogpeppe> fwereade: cool
<rogpeppe> fwereade: still in the UK presumably?
<rogpeppe> fwereade: hope you've got the same lovely weather we have. perfect for half days...
<TheMue> rogpeppe: Could it be that the watcher problems are since about one week?
<rogpeppe> TheMue: it's possible.
<rogpeppe> TheMue: i couldn't say for sure i'm afraid
<rogpeppe> TheMue: have you got a possible candidate for the source of the problems?
<TheMue> rogpeppe: Just found the change introducing the "initialValue" flag in the content watcher.
<rogpeppe> TheMue: it's entirely possible that's the problem!
<TheMue> rogpeppe: And the problem I've got here in my test belong to the content watcher.
<rogpeppe> TheMue: it may easily all be my fault.
<rogpeppe> TheMue: if that's true, we need better tests in the watcher package :-)
<TheMue> rogpeppe: *lol*
<TheMue> rogpeppe: What's the intention behind this flag?
<rogpeppe> TheMue: it's so that the initial value is always sent, regardless of what it is
<rogpeppe> TheMue: perhaps your tests are assuming otherwise...
<rogpeppe> TheMue: s/tests are/code is/
<TheMue> rogpeppe: That's possible, the old behavior has been different.
<rogpeppe> TheMue: i think maybe the old behaviour didn't send something if the initial contents were blank
<rogpeppe> TheMue: that might actually be the behaviour we want, i guess.
<rogpeppe> TheMue: one question: why doesn't the test fail all the time?
<TheMue> rogpeppe: It might or it is?
<rogpeppe> TheMue: i dunno. depends what we're after.
<TheMue> rogpeppe: Yes, that's an interesting question.
<rogpeppe> TheMue: i think it's probably because it depends whether the content watcher sees the initial blank value before the test changes it.
<TheMue> rogpeppe: So we're back to the race condition.
<rogpeppe> TheMue: yeah, but we know what the problem is, and it's easy to solve - just put a sleep before the initial contents set.
<TheMue> rogpeppe: Yip
<rogpeppe> TheMue: woof
<niemeyer> Hello all
<Aram> hi.
<niemeyer> yo.
<niemeyer> Aram: Monday is the day then?
<Aram> yes.
<niemeyer> nice.
<niemeyer> that's.
<niemeyer> awesome.
<niemeyer> :-)
<Aram> yes :).
<niemeyer> rogpeppe, TheMue, fwereade: Aram is joining us officially on Monday
<rogpeppe> niemeyer: cool. i heard from robbiew yesterday.
<rogpeppe> Aram: welcome!
<TheMue> niemeyer: Moin. Great news!
<Aram> thanks, hi all!
<TheMue> Aram: Welcome.
<andrewsmedina> fwereade: thanks for your review :D
<fwereade> andrewsmedina, a pleasure :)
<fwereade> Aram, welcome!
<fwereade> niemeyer, heyhey
<Aram> thanks/.
<niemeyer> andrewsmedina: Heya
<niemeyer> andrewsmedina: Aram is joining the team on Monday, btw
<niemeyer> fwereade: Heya
<niemeyer> rogpeppe: Wanna have another look at SignedURL?
<rogpeppe> niemeyer: looking already
<niemeyer> rogpeppe: Cool, thanks for the review, looks even better now :)
<rogpeppe> niemeyer: "
<rogpeppe> Corrupting the returned URL is testing the behavior of something that is not
<rogpeppe> what the method produces.
<rogpeppe> "
<rogpeppe> i was suggesting, i think, that we test that the unsigned URL would fail where the signed URL would succeed
<rogpeppe> i'm not sure that the tests verify that currently - we're just assuming that Get of the unsigned URL will fail because the object is private
<niemeyer> rogpeppe: And I'm suggesting that this is not testing SignedURL
<niemeyer> rogpeppe: Because you're testing something that is *not* the signed URL
<andrewsmedina> Aram: welcome :D
<rogpeppe> niemeyer: the assumption of those tests is that the signed URL is giving us something that the unsigned URL does not, right?
<niemeyer> rogpeppe: There are positive and negative tests in place already, and both of them verify the real URL.
<niemeyer> rogpeppe: No
<niemeyer> rogpeppe: The unsigned URL is completely irrelevant
<niemeyer> rogpeppe: The behavior of a signed URL stands on its own.. if you corrupt the URL, you're not testing the method anymore
<Aram> andrewsmedina: thanks.
<rogpeppe> niemeyer: if the server is letting everything through anyway, then the signing is irrelevant and the test isn't testing anything.
<rogpeppe> niemeyer: my suggestion makes sure that the test *is* actually testing something.
<niemeyer> rogpeppe: If the server is letting everything through, the tests will fail, and they *did fail*!
<niemeyer> rogpeppe: They are testing something.. please take the code and set authIsBroken to 1
<andrewsmedina> Aram: were are you from?
<niemeyer> rogpeppe: or 0
<niemeyer> rogpeppe: It will break, because the auth is letting everything through in the fake server
<TheMue> robbiew: Hi. I'm ready.
<rogpeppe> niemeyer: for instance, i think the test would succeed ok if the server verified expiration time without testing the signature
<Aram> andrewsmedina: I am from romania but I currently live in austria.
<TheMue> Aram: Austria? Ah, so you now may be the nearest to me. Been twice in Vienna, a wonderful town.
<niemeyer> rogpeppe: Heh.. the server may be broken in all kinds of ways, yes
<andrewsmedina> Aram: I'm from Brazil
 * TheMue should measure, Roger still has good chances to be nearer.
<Aram> TheMue: where are you from?
<TheMue> Aram: Oldenburg in North Germany
<rogpeppe> niemeyer: it's ok, i don't mind if we don't do the check. it was just a suggestion.
<Aram> andrewsmedina: same place as niemeyer?
<niemeyer> rogpeppe: If it allowed you to see a *private* object, just because you provided a URL with an expire time, what is broken is not the SignedURL method, though.
<niemeyer> rogpeppe: Thanks for the suggestion. I've read it, understood it, and replied.
<robbiew> TheMue: g+ invite on the way
<TheMue> robbiew: ack
<rogpeppe> niemeyer: true. but it does mean we haven't verified SignedURL. i don't mind much.
<niemeyer> rogpeppe: No, it doesn't mean that. I don't mind much that you don't agree either, though.
<andrewsmedina> Aram: no, I live in Rio de JAneiro
<niemeyer> rogpeppe: Because we tend to get in those interminable arguments very frequently.
<rogpeppe> niemeyer: sorry about that
<niemeyer> rogpeppe: The test not only worked, but it detected the fact that the auth in the server was broken. There's a positive and a negative test, both verifying access to a private object.
<niemeyer> rogpeppe: Your argument is that you wanted to see that a plain URL, *without signature*, works.
<niemeyer> rogpeppe: It's beyond me how you're willing to argue that to death.
<niemeyer> </argument>
<rogpeppe> niemeyer: i'm not. i've stopped. i'm fine with what you've got.
<niemeyer> rogpeppe: You've stopped with "it does mean we haven't verified SignedURL". Thanks, but that sucks.
<niemeyer> rogpeppe: Enjoy the method, though. Happy to help.
<rogpeppe> niemeyer: oh sorry. i meant that only in the case that the server is broken so it checks the expiry time only.
<Aram> TheMue: by air you seem to be the same distance from me and rogpeppe, by land I'm closer because there's no water to cross.
<rogpeppe> niemeyer: "// URL returns a non-signed URL that allows retriving the object at path".
<rogpeppe> s/retriving/retrieving/
<rogpeppe> :-)
<rogpeppe> and i'd s/a non-signed/an unsigned/ too
<rogpeppe> but i'm too late ...
<niemeyer> rogpeppe: Fixed.
<TheMue> Aram: Yes, it's funny how near the UK is, only the North Sea between us.
<niemeyer> Heading to lunch
<fwereade> hey all: despite the very long lunch, I'm pretty sure I've done my half day now
<rogpeppe> fwereade: np. you might want to have a brief look at this, if you have any time at all: https://codereview.appspot.com/6209094
<rogpeppe> fwereade: it adds a "public-bucket" field to the ec2 config
<rogpeppe> fwereade: i'm not entirely sure that's the best name for it, but gotta start somewhere...
<rogpeppe> i have to have some lunch too
<Aram> what's your setup guys, do you use a stable or unstable ubuntu release for development?
<Aram> (so the hidden question really is if a stable version is good enough).
<fwereade> rogpeppe, it suddenly seems to me as if we're going to a lot of trouble to accommodate something in juju (tool upload) that could be entirely replaced with (1) the public-bucket and (2) a tiny separate script that just nukes the bucket and repaves with fresh local builds
<fwereade> rogpeppe, maybe my problem is derived from the name though... but surely the public source will always be the same for everybody? so, not necessarily a bucket, just a url prefix
<fwereade> rogpeppe, and the only reason (that I can see) to make it configurable is for testing
<fwereade> rogpeppe, have I just gone off into the weeds?
<fwereade> Aram, stable, fwiw
<fwereade> rogpeppe, I do need t be off now; I've left a tab open, it'll be the first thing I see when I come back :)
<fwereade> later :)
<rogpeppe> Aram: i use vanilla 12.04
<TheMue> Aram: I'm using stable (so now 12.04) as "production environment" in a VM and test unstable releases in another VM.
<Aram> I see.
<rogpeppe> fwereade: i don't know. maybe you're right and we should just strip all that stuff out.
<rogpeppe> niemeyer: what do you think?
<niemeyer> fwereade: You mean implementing the exact same logic as tool upload, but outside of juju?
<niemeyer> rogpeppe: If that's what fwereade means, I'm not sure.. feels like we'll have the exact same trouble on both sides.. we're just delegating it to a second class citizen that is not as well integrated.
<rogpeppe> niemeyer: i guess it would mean that we could do a simple uploader for each provider type without having to have all the --upload-tools flags and args to Bootstrap etc
<niemeyer> rogpeppe: All? You mean two? :)
<rogpeppe> niemeyer: :-)
<rogpeppe> niemeyer: the logic for finding tools would be a little simpler too. only one Storage per environ.
<niemeyer> rogpeppe: I don't have a strong feeling about it, but I'm not convinced it's getting us much. Having convenient mechanisms for people to develop juju itself feels like a valuable goal.
<niemeyer> rogpeppe: How so? public-bucket is a storage
<niemeyer> rogpeppe: And it has to exist for every single storage, despite uploading tools
<niemeyer> rogpeppe: Because the real tool deployments uses the public storage too
<niemeyer> rogpeppe: The only difference is that FindTools would look in a single storage, rather than in two. So 3 or 4 lines rather than 6 or 7
<rogpeppe> niemeyer: yeah, that's true. we need both regardless.
<rogpeppe> niemeyer: did you take a look at the public-bucket branch, BTW?
<rogpeppe> niemeyer: (i'm quite happy with the direction we're going actually. it's always good to try and take a step back and think "is this really the right way to do it?" though.)
<niemeyer> rogpeppe: No, I had a look at a nice lunch just now :)
<rogpeppe> niemeyer: cool. hope your weather is as nice as ours ...
<niemeyer> rogpeppe: What's the public-bucket setting about?
<niemeyer> rogpeppe: It's surprisingly good
<niemeyer> rogpeppe: Feels like we're going back onto summer, even though it's quite the opposite
<rogpeppe> niemeyer: public-bucket is where the provider looks for public storage.
<rogpeppe> niemeyer: i.e. to get the publicly accessible tools downloads from
<niemeyer> rogpeppe: Ah, nice
<rogpeppe> niemeyer: perhaps in the future we might want to support a simple URL, so we don't have to host the binaries in EC2, but i think that a bucket should be ok for now. and it makes it easy to write to for testing purposes :-)
<niemeyer> rogpeppe: Btw, are you seeing my messages in that thread with William?
<niemeyer> rogpeppe: Or just William's?
<rogpeppe> niemeyer: the "serve metadata and config" thread?
<niemeyer> rogpeppe: Yeah
<rogpeppe> niemeyer: yeah, i'm seeing them. although i haven't read it in a couple of hours
<niemeyer> rogpeppe: Ok.. np
<niemeyer> rogpeppe: negronjl seems to missing my messages that go to the list for some reason
<niemeyer> negronjl: Apparently they're reaching the list, at least
<rogpeppe> niemeyer: well, i *might* be missing some of your messages... :-)
<rogpeppe> niemeyer: oh yeah, just seen negronjl's email. perhaps you count as spam :-)
<niemeyer> rogpeppe: It might be half-right.. :-)
<negronjl> niemeyer, rogpeppe:  lol ...
<negronjl> niemeyer: I am checking my spam folder just to be sure ...
<rogpeppe> niemeyer: while i remember... are we going to try to have monday meetings as planned?
<niemeyer> rogpeppe: Yeah, we should
<niemeyer> rogpeppe: This one might be a tricky one as I'll be in London, but let's make an effort at least
<rogpeppe> niemeyer: does that mean three of us will all be in the UK at once? woo!
<negronjl> nothing on my spam from niemeyer ....
<niemeyer> rogpeppe: Unfortunately not.. William is back to Malta next week
<niemeyer> negronjl: I really can't tell then.. maybe we should ask IS
<negronjl> niemeyer: I'll ask
<niemeyer> negronjl: Thanks, please let me know.. curious about what could be going on as well
<negronjl> rt.admin.canonical.com #53190
<negronjl> niemeyer: ^^
<niemeyer> negronjl: Don't have access to it, but thanks.. let's see what they find
<rogpeppe> i'm off to enjoy the sunshine
<rogpeppe> see y'all tomorrow
<niemeyer> rogpeppe: Enjoy
<Aram> hi davecheney.
<davecheney> howdy
<niemeyer> davecheney: Morning
<davecheney> morning
<niemeyer> davecheney: 181 Â»       Â»       panic(fmt.Sprintf("configuration was %T, expected %T", cfg, new(
<niemeyer>      providerConfig)))
<niemeyer> davecheney: That's not necessary
<niemeyer> davecheney: c, ok := cfg.(*providerConfig) => s/, ok//
<niemeyer> davecheney: The compiler can do that for us
<niemeyer> davecheney: Re-reviewed the branch, mentioned a couple of details, and LGTM
<davecheney> niemeyer: thanks, i'll finalise and submit later today
<davecheney> if you have time coudl you also take a look a the NewConfig banch
<davecheney> branch
#juju-dev 2012-05-24
<niemeyer> davecheney: Will do
<davecheney> niemeyer: ty
<niemeyer> davecheney: The NewConfig branch seems polluted by an undeclared pre-req
<niemeyer> davecheney: https://codereview.appspot.com/6209092/diff/10001/environs/ec2/ec2.go?column_width=80
<davecheney> yeah, only open.go and open_test.go
<davecheney> I will resubmit it once SetConfig is landed
<niemeyer> davecheney: Ok, it'd be nice to get the diff fixed
<davecheney> if you coudl take a cursory glance at environs/open{,_test}.go
<davecheney> that is the only part changed
<niemeyer> davecheney: It looks good
<davecheney> its not a big change, just exposing the EnvironConfig before it gets passed to Open
<niemeyer> davecheney: Yeah, it looks fine if that's all, but it'd be nice to clean up the diff before getting it in
<davecheney> i will certainly do that
<niemeyer> davecheney: Cool, should be a trivial LGTM after that
<twobottux> aujuju: Creating volume group in nova-volume Juju charm <http://askubuntu.com/questions/141552/creating-volume-group-in-nova-volume-juju-charm>
<TheMue> morning
<fwereade> heya TheMue
<rogpeppe> TheMue, fwereade: yo!
<TheMue> rogpeppe, fwereade: Heya
<fwereade> rogpeppe, heyhey
<rogpeppe> fwereade: i bet this weather is just what you're used to... were you really pining for a bit of old-fashioned english drizzle? :-)
<fwereade> rogpeppe, well, it *is* really nice for about 10 minutes
 * rogpeppe likes a bit of rain too
<rogpeppe> but not for months on end :-)
<fwereade> rogpeppe, after that I start to congratulate myself for moving ;)
 * TheMue sits again on the veranda, right now 27Â°C and wonderful sun.
<TheMue> So, short break, have to bring daughter to work.
<Aram> morning.
<fwereade> hey ra
<fwereade> hey Aram
 * davecheney waves to rogpeppe 
 * rogpeppe waves to davecheney
 * rogpeppe waves to davechen1y
<davecheney> so, how would you like to manage this merge ?
<rogpeppe> davecheney: good question. if the ImageSpec->instanceSpec stuff can go in today, it would be quite nice if i could do them both at once.
<davecheney> how is imagespec looking ?
<rogpeppe> davecheney: i've spent... too much time pissing around with splitting up branches and merging and remerging
<davecheney> f'yeah, i'm over reqs
<rogpeppe> davecheney: it should be good to go, but it needs gustavo's go ahead because i'm not quite sure what he wants in the doc comments.
<davecheney> that is cool
<rogpeppe> davecheney: it should be less important now that i've unexported the types.
<davecheney> i'm happy to wait for those two to go in
<davecheney> then I'll merge and adjust as needed
<davecheney> gostavo has LGTM in principal the various bits
<rogpeppe> davecheney: thanks.
<davecheney> i'm sad that the proxy idea didn't get up
<davecheney> but at the end of it I couldn't offer a strong enough argument over Environ.SetConfig
<rogpeppe> davecheney: i think it made sense. but i suppose it only makes sense when the provider has no state
<davecheney> possibly if there was going to be more than one PA
<davecheney> once we have somethig working
<davecheney> i'll hvae another crack at introducing the proxy idea
<davecheney> as a way of seperating concerns
<rogpeppe> davecheney: i quite liked the "return a proxy within ec2" idea
<davecheney> I was thinking proxy.(*proxyEnviron).SetConfig(config)
<davecheney> which would reload it on command
<davecheney> which might resolve gustavo's concerns about control
<davecheney> from my vantage point, the Environs themselves contain almost no state
<rogpeppe> davecheney: i was thinking type Proxy struct {env environs.Environ}
<davecheney> yup
<davecheney> or even
<davecheney> type Proxy struct { environs.Environ }
<rogpeppe> davecheney: and then environs.config.Open returns Proxy{ec2environ}
<davecheney> but that has visilibity problems
<davecheney> anyway, i'll wait til we have something that works
<rogpeppe> davecheney: yeah, i think you want to be sure that all methods are reimplemented
<davecheney> rather than spinning on this
<rogpeppe> davecheney: yeah, the current thing will work.
<davecheney> from my POV I need some commands to do some functional testing on the PA
<davecheney> but I was wondering how jujuc is coming along ?
<rogpeppe> davecheney: it needs more stuff.
<davecheney> from a quick look at the supercommand stuff
<rogpeppe> davecheney: i'm sure fwereade will be cranking out more commands soon though
<davecheney> i can add extra commands to the framework pretty simply
<rogpeppe> davecheney: by "functional testing" you mean ad-hoc testing?
<davecheney> I basically need to invoke, addmachine and remove machine
<davecheney> in another window
<rogpeppe> davecheney: can't you write tests that use the State methods to do that?
<davecheney> i also need to figure out how to hook the provisioning command up to StateTest so I can do unit tests against dummy with it
<rogpeppe> davecheney: i tend not to do any ad-hoc testing, but maybe that's wrong. that was always the only method i ever used, in previous lives...
<davecheney> i've found there is value in doing both
<davecheney> for example I wrote a quick SOCKS over ssh proxy
<davecheney> then left that running on the open internet for a day to load test the ssh client code
<rogpeppe> davecheney: these days i tend to put the effort i would spend on thinking of ad hoc tests into thinking of reusable tests
<rogpeppe> davecheney: there's definitely room for functional testing, but even then i think there's value in scripting it.
<rogpeppe> davecheney: and until we have a framework for that, maybe we should just spend time writing unit tests as good as we can think of.
<davecheney> of course, hence the need for simple cli tools to push the state around so I can watch the PA recact to it
<rogpeppe> davecheney: i think i'd just hook them both up to the same zk and use the state methods and check that the pa reacts in the correct way.
<rogpeppe> davecheney: perhaps using the dummy provider so you can read actions as they take place.
<davecheney> yup, i'm not sure how much of that harness exists atm
<davecheney> i know environs/jujutest does some of it
<rogpeppe> davecheney: cmd/juju does a certain amount of it.
<rogpeppe> davecheney: really, i think it would be nice if the dummy provider could be configured to start zk
<rogpeppe> davecheney: then we get an almost-complete environment for testing.
<davecheney> i'll look into that
<davecheney> i don't think it does that at the moment
<rogpeppe> davecheney: it doesn't
<rogpeppe> davecheney: it shouldn't be hard though, particular with the recent refactoring of it i've done.
<davecheney> in fact I think it works the opposite, it waits for the juju/testing to make a zk
<davecheney> indeed
<davecheney> i'll look at it tomorrow
<davecheney> rogpeppe: can you chain req branches ?
<rogpeppe> davecheney: yeah
<davecheney> ie propose a, propose b -req a, propose c -req b
<rogpeppe> davecheney: although it's a hassle, because once you've changed the base, you have to remerge all the subsequent branches in the chain
<davecheney> yes, that doth blow
<rogpeppe> davecheney: but tbh, it's not as bad as it was. you really only need to remerge when you come to actually submit the next branch.
<davecheney> i'm looking forward to phase of th eproject when we've got something we can itterate on
<rogpeppe> davecheney: definitely.
<davecheney> when I was building the platform at atlassian we got a prototype working very quickly
<rogpeppe> davecheney: i'm just about to propose a branch that gets the go tools and runs them on the server side, BTW
<davecheney> but then there was a big gap of nearly 2 months before we got the next version working
<rogpeppe> davecheney: we've already got a prototype - the python version...
<davecheney> rogpeppe: nice
<davecheney> rogpeppe: the first version of the atlassian platform was three desktops that we built by hand
<davecheney> but the next version took a long time because we decided to build all the automation first
<davecheney> but, once it was running itteration was very fast
<rogpeppe> davecheney: i think that the Go version will be nice to iterate on when we've got it up.
<davecheney> yup
<rogpeppe> davecheney: i'm quite happy with how the overall structure is turning out
<rogpeppe> davecheney: slight misgivings about the whole necessity of the State abstraction, as you pointed out, but i'm ok with it.
<davecheney> rogpeppe: tell me more
<rogpeppe> davecheney: i think it was you that suggested that all of state could be represented more compactly by a simple parent/child/attribute abstraction.
<davecheney> I was thinking that I would like it if I could as the topology for a watcher, rather than the way watchers are quite chummy with the zk nodes themselves
<rogpeppe> s/as the/ask the/ ?
<davecheney> ya
<davecheney> for the particular example of wtching machines
<davecheney> I want the ability to subscribe to a part of the toploogy
<davecheney> what we have now works fine
<rogpeppe> davecheney: +1
<davecheney> watch /topology, then parse the full thing
<davecheney> but given that the State already has the whole topology as a data structure
<davecheney> it would make sense that it could push data to any watcher directly
<davecheney> but I wonder if gustavo has plans for the topology
<davecheney> as it is a crutch for the limitations of zk
<rogpeppe> davecheney: i think the eventual idea is that we push all of state behind a server API
<davecheney> at which point all the watchers are going to be pub/sub kind of things
<rogpeppe> davecheney: then we don't have to worry about atomic changes to the topology because the server can mediate that
<rogpeppe> davecheney: exactly
<davecheney> speaking of Environs
<rogpeppe> davecheney: it's interesting to think about what level we might want the server API
<davecheney> I was going to try a branch that removes the requirement to pass state.Info to Environ.StartInstance
<rogpeppe> davecheney: how would the new instance know how to connect to the state then?
<davecheney> rogpeppe: i hope it looks like a URL tree
<davecheney> pretty much the same as the zk topology
<davecheney> but with REST calls that you can subscribe to changes to a path and it's children, etc
<rogpeppe> davecheney: yeah, but what level of abstraction are the URLs?
<davecheney> without giving it any thought
<rogpeppe> davecheney: i dunno. i was wondering about something higher level.
<davecheney> i think the URL's should look like the current ZK path
<davecheney> but that is without giving it any serious consideration
<rogpeppe> davecheney: i don't think so, actually. i think that constrains the possibilities to db optimisation.
<rogpeppe> s/to db/for db/
<rogpeppe> davecheney: higher level API gives more scope for changing the impl behind the scenes
<rogpeppe> davecheney: anyway... back to ground level
<davecheney> yeah
<rogpeppe> davecheney: how does StartInstance work without having a StateInfo?
<rogpeppe> s/does/would/
<davecheney> there is a StateInfo method on the Environ itself
<davecheney> ergo, it already knows it's state
<davecheney> err, state.Info
<rogpeppe> davecheney: not necessarily. i wanted to keep open the possibility that one environ might connect to a state from elsewhere.
<rogpeppe> davecheney: there's no essential connection between a state and a given environ.
<rogpeppe> davecheney: for instance, at some point in the future, we want to have the possibility of several environments using the same "bootstrap" node.
<fwereade> rogpeppe, hmm, my reading of that had been that they'd still be separate states despite all happening to coexist within the same zookeeper (there's a chrooty thing for ZK, right?)
<fwereade> rogpeppe, it feels to me like a state is pretty strongly associated with an environ
<davecheney> fwereade: i agree
<davecheney> so either Environ gets closely bound to a state.Info, or is completely abstracted from it
<davecheney> either way, the requirement to pass state.Info in StartInstance will go away
<fwereade> rogpeppe, davecheney: yeah, that param definitely feels wrong to me
<davecheney> fwereade: rogpeppe i'm guessing it's there to pass details to the machine agent on how it can find the ZK and bootstrap itself so to speak ?
<rogpeppe> davecheney: yes, at least that. i'm trying to remember the other scenarios.
<davecheney> or to say it another way, how it can build and Environ that matches that of the Provisioning Agent that created it
<fwereade> davecheney, rogpeppe: sorry, I'm confused... doesn't the environ already know its own state info?
<davecheney> fwereade: yes, it must, because there is this method
<davecheney> Environ.StateInfo
<rogpeppe> fwereade: only if you've bootstrapped it.
<davecheney> rogpeppe: i'm happy with that restriction, the PA doesnt' run til bootstrapping is complete, unless i'm mistaken
<rogpeppe> fwereade: but you could reasonably have an environ that you hadn't bootstrapped, but were using with a provisioning agent and a state from somewhere else, i think.
<fwereade> rogpeppe, ok, but if you haven't bootstrapped it you don't have a state: you just know that there *will* be one on localhost once bootstrap is complete
<davecheney> rogpeppe: ah, i see what i'm missing
<davecheney> bootstrapping uses Environ.StartInstance to start machine(0)
<rogpeppe> davecheney: actually, it uses startInstance; slightly different.
<davecheney> still, possibly a strategic nil could be passed, suggesting, use whatever state you have internally
<rogpeppe> part of the problem is we don't want to pass the ec2 secret key in the cloudinit script
<fwereade> davecheney, rogpeppe: it feels to me like bootstrap is the special case, and it's misleading to expose those details to the outside world
<davecheney> i concur
<rogpeppe> hmm, i think it feels quite clean actually. we explicitly tell the new machine how to connect to the juju state.
<rogpeppe> it's always going to need to do that, i think.
<fwereade> rogpeppe, but the environ is so closely tied to the state that it feels wrong to have to tell it again
<rogpeppe> fwereade: i don't see the coupling as that close actually.
<fwereade> rogpeppe, Environ.StateInfo implies otherwise, to me
<rogpeppe> fwereade: i like the idea of being able to have the machines in a single environ controlled by multiple states, for example.
<fwereade> rogpeppe, sure, that has potential: but I'm pretty sure that's not what we're implementing right now :p
<fwereade> rogpeppe, would you say that Environ.StateInfo was a misstep that we should remove?
<davecheney> fwereade: rogpeppe it appears to me that if the environ has a StateInfo method, then it knows it's own state, so StartInstance shuldn't need to be told a stateinfo
<fwereade> davecheney, exactly so
<davecheney> on the other hand, if the envion shuldn't need to know about the stae, then the StateInfo method is misleading
<rogpeppe> davecheney: no, i don't think an environ does know its own state
<davecheney> as it stands, it's not a big deal to do si, _ := environ.StateInfo(); environ.StartInstance(99, si)
<rogpeppe> davecheney: the private keys are not accessible to an environ for example
<davecheney> rogpeppe: then what is returned from environ.StateInfo() is not actually useful then ?
<davecheney> oh, hang on
<davecheney> i'm talking about stateInfo, which is just a pretty connection string to find zookeepers
<fwereade> davecheney, I must have missed something; how does that change things?
<davecheney> fwereade: 22:12 < rogpeppe> davecheney: the private keys are not accessible to an environ for example
 * rogpeppe is thinking
<davecheney> ^ i interprited that to mean more than it did
<fwereade> davecheney, ah, cool, thanks
<rogpeppe> davecheney: yeah, and i'm not sure i meant what i said.
<davecheney> fwereade: rogpeppe i'm sure this is over simplifying this, but for the privisioning, machine and unit agents
<davecheney> to get up and running
<davecheney> all then need to know is the argument they are passed to jujud, and a stateinfo, which is really just a string passed to --zookeeper-servers
<fwereade> davecheney, yes, essentially
<davecheney> and so, Environ.StartInstance() is shorthand for 'start a new machine, and run this daemon passing these arguments'
<fwereade> davecheney, the PA also needs to know the environ, and the UA/MA need to know what U/M they's acting for
<davecheney> yeah, there is clearly more too it than I just glossed over
<davecheney> but from the point of view of 'how do I find zookeeper so I can watch stuff'
<rogpeppe> davecheney: one issue is the possibility that we need a different internal vs external IP address. i'm not *sure* that the environ is in a good place to know which to use.
<fwereade> davecheney, sure, understood
 * fwereade thinks
<davecheney> rogpeppe: for the short term (read ec2 and workalikes) I think we can use the assumption that all machiens in the same environment areusing the same network
<rogpeppe> davecheney: what about the juju client?
<fwereade> davecheney, it still seems to me that the fact that there's a different ZK address is an incredibly minor aspect of the things that are different when starting the first node
<davecheney> rogpeppe: good point, i don't have an answer to that, other than saying we probably need two state.Info's
<davecheney> external and internal
<davecheney> or something that clearly identifies external (as in not inside the environment) and internal (that are)
<davecheney> otherwise we'll go mad with the permutations
<rogpeppe> davecheney: how does a machine know if it's internal or external?
<davecheney> jujuc == external, jujud == internal
<davecheney> as a start
<fwereade> davecheney, so it feels wrong to push that one specific difference up to that level while leaving the others implicit
<fwereade> davecheney, wait, what?
<davecheney> but trying to make one connectino string serve two masters sounds worse
<davecheney> rogpeppe: as I understand it, all machines are internal; the are inside the environment
<rogpeppe> currently StateInfo returns the external address.
<davecheney> if ec2 lets that work, then it's a solved problem for now
<rogpeppe> davecheney: the difficulty is that Environ is used by the client as well as the cloud machines
<rogpeppe> davecheney: i think the connection methods are different too. i'm not sure we use ssh between cloud machines.
<davecheney> ahh, so the reason StartInstance takes a state.Info is to allow us to construct an 'internal' connection string
<rogpeppe> davecheney: that's certainly one reason, yes.
<fwereade> davecheney, rogpeppe: hmm... except when bootstrapping, the state info comes from an Instance, right?
<fwereade> davecheney, rogpeppe: which already knows both internal and external addresses
<rogpeppe> fwereade: no, it comes from the private storage
<fwereade> rogpeppe, how did it get there?
<rogpeppe> fwereade: it was put there at bootstrap time.
<fwereade> rogpeppe, if not from the Instance constructed on bootstrap
<rogpeppe> fwereade: at that time we don't know the DNS addresses of the instance.
<davecheney> rogpeppe: fwereade reviewing state/open.go, State info has a slice of addresses to connect to, is that what you mean by internal and external ?
<fwereade> rogpeppe, wait, you just said the information was put in provider storage at bootstrap time
<rogpeppe> fwereade: the instance id only
<fwereade> davecheney, no, the slice is all ZKs; internal/external is just about what address we use to connect to a ZK or group thereof
<rogpeppe> so... if i understand the general idea, we could have ...
<fwereade> rogpeppe, ok, and the instance id is used at some point to get an Instance, from which we get the public address and discard the private?
<rogpeppe> one mo, tel call
<fwereade> rogpeppe, ...and we're now discussing how we go about reacquiring the private address?
<rogpeppe> fwereade: yeah
<fwereade> rogpeppe, it seems to me that the trouble is in throwing away half the information in the first place
<rogpeppe> so... if i understand the general idea, we could have a "local" argument to StateInfo which determines which info to get.
<rogpeppe> 'cos we can't get rid of StateInfo entirely, right?
<rogpeppe> so we're suggesting that StartInstance call StateInfo(local) inside itself, rather than taking an explicit state.Info arg?
<fwereade> rogpeppe, perhaps something like that: I haven't thought it through entirely
<davecheney> rogpeppe: i'd like to try that
<fwereade> rogpeppe, but it feels like a potentially fruitful avenue
<rogpeppe> i don't really see the point. i don't think it'll make anything significantly simpler
<davecheney> or at least have StartInstance(314, nil) as a sentinal for 'pass whatever state.Info you know/makes sense/sounds best'
<davecheney> rogpeppe: as it sounds like the reason StartInstance takes a state.Info is to facilitate bootstrapping
<fwereade> davecheney, doesn't StartInstance imply non-bootstrap in the first place?
 * davecheney shrugs
<rogpeppe> fwereade: yes
<fwereade> davecheney, rogpeppe: I'm not bothered what we pass to startInstance, but I'm concerned about dirtying up StartInstance when in practice it will *always* be called with nil... won;t it?
<fwereade> davecheney, rogpeppe: ... or  .StateInfo
<fwereade> davecheney, rogpeppe: depending on how we do the details
<rogpeppe> fwereade: it means that StartInstance will have to read the provider local storage each time, i think.
<davecheney> fwereade: or to put it another way, if the PA has to call StartInstance when it sees a machine appear in the topology, what is it supposed to use for that argument
<davecheney> at this moment the only thing it can use is the value from a.Conf.StateInfo
<davecheney> which comes from the jujud super command framework
<fwereade> davecheney, rogpeppe: how does the same argument not apply to Environ.StateInfo?
<davecheney> fwereade: that is an intersting one
<rogpeppe> fwereade: if the caller already has a StateInfo, it can pass that to StartInstance each time.
<davecheney> the PA is connected to a state.State
<davecheney> it find a *ConfigNode by watching /enviornment, and passes that to environs.NewEnviron
<davecheney> and gets an Environ
<rogpeppe> if at some point we want to be able to adjust the set of database machines at runtime, it will be good to be able to do that through the existing state.
<davecheney> what that environ.StateInfo() returns is a damn good question
<rogpeppe> davecheney: the bucket will be configured in the attributes it passes to SetConfig
<rogpeppe> davecheney: and that's what StateInfo will use to return the state.Info
<fwereade> rogpeppe, we surely will at some stage... in which case surely the right answer *is* to read it every time
<fwereade> rogpeppe, but this seems essentially orthogonal to the question of whether it needs to be passed to StartInstance
<rogpeppe> fwereade: i'm not sure. i think the provider storage is good for bootstrap, but we can use other means when we're up and running.
<rogpeppe> fwereade: bootstrap and client connections, of course.
<rogpeppe> fwereade: there's no way of *watching* provider storage, i think
<fwereade> rogpeppe, I thought you said bootstrap didn't use StartInstance
<fwereade> rogpeppe, davecheney: oh hell, EOD, got to go to the bank :(
<rogpeppe> BTW the provider state is written *after* the bootstrap node is successfully started.
<fwereade> rogpeppe, davecheney: hope I wasn't just muddying the waters for you
<rogpeppe> fwereade: no, it's all good stuff
<fwereade> rogpeppe, I know; don't get current significance though
<davecheney> +1
<rogpeppe> fwereade: it means that the bootstrap node doesn't necessarily have access to the provider state when it starts.
<rogpeppe> of course in practice with ec2, it's so slow that it will in fact.
<fwereade> rogpeppe, but we bake the stateinfo into cloud-init regardless
<rogpeppe> fwereade: we can't do that if we can't pass it in to StartInstance
<rogpeppe> fwereade: because StartInstance will, of necessity, retrieve the state info from the provider storage bucket
<fwereade> rogpeppe, but the environ *knows* it's bootstrapping, and it fully controls the cloud-init, and it *knows* it's starting a ZK and relevant agents on that instance, and hence it *knows* that it can use localhost
<fwereade> rogpeppe, and did I misunderstand about StartInstance not actually being used to bootstrap anyway?
<rogpeppe> fwereade: i'm not sure that the Environ knows that it's on a bootstrap node.
<davecheney> if I can interrupt for a moment
<rogpeppe> davecheney: please do
 * fwereade listens
<davecheney> is the reason we're saying 'we can't just let the Environ pass what it' thinks is it's state info to any new machine via StartInstance' is because it might have 'localhost' as its zk host ?
<rogpeppe> davecheney: that's not the only reason.
<rogpeppe> davecheney: the reason we were just talking about was that there's more than one source for a StateInfo
<rogpeppe> davecheney: it might come from the provider storage; or it might come from being passed into the cloudinit script.
<davecheney> rogpeppe: but from the POV of the PA, there is only one stateinfo
<davecheney> their are many, but this one belongs to the PA
<rogpeppe> davecheney: yeah, the one that's passed on the command line
<fwereade> davecheney, rogpeppe: I do actually have to go now; ttyl :)
<rogpeppe> fwereade: k
<davecheney> kk
<rogpeppe> fwereade: c u tomorrow
 * rogpeppe also has to go to lunch soon
<davecheney> fair enough, lets wrap up
<davecheney> from what i've read today, the best course of action is to not change naything
<davecheney> the PA calls environ.StartInstance(999, a.Conf.StateInfo)
<davecheney> we'll revisit it later, if necessary
<rogpeppe> davecheney: i *think* that's good. i'm open to the idea of a change, but in this case i'm not sure it helps much.
<rogpeppe> davecheney: passing in StateInfo explicitly means the environ doesn't need to care how we got it.
<rogpeppe> davecheney: which i *think* is a reasonable separation of concerns. but please keep on considering alternatives!
<rogpeppe> right, lunch!
<rogpeppe> davecheney: see you tomorrow, i hope, if i get up early enough.
<davecheney> rogpeppe: kk, i'll catch you on the next rotation
<andrewsmedina> rogpeppe: hi
<andrewsmedina> rogpeppe: I sent another review
<rogpeppe> andrewsmedina: thanks. i'll take a look.
 * niemeyer waves
<rogpeppe> niemeyer: yo!
<rogpeppe> andrewsmedina: LGTM
<andrewsmedina> rogpeppe: thanks \o/
<rogpeppe> niemeyer: i updated https://codereview.appspot.com/6188068/ to unexport the types and change the docs. hopefully should be better now.
<niemeyer> rogpeppe: Cool, thanks
<niemeyer> rogpeppe: Agreed regarding it going out of the ec2 package eventually
<niemeyer> rogpeppe: We don't have to do that now, but let's just watch out to lead the code towards making them sensible there (it's already going in that direction it seems)
<rogpeppe> niemeyer: will do.
<TheMue> niemeyer: Moin and thx for the reviews.
<niemeyer> rogpeppe: It actually feels like going backwards a bit
<rogpeppe> niemeyer: oh?
<niemeyer> rogpeppe: All those field name changes will have to go back again in the next branch that does the actual switch over
<niemeyer> rogpeppe: The way we have colseries and colServer also feels quite awkward
<niemeyer> rogpeppe: These names should be internally consistent
<rogpeppe> niemeyer: i'm not sure. it's only the instance contraint that will have to change
<rogpeppe> niemeyer: colseries is my mistake - a global replace too enthusiastic!
<niemeyer> rogpeppe: Ah, phew
<niemeyer> TheMue: Heya!
<niemeyer> TheMue: np
<rogpeppe> niemeyer: i think that instanceSpec will remain private
<rogpeppe> niemeyer: and instanceConstraint will be significantly different
<niemeyer> rogpeppe: I'm tempted to suggest moving the InstanceConstraints to environs right now
<rogpeppe> niemeyer: so i don't mind it being private now
<niemeyer> rogpeppe: If it will be significantly different, we're doing it wrong
<rogpeppe> niemeyer: can we leave it as it is - i've got three upstream branches that i'm juggling on top of this one
<rogpeppe> niemeyer: then i can do the constraints stuff later
<niemeyer> rogpeppe: Yep, sure
<niemeyer> rogpeppe: Done
<rogpeppe> niemeyer: thanks a lot
<niemeyer> rogpeppe: np
<rogpeppe> niemeyer: remind me again how i can get the full revision id for the current branch out of bzr, please.
<rogpeppe> niemeyer: my google-fu has deserted me
<niemeyer> rogpeppe: revision-info should do
<rogpeppe> ah!
<rogpeppe> niemeyer: i was trying revno and info
<niemeyer> rogpeppe: revision-info --tree is relevant in some cases
<rogpeppe> revno -v should do it, i reckon
<smaddock> bcsaller: ping
<twobottux> aujuju: Juju error: Server refused to accept client <http://askubuntu.com/questions/141687/juju-error-server-refused-to-accept-client>
<niemeyer> Lunch, biab
<niemeyer> TheMue: ping
<TheMue> niemeyer: pong
<niemeyer> TheMue: yo
<niemeyer> TheMue: Would you mind if we at least delayed the addition of that method?
<niemeyer> TheMue: It feels like an awkward interface to me.. I can't see what this method is offering that we can't do with the other methods we have
<TheMue> niemeyer: So far I've got no problem. The original is used two or three times (beside the tests).
<niemeyer> TheMue: So it'd be awesome if we could keep that out while we figure the answer for that
<TheMue> niemeyer: So I'll see when I reach this point.
<niemeyer> TheMue: I'm sure it's used, but I think we have a better interface that may enable its original use case withou tit
<andrewsmedina> niemeyer: https://codereview.appspot.com/6099051/
<niemeyer> andrewsmedina: Neat, will give it a final pass
<TheMue> niemeyer: OK, so I'll kick it with the next proposal.
<niemeyer> TheMue: Cheers, please ping me once you're done.. I suspect it's ready for going in, and will check as soon as you push
<niemeyer> andrewsmedina: Why network.Subnet has no xml tag?
<andrewsmedina> niemeyer: its only used to create a new net
<andrewsmedina> niemeyer: this attribute is not returned in the xml generated by virsh
<niemeyer> andrewsmedina: Cool
<niemeyer> andrewsmedina: The tests will fail without root, right?
<andrewsmedina> niemeyer: wrong
<niemeyer> andrewsmedina: Ah, you're faking the whole thing.. hmmm
<niemeyer> andrewsmedina: That's both nice, and not'''
<andrewsmedina> niemeyer: yes...
<niemeyer> andrewsmedina: Either way, we can move forward like that and address it later
<niemeyer> andrewsmedina: It's better then the opposite I guess
<andrewsmedina> niemeyer: but it was the only way that I found not to change the network configuration of the machine on which the tests have been run
<niemeyer> andrewsmedina: Yeah, that's what I mean.. your approach is better than being *just* on the other side of the fence
<niemeyer> andrewsmedina: At some point in the future, once that's all working well, we can add some extra tests to *optionally* guarantee that it works for real
<niemeyer> andrewsmedina: Or maybe not.. external functional tests would deal with that too
<niemeyer> andrewsmedina: Either way, you have a review with a few trivialities, and LGTM
<niemeyer> andrewsmedina: Please ping me once these are done and I'll submit
<niemeyer> andrewsmedina: (rog had a trivial too)
<andrewsmedina> niemeyer: ok
<andrewsmedina> niemeyer: what trivial rog had?
<niemeyer> andrewsmedina: It's in the CL
<andrewsmedina> where?
<niemeyer> andrewsmedina: https://codereview.appspot.com/6099051/diff/15001/environs/local/network.go#newcode92
<niemeyer> andrewsmedina: Also part of the LGTM mail he sent
<andrewsmedina> niemeyer: I will work on these issues
<andrewsmedina> brb
<TheMue> niemeyer: ping, propose is in.
 * hazmat lunches
<rogpeppe> niemeyer: this should be trivial: https://codereview.appspot.com/6257043/
<niemeyer> TheMue: Cheers
<niemeyer> rogpeppe: Cool
<rogpeppe> niemeyer: as should this: https://codereview.appspot.com/6249049/
<rogpeppe> niemeyer: and the one after that is the important one... just doing final amazon test now before proposing
<niemeyer> rogpeppe: LGTM on first
<rogpeppe> niemeyer: thanks a lot
<niemeyer> rogpeppe: LGTM on second too, thanks
<niemeyer> TheMue: Going through it now
<rogpeppe> niemeyer: lovely jubbly, ta
<TheMue> niemeyer: great, thx
<rogpeppe> niemeyer: here's the boy: https://codereview.appspot.com/6245048
<rogpeppe> niemeyer: i tried to split it up more, but it wasn't having any of it (too much interdependency)
<rogpeppe> niemeyer: if it's any consolation, the code ends up 200 lines *shorter* after this CL...
<rogpeppe> niemeyer: gotta go now.
<niemeyer> rogpeppe: Super, thanks a lot
<rogpeppe> niemeyer: i'll be quite excited if this goes in
<rogpeppe> niemeyer: see ya tomorrow
<niemeyer> rogpeppe: Well it will certainly go in
<niemeyer> rogpeppe: Cheers
<rogpeppe> niemeyer: well, you might say it's too big or just wrong or something :-)
<niemeyer> rogpeppe: I can't anticipate that, but the concept of the feature we want for sure
<niemeyer> TheMue: Sorry, ended up sidelined by other conversations.. back to the review
<TheMue> niemeyer: np
<niemeyer> TheMue: "It's allowed today in endpoint.py. Maybe it's an error there too?"
<niemeyer> TheMue: I believe it is.. can you please drop it so that we can be sure about what's going in?  If we find issues, it'll be a nice insight
<TheMue> niemeyer: Ok, will be dropped.
<niemeyer> TheMue: "// /topology node in ZooKeeper.s"
<niemeyer> TheMue: Extra "s" at the end, topology.go line 46
<niemeyer> TheMue: LGTM with these sorted. Thank you!
<niemeyer> TheMue: Review delivered for the record too
<TheMue> niemeyer: Thanks. The s may be from saving without ctrl. ;)
<niemeyer> TheMue: Hehe, I know how these go :)
<TheMue> niemeyer: Sh.., had to leave computer, "accident" here. My wife lost her earring into the plughole. *sigh*
<niemeyer> TheMue: Wow, what/
<niemeyer> ?
<niemeyer> TheMue: Oh, ok
<niemeyer> TheMue: Did you pick it up?
<niemeyer> robbiew: Is it meeting time?
<robbiew> yesss!...one sec
<robbiew> g+ invite sent
<niemeyer> Cheers
<TheMue> niemeyer: So, final propose before LGTM. And the earring has to wait, I had to bring my daughter to before. Tasks when the kids grow. ;)
<niemeyer> TheMue: Cool :)
<niemeyer> TheMue: With robbiew, will check it right afterwards
<TheMue> niemeyer: Fine, so I can submit it later or as my first task tomorrow morning.
<niemeyer> TheMue: LGTM
<TheMue> niemeyer: Great, thx. A good step for the end of the day.
<niemeyer> TheMue: +1, very happy to see this going n
<niemeyer> in
<TheMue> niemeyer: Me too.
 * hazmat likes the new golang book
<hazmat> the mark summerfield one
<niemeyer> hazmat: Good to hear
<niemeyer> hazmat: Haven't checked it out yet
<hazmat> niemeyer, i've read some of his others pyqt.. etc.. he's a good author
<niemeyer> davecheney: Good morning
<niemeyer> hazmat: Yeah, haven't read those, but I did see his name around books for a while
<davecheney> niemeyer: morning
<niemeyer> davecheney: Had to put https://codereview.appspot.com/6229046/ back to WIP unfortunately.. a race was introduced in the latest change
<davecheney> yup
<davecheney> i'll address that now
<niemeyer> I'm heading off to dinner.. will be back later to review rog's latest goodness
<davecheney> ok
<robbiew> davecheney: ping
<andrewsmedina> niemeyer: I sent the new proposal
<davecheney> robbiew: ack
<davecheney> i should be on skype
<robbiew> davecheney: cool...one sec
<davecheney> niemeyer: out for review: https://codereview.appspot.com/6244053
#juju-dev 2012-05-25
<davecheney> niemeyer: new revision of SetConfig incoming
<davecheney> you may want to see more caching
<davecheney> niemeyer: ready for review, https://codereview.appspot.com/6229046
<niemeyer> davecheney: Review on first one delivered
<niemeyer> davecheney: "Patch set 8 incorporates https://codereview.appspot.com/6244053."
<niemeyer> davecheney: I guess it shouldn't?  Otherwise what's the point of having the other CL?
<davecheney> niemeyer: that was a cryptic way of saying i rolled that change up into setconfig
<davecheney> but given your comments on the makeBucket branhc
<davecheney> that will turn out to be a bigger piece of work
<niemeyer> davecheney: The comment remains.. what's the point of having a second CL if it's included in another one
<davecheney> i'm finding it hard to manage branches that have multiple parents
<davecheney> in the future I won't merge the like patch #8
<davecheney> i'll wait til the upstream passes review
<niemeyer> davecheney: Ideally things are either independent, or you use -pre-req
<niemeyer> davecheney: Yeah, that's a good plan in general
<davecheney> so, talking about https://codereview.appspot.com/6244053 for a second
<davecheney> you belive this is a bug introduced recently
<davecheney> ?
<niemeyer> Ok
<niemeyer> davecheney: Yeah, definitely.. this logic was worked on by rog last week or so
<davecheney> ah, ok
<davecheney> i'll review the commit history and try a different approach
<davecheney> i'll put SetConfig back into WIP
<niemeyer> davecheney: I'm having a look at it just now
<davecheney> if you could give it a cursory view that would be good
<niemeyer> davecheney: I'm a bit unhappy with the strategy that is spreading through there
<niemeyer> return &storage{bucket: e.s3().Bucket(config.bucket)}
<niemeyer> davecheney: This is creating a storage+S3+bucket every single time we need to use the bucket
<niemeyer> davecheney: Imagine what happens in a trivial loop like this:
<niemeyer> for {
<niemeyer>     ... whatever ...
<davecheney> yup, understood
<niemeyer>     environ.PutFile(file)
<niemeyer> davecheney: *tons* of garbage
<davecheney> I am not overly concerned with performance at this stage
<davecheney> as this is talking to a network service
<davecheney> but I will add caching
<niemeyer> davecheney: For no great reason.. we got into this situation after trying to avoid the caching, which wasn't a big deal at all
<davecheney> understood
<niemeyer> davecheney: I'm not *concerned* either
<niemeyer> davecheney: I just feel bad that we're giving up on both simplicity *and* performance at the same time
<davecheney> i'm trying for something that is correct with respect to SetConfig
<davecheney> i hope that caching could be added later
<davecheney> going back to makeBucket for a second, 'cos that is now a blocker
<davecheney> I can't see there the bug was introduced
<davecheney> not in the last week at least
<niemeyer> davecheney: THe makeBucket issue was created while avoiding the caching as well
<niemeyer> davecheney: Well, some of the issues, maybe not makeBucket
<niemeyer> davecheney: The race from the last review, for example
<davecheney> niemeyer: i'm concerned that making the environs reloadable is becoming a blocker
<davecheney> I understand it is important for the final product
<davecheney> but would it be possible to propose a provisioning agent that doesn't reload it's configuration after starting
<niemeyer> davecheney: No, we have spent a lot of time on this, let's not throw all the work and discussion now that it's pretty much done
<niemeyer> davecheney: Hindsight is good, but let's not ignore what was done now that we have it
<davecheney> i am concerned that the adding reloadability to environs is a moving target
<davecheney> i'm wondering if it would be better to address then after we have a PA that we can use to bring up a trial juju on ec2
<niemeyer> davecheney: It wasn't a moving target.. I suggested not changing caching, for example, because it worked, and then you found it a better approach.. that broke things further.. and that had to be fixed.. and ...
<davecheney> exactly
<davecheney> so lets put all that to once side
<davecheney> one side
<niemeyer> davecheney: Yeah, let's *fix the problems* and not change further
<niemeyer> davecheney: If you're feeling overwhelmed, I can do that myself
<davecheney> niemeyer: not overwhelmed, just concerned that all the changes that the PA needs to environs/Environ keeps pushing back the PA itself
<niemeyer> davecheney: Yeah, so ignore the madeBucket stuff, if that's what you mean
<niemeyer> davecheney: Just drop it, and we'll put on someone else's todo list
<niemeyer> davecheney: That sounds fine
<davecheney> i'm suggesting a middle ground of delivering a PA that doesnt' try to respond to changes in /environment, so has no requirement for Environ.SetConfig
<niemeyer> davecheney: But I do want to see caching added back, and all that extra stuff removed
<niemeyer> davecheney: Because that's part of the "never ending roll"
<davecheney> niemeyer: yes, i want to get out of this spiral
<davecheney> and I can do that if the PA can skip having to reload it's environ once it's up and running
<davecheney> then when the rest of the PA is working
<niemeyer> davecheney: Again, you spent days on this, and I spend hours reviewing it.. I'm not happy with trashing both your and my time
<davecheney> reloading environ configs can be tackled as seperate branch
<davecheney> i'm not saying throw it awwy
<davecheney> just put it on hold for a while
<niemeyer> davecheney: That doesn't work.. if it stays out, it will be broken next week
<niemeyer> davecheney: Let's fix it, and move on
<davecheney> ok, understood
<niemeyer> davecheney: Happy to drop make/madeBucket, though
<niemeyer> davecheney: Didn't happen in this branch..
<davecheney> ok, so to review, I will continue to work on SetConfig. I will restore all the cached values like ec3Unlocked and friends
<davecheney> and makes changes to environ.config atomic with changes to {public,}storage.bucket
<davecheney> /s/ec3/ec2/
<niemeyer> davecheney: Heh, madeBucket was born broken
<davecheney> niemeyer: thought so
<niemeyer> davecheney: Sorry about that
<niemeyer> davecheney: I suspect the second problem is a NOOP once you deal with the first
<niemeyer> davecheney: You got into the second issue because replacing the bucket was done with s3(), which needed a lock
<niemeyer> davecheney: Once you have the cached values, it's all done within the lock
<davecheney> yup, that'll work
<davecheney> seriously, i should stop listening to roger's advice
<davecheney> that was his suggestion :)
<davecheney> anwywya, i have to go
<niemeyer> LOL
<niemeyer> Rog have some great advices, though
<niemeyer> ''''''''''''''''has
<davecheney> i know, it's hard to sift one from the other
<davecheney> anyway, i have to go
<niemeyer> davecheney: Cheers
<davecheney> will submit another round tonihgt
<davecheney> thank you for your time
<niemeyer> davecheney: My pleasure
<andrewsmedina> niemeyer: are you working 24/7 ?
<andrewsmedina> :-p
<niemeyer> andrewsmedina: Nah.. :-)
<andrewsmedina> :D
<andrewsmedina> niemeyer: I submitted a new proposal
<niemeyer> andrewsmedina: Thanks!
<niemeyer> andrewsmedina: This was probably the last one.. with those minors, it should be good to go in
<andrewsmedina> niemeyer: thanks :D
<andrewsmedina> niemeyer: Rodrigo Senra will be here tomorrow
<niemeyer> andrewsmedina: Wow, really?
<niemeyer> andrewsmedina: This is awesome.. small world
<niemeyer> andrewsmedina: Please deliver him a hug from me :)
<andrewsmedina> niemeyer: ok, I will
<niemeyer> andrewsmedina: aaaaand your branhc is going in
<andrewsmedina> niemeyer: nice!
<andrewsmedina> niemeyer: thanks
<niemeyer> andrewsmedina: Thank *you* for keeping up with it
<andrewsmedina> niemeyer: now I will work on local provider environ interface =D
<niemeyer> andrewsmedina: Sweeeeet
<niemeyer> andrewsmedina: Small increments is the key
<niemeyer> andrewsmedina: Don't feel bad to push half baked stuff
<niemeyer> andrewsmedina: As long as it's a correct step forward, it's fine to go in
<niemeyer> andrewsmedina: E.g. a type that implements all interface methods returning errors in all of them is a correct step forward
<niemeyer> andrewsmedina: the implementation of a single one of these methods is a correct step forward to
<niemeyer> too
<niemeyer> error: Failed to load data for branch bzr+ssh://bazaar.launchpad.net/~andrewsmedina/juju/go-local-network/: Get https://api.launchpad.net/devel/branches?ws.op=getByUrl&url=bzr%2Bssh%3A%2F%2Fbazaar.launchpad.net%2F~andrewsmedina%2Fjuju%2Fgo-local-network%2F: dial tcp 91.189.89.224:443: connection timed out
<niemeyer> error: Failed to load data for bran
<niemeyer> Ah, dave isn't here
<andrewsmedina> niemeyer: Why this error??
<niemeyer> andrewsmedina: NEvermind.. I was going to point to Dave that it wasn't just on him that it was blowing up :)
<andrewsmedina> niemeyer: I will sent small proposes
<niemeyer> andrewsmedina: Cheers!
<niemeyer> andrewsmedina: It's in!
<niemeyer> and the queue is zeroed!
<niemeyer> Ah, the feeling of mission accomplished.. well, at least for a few hours.
<niemeyer> Will take the chance to have a nice sleep
<niemeyer> Night all!
<andrewsmedina> niemeyer: good night!
<davecheney> rogpeppe: whoop whoop!
<rogpeppe> davecheney: haroo!
<fwereade> davecheney, rogpeppe: heyhey
<rogpeppe> fwereade: yo!
<fwereade> davecheney, btw, what awful time of day is it for you?
<davecheney> fwereade: 7pm
<davecheney> rogpeppe: this is what i'm thinking http://paste.ubuntu.com/1006126/
<fwereade> davecheney, ok, not so awful then :)
<rogpeppe> fwereade: you might want to fix this bug BTW: cmd/jujuc/server/util_test.go:37: not enough arguments in call to st.AddCharm
<davecheney> not aweful at all
<fwereade> rogpeppe, oof, thanks
<davecheney> s/awe/aw/g
<rogpeppe> davecheney: that won't work, sadly
<davecheney> poop, why not
<rogpeppe> davecheney: look at the implementation of deleteAll
 * davecheney looks
<rogpeppe> davecheney: it wouldn't be good to have the bucket changing underfoot during the execution of that method
<rogpeppe> davecheney: hence my "immutable" suggestion.
<rogpeppe> davecheney: i thought that's what you meant when you said it was "a bit harder" BTW
<davecheney> rogpeppe: good point, but consider why the bucket would be changing
<rogpeppe> davecheney: if we change the bucket, does it make sense to have an ongoing delete method suddenly start to remove objects that were listed in one bucket from another bucket?
 * davecheney thinks
<rogpeppe> davecheney: i think that an "immutable" method gives a nice understandable semantic - if you've started an operation with one set of settings, it will complete with the same.
<davecheney> rogpeppe: if only it were that easy, for environ.SetConfig doesn't create a new storage, but alters it's bucket value
<davecheney> however, that can be fixed
<rogpeppe> davecheney: that's fine, i hadn't intended that it did
<fwereade> rogpeppe, trivial: http://paste.ubuntu.com/1006133/
<rogpeppe> davecheney: did you look at my immutable method?
<davecheney> yup, you look to be taking another reference to the *storage
<rogpeppe> fwereade: oh cool, i thought maybe it needed a real sha hash
<fwereade> rogpeppe, it may do at some stage
<rogpeppe> davecheney: no, i'm copying the contents of the storage.
<fwereade> rogpeppe, but we're not at that stage yet :)
<rogpeppe> davecheney: well, the contents of the storage struct anyway
 * davecheney reads again
<davecheney> 7~oooooooooh
<davecheney> wow, i never considered that
<davecheney> mainly 'cos i've never used a lanugage that could do that
<rogpeppe> davecheney: it does mean we've got another allocation each time, but i think that's irrelevant, as mentioned earlier
<davecheney> i agree about allocations
<davecheney> i don't think allocation counting is important when we're talking to a slow arse ec2 provisioning service
<rogpeppe> davecheney: and actually i think probably the compiler can tell that the value never escapes, so it might not even do one.
<rogpeppe> davecheney: absolutely
<rogpeppe> davecheney: particular when PutReader allocates 1700 times!
<rogpeppe> s/ar/arly
<davecheney> yeah, but gustavo had a point that I was touching too much code
<davecheney> rogpeppe: what if the exported Storage/PublicStorage methods always returned an immutable *storage ?
<davecheney> then DeleteAll would just work
<fwereade> rogpeppe, btw, can I just submit that directly please?
<davecheney> hmm, that might need a little more finesse, deleteAll isn't part of the public Storage API
<rogpeppe> fwereade:  i think you should make a CL, i'll LGTM it and you submit it immediately. that way the commit log looks consistent.
<fwereade> rogpeppe, SGTM
<rogpeppe> davecheney: i don't think that would work
<davecheney> rogpeppe: would you say that deleteAll is the exception that breaks the rule
<rogpeppe> davecheney: because there's nothing stopping someone holding on to a Storage object, but i guess we want it to change when the settings change.
<rogpeppe> davecheney: ... or maybe not. i wonder.
<davecheney> rogpeppe: there are arguments for an against
<fwereade> rogpeppe, https://codereview.appspot.com/6243054
<rogpeppe> fwereade: you've checked that all tests pass?
<fwereade> rogpeppe, heh, I'll do a full run through
<rogpeppe> fwereade: i think it's worth it :-)
<davecheney> rogpeppe: which do you think gustavo had in mind ?
<fwereade> rogpeppe, quite so :)
<rogpeppe> davecheney: i dunno. as you say, arguments for and against.
<davecheney> if we call Environ.SetConfig, should that apply to any Storage that was requested, or should the Storage you previously asked for remain valid ?
<davecheney> given that the reason for calling SetConfig is most likely to update some credentials
<rogpeppe> davecheney: exactly. that's the question
<davecheney> it's likely that your previous Storage would be inoperable
<fwereade> rogpeppe, huh, strange things are going on elsewhere; I'll poke around and see if I can figure out what's up
<davecheney> actually, SetConfig can _only_ change credentials, or the bucket
<davecheney> as those are the only configuration values consumed
<rogpeppe> davecheney: if we're changing credentials, to avoid screwing things up, we'll have to change the credentials *before* deprecating the old credentials
<davecheney> rogpeppe: how about this, SetConfig never changes the value of storage.bucket, that is immutable once created
<rogpeppe> davecheney: that won't work if we want a Storage to work long term
<davecheney> it simply replaces (and safely publishes) new storage and publicStorage references
<davecheney> right, so then the opposite is the better chioce
<davecheney> the mutex guards storage.bucket, and changes to that are visible immediately
<rogpeppe> davecheney: i think the crux is deleteAll - do we treat that as an "external" method, so the bucket can change underfoot as it's progressing?
<rogpeppe> davecheney: what i'm concerned about is someone changing not just the security credentials (but leaving them pointing to the same original AWS account) but the name of the bucket too.
<rogpeppe> s/(but /c/(/
<davecheney> so, what if I implement 'immutable' inside deleteAll, as it is the special case
<davecheney> or at least use it in just the deleteAll case
<rogpeppe> davecheney: tbh i think we might be better for all concerned if we make storage not change with SetConfig
<davecheney> rogpeppe: i would be more than happy with that
<rogpeppe> davecheney: and we can document that SetConfig works only on the Environ and not on storage objects returned from it
<davecheney> what is your feeling about gustavo's view on this ?
<rogpeppe> davecheney: then anything that calls Storage gets a nice coherent view
<rogpeppe> davecheney: i *think* he'd be ok with that.
<davecheney> i've seen you raise the point a few times
<rogpeppe> davecheney: i have mixed feelings - there are trade-offs all over
<TheMue> morning
<rogpeppe> davecheney: but my current feeling is that this is the sweet spot
<davecheney> hey
<rogpeppe> TheMue: hiya!
<davecheney> rogpeppe: i'm happy to float that
<davecheney> rogpeppe: in your extimation, who are these long term holders of Storage references ?
<rogpeppe> davecheney: i don't think there are any
<rogpeppe> davecheney: that's the reason for my current thoughts
<davecheney> rogpeppe: then why suggest that storages' not change with SetCOnfig ?
<rogpeppe> davecheney: because even though there are no long term holders of Storage references, there will be *concurrent* holders of Storage references
<rogpeppe> davecheney: so we can provide them with a coherent view
<rogpeppe> davecheney: by letting each call to Storage return a new *storage instance.
<davecheney> rogpeppe: yup, i'd support that
<rogpeppe> davecheney: in fact it's easier than that
<rogpeppe> davecheney: we just alloc a new storage on each SetConfig
<rogpeppe> davecheney: and return that from Storage
<rogpeppe> davecheney: easy peasy
<rogpeppe> davecheney: (changing the type of the storage field in environ to be a pointer, of course)
<rogpeppe> davecheney: then the current storage implementation stays identical to current.
<davecheney> let rogpeppe it's slightly less easy, 'cos we have to publish that reference properly
<rogpeppe> doh, s/identical to current/the same/
<rogpeppe> davecheney: that reference is inside the environ instance, no?
<davecheney> i'd vote for constructing a new storage for each call to environ.Storage() using the current values of config()
<rogpeppe> davecheney: i don't think there's a need to publish it
<rogpeppe> davecheney: we will get the new instance when we call Storage again
<davecheney> publish across goroutines
<rogpeppe> davecheney: why do we need to do that?
<rogpeppe> davecheney: we've said that there are no long-term users of Storage instances, right?
<rogpeppe> davecheney: which means that when the current storage operations are complete, the goroutines will call Storage again and get the new version.
<rogpeppe> davecheney: there, published.
<davecheney> if goroutine X calls SetCOnfig, we need a mutex to ensure the value of environ.storage is properly published
<rogpeppe> davecheney: yeah, obviously we still need the mutex on the environ
<rogpeppe> davecheney:  but we make sure that any Storage returned from Environ is immutable.
<davecheney> rogpeppe: yup, i can do that
<rogpeppe> davecheney: it's important, then, that environ doesn't call deleteAll directly on the storage instance inside the environ. it should call Storage() first, and then cast .(*storage).
<davecheney> i'll add a comment block if appropraite
<davecheney> rogpeppe: this it looking like a much smaller diff
<davecheney> storage.go is almost untouched
<rogpeppe> davecheney: that's what i hoped
<davecheney> # launchpad.net/juju/go/environs/ec2_test
<davecheney> ./live_test.go:281: undefined: "launchpad.net/juju/go/environs/ec2".DeleteStorageContent
<davecheney> o_O
<davecheney> this is against trunk
<davecheney> let me check I haven't delted that by accident
<rogpeppe> davecheney: trunk works for me
<davecheney> ignore that error, ny fault
<davecheney> my fault
<TheMue> *rofl*
<fwereade> rogpeppe, ok, I just plain cannot figure out what is up with the store tests... I think I've nuked everything and started perfectly clean, and I still get 1 pass, 2 fixture panic, 25 missed
<fwereade> rogpeppe, have you seen this before?
<rogpeppe> fwereade: yeah, i see it every so often. i'll try again and see if i can reproduce it
<davecheney> rogpeppe: https://codereview.appspot.com/6256050, tell me what you think
<davecheney> specifically around immutable storages
<rogpeppe> davecheney: LGTM
<rogpeppe> davecheney: and i think this shows the goodness of the approach actually
<davecheney> i'm relieved that storage.go is virtually unaffected
<rogpeppe> davecheney: it means that we can still continue to use environs.EmptyStorage when there's no public storage
<rogpeppe> davecheney: otherwise that wouldn't work.
<davecheney> coolcoolcool
<davecheney> so, from the POV of someone who wrote Storage, are you happy with the tradeoffs there ?
<davecheney> ie, Storage() gives you an immutable reference
<rogpeppe> davecheney: yeah, i think this works nicely
<rogpeppe> davecheney: it means that someone external can write the equivalent of deleteAll without worrying about things changing underfoot
<davecheney> rogpeppe: the only time I had to step outside the API was in instance.Destroy, but it was only two lines
 * rogpeppe didn't notice that. one mo, while i look.
<rogpeppe> davecheney: oh, you mean e.Storage().(*storage) ?
<davecheney> yup
<davecheney> e.Storage() gives you your own reference, the cast it too (*storage)
<rogpeppe> davecheney: yeah, i think that's inevitable, unless we provide a Destroy method on environs.Storage
<davecheney> this is just so you go through the mutex to get the properly published reference to environ.storageUnlocked
<rogpeppe> davecheney: because the bucket's existence isn't something dealt with in the storage API
<rogpeppe> davecheney: exactly; i think i mentioned this case above.
<rogpeppe> [11:00:00] <rogpeppe> davecheney: it's important, then, that environ doesn't call deleteAll directly on the storage instance inside the environ. it should call Storage() first, and then cast .(*storage).
<davecheney> jynx
<davecheney> i'll fix the typos
<davecheney> whoa, how did that change to storage slip into NewConfig
<davecheney> right - time for a break, and probably a scotch
<fwereade> rogpeppe, after much messing around it appears that the only thing that makes any difference is adding a ludicrously long sleep after starting the server in MgoSuite
<fwereade> rogpeppe, by "ludicrously", I mean that sometimes 10s isn't long enough but it hasn't yet failed with 20s
<rogpeppe> fwereade: hmm.
<rogpeppe> fwereade: how is it failing again?
<fwereade> rogpeppe, failing to dial
<hazmat> fwereade, the server hasn't started?
<fwereade> rogpeppe, no reachable servers
<fwereade> hazmat, yeah
<rogpeppe> fwereade: maybe the dial should retry?
<fwereade> rogpeppe, reading the docs it appears it should anyway
<fwereade> rogpeppe, I'll try DialWithTimeout again (10s is usual I think; I tried 20s, but I should probably try 30s to match the always-succeed-with-sleep rate)
<rogpeppe> fwereade: sounds like a plan
<rogpeppe> fwereade: what is it with these servers that forever to come up?
<fwereade> rogpeppe, I have no idea, I really wasn't expecting that
<fwereade> rogpeppe, I've spent all morning poking around at entirely unrelated things :/
<rogpeppe> fwereade: i know the feeling!
<rogpeppe> fwereade: hmm, having fixed the bug in madeBucket, one of my old tests fails. it *should* fail given the new semantics, but i'm not sure now if they're right.
<fwereade> rogpeppe, heh :(
<rogpeppe> fwereade: the test scenario is we have two Environs, both pointing to the same remote environ, e and e2; we do this: e.Bootstrap(); e2.Destroy(); e.Bootstrap()
<rogpeppe> fwereade: do we care about that working or not?
<fwereade> rogpeppe, tbh that feels somewhat edgey to me
<rogpeppe> fwereade: me too. when are we ever gonna call Bootstrap twice on the same Environ, right?
<rogpeppe> Aram: yo!
<Aram> hi.
<fwereade> rogpeppe, indeed; if reported the response should probably be "don't do that please"
<rogpeppe> fwereade: yeah. i think i'll change the test.
<rogpeppe> fwereade: and perhaps the docs: "When Destroy has been called, all Environs referring to the same remote environment may become invalid", or something.
<fwereade> rogpeppe, yeah, sounds pretty clear
<rogpeppe> fwereade: you gonna submit that AddCharm fix?
<fwereade> rogpeppe, yeah, I got distracted with trying to make all tests pass :/
<fwereade> rogpeppe, I'll submit that one separately right now
<rogpeppe> fwereade: there are still more that fail too. i get sporadic StateSuite.TestUnitWatchPorts failures.
<fwereade> rogpeppe, I saw that one once, indeed
<rogpeppe> fwereade: i think TheMue has it in hand
<fwereade> rogpeppe, cool
<TheMue> rogpeppe, fwereade: Not yet really, only the idea of a race condition.
<TheMue> rogpeppe: We talked about inserting a little sleep. But it could be a similar flaky behavior in production environment. So I've got to analyze your laste change deeper and how to avoid this behavior directly in the content watcher.
<rogpeppe> TheMue: no need to insert a sleep.
<rogpeppe> TheMue: i can see the problem, and i know the fix.
<TheMue> rogpeppe: Oh, so feel free. I've got two branches in queue before.
<TheMue> rogpeppe: What is your idea now?
<rogpeppe> TheMue: i think! i haven't tested it yet, but i've just written the code (one line)
<TheMue> rogpeppe: Small and beautiful ;)
<rogpeppe> TheMue: insert a null op as the first test in unitWatchPortsTests
<rogpeppe> TheMue: expecting an empty ports array
<TheMue> rogpeppe: It's not only the port test.
<TheMue> rogpeppe: There are more failing, all based on the content watcher.
<rogpeppe> TheMue: the same would apply to any similar test
<TheMue> rogpeppe: But so you fix the test, but not the nehavior.
<rogpeppe> TheMue: the tests need to take account of the fact that the watcher *always* sends an initial value on the channel
<rogpeppe> TheMue: i think that's reasonable behaviour
<rogpeppe> TheMue: if we don't think that, then we can fix the behaviour
<TheMue> rogpeppe: The idea of always sending is ok. But the fact, that we sometimes get errors and sometimes not.
<rogpeppe> TheMue: but that behaviour was deliberately made that way; i don't mind if we change it to be different.
<rogpeppe> TheMue: that's a bug in the tests.
<rogpeppe> TheMue: they're creating the watcher and setting the content immediately after. there's a race there.
<TheMue> rogpeppe: And what do you do when this happens in production?
<rogpeppe> TheMue: there's no race unless you're trying to predict what the events are coming on to the channel, as the tests are.
<rogpeppe> TheMue: each of those events is a valid representation of the contents at some point in the past.
<rogpeppe> TheMue: and that's all we want from the contents watcher
<TheMue> rogpeppe: I'll take a look at your change.
<rogpeppe> TheMue: i'll push it in a mo
<TheMue> fwereade: I've got to change the endpoint related methods to not separate between provider/requirer and peer. Instead I shall change it to â¦(endpoints â¦RelationEndpoint). OK, less functions, but more complex code imho.
<fwereade> TheMue, yeah, I think that's a shame, IMO it makes it much harder to see what's going on
<fwereade> TheMue, but hey ho
<fwereade> TheMue, sorry for the noise, I guess :(
<TheMue> fwereade: np ;)
<rogpeppe> TheMue: there's an issue with ResolvedWatcher
<rogpeppe> TheMue: when the content is empty, parseResolvedMode fails.
<TheMue> rogpeppe: Will have to look at the change history. IMHO I've tested it.
<rogpeppe> TheMue: yes, but before we weren't seeing that initial blank value. i guess it's being created blank, but perhaps it should be created with a valid resolved content.
<rogpeppe> TheMue: ah, no! i guess that it just didn't exist before.
<TheMue> rogpeppe: Yep.
<rogpeppe> TheMue: so the first value on the content-changed channel is "no node existing"
<rogpeppe> TheMue: what should the value of "resolved" be then?
<rogpeppe> TheMue: maybe it should just be ResolvedNone if there's no node.
<TheMue> rogpeppe: Sorry, I'm in a totally different context currently. IMHO the resolve change is relative new even for the Python version. Maybe fwereade can help.
<TheMue> rogpeppe: I have to take some to to see the change history here too and look why it now fails.
<TheMue> some time
<rogpeppe> TheMue: i think it's quite straightforward - if you do WatchResolved on a unit without a resolved node, what should you read from the channel?
<rogpeppe> TheMue: ah! the answer is there in Unit.Resolved
<rogpeppe> TheMue: ResolvedNone is correct.
<TheMue> rogpeppe: Didn't the original implementation did its first shot when the node is created, not before?
<rogpeppe> TheMue: yes, but in keeping with the current thinking on watchers, we want to send the current state as the first event on the channel.
<rogpeppe> TheMue: which in this case is ResolvedNone.
<TheMue> rogpeppe: Yes, that's ok. But I'm a bit astonished that the change is in the code based but dependencies havn't been changed so far.
<TheMue> code base
<rogpeppe> TheMue: which dependencies?
<TheMue> rogpeppe: You now see several follow-up, like in the tests.
<rogpeppe> TheMue: sorry, i don't understand
<TheMue> rogpeppe: The checkin of the changed watcher has been too early. The same branch IMHO has to change the tests or the problem with the parseResolvedMode.
<rogpeppe> TheMue: i haven't checked in anything
<rogpeppe> TheMue: i'm fixing those problems currently.
<rogpeppe> TheMue: and the fact that ChildWatcher doesn't always send a first event.
<TheMue> rogpeppe: The behavior of the content watcher changed. That's what I'm talking about.
<rogpeppe> TheMue: ah yes. that's true. the problem was that the tests usually passed!
<rogpeppe> TheMue: so i'm fixing that now.
<TheMue> rogpeppe: Yip, here we have to find a proper way, like a continous integration server running all tests after each submit.
<rogpeppe> TheMue: i think it's fine. as niemeyer said recently, we just need to make sure we run the tests ourselves before submitting.
<rogpeppe> TheMue: ok, i have all tests passing reliably (i think!) now.
<TheMue> rogpeppe: Do you run all tests? Off each package?
<TheMue> rogpeppe: Great, cheers.
<rogpeppe> TheMue: just the state package currently.
<TheMue> rogpeppe: The cheers is for the fixing. ;)
<TheMue> rogpeppe: I would like some kind of running all (!) tests at least once per day.
<rogpeppe> TheMue: i'm gonna fix the watcher tests too.
<rogpeppe> TheMue: we all run the tests at least once a day, i hope!
<TheMue> rogpeppe: Do you have a script for it or do you go into each package manually?
<rogpeppe> TheMue: go test launchpad.net/juju/go/...
<rogpeppe> TheMue: or just go test ./...
<rogpeppe> from the root of the go hierarchy
<TheMue> rogpeppe: Oh, the elipses works? I didn't know, great, that helps a lot.
<rogpeppe> the go tool is lovely like that
<TheMue> rogpeppe: Wow, thx for that hint.
<rogpeppe> TheMue: the ellipses are a wildcard - you can do go test .../state
<Aram> rogpeppe: do you use bzr manually or do you use niemeyer's wrapper?
<rogpeppe> or go install .../test/.../foo
<rogpeppe> Aram: i use niemeyer's cobzr wrapper
<Aram> ok, I'll look into it.
<rogpeppe> Aram: it makes the branches work well with $GOPATH
<rogpeppe> Aram: otherwise it's a right pain
<rogpeppe> TheMue: https://codereview.appspot.com/6245053
<TheMue> *click*
<rogpeppe> TheMue, fwereade: i'm going to be away for an hour or two soon.
<TheMue> rogpeppe: OK
<fwereade> rogpeppe, np, I'm finishing in a mo too
<fwereade> rogpeppe, might manage to push a couple of CLs before I do
<rogpeppe> fwereade: cool. will have a look later.
<TheMue> Btw, I've got a public holiday on Monday. So don't wonder if you don't reach me.
<rogpeppe> TheMue: ah, ok. sometime we're going to have to try to do a meeting!
<rogpeppe> TheMue: you know Aram is starting on monday, right?
<TheMue> rogpeppe: Yep, good news. ;)
<rogpeppe> TheMue: indeed. maybe we should have a meeting on tuesday?
<TheMue> rogpeppe: Would be fine to me.
<rogpeppe> fwereade: ?
<rogpeppe> fwereade: i.e. would you be up for a meeting on tues?
<fwereade> rogpeppe, tuesday would be fine
<rogpeppe> fwereade: probs around 8pm oz time
<fwereade> rogpeppe, is there only one time zone there?
<rogpeppe> fwereade: which i think was what we agreed before
<rogpeppe> fwereade: no idea. i mean sydney time, of course.
<fwereade> rogpeppe, ok, so my midday I guess; that sounds great
<rogpeppe> fwereade: not so good for niemeyer i think
<TheMue> rogpeppe: Yep, midday is ok, lunch is later. :)
<rogpeppe> fwereade: maybe he should call the time.
<TheMue> rogpeppe: It's 7:00 then for him.
<rogpeppe> TheMue: exactly.
<rogpeppe> TheMue: maybe 9pm and 8am might be better
<fwereade> rogpeppe, TheMue: I'm easy, but as you say niemeyer should probably call it
<TheMue> rogpeppe: Isn't he in UK next week?
<rogpeppe> TheMue: that's true
<Aram> what timezone is dave cheney in?
<rogpeppe> Aram: sydney, australia
<Aram> ah.
<TheMue> rogpeppe: LGTM
<rogpeppe> TheMue: thanks
<rogpeppe> niemeyer: yo!
<Aram> hi ni
<Aram> niemeyer:
<niemeyer> Hello!
<rogpeppe> niemeyer: you've arrived just as i'm leaving for a couple of hours
<rogpeppe> niemeyer: got a couple more CLs for you to look at though. just bug fixes.
<niemeyer> rogpeppe: Super
<niemeyer> rogpeppe: Thanks
<TheMue> niemeyer: Moin
<niemeyer> Heya
<rogpeppe> see y'all in a bit
<TheMue> niemeyer: Just for info, the earring is back. ;)
<niemeyer> TheMue: Hah, awesome :)
<TheMue> niemeyer: One of the side jobs: plumber.
<niemeyer> TheMue: Hehe, I know how it feels :)
<Aram> meh, vmware website is getting worse every day.
<TheMue> niemeyer: Could you please take a look at topology.py line 567 ff?
<niemeyer> TheMue: Local or branch?
<TheMue> niemeyer: Current trunk (python).
<niemeyer> TheMue: Ah, .py
<TheMue> niemeyer: There the interface of the actual relation in the loop is tested only against the interface of endpoint 0.
<niemeyer> TheMue: There
<niemeyer> TheMue: Ok
<niemeyer> TheMue: ?
<TheMue> niemeyer: When doing a test now with two endpoints with different interfaces, but the rest is ok, it seems to return a key instead of an error.
<TheMue> niemeyer: Is that ok?
<niemeyer> TheMue: How could this possibly happen?
<niemeyer> TheMue: That would mean there's a relation with two different interfaces on it?
<TheMue> niemeyer: No, adding is tested ok. But the user of the function may test two invalid endpoints (I've done so in my unit test, extra).
<TheMue> niemeyer: IMHO a test that all passed endpoints share the same interface could be added.
<niemeyer> TheMue: Yeah, that sounds fine
<TheMue> niemeyer: OK, will add it, the rest of the new function RelationKey() has been changed according to your review. That's the last point.
<niemeyer> TheMue: Super, thanks!
<TheMue> niemeyer: For the NoRelationError string I've tried %+v, which is really good due to the field names, but long, and %v, which only shows the values of the endpoints. What do you prefer, short or long?
<niemeyer> TheMue: Neither.. that's an error message.. should be oriented towards the user, not with types and debugging information
<niemeyer> TheMue: An endpoint is uniquely identified by service name + relation name
<TheMue> niemeyer: Hehe, that's why I wrote a RelationEndpoint.String() where you told me that it's not needed, instead %v.
<niemeyer> TheMue: There can't be two different relations between two services with the same (service name, relation name) => (service name, relation name) characteristics
<niemeyer> TheMue: Yeah, because you were *already* printing debugging information there
<niemeyer> TheMue: In a way that is not understandable to a user nor to a developer
<TheMue> niemeyer: OK, what about a method Id() string for RelationEndpoint returning exactly this with a colon?
<niemeyer> TheMue: That's what I was against
<TheMue> niemeyer: And I would use that id in the error generation.
<niemeyer> TheMue: foo:bar:baz:baco:bar is not readable
<TheMue> error message
<niemeyer> TheMue: Yeah, sounds fine in principle
<niemeyer> TheMue: Not sure I have the whole picture right now, but we can try it
<niemeyer> I'll head to lunch and be back in a bit
<TheMue> niemeyer: The message would be like ("â¦ between %q and %q", ep1.Id(), ep2.Id()) , where %q later is "wordpress:blog" etc.
<niemeyer> TheMue: I suggest using String()
<niemeyer> TheMue: So you can say between %s and %s
<niemeyer> , ep1, ep2,
<rogpeppe> niemeyer, TheMue, fwereade: yo!
<niemeyer> rogpeppe: Welcome back!
 * niemeyer sips a delicious post-lunch coffee
<rogpeppe> niemeyer: looks out over the sunny Tyne river
 * rogpeppe looks out over the sunny Tyne river :)
<rogpeppe> i just took a little time to see a final year recital from a friend who's a folk degree students here. t'was amazing. now using the free wi-fi at the Sage, Gateshead.
<niemeyer> rogpeppe: Where's that?
<rogpeppe> niemeyer: http://en.wikipedia.org/wiki/The_Sage_Gateshead
<rogpeppe> niemeyer: am currently sitting just inside the glass just above the smaller of the two boats you can see in that picture. and the sky is just as blue.
<rogpeppe> niemeyer: it's a fantastic building
<rogpeppe> niemeyer: have you got any suggestions for the best way to do the putFakeTools thing?
<niemeyer> rogpeppe: Woah
<niemeyer> rogpeppe: This is amazing
<rogpeppe> niemeyer: what is?
<niemeyer> rogpeppe: That building
<rogpeppe> niemeyer: it certainly is
<rogpeppe> niemeyer: although some have compared it to a sea slug :-)
<niemeyer> rogpeppe: It reminds me of the rubber that protects a car's wheel transmission
<niemeyer> rogpeppe: Has the same shape
<rogpeppe> :-)
<niemeyer> rogpeppe: But I digress wildly :)
<rogpeppe> niemeyer: i'm sitting just near to where the woman on the left is in this picture: http://en.wikipedia.org/wiki/File:Newcastle_the_sage_innen.jpg
<niemeyer> rogpeppe: Regarding the putFakeTools, if you want to keep the tools fake, sounds fine, but I still see that as a lot of logic for what is basically "List + Get"
<rogpeppe> niemeyer: and that arc you can see outside above her head is also an amazing piece of engineering, the millenium bridge
<rogpeppe> niemeyer: well, yes.
<rogpeppe> niemeyer: i'm mildly inclined to export ToolsPath from environs
<niemeyer> rogpeppe: Sounds sensible too
<rogpeppe> niemeyer: which would simplify the logic a lot, but slightly clutter the public API for the sake of testing.
<rogpeppe> niemeyer: which i don't like doing.
<niemeyer> rogpeppe: I'd prefer that we had at least a couple of variables on it, to make it explicit what it is depending upon
<rogpeppe> niemeyer: ? variables on what?
<niemeyer> rogpeppe: environs.ToolsPath(version, arch, "12.04")
<rogpeppe> niemeyer: definitely. i think that function already might exist.
<rogpeppe> niemeyer: just not exported
<rogpeppe> 12.04 or precise? :-)
<rogpeppe> niemeyer: BTW, i caught this piece of your conversation last night with davecheney: http://paste.ubuntu.com/1006651/
<rogpeppe> niemeyer: i beg to disagree :)
<niemeyer> rogpeppe: Yeah, good question, I kind of put the literal to talk about it :)
<rogpeppe> niemeyer: do you have any idea how many allocations PutFile does?
<niemeyer> rogpeppe: I don't, and I see no reason to increase it whatever that number looks like
<rogpeppe> niemeyer: 1780
<rogpeppe> more than
<niemeyer> rogpeppe: Nice, let's make it larger then! Not.
<rogpeppe> niemeyer: 3 allocations is lost in the noise
<rogpeppe> niemeyer: (literally - it's within the standard deviation)
<niemeyer> rogpeppe: Yeah, sounds great. Let's do caching please, it's simpler.
<rogpeppe> niemeyer: the code is more complex actually. i don't mind too much, but i just had to take a little issue with the reasoning.
<rogpeppe> niemeyer: anyway, "precise" vs "12.04"
<niemeyer> rogpeppe: No, it's not, and we don't have to discuss this because we have evidence.. two different bugs were already introduced on the way of removing caching..
<rogpeppe> niemeyer: i like the former because it's obviously an ubuntu release name. i like the latter because it's more amenable to program manipulation.
<niemeyer> rogpeppe: Well, let's have "precise" then.. it's already around the code base, and we can actually manipulate it too now that I think of it.
<niemeyer> rogpeppe: Still curious about what happens after 'z', but I'm sure *some* form of the logic will remain
<rogpeppe> niemeyer: yeah, i'm slightly concerned about that.
<rogpeppe> (if the code's simpler, i'm all for it in general - it was just the "it allocates more therefore let's do it differently" reasoning i wasn't keen on.
<rogpeppe> )
<niemeyer> rogpeppe: If you read properly, you'll see in the context that this is just yet another argument among the others
<niemeyer> rogpeppe: and that argument is *also* true.
<niemeyer> fwereade: ping
<rogpeppe> niemeyer: fair enough
<niemeyer> fwereade, rogpeppe, TheMue, Aram: Folks, just as a reminder, next week I'll be in London
<rogpeppe> niemeyer: do you think you'll be able to make a meeting on Tues?
<niemeyer> I'm not sure about how the flow of the sprint will look like
<niemeyer> But I'll try to keep on top of reviews and communication as usual
<TheMue> niemeyer: *envy*
<Aram> btw niemeyer, do you think I'll be able to access the wiki by monday?
<Aram> I don't have permission right now :).
<niemeyer> Aram: I'm hoping so
<TheMue> niemeyer: I only have been there on the airport so far. Would like to see more.
<niemeyer> Aram: There's a "nice" list of tasks for you to go through after starting
<niemeyer> TheMue: I won't be able to see much more than the airport, and the office, I'm afraid :)
<niemeyer> TheMue: Well, the office is still in a fantastic spot, though..
<niemeyer> That's changing soon unfortunately
<niemeyer> Aram: That said, hmm
<niemeyer> Aram: DOn't be super enthusiastic about it.. As far as juju goes, it's pretty much all public
<TheMue> niemeyer: The new one seems to be nice too. Only not such a good outlook.
<niemeyer> rogpeppe: Most probably, re. the meeting
<rogpeppe> niemeyer: i'm thinking that it would be nice if we could attempt at least one face-to-face with Aram next week.
<niemeyer> rogpeppe: Wasn't it Monday, btw?
<niemeyer> rogpeppe: Yeah, that sounds like a great idea actually
<rogpeppe> niemeyer: yeah, but TheMue has a public holiday on mon
<rogpeppe> niemeyer: how about early one morning?
<niemeyer> rogpeppe: Ah, sounds good then
<niemeyer> rogpeppe, Aram: What about Friday?
<rogpeppe> niemeyer: then it'll be ok for davecheney
<Aram> niemeyer: next friday? as in one feek from now?
<Aram> sure.
<Aram> anytime.
<niemeyer> Aram: Yeah, in London
<niemeyer> :)
<niemeyer> rogpeppe: Sounds good.. good chance given we're all closer in terms of timing
<rogpeppe> niemeyer: BTW by "face to face" i meant G+...
<niemeyer> rogpeppe: ROTFL
<niemeyer> rogpeppe: That's what face-to-face has turned into? :-)
<niemeyer> rogpeppe: I actually mean *real* face-to-face.. :)
<rogpeppe> niemeyer: more so than IRC :-)
<niemeyer> Aram: Can you get to London on Friday?
<rogpeppe> niemeyer: actually next fri would work for me
<niemeyer> Aram: How troublesome would it be for you?
<rogpeppe> niemeyer: seems like a long way to come for a short time though.
<niemeyer> rogpeppe: It pays off..
<niemeyer> rogpeppe: The next several weeks will be a lot nicer for Aram after having met people personally
<Aram> niemeyer: I guess I could come, have to check up flights and whatnot, never bought tickets from here.
<rogpeppe> niemeyer: FWIW i can get to london in about 3h15m
<rogpeppe> niemeyer: yeah, i agree with that
<rogpeppe> niemeyer: when do you fly back?
<niemeyer> rogpeppe: Friday too
<niemeyer> rogpeppe: EOD
<niemeyer> Maybe Thursday would be better
<niemeyer> Aram: Which city are you in again?
<Aram> vienna
<niemeyer> Aram: I'll send a probe to the agency that books flights for us
<niemeyer> Aram: Will CC you
<Aram> please, thanks
<rogpeppe> looks like flights from vienna to heathrow are ok
<rogpeppe> niemeyer, Aram: http://www.skyscanner.net/flights/vie/lond/120530/120602/airfares-from-vienna-to-london-in-may-2012-and-june-2012.html
 * Aram has to leave for a few hours, sorry.
<Aram> I'll be back.
<rogpeppe> Aram: see you monday!
<niemeyer> Mail sent
<niemeyer> rogpeppe: Yeah, not to bad
<niemeyer> too
<niemeyer> Aram: Cheers
<rogpeppe> niemeyer: i've just proposed a new version of https://codereview.appspot.com/6245048 with simplified putFakeTools
<niemeyer> rogpeppe: Looking
<rogpeppe> niemeyer: i've got to head off now. will poke my head in later.
<rogpeppe> niemeyer, TheMue, Aram, fwereade: have a great w/e
<TheMue> rogpeppe: Thx, you too.
<niemeyer> rogpeppe: Review delivered
<niemeyer> rogpeppe: Thanks for the quick turn around
<TheMue> niemeyer: Btw, changed Id() to String() of the RelationEndpoint too.
<niemeyer> TheMue: Sweet, is it ready for another round?
<TheMue> niemeyer: Yep, it's in the box. ;)
<niemeyer> TheMue: Checking
<niemeyer> TheMue: Diff looks bogus
<niemeyer> TheMue: https://codereview.appspot.com/6198055/
<TheMue> niemeyer: Ooops, but I set the prerequisite.
<niemeyer> TheMue: Yeah, this is a bug in lbox that I still haven't fixed
<TheMue> niemeyer: Hmm, but that is now in the trunk. May that be the reason?
<niemeyer> TheMue: The problem is that the pre-req has been merged
<niemeyer> TheMue: But lbox is still comparing against the pre-req
<niemeyer> TheMue: I'm sorry about that
<niemeyer> TheMue: To fix it, merge trunk on the pre-req, and push it
<niemeyer> TheMue: (with bzr push only)
<niemeyer> TheMue: Then, re-propose the follow up
<niemeyer> TheMue: This will get the diff clean again
<niemeyer> Ouch, forgot to buy the new battery
<TheMue> niemeyer: OOK, wait a moment, will try.
<TheMue> niemeyer: Ah, great, thx, looks cleaner now.
<niemeyer> TheMue: Wow, yeah :)
<niemeyer> TheMue: Review delivered
<niemeyer> TheMue: The logic there is not quite right
<TheMue> niemeyer: In which way?
<niemeyer> TheMue: Please check the review and let me know if you need further details
<TheMue> niemeyer: Done, that's why I ask. ;)
<niemeyer> TheMue: Ok.. so you don't know that a *relation name* and a *service name* are not the same thing?
<TheMue> niemeyer: I took line 571 in topology.py as example
<TheMue> niemeyer: There is service["name"] != endpoint.relation_name
<niemeyer> TheMue: The code is misleading indeed, but it's important to keep an eye on what you are actually doing
<niemeyer> TheMue: What is inside that "service" dictionary?
<niemeyer> TheMue: Have a look at assign_service_to_relation
<TheMue> niemeyer: Ah, the relation name of the service.
<niemeyer> TheMue: Do you have a good mental model of what a relation name is?
<niemeyer> TheMue: That's not a trick question.. if you don't, let's talk about it
<TheMue> niemeyer: It's a part of the relation identifier, but I find the name "name" mislieading.
<niemeyer> TheMue: Ok.. so let's pick a metadata.yaml to look at
 * niemeyer looks
<TheMue> niemeyer: One moment, have to help my wife.
<niemeyer> TheMue: https://juju.ubuntu.com/docs/charm.html#the-metadata-file
<niemeyer> TheMue: At the end of the doc there are some samples
<niemeyer> TheMue: The relation name is that "db" under provides or requires
<niemeyer> TheMue: Now, imagine that I have an application called mywiki that takes a couple of mongodbs
<niemeyer> TheMue: One of them is for caching, and the other contains the actual app data
<niemeyer> TheMue: Both of these relations will have the "mongodb" interface, for example
<niemeyer> TheMue: But we need to disambiguate them so we can refer to them
<niemeyer> TheMue: So we might call one of them "data" and the other "cache"
<niemeyer> TheMue: Both with interface "mongodb"
<niemeyer> TheMue: So when I'm adding a relation, I could say something like
<niemeyer> TheMue: juju add-relation mywiki:data themongoservice
<niemeyer> For example..
<niemeyer> TheMue: TO connect that specific relation
<niemeyer> TheMue: So it is indeed a "relation name"
<niemeyer> TheMue: Makes sense?
<TheMue> niemeyer: Ah, ok, thank you, makes it more clear.
<niemeyer> TheMue: np
<TheMue> niemeyer: OK, so I'll change it (with a glas of whine beside me)
<TheMue> niemeyer: For the interface error I used a plain error to return informations about the interface name.
<niemeyer> TheMue: Will keep that in mind when reviewing.. LOL
<TheMue> niemeyer: But indeed, I can change it to a NoRelationError too.
<TheMue> niemeyer: Hehe, only one wine (or two or three *hicks*)
<niemeyer> ;-)
<TheMue> niemeyer: One question, coult you please take a look at https://codereview.appspot.com/6200044/diff/3001/state/topology.go.
<TheMue> niemeyer: This has been my first approach before UDS. In line 50 and 55 you see that I used a type to contain the name information.
<niemeyer> TheMue: Checking
<niemeyer> TheMue: Ok?
<TheMue> niemeyer: After the discussion with William I switched to a map of role to service key. Our idea has been that the name relates to the service name and that the key is better here. A wrong assumption, as you now showed to me.
<TheMue> niemeyer: So now I would like to reintroduce this type, but containing the serviceKey (like today the sting in the map) and the relationName. What do you think?
<niemeyer> TheMue: Sounds reasonable
<TheMue> niemeyer: Fine, thanks.
<niemeyer> TheMue: Thank you!
<niemeyer> go-environs-setconfig is ready to go.. OMG.. so happy
<andrewsmedina> niemeyer: nice!
<niemeyer> Woohoo.. clean queue again
<niemeyer> I'm heading out for some exercising
<TheMue> niemeyer: I have to disappoint you. Just reproposed the latests branch. ;)
#juju-dev 2012-05-26
<davecheney> rogpeppe: i ran into your 'can't go test jujuc'
<davecheney> today (again)
<davecheney> i think it is related to testing in main packates
<davecheney> packages
<Aram> morning,
<davecheney> evening!
#juju-dev 2012-05-27
<Aram> morning.
<Aram> hi davecheney
<davecheney> hi there
<davecheney> or should I say, welcome
<Aram> thanks,
<Aram> davecheney: I'm trying to setup everything to all tests for juju's dependencies pass before I try to do anything. go test launchpad.net/mgo, do you know if I need to setup mgo beforehand somehow?
<Aram> the zookeeper tests launch zookeeper with a test configuration.
<Aram> I guess this is not the case for mgo?
<Aram> s/go test launchpad.net/mgo/go test launchpad.net/mgo fails/
<davecheney> ahh yes
<davecheney> apt-get install mongodb-server zookeeperd libzookeeper-mt-dev
<davecheney> should get you going
<Aram> I installed all those, but:
<Aram> http://pastebin.com/z8Qmwfk3
<davecheney> ok,maybe you need to start the service
<Aram> I guess, though it seems like the tests want to start mongo manually.
<Aram> mmm.
<davecheney> yeah, the zk tests run it up on a spare port
<Aram> I think I solved it.
<Aram> I had to install supervisor
<Aram> well, now I'm waiting to see if it did anything.
<Aram> no
<Aram> meh
<davecheney> Sun May 27 15:11:29 ReferenceError: rs1a is not defined wait.js:42
<davecheney> that sounds like the problem
<davecheney> you're using precise right ?
<Aram> yes
<davecheney> weird
<Aram> yeah, I believe this fails: rs1a = new Mongo("127.0.0.1:40011").getDB("admin")
 * Aram knows nothing about mongodb
<Aram> the default system provided instance works.
<Aram> but in the test a javascript file is used to create new instances.
<hazmat> m_3, installing zookeeperd runs zookeeper.. for the tests you only need libzookeeper-java and zookeeper-server
<hazmat> whoops.. that was for Aram ^
<Aram> hazmat: thanks, I figured.
<hazmat> Aram, out of curiosity, is doozerd dead?
<Aram> I think so.
<m_3> hazmat: np
#juju-dev 2013-05-20
<thumper> davecheney: does go test buffer output?
 * thumper loses davecheney
<thumper> hmm...
 * thumper waits 
<jam> wallyworld_: if you prefer today, https://plus.google.com/hangouts/_/750982817f7631223162e5af2eceb6726c27bfe3
<jam> thumper: /wave
<rogpeppe1> mornin' all
<dimitern> morning!
<thumper> jam: oh hai
<thumper> jam: just read your email respons
<jam> hi thumper
<thumper> jam: I'll pop back in in 30min to see if fwereade is around
<jam> I'm pretty sure william is gone today
<thumper> ah, that's right
<thumper> leave
<thumper> geez
<jam> so we can check in 30min, but probably just going for tomorrow might work better
<dimitern> thumper: fwereade is gone all week btw
<jam> dimitern: thanks for the clarification.
<jam> he still wanted to meet with us during that time, but it is hard to meet up today.
<jam> thumper: I can do the same time tomorrow
<jam> thumper: does it work for you?
<dimitern> jam: not sure about that - he'll probably pop in at some point in irc at least (provided he has decent connection in backwater gozo :)
<thumper> jam: can you do now?
<thumper> hi dimitern
<jam> thumper: I could do now for a bit. I techincally have a call with dimitern in 30min, but I think he would accept moving it around a bit.
<jam> obviously mark won't be there.
<dimitern> thumper: do you have 5m about a couple of bzr pipes question?
<thumper> jam: we could be done in 30 minutes
<thumper> jam: I don't think there is that much to discuss :)
<dimitern> thumper: hiya
<thumper> dimitern: leave the question and I'll answer as fast as I can
<dimitern> jam: sure, np moving it later
<thumper> jam: I'm fine with mark not being there
<thumper> jam: lets hang out
<thumper> oh for the love of god why did hangouts get so hard to start?
<jam> thumper: https://plus.google.com/hangouts/_/16239e7d8ea236f8aa18ed4ada6c87f9e6dc3dfc
<dimitern> thumper: ah, actually thanks to the very good docs you wrote I managed :) cheers
<dimitern> jam: ping me when you're done
<jam> dimitern: poke
<jam> would you prefer hangout to mumble?
<dimitern> jam: don't mind a hangout
<jam> dimitern: https://plus.google.com/hangouts/_/e7c22066f25817d34f2bb47a6c1eabc32def3fc6d
<dimitern> jam: i'm there
<jam> dimitern: I don't see you yet, I'm also in mumble if it is easier for you to get there.
<dimitern> jam: hmm - ok. let's mumbe then
<mgz> mum-be!
<thumper> night all
<jam> let's get ready to mummmbleeee!!!
<dimitern> mgz: ping
<mgz> pong
<dimitern> mgz: sorry, stupid question maybe, but i'm trying to unlearn cobzr workflow and switch to normal bzr one - i'm using a LW checkout of juju-core, my current branch is trunk, how to create a feature branch out of it? tried bzr branch . new-branch, but it seems to do more than it should
<mgz> `bzr switch -b new-feature`
<dimitern> mgz: and running go build ./... && go test ./... afterwards gives me lots of errors: http://paste.ubuntu.com/5683245/
<mgz> your commands tells bzr to create a new copy of the current branch under the same tree structure, which no doubt confuses go
<mgz> (fix by `rm -rf new-branch`)
<dimitern> mgz: ah, ok, so how can I fix what I messed up? I did the rm -fr new-branch, but how to switch back to trunk? (040-move-set-agent-alive-out-of-machiner) dimitern@kubrik:~/work/juju-core$ bzr switch trunk
<dimitern> bzr: ERROR: Not a branch: "/home/dimitern/work/go/src/launchpad.net/juju-core/trunk/".
<mgz> what does `bzr branches` say?
<dimitern> mgz: bzr: ERROR: Not a branch: "/home/dimitern/work/go/src/launchpad.net/juju-core/040-move-set-agent-alive-out-of-machiner/".
<mgz> exciting.
<dimitern> mgz: oh well, maybe I should scrach everything and start over - nothing done yet, so nothing to lose
<mgz> edit .bzr/branch/branch.conf
<mgz> or yeah, just start over
<dimitern> mgz: thanks
<mgz> so, are you aiming for native colo,  or a shared repo?
<mgz> the general idea is the same, but what you need in the go world is slightly different
<dimitern> mgz: i want to have ~/src/juju-core shared repo with no trees, have $GOPATH/../j-c bound to that lwco and have a symlink in ~/work/juju-core to safe typing
<mgz> okay, like jam does. pretty sure he still uses switch, but also has some locations magic
<dimitern> mgz: i'm a bit confused that i have to specify paths for switch and merge - with cobzr i needed just the branch names
<dimitern> mgz: so generally, when i'm in trunk to create a new branch I just need bzr branch -b new-branch and it'll switch to it as well?
<mgz> no, `bzr switch -b`, which creates the branch
<dimitern> mgz: ah, right. and then while i'm in new-branch to merge trunk in: bzr merge trunk ?
<mgz> you need a path for merge, but often you just want `bzr merge :parent`
<dimitern> mgz: cool! i'll add alias for this
<dimitern> mgz: strange though while in new-branch "bzr branches" shows * . only and "bzr show-pipeline" shows what I expected: * new-branch
<mgz> yeah, branches go borked about the shared repo case again I think
<mgz> *got
<dimitern> mgz: but i did a fresh clone of trunk
<dimitern> mgz: and now, while in new-branch i did bzr switch trunk, then bzr remove-branch new-branch and it said: bzr: ERROR: Branch is active. Use --force to remove it.
<mgz> what does `bzr info` say now?
<dimitern> mgz: http://paste.ubuntu.com/5683309/
<mgz> that still seems sane
<dimitern> mgz: light chechout root: . seems fishy - should it be like that?
<mgz> ah, wait
<mgz> no, should be okay
<mgz> it may be just a confused remove-branch somehow
<dimitern> mgz: how to manually remove it?
<mgz> it's not very useful for the branch-in-dir layout anyway, as you want to rm the directory after, and that's what you need to remove the branch too
<mgz> dimitern: just `rm -rf ~/src/juju-core/new-feature`
<mgz> you can try `bzr rmbranch ~/src/juju-core/new-feature` first
<mgz> and use --force if that complains
<dimitern> mgz: too late :)
<mgz> new experiment time :)
<mgz> just `bzr switch -b new-feature && bzr switch trunk` then go
<dimitern> mgz: i'll try again
<mgz> (or `bzr branch . ~/src/juju-core/new-feature`)
<dimitern> mgz: in addition to --no-trees what else should i specify to init-repo so there are no files in there? with cobzr there were just empty dirs for branches and the root dir was empty as well (safe for .bzr)
<mgz> just that
<dimitern> mgz: oh, I see the problem - instead of doing init-repo i just did mkdir juju-core && bzr branch lp:juju-core trunk before
<dimitern> mgz: now it works and bzr branches reports * new-branch \n trunk as expected
<dimitern> mgz: i added bash completions for bzr commands (very handy!) and a small script to show me the current branch name in the prompt. it will be really useful if somehow i can complete branch names as well in bzr commands - any idea what can i use?
<mgz> not off hand
<dimitern> oh well, it's not that bad with a few aliases and such
 * dimitern bbiab
<dimitern> back
<dimitern> rogpeppe1: ping
<rogpeppe1> dimitern: pong
<dimitern> rogpeppe1: if i move setagentalive in MA.Entity(), when should I stop the pinger then?
<rogpeppe1> dimitern: i don't think that's the right place for it
<dimitern> rogpeppe1: where then? runloop?
<rogpeppe1> dimitern: possibly. let me look again, one mo
<rogpeppe1> dimitern: hmm, how many pingers do we want going at once?
<dimitern> rogpeppe1: how many? just one i think
<dimitern> rogpeppe1: for the machine entity the MA is responsible for
<rogpeppe1> dimitern: in which case i think that the current way is perhaps the only decent solution
<rogpeppe1> dimitern: unless we make another worker specifically to maintain the pinger
<dimitern> rogpeppe1: i thought we agreed to move it
<rogpeppe1> dimitern: yes, but i'm not sure it's right
<rogpeppe1> dimitern: in the machine agent there are two concurrent runLoops
<dimitern> rogpeppe1: which are?
<rogpeppe1> dimitern: the API server and the rest of the workers
<rogpeppe1> dimitern: and we will have three
<rogpeppe1> dimitern: as we move towards using the API for the workers
<dimitern> rogpeppe1: william mentioned you agreed to merge some of these loops
<dimitern> rogpeppe1: not sure what was it
<rogpeppe1> dimitern: i'm not sure that's what we arrived at
<dimitern> rogpeppe1: well this is kinda blocking me
<rogpeppe1> dimitern: in my understanding we decided to have a separate loop for workers which connect to mongo and workers which connect to the API
<dimitern> rogpeppe1: i can switch to the other task - machine api instead
<dimitern> rogpeppe1: and leave the AA stuff until next week to have an agreement
<rogpeppe1> dimitern: i think that's not a bad idea, particularly as fwereade doesn't appear to be around currently.
<rogpeppe1> dimitern: sorry not to be more help
<dimitern> rogpeppe1: it's ok, i just need to start from somewhere. can do the AA stuff later, it's orthogonal
<dimitern> rogpeppe1: do you have 20m for a quick hangout about the machine api?
<rogpeppe1> dimitern: sure
<dimitern> rogpeppe1: i'll start one as soon as we finish the standup
<rogpeppe1> dimitern: ok
<dimitern> rogpeppe1: they changed *everything* on g+ - how do i start a hangout now??
<rogpeppe1> dimitern: i'll try!
<rogpeppe1> dimitern: do you think "hangouts" are the same as "hangouts on air" ?
<dimitern> rogpeppe1: no - those are live streaming ones
<dimitern> rogpeppe1: public
<rogpeppe1> dimitern: https://plus.google.com/hangouts/_/efbe73ddf30405282d381b4767ff0762c0dee700?authuser=0&hl=en-GB
<ahasenack> hi guys, I'm getting "ERROR: Not a branch" for goose when I run go get -v -u launchpad.net/juju-core/... : http://pastebin.ubuntu.com/5683526/
<ahasenack> I didn't see anything in the juju-dev@ mailing list about this, or about some branch changing
<dimitern> ahasenack: you probably didn't update goose for a while
<ahasenack> dimitern: I run go get -v -u everyday
<dimitern> ahasenack: trunk moved to another location owned by the go-bot
<ahasenack> dimitern: if I trash everything under $GOPATH/src and run go get -v launchpad.net/juju-core/... will it be using the right branch?
<dimitern> ahasenack: you'll need to pull it from lp:goose and use --remember and --overwrite
<dimitern> ahasenack: that should work if you can do it\
<ahasenack> sure, it's fast
<ahasenack> dimitern: thanks
<dimitern> ahasenack: but use go get -v -u
<dimitern> mramm2, rogpeppe1: kanban?
<dimitern> rogpeppe1: you did godef right?
<rogpeppe1> dimitern: yeah
<dimitern> rogpeppe1: awesome! it turns out go mode for emacs supports it as an alternative to searching for tags
<dimitern> rogpeppe1: and it works way better than tags as well
<rogpeppe1> dimitern: ah, cool. i knew someone integrated it at some point
<rogpeppe1> dimitern: i wasn't sure if it was available anywhere
<rogpeppe1> dimitern: it doesn't *always* work, but it's not bad
<dimitern> rogpeppe1: in go 1.1 it's used
<rogpeppe1> dimitern: the go parser it uses is an ancient fork of the go parser package
<rogpeppe1> dimitern: i really want to update it to use go/types some time, but y'know, "in my copious spare time" :-)
<dimitern> rogpeppe1: if it doesn't work for some code i'll tell yu :)
<rogpeppe1> dimitern: someone should make it work for vim too
<dimitern> rogpeppe1: yeah
<rogpeppe1> dimitern: it doesn't know about import .
<rogpeppe1> dimitern: as in the way we import gocheck
<dimitern> rogpeppe1: yeah
<rogpeppe1> dimitern: that's the main limitation i know about currently
<rogpeppe1> dimitern: i find the functionality indispensible when browsing through other people's code (and our own, actually, come to that)
<rogpeppe1> dimitern: you can get it to tell you type information too
<rogpeppe1> dimitern: although i rarely use that functionality
<dimitern> rogpeppe1: i'm content with having a "jump" shortcut that works for most cases, rather than having to grep
<rogpeppe1> dimitern: yeah. that's the main thing.
<rogpeppe1> dimitern: it should work with any identifier
<rogpeppe1> dimitern: including field names, type names, method names etc
<dimitern> rogpeppe1: yeah, just tested it within the apiserver - sweet! ws.ws.x.y.z.Stop() - no problem :)
<rogpeppe1> dimitern: yup, even when ws itself comes from a for loop or something
<dimitern> rogpeppe1: really nice
<rogpeppe1> dimitern: at one point i instrumented the godoc web interface so that every identifier was shown as a link. that was quite cool.
<dimitern> rogpeppe1: they actually have that now in go 1.1
<dimitern> rogpeppe1: not sure how
<rogpeppe1> dimitern: not really - only in the doc pages, not in the source code itself.
<dimitern> rogpeppe1: ah, i see - well it's a start at least
<rogpeppe1> dimitern: yeah, it's a great improvement
<dimitern> rogpeppe1: why isn't there a srvService? ServiceExpose/Unexpose are global
<rogpeppe1> dimitern: ServiceExpose and ServiceUnexpose are client methods
<rogpeppe1> dimitern: we need srvService too
<rogpeppe1> dimitern: (and that will not have an Expose method)
<dimitern> rogpeppe1: hmm.. i have no way to see how some methods on srvMachine should look like
<rogpeppe1> dimitern: i suggest you add methods on srvMachine only as they are needed
<dimitern> rogpeppe1: let me paste you a few i have so far and tell me if i'm on the right track
<rogpeppe1> dimitern: these are all the calls i think we need: http://paste.ubuntu.com/5684020/
<dimitern> rogpeppe1: a bit more we need in srvMachine
<rogpeppe1> dimitern: oh yes?
<dimitern> rogpeppe1: http://paste.ubuntu.com/5684027/
<rogpeppe1> dimitern: i did the survey that led to that list ages ago, so things have probably changed since then
<rogpeppe1> dimitern: that looks right. we've only done proper lifecycles and status setting relatively recently
<dimitern> rogpeppe1: so it looks sane?
<rogpeppe1> dimitern: yes apart from SetStatus
<rogpeppe1> dimitern: API calls can have at most one argument
<dimitern> rogpeppe1: why?
<rogpeppe1> dimitern: and that argument must be a struct
<rogpeppe1> dimitern: because we need the parameters to be named
<rogpeppe1> dimitern: and function params have no associated names
<dimitern> rogpeppe1: ah, ok - so params.MachineSetStatus ?
<rogpeppe1> dimitern: read go doc launchpad.net/juju-core/rpc
<rogpeppe1> dimitern: no, SetStatus is fine
<rogpeppe1> dimitern: oops
<rogpeppe1> dimitern: yes
<rogpeppe1> dimitern: that sounds about right
<dimitern> rogpeppe1: yeah, there's unit setstatus as well
<dimitern> rogpeppe1: but it's the same
<rogpeppe1> dimitern: perhaps just params.SetStatus then
<dimitern> rogpeppe1: ok
<dimitern> rogpeppe1: and i need to do state.Life -> params.Life conversion as well
<rogpeppe1> dimitern: yes. i think you can just use state.Life.String actually.
<dimitern> rogpeppe1: ah, cool, even better
<dimitern> rogpeppe1: i just realized i have to implement a whole Pinger similarly to EntityWatcher in apiclient as well
<rogpeppe1> dimitern: not nearly that complex.
<rogpeppe1> dimitern: it's much more similar to  AllWatcher
<rogpeppe1> dimitern: but simpler than that
<rogpeppe1> dimitern: it'll only have one method, Stop.
<dimitern> rogpeppe1: not only that - Alive() as well
<rogpeppe1> dimitern: ah yes, that too
<rogpeppe1> dimitern: but no complex logic - just a call-through to the api
<dimitern> rogpeppe1: yeah seems so
<dimitern> rogpeppe1: the problem is pingers are in state/presence, and I don't see how to use that when I'm supposed to have st *State only
<rogpeppe1> dimitern: you'll need to define an api.Pinger type
<rogpeppe1> dimitern: oh sorry, wrong problem.
<dimitern> rogpeppe1: by api you mean put it in apiclient.go
<rogpeppe1> dimitern: i'm not sure i see your problem
<dimitern> rogpeppe1: yes, i'm there
<rogpeppe1> dimitern: what's the difficulty with using a pinger in the API server?
<dimitern> rogpeppe1: well the api.EntityWatcher uses stuff like err := w.st.call(w.etype, w.eid, "Watch", nil, &id);
<dimitern> rogpeppe1: but i need to call stuff not in state/ but state/presence/
<rogpeppe1> dimitern: from the API client? or the server?
<dimitern> rogpeppe1: from the client
<rogpeppe1> dimitern: why so?
<dimitern> rogpeppe1: ok, think for a moment - SetAgentAlive -> *pinger, error
<rogpeppe1> dimitern: you will need a new srvPinger type, if you hadn't realised that already
<dimitern> rogpeppe1: in the api it's params.PingerId instead
<dimitern> rogpeppe1: ah
<rogpeppe1> dimitern: so do just what the watcher stuff does
<dimitern> rogpeppe1: yeah
<dimitern> rogpeppe1: ok, sorry :)
<rogpeppe1> dimitern: np
<dpb1> Hey -- juju-core has a rather nasty behavior for relation-list if you are calling it on something like a peer relation with only one unit.  Should it be a bug?  It breaks charm suppor and probably many other charms:  See here for an example: https://pastebin.canonical.com/91346/
<dpb1> FYI -- I have filed: https://bugs.launchpad.net/juju-core/+bug/1182224
<_mup_> Bug #1182224: relation-list returns null with json-output <juju-core:New> <https://launchpad.net/bugs/1182224>
 * kyhwana pokes Makyo idly
<Makyo> Oh, hey Kyh.
<kyhwana> Makyo: oh his :)
<kyhwana> Is it idly time around here? Was planning on trying to write a smokeping charm, since there doesn't seem to be one.
<Makyo> kyhwana, Yeah, it's past EOD for a lot of folks.  #juju might be more active?
<kyhwana> ahh ok
<Makyo> kyhwana, mmmmaaybe marcoceppi or m_3 would know a bit more, too, if you need :)
<kyhwana> well, will ask around later. That'll be today's After Work project :) I keep breaking my smokeping install at home :|
<Makyo> Ah, yeah :S
<Makyo> I keep wanting to do some random charm, but every time I think I have a good idea, I find it's already been done :P
<kyhwana> hah, I checked the charm list and there's no smokeping one. (otherwise I would've deployed it myself)
<Makyo> Yeah, well, hey, there's your chance \o/
#juju-dev 2013-05-21
 * thumper loos for wallyworld_
<thumper> s/loos/looks/
 * thumper sighs
<wallyworld_> hello
<thumper> wallyworld_: s'up?
<wallyworld_> finishing some work to make juju easier to use
<wallyworld_> using simplestreams metadata, public bucket url in keystone catalog etc
<wallyworld_> i have something like 6 branchs in review
<thumper> heh
<wallyworld_> what you doing?
<wallyworld_> you were off last  week?
<thumper> I'm just about to put up the three branches I have to implement plugins fully
<thumper> yes, last week was a holiday for me
<wallyworld_> yay
<thumper> about to spend the afternoon breaking down work items into bite sized chunks
<wallyworld_> did you go away anywhre
<wallyworld_> apparently me, you and frank are on containers
<thumper> wallyworld_: no, stayed at home and hacked on my personal project
<thumper> made good progress
<marcoceppi> Makyo: did you need help with anything in particular?
<marcoceppi> err, kyhwana ^^
<kyhwana> marcoceppi: not yet ;)
<kyhwana> still work time *power cycles some more boxeS*
<kyhwana> XFS doesn't like it when you're building a kernel on it and you power cycle the box, even with RAID BBUs :(
<marcoceppi> Well, it's sleep time here, wanted to make sure you weren't stuck on something before I powered down
<kyhwana> oh thanks, will see how I get on
<marcoceppi> kyhwana: feel free to ping in #juju if you have a problem. A lot more charmers tend to lurk in there
<marcoceppi> cheers o/
<dimitern> g'morning
<rogpeppe1> dimitern: hiya
<dimitern> rogpeppe1: heyhey
<dimitern> rogpeppe1: still struggling with (srv)Pinger stuff
<dimitern> rogpeppe1: but i hope it'll end soon
<rogpeppe1> dimitern: ok. feel free to ask if you think there's something i could help with.
<dimitern> rogpeppe1: cheers
<dimitern> rogpeppe1: so to be able to use the pinger client-side would I need to define func (r *srvRoot) Pinger(id string) (srvPinger, error) ?
<rogpeppe1> dimitern: yes
<dimitern> rogpeppe1: not just have func (m *srvMachine) SetAgentAlive() (params.PingerId, error)
<rogpeppe1> dimitern: well, probably you'd return *srvPinger
<rogpeppe1> dimitern: yeah, you'd need that too
<dimitern> rogpeppe1: but EntityWatcher() returns just srvEntityWatcher
<rogpeppe1> dimitern: ah, probably because it's only wrapping a pointer.
<dimitern> rogpeppe1: and errUnknownWatcher
<dimitern> rogpeppe1: it looks like it's just an empty struct rather than wrapping a pointer
<dimitern> rogpeppe1: s/empty/uninitialized/
<dimitern> rogpeppe1: oh.. you know what i mean :)
<rogpeppe1> dimitern: it's only returning a zero struct when it returns an error
<dimitern> rogpeppe1: should I follow this and define an error as well or just return a pointer?
<rogpeppe1> dimitern: otherwise it wraps the *srvWatcher
<rogpeppe1> dimitern: those are weird alternatives :-)
<rogpeppe1> dimitern: type srvPinger {*srvWatcher} would seem to be the right approach
<dimitern> rogpeppe1: I prefer a pointer actually
<rogpeppe1> dimitern: there's no point :-)
<dimitern> rogpeppe1: yeah, I did that already
<rogpeppe1> dimitern: it just means one more allocation for no particular reason
<dimitern> rogpeppe1: so have errUnknownPinger and use the same pattern
<rogpeppe1> dimitern: yeah
<dimitern> rogpeppe1: ok
<dimitern> rogpeppe1: so which methods do I need to implement on srvPinger? there are Start, Stop and Kill defined, but for srvEntityWatcher I can see only Next() defined and there are more in state.EntityWatcher
<rogpeppe1> dimitern: i think you probably want Stop and Kill
<dimitern> rogpeppe1: but how to start it?
<rogpeppe1> dimitern: SetAgentAlive starts it
<dimitern> rogpeppe1: ah, right!
<rogpeppe1> dimitern: BTW, i've just noticed something
<dimitern> rogpeppe1: yeah?
<rogpeppe1> dimitern: you probably the resource to Kill the pinger rather than Stopping it
<rogpeppe1> dimitern: which means you won't be able to just embed the srvResource
<dimitern> rogpeppe1: why?
<rogpeppe1> dimitern: because when a connection goes away, i *think* we want the agent to be reported as dead immediately
<dimitern> rogpeppe1: seems right
<dimitern> rogpeppe1: but this means I need to both embed srvResource and define Kill
<dimitern> rogpeppe1: ah, I see your poing - implement Stop() as Kil()
<rogpeppe1> dimitern: you'll need srvResource as a field, and yes, you'll need to define Stop and Kill on srvPinger
<rogpeppe1> dimitern: yeah
<dimitern> rogpeppe1: I need srvResource because?
<rogpeppe1> dimitern: i think the naming distinction between Stop and Kill in Pinger is a bit too subtle
<dimitern> rogpeppe1: s/need/need to embed/
<dimitern> rogpeppe1: actually wrt to the logic above, do I need Stop at all?
<rogpeppe1> dimitern: not strictly speaking
<rogpeppe1> dimitern: if we don't currently use it
<rogpeppe1> dimitern: i think that we should probably have Pinger.Stop and Pinger.StopButLeaveRunning :-)
<rogpeppe1> dimitern: or something like that
<dimitern> rogpeppe1: why the second one?
<rogpeppe1> dimitern: that's what Stop does currently
<dimitern> rogpeppe1: I can't see Stop being used - perhaps i'm not looking in the right place
<rogpeppe1> dimitern: currently Stop is the only method being used
<rogpeppe1> dimitern: which is perhaps not quite right.
<dimitern> rogpeppe1: I meant presence.Pinger.Stop
<rogpeppe1> dimitern: that's what i meant too
<rogpeppe1> dimitern: basically, we should really only be using pinger.Stop when we know we're upgrading
<dimitern> rogpeppe1: where is it used?
<rogpeppe1> dimitern: in worker/machiner/machiner.go:/watcher.Stop
<dimitern> rogpeppe1: hmm
<rogpeppe1> dimitern: but actually...
<dimitern> rogpeppe1: when upgrading we need the pinger stopped but alive somehow?
<rogpeppe1> dimitern: well yes - we don't want to see that the agent has died when in fact it's just restarting
<dimitern> rogpeppe1: it *is* dead until restarted, isn't it?
<rogpeppe1> dimitern: just temporarily inactive :-)
<dimitern> rogpeppe1: i don't see how this matters - it might not come up again
<dimitern> rogpeppe1: and when it does it'll start a new pinger anyway
<rogpeppe1> dimitern: yeah, in which case the normal timeout will apply
<rogpeppe1> dimitern: it will start a new pinger which will take over from the previous one, and noone will know
<dimitern> rogpeppe1: that's very subtle logic
<rogpeppe1> dimitern: that was part of the design
<rogpeppe1> dimitern: well, not the "subtle" bit :-)
<dimitern> rogpeppe1: so what then - sP.Stop() calls P.Kill and have sP.Pause() ?
<dimitern> rogpeppe1: calling P.Stop()
 * rogpeppe1 inspects the logic inside presence
<dimitern> rogpeppe1: or just don't complicate things and have sP.Kill() -> P.Kill() and sP.Stop() -> P.Stop()
<dimitern> rogpeppe1: ?
<rogpeppe1> dimitern: yes, but a bit more too
<dimitern> rogpeppe1: please explain
<rogpeppe1> dimitern: i think you want: type pinger struct {p *presence.Pinger}; and pinger.Stop() -> pinger.p.Kill
<dimitern> rogpeppe1: by "pinger" you mean srvPinger or it's a separate type?
<kyhwana> hrm, going through the juju getting started at https://juju.ubuntu.com/docs/getting-started.html in 12.04, trying to use lxc, in "In ~/.juju/environments.yaml, add a section for type "local":" I get error: environment "sample" has an unknown provider type "local" when I run juju bootstrap
<rogpeppe1> dimitern: it's a separate type
<dimitern> kyhwana: Go juju (juju-core) does not yet support the local provider
<dimitern> rogpeppe1: ok, I'm confused now.. hangout?
<rogpeppe1> dimitern: sure
<kyhwana> dimitern: what. Why does the getting-started then have instructions for LXC/local?
<dimitern> rogpeppe1: I'll start one this time
<dimitern> kyhwana: it's about Python juju mostly, if you want to use the local provider you should stick with Py juju for now
<dimitern> rogpeppe1: https://plus.google.com/hangouts/_/9849418108b0af37525e7611bc4b9e20837c241d?authuser=0&hl=en
<kyhwana> dimitern: whats the difference?
<dimitern> kyhwana: juju-core is actively developed, while py-juju is just maintained
<kyhwana> ahhh
<kyhwana> so I should try it in 13.04?
<kyhwana> right, works in 13.04
<kyhwana> I know it says "12.04" is in beta, but maybe the getting started page should say "requires 13.04"
<dimitern> kyhwana: so in 13.04 juju-core is released for the first time, but it does not implement all the features of py-juju; it implements more things as well and generally is the way forward for juju. In 13.10 there is a lot more stuff planned (including the local provider among other things)
<kyhwana> hmm
<dimitern> rogpeppe1: all done on the server side i think, now I have a question about SetAgentAlive -> *Pinger, error
<dimitern> rogpeppe1: (client-side)
<rogpeppe1> dimitern: ok
<dimitern> rogpeppe1: I need both to call srvMachine.SetAgentAlive and start a pinger client-side, right?
<rogpeppe1> dimitern: you just need to call SetAgentAlive
<rogpeppe1> dimitern: and store the pinger id in the *Pinger that you return from tha
<rogpeppe1> t
<dimitern> rogpeppe1: but how about the  loop stuff and Pinger at client-side?
<rogpeppe1> dimitern: it's all done server side
<rogpeppe1> dimitern: no loop necessary
<dimitern> rogpeppe1: so why EntityWatcher defines a loop at client-side then?
<rogpeppe1> dimitern: because it needs to turn the API calls into channel sends
<rogpeppe1> dimitern: there's no need for that with a pinger
<dimitern> rogpeppe1: ah, right; but I still need a Pinger type at client side
<rogpeppe1> dimitern: yup
<rogpeppe1> dimitern: type Pinger {id string}
<dimitern> rogpeppe1: which does what?
<rogpeppe1> dimitern: it'll just define a Stop method
<rogpeppe1> dimitern: actually, type Pinger {st *State; id string}
<dimitern> rogpeppe1: I see
<rogpeppe1> dimitern: then func (p *Pinger) Stop() error {return p.st.call("Pinger", p.id, "Stop", nil)}
<rogpeppe1> or something like that
<dimitern> rogpeppe1: cool :)
<dimitern> rogpeppe1: how about the result of the call to SetAgentAlive? It returns params.PingerId and error
<dimitern> rogpeppe1: do I need something like struct result {id params.PingerId, err error}?
<rogpeppe1> dimitern: no
<rogpeppe1> dimitern: just pass a PingerId to the result paramter of call
<dimitern> rogpeppe1: and the error is returned by m.st.Call ?
<rogpeppe1> dimitern: yup
<dimitern> rogpeppe1: and for methods w/o arguments and an error result, like EnsureDead? -> return m.st.call("Machine", m.id, "EnsureDead", nil, nil)
<rogpeppe1> dimitern: yup
<dimitern> rogpeppe1: i like  it already :)
<rogpeppe1> dimitern: cool
<dimitern> rogpeppe1: and Life? there's no error result, just params.Life -> m.st.call("Machine", m.id, "Life", nil, &life); return life?
<rogpeppe1> dimitern: every API call *must* return a struct
<rogpeppe1> dimitern: so you'll need a struct with a single Life member
<dimitern> rogpeppe1: ah, ok
<dimitern> rogpeppe1: does it have to be a struct with named fields? will type Life struct { string } work?
<rogpeppe1> dimitern: it should be with named fields, yes
<rogpeppe1> dimitern: that's kinda the point
<dimitern> rogpeppe1: so Life { Life string } then
<rogpeppe1> dimitern: it means that we can add fields later without breaking the API
<rogpeppe1> dimitern: yeah, that sounds reasonable.
<rogpeppe1> dimitern: although...
<rogpeppe1> dimitern: i'm wondering if we need to define "type Life string" in params
<rogpeppe1> dimitern: so that ServiceInfo can use it
<rogpeppe1> dimitern: and that means we need to think of another name...
<dimitern> rogpeppe1: I did that initially, but now I have: http://paste.ubuntu.com/5686558/
<rogpeppe1> dimitern: so no state.Life type?
<dimitern> rogpeppe1: well state.Life is an int8
<rogpeppe1> dimitern: yes, but lots of things in the code use state.Life, don't they?
<rogpeppe1> dimitern: (or maybe they don't actually...)
<dimitern> rogpeppe1: all of them most probably
<dimitern> rogpeppe1: Life.String() is used in commands only I think and logs
<dimitern> rogpeppe1: so then I should have params.LifeResults with Life state.Life? can I do that in params?
<rogpeppe1> dimitern: hmm, i wouldn't mind just having life as an untypes string
<rogpeppe1> untyped
<rogpeppe1> dimitern: but others may well disagree
<rogpeppe1> dimitern: no
<dimitern> rogpeppe1: yeah, thought so
<rogpeppe1> dimitern: you'd need to define params.Life type
<rogpeppe1> dimitern: e.g. (in params) type Life string
<dimitern> rogpeppe1: so why not have type Life int8 in params and use that, so there's no fishy string conversions?
<rogpeppe1> dimitern: i'm not sure i want those constant values to be hardcoded in the javascript
<dimitern> rogpeppe1: but how about the agents?
<rogpeppe1> dimitern: i think we should choose a single representation across the whole API
<dimitern> rogpeppe1: these values has to be defined somewhere if you need to use them - strings or ints
<rogpeppe1> dimitern: sure.
<rogpeppe1> dimitern: i'd do (in params) type Life string
<dimitern> rogpeppe1: but I agree numbers in js are not real numbers, so strings is better
<rogpeppe1> dimitern: and then 'type LifeResult struct { Life Life }'
<dimitern> rogpeppe1: ok (and let the bikeshedding begin during the review :)
<rogpeppe1> dimitern: s/Result/Results/
<rogpeppe1> dimitern: i think that's relatively uncontroversial actually
<dimitern> rogpeppe1: you never know :)
<rogpeppe1> dimitern: v true :-)
<dimitern> rogpeppe1: so any API call can return: (1) an error, (2) struct, (3) both - and these are all the cases
<rogpeppe1> dimitern: yeah - read the rpc documentation!
<rogpeppe1> dimitern: it's all there
<dimitern> rogpeppe1: sorry for bugging you so much, but it's easier for me to understand things as I go (in context), then the docs will be more useful (as a reminder)
<rogpeppe1> dimitern: np. it is worth reading (and trying to understand) the docs at least once though, to start with
<dimitern> rogpeppe1: I agree, my bad
<rogpeppe1> dimitern: and if you find them incomprehensible, i want to know so i can improve them.
<dimitern> rogpeppe1: sure
<dimitern> rogpeppe1: how about doc comments - do I need to copy the state method's comment for both client and server methods?
<rogpeppe1> dimitern: i'd copy existing convention. definitely copy doc comments for the client methods, but not necessarily for the server methods
<dimitern> rogpeppe1: or the client one is more important and the server one can be shorter (e.g. EnsureDead is several lines long)
<rogpeppe1> dimitern: but... we might want that to change actually
<dimitern> rogpeppe1: right
<rogpeppe1> dimitern: if we start auto-generating docs for the API
<rogpeppe1> dimitern: but for the time being, i'd go with that
<dimitern> rogpeppe1: why in the permission tests allow specifies both machine-0 and machine-1 when (e.g. SetPassword) only affects machine 1?
<rogpeppe1> dimitern: i'm not sure i understand the question
<dimitern> rogpeppe1: should i follow this pattern?
<dimitern> rogpeppe1: opMachine1SetPassword acts on machine 1 only, not both 0 and 1
 * rogpeppe1 goes to have a look
<dimitern> rogpeppe1: I think the machine-0 in allow is not needed
<rogpeppe1> dimitern: ah, it's correct but there should be a comment
<dimitern> rogpeppe1: explain please?
<rogpeppe1> dimitern: machine-0 is allowed to set the password because it's an environment manager
<dimitern> rogpeppe1: but for all other cases only the machine we're acting upon has to be specified in allow, right?
<rogpeppe1> dimitern: yes.
<dimitern> rogpeppe1: I'll add a comment about opMachine1SetPassword in the table then
<rogpeppe1> dimitern: sounds good
<dimitern> rogpeppe1: so the func() that's being returned resets whatever the operation did?
<rogpeppe1> dimitern: yes
<rogpeppe1> dimitern: hmm
<rogpeppe1> dimitern: that may mean you need to provide a Kill operation
<dimitern> rogpeppe1: fortunately for EnsureDead it's a no-op, because it won't affect the machine which is alive
<dimitern> rogpeppe1: oh, for SSA() yeah..
<rogpeppe1> dimitern: SAA?
<dimitern> rogpeppe1: or just call pinger.Stop?
<dimitern> rogpeppe1: SetAgentAlive
<rogpeppe1> dimitern: actually, that should be fine
<dimitern> rogpeppe1: ok then
<rogpeppe1> dimitern: it doesn't matter if we're still notionally alive
<jam> dimitern, mgz, wallyworld_, danilos: greetings all
<wallyworld_> hello
<dimitern> jam: hey, just staring mumble now
<jam> danilos: are you there? We see you in mumble but you are deafened and muted
<dimitern> rogpeppe1: so wrt permissions should all these machine methods be accessible only by the machine's own agent: SetStatus, Life, EnsureDead, SetAgentAlive?
<dimitern> rogpeppe1: not sure about Life
<rogpeppe1> dimitern: not Life, but the others, yes
<rogpeppe1> dimitern: one other thing
<rogpeppe1> dimitern: the environ manager needs to be able to call SetStatus
<dimitern> rogpeppe1: why?
<rogpeppe1> dimitern: what happens when the provisioner can't provision a machine?
<dimitern> rogpeppe1: it tries again
<rogpeppe1> dimitern: forever?
<dimitern> rogpeppe1: let me check
<rogpeppe1> dimitern: we need to think about what happens if we can't bring up a machine. i'm thinking that the environ manager probably needs to be able to call EnsureDead in that case actually.
<dimitern> rogpeppe1: actually if the PA fails to start a machine it sets it to an error state and will skip it next time
<rogpeppe1> dimitern: that's what i thought
<rogpeppe1> dimitern: so it calls SetStatus, right?
<dimitern> rogpeppe1: ah, I see your point
<dimitern> rogpeppe1: yeah, m0 needs to SetStatus
<dimitern> rogpeppe1: and how about Life() - what should be in allow and deny?
<rogpeppe1> dimitern: Life shouldn't be in the perms checks
<rogpeppe1> dimitern: sorry, i just remembered :-)
<rogpeppe1> dimitern: it's just a field in params.Machine
<rogpeppe1> dimitern: not a separate API call at all
<dimitern> rogpeppe1: hmm..
<dimitern> rogpeppe1: like InstanceId ?
<rogpeppe1> dimitern: exactly like that
<dimitern> rogpeppe1: ok, I was wondering the same actually
<rogpeppe1> dimitern: sorry for not catching that earlier
<dimitern> rogpeppe1: np
<rogpeppe1> dimitern: we did mention it yesterday though
<danilos> jam, dimitern, mgz, wallyworld_: hey, sorry, got too focused on coding :/
<jam> danilos: how's it going?
<danilos> not that that's necessarily a bad thing, just forgot about the meeting
<danilos> jam, it's pretty good, thanks
<jam> danilos: finishing up the bootstrap stuff?
<danilos> jam, yeah
<dimitern> rogpeppe1: so no srvMachine.Life(), only client-side
<rogpeppe1> dimitern: yup
<rogpeppe1> dimitern: and some tests to check that refresh refreshes the life, etc
<dimitern> rogpeppe1: it'll be difficult to change the lifecycle with the current scenario
<rogpeppe1> dimitern: how do you mean?
<dimitern> rogpeppe1: all machines have assigned units, so cannot be removed or set to dying
<rogpeppe1> dimitern: isn't that the same as it is now?
<rogpeppe1> dimitern: you must remove the units, then the machine
<dimitern> rogpeppe1: ah, ok
<dimitern> rogpeppe1: and then restore the scenario
<rogpeppe1> dimitern: oh, i see, when testing
<rogpeppe1> dimitern: don't bother
<dimitern> rogpeppe1: wait - are we talking about perm checks or individual api calls tests?
<rogpeppe1> dimitern: the perms checks
<rogpeppe1> dimitern: just try to change the lifecycle and check you get the right kind of error
<dimitern> rogpeppe1: yeah - i could add a hanging machine with no units that can change life
<rogpeppe1> dimitern: don't even bother to do that
<dimitern> rogpeppe1: sgtm
<rogpeppe1> dimitern: just try to change the life of one of the machines
<rogpeppe1> dimitern: you should get a "machine has units" error
<rogpeppe1> dimitern: if you don't, there's a problem
<dimitern> rogpeppe1: that's exactly what i did :)
<rogpeppe1> dimitern: but for the individual API test, you should create a machine and test that it actually works
<dimitern> rogpeppe1: sure
<dimitern> rogpeppe1: does this look ok to you? http://paste.ubuntu.com/5686802/
<dimitern> rogpeppe1: because i'm getting permission denied
<rogpeppe1> dimitern: i think i'd implement isOwnAgent on srvRoot, not srvMachine - it's quite generic
<rogpeppe1> dimitern: (not that that effects the correctness of the code)
<dimitern> rogpeppe1: i though about that
<dimitern> rogpeppe1: but otherwise?
<rogpeppe1> dimitern: what's the actual output from your test
<rogpeppe1> dimitern: (yes, i do think i see a potential problem)
<dimitern> rogpeppe1: http://paste.ubuntu.com/5686827/ (disregard the messed up Life value - fixed that)
<rogpeppe1> dimitern: look at opClientResolved
<rogpeppe1> dimitern: you've got a spelling error in your error message for a start
<rogpeppe1> dimitern: and if you get a permission-denied error, you should return it
<rogpeppe1> dimitern: on line 29, you can just do c.Assert(err, IsNil)
<dimitern> rogpeppe1: so I should expect api.CodeUnauthorized at first (Machine("1")) ?
<rogpeppe1> dimitern: hmm, no you can't
<rogpeppe1> dimitern: hmm, interesting
<dimitern> rogpeppe1: I was wondering why about that - why check + return empty func instead of assert
<rogpeppe1> dimitern: because the actual permission-denied error will happen when trying to get the machine, not on the later call
<dimitern> rogpeppe1: ah..
<dimitern> rogpeppe1: i'll poke it a bit more
<rogpeppe1> dimitern: i think it (and opMachine1SetPassword) could use some improvement actually
<rogpeppe1> dimitern: they're not actually testing the right thing.
<dimitern> rogpeppe1: but first - if I move isOwnAgent into srvRoot, how can I get the m.m.Tag() there?
<rogpeppe1> dimitern: maybe call it m.root.loggedInAs(m.Tag())
<dimitern> rogpeppe1: there's no m anymore
<dimitern> rogpeppe1: I have r *srvRoot only
<rogpeppe1> dimitern: i mean in EnsureDead, you'd call m.root.loggedInAs(m.Tag()) rather than m.isOwnAgent()
<dimitern> rogpeppe1:  i see.. would've liked to shorten this somehow
<dimitern> rogpeppe1: because I have also isOwnAgentOrManager - extracted the check from setpassword
<dimitern> rogpeppe1: maybe I can make it loggedInAs(tag, job) - and job can be nil
<rogpeppe1> dimitern: hmm, no i don't think that's quite right.
 * rogpeppe1 thinks
<dimitern> rogpeppe1: that's what I need, but the name might be better
<dimitern> *could b* better
<dimitern> rogpeppe1: hasPermission(tag, *job) ?
<rogpeppe1> dimitern: i'm trying to think of a good name too
<rogpeppe1> dimitern: this is going to be used a lot, so it's worth coming up with something that works well
<dimitern> rogpeppe1: canAccess? are you ok with the *job arg?
<rogpeppe1> dimitern: i'm not sure. i think i'd prefer to keep the two checks separate.
<dimitern> rogpeppe1: we can have multiple: canAccess(tag), canManage(tag, job) ?
<dimitern> or just canManage and hard code JobManageEnviron check inside
<rogpeppe1> dimitern: i don't find the meanings of those verbs very obvious
<dimitern> rogpeppe1: why?
<rogpeppe1> dimitern: what does canAccess mean? who can access what?
<dimitern> rogpeppe1: we're asking "can <user> access <tag>"?
<rogpeppe1> dimitern: the answer to that depends on the operation, no?
<dimitern> rogpeppe1: not really - it's ancilliary to the operation
<rogpeppe1> dimitern: the verb seems to imply that it's universal, to me at any rate.
<dimitern> rogpeppe1: I mean the operation itself has to define what is done
<dimitern> rogpeppe1: that's why I started with isX
<dimitern> rogpeppe1: userCanAccess? :)
<dimitern> rogpeppe1: it has to be not to long as well
<rogpeppe1> dimitern: personally i think readability is probably more important than length here.
<dimitern> rogpeppe1: ok, let's dice it step by step - 1) get logged in user's tag, 2) check it matches the passed entity's tag
<dimitern> rogpeppe1: userMatches(x) -> e.Tag() == x.Tag()
<rogpeppe1> dimitern: how about "isOwner" as a name
<rogpeppe1> dimitern: as in m.root.isOwner(m.m)
<rogpeppe1> hmm, not quite right too
<dimitern> rogpeppe1: sgtm
<dimitern> rogpeppe1: we're coming back to isOwnAgent :)
<dimitern> "agent" here in a more general sense (s/o acting on behalf of)
<rogpeppe1> dimitern: yeah, i didn't really have a problem with isOwnAgent except the name and the fact we'd need to duplicate similar code for other entities.
<rogpeppe1> dimitern: although, come to think of it, probably only srvUnit
<dimitern> rogpeppe1: no, I mean have isOwnAgent in srvRoot
<dimitern> rogpeppe1: actually you're right - we don't need other perm checks like that (what other agents are there that "own" entities?)
<dimitern> units and machines only
<rogpeppe1> dimitern: yeah
<rogpeppe1> dimitern: how about this? http://paste.ubuntu.com/5686906/
<rogpeppe1> dimitern: then we can use the general Auth suffix to apply to various kinds of authorization primitive
<rogpeppe1> dimitern: where ownerAuth is your isOwnAgent by another name
<dimitern> rogpeppe1: i like it - we can have envrionManagerAuth on srvRoot actually
<rogpeppe1> dimitern: ah yes, good point
<dimitern> rogpeppe1: sgtm, will make it so
<rogpeppe1> dimitern: cool, thanks for bearing with me
<rogpeppe1> dimitern: hmm, maybe "auth" would be better as a prefix rather than a suffix
<rogpeppe1> dimitern: i can't quite decide
<dimitern> rogpeppe1: i thought the same
<rogpeppe1> dimitern: in which case, let's do that
<dimitern> rogpeppe1: let's make it a prefix and also - how about putting authOwner on srvRoot getting x with .Tag()?
<dimitern> rogpeppe1: so if !m.authOwner(m.m) && !m.authEnvironManager() { return errPerm }
<rogpeppe1> dimitern: sgtm
<dimitern> rogpeppe1: cool
<dimitern> rogpeppe1: what did you say can be improved about opMachine1SetPassword?
<dimitern> rogpeppe2: kanban
<dimitern> rogpeppe2: I cannot test SetAgentAlive properly without having AgentAlive() as well
<dimitern> rogpeppe2: and using the state machine's AgentAlive won't work because the pingers are different
<dimitern> rogpeppe2: but I can just call it and see it returns no error and calling stop on the returned pinger is also successful
<rogpeppe2> dimitern: for the permissions check or the actual test?
<dimitern> rogpeppe2: actual test
<dimitern> rogpeppe2: permissions test fine
<rogpeppe2> dimitern: why can't you use AgentAlive on the state Machine ?
<dimitern> rogpeppe2: it returns false after api.SetAgentAlive
<rogpeppe2> dimitern: really? that seems a bit weird.
<dimitern> rogpeppe2: hmm.. maybe something's wrong with my pinger code
<rogpeppe2> dimitern: ah, you probably need to call state.Sync
<dimitern> rogpeppe2: oh, forgot about that :)
<dimitern> rogpeppe2: all done and all tests pass!  https://codereview.appspot.com/9614043
<rogpeppe2> dimitern: woo!
<dimitern> rogpeppe2: scrutiny appreciated :)
<rogpeppe2> dimitern: looking now
<rogpeppe2> dimitern: reviewed
<dimitern> rogpeppe2: tyvm
<dimitern> rogpeppe2: the issue with setstatus is you cannot change it back to pending once it's advanced
<dimitern> rogpeppe2: that's why I added default status and changed the scenario
<rogpeppe2> dimitern: ah
<rogpeppe2> dimitern: so you have to set the correct status
<dimitern> rogpeppe2: what do you mean?
 * rogpeppe2 thinks we do make life hard for ourselves sometimes. more code breeds more code.
<rogpeppe2> dimitern: ah, i see, it's just pending you can't rewind
<dimitern> rogpeppe2: yes
<dimitern> rogpeppe2: and anyways, i think the scenario should have all machines in started status
<TheRealMue> dimitern: another review
<rogpeppe2> dimitern: you may well be right.
<dimitern> TheRealMue: cheers
<dimitern> TheRealMue: I feel the same about the 1 or 2 char vars throughout, but changing that is out of scope I think
 * dimitern thinks now once I picked up speed I can ravage through the next agent's needed API calls easily (PA I think) :)
<TheRealMue> dimitern: But otherwise it never will be changed. ;)
<dimitern> TheRealMue: not quite - can be changed independently as a whole
<dimitern> rogpeppe2: HasAssignedUnitsError is a tricky one - what should the CodeHasAssignedUnits be? it's not a const
<rogpeppe2> dimitern: it should be a const like the others
<dimitern> rogpeppe2: it can be a format string, but that will duplicate the logic in state
<rogpeppe> dimitern: i'm not sure i see the problem.
<rogpeppe> dimitern: why can't it just be a const code like the other codes?
<dimitern> rogpeppe: func (e *HasAssignedUnitsError) Error() string { return fmt.Sprintf("machine %s has unit %q assigned", e.MachineId, e.UnitNames[0]) }
<rogpeppe> dimitern: the message is separate from the code
<dimitern> rogpeppe: ah, ok
<rogpeppe> dimitern: take a look at the serverError implementation
<dimitern> rogpeppe: ok, all done, will repropose soon and I'd appreciate one last look (running tests now)
<rogpeppe> dimitern: ok
<dimitern> rogpeppe: https://codereview.appspot.com/9614043
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: thanks
<dimitern> rogpeppe: I don't get what's wrong with the error code check in opMachine1EnsureDead - do you mean to move the error check into a test similar to TestMachineEnsureDead instead?
<rogpeppe> dimitern: yes. the op checks are only for sanity - they should not be used to test specific functionality
<dimitern> rogpeppe: ok
<dimitern> rogpeppe: will do that and land it then
<dimitern> rogpeppe: ah, perhaps you meant add it to errorTransformTests ?
<dimitern> rogpeppe: or that as well as the separate test case for EnsureDead?
<mgz> dimitern: poke me if any of my review comments are unclear or just silly
<dimitern> mgz: already replied :)
<Makyo> Anyone up for a review of https://codereview.appspot.com/9566045/ - refactor deploy to statecmd?
<mgz> dimitern: you can carry forward my lgtm
<dimitern> mgz: cheers
<mgz> Makyo: done, poke rog or dimitern as well though
<Makyo> mgz, cool, thanks.  Will poke around the args, too, see what can be reduced.
 * rogpeppe has got to go. g'night all
<rogpeppe> dimitern: yeah, it should be in errorTransformTests
<rogpeppe> dimitern: which probably means you don't need a separate test for the error return
<dimitern> rogpeppe: well I did both
<dimitern> rogpeppe: they're different anyway
<dimitern> rogpeppe: one tests the flow and the other changes to serverError
<dimitern> ok, i'm off - mass effect is calling for me ;)
<dimitern> g'night all
<thumper> morning mramm
<mramm> morning
<thumper> morning wallyworld_
<wallyworld_> g'day
<wallyworld_> thumper: in your branch, i can't see where DescriptionTimeout is used to kill the process
#juju-dev 2013-05-22
<thumper> wallyworld_: ah, removed that
<thumper> wallyworld_: any reference there now is by mistake
<wallyworld_> ah ok :-)
<wallyworld_> i'll let you fix that and take another look after i finish this bit of work i'm doing
<thumper> kk
<thumper> I'm beginning to realise that my mental model of the state stuff is just plain wrong
<wallyworld_> at least you had a mental model
<davecheney> ooooooooooooh, juju totally doens't work with the mongodb we ship with raring
<davecheney> that is a slight problem
<thumper> davecheney: plz fix, kthxbye
<thumper> :)
<thumper> dimitern: hey there
<thumper> dimitern: for the tasks you are doing or done, put your launchpad id in square brackets at the start
<dimitern> thumper: heyhey
<dimitern> thumper: in the blueprint?
<thumper> dimitern: yeah, the work items
<dimitern> thumper: ok, will do
<thumper> so like this: [thumper] doing foo: INPROGRESS
<thumper> dimitern: I'm just touching up the last part of plugins, using text/template etc
<dimitern> thumper: ah, cool!
<dimitern> thumper: my point (not that i explained it properly i think) with time.duration vs. floats is you can convert one to the other when generating the script
<dimitern> thumper: and still have a nice to read code rather than floats
<thumper> hmm... yeah, perhaps for a public thing, but really worth it for a test?
<thumper> also, with some refactoring on the templates, things are more obvious
<dimitern> well, tests are people too :)
<dimitern> i mean we need to read and maintain them as well as other code, and readability helps (with that i'll leave it to your discretion, i don't absolutely insist on it)
<dimitern> thumper: wrt to api authorization, who do you think should be allowed to call machine.Series(), m.Constraints() and m.Status() - everyone, only admins or only the environment manager node?
<dimitern> thumper: the cli will need to use all of these in different commands, so maybe restrict them to whoever can use the cli and stop there?
<thumper> dimitern: sorry, not thought too much about api auth, so not much help here
<dimitern> thumper: np. how's it going in containerland?
<thumper> dimitern: emailed people earlier
<dimitern> thumper: ah, missed that, thanks
<dimitern> thumper: have you seen my mail about the copyrights headers?
<dimitern> thumper: reading the containers page you wrote, and i remember a discussion we had with william on the way back from oakland about nesting:
<dimitern> thumper: the idea basically was to expand machine-id to contain a "path" like <root-id>/<type>:<child-id>/<type>:<child-id>/... where all ids are still numbers, and the type is the container type used (lxc, kvm, etc.) e.g. "0/lxc:1/kvm:2"
<dimitern> thumper: this has some advantages: (1) scoping/namespacing, (2) ability to "move" up the "tree" when needed (e.g. set an ip address request going down from root to most deeply nested level, then back up (setting up networking on each level, etc).
<thumper> yeah, I remember the discussion, but we also talked about having the machine namespace flat
<dimitern> thumper: not sure it's the same discussion (i wasn't there on the one you mention, we talked about this later), but william seemed to like the idea; anyways just mentioning lest i forget
 * dimitern bbiab
 * thumper is away now
<thumper> dimitern: if I have time.Duration in the struct instead of a float, how do I get the Seconds in the template?
<thumper> with a float {{.Seconds}} works
<thumper> sorry, with a float {{.Sleep}} works
<thumper> but I need Sleep.Seconds()
<thumper> can't see how to do that in the template
 * thumper now away to eat
<dimitern> thumper: well, i suppose something like this should work: http://play.golang.org/p/5g5fsv-3zn
<dimitern> thumper: btw sleep(1) also supports arguments like "1m 23s"
<thumper> dimitern: that isn't inside the template {{ ... }} bits
<dimitern> thumper: look at {{printf}}
<TheMue> *: morning
<thumper> dimitern: nm {{.Sleep.Seconds}} works
<dimitern> TheMue: hiya
<dimitern> wallyworld_: consistency isn't everything you know :)
<thumper> dimitern: last proposal updated, I'm signing off for the night now
<thumper> ciao everyone
<rogpeppe1> mornin' all
<TheMue> rogpeppe1: morning
<dimitern> rogpeppe1: morning
<dimitern> rogpeppe1: i have a question already
<dimitern> :)
<rogpeppe1> dimitern: ok
<dimitern> rogpeppe1: so in order to implement state.AllMachines() -> []*state.Machine, error
<dimitern> rogpeppe1: what should I put in AllMachinesResults.Machines? []params.Machine or params.MachineInfo? the client expects to have actual machine objects there
<rogpeppe1> dimitern: []params.Machine
<rogpeppe1> dimitern: as long as we're keeping Machine separate from MachineInfo
<dimitern> rogpeppe1: ok
<rogpeppe1> dimitern: or []*params.Machine if it's more convenient
<dimitern> rogpeppe1: and then the user can do like machines[0].Id() right?
<rogpeppe1> dimitern: api.State.AllMachines should look just as it does in state
<rogpeppe1> dimitern: so you'll need to map the slice, making a *api.Machine for every params.Machine
<dimitern> rogpeppe1: wait.. actually params.Machine is not right - I need apiclient.Machine
<dimitern> rogpeppe1: oh yeah
<dimitern> rogpeppe1: thanks
<rogpeppe1> dimitern: np
<dimitern> rogpeppe1: already implemented LifecycleWatcher and EnvironConfigWatcher - no trouble so far
<rogpeppe1> dimitern: cool
<dimitern> rogpeppe1: hmm.. params does not import api though - will there be a problem if I import it?
<rogpeppe1> dimitern: yes
<rogpeppe1> dimitern: why do you want to import api into params?
<dimitern> rogpeppe1: to define the slice of machines
<dimitern> rogpeppe1: in AllMachinesResults
<rogpeppe1> dimitern: that must be []params.Machine, no?
<dimitern> rogpeppe1: ah, it'll be api.Machine at client side, i see
<rogpeppe1> dimitern: yup
<dimitern> rogpeppe1: do you think it's worth having if len(machines) == 0 { return params.AllMachinesResults{}, nil } (or maybe Machines: make([]*params.Machine, 0) is better) ?
<rogpeppe1> dimitern: i don't mind. whatever is easiest.
<dimitern> rogpeppe1: wasn't sure how to convey best the case "empty slice"
<dimitern> rogpeppe1: but it's up to the client I guess anyway
<rogpeppe1> dimitern: i think it may be better to always return an empty slice. that way the json representation is [] not nil.
<dimitern> rogpeppe1: yeah, that what i was thinking
 * rogpeppe1 wishes a nil slice encoded as json [] anyway
<dimitern> rogpeppe1: strictly speaking a nil slice should be either null or undefined in json
<rogpeppe1> dimitern: i'm not sure. the difference between a nil slice and an empty slice in Go is a subtle one that really isn't that useful to leak out into json.
<rogpeppe1> dimitern: i can't see a particular use case for allowing a slice to encode as null
<dimitern> rogpeppe1: yeah actually json doesn't have such a distinction
<rogpeppe1> dimitern: well anything can be null in json
<dimitern> rogpeppe1: i'm thinking it's worth having some helper to convert from state.Machine to params.Machine, so we have it at once place, right?
<rogpeppe1> dimitern: definitely
<dimitern> rogpeppe1: so can we assume all places using state.IsNotFound(err) will be changed to use api.ErrCode(err) == api.CodeNotFound?
<dimitern> rogpeppe1: or we need a helper for each such case
<dimitern> rogpeppe1: ping
<rogpeppe1> dimitern: pong
<dimitern> rogpeppe1: ^^
<rogpeppe1> dimitern: ah, sorry, i didn't see that
<rogpeppe1> dimitern: yes, we should assume that
<dimitern> rogpeppe1: cool
<rogpeppe1> dimitern: although we could add a help too if we wanted
<rogpeppe1> helper
<dimitern> rogpeppe1: let's see how it'll work when we start changing the agents
<rogpeppe1> dimitern: +1
<dimitern> rogpeppe1: i'm having difficulty with state.AllMachines at the client side - what should I put in the objType arg of call() ?
<rogpeppe1> dimitern: i think we probably need a srvState type
<dimitern> rogpeppe1: ok, will add it then
<rogpeppe1> dimitern: and srvRoot.State(id string) (*srvState, error)
<rogpeppe1> dimitern: then you'll use State as the obj type
<dimitern> rogpeppe1: yeah, like the other objects
<rogpeppe1> dimitern: yup
<dimitern> rogpeppe1: but what about st.id ?
<dimitern> rogpeppe1: i can add it but how shoud i initialize it?
<rogpeppe1> dimitern: st.id ?
<dimitern> rogpeppe1: st.call("State", st.id ...
<rogpeppe1> dimitern: ah, i think we should enforce it as blank
<dimitern> rogpeppe1: in srvRoot State(id string) ?
<rogpeppe1> dimitern: at some point in the future, we'll potentially be able to connect to several states through the API, and then the id will be useful
<rogpeppe1> dimitern: yes
<dimitern> rogpeppe1: ok
<rogpeppe1> dimitern: same as with Client
<danilos> whoops, forgot about OCR :)
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: danilos | Bugs: 8 Critical, 68 High - https://bugs.launchpad.net/juju-core/
<rogpeppe1> danilos: hiya
<danilos> rogpeppe1, heya
<dimitern> danilos: yo
<dimitern> rogpeppe1: actually AllMachinesResults now will have Machines []string instead (ids) and client-side I'll range over it and call st.Machine(id) for each, right?
<rogpeppe1> dimitern: i don't think you should do that
<rogpeppe1> dimitern: i think you should return all the info
<dimitern> rogpeppe1: I only need the id to wrap it with a api.Machine
<dimitern> rogpeppe1: but that I don't have in params.Machine
<rogpeppe1> dimitern: i don't think you want an extra n round trips
<rogpeppe1> dimitern: ah, well params.Machine should have it
<dimitern> rogpeppe1: all details?
<dimitern> rogpeppe1: or only the ones I need right now
<rogpeppe1> dimitern: just what you need right now
<dimitern> rogpeppe1: ok, this also means changing srvMachine.Get() to return the Id (now both this and the range loop in AllMachines use the same helper: stateMachineToParams(stm *state.Machine) *params.Machine)
<rogpeppe1> dimitern: that seems fine. it should only affect stateMachineToParams, right?
<dimitern> rogpeppe1: yeah
<dimitern> rogpeppe1: and I'll have the reverse helper in apiclient: paramsMachineToAPI(?)
<rogpeppe1> dimitern: sounds reasonable.
<dimitern> rogpeppe1: except I can't do it that
<rogpeppe1> dimitern: no?
<dimitern> rogpeppe1: ah, no sorry api.Machine has doc which is params.Machine
<rogpeppe1> dimitern: yu
<rogpeppe1> p
<dimitern> rogpeppe1: who should be able to call m.Constraints() and m.Status() wrt permission checks? everyone (deny: "user-admin", "user-other") ?
<rogpeppe1> dimitern: i think allow: "machine-0", "machine-1"
<dimitern> rogpeppe1: how about juju status?
<rogpeppe1> dimitern: that wil be part of the client API
<dimitern> rogpeppe1: really? that simplifies things then ok
<rogpeppe1> dimitern: yeah, we already have an isAgent check in srvRoot.Machine, i think
<dimitern> rogpeppe1: strictly speaking only the environ manager uses these two now
<dimitern> rogpeppe1: so allow "machine-0" to be as restrictive as possible
<rogpeppe1> dimitern: i think it's reasonable that a machine can get its own info
<dimitern> rogpeppe1: it *sounds* reasonable but we never cared about this in the machiner
<rogpeppe1> dimitern: i'm ok if you want to deny access to everything but the environ manager
<dimitern> rogpeppe1: let's do that for now, we can expand it later when we get to the MA itself and the other tasks
<jam> wallyworld_: is there anything particularly in your chain of branches that I can unblock. I've reviewed a bunch of stuff, but I don't have the knowledge of the thread that you do.
<wallyworld_> jam: thanks for asking. just one more :-) https://codereview.appspot.com/9606046/
<wallyworld_> thanks very much. you and dimitern have been very awesome in doing all the reviews
<jam> wallyworld_: yeah, I was looking at that one now
<jam> though it seems not in-the-critical-path to getting the others landed
<wallyworld_> it needs to be done so that https://codereview.appspot.com/9603044/ can land
<jam> danilos: poke
<wallyworld_> danilos: if you want a review to do https://codereview.appspot.com/9545045/
<wallyworld_> it's an easy one
<danilos> wallyworld_, you've got conflicts in there (or at least the conflict markers)
 * wallyworld_ checks
<wallyworld_> jam: i got permission denied trying to add you/mark etc as a watcher on my rt #61613
<_mup_> Bug #61613: Can't run liferea with main window hidden <Dapper Backports:Fix Released> <liferea (Ubuntu):Fix Released> <https://launchpad.net/bugs/61613>
<jam> wallyworld_: I get an error as well.
<wallyworld_> :-(
<dimitern> rogpeppe1, TheMue: sorry guys I'll have to skip kanban today - i did move all my cards accordingly
<jam> wallyworld_: try just replying to the ticket, and then including john.meinel@canonical.com in a CC field
<rogpeppe1> dimitern: np
<jam> wallyworld_: I just sent an email to derykc
<jam> deryck
<wallyworld_> i just cc'ed
<jam> wallyworld_: I saw the email
<wallyworld_> that was quick
<wallyworld_> i only *just* hit send
<jam> wallyworld_: though it just puts me in a one-time message
<jam> vs the default CC
<jam> so we want someone to update it.
<wallyworld_> better than nothing i guess
<TheMue> dimitern|afk:ok
<hazmat`> mgz, ping
<TheMue> rogpeppe1: btw, I know found on my own what I wanted to ask by digging into the whole coudinit stuff
<rogpeppe1> TheMue: cool
<jam> mgz: I figured out the bug for tarmac
<danilos> jam: btw, the roundtrips to mongo are 70ms? that sounds like a lot: is that on the unix socket or over tcp/ip?
<jam> danilos: that is running from the client sitting in the hotel in Oakland
<jam> stuff like 'juju status' is evaluated client-side ATM
<danilos> jam: oh, right, I see
<jam> mgz: It turns out that Tarmac uses launchpad's API to find "bzr_identity" for a given branch.
<danilos> jam: if it went through api server it would have gone much quicker, understood
<jam> so if you say [lp:~go-bot/goose/trunk] tarmac connects to lp, but then tries to load the config item for lp:goose
<jam> (the canonical name)
<jam> and since that doesn't have any config, it just says "ok, everything Approved is ready to land"
<jam> danilos: presumably the api server has much lower overhead, yes.
<jam> mgz: which also explains why everything worked in testingb
<jam> because it *wasn't* lp:goose yet
<danilos> jam: ack, thanks for explaining it
<rogpeppe1> danilos, jam, mgz, TheMue: i'd appreciate a review of this: https://codereview.appspot.com/9643043
<danilos> rogpeppe1, taking a look
<TheMue> rogpeppe1: *click*
<TheMue> rogpeppe1: done
<rogpeppe1> TheMue: thanks
<rogpeppe1> TheMue: responded
<TheMue> rogpeppe1: *clickAgain*
<niemeyer> Heya
<niemeyer> rogpeppe1: We have a meeting in ~50
<rogpeppe1> niemeyer: ah, thanks - i hadn't seen an invite
<rogpeppe1> niemeyer: anything you want to chat about prior to it?
<niemeyer> rogpeppe1: You seem to be there in the event
<niemeyer> rogpeppe: Hmm.. given it's still a preliminary meeting, we can probably exchange ideas during it
<niemeyer> rogpeppe: Have you read that document + comments?
<rogpeppe> niemeyer: not ye
<rogpeppe> t
<TheMue> rogpeppe: replied again
<niemeyer> rogpeppe: That might be good to have a quick look before the call
<rogpeppe> niemeyer: i definitely will do
<rogpeppe> danilos: responded
<danilos> rogpeppe, LGTM, worker.Runner/worker.Worker does sound better, but ultimately your call (this might make it easier to extract in the future)
<allenap> bac: Eyup lad. Are you using anything like socket.io in juju-gui?
<bac> allenap: no
<allenap> bac: Just plain websocket, or something in YUI? Btw, "eyup lad" is a greeting in gmblish.
<allenap> In case you didn't know whether to feel insulted or not.
<bac> allenap: i always default to insulted until proven otherwise
<bac> allenap: we use the reconnecting-websocket.js package
<allenap> bac: Was there a conscious decision to use that over something else, or was it just convenient and close to hand? (Wow it's small; that's neat.) Would you recommend it?
<bac> allenap: i wasn't in on that decision.  you may want to quiz hazmat
<hazmat> allenap, its a very thin layer over websocket.. to you know reconnect ;-)
<hazmat> allenap, we're basically using a raw  websocket, the gui has several abstractions built on that.. the gui itself while yui.. focuses mostly on the backbone app framework style of yui
<hazmat> allenap, socket.io is basically just particular conventions around websocket,  pub/sub .. chatrooms, etc.  most of which aren't relevant here.
<allenap> hazmat: Have you had any issues with websockets that might have been dealt with by something like socket.io, or does the reconnecting logic cover it?
<hazmat> allenap, none
<allenap> hazmat: Brilliant, thanks, and thank bac too.
<allenap> s/thank/thanks
 * hazmat relocates
 * rogpeppe is done for the day. see y'all tomorrow.
<dpb1> jam: any way to get some action on #1182224  I think it should be quite easy, at least from the outside looking in?
<_mup_> Bug #1182224: relation-list returns null with json-output <juju-core:New> <https://launchpad.net/bugs/1182224>
<thumper> good morning juju folks
<kyhwana> Wait, it is actually morning somewhere else!
<thumper> sure...
<kyhwana> thumper: oh, that's because you're in the same country as me
<thumper> 9:21am in NZ
<thumper> kyhwana: where are you?
<kyhwana> The 'tron
 * thumper doesn't know where the 'tron is
<kyhwana> O.o
<kyhwana> Hamiltron?
<thumper> oh...
 * thumper is in Dunedin, so not up with all that north island slang
<kyhwana> ahh
<thumper> kyhwana: what do you do for work, and what's your interest level with juju?
 * thumper is curious
<kyhwana> thumper: I'm a "systems engineer" for a uh, company that makes "packet acquisition and generation" cards+appliances.
<kyhwana> and interest in juju is mostly personal admin stuff
 * thumper nods
<kyhwana> though, I bet I could use it to deploy build environments. Hmm
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: - | Bugs: 8 Critical, 68 High - https://bugs.launchpad.net/juju-core/
<sidnei> thumper: re https://launchpad.net/bugs/1182224 seems the problem is in json.Marshal, it should marshall an empty list to [] not nil: http://play.golang.org/p/QIMUVJMVPR
<_mup_> Bug #1182224: relation-list returns null with json-output <papercut> <uniter> <juju-core:Triaged> <https://launchpad.net/bugs/1182224>
<thumper> sidnei: it does
 * sidnei just wrote his first go program :)
<thumper> sidnei: it isn't bein given an empty slice
<thumper> sidnei: it is a nil
<thumper> which relates to null in json
 * thumper is fixing the code
<sidnei> sure it's a nill not an empty slice?
 * sidnei staring at uniter/context.go
<thumper> sidnei: I did
<sidnei> UnitNames starts with units []string, and returns that at the bottom, i would expect it to be an empty slice or whatever that's called
<thumper> sidnei: it starts with an uninitialized slice
<thumper> which is nil
 * thumper is adding one line of code
<sidnei> i see, uninitialized
<thumper> 	units = []string{}
<thumper> append can handle a nil to append to
<thumper> and interestingly, if you have 'var value []string'
<thumper> v := reflect.ValueOf(value)
<thumper> v.Len() == 0
<thumper> v.IsNil() -> true
<thumper> v.Kind() -> "slice"
<thumper> so the HasLen matcher succeeds with 0 on nil
<thumper> I've changed the tests to be "DeepEquals, []string{}"
<sidnei> thumper: so why this results in nil: http://play.golang.org/p/6T6mLG2FY0
<thumper> oh poos
<sidnei> phew, so im not crazy
 * thumper pokes it with a stick
 * sidnei goes drink a beer
<thumper> hmm..
<thumper> sidnei: you are crazy
<thumper> sidnei: it just took me some time to work it out
<thumper> sidnei: http://play.golang.org/p/xHa0rd-0cn
<mwhudson> has there been any progress with juju-core deploying to armhf recently?
<thumper> mwhudson: I believe it is being looked at actively
<thumper> mwhudson: particularly as with go 1.1, it fixes some things
<thumper> we are pushing to get go 1.1 in the repos, and arm is kinda dependent on that
<mwhudson> ok
<mwhudson> i think that was going to be my next question :)
<thumper> we are getting arm deployments into a CI system for us
<mwhudson> is there a ppa with go 1.1 in it yet?
<thumper> probably
<thumper> look for ~gophers
<mwhudson> ... for armhf
<thumper> hmm... I think PPAs still need special tweaking to build for armhf don't they?
<thumper> as it isn't a default target (ISTR)
<mwhudson> yeah
<mwhudson> is go mostly written in go?
<mwhudson> i can imagine there are bootstrapping issues looming
<thumper> possibly
<thumper> I think davecheney has tested arm locally
<thumper> and it is all good
<thumper> but for what value of good, I don't know
 * thumper is spending time considering jam's comments
#juju-dev 2013-05-23
<hazmat> davecheney, any comment re juju for armhf? ^
 * hazmat notices previous nick ref too late
<davecheney> hazmat: recap ?
<hazmat> davecheney, small. but http://paste.ubuntu.com/5692238/ mwhudson's been asking around
<davecheney> TTBOMK go 1.1 is not avilalble in any ubuntu form for armhf
<davecheney> 1.1 was imported from upstream into saucy proposed, but it didn't build on armhf
<davecheney> that is all I know at the moment
<davecheney> If you're happy to use a tarball, http://dave.cheney.net/unofficial-arm-tarballs
<wallyworld_> davecheney: how closed is the goamz group? I have a goamz branch i want to land. should i bother asking for group membership or is it best to get one of the group members to land it?
<davecheney> wallyworld_: dunno
<davecheney> is this another 'we're all not members of gophers' problem ?
<wallyworld_> i'm not a member of ~goamz
<wallyworld_> which owns goamz project
<wallyworld_> you and rodger are
<thumper> wallyworld_: congrats on landing all those branches
<wallyworld_> thumper: yeah, finally!
<davecheney> wallyworld_: well, that is fucked
<davecheney> let me see if I can fix that
<wallyworld_> davecheney: thanks!
<thumper> davecheney, wallyworld_: here is a trivial branch to review: https://codereview.appspot.com/9677044/
<davecheney> nope, can't fix
<davecheney> gustavo owns it
<wallyworld_> restricted team :-(
<wallyworld_> ok, will ask him
<thumper> davecheney: we can fix it... with the right request :)
<mwhudson> davecheney: ok thanks
<wallyworld_> i just wasn't sure how protective he is of that project
 * wallyworld_ looks at thumper's branch
 * thumper considers a constraint of "machine=0"
<thumper> --force-machine is all well and good, but doesn't capture intent in state
<thumper> I'm trying to work out a way to have state represent the desire for maximum density...
<thumper> a.k.a. cheap solution with everything on one ec2 instance
<wallyworld_> s/ec2/openstack as well :-)
<thumper> wallyworld_: exactly
<thumper> wallyworld_: I've been reading through jam's comments on the wiki page
<thumper> I don't want to go too far down the path of explicitness when we know we need intent
<wallyworld_> thumper: force-machine was always a *very* short term quick fix AFAIUI, so we had better not propogate it's use
<thumper> that is my thinking too
 * thumper considers a multitude of edge cases
<wallyworld_> i think the comments we more along the lines of - we can do the mechanics first (and require explicitness), and once the machinery is there, automate the intent
<thumper> juju deploy -n 2 wordpress --constraints "machine:0"
<wallyworld_> which i agree with at first read
<thumper> should barf
<wallyworld_> what does -n do?
<thumper> yeah, I've been thinking about that too
<thumper> deploy two units
<thumper> of the service
<wallyworld_> for HA?
<wallyworld_> load balancing
<thumper> yes
<thumper> although...
<thumper> would be possible if (and only if) containerised
<wallyworld_> yes, makes sense to allow that scenario for testing, even if on the one physical machine
<thumper> however...
<thumper> what happens if you have the initial environment...
<thumper> where everything was deployed onto machine/0
<thumper> then you want to scale out
<thumper> and add two more units of "foo"
<thumper> you don't want it to use the contstraint "machine:0:
<thumper> you want it to make new machines...
<thumper> ish
<thumper> maybe
<thumper> geez
<wallyworld_> so you wouldjust "juju deploy foo" without the constraint bit?
<wallyworld_> then it would allocate a new machine?
<thumper> no, it is deployed
<thumper> you use add-unit
<wallyworld_> right, yes
<thumper> it already has constraints though
<thumper> how about a constraint...
<wallyworld_> so you override those?
<thumper> yeah
<thumper> but I image it would get icky
<wallyworld_> perhaps last specified constraint wins
<wallyworld_> or maybe a constraint for "just this one operation"
<thumper> yeah...
<wallyworld_> eg policy is to allocate a new machine, but just for this add-unit, do it here
<thumper> a unit constraint
<thumper> rather than a service constraint
<wallyworld_> yes
<thumper> that _may_ work
<wallyworld_> still complicated for users to internalise the model though
<wallyworld_> juju status would need serious work i reckon
<thumper> yeah, I'd prefer something semantically simple
<thumper> to work with maximum unit density
<wallyworld_> i reckon we need a "tell me what you are about to do but let me conform first" switch
<wallyworld_> confirm
<thumper> I think the default will soon be to have machines with lots of units on them
<thumper> one machine to do multiple things
<wallyworld_> i've been wanting that for ages
<wallyworld_> didn't understand why we never had it to start with
<thumper> yeah, unit density is the primary outcome for containers I think
<thumper> because it is simpler to start a new machine for everything?
<wallyworld_> maybe
<thumper> the trick is to record intent
<thumper> but to have intent flexible
<thumper> and easily understandable
 * thumper waves his hands
<thumper> and magic happens
<thumper> that shit
<wallyworld_> USB plugged into the back of users' heads should do the trick
<thumper> wasn't there a movie ages ago called "free jack" ?
<wallyworld_> not sure, perhaps
<wallyworld_> maybe only in NZ
<wallyworld_> thumper: so we still have lots to discuss before we can fire up our IDEs and start hacking me thinks
<thumper> wallyworld_: well, there should be some things we can start on soon
<thumper> wallyworld_: but the subtleties are the hard bit
<thumper> and I'd like williams input
<thumper> when he is back next week
<wallyworld_> yes. i fear even the "simple" things could turn nasty unless we fully understand the solution we are aiming for
<wallyworld_> although if we agree containers are machines with parent machines we could start reworking the relevant interfaces
<thumper> yes
<thumper> and having a create-machine command could be useful
<thumper> we already have a destroy-machine
<wallyworld_> as in create-machine just ultimately calls the provider's startInstance() and registers the machine in the env but there's nothing deployed to it?
<thumper> right
<thumper> but "create-machine 0:lxc" could create an lxc container on machine 0
<wallyworld_> if we were to do that, and come back later and do a deploy, that machine should then be used
<thumper> and put a machine agent on it
<wallyworld_> rather than creating a new one
<thumper> maybe...
<thumper> sounds reasonable at least
<wallyworld_> i think your n:lxc thing works also
<thumper> "create-machine :lxc" would create a new machine, and a container
<thumper> this way, we could have nested containers
<wallyworld_> but there's no point doing those things as commands for the user to run manually unless the result machine/container were to be used by juju later
<thumper> as the container is represented as a machine
<thumper> wallyworld_: unless we had a constraint "machine x"
<thumper> but I agree, it should be deployed to
<thumper> if it is empty
<wallyworld_> hmmm. that would require the user to keep track of a lot of stuff
<thumper> and fits the requested constraints
<thumper> wallyworld_: not necessarily
<wallyworld_> yes, fits the constraints is essential
<thumper> and it is only a first cut
<thumper> and it gives a very definite "first thing" to do
<wallyworld_> thumper: ok, so i'll create a branch for the start-machine command
<thumper> ok.
<thumper> initially take no params
<wallyworld_> initially without :lxc
<thumper> and just create a new machine from the provider
<thumper> ack
<wallyworld_> yes
<thumper> also...
<wallyworld_> since lxc not done yet
<thumper> hmm...
<thumper> we do have a weird case
<thumper> we explicitly don't reuse machines
<thumper> which means, if a machine is sitting there empty
<wallyworld_> we could do if they are marked as empty
<thumper> we explicitly don't use it
<wallyworld_> let's add an "is empty"  param
<wallyworld_> attribute
<thumper> well...
<thumper> it is harder than that
<thumper> we effectivly have "is empty" by "no prinicple units"
<thumper> and there is code there already
<wallyworld_> ok
<thumper> perhaps what we need is "clean"
<thumper> a machine is clean if it is new
<wallyworld_> sounds better
<thumper> if a unit is deployed, not in a continer
<thumper> and then removed
<thumper> the machine is "unclean"
<wallyworld_> yes
<thumper> if a unit is deployed in a container
<thumper> and removed
<thumper> the machine is clean
<wallyworld_> but if container is nixed, machine is clean
<thumper> we can use clean machines
<wallyworld_> yes
<thumper> not "unclean"
<wallyworld_> +1
<thumper> ok, this is almost making sense
<wallyworld_> i'll start hacking and see where the yellow brick road takes me
<wallyworld_> suspect there will be a few branches
<thumper> wallyworld_: start with a create-machine
<wallyworld_> yes
<thumper> I'll try to document our thought process above
<wallyworld_> ok, thans
<wallyworld_> add it to your document
<wallyworld_> i can't wait for this stuff to mature, i really want to fire up lots of stuff on my local machine
<thumper> yeah
<thumper> we'll make it fully manured
<wallyworld_> lol
 * thumper goes to shower
<wallyworld_> bad visual image
<niemeyer> wallyworld_: Not too protective.. just want to give access to people actually doing good work on it.
<niemeyer> wallyworld_: If you have stuff for it, I'll add you to the team
<wallyworld_> niemeyer: thank you. i just wanted to make sure i wasn't stepping over any boundaries
<niemeyer> wallyworld_: Not at all, thanks for contributing
<wallyworld_> niemeyer: my branch fixes a user reported bug - it allows for EC2_ env vars to be reconised for credentials
<niemeyer> wallyworld_: Oh, haven't heard of those
<niemeyer> wallyworld_: Thanks
<niemeyer> wallyworld_: That's done
<wallyworld_> niemeyer: is there a landing bot or do we run tests locally and push manually
<wallyworld_> thanks
<niemeyer> wallyworld_: The latter
<niemeyer> wallyworld_: "LGTM" + test + lbox submit
<wallyworld_> niemeyer: thanks. bug 1174874 fwiw
<_mup_> Bug #1174874: EC2_* style Amazon environment variables not consulted <juju-core:Confirmed for wallyworld> <https://launchpad.net/bugs/1174874>
<niemeyer> wallyworld_: Huh..
<niemeyer> wallyworld_: Where's the reference for these environment variables?
<niemeyer> wallyworld_: I mean, who uses it?
<wallyworld_> not sure. i know i had them set up originally when i was got my ec2 account
<niemeyer> wallyworld_: As John points out, the EC2 tools actually use the AWS_* ones
 * niemeyer googles
<wallyworld_> i think they may have come from the aws account sign up perhaps
<wallyworld_> not sure
<wallyworld_> but i have . files with them in from ages ago
<niemeyer> eucatools
<niemeyer> Okay, +1
<wallyworld_> ok, thanks
<wallyworld_> just gives a nice robustness for the user
<niemeyer> Yeah
<wallyworld> niemeyer: when i run the tests, there's an issue with the test http server already being started. from what i can see, a few tests start separate test servers and don't clean up?????
<wallyworld> this issue is unrelated to my changes but best not to land something with test failures
<niemeyer> wallyworld: In aws_test.go?
<wallyworld> var testServer = testutil.NewHTTPServer() is in several test files
<wallyworld> s3_test, iam_test etc
<wallyworld> running go test ./... from the root dir exposes the issue
<wallyworld> example error
<wallyworld> PANIC: s3_test.go:28: S.SetUpSuite
<wallyworld> ... Panic: listen tcp 127.0.0.1:4444: address already in use (PC=0x413D81)
<wallyworld> /usr/lib/go/src/pkg/runtime/panic.c:229
<wallyworld>   in panic
<wallyworld> /home/ian/juju/go/src/launchpad.net/goamz/testutil/http.go:47
<wallyworld>   in HTTPServer.Start
<wallyworld> s3_test.go:29
<wallyworld>   in S.SetUpSuite
<wallyworld> so several tests call testServer.Start()
<wallyworld> maybe there needs to be a Stop() method called from TearDown
<wallyworld> niemeyer: i fixed the issue https://codereview.appspot.com/9545045
<wallyworld> go test ./... from toor dir now works
<wallyworld> root
<niemeyer> wallyworld: Hmm
<niemeyer> wallyworld: I'd prefer to not mix the two sets of variables
<wallyworld> which variables? the listener and testServer?
<niemeyer> wallyworld: if auth.AccessKey == "" || auth.SecretKey == "" { ... set both of them again using EC2_* ... }
<wallyworld> oh, you mean the bug fix
<wallyworld> ok, i can change that
<niemeyer> wallyworld: As it is, it's mixing the two sets of variables, which may end up pretty confusing
<wallyworld> sure, will change it
<wallyworld> you agree with the test fixes?
<niemeyer> wallyworld: Yeah
<wallyworld> with the variables thing, i just did what we did for goose
<wallyworld> so you won't agree with goose either :-)
<niemeyer> wallyworld: I suppose :)
<wallyworld> maybe we'll do a drive by fix sometime
<niemeyer> wallyworld: On Stop(), might be worth adding listener = nil at the end, although that's really minor
<wallyworld> ok
 * thumper laughs
<thumper> grr
<thumper> I was going to bitch at niemeyer
 * thumper wonders if he'll come back
<thumper> davecheney, wallyworld: updated that branch https://codereview.appspot.com/9677044/ to follow gustavo's suggestion
 * wallyworld looks
<thumper> wallyworld: ta
<wallyworld> np
<thumper> imagine my surpise when I found a test that actually tested the use case, and it passed
<wallyworld> another Go gotcha
<thumper> I'm pleased it isn't idiomatic to return None in python instead of []
<thumper> 'cause that kinda blows
<wallyworld> yes indeed
<wallyworld> thumper: i wonder, should the new machine being created use constraints stored in the env (for mem etc) or allow the user to specify different ones
<thumper> yes
<thumper> wallyworld: both
<wallyworld> ok, so if none specified, use the env ones
<thumper> wallyworld: specifying new ones override the existing env, but become a superset
<thumper> not an override, but a merge
<thumper> with specified overriding others
<thumper> make sense?
<wallyworld> yes
<wallyworld> aye
 * thumper looks at the assignment policies
<thumper> again...
<thumper> actually, I'll go pick up the sushi first
<thumper> davecheney: what is our next juju-core milestone?
<dimitern> morning all
<TheMue> morning
<dimitern> TheMue: hiya
<TheMue> dimitern: just playing around with the gmail account ;)
<dimitern> TheMue: oh, yeah?
<TheMue> dimitern: yep, I've used the automatic filtering so far. now I've got a lot of labels for each imap folder and I want to clean it up.
<dimitern> TheMue: i have like 60 filters in thunderbird already, didn't want to go through the hassle of migrating to gmail, so I opted out
<TheMue> dimitern: hehe
<TheMue> dimitern: I dislike client side filters as I access my mail (private as work) from multiple devices
<jam> dimitern, TheMue: my point as well. I did client side for a while, but I access the account from multiple locations (phone, laptop, desktop), and client-side doesn't work so well there.
<jam> I was trying it back when I ran my own mailserver, but server-side just works a lot better.
<TheMue> jam: yes. I run my private mail server (for my family and some friends) too and their I also provide server-side filtering
<jam> TheMue: gmail does ok. There are bits I'd like to be better, but at least to set up a filter you can say "filter messages like this" and it will detect the list header and set it for you from the start.
<jam> It doesn't work with custom Launchpad headers.
<jam> but for lists, it is quite easy to set up.
<dimitern> rogpeppe1: ping
<TheMue> jam: yes, looks fine so far. will now only clean up the migrated old folder. I kept too much unneeded stuff
<rogpeppe1> dimitern: pong
<dimitern> rogpeppe1: hey, i've been trying to figure out the correct client-side logic for the LifecycleWatcher loop, in particular when to call next initially
 * rogpeppe1 goes to look at state.LifecycleWatcher
<dimitern> rogpeppe1: I should call it once initially and then after the select (like in entitywatcher), right?
<davecheney> thumper: milestone ?
<davecheney> you mean release number
<davecheney> nfi
<davecheney> i hope someone tells me
<rogpeppe1> dimitern: that sounds plausible, yes
<rogpeppe1> dimitern: actually, no
<rogpeppe1> dimitern: i think you should follow the pattern of entitywatcher
<dimitern> rogpeppe1: but for that initial call I shouldn't care as much for error handling - I mean return whatever error I got from Next, otherwise for the second call (after the select), I use the same logic as in the entitywatcher loop
<dimitern> rogpeppe1: well, I need some changes []string to send on the out channel
<rogpeppe1> dimitern: why should the error handling be different for the first call and the second call?
<dimitern> rogpeppe1: and the select appears before the Next call
<rogpeppe1> dimitern: ah, yes, the logic will need to be a bit different - you'll need to set the out channel to non-nil only when you've got something to send
<dimitern> rogpeppe1: ah, right - that tricky case
<dimitern> rogpeppe1: so set w.out to nil initially to bypass the select and set it to not nil after i have changes?
<rogpeppe1> dimitern: yup
<rogpeppe1> dimitern: well, probably have a local "out" variable to do that
<rogpeppe1> dimitern: just before the select, set it to non-nil iff there's something to send.
<dimitern> rogpeppe1: yes, I have out := w.out; w.out = nil; var lastChanges []string
<dimitern> rogpeppe1: and w.out <- lastChanges (both of these are set when relevant)
<rogpeppe1> dimitern: i wouldn't ever set w.out to anything different than its original value
<rogpeppe1> dimitern: i think it's easier to see correctness when mutable state is in local variables
<dimitern> rogpeppe1: so use out < lastChanges instead of w.out in the select then?
<dimitern> rogpeppe1: and have var out chan []string before the for loop
<rogpeppe1> dimitern: no, do: for { var out chan []string; if lastChanges != nil { out = w.out }; select {case out <- lastChanges: lastChanges = nil; case lastChanges = <-in: } }
<rogpeppe1> or something like that
<rogpeppe1> dimitern: you'll probably need to have a loop calling Next in its own goroutine
<dimitern> rogpeppe1: I didn't get the last "case" - why that?
<rogpeppe1> dimitern: how are you going to receive new changes?
<dimitern> rogpeppe1: after the select I have if changes, err := callNext(); err != nil .....
<rogpeppe1> dimitern: you can't do that
<dimitern> rogpeppe1: why not?
<rogpeppe1> dimitern: because the call to Next can block indefinitely
<rogpeppe1> dimitern: which means you won't be able to stop the watcher
<dimitern> rogpeppe1: hmm - why is it working for the entitywatcher's loop then?
<dimitern> rogpeppe1: no gorouting for calling next there
<rogpeppe1> dimitern: hmm, yes. le me think a mo
<rogpeppe1> dimitern: (the other works because Stop is called in a goroutine)
<dimitern> rogpeppe1: i suspected as much
<dimitern> rogpeppe1: so with the same goroutine for stop it should work w/o another for next?
<rogpeppe1> dimitern: i don't think it would be quite right
<rogpeppe1> dimitern: the key is in the "Note that because the change notification contains no information" comment
<rogpeppe1> dimitern: if we don't have the Next in a separate goroutine, if the receiver is slow, we'll be trying to send stale information.
<rogpeppe1> dimitern: we could decide that we don't care about that, but i suspect that william might not be keen
<dimitern> rogpeppe1: hmm.. so how about having another go func() block before the loop for calling next, and moving all the logic & error handing around it there
<rogpeppe1> dimitern: i think that sounds reasonable
<rogpeppe1> dimitern: if it gets an error, it can kill the tomb and die
<dimitern> rogpeppe1: thus inside the loop only the select and out channel switching will remain, and the lastChanges will be a closure used by that goroutine
<rogpeppe1> dimitern: i think lastChanges will be a local slice variable
<dimitern> rogpeppe1: local to the next() goroutine?
<rogpeppe1> dimitern: local to the select loop
<dimitern> rogpeppe1: how will the next goroutine report the changes to back?
<dimitern> rogpeppe1: s/to//
<rogpeppe1> dimitern: with a channel (i called it "in" above)
<dimitern> rogpeppe1: ah! cool
<dimitern> rogpeppe1: does it have to take it as an argument or it can be a closure?
<rogpeppe1> dimitern: huh? "it" ?
<dimitern> rogpeppe1: the in chan
<rogpeppe1> dimitern: how can a channel take an argument or be a closure?
<dimitern> rogpeppe1: sorry, i'm not explaining well enough - if I have in := make(<-chan []string); go func() { ... using in directly .. }() will it work or I have to use go func(in<-chan []string){ ... }
<rogpeppe1> dimitern: ah, i see. yeah the former is just fine.
<dimitern> rogpeppe1: good
<dimitern> guys is the call now or in 1h ?
<rogpeppe1> dimitern: i think it's in 5 mins
<dimitern> rogpeppe1: ok, I set my calendar correctly this time :)
<dimitern> rogpeppe1: and in addition to the in chan, i'll need a done chan error as well in the next goroutine, right?
<rogpeppe1> dimitern: i don't think so
<rogpeppe1> dimitern: i think you can use the tomb for that
<dimitern> rogpeppe1: kill the tomb on error?
 * danilos is installing latest hangout plugin :/
<dimitern> rogpeppe1: take a look at this please http://paste.ubuntu.com/5692992/
<dimitern> rogpeppe1: does it look sane more or less?
<rogpeppe1> dimitern: i'll look after the meeting
<dimitern> rogpeppe1: sure, np
<dimitern> rogpeppe1: so?
 * rogpeppe1 looks
<mramm> I think if we are going to keep the containerization and API everywhere lanes, we should kill core 1 and core 2 -- the idea there was one lane per feature group anyway
<dimitern> rogpeppe1: haven't compiled it yet (just realized I need to add panic("unreachable") in the next goroutine for go1.0x compatibility)
<mramm> and we can move some of the existing stuff that is in the core1/2 lanes back into the backlog if they are just providing too much visual clutter
<rogpeppe1> dimitern: i'd like to know if we still need to be go1.0.2 compatible
<dimitern> mramm: and change the lanes as we progress down the roadmap?
<mramm> (which I think is why thumper created the new lanes)
<mramm> dimitern: well, I did make them generic so we don't have to change them
<dimitern> rogpeppe1: i really don't want to care about go1.0[23] anymore, but we need to remove a lot of these panics throughout and possibly some other improvements are in order
<mramm> but I honestly don't care, changing them occasionally is no big deal
<thumper> mramm: if we do that, then add a general bugs lane
<mramm> I did
 * thumper has another call now
<mramm> (at the top)
<thumper> oh
<thumper> ok
<thumper> good
<thumper> :)
<rogpeppe1> dimitern: i was wondering about other stuff. for instance my recent tasks branch uses a go1.1 feature
<rogpeppe1> dimitern: (method values)
<dimitern> we should've brought that up in the call, i just realized now
<dimitern> let's make it official and send a mail to juju-dev "We're now using go1.1 only"
<dimitern> TheMue, davecheney, jam, mgz, rogpeppe1, danilos, wallyworld_, mramm: any objections to the above? ^^
<wallyworld_> Go for it :-)
<rogpeppe1> dimitern: code looks reasonable
<wallyworld_> the sooner the better
<rogpeppe1> dimitern: i might be tempted to factor out the Next error handling logic into a function, so it's not duplicated
<jam> wallyworld_, dimitern: I believe the original discussion ended in "we'd like to be able to compile with the platform tools". Already it is really hard to backport to precise.
<jam> because we don't have the tool there.
<danilos> dimitern, if we've got golang packages lined up for people wanting to do development on precise and raring, I am +1
<jam> But if 1.1 is officially at least in Saucy, we can probably go with it.
<jam> And then we have to backport 1.1 for at least P, and probably Q and R
<mgz> dimitern: only?
<wallyworld_> can we put 1.1 in backportsd?
<jam> mgz: his point was only
<mgz> how do we get that on prrecise?
<jam> mgz: he just wants to make the release process harder on you. :)
<jam> mgz: the same way we'll get != go 1.0 ?
<jam> mgz: arguably if we have to do something to get go 1.0.3, we might as well get 1.1
<jam> instea
<jam> instead
<mgz> I'm not *sure* we do yet...
<rogpeppe> mgz: go1.1 works fine on precise - you just won't be able to use the standard golang package
<jam> mgz: I'm about 75% sure go 1.0 won't pass the test suite.
<mgz> but we could SRU an new minor version, a new major version would be... not sruable
<jam> (as I have precise and I have to install go from the ppa to work on juju-core)
<jam> I *know* lbox is broken with go 1.0, I think juju-core might be as well.
<dimitern> so the agreement is not yet there - ok then, to early for an announcement on the list
<wallyworld_> mgz: major = .x ?
<dimitern> rogpeppe: what is duplicated about the error handling?
<jam> mgz: so I do believe that go 1.1 is source-compatible with go 1.0
<wallyworld_> dimitern: if stuff being committed break with 1.0.3, we should tell people
<mgz> jamespage is shaking his head
<jam> it is intended to be, at least, because of the "minor" bump
<rogpeppe> dimitern: it's exactly the same logic as in EntityWatcher.loop
<mgz> we can't use backports
<rogpeppe> i feel strongly that we should move to using go 1.1
<dimitern> rogpeppe: ah, good point, ok
<mgz> when you do a backport, it doesnt build against backported packages
<jam> mgz: because of the bug that backports can't build with other backports?
<mgz> jam: indeed
<rogpeppe> but if it's not possible to backport to precise, then perhaps we should stay away from 1.1-specific features
<mgz> wallyworld_: I'm reading "1.1" as major and "1.0.3" as minor, but it's all pretty fuzzy, the policies of the project matter more than the actual numbers
<rogpeppe> that means we should probably gate commits on 1.0.3 compatibility
<mgz> rogpeppe: I'm not sure what the current status is, or what our priorities are
<wallyworld_> yeah :-(
<jam> rogpeppe: so while I agree that use 1.1 when available, especially for official binaries. However, is there something in 1.1 which would prevent it working with 1.0.3 compiler?
<jam> I thought it was at least supposed to be syntactically the same.
<jam> maybe lib changes?
<rogpeppe> jam: there are language additions, yes
<mgz> if we want newest juju installable on precise without a ppa, that then does make what version of go we run against more limited, unless we have other workarounds
<rogpeppe> jam: 1.0.3 code is compatible with 1.1 but not vice versa
<dimitern> jam: e.g. panic("unreachable") is no longer needed in funcs with a for loop + select only  (most of the watchers, etc.) - it's a compile error in go1.0.3, not in go.1.1; there are nice features we can use in go1.1
<dimitern> and there's the performance improvements, method values, race detection..
<rogpeppe> jam: another nice feature is you can use a method as a value (it becomes a closure like you might expect)
<rogpeppe> jam: the changes are all here: http://golang.org/doc/go1.1
<jam> dimitern: performance improvements and race detection can be used without changing syntax
<jam> method values are nice, but you can use closures, and not having panic is also nice but a bit trivial
<dimitern> jam: true, but if we're not allowed to use go1.1 due to (stupid?) packaging/backporting issues..
<jam> so I think stating that for now, 1.0.3 compatible, but use 1.1 when you like.
<rogpeppe> jam: the issue i have is that it's very easy to fall outside 1.0.3 compatibility.
<jam> rogpeppe: thats what the bot is for
<jam> If it is running precise (the one I have currently is), then we have that gate in place already.
<rogpeppe> jam: so the bot would test against 1.0.2 and 1.1 ?
<jam> rogpeppe: I would tend to not do a multi-compiler pre-commit check, though that could be done.
<jam> right now, goose is still 1.0 compatible (and passes the test suite there)
<jam> so I didn't have to even try to get 1.0.2/3 there.
<dimitern> rogpeppe: how about http://paste.ubuntu.com/5693088/?
<jam> rogpeppe: the issue is trading off time-to-land-in-trunk with how much you expect to actually benefit from spending the time. A CI system is much better at doing multiple-platform-and-compiler level testing, vs the pre-commit testing
<rogpeppe> jam: right
<jam> so the "most expected platform/system" is the pre-commit check, and the CI lets you know that you could release this across all your supported platforms.
<jam> you could certainly do 1.0.3 in CI, but I would put that in "most expected to fail" vs 1.1
<rogpeppe> dimitern: how about this instead? http://paste.ubuntu.com/5693094/
<rogpeppe> dimitern: oops, with a tomb argument too, i guess
<dimitern> rogpeppe: yeah
<dimitern> rogpeppe: and the call()?
<rogpeppe> dimitern: i think that bundling the error checking with the call itself makes for a slightly more coherent function
<rogpeppe> dimitern: see callWatch in EntityWatcher.loop
<dimitern> rogpeppe: so call has the be an arg as well - call func() error
<rogpeppe> dimitern: with one extra argument
<rogpeppe> dimitern: yeah, you'd do: if err := watcherNext(call, &w.tomb); err != nil {w.tomb.Kill(err)}
<dimitern> rogpeppe: what if we have a commonWatcher with next() error and pass that instead? or even make this a method of the commonWatcher & embed it in both watchers?
<dimitern> rogpeppe: having commonWatcher will allow all this to be handled inside it's next() method
<rogpeppe> dimitern: i'm not sure i see a particular advantage in using a commonWatcher type rather than a closure
<rogpeppe> dimitern: but there may be something i'm not seein
<rogpeppe> g
<dimitern> rogpeppe: well, there are going to be at least 3 watchers with this logic - entity, life and environconfig
<rogpeppe> dimitern: what do you see going into commonWatcher other than next() ?
<dimitern> rogpeppe: right, we need different next calls for each one
<rogpeppe> dimitern: anything else?
<dimitern> rogpeppe: so watcherNext(call func(method string, result interface{}), err error, wtomb *tomb.Tomb) error {..} then
<rogpeppe> dimitern: watcherNext(call func(method string, result interface{}) error, wtomb *tomb.Tomb) error {..}
<rogpeppe> dimitern: i think
<dimitern> rogpeppe: oh yeah, the err is internal now
<dimitern> rogpeppe: sgtm
<rogpeppe> dimitern: i may very well change my position on commonWatcher if we find more stuff that can be factored out
<rogpeppe> dimitern: but for the time being, i think it makes things less obvious and would use more code
<rogpeppe> dimitern: actually, i think i see a way to factor out almost all of the code
<dimitern> rogpeppe: except next is different for each watcher - entitywatcher returns nothing but an error, while the lifecyclewatcher returns a slice as well
<rogpeppe> dimitern: i think entitywatcher is different
<rogpeppe> dimitern: i'm thinking about other watchers that all return some actual data
<dimitern> rogpeppe: so?
<rogpeppe> dimitern: go with just factoring out the next call for the time being.
<rogpeppe> dimitern: the other factor can be done with the next watcher
<dimitern> rogpeppe: you mean factoring out only the error handling, and not call next() inside it?
<rogpeppe> dimitern: no, i mean call next inside it, as i suggested. i *think* that looks nicer.
<dimitern> rogpeppe: what about the results?
<rogpeppe> dimitern: that's just an extra arg
<dimitern> rogpeppe: no, i mean we should return error & the results we got, right?
<rogpeppe> dimitern: ha, good point.
<rogpeppe> dimitern: watcherNext(results interface{}, call func(method string, result interface{}) error, wtomb *tomb.Tomb) error
<dimitern> rogpeppe: yeah, sgtm
<dimitern> rogpeppe: params.EnvironConfigWatcherNextResults :) that starts to look like java
<rogpeppe> dimitern: :-)
<rogpeppe> dimitern: at least you'll only be using it in about two places
<dimitern> rogpeppe: yeah
<dimitern> rogpeppe: I can think a couple of cases where commonWatcher is useful
<dimitern> rogpeppe: e.g. Stop() and Err() are the same
<rogpeppe> dimitern: i've got a suggestion coming up
<dimitern> rogpeppe: in addition we can have callNext call as a field in commonWatcher as well
<dimitern> rogpeppe: and the tomb and wg
<dimitern> and st
<dimitern> rogpeppe: although "commonWatcher" might be incorrect, as it's just a few bits of common code extracted, but not a complete watcher
<dimitern> rogpeppe: maybe "watcherBase" is better - we can have the go func() { .. spin the loop } in newBaseWatcher
<rogpeppe> dimitern: something along these kinds of lines? http://paste.ubuntu.com/5693203/
<mgz> was there a message to list about the code.google.com/p/go.crypto dep?
<rogpeppe> dimitern: it's not *quite* right, but i think it's not too bad
<rogpeppe> mgz: i don't remember one
<dimitern> rogpeppe: mostly ok, except for a few things
<dimitern> rogpeppe: call needs to handle the case when results is nil
<rogpeppe> dimitern: i think it does that ok, doesn't it?
<mgz> we should probably do such things when adding deps
<dimitern> rogpeppe: not sure - shouldn't the last arg of st.call be &results ?
<rogpeppe> dimitern: i don't think so
<rogpeppe> dimitern: result is already a pointer (created by the newResults function)
<dimitern> rogpeppe: ah, ok
<dimitern> rogpeppe: should work then
<rogpeppe> dimitern: actually, i think there's another way of avoiding a bit more work in each watcher
<rogpeppe> dimitern: one mo
<dimitern> rogpeppe: how about overriding init() and setting there the out chan, call and newResult? then we can just spin up the loop goroutine right after that
<rogpeppe> dimitern: i'm not quite sure what you mean
<dimitern> rogpeppe: i'd like to factor out this somehow as well: http://paste.ubuntu.com/5693224/
<dimitern> rogpeppe: except out's type is different so it won't work probably (damn generics!)
<rogpeppe> dimitern: yeah, i think the wg.Wait can go (suggestion in a moment) but the rest should probably stay
<rogpeppe> dimitern: marginally better, perhaps: http://paste.ubuntu.com/5693269/
<dimitern> rogpeppe: nice; only in the case of if !ok { should return some error }
<rogpeppe> dimitern: no need to return an error - the tomb will already have been killed with an appropriate error
<dimitern> rogpeppe: and this will work for entitywatcher as well, right?
<dimitern> rogpeppe: it won't compile with just "return"
<rogpeppe> dimitern: i think it will. it won't be *quite* as efficient, but i don't think we mind.
<rogpeppe> dimitern: yeah, it should be return nil probably
<dimitern> rogpeppe: ok, and i'll put a comment about the tomb there as well
<dimitern> rogpeppe: thanks
<rogpeppe> dimitern: np
<dimitern> rogpeppe: so if we return nil from the loop, and in commonLoop we already killed the tomb with an error, won't the "return nil" wipe out that original error?
<rogpeppe> dimitern: no
<rogpeppe> dimitern: the first non-nil, non-ErrDying error will stick
<dimitern> rogpeppe: i see, ok
<dimitern> rogpeppe: wasn't immediately obvious for me looking at tomb.Kill
<rogpeppe> dimitern: rtfm :-)
<rogpeppe> dimitern: http://paste.ubuntu.com/5693339/
<dimitern> rogpeppe: isn't w.commonWatcher.init() the same as w.init() ?
<rogpeppe> dimitern: yeah. i guess i prefer the explicitness there.
<dimitern> rogpeppe: sure
<dimitern> rogpeppe: so either the TestMachineWatch logic is wrong or the commonLoop logic is wrong, because Next never returns an inital event
<rogpeppe> dimitern: interesting. the Next call is being made and doesn't return?
<dimitern> rogpeppe: yes - it returns only after the timeout
<dimitern> rogpeppe: http://paste.ubuntu.com/5693378/ - added debug logs after next returns and one after sending to in
<rogpeppe> dimitern: ah, this is interesting. it looks like i designed it that way deliberately.
<dimitern> rogpeppe: :) so? is the test wrong or the loop?
<rogpeppe> dimitern: look at srvMachine.Watch - it reads the channel
<dimitern> rogpeppe: yeah, it does, but shouldn'y
<rogpeppe> dimitern: actually, i think it should
<rogpeppe> dimitern: and i think the test is right, as is the current loop
<rogpeppe> dimitern: the suggested implementation should be different though
<dimitern> rogpeppe: what then?
<dimitern> rogpeppe: it shouldn't call next initially?
<rogpeppe> dimitern: we should return the initial data with the initial watch request
<rogpeppe> dimitern: so that we don't do two round-trips whenever we start a watcher
<rogpeppe> dimitern: that was the reason for doing things this way anyway
<dimitern> rogpeppe: saving a roundtrip seems sane
<jam> danilos, wallyworld_, w7z: poke
<dimitern> rogpeppe: but what does this mean for the loop?
<rogpeppe> dimitern: something like this: http://paste.ubuntu.com/5693401/
<dimitern> rogpeppe: move the next call after the select and have in := w.in
<dimitern> rogpeppe: ah, right
<dimitern> rogpeppe: but for the entitywatcher, this should be changes := struct{}{} instead
<rogpeppe> dimitern: no need to have a changes variable for the entity watcher
<dimitern> rogpeppe: wait, so the entitywatcher is the one not working, haven't written tests for the others yet
<rogpeppe> dimitern: the entity watcher was working before, right?
<dimitern> rogpeppe: yes
<dimitern> rogpeppe: for the lifecycle and environconfig watchers what you suggested should work
<rogpeppe> dimitern: so you're just changing it to use the commonWatcher to start with?
<dimitern> rogpeppe: yes, but the entitywatcher doesn't have results, so calling etype, eid, "Watch" won't report anything
<dimitern> rogpeppe: maybe you meant out = w.out initially for entitywatcher loop
<dimitern> rogpeppe: yeah, it works like this
<rogpeppe> dimitern: the entitywatcher loop will be a bit different
<dimitern> rogpeppe: that was the only change needed
<dimitern> rogpeppe: the test now pass
<rogpeppe> dimitern: cool
<dimitern> rogpeppe: so the same has to be applied for lifecycle and environconfig watchers? calling Changes in Watch* at server-side?
<rogpeppe> dimitern: yeah, i think so
<rogpeppe> dimitern: it's probably worth checking that all those things always produce an initial event
<dimitern> rogpeppe: so the params.*WatcherId should embed params.*WatcherNextResults then
<wallyworld__> jam: you forgot to ask me what i did, but that's ok. one thing for you - i talked to is about the rt, got the cc list updated, and they have also bumped its priority so hopefully will be done real soon now
<jam> wallyworld__: yay, sorry I missed you. You had been talking about landing your other branches, so I thought you were done. But good to hear about the rt.
<rogpeppe> dimitern: i don't think you can embed it, but yes, the same value should be there. and i'd probably call it, e.g. params.LifecycleWatchResults
<rogpeppe> rather than WatcherId
<dimitern> rogpeppe: ok
<wallyworld__> jam: i also fixed goamz - tests weren't cleaning up and so were failing, and i didn't want to land my changes without fixing even though my changes weren't responsoble
<jam> wallyworld__: so instead of cleaning up at the start of the test we cleanup at the end?
<wallyworld__> jam: each test was creating a socket and then just leaving it opened and not closing, so subsequent tests would fail
<wallyworld__> so i added cleanup to the suite teardown
<dimitern> rogpeppe: and afaics both lifecycle and environconfig watchers send initial event
<wallyworld__> jam: also talked to tim about containerisation, agreed on some work items,  and started work on the first branch
<frankban> rogpeppe, anyone else: it seems there is something like a logging loop in machine 0 when you use --force-machine. /var/log/juju/all-machines.log grows about 2 MB per second. and debug-log is unusable
<rogpeppe> frankban: what are all those log messages?
<frankban> rogpeppe: it seems to repeat the same messages again and again (e.g. messages from hooks execution)
<rogpeppe> frankban: what messages?
<dimitern> rogpeppe: should I use params.LifecycleWatchResults for both Watch* and Next then ?
<frankban> rogpeppe: http://pastebin.ubuntu.com/5693483/ just an example. the problem is the same logs are repeated in a loop.
<dimitern> rogpeppe: and just pass the id in next as well
<rogpeppe> dimitern: i suppose you could. or just leave the id blank for next.
<dimitern> rogpeppe: ok
<rogpeppe> frankban: could you paste the log with at least 3 repetitions of the cycle, so i can get an idea for what's going on, please?
<rogpeppe> s/the log/a portion of the log/
<frankban> rogpeppe: sure, grabbing it. however, to dupe: bootstrap a juju-core env, deploy wordpress (or the GUI) using --force-machine 0.
<frankban> rogpeppe: http://pastebin.ubuntu.com/5693508/
<rogpeppe> frankban: thanks
<frankban> np
<frankban> .me lunches
<dimitern> rogpeppe: I don't have to add tests like TestServerStopsOutstandingWatchMethod for the other watchers, right?
<rogpeppe> dimitern: looking
<rogpeppe> dimitern: i think that functionality should be tested for each watcher actually
<rogpeppe> dimitern: i think it's probably possible to make the test code generic though
<dimitern> rogpeppe: but all of them are resources and handled all the same in stopAll, right?
<rogpeppe> dimitern: that test is more about testing the client-side stop logic actually
<rogpeppe> dimitern: stopAll isn't invoked in that test, i don't think
<rogpeppe> dimitern: oops
<rogpeppe> dimitern: sorry, i'm wrong
<dimitern> rogpeppe: it's on the server side when stopping
<dimitern> rogpeppe: so I can transform that test into a table-based with setup funcs for each watcher
<rogpeppe> dimitern: it might be good to make that test something that starts one of every kind of watcher, and checks to see that they're all torn down appropriately when the server is stopped.
<dimitern> rogpeppe: by "torn down" you mean returning CodeStopped, right?
<rogpeppe> dimitern: yeah.
<dimitern> rogpeppe: ok
<rogpeppe> dimitern: i'd start all the watchers at once, then tear down the state just once.
<dimitern> rogpeppe: yeah, so some state setup code first, start all & wait for initial events from everyone, then stop the server and try calling changes on each one and check for CodeStopped
<jam> mgz: we missed you earlier. How's your day going?
<mgz> fighting dpkg a little, otherwise okay
<mgz> (should have mentioned I'd be out over lunch, but you probably saw me discussing that with gavin)
<jam> mgz: so are there plans to get go-1.0.2 or 3 into precise?
<jam> I think we have a failure in the test suite from using go-1 in gobot
<jam> (it fails reliably on gobot, but succeeds on my local machine)
<mgz> not currently, but we could form some
<mgz> we're the only people who'd drive that, juju is the only real user for go in precise
<dimitern> rogpeppe, mramm: kanban?
<rogpeppe> dimitern: ah, good point
<mgz> jam: the go bot kinda hates life it seems
<jam> mgz: well, its doing things slightly differently than what the rest of us have all worked around manually.
<dpb1> Hi all -- I'm getting an error with the latest updated juju-core: http://paste.ubuntu.com/5694171/  <-  What am I missing?  (I'm using the 'cstack' environment)
<dpb1> Man, I just enabled mouse mode for weechat and byobu.  It's working really great.  Even cut and paste is working better now.
<dpb1> wow, that was the wrong window.
<rogpeppe1> dpb1: mgz or jam or dimitern might know what's going on there
<mgz> looks like the container was not marked public...
<mgz> dpb1: can you pastebin `nova endpoints` with your novarc sourced? I'm assuming e8231fcc9d9546c9961caf858676ea4e is your tenant id
<dpb1> checking
<dpb1> mgz: http://paste.ubuntu.com/5694222/
<mgz> dpb1: for now, you can probably just set public-bucket-url... though I just checked and that seems to not have the streams data
<mgz> dpb1: can you file a bug please?
<dpb1> mgz: sure, what I have in those two pastes enough?
<dpb1> mgz: also, what should I set public-bucket-url to?
<mgz> https://swift.canonistack.canonical.com/v1/AUTH_526ad877f3e3464589dc1145dfeaac60
<mgz> right, I'm off now
<mgz> may be around later tonight
<hatch> umm well we are still on the upswing of this project so we have a support team of 5 and a core team of 7
<hatch> so 12 fulltime
<hatch> plus some floaters if needed for specific things we don't have the experience for already
<hatch> damn
 * rogpeppe1 is very happy with the new Task abstraction
<rogpeppe1> anyone around to review a branch that i think's trivial that unbreaks trunk for go 1.0.* ?
<rogpeppe1> if so, it's at https://codereview.appspot.com/9682047
<thumper> morning
<kyhwana> Yep *chugs coffee*
<thumper> niemeyer: you around?
#juju-dev 2013-05-24
<niemeyer> thumper: Yo
<thumper> niemeyer: oh hai
<thumper> I had a go question...
<thumper> if you have a select and each of the cases are waiting on a channel
<thumper> what is happening under the covers?
<thumper> and that select is inside a for { }
<thumper> like the provisioner
<niemeyer> thumper: An epoll
<niemeyer> thumper: Not sure if that's your question, though?
<thumper> niemeyer: I was just wanting to make sure it wasn't doing stupid looping when nothing is happening
<thumper> I considered wrting a test app to check, but got busy thinking of something real
<niemeyer> thumper: Ah, no, it will block
<niemeyer> thumper: Unless you have a default in the select loop.. then it would spin like crazy
 * thumper nods
<niemeyer> s/select loop/select clause/
<wallyworld_> thumper: if you have a moment at some point, https://code.launchpad.net/~wallyworld/juju-core/create-machine-command/+merge/165518
<thumper> wallyworld_: sure
<wallyworld_> thumper: i didn't update machine state to include a Clean attribute. still not sure if we want that or not
<thumper> wallyworld_: I'm about to email the group about my proposed machine state changes as there are a lot of them
<wallyworld_> i'm thinking if we do add it, we can change the default assignment policy
<wallyworld_> from AssignNew to AssignUnused
<wallyworld_> since we can look for a "clean" machine
<thumper> right...
<thumper> there is more to it though :)
<thumper> which I'll be putting in the email
<wallyworld_> sure, containers complicate things
<wallyworld_> what i say above gives us the same thing as we have now
<wallyworld_> since without it, create-machine just wastes resources
<thumper> wallyworld_: well, you sould be able to use --force-machine
<thumper> wallyworld_: does that work?
<wallyworld_> ah,true
<wallyworld_> should work
<wallyworld_> the machine is created, spun up, and is just sitting there
<wallyworld_> with no principal units or anything running
<wallyworld_> i'll check it out
<wallyworld_> thumper: all works as expected :-)
<thumper> wallyworld_: good
<wallyworld_> thumper: i think juju status should print out the constraints for the machine
<wallyworld_> right now it doesn't seem to do that
<thumper> perhaps a flag
<thumper> we are getting too much info by default
<thumper> much more than is usually useful
<wallyworld_> you think?
<wallyworld_> i sorta like what's there now
<wallyworld_> there's only 5 lines for eacdfh machine
<wallyworld_> adding one more for constraints doesn't seem too bad
<thumper> well, that is me about done for the day
<rogpeppe1> mornin' all
<kyhwana> evening
<dimitern> morning
<rogpeppe1> kyhwana: :-)
<rogpeppe1> dimitern: hiya
<dimitern> rogpeppe1: hey
<dimitern> rogpeppe1: i couldn't finish yesterday because just as i was nearing the end and only a few more test cases were needed, i started getting test failures or panics when running the apiserver tests; but not all the time, like 1 in 10 cases
<rogpeppe1> dimitern: hmm, that's not entirely surprising for that kind of logic.
<rogpeppe1> dimitern: have you run it with race detection on?
<dimitern> rogpeppe1: yes, multiple times - no races
<rogpeppe1> dimitern: well, not a memory race anyway, eh? :-)
<dimitern> rogpeppe1: so there are 2 kinds of failures - timeouts for the watchers (always the same - next() returns a stopped error before stopping either the server or the watcher - and in the logs i can see it is indeed getting stopped somehow)
<rogpeppe1> dimitern: is that one kind of failure or two?
<dimitern> rogpeppe1: another issue is the pretty long and somehow not very legible panic, usually mentioning PutCharm some 50 frames down and having mgo stuff close to the top
<rogpeppe1> dimitern: could you paste the panic here?
<dimitern> rogpeppe1: running them again
<dimitern> rogpeppe1: http://paste.ubuntu.com/5696111/
<rogpeppe1> dimitern: interesting, looks like a runtime bug
<rogpeppe1> dimitern: you can reproduce this ok?
<dimitern> rogpeppe1: trying more times
<dimitern> rogpeppe1: how can you tell is likely a runtime bug?
<rogpeppe1> dimitern: it's dying during garbage collection
<dimitern> rogpeppe1: wow :) these kind of things always happen to me :D
<rogpeppe1> dimitern: so it seems.
<rogpeppe1> dimitern: please could you try and see if you can reproduce the behaviour using gustavo's new Go-only yaml implementation
<dimitern> rogpeppe1: it happened again - almost the same, but trace order is changed a bit
<dimitern> rogpeppe1: http://paste.ubuntu.com/5696122/
<dimitern> rogpeppe1: so how to get it?
<rogpeppe1> dimitern: i'm just looking for it
<dimitern> rogpeppe1: hmm now it happens on every run - yesterday happened less often.. also it doesn't happen if I run individual tests
<rogpeppe1> dimitern: https://code.launchpad.net/~niemeyer/goyaml/go-port
<dimitern> rogpeppe1: so what - delete the goyaml in gopath and get this instead?
<rogpeppe1> dimitern: yeah - although you'll need to manually branch it into launchpad.net/goyaml
<dimitern> rogpeppe1: or just remove the files and replace them
<rogpeppe1> dimitern: yeah, that's probably easier
<TheMue> morning
<rogpeppe1> dimitern: fairly trivial CL: https://codereview.appspot.com/9681045
<rogpeppe1> TheMue:  hiya
<dimitern> TheMue: hey
<dimitern> rogpeppe1: looking
<dimitern> rogpeppe1: LGTM
<rogpeppe1> dimitern: ta
<dimitern> rogpeppe: ha! i happened at first with goyaml-go, but wasn't sure it's clean, so I went in pkg/linux_*/launchpad.net/ and deleted goyaml.a
<dimitern> rogpeppe: now there's no panic, just some failures
<rogpeppe> dimitern: hmm. keep trying
<dimitern> rogpeppe: http://paste.ubuntu.com/5696153/
<rogpeppe> dimitern: it may still be a runtime bug
<rogpeppe> dimitern: just that you're not triggering it any more for some reason
<dimitern> rogpeppe: oops there is it again: http://paste.ubuntu.com/5696157/
<rogpeppe> dimitern: because i'm not sure the code path you're testing uses yaml much
<rogpeppe> dimitern: ah, good. (well, kinda :-])
<rogpeppe> dimitern: could you push the branch - i'll see if i can reproduce
<dimitern> rogpeppe: sure
<dimitern> rogpeppe: ha! now all passed
<rogpeppe> dimitern: run it in a loop
<TheMue> *: any probs with the current goyaml?
<dimitern> rogpeppe: lp:~dimitern/juju-core/041-provisioner-api-calls
<dimitern> rogpeppe: how do you mean in a loop?
<dimitern> rogpeppe: it seems it happens in the TestOperationPerm ... except now it happened later + a failure: http://paste.ubuntu.com/5696171/
<rogpeppe> dimitern: while true; do go test; done
<dimitern> rogpeppe: running
<rogpeppe> dimitern: i've just reproduced it
<dimitern> rogpeppe: cool
<dimitern> rogpeppe: w/o goyaml-go?
<rogpeppe> dimitern: no - i'll just try with non-cgo goyaml
<dimitern> rogpeppe: ok, if it's indeed a runtime bug now begins the tedious hunting for a minimal case to reproduce i guess..
<rogpeppe> dimitern: yeah.
<dimitern> rogpeppe: while you're looking at this, i'll finish the rest of the tests, so hopefully if there's a way around the panic it'll be ready for proposing
<dimitern> rogpeppe: if you don't mind
<rogpeppe> dimitern: you could work against 1.0.3 for the time being
<dimitern> rogpeppe: does it work like that?
<rogpeppe> dimitern: it certainly should do
<rogpeppe> dimitern: we're trying to keep 1.0.3 compatibility after all
<rogpeppe> dimitern: if it doesn't, it's a bug
<dimitern> rogpeppe: i'm getting 1.0.3 again to try
<rogpeppe> dimitern: if you're using go from source, you can just do hg update go1.0.3
<rogpeppe> dimitern: then run all.bash
<dimitern> rogpeppe: I'll put it in a separate dir and just wipe out the gopath/pkg/*, i messed it up last time i tried to setup multiple go versions + gopaths so i can switch between them easily, i'll be more careful this time
<rogpeppe> dimitern: i have an entirely separate tree for working with separate go versions
<rogpeppe> dimitern: (which is actually very useful because i can be testing against one branch while developing another)
<dimitern> rogpeppe: well, I tried having ~/go/<ver> for goroot, and ~/work/go/<ver> for gopath, but then it didn't work with emacs somehow after the raring upgrade
<dimitern> rogpeppe: (come to think about it, it still doesn't work - i need to run emacs from a terminal rather than from the launcher, otherwise the gopath/root i set in .bashrc are not visible
<rogpeppe> dimitern: that's not surprising. emacs won't run .bashrc
<rogpeppe> dimitern: did you try putting the settings in your .profile ?
<dimitern> rogpeppe: but it worked in quantal! now it cannot find gofmt etc. unless run from the shell
<dimitern> rogpeppe: i tried .profile, .Xsessionrc, /etc/profile, everything.. no joy
<rogpeppe> dimitern: try removing it from your .bashrc, adding it to your .profile, logging out and logging in again
<rogpeppe> dimitern: .bash_profile might work too
<dimitern> rogpeppe: tried that as well, just emacs somehow defies all known ways it should cooperate with my environment :)
<rogpeppe> dimitern: i'd be surprised. this is an environment-variable/shell issue, not emacs, i think
<dimitern> rogpeppe: well, that very well maybe, but haven't seen a problem with another app so far
<rogpeppe> dimitern: that's probably because no other app needs $GOPATH and $GOROOT
<rogpeppe> dimitern: or relies on any setting in your .bashrc, i suspect
<dimitern> rogpeppe: btw - if i just rm -fr $GOPATH/pkg/ and then run should work, or I should leave pkg empty?
<rogpeppe> dimitern: that should work fine.
<rogpeppe> dimitern: directories are created on demand
<dimitern> rogpeppe: hmm apparently goyaml-go uses go1.1 features (function ends without a return statement)
<dimitern> rogpeppe: switching to goyaml-c
<rogpeppe> dimitern: ha
<dimitern> wallyworld_: ping
<wallyworld_> hi
<dimitern> wallyworld_: hey, looking at create-machine
<wallyworld_> thanjs
<dimitern> wallyworld_: why do we need this at all? why not have an argument to deploy instead? don't we want it to be atomic?
<wallyworld_> it's part of the containerisation work
<wallyworld_> maybe read tim's email
<dimitern> wallyworld_: i know; i read it
<dimitern> wallyworld_: but still now convinced it's a good idea to have a separate command
<wallyworld_> the next step is to allow containers to be created inside a machine
<dimitern> wallyworld_: i see that, ok, but still - using 2 steps to deploy inside a container seems wrong
<wallyworld_> ultimately the common operation ie deploy will be atomic. this is a step along the way
<dimitern> rogpeppe: with go 1.0.3 i see no panics so far, but still some tests failures with the watchers (again, not on every run)
<wallyworld_> just constructing the jigsaw peices
<dimitern> wallyworld_: so you're saying create-machine isn't there to stay?
<wallyworld_> not sure yet
<wallyworld_> all up for prototyping and discussion
<wallyworld_> will aid in testing during the development etc
<wallyworld_> plus it ma turn out to be useful longer term, not sure yet
<dimitern> wallyworld_: yeah, that's just the thing - adding stuff we're not sure we need right now will surely bend the direction in the future to finding ways to use them
<dimitern> wallyworld_: anyway, just thinking out loud - will take my concerns to tim's email :)
<wallyworld_> fair questions. i started this work before the email. it's all very much in a state of flux
<wallyworld_> but as a developer, the ability of create machines/containers will be very usrful
<dimitern> wallyworld_: sure when starting on a big new feature there's always uncertainty where to begin
<wallyworld_> eg may want to spin up a few containers ahead of time
<dimitern> wallyworld_: i agree, but i doubt a command is the necessary for that
<wallyworld_> and then scale out later
<wallyworld_> how else would it be done if there were no command to so it
<dimitern> wallyworld_: you see "juju's way" of doing things afaiui is telling it what you need and letting it do it for you
<dimitern> wallyworld_: what mark (sabdfl) said scaling out a service + containers could be automated
<wallyworld_> sure. but having some manual control would be nice too
<dimitern> wallyworld_: having a service deployed on 1 machine + some containers and this is 1 unit, adding a unit could replicate that and the containers
<dimitern> wallyworld_: could be, not sure yet, i think it's worth some careful consideration
<wallyworld_> think of it like being able to add an existing running machine/vm to the env
<wallyworld_> it's just another tool in the kit
<dimitern> wallyworld_: you could do that with manual provisioning (eventually), but this is just a special case - why complicate things adding more special cases like that?
<rogpeppe> dimitern: https://code.google.com/p/go/issues/detail?id=5554
<wallyworld_> dimitern: for me it's a useful tool. ymmv. the command bit is a small portion of the logic. i like having manual control
<dimitern> rogpeppe: cool! thanks for filing a bug
<rogpeppe> dimitern: i'm currently testing against tip. it's possible the problem is already fixed.
<dimitern> rogpeppe: so with go 1.0.3 more often than not tests pass (1 in 5 failure so far)
<rogpeppe> dimitern: i'm sure there's a problem you need to fix too :-)
<rogpeppe> dimitern: i'm not looking into it any further for the moment, because there's lots of stuff i want to get proposed today before i go away next week
<dimitern> rogpeppe: it'll be helpful if i manage somehow to reproduce it consistently
<dimitern> rogpeppe: sure
<dimitern> rogpeppe: but I want to propose a split of apiserver.go into multiple submodules
<rogpeppe> dimitern: different packages?
<rogpeppe> dimitern: if you're just talking about splitting the file, +1
<dimitern> rogpeppe: the same package state/apiserver/ but having things like root.go, client.go, machine.go, etc.
<dimitern> rogpeppe: ok, i'll do it after i finish this
<rogpeppe> dimitern: i'd name all the ones implementing the actual api to start with "api"
<rogpeppe> dimitern: e.g. apiroot.go, apiclient.go, apimachine.go
<rogpeppe> dimitern: so it's easy to see which files implement the core logic
<dimitern> rogpeppe: also i'd like to split api_test.go into perm_test.go, machine_test.go, client_test.go, etc.
<rogpeppe> dimitern: seems reasonable
<dimitern> rogpeppe: i'm fine with api*.go
<dimitern> rogpeppe: cool
<rogpeppe> dimitern: if you do it, make it a branch which is entirely mechanical - no code changes at all, please.
<rogpeppe> dimitern: i'm not seeing any panics against tip, BTW
<dimitern> rogpeppe: sure, but after i'm done with this; don't want to make it harder for me to merge later
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: will it impede the run time too much if we split agent and client perm tests into 2 files and 2 tables?
<rogpeppe> dimitern: i'm not sure i see the point
<rogpeppe> dimitern: the point of those tests is to test permissions against the entire API surface area
<rogpeppe> dimitern: you could split the table, i suppose
<dimitern> rogpeppe: hmm, ok - but only perm checks should be in there; error conversion tests, bad logins, etc should be separate
<rogpeppe> dimitern: agreed
<rogpeppe> dimitern: i have some vaguely formed thoughts about how we can make the perm checks much smaller and faster
<dimitern> rogpeppe: oh?
<dimitern> rogpeppe: (while i'm at splitting stuff i might as well split apiclient to multiple files as well)
<rogpeppe> dimitern: that's a fair bit smaller - i wonder if it might be better just as two files - one for the client api and one for the agent api
<rogpeppe> dimitern: or actually
<rogpeppe> dimitern: it's perhaps good to have a 1-1 file mapping between apiserver and api
<dimitern> rogpeppe: exactly my point
<rogpeppe> dimitern: for the pieces implementing the api
<dimitern> rogpeppe: good, will do then
<rogpeppe> dimitern: i've run the apiserver tests 17 times so far against tip and no panic
<dimitern> rogpeppe: good! so it seems fixed
<rogpeppe> dimitern: or the bug just isn't being tickled any more
<dimitern> rogpeppe: well, it might also mean someone will recognize it and say it's fixed
<rogpeppe> dimitern: yeah. i'm hoping someone will say "ah, we fixed it then", and then they'll issue a new point release
<rogpeppe> wallyworld_: ping
<wallyworld_> rogpeppe: hi, i'm just about to leave for soccer
<rogpeppe> wallyworld_: v quick question
<wallyworld_> ok
<rogpeppe> wallyworld_: in environs/cloudinit, you did syslogConfigRenderer, right?
<wallyworld_> i think so, let me check
<rogpeppe> wallyworld_: is there a particular reason why the config is rendered after bootstrap-state is called?
<rogpeppe> wallyworld_: because i'm thinking about moving it earlier in the script
<wallyworld_> rogpeppe: i think it will be ok to move. i think it was put where it is because that's where some previous logging stuff was initialised
<wallyworld_> but i'm not entirely sure
<rogpeppe> wallyworld_: ok, cool
<wallyworld_> all it does anyway is write a conf file
<wallyworld_> so that can go anywhere really
<rogpeppe> wallyworld_: that's what i thought; just wanted to check there wasn't anything subtle going on.
<rogpeppe> wallyworld_: thanks
<wallyworld_> np.
<dimitern> rogpeppe:
<rogpeppe> dimitern:
<dimitern> rogpeppe: oops, I have a question :)
<rogpeppe> dimitern: go on
<dimitern> rogpeppe: in this log: http://paste.ubuntu.com/5696381/
<dimitern> rogpeppe: is it normal to have an Error response to the Stop request?
<rogpeppe> dimitern: i don't see that you are
<rogpeppe> dimitern: which line are you thinking is an error response to the stop request?
<rogpeppe> dimitern: i am seeing that i need to sort out the rpc logging - it's logging each message twice
<dimitern> rogpeppe: ah, right - [LOG] 94.86663 DEBUG rpc/jsoncodec: -> {"RequestId":4,"Error":"watcher has been stopped","ErrorCode":"stopped","Response":{}}
<dimitern> this is the Next response
<rogpeppe> dimitern: (i know why)
<rogpeppe> dimitern: that's a response to a Next
<rogpeppe> dimitern: note the RequestId
<dimitern> rogpeppe: yeah
<dimitern> rogpeppe: i think we have a problem with the current watcher implementation in the client, but only the environ config one fails because it needs to do extra work on marshaling/unmarshaling the AllAttrs when receiving a response
<dimitern> rogpeppe: it seems next should quit as soon as stop was called
<dimitern> rogpeppe: (the next goroutine in the client, that is)
<rogpeppe> dimitern: on the server side?
<rogpeppe> dimitern: why's that?
<dimitern> rogpeppe: because otherwise we have this issue
<dimitern> rogpeppe: and arguably it's an error to try reading a stopped watcher anyway
<rogpeppe> dimitern: perhaps you could try describing the issue and why it's happening
<dimitern> rogpeppe: well, i *think* as soon as stop is called the next goroutine does not exit immediately and tries to read, getting the error, and then exits, but the error is dispatcher to the client outside
<dimitern> rogpeppe: but really it shouldn't report that error if it knows for sure the server-side watcher was stopped
<rogpeppe> dimitern: isn't that what the ErrStillAlive check is about?
<dimitern> rogpeppe: if it worked i shouldn't have received anything on the in channel, right?
<dimitern> rogpeppe: when next encounters a stopped state
<rogpeppe> dimitern: ha, yes
<rogpeppe> dimitern: it should return after the Kill
<dimitern> rogpeppe: ah, I think I see the error - after w.tomb.Kill(err) i should return rather than trying the select
<dimitern> :)
<dimitern> yeah
<dimitern> rogpeppe: that fixed it
<rogpeppe> dimitern: cool
<dimitern> rogpeppe: only one things bothers me - i think it's maybe worth panicking when i cannot unmarshal the environ config i got on the in channel, rather than logging an error
<rogpeppe> dimitern: i think the watcher should die if that happens
<dimitern> rogpeppe: it does, but if we got that far something is very wrong, hence a panic is in order i think
<rogpeppe> dimitern: can't that happen if the server sends some dubious data back?
<dimitern> rogpeppe: and where did it get it from? state - so it think it's still worth panicking
<rogpeppe> dimitern: definitely not
<dimitern> rogpeppe: explain?
<rogpeppe> dimitern: in general we don't panic when an external data source produces something unexpected
<dimitern> rogpeppe: hmm
<rogpeppe> dimitern: it might be talking to a version of the API with a subtly different interface
<dimitern> rogpeppe: ok then, i'll leave just the error log
<dimitern> rogpeppe: fair point
<rogpeppe> dimitern: you don't need to log the error - you can kill the watcher with an appropriate error
<rogpeppe> dimitern: then it will be picked up as usual by the rest of the logic
<rogpeppe> dimitern: causing the agent to die, or whatever
<dimitern> rogpeppe: i'm killing it, and logging the error
<rogpeppe> dimitern: i'm not sure that the log is necessary, but i guess
<dimitern> rogpeppe: it won't hurt when debugging incompatible api server/client connections, i think
<dimitern> rogpeppe: btw you copied my branch when you filed the golang issue, right?
<rogpeppe> dimitern: yes
<dimitern> rogpeppe: cool, i'm proposing it soon
<dimitern> rogpeppe: so with the newly gained understanding of the api internals hopefully, i'll be able to help the other guys next week while you're gone
<rogpeppe> dimitern: that would be great, thanks
<rogpeppe> dimitern: another small CL: https://codereview.appspot.com/9710044
 * dimitern looking
<dimitern> rogpeppe: what's "bootstrap-state" ?
<rogpeppe> dimitern: jujud bootstrap-state
<rogpeppe> dimitern: the thing that initialises the mongo state
<rogpeppe> dimitern: and sets the initial password for the machine agent
<dimitern> rogpeppe: i see (never heard of it before)
<dimitern> rogpeppe: LGTM
<dimitern> rogpeppe: do you think you'd manage adding the agent API connection stuff today?
<rogpeppe> dimitern: that's the aim
<rogpeppe> dimitern: i have a branch that does it
<rogpeppe> dimitern: but there are one or two branches that need to go in first
<dimitern> rogpeppe: that will be awesome
<rogpeppe> dimitern: well, let's sat i have a branch that does it *in theory* :-)
<dimitern> rogpeppe: i was thinking once i started the api stuff i might as well do most (if not all of it), so others can concentrate on replacing state calls with api calls next week, once we have the connection
<rogpeppe> dimitern: that sounds like a good plan
<rogpeppe> dimitern: BTW the API connection stuff involved refactoring the machine agent, and it's turned out really quite nice
<dimitern> rogpeppe: good to hear
<rogpeppe> dimitern: did you look at the Runner CL?
<rogpeppe> dimitern: i don't think you reviewed it, which was a bit surprising :-)
<dimitern> rogpeppe: this one? https://codereview.appspot.com/9643043/
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: I looked at it briefly, but by the time I got the time to review it it already landed :)
<dimitern> rogpeppe: https://codereview.appspot.com/9721043/ is ready
<dimitern> rogpeppe: will look at yours once more more carefully
<rogpeppe> dimitern: np. more eyes always appreciated - it's a moderately complex piece of code.
<rogpeppe> dimitern: i'm slightly surprised you've done all those api calls in one CL. i thought they might work well as many separate ones.
<rogpeppe> dimitern: it's not a particular problem, but i might not get it reviewed today
<dimitern> rogpeppe: they're closely related - all needed by the provisioner and really most are trivial
<dimitern> rogpeppe: np, go as far as you can
<rogpeppe> dimitern: i guess i just prefer separable changes to be separate :-)
<dimitern> rogpeppe: i guess watchers and machine calls can be separated
<rogpeppe> dimitern: that might be good, if you don't mind too much.
<dimitern> rogpeppe: uhh.. you're right actually - just now glanced at the diff - it's in excess of 1000 lines
<rogpeppe> dimitern: yeah, it's pretty big
<dimitern> rogpeppe: didn't look like that when i was doing it :) anyway - will try bzr pipes to see if i can split it easily
<rogpeppe> dimitern: you might not need pipes too much
<rogpeppe> dimitern: most of the changes are orthogonal
<rogpeppe> dimitern: if you do the commonWatcher thing first (making the EntityWatcher use it only) then all the rest are likely to be independent
<dimitern> rogpeppe: good point, will do
<dimitern> rogpeppe: ok, disregard that CL then, I'll propose it after splitting
<rogpeppe> dimitern: thanks
 * dimitern hates bzr diff!!!!
<mgz> ?
<mgz> you know it takes arguments, right?
<dimitern> mgz: so i've been in this situation before
<dimitern> mgz: i have trunk and a branch off it
<dimitern> mgz: i need to see changes in a particular path between the two branches
<dimitern> mgz: but none of the suggested args help
<dimitern> mgz: tried --old trunk while on the new branch; tried -r-2, but i need a path as well, not the changes in the whole tree
<dimitern> mgz: tried --old trunk --new feature-branch - still frustratingly no results!
<dimitern> mgz: please help :)
<dimitern> mgz: (if it makes a difference - both branches are clean - no uncommitted changes)
<mgz> hm, I'd expect --old to work
<mgz> have you got qbzr intalled?
<dimitern> mgz: well it doesn't; yes I have qbzr
<dimitern> mgz: with qdiff is still the same (No changes)
<dimitern> mgz: all i need here is diff trunk:tip and feature:tip and show me the changes in state/api/ only
<mgz> so, it works for me with standalone branches
<dimitern> mgz: so i can cherry pick some of them and split the feature into multiple smaller ones
<mgz> but something seems to be unhappy with native colo
<mgz> but, I'd expect if you cd ~ then do `bzr diff --old .../src/real/branch/location/trunk --new .../src/.../feature state/api` it'll work
<dimitern> mgz: here's what bzr reports: http://paste.ubuntu.com/5696558/
<dimitern> mgz: ah, i'll try that
<mgz> right, so try --old ~/src/juju-core/trunk --new ~//src/juju-core/041-provisioner-api-calls
<mgz> there might be an issue with checkouts, I'll see if we have a bug filed
<dimitern> mgz: hallelujah! it works like that
<dimitern> mgz: although i'd expect it to work in a LW checkout as well..
<mgz> looks like bug 596785 plus bad location resolution code in the diff command
<_mup_> Bug #596785: 'bzr diff --old|new PATH' does silently exists if arg does not exist <diff> <patch-needswork> <Bazaar:Confirmed> <https://launchpad.net/bugs/596785>
<mgz> can you file a new bug with your symptoms please dimitern?
<dimitern> mgz: a new one or amend this
<mgz> a new one
<dimitern> mgz: sure
<mgz> the location code being wrong I don't see any entry for, can do both issues in one merge possibly, but might make sense to do them seperately
<dimitern> mgz: bzr diff --old/--new does not report and changes with a lightweight checkout of a native repo?
<dimitern> mgz: as summary
<mgz> s/and/any/ and seems reasonable
<mgz> and just end after "checkout"
<dimitern> mgz: not sure i get you - where's the "and" ?
<dimitern> mgz: ah
<mgz> :)
<dimitern> mgz: bug 1183776
<_mup_> Bug #1183776: bzr diff --old/--new does not report and changes with a lightweight checkout of a native repo <amd64> <apport-bug> <bzr> <diff> <raring> <bzr (Ubuntu):New> <https://launchpad.net/bugs/1183776>
<mgz> thanks!
<dimitern> mgz: any idea how to make my life easier and cherry pick interactively what to merge? rather than having to manually edit the diff and run patch with it
<danilos> dimitern, mgz: I need to step out to sign some documentation in the court, likely to not make it for the stand-up
<dimitern> danilos: ok
<rogpeppe> dimitern, TheMue: another small CL: https://codereview.appspot.com/9723043
<mgz> dimitern: use shelve
<dimitern> rogpeppe: LGTM
<rogpeppe> dimitern: ta!
<dimitern> mgz: i'll try
<dimitern> mgz: how will shelve help with committed trees?
<mgz> cherrypick the feature branch across to your new integration branch
<mgz> then set EDITOR and use shelve interactive to edit out the bits you don't want
<dimitern> mgz: it seems way too complicated.. i'll just branch the old feature and vistually merge into the new one with directory comparison
<mgz> it's two commands...
<mgz> but depending on how hard the changes are to unpick, interactive mode may be useful or not
<mgz> oh, and I'm not actually sure what `bzr merge -i` does
<mgz> what you want, it seems
<mgz> so, you have feature, which you want to split out
<mgz> dimitern: so, in your checkout, on trunk, `bzr switch -b prefeature` then `bzr merge -i .../feature`
<dimitern> mgz: i'll try next time, thanks; spent too much time already fiddling around with tools, so i'm back to good old manual methods + meld for visual merging
<TheMue> rogpeppe: done
<dimitern> mgz, wallyworld_: are we doing the standup?
<mgz> lets
<mgz> can just say hi at least :)
<dimitern> rogpeppe, mgz: https://codereview.appspot.com/9667048 first part
<rogpeppe> dimitern: that's more like it :-) :-)
<dimitern> TheMue: ^^ reviews appreciated
<dimitern> there are 3 more CLs coming in this pipeline
<TheMue> dimitern: done
<dimitern> TheMue: cheers
<dimitern> TheMue: actually making them arguments to init() will make the code less readable
<dimitern> TheMue: and I don't see an advantage in doing that
<dimitern> TheMue: could you explain your reasoning?
<TheMue> dimitern: having them as arguments would let you check if they are set. that's the only reason.
<dimitern> TheMue: it won't compile anyway if they're not set
<dimitern> TheMue: i'll add a check in init() to ensure they are set, but not as arguments
<TheMue> dimitern: it won't compile?
<dimitern> TheMue: no, you're right - it will compile, but panic at runtime
<dimitern> TheMue: i'll add a check, thanks
<TheMue> dimitern: you set them after init(), so a check there would fail.
<dimitern> TheMue: i moved init after setting the,
<TheMue> dimitern: so please also comment that those values have to be set before calling init().
<dimitern> TheMue: sure
<TheMue> dimitern: great, thx
<dimitern> thanks guys, submitting with the changes
<dimitern> rogpeppe, mgz, TheMue: next in line: https://codereview.appspot.com/9714044
<dimitern> danilos: you as well? ^^
<rogpeppe> dimitern: you have a review
<dimitern> rogpeppe: tyvm
<rogpeppe> dimitern: much better split into bits like this!
<dimitern> TheMue: calling init() in commonLoop() is not quite the same, because it's needs to be called before the go routine spins up
<TheMue> dimitern: due to the created channel?
<dimitern> TheMue: yeah, that as well
<TheMue> dimitern: ok, could lead to a race otherwise. what else?
<dimitern> TheMue: actually I think this is it - couldn't find other reason
<TheMue> dimitern: ok, fine
<dimitern> thanks for the reviews, submitting soon; 2 more to go
<dimitern> rogpeppe: kanban
<dimitern> rogpeppe, TheMue: last one about the provisioner: https://codereview.appspot.com/9708044/
<TheMue> *click*
<TheMue> dimitern: done
<dimitern> TheMue: thanks!
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> rogpeppe: we you're saying the agent shouldn't use SetMongoPassword at all, and use SetPassword instead?
<rogpeppe>  dimitern: yes
<dimitern> rogpeppe: ok, even better
<rogpeppe> dimitern: the API server knows when SetMongoPassword is appropriate, and does it
<dimitern> rogpeppe: it's getting smarter then :)
<rogpeppe> dimitern: that's kinda the idea of the API :-)
<dimitern> rogpeppe: indeed
<dimitern> rogpeppe: does the same apply for unit.SetMongoPassword?
<rogpeppe> dimitern: yes. hmm, that's an interesting point actually
<rogpeppe> dimitern: currently SetPassword should be doing SetMongoPassword for all agents
<dimitern> rogpeppe: so for unit.SetMP i'll use the same logic  as for a machine
<rogpeppe> dimitern: yeah - i.e. it should always be set.
<rogpeppe> dimitern: presumably you've merge trunk?
<mramm> anybody know how sync-tools works
<dimitern> rogpeppe: ok
<rogpeppe> merged?
<dimitern> yeah
<rogpeppe> mramm: i've never used it
<dimitern> rogpeppe: it's used to sync ec2 public tools to your private bucket
<mramm> and have a minute to walk danwest through getting tools into MAAS?
<rogpeppe> dimitern: yeah i know roughly what it does
<rogpeppe> dimitern: but i've never actually *used* it
<dimitern> rogpeppe: ops, sorry - that was for mramm actually
<dimitern> rogpeppe: got too used to addressing you lately :)
<rogpeppe> dimitern: np - i do that too sometimes
<mramm> can you use sync-tools as an argument to bootstrap
<mramm> and IIRC you need to put your EC2 credentials somewhere to make it work
<mramm> is this documented anywhere that we can just point danwest too?
<dimitern> mramm: no
<mramm> :(
<dimitern> ramit's a separate command
<mramm> ok
<dimitern> mramm: there was a mail from jam about how it works
<mramm> so bootstrap and then sync tools?
<mramm> ok, will search for that
<dimitern> rogpeppe: do you think unit tests for agents/workers in general should also only use the api?
<rogpeppe> dimitern: that's an interesting question.
<rogpeppe> dimitern: i'm not entirely sure it's possible
<rogpeppe> dimitern: you'd need to juggle at least two API connections and send the right commands to the right one
<dimitern> rogpeppe: yeah
<dimitern> rogpeppe: and if we leave them using state directly how bad will it be?
<rogpeppe> dimitern: i'm not sure that's bad at all
<rogpeppe> dimitern: the actual code being tested will need to use the API
<rogpeppe> dimitern: it's just the setup code which will talk directly to mongo (and probably some verification code too)
<dimitern> rogpeppe: yeah, exactly, the tests can still access it directly
<rogpeppe> dimitern: yup
<dimitern> rogpeppe: and btw this will save a *huge* amount of work
<rogpeppe> dimitern: yeah
<rogpeppe> i hadn't seen this error before: cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again.
<dimitern> rogpeppe: it seems like an ec2 error
<rogpeppe> dimitern: yeah it is. just another one i hadn't seen before
<rogpeppe> dimitern: to join the collection of sporadic live test failures.
<rogpeppe> dimitern: getting live tests to pass reliably is gonna be really hard
<dimitern> rogpeppe: yup, unfortunately
<danwest> dimitern: mramm pointed me your way for juju-core/MAAS bootstrap issues
<dimitern> danwest: hm, ok - what do you need?
<danwest> dimitern: new raring install with apt-get juju-core
<danwest> trying to bootstrap against MAAS provider and get the following:
<danwest> ~$ juju -v bootstrap
<danwest> 2013/05/24 11:27:45 INFO environs: reading tools with major version 1
<danwest> 2013/05/24 11:27:45 INFO environs: falling back to public bucket
<danwest> 2013/05/24 11:27:45 ERROR command failed: no tools available
<danwest> error: no tools available
<dimitern> danwest: can you try juju-core from source and see if it helps?
<mramm> I expect that nobody has tested the sync tools path with MAAS
<mramm> it landed at the very last second for the raring cycle
<mramm> I know I have not tested it myself :/
<dimitern> mramm: me too, and maas landed at the last moment as well, so it might not work from the released version
<danwest> dimitern: mramm: I can try from source but that might take me a couple to get setup
<mramm> dimitern: dan is working on the "cloud installer" and any help we can give him would be good
<danwest> thanks mramm
<mramm> dimitern: we want to keep him unblocked ;)
<dimitern> mramm: sure, i'll help if i can, but have never tried the maas provider
<mramm> yea, but william is out
<dimitern> danwest: we fixed a bunch of issues with tools and bootstrapping after the raring release, hence my suggestion to run from source
<danwest> dimitern: k
<mramm> danwest: once you have a working source install
<mramm> the upload tools thing you were pointed at a while ago will probably work
<danwest> dimitern: mramm: grabbing source now
<mramm> danwest: sorry this is not a better path yet
<mramm> I have to run out and get some food, back in a few
<danwest> same here - just grabbing source first
<mramm> william will also be back on monday, and I know he has tested this
<danwest> k
<Makyo> I'm seeing http://pastebin.ubuntu.com/5697229/ in trunk, running in a precise lxc.  Is this a known thing?
<dimitern> Makyo: don't tell me you never updated from lucid? :)
<Makyo> dimitern, Haha, nah, running raring, but tests were failing there, thus the precise lxc dev setup
<dimitern> Makyo: well, lucid is definitely not among the expected series iirc
<dimitern> Makyo: can you reproduce it consistently?
<Makyo> dimitern, Every time.
<Makyo> Let me run with -gocheck.v
<Makyo> No more info.
<dimitern> Makyo: running with -gocheck.v in environs/imagemetadata/
<Makyo> dimitern, Yeah, no extra info on the failure, others pass.
<Makyo> dimitern, lucid is specified in the test, environs/imagemetadata/simplestreams_test.go:463
<dimitern> Makyo: file a bug; maybe they decided to drop lucid from the cloud archive just today
<Makyo> dimitern, got it.  Thanks.
<dimitern> Makyo: thanks
<dimitern> rogpeppe: ping
<rogpeppe> dimitern: pong
<dimitern> rogpeppe: actually why do you insist on having api prefix to filenames when split?
<rogpeppe> dimitern: because i like having error.go and server.go stand out as files related to core api server functionality, not the actual API methods
<dimitern> rogpeppe: hmm
<dimitern> rogpeppe: well, ok
<dimitern> rogpeppe: so apiserver.go will be gone, to be replaced by apiroot.go, apistate, apiclient, apimachine, etc, right?
<rogpeppe> dimitern: there is actually a possibility that the api* files could be moved into their own package actually (in the api server anyway)
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: that sounds better - so apiserver/api/*
<rogpeppe> dimitern: yeah, but it's only a possibility - i'd prefer not to do it for now
<dimitern> rogpeppe: what are your concerns about it?
<rogpeppe> dimitern: it feels unnecessary
<dimitern> rogpeppe: why not rename server to apiserver, errors to apierrors and leave the rest uprefixed?
<dimitern> rogpeppe: they's stand out even more
<rogpeppe> dimitern: that's not a bad idea
<dimitern> rogpeppe: cool, i'll make it so then
<rogpeppe> dimitern: sgtm
<dimitern> rogpeppe: and the same for the client?
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: nice
<danwest> dimitern: any tricks to building juju-core from source?
<dimitern> danwest: so if you do go get -v -u launchpad.net/juju-core/... (note the ...)
<dimitern> danwest: once done, you'll just need to do go install launchpad.net/juju-core/cmd/juju
<danwest> k
<dimitern> danwest: and then use juju <command> as usual
<dimitern> rogpeppe: still there?
<rogpeppe> dimitern: aye
<dimitern> rogpeppe: i'm done with the splitting, proposing in 5m, and i'd like if you take a look
<rogpeppe> dimitern: ok. i'm gonna be gone quite soon though, and i'm really trying to get this branch proposed
<dimitern> rogpeppe: if you can; it's mechanical, but mostly if it seems ok as an arrangement https://codereview.appspot.com/9746043
<rogpeppe> dimitern: well remembered about the copyright notices :-)
<dimitern> rogpeppe: oh yeah :)
<rogpeppe> dimitern: i get "upload in progress" at https://codereview.appspot.com/9746043/diff/1/state/apiserver/apiserver.go?column_width=90
<rogpeppe> dimitern: i think that's because codereview is confused though.
<dimitern> rogpeppe: hmm.. well it's just what server.go used to be
<dimitern> rogpeppe: probably the rename + modify somehow confuses rietveld
<rogpeppe> dimitern: utils.go rather than helpers.go ?
<dimitern> rogpeppe: np
<rogpeppe> dimitern: i'm wondering if each test file should have its own suite
<dimitern> rogpeppe: we can do that, but let it be a follow up
<rogpeppe> dimitern: but i don't mind that much
<rogpeppe> dimitern: yeah, definitely
<dimitern> rogpeppe: i think it should be considered a trivial CL
<dimitern> rogpeppe: and would really want to land it today, to save me some trouble tracking moving trunk
<rogpeppe> dimitern: where have the auth* functions ended up?
<dimitern> rogpeppe: in root.go
<rogpeppe> dimitern: ah, that seems goo
<rogpeppe> d
<rogpeppe> dimitern: it all seems great actually
<dimitern> rogpeppe: cool enough to land?
<rogpeppe> dimitern: LGTM trivial
<dimitern> rogpeppe: great, tyvm!
<rogpeppe> dimitern: i think that's the largest "trivial" change ever :-)
<dimitern> rogpeppe: yeah :)
<rogpeppe> dimitern: in case you're still around, you might want to take a look at this: https://codereview.appspot.com/9738043
<dimitern> rogpeppe: will do
<dimitern> rogpeppe: LGTM
<rogpeppe> dimitern: thanks. i won't submit until the next one's ready though
<dimitern> rogpeppe: ok
<dimitern> rogpeppe: btw "biggest diff ever": Diff against target:	7640 lines (+3825/-3612) 26 files modified
<rogpeppe> dimitern: what's that in?
<dimitern> rogpeppe: the one I just landed
<rogpeppe> dimitern: :-)
<rogpeppe> dimitern: not as big as the error one which may happen sometime, probably
<dimitern> rogpeppe: in lp - doesn't show the full diff, just the first 5000 lines, but the summary at the top shows the info above
<dimitern> rogpeppe: oh, yeah - we should do that
<rogpeppe> dimitern: i didn't get it working yet - i was spending too much time on it, so put it on hold
<dimitern> rogpeppe: yeah, it'll come to that
 * dimitern is still not quite sure i didn't screw up trunk with that rename
<dimitern> I pulled and run all tests pass
<dimitern> now i'm doing the same in a fresh clone of trunk, just to be shure
<dimitern> *sure
 * dimitern is relieved.. all tests pass on trunk.. *whew*
<dimitern> time to go
<dimitern> rogpeppe: good night and have a nice holiday!
#juju-dev 2013-05-26
<kyhwana> hmm, what configuration options than the smokeping configs in /etc/ should I be setting up/letting people change before deploying a smokeping charm?
<wallyworld_> thumper: morning. good weekend?
<thumper> wallyworld_: hi there, yes thanks
<thumper> wallyworld_: I'd like to have a chat with you soonish
<thumper> just replying to a long email
<wallyworld_> sure. i was hoping for the same thing
<wallyworld_> i think i know what i want to do next
<wallyworld_> want to coordinate
<thumper> ack
#juju-dev 2014-05-19
<wallyworld> waigani: hi
<waigani> wallyworld: could I hangout with you for 5min?
<wallyworld> sure
<perrito666> nites/morning people
<waigani> wallyworld: just swapped two occurrences of GetMockBuildTools for GetMockBundleTools in one file: http://paste.ubuntu.com/7486270/
<waigani> wallyworld: tests now pass
<wallyworld> great
<waigani> I'll leave it there for now unless you/william think it is urgent to get rid of GetMockBuildTools all together.
<waigani> I need to start looking at identity now
<hazmat`> g'morning
<waigani> morning hazmat`
<waigani> morning perrito666
<wallyworld> axw: morning, good weekend?
<axw> wallyworld: morning, yeah not too shabby
<axw> never get to rest anymroe on the weekend though
<wallyworld> i had to give my son a driving lesson, very stressful :-)
<axw> ooh :)
<wallyworld> good news on the house?
<axw> mine was slightly stressful too ;)
<axw> yes
<wallyworld> \o/
<axw> it is ours
<wallyworld> great :-)
<wallyworld> we should have martin back today, so standup tonight
<axw> cool, sounds good
<wallyworld> feel free to ping me before then if required. you've got some wip and i've made a couple of cards high propority
<axw> no worries
<wallyworld> the cards i've bumped are the cause of the last few test failures
<wallyworld> on jenkins
<thumper> axw, wallyworld: local tests are getting better, 5 times through only one failure, replicaset
<wallyworld> yeah, getting there :-)
<thumper> wallyworld: we need to chat, can I grab you in about 15 minutes?
<wallyworld> sure, anytime
<axw> thumper: glad to hear that
<axw> the peergrouper one was a pain in the arse
<thumper> any plans on how to fix the replica set failures?
<thumper> they seem problematic
<wallyworld> one fix has gone in already
<wallyworld> to retry after starting mongo since the replicaset may not be ready just yet
<wallyworld> just need to keep plugging away at it, mongo seems quite flakey at times
<waigani_> just spotted this in trunk: provider/openstack/storage.go:59: assignment count mismatch: 3 = 2
<waigani_> anyone on it?
<wallyworld> waigani_: your goose deps are wrong
<waigani_> oooh
<wallyworld> godeps will fix it
<waigani_> wallyworld: I just filed a bug. Is there a way to delete it or should I mark it as invalid?
<wallyworld> mark it as invalid
<waigani_> done
<axw> wallyworld: are our architecture codes meant to match simplestreams? if not, is there a mapping somewhere?
<wallyworld> the codes in arch.go?
<axw> wallyworld: yeah, i386, armhf, etc.
<wallyworld> there's a regexp transformation in arch.go
<wallyworld> NormaliseArch()
<axw> wallyworld: that's for uname
<axw> isn't it?
<axw> yeah, uname
<wallyworld> ah, sorry was going from memory
<axw> wallyworld: I found that the tests in environs/instances use dummy simplestreams data that has an entry for "arm", whereas we expect armhf
<wallyworld> in that case i think the codes are meant to match
<axw> wallyworld: ok, so I will change that dummy data
<wallyworld> yeah, that may well be wrong
<davechen1y> https://bugs.launchpad.net/juju-core/+bug/1320738
<_mup_> Bug #1320738: constraints: gccgo test failure <gccgo> <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1320738>
<davechen1y> bugger
<waigani> Gobot reports environs/sync [build failed], but when I run the tests locally it passes... hmmm
<waigani> ah, I see why, ignore me
<jam> thumper: are we generally cancelling our weekly 1:1? Did you find a better time?
<waigani> I'm hitting this on the gobot:
<waigani> PANIC: sync_test.go:420: badBuildSuite.SetUpTest
<waigani> ... Panic: cannot share a state between two dummy environs; old "only"; new "only"
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1320738/comments/1
<_mup_> Bug #1320738: constraints: gccgo test failure <gccgo> <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1320738>
<davecheney> annoying
<davecheney> thoughts, http://paste.ubuntu.com/7487106/ ?
<wallyworld_> axw: did you confirm your mp fixes the tests on i386 by doing a test run, perhaps with version.Current.Arch hardcoded or something?
<axw> wallyworld_: yes that's what I did
<wallyworld_> great
<axw> wallyworld_: modified juju/arch/arch.go
<wallyworld_> ok
<wallyworld_> axw: ultimately i'd like to mature the ToolsFixure and consolidate a lot of the older code
<axw> wallyworld_: sounds good. less reliance on version.Current would be nice
<wallyworld_> yep. it's all very messy in that area
<axw> wallyworld_: are you going to look at the prereq too, or leave it for fwereade?
<wallyworld_> looking now
<dimitern> morning all!
<wallyworld_> o/
<axw> hye dimitern
<axw> hey*
<wallyworld_> axw: i also want to get rid of the hard coded json metadata cause we have marshalling capability now also
<wallyworld_> oh, and don't forget to update the mp covering description of the 2nd branch
<axw> wallyworld_: hm yeah. we could just have an init() function that renders the images/tools objects
<axw> wallyworld_: will do
<wallyworld_> we can set up a slice of metadata structs and just marshal those
<wallyworld_> much easier than all the json
<axw> yeah, very finicky
<axw> miss a comma and it's all over
<wallyworld_> yep, happened to me today
<wallyworld_> had to run it through a parser to find the rrror
<voidspace> morning all
<mgz> morning!
<TheMue> morning
<jam> mgz: good morning
<axw> wallyworld_: I think you're going to have to merge changes for i386 fixes in provider/joyent
<wallyworld_> axw: yeah, figured as much when i saw your work :-)
<axw> wallyworld_: any chance  we could delay the standup for till quarter past?
<axw> -for
<wallyworld_> axw: no problems
<axw> thanks
<wallyworld_> fwereade: hiya, will you be free for a chat in say 45 minutes?
<fwereade> wallyworld_, sure
<wallyworld_> awesome, i'll ping you after our standup
<axw> wallyworld_ mgz ready when you are
<dimitern> so is it intentional that virtually all team names (except onyx) are blue stones?
<dimitern> :)
<dimitern> i didn't know how moonstone or tanzanite look like, but both are blue and the latter is almost exactly the same color as sapphire
<thumper> jam: yes, we need to find a better time...
<jam> dimitern: tanzanite is actually a bit more of a purple, and moonstone is white.
<jam> emerald is more green
<mgz> dimitern: we just won, everything is a shade of blue now
<dimitern> jam, yeah, I forgot about emerald :)
<dimitern> mgz, exactly ;)
<dimitern> mgz, welcome back btw
<mgz> thanks dimitern :)
<wallyworld_> fwereade: you ok for a chat now?
<fwereade> wallyworld_, yeah
<wallyworld_> https://plus.google.com/hangouts/_/g6w7cwvuro6wshqar324ytpg5ma?hl=en
<jam> fwereade: if you're done chatting with wallyworld, I wouldn't mind an ear to bounce a couple more API thoughts off of
<fwereade> jam, sure
<fwereade> jam, done chatting with mgz as well :)
<jam> fwereade: https://plus.google.com/hangouts/_/canonical.com/juju-sapphire
<jam> we have our standup in 10m anyway
<axw> mgz: I think I have the instancepoller test under control, will let you know later if I don't think I will get it fixed soon
<mgz> axw: ace
<perrito666> morning everyone
<jam> vladk: dimitern: TheMue: standup ?
<TheMue> jam: yep, lost the link :D
<perrito666> fwereade: hi, regarding your comment in https://codereview.appspot.com/92320043/ do you have any objections regarding merging this?
<jam> wallyworld_: have a good evening
<wallyworld_> jam: will do, sorry long day today, very tired
<jam> np
<fwereade> perrito666, I think it's fine to merge
<fwereade> perrito666, this feels like one of those cases where we really need some sort of docs though
<fwereade> perrito666, just to make the expected usage and situations and so on clear
<perrito666> fwereade: yup, we have doc duty too, but I did not want to propose a pull request for the docs before merging the code
<fwereade> perrito666, one question: does this fail properly when run against an env with a running state server that *isn't* the bootstrap node?
<fwereade> perrito666, I think we all agreed that docs come first
<fwereade> perrito666, they don;t need to be *good* but they do need to go to evilnick early so he can massage them into shape
<fwereade> perrito666, and so that we have some sort of idea what we're actually implementing, too ;)
<perrito666> fwereade: agreed, specially since this use case is not all that intuitive
<fwereade> perrito666, cool
 * perrito666 pull requests first
<perrito666> fwereade: mmm, good question about the node, it fails on non HA, Ill make a test run for that case just to be sure
<voidspace> perrito666: morning
<voidspace> axw: ping
<voidspace> if you're still around
<voidspace> natefinch: morning
<voidspace> natefinch: who are you interviewing, anyone interesting?
<jam> dimitern: did you want to talk about addressible containers with vladk ?
<natefinch> voidspace: Not particularly... seems an ok candidate
<voidspace> natefinch: cool
<voidspace> natefinch: I'm down a rabbit hole of rsyslog changes (again)
<natefinch> voidspace: ug
<voidspace> natefinch: whilst flowing host ports around I discovered one serious bug in our current implementation
<voidspace> natefinch: previously it was only port/cert that could change - so in the worker handler there's a shortcut that doesn't render/restart if those haven't changed
<axw> voidspace: I'm here atm, but may disappear at a moment's notice
<voidspace> natefinch: whereas now we can add a new state server without the port or cert changing
<voidspace> natefinch: that was easy to fix
<voidspace> axw: are you normally around until about this time?
<voidspace> axw: I can ping you earlier tomorrow
<axw> voidspace: no, not normally
<voidspace> ah...
<axw> voidspace: usually clock off about 1h40m ago
<voidspace> axw: ok, I'm usually on well before then
<voidspace> natefinch:  fwereade: FTR axw says that the rsyslog port and cert are in the environment "at the time the environment is bootstrapped"
<axw> ooh wait
<voidspace> natefinch: fwereade which means that we are fine to only watch APIHostPorts in the general case
<voidspace> axw: haha, ok...
<axw> voidspace: I may have brain farted
<voidspace> axw: would you rather continue this tomorrow - I have plenty of other stuff in the meantime
<natefinch> jam: FYI - I have to skip our meeting today to get the kids prepped early so I can be ready for the interview in a little over an hour.
<axw> voidspace: probably better, I don't want to keep feeding you lies
<axw> voidspace: I think the rsyslog-ca-cert is actually generated post-bootstrap
<voidspace> axw: hmm... ok
<axw> that doesn't need to be kept like that though
<axw> was just to consolidate bootstrap/upgrade logic
<voidspace> axw: you can point me to the code tomorrow
<voidspace> axw: so long as it is *before* the rsyslog worker is created we're ok, but that may not be the case
<axw> voidspace: it's not, it's created by the rsyslog worker
<axw> voidspace: see "ensureCertificates" in worker/rsyslog/worker.go
<voidspace> axw: ah, which also doesn't matter then
<voidspace> it sounds like we're good
<voidspace> axw: I'll spelunk that code a bit more
<axw> cool
<axw> fwiw I think we could extract that bit out to bootstrap time, and make rsyslog-ca-cert immutable in 1.20
<voidspace> ok
<voidspace> lunch
<perrito666> voidspace: you have the most random lunch pattern ever
<jam1> perrito666: I've got 10 minutes before my next meetingâ¦ Lunch!
<voidspace> perrito666: yup
<voidspace> perrito666: I *aim* for 1pm (now) - but often miss it by a couple of hours
<jam> natefinch: one thought, the test here: http://juju-ci.vapour.ws:8080/job/local-deploy/1309/console is running on a Precise machine. which *might* be why it would be seeing different results.
<voidspace> perrito666: although moving our standup to 3pm made that harder
<voidspace> perrito666: so this sort of time will probably be more normal for me now
<perrito666> voidspace: good luck then :)
<perrito666> fwereade: nice spot there, turns out that restore will restore if there are other state-machines as long as machine 0 is dead
<fwereade> perrito666, cool
<fwereade> perrito666, well, not cool
<fwereade> ykwim
<perrito666> hehe, yep
<fwereade> perrito666, is it that we're not storing the current set of state servers on the provider now?
<TheMue> jam: I have to step out for some minutes. there are some open questions in our doc, could you please take a quick look? tia
<perrito666> fwereade: I think it is the way restore checks if stateservers are up, as long as one is down it will consider that it is safe to restore
<jam> TheMue: I tried to respond to them, there are a few more "comments" in the dropdown box that seem to be related to stuff you've already deleted.
<jamespage> fwereade, fyi https://bugs.launchpad.net/juju-core/+bug/1314686 is blocking any SRU activity of 1.18.x into trusty right now
<_mup_> Bug #1314686: Please add support for utopic <packaging> <juju-core:Fix Released by wallyworld> <juju-core 1.18:Fix Committed by wallyworld> <juju-core (Ubuntu):Triaged> <https://launchpad.net/bugs/1314686>
<wallyworld_> jamespage: we can't help unless the extra mongo logs are attached as requested so we can see what is going wrong
<jamespage> wallyworld_, rbasak is just reproducing again to get that
<wallyworld_> great, thanks :-)
<wallyworld_> we may need to set up a utopic vm to test with
<jamespage> wallyworld_, sure - we have pubished cloud images on the daily stream for utopic now
<wallyworld_> the initial fix was for the core issue with juju which was lack of utopic knoledge
<wallyworld_> juju now knows about utopic but i guess mongo is unhappy
<jamespage> wallyworld_, maybe - I'm just upgrading a server to utopic to repro as well
<wallyworld_> ok. i was hoping teaching juju about utopic would be enough
<wallyworld_> a new bug may be pertinent as there are 2 separate issues to address
<TheMue> back again
<TheMue> jam: thx, will take a look
<voidspace> is there any way of building all test packages *as well*
<voidspace> I would like to find compile errors in the tests (I've changed some apis) and "go build ./..."
<voidspace> omits test packages
<voidspace> hmmm... maybe it has a flag
<perrito666> fwereade: you where right
<perrito666> <fwereade> perrito666, is it that we're not storing the current set of state servers on the provider now?
<fwereade> perrito666, cool
<perrito666> fwereade: not really :p
<perrito666> hehehe
<fwereade> perrito666, on this occasion, I men it's cool that my psychic debugging skills are still here
<fwereade> perrito666, and at least there should be a path to fixing that, right?
<perrito666> yep :)
<voidspace> natefinch: you still interviewing?
<natefinch> voidspace: sorry, no, got caught up in writing down my thoughts about it.  I'll be right there
<voidspace> natefinch: cool
<natefinch> wwitzel3: standup?
<mattyw> natefinch, perrito666 Hi guys, I've been working on other things today so I'll be taking a look at the ensureavailabilty command tomorrow, just wanted to let you know
<natefinch> mattyw: cool, no problem
<natefinch> mattyw: we're doing a standup right now, would you like to join?
<mattyw> natefinch, I certainly can do
<mattyw> natefinch, moonstone?
<natefinch> mattyw: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1
<natefinch> yep
<voidspace> natefinch: this is a diff against Wayne's branch
<voidspace> https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-api/+merge/220087
<voidspace> natefinch: this is before fixing the old tests that are now broken and adding new tests for the changed behaviour
<voidspace> natefinch: lines 339-343 of that diff is the removal of the obsolete check (without which the existing CL is broken)
<perrito666> uff, I just missed dimitern
<fwereade> I have https://codereview.appspot.com/12841044 and https://codereview.appspot.com/94540043/ up and would appreciate reviews
<fwereade> yes, that is a 9-month gap between the prereq and the followup :/
<mgz> fwereade: looking
<perrito666> fwereade: we love you anyway
<mgz> huh delete(someMap, keyNotInMap) doesn't have any error-y-ness
 * perrito666 gets scared to death when her wife is followed by a twitter botnet and her tablet, next to him, starts making sounds as if being possessed 
<mgz> her? is there something you're not telling us?
<perrito666> mgz: her tablet?
<perrito666> next to me?
<perrito666> its interesting what 150 notifications make to android
<mgz> "/me gets... when her wife is"... the her has the wife, clearly
<perrito666> heh
<voidspace> natefinch: this is the mp for just removing the "dangerous check" (plus a "now obsolete and failing" test)
<voidspace> https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-shortcut-removal/+merge/220105
<natefinch> voidspace: thanks
<voidspace> right, off to krav maga
<voidspace> back shortly
<natefinch> voidspace: kick some butt
<voidspace> natefinch: I'll talk to wwitzel3 about that branch
<natefinch> voidspace: cool
<voidspace> natefinch: he might as well merge it into his before landing
<voidspace> heh
<voidspace> we don't kick butts much
<voidspace> not an effective place to kick
<voidspace> groin or the back of the knee
<voidspace> or the chest if someone is coming towards you
<voidspace> c'ya later all
<natefinch> voidspace: and this is why I'm glad you're on my team and not someone else's :)
 * perrito666 is glad you are all 14k of his but, groin and back of knee
<voidspace> hehe, nice
<perrito666> fwereade:  doc/architectural-overview.txt line 118 , care to tell me the end of that tale? :)
<jcw4> dumb bzr question
<jcw4> if I'm in a cobzr branch and merge in juju-core
<jcw4> and commit
<jcw4> and then do go get -u ./...
<jcw4> should I expect to lose my locally committed changes irretrievably?
<natefinch> don't use go get when developing on a branch
<jcw4> yeah... :(
<natefinch> or rather: don't use go get when developing
<jcw4> I usually switch to master and do that to get updated dependencies
<natefinch> you shouldn't ever be able to lose your changes forever
<natefinch> it's version control, that's what it's for
<jcw4> natefinch: that's what I would expect
<jcw4> natefinch: but I can't find the revisions and don't know enough about bzr innards to know where to look
<natefinch> so, after doing a go get -please-screw-up-my-workspace .... hopefully a simple bzr revert would put you back in the right spot, but I'm not 100% sure
<jcw4> haha
<jcw4> sadly bzr status looked happy and I switched to master and back and that's when I noticed my missing revisions
<natefinch> did you push the changes or just commit locally?
<jcw4> just local
<jcw4> if I'd pushed them I'd just blow my local branch away
<jcw4> :(
<natefinch> right
<jcw4> If I'm in a branch and merge in juju-core... and juju-core has new dependencies... how should I retrieve them if not go get -u ./...
<jcw4> ?
<jcw4> manually for each import error?
<perrito666> jcw4: godeps
<natefinch> so, the proper way to update dependencies in juju is to go to the root directory of juju-core and do godeps -u dependencies.tsv, which will sync all external branches.  Occasionally that'll return an error like "abc1234 is not a valid revision on branch foo" which just means that you need to go do a bzr pull on that repo, because dependencies.tsv references a revision that exists on launchpad but not on your local machine
<jcw4> godeps is what gave me the error
<jcw4> because I didn't have the new errors package downloaded
<natefinch> righ
<natefinch> right
<jcw4> so; manual get of dependencies in that case...
<jcw4> sigh.  so much to learn
<natefinch> yeah, it's a failing of godeps
<jcw4> :)
<jcw4> tx natefinch perrito666
<perrito666> natefinch: jcw4 yep, godeps falls short in a few very common cases :(
<natefinch> there's two failings - one is repos it's never heard of before, like the errors packages.... that doesn't happen very often since we don't add dependencies very often.  The one about revisions not existing locally happens all the time, unfortunately.  We should really spend some time fixing it up so it just does the right thing all the time.
<natefinch> neither of those two problems is really that hard to fix, since the solution is blatantly obvious each time
<jcw4> If I didn't misunderstand it looks like internal Google teams use godeps too?
<natefinch> they may use godep .... which has a very similar name, but does very different things
<jcw4> natefinch: oh.  Interesting.  Maybe that's what I saw.
<natefinch> well, they don't use godep, they use something LIKE godep, which they aren't sharing ;)
<jcw4> teh suk
<jcw4> ;)
<natefinch> Yeah, Google doesn't open source enough of their stuff, unfortunately.
<jcw4> I imagine that that tool would be getting ticklishly close to their competitive advantage in managing vast complex code-base
<natefinch> probably. They do just have everything in one hugely massive repo.... so all third party code is also in the repo (which means it needs to have its import statements fixed up to point at google's repo and not github or whatever)
<jcw4> hmmm; the 'vendor' folder approach
<natefinch> exactly.  Honestly, that's the only way to be 100% sure your code will always build.  We've avoided that with juju because all our dependencies are really controlled by us, or people who work at the company.  So if Gustavo's mgo package suddenly breaks, we can go hound him about it ;)
<jcw4> dependencies in juju haven't become tedious enough for us to adopt that approach I suppose.  right
<jcw4> fun
<niemeyer> natefinch: No need to change imports for that.. a controlled GOPATH environment would work fine
<natefinch> niemeyer: yeah, that requires a little more futzing with the local environment, though
<jcw4> niemeyer: so... using a vendor folder + GOPATH that points to it?
<niemeyer> natefinch: Well, just using a GOPATH with content that is under your control rather than arbitrarily taken from the net
<niemeyer> s/natefinch/jcw4
<jcw4> niemeyer: I see.  Makes sense
<niemeyer> That's what godeps does, for example
<niemeyer> Erm godep
<natefinch> ahh, ok, I was mistaken about how godep works, I thought it was rewriting import paths.  I know camlistore has a tool they use to do that.  It's hard keeping all these tools straight :(
<bodie_> how exactly does it expect the dependency entry to be formatted?
<bodie_> e.g., launchpad.net/tomb      bzr     gustavo@niemeyer.net-20130531003818-70ikdgklbxopn8x4    17
<bodie_> I'm not sure what the third and fourth entries are
<bodie_> or where to look for that info, anyway
<jcw4> I'd hazard a guess at #3 (timestamp + hash?), but #4 defies even guessing :)
<bodie_> I think it's specified in CONTRIBUTING
<bodie_> yep
<bodie_> line 234
<bodie_> tab-separated
<jcw4> according to godeps source it looks like #3 and #4 are vcs specific
<jcw4> revid, revno
<bodie_> godeps -t $(go list launchpad.net/juju-core/...) > dependencies.tsv
<bodie_> :)
<jcw4> and a #5 an optional 'clean' flag
<jcw4> oops optional revno
<bodie_> hmmm
<bodie_> this doesn't want to work properly
<mattyw> natefinch, ping?
<natefinch> here is but in a meeting
<natefinch> -ish
<mattyw> natefinch, I've got my leankit account setup, was trying to find the ensureavailability more logging card on the board
<mattyw> natefinch, ah, I see it
<fwereade> perrito666, there's an orphaned line 20 or so further down
<fwereade> perrito666, fixed in followup (actually just removed)
<perrito666> tx
<menn0> morning ppl
<perrito666> menn0: morning :p
<waigani> morning all
<waigani> I'm getting this panic from gobot that I can't reproduce locally: http://pastebin.ubuntu.com/7490387
<waigani> Panic: cannot share a state between two dummy environs; old "only"; new "only" (PC=0x414311)
<perrito666> waigani: poisoned env?
<waigani> perrito666: what does that mean?
<perrito666> The env where that is being run is dirty?
<perrito666> waigani: you might want to go in there and clean the bot's env by hand
<waigani> perrito666: right, I've never done that.
<perrito666> waigani: neither did I
<waigani> i assume Its sshing in and rming go/pkg
<perrito666> waigani: yup
<waigani> but I might wait and check with wallyworld_ (I also don't know where to ssh into)
<wallyworld_> wot
<waigani> hey :)
<perrito666> wallyworld_: aha, lurking
<wallyworld_> coding :-)
<waigani> it's early for you right wallyworld_ ?
<wallyworld_> yeah
<wallyworld_> not toooo early
<waigani> well in that case ...
<perrito666> waigani: waigani has tarmac shouting him stuff that he can reproduce
<waigani> looks like I need to kick the bot, but I have not done it before
<perrito666> I meant wallyworld_
<waigani> *can't reproduce
<wallyworld_> what do you mean "ick the bot"
<wallyworld_> kick
<waigani> wallyworld_: Panic: cannot share a state between two dummy environs; old "only"; new "only" (PC=0x414311)
<wallyworld_> right, but what does "kick" mean
<waigani> do I need to ssh in a rm go/pkg?
<wallyworld_> not sure
<waigani> kick means ^
<wallyworld_> i'm landing a branch at the moment
<wallyworld_> we can see how that foes
<wallyworld_> goes
<waigani> cool, sounds like a plan
<wallyworld_> so far so good, stil running
<wallyworld_> after 10 minutes
<wallyworld_> my guess is it's a bug in your code :-)
<waigani> wallyworld_: okay I'll look harder
<wallyworld_> my branch will befinished soon so we can see for sure then
<perrito666> heh dont rub your code correctness on the man
<wallyworld_> waigani: my branch just landed so bot is ok
<waigani> hmph
<waigani> wallyworld_ perrito666: I just reproduced it on my machine! So yep my fault. Sorry for the red herring. Weird that I couldn't reproduce it the first time.
<wallyworld_> no problem
<wallyworld_> arosales: g'day, you around?
<arosales> wallyworld_: hello
<wallyworld_> hi, i was hoping you could help. i have some pull requests on github for the joyent libraries we need landed for core. the've been there a week and no one has looked at them. i tried emailing danile also. is there any contact in joyent we can prod?
<arosales> wallyworld_: for sure can you point me to the pull requests I can send a mail and cc you.
 * arosales looking around https://github.com/joyent . . .
<wallyworld_> arosales: sure, i'll email you - there are 4 of them
<arosales> wallyworld_: thanks, I'll look for your mail.
<wallyworld_> arosales: thanks, email sent
<arosales> wallyworld_: thanks, we should hopefully have some folks taking a looks soon. Thanks for finding those fixes and perf. improvments.
<wallyworld_> no problem
<wallyworld_> thanks for helping
<arosales> wallyworld_: anytime.
 * wallyworld_ bbiab
 * arosales grabs some dinner
#juju-dev 2014-05-20
<sinzui> guess who types rm -r in his home directory
<sinzui> axw, I have no notes, but I recall I wanted to ask how you did a full i386 test run
<sinzui> axw I saw timeouts in the CI runs. I am sure you fixed the tools issue. I may need to procure bare metal or try running the tests in an lxc in a m1.large amd64
<wallyworld_> sinzui: afaik, we hard coded the arch in arch.go to i386
<wallyworld_> so it was faked
<sinzui> wallyworld_, okay
<sinzui> wallyworld_, is there an env setting to extend the timeouts in the test suite
<axw> sinzui: one time I actually ran with i386 on canonistack
<axw> sinzui: after all my changes I only ran locally with arch faked to i386
<wallyworld_> sinzui: you use command line arg to test
<sinzui> axw okay that works I have a sleve there
<wallyworld_> test.timeout of something
<sinzui> thank you wallyworld_
<wallyworld_> -timeout is the raw go test arg
<sinzui> I think canonistack my be viable, even though I have tried to not use it
<wallyworld_> i think you need test.timeout if used with gocheck, can't recall exactly without trying
<sinzui> wallyworld_, thank you
<sinzui> anyway I need to rebuild my computer. I don't think I will accomplish much tomorrow.
<wallyworld_> sinzui: np. here's the syntax:   "go test -gocheck.v -test.timeout=xxxxms"
<wallyworld_> if you use gocheck
 * sinzui pastebin's that
<wallyworld_> sinzui: did we need to schedule a chat about jenkins lander for core?
<sinzui> wallyworld_, not just yet. I don't have time to start since I still need to get ppc tested every rev
<wallyworld_> ok, np
<sinzui> and the script that would build ppc64 packages just emptied my home directory
<wallyworld_> sinzui: if you like, you can give us epermission to set up a job in your jenkins and we can experiment
<wallyworld_> oh no!
<sinzui> At least I know the script wont fill up the stilson disk space again.
<wallyworld_> did you have backups?
<sinzui> Looks like I have one from 4 days ago
<wallyworld_> better than nothing, but still :-( :-(
<wallyworld_> sorry to hear that
<sinzui> I don't backup my work or hacking dirs. I need to rebuild my repos
<wallyworld_> sinzui: would letting giving us permission to create a new jenkins job on your instance be viable? we could then experiment and get the lander working
<sinzui> wallyworld_, sure
<wallyworld_> not sure how far we'll get
<wallyworld_> but still, having more potential resources to help would be positive
<wallyworld_> not that i'm in *any* hurry to leav lp
<wallyworld_> git makes me sad the more i learn about it
<wallyworld_> br is so much nicer
<wallyworld_> bzr
<wallyworld_> axw: morning, i didn't hear back from martin about how he got on looking at the TestEnsureAdminUser failure and he also hasn't assigned himself to the card. so in terms of stuff todo, there's that plus figuring out what's going on with juju on utopic. did you have any preference?
<axw> wallyworld_: I can take a look at TestEnsureAdminUser
<wallyworld_> ok, ta
<wallyworld_> that test has been failing a lot
<axw> wallyworld_: how'd you go with the PRs to joyent?
<wallyworld_> axw: we now have a committment that they will look at them in the next day or so
<axw> cool
<wallyworld_> so, i'm budgetting for eow by the time it goes around
<davecheney> tiny change https://codereview.appspot.com/100580045/
<perrito666> davecheney: lgtm, if the readme file is in that directory :)
<menn0> review please: https://codereview.appspot.com/92520043/  (new "user" supercommand)
<jimmiebtlr> Is it best to always post here when a new codereview is up?
<jimmiebtlr> A proposal for a bug fix
<davecheney> jimmiebtlr: seems to work well to get people to look at it if it is urgent
<davecheney> any members of juju-core will get an email that the code review is reday
<davecheney> actually, they'll get a lot of mail
<jimmiebtlr> Nothing urgent
<jimmiebtlr> low priority bug, just getting a feel for committing fixes
<davecheney> jimmiebtlr: i haven't seen anyting coming through email
<davecheney> do you have a link to the code review ?
<jimmiebtlr> https://codereview.appspot.com/98410043/
<davecheney> jimmiebtlr: toptip:
<davecheney> lbox propose -bug=1313793 to link the review to the bug
<davecheney> you can do this now
<davecheney> and it will do the linkage
<jimmiebtlr> that returned an error "User does not have sufficient permission to edit the bug task milestone"
<davecheney> jam: wallyworld_ can you fix this ?
<wallyworld_> hmmm, maybe. but we don't want to open permissions to change any milestone
<wallyworld_> might be that lbox needs looking at to see what it is trying to do
<wallyworld_> i wonder if it can be linked via the lp gui
<jimmiebtlr> Are you referring to the codereview being linked to the bug, or the branch being linked to the bug?
<wallyworld_> branch -> bug
<wallyworld_> the code review will then pick up the link
<jimmiebtlr> I linked the branch and the bug via the lp gui several days ago
<jimmiebtlr> or related it anyway
<wallyworld_> hmm, ok. looks like it all went through ok in any case
<wallyworld_> i suspect an issue with the lbox code trying to do something unexpected/unneeded but can't be sure without digging into it
<jimmiebtlr> thanks for the help, I'm off
<davecheney> protip: if you want to do a proposal, but aren't sure what the total diff will look like
<davecheney> you can use the lbox -wip flag to get a proposal and not mail it
<jcw4> yay for protip
<jcw4> what about bzr missing?
<jcw4> if I do an lbox propose, and then get feedback and make changes, should I do another lbox propose to propose the updated changes?
<jcw4> fwereade, mgz ? ^^
<davecheney> Technically, this is a 503 error and has been caused by our database being temporarily offline.
<davecheney> sad trombone
<davecheney> jcw4: just do lbox propose
<davecheney> again
<davecheney> jcw4: you are always proposing a branch, not a patch
<davecheney> so propose will always roll up your commits
<davecheney> so
<davecheney> bzr commit -m '...';
<davecheney> until you are happy
<davecheney> then lbox propose to ask people to take another look
<jcw4> davecheney: cool, tx!
<jcw4> mgz, fwereade  updated the MR: https://codereview.appspot.com/98260043/
 * jcw4 is off to bed... early morning meeting at 1400 UTC
<jcw4> :)
<voidspace> morning all
<fwereade> voidspace, heyhey
<TheMue> morning from a sunny veranda too ;)
<voidspace> we just had the most incredible downpour of rain and thunder for about half an hour
<voidspace> and now we're back to sun
<fwereade> we even had some rain here yesterday
<jam> morning voidspace
<jam> fwereade: https://codereview.appspot.com/93500044 is changing Resource to allowed named resources, and changing Pinger to use that functionality.
 * fwereade bbiab
<jam> Its independent of my other API changes.
<fwereade> jam, sorry, I'll take a look when I get back
<jam> fwereade: np, no rush
<jam> dimitern: you're welcome to look at ^^ as well, it is part of my changing of Facades to be versioned.
<jam> not directly related, but a necessary precursor to unifying how Facades are created.
<dimitern> jam, will look in a bit, thanks
<voidspace> dimitern: morning
<voidspace> jam: morning to you too
<dimitern> hey voidspace!
<voidspace> dimitern: how's life?
<dimitern> voidspace, not bad at all :) i wish the weather was so good
<dimitern> voidspace, how was the grand canyon?
<voidspace> dimitern: the grand canyon was spectacular, really amazing
<voidspace> dimitern: I posted my pictures of our shooting expedition and the grand canyon on flickr
<dimitern> voidspace, nice, I'll check them out
<voidspace> dimitern: https://www.flickr.com/photos/mfoord/sets/
<voidspace> dimitern: I didn't get a picture of us both together at the shooting, shame :-/
<dimitern> voidspace, I still haven't got to posting some pictures yet
<voidspace> dimitern: it was awesome though, one of the highlights of the trip for me
<dimitern> voidspace, yeah, but the photos otherwise kick ass ;)
<voidspace> dimitern: hehe, yep - and the videos
<dimitern> voidspace, indeed
<dimitern> voidspace, next time - shotguns ;)
<voidspace> dimitern: hehe, fair enough :-)
<voidspace> I've shot a shotgun before, clay pigeon shooting (once). I was getting the hang of it too.
<voidspace> all the guns we shot were fun.
<dimitern> yep, unexpectedly for me, it *was* fun
<perrito666> morning
<voidspace> perrito666: morning
<jam> dimitern: fwereade: A followup which uses a Named StringResource to register "dataDir" so that Client's Facade constructor looks like all the other Facades: https://codereview.appspot.com/99410043
<dimitern> jam, looking
<jam> dimitern: did you look at the earlier patch? I didn't see a comment from you
<dimitern> jam, no, sorry got caught up deep in goamz stuff
<jam> np, just figured you might want to look at that first
<dimitern> jam, LGTM, i'm looking at the prereq now
<TheMue> jam: Iâm mostly happy with the document now, so I would like you to take another look. Iâll start with the table now.
<jam> TheMue: do you have specific questions for me, or you just want me to audit the whole doc?
<mattyw> morning folks - does this look odd or normal: https://bugs.launchpad.net/juju-core/+bug/1321212
<_mup_> Bug #1321212: Odd behaviour when using ensure-availability <juju-core:New> <https://launchpad.net/bugs/1321212>
<TheMue> jam: a quick audit of the whole doc. I think everything is now clean and the open questions have been discussed yesterday.
<jam> mattyw: try running "juju bootstrap" and then "juju status" and then a loop of "juju status". You can check what the first line in ~/.juju/environments/$ENV.jenv says are the API servers.
<dimitern> jam, the prereq LGTM as well
<jam> There *was* a bug that it was getting all the possible API server endpoints listed (and one of those is MachineLocal's 127.0.0.1) which is obviously not an address we want to store in clients
<jam> I would *guess* that this isn't because of ensure-availability, but because of First Login issues
<jam> mattyw: you could also try "juju status --debug" to see what addresses it is trying to connect to
<jam> the really weird part is that 127.0.0.1:37017 looks like it is fallback code (we should never usually try to connect to 37017 which is the DB port)
<mattyw> jam, ok thanks - I "sort of" remember that stuff, I'll take another look
<voidspace> brb, grabbing coffee
<voidspace> jam: doesn't look like natefinch is around yet
<jam> voidspace: yeah, I'm a bit surprised he picked that early for him, but when he comes around we'll chat
<voidspace> jam: cool :-)
<voidspace> question about our watcher architecture
<voidspace> suppose a client has a watcher on something (say environ-config changes for example)
<voidspace> and three changes occur before the client makes a "Next" request
<voidspace> will the subsequent two "Next" requests return *immediately*, or will the next "Next" wait for *another* change?
<voidspace> does the apiserver maintain a queue of changes for watchers?
<jam> voidspace: they get collapsed, I believe the Watcher collapse interval is 5s
<jam> so if there are 3 changes within 5s, then you only get 1 Next() event
<voidspace> jam: ok
<voidspace> what if there are 3 events each 5 seconds apart, and *then* the client makes a Next call - are they all collapsed too? Or is the collapse interval a *maximum* of five seconds.
<jam> voidspace: I'm pretty sure we don't maintain a "you have X events ready" we just have "something has changed" so a single bit
<voidspace> right - so they're collapsed all the time, but also "close" events are collapsed automatically anyway
<voidspace> so the watchers have a minimum granularity of 5 seconds
<voidspace> thanks
<mattyw> jam, have you got a moment?
<jam> mattyw: what's up?
<mattyw> jam, tasdomas and I are working on adding output to ensure-availability so it prints what it's doing
<jam> sure
<mattyw> at the moment we're changing the ensureavailbilityintent struct (addmachine.go:616) to be exported, but we'd have to make all of the values in it exported as well
<mattyw> but that seems a bit odd since you don't really want to be able to change any of the values within ensureavailbilityintent one at a time
<mattyw> nate suggested we could ask you these kind of questions in his absense - just wondered if you had an opinion?
<jam> mattyw: looking into it
<jam> mattyw: so from what I see, there is just 4 lists of Machines that we are operating on, but we don't want to actually have the Machine objects themselves in the API
<jam> rather, we should probably just expose lists of machine-tags
<jam> mattyw: so I'm thinking we shouldn't just make that type exposed, but create a new params.EnsureAvailabilityResult (Intent, Actions, ... ?)
<mattyw> jam, the apiserver was going to convert this type into a Results type anyway
<jam> mattyw: so... exposing the attributes is fine, they are all intended as Readonly anyway, and actually changing them doesn't have any actual effect.
<jam> however, we have precedent for state could just return the polished up params.EnsureAvailabilityResults itself
<jam> and then we don't have to expose raw Machine entities to apiserver/client code.
<jam> we *can* do it with that layering
<jam> certainly the AddMachine api works in real Machine objects and then wraps the data into returning the Id(), though *really* *really* AddMachine should be returning Tags not Ids
<jam> the API talks in terms of Tags
<jam> the internals talk in terms of Ids
<jam> time for an AddMachinesV3 :)
<mattyw> jam, I like the idea of the params.params.EnsureAvailabilityResults
<dimitern> jam, standup?
<jam> dimitern: yeah, brt
<jam> voidspace: looks like natefinch didn't make it online before his wife's dentist appointment
<jam> I have to work on my son's homework in a bit, but I can chat with you after that.
<TheMue> lunchtime, ended exactly when my wife called ;)
<voidspace> jam: ok
<voidspace> jam: if you're not around we can postpone appropriately
<voidspace> jam: don't hang around on my behalf
<perrito666> fwereade: hey, I am implementing what we spoke yesteday, about ServerInstances, the get/set methods you suggested should operate against Storage right?
<fwereade> perrito666, not necessarily
<fwereade> perrito666, we want them to be Environ method
<fwereade> perrito666, but we also need a common implementation that most environs can use to implement them
<fwereade> perrito666, and that would take a storage and keep working the same as before
<fwereade> perrito666, the important thing really is that those Environ interface methods *not* metion storage
<fwereade> perrito666, am I sounding sane?
<perrito666> fwereade: most of the time
<fwereade> perrito666, how about in the last 2 minutes?
<perrito666> sane enough :) tx
<fwereade> perrito666, cool :)
<perrito666> what still bothers me a bit is that everywhere seems to be implying that the current storage Should already have that info
 * perrito666 looks the rain outside and smiles on his non commuting job
<fwereade> perrito666, that is still true
<fwereade> perrito666, which is why almost everything should use that method
<wwitzel3> hello
<fwereade> perrito666, pretty sure that manual and local will be different to some degree, though
<perrito666> fwereade: expected
<fwereade> perrito666, and more to the point, we will soon have to deal with some cloud that *doesn't* have storage
<perrito666> fwereade: I see
<fwereade> perrito666, (also, wallyworld_ is moving storage into gridfs soon as well -- so we'll be dropping storage from Environ anyway)
<perrito666> ok then, I should assume storage is not there for my new methods and then do everything use those
<voidspace> lunch
<fwereade> perrito666, yeah, all the implementations will use storage, but the interface shouldn't mention it
 * fwereade should have known the "JoinedRelations" API method was a bad idea, because it should now be "RelationsInScope"
 * fwereade kicks himself a couple of times
 * fwereade can't wait for the versioned API
<ahasenack> hi, can someone see what's wrong with the simplestreams in this bug, it was produced by juju's 1.19.2 metadata command, and can't bootstrap
<ahasenack> https://bugs.launchpad.net/juju-core/+bug/1321009
<_mup_> Bug #1321009: juju-metadata doesn't produce content that bootstrap can use <landscape> <juju-core:New> <https://launchpad.net/bugs/1321009>
<wallyworld_> sinzui: hi, the failing trusy tests can't run bzr, i have no idea why off hand, bzr add results in an error
<wallyworld_> ... Panic: command failed: bzr add
<wallyworld_>  (PC=0x4143E6)
<wallyworld_> i haven't seen that before
<wallyworld_> looks like it's external to juju?
<wallyworld_> the PC=xxxx is from the br add output
<wallyworld_> bzr
<wallyworld_> ahasenack: i commented on your bug - i think you just need to delete your old jenv file and/or first destroy your existing environment
<sinzui> wallyworld_, :( I see this hack is still in place when CI prepares
<sinzui> bzr whoami 'J. Random Hacker <jrandom@example.org>'
<wallyworld_> that jrandom doesn't look like it comes from juju
<ahasenack> wallyworld_: ok, will try, thanks
<dimitern> niemeyer, hey
<niemeyer> dimitern: Hey hey
<dimitern> niemeyer, I have a small review for you if you can spare 10m - https://codereview.appspot.com/98430044/ (for goamz)
<dimitern> jam, vladk, fwereade, ^^ another review appreciated
<ahasenack> wallyworld_: same problem, let me paste
<niemeyer> dimitern: Not exactly a *small* review, but will surely look :)
<dimitern> niemeyer, :) thanks
<ahasenack> wallyworld_: http://pastebin.ubuntu.com/7492988/
<niemeyer> dimitern: At a glance, it's easy to tell the code is under-covered by tests
<niemeyer> dimitern: Maybe that's okay, given that the fake server itself is meant to be the test
<niemeyer> dimitern: (how far do we test the test)
<dimitern> niemeyer, I've included the most important parts for juju in the tests
<niemeyer> dimitern: Why do we have the rest of the logic in the fake server if it's not relevant?
<dimitern> niemeyer, which logic?
<niemeyer> dimitern: You said you've included the most important parts for juju in the tests
<dimitern> niemeyer, yes - that is the assumption in juju that all instances have 1 nic after launching
<niemeyer> dimitern: Not sure about how that reflects into the above points
<perrito666> voidspace: wwitzel3 could anyone of you jump into the moonstone channel? I am trying to set up my hangout
<dimitern> niemeyer, well, i've always assumed the test server should (when possible) mimic ec2 as close as possible
<voidspace> perrito666: ok
<dimitern> niemeyer, what do you think needs more test coverage?
<niemeyer> dimitern: I'm mainly wondering about what you were saying.. I said there's certainly a good portion of untested logic in the server, given how extensive the code is and how slim the tests are
<niemeyer> dimitern: then you said something along the lines of "what matters for juju is tested"
<niemeyer> dimitern: and my question was "why do we have anything that does not matter for juju in there"?
<dimitern> niemeyer, ok, let's backtrack a bit, sorry
<dimitern> niemeyer, imho it doesn't make sense to implement "create a default network interface when none given in RunInstances" and not do "create/use network interfaces if specified in RunInstances"
<mbruzek> Hi guys I am seeing an SSLError from deployer when I try to run some amulet tests.  http://pastebin.ubuntu.com/7493011/
<hazmat> mbruzek, update your version of jujuclient
<dimitern> niemeyer, the code it almost the same in both cases
<dimitern> niemeyer, and the tests indeed cover those 2 cases - the implementation is actually longer than the tests need to be I think
<mbruzek> hazmat, python-jujuclient is already the newest version.
<hazmat> mbruzek, are you pulling things from pypi or from packages?
<dimitern> niemeyer, there are very few changes in behavior not really tested - like "ensure you can't launch more than 1 instance when you specify an existing NIC ID"
<hazmat> mbruzek, if you pull from packages it should be fine, but if you start mixing in some from both there's a python-websocket default behavior change in ssl cert verification that needs a corresponding version change.
<dimitern> niemeyer, or a test that creates 2 NICs
<hazmat> mbruzek, what version?
<hazmat> mbruzek, pip -U python-jujuclient
<hazmat> to upgrade
<mbruzek> hazmat no such option: -U
<dimitern> niemeyer, on the same instance, but it is supported, I can add a second RunNetworkInterface item to RunInstances and it will do/test that
<mbruzek> hazmat it looks like 0.17.5 is the version I have
<mbruzek> $ pip freeze | grep jujuclient
<mbruzek> jujuclient==0.17.5
<niemeyer> dimitern: I somewhat doubt that, given the sheer volume of logic in the code which is not represented in the tests
<hazmat> mbruzek, can we move this to #eco
<niemeyer> dimitern: But as I said, I'm won't argue much.. this is test code being tested
<niemeyer> I won't
<dimitern> niemeyer, yes, it is test code
<dimitern> niemeyer, but I'll be happy to add more tests if you have something particular in mind
<niemeyer> dimitern: I have everything there in mind :)
<dimitern> niemeyer, :)
<niemeyer> dimitern: Lots of these lines are unchecked
<niemeyer> dimitern: I'm not going to go over the trouble of telling you every single one
<niemeyer> dimitern: If you really care, you might go over the diff and check yourself
<dimitern> niemeyer, I'll look some more into it, thanks
<jcw4> thanks fwereade
<bodie_> fwereade, wasn't sure if there was any more you wanted to add, but everyone dropped off :)
<jcw4> wrt. Txn loop: you had sent me some example code which I subsequently misplaced....
<fwereade> jcw4, bodie_: I'd be keen to talk about txn loops -- mgz, not sure if you've delved deeply there before?
<fwereade> jcw4, heh, no worries, I'll look that up again -- or, tell you what, I'll do a quick comment on your CL if you send me the link again
<jcw4> fwereade: +1
<bodie_> my only knowledge is that it's a technique essentially for pipelining State's database queries and upserts through a single port of authority
<jcw4> fwereade: https://codereview.appspot.com/98260043/
<fwereade> jcw4, bodie_, mgz: a live chat might be worthwhile then, I'll find the comment I did for perrito666
<perrito666> voidspace: apparently https://bugs.launchpad.net/ubuntu/+source/linux/+bug/950490
<_mup_> Bug #950490: External microphone does not work on Zenbook UX31 <amd64> <apport-bug> <precise> <verification-done-precise> <linux (Ubuntu):Fix Released by diwic> <linux (Ubuntu Precise):Fix Released> <https://launchpad.net/bugs/950490>
<voidspace> perrito666: ah, so it maybe Ubuntu's fault
<perrito666> alsa, actually
<dimitern> fwereade, do you have some time for a goamz review? https://codereview.appspot.com/98430044/
<voidspace> perrito666: I have a similar issue - when you have any alternative audio input source the "line in" is no longer available to select
<dannf> how can i specify a constraint in a bundle?
<dimitern> niemeyer, In any case, I'd appreciate your review
<niemeyer> dimitern: Have been doing it since
<niemeyer> dimitern: The "10 minutes" :)
<dimitern> niemeyer, thanks! :)
<niemeyer> dimitern: https://codereview.appspot.com/98430044/
<dimitern> niemeyer, cheers!
<niemeyer> dimitern: np, please let me know how you feel about these
<dimitern> niemeyer, will do
<bac> marcoceppi: super trivial charm-tools review at https://codereview.appspot.com/98390047 when you have time
<marcoceppi> bac: thanks!
<jam> fwereade: yay, Client and Pinger are now no longer special in "api-use-register-standard-facade" so now everything is *just* another Facade with a similar factory.
 * fwereade cheers
<dimitern> niemeyer, updated https://codereview.appspot.com/98430044/ with most of your suggestions, and added some replies
 * dimitern reached EOD
<tasdomas> hi
<tasdomas> I am getting a test failure in cmd/juju/machine_test.go:253
<tasdomas> (on trunk) : http://paste.ubuntu.com/7493538/
<jcw4> fwereade: 6pm too late to discuss txn loop?
<fwereade> jcw4, that's fine
<fwereade> jcw4, was that link remotely relevant, or missing too much context?
<jcw4> fwereade: I reviewed and looks interesting.. one question
<jcw4> fwereade: should I build the ops in the loop?
<jcw4> fwereade: I assume so
<jcw4> fwereade: generate the Id outside the loop but everything else inside?
<fwereade> jcw4, yeah, exactly
<jcw4> fwereade: and is there a specific reason for 3 iterations?
<fwereade> jcw4, it may not *always* be necessary to do so, but it's generally the right thing to do
<jcw4> fwereade: k
<fwereade> jcw4, none whatsoever
<jcw4> fwereade: ok... Shall I start with 3?
<fwereade> jcw4, some use 5; some use fewer because we can guarantee that >2 implies something wrong
<fwereade> jcw4, sgtm
<jcw4> fwereade: makes sense
<fwereade> jcw4, at some point I'll probably get round to a generalised txn build/run/backoff-retry thing
<jcw4> fwereade: ok.  also, if the txn completes successfully we just break out of the loop right?
<fwereade> jcw4, better yet, if err != txn.ErrAborted { return err }
<fwereade> jcw4, that covers both success and surprising failures like dropped connection
<jcw4> fwereade I see.  so on success err will be nil therefore we return nil error
<fwereade> jcw4, yeah
<jcw4> fwereade: and if its anything else but ErrAborted we return the suprising error
<jcw4> fwereade: and if the failure is ErrAborted we just try again
<jcw4> k
<fwereade> jcw4, yeah, just try to build a fresh txn against whatever state needs to be updated
<fwereade> jcw4, in your case, check the unit's life again
<jcw4> fwereade: until we finish the loop, and if we get there we return ErrAborted?
<jcw4> fwereade: ok
<fwereade> jcw4, fall out of the loop and return ErrExcessiveContention
<jcw4> fwereade: perfect
<fwereade> jcw4, although what that *really* means in every case I have yet seen is "some programmer ballsed things up"
<fwereade> jcw4, one detail
<jcw4> fwereade: hehe
<fwereade> jcw4, you'll want to clone the unit rather than Refresh() it in the loop
<fwereade> jcw4, we prefer not to update fields whose update isn't implies by the op you're running
<jcw4> fwereade: hmm.  Okay
<fwereade> jcw4, ie you should be able to call a bunch of methods in a row and not have to worry about in-memory state changing underneath you
<fwereade> jcw4, (although if you SetFoo you do expect foo to be set)
<jcw4> fwereade: I think I get it.  I should be able to code this part up fairly quickly... I forget where the example of how to test txn loop stuff was though
<bodie_> fwereade, jcw4, I'm worrying about the technique I'm using to validate the actions.yaml as valid json-schema
<bodie_> right now, our actions.yaml is relatively simple, as specified in the MR
<fwereade> jcw4, the funcs are defined in export_test.go -- SetTransactionHooks et al
<jcw4> fwereade: right. tx!
<bodie_> it appears to parse as valid json-schema, but i'm looking at the json-schema spec right now and, among other things it requires a root level key of $schema
<bodie_> which isn't present in my actions.yaml
<fwereade> jcw4, searching for Hooks in the state package should get yu plenty of examples
<jcw4> fwereade: +1
<fwereade> bodie_, hmm, can you get it to fail at all? :)
<bodie_> it could be that it's returning an empty json-schema, and i'm not checking for that.  i guess that's the point i need to dig down to
<bodie_> or, perhaps simply loading a k/v map as a jsonschemadoc isn't sufficient to validate it
<fwereade> bodie_, the examples on json-schema.org don't necessarily include that
<bodie_> http://json-schema.org/latest/json-schema-core.html#anchor23
<bodie_> good point
<bodie_> in that case, I don't understand the use of "MUST" in this spec ;)
<fwereade> bodie_, "MUST if you feel like it ehh who cares"
<bodie_> MUST be optional
<bodie_> :/
<bodie_> AH
<bodie_> ah* I misread it
<bodie_> IF it exists, it must be at the root; it is "RECOMMENDED" that schema authors include this keyword
<bodie_> need to beef my reading comprehension.  Go is making me lazy....
<fwereade> bodie_, ah cool
<bodie_> order matters in short-circuit evaluation of English clauses...
<bodie_> I guess I'm having a hard time thinking of valid YAML that isn't also valid JSON-Schema
<fwereade> bodie_, stick a $schema key in *not* at the root :)
<bodie_> good one
<fwereade> tasdomas, yeah, that's clearly a real problem, good that you can repro
 * fwereade beholds the horror that is cmd/jujud and sighs heavily
<jcw4> fwereade :)
<tasdomas> fwereade, if there's anything I can do to help resolve this, let me know - I'd be glad to help
<jcw4> fwereade: *how* do I clone a unit?
<tasdomas> also, lbox propose gives me this error: error: Failed to update merge proposal log: Server returned 401 and body: (<BranchMergeProposal at 0x2af0e8f83850>, 'description', 'launchpad.Edit') - no idea, what's it about, though.
<jcw4> fwereade:  st.Unit(id) /??
<fwereade> jcw4, that's probably the easiest way, yeah
<jcw4> fwereade: tx
<fwereade> tasdomas, sorry, that bit doesn't ring a bell
<jcw4> tasdomas: isn't 401 forbidden?
<jcw4> tasdomas: maybe some auth issue?
<fwereade> tasdomas, I can explain what the problem is but I kinda want to check with wallyworld_ to see if there's some reason it's like that
<jcw4> tasdomas: Unauthorized (not forbidden)
<mgz> tasdomas: mat be worth deleting your launchpad login cookie bits and going through the auth in browser bit again
<tasdomas> mgz, jcw4 - thanks for the advice, solved the issue (had not granted lbox the proper permission level)
<mgz> tasdomas: rm ~/.lpad_oauth
<mgz> tasdomas: ah yes, that one... selecting anything other than "change everything" is a bear trap
<mgz> these days launchpad lets the app say what access it wants, lpad probably need updating
<jcw4> tasdomas: cool
<fwereade> tasdomas, you can "fix" it by moving the setupContainerSupport call up just after the `runner := newRunner(...` line
<mgz> I say these days, 2010... <http://jameswestby.net/weblog/tech/16-Improving-the-usability-of-launchpadlib-using-code.html>
<fwereade> tasdomas, I would probably LGTM that if you proposed that alone
<tasdomas> fwereade, great - will do
<fwereade> tasdomas, but... that whole method is evidently all screwed up
<fwereade> tasdomas, if I suddenly figure out why that's a terrible idea, I apologise
<tasdomas> fwereade, no problem - at least you've pointed my in some direction, I'll go take a look at it
 * perrito666 starts doing ddd before going crazy
 * fwereade approves
<voidspace> yay, they're retiring canonicaladmin!!!
<voidspace> awesome
<mgz> voidspace: not quite, does expenses still
<voidspace> mgz: ah, well not too bad I guess
<voidspace> which reminds me, I haven't done my expenses for vegas yet
<mattyw> voidspace, tasdomas and I were have that exact conversation just now
<voidspace> heh
<voidspace> EOD folks
<voidspace> g'night
<wwitzel3> voidspace: see ya
<perrito666> fwereade: I am trying to write a watcher for instance id changes but I am not sure what to watch
<perrito666> brb
<fwereade> perrito666, I think the existing API-addresses watcher should be sufficient to inform you of any changes to the set of state servers; then you can just look up the instances on the state server machines, I think
<fwereade> perrito666, btw, you're writing a new API, so please don't just use the same name for the watch method -- you can use the same state method to start the watch, but make the API name reflect its purpose not its implementation
<jcw4> fwereade: what kind of action would disrupt a transaction that's adding an action to the state.actions collection?
<fwereade> jcw4, I can't think of anything; you should probably just check for unit dead
<fwereade> jcw4, you've already got a unique id
<jcw4> fwereade: yep, but I want to write a failing test first
<fwereade> jcw4, so it should be impossible for the DocMissing to fire
<jcw4> so something that would perturb the transaction :)
<fwereade> jcw4, ok, so set a txn hook that makes the unit Dead in its Before
<jcw4> fwereade: doh~!  tx!!
<fwereade> jcw4, you'll want to call preventUnitDestroyRemove() before you Destroy, otherwise it'll fast-forward straight through to removed (which you'll also want to test, but you can't make it Dead unless you do that, Destroy, EnsureDead
<jcw4> fwereade: ah.. ok.  I already did the Destroy / EnsureDead for another test, but didn't call preventUnitDestroyRemove() first...
<jcw4> test seemed to work as I expected though... prolly just lucky
<fwereade> jcw4, I'm sure it *was* a valid test, just not the one you thought you were writing ;)
<jcw4> lol
<perrito666> fwereade: I was going for WatchAPIInstances not sure if that is what you meant with the naming
<jcastro> http://askubuntu.com/questions/448444/juju-security-model-issues
<jcastro> Can someone from core check out that question? I've tacked a bonus 200 point bounty!
<sinzui> perrito666, the ha-backup-restore test is queued. I can enable it when you land.
<natefinch> jcastro: I can answer it, but they might not like the answer :)
<jcastro> that's fine
<jcastro> natefinch, towards the end of the answer encourage him to post to the list if he wants to follow up
<natefinch> ahh good idea
<jcastro> It'd be interesting to get real details on what he's planning on doing, etc.
<fwereade> perrito666, let's say WatchStateServerInstances?
<natefinch> jcastro: responded, btw
<jcw4> using mgo txn.Assert... how can I see which Assertion failed?
<jcw4> mgo seems to only return ErrAbort if something fails
<natefinch> sorry, no idea
<fwereade> jcw4, you can't
<fwereade> jcw4, that's one of the big reasons for the loop structure
<fwereade> jcw4, go back to the top, start building txn again, checking for sanity against latest known state
<fwereade> jcw4, assuming your in-memory checks map correctly to your assertions, you'll fail out appropriately
<jcw4> Boom
<jcw4> I see now
<jcw4> :)
<fwereade> jcw4, sweet
<jcw4> with enough repetition even granite will crack
<fwereade> jcw4, not at all, you have taken less instruction than most -- it's really not a familiar way of working for almost anyone
<mbruzek> Hello juju-devs, I am running juju local, and all my systems are in "pending" state.  I remember a similar problem when I had an incomplete image.  Can someone refresh my memory where juju stores the local images?
<jcw4> fwereade: thx... I'm loving the 'go' way of thinking, so this has been a fun project
<fwereade> jcw4, awesome
<fwereade> mbruzek, I will poke around but might have to go in a sec, someone else who knows for sure should certainly answer if they can
<mbruzek> http://pastebin.ubuntu.com/7494454/
<mbruzek> That is the symptom, all my machines are in pending.
<mbruzek> I am looking in /var/lib/containers/ but I don't know what I am looking for.
<natefinch> mbruzek: check ps -A | grep lxc  - sometimes my lxc gets stuck and there will be a half dozen lxc processes started but not doing anything... I end up having to reboot to fix it.
<fwereade> mbruzek, it's /var/lib/juju/containers FWIW
<mbruzek> natefinch, I see nothing when running ps -A | grep lxc
<fwereade> mbruzek, it might be worth checking the logs for "juju.container.lxc" messages
<mbruzek> fwereade, the logs in ~/.juju/local/log/  have no such messages,
<mbruzek> I screwed something up here, I can tell
<fwereade> mbruzek, well, it might have been us
<fwereade> mbruzek, how about "provisioner" messages?
<mbruzek> It was running earlier today
<mbruzek> /var/log/juju-mbruzek-local$ grep provisioner *
<mbruzek> all-machines.log:machine-0: 2014-05-20 19:44:16 INFO juju.worker.singular singular.go:101 starting "environ-provisioner"
<mbruzek> all-machines.log:machine-0: 2014-05-20 19:44:16 INFO juju.worker runner.go:260 start "environ-provisioner"
<fwereade> mbruzek, hmm, this exact environment was running before? or the "same" one, but destroyed and recreated?
<fwereade> brb
<mbruzek> I destroyed the environment a few times today.  Had to control+C out of a bootstrap at one point.
<mbruzek> probably should not have done that (I realize)
<mbruzek> All looks well when I destroy-environment and bootstrap
<mbruzek> http://pastebin.ubuntu.com/7494490/
<natefinch> yeah, we try to handle control-C, but it's still not always 100%
<mbruzek> But when I try to deploy all machines stay in pending state and I get no unit logs.
<mbruzek> fwereade, When you get back it is the "same" environment destroyed and recreated.
<natefinch> mbruzek: what does sudo lxc-ls tell you?
<mbruzek> I just destroyed the environment again natefinch, sudo lxc-ls reports nothing.
<mbruzek> would you like me to bootstrap and run it again?
<natefinch> mbruzek: bootstrap, add a machine, wait 30 seconds, then run sudo lxc-ls
<mbruzek> natefinch,  http://pastebin.ubuntu.com/7494534/
<mbruzek> Still nothing when I run a second sudo lxc-ls --fancy
<natefinch> mbruzek: yeah, something's wonky, should look more like this: http://pastebin.ubuntu.com/7494535/
<mbruzek> How do I ask juju to re-download the image files?
<natefinch> mbruzek: they should be in /var/lib/juju/containers   if they're not there.... lemme check, I'm not 100% sure
<mbruzek> I see directories at that location juju-precise-template  juju-trusty-template  mbruzek-local-machine-1
<mbruzek> they contain clout-init lxc.conf, container.log and console.log no binary image.
<natefinch> mbruzek: same here
<mbruzek> How about your lock directory?
<mbruzek> root@workhorse:/var/lib/juju/locks# ls
<mbruzek> juju-precise-template
<natefinch> no locks for me
<mbruzek> I still have the lock when I have destroyed the environment.
<natefinch> mbruzek: try rebooting. I know it's wacky, but it helps a lot of the time (unless you've already tried that)
<mbruzek> Can I safely remove the /var/lib/juju/locks/juju-precise-template when the system is destroyed?
<mbruzek> natefinch, I have not rebooted, seems a bit windows-ish, but I am not against doing that.  As long as you are back here when I get back!
<natefinch> mbruzek: I have no idea.  My knowledge of lxc is pretty limited, unfortunately.
<natefinch> I'll be here for another half hour
<mbruzek> let me try removing the directory under locks so mine looks like yours firt
<mbruzek> first
<sinzui> natefinch, I don't understand this bug. I don't think it is about virtual private clouds https://bugs.launchpad.net/juju-core/+bug/1321442
<_mup_> Bug #1321442: Juju does not support EC2 with no default VPC <juju-core:New> <https://launchpad.net/bugs/1321442>
<mbruzek> natefinch, fwereade deleting /var/lib/juju/locks/juju-precise-template looks like has done the trick.
<mbruzek> I am able to get machines started now, and charms deployed!
<natefinch> sinzui: we definitely don't require VPC
<natefinch> mbruzek: awesome
<mbruzek> natefinch, Thanks for hanging in there with me.
<natefinch> mbruzek: no problem... I've been there
<mbruzek> I am sure fwereade would have too if he did not have to go.
<natefinch> mbruzek: seems like our cleanup code probably needs a little more attention to make sure we don't get that lock file stuck there
<mbruzek> yes that would be something to check.
<natefinch> mbruzek: yeah, it's 9:30pm his time, sorta understandable
<mbruzek> Yes, well thanks for both of you walking me through it.  I am not blocked any longer.
<natefinch> awesome
<bodie_> if a charm's actions.yaml has no params defined for an action, should its Params be nil or empty?
<bodie_> s/its/the Action's/
<bodie_> I'm assuming empty, but I want to make sure
<waigani> anyone know how to resolve a ghost in the ancestry of a bzr branch? (I feel like I'm playing a fantasy game)
<waigani> Could not determine revno for {jesse.meek@canonical.com-20140519235723-weky2oyq7yc20kqc} because its ancestry shows a ghost at {tarmac-20140413170514-ii8w6q2fht3eiizi}
<wwitzel3> EOD for me
<jcw4> bodie_: I vote for empty
<waigani> cya wwitzel3
<wallyworld_> fwereade: did you have any followup comments to my followup comments to your comments? :-)
<jcw4> wallyworld_: fwereade will probably followup with you on that ...
<wallyworld_> :-)
<sinzui> wallyworld_, I would like your help triaging bug 1320794
<_mup_> Bug #1320794: After bootstrapping an OpenStack environment, juju tries to connect to the internal IP <addressability> <landscape> <openstack-provider> <juju-core:New> <https://launchpad.net/bugs/1320794>
<wallyworld_> sinzui: ok, one sec otp
<sinzui> wallyworld_, I understand juju avoids public ips for internal communication, and that it (1.18.x) doesn't understand multiple networks. I think the issue may be fixed in 1.19.x and possibly related to bug 1283866
<_mup_> Bug #1283866: Openstack (>=grizzly) instanses can't accept floating IPs <addressability> <api> <days> <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1283866>
<wallyworld_> sinzui: in 1.19 (and possibly later 1.18 releases), work was done to make juju smarter about using the right network. it may be that the above bugs are the result of issues with the implementation. i'll talk to martin about it and update the bugs accordingly
<sinzui> fab. Thank you wallyworld_
<wallyworld_> sinzui: we need to talk about ppc, can i meet with you this time tomorrow, or about 30 minutes earlier?
<sinzui> sure
<wallyworld_> ok, i'll ping you
 * sinzui talked with dave and alexisb  last hour about it
<wallyworld_> yeah, i just got asked to get involved
<sinzui> wallyworld_, sure ping me when you are ready, caffeinated, and waring cozy slippers
<wallyworld_> will do
<wallyworld_> sinzui: also, i'm at a loss about the bzr failures, i'll need to talk to john or someone to try and understand why it's ding like that
<wallyworld_> dying
<sinzui> wallyworld_, I just verified the new CI build scripts will build PPC debs and tools for testing each time we test a revision. I will put this change into operation tomorrow
<wallyworld_> oh great. that's one of the things i wanted to talk abot
<sinzui> wallyworld_, I was pondering python version and bzr
<wallyworld_> did that change?
<sinzui> wallyworld_, I don't know.  I am not firing on all cylinders. I deleted my home directory last night; I am still numb.
<wallyworld_> :-(
<wallyworld_> leave it with me, we can catch up about it tomorrow
<wallyworld_> sinzui: i find a large glass of red helps in such situations :-)
<sinzui> Anne bought me cider :)
<wallyworld_> \o/
 * wallyworld_ bbiab
<perrito666> hello night shift
<rick_h_> wallyworld_: ping when you get back
<davecheney> lucky(~/src/launchpad.net/juju-core/state) % go test
<davecheney> 2014-05-20 23:46:31 INFO juju.testing mgo.go:453 reset successfully reset admin password
<davecheney> 2014-05-20 23:47:02 INFO juju.testing mgo.go:453 reset successfully reset admin password
<davecheney> 2014-05-20 23:47:25 INFO juju.testing mgo.go:453 reset successfully reset admin password
<davecheney> OK: 455 passed
<davecheney> ^ something leaking out the the tests
<davecheney> PASS
<davecheney> ok      launchpad.net/juju-core/state   125.341s
<davecheney> i didn't used to leak this output
<wallyworld_> rick_h_: g'day
<rick_h_> wallyworld_: hey man, got time to catch up at some point tonight/today?
<wallyworld_> sure, now?
<rick_h_> wallyworld_: sure
<wallyworld_> i'll start a hangout
<wallyworld_> https://plus.google.com/hangouts/_/gwwgzrr4og3azmmxmywivftyxua?hl=en
#juju-dev 2014-05-21
<davecheney> bugs, so many bugs
<bac> hi bigjools
<bigjools> o/
<bigjools> OTP
<bac> np, just saying hello
<bigjools> hello bac :)
<bodie_> I need some guidance here: http://bazaar.launchpad.net/~binary132/juju-core/charm-actions/view/head:/charm/actions.go
<bodie_> this bit in particular (line 58): actionsSpec.ActionSpecs[name].Params = map[string]interface{}{}
<bodie_> I'm trying to initialize the Params member with an empty interface (I've also tried make( map[string]interface{} ) with the same results but it's giving me the following error:
<davecheney> looking for a review on this
<davecheney> https://github.com/juju/errgo/pull/6
<davecheney> wallyworld_: nate, jimmiebtlr_
<bodie_> cannot assign to actionsSpec.ActionSpecs[name].Params
<davecheney> no, jam
<bodie_> presumably because Params is an uninitialized map
<davecheney> bodie_: i don't think reflect is necessary
<bodie_> you can do equality on a potentially nested map without reflect?
<davecheney> if actionSpec.ActionSpec[name].Params == nil { actionSpec.ActionSpec[name].Params = make(map[string]interface{})
<wallyworld_> davecheney: lgtm, i like it
<davecheney> }
<bodie_> I tried that, but it gives me the same error
<bodie_> it doesn't want me to assign values to that member... I'm not seeing why
<davecheney> because maps are not addressible
<davecheney> bodie_: why do you need to initalise that map
<davecheney> line 60 will work fine with the zero value
<bodie_> well, I don't necessarily need to.  what's happening is that I'm expecting an empty map in my test but getting nil
<bodie_> which got me thinking
<davecheney> 	var actionsSpec *Actions
<davecheney> 	if err := goyaml.Unmarshal(data, &actionsSpec); err != nil {
<bodie_> perhaps if the "params" key has no values, it should be defined as an empty value, rather than nil, that way it can be handled without nil checks
<davecheney> ^ ???
<bodie_> what's the question
<davecheney> your passing **Action to Unmarshal
<davecheney> shouldn't this be
<bodie_> one moment sorry
<davecheney> var actionSpec Actions
<bodie_> i'll be back
<davecheney> goyaml.Unmarshal(date &actionSpec)
<bodie_> okay, looking
<bodie_> child just awoke suddenly...
<bodie_> mh
<bodie_> hm*
<axw> wallyworld_: hey. are you on mgz's box?
<wallyworld_> no
<axw> k
<davecheney> wallyworld_: follow up for ya https://code.launchpad.net/~dave-cheney/juju-core/178-update-errgo-and-loggo-dependencies/+merge/220340
<wallyworld_> rightio
<wallyworld_> land that sucker
<axw> wallyworld_: are you looking at the bzr error, or should I?
<wallyworld_> axw: i haven't looked yet
<wallyworld_> it did happen after my branch landed
<axw> hrm ok
<wallyworld_> but off hand i can't see the causal link
<axw> I wonder if it's due to the bzr config not being found in $HOME
<wallyworld_> leave it and doing another one if you want, likr the new mongo failure
<axw> ok
<wallyworld_> maybe
<wallyworld_> axw: but it does pass here locally
<axw> I see
<wallyworld_> so could be pythin version related according to curtis
<bodie_> davecheney, seems weird that it would work even though I had the error you pointed out
<bodie_> anyway, corrected that but it's still giving me the same thing
<bodie_> with make as well as the way I had it
<wallyworld_> axw: it does seem that in CI, that ensure admin user stuff is happy now
<axw> hooray
<wallyworld_> but of course we have the new failures :-(
<axw> these last couple have been a massive PITA
<axw> yeah
<davecheney> faaaaaaaaaaaaaaark
<davecheney> https://code.launchpad.net/~dave-cheney/juju-core/178-update-errgo-and-loggo-dependencies/+merge/220340
<davecheney> this is really gettig me down
<axw> never ending
<wallyworld_> yep :-(
<davecheney> bodie_: can you paste the error
<davecheney> please
<bodie_> sure thing
<bodie_> cannot assign to actionsSpec.ActionSpecs[name].Params
<davecheney> bodie_: i need the code, the line number and the error
<axw> davecheney: I think that means the bot needs manually updating
<wallyworld_> davecheney: i'm not a git expert so have no idea wtf that error means
<davecheney> wallyworld_: what axw said
<axw> wallyworld_: godeps -u doesn't actually pull new revs
<davecheney> axw: nope, because godeps creates disconneted heads
<wallyworld_> so godeps needs fixing?
<davecheney> which is a git word for, i dunno, branches that have no upstream
<davecheney> wallyworld_: yup
<wallyworld_> i can manually update the bot
<davecheney> wallyworld_: thanks
<wallyworld_> davecheney: can you file a bug for the godeps thing?
<bodie_> davecheney, charm/actions.go:58: cannot assign to actionsSpec.ActionSpecs[name].Params
<bodie_> http://bazaar.launchpad.net/~binary132/juju-core/charm-actions/view/head:/charm/actions.go
<bodie_> same error if I use a composite literal of map[string]interface{}
<bodie_> er... append {}
<davecheney> bodie_: ok
<davecheney> sorry
<davecheney> there are two things going on
<davecheney> 1. map values may not be assignable
<davecheney> but
<davecheney> 2. you probaly don't need to do all that defensive coding
 * davecheney reads the code
<wallyworld_> bot fixed, tests running
<davecheney> wallyworld_: got another one for you https://code.launchpad.net/~dave-cheney/juju-core/179-remove-thirdparty-pbkdf2-dependency/+merge/220341
<wallyworld_> ok
<bodie_> I don't necessarily want to assign anything except an empty but non-nil value
<bodie_> just so I don't have people stumbling on nil param sets if the action doesn't take any parameters
<bodie_> I had a devil of a time trying to figure this out earlier
<bodie_> but, I guess I can leave it nil -- I just don't want inability to be what prevents me from doing it right
<bodie_> frustrating
<davecheney> bodie_: leave it as nil
<davecheney> bodie_: i'm really not sure about all the use of reflect in that file
<davecheney> it shouldn't be necessary
<davecheney> in fact, why won't that struct just unmarshall directly ?
<davecheney> bodie_: dont you just need to add some `yaml:...` tags to the ActionSpec struct
<davecheney> and it should unmarshal directly
<davecheney> without having to do it by hand
<davecheney> i'm clearly missing something
<bodie_> I'm probably not fully understanding the difference between what you're suggesting and what I have here -- I'm not certain I can do it justice, but I've been over it with William several times and he thought it important to leave room for metadata that we specify, in addition to a more flexible Params map per action which only need conform to JSON-Schema
<davecheney> bodie_: can you show me a sample actions.yaml file
<bodie_> the deep equality was because the compiler complained about the use of normal equality, and it seemed sensible since there could be nested maps
<bodie_> yeah, there's an example in my mr... let's see
<bodie_> https://codereview.appspot.com/94540044
<davecheney> bodie_: it feels like you are fighting the language
<bodie_> perhaps a bit :)
<davecheney> and you'll loose, 'cos its really stubourn
<davecheney> so i'd like to help find a better way to do what you want to do, rather than help you win a small battle with the compiler
<bodie_> that usually means I'm not properly understanding the way it wants me to do things :)
<davecheney> so looking at that definition
<davecheney> you wnt to do somethign like
<davecheney> type ActionSpec struct { Description string ; Params map[string]interface{} }
<davecheney> actionSpecs := make(map[string]ActionSpec)
<davecheney> goyaml.Unmarshal(r, &actionSpecs)
<davecheney> that should be pretty close
<davecheney> then you'll need to go through the params of each ActionSpec to do the json validation
<davecheney> but that should be a separate function
 * davecheney doesn't even know what json validatoin means
<davecheney> it sounds like an oxymoron
<bodie_> json-schema is a subset of json which has to conform to a spec
<bodie_> it's to define the format of json objects which will be passed as the arguments to the action
<bodie_> invalid params will be rejected
<bodie_> and you can just check it against the JsonSchemaDoc to validate the parameters
<bodie_> this is the argument in favor of this
<davecheney> sure
<davecheney> but that doens't need to be wedded to the yaml decoding process
<bodie_> so, we're ensuring that this yaml matches the format we need for json-schema, since charm coders like yaml, or something
<davecheney> it should just be a function that takes a map[string]interface{} and returns an error
<wallyworld_> arosales: did you end up meeting with the joyent folks? or is that yet to happen? i don't think i need to be there - hopefully the pull requests are self explanatory
<bodie_> I think the goyaml.Unmarshal bit is making the map, but I could be wrong
<bodie_> davecheney, we're just making sure at the time of unmarshaling that it's valid
<bodie_> the jsonschema bit just checks that, does nothing with the result
<bodie_> then, I'm going through each action in the list, making sure it has an OK name, and then checking to make sure its params do
<bodie_> that's really all
<bodie_> if the params map is nil, I'm trying to make it empty instead
 * davecheney twiddles thumbs waiting for the bot
<axw> jam: you put up a branch fixing this issue before, right? but it didn't get proposed for trunk? #1320794
<_mup_> Bug #1320794: After bootstrapping an OpenStack environment, juju tries to connect to the internal IP <addressability> <landscape> <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1320794>
<arosales> wallyworld: I haven't meet with them yet, but I think they are working on the pull requests
<wallyworld> arosales: great, thanks, let me know if i can do anything
<arosales> wallyworld: np will do
<jam> axw: IIRC we put one together that made it so we didn't use networks marked MachineLocal, however, it is possible the auto-detection in Openstack is wrong.
<jam> I don't think I did the fix
<axw> jam: ok. thought you did something for ODS
<axw> maybe that was it
<jam> axw: ah the ODS thing was attached to a bug but never proposed because it didn't have any tests.
<jam> axw: that was https://bugs.launchpad.net/juju-core/+bug/1308767
<_mup_> Bug #1308767: juju client is not using the floating ip to connect to the state server <addressability> <juju-core:Triaged> <https://launchpad.net/bugs/1308767>
<jam> it does have a branch associated, but without tests or someone at least trying it in the wild, I wasn't confident about it.
<jam> you're welcome to pick it up and finish it off
<jam> it *seems* correct to me
<axw> jam: thanks, that's what I was thinking about. I will take a look after my current work
<wallyworld> jam: do you have any idea what might cause "bzr add" to crash hard and print out "(PC=0x4143E6)"
<jam> wallyworld: that looks like a stack trace, which would sound like a C sort of extension, which *sounds* like a plugin.
<jam> try "bzr add --no-plugins" ?
<jam> well s/stack trace/core dump/
<wallyworld> jam: this is on jenkins, in some of the tests
<wallyworld> bug 1321282
<_mup_> Bug #1321282: PANIC: branch_test in trusty <ppc64el> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1321282>
<wallyworld> i guess we could modify the tests
<wallyworld> to add the --no-plugins arg
<jam> wallyworld: so the PC=... stuff is Go
<jam> not bzr
<wallyworld> i could have msiread the source code. it looked like when the cmd.exec failed it printed the failing cmd line and then the stdout
<wallyworld> i'll look closer
<jam> wallyworld: output is a []byte, so maybe it is giving the array rather than the string content?
<wallyworld> seems strange that it's just the bzr commands, hence my suspicion based on no facts that bzr might be involved
<wallyworld> could be yeah
<jam> wallyworld: my guess is that the failure is actually that bzr isn't present
<jam> note that we aren't checking "err"
<jam> which could be saying "ENOENT"
<wallyworld> yeah, true
<wallyworld> i'll modify  the test to give better output
<wallyworld> i haven't really looked that closely
<davecheney> does anyone know anything about a Tagger interface ?
<davecheney> for things that have, uh, a tag ?
<fwereade> davecheney, I think all state entities have one
<fwereade> davecheney, the tag is the definitely-unambiguous identifier we use over the API
<fwereade> davecheney, ie unit-mysql-3, machine-7-lxc-1, service-blah, relation-some-horrible-mess
<fwereade> davecheney, except we don't always use it over the API, and sometimes for no explicable reason someone allows them to leak into the UI
<davecheney> fwereade: there is no state.Tagger interface definition
<davecheney> nm, i just made one up
<davecheney> fwereade: i can easily move that definition into the state package for better feng shui
<fwereade> davecheney, heh, it's state.Entity
<davecheney> fwereade: right
<davecheney> will fix
<davecheney> or I can fix in a followup
<TheMue> morning
<dimitern> hey TheMue
<axw> wallyworld: FYI, bzr tests are barfing on my laptop
<wallyworld> oh great :-(
<axw> wallyworld: I can take that next if you're not on top of it
<wallyworld> axw: i've been caught up with other stuff o haven't yet got root cause sussed out. i'll take another look and we can discuss at standup
<wallyworld> bug 1321009 is pretty important
<axw> wallyworld: sure
<axw> wallyworld: is that what you're looking at now?
<axw> wallyworld: or I should take a look?
<wallyworld> nah, gccgo stuff
<axw> ok
<wallyworld> feel free
<voidspace> morning all
<dimitern> morning voidspace
<dimitern> (just how it sounds :)
<wallyworld> axw: i just got the failing bzr tests as well although i could have sworn they passed before as i did a complete test run. go figure
<fwereade> hey all
<fwereade> I'd love a review of https://codereview.appspot.com/94540043/
<axw> wallyworld: me too, though I did a pull earlier... I'm pretty sure I had all of your changes already though
<dimitern> fwereade, jam, vladk|offline, I'd really appreciate a second look on this goamz CL https://codereview.appspot.com/98430044/
 * dimitern is away for a while
<tasdomas> fwereade, hi
<fwereade> tasdomas, heyhey
<fwereade> dimitern, oops, looking
<fwereade> tasdomas, (I can talk and review at the same time, at least to some degree)
<tasdomas> fwereade, was thinking about the ensure-availability MP - should we display the Promote/Demote/Remove information always or just in verbose mode?
<axw> wallyworld: I think it will be very difficult to backport that fix for 1.18
<axw> it relies on a bunch of existing changes in 1.19 that allow dialling multiple addresses
<wallyworld> axw: ok, it seemed like something we would have wanted
<fwereade> tasdomas, I think we should generally display feedback, and have a --quiet option; --verbose remains meaningful, it just means *loads* of output
<wallyworld> axw: i think a 2nd opinion to mine as to how important it is would be desireable
<axw> wallyworld: ideally yes, but I think it should probably just wait for 1.20 unless it's critical (which I assume it's not, since this is nothing new)
<wallyworld> ok
<tasdomas> fwereade, so would this sort of output look sane to you: http://paste.ubuntu.com/7496511/ ?
<wallyworld> axw: FFS :-( https://codereview.appspot.com/100590048
<mgz> wallyworld: aha
<mgz> wallyworld: what's the reason?
<wallyworld> mgz: i was dumb
<axw> mgz: bzr wants a home
<axw> it tries to read /.bazaar if $HOME==""
<mgz> oh yeah, \bsse just removes the envvar
<axw> wallyworld: mgz sorry to be a pain, going to be a little late again - gotta get my daughter to have a shower... back real soon
<wallyworld> yep
<wallyworld> ok, np
<fwereade> tasdomas, sorry: I'd say --quiet can suppress all output, and it would be perfect if normal mode output the final list of expected sate servers
<tasdomas> fwereade, ok - what do you mean by "final list of expected state servers" ?
<fwereade> tasdomas, if we add a new one it isn't a state server *yet*, but we expect that in a couple of minutes it will be
<axw> wallyworld mgz: back
<tasdomas> fwereade, so instead of printing the promotion/demotion lists, I should just print a list of what state servers are expected to be operating once the ensure-availability process completes (i.e. collapse those lists)?
<fwereade> tasdomas, yeah, I reckon the lists are maybe best for --verbose
<fwereade> tasdomas, the thing most people care about is "which ones are now my state servers"
<fwereade> tasdomas, with an option on "which ones used to be but aren't any more"
<tasdomas> fwereade, ok - I will update the branch
<dimitern> fwereade, how's the looking going? :)
 * fwereade glances guiltily at dimitern, changes tabs
<dimitern> :)
<dimitern> np
<mgz> wallyworld: I wonder if we should switch over goose or something first, less active than core and may avoid some early pain
<wallyworld> mgz: not a bad idea. but i guess we already have other things in github, but not the landing bot support. let's see how we go over the next day or so. at this stage i think we should just rip off the band aid
<wallyworld> also, the guys getting store out of core are waiting for us
<wallyworld> gotta run though, back later maybe
<mgz> wallyworld: goose is on the landing bot atm I mean, so we need both and it's the same steps
<mgz> just doing the smaller one first will give a day or whatever to find glaring issues
<voidspace> router firmware update
<voidspace> going offline briefly
<fwereade> dimitern, so, it is looking pretty sane to me, but I'm worried I'm not the best person to review it -- jam, do you feel ~at home with the ec2 testing double? I've never really worked with it at all
<fwereade> dimitern, all I really have is a whine that one of the methods is awfully big
<dimitern> fwereade, it is indeed, and that's after splitting it up a bit
<dimitern> fwereade, I welcome suggestions how to simplify it further though
<fwereade> dimitern, I was thinking that at least the if len(ifaces) == 0 block could move out
<fwereade> dimitern, it was less clear whether it'd be a win to extract that massive switch
<dimitern> fwereade, ah, good point
<dimitern> fwereade, it's really not that easy to parse that huge complicated form
<fwereade> dimitern, I suspect that if you were to do that it'd be a matter of a new type that sort of accumulates state in its fields
<dimitern> fwereade, hence the switch
<dimitern> fwereade, expand a bit please?
<fwereade> dimitern, some of those switch branches are pretty big themselves, so it could be good to extract them; but maybe it isn't even a type, it's just a func that takes a *iface and modifies it in place
<fwereade> sorry forget anything I said about an extra type
<fwereade> dimitern, just consider breaking the longer switch branches out so it's easier to see the structure of the whole method without getting lost
<dimitern> fwereade, i kinda feel there is indeed a type waiting to emerge around parsing stuff in the test server, but it will require bigger refactoring
<dimitern> fwereade, alright then, sgtm
<fwereade> dimitern, I heartily support the "refactor first, so that the change you want to make becomes trivial" approach, so I'm not going to complain if you do do that, but if it's not clear what to do maybe best to skip it ;p :)
<dimitern> fwereade, yeah, it will blow up the CL without much immediate gain, and I just want to get it landed so I can concentrate on juju work
<fwereade> dimitern, +1
<pindonga> hi, I'm having some issues trying to deploy charms on canonistack; I've filed 2 bugs in case someone can help me figure out what's wrong with this
<pindonga> https://bugs.launchpad.net/juju-core/+bug/1321686
<_mup_> Bug #1321686: initial deploy using local provider fails because template creation takes too long <juju-core:New> <https://launchpad.net/bugs/1321686>
<pindonga> https://bugs.launchpad.net/juju-core/+bug/1321683
<_mup_> Bug #1321683: juju takes a long time to deploy using the local provider <juju-core:New> <https://launchpad.net/bugs/1321683>
<tasdomas> is `juju ensure-availability` supposed to work with the local provider?
<jam> tasdomas: it is currently blocked from  working there
<jam> well, it doesn't work there
<jam> enabling it properly is a big pile of work
<voidspace> natefinch: https://codereview.appspot.com/94510043/
<jam> doing an approximation of it is just a matter of disabling some checks
<tasdomas> jam, just to check that I understand this correctly - enabling it would mean changing the location of the state server in local provider (as I understand it, localhost is the state server in local provider)
<jam> tasdomas: enabling it would mean that we would add a state server also in a container. To enable it properly that state server would then need something to talk to running on your host that could start more containers
<jam> because from within a container, you cannot spawn other containers
<tasdomas> jam, thanks
<jam> fwereade: want to join our standup?
<fwereade> jam, why not :)
<jam> https://plus.google.com/hangouts/_/canonical.com/juju-sapphire
<voidspace> natefinch: actually there was a review from fwereade that I missed - with a few changes needed on that mp
<voidspace> natefinch: so switching to completing that
<natefinch> ok cool
<voidspace> fwereade: ping
<fwereade> voidspace, pong
<voidspace> fwereade: about your review comments on the rsyslog branch
<voidspace> fwereade: specifically, dropping the Port field on RsyslogConfig
<voidspace> fwereade: I did start down that road
<fwereade> voidspace, oh yes?
<voidspace> fwereade: it means having RsyslogWorker creation plus template rendering use HostPorts instead of the global Port
<fwereade> voidspace, yeah, that was the plan in my mind
<voidspace> unfortunately at bootstrap time we have a global port and not a set of HostPorts available
<voidspace> so it means threading this api change through several layers of architecture in two places (initial creation plus in the worker handle)
<voidspace> that was quite an invasive change (not *too bad*)
<voidspace> https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-api/+merge/220087
<voidspace> I made the code changes, but that broke a metric crapload of tests *too* that would also need fixing
<voidspace> (and I'm a little uncertain about what I've done with worker creation - but I can talk that through with someone)
<voidspace> fwereade: so it seemed like quite a lot of extra work for something we *might* use in the future (switch away from a global port)
<voidspace> fwereade: so in discussion with natefinch it seemed like defering that work until we actually needed this capability was more sensible
<voidspace> fwereade: would you rather I spent the time on it now?
<fwereade> voidspace, heh, I see -- tbh those changes do look like a win to me (although there's still the odd Port sitting around?), but let me think a sec
<fwereade> voidspace, the main thing is that if we have HostPorts in the API but don't use it, there doesn't seem much point to that
<voidspace> it's an initial step...
<voidspace> it means when we do switch the GetRsyslogConfig already supports it
<voidspace> the *API* already supports it
<fwereade> voidspace, right, but unless that step is imminently followed by further steps in the same direction its value is somewhat limited
<voidspace> well, it means when we do make the switch it doesn't require a backwards incompatible API change
<fwereade> voidspace, versioning is coming along apace, thanks be to jam
<voidspace> switching template rendering etc to use it - but still *actually* only storing and using a global port would be just as pointless
<voidspace> but more work
<voidspace> even with these changes (plus maybe pulling the Port out of syslogConfig altogether) wouldn't change the fact that we're only *really* using a global Port
<voidspace> it would just mask it further
<fwereade> voidspace, agree -- but each, in the absence of the other, is just a speculative change, and they rarely work out well
<voidspace> so yes, fewer changes to make in the future - but no actual change in behaviour
<fwereade> voidspace, getting the mask in place between the api server and the rest of the world is imo a good sort of masking
<voidspace> fwereade: but even with this extra work, it's *still* a speculative change
<voidspace> fwereade: right, yep
<voidspace> that is true, it leaves the only place that thinks in terms of a global port being the internal database
<fwereade> voidspace, it eliminates the assumption of a single port among the api clients
<voidspace> fwereade: it sounds like you'd rather I did it
<fwereade> voidspace, what happens under the covers is exposed to far fewer people ;)
<fwereade> voidspace, I would, yeah
<voidspace> ok
<voidspace> np
<fwereade> voidspace, tyvm
<natefinch> sorry to throw a wrench in things, guys.  It just sounded like more work was being added to support a feature that may or may not ever be implemented, and I wanted Michael to be able to move on to other projects.
<voidspace> fwereade: natefinch: shall I do this as a separate CL - it's going to be quite a big diff against the existing branch
<voidspace> so I can address the *other* comments from fwereade in the existing CL and then work on the new one
<natefinch> voidspace: that sounds like a good idea
<voidspace> fwereade: ah, there's a difficulty with eliminating use of Port altogether
<voidspace> fwereade: when we start a new state server we need to pick a port to use
<voidspace> fwereade: and the "State" needs to *know* which port we picked, because it needs to associate the state server with that port to return in RsylogConfig.HostPorts
<fwereade> voidspace, does that happen at the same time as we generate the cert?
<voidspace> fwereade: to be honest I'm not sure how we pick the initial port - let me check
<voidspace> fwereade: hmmm... jujud/agent doesn't pass a port and the worker is started with a port of 0
<fwereade> voidspace, so, wait, we have an env config setting that's just ignored?
<voidspace> ensureCertificates doesn't set a port
 * fwereade wonders why it's env config in the first place then :/
<voidspace> fwereade: we must be setting it somewhere - because HA does set the correct port
<voidspace> I just haven't found it yet
<voidspace> fwereade: hah
<voidspace> fwereade: we have DefaultSyslogPort in environs/config/config.go
<fwereade> voidspace, that syllable rarely preceds good news
<voidspace> fwereade: what happens is that we use port 0
<voidspace> fwereade: when we set the cert it causes a rewrite of the config which requests the port from EnvironConfig
<voidspace> fwereade: which returns the DefaultPort, which is what we use
<voidspace> and port 6541 is what I always see used (the default)
<voidspace> we have an api to change the port though
<fwereade> voidspace, ah, ok, but someone might have set it to something different, and in that case it would be used?
<voidspace> let me see where that's called from
<voidspace> fwereade: yes, if it's changed we would (previously anyway!) see the change and rewrite
<voidspace> fwereade: axw is certain that the port is in the config when the worker is created - so we should be safe (and indeed we will make the port immutable once set)
 * fwereade is somewhat reassured
<voidspace> heh
<voidspace> hmmm... no, we only have an api to set the cert
<voidspace> setting the port has to be done through environconfig I believe
<voidspace> fwereade: sigh, so I don't think it's set deliberately in our code - we always use the default
<voidspace> fwereade: so the only way it would be set would be by users doing it - and we wouldn't pick up that change with our new code
<voidspace> fwereade: so I need to talk to axw again about the motivation for allowing it to be changed
<voidspace> fwereade: if there's a real use case then we need to still support it
<voidspace> he said it was because previously it could pick a privileged port and rsyslog refuse to start
<voidspace> but with our default of 6541 I can't see that being the case now
<voidspace> it maybe that the initial starting of the rsyslog worker was picking a privileged port when we give it port 0
<voidspace> but as far as I can see we *always* overwrite that immediately - we don't set back the port the system gives us
<wwitzel3> hello
<voidspace> wwitzel3: morning
<voidspace> fwereade: what that means though is that we *don't* have a way to pick a port for rsyslog - we use the default or one the user has set
<voidspace> fwereade: so a new state server coming online either has to have a new way of picking a port, and that needs to be stored associated with the server
<voidspace> fwereade: or we still have to use a global one
<voidspace> fwereade: meaning I can't eliminate it from the api client
<voidspace> fwereade: we could call it "DefaultPort", and it would only be used by state servers not by units
<voidspace> and setting the environ config would only change the default port for new state servers
<fwereade> voidspace, that doesn't seem quite right either
<fwereade> voidspace, although... there's something funny about having the same RsyslogConfig for servers and clients, isn't there? I'm not sure we should be getting the info from the same call
<voidspace> fwereade: the part of the state server config that specifies the port to listen on is a separate part to the bit where we specify forwarding rules
<voidspace> fwereade: so conceptually there is a difference between "the HostPorts to forward to" and "the port to listen on"
<voidspace> and if we eliminate Port now, it's harder to communicate a *port change* which is a compatibility issue
<fwereade> voidspace, ok, so, if we keep Port and document it as "the port this server should listen on"; and also have HostPorts meaning "where you should send your logs", we're essentially fine modulo the semantic drift, which probably doesn't hurt us much?
<voidspace> fwereade: I concur
<fwereade> voidspace, and again the fact that Port is always the same is something we *can* vary per-server if we want to , without changing anythying outside the API
<fwereade> voidspace, ok, cool
<fwereade> voidspace, keep Port but make it clear what it's for
<voidspace> we can't really eliminate Port from the API client until we start properly storing HostPorts in the db, and have state servers pick a port and store that information
<voidspace> yep, we can still do that without changing the api
<fwereade> voidspace, ok, sgtm, thanks
<voidspace> wwitzel3: I'm addressing William's comments on our CL
<wwitzel3> voidspace: got my coffee, reading the review and chat here
<voidspace> I'm going on lunch
<voidspace> wwitzel3: I'll sync up with you when I'm back
<wwitzel3> voidspace: sounds good
<axw> voidspace: the use case was purely for supporting upgrading existing environments to TLS without triggering the privileged port/drop-to-root race in rsyslog
<axw> voidspace: we don't need to do this anymore, as of 1.19.x
<axw> voidspace: I think let's just make it immutable again
<axw> (it used to be)
<axw> voidspace: also, the default used to be a privileged port
<voidspace> dammit, dammit
<voidspace> I mean, what axw said is *great*
<voidspace> but I forgot I have a dentist appointment today
<voidspace> so I'm back off lunch, but will be going out shortly and not back until later
<wwitzel3> gotta keep those teeth healthy
<voidspace> heh, yeah
<perrito666> voidspace: uh, you reminded me
<voidspace> perrito666: sorry :-)
<voidspace> I completely forgot about it until now
 * perrito666 changes his dentist appointment by one week bc he accidentally let himself without health care for a month
<hazmat> anyone know the  reason add-unit does a tools lookup instead of using the env existing tools?
<natefinch> I'm sure someone knows
<natefinch> but not me
 * hazmat files a bug
<voidspace> fwereade: if we no longer support setting rsyslog port at all, should I remove upgrades/rsyslogport.go?
<voidspace> fwereade: it would only ever set the port to the DefaultSyslogPort anyway
<voidspace> fwereade: the use case was for an older version of juju anyway
<fwereade> voidspace, I would prefer not to drop anything from upgrades until I can no longer think of any way anyone could possibly be running an old enough version to qualify
<voidspace> fwereade: well, it's just that if the port is immutable then that code can't succeed
<voidspace> fwereade: unless we make it "immutable unless switching to the default"
<voidspace> fwereade: but we'd like to be able to guarantee that the port can't change
<voidspace> fwereade: so the watcher can ignore environ config altogether
<fwereade> voidspace, or immutable unless switching from unset (and being sure that it's always set on bootstrap ofor newer envs)
<fwereade> voidspace, wait, possibly that made no sense
<voidspace> fwereade: we *always* use default, except on an existing system where it may have been changed
<voidspace> fwereade: upgraded from 1.16 to 1.18 will switch to default
<voidspace> fwereade: and we don't support 1.16 to 1.20 directly I don't believe
<fwereade> voidspace, but it's in env config, won't we use whatever was already there?
<voidspace> so 1.20 doesn't have to consider a *change*
<voidspace> we will use whatever is already there
<voidspace> the question is - do we have to watch for changes
<voidspace> the use case for permitting change was for the upgrade
<voidspace> there's no such value as "unset" really I believe - just Default or non-default. If we allow changing away from default *at all*, then we need extra complexity in the watcher to handle the port changing
<voidspace> and post 1.18 there's no actual use case for that
<voidspace> is the "upgrades/*" code used by the *client*? If a 1.20 client might run against an earlier server then maybe we still need that code.
<voidspace> But that code should fail against a 1.20 server.
<fwereade> voidspace, the upgrades code should not be used by the client, no, I think we're ok there
<fwereade> voidspace, (sorry, workman interruption, reloading state)
<fwereade> voidspace, IIRC you don't have to watch for changes anyway
<fwereade> voidspace, the watcher will always fire once when the agent bounces, and it'll always bounce on upgrade
<voidspace> fwereade: right, but it's not the upgrade that's the problem - it's if we don't make it totally immutable it could change at *other* times
<voidspace> and *then* we'd have to watch for changes
<voidspace> unless I can make it "immutable except during upgrade"
<voidspace> there's no "unset" value I don't think
<fwereade> voidspace, well, before it existed it would simply not have been there, right?
<fwereade> voidspace, ah, ok, but it's inserted automatically when we read a config without it
<fwereade> voidspace, hmm, in which case, yeah, I don't see the point of that upgrade step at all
<fwereade> voidspace, I think you can drop it, but get axw to review it as well
<fwereade> voidspace, on a separate note
<fwereade> voidspace, jobs in agent config vs jobs from the API
<voidspace> ok
<fwereade> voidspace, which we happen to use at any given time seems pretty random
<fwereade> voidspace, is there a pattern I'm not seeing?
<voidspace> fwereade: what do you mean by "jobs"? :-)
<fwereade> voidspace, entity.Jobs() vs agentConfig.Jobs()
<natefinch> fwereade: if there's two sources of truth, a programmer will use whichever one he stumbles on first
<fwereade> voidspace, quite so
<voidspace> fwereade: I don't see either used in our CL
<fwereade> voidspace, I had a vague recollection that you were doing something around that area in your first sprint, but maybe you were just in the conversation
<voidspace> fwereade: ah, not something I recall I'm afraid
<fwereade> voidspace, no worries
<fwereade> dimitern, does the Jobs() thing above ring a bell for you?
 * dimitern reads scrollback
<natefinch> fwereade: I've faced that dilemma - where to get it.  is there a right place and a wrong place?  If we have two places, can we consolidate them, or at least hide one so we don't confuse ourselves?
<natefinch> fwereade: I was talking about this with perrito666 because he was doing work in APIWorker to only connect to localhost for the API if we're on a state server, but it seemed like we needed to connect to the API first to determine our jobs.  Maybe we missed where the jobs were held in the agent config
<fwereade> natefinch, well, the jobs from the state server should be authoritative
<dimitern> fwereade, it's not random at all - Jobs from agent config is used only before we have an api connection, then we use API Jobs
<dimitern> voidspace, ^^
<fwereade> dimitern, but we don't update config jobs based on api results, right?
<fwereade> dimitern, mainly because jobs don;t change -- yet -- I guess
<dimitern> fwereade, no, they're not updated
 * perrito666 feels summoned
<fwereade> dimitern, do you remember why we need jobs before we have an api connection?
<fwereade> natefinch, fwiw there's a Jobs method on agent config
<natefinch> fwereade: well, in the example I was talking about, we were using jobs to determine which API to connect to
<dimitern> fwereade, to know what workers to start in the agent?
<fwereade> dimitern, what workers need to be started before we're connected, that can't be determined trivially by looking for whether we have api-connect info and/or state-connect info?
<fwereade> dimitern, natefinch: ie, always connect to the api; and if we can connect to state, do so
<voidspace> dimitern: thanks
<dimitern> fwereade, i'm not saying it's not shit, it is :)
<voidspace> ok, folks - off to the dentist
<voidspace> it's in the next town over, so I'll be a while
<dimitern> fwereade, we should not use agent config Jobs or other "cached" stuff for that matter before we connect to the api
<wwitzel3> voidspace: gl :)
<fwereade> dimitern, I'm not quibbling there, I'm really just trying to determine the diet that led to the substance in question
<perrito666> voidspace: beautiful, across town travel with half face asleep
<fwereade> voidspace, take care
<TheMue> jam: you discussed about the charm implications of the network model. thatâs not in the agenda table. add?
<dimitern> fwereade, for agent config Jobs() existence specifically, I'm to blame, but I just moved them from the Values map to a config field
<fwereade> dimitern, no worries, I'm happier having them there than in Values
<dimitern> fwereade, you can probably find the reason following the changes around adding the jobs to the map in the first place
<fwereade> dimitern, just trying to unpick jujud a little bit, it's triped all my needs-fixing flags :)
<dimitern> fwereade, but i really am not the best person to ask why do we have them
<fwereade> dimitern, not to worry, thanks for the nuggets you supplied :)
<dimitern> fwereade, thumper / rog know more most likely
<dimitern> fwereade, ;)
<fwereade> dimitern, hmm, I did not actualy mean nuggets of shit there, sorry
 * dimitern hears for that type of nugget for the first time :)
<natefinch> lol  when you have kids, you get to know lots of types of nuggets
<natefinch> and shit
<jcw4> fwereade: regarding the CL https://codereview.appspot.com/98260043/ ... you'd mentioned cloning the Unit.  I thought that would be at least the same db impact as what I'm doing by checking the u.doc.Life in mongo directly?
<fwereade> jcw4, the trick is only hitting the db after you've failed a transaction, and doing that is often a bit tedious to arrange cleanly
<fwereade> jcw4, "if i != 0" is about as nice as it gets iirc
<fwereade> jcw4, and that still has a little ickiness about it
<jcw4> fwereade: I see.  Other than that and punting on the contention simulation I think everything else is ready to submit...
<jcw4> fwereade: I was kinda waiting for your cleanup refactor to land to introduce actions cleanup
<fwereade> jcw4, excellent, I'll take a quick look and approve in a bit
<jcw4> fwereade: tx
<fwereade> jcw4, good point
<fwereade> natefinch, dimitern: either of you free to review https://codereview.appspot.com/94540043/ ?
<jcw4> fwereade: mgz may be interested too?
<fwereade> jcw4, good point, yeah; mgz ^^
<dimitern> fwereade, i can look a bit later if not too late
<natefinch> fwereade: sure
<jcw4> forgot to lbox propose it again...
<jcw4> okay done
<fwereade> oh WTF
<dimitern> fwereade, vladk, https://codereview.appspot.com/98430044/ updated - final look while i'm waiting for gustavo?
<fwereade> we don't even use the jobs in agent config, except in the StateWorker, which we only start if we have StateServingInfo available in the first place
<fwereade> and while it might indeed be nice to separate jobs that connect to state from jobs that cause state to exist, we don't have a distinction there at all currently
<fwereade> next does-anyone-know question: (1) we still have minunitsworker running against state, does anyone have any idea why? (2) we have all this "singular" crap in stateworker, does anyone know why?
<natefinch> fwereade: the singular stuff is to ensure we only have one of them running in the environment, in case of HA where we have multiple state servers
<fwereade> natefinch, right, but those are all tasks that are meant to work with multiples
<wwitzel3> natefinch: standup?
<natefinch> wwitzel3: thanks for reminding me
<wwitzel3> np :)
<fwereade> natefinch, it's quite nice to have minunitsworker singular I guess
<fwereade> natefinch, but resumer should certainly be running everywhere
<fwereade> natefinch, and cleaner should also be fine, I think
<fwereade> (grumble grumble, why is resumer only started after upgrade?)
<natefinch> fwereade: voidspace did that work with roger I think.
<fwereade> natefinch, cool, I'll see what he remembers when he gets back
<dimitern> niemeyer, hey hey
<dimitern> niemeyer, i've updated again https://codereview.appspot.com/98430044/ - included more tests and simplified the code a bit, can I get an LGTM? :)
<niemeyer> dimitern: Will have a look
<dimitern> niemeyer, thanks!
<niemeyer> dimitern: Have you found anyone with good knowledge on those areas of EC2 to have a look over it?
<dimitern> niemeyer, it seems only vladk has more experience with ec2/vpc/nic stuff, I asked for a review
<dimitern> niemeyer, but in any case, i have a parallel juju branch that depends on those changes and it does seem to work for what we need
<niemeyer> dimitern: Can I nitpick a bit about the comment?  It'll probably be useful in future code..
<niemeyer>   404                         // If default-vpc were provided, create them and their
<niemeyer>   405                         // subnet.
<niemeyer> This is right after
<dimitern> niemeyer, yes?
<niemeyer> 403                 if attrName == "default-vpc" {
<niemeyer> dimitern: At this point, it's not "if".. the attribute *was* provided
<niemeyer> dimitern: and "them" is context-less there.. you are not creating attributes, I assume
<dimitern> niemeyer, "them" refers to the default vpc
<niemeyer> dimitern: That's not what the first part of the sentence refers to
<niemeyer> dimitern: It's talking about the defalut-vpc attribute
<natefinch> fwereade: looking at the review you mentioned about setting departed on dying units' relations... what does it mean for a relation to be in scope?
<niemeyer> dimitern: I'd suggest something along the lines of
<dimitern> niemeyer, well, it's both the attribute name and what it signifies
<niemeyer> dimitern: // The default-vpc attribute was provided, so create the respective VPCs and their subnets.
<niemeyer> dimitern: No, it's not
<dimitern> niemeyer, sgtm, will change it
<niemeyer> dimitern: It's the attribute that was provided
<niemeyer> dimitern: You cannot provide the VPC object
<niemeyer> dimitern: Either way, that's FWIW really..
<dimitern> niemeyer, that's correct, you just provide it's id in the values slice of the attribute named "default-vpc"
<dimitern> niemeyer, I see how it's confusing and will rephrase it as you suggest
<perrito666> Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´
<perrito666> Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´Â´'
<perrito666> ouch sorry dropped my external kb over the laptop
<niemeyer> dimitern: Thanks.. I'd not bikeshed for this instance alone.. the comment might even go away entirely since the code is clear enough, but keeping that kind of thing in mind has a chance of improving other comments in future code
<fwereade> natefinch, I haven't written that bit up yet: quick hangout?
<natefinch> fwereade: sure, cool
<niemeyer> perrito666: I thought it was a big quote for an upcoming text file!
<fwereade> natefinch, popping into moonstone
<niemeyer> perrito666: Glad it wasn't ;)
<niemeyer> dimitern: The trick of having the comment inside the if block is a good one I learned not too long ago
<niemeyer> dimitern: It enables phrasing such as "We have a foo, do bar."
<dimitern> niemeyer, +1 good point
<niemeyer> dimitern: Instead of "If the world is in the proper conditions then we'll have a foo, and be able to do bar."
<wwitzel3> lol .. when I read fwereade cooment to natefinch I added mentally replaced that second p with an o .. now I can't stop laughing.
 * wwitzel3 is mature.
 * dimitern is afk for some time
<fwereade> haha
<natefinch> wwitzel3: lol
<TheMue> *rofl*
<niemeyer> dimitern: I still don't get the logic around this:
<niemeyer> if len(ifaceToCreate.PrivateIPs) > 0 {
<niemeyer>         primaryIP = ifaceToCreate.PrivateIPs[0].Address
<niemeyer> The new comment you added says
<niemeyer> 	  732                 // To simulate that, we need to set PrimaryIPAddress of the
<niemeyer>   733                 // NIC and empty the PrivateIPs when there is only one address
<niemeyer>   734                 // specified there.
<niemeyer> But nowhere does it check that there is a single entry in that list
<niemeyer> It just picks up the first one blindly and considers it as the primary ip
<niemeyer> At the other end of it, we see
<niemeyer>     614                                 iface.PrivateIPs = append(iface.PrivateIPs, ec2.PrivateIP{
<niemeyer> dimitern: I must be missing something
<dimitern> niemeyer, just a sec
<dimitern> niemeyer, it my tests AWS behaves like this: if you specify explicit NICs and private IPs in RunInstances, when you get the instance details, the NetworkInterfaces[x].PrivateIPs is populated as you specified
<dimitern> niemeyer, OTOH if you don't specify NICs, AWS creates one for you with a single private ip assigned, which is the primary, and sets NetworksInterface.PrivateIPAddress and PrivateDNSName, but leaves the PrivateIPs slice empty, i.e. the single ip is not duplicated in the slice and the field
<dimitern> niemeyer, i forgot to mention that in the first case, PrivateIPAddress is not set
<niemeyer> dimitern: and what prevents that slice from having 10 entries?
<niemeyer> dimitern: and why would the first one be primary in that case?
<dimitern> niemeyer, the first one is the primary, when there's anything in the slice at all - that's my observation
<dimitern> niemeyer, the slice can have many entries
<niemeyer> dimitern: Sorry, but I don't get the logic
<niemeyer> dimitern: There's a: case "PrivateIpAddresses":
<dimitern> niemeyer, i don't get it either - it's just what I've seen AWS respond with
<niemeyer> dimitern: In your code
<niemeyer> dimitern: That adds entries into the iface.PrivateIPs
<dimitern> niemeyer, yes, that's where PrivateIPs gets deserialized from
<niemeyer> dimitern: slice
<niemeyer> dimitern: and has a privateIP.IsPrimary = val
<niemeyer> dimitern: Assignment
<niemeyer> dimitern: Based on a "Primary" field
<dimitern> niemeyer, yes
<niemeyer> dimitern: There's zero consideration of ordering in that logic
<niemeyer> dimitern: Why on earth a few instructions afterwards the first one would magically be the primary?
<dimitern> niemeyer, sorry, I don't understand that question
<niemeyer> dimitern: It
<niemeyer> dimitern: It's the only one I've been asking so far :)
<niemeyer> dimitern: On line 607 there's logic appending to PrivateIPs and handling a Primary boolean
<dimitern> niemeyer, AWS, when there is more than one private ip, returns a populated PrivateIPs, and usually the first one in there is the primary
<niemeyer> dimitern: On line 736 there's logic that considers the first IP on PrivateIPs  to be the primary, whatever happened on line 607
<niemeyer> dimitern: You've said that many times, but it's completely irrelevant
<dimitern> niemeyer, these are two different things
<niemeyer> dimitern: Okay, phew.. can you explain how?
<dimitern> niemeyer, in parseRunNetworkInterfaces, I'm taking AWS request and I'm parsing is as best as I can, because that's what the ec2 provider in juju will be using
<niemeyer> dimitern: Okay?
<dimitern> niemeyer, in createNICsOnRun I'm using that to create fake NICs for the instance, so when I get the instance info in juju it will have a NIC
<niemeyer> dimitern: Well, yes, you've described what both of these functions do
<niemeyer> dimitern: per their documentation
<niemeyer> dimitern: Now, what gives them a free pass to create completely bogus data?
<niemeyer> dimitern: Based on blind assumptions?
<dimitern> niemeyer, why bogus?
<niemeyer> dimitern: because parse takes a list of interfaces which explicitly define what is primary.. and create ignores that information entirely and picks the first entry out of the slice
<niemeyer> dimitern: Again, this is the same point I've been making over and over
<niemeyer> dimitern: and so far there's no good answer for why that'd be okay
<dimitern> niemeyer, well, because i'm trying to accommodate an arbitrary user-sent RunInstances + NICs request as AWS would do (all quirks included)
<niemeyer> dimitern: What quirks?  Do you mean Amazon ignores a Primary: true definition for an interface?
<niemeyer> dimitern: THat'd be *very* surprising
<dimitern> niemeyer, and in case the user didn't give any NICs, create one as if the user did, and in that case i control what's in PrivateIPs
<dimitern> niemeyer, that part of AWS API is unnecessary complicated i think - why have both PrivateIpAddress as a separate field, and a list of PrivateIpAddresses, which can be primary or secondary
<dimitern> niemeyer, you can even combine both in the same request up to point
<niemeyer> dimitern: Okay, we've been talking about this for half an hour, and I'm very unconvinced
<niemeyer> dimitern: I'm copy & pasting that conversation somewhere we can refer to, and will mark the issue as not lgtm
<dimitern> niemeyer, ok, so to summarize - you suggest to change the code in createNICs to either use the stand-alone PrivateIPAddress as primary, or scan PrivateIPs to find it
<niemeyer> dimitern: Yes, I suggest doing something minimally sensible and internally consistent..
<dimitern> niemeyer, will that be ok?
<dimitern> niemeyer, ok, will do it then and repropose
<dimitern> niemeyer, in the mean time I'd appreciate other suggestions on the rest of the CL, if you have them
<niemeyer> dimitern: Sent
<dimitern> niemeyer, cheers
<perrito666> wwitzel3: if you have a moment let me know if that mail I sent is accurate, so I can actually write docs with it
<perrito666> wwitzel3: I am a bit rust at mecanography, at least in english
<wwitzel3> perrito666: sure, looking now
<dimitern> niemeyer, updated https://codereview.appspot.com/98430044/ again - this time the code around PrivateIPs in createNICs should be straightforward; also fixed the comment and the error message
<dimitern> hazmat, ping
<hazmat> dimitern, pong
<dimitern> hazmat, quick question: are you comfortable enough with EC2 networking API for VPC to review this? https://codereview.appspot.com/98430044/
<hazmat> dimitern, looking.
<dimitern> hazmat, the changes are mostly in the ec2test testing server, improving it to behave closer to the real AWS wrt running instances with/without specified NICs
<dimitern> hazmat, thanks!
<bodie_> anyone know offhand what tz mgz is in?
<hazmat> dimitern, fwiw. default vpc on an acct starts with subnets in several azs
<hazmat> bodie_, gmt
<bodie_> thanks
<dimitern> hazmat, yes, a subnet per AZ
<hazmat> dimitern, btw. the aws unified cli is a great tool
<dimitern> hazmat, the ec2-xxx one?
<hazmat> can do raw request return, and has wide api coverage
<hazmat> dimitern, no.. the python one.
<hazmat> dimitern,  https://aws.amazon.com/cli/
<dimitern> hazmat, haven't used any aws cli that much - will check it out, thanks
<hazmat> dimitern, they used to random hodgepodge by each team in different languages, they rewrote based on json description of the api with python frontend, much nicer to work with, but also easy to determine exact params on apis
<dimitern> hazmat, that is indeed useful, would've saved me a few hours of wireshark usage :)
<hazmat> dimitern, default-vpc doesn't have a name
<dimitern> hazmat, name? i don't think any one has
<mgz> bodie_: gmt+1 atm
<hazmat> dimitern, nevermind.. i got thrown by 'default-vpc' == Attributes[0].Name .. but that's just  describe account attributes resposne.
<dimitern> hazmat, ah :)
<bodie_> mgz, hua
<hazmat> dimitern, re awscli fwiw.. http://pastebin.ubuntu.com/7497943/
<dimitern> hazmat, interesting - the api docs do not describe most of these
<hazmat> dimitern,  these commands also have builtin detailed parameter help
 * dimitern is going now, might be back later
<bodie_> how does the frontend talk to the client api?
<bodie_> is there an http service or does it go through sockets?
<bodie_> I'm familiar with the CLI client api interface
<bodie_> I suppose that info will be in doc/api.txt :)
<hazmat> bodie_, via an api client that uses websockets
<hazmat> bodie_, parts of the api are over https (charm upload, log download), but most goes over the websocket
<bodie_> so via cmd/*?
<hazmat> bodie_, yes.. cmd/* are cli commands that use state/api  client
<hazmat> bodie_, did you guys know go before you started this project?
<bodie_> I just want to make sure I'm understanding you right, the web frontend also uses that command client?
<bodie_> I'd experimented with and explored it a bit but I wouldn't say I was quite intermediate even really :)
<jcw4> hazmat: some, but this is my first production code Go project
<bodie_> I'm from a C / Java background so it's been a pretty natural fit outside of some of the RPC and interface{} bits
<bodie_> why do you ask?
<hazmat> bodie_, the webfront end uses the a javascript implementation of the same
<hazmat> bodie_, the webfront talks via the websocket protocol using a js client implementation of the api.. there's also a python-jujuclient that exposes the same client api in python.
<hazmat> bodie_, ie. api not cli implementation
<hazmat> bodie_, jcw4  just curious about ramp-up time
<bodie_> I see
<bodie_> so we expose the stateservice endpoints and the js client will have methods written to map to them, I suppose
<jcw4> hazmat: for me the big ramp up time is learning juju, not so much Go
<jcw4> hazmat: although to be fair I do need to learn and internalize some of the Go idioms better
<hazmat> bodie_, yes
<jcw4> <fwereade> natefinch, dimitern: either of you free to review https://codereview.appspot.com/94540043/ ?
<jcw4> mgz?  :-D
<mgz> jcw4: on it
<jcw4> mgz: ta
<mgz> (thanks for the poke :)
<jcw4> :)
<jcw4> mgz: any luck on that? :)
<mgz> jcw4: need to hit m, will do shortly
<jcw4> mgz: yay :)  tx
<jcw4> mgz: looking forward to more fun comments ;)
<hatch> hi dpritchett :)
<dpritchett> hi there
<fwereade> jcw4, bah, I thought you were looking for a review for yourself, and I jumped straight on it, then...hmm
<jcw4_> lol
<jcw4> mgz: is almost done
<mgz> yup
<mgz> only nits really, just got distracted earlier
<jcw4> I like nits
<jcw4> better than "what is this shyte?"
<jcw4> fwereade: any eta on your cleanup refactor landing?
<jcw4> fwereade: should I just make a dependent branch on that one?
<fwereade> jcw4, isn't that the one you just ask mgz for a review of? it's rather in his hands ;p
<jcw4> I thought he was reviewing the one you just finished reviewing of mine?
<jcw4> fwereade: I'm referring to https://code.launchpad.net/~fwereade/juju-core/use-prepare-leave-scope
<jcw4> fwereade: I *think* mgz is reviewing https://code.launchpad.net/~johnweldon4/juju-core/action
<fwereade> jcw4, ehh, who knows, but just above you quoted me asking for a review on mine
<jcw4> fwereade: doh... I just saw the last few digits of the number and assumed...
<fwereade> jcw4, there is definitely something funny about codereview.appspot.com, "043" comes up alarmingly often
<jcw4> fwereade: I'm actually glad... I want that one in soon!  42 +1?
<mgz> I can do both!
<jcw4> woo hoo
<fwereade> haha
 * fwereade cheers
<mgz> fwereade: yours seems to have some overlap with one I already reviewed?
<mgz> or I'm getting confused?
<jcw4> :)
<fwereade> mgz, yeah, it's the followup to that one
<jcw4> mgz: mine is https://codereview.appspot.com/98260043/
<mgz> k
<fwereade> mgz, it uses it and renames a bunch of stuff just to keep your life interesting
<mgz> jcw4: will payload change type or remain map[string]interface{}?
<jcw4> afaik it will remain
<jcw4> mgz: the actual definition is configurable per action, per charm so that's the strongest typing we can achieve imo
<bodie_> mgz, that's where the json-schema comes into play, so we don't know how the charmer will define his or her params spec
<bodie_> I'm working through some potential edge cases right now in my tests
<bodie_> the map[string]interface{} represents a list of arguments, but those arguments could have complex values (is my understanding)
<mgz> bodie_: yup
<jcw4> tx mgz
<jcw4> so mgz : I should add that ErrMatches and you want me to comment Name and Payload in the actionDoc for now?
<mgz> yeah, that would be nice
<jcw4> Okay.  No issues with ErrMatches, just cautious about Name/Payload comment, but since we're not using it now and as you suggest we may want to clarify our thinking on it I'll do that and re-submit
<bodie_> does anyone know the default behavior of json-schema if the "optional" or "required" keys are omitted?
<perrito666> hey what is the criteria for naming things such as addresser
<mgz> no erers
<perrito666> and I am mostly curious about the "er"
<perrito666> mgz: @more
<mgz> try not to make it too weird
<perrito666> ok
<bodie_> lol
<bodie_> uniter is pretty weird
<bodie_> at least there's no errer
<jcw4_> lol
<jcw4> mgz: done
<mgz> jcw4: I think you can go ahead and land then, fwereade any reason not to?
<jcw4> mgz: pending fwereade approval, do I do something to land that branch?
<mgz> jcw4: basically mark the proposal on launchpad as approved and set the commit message (to something similar to the description)
<jcw4> mgz: +1 I'll do that once fwereade approves.  Tx again
<bodie_> woot, I think I have a good MR here.  next stop will be the Validate function so I'm sure there will be more interesting stuff to come
<jcw4> bodie_: yay
<bodie_> https://codereview.appspot.com/94540044
<bodie_> brb
 * jcw4 leaves for a couple hours 
<fwereade> jcw4, can't figure out why name/payload are commented; otherwise looks fine
<fwereade> mgz, thanks for the review, good points
<jcw4> fwereade: per mgz
<jcw4> fwereade: maybe I misunderstood mgz ...
<fwereade> jcw4, I think he meant "describe them with comments" rather than "comment them out"
<jcw4> ROFL
<jcw4> seriously
<jcw4> okay getting up again
<jcw4> doh
<fwereade> jcw4, but to be fair there should probably also be tests that what we getout matches what we put in
<fwereade> jcw4, I might actually recommend getting some sleep ;p
<jcw4> I'll uncomment and then comment and then add tests and then resubmit
<jcw4> ;)
<fwereade> jcw4, cheers
<fwereade> jcw4, I think I'll be in bed by then, I'm afraid
<jcw4> no worries. I'll just let it wait til your morning fwereade , thanks
 * jcw4 mutters to himself... I thought commenting out was a little weird
<mgz> jcw4: doh, yeah I did, sorry, not clear enough
<jcw4> :)
 * jcw4 leaves for real this time
<fwereade> bodie_, https://codereview.appspot.com/94540044/ reviewed
 * fwereade goes for a ciggie, might go straight to bed afterwards, but ping me if you need me in case I don'y
<wallyworld> sinzui: hi, did you have time for a quick catch up?
<sinzui> I will in 6 minutes
<wallyworld> ok
<sinzui> wallyworld, I am ready
<wallyworld> sinzui: https://plus.google.com/hangouts/_/gtg7auay67vchcqri3to3a6g4qa
<sinzui> wallyworld, it that the full url? I am told the party is over
<wallyworld> sigh, let me try again
<wallyworld> https://plus.google.com/hangouts/_/gwbfwjemvxm5p7u4jhegwu2aima
<sinzui> wallyworld, try inviting me. Google doesn't want me you interrupt your party
<wallyworld> ok, invite sent
<waigani> So Actions in juju. What is that all about? what are 'Actions'?
<rick_h_> waigani: the idea of an action is a prepackged script that can be run on demand. The primary example is a mongodb "backup" action that can be triggered and will perform a mongodump and place the backup at a location that might be configurable in the action call.
<rick_h_> waigani: so actions can take some pre-defined input as if script --params
<waigani> rick_h_: thanks. What would trigger the backup (to go with your analogy)?
<waigani> i.e. is this pub/sub or command line calls or ...
<rick_h_> waigani: a command. So you'd juju xxxx mongodb backup --output=/tmp/xxxx.back
<rick_h_> waigani: so command line calls, but available over the api so the GUI will report and allow users to submit them from there as well
<rick_h_> waigani: in the future there might be a place for automating actions based on some external controls
<waigani> rick_h_: right, cool.
<wallyworld> waigani: you were looking at this problem, right?   cannot share a state between two dummy environs; old "dummyenv"; new "dummyenv"
<waigani> wallyworld: I fixed that one. I had to dummy.Reset() in TearDownTest
<wallyworld> waigani: it's still appearing on the bot
<waigani> hmph
<wallyworld> might be in a different place
<wallyworld> waigani: can you take aquick peak and see if it is failing in the place you fixed? http://juju-ci.vapour.ws:8080/job/walk-unit-tests-amd64-trusty/317/console
#juju-dev 2014-05-22
<menn0_> anyone able to do a (likely quick) review: https://codereview.appspot.com/100620046/
<perrito666> menn0: this is is a purely personal thought, but decodeyaml could escalate that error to decodeyamlfrostdout who could escalateit up and so the actual error assertion would be in the test using it
<menn0> perrito666: the reason for doing it the way I did is that it avoids boilerplate in each test method
<menn0> perrito666: there's no backtrace in the failure output is there?
<perrito666> menn0: iirc no
<menn0> perrito666: my Python background is showing :)
<menn0> perrito666: I'll change the tests according to your suggestion.
<perrito666> oh, I am as pythonish as you, worry not
<perrito666> menn0: you might want to get a better review, I am relatively new here too
<menn0> perrito666: understood. thanks for looking.
<wallyworld> morning axw
<axw> hey wallyworld
<wallyworld> sadly jenkins is not very happy, but the failures are seemingly random, one off things
<axw> :(
<wallyworld> for now though, there's a couple of bugs that need looking at for a 1.19.3 release. i plan to grab one after i run some gccgo tests, so if you are at a loose end, feel free to grab one also
<axw> sure
<axw> I was going to fix that utopic one first
<axw> then look at gccgo things
<wallyworld> sure ok
<axw> wallyworld: turns out 1.18 *does* use juju-mongodb on local, but *only* on trusty
<axw> should be trusty onwards
<wallyworld> lol ok
<wallyworld> axw: feel free then to look at gccgo stuff after the mongo fix - first thing we need to do is fully understand how many test failures are left to fix
<axw> nps
<wallyworld> hopefully it won't be too much effort to ensure that bugs in lp reflect the situation - one bug tagged ppc64el / test-failure per issue
<wallyworld> then we can drive that count down to 0
<waigani> wallyworld: ping
<waigani> wallyworld: JujuConnSuite already calls dummy.Reset() in its TearDownTest. That was basically my fix :(
<wallyworld> ah ok
<wallyworld> well there's still something wrong sadly :-(
<waigani> yep
<wallyworld> we can see how often it comes up
<waigani> okay, you know where to fine me
<axw> wallyworld: https://codereview.appspot.com/99410053/ please
<wallyworld> looking
<bodie_> to check whether my dependencies.tsv is suitable for use with gccgo, I can just "make" in the juju-core root directory, right?
<jam> bodie_: are you wanting to check that the dependencies themselves are compatible with gccgo?
<jam> bodie_: AFAICT the Makefile knows nothing about dependencies.tsv, though we should probably fix that
<bodie_> well, I figured if I've aligned deps with godeps, then if I make with gccgo, I should be able to tell whether the build is broken or not
<bodie_> I just don't know how to make with gccgo, which I'm sure is something simple :) just not seeing it
<jam> bodie_: so "make" will only switch to gccgo if you aren't on x86, armel or armhf
<bodie_> yeah, I was just noticing that
<jam> go build -compiler=gccgo launchpad.net/juju-core/...
<bodie_> okay, thanks
<jam> the "..." means recursively
<bodie_> that's simpler than I expected
<davecheney> jam: bodie_ that is an accurate summary
<davecheney> iff you are using trusty you can test with both compilers on your amd64 machine
<davecheney> just use the -compiler=gccgo flag to switch to gccgo
<davecheney> if that passes, that is all that we expect
<bodie_> hmmm
<bodie_> I thought I was on 14.04, but I'm on saucy
<bodie_> I'll have to try on my pc
<davecheney> bodie_: we can probably get the compiler in a backport ppa
<davecheney> but you probably want to upgrade to trusty pretty soon
<davecheney> a. it'll be easier
<davecheney> b. saucy suport expores at the end of JUly ? I think
<davecheney> c. it's the smoothest upgrade i've had, it's very polished from saucy -> trusty
<bodie_> makes sense :) there was some reason I'd set up our dev remote as saucy... I think there was some kind of weird build issue when we were coming onboard
<davecheney> and this laptop has gone from P -> Q -> R -> S
<bodie_> nice
<davecheney> -> T
<davecheney> axw: did you take https://bugs.launchpad.net/juju-core/+bug/1321492 ?
<_mup_> Bug #1321492: provider/openstack: gccgo test failures <gccgo> <ppc64el> <test-failure> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1321492>
<axw> davecheney: I did
<axw> davecheney: working through gccgo bugs now
<davecheney> sorry, just proposed a fix
<axw> no worries, was trivial anyway
<davecheney> axw: i'm going to fix all the trivials in the ppc tests today
<davecheney> then we'll be able to see the underlying problems better
<davecheney> there are, roughtly
<davecheney> 3
<davecheney> one tools related failure
<davecheney> a timeout with the joyent tests
<davecheney> and a runtime or compiler crash
<davecheney> i want to expose those as the only failures
<davecheney> https://codereview.appspot.com/95550045
<axw> ok, cool
<davecheney> the rest are just noise
<axw> davecheney: the one in worker may not be trivial, if that's the runtime/compiler crash you're referring to
<davecheney> yup
<axw> looks like it's related to the receive loop in init()
<davecheney> axw: you take a look at https://bugs.launchpad.net/juju-core/+bug/1303583
<_mup_> Bug #1303583: provider/azure: new test failure <gccgo> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1303583>
<davecheney> i see you've snagged it
<axw> davecheney: already proposed
<davecheney> link ?
<axw> https://codereview.appspot.com/93520047/
<davecheney> axw: do you use lbox propose -bug=NNN
<davecheney> the it links the bug to the branch
<axw> davecheney: I don't because of the milestone thing
<davecheney> so when you get an email that the branch is merged
<axw> I usually link manually, but sometimes forget
<davecheney> you can click through to the bug
<davecheney> fix the milestone and mark it fix comitted
<davecheney> axw: fwiw i tag all these bugs as gccgo and pcc64el
<axw> davecheney: yup, have been looking at the gccgo tag so far
<davecheney> i consider them to be synonyms, but others like to track them independanty
<bodie_> I think my MR is mostly ready to roll, but I'm having some trouble when I try to build with GCCGo.  I'm not sure if I'm doing something wrong.  Can anyone else verify?
<bodie_> https://codereview.appspot.com/94540044/
<bodie_> it adds a few dependencies and one of them doesn't seem to be happy.
<davecheney> bodie_: which one
<davecheney> i'll try
<bodie_> github.com/sigu-399/gojsonpointer
<davecheney> % cover
<davecheney> PASS
<davecheney> coverage: 66.7% of statements
<davecheney> hmmm ...
<davecheney>  % go test -compiler=gccgo
<davecheney> PASS
<davecheney> ok      github.com/sigu-399/gojsonpointer       0.047s
<davecheney> lucky(~/src/github.com/sigu-399/gojsonpointer) %
<davecheney> works for me
<davecheney> bodie_: if you are using saucy
<davecheney> you will have gccgo-4.8
<davecheney> which is, sadly, not up to the task
<bodie_> I'm on Trusty on my PC here
<davecheney> bodie_: can you show what you see, use paste.ubuntu.com
<davecheney> or install pastebinit
<bodie_> yeah
<bodie_> http://paste.ubuntu.com/7500153/
<bodie_> perhaps it's because I'm using zsh...
<davecheney> oh
<bodie_> negative
<bodie_> same issue using bash
<davecheney> that is a differnt problem
<davecheney> here is what's happened
<davecheney> 1. you did go get -u -v launchpad.net/juju-core/...
<davecheney> which is going to fetch all the _current_ juju dependencies
<bodie_> right
<davecheney> then you switch to your branch
<davecheney> which essentially has new dependencies
<davecheney> which is why godeps is whinging
<davecheney> the simplest solution for today would be
<davecheney> go get -u -v github.com/sigu-399/gojsonpointer
<bodie_> I figured I should switch before doing a godeps so that I get the same version I had in my code
<davecheney> and the same for the other two new deps
<davecheney> then godeps should be happy
<bodie_> oh.... but it's complaining because my code is an older version of core...?
<bodie_> wouldn't ... oh
<bodie_> I see
<bodie_> could I go get lp:~binary132/juju-core/charm-actions ?
<axw> davecheney: I got to the bottom of the issue in worker
<axw> davecheney: seems that closing a channel twice in gccgo will abort hte process, even if you recover the error
<axw> panic*
<davecheney> axw: hmm
<davecheney> is that worth a bug upstream ?
<davecheney> one nice thing about gccgo is gdb actuially works
<axw> davecheney: http://play.golang.org/p/eUJw-e1GRJ
<axw> davecheney: yeah I think so
<davecheney> axw: two secs
<davecheney> bodie_: are you going ok for the moment ?
<waigani> how do I patch name, username and id of the current user for testing? I've tried s.PatchEnvironment("USER", "admin001"), but in the test switch still prints "jesse"
<davecheney> axw: what happens if you make the call stack a bit longer
<davecheney> i suspect that the main function, or main gorotuine might be a bit special in gccgo
<bodie_> yes, I'm about to head to bed actually (UTC-4)
<axw> davecheney: it wasn't in the main function in the test, that's just a minimal repro
<bodie_> would the right way to do things from the beginning be to fetch via lp:~binary132/juju-core/charm-actions directly?
<davecheney> axw: http://play.golang.org/p/VNezaP7ADp
<davecheney> bang on
<axw> davecheney: FYI it's in worker/simpler.go, the Kill method
<axw> we can work around it, but a upstream bug is warranted
<bodie_> hrm, gccgo not found on path on trusty either
<davecheney> axw: do you want to raise it ?
<davecheney> or i can do it
<axw> I can
<davecheney> cool, excellent
<davecheney> surely double closing a channel is a bug
<davecheney> and using recover to cover for that is a horrid smell
<axw> yeah
<davecheney> axw: raise the bug on golang.org/issue
<axw> will do
<davecheney> i don't think the issuetracker on the gofront-end project is used as much
<bodie_> can I just apt-get install gccgo?
<bodie_> will that be suitable?
<davecheney> bodie_: yes
<bodie_> okay, that's nice
<davecheney> bodie_: actually, you'll have to do
<davecheney> apt-get install gccgo-4.9
<bodie_> uh oh, already started build.  do I need to clean
<bodie_> ?
<bodie_> er, it looks like I may have installed gccgo-4.9 by default (since trusty)
<davecheney> gccgo -v
<davecheney> % gccgo -v 2>&1 | tail -n1
<davecheney> gcc version 4.9.0 20140405 (experimental) [trunk revision 209157] (Ubuntu 4.9-20140406-0ubuntu1)
<bodie_> http://paste.ubuntu.com/7500172/
<davecheney> bodie_: run godeps -u dependencies.tsv
<davecheney> if it whinges, fix the problem and run it again
<axw> davecheney: filed https://code.google.com/p/go/issues/detail?id=8070
<axw> davecheney: I can fix it in our code, unless you're already doing that
<davecheney> added some code
<davecheney> sorry, tags
<davecheney> axw: if you're there have a crack
<davecheney> i'm doing another ppc test run to find some more issues
<bodie_> all righty
<bodie_> there we go, seems to be working
<bodie_> thanks
<bodie_> and built!
<davecheney> win
<davecheney> is the bot running ?
<davecheney> it's been a long time since I marked that change as accepted
<davecheney> approved
<bodie_> bot?
<davecheney> bodie_: the commit landing bot
<davecheney> aww crap
<davecheney> i see the problme
<bodie_> I need to lbox propose again, right?
<davecheney> always
<davecheney> propose early, propose often
<jcw4_> if in doubt.... propose again
<bodie_> then propose s'more!
<bodie_> yeah, my last propose was -wip
<bodie_> and there it should be
<bodie_> https://codereview.appspot.com/94540044
<davecheney> wallyworld: axw what are we going to do about the joyent tests ?
<wallyworld> which bit?
<wallyworld> the time taken to run them?
<wallyworld> i'll got pull requests waiting to be merged by upstream
<wallyworld> that will fix the execution tim
<wallyworld> is there anything else?
<davecheney> that's basically it
<davecheney> they run so slowly on ppc they always timeout
<wallyworld> yeah. i did fix it over a week ago
<wallyworld> just need to get my changes to upstream libs merged in, hopefully this week
<wallyworld> davecheney: axw : so, do you have a feel for how long we need to get the tests running under gccgo and ppc (assuming joyent ones are fixed)? 1 week? 2 weeks?
<davecheney> wallyworld: today i feel confident that we can close this off in a week
<wallyworld> o/
<wallyworld> \o/
<davecheney> ask me tomrrow, we might have a different answer
<wallyworld> lol ok
<davecheney> but it's looking pretty good so far
<wallyworld> is there anything outside our control? what's the exposure to compiler issues?
<davecheney> ask axw, i think it's managable
<axw> wallyworld: so far seems fine
<wallyworld> compiler issues?
<axw> wallyworld: only one gccgo-specific issue, but that's because our code is a bit crap
<axw> brtb
<wallyworld> ok, so we can fix our code hopefully
<axw> yes I am fixing that one atm
<wallyworld> axw: looking at your address polling branch - it seems that DNSName is not really used with the changes made
<davecheney> http://paste.ubuntu.com/7500164/
<davecheney> some of these have fixes waiting to be landed by the bot
<axw> wallyworld: it's not meant to be anymore
<wallyworld> axw: yeah, i had heard that i thin, just wanted to confirm. so we should follow up and remove it i guess
<wallyworld> davecheney: that pastebin doesn't look too bad
<axw> wallyworld: oh sorry, misunderstood
<axw> wallyworld: if it's not used anywhere, definitely
<axw> wallyworld: I thought it might still be used in status
<wallyworld> axw: it doesn't *seem* to be at first glance. not sure about status
<axw> but it wouldn't... it uses machhine's public address now
<wallyworld> yeah
<axw> wallyworld: I will follow up and wipe it out
<wallyworld> only issue is we would want to give people te dns name to connect to
<wallyworld> rather than an ip address
<axw> we can do that still, with Addresses
<axw> there's an address type
<wallyworld> ah yes, true
<bodie_> 'night all
<axw> night
<wallyworld> seeya
<wallyworld> axw: ok, i'll get a 2nd opinion on getting rid of dnsname and we can nuke it
<axw> cool
<axw> davecheney: https://codereview.appspot.com/98480046 fixes worker
<davecheney> man, that is a nasty code smell
<davecheney> why isn't this code using a tomb ?
<davecheney> this is exactly what a tomb is for
 * axw shrugs
<axw> tomb seems a little heavy here, it's pretty trivial code
<davecheney> lol
<jcw4> wallyworld: dumb question... jc.DeepEquals is not the same as gc.DeepEquals right?
<jcw4> wallyworld: if I want to use jc.SameContents what import is that?
<jam> jcw4: IIRC it differs based on whether []slice(nil) == []slice()
<wallyworld> jcw4: same end result, but jc.DeepEquals gives far better errors, so use that
<jam> is the empty slice the same as a nil slice
<wallyworld> plus the slice difference, yeah
<jcw4> jam, wallyworld cool thanks
<wallyworld> gc.DeepEquals is considered deprecated
<jcw4> what is the import for jc.*
<jcw4> juju-core?
<wallyworld> launchpad.net/juju-core/checkers i think
<jcw4> k, tx!
<wallyworld> no
<wallyworld> bad memory
<wallyworld> github.com/juju/testing/checkers
<jcw4> wallyworld: I see it now... it's in a handful of tests already
<wallyworld> yeah
<jcw4_> hmm;  gc.DeepEquals is okay for maps, but use jc.SameContents for slices?
<jcw4_> it looks like even gc.DeepEquals is intended for slices...
<jcw4_> is there any similar comparison tool for maps?
<jcw4_> Or do I need to compare the keys and values separately?
<dimitern> jam, hey
<jam> morning dimitern
<dimitern> jam, so everybody else than niemeyer is chickening out of giving me an lgtm for this goamz branch, 3rd day in a row, and i'm starting to get frustrated.. https://codereview.appspot.com/98430044/
<wallyworld> jcw4: DeepEquals works for slices sure, but it fails if order is different
<dimitern> and he even gave me a not lgtm, but i fixed what we discussed, and i'll be waiting for him to appear later today and hopefully approve it
<wallyworld> SameContents doesn't care about order, so you can think of it as treating a slice like a set
<wallyworld> well, except sets can't have dupes
<jcw4> wallyworld: I'm getting an error using SameContents with two maps
<wallyworld> use DeepEquals
<jcw4> wallyworld: ok
<jcw4> wallyworld: tx!
<wallyworld> np
<davecheney> http://paste.ubuntu.com/7500334/
<davecheney> axw: getting closer
<axw> nice
<jcw4> mgz, fwereade : I think this is the last go round: https://codereview.appspot.com/98260043
<jam> dimitern: I have some comments, but adding reviews just adds more chefs to the pot :). I still feel like you're in the best position to be comfortable with it, and if you are, LGTM
<dimitern> jam, thank you!
<rogpeppe> mornin' all
<dimitern> hey rogpeppe
<rogpeppe> dimitern: yo!
<dimitern> rogpeppe, just back from holiday? or you're hanging in other channels usually?
<dimitern> :)
<rogpeppe> dimitern: just back from holiday
<rogpeppe> dimitern: was in Cypress for two weeks
<dimitern> rogpeppe, excellent!
<rogpeppe> dimitern: yeah, it was!
<davecheney> night all
<dimitern> 'night davecheney
<dimitern> jam, so no standups today, just the x-team meeting @14.30 utc?
<wallyworld_> axw: i will add another test before landing - you're right, it's messy to do
<axw> wallyworld_: okey dokey
<wallyworld_> i was being lazy :-)
<axw> wallyworld_: I just noticed there's a card "TestEnsureAdminUser is still broken"
<axw> wallyworld_: did CI fail again?
<wallyworld_> yeah, the latest test run has it failing, and also several over the past few days
<wallyworld_> not every time
<wallyworld_> but often enough
<wallyworld_> the other failures all seem to be one off
<axw> wallyworld_: latest run of "walk tests" on trusty is buggered, the machine is out of space
<wallyworld_> this is the one i looked at just before
<wallyworld_> http://juju-ci.vapour.ws:8080/job/walk-unit-tests-amd64-trusty/330/console
<axw> ok
<wallyworld_> plus a few others over the past few days
<wallyworld_> it seems much better but not quite fully fixed
 * wallyworld_ -> soccer
<axw> later
<jam> dimitern: right now we didn't have the team standup because of the x-team meeting, I might reintroduce it, but we hadn't talked about it, so I'll skip it today
<dimitern> jam, ok, do any of us need to be at the x-team?
<jam> dimitern: the goal is that if you *can* be there you should, as we'd like people to go to 2 out of 3 of the meetings.
<jam> I will not be able to be there tonight, as it is my 10-year anniversary weekend
<jam> but I'll probably be going to most of them.
<dimitern> jam, right, I'll try to attend then - it is one of the poor-quality-voip-conference calls right?
<TheMue> morning
<jam> dimitern: ah sorry, this is the whole team juju meeting at 18:00 UTC
<jam> dimitern: sorry, the x-team meeting at 14:30UTC is not one you have to be at
<jam> we need a smattering of people from Juju to be there.
<dimitern> jam, ah, ok
<jam> dimitern: the one at 18:00UTC is the replacement for the weekly whole team meeting that we used to have at 10:00UTC on Thursdays
 * TheMue is looking forward to the 1800UTC meeting, see some of the new US colleagues.
<dimitern> jam, it'll be more difficult to attend the one at 18 utc - it is the only one this week and will rotate next week at another time iirc?
<jam> I think 18:00 is around 20:00 for you, right?
<jam> dimitern: right, next week it will be 8 hours later
<jam> (16 hours earlier, depends on POV)
<TheMue> jam: it is 20 CEST, yes
<dimitern> jam, :) so +8h each week?
<jam> dimitern: right
<dimitern> hey TheMue
<jam> we'd like people to make it to 2 out of 3
<TheMue> dimitern: heya
<dimitern> jam, ok, so 18 utc, 2 utc and 10 utc
<TheMue> jam: but donât expect me to join the 200UTC meeting :D only if I cannot sleep
<TheMue> dimitern: yep
<jam> TheMue: yep.
<jam> It is at 6am here, so I can *just* make it.
<dimitern> jam, why 2 out of 3 ?
<jam> dimitern: it is an approximate, but hopefully we can make that work
<TheMue> 6am is hard enough *yawn*
<jam> we want people to see each other as much as we can
<jam> dimitern: it isn't hard and fast, but we used to see everyone every week, but can't actually do that with 21 people
<dimitern> jam, i see - that way at least for one of the meetings everyone will make an effort to attend at an inconvenient time
<jam> it is possible that we could make it at the start of and end of some people's work day
<jam> but it is too hard to actually try to do the numbers to actually figure out what time of day that would be
<jam> this way at least 1 per 3 weeks should definitely be in your normal TZ
<dimitern> or perhaps make 4 meetings instead of 3, that way the intended goal is easier to achieve i think
<dimitern> each one 6h apart
<dimitern> jam, was that an option?
<jam> it wasn't ever brought up, it would be something to consider
<voidspace> morning all
<jam> dimitern: the next patch in my API versioning saga: https://codereview.appspot.com/97630045
<dimitern> jam, looking
 * jam takes his dog to the kennel for the weekend, bb < 1hr
<perrito666> morning
<voidspace> perrito666: morning
 * jam is back
<jam> morning perrito666 and voidspace
<voidspace> jam: morning
<mgz> have I actually got internet right now...
<wallyworld_> mgz: yes you have :-) you ok for 1:1?
<mgz> now google is responding, I'm there
<voidspace> gah, the srvRoot authorizer methods aren't tested
<voidspace> I've added a new one
<voidspace> so testing it means working out how to test those methods at all
<voidspace> unless they're tested elsewhere
<voidspace> jam: ping
<jam> voidspace: ?
<voidspace> jam: state/apiserver/root.go
<voidspace> jam: srvRoot represents a client's connection
<jam> well, it represents the API
<voidspace> jam: and it is the main implementation of the Authenticator interface
<jam> for clients or agents
<voidspace> jam: the Authenticator methods on it aren't tested directly that I can find
<voidspace> and in fact most tests use the FakeAuthorizer
<jam> voidspace: I'm pretty sure they are all tested indirectly, but yes, there are no direct tests of srvRoot behavior
<voidspace> I've added a (simple) method to this, and the only test is against the FakeAuthorizer (which also now has this method)
<jam> voidspace: so things that test the APIServer directly use FakeAuthorizer, but there are a bunch of client-side tests that go end-to-end that have a real srvRoot in the middle.
<jam> voidspace: but if you can write some direct tests, I'd be happier to see them.
<voidspace> jam: so the only way to make that work would be to create a client that is neither representing a MachineAgent nor a UnitAgent
<voidspace> jam: and we struggled to do that as it seemed the way to get the API was "OpenAPIAsMachine"
<jam> voidspace: we have 3 types of entities that access the API, Client, MachineAgent and UnitAgent
<voidspace> right, but how to get a test client entity wasn't obvious
<jam> voidspace: so opening the API as the admin user is not a machine or unit agent
<voidspace> ah, ok
<jam> voidspace: things that access the "Client" facade are not machine or units either.
<dimitern> jam, reviewed
<voidspace> if a direct test is preferable (srvRoot is a private type) can you suggest how I would do that or point me at something that gets at it for testing
<jam> voidspace: I know of nothing that tests it directly, and I'm working on that right now for some API versioning tests
<voidspace> ah right
<jam> *I* like to have direct Unit tests of each layer
<jam> not everyone felt the same
<jam> namely, the people who implemented this the first time
<voidspace> yep, me too - it seems wrong to only indirectly test
<jam> felt that it was better to test from the actual client code
<voidspace> because those tests can be changed or removed as it's not obvious what they're testing
<jam> dimitern: thanks
<voidspace> unless you add a note
<voidspace> it's also not obvious where to go to find them / update them
<fwereade> voidspace, yeah, +1 to direct tests
<voidspace> well, +1 in theory
<fwereade> voidspace, it helps with interface sanity more than anything else I'm aware of
<voidspace> private types that are hard to construct make it difficult
<voidspace> obviously not built with testability in mind
<fwereade> voidspace, +1 also to tiny weeny little packages that don't have internal layers that really need tests but don't have them
<jam> voidspace: fwiw I'm currently overhauling srvRoot a *lot* so you may not want to poke at it too much.
<voidspace> jam: ok, it's literally a one line method
<voidspace> jam: maybe it's not warranted as there's only one use for it - if I see the need again I'll add it
<voidspace> jam: and by then it should be easier to test due to your work
<jam> sgtm
<jam> fwereade: don't forget the team leads call in 5min. If you're available, I have a quick question wrt API that I'd like to run by you (if you can join early)
<voidspace> fwereade: in terms of making GetRsyslogConfig bulky, it takes no params
<voidspace> fwereade: so is it just a question of renaming to Configs and returning a slice of results?
<fwereade> voidspace, shouldn't we be asking for the rsyslog config for ourselves, though,much as we watch the config for ourselves?
<jam> dimitern: https://codereview.appspot.com/97570051 implements the "Does this Facade return the exact type I expected"
<voidspace> fwereade: no, units and state servers all need to log to "all the state servers"
<voidspace> fwereade: so we want "the global config for everything" in all cases
<jam> voidspace: fwiw ^^ removes some of the coupling so you can create one with apiserver.TestingSrvRoot(State) which would let you write the tests you wanted. (once all this stuff lands)
<voidspace> jam: ah, cool
<fwereade> voidspace, the answers for "where should machine 7 log to" and "where should machine 9 log to" may happen to always be the same
<fwereade> voidspace, but I think they're different questions
<fwereade> voidspace, and I think it's a bit nicer to keep the object of the sentence implied by the API call explicit ratherthan implicit in the connection
<fwereade> voidspace, any evidence of sanity there?
<voidspace> fwereade: it sounds like unnecessary complexity in the name of trying to be consistent
<voidspace> fwereade: the *explicit task* we are achieving is "make sure everything logs to all the state servers"
<voidspace> that's the goal of HA - present a single world view with redundancy
<voidspace> so to provide an api that looks like we could have multiple world views actually breaks that model
<voidspace> (conceptually)
<fwereade> voidspace, it's more about (1), yes, consistency, but (2) trying to make it easy to build the API out of easily composable chunks: and methods with implicit params don't really help there
<fwereade> voidspace, I may have to think about that for a mo, but I'm not sure I agree
<voidspace> I defer to you of course, but I'm a bit wary of the "make all API calls bulk calls even when not appropriate" approach
<voidspace> I'd rather we think about whether it makes sense for each endpoint (and yes bearing the future in mind because API churn is a pain)
<voidspace> but I still defer to you :-)
<voidspace> but we *want* a single logging endpoint for the user - and we want HA to provide redundancy for that *behind the scenes*
<dimitern> jam, thanks, will look in a bit
<voidspace> so it seems to me that the model here is a single call
<fwereade> voidspace, it is certainly less compelling than it originally was, because at long last we're getting api versioning, so the cost and complexity of changing an API is much smaller
<fwereade> voidspace, there are 2 questions in play though
<voidspace> versioning isn't a silver bullet - you still have to maintain the obsolete version, which can be a real nuisance
<voidspace> so better to get it right :-)
<fwereade> voidspace, one is, should inform the server who's sending the logs when we ask where they should be sent, and I think that's a yes
<voidspace> "should (?) inform the server" ?
<fwereade> voidspace, "we" or "the client" or whoever
<fwereade> voidspace, "where do all logs go always" != "where do $entity's logs go", even if the answers are currently the same
<voidspace> that's the one I'm disputing I think - we explicitly want all logs to go to the same place
<voidspace> doorbell - back in 2
<wwitzel3_> hello
<voidspace> wwitzel3: hello
<fwereade> voidspace, the other is "is the cost/benefit of mandating all-bulk calls likely to pay off" and I *think* they actually are: the consistency argument is stronger than you might think, because people do what they see us doing already; and IMO enough calls are worth bulking that it's a win to provide a consistent API that always allows it even if we don't always use it
<voidspace> fwereade: if we start sending different logs to different places, we'll be breaking the HA model - which is, all logs are always available and it doesn't matter which state server you contact
<voidspace> fwereade: right, I certainly don't want to fight strongly on that point
<fwereade> voidspace, strawman: additional external logging targets for units of particular services
<voidspace> fwereade: so, the args should be a slice of entities and the result a slice of configs?
<fwereade> voidspace, yes please
<voidspace> fwereade: hmmm...
<voidspace> fwereade: ok
<fwereade> voidspace, my argument is not that it always makes sense; it's that it often does, and that the bonus from consistency pushes it past the line to just-always-do-this
<voidspace> fwereade: ok
<voidspace> fwereade: for bulk calls, is the convention that, assuming no "global error" (causing the whole call to fail) happens, you return "result, nil" - where result is a collection (.Results) with an Error field for individual errors
<rogpeppe> voidspace: yes
<voidspace> so even where this one call with one error, you return nil for the error - but result.Results[0].Error has the real error
<voidspace> cool
<rogpeppe> jam, fwereade: hiya, i was talking earlier with frankban about moving store out of core, and we think that it's a reasonable principle that nothing outside juju-core imports juju-core's TestBase. does that seem right to you?
<fwereade> rogpeppe, agreed; but ideally we'd be moving stuff to github.com/juju/testing rather than duping or  dropping functionality
<rogpeppe> fwereade: yup
<fwereade> rogpeppe, perfect
<rogpeppe> fwereade: but TestBase in particular has very core-specific functionality
<rogpeppe> fwereade: but packages like filetesting will move too
<fwereade> rogpeppe, yep, +1 to that
<jam> rogpeppe: in general we'd rather have things not depend on code inside juju-core, and have things nicely pulled out into smaller modules. I'd be willing to be a bit pragmatic about it
<rogpeppe> jam: i hope we can be good about it. in particular, i very much hope that we can make a strictly non-cyclic dependency graph between repositories.
<jam> fwereade: https://codereview.appspot.com/100460045/ is the one patch along the way that didn't actually get an LGTM, I think I've finished the work on it that I wanted to do.
<jam> dimitern: ^^ if you want to give that a look as well
<jam> it is *mostly* a mechanical application of the previous patches.
<voidspace> rogpeppe: I didn't notice that reply came from you
<voidspace> rogpeppe: thanks, and hi
<rogpeppe> voidspace: hiya :-)
<perrito666> hey rogpeppe wb
<rogpeppe> perrito666: ta!
<jam> fwereade: https://codereview.appspot.com/97630045/ is a followup that Dimiter reviewed, and https://codereview.appspot.com/97570051/ is one that needs reveiew
<dimitern> jam, reviewed https://codereview.appspot.com/97570051/
<jam> dimitern: thanks
 * perrito666 wonders why ctrl+w doesnt work on the screen he is looking at instead of the one with focus
<voidspace> perrito666: heh, I do that all the time
<voidspace> hah, I wondered why I suddenly had all these failures on our rsyslog branch
<voidspace> wwitzel3: ping
<voidspace> wwitzel3: revision 2755 (most recent one) of your rsyslog branch merges a branch of mine that was a dead end
<voidspace> wwitzel3: I restarted the work in a different branch
<voidspace> wwitzel3: can you back out that revision?
<voidspace> wwitzel3: I have further commits so it's harder for me to do
<voidspace> in the meantime
 * voidspace lunches
<jam> voidspace: "bzr merge -r 2755..2754" should do what you want, fwiw
<wwitzel3> voidspace: was eating breakfast and getting Jessa off to work, ping me when you're back, I don't even see that revision on my branch of rsyslog-api
<tasdomas> fwereade, hi
<tasdomas> fwereade, I've started working on moving juju-core/cmd to a separate package (github.com/juju/cmd probably) - what is the process of creating a new juju repo on github?
<fwereade> tasdomas, I'm a bit surprised by cmd... oh, yeah, the store commands use it now
<fwereade> tasdomas, are you in the team on github?
<voidspace> wwitzel3: http://bazaar.launchpad.net/~wwitzel3/juju-core/009-ha-rsyslog-api/revision/2755
<voidspace> wwitzel3: is that not your branch? https://code.launchpad.net/~wwitzel3/juju-core/009-ha-rsyslog-api
<wwitzel3> voidspace: huh, I just don't see it locally I guess
<voidspace> heh
<voidspace> wwitzel3: the command that jam gave is the one you should run
<wwitzel3> voidspace: it is indeed my branch, but locally when I look at the log, I see 2747 as the latest
<voidspace> it would be more painful for me
<voidspace> hah
<wwitzel3> voidspace: ok, try now? ..
<wwitzel3> not sure if I should cross my fingers or plug my ears
<wwitzel3> both? ..
<voidspace> hah
<voidspace> wwitzel3: now I see the latest revision as 2747 (!?) but with the offending revision still in it
<voidspace> I believe
<voidspace> when I merged I got no changes anyway
<wwitzel3> ok what is the offending revision # now?
<voidspace> 2747
<wwitzel3> voidspace: ok, pushed up the revert of that (I hope)
<voidspace> wwitzel3: ah, I see what happened I think
<voidspace> wwitzel3: you intended to merge in my changes removing the obsolete check that Port and CACert had changed
<voidspace> wwitzel3: this was the branch that should have been merged
<voidspace> https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-shortcut-removal/+merge/220105
<voidspace> wwitzel3:  lp:~mfoord/juju-core/ha-rsyslog-shortcut-removal
<wwitzel3> ok, merged and pushed
<voidspace> wwitzel3: and rsyslog worker tests are passing again for me!
<voidspace> and I don't think we lost anything...
<wwitzel3> great :)
<voidspace> hmmm...
<voidspace> except this on your branch
<voidspace> wwitzel3:
<voidspace>     Diff: 12160 lines (+2221/-2600) 262 files modified (has conflicts)
<wwitzel3> great
<voidspace> wwitzel3: you may need to merge trunk and resolve conflicts
<voidspace> :-o
<wwitzel3> yep, doing that now
<wwitzel3> 11 conflicts encountered.
<wwitzel3> awesome
<voidspace> weird
<voidspace> wwitzel3: files you've not touched you can resolve with --take-other
<wwitzel3> well that is handy :)
<voidspace> wwitzel3: and in fact, the conflicts are pretty much all in files we've not touched
<wwitzel3> voidspace: pushed
<voidspace> wwitzel3: thanks
<wwitzel3> voidspace: we should pair after this call
<voidspace> wwitzel3: ok
<voidspace> anyone recognise this error: "missing agent version in environment config"
<voidspace> in provider/dummy
<voidspace> actually comes from testing.AssertStartInstance
<perrito666> voidspace: your config file is being generated without version ?
<voidspace> perrito666: sure, but it's from a test
<voidspace> sorry, yeah - test failure error
<voidspace> ah, never mind
<voidspace> I have build failures
<wwitzel3> hah
<wwitzel3> priorities
<wwitzel3> :P
<voidspace> in the openstack provider
<voidspace> I wonder if dependencies.tsv wasn't updated on trunk
<voidspace> if I merge from trunk it says nothing to do
<voidspace> but if I switch to trunk and run godeps it *does* update some dependencies
<voidspace> wwitzel3: you still have an 11000 line diff
<voidspace> wwitzel3: I think you somehow collapsed a bunch of changes into that single revision
<voidspace> or something
<voidspace> somehow our branch has undone a load of changes
<voidspace> goddammit
<wwitzel3> well, I can undo the merge I did earlier
<wwitzel3> where I jumped back a revision
<voidspace> right, that would bring back the unwanted changes from my branch
<voidspace> let's look at the revision history and see if we can work out where we want to go back to
<voidspace> wwitzel3: it seems like you want to go back to 2746 (which I thought was what you'd done previously)
<voidspace> and then merge trunk in
<voidspace> which should leave just our changes
<voidspace> ah
<voidspace> so the problem is that trunk has already been merged into your branch
<voidspace> so the reverse merge *undoes* trunk revisions
<voidspace> merging trunk again says "Nothing to do" because it sees those revisions in the merge history
<voidspace> mgz: ping
<dimitern> niemeyer, hey
<niemeyer> dimitern: Heya
<dimitern> niemeyer, I fixed what we discussed in https://codereview.appspot.com/98430044/, can you please take a look if it's good to land?
<niemeyer> dimitern: Will do, thanks
<bodie_> fwereade / others, looks like all the libs in question are apache2-licensed in their source code but LICEN(S|C)E[.txt] isn't included in the two deps for gojsonschema.  should I open MRs or is it sufficient to have the license in the files themselves?
<voidspace> wwitzel3: I'm going to try and fix it by branching trunk and merging from revision 2747 of your branch
<wwitzel3> voidspace: ok
<voidspace> wwitzel3: that *seems* to have worked
<voidspace> wwitzel3: we may need to abandon your one - the history seems to be irrevocably screwed now (?)
<voidspace> wwitzel3: I'm in moonstone by the way
<niemeyer> dimitern: Does it make sense to have the same domain name for different private IPs?
<niemeyer> dimitern: Ah, nevermind
<niemeyer> dimitern: The loop breaks out on the first entry
<dimitern> niemeyer, right
<niemeyer> dimitern: Okay, LGTM
<dimitern> niemeyer, thanks!
<voidspace> wwitzel3: https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-good/+merge/220667
<voidspace> wwitzel3: https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-good/+merge/220669
<wwitzel3> voidspace: second link looks right
<bodie_> someone help me understand what fwereade means by "d" in these comments? https://codereview.appspot.com/94540044/diff/60001/charm/actions_test.go
<perrito666> bodie_: delete
<voidspace> wwitzel3: https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-2/+merge/220603
<mgz> voidspace: hey, sorry missed ping earlier
<bodie_> o/ mgz
<mgz> hey bodie
<voidspace> mgz: we have a screwed bazaar branch that we can't recover
<voidspace> mgz: we've mostly worked round it now and are abandoning the branch
<voidspace> mgz: but I thought you might be able to help
<mgz> voidspace: fallback is just pull out the changes in the tree into a fresh branch normally
<voidspace> mgz: right, that's what I'm doing
<voidspace> mgz: except it's changes from three different branches we had consolidated
<voidspace> and it was the consolidation that screwed us
<mgz> what fun
<voidspace> and then unfortunately there was a merge back the other way - making it hard to back out changes without also reverting a load of trunk changes
<voidspace> which is what we did
<voidspace> and why we're screwed
<voidspace> we reverted 11000 lines of changes from trunk
<voidspace> and the history shows those changes as already merged - so we can't bring them back again
<wwitzel3> they probably weren't needed anyway
<voidspace> yeah, probably
<voidspace> wwitzel3: this is now the full consolidated branch
<voidspace> https://code.launchpad.net/~mfoord/juju-core/ha-rsyslog-good/+merge/220669
<mgz> voidspace: losing the history shouldn't matter much, as we're going to be losing history anyway next week
<voidspace> mgz: really, we're just dumping into github rather than converting the repo?
<hazmat> mgz, losing history?
<hazmat> what?
<hazmat> easy to convert with history
<mgz> voidspace: we're converting, but that's a lossy conversion
<voidspace> oh, ok
<voidspace> we'll just pretend it's git's fault
<voidspace> I mean, I'm sure nothing like this will ever happen once we're using git
<voidspace> git is not *at all* renowned for mangling repos into completely unusable states
<perrito666> voidspace: nope, git users are though
<sinzui> voidspace, perrito666, bodie_ is anyone planning to land a branch in the next hour? I want to take CI offline to upgrade how it builds test packages
<perrito666> sinzui: nope
<jcw4_> sinzui I'm hoping to in a couple hours
<bodie_> hoping to get one in asap, but struggling a little with tests right now.  I can push it back
<voidspace> perrito666: git will happily chainsaw your repo into an unusable mess
<voidspace> and smile whilst doing it
<voidspace> sinzui: an hour should be fine
<perrito666> voidspace: but will blame me
<perrito666> so technically its my fault
<voidspace> sure...
<voidspace> I'll happily blame you too if you like :-D
<sinzui> bodie_, does the landing bot hate you?
<perrito666> voidspace: usually when I use git, which I like, I feel like barely above using diff+patch+ftp
<bodie_> I assume I would know if it did
<voidspace> heh
<bodie_> and I don't, so it must not
<voidspace> fwereade: we've created a new CL as we managed to "break" the other one
<voidspace> fwereade: https://codereview.appspot.com/91630045/
<voidspace> fwereade: that addresses your previous review comments as discussed (I believe)
<bodie_> jcw4, mgz, I'm not certain I'll be able to have a voice session at 1600 -- michelle will be getting home any minute now and i failed to communicate the plan to her so I might have to do text this time
<jcw4_> I think we're preferring IRC right now anyway
<mgz> that's fine
<jcw4_> #jujuskunkworks to reduce the noise here though?
<mgz> lets
<jcw4> fwereade: I'm assuming this is what you wanted with the err handling and return of newActionID: http://bazaar.launchpad.net/~johnweldon4/juju-core/action/view/head:/state/unit.go#L1343
<voidspace> mgz: ping
<fwereade> jcw4, yeah, looks good
<jcw4> fwereade: ta
<voidspace> fwereade: so I assume that when we release 1.20, what is currently "upgrades/steps118.go" will be changed to "upgrades/steps120.go"
<bodie_> fwereade, https://codereview.appspot.com/94540044
<bodie_> hopefully this is satisfactory :)
<bodie_> I made a few tweaks to the regexp that I'd not caught until I added the param name tests you'd requested
<fwereade> voidspace, no? those are needed for anything pre-1.18 to be valid for 1.18; they don't need to be run again from 1.18 to 1.20; others might, and they'd be 1.20 ones (or I guess 1.19 ones, I hope we'd discover them earlier than 1.20...)
<mgz> voidspace: hey
<voidspace> fwereade: but we don't have steps116.go, I thought we only supported 1.18  -> 1.20
<voidspace> mgz: hi, I was going to ask what I'm asking will
<bodie_> right -- so the deps I added with gojsonschema have apache v2 license in the source files but not repos themselves
<voidspace> I'd better my changes
<bodie_> do I need to get the author to put LICENSE.txt in the repos or are we good?
<voidspace> *revert my changes
<fwereade> voidspace, the idea is that we *should* be able to support arbitrary upgrades, and that we run them from src->dst in order
<coreycb> I have a local provider deployment of mysql to kvm that hangs in pending state.  any idea if that is fixed by bug 1317197?
<_mup_> Bug #1317197: juju deployed services to lxc containers stuck in pending <oil> <juju-core:Fix Committed by axwalk> <https://launchpad.net/bugs/1317197>
<voidspace> fwereade: ok, I'll leave the syslog port upgrade step in place
<coreycb> sinzui, maybe you know ^
<sinzui> coreycb, sorry, the bug is unrelated. That bug is about a lock dir used when working with lxc templates
<coreycb> sinzui, ok need a bug for my issue?
<sinzui> coreycb, common reasons for localhosts stuck in pending is a bad cloud image cache. I don't know how kvm manages that
<sinzui> coreycb, I think we do need a bug https://bugs.launchpad.net/juju-core/+bugs?field.tag=kvm doesn't describe the issue
<jcw4> fwereade, or mgz : if you get a chance to review -- https://codereview.appspot.com/98260043/
<coreycb> sinzui, thanks, I opened bug 1322281
<_mup_> Bug #1322281:  local provider deployment of mysql to kvm hangs in pending state <juju-core:New> <https://launchpad.net/bugs/1322281>
<sinzui> thank you
<natefinch> Ahh, there's nothing like starting the day at 4am with a puking 11 month old.
<rick_h_> natefinch: all uphill from here
<natefinch> haha, yeah
<wwitzel3> I've ended the night with a puking 30 year old before, if that makes you feel any better :P
<rick_h_> wwitzel3: yourself doesn't count :P
<wwitzel3> I'd puke, thats fiscally irresponsible
<wwitzel3> I don't
<jcw4> wwitzel3: I don't know... with a 30 yro you can laugh at them
<jcw4> and... walk awy
<wwitzel3> I feel like you can laugh at an 11 month old too, doubt they would remember
<jcw4> haha
<wwitzel3> oh, yeah, well the walk away part .. not so much
<natefinch> At least I didn't have to catch her puke in my hands this time, that was nice.
<wwitzel3> I see as a parent your definition of nice has to change slightly
<natefinch> lol yup
<jcw4> haha yep
<voidspace> natefinch: morning
<natefinch> voidspace: morning :)
<voidspace> hmmm... so adding an attribute to the slice of immutableAttributes in config.go isn't sufficient to make it actually immutable
<voidspace> I thought that was a bit hopeful
<voidspace> ah, wait
<voidspace> I didn't go install first
<voidspace> maybe it still is
<voidspace> and it looks like it is
<voidspace> ERROR cannot change syslog-port from 6514 to 3030
<voidspace> that's what I wanted :-)
<natefinch> nice
<voidspace> ship it!
<voidspace> I should probably test it
<bodie_> rogpeppe, you around?
<voidspace> but that can wait until the morning
<voidspace> g'night all
<voidspace> EOD
<bodie_> o/
<alexisb> natefinch, ping
<natefinch> alexisb: howdy
<alexisb> given you are the lead that is awake :)
<natefinch> heh
<alexisb> can you take a look at 2 bugs real fast and get them to triaged state
<alexisb> most likely they are both critical given they are blocking teams but I want someone to take a quick look first
<alexisb> 1322302 and 1322281
<natefinch> ok
<alexisb> thanks
<bodie_> natefinch, does that mean you can give some input on whether we should use a dep of relatively unknown quality from github?  rogpeppe indicated some concern
<natefinch> bodie_:
<natefinch> bodie_: er yep
<natefinch> bodie_: what's the repo?
<bodie_> https://github.com/xeipuuv/gojsonschema
<bodie_> AFAIK, it's the go-to JSON-Schema implementation for Go -- the only other github repo for gojsonschema is a fork of it which was recently merged
<bodie_> I could loop you into the MR email thread if you'd like
<natefinch> bodie_: I think I saw part of it at leasty
<bodie_> about licensing, right
<bodie_> I should let you take care of what alexisb wanted, but let me know if you have any input
<natefinch> bodie_: I'll take a look at it.
<natefinch> alexisb: I made the first one high, since slow is not the same as "doesn't work"
<natefinch> alexisb: the second one is "doesn't work" so I made it critical
<natefinch> alexisb: both need investigation by dev or QA to confirm it's reproducible, though
<alexisb> natefinch, thank you, can you make sure to comment on any needs that block forward progress from juju-core in the bug?
<natefinch> alexisb: sure
<perrito666> natefinch: here is the everybody is happy patch https://codereview.appspot.com/92320043
<natefinch> perrito666: nice
<waigani> morning all
<waigani> how do you find the current juju user?
<arosales> wallyworld_: et'all are folks seeing issues on HP Cloud
<arosales> mbruzek: is seeing hex output in juju status
<wallyworld_> oh really? 1.18.3?
<arosales> wallyworld_:  1.19.3 I think
<mbruzek> I am seeing some very strange output
<mbruzek> http://paste.ubuntu.com/7503161/
<mbruzek> juju status
<wallyworld_> ok, so crappy error message :-( looks like the security group quota may have been exceeded perhaps
<wallyworld_> let me have a look to see if i can figure it out
<wallyworld_> mbruzek: so, looking at the hp console, it seems your machine 1 reports an " Error(spawning) "
<wallyworld_> so at first glance it looks like juju tried to start the machine and got an error back from hp cloud
<wallyworld_> and reported the error in juju status but in a crappy way
<wallyworld_> you could try destroying the machine in juju
<mbruzek> wallyworld_, I will do that now
<wallyworld_> and possibly manually deleting from hp cloud
<wallyworld_> perhaps file a bug about the poor error message
<mbruzek> wallyworld_, this is my second time seeing these problems, I believe they will persist after I destroy-environment and re-bootstrap
<wallyworld_> mbruzek: it seem though that some machines start ok?
<mbruzek> correct, but there always seems to be one that does not.
<wallyworld_> i tried clicking on the console log in the hp cloud admin page page it just hung
<mbruzek> $ juju destroy-machine 1
<mbruzek> ERROR no machines were destroyed: machine 1 has unit "cassandra/0" assigned
<mbruzek> Do I need to --force?
<wallyworld_> you either need to destroy the unit/service first or i think there's now a --force option?
<wallyworld_> perhaps the all-machine.log on the start server will give clues
<wallyworld_> you could juju debug-log in another terminal window as you try and deploy charms
<wallyworld_> and then see what it says if the machine provisioning on the cloud fails
<wallyworld_> not start server, state server
<wallyworld_> arosales: you meeting with the joyent folks today?
<mbruzek> wallyworld_, I already had juju debug-log running in another window!
<wallyworld_> what did it say as machine 1 failed to start?
<wallyworld_> pastebin it if you want
<arosales> wallyworld_: they defered till next wed
<mbruzek> Is there somewhere all-machine.log is on my system?  I am cut and pasting this to pastebin and it is not very fast
<arosales> wallyworld_: any traction on your pull requests?
<wallyworld_> arosales: no :-(, still pending. we really need them merged
<wallyworld_> mbruzek: the log file is on the state server
<mbruzek> wallyworld_, Here is the top of the juju debug-log command after I started the deployement
<mbruzek> http://pastebin.ubuntu.com/7503212/
<arosales> wallyworld_: ugh. ok I'll ping again. Really they shouldn't have to meet with me to get the pull request merged. I thought they were taking a look.
<mbruzek> wallyworld_, machine-0: 2014-05-22 19:48:53 ERROR juju.provisioner provisioner_task.go:417 cannot start instance for ma
<mbruzek> chine "1": cannot run instance: failed to run a server with nova.RunServerOpts
<wallyworld_> arosales: part of the issue is that the ppc tests keep timing out because of the issue, and we are under prssure to get those sorted out
<wallyworld_> arosales: maybe they are looking at them :-) i'll check later today to see if they're still pending
<wallyworld_> mbruzek: so, it's saying that the cloud has reported an error starting the instance but doesn't really say what the error is (eg error code)
<arosales> wallyworld_: ok I'll add that into my ping.
<wallyworld_> thank you :-)
<wallyworld_> mbruzek: did you just destory environment?
<mbruzek> wallyworld_, Yes I wanted to try a simpler case.
<wallyworld_> the console shows just a state server machine now which is green, but also an old machine 1
<wallyworld_> and i think that's the issue
<wallyworld_> the old machine 1 isn't getting removed because it's broken
<mbruzek> I am on the hpconsole but not seeing that information what page are you on?
<wallyworld_> https://console.hpcloud.com/compute/az-1_region-a_geo-1/servers
<wallyworld_> if the old machine 1 is there, juju can't start another one with the same name
<mbruzek> OK I see
<wallyworld_> that would be a logical explanation for the error
<mbruzek> Yes
<mbruzek> I just bootstrapped so I should only have 1 server out there.  Let me destroy and clean that one up manually
<wallyworld_> yep, i reckon that will solve your issue (i hope)
<arosales> wallyworld_: pinged again re joyent pull requests
<wallyworld_> awesome, thanks
<mbruzek> wallyworld_, Terminating juju-hp-mbruzek-machine-1 seems to have done the trick.  My deployment is looking to be in better shape.
<wallyworld_> great :-)
<wallyworld_> my guess there was an error starting it in the first place and hence a subsequent desroy env couldn't kill it
<mbruzek> I agree
<wallyworld_> hazmat: you around?
<hazmat> wallyworld_, am now
<hazmat> waigani, what's up
<waigani> hazmat: hey :)
<hazmat> doh.
<hazmat> :-)
<waigani> ah you wanted wallyworld_
<wallyworld_> hazmat: with dan's bug report about lxc staying in pending a long time - that's just because the first machine has to download the lxc image
<hazmat> wallyworld_, hmm..
<wallyworld_> we added cloning for them so the next one started fast
<hazmat> wallyworld_, so let me pose it differently
<hazmat> i agree though that's possibly the primary issue there
<wallyworld_> we could do something like allow juju to copy an image across at bootstrap
<hazmat> wallyworld_, if i have containers already running (via manual).. it still takes up to a minute for them to show as started
<hazmat> during which they go through various odd permutations in status agent-state: (started)
<wallyworld_> ok, that's a bit different
<hazmat> it feels like the pinger/heartbeat is a bit schizo there
<wallyworld_> that seems like a sepaate but legitimate issue
<hazmat> wallyworld_, the fix there would be lxc template hooks respecting proxy settings
<hazmat> re dan's issue
<hazmat> that would decrease the time
<hazmat> alternatively downloading the image while giving feedback to the user
<wallyworld_> adding feedback is on the todo list
<hazmat> ie.. we don't call out 'install' hook running.. we just leave things in pending
<wallyworld_> i'm not sure if fetching the template honours the proxy settings
<wallyworld_> you think it doesn't?
<hazmat> its just doing a wget, but not sure the env variables propagate
<hazmat> from juju's invocation
<wallyworld_> ok, we can check that
<wallyworld_> there's no good short term soluation though sadly
<wallyworld_> apart from the proxy thing
<hazmat> wallyworld_, any feedback to the user via status would be helpful to users re long running ops
<wallyworld_> agree +100, but it's not necessarily trivial, it's on the todo list for sure
<hazmat> ack
<wallyworld_> i hope that dan's group can bear the pain of the initial download and use cloning for speedier deployment for the thers
<wallyworld_> others
<wallyworld_> i'll comment on the bug
<wallyworld_> hazmat: seems also bug 1322281 which was just raised is the same thing
<_mup_> Bug #1322281:  local provider deployment of mysql to kvm hangs in pending state <cloud-installer> <juju-core:Triaged> <https://launchpad.net/bugs/1322281>
<wallyworld_> "hangs" might mean slow template download
<wallyworld_> or no http egress except via proxy perhaps
<wallyworld_> so i think if we check the proxy thing, that may be what we can do short term
<wallyworld_> i'm sort a hoping we do not currently use the proxy settings so we have something to fix
<hazmat> wallyworld_, its more than proxy.. its respect  a cache
 * wallyworld_ -> breakfast bbiab
<hazmat> although typically the same
<hazmat> most of thse are orange boxes
<wallyworld_> which cache?
<wallyworld_> a squid cache?
<wallyworld_> which i guess the proxy would point to
<hazmat> wallyworld_, yeah.. squid-deb-proxy cache.. also orange boxes have full archive mirrors
<hazmat> which we don't configure/use
<wallyworld_> ok, i think we can/should add in proxy support for getting template
<wallyworld_> that should be a simple fix
<hazmat> sounds good
<wallyworld_> thanks for the input
<wallyworld_> now i am really off to eat and caffeinate myself
<wallyworld_> hazmat: i've done some checking. the lxc-create which calls wget is invoked via golang's exec.Command(). this does pass through env vars like http_proxy. so obstently, if they were to set http-proxy and /or apt-htty-proxy in their env config, those should be passed through and used when the template is fetched
<wallyworld_> do you know if they set http-proxy in their env?
<hazmat> wallyworld_, talking to kirkland atm about setting up a transparent proxy on the orangebox, so everything hits the proxy cache.
<hazmat> wallyworld_, i've heard different reports from different folks,  i don't have a solid baseline for analysis
<wallyworld_> hazmat: great, so i hope/expect that will solve the bug
<wallyworld_> can you let me know how you get on?
<wallyworld_> so i can schedule juju work if needed
<wallyworld_> even without a transparent cache, setting http-proxy in env config should work
#juju-dev 2014-05-23
<hazmat> wallyworld_, looking at the file dan uploaded .. there's  a wierd 5 minute gap http://paste.ubuntu.com/7502559/
<hazmat> where the agent restarts itself
<hazmat> oh.. right after it reconfigures logging to WARN.. hence no log
<sinzui> Is anyone about to help me diagnose a local kvm issue on a ppc64 machine in the lab?
<sinzui> http://pastebin.ubuntu.com/7503638/
<davecheney> sinzui: my gut says you are missing a package
<davecheney> juju-local does not depend on the entire set of packages required to boot ubuntu images
<sinzui> thank you davecheney I will investigate
<davecheney> (based on my experiences 6 weeks ago)
<davecheney> something like ubuntu-cloud-template
<davecheney> hang on
<davecheney> i'll stop guessing
<sinzui> davecheney, I did install the list of packages it suggested when I tried it first, but I understand the list might no be accurate
<davecheney> possibly you need lxc-templates
<sinzui> davecheney, lcv works fine. CI now tests lxc on ppc64el for every rev we change
<davecheney> ok, i don't know jack about kvm
<davecheney> it didn't even work on ppc when I was using it
<davecheney> as all the machines i had access to were themselves inside kvm
<davecheney> which might be the problem
<sinzui> davecheney, yeah. I know kvm in kvm works. This is my last hope to get this working. I can see no cloud images were downloaded. Maybe there aren't ppc64el cloud-images suitable for this case
<davecheney> wallyworld_: axw http://paste.ubuntu.com/7503679/
<davecheney> getting closer every day
<wallyworld_> farking awesome
<axw> davecheney: nice
<axw> hmm, what's that replicaset one
<davecheney> replica set stuff is always busted
<davecheney> sorry to whoever wrote it
<davecheney> its really unreliable
<axw> yeah I know, I've been working on making it more reliable
<wallyworld_> also on golang go :-(
<axw> apparently still not
<axw> davecheney: I have a fix for the utils one
<wallyworld_> davecheney: TestSecureClientNoAccess failure - that's because we're trying to connect to a non local address
<axw> just about to approve
<axw> wallyworld_: it's cos of $http_proxy
<wallyworld_> ah right
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1322392
<axw> I've got a CL to clear it
<_mup_> Bug #1322392: utils: test failure due to proxy <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1322392>
<wallyworld_> great
<davecheney> ^ sorry, just raised this, it's probably a due
<davecheney> dup
<axw> yeah I opened one yesterday
<wallyworld_> do we have a bug for TestConstraintsValidator?
<sinzui> wallyworld_, no
<wallyworld_> ok, that's an easy fix anyway
<axw> wallyworld_: can you please take a look at https://codereview.appspot.com/100730043/ to make sure I haven't buggered up the intent of the test
<wallyworld_> sure
<axw> davecheney: seen this yet? https://code.google.com/p/gosmith/
<wallyworld_> axw: looks fine
<axw> wallyworld_: thanks
<davecheney> axw: yeah, dmitry has been ripping the compiler a new one with that
<davecheney> it's great
<axw> now it's llgo's turn
<sinzui> axw, Do you have any insights into this? http://pastebin.ubuntu.com/7503710/
<axw> sinzui: 'fraid not
<sinzui> thank you axw
<axw> about to get on a meeting, I will take a look at the kvm code later to see if there's somethign else you can run
<sinzui> axw: I found the issue.
<sinzui>  uvt-kvm: error: no images found that match filters ['release=trusty', 'arc
<sinzui> h=ppc64'].
<davecheney> wap, wap, wah
<davecheney> hang on
<davecheney> is that 'cos the arch is wrong
<davecheney> it should be ppc64el, or ppc64le
<sinzui> davecheney, exactly what I suspect.
<sinzui> I see ppc64el release
<sinzui> http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.json
<sinzui> davecheney, but lxc works so WTF
<davecheney> sinzui: we have several functions inside juju that translate btween the various aliases
<davecheney> it is possible lxc knows more than kvm
<davecheney> or even that the lxc images have a different naming scheme
<sinzui> I could test this by creating an alternate stream, but my weekend has started. I am not keen to start a project at this hour
<sinzui> davecheney, before the ports were opened of for stilson-06, I hacked up a https server and the hosts file on the machine to help juju find the tools
<axw> sinzui: any chance I could get direct access to the trusty CI machine that runs the unit tests? there's one in particular that I cannot reproduce anywhere else
<sinzui> axw, absolutely. one moment while I install your key into one of them
<sinzui> axw, who else is in tanzanite?
<axw> sinzui: wallyworld_ and mgz
<wallyworld_> yes please
<axw> davecheney: should be a couple less test failures on trunk now
<davecheney> axw: i'm working on a mega patch that will remove 90% of the uses of DeepEquals in our tests
<davecheney> IMO they are almost all wrong
<axw> mmkay
<davecheney> for example
<davecheney> c.Assert(manifest, jc.DeepEquals, set.NewStrings(dummyManifest...))
<davecheney> this is a set
<davecheney> order is unimportant
<axw> set is a map underneath I think
<axw> so DeepEquals will work anyway?
<davecheney> logically your testing for set equality
<davecheney> not ordering
<sinzui> wallyworld_, axw, and mgz: I sent an email about accessing all the ppc64 env I manage
<wallyworld_> thanks
<sinzui> wallyworld_, axw. I reported bug 1322401 about the issue. I will work on other tests next week
<_mup_> Bug #1322401: kvm on ppc64el cannot fine images <ci> <kvm> <local-provider> <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1322401>
<axw_> sinzui: sorry if you've been talking to me, my client didn't realise it was disconnected
<sinzui> axw, I wasn't all is fine. and I didn't say the bug had to be fixed this week
<axw> okey dokey
<sinzui> oh, axw, or wallyworld_ I was supposed to rebuild 1.19.2 to make ppc64el and arm64 tools. Since ppc64el has so many names, should I always make the tool with a specific name? when the script sees "ppc64el" debs, make a "ppc64" tools?
<axw> ppc64 is the arch name we use in the juju code
<sinzui> axw, to restate your brief remark. Juju knows about ppc64. It will look for tools by that name. when the assemble script sees ppc64el, ppc64le, or power8, it will make a ppc64 tools from the first one, and ignore the rest.
<axw> ah right. sounds reasonable to me
<sinzui> axw, I think that is right since that is the rule I hacked together to deploy the jenkins slave to stilson-06. I didn't want to put it in production without a dev having a chance to say It was dangerously wrong
<axw> sinzui: davecheney would probably know better, but AFAIK ppc64el is the right dpkg arch to match against
<sinzui> axw, that is the common one.
<sinzui> axw, http://ports.ubuntu.com/pool/universe/j/juju-core/
<sinzui> ^ powerpc is still there, there was a ppc64le for a few weeks
<axw> powerpc eh... fairly sure we don't care about that
<sinzui> The assembly script makes a tool for each one. There are not tools for ppc64
<sinzui> maybe I should make an extra tool each time a ppc64el is found
<sinzui> davecheney, I /think/ juju is looking for ppc64 tools, not ppc64el tools. Should I be making I make tools named arm64 each time I find a new ppc64el deb?
<sinzui> davecheney, I think I need to do that because the streams I made to deploy jenkins-slave to stilson-06 didn't work with ppc64el
<davecheney> axw: I think you've fixed it https://bugs.launchpad.net/juju-core/+bug/1322392
<_mup_> Bug #1322392: utils: test failure due to proxy <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1322392>
<davecheney> sinzui:  I don't know
<davecheney> this keeps changing
<axw> davecheney: excellente
<davecheney> afaik when you use --upload-tools the name of those tools were -ppc64el.tar.gz
<sinzui> davecheney, users aren't supposes to use it
<davecheney> sure
<sinzui> davecheney, since I have an env with tools that were never published, and my computer is amd64, upload-tools is usless
<davecheney> but that leads me to think that the the published name of the tools should also have the same exension
<sinzui> so
<sinzui> https://juju-ci.s3.amazonaws.com/tools/streams/v1/com.ubuntu.juju:released:tools.json
<sinzui> ^ I created a stream of just the tools we run ci on, including a tools for ppc64. I chose that label because ppc64el didn't work
<davecheney> sinzui: https://bugs.launchpad.net/juju-core/+bug/1290654
<_mup_> Bug #1290654: juju must not rely on the output of uname -m <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1290654>
<davecheney> possibly related
<sinzui> davecheney, yep, suffered that bug with i386 too
<sinzui> I have a hack in the run-unit-tests jobs to ensure i386 is matched correctly
<sinzui> I think the safest thing to do is to make two tools for each ppc64el
<axw> sinzui: do you have a fixed ec2 instance for running the amd64/trusty unit tests? or do you start/stop instances each time?
<sinzui> axw, neither, we start an instance with an ami, then run apt-get update (and possibly add ctools)
<axw> sinzui: each run?
<sinzui> yes
<axw> k
<axw> which AMI? I want to try to reproduce this EnsureAdminUser test failure
<axw> cannot repro anywhere
<sinzui> axw, http://bazaar.launchpad.net/~juju-qa/juju-ci-tools/trunk/files < run-unit-test is for amd64 and i386
<axw> thanks
<sinzui> axw, run-unit-tests m1.xlarge ami-018c9568
<sinzui> axw, some of the tests now have ami's embedded in them. It was a problem when many bit rotten while we were in vega
<axw> alright, will give that a shot...
 * sinzui updates each test in http://juju-ci.vapour.ws:8080/ to show what AMIs are used
<sinzui> axw, This is the whole test script http://pastebin.ubuntu.com/7503961/
<axw> sinzui: thanks
<wallyworld_> davecheney: i'm about to fix the TestConstraintsValidator test failure for gccgo in the joyent tests
<wallyworld_> are you alreayd doing that one?
<axw> wallyworld_: can you please take a look at https://codereview.appspot.com/96540047/
<wallyworld_> sure
<wallyworld_> axw: take my +1 with a grain of salt since i got it wrong the first time
<axw> sure :)
<wallyworld_> axw: so it seems like you've changed it to how it should have been done first time round when it ws originally written - take use tomb
<axw> yep
<axw> nfi why it wasn't to start with
<wallyworld_> nate wrote it originlly and roger +1ed :-)
<wallyworld_> i think
<axw> it was fine, but only on gc
<wallyworld_> yeah :-(
<wallyworld_> axw: remind me - we reduced the mgo dial timeout to 30 seconds because it used to be 10 minutes and sometimes tests wouldn't notice a connection had disappeared and so waited 10 minutes for the timeout to expire, right?
<axw> wallyworld_: hey sorry, otp
<axw> just a mo
<axw> wallyworld_: it was reduced to 30s because it shouldn't need to be any longer than that; other connections should time out and retry. Yes, some tests were failing to connect for 10m and then the test got killed
<axw> so we wouldn't get useful output
<wallyworld_> axw: so i might have to add a retry look around it because of the bug where a user's slow machine can't start mongo fast enough and bootstrap dies
<axw> wallyworld_: around what?
<wallyworld_> MaybeInitiateMongoServer
<wallyworld_> or an upstream caller
<axw> wallyworld_: ahh.. right, in bootstrap it doesn't retstart
<axw> restart*
<axw> bugger, sorry I missed that detail
<wallyworld_> in bootstrap if it can't connect after 30s yeah it dies
<wallyworld_> np, simple enough fix
<axw> wallyworld_: you could just set the timeout explicitly in bootstrap
<axw> rather than use the default
<wallyworld_> could do
<wallyworld_> i think a loop and smaller timeouts are perhaps better
<wallyworld_> maybe not, doesn't really matter for this situation i don't think
<axw> I don't think it matters. whatever works and is elegant
<axw> with preference to the former :)
<wallyworld_> sure
<axw> I have nfi about this EnsureAdminUser thing. I can't reproduce on the same instance type as used in CI
<axw> also trying a faster instance type on ec2
<axw> no luck
<axw> I'm going to try crafting up a separate program soon to cut down the time initialising mongo, speed up the test cycle
<wallyworld_> davecheney: have you done a recent gccgo test run? on my machine, i get compiler errors and seg faults
<wallyworld_> lots of tests pass though
<wallyworld_> davecheney: when you get a chance could you do another test run and pastebin the results?
<menn0> review please (fairly easy one): https://codereview.appspot.com/94710046
<axw> wallyworld_: I may go back on what I said before about longer timeout vs. retry
<axw> wallyworld_: if we use a retry loop, it may help with these i/o timeout errors
<wallyworld_> that's sorta what i was thinking
<wallyworld_> i'll add a loop
<wallyworld_> bbiab, gotta go get kid from train station
<axw> k
<axw> wallyworld_: interesting thing- on CI, the agent and agent/mongo package tests take a long time to run (>15s). on my machine, they take ~1s each
<axw> on another run, they took closer to 40s each
<axw> on CI
<axw> just a guess, but maybe the machine is still busy initialising when the tests start up
<wallyworld_> could be
<wallyworld_> maybe we delay starting the tests
<wallyworld_> axw: do you know how dave has been running his tests? on amd64 using -compiler=gccgo or on ppc?
<axw> on ppc I think
<rogpeppe> hi all,
<axw> hey rogpeppe
<rogpeppe> axw: yo!
<rogpeppe> axw: how's tricks?
<axw> not too shabby
<axw> how are you? have a good break?
<TheMue> morning
<axw> morning TheMue
<rogpeppe> TheMue: hiya
<axw> wallyworld_: I don't think it's worth expending any more effort on these i/o timeout bugs until we try adding a delay at the beginning of CI
<wallyworld_> agreed
<axw> wallyworld_: I can extend timeouts and so on, but I think it'll be neverending
<axw> I'll pick something else up for now
<wallyworld_> 1305014?
<axw> yep
<wallyworld_> or 1322281
<wallyworld_> we are sort of expecting more info on those top 2 cards to come in
<wallyworld_> the cloud init logs
<wallyworld_> davecheney: you around per chance?
<rogpeppe> axw: yeah, had a great break thanks
<rogpeppe> axw: two weeks on cyprus
<axw> nice
<wallyworld_> fwereade: hiya, i've done another pass over that doc if you have a chance to look, no rush
<fwereade> wallyworld_, cheers
<wallyworld_> fwereade: i have standup soon and then will be back from soccer in about 4 hours if you needed to chat or whatever
<fwereade> wallyworld_, nothing hugely springs to mind, but I'm here if you want me :)
<voidspace> morning all
<wallyworld_> fwereade: i meant about the doc
<wallyworld_> once you read it
<fwereade> wallyworld_, I have lots of other stuff I have to figure out today so I might not get there in time, but yu never know your luck
<wallyworld_> np, let's tak monday
<wallyworld_> mgz: you around for standup?
<mgz> joining
<perrito666> morning
<mgz> mornin' perrito666
<TheMue> mgz, perrito666: morning
<wwitzel3> hello
<voidspace> my connection is ropey today
<rogpeppe> anyone know if the bot automatically pulls new deps? i *thought* it did now, but this seems to indicate otherwise: https://code.launchpad.net/~frankban/juju-core/update-testing-package/+merge/220772
<mgz> rogpeppe: it doesn't
<frankban> mgz: what's the process for updating a git repo in tarmac? should we ask Curtis?
<mgz> frankban: I'll do it
<frankban> mgz: thanks
<frankban> mgz: since we want to migrate some packages to github, we will need to update the dependencies file several times. I am also taking notes about the migration process. It would be nice to have the bot to pull changes automatically, but for the time being, is it ok if I include something like "ask mgz or sinzui to pull the last revision of the updated package in the tarmac machine before landing the dependencies.tsv c
<frankban> hange"?
<mgz> frankban: this is all changing, this week
<frankban> mgz: as part of the git migration?
<mgz> yeah
<mgz> frankban: for now, bugging me is fine
<frankban> mgz: cool thanks, please ping me when I can re-submit the branch
<rogpeppe> anyone have strong feelings about breaking the utils.Apt* functions into a separate package?
<natefinch> I'm a big fan of small, focused package, so seems like a good idea to me
<rogpeppe> also potentially moving the proxy-specific functions in osenv into their own package too
<rogpeppe> fwereade: ^
<wwitzel3> +1 to small, focused packages
<rogpeppe> i'm focusing more on dependency hygiene than small packages per se
<rogpeppe> i don't mind larger packages if they're coherent
<TheMue> coherent packages, small files
<TheMue> e.g. focussing on individual types
<rogpeppe> i don't mind large files either, if the contents of each file makes sense as whole
<rogpeppe> s/as whole/as a whole/
<wwitzel3> I find it tought to build large packages with large files if I am making things coherent
<fwereade> rogpeppe, +1 to dependency hygiene, small files, and small packages
<rogpeppe> i think the most important thing is that for every exported thing, it's easy to say "that thing definitely *belongs* in this package"
<fwereade> rogpeppe, I think you have a greater tolerance for the latter two than many of us
<rogpeppe> fwereade: possibly. i like to follow the examples demonstrated by the go stdlib
<rogpeppe> pwd
<fwereade> rogpeppe, I get antsy past a few hundred lines in a file, and past a few files in a package, tbh
<fwereade> rogpeppe, mybrain has to start swapping things out at around that point
<rogpeppe> fwereade: i think there's a lot to be said for intra-package hygiene too.
<rogpeppe> fwereade: (for avoiding that situation)
<fwereade> rogpeppe, definitely, but it's *really* hard to get that right in a large team -- package boundaries don't get violated accidentally, but it's easy to miss those sorts of screwups within a package
<wwitzel3> rogpeppe: I'm hesitant using the stdlib (of any language) as a guide for how you should craft packages that aren't part of the stdlib.
<fwereade> rogpeppe, sometimes they still get violated deliberately because someone doesn't understand the situation, but it's usually more obvious that the exposed surface has just changed significantly
<rogpeppe> fwereade: with too many packages, it's very easy to have packages with no proper raison d'etre - they tend to have fuzzy boundaries and end up having random collections of not-really-related functionality in them
<voidspace> axw: ping, you still around
<fwereade> rogpeppe, IME that's more a problem with large packages -- if the bias is to create small packages at the drop of a hat, existing packages keep their focus
<fwereade> rogpeppe, and if two packages really are so closely related as to be better as one, it's an awful lot easier to merge than to split
<rogpeppe> fwereade: it's all trade offs as usual :-)
<fwereade> rogpeppe, yeah :)
<rogpeppe> lc
<rogpeppe> fwereade: tbh i've found it pretty easy to split packages in the past
<dimitern> TheMue, standup?
<natefinch> wwitzel3: the standard library for Go is pretty good standard to go by for coding standards.  Occasionally there are functions which are hard for mere mortals to follow, but in general, it's a pretty good template.
<voidspace> natefinch: morning
<natefinch> voidspace: morning
<wwitzel3> natefinch: I guess I've just found that things done for the sake of the "stdlib" are not always the best conventions to be used in your own packages.
<wwitzel3> natefinch: that said, my exposure to the Go stdlib at this point is still limited, so I'm still open to the idea
<wwitzel3> natefinch: and I've always pissed and moaned that the stdlib for a language wasn't the defacto go to place to learn good practices for building packages, so hopefully as I get more exposure to the Go stdlib, it will be able to promote it as the language that does just that.
<natefinch> wwitzel3: I hope so :)
<wwitzel3> fwereade: you have a moment for a hangout?
<fwereade> wwitzel3, sure
<fwereade> wwitzel3, moonstone?
<wwitzel3> fwereade: yep
<rogpeppe> trivial automatically-made change to review anyone? https://codereview.appspot.com/97670044
<rogpeppe> natefinch, fwereade, wwitzel3, voidspace: ^
<rogpeppe> (it just sorts all imports standardly)
<axw> voidspace: sorta here
<natefinch> rogpeppe: LGTM
<rogpeppe> natefinch: ta
<axw> voidspace: (re rsyslog review) that's why I suggested to remove the upgrade step; the immutability of syslog-port will cause the upgrade-step to fail now
<axw> voidspace: although... there is a way to open a state connection that doesn't validate environ config changes
<voidspace> axw: so, when an upgrade is done is the code upgraded first - or the upgrade steps run *then* the code update
<voidspace> axw: hmmm... the upgrade must be run with the current codebase
<voidspace> axw: so it can't work
<axw> voidspace: it runs with the new code
<voidspace> axw: do we want to *aim* to allow a 1.16 -> 1.20 upgrade
<voidspace> if so we need to bypass validation
<voidspace> if not we need to remove those upgrade steps
<axw> voidspace: I think that was the aim, yeah. so I guess we should leave it in but reopen state without the policy
<voidspace> axw: how do I do that? :-)
<fwereade> voidspace, axw: we certainly aim for that to work, yes
<voidspace> so yes, with my current mp the upgrade tests fail
<axw> fwereade voidspace: can we just leave a TODO for that for now?
<voidspace> for the obvious reason
<axw> ah
<voidspace> I should have run that before proposing of course
<fwereade> axw, voidspace: well, we want 1.20 relatively soon, so I'm not sure how good an idea that it
<voidspace> axw: how hard is it to bypass validation for an upgrade step?
<axw> voidspace: not sure, just taking a look now
<voidspace> axw: thanks, I appreciate it
<axw> voidspace: the upgrade steps get an agent.Config, so can do state.Open
<axw> voidspace: state.Open has an optional policy argument, which you'd want to pass in as nil
<axw> voidspace: hrm, but I'm not sure if it'll work, because syslog-port is provider-independent
<rogpeppe> wow
<rogpeppe> ok  	launchpad.net/juju-core/provider/joyent	388.185s
<axw> voidspace: I don't think it's going to work
<axw> rogpeppe: yeah, wallyworld_ has a pull request to the joyent packages to fix that
<voidspace> axw: hmm
<axw> apparently it's due to repeated crypto calls
<rogpeppe> axw: it wouldn't be the first time...
<axw> voidspace: so I don't see an easy fix here. it's possible to modify the settings docs in mongo directly, which is a little bit horrible
<voidspace> axw: instead of adding it to the immutableAttributes, can we have custom validation?
<voidspace> axw: which will allow setting it to the default but disallow all other values
<voidspace> axw: so the upgrade will work
<axw> voidspace: actually I may be wrong, I think it *does* skip all config validation. I'll investigate a bit more...
<voidspace> axw: let me look at the failing test, it may be failing when it tries to set the value *before* running the upgrade
<axw> ehh, could do that, but I'd rather not have super special case things
<voidspace> axw: sure
<voidspace> nope, it's running UpdateRsyslogPorts that fails
<rogpeppe> fwereade, natefinch, wwitzel3, voidspace: another fairly trivial review. just a code move. please ignore the dubious import formatting - i wanted to avoid extra diffs from merging the above CL before it landed. https://codereview.appspot.com/100730044
<rogpeppe> this is just a code move (plus some associated renaming)
<voidspace> natefinch: where is your tool for stripping log output from test runs?
<axw> voidspace: opening without the policy allows you to bypass the validation altogether
<axw> provider-specific or not
<axw> just tested it in the state package
<natefinch> voidspace: go get github.com/natefinch/nolog
<voidspace> natefinch: thanks
<axw> voidspace: so, in the upgrade step- state.Open(agentConfig.StateInfo(), state.DefaultDialOpts(), nil)
<voidspace> awesome
<voidspace> I was just wondering how to get the Info
<natefinch> rogpeppe: lgtm
<rogpeppe> natefinch: thanks!
<natefinch> I love the way Go namespacing means you don't have to put "Apt" in the name of all the types and functions
<rogpeppe> fwereade, dimitern, natefinch: ISTM that we should probably allow a single `import "foo"` statement in a file so that we can work nicely with goimports, which will format things that way if it adds a single import.
<rogpeppe> fwereade, dimitern, natefinch: it would make life easier for ourselves, and it seems a fairly trivial thing
<natefinch> rogpeppe: yeah, that seems fine.  not wrestling with standards that conflict with goimports seems like a good idea in case anyone wants to use it.
<rogpeppe> natefinch: lots of people run goimports on save
<natefinch> rogpeppe: yep.  Certainly for things like this where it doesn't really matter either way, if one makes goimports happy, that seems a reasonable standard.
<voidspace> my lunch date is here
 * voidspace lunches
<dimitern> rogpeppe, goimports leaves it alone if you reformat it
<dimitern> rogpeppe, but i'm fine either way for a single import, as long as it's consistent across the code base
<rogpeppe> dimitern: yeah, but if it adds a new import when there was none already, it doesn't add the ()
<rogpeppe> dimitern: i'm happy allowing either form, tbh
<rogpeppe> dimitern: i don't think total consistency matters that much here
<jcw4> fwereade, mgz - any chance of getting https://codereview.appspot.com/98260043/ reviewed and landed?
<fwereade> jcw4, approved, keep half an eye on it to make sure it lands
<perrito666> fwereade: go bot will publicly shame him if it doesnt :p
<jcw4> lol... tx fwereade
<voidspace> natefinch: this is my fix https://codereview.appspot.com/91660043/patch/20001/30010
 * perrito666 sees the noise in the standup was generated by a trunk pumping an overflowed sewer... thanks cthulu my nose is a bit clogged today
<voidspace> heh
<mgz> jcw4: sorry, I thought we'd said yesterday to just land it,
<mgz> jcw4: branch seems to have a text conflict with trunk which will need fixing
<mgz> jcw4: then it needs a commit message, which the bot should then pick up and merge
<natefinch> bodie_: you around?
<bodie_> hullo
<natefinch> I was looking at the json schema package
<bodie_> aye
<natefinch> I have some concerns
<bodie_> all ears!
<natefinch> it looks like it could be ok if it were fixed up
<bodie_> I was wondering what Dave Cheney meant about 66% coverage -- the author claims 246/248 tests passing on json-schema
<bodie_> that's the only reason I thought it would be acceptable
<natefinch> I think he means code coverage, so like 33% of the code doesn't get touched by the tests
<bodie_> ah, is there a way to check that that I'm not aware of?
<natefinch> bodie_: go test -cover :)
<jcw4> mgz: thanks... I saw the text conflict message, I'll try to figure out how to resolve it... should I just merge trunk into my branch?
<natefinch> looks like 80.6% coverage when I run it right now
<natefinch> also, failing tests is pretty bad
<natefinch> bodie_: more concerning for me is when I see stuff like this function: https://github.com/xeipuuv/gojsonreference/blob/master/reference.go#L84
<natefinch> bodie_: which says it returns an error, but actually always returns nil
<bodie_> oooo
<bodie_> hmm
<bodie_> okay, I think those concerns make sense
<bodie_> ... obviously they do, I mean I agree and understand :)
<natefinch> bodie_: the last one is the most important....  that's a function that is obviously encountering errors and throwing them away for some reason.  That's pretty bad.  Who knows what other problems exist.
<natefinch> The code has a lot of test cases which is great, and it might be a good starting point to make it into something production ready... but it's not production ready as-is, IMO.
<bodie_> I'm not sure why I didn't think to look through it -- I think my understanding was that we went with JSON-Schema because there were solid Go implementations already in place, so I made an assumption there
<bodie_> Do you think we need to stop and dig in to get this thing shipshape before we continue the Actions implementation, or do you think it makes more sense to chain things together from beginning to end and then attack the pain points?
<bodie_> I mean, my original implementation of an Actions spec from YAML was a little clumsy, but conformed the spec to a basic static schema
<bodie_> something like actions: {description: ..., params: {name: {description: ..., type: ..., default: ...}, name: ...}
<bodie_> oops, insert multiple names of actions as keys to the params values
<bodie_> or, we could use the code I have now which validates via gojsonschema, but instead use a generic "Validate" function that, for now, just ensures the spec conforms to the basic actions.yaml format I laid out there ^
<mgz> jcw4: yeah, just merge in trunk and resolve/commit
<bodie_> then as we move forward, put work into gojsonschema, and once it's in shape, it'll be a drop-in replacement for the Validate we use
<bodie_> I'm not trying to be lazy, just expeditious
<natefinch> I would not feel comfortable using that package, and I think fixing it up is going to take more time than value we'll actually get out of it, in my opinion.... but other people might have differing opinions.  fwereade might want to chime in
<bodie_> would agree there
<bodie_> another option in that case is that we simply Validate against a tiny subset of json-schema, something like the actions.yaml schema I specified
<fwereade> natefinch, bodie_: heh, yes, I should probably not have taken its quality on trust myself :/
<fwereade> natefinch, bodie_: yeah, that method alone is a big ol' WTF-do-not-touch-this-with-a-bargepole
 * fwereade sighs heavily
<bodie_> lol, fair enough
<fwereade> bodie_, can you reasonably estimate the effort to get a sane gojsonschema library written? even if it's just a conformant but useful subset?
<jcw4> mgz: I committed my merge fix, but I'm afraid I can't see how to update the commit message and land without proposing a new review?
<fwereade> bodie_, at least we can probably copy the test cases ;p
<mgz> jcw4: you can just edit the launchpad page
<mgz> jcw4: copy in the description to the commit message
<voidspace> jcw4: the commit message you normally set directly on launchpad
<fwereade> jcw4, I just did your commit message, I should have done that when I approved in the first place :/
<jcw4> I see
<jcw4> thx mgz, voidspace, and fwereade
<jcw4> fwereade: go bot kicked it back into needs review
<bodie_> fwereade, I can't say with any realism right now.  do you think the first step should be to think about what our needs are from it, first?  it seems very reasonable to me to implement a subset, or even a validator similar to the one in Config.  we could pass a json-schema rendering to the frontend
 * jcw4 will be back in 15 minutes... dropping kids off at school
<bodie_> right now I'm thinking very much in terms of "what could and would work" rather than "what would be best" so forgive me for being less than rigorous
<bodie_> just ideating, I expect you to knock these down
<mgz> sinzui: do you have some time to talk git?
<natefinch> why can't people just say brainstorming anymore?  Why is it ideating now?
<bodie_> haha
<sinzui> mgz, I and going into a meeting. I can talk in about 30 minutes
<bodie_> at least I didn't say incepting
<mgz> sinzui: sounds good
<perrito666> natefinch: what does ideating mean?
<bodie_> making ideas
<natefinch> perrito666: brainstorming
<perrito666> natefinch: meh, why replace a perfectly working buzzword for another
<natefinch> see?
<natefinch> sounds more hip, I guess.
<bodie_> that's certainly not the intention... I just like the way it sounds :)
 * natefinch likes the image of a storm of brains swirling in the dark clouds
<perrito666> sinzui: where can I find the ha b&r test?
<bodie_> cleaning up after a brainstorm is the worst.
<perrito666> natefinch: http://slurmed.com/fanfics/coldangel_1/blame-it-to-the-brain-part-2/003.jpg
<natefinch> lol nice
<bodie_> let's see here.  gojsonschema relies on gojsonpointer and gojsonreference....
<bodie_> gojsonpointer is about 400 lines in total
<bodie_> as is gojsonreference
<bodie_> gojsonschema itself is probably around 3000 lines, in about 8-12 files
<bodie_> that's really not so bad
<sinzui> perrito666, http://pastebin.ubuntu.com/7505941/
<bodie_> as dave mentioned, the test coverage is low
<perrito666> sinzui: tx
<bodie_> the question is, how much bad code is there, how much extra is needed to pull it apart into the good bits / organizational concepts, grind out the rot, and how much complexity was the bad stuff glossing over
<perrito666> sinzui: just fyi, along with the HA backup/restore patch I added doc on what was the expected behavior
<natefinch> bodie_: the other option is to take a well-regarded implementation in another language and port it to go.   That might be easier
<natefinch> bodie_: https://github.com/JamesNK/Newtonsoft.Json/tree/master/Src/Newtonsoft.Json/Schema
<bodie_> perhaps it would be intelligent simply to harness an existing implementation externally to our code
<bodie_> I know it's an ugly option, but it's an option
<bodie_> for an obvious example, we didn't implement our own database
<tasdomas> fwereade, I'm working on separating juju-core/cmd from juju-core. The package cmd depends on loggo - do you think that dependency should be broken (making logging something command implementers need to set up themselves)?
<jcw4> woo hoo, branch landed ... yay
<voidspace> natefinch: ping
 * bodie_ buys jcw4 a beer
 * jcw4 downs it and burps
<hazmat> anybody know where the cloudsigma provider stuff lives?
<perrito666> brb lunch
<natefinch> voidspace: how goes?
<voidspace> natefinch: cool, I need a new task
<voidspace> natefinch: I've started to look at HA: generate/update StateServingInfo in mongo.EnsureServer
<voidspace> natefinch: as that sounds plausible that I can do it :-)
<voidspace> natefinch: would you prefer I work on something else?
<natefinch> voidspace: haha... I'm not sure exactly what that's buying us... it's an old card.  rogpeppe - what was that task about generating StateServingInfo in EnsureMongoServer doing for us?
<rogpeppe> natefinch: in a call - be with you in a bit
<voidspace> natefinch: ah, ok
<natefinch> rogpeppe: sure thing, thanks
<voidspace> natefinch: which card you prefer I look at
<natefinch> voidspace:  you can do the change mongo to write majority one.  I had started on that, but it was causing problems for some reason.  The code change is simple,  but figuring out why it breaks things may be tricky
<voidspace> natefinch: ok
<voidspace> natefinch: do you have a branch you've started on?
<natefinch> voidspace: lp:~natefinch/juju-core/052-write-majority
<voidspace> natefinch: ok, I'll take a look
<voidspace> natefinch: ooh, lots of test failures
<voidspace> I wonder if it's even an mgo issue
<bodie_> sigh
<rogpeppe> anyone know what the current status of network routing to containers under ec2 is?
<rogpeppe> natefinch: hmm, i can't remember about that card. will have a look now.
<rogpeppe> natefinch, voidspace: i *think* the point is that the StateServingInfo in the agent.conf should not be generated by environs/cloudinit
<voidspace> natefinch: do we *always* have a replicaset?
<perrito666> voidspace: yup
<perrito666> learn to love it
<voidspace> perrito666: thought so
<voidspace> damn
<voidspace> I thought maybe write-majority was failing when we didn't
<voidspace> as the error message says "cannot create database index: norepl"
<voidspace> and indeed, google confirms that this is possibly the case
<voidspace> for "norepl"
<voidspace> This error happens when you try to use the w option without replication enabled.
<perrito666> mm, replicaset should be enabled always,that is what broke restore last time
<voidspace> perrito666: well, except maybe in tests
<perrito666> ah
<natefinch> sorry, was away for lunch.   Yes, we always run with a replicaset, except for localhost (which we actually want to fix)
<voidspace> natefinch: "except for localhost"
<voidspace> natefinch: I suspect that's what's breaking all the tests
<perrito666> breaking is the word of the day >p
<voidspace> natefinch: setting WMode without replicasets being enabled causes mongo to throw "norepl" errors
<voidspace> natefinch: and that's what I'm seeing in the tests
<voidspace> natefinch: using mgo session can we check for this? so we don't unconditionally set WMode?
<voidspace> session.SetSafe doesn't return anything, so we can't check there for error
<natefinch> voidspace: I wonder if we shouldn't run the test mongo instances with replicasets, to better replicate our running environments, rather than making them special
<perrito666> +1
<voidspace> natefinch: that sounds better - are they only used for tests? Or by "localhost" do you mean local provider too.
<natefinch> voidspace:  there's a special testing framework that we use to spin up mongo instances for testing
<voidspace> natefinch: JuJuConnSuite ?
<natefinch> voidspace: we do want to fix the local provider too, but that's a separate issue
<voidspace> natefinch: does the local provider currently *not* use replicasets?
<natefinch> voidspace: correct
<voidspace> natefinch: in which case setting WMode in state.Open will be broken for that too
<voidspace> so we either need to fix *both*, or not set WMode unless we have a replicaset
<natefinch> voidspace: correct
<voidspace> do you think fixing both is plausible within the scope of this card?
<natefinch> voidspace: I can make another card to fix local provider
<perrito666> voidspace: you cant fix one and make wmode not set when there is no replicaset?
<natefinch> we're supposed to do both, so might as well do the prerequisite first
<voidspace> perrito666: well, the fix is either "always use replicaset" or "don't use WMode when not using a replicaset"
<voidspace> perrito666: so saying "fix one and don't use WMode sometimes" is like the worst of both worlds :-)
<voidspace> natefinch: ok, so if you create the card I'll switch to that
<voidspace> or I can create the card
<perrito666> speaking of which can I create a card for what I am working now?, since I split the HA restore story in 2
<perrito666> or should I wait for you to create it?
<natefinch> voidspace: https://canonical.leankit.com/Boards/View/103148069/109980847
<natefinch> perrito666: you can do it
<voidspace> natefinch: thanks
<perrito666> natefinch: going to
<natefinch> voidspace: there's stuff in agent/mongo/    there's a withHA flag to remove
<voidspace> natefinch: ok
<voidspace> so there's a function "shouldEnableHA", that returns literally "providerType != provider.Local"
<voidspace> I've tried changing that to "true" and am running tests...
<voidspace> natefinch: *why* didn't we use replicasets for local - for perf reasons?
<voidspace> rogpeppe: ^^
<natefinch> voidspace: a couple people experienced problems
<voidspace> ah
<voidspace> natefinch: do you recall the nature of those problems?
 * perrito666 did some clean up of his cards
<natefinch> voidspace: nope.  There's a bug somewhere I think.
<natefinch> voidspace: I believe ian was one of the people who reported having problems
<voidspace> no failures on my machine so far
<voidspace> job done ;-)
<natefinch> voidspace: I believe the nature of the problem was long the lines of "bootstrap never completes" or something critical like that
<natefinch> voidspace: nothing too subtle
<voidspace> well, so long as it's only for Ian I'm not too worried
<voidspace> heh, I'll talk to him on Tuesday
<voidspace> and look for the bug
<natefinch> cool
<voidspace> so, apiserver/client tests fail
<voidspace> digging in
<voidspace> ah, but pass on their own
<natefinch> voidspace: I think it was less of a tests problem and more of a production problem.  But most of us could run it without a problem.
<voidspace> sure, but getting the code right is the *first* step - then I can try and *find* the production problems
<voidspace> it's ten to six on Friday night before a bank holiday though
<voidspace> gonna look for this bug then call it a night I think
<rogpeppe> voidspace: try looking in the bugs that were closed by the change that made juju not used replicaset on the local provider
<bodie_> separately from the issue of code quality, gojsonschema's deps are prepended with Apache2, but there's no LICENSE in their root directories
<voidspace> rogpeppe: ok, thanks
<bodie_> I'm not sure where that puts us
<bodie_> any input welcome
<rogpeppe> bodie_: hi
<natefinch> bodie_: I think licenses in the code is fine even if there's no license file.
<rogpeppe> bodie_: you pinged me last night, but i wasn't around
<bodie_> hullo, just was going to ask about the gojsonschema code quality and what my next steps ought to be.
<rogpeppe> bodie_: at first glance, it's not great at all
<rogpeppe> bodie_: i could see lots of opportunities for simplification and making the code more Go-idiomatic
<bodie_> yeah, my mistake there -- I completely assumed we'd vetted it for some reason, since I was under the impression we chose JSON-Schema for use with Juju due to the existence of a quality implementation for Go
<bodie_> since xeipuuv's implementation appears to be the go-to for Go, I wrongly made that connection
<rogpeppe> bodie_: i expect it was chosen because people know about it
<bodie_> i'll be back in a minute, kid is melting down
<rogpeppe> bodie_: personally, i think the standard is much more complex than it needs to be, but that's just me reacting to most of the modern computing world probably :-)
<rick_h_> bodie_: it was mainly chosen due to the JS side and the idea it could be supported in Go
<rick_h_> bodie_: and that it's got a definition that's standard
<rogpeppe> rick_h_: that's definitely a plus point
<rick_h_> jeremy has written a very good JS side of it and making it work down to Go is part of the work in making this work out
<rogpeppe> rick_h_: jeremy's written the JS schema verification engine?
<rick_h_> rogpeppe: yes, it generates UI and does validatio on it and such. mark S found it and brought him into the project because of it. https://github.com/jdorn/json-editor/
<rogpeppe> rick_h_: cool
<rogpeppe> rick_h_: i think it would be worth moving towards having our own high quality Go implementation, whether that entails from-scratch or mutating gojsonschema
<rick_h_> rogpeppe: yea, I think there was hope there was some good starting points out there for Go
<rogpeppe> rick_h_: i'd probably be tempted to start from scratch but with some close contemplation of existing implementations. it's very useful that there's an existing comprehensive test suite.
<bodie_> and back --
<natefinch> rogpeppe: yeah, the test suite was what I was most interested from that package.
<rogpeppe> natefinch: i was thinking mostly of the set of examples provided by json schema itself
<natefinch> rogpeppe: ahh
<natefinch> rogpeppe: my thought was to port an implementation from another language, since that's generally pretty easy, and gets us ahead of the game, rather than having to sit down and start from scratch.   Maybe we could fix up that go repo, but I honestly haven't given its approach any thought after I looked at some specific code samples
<bodie_> I'm hesitating to move forward in a specific direction since I already made a bad call here, but I chatted with fwereade and I think my next step is to start digging through the existing implementation to see whether it can be salvaged and how much rot there is to it
<rogpeppe> natefinch: the actual test suite in gojsonschema looks as if it could easily be more driven by the data inside json_schema_test_suote
<rogpeppe> bodie_: that's a good idea
<rogpeppe> bodie_: also perhaps study the standard and see if you can think of a nice obvious way to implement it simply and efficienctly.
<rogpeppe> bodie_: BTW i wouldn't say you made a bad call - grabbing the only json schema implementation is the obvious thing to do. it's a pity there's not a really nice implementation already out there.
<bodie_> I did some digging through the standard last night and I think "simple" and "efficient" are not the choicest words I would select to describe it
<rick_h_> lol
<bodie_> rogpeppe, thanks for that -- definitely feeling pretty embarrassed here
<bodie_> I wonder --
<bodie_> a functional implementation might actually make it much simpler.
<rogpeppe> bodie_: yeah, it's the usual completely bloated web standard
<perrito666> bodie_: cmon, there is no reason to feel embarrassed
<rogpeppe> bodie_: and fairly impenetrable, also as usual
<bodie_> I'm reading an article about how a J-schema validator was implemented in Clojure (my pet language) and it's really quite beautifully done
<bodie_> perhaps could rip off -- er, find inspiration in some of the ways this deals with it
<bodie_> me likey fp
<rogpeppe> bodie_: yeah, that might work nicely
<bodie_> rick_h_, rogpeppe -- I think there's a lot to be said for the value of a high quality json schema utility in Go
<bodie_> brb again --
<rogpeppe> bodie_: one thing i'd like to know is if it might be possible to define the actual json schema syntax as go structs
<voidspace> EOD, g'night all
<perrito666> voidspace: gnight
<natefinch> night voidspace
<voidspace> thanks all
<voidspace> see you on Tuesday
<wwitzel3> voidspace: have a good weekend
<perrito666> why tue?
<natefinch> perrito666: UK has one of their poorly named bank holidays on monday
<natefinch> perrito666: where the name of the holiday is "Bank Holiday"
<perrito666> according to wp is "Spring Bank Holiday"
<perrito666> lol and its determined by Last Monday in May
<natefinch> perrito666: btw, I and presumably wwitzel3 will also be out monday, since it's a holiday for the US, too - Memorial Day
<perrito666> well Ill then be a bit late since sunday its my birthday and I presume I might get a zip or two of wine
<natefinch> perrito666: happy birthday, and you can be as late as you want, because the rest of us will be not working :)
<perrito666> lol
<perrito666> tx
<bodie_> back, sorry.  michelle is sick and i'm watching the babe -- she's down for a nap now
<bodie_> rogpeppe, thinking about what you said
<bodie_> rogpeppe, right now we're parsing it out into a map[string]interface{}
<rogpeppe> bodie_: going down some of the schema document, it *might* be possible to parse into something like: http://paste.ubuntu.com/7506473/
<bodie_> then dumping that into a JsonSchemaDoc
<bodie_> if it complains, we reject it as invalid jsonschema
<rogpeppe> bodie_: obviously i've omitted most of the properties there
<bodie_> so, in the state, our schema will be defined as a nested map
<rogpeppe> bodie_: and there might be some kind of meta-attribute thing going on which makes the approach infeasible
<bodie_> to validate a json argument map from the frontend, they'll just GetJsonSchema or whatever
<bodie_> taking a look
<bodie_> ah, I see what you mean
<bodie_> interesting
<bodie_> an actual Go implementation of JSON-Schema, basically, rather than a parser / validator
<rogpeppe> omg, there's the whole email address syntax in there
<natefinch> lol
<bodie_> lol
<bodie_> and much more!
<bodie_> yours today for only 19.99 man-months...
<rogpeppe> bodie_: the point that we *might* be able to leverage encoding/json to do a lot of the initial verification for us
<natefinch> rogpeppe, bodie_:  A struct that might look like this (if you don't know C#  bool?  is a nullable bool, sorta like *bool in Go)
<natefinch> ^^^  https://github.com/JamesNK/Newtonsoft.Json/blob/master/Src/Newtonsoft.Json/Schema/JsonSchema.cs#L43
<rogpeppe> bodie_: then you wouldn't need to do nearly so much dynamic-value mangling to parse the schema
<bodie_> my eyes!!!
 * rogpeppe needs to go
<natefinch> haha... C# is pretty good.  And it's easy to translate to Go.
<sinzui> perrito666, I am not really at work today, but I have good news for you. http://juju-ci.vapour.ws:8080/view/Functions/
<natefinch> see ya rogpeppe
<rogpeppe> natefinch, bodie_: see ya
<bodie_> have a good weekend!
<sinzui> C# is nicer than java
<bodie_> fair enough
<bodie_> I don't personally find either very pleasant :)
<bodie_> but maybe I just haven't had enough exposure to C#
<perrito666> sinzui: sweet, pretty good for not your best day off
<perrito666> sinzui: why are there several runs for the same r?
<sinzui> perrito666, The failed test runs related to resource contention with other tests I adjusted the envs they run it.
<sinzui> perrito666, CI will run tests up to 5 times before cursing them. While juju might be at fault, clouds, network, and resource contention by be the real cause
<natefinch> bodie_: the reason I went with C# is that people generally don't try to do too much magicky stuff in it, which means it might have a valid implementation that would port well to Go.  It might be a little class-heavy, but probably a lot less so than a java implementation
<sinzui> perrito666, and in the case that azure is just dead to juju and cloud-init, its not juju's fault that azure tests fail
<perrito666> sinzui: well It would have made a pretty heavy weekend for me if I was responsible for the testing cursing after my commit
<sinzui> perrito666, I named the new test -devel, telling CI that the test does not get a vote to curse. The test certainly was at fault for the first few runs
<jcw4> I like the idea of patterning a Go json schema after NewtonSoft's implementation
<sinzui> perrito666, If the test reliably runs over the next few days, I will promote it by removing the -devel
<natefinch> bodie_: I've used the Newtonsoft stuff in C#, it's basically the go-to implementation for Json, since there isn't built-in support in the .net framework (or at least there wasn't as of about a year ago)
<jcw4> I believe that library has become the standard JSON impl in .NET vNext
<natefinch> jcw4: oh, cool
<jcw4> natefinch: looking for the link I read recently...
<jcw4> natefinch: https://twitter.com/JamesNK/status/466375482850963458
<jcw4> asp.net v next
<bodie_> lol
<natefinch> nice chart
<jcw4> :)
<bodie_> I need to have a look over this jsonschema stuff over the weekend.  my family is sick and I got very little sleep.  I hope to have something better to show for my investigation over the next couple of days.
<bodie_> enjoy your weekends :)
<natefinch> bodie_: good luck
#juju-dev 2014-05-25
<bodie_> oh, json-schema would be really easy in a language like haskell or scala, i think.
<bodie_> hrmph
<thumper> morning folks
<rick_h_> howdy thumper
<thumper> o/ rick_h_
<thumper> snow day here
<thumper> all the kids off school
<rick_h_> hah
<thumper> at least rachel is staying at home too
<rick_h_> national holiday tomorrow here but swapping it out wheee
<rick_h_> hah cool
<thumper> so I can get some work done
<thumper> trawling my way through the 400 odd emails of last week
<rick_h_> wheeee
<bodie_> hullo
<thumper> o/ bodie_ meetingology
<thumper> oops
<thumper> I mean menn0
<thumper> tab fail
<rick_h_> hey bodie_
<menn0> hi hi
<bodie_> lol
<thumper> down to 210 unread emails...
<thumper> coffee time
<menn0> anyone able to look at this?: https://codereview.appspot.com/94710046/
<thumper> o/ wallyworld
<wallyworld> g'day
<wallyworld>  good "holiday"?
<thumper> wallyworld: yeah...
<thumper> wishing I was in queensland right now
<thumper> heavy snow here
<wallyworld> \o/
<thumper> kids all off school
 * wallyworld loves snow
<wallyworld> we never have snow days here :-(
<bodie_> "holiday" == Memorial Day
<bodie_> :)
<bodie_> or did I fail to parse some context
<thumper> wallyworld: I don't get a snow day
<thumper> bodie_: no, I took leave last week
<thumper> bodie_: my holiday was me working on a different project
<bodie_> ah, I see.  fun
<davecheney> good news everyone, save joyent, http://paste.ubuntu.com/7517545/
<davecheney> ppc64el is passing
#juju-dev 2015-05-18
<mwhudson> davecheney: can you run lspci?
<davecheney> not at the moment
<davecheney> the machine went away
<davecheney> i suspect brad is playing kernel games
<mwhudson> also possible
<menn0> thumper or wallyworld: i'm going to ask niemeyer to look at this before merging but if you could take an initial look: http://reviews.vapour.ws/r/1705/
<menn0> this is the "hard part" of the fix for keeping the txns collection at a sane size
<wallyworld> menn0: i really hope this gets into mgo were it belongs
<menn0> wallyworld: agreed. this is just to get something into juju sooner.
<davecheney> thumper: menn0 https://go-review.googlesource.com/#/c/10135/2
<davecheney> fix for websocket issue
<davecheney> going to try to get it upstreamed ASAP
<davecheney> or otherwise I will deliver it on a branch
<thumper> davecheney: thanks
<mwhudson> luckily brad seems to be an insomniac workaholic :)
<mwhudson> well, i guess it's not that late for him, it is sunday though
<davecheney> his loss is our gain
<davecheney> ok, patch comitted
<davecheney> thumper: menn0 https://github.com/juju/juju/pull/2344
<cherylj> btw, davecheney, there is an actual file handle leak for bug 1420057
<mup> Bug #1420057: agents see "too many open files" errors after many failed API attempts <juju-core:Triaged by dave-cheney> <juju-core 1.22:In Progress by dave-cheney> <juju-core 1.23:In Progress by dave-cheney> <juju-core 1.24:In Progress by dave-cheney> <https://launchpad.net/bugs/1420057>
<cherylj> I saw it after testing the websocket changes
<cherylj> I just updated the bug with info
<cherylj> testing a fix now
<davecheney> cherylj: ok
<davecheney> so, https://github.com/juju/juju/pull/2344
<davecheney> i'm patching that back all the way back to 1.22
<menn0> thumper: 1:1?
<thumper> menn0: there shortly
<menn0> thumper: np
<menn0> cherylj: nice job finding another leak
<cherylj> davecheney: sounds good.  I think the combination of that, plus this fix I'm testing is required
<cherylj> menn0: thanks :)
<davecheney> cherylj: ok, i don't know how to track backporting the websocket dependency
<davecheney> change
<davecheney> so i'll just pretend like someone else does
<davecheney> thumper: ... value *errors.Err = &errors.Err{message:"", cause:(*net.OpError)(0xc21054c440), previous:(*errors.Err)(0xc2106a26e0), file:"github.com/juju/juju/state/open.go", line:69} ("cannot create database index: local error: bad record MAC")
<davecheney> ... error stack: local error: bad record MAC
<davecheney> annnd there goes another 20 minutes
 * thumper sighs
<davecheney> thumper: merge to trunk successful
<davecheney> 1.23 and 1.24 branches incoming
<davecheney> 1.22 unaffected, doesn't use x/net/websocket
<davecheney> thumper: ok
<davecheney> 1.22 will still be using code.google.com
<davecheney> do I need to update all the deps in 1.22 to use the github version of the websocket library ?
<davecheney> thumper: ?
<thumper> oh...
<thumper> um...
<thumper> ideally
<thumper> sed?
<davecheney> it's not a hard job
<davecheney> just a slightly large one
<davecheney> i'll do it afte rlunch
<thumper> kk
<thumper> ta
 * thumper afk to collect daughter from school
<thumper> bbs
<davecheney> godeps: cannot update "/var/lib/jenkins/workspace/github-merge-juju/tmp.BEr8lKzge2/RELEASE/src/bitbucket.org/kardianos/service": cannot create repo: cannot find project root: Get https://api.bitbucket.org/1.0/repositories/kardianos/service: EOF
<davecheney> great
<davecheney> just peachy
<davecheney> can't build 1.24 anymore
<davecheney> yay reproducible builds
<davecheney> any on call reviewers in the channel ? http://reviews.vapour.ws/r/1711/
<davecheney> any on call reviewers in the channel ? http://reviews.vapour.ws/r/1711/
<TheMue> morning o/
<dimitern> morning TheMue
<dimitern> TheMue, sorry, I'll be 10m late for our 1:1
<TheMue> ok
<dimitern> TheMue, I'm in the hangout
<TheMue> ok, coming
<davecheney> mwhudson: scaleway servers don't support neon
<davecheney> how the F did they manage that !?!
<mwhudson> davecheney: ah yeah was going to paste https://twitter.com/edouardb_/status/600211988552224768
<mwhudson> well, less transistors doing simd means more cpus on a die or something i guess
<mwhudson> seems an odd choice
<mwhudson> alternatively marvell had a bucket of armada 370s they wanted to get rid of
<sinzui> mgz, did you look into the maas 1.7 failures?
<sinzui> looks like it is thoroughly dead.
<mgz> sinzui: I saw them, and wimped out
<mgz> sinzui: if you're going to do some maas resurrection now I'd like to tag along
<sinzui> mgz: Me. The vms are paused and the nodes are gone. I don't think jog dismantled the maas, but something has
<sinzui> mgz: I want to consult with jog first. I think we need to unpause 1.7, probably restart it too, then use the utility script to make 15 new nodes
<mgz> hm, I sohuld have disabled the jobs on jenkins, didn't think of that
<sinzui> mgz, can you review this to unblock 1.22.4 testing http://reviews.vapour.ws/r/1713/
<mgz> sinzui: shipit
<katco> ericsnow: stand up
 * katco just noticed that out of context, her previous comment can be regarded as a weird request that ericsnow stop sitting.
<ericsnow> :)
<mgz> katco: having a space in standup makes it seem more like a command :)
<perrito666> mgz: who says it isnt?
<katco> mgz: hehe yeah. maybe i should make it a proper noun "Standup"
<perrito666> katco: you could upgrade with a : now!
<katco> perrito666: that would just be rude :|
<perrito666> well it works for shutdown
<perrito666> why would it not work for stand up :p
<ericsnow> cherylj: the solution for systemd/cloudinit that you wrote up in the bug sounds correct
<katco> lol, for shutdown i'm commanding a machine
<cherylj> cool, thanks ericsnow
<ericsnow> cherylj: it may be worth double-checking with smoser about it
<ericsnow> cherylj: I expect he wrote the systemd configs for cloudinit
<cherylj> ericsnow: thanks, will do
<ericsnow> cherylj: let me know when you have a patch and I'll review it
<ericsnow> cherylj: or ping me if you want any other feedback
<cherylj> ericsnow: thanks.  I'll probably have it ready this afternoon.  I'm working on a different bug this morning.
<ericsnow> cherylj: np
<voidspace> ericsnow: every time I do a full test run I get systemd broadcast messages spammed to every terminal
<voidspace> ericsnow: and I blame you...
<voidspace> haven't tracked down which test it is yet
<ericsnow> voidspace: I accept it :/
<voidspace> hehe
<ericsnow> voidspace: what are the messages?
<ericsnow> voidspace: you're running vivid?
<voidspace> ericsnow: yes
<ericsnow> voidspace: (by which I mean systemd)
<voidspace> ericsnow: debug level log messages
<voidspace> np
<voidspace> * lucap_ (l
<voidspace> gah
<voidspace> Broadcast message from systemd-journald@UbuntuBox (Mon 2015-05-18 16:04:10 BST):
<voidspace> [23762]: 2015-05-18 15:04:10 DEBUG juju.apiserver apiserver.go:273 <- [7] machine-1 {"RequestId":6,"Type":"NotifyWatcher","Id":"2","Request":"Stop","Params":"'params redacted'"}
<voidspace> etc...
<voidspace> to *every* damn terminal
<voidspace> including vim
<voidspace> :-)
<ericsnow> voidspace: yuck
<ericsnow> voidspace: hmm, this is probably going to require more than a minute to track down...
<ericsnow> voidspace: would you mind opening a bug on this (against 1.23 and up)
<voidspace> ericsnow: ok
<voidspace> thanks
<voidspace> ericsnow: I was only complaining...
<ericsnow> voidspace: :)
<ericsnow> voidspace: I'm sure it will bug everyone at some point and it *could* imply some other issue
<voidspace> :)
<ericsnow> wwitzel3: I just realized I never applied your rb_webhooks patch to our install :(
<wwitzel3> ericsnow: do it :)
<ericsnow> wwitzel3: I've done so now (PRs should now be links)
<wwitzel3> ericsnow: thanks :)
<ericsnow> wwitzel3: unfortunately it only helps with new PRs
<wwitzel3> ericsnow: yeah, I briefly looked at what to do for historical ones and decided it wasn't worth it
<ericsnow> wwitzel3: yeah that shouldn't matter within a few weeks
<ericsnow> wwitzel3: just wish I'd applied it when I merged your PR :)
<voidspace> http://www.cattlerap.com/
<jcastro> mgz: anything new wrt. dreamcompute?
<perrito666> voidspace: that is actually quite catchy
<voidspace> perrito666: :-)
<voidspace> it's ridiculous
<voidspace> wwitzel3: afternoon
<perrito666> voidspace: its not really unpleasant as background noise
<wwitzel3> voidspace: hey :)
<voidspace> o/
<mgz> jcastro: you saw my reply right?
<perrito666> I should try with spanish auctions but since we use less words it might not be so rap like
<voidspace> ericsnow: https://bugs.launchpad.net/juju-core/+bug/1456258
<mup> Bug #1456258: systemd broadcast message spam during test run <papercut> <unit-tests> <juju-core:New> <https://launchpad.net/bugs/1456258>
<ericsnow> voidspace: thanks
<voidspace> ericsnow: couldn't see how to mark it as affecting 1.23 without targetting to a milestone
<voidspace> which seemed innapropriate
<ericsnow> voidspace: yep
<ericsnow> voidspace: np
<ericsnow> voidspace: you only get the spam when running the test suite, right?
<voidspace> ericsnow: yeah, so far
<voidspace> ...
<voidspace> ericsnow: I haven't narrowed down which test it is I'm afraid
<ericsnow> voidspace: no worries
<voidspace> ericsnow: the messages appear (always) before the output line for cmd/juju
<voidspace> ericsnow: but running those tests alone doesn't produce it
<mup> Bug #1456258 was opened: systemd broadcast message spam during test run <papercut> <unit-tests> <juju-core:New> <https://launchpad.net/bugs/1456258>
<mgz> jcastro: was hoping to get some juju hacking time on sunday, but have now filed bug 1456265 at least to track it
<mup> Bug #1456265: Openstack provider should with without object-store <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1456265>
 * natefinch just hit esc :wq from sublime :/
<perrito666> and that doesnt work?
<perrito666> I am surprised
<perrito666> :p
<natefinch> heh
<natefinch> You could probably configure it to work
<perrito666> I believe there is a vim mode for almost everything
<mup> Bug #1456265 was opened: Openstack provider should with without object-store <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1456265>
<natefinch> heh, just accidentally proved that the CI test I wrote fails on an unfixed version of Juju
<natefinch> "Why the hell isn't this working? ..... oh, this is an old version of 1.24, duh"
<natefinch> mgz, sinzui: I have a test charm that I need to call actions on... am I correct in thinking the current jujupy doesn't have support for calling actions?
<sinzui> natefinch, no yet.
<sinzui> natefinch, We certainly intend to add actions to you dummy charms, and jujupy needs to grow support
<natefinch> sinzui: cool, well, maybe I can help out, since I need it right no
<natefinch> now
<sinzui> natefinch, fab, we love contributions
<voidspace> right, EOD
<voidspace> g'night all
<mup> Bug #1362324 changed: Unit tests dying on PPC64el on 1.20.6 <intermittent-failure> <ppc64el> <juju-core:Invalid> <https://launchpad.net/bugs/1362324>
<mup> Bug #1393986 changed: TestStartInstanceWithUnknownAZError nil deref panic <intermittent-failure> <test-failure> <juju-core:Fix Released by gz> <https://launchpad.net/bugs/1393986>
<perrito666> rick_h_: can I ask you the "gvim question of the month" ?
<rick_h_> perrito666: sure thing
<perrito666> do know to what settings do the gui components other than the editor obey? (especially the tabs)
<cherylj> hey ericsnow, turns out that there's an option in cloud-config for cloud init to halt the system when it's done:  https://cloudinit.readthedocs.org/en/latest/topics/examples.html#reboot-poweroff-when-finished
<ericsnow> cherylj: perfect
<perrito666> rick_h_: I have  some visual effort to determine the current tab
<rick_h_> perrito666: parsing
<rick_h_> perrito666: so do you mean like the menu bar and such?
<perrito666> yes, I am not sure if it is gtk, because it looks different than the rest of my system
<rick_h_> perrito666: oh, yea, there's vim-gtk the package that uses gtk
<perrito666> and gvim tabs is a very polluted search
<rick_h_> perrito666: maybe check lxappearance if you're not finding it listening
<rick_h_> as there's gtk and then there's the rest of unity stuff? /me doesn't have any decorations so can't say for sure tbh
<perrito666> rick_h_: tx, ill hack a bit the gtk theme to see if I can get the active tab to be more evident
<rick_h_> perrito666: stop using tabs :P
<rick_h_> perrito666: I fought against vim forever saying I couldn't use it until it had tabs. Then it got tabs and now I realize if I'm using tabs I messed up
<rick_h_> perrito666: but say that tongue in cheek.
<perrito666> lol, I use tabs to create a visual path of the code, very useful when you have 6 or 7 lovely levels of indirection
<perrito666> console vim tabs are easy to style but gtk ones seem a bit harder
<natefinch> sinzui: I'm a little lost with the CI tests... are the subclasses of TestCase the actual CI tests... or are those just unit tests for the test code itself?  If the latter, how does one construct a CI test?
<natefinch> sinzui: (as always, if there's documentation, feel free to just send me to that)
<sinzui> natefinch, test_*.py are tests for the test harnesses. the *.py scripts setup tests
<sinzui> natefinch, for ever feature added jujupy.py, we expect to see it exercised in test_jujupy.py
<natefinch> sinzui: ok
<natefinch> sinzui: I'm still not sure I understand, actually.  When you say "tests for the test harness" does that mean "tests that test the testing code" or "tests that show up in jenkins as failed tests indicating juju bugs"?
<sinzui> natefinch, none of the scripts in juju-ci-tools are specific tests for anything
<sinzui> natefinch, juju-ci-tools containts the libraries and scripts to create juju tests in Jenkins.
<natefinch> sinzui: ok, good to know.  So I would add the action-running functions there.....    how does one "create juju tests in Jenkins"?
<sinzui> natefinch, working examples include deploy_stack.py, quickstart_deploy, and assess_recovery.py.
<natefinch> sinzui: I thought you said none of those tests were specific tests for anything?
<sinzui> natefinch, We wrote those scripts to work on our machines, using cloud-city, then setup in jenkins that pass the arguments to test a version of juju in a specific environment
<sinzui> natefinch, they don't because without a version of juju and environment written in a jenkins job, it is generic
<sinzui> natefinch, in http://reports.vapour.ws/releases/2657/job/aws-deploy-precise-amd64/attempt/2649, the actual command to for that test is
<sinzui> timeout -s INT 30m /mnt/jenkinshome/juju-ci-tools/deploy_job.py --new-juju-bin ./extracted-bin/usr/lib/juju-1.25-alpha1/bin --series precise test-release-aws /var/lib/jenkins/jobs/aws-deploy-precise-amd64/workspace/artifacts aws-deploy-precise-amd64
<mup> Bug #1456315 was opened: Received disconnect from ...: 2: Too many authentication failures for ubuntu <juju-core:New> <juju-core 1.23:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1456315>
<natefinch> my #1 complaint about python:  I have no f'ing clue what any particular function will return.
<perrito666> natefinch: have faith
<perrito666> it is also my number one complaint aboubt declaring things with := instead of explicit declarations in go, and you dont see me complaining about :p
<perrito666> natefinch: just assume it will return something whith the api you need, and it will almost always be the case
<natefinch> perrito666: yes but I mean, even when *looking at the source* I can't tell what a python function returns
<natefinch> at least in Go the types returned are declared in the function signature
<perrito666> natefinch: well if the code is properly written you can see what the returned object looks like
<perrito666> not knowing the type "is a feature"
<perrito666> but I always said that python could benefit from interfaces
<perrito666> or some sort of contract
<perrito666> as a side effect, it makes you choose more carefully who you code with :p
<natefinch> sinzui: is the way that we make a test fail is to have it raise an exception, or is there some other way to signal failure?
<sinzui> natefinch, jenkins only uses exit codes :(. 0 as pass and anything else is fail. There is no concept of skipped or error in setup.
<natefinch> sinzui: ok.  Looks like the standard setup is to just wrap everything in a try/except and exit(1) if an exception is caught, which works well enough for my purposes.
<sinzui> natefinch, yep, because python will set an exit code. We also like to read the error to diagnose why it failed
<ericsnow> wwitzel3: PTAL: http://reviews.vapour.ws/r/1716/
<davecheney> thumper: nope, still busted, https://github.com/golang/go/issues/10844#issuecomment-103086849
<thumper> davecheney: heh
<davecheney> to everyone to gophercon
<davecheney> have you used the hotel discount codes on the website ?
<cherylj> davecheney: I had the travel provider use the code when she booked the hotel
<davecheney> thanks!
<mup> Bug #1456343 was opened: APIServer Run method doesn't validates if Machines, Services or Units are present on the request. <juju-core:In Progress by niedbalski> <https://launchpad.net/bugs/1456343>
<wwitzel3> ericsnow: looking
<ericsnow> wwitzel3: ta
<mup> Bug #1456343 changed: APIServer Run method doesn't validates if Machines, Services or Units are present on the request. <juju-core:In Progress by niedbalski> <https://launchpad.net/bugs/1456343>
<davecheney> and for my next magic trick, I will spend all day git bisecting which revision of the gc broke juju
 * davecheney pulls rabbit from hat
<mup> Bug #1456343 was opened: APIServer Run method doesn't validates if Machines, Services or Units are present on the request. <juju-core:In Progress by niedbalski> <https://launchpad.net/bugs/1456343>
<wwitzel3> ericsnow: ok, actually looking now, sorry, was emailing
<ericsnow> wwitzel3: np
<wwitzel3> ericsnow: ptal at my email when you can :)
<ericsnow> wwitzel3: already started :)
<ericsnow> wwitzel3: I've addressed your review comments on 1716
<axw> wallyworld: just reading your email about bootstrap. am I crazy, or did you not actually propose to change the default behaviour of auto-upgrading in the end?
<axw> wallyworld: "bootstrap with no version specified" sounds like what we do now?
<axw> except without an agent restart
<wallyworld> axw: yeah, that's what we do now. i was advocating keeping that for compatability and introducing a new option for repeatability
<wallyworld> axw: but the current option would no longer reboot agents
<axw> wallyworld: which is basically just moving agent-version from environments.yaml to a CLI option
<axw> ok
<axw> sounds fine, I just thought there was going to be a bigger change
<wallyworld> yes, - so it's a cleaner approach, and will avoid agent restarts
<wallyworld> i tried to keep it as minimal as possible in terms of change
<wallyworld> but getting us to a better place
<axw> wallyworld: how do you prevent restart? don't start until it's got to the target version?
<axw> I mean, don't accept conns
<wallyworld> axw: not 100% clear yet - need to work through implementation
<axw> okey dokey
<wallyworld> but could be something like that
<wallyworld> axw: i was going to leave that up to anastasiamac to figure out :-)
<axw> ah :)
<anastasiamac> hmm :... thanks wallyworld :D... i think :))
<axw> bbl
#juju-dev 2015-05-19
<perrito666> katco: ping
<davecheney> thumper: https://bugs.launchpad.net/juju-core/+bug/1456398
<mup> Bug #1456398: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>
<davecheney> this is me today
<davecheney> git bisect for the git gods
<mup> Bug #1456398 was opened: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>
<mup> Bug #1456398 changed: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>
<mup> Bug #1456398 was opened: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>
<menn0> davecheney: i'm a big fan of git bisect
<thumper> axw: around?
<axw> thumper: hey
<thumper> axw: oh hai
<thumper> axw:  just talking to wallyworld about the storage provider and multiple environments
<thumper> axw: just bitten us
<axw> thumper: bugger. what's the problem?
<menn0> thumper or wallyworld: txn pruning integration with Juju: http://reviews.vapour.ws/r/1720/
<thumper> axw: runner exited: watching volumes: permission denied
<thumper> axw, wallyworld: likely to be a simple fix
<thumper> just finding the right line to add
 * thumper tries to run up to speed
<thumper> axw: the storage provider is a machine agent worker?
<axw> thumper: the "storageprovisioner" worker is run twice: once in the env manager, once in each machine agent
<axw> thumper: kinda like how we have a env-level provisioner and one per machine for containers
<thumper> axw: well... we have an env-level provisioner per environment
<thumper> not just one on the state server machine
<thumper> I'm wondering if this is the gotcha
<axw> thumper: quite possibly
<thumper> axw: what is the worker called?
<thumper> found it
<axw> thumper: if you get stuck, and can give me a repro, I can debug it
<thumper> the environ-storageprovisioner is running per environment
<thumper> so that should be fine
<thumper> waigani: where was the error exactly?
<thumper> axw: it was waigani that ran into the issue, you two can talk directly so I can go smash the cli some more
<axw> okey dokey
<waigani> thumper: ERROR juju.worker runner.go:219
<thumper> waigani: hey, there is this cool function you can add...
 * thumper goes to search for it
<thumper> I wrote it for this type of situation...
<thumper> I think
<waigani> thumper: I created a hosted env, deployed wordpress, work-load status hangs on: message: Waiting for agent initialization to finish. Debug-log then is full of perm denied errors
<anastasiamac> axw: if it helps, I saw similar error in featuretests - i wasn't on multiple env tho
<thumper> LoggedErrorStack
<anastasiamac> axw: i was runing one cli command and than followed by another in the same test
<wallyworld> menn0: review done
<waigani> so there is this line: return isEnvironManager && tag == st.EnvironTag()
<thumper> see worker/provisioner/provisioner.go line 181
<anastasiamac> axw: i was going to talk to u about it but got distracted :(
<waigani> storageprovisioner.go:78
<thumper> if the error is not nil and the feature flag is set, it logs the error stack
<thumper> very handy for debugging
<thumper> when you really don't expect an error return there
<axw> waigani: st should be scoped to the env at that point? or not?
<waigani> if it is okay to watch volumes in hosted envs, the fix should just be a matter of removing the isEnvironManager line
<waigani> axw: yes, it should be
<waigani> axw: I can double check though
<menn0> wallyworld: thank you
<wallyworld> np
<axw> waigani: don't understand, why would you want to remove that line?
<menn0> anastasiamac: thanks for your review too
<waigani> axw: I haven't tested, but it seems that would be what is stopping volumes from being watched in the hosted env
<axw> waigani: we only want state servers to be able to access environments
<waigani> axw: as it will return false if it's hosted
<waigani> axw: you've got this comment in the code:
<waigani> / TODO(axw) allow watching volumes in alternative
<waigani> 				// environments? Need to check with thumper.
<anastasiamac> menn0: always nice :D tyvm for PR
<axw> waigani: yeah, I don't think that's relevant. that was before all the JES stuff went in, now I see we have one worker per env
<axw> waigani: before I thought we might have one worker that spoke to multiple envs
<waigani> axw: ah okay
<thumper> waigani: where is the isEnvironManager line you are talking about?
<waigani> thumper: line 79
<thumper> waigani: of which file
<waigani> storageprovisioner.go
<waigani> apiserver
<thumper> no one will ever log in to an apiserver with an environ tag
<thumper> that block should be removed
<thumper> the only things that log in are machines, units or users
<waigani> should we add cases for units and users?
<waigani> as it currently doesn't have any
<axw> thumper: what do you mean nothing will log in with an environ tag? what bit of code leads you to think that's happening?
<thumper> line 72 or apiserver storageprovisioner.go
<axw> thumper: the "case names.EnvironTag" is not the tag used to log in, it's a tag that the storage provisioner uses to scope things
<thumper> uuurrrggghhhh....
<thumper> this seems overly complicated and ackward....
<axw> thumper: no worker ever passes the login tag to canAccess
<thumper> I think more logging around this access would show where the problem is
 * thumper leaves it to waigani
 * thumper runs away
<waigani> haha
<waigani> thanks :/
<axw> waigani: the gist is: the worker asks to watch volumes. it passes a scope tag, and that is either a names.MachineTag or a names.EnvironTag
<axw> waigani: if it's an EnvironTag, the apiserver will watch volumes scoped to that environ
<axw> waigani: if it's a MachineTag, it'll watch volumes scoped to that machine
<waigani> axw: right, that makes sense
<axw> waigani: so, if it's an EnvironTag we want to make sure the logged-in user is an environ manager
<axw> and also that the state's EnvironTag matches the scope
<axw> waigani: my understanding of JES is limited, so that last bit may be where things are broken
<waigani> axw: when you say scope, you mean the state passed in to NewStorageProvisionerAPI ?
<axw> waigani: no, I mean the tag passed in the API request, in WatchVolumes/WatchFilesystems
<axw> waigani: those methods take params.Entities, each of which is a MachineTag or EnvironTag as described above
<waigani> axw: yeah but "matching the scope" is matching the EnvrionTag passed in with that of the state obj used to make the api, right?
<axw> waigani: yes
<waigani> axw: so the EnvironTag case is never going to pass in a hosted env
<axw> <axw> waigani: st should be scoped to the env at that point? or not?
<axw> <waigani> axw: yes, it should be
<axw> ?
<axw> so the "state.State" is *not* scoped to the env?
<axw> (hosted env)
<waigani> axw: yes, it should be scoped, but I'll check. I'm just looking to see which user is isEnvironManager() checking?
<axw> waigani: AuthEnvironManager
<axw> i.e. a machine agent with JobManageEnviron
<waigani> axw: so in the hosted env scenario, which machine would that be?
<axw> waigani: you're asking me how JES works? :)
<axw> it should be a state server machine?
<waigani> axw: haha, okay I'm going to dig into isEnvironManager
<waigani> axw: if it's not the state server machine, it's going to fail. Also, as you point out if st isn't scoped, it's going to fail
<axw> waigani: yup
<waigani> axw: so let me dump out some info and we can go from there
<waigani> axw: shall I remove your comment?
<axw> waigani: yes please, if you're making changes
<axw> thanks
<waigani> axw: I'll take it out now, in case this turns into a PR
<waigani> np
<wallyworld> axw: it does appear that azure really wants people to stop using affinity groups which is good for us. let me know if you have time for a quick chat as there's a couple of points i'd like to clarify. no rush
<axw> wallyworld: yeah, that sounds like the case. can chat now, tanzanite standup?
<wallyworld> sure
<waigani> got it
<waigani> so st is scoped to hosted env, but the tag passed in is always the server env, so they never match. jujud's  ReadConfig (cmd/jujd/agent/agent.go:106) takes a string tag and uses that to determine the location of the agent's config file to read.
<waigani> the string tag is "machine-0"
<waigani> (which can no longer be assumed to be the state server, as the first machine in a hosted env is also machine-0)
<waigani> agent.ConfigPath returns the agent.conf of machine-0 the state server, not machine-0 the first machine in hosted env
<waigani> .juju/local/agents/machine-0/agent.conf
<anastasiamac> waigani: well done :D
<waigani> axw: thumper: ^^
<axw> waigani: are you saying you can have two machines called machine-0?
<waigani> axw: yes
<axw> sounds crazy, but ok
<axw> waigani: so... is that a jujud bug? that it's reading the agent.conf for the wrong agent?
<waigani> axw: yep
<waigani> axw: so will most likely have side effects elsewhere
<waigani> axw: juju environment create env1; juju deploy wordress
<waigani> axw: the juju status and checkout the machine name
<thumper> waigani: I'm pleased you found it
<axw> waigani: hrm, I'm a little confused. I would think that machine-0 the state server would be running env workers per hosted env?
<thumper> axw: machine-0 machines are in different environments
 * thumper takes the dog out while it is still light
<waigani> thumper: can we talk this one over tomorrow morning?
<waigani> I need to run also
 * thumper heads off to make dinner
<thumper> laters
<axw> wallyworld: looks like the whole azure model has changed. the little bit I've read makes it sound a lot more like other IaaS now, and Cloud Services are no more (with the new ARM API). portal is excruciatingly slow though, so can't actually tell for sure...
<axw> wallyworld: if that is the case, we might be able to migrate away from what the either-or HA/placement implementation
<wallyworld> axw: yeah, they are changing terminology to use Region etc as well - so i think it's going to be goof
<wallyworld> good
<axw> wallyworld: yes, looks good
<axw> will be nice to kill some of that old hacky code
<axw> well
<axw> relegate it anyway
<rogpeppe3> anyone fancy a small review of a LoggingSuite change? https://github.com/juju/testing/pull/69
<voidspace> dimitern:  http://reviews.vapour.ws/r/1714/
<dimitern> voidspace, looking
<voidspace> dimitern: plus http://reviews.vapour.ws/r/1715/
<dimitern> voidspace, will do
<voidspace> dimitern: basically the same code as landed, with minor fixes for 1.24/master plus adding the upgrade step to steps124 too
<voidspace> dimitern: thanks
<dimitern> voidspace, have a look at our shiny new and empty board https://canonical.leankit.com/sapphire
<voidspace> dimitern: oooh...
<voidspace> dimitern: shiny :-)
<dimitern> voidspace, we'll start using it by friday I hope
<voidspace> dimitern: cool
<TheMue> dimitern: but then we'll soon fill it and it won't be so shiny anymore *scnr*
<TheMue> dimitern: hehe, and opening the parking lot show's already some cards
<dimitern> TheMue, :) that's right, yeag
<dimitern> yeah even
<dimitern> I'll groom the parking lot cards a bit and see what we didn't manage to finish by thursday
<TheMue> dimitern: now it only has to be a visual frontend to our tasks in launchpad ;)
<dimitern> voidspace, both reviewed
<voidspace> dimitern: ta
<voidspace> dimitern: I'll have to go and fix those on 1.23 as well
<dimitern> voidspace, well, at least check if with 1.23 and string ids there the test still passes, other than that it's too late for 1.23 I think
<voidspace> dimitern: the tests pass
<voidspace> dimitern: the branch landed and I ran tests for 1.24 and master
<dimitern> voidspace, cool, I think it's fine then
<voidspace> on 1.24 (current version not my branch) featuretests time out
<voidspace> they pass on master
<voidspace> odd and annoying
<dimitern> hmmm
<rogpeppe3> dimitern: where's the right place to submit bugs against juju-core - github or launchpad?
<dimitern> rogpeppe3, lp I guess
<rogpeppe3> dimitern: "you guess"? :)
<rogpeppe> dimitern: ok, will submit against lp
<dimitern> rogpeppe, I guess >> it was like that originally, but I know there are some GH issues filed now
<rogpeppe> dimitern: ok.
<rogpeppe> dimitern: i'll file on lp anyway
<rogpeppe> i guess :)
<rogpeppe> dimitern: https://bugs.launchpad.net/juju-core/+bug/1456519
<mup> Bug #1456519: apiserver/client: clientSuite.TestProvisioningScript is flaky <juju-core:New> <https://launchpad.net/bugs/1456519>
<rogpeppe> i guess noone else is using go1.3 or later to run juju tests
<rogpeppe> dimitern: any chance of a review of https://github.com/juju/testing/pull/69 please?
<rogpeppe> dimitern: (it's very small)
<rogpeppe> or anyone else ( voidspace? fwereade?)
<fwereade> rogpeppe, ship it
<rogpeppe> fwereade: ta!
<dimitern> rogpeppe, I'll have a look after standup
<dimitern> rogpeppe, ah, ok then
<rogpeppe> fwereade: it would be nice to have your LGTM on it
<fwereade> rogpeppe, it does? http://reviews.vapour.ws/r/1724/
<rogpeppe> fwereade: (we need two, and an explicit signoff from -core is always good)
<rogpeppe> fwereade: ha! i'm not used to reviews.vapour.ws!
<rogpeppe> fwereade: i was looking at github only
<rogpeppe> fwereade: ta!
<mup> Bug #1456519 was opened: apiserver/client: clientSuite.TestProvisioningScript is flaky <juju-core:New> <https://launchpad.net/bugs/1456519>
<dimitern> reviews on http://reviews.vapour.ws/r/1725/ please - it fixes bug 1442257
<mup> Bug #1442257: lxc network.mtu setting not set consistently across hosts <addressability> <lxc> <network> <juju-core:Triaged> <juju-core 1.23:In Progress by dimitern> <juju-core 1.24:Triaged by dimitern> <https://launchpad.net/bugs/1442257>
<perrito666> morning
<dimitern> voidspace, TheMue, dooferlad, ^^
<dooferlad> dimitern: *click*
<dimitern> dooferlad, ta!
<TheMue> dimitern: back from lunch, looking too
<dimitern> TheMue, cheers
<jam> wallyworld: perrito666: we're still on for the relation status discussion, right?
<wallyworld> yep
<jam> wwitzel3: we're also supposed to meet sometime, right? Do you want to meet before the retrospective tomorrow?
<perrito666> yep
<wwitzel3> jam: that sounds good to me
<jam> perrito666: on the concept of VR hangouts, I'm told that having the world move for you is the really-really big producer of nausea
<perrito666> jam: I only used a VR headset once and I got really confused about not seeing my hands
<perrito666> the movement was not very annoying when sitting
<alexisb> jam, I was just about to schedule a planning meeting in that slot :)
<jam> alexisb: for our scheduling?
<perrito666> alexisb: did you just cancel the interlock?
<alexisb> perrito666, yes I forgot to do it yesterday
 * perrito666 notices google calendar loves a refresh after a couple of weeks up
<alexisb> cloudbase is at ODS this week
<perrito666> ah true
<alexisb> jam, yes
<jam> alexisb: sure. if you get it scheduled first you win :). I also sent a message to cmars about if there is another time we could use
<alexisb> yeah the hour after the leads call is definitely my preference
<alexisb> but I am going to grab that weds slot just in case
<jam> sure
<katco> natefinch: standup
<ericsnow> voidspace: weren't you the one that moved us away from "anything" on our SSL cert code?
<sinzui> voidspace, is this bug Fix Committed in 1.24 https://bugs.launchpad.net/juju-core/+bug/1348663
<katco> fwereade: hey want to chat about the container stuff a bit early? we're all in moonstone?
<mup> Bug #1348663: DHCP addresses for containers should be released on teardown <maas-provider> <network> <oil> <juju-core:Triaged by mfoord> <juju-core 1.24:Triaged by mfoord> <MAAS:Invalid> <https://launchpad.net/bugs/1348663>
<sinzui> voidspace, I think your last commit to 1.24 fixes bug 1441206
<fwereade> katco, just making coffee, will join you in there though if that makes sense?
<mup> Bug #1441206: Container destruction doesn't mark IP addresses as Dead <destroy-unit> <network> <juju-core:In Progress by mfoord> <juju-core 1.24:In Progress by mfoord> <https://launchpad.net/bugs/1441206>
<katco> fwereade: yeah, we're all in ther efrom the standup
<katco> fwereade: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1&hceid=Y2Fub25pY2FsLmNvbV9pYTY3ZTFhN2hqbTFlNnMzcjJsaWQ5bmhzNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t.7g85fo35lqi339e7ppmrdgeoi0
<voidspace> ericsnow: no, that was wwitzel3
<voidspace> sinzui: it does fix a bug
<voidspace> sinzui: let me confirm
<voidspace> sinzui: I've marked 1441206 as fix committed for 1.24, still trying to merge into master
<sinzui> voidspace, thank you
<natefinch> jw4: you around?
<voidspace> command line tool to open hangouts at your next meeting
<voidspace> https://t.co/gNRi41ELLk
<jw4> natefinch: yeah - I'm OTP for the next 30 minutes or so, but I'm somewhat around
<natefinch> jw4: quick question - am I correct in my observation that action results are always converted to strings?
<jw4> natefinch: hmm; that doesn't seem quite right to me
<natefinch> jw4: for example, I set a value of 16, but it gets converted into "16"
<jw4> natefinch: I don't know for certain, but I *thought* result types would be preserved
<jw4> natefinch: bodie_ worked on that part so he may remember more accurately
<jw4> natefinch: I'll investigate too when I get a chance
<natefinch> jw4: ok no worries.  Could well be my fault, too.
<natefinch> heh, interesting, from python's time formatting, about parsing seconds: "The range really is 0 to 61; this accounts for leap seconds and the (very rare) double leap seconds."
<dimitern> reviews on http://reviews.vapour.ws/r/1726/ are most welcome
<TheMue> dimitern: *click*
<mup> Bug #1454697 was opened: jujud leaking file handles <cpec> <stakeholder> <juju-core:Triaged> <juju-core 1.22:In Progress by cherylj> <juju-core 1.23:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1454697>
<alexisb> katco, you around?
<katco> alexisb: yep
<natefinch> one thing I have to give it to python for, it is very googable for common questions
<abentley> tvansteenburgh: Thanks for the merge.
<jw4> natefinch: it looks like all the result values are converted to strings...
<jw4> natefinch: the tests seem to explicitly expect that
<natefinch> boo!
<natefinch> why even use yaml if you can't use nice types? :/
<jw4> teh suk
<natefinch> well, at least I'm not crazy
<jw4> natefinch: correction "at least this situation doesn't demonstrate whether you're crazy"  - you may still be but not as shown by this situation :)
<natefinch> jw4: I was going to make that joke, but couldn't figure pout a way to word it.
<jw4> natefinch: well, I didn't do too well either
<jw4> :)
<natefinch> https://bugs.launchpad.net/juju-core/+bug/1456703
<mup> Bug #1456703: action-set converts everything to strings <actions> <juju-core:New> <https://launchpad.net/bugs/1456703>
<jw4> boom
<jw4> :)
<natefinch> at least it's recorded now
<mup> Bug #1456703 was opened: action-set converts everything to strings <actions> <juju-core:New> <https://launchpad.net/bugs/1456703>
<natefinch> way to be on the ball there, mup
<jw4> haha
 * natefinch really needs pyformat
<natefinch> python peeps... is there an official or semi official pyformat?
<natefinch> I see pyformat, I see autopep8 ... what should I use and do I have to worry that it'll blow up my code?
<ericsnow> natefinch: nothing official
<voidspace> natefinch: autopep8 is the closest
<natefinch> good enough for me
<rick_h_> natefinch: run flake8 and that's the basics
<rick_h_> natefinch: it's what we've been using + a few manual ideas copied from LP days for all our python stuff
<mup> Bug #1456714 was opened: assignCleanSuite.TearDownTest fails <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1456714>
<katco> wallyworld: hey got a few minutes to talk about a spec?
<wallyworld> katco: yeah, give me a minute
<katco> wallyworld: ack
<katco> wallyworld: doh, nevermind the family just got home. i'll catch up with you later
<wallyworld> ok, ping me , i ave a meeting in an hour but free after that
<thumper> cmars: around?\
<cmars> thumper, hey
<cmars> thumper, yeah, hang on
<thumper> cmars: I'm in our hangout
<noodles775> wwitzel3: Hi! Just wondering if the reason you switched the status of bug 1453644 was related to the bug itself, or just simply you've got other priorities you need to switch to etc.
<mup> Bug #1453644: destroy-env leaves one lxc which remains pending on redeploy <destroy-environment> <local-provider> <lxc> <reliability> <repeatability> <juju-core:Triaged by wwitzel3> <https://launchpad.net/bugs/1453644>
<davecheney> thumper: git bisect success
<davecheney> machine handed off to rsc
<davecheney> i'm off for a coffee
<thumper> \o/
<davecheney> and the offending diff looks like a plausible candidate this time
<davecheney> not a change to an unrelaed .h file
<mwhudson> davecheney: \o/
<wallyworld> axw: i can't get the meeting link to work, can you?
<axw> wallyworld: nope
<sinzui> wallyworld, do you have a moment to review http://reviews.vapour.ws/r/1729/
<wallyworld> sure
<sinzui> wallyworld, 386 unit tests might fail for 1.24. The failures for the first 2 tries are flaky, so I am declaring build 2664 releasable for beta4.
<wallyworld> \o/
<thumper> yay
<thumper> wallyworld: I may be slightly late to the 1.23 retrospective, but I will be there
<wallyworld> ok
<davecheney> mwhudson: thanks for breaking the seal on the 'compiler is slow argument, let's no gc'
<davecheney> i was thinking of writing the same to russ privately
<davecheney> but public is better for everyone's karma
#juju-dev 2015-05-20
<mwhudson> yeah, i was profiling the linker and it stabs you in the face really
<davecheney> it's all work that the 1.4 compilers never did
<mwhudson> indeed
<davecheney> and in my testing, that acconts for 3x slowdown
<mwhudson> GOGC=off makes things about 40% faster in my tests i think
<mwhudson> varies a bit with case, of course
<davecheney> yup
<mwhudson> not sure what a good sort of gc is for this sort of thing really
<davecheney> none, it's a pathlogical case
<davecheney> gc only works for allocations you intend to free
<davecheney> preferably shortly
<mwhudson> yeah
<davecheney> i suspect the biggest problem to GOGC=off or using off heap memory will be political
<mwhudson> generational would help a bit, but all the new Nodes get old->young pointers immediately
<mwhudson> which kinda screws over the generational hypothesis
<davecheney> generational collectors are good for avoiding heap fragmentation
<davecheney> apart from that, they actaully don't help
<davecheney> as anything which uses a lot of memory, by defintition has big data structures which are long lived
<davecheney> think reddis, memcache, casandra
<mwhudson> they help if you generate piles of short lived garbage
<davecheney> all of those get promotoed, or are reachable from the promoted set
<mwhudson> e.g. if you are writing python
<davecheney> they are good for request/response servers
 * mwhudson spots some java experience
<davecheney> where they genrate a lot of allocations relative to the incoming connection, then free them at the end
<davecheney> mwhudson: i'll show you the place that Java touched me, later
<davecheney> welp, keith is intersted at least
<davecheney> i don't really care about the solution
<davecheney> only they engage with the idea that the gc is not helping the compiler
<davecheney> menn0: thumper http://paste.ubuntu.com/11234864/
<davecheney> uh oh
<davecheney> this is our old friend
 * menn0 looks
<menn0> davecheney: where are you seeing that?
<davecheney> failure on ppc64
<davecheney> .../jujud/agent
<menn0> davecheney: is it consistent?
<davecheney> menn0: yup
<davecheney> i smell a data race
<thumper> anastasiamac, wallyworld: I need to run off in about 10 minutes, pepper is booked in for a hair cut
<thumper> :)
<mwhudson> davecheney: turns out being in a hangout makes compiles even sloweer
<natefinch> hangouts DEFINITELY slow down compilation (and everything else)
<davecheney> menn0: go test -race .../jujud/agent
<davecheney> OK: 80 passed, 1 skipped
<davecheney> PASS
<davecheney> Found 13 data race(s)
<natefinch> since it's a prime number, they cancel out, right?
<davecheney> natefinch: so, should I add more races, or take some away ?
<natefinch> davecheney: no no, if you fix one, it won't be prime, and they won't cancel out anymore
 * natefinch is sure that's a thing.
<davecheney>   github.com/juju/juju/apiserver.(*changeCertConn).Read()
<davecheney>       <autogenerated>:37 +0xa3
<davecheney> did someone recently add support to the apiserver to change certificates on the fly ?
<natefinch> davecheney: on the upside, coverage of jujud/agent is better than I thought, 78.5% - which means we're only probably missing ~3 data races not covered in tests
<davecheney> natefinch: superb
<menn0> davecheney: wallyworld added the cert swapping thing
<menn0> it's needed to support upgrade IIRC
<menn0> s
<wallyworld> recently = 1.22
<wallyworld> it's needed to support secure connections to state servers from cloud nodes
<davecheney> excellent
<davecheney> it's not clear if those races are just in the tests
 * davecheney throws table at mocking functions at test time
<davecheney> but it's certainly a big problem
<davecheney> data race == program is unknowable
<davecheney> that could explain the runtime crashes we see
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1456851
<mup> Bug #1456851: cmd/jujud/agent: multiple data races detected <juju-core:Confirmed> <https://launchpad.net/bugs/1456851>
<mup> Bug #1222413 changed: openstack provider Instances suppresses errors <openstack-provider> <tech-debt> <juju-core:Fix Released by gz> <juju-core 1.24:Fix Released by gz> <https://launchpad.net/bugs/1222413>
<mup> Bug #1450129 changed: vsphere provider is missing firewaller, networking implementations <tech-debt> <vsphere-provider> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1450129>
<mup> Bug #1450701 changed: Juju CLI compatibility option <status> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1450701>
<mup> Bug #1451283 changed: deployer sometimes fails with a unit status not found error <blocker> <ci> <intermittent-failure> <landscape> <regression> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1451283>
<mup> Bug #1452114 changed: Unnecessary errors emitted during init system discovery <systemd> <upstart> <vivid> <juju-core:Fix Released by wwitzel3> <juju-core 1.23:Fix Released by wwitzel3> <juju-core 1.24:Fix Released by wwitzel3> <https://launchpad.net/bugs/1452114>
<mup> Bug #1452535 changed: default storage constraints are not quite correct <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1452535>
<mup> Bug #1453801 changed: /var/spool/rsyslog grows without bound <stakeholder> <juju-core:Fix Released by axwalk> <juju-core 1.22:Fix Committed by axwalk> <juju-core 1.23:Fix Committed by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1453801>
<mup> Bug #1454043 changed: InstancePoller compares wrong Address list and always requests updated state Addresses <cpec> <network> <stakeholder> <juju-core:Fix Released by thumper>
<mup> <juju-core 1.22:Fix Committed by thumper> <juju-core 1.23:Fix Committed by thumper> <juju-core 1.24:Fix Released by thumper> <https://launchpad.net/bugs/1454043>
<mup> Bug #1454676 changed: failed to retrieve the template to clone - 500 Internal Server error - error creating container juju-trusty-lxc-template - <oil> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1454676>
<mup> Bug #1454829 changed: 1.20.x client cannot communicate with 1.22.x env <blocker> <compatibility> <status> <juju-core:Fix Released by wallyworld> <juju-core 1.22:Fix
<mup> Committed by wallyworld> <juju-core 1.23:Fix Committed by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1454829>
<mup> Bug #1456851 was opened: cmd/jujud/agent: multiple data races detected <juju-core:Confirmed> <https://launchpad.net/bugs/1456851>
<menn0> thumper: ok to merge this? https://github.com/juju/txn/pull/10
<menn0> no landing bot it seems
<menn0> thumper: i have permission to merge, just want someone else to agree
<davecheney> thumper: um, we have a serious problem
<davecheney> the cert change listner basically doesn't work
<davecheney> and cannot be fixed in its current form
<davecheney> thumper: menn0 https://bugs.launchpad.net/juju-core/+bug/1456857
<mup> Bug #1456857: apiserver: updateCert has data race, corrupts certificate information <juju-core:New> <https://launchpad.net/bugs/1456857>
<menn0> wallyworld: ^^^
<mup> Bug #1415176 changed: debug-hooks exit 1 , doesn't mark hook as failed <cts> <debug-hooks> <juju-core:Fix Released by hduran-8> <juju-core 1.23:Fix Released by hduran-8> <juju-core 1.24:Fix Released by hduran-8> <https://launchpad.net/bugs/1415176>
<mup> Bug #1420057 changed: agents see "too many open files" errors after many failed API attempts <juju-core:Fix Released by dave-cheney> <juju-core 1.22:Fix Committed
<mup> by dave-cheney> <juju-core 1.23:Fix Committed by dave-cheney> <juju-core 1.24:Fix Released by dave-cheney> <https://launchpad.net/bugs/1420057>
<mup> Bug #1429790 changed: debug-hooks not working with manually provisioned machines <debug-hooks> <manual-provider> <juju-core:Fix Released by alesstimec> <juju-core 1.24:Fix Released by alesstimec> <https://launchpad.net/bugs/1429790>
<mup> Bug #1437266 changed: Bootstrap node occasionally panicing with "not a valid unit name" <deploy> <destroy-machine> <destroy-service> <juju-core:Fix Released by themue> <juju-core 1.24:Fix Released by themue> <https://launchpad.net/bugs/1437266>
<mup> Bug #1441206 changed: Container destruction doesn't mark IP addresses as Dead <destroy-unit> <network> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <https://launchpad.net/bugs/1441206>
<mup> Bug #1441913 changed: juju upgrade-juju failed to configure mongodb replicasets <canonical-is> <mongodb> <upgrade-juju> <juju-core:Fix Released by menno.smits> <juju-core 1.23:Fix Committed by menno.smits> <juju-core 1.24:Fix Released by menno.smits> <https://launchpad.net/bugs/1441913>
<mup> Bug #1442012 changed: persist iptables rules / routes for addressable containers across host reboots <addressability> <network> <juju-core:Fix Released by dooferlad> <juju-core 1.23:Fix Committed by dooferlad> <juju-core 1.24:Fix Released by dooferlad> <https://launchpad.net/bugs/1442012>
<mup> Bug #1444861 changed: Juju 1.23-beta4 introduces ssh key bug when used w/ DHX <blocker> <dhx> <regression> <ssh> <juju-core:Fix Released by hduran-8> <juju-core 1.23:Fix Released by hduran-8> <juju-core 1.24:Fix Released by hduran-8> <https://launchpad.net/bugs/1444861>
<mup> Bug #1446264 changed: joyent machines get stuck in provisioning <bootstrap> <joyent-provider> <reliability> <repeatability> <juju-ci-tools:Fix Released by gz> <juju-core:Fix Released by gz> <juju-core 1.23:Fix Committed by gz> <juju-core 1.24:Fix Released by gz> <https://launchpad.net/bugs/1446264>
<mup> Bug #1449301 changed: storage: storage cannot be destroyed <storage> <tech-debt> <juju-core:Fix Released by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1449301>
<mup> Bug #1449390 changed: storage: charms must wait for storage to be attached before running "install" hook <storage> <tech-debt> <juju-core:Fix Released by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1449390>
<mup> Bug #1449822 changed: storage: storage-detached should be storage-detaching <storage> <tech-debt> <juju-core:Fix Released by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1449822>
<mup> Bug #1450118 changed: vsphere provider should use OVA instead of OVF from cloud images. <tech-debt> <vsphere-provider> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1450118>
<mup> Bug #1451674 changed: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <golang> <upgrade-juju> <vivid> <juju-core:Fix Released by menno.smits> <juju-core 1.23:Fix Released by menno.smits> <juju-core 1.24:Fix Released by menno.smits> <https://launchpad.net/bugs/1451674>
<mup> Bug #1452113 changed: log files are lost when agents are restarted under systemd  <regression> <systemd> <vivid> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.23:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1452113>
<mup> Bug #1452207 changed: worker/uniter: charm does not install properly if storage isn't provisioned before uniter starts <storage> <juju-core:Fix Released by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1452207>
<mup> Bug #1452511 changed: jujud does not restart after upgrade-juju on systemd hosts <regression> <systemd> <vivid> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.23:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1452511>
<mup> Bug #1454481 changed: juju log spams ERROR juju.worker.diskmanager lsblk.go:111 error checking if "sr0" is in use: open /dev/sr0: no medium found <juju-core:Fix Released by axwalk> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1454481>
<mup> Bug #1454599 changed: firewaller gets an exception if a machine is not provisioned <cpec> <stakeholder> <juju-core:Fix Released by hduran-8> <juju-core 1.22:Fix
<mup> Committed by hduran-8> <juju-core 1.23:Fix Committed by hduran-8> <juju-core 1.24:Fix Committed by hduran-8> <https://launchpad.net/bugs/1454599>
<mup> Bug #1454870 changed: Client last login time writes should not use mgo.txn <tech-debt> <juju-core:Fix Released by thumper> <juju-core 1.22:Fix Committed by thumper> <juju-core 1.23:Fix Committed by thumper> <juju-core 1.24:Fix Released by thumper> <https://launchpad.net/bugs/1454870>
<mup> Bug #1456857 was opened: apiserver: updateCertificate has data race, corrupts certificate information <juju-core:New> <https://launchpad.net/bugs/1456857>
<thumper> spammy mup
<thumper> menn0: my guess is no bot
<thumper> menn0: so just merge it (assuming all tests pass :-)
<menn0> thumper: yep they do
<davecheney> protip: whenever you use PatchValue, you're probably creating a data race
<davecheney> please don't use PatchValue
<wallyworld> davecheney: what do you mean doesn't work? if it didn't work, lxc image caching would not work
<wallyworld> data race != doesn't work
<marcoceppi> hey, emergency for demo
<marcoceppi> getting this error
<marcoceppi> WARNING failed to load charm at "/home/ubuntu/charms/trusty/rally": YAML error: line 20: did not find expected key
<marcoceppi> kind of cryptic error
<wallyworld> what version of juju? what is charm yaml?
<marcoceppi> 1.24-beta3
<marcoceppi> https://github.com/juju-solutions/rally
<wallyworld> marcoceppi: sadly the error is from inside the yaml lib and it has been provided with no context :-(
<wallyworld> marcoceppi: looks like action yaml
<marcoceppi> wallyworld: yeah, missing "
<marcoceppi> wallyworld: thanks@
<marcoceppi> never have seen that error before
<wallyworld> np
<marcoceppi> and proof didn't pick it up
<wallyworld> something to fix :-)
<mup> Bug #1454466 was opened: Deployment times out waiting for relation convergence - neutron-gateway in installing state <oil> <juju-core:New> <juju-deployer:Invalid> <neutron-gateway (Juju Charms Collection):New> <https://launchpad.net/bugs/1454466>
<natefinch> waigani_: you around?
<waigani_> natefinch: yep
<natefinch> waigani_: #1 - thanks for fixing my dumb windows bug in the log rotation tests
<waigani_> natefinch: np :)
<natefinch> waigani_: #2 - since you wrote a script to run CI, maybe you can help me get my CI script running.... what should "job_name" be?  my new test is log_rotation.py - does that mean the job name should be log_rotation?  or log_rotation.py  or something else entirely?
<waigani_> natefinch: from what I can see, job_name is just used to name the environment
<waigani_> natefinch: so the CI guys can keep track of what envs are for what jobs I'm guessing
<natefinch> ahh ok.. do you know if there's a way to run just my one test, and not all of CI?
<waigani_> natefinch: what's your one test?
<jam> menn0: I had a couple quick thoughts about your scanner patch, are you around?
<waigani_> natefinch: is your script up somewhere I can have a squizz?
<menn0> jam: yep I'm here
<natefinch> waigani_: I wrote a python script following the pattern set by some of the other tests, like assess_bootstrap.py
<natefinch> waigani_: here's the code: https://gist.github.com/natefinch/e377eacd6b2316b2a884
<jam> menn0: so one thing I was thinking about is that since we have to read the whole DB it really is a bit too often to do it every 2hrs (i think). so I was trying to think of ways to make it more logical
<jam> menn0: one that I thought could be really good is to track how many txns are in the collection and only prune when they grow by a certain amount
<jam> like say 2x
<jam> menn0: as this is really "don't let TXNs grow without bound and take up 99.99% of the total DB size"
<jam> but since there is one TXN for every other doc, it is fair to expect TXN to be as much as 50% of the total DB size.
<jam> ideally we would record the size of the collection after the last pruning
<natefinch> waigani_: it requires a new test charm that exists here for now: https://github.com/natefinch/fill-logs
<jam> and then only once it has bloated do the next GC
<jam> as a poor man's approximation we could track that info in memory
<natefinch> waigani_: just needs to be manually copied into repository (again, just for now)
<jam> (always GC on the first inspection, track the final size of the DB, and then only GC again once the count() is 2x the original count())
<jam> or whatever is the cheapest thing to measure.
<jam> We could do db.txn.stats() but I'm not sure if that shrinks after a big prune
<menn0> jam: hmmm ok
<natefinch> waigani_: ah crap, the script requires modifications to the ci-tools that aren't pushed yet
<menn0> jam: i'm pretty sure the stats tell you the allocated size and the in-use size
<waigani_> natefinch: so, when it's ready, the test charm should probably live here: lp:juju-ci-tools/repository
<jam> menn0: so db.collection.stats() if it accurately tracks what pages are in use would probably be a very cheap chekck
<jam> check
<natefinch> waigani_: yep, got that. But I need to be able to test it to know if it's ready :)    Actually, I've done a lot of manual testing on the charm, so pretty sure it's fine.
<jam> menn0: db.txn.count() *could* be cheap depending on how mongo tracks documents.
<menn0> jam: count is very cheap
<menn0> jam: it's tracked separately
<jam> menn0: I think in mongo 3 because of MVCC it changes to be not so cheap
<natefinch> waigani_: deploy_job is giving me ImportError: No module named boto   .... where do I get boto, pip?
<menn0> jam: that makes sense
<waigani_> natefinch: sure. With the JES stuff, I took the approach of writing a new job - using deploy_job.py as a template
<jam> menn0: anyway, if count is cheap today we can go with it
<natefinch> waigani_: oh interesting
<jam> as we're a fair bit off, perhaps a code comment to evaluate if this stays cheap.
 * menn0 nods
 * natefinch grumbles that this whole "write a CI test" thing would be a hell of a lot easier if there were documentation about how to do it.
<waigani_> natefinch: so I've got deploy_jes_job.py - which builds ontop of deploy_job.py
<menn0> jam: well the pruning change - mostly as you reviewed it - is merging for 1.22 as we speak :)
<menn0> jam: but i can iterate
<waigani_> natefinch: so maybe you could do something similar?
<menn0> jam: basically what you're after is: only prune if there's actual useful gains to be made
<menn0> so that we're not loading the whole DB unecessarily
<jam> menn0: k. so my thoughts are generally that we want to have *some* GC so that things don't grow without bound, but obviously we can functionally cope with a fair amount of garbage, and we don't want to saturate our system just checking for garbage that isn't there.
<jam> menn0: right
<jam> menn0: the fact that our current GC is really expensive because we aren't doing incremental
<jam> menn0: I was going to say we could just drop the poll time to 1/day or 1/week even
<jam> but doing it when we expect to be able to clean things seems a better path
<natefinch> waigani_: yeah, I can look into that
<natefinch> I love the way every time something goes wrong with a python script I get a huge useless stack trace
<menn0> jam: so to recap: track the count of the txns collection after each prune, and only try to prune if the count grows to 2x the previous value
<menn0> jam: (and prune the first time if there's no count recorded)
<wallyworld> natefinch: as opposed to a panic in go? the stack trace is useful for diagnosis :-)
<natefinch> so, the CI script gives me "ImportError: No module named boto" pip install boto givees me "ImportError: cannot import name IncompleteRead"  ...  my kingdom for a statically linked binary that just f'ing works.
<jam> menn0: right. IMO ideally we would save the count after the last GC so that we don't always GC on startup
<jam> but we can live with that
<natefinch> wallyworld: a stack trace from pip is useless to the end user
<natefinch> (i.e. me)
<natefinch> it's just ugly
<wallyworld> so are go panics
<jam> menn0: also, I wanted to make sure that you don't GC immediately on startup (while load is the greatest on the machine), but I'm pretty sure you don't
<natefinch> wallyworld: your code shouldn't panic unless there's something hugely drastically wrong
<jam> menn0: can you make sure there is a test that you don't GC immediately ?
<menn0> jam: the first prune doesn't happen until 2hrs after startup anyway
<natefinch> wallyworld: like, programmer error, generally
<natefinch> wallyworld: python scripts throw exceptions if you look at them the wrong way
<wallyworld> natefinch: same with python - the programmer is just lazy not to deal with the error
<menn0> jam: and if the count-at-last-prune is kept in the DB then it'll only happen when we want it to
<natefinch> wallyworld: then almost every python programmer ever is lazy
<jam> menn0: so in a healthy system (once we've fixed the address updater bug), I don't think we'll generate much garbage.
<jam> menn0: like, I would expect it to take us weeks to actually grow to 2x
<wallyworld> natefinch: and go programmers aren't?
<menn0> jam: agreed
<jam> menn0: I'd probably like INFO level logs that a GC is actually started (since we wouldn't be GC every 2 hrs)
<jam> menn0: the only reason not to record it in the DB is that we don't have a great place to put the info.
<menn0> jam: there is a log at debug but I can bump it
<natefinch> wallyworld: sure they are.  But the default in go is not to show a huge useless ugly stack trace to your users.
<menn0> jam: I might add a txns.prune collection or something
<natefinch> wallyworld: regardless.... do you know how to fix this problem?  I presume the proper way to get boto is through pip, and yet my pip seems sad.
<menn0> jam: or better yet txns.gc
<wallyworld> natefinch: same with python if you wrap main in a try catch, all of 3 lines of code
<waigani_> natefinch: as far as running one test, looking at assess_bootstrap.py you can execute that directly
<jam> menn0: so I think an actually informative single-line message every 2 hrs would be ok. "checked for pruning transactions, found 1M current vs 0.5M old"
<menn0> jam: with just a single doc
<waigani_> natefinch: python assess_bootstrap.py $(which juju) local
<wallyworld> natefinch: i've not sed boto sorry
<wallyworld> used
<waigani_> natefinch: that works for me - it's currently bootstrapping on my local machine
<jam> menn0: so if we're going to create a table, I think recording the results of actual GC runs would also be useful
<jam> for being able to track "how fast am I growing garbage, how often is GC actually running, etc"
<natefinch> waigani_: ah, ok.  I tried to ask mgz and sinzui that... but I think I confused them by not wanting to run everything
<jam> menn0: *not* recording every 2hrs, but recording the actual successful runs.
<jam> menn0: thoughts?
<natefinch> waigani_: do you know how to get boto?
<menn0> jam: yeah I guess that would be nice
<menn0> jam: although that collection would grow without bounds  :)
<jam> menn0: so we *could* just put that info into logs and then get it from log scraping
<jam> menn0: but with log rotation you are likely to never have enough history to be useful.
<menn0> jam: yeah... i'm not sure it'll be that valuable in the db
<jam> menn0: it does, hence the "only record successful runs"
<jam> menn0: I would also be ok with capping the amount of history if we are worried about it
<waigani_> natefinch: I must have already had that, I didn't hit that err. Have you tried pip?
<jam> say 1000 successful GC runs should be big enough for anyone :)
<natefinch> waigani_: yeah, my pip is broken, too.  Google/stackoverflow says easy_install -U pip  .... which is hilarious
<menn0> jam: until it's not :)
<jam> menn0: hence why I used that phrasing
<menn0> jam: but seriously... that's not a major concern
<waigani_> haha
<jam> menn0: so some of it is protecting us against the unknown
<jam> knowing that X grows without bound and we may have an env that runs for 10 years
<jam> menn0: also multi-environment state servers magnify this sort of problem.
<jam> menn0: that's actually one of the bigger reasons to not read-the-world every 2 hrs
<jam> menn0: as you start removing the constraint that the DB is small.
<menn0> jam: yep - agreed
<natefinch> https://twitter.com/gardaud/status/357638468572151808
<jam> natefinch: how do you get easy_install "apt-get install easy_install" :)
<jam> (not actually true)
<natefinch> jam: right?  :) I forget how I got easy_install... I'm sure I got it just to install pip
<jam> natefinch: I believe easy_install is a single python file so many people 'wget' it
<jam> natefinch: in proper security fashion: wget https://bootstrap.pypa.io/ez_setup.py -O - | python
<jam> https://pypi.python.org/pypi/setuptools
<jam> natefinch: I guess at least now they have you download it over HTTPS it used to be raw HTTP I believe.
<natefinch> jam: well, at least that
<jam> natefinch: though they also give you the "--no-check-certificate" and "--insecure" version of installation just to make sure you can root yourself to the world.
<jam> menn0: does it seem reasonable to do the "only GC if big enough" ? I don't want to spend huge amounts of time on it, but it seemed a pretty cheap win
<menn0> jam: i think it's worth doing
<menn0> jam: i'd feel a lot less worried about this going out if that was in place
<menn0> jam: aside from the potential i/o load i was concerned about what this would do to mongodb cached pages. it could hurt performance for a while.
<menn0> (after each prune)
<jam> menn0: did gustavo ever comment on the pruning changes to mgo/txn ?
<jam> menn0: IIRC we also had some patches to PruneMissing that would also GC the txn-queues
<jam> rogpeppe I think had done that.
<jam> anyway /me needs to go run some errands. be back later
<menn0> jam: unfortunately i haven't been able to get a response from gustavo so far
<menn0> jam: i have a PR with that fixes PurgeMissing for the "huge txn-queues" situation (and various drive-by fixes)
<menn0> jam: and he hasn't looked at the pruning fixes
<wallyworld> axw_: reviewed, i've been adding todo cards to the backlog, some of which your branch now obsoletes. there's a few still to add. one i highlighted as high priority is an api compatability issue if we ship 1.24 with it
<axw_> wallyworld: thanks. I will take a look after I address your comments
<wallyworld> axw_: ok, np, i'm relocating so will be afk for a bit
<axw> okey dokey
<natefinch> ahh, python... http://cdn.meme.am/instances/500x/62360284.jpg
<natefinch> if python were compiled, I'd be in bed by now :/
<natefinch> sonofa .... juju action evidently does not take the -e <provider> flag
<natefinch> er, environment, not provider
<rogpeppe> mgz: hiya
<dimitern> mgz, ping
<TheMue> hmm, got no camera and sound, will restart browser
<TheMue> aaargh, fan spins up and browser blocks :(
<mup> Bug #1456957 was opened: rsyslog worker should not add machines that are not ready yet <cpec> <logging> <rsyslog> <juju-core:Triaged> <https://launchpad.net/bugs/1456957>
<TheMue> dooferlad: is that you http://www.reddit.com/r/golang/comments/36lf6o/golang_and_openid/ ?
<dooferlad> TheMue: no
<TheMue> dooferlad: sadly so far no good answer there
<mup> Bug #1456989 was opened: cloud-init 0.6.3 on precise generates invalid apt-get install command line <cloud-init> <precise> <regression> <juju-core:Triaged by dimitern> <juju-core 1.24:Triaged by dimitern> <https://launchpad.net/bugs/1456989>
<mgz> dimitern: hey
<dimitern> mgz, hey, I've filed the bug above ^^ which might be causing some CI failures wrt precise
<mgz> dimitern: interesting - for real deployment right, not unit tests?
<dimitern> mgz, yes
<dimitern> mgz, also I wanted to ask about that PR you reverted yesterday re init discovery
<mgz> 1.24/master have been fine on CI since the centos change - and we do have precise testing charms
<mgz> dimitern: see the last 1.23 run for full breakge
<mgz> I only reverted on 1.24 for the release, so 1.23 and master runs will still be borked
<dimitern> mgz, I'm running into an issue where I get http://paste.ubuntu.com/11243099/
<mgz> http://reports.vapour.ws/releases/2666
<mgz> dimitern: yup, that's the deployment breakage
<dimitern> mgz, so far only on 1.24 - and seems related to https://github.com/juju/juju/pull/2359
<dimitern> mgz, what's interesting is that the same code is also on master (https://github.com/juju/juju/pull/2358), but I'm not seeing the same issue
<mgz> dimitern: I don't see that *after* the revert
<dimitern> mgz, the "[[: not found" error?
<mgz> dimitern: fun, we probably need a master through CI then
<mgz> dimitern: the breakage at all, the last 1.24 run was clean (ish - precise unit tests still failed, known flakiness)
<dimitern> mgz, yeah, I'm currently trying to file a bug for that, but I'm having a bit of trouble pinning it down exactly
<mgz> dimitern: one reason I just did the revert... I should have filed a bug after though
<dimitern> mgz, it seems that when I reproduce the issue, replacing [[ and ]] with [ and ] in that init discovery script solves the errors
<mgz> I wonder if this is a shebang issue
<dimitern> mgz, and the reason is simple - http://paste.ubuntu.com/11243163/
<mgz> I *thought* we were careful to use bash for everything though
<dimitern> mgz, the script is rendered with a #!/usr/bin/env bash shebang
<dimitern> mgz, however runcmd in cloud-init starts with a #!/bin/sh shebang
<dimitern> you see where this is getting..
<mgz> mehe, okay, well, that's the bug then
<dimitern> the repercussions are potentially enormous - any runcmd script that requires bash in cloud-init user-data has to be pre-rendered somewhere and executed, rather than included inline like $(...)
<mgz> we can also just not use bashism
<dimitern> damn right :)
<dimitern> mgz, FYI - bug 1456989
<mup> Bug #1456989: cloud-init 0.6.3 on precise generates invalid apt-get install command line <cloud-init> <precise> <regression> <juju-core:Triaged by dimitern> <juju-core 1.24:Triaged by dimitern> <https://launchpad.net/bugs/1456989>
<mgz> dimitern: I saw, thanks
<dimitern> mgz, no, sorry not that one
<mgz> so, my inclination is to go ahead and back the chane out on 1.23 and master as well
<mgz> I saw mup say it in another channel :)
<mgz> bug 1457011
<mup> Bug #1457011: init system discovery script fails with: [[: not found <cloud-init> <compatibility> <regression> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1457011>
<dimitern> mgz, that one yes
<mup> Bug #1457011 was opened: init system discovery script fails with: [[: not found <cloud-init> <compatibility> <regression> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1457011>
<mup> Bug #1457022 was opened: state server panic: "rescanned document misses transaction in queue" <cpec> <mongodb> <juju-core:In Progress by fwereade> <juju-core
<mup> 1.22:In Progress by fwereade> <juju-core 1.23:In Progress by fwereade> <juju-core 1.24:In Progress by fwereade> <https://launchpad.net/bugs/1457022>
<mattyw> perrito666, https://github.com/juju/charm/pull/129
<perrito666> mattyw: you are loosing communication skills :p
<perrito666> ah, you want me to merge that :p
<mattyw> perrito666, just letting you know it's been updated :)
<mattyw> perrito666, I've pinged you about others that I think can be closed
<perrito666> mattyw: yes, sorry notifications of github get lost on the sea of notifications
 * perrito666 sees no other pings from mattyw 
<mattyw> perrito666, I prefer calling them github "notifications"
<perrito666> mattyw: I call them spam
<mup> Bug #1457031 was opened: Juju cannot deploy to any substrate <blocker> <bootstrap> <ci> <regression> <juju-core:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1457031>
<perrito666> mattyw: done
<perrito666> well look at that, this call is full
<TheMue> perrito666: yes, just tried it too
<perrito666> odd
<TheMue> perrito666: have been in another meeting so far and now cannot jump into this one
<perrito666> seems that google is not that smart into letting you know if you invite more people than possible
<perrito666> and there isnt a way to be just an expectator
<mup> Bug #1457031 changed: Juju cannot deploy to any substrate <blocker> <bootstrap> <ci> <regression> <juju-core:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1457031>
<perrito666> ohhh, what now
<TheMue> perrito666: see canonical #juju, they trey something with bundling lines
<mup> Bug #1457031 was opened: Juju cannot deploy to any substrate <blocker> <bootstrap> <ci> <regression> <juju-core:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1457031>
<katco> fyi perrito666's internet has gone down
<fwereade> katco, ericsnow, wwitzel3: sorry, can we push our meeting back 30 mins please?
<katco> fwereade: certainly
<mup> Bug #1457022 changed: state server panic: "rescanned document misses transaction in queue" <cpec> <mongodb> <juju-core:In Progress by fwereade> <juju-core 1.22:In Progress
<mup> by fwereade> <juju-core 1.23:In Progress by fwereade> <juju-core 1.24:In Progress by fwereade> <mgo:In Progress by fwereade> <https://launchpad.net/bugs/1457022>
<natefinch> python peeps... is there some static analysis tool that'll tell me when I've typoed function names etc?  You know, the stuff you get for free from a compiler?
<ericsnow> natefinch: pyflakes
<Spads> flake8
<wwitzel3> tests?
<ericsnow> natefinch: yeah, that one's better ^^
<Spads> natefinch: as a bonus, flake8 can do McCabe cyclical complexity metrics
<ericsnow> wwitzel3: +1 :)
<natefinch> wwitzel3: I was waiting for that one... but I'm *writing* tests... where do the tests end?
<natefinch> how do I test the tests?
<natefinch> of the tests
<ericsnow> natefinch: it's tests all the way down
<natefinch> evidently
<mup> Bug #1457022 was opened: state server panic: "rescanned document misses transaction in queue" <cpec> <mongodb> <juju-core:In Progress by fwereade> <juju-core 1.22:In Progress
<mup> by fwereade> <juju-core 1.23:In Progress by fwereade> <juju-core 1.24:In Progress by fwereade> <mgo:In Progress by fwereade> <https://launchpad.net/bugs/1457022>
<natefinch> if I have to do sudo pip install,  does that mean that I've screwed up my environment?
<natefinch> or is that correct?
<wwitzel3> natefinch: not, it just means you haven't explicitly isolated your environment and you are installing packages in to the system Python
<natefinch> if I don't sudo, I get some massive traceback
<mup> Bug #1457022 changed: state server panic: "rescanned document misses transaction in queue" <cpec> <mongodb> <juju-core:In Progress by fwereade> <juju-core 1.22:In Progress
<mup> by fwereade> <juju-core 1.23:In Progress by fwereade> <juju-core 1.24:In Progress by fwereade> <mgo:In Progress by fwereade> <https://launchpad.net/bugs/1457022>
<perrito666> yey, internet is sort of back
<dimitern> natefinch, pip is supposed to be run inside a virtualenv
<dimitern> natefinch, that might be the problem
 * dimitern waves at voidspace 
<natefinch> I really don't care enough to mess with virtualenv
<perrito666> and off course, you look for something about vi in google and it returns answers about rick_h_ very often
<perrito666> btw rick_h_ your screencast about bundle jugler is down
<perrito666> natefinch: you will be sorry whenever you try to do something else and your system python is al screwed
<natefinch> perrito666: I'll just complain about how much python sucks and make you fix it ;)
<natefinch> s/make you/ask you nicely to/
<perrito666> you cannot do that anymore, now wallyworld does
<perrito666> :p
<wallyworld> what?
<dimitern> :D
<wallyworld> python is awesome, way better than Go
<dimitern> it feels like it's friday
<TheMue> ouch
<natefinch> wallyworld: I don't need 3 different package installer / environment handlers to run simple go code
<wallyworld> me either
 * TheMue fetches some cakes and coke and then watches the fight
<natefinch> easy_install, pip, virtualenv
<wallyworld> you don't have to have all those
<dimitern> nope, just the first two - the last is not a package manager
<wallyworld> i never have
<natefinch> except people complain that I don't have it, implying that to do it the "right way" I should be using it.
<dimitern> you can always use tarballs, like in the good old slackware days
<wallyworld> at least python has such things available
<natefinch> wallyworld: go doesn't have them because you don't need them
<wallyworld> lol
<wallyworld> yeah, just pull everything from tip, what could possibly go wrong
<wallyworld> who needs package management
<wallyworld> or config management
<wallyworld> or versioning
<katco> wallyworld: to be fair, go's solution to that is linking
<katco> wallyworld: it just hasn't landed yet
<wallyworld> don't get me started on static linking
<natefinch> wallyworld: what do you mean by config management?
<perrito666> katco: nanan, invalid point, if it is not there its not there
<katco> perrito666: it's there, just not in a release yet :)
<dimitern> just wait until 1.5 is out
<wallyworld> and you can still statically link bad code without proper versioning
 * perrito666 has dealt with academics too much to accept the answer "theoretically this is the solution, we just need to wait until the computer able to run it exists"
<dimitern> not only that - you would be able to do it on ppc64 and arm64 as well :D
 * wallyworld is too tired to argue anymore, need sleep
<katco> tc wallyworld
<wallyworld> next time we can discuss over drinks :-)
<katco> :)
<TheMue> just when it began to get funny
<wallyworld> TheMue: it's past 12am here :-) you can pick up the discussion and preach how good erlang is :-)
<dimitern> i'm outta here :D
<TheMue> wallyworld: good idea, or pony (but it's very young)
<TheMue> wallyworld: thanks for this great idea
<wwitzel3> the fastest way to a ruin a perfect language is to program something in it
<TheMue> so *, I'm open
<wallyworld> lol
<TheMue> did I evenr mentioned Smalltalk?
<TheMue> *duck*
<wwitzel3> I <3 Smalltalk, my first real programming job was Smalltalk.
 * TheMue hugs wwitzel3
<perrito666> mm, I think I lost the chance to insert the classic  C is the only real language
<perrito666> and get the classic answer
<perrito666> C is ASM for weaks
<katco> i'm enjoying learning common lisp. it's a neat language
<TheMue> I've once done Scheme and liked it. Always wanted to do Common Lisp too.
<perrito666> wasnt that a kathy perry song?
<perrito666> "I did Scheme and I liked it."
<katco> TheMue: if you do, do yourself a favor and get quicklisp first: https://www.quicklisp.org/beta/
<TheMue> katco: will try to remember
<katco> TheMue: CL has some cruft from the spec being ratified in the 80's, but it's actually a very practical language
<katco> lots of libraries
<TheMue> katco: currently I'm looking into pony http://www.ponylang.org/
<katco> yeah i saw your tweet and took a peek at it
<TheMue> katco: that's an actor base languag, very clean
<TheMue> hmm, loosing chars. "based language"
<katco> ericsnow: hey sorry for the time change. standup time
<ericsnow> katco: trying...hangouts is misbehaving for me
<katco> ericsnow: ah okl
<mup> Bug #1457068 was opened: bootstrap failed, no tools available <juju-core:New> <https://launchpad.net/bugs/1457068>
<katco> fwereade: we're ready to argue!
<katco> ;)
<katco> https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1
<katco> fwereade: there?
<natefinch> evilnickveitch: you around?
<mup> Bug #1457089 was opened: reboot request in charm hook context is silently ignored in the case of actions <juju-core:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1457089>
<evilnickveitch> natefinch, yup
<natefinch> evilnickveitch: nevermind, my question was answered by this bug: https://github.com/juju/docs/issues/405  Add "1.23" docs and update "devel" to 1.24.
<evilnickveitch> ok, cool
<natefinch> wwitzel3, ericsnow: is there documentation for GCE that should be going into jujucharms.com/docs?
<ericsnow> natefinch: just what's in the release notes
<natefinch> ericsnow: we need to convert that into a markdown document to put up on the webpage
<natefinch> katco: ^^
<natefinch> katco: sorry, the lack of docs is my fault, since it happened on my watch.
<katco> natefinch: thx for the ping, i'll add it to the backlog
<katco> ericsnow: wwitzel3: fwereade: such good conversation/work. i feel good about this direction.
<wwitzel3> katco: me too
<wwitzel3> jam: we never got a chance to meet with all the back-to-back meetings
<Johncr1> juju cannot be installed because of a possible issue with python-pygment package.
<ericsnow> mattyw: ping
<rick_h_> perrito666: :(
<mattyw> ericsnow, pong
<ericsnow> mattyw: could you take another look at http://reviews.vapour.ws/r/1733/?
<ericsnow> mattyw: also http://reviews.vapour.ws/r/1728/
<mattyw> ericsnow, would be my pleasure
<ericsnow> mattyw: thanks!
<ericsnow> wwitzel3: I'm going to take lunch early so I'll be back in about 1.5 hours
<mattyw> ericsnow, you mention an upcoming proper fix in the pr. when you say in that comment it just needs to be none windows, for now, do you mean until that fix?
<ericsnow> mattyw: correct
<wwitzel3> ericsnow: sounds good
<mattyw> ericsnow, can you mention the bug number in that comment, and say it will change when a fix for that bug lands?
<ericsnow> mattyw: sure
<mattyw> ericsnow, I'll take another quick look after that but basically LGTM
<ericsnow> mattyw: thanks again
<katco> ericsnow: are you looking into bug 1457031?
<mup> Bug #1457031: Juju cannot deploy to any substrate <blocker> <bootstrap> <ci> <regression> <juju-core:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1457031>
<katco> wwitzel3: has anyone reached out to natefinch to help with those CI tests?
<wwitzel3> katco: I can after I finish stuffing my face with food
<katco> wwitzel3: your priorities are correct sir! ;)
<natefinch> food sounds like a good idea, I'll do that too
<ericsnow> katco: I will after lunch
<mattyw> ericsnow, reviewed
<katco> ericsnow: cheers
<mattyw> evilnickveitch, ping?
 * perrito666 tries for week 3 to obtain a more decent internet provider.... the one I tried today cannot give me details over the internet they asked me to make a phone call...
<mup> Bug #1457122 was opened: local data dir handling for init services should be handled independently <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1457122>
<mup> Bug #1457124 was opened: Panic: FilterSuite.TearDownTest <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1457124>
<evilnickveitch> mattyw, pong
<wwitzel3> natefinch: ping
<natefinch> wwitzel3: let's jump on moonstone
<wwitzel3> natefinch: sounds good
<natefinch> wwitzel3: let me know if you have any questions or need help getting the environment set up
<wwitzel3> natefinch: yeah, I closed the hangout because it was sucking CPI
<wwitzel3> CPU
<natefinch> wwitzel3: basically the test just deploys my charm, runs the "add 300 megs of data to the unit agent log" action, and then runs the "return the size of all unit agent logs" action... and verifies the output
<natefinch> wwitzel3: totally understand
<wwitzel3> natefinch: so what should I be passing as an env?
<wwitzel3> natefinch: also JUJU_REPOSITORY , I assume that is pointing to your charm?
<natefinch> wwitzel3: so, I just run from the juju-ci-tools directory.  You need to checkout lp:juju-ci-tools/repository under the juju-ci-tools directory
<natefinch> then JUJU_REPOSITORY=./repository works
<natefinch> wwitzel3: you need to copy the charm dir under ./repository/trusty
<natefinch> er the fill-logs charm dir that is
<natefinch> and env is the name of an environment in your environments.yaml that you would like to deploy to
<wwitzel3> natefinch: got it, ok, running it now
<wwitzel3> natefinch: and what is the issue that needs resolving, it isn't clear from the LP ticket
<natefinch> wwitzel3: this is just a CI test for log rotation that I'm writing... right now it's timing out while running one of the actions... probably just not waiting long enough for the action to finish
<natefinch> wwitzel3: there's action_fetch and action_do that I added in jujupy.py which could potentially have problems as well... though they seem to be fine.
<natefinch> wwitzel3: just pushed a fix to the juju-ci-tools branch I'm working on
<natefinch> wwitzel3: ha, now it just passes entierly
<wwitzel3> natefinch: nice :)
<wwitzel3> natefinch: I'm getting a regex issue atm, but haven't tried your latest fix
<natefinch> wwitzel3: huh, I thought I fixed all the regex issues
<wwitzel3> natefinch: Exception: Rotated unit log name '/var/log/juju/unit-fill-logs-0.log' does not match pattern '/var/log/juju/unit-fill-logs-0-(.+?)\.log'.
<natefinch> wwitzel3: oh yeah, that was what I fixed
<natefinch> haha sorry
<wwitzel3> natefinch: ok, cool, running it now
<wwitzel3> natefinch: if I get a successful run then LGTM
<wwitzel3> natefinch: and I did .. no failures here
<natefinch> I have to add the machine log rotation checks, but that'll be mostly copy and paste
<natefinch> (and modify the regexes etc
<natefinch> and/or abstract out the differences
<natefinch> wwitzel3: anyway, I can finish that up, Thanks for verifying
<natefinch> \
<jam> wwitzel3: yeah, the meetings today took up our normal slots. are you around now?
<wwitzel3> jam: yep
<wwitzel3> natefinch: cool, np
<wwitzel3> natefinch: let me know if you need me to do any more verifying
<wwitzel3> jam: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=0
<ericsnow> wwitzel3: back
<ericsnow> wwitzel3: I have a couple bugs to look at really quickly
<wwitzel3> ericsnow: in moontsone talking to jam about the spec
<ericsnow> wwitzel3: k
<perrito666> I just heard this from a sales person at my ISP "We will need to replace your modem for one larger so the 50M we will provide you fit" (it is not awkwardly translated, in spanish the person actually spoke about volume)
 * perrito666 was forced to buy the biggest residential connection available to get 7M upload
<natefinch> heh
<natefinch> sorry
<natefinch> I think the CI test I just wrote actually found a bug in lumberjack
<natefinch> quite by accident, but still.. handy
<natefinch> https://github.com/natefinch/lumberjack/issues/12
<natefinch> well, guess I know what I'm working on tonight
<natefinch> Going to go make dinner, will be back in ~4.5 hours
<natefinch> (when the kids are asleep)
<mup> Bug #1457205 was opened: Subordinate charm Action data not reported by API <juju-core:New> <https://launchpad.net/bugs/1457205>
<katco> wwitzel3: hey where did you and natefinch-afk leave the CI tests?
<wallyworld> menn0: cherylj: is your work for a. txn fixes, and b. file handle leaks committed to 1.22?
<wallyworld> i see a txn fix merged to 1.22
<katco> wallyworld: see #juju@can
<menn0> wallyworld: yes I committed a fix for the txns issue which is good enough
<menn0> wallyworld: as it was merging jam started talked to me about some improvements
<wallyworld> menn0: ok, i'll mark the bug as fix committed, ty
<wallyworld> for 1.22 at least
<menn0> wallyworld: no hang on :)
<wallyworld> ok
<menn0> wallyworld: i've almost got the improvements ready
<wallyworld> rightio, we are waiting on another fix anyway
<menn0> wallyworld: i think it's worth getting those in to 1.22 as well
<wallyworld> ack
<menn0> it significantly lowers the performance hit of the pruning change
<menn0> wallyworld: are we aiming for the next 1.22 release today-ish?
<katco> menn0: i don't think so
<wallyworld> menn0: sorta - we are waiting for william's fix so it will likely be a bit later than just 1 day
<menn0> wallyworld, katco: cool. well this is my top priority regardless. I'll definitely be done with this today. (for 1.22 at least if not all the branches)
<wallyworld> ty :-)
<katco> menn0: you are, as always, awesome :D
<mup> Bug #1457218 was opened: failing windows unit tests <juju-core:In Progress by ericsnowcurrently> <juju-core 1.23:In Progress by ericsnowcurrently> <juju-core 1.24:In Progress by ericsnowcurrently> <https://launchpad.net/bugs/1457218>
<ericsnow> could I have someone look at the patches I have up for review (for critical bugs): http://reviews.vapour.ws/r/1737/ and http://reviews.vapour.ws/r/1738/
<ericsnow> I could also use a review on http://reviews.vapour.ws/r/1728/
<mup> Bug #1457225 was opened: Upgrading from 1.20.9 to 1.23.3 works, but error: runner.go:219 exited "machiner": machine-0 failed to set status started: cannot set status of machine "0": not found or not alive <cts> <sts-stack> <juju-core:New> <https://launchpad.net/bugs/1457225>
<perrito666> why, why oh god why is it so hard to write a proper unit test :(
<mup> Bug #1457225 changed: Upgrading from 1.20.9 to 1.23.3 works, but error: runner.go:219 exited "machiner": machine-0 failed to set status started: cannot set status of machine "0": not found or not alive <cts> <sts-stack> <juju-core:New> <https://launchpad.net/bugs/1457225>
<mup> Bug #1457225 was opened: Upgrading from 1.20.9 to 1.23.3 works, but error: runner.go:219 exited "machiner": machine-0 failed to set status started: cannot set status of machine "0": not found or not alive <cts> <sts-stack> <juju-core:New> <https://launchpad.net/bugs/1457225>
<cherylj> Can I get a review for the file handle leak bug 1454687: http://reviews.vapour.ws/r/1740/
<mup> Bug #1454687: add NX 842 hw compression patches <architecture-ppc64> <bot-comment> <bugnameltc-124979> <severity-medium> <targetmilestone-inin1510> <linux (Ubuntu):Triaged by arges> <https://launchpad.net/bugs/1454687>
<cherylj> oops, wrong bug
<cherylj> bug 1454697
<mup> Bug #1454697: jujud leaking file handles <cpec> <stakeholder> <juju-core:Triaged> <juju-core 1.22:In Progress by cherylj> <juju-core 1.23:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1454697>
#juju-dev 2015-05-21
<menn0> wallyworld, thumper: txn pruning improvements http://reviews.vapour.ws/r/1741/
<waigani> thumper, axw: storage worker fix: http://reviews.vapour.ws/r/1742/
<waigani> I tested 1.23, 1.24, 1.25 - bug is only in 1.24 and 1.25
<wallyworld> menn0: done
<menn0> wallyworld: thank you. you're a review machine.
<wallyworld> sometimes
<anastasiamac> menn0: wallyworld is always
<axw> waigani: ah, so the state that the agent opens isn't scoped. thanks for looking at that
<thumper> waigani: shipit
<waigani> thumper, axw thanks - I'll land in 1.24 now and follow up in 1.25
<thumper> ack
<thumper> waigani: I was just looking at my local machine-0.log wondering why it was 16 meg already
<thumper> waigani: was due to the worker constantly restarting in two environments :-)
<natefinch-afk> katco: the CI test is working.  It actually inadvertently discovered a bug in lumberjack, due to a combination of how I had to do one of the tests and the fact that destroying the local environment leaves logs behind.
<thumper> natefinch-afk: funny how CI tests find bugs eh?
<waigani> thumper: ha
<natefinch> thumper: yep.  really just a huge coincidence in how everything came together, but I'm certainly glad to find the bug, even if it is a pretty edge case.
<natefinch> for those curious: https://github.com/natefinch/lumberjack/issues/12
<natefinch> I love how sloppy I can be and let gofmt clean everything up for me.  Saves me a ton of time fiddling with indenting etc
<waigani> thumper: hosted upgrades:  http://reviews.vapour.ws/r/1639/ - needs a follow up review (not urgent, I've found another bug to look into)
<axw> natefinch: I find I'm much worse about writing in other languages because of that now :/
<natefinch> axw: haha yep
<waigani> natefinch: +1 if it doesn't format I know I've got a syntax error somewhere, that's been handy too
<natefinch> waigani: yep... and in Sublime, if you're using gosublime it'll put a dot next to the line that is invalid
<waigani> natefinch: true! too easy :)
<menn0> wallyworld, thumper : a quick one before the meeting? http://reviews.vapour.ws/r/1743/
<menn0> this is the final bit of the txn pruning work (for 1.22 at least)
 * thumper looks
<thumper> menn0: if there was no previous, how long does it wait?
<thumper> menn0: also, what about those long running ones with 500k transactions pruned
<menn0> thumper: that was all in the PR for juju/txn...
<menn0> thumper: pruning will first be checked 2 hrs after jujud startup
<menn0> thumper: if there is no record of a previous prune, then pruning happens
<thumper> k
<thumper> well, all that can be tweaked later if necessary
<menn0> thumper: there after it only happens when necessary
<menn0> thumper: where necessary at the moment means "txn count has doubled since last prune"
<thumper> right, just considering the pathalogical cases
<menn0> thumper: so for small envs it'll probably never happen
<menn0> thumper: or very infrequently
<menn0> thumper: for the currently huge envs, they'll have a massive prune the first time around but should then be fairly infrequent
<menn0> thumper: there's logging of the txn counts etc in juju/txn each time MaybePruneTransactions is called
<natefinch> super quick review anyone?  Just updating dependencies.tsv to include the bugfix I just made to lumberjack: http://reviews.vapour.ws/r/1744/diff/#
<natefinch> davecheney: thanks
<thumper> axw: machine-0: 2015-05-21 02:26:15 ERROR juju.worker.diskmanager lsblk.go:111 error checking if "sr0" is in use: open /dev/sr0: no medium found
<thumper> axw: should I be worried?
<axw> thumper: have you pulled trunk lately? that should be fixed a couple of days ago
<thumper> axw: no, using jes-cli, behind trunk by a week or two
<thumper> axw: good to hear it is fixed though
<thumper> I'll merge master soonish
<axw> ok, good to know it's probably not still broken :)
<perrito666> feature branches feel a bit like the explanation of time travel by the doc to marty
<perrito666> in axw and my master is fixed, but in your version of master it is still borken
<axw> just need to get the almanac off biff and it'll be OK
<natefinch> gah.... just realized my CI test won't work on vivid because it assumes we're using upstart.  le sigh
 * natefinch reboots to see if he can get his lxc containers to stop getting stuck for no reason
<natefinch> heh... I bet this is that CI blocker that is hosing tip.... let's see if 1.24 works....  yep, that was it.
 * natefinch waits for his CI test to write 1.8 GB of data to disk
 * natefinch does a little dance
<natefinch> success!
<natefinch> wallyworld: you around?
<wallyworld> yeah
<natefinch> wallyworld: there's no docs on status-set on jujucharms
<wallyworld> i gave doc to nic
<wallyworld> ages ago
<natefinch> wallyworld: oh, huh, ok.  I'll bug him then.
<wallyworld> ball is in his court
<natefinch> evilnickveitch: ^^
<wallyworld> not enough hours in the day - we need more doc people
<wallyworld> like 2 or 3
<natefinch> yep
<davecheney> thumper: http://paste.ubuntu.com/11256308/
<davecheney> not terrible
<davecheney> but some important ones to fix
<thumper> davecheney: ack
 * thumper tries to work out where to add this test...
<cherylj> hey davecheney, quick question for you on your review - I'm not clear how the test case I've added is a race.  We're not actually starting the unit agent, we're just invoking its start method directly.  Is there something I'm missing?
<thumper> I wish git stash worked more like bzr shelve
<natefinch> what's the difference?
<davecheney> cherylj: maybe i was mistaken
<davecheney> normally when patch value is used, it's vbecause it's changing the value of a thing that will run in another goroutine
<axw> wallyworld: FYI, http://reviews.vapour.ws/r/1751/
<wallyworld> looking
<axw> ericsnow: the GitHub links in RB seem to be messed up now (looks like unrendered markdown links). did you change something?
<axw> wallyworld: I'm going to look at tagging volumes now, since if we don't tag them we won't later be able to list them
<wallyworld> axw: yep, sounds great
<axw> wallyworld: after that I'll look at persistent volume deletion unless you think there's something more important
<wallyworld> axw: i need to go over our cards, we can talk tomorrow
<axw> ok
<wallyworld> but if you get done before we talk, then vol deletion sounds good
<wallyworld> axw: i think the other good card would be storage reattachment?
<wallyworld> maybe before volume deletion?
<axw> wallyworld: ok
<dimitern> voidspace, hey there
<dimitern> voidspace, i'm in the hangout
<voidspace> dimitern: omw
<mfoord> dimitern: and back...
<mfoord> dimitern: need to reconfigure virt-manager before I try that again...
<TheMue> morning o/
<mfoord> TheMue: o/
<dimitern> :) ok
<dimitern> morning TheMue
<TheMue> mfoord: no voidspace today?
<mfoord> TheMue: heh
<mfoord> TheMue: I had to force kill my machine - virt-manager wouldn't release the mouse and keyboard
<dimitern> aaargh.. RB hates me today
<dimitern> https://github.com/juju/juju/pull/2392 - reviewers welcome
<TheMue> mfoord: ah, ok
 * TheMue starts the day with a review
<TheMue> dimitern: reviewed
<dimitern> TheMue, ta!
<TheMue> dimitern: I like the proper use of LXC... instead of Lxc... ;)
<mfoord> dooferlad: TheMue: we're going to be a few minutes late to standup - in another meeting
<mfoord> dooferlad: TheMue: that is dimitern and I
<TheMue> ok
<dimitern> mfoord, when you're ready
<mfoord> dimitern: TheMue: dooferlad: I'd like to grab a drink
<mfoord> 5 mins
<mfoord> TheMue: dooferlad: dimitern: right, ready
<TheMue> mfoord: we're in
<mfoord> omw
<jam> menn0: about the pruning changes, you're not tracking the history of successful GC runs and how much they pruned?
<jam> You just overwrite 'last' each time
<menn0> jam: yep, that's intentional
<menn0> jam: I thought we decided it wasn't that important
<jam> menn0: I don't think it is critical, but I do think when adding something new like this being able to see how its working is pretty useful.
<menn0> jam: well there's still the logs
<jam> menn0: unless they have DEBUG logging on and it only captures 5hrs before it rotates.
<menn0> jam: sure
<menn0> jam: if you feel strongly about it i can change it
<jam> menn0: I feel like having that information isn't going to be an OPs problem unless we have something else that's really wrong, and would give us some feedback as to how GC is working in the wild.
<menn0> jam: i see your point
<menn0> jam: i'll try and get something in tomorrow (it's late here)
<jam> menn0: rest well
<menn0> jam: i guess you also want the start timestamp and the start txn count? :)
<jam> menn0: yeah, so we know Why we tried to run GC, how much GC helped, and how long GC took to run.
<menn0> jam: what about the issue of the txns.prune collection growing forever?
<menn0> jam: I guess the growth rate is pretty small
<jam> menn0: if it grows by 1 record every 2 weeks, and that record is 200bytes in size
<jam> we consume 1MB in...
<jam> menn0: 192 years
<jam> menn0: I can live with that growth rate.
<menn0> jam: hmmm I don't know... :)
<wallyworld> jam: did you have an opinion on the latest emails re: relation status?
<jam> wallyworld: I'll reply, I don't have particularly strong feelings, nor was I in the conversations in Nuremberg
<wallyworld> jam: np, i like your input because 99.99999% of the time you are usually right on the money with what should be done :-)
<dimitern> reviewers, please have a look at http://reviews.vapour.ws/r/1756/ - fixes bug 1456989 for 1.24
<mup> Bug #1456989: cloud-init 0.6.3 on precise generates invalid apt-get install command line <cloud-init> <precise> <regression> <juju-core:Triaged by dimitern> <juju-core 1.24:In Progress by dimitern> <https://launchpad.net/bugs/1456989>
<natefinch> linux peeps... I'm trying to set gopath in my install hook... but nothing seems to work.. anyone have any ideas on how to do this?  I tried echo "export GOPATH=$HOME" >> .bashrc  (and same to .profile).... no matter what I do, "go get ..." running in my start hook always says GOPATH not set.
<natefinch> (er sorry, that's ~/.bashrc of course)
<perrito666> natefinch: I am not sure .bashrc is used for the hooks env
<tvansteenburgh> are you sourcing the file after you do that?
<natefinch> tvansteenburgh: can I usefully source a file from inside a hook script?
<perrito666> natefinch: also, when you speak of your ~ where exactly is that?
<perrito666> natefinch: yes, you can, it is a shell after all
<dimitern> mfoord, dooferlad, TheMue ^^
<dimitern> (the review I posted a while ago)
<TheMue> dimitern: ic, will review
<natefinch> perrito666: I dunno... where-ever it is.  I presume ~ is the same for the user is running every hook
<dimitern> TheMue, thanks!
<perrito666> ah you are doing that IN the hook, while running it
<perrito666> natefinch: why dont you just run the export?
<natefinch> perrito666: it doesn't work if I just export it
<tvansteenburgh> he needs it in a later hook right?
<perrito666> natefinch: that is... wrong somehow
<natefinch> perrito666: go get <thingy> says "no GOPATH set"
<perrito666> natefinch: ahhhhhhhh
<natefinch> What I want is a working version of this as my install script: yes, what I want is for my
<natefinch> er http://pastebin.ubuntu.com/11264398/
<natefinch> however the hell you make that work right
<perrito666> natefinch: what is HOME?
<perrito666> I mean, what exactly does it point to?
<perrito666> that you just did there should work for your current shell and its forks iirc
<natefinch> ok, I'll test it
<perrito666> natefinch: but then again, also exporting it
<perrito666> natefinch: assuming you have access to write the actual home of the user, you are tainting the machine
<natefinch> exporting it definitely does not work
<perrito666> natefinch: ok are you inside the hook tmux?
<dooferlad> dimitern: *click*
<perrito666> if so pastebin env
<perrito666> natefinch: you can also GOPATH=/what/ever go get blah
<natefinch> perrito666: hang on, let me rerun things with the new install script
<natefinch> perrito666: that doesn't work either
<natefinch> perrito666: I think it's forking processes or something that somehow are not getting the root process' env
<perrito666> natefinch: then there definitely is something wrong there
<natefinch> go get is, that is
<perrito666> yes
<perrito666> but forking a process should get the exported vars
<perrito666> its like the only good reason to export
<natefinch> haa well, $HOME is nothing, which is fantastic
<natefinch> maybe that's the problem
<perrito666> natefinch: it is appropriate, you are in a chrooted env most likely
<dimitern> dooferlad, thanks! any reason not to check Ship it ?
<perrito666> natefinch: you have superpowers though, you can use the system gopath
<dooferlad> dimitern: mostly beause I don't have those powers. I haven't graduated, so I try not to.
<dimitern> dooferlad, well, you could still express intent :)
<dooferlad> dimitern: OK, will do next time!
<natefinch> perrito666: ok, so I guess it was the fact that I was setting GOPATH to $HOME which was nothing... setting it to /home/ubuntu works fine
<perrito666> natefinch: and makes your charm also ubuntu bound
<TheMue> dimitern: reviewed
<perrito666> natefinch: just use /go
<natefinch> perrito666: k
<perrito666> natefinch: or iirc /usr/shar/go?
<perrito666> I never remember but there is a proper place for those things
<dimitern> TheMue, thanks
<TheMue> dimitern: only thought that it could make sense to test cloudinit.New() for all releases
<dimitern> TheMue, it's tested in other places
<TheMue> dimitern: ah, fine
<katco> wwitzel3: ericsnow: natefinch
<katco> sorry internet is being flaky the past few days
<wwitzel3> katco: no worries
<ericsnow> katco: np, we basically finished after you left
<ericsnow> katco: I believe natefinch was going to ask about the blog :)
<katco> i swear to god if this lady tells me one more time i have to power cycle my router i'm going to throw this phone.
<perrito666> katco: but have you?
<cherylj> ha, that's the worst.  Hang in there, katco :)
<cherylj> I remember calling the "help desk" while I was at IBM and them not understanding when I told them I don' t have a start button because I'm using linux.
<perrito666> the only good thing about my isp is that it is easy to know when there is a problem outside your immediate network, they just dont answer your call until its fixed
<perrito666> cherylj: they are quite ok with that here, they tell you the command in linux to diagnose the network
<perrito666> the people installing the connection are even cool with doing so in a computer running ubuntu, which is surprising
<perrito666> they still require for you to have a computer with an ethernet por though, which is usually a problem around here
<perrito666> katco: just mtr to google and see if you are dying before leaving the gw or after
<katco> perrito666: that's exactly what i was doing
<katco> perrito666: unfortunately, tier 1 lady said that there's no other way to troubleshoot the situation than to "bypass your router"
<perrito666> katco: and by router they mean you r home one, not the one provided by them right?
<katco> perrito666: yeah, not the modem
<katco> perrito666: she didn't understand i can't just change my entire network to wait for the internet to go out again
 * katco is just so frustrated
<perrito666> katco: cant you just ssh into your router?
<katco> perrito666: i can do lots of things, but it won't solve a problem in their network
<perrito666> katco: lol
<perrito666> if the problem its in their network it will solve itself eventually
<katco> perrito666: yeah, usually i ride these out, but it's been dropping me pretty frequently the past few days
<perrito666> my isp still hasnt called me to schedule the installation of my decent speed line because they cannot resolve their installer guy schedule (that is the level of inutility)
 * TheMue passes katco a piece of chocolate to calm her nerves
<perrito666> katco: cant you just ask for a rewiring/new modem?
<perrito666> katco: just in case?
<perrito666> i am not sure what is the policy there
<katco> i mean... side by side, ping to modem, ping to 8.8.8.8. ping to modem fine, ping to 8.8.8.8 dropping. why is this not crystal clear?
<natefinch> katco: comcast?
<katco> natefinch: charter
<natefinch> katco: ug, possibly worse
<katco> natefinch: not sure if you guys have that up there
<perrito666> I had a friend shot at the main telco distribution box because their policy only allowed to replace cabling if the cable had more than N meters missing (otherwise just patching) and only went to check the box if the box was completely dead
<natefinch> katco: we have charter in town, actually... but since our property is on the town line of the next door town, which has fios, I just lied to verizon and said my address is in the next town (and put my billing address as my own town)
<katco> lol
<perrito666> so we got an insider tell us this (they where ignoring the rewiring request) so we basically chopped the cable by more than the required meters and shot the box (it is a place where said thing is not all that suspicious)
<katco> ericsnow: natefinch: wwitzel3: was going to mention, i'm out for the next hour for a dr. appt. wwitzel3, if i'm late for our 1:1 that's why
<katco> and if i drop out, now you know why >:|
<wwitzel3> katco: ok, if I am late, I have no good excuse
<katco> wwitzel3: lol
<katco> k bbiab guys, ty for the moral support haha
<natefinch> perrito666: nice work of vandalism
<natefinch> perrito666: not that I disapprove :)
<perrito666> natefinch: tx, the whole neigborhood was very happy
<perrito666> they got new phone lines that dont go down on rainy days
<natefinch> nice
<perrito666> telcos are the worse, there are only 2 in the whole country one in the south and one in the north
<perrito666> they are a very bad monopoly and usually cut so many corners that they are a circle
<natefinch> perrito666: there's a bunch in the US, but we have government supported monopolies so that they don't ever have to compete in any particular town
<perrito666> natefinch: I heard of people actually set their posts on fire so they got underground wiring
<natefinch> perrito666: haha, wouldn't work here... they'd just install new posts
<perrito666> natefinch: they dropped the posts 15 years ago
<perrito666> they just decided not to replace them in some places until it was necessary
<perrito666> to cut costs
<perrito666> they have an explicit internal rule for service repairmen not to replace anything until is not working 80% of the time or more
<Web> Big thanks to Juju team for all your work.  Without you my graduate project wouldn't have been such a success!
<perrito666> Web: excelent, congrats on graduation? (will you do a blog post?)
<cherylj> Web: I'm curious, what was your project?
<perrito666> natefinch: in my birth town they even dropped the tech service, they send 2 people every month for a day and they just fix as much as they get to in that day and the rest just are postponed, there is a guy that works as an individual contractor for people to fix these things because no one expects the company to come anymore
<natefinch> perrito666: that's pretty bad
<cherylj> can I get a sanity check from someone on a review comment I received yesterday?  dave commented that there was a race in a particular test that I wrote, and I don't think there is:  http://reviews.vapour.ws/r/1740/
<perrito666> natefinch: not for the individual contractor guy
<natefinch> haha
<Web> perrito666: I will be posting a detailed blog article soon on http://mastersproject.info.
<cherylj> In the test, we synchronously call a function, APIWorkers which will call the reportClosedAPI function we're patching
<cherylj> I just want to make sure I'm not missing something.
<Web> cherylj: SaaS delivering questionnaire services with developer access.  Primarily a Node.js based application.  http://themindspot.com
<cherylj> neato!  reading now
<Web> Now I can start on cleaning it up and expanding the api to make native applications.
<perrito666> cherylj: I am a bit out of context, what does reportClosedAPI does exactly
<cherylj> it is just a function added to test that we actually close the api we opened in the case of failure.  In the non-test case, it is an empty function.  In the tests, we patch it to set some var to true to indicate that it was called.
<Web> cherylj: Still alot of work to do to make it a viable service but the value and product was delivered on time and as expected with good software engineering practices.
<cherylj> You can see how a similar pattern is used here:  https://github.com/juju/juju/blob/master/cmd/jujud/agent/machine_test.go#L858
<cherylj> perrito666: ^^
<cherylj> perrito666: but in the above case, they are creating a new goroutine and that's why they're using channels
<perrito666> cherylj:  I am tracing the code
<cherylj> thanks, perrito666
<cherylj> Web: how was your experience using Juju?
<perrito666> I am not sure here if deferreds run before or after returning from a function, natefinch ?
<perrito666> I do believe they are blocking
<perrito666> cherylj: anyway, given the logic there it think it would be mor go-ish and safer to use a channel and wait on it isntead of a bool
<perrito666> more*
<cherylj> thanks, perrito666!
<Web> cherylj:  I have been enjoying Juju from the beginning.  I haven't been a charmer for awhile now but have always boasted Juju.  Thats why I knew it was perfect for creating my project proposal.
<cherylj> Web: how did you first hear about it?
<sinzui> katco, can you ask someone to triage bug 1457225. The case is somewhat extraordinary, I don't understand the severity or labour.
<mup> Bug #1457225: Upgrading from 1.20.9 to 1.23.3 works, but error: runner.go:219 exited "machiner": machine-0 failed to set status started: cannot set status of machine "0": not found or not alive <cts> <sts-stack> <upgrade-juju> <juju-core:New> <https://launchpad.net/bugs/1457225>
<natefinch> perrito666: deferred functions run before the function exits, but after return values are set.  They will be required to finished before the function exits
<perrito666> natefinch: I assumed so which makes dave's comment wrong
<perrito666> natefinch: but I believe that channel over flags is good anyway
<alexisb> Web, glad to hear it! and congrats on a success graduate project!
<natefinch> perrito666: http://play.golang.org/p/P2zB4t1Alf
<perrito666> natefinch: and how is it handled in the low level?
<perrito666> natefinch: is the defered a go routine with access to the namespace of the function or is  an appended func call or what?
<Web> cherylj:  I don't even remember but I worked on a early version of node-app to implement some features m3 (Mark Mims) wanted.
<natefinch> perrito666: it's not a goroutine.  I'm not sure exactly how it runs.  It does have a non-negligible performance impact, though not enough to care about unless it's in a really hot path
<Web> alexisb: thank you very much.
<perrito666> natefinch: uff, I assume its something like a try/catch
<perrito666> natefinch: I am now tempted to read the go source for that
<Web> Now I need to beautify and get some money to upgrade servers from t1.micro's if I want to make this a real service. :)
<cjwatson> Hi, is there any way I can get juju's private mongod to not sit there spinning and calling select 100 times a second?  It's quite literally causing me to have to open a window to dissipate laptop heat any time I'm using juju
<cjwatson> (jcastro sent me here from #juju)
<katco> sinzui: ack
<katco> ericsnow: wwitzel3: natefinch: back if you need anything
 * natefinch has to spend a little time writing unit tests for his CI tests
<katco> ...
 * katco is actually not sure how she feels about that
<natefinch> katco: https://code.launchpad.net/~natefinch/juju-ci-tools/logrot/+merge/259750
<natefinch> katco: I know... on the one hand, it's nice to double check that the tests aren't malfunctioning, but on the other hand, it seems like a belt and suspenders and ... something else that holds up your pants... kind of approach
<perrito666> natefinch: I use being fat for that, the belt/suspenders are just for aestethic reasons
<katco> yeah... i honestly haven't ever thought about that enough to have an opinion, but the gut check says it seems excessive
<marcoceppi> Web: congrats!
<Web> marcoceppi: big thanks to you too!  Your always there when I've needed advice.
<mup> Bug #1457566 was opened: lock timeout exceeded <deploy> <intermittent-failure> <juju-core:Incomplete> <juju-core 1.22:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1457566>
 * perrito666 discovers that his mobile company uses the emergency broadcast alerts to send spam...
<katco> i need a volunteer to look at bug 1457225
<mup> Bug #1457225: Upgrading from 1.20.9 to 1.23.3 works, but error: runner.go:219 exited "machiner": machine-0 failed to set status started: cannot set status of machine "0": not found or not alive <cts> <sts-stack> <upgrade-juju> <juju-core:New> <https://launchpad.net/bugs/1457225>
<katco> currently only triage needed
<katco> cherylj: perrito666: dooferlad: voidspace: wwitzel3: ericsnow: natefinch-afk: TheMue: ^^
<wwitzel3> katco: in a hangout with ericsnow, whit, and lazyPower
<katco> wwitzel3: k lmk when you're ready
<Web> Okay hadn't thought about applying to Canonical until today.  If you hear my name (Brandon L. Clark) in passing give me some praise.   And any advice?  :)
<wwitzel3> katco: ping
<katco> wwitzel3: pong
<peterklipfel> Hi all, I'm wondering if anyone has any tips on where I can start with modifying the juju source. I've been using the project for ~1.5 years, but haven't contibuted anything. I'm not so good at go. The thing that I want to do is enable custom image usage on openstack on a per-machine basis. It's possible that this might be a big refactor due to (for example) colocating services on machines. Thoughts?
<natefinch-afk> peterklipfel: using go will likely not be your biggest hurdle.  The language itself is quite small and easy to pick up if you know pretty much any other imperative language.  The codebase is quite large, which is the biggest hurdle for new developers.
<natefinch-afk> nick natefinch
<natefinch> peterklipfel: I woudl start here: https://github.com/juju/juju/blob/master/CONTRIBUTING.md
<peterklipfel> natefinch: Right, I'm struggling to wrap my head around the codebase. I have the repository up and running, and have read through the contributing doc. I'm wondering if there's an architecture doc or something somewhere
<natefinch> peterklipfel: adding the ability to use custom images is something we've been wanting to do for forever, but just haven't been able to put it up to the top of the priority list
<natefinch> peterklipfel: I don't know if we really have a good architecture document.  There's some docs here: https://github.com/juju/juju/tree/master/doc   but a bunch are very old (note that some of them say "2 years ago"... which is a lifetime in the life of juju)
<natefinch> peterklipfel: luckily... using custom images would just be something to be able to specify in the communication to the provider (openstack in this case), and the providers are pretty self-contained
<peterklipfel> natefinch: I totally missed the "doc" folder. Thanks! Will be back later
<natefinch> peterklipfel: probably you'd want to make image a constraint, as defined here: https://github.com/juju/juju/blob/master/constraints/constraints.go   and then figure out how to get the openstack provider to do the right thing with it inside its StartInstance method here: https://github.com/juju/juju/blob/master/provider/openstack/provider.go#L944
<peterklipfel> natefinch: awesome. Thanks again
<katco> natefinch: wrapping up last meeting
<natefinch> katco: ok
<bac> hey whit, thanks for the note on that juju-deployer bug.  any plans to get that moving? i think tvansteenburgh has commit rights now, which may speed things up.
<whit> bac, yeah...
<whit> bac, hazmat requested a test, and I've not had a chance to get back to it
<bac> yeah, no one's favorite project.  :)
<whit> it's a bit wierd because of the fact that it's dependent on git
<whit> (testing that is)
<whit> well, no single maintainer.
 * whit has no ill will towards deployer
<hazmat> whit: just set HOME=tmpdir()  and record identity .. you can setup a temp repo that way in the test without pulling down anything or checking in a repo
<hazmat> alternatively just checkin a small git repo
<hazmat> whit: the particular error that i saw was that it didn't respect JUJU_REPOSITORY env var
<whit> hazmat, not sure about that one
<mup> Bug #1457645 was opened:  warning: log line attempted over max size - leadership related <landscape> <juju-core:New> <https://launchpad.net/bugs/1457645>
<mup> Bug #1457645 changed:  warning: log line attempted over max size - leadership related <landscape> <juju-core:New> <https://launchpad.net/bugs/1457645>
<mup> Bug #1457645 was opened:  warning: log line attempted over max size - leadership related <landscape> <juju-core:New> <https://launchpad.net/bugs/1457645>
<natefinch> what the heck is this even supposed to mean?  "missing namespace, config not prepared"
<perrito666> the config is not prepared, you are missing a namespace
<natefinch> thanks :/
<perrito666> natefinch: where do you see that?
<natefinch> when I do juju status on my local environment that is currently running
<natefinch> I have a feeling something in the CI tests fubared my environment
<natefinch> are we still supposed to have jenv files?
 * natefinch notes he no longer has a local.jenv
<perrito666> natefinch: the comment in the code does not make much sense either
<natefinch> I think we are.  I think something in the CI tests blew away my jenv without destroying the CI tests
<natefinch> yeah
<natefinch> er sorry, without destroying the environment
<thumper> natefinch: the namespace is added to the local config at environ prepare time
<thumper> so if you aren't getting it, something has gone wrong
<natefinch> thumper: I had no jenv for my local environment for some reason
<thumper> natefinch: are you running with the jes flag?
<natefinch> thumper: nope
 * thumper thinks
<thumper> natefinch: in the jes-cli branch, we now have 'cache.yaml' instead of .jenv files
<thumper> but this won't be impacting you
<natefinch> luckily, I have a script to blow away my local environment for just this kind of situation.  Notably it outputs "rm: cannot remove â/home/nate/.juju/environments/local.jenvâ: No such file or directory"
<natefinch> thumper: honestly, it was the error message more than anything that confused me.
<thumper> hmm..
<natefinch> thumper: totally not a useful error message for an end user, unless there's a juju prepare-config command I don't know about
<thumper> agreed
<thumper> some developer error messages should really be caught and translated into 'user speak'
<natefinch> but hey, maybe it's different with jes now... might get lucky
<natefinch> yep
<natefinch> dinner time.  Back later folks.
<ericsnow> could I get some eyes on patches revert changes that are breaking CI: http://reviews.vapour.ws/r/1757/ http://reviews.vapour.ws/r/1758/ http://reviews.vapour.ws/r/1759/
<mwhudson> davecheney: i'm still amazed by the effectiveness of my golang-dev trolling about gc
<davecheney> mwhudson: yeah
<davecheney> it's been rather productive, we've now ruled out custom allocators
<davecheney> and people are talking seriously about making Node and Proj smaller
<mwhudson> i wonder if people have noticed how very slow the linker is
<mwhudson> i could try some trolling in that direction
<mwhudson> davecheney: you know that bit in cmd/go's build.go about link order for gccgo builds
<mwhudson> davecheney: that we've fixed 1 million times
<mwhudson> davecheney: if someone were to hypothetically rewrite it entirely, how would they know they haven't broken it?
<ericsnow> katco: could you take a quick look at the other two as well (strictly reverts)
<ericsnow> thanks davecheney
<davecheney> mwhudson: try to build juju is a good teest
<cherylj> davecheney: could you take a look at my changes for your review comments? http://reviews.vapour.ws/r/1740/
<mwhudson> davecheney: ack
<mwhudson> davecheney: any idea how long "go test -compiler gccgo github.com/juju/juju/..." will take?
<mwhudson> 30 mins or so?
<voidspace> mwhudson: hey michael
<voidspace> mwhudson: o/
<mwhudson> voidspace: hi!
<voidspace> mwhudson: I'm up late doing (another) maas install
<voidspace> mwhudson: how are things?
<mwhudson> hm, pretty sure the go process isn't supposed to spin using 100% of cpu :(
<voidspace> :-)
<mwhudson> voidspace: ok, trying to actually make debian packages that contain go shared libraries now
<voidspace> mwhudson: very cool
<mwhudson> which involves a surprising and unfortunate amount of perl
<voidspace> hehe, my commiserations
<mwhudson> voidspace: how about you? are you sleeping with tcp illustrated under your pillow? :)
<voidspace> mwhudson: I should be!
<voidspace> gradually learning
<voidspace> mostly learning the peculiarities of maas recently
<voidspace> not sure that's such a transferable skill
<voidspace> I like maas though
<mwhudson> heh
<mwhudson> i've wrangled it a bit in the past
<voidspace> about to test bleeding edge apis around container IP address releasing
<voidspace> which means installing bleeding edge maas
<voidspace> which fails to configure when installed from the dailybuilds ppa
<voidspace> so back to a vanilla install and then upgrading
<voidspace> seeing as I can't work out how to manually run the configure script
<voidspace> thankfully so far their upgrades have always done the right thing
<voidspace> unlike ubuntu
<voidspace> known issue - if you have an encrypted hard drive then upgrading to 14.10/15.04 means you can no longer enter the password
<voidspace> fortunately you can still boot through recovery mode, which has a text password entry
<voidspace> but given that there's no supported way of removing encryption I'm doing a recovery mode boot every time
<mwhudson> ha oops
<voidspace> :-)
<mwhudson> i avoided that problem by having my laptop fall apart just before 15.04 was released
<voidspace> that's one way I guess
<voidspace> I haven't dared upgrade my laptop
<mwhudson> although the gyrations required to mount my encrypted home dir from the drive after i'd pulled it from the laptop and stuck it in a caddy were... contorted
<mwhudson> partly because i'd installed with lvm
<voidspace> :-/
<voidspace> computers
<mwhudson> yeah, otoh usb3 + ssd = pretty fast restore
<mwhudson> voidspace: about configuring maas, you should talk to rbasak
<voidspace> mwhudson: thanks
<mwhudson> his point is that setting up maas requires orchestration, i.e. juju
<voidspace> mwhudson: if this doesn't work I will do
<mwhudson> but there's a bootstrapping problem there :-)
<voidspace> heh
<mwhudson> he had some crazy scheme to use it regardless but i don't know what it was
<mwhudson> i don't know if he would be of practical help
<voidspace> installing maas is supposed to run configuration
<voidspace> and if you're *supposed* to do it a different way they should document that
<mwhudson> i'm pretty sure installing maas is supposed to work, yea :)
<voidspace> it certainly always used to work
<mwhudson> Broadcast message from systemd-journald@glamdring (Fri 2015-05-22 11:27:36 NZST):
<mwhudson> [14085]: 2015-05-21 23:27:36 INFO juju.testing mgo.go:514 reset successfully reset admin password
<mwhudson> wtf
<voidspace> yeah, running the test suite on vivid spams systemd messages to the terminal
<voidspace> ericsnow knows and cares
<ericsnow> mwhudson: see #1456258
<mup> Bug #1456258: systemd broadcast message spam during test run <papercut> <unit-tests> <juju-core:Triaged> <juju-core 1.23:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1456258>
<mwhudson> ah cool
<voidspace> right, I've failed to get MAAS working
<voidspace> I've got as far as internal server error
<voidspace> can't connect to cluster in the logs
<voidspace> I'm going to bed
<voidspace> g'night all
<mwhudson> i think running the tests with gccgo ran my machine to close to out of memory
#juju-dev 2015-05-22
<jhobbs> how can I enable trace logging for golxc when using juju?
<wallyworld> jhobbs: juju set-env logging-config="<root>=INFO;unit=DEBUG;golxc=TRACE"
<wallyworld> that's after a system is running
<wallyworld> can also specify at bootstrap using --logging-config="blah"
<cherylj> Can I get a review for my file handle bug fix?  http://reviews.vapour.ws/r/1740/
<thumper> cherylj: shipit
<cherylj> thanks, thumper!
<davecheney> thumper: so i'm trying to fix a race in github.com/juju/juju/apiserver_test.(*pingerSuite).TestAgentConnectionDelaysShutdownWithPing()
<davecheney> and i'm suspecting that apiserver.Pinger is not actually used
<davecheney> two qestions
<davecheney> 1. should I make a bug for each of these races
<davecheney> 2. can I just delete apiserver.Pinger ?
<davecheney> oh, i see how it's used
<davecheney> ok
<natefinch-afk> it's amazing how little time it takes writing in a language with exceptions to remind me why I hate them
<axw_> anastasiamac: I ignored your "not ready for review", and reviewed ;)
<axw_> anastasiamac: mostly looks good anyway
<anastasiamac> axw_: oh! the headache ...
<anastasiamac> axw: :D
<anastasiamac> axw: it was marked this way as it does not work and m using the wrong glue
<anastasiamac> axw: i think
<anastasiamac> axw: tyvm for review... i m fixing it all now anyway :D
<axw> anastasiamac: oh? it looked like it would work
<axw> nps
<anastasiamac> axw: nuh... it would not and it does not link properly... m going to hijack ian after ur 1:1 and nail this PR :D
<wallyworld> natefinch: i have the opposite - amazing how using Go and putting if err != nil {return err} EVERYWHERE makes me miss exceptions so much
<natefinch> wallyworld: my current complaint is that anything can fail, and you have no idea it can until it does.  And having the real code in your function be wrapped in 3 levels of indented try's and 20 lines of exception handling at the end... . and then you have the exception handling *inside* your except blocks, which if you forget, then you mask the original error....
<wallyworld> natefinch: i guess it's like anything - it can be done nicely or poorly :-)
<natefinch> wallyworld: I find it a lot easier to do error returns nicely... plus then if your function can fail, it's right there in the function signature.
<natefinch> meanwhile, I think python's function-scoped variables is screwing me somehow
<wallyworld> natefinch: true but what tends to happen is that we err != nil retrun err and just ignore the error
<natefinch> wallyworld: that's not really any different than just letting an exception fly
<wallyworld> natefinch: true, but doing exceptions is far less useless boilerplate
<natefinch> wallyworld: is it? http://pastebin.ubuntu.com/11275221/
<natefinch> at least with error returns the code paths are 100% straight forward if statements... not this convoluted jumping around in the code.
<wallyworld> natefinch: just in meeting, will look in a bit
<natefinch> wallyworld: np
<natefinch> someday I'll finish up my blog post about why exceptions are evil.... maybe when all the kids are in school finally.
<natefinch> anyone good at python want to help me out with figuring out why the wrong exception is being printed from this code?  http://pastebin.ubuntu.com/11275221/
<natefinch> something in lines 19-24 throwing an exception, and then dump_env_logs on line 29 is throwing....  line 34 should be raising e, not e2, right?    Because right now, line 39 is logging e2 somehow.
<natefinch> I added the try/catch around the dump_env_logs in order to try to keep that exception from masking the real problem... but somehow line 33 is logging the exact same error as line 39
<natefinch> thumper: ^
<natefinch> afk a bit
<thumper> natefinch-afk: would really help to see the stack trace
<axw> wallyworld: sorry, didn't want to miss my delivery
<wallyworld> axw: np, we were finished anyway
 * axw goes to play with new modem
<wallyworld> enjoy
<natefinch> thumper: full code (for correct line numbers): http://pastebin.ubuntu.com/11275530/      stack traces: http://pastebin.ubuntu.com/11275558/
<thumper> natefinch: what are you expecting?
<natefinch> thumper: I'm expecting that the raise on line 197 is going to raise e (from line 186) and it seems to be raising e2.
<thumper> If no expressions are present, raise re-raises the last exception that was active in the current scope.
<thumper> e2 was the last exception in the current scope
<thumper> if you explicitly want e, go 'raise e'
<natefinch> won't that screw up the stack trace of e?
<thumper> not sure
<thumper> try it
<natefinch> it does
<natefinch> that seems amazingly broken.
<natefinch> tiny repro case: http://pastebin.ubuntu.com/11275739/
<natefinch> (pay no attention to my tabs instead of spaces.... wrote it before I saved, so my editor didn't know to convert my tabs into spaces)
<natefinch> I guess the solution is - you have to put that cleanup code in a separate function.
<thumper> natefinch: yeah, I was just writing that out
<thumper> had the same thought
<thumper> make the second exception live in a different scope
<natefinch> which is not horrible, it's just surprising
<thumper> python scopes != indentation level
<natefinch> yeah
<davecheney> http://reviews.vapour.ws/r/1762/
<natefinch> I knew the function-level scope was screwing me somehow
<menn0> thumper: so jam caught me late last night and wanted one more improvement to the txn pruning work.
<menn0> thumper: so here it is: http://reviews.vapour.ws/r/1763/
<thumper> menn0: oh?
<menn0> thumper: he wanted historical stats to be kept. which is probably a good idea.
<menn0> thumper: especially given that pruning is quite infrequent now and so you won't see much in the logs once they get rotated.
 * thumper nods
<menn0> thumper: so review pls? :)
<thumper> menn0: ack, was just reviewing davecheney's one
<menn0> thumper: cheers
<menn0> thumper: does your switch PR need another review?
<thumper> menn0: yeah
<menn0> thumper: k i'll have a look at that
<thumper> menn0: does the txn package use juju/errors yet?
<menn0> thumper: no, that's why i didn't introduce it
<menn0> thumper: is it supposed to?
<thumper> fair enough... for now,
 * thumper shrugs
<thumper> probably one day
<menn0> thumper: I almost started using it but then noticed it wasn't used so thought that might have been on purpose to reduce coupling or something
<menn0> it should probably use it
<davecheney> https://github.com/juju/juju/pull/2402
<axw_> wallyworld: would you mind taking a look at https://github.com/go-amz/amz/pull/50 for me?
<wallyworld> sure
<wallyworld> axw_: haven't looked yet, just finishung with anastasia, will look real soon
<axw_> wallyworld: no worries
<menn0> thumper: is it possible for a system and an env to have the same name?
<thumper> yes
<thumper> normally when you bootstrap
<thumper> of have access to the initial environment
<menn0> thumper: so how does switch tell the difference?
<thumper> menn0: it doesn't, environments can be used as systems
<menn0> right
<thumper> but systems can't be used as environments
<menn0> ok
<natefinch> wait, what's a system?
<thumper> so if you have access to the environment "test" and system "test", then it will always use the environment
<natefinch> is that the base environment that all the sub-environments run on top of?
<thumper> natefinch: kinda
<menn0> could it be possible for someone to shoot themselves in the foot and name the non-initial env with the same name as another env or system?
<thumper> natefinch: the word we were told to use
<thumper> was server, now it is system
<thumper> as it could be more than one server...
<thumper> ha and all that
<menn0> thumper: ^^^
<menn0> (my Q a few lines back)
<natefinch> thumper: makes sense
<thumper> menn0: no, as the configstore treats that as an error
<thumper> configstore has tests for all that
<menn0> thumper: ok cool. just checking that this has been considered.
<thumper> :-)
<aisrael> alexisb: Here's the bug I mentioned the other day that's causing juju to hang after upgrading 1.24 betas: https://bugs.launchpad.net/juju-core/+bug/1457728
<mup> Bug #1457728: `juju upgrade-juju --upload-tools` leaves local environment unusable <juju-core:New> <https://launchpad.net/bugs/1457728>
<axw> wallyworld: please see my reply about non-contiguous sequence numbers in request params
<wallyworld> ok
<wallyworld> axw: sounds good
<axw> wallyworld: ta
<menn0> thumper: one line review pls http://reviews.vapour.ws/r/1765/
<menn0> thumper: forgot to say... review done
<menn0> thumper: for the switch change
<mup> Bug #1457728 was opened: `juju upgrade-juju --upload-tools` leaves local environment unusable <juju-core:New> <https://launchpad.net/bugs/1457728>
<menn0> wallyworld or axw: one line review pls? http://reviews.vapour.ws/r/1765/
<mup> Bug #1457728 changed: `juju upgrade-juju --upload-tools` leaves local environment unusable <juju-core:New> <https://launchpad.net/bugs/1457728>
 * menn0 is keen to be done with this
<axw> menn0: done
<menn0> axw: cheers
<wallyworld> too quick for me
<mup> Bug #1457728 was opened: `juju upgrade-juju --upload-tools` leaves local environment unusable <juju-core:New> <https://launchpad.net/bugs/1457728>
<wallyworld> axw: a small one for friday arvo if you have a sec http://reviews.vapour.ws/r/1766
<axw> wallyworld: ok
<axw> wallyworld: I've just sent a PR for EC2 tagging, but won't tick the card over yet because I need to do OpenStack too
<wallyworld> ok
<wallyworld> maybe we needed 2 cards
<axw> wallyworld: I don't think this change is safe. It would lead to machines being destroyed while there are units on it. Pretty sure the units are supposed to run stop hooks etc. before the machine is destroyed
<wallyworld> axw: how does it do that? if there is a pending cleanup operation, the machine likfecycle is not advanced, otherwise it's just the same as now
<wallyworld> the only change is that if there is one unit on the machine which is part of a dying service, the machine destroy doesn't error and also doesn't do anything
<axw> wallyworld: "juju deploy mysql --to 1", <wait for unit to come up>, "juju desroy-unit mysql/0" immediately followed by "juju destroy-machine 1"
<wallyworld> that will behave as normal
<wallyworld> it's only if the service is dying that the new behaviour kicks in
<axw> wallyworld: ok, so swap "destory-unit" with "destroy-service mysql"
<wallyworld> and the new behaviour is not to error, and also not to advance machine lifecycle
 * axw looks again
<wallyworld> maybe a hangout to explain
<axw> wallyworld: sorry, I thought it was still running ops
<wallyworld> ah np
<axw> wallyworld: I think this does break the contract of Destroy though. there's no guarantee that another service won't be added right?
<axw> if Destroy returns, then the lifecycle should be Dying
<wallyworld> axw: true, but then the cleanup won't run i don't think
<axw> wallyworld: precisely
<axw> so Destroy did not in fact do what it was supposed to
<wallyworld> but the user doesn't get an error is what you're sayinh
<wallyworld> so we are stuck either way
<wallyworld> i guess it could be a warning to the user instead
<wallyworld> ErrDestroyPendingButMightNotHappenIfSomeoneAssignsANewUnitInTime
<wallyworld> axw: school pickup time, bbiab, maybe introduce a new error as per above
<axw> wallyworld: maybe... I think it would be better if we allowed the machine to become Dying when it has units, but prevent it from becoming Dead until it has none. then worker/machiner would have to wait until there are no units before callign EnsureDead
<wallyworld> axw: ok, so not a no-op, thus preventing new units, but modify the cleanup worker
<wallyworld> that sounds like a good idea
<axw> wallyworld: you'd have to modify worker/machiner (at least)
<wallyworld> yep
<axw> wallyworld: it just waits until life = Dying, then calls EnsureDead
<wallyworld> will do that, a nicer solution, thanks
<wallyworld> gotta run, biab
<axw> ttyl
<natefinch> so I accidentally renamed a file outside of bzr, and now it thinks I have removed and readded it with a different name. Is there an easy way to turn that into a bzr rename?
<natefinch> oh, I guess just doing bzr mv worked... nevermind
<natefinch> man bzr has terrible error messages
<natefinch> I have a checkout of  lp:juju-ci-tools/repository  with some changes I've made.. I want to push those up to a branch under my own namespace.... I can't figure out the right incantation to make that happen
<natefinch> thumper: ^ ?
<natefinch> or wallyworld ^
<thumper> natefinch: checkout or branch?
<natefinch> thumper: I don't know how to tell
<thumper> natefinch: plz do this and pastebin "bzr info -v"
<natefinch> thumper: http://pastebin.ubuntu.com/11278010/
<thumper> natefinch: yeah, checkout is so the wrong thing to do
<natefinch> looks like checkout, but I don't know how to branch, because branch seems to expect I'm branching from somewhere else... and I just want to branch from what's right here
<thumper> natefinch: you have something that will be remotely trying to commit to the branch on launchpad
<natefinch> bzr branch . my-new-branch
<natefinch> except that of course does not work
<aisrael> bzr push lp:~username/juju-cli-tools/repository ?
<natefinch> then I get  bzr: ERROR: Permission denied: "~natefinch/juju-cli-tools/repository/": : Project 'juju-cli-tools' does not exist.
 * thumper tries to recall the incantation
 * natefinch tried switch, too, to no avail
<natefinch> $ bzr switch fill-logs
<natefinch> bzr: ERROR: Permission denied: "Cannot create 'fill-logs'. Only Bazaar branches are allowed."
<thumper> natefinch: bzr reconfigure --tree .
<thumper> natefinch: that should unbind your checkout from the remote branch
<thumper> natefinch: then
<thumper> bzr push lp:~natefinch/juju-ci-tools/foo-bar
<natefinch> it really does not like juju-ci-tools - bzr: ERROR: Permission denied: "~natefinch/juju-cli-tools/fill-logs/": : Project 'juju-cli-tools' does not exist.
<natefinch> bzr: ERROR: Permission denied: "~natefinch/juju-cli-tools/repository/": : Project 'juju-cli-tools' does not exist.
<natefinch> I'm dumb
<natefinch> the project is juju-ci-tools
<natefinch> except.... I think I fucked it up..
 * natefinch should not bzr at 1:30am
<natefinch> haha ... yeah, man.. just hadn't commited.  ok, definitely time for bed.
<natefinch> thumper: thanks for the help
<thumper> :)
<thumper> np
<fwereade> TheMue, dooferlad, voidspace: dimitern's had a power cut
<TheMue> fwereade: ouch, thanks for the info
<voidspace> fwereade: ok, thanks
<dooferlad> TheMue: before the meeting (and dimitern gets his power back), what can I do to help with the planning board? Is there anything to add or review?
<TheMue> dooferlad: not yet, the old open cards are already all moved by dimitern
<dooferlad> TheMue: ok, thanks.
<TheMue> dooferlad: you can take a look at the parking lot
<TheMue> dooferlad: currently there are no cards assigned to you
<dooferlad> TheMue: Isn't that how it is supposed to be if we are being "agile"?
<dooferlad> TheMue: only take a card when you are ready to do something new?
<TheMue> dooferlad: that's moving it from iteration backlog into actively working
<TheMue> dooferlad: in my case in the backlog all those cards belong to a family of tasks. so it makes sense to assign them to one person. but that does not mean to do them all at the same time.
<TheMue> dooferlad: first backlog, then planning of an iteration, then getting active
<fwereade> TheMue, dooferlad, voidspace: thought/suggestion: preassigning is an antipattern. short-term impact of picking up work you're not immediately familiar with: slows down you and likely the person who write the code. long-term impact: everyone knows everything better, less silos, less bugs, more general goodness
<TheMue> fwereade: yeah, the intention here isn't to create silos. only grouping short-term related work. but it's worth discussing even that.
<TheMue> dooferlad: voidspace: dimitern suggests we already start the meeting, he will join later
<dooferlad> TheMue: OK, there in a second.
<voidspace> ok
<voidspace> fwereade: I think I missed the context for those  comments
<voidspace> fwereade: I agree on both points though
<mup> Bug #1457797 was opened: Juju bootstrap fail everytime <bootstrap> <juju> <juju-core:New> <https://launchpad.net/bugs/1457797>
<perrito666> good morning all
<TheMue> perrito666: o/
<voidspace> I'm getting really good at installing trusty into kvm images
<voidspace> I've done it about eight times in the last 48 hours
<voidspace> and I still don't have a working maas
<voidspace> although I got closer than ever before this time
<voidspace> I had a working maas with boot images imported and a commissioned node
<voidspace> but the upgrade to maas 1.8 failed
<voidspace> this time I'll try again and go directly to 1.8 instead of via the "install and upgrade" approach
<voidspace> jw4: you will like this
<voidspace> jw4: https://www.youtube.com/watch?v=1Zfv1LZlU9U
<voidspace> jw4: video made by a close friend of mine, singer is our babysitter...
<voidspace> jw4: last night it won Best Christian Music Video at the UK Christian Film Festival Awards.
<dimitern> http://reviews.vapour.ws/r/1769/ << reviews welcome
<TheMue> dimitern: *click*
<TheMue> dimitern: any differences to the already done fix?
<dimitern> TheMue, none
<TheMue> dimitern: then it's simple ;)
<dimitern> TheMue, indeed :)
<TheMue> dimitern: so, done
<dimitern> TheMue, thanks!
<benji> voidspace: good stuff; I like the spare arrangement
<rogpeppe1> dimitern, fwereade, jam: i'm wondering about adding a function to the juju/api which tries to dial [][]network.HostPort (the same thing returned by api.State.APIHostPorts). does that sound reasonable to you?
<rogpeppe1> s:juju/api:juju/api package:
<dimitern> rogpeppe1, instead of using Open?
<rogpeppe1> dimitern: it would probably layer on top of Open
<fwereade> rogpeppe1, that does ~the same thing as we do here and there re connecting to all of them in parallel and returning the first one that works?
<rogpeppe1> dimitern: atm every external client needs to implement parallel dialing
<rogpeppe1> fwereade: yes
<fwereade> rogpeppe1, so long as you replace the existing parallel dialers with your new one, +10 :)
 * rogpeppe1 thinks again :)
<rogpeppe> fwereade: there's more than one?
<fwereade> rogpeppe, I'm assuming at least the clients and the agents?
<rogpeppe> fwereade: i was kinda hoping they used the same code. it's been a while
 * rogpeppe looks
<voidspace> benji: spare?
<dimitern> rogpeppe, sgtm
<voidspace> benji: do you mean sparse?
<benji> voidspace: just piano and vocals
<benji> yep
 * dimitern is afk for a while
<voidspace> cool, yeah it's nice
<benji> the massive verb put me off at first, but it works once the song gets going
<benji> I listened to it twice, so it must have been enjoyable :)
 * TheMue is afk for a while, fetching his new car
<perrito666> nice
<rogpeppe> fwereade: AFAICS there's only one implementation because in agent, configInternal.SetAPIHostPorts selects only the internal addresses, so it can use the existing dial logic in api
<fwereade> rogpeppe, doesn't it still have a bunch of addresses to choose from?
<rogpeppe> fwereade: yes, but that logic is dealt with in api.Open
<rogpeppe> fwereade: it's possible that i should just jam all the addresses into one slice
<rogpeppe> fwereade: excluding any local addresss, i guess
<fwereade> rogpeppe, ok, I'm a bit lost -- api.Open handles *some* multiple-address stuff but... doesn't parallelise?
<rogpeppe> fwereade: there are two possible levels of parallelisation
<rogpeppe> fwereade: because we've got [][]HostPort, not []HostPort
<rogpeppe> fwereade: api.Open handles []HostPort but not [][]HostPort
<fwereade> rogpeppe, ohhhhh
<fwereade> rogpeppe, the "jam all addresses into one slice" feels reasonable offhand?
<rogpeppe> fwereade: it kinda makes sense to do all servers in parellel, but within each server, do each address in turn
<rogpeppe> fwereade: i dunno. there can be a lot of them
<fwereade> rogpeppe, does it? isn't the reason for dialing multiple addresses for the same server that we don't know which might be reachable?
<fwereade> rogpeppe, if you can somehow sort each of the inner slices by bestness, then I can see the point
<rogpeppe> fwereade: once we know that an address *is* reachable, we should probably use that by preference for a given server
<rogpeppe> fwereade: yes
<fwereade> rogpeppe, I *think* there's some code that does that already
<fwereade> ...somewhere :/
<rogpeppe> fwereade: yeah, i seem to remember that too
<rogpeppe> fwereade: i think it might be in api.Open
<rogpeppe> jam: do you remember where that logic is, by any chance?
<rogpeppe> oh bother, it's friday
<rogpeppe> ha! found it. api.slideAddressToFront
<rogpeppe> fwereade: ^
<jw4> voidspace: I saw the comment on twitter a couple days ago... very nice
<rogpeppe> fwereade: i discovered this, so i think i'll go with a similar approach for the time being. there's a lot of logic there and combined with api.Open; a more cohesive approach would probably be good at some point. http://godoc.org/github.com/juju/juju/juju#PrepareEndpointsForCaching
<wwitzel3> ericsnow: ping
<ericsnow> wwitzel3: present
<wwitzel3> ericsnow: going to continue to go through what you did and make some stories on the board from it
<perrito666> ericsnow: hb
<ericsnow> wwitzel3: want to talk through them with me?
<mgz> ericsnow: lots of presents? :)
<wwitzel3> ericsnow: up to you :) I'm happy to, but don't feel obligated
<ericsnow> perrito666, mgz: yeah, waking up early to realize my alarm clock is broken so I had better just get up :)
<perrito666> ericsnow: hope you got a new alarm clock
<mgz> that seems like an unexciting gift
<ericsnow> perrito666: just shopping for one :)
<perrito666> who uses alarm clocks these days, mine has been disconnected for years, I just have it on the night table because it is nice
<perrito666> I had it because it was one of those ipod docks, but then apple graciously changed the connector and it no is just a nice white cube
<ericsnow> wwitzel3: on moonstone
<anastasiamac> ericsnow: âââ[Ì²ÌÌHÌ²Ì][Ì²ÌÌAÌ²Ì][Ì²ÌÌPÌ²Ì][Ì²ÌÌPÌ²Ì][Ì²ÌÌYÌ²Ì] [Ì²ÌÌBÌ²Ì][Ì²ÌÌIÌ²Ì][Ì²ÌÌRÌ²Ì][Ì²ÌÌTÌ²Ì][Ì²ÌÌHÌ²Ì][Ì²ÌÌDÌ²Ì][Ì²ÌÌAÌ²Ì][Ì²ÌÌYÌ²Ì]âââ Â¸.â¿Â¨Â¯`â¿Â´Â¸Â¸.â¿Â¨Â¯`â¿
<Spads> ð
<alexisb> happy b-day ericsnow!
<mup> Bug #1457895 was opened: azure-mode should not forbid targeted deployments <juju-core:New> <https://launchpad.net/bugs/1457895>
<mup> Bug #1457895 changed: azure-mode should not forbid targeted deployments <juju-core:New> <https://launchpad.net/bugs/1457895>
<mup> Bug #1457895 was opened: azure-mode should not forbid targeted deployments <juju-core:New> <https://launchpad.net/bugs/1457895>
<mup> Bug #1457797 changed: Juju bootstrap doesn't work behind proxy <bootstrap> <juju> <juju-core:Invalid> <https://launchpad.net/bugs/1457797>
<natefinch_> jw4: I wish action fetch and status would tell me the action name that go with the results.
<natefinch_> katco: can you resend me that link?  I was logged into the wrong account
<katco> natefinch_: yeah
<mup> Bug #1424961 changed: provisioning stops working with local provider <deploy> <local-provider> <juju-core:Invalid> <https://launchpad.net/bugs/1424961>
<natefinch> wwitzel3: embiggen plz
<jw4> natefinch: really good point!
<jw4> natefinch: I'm gonna use Actions 2.0 to trojan in a bunch of enhancements hopefully
<jw4> natefinch: I'm not sure exactly how we're going to fix a bunch of the usability issues with actions 1.0 - I wish we'd clearly labeled the api and cli as beta
<natefinch> jw4: seems like adding a new key to the output wouldn't hurt anything.. parsers should ignore extra data they don't understand anyway
<natefinch> course... you never know when someone might have a poorly thought out script
<jw4> natefinch: true - your request shouldn't change much... there's just a bunch of fiddly little things
<mup> Bug #1457797 was opened: Juju bootstrap doesn't work behind proxy <bootstrap> <juju> <juju-core:Incomplete> <https://launchpad.net/bugs/1457797>
<alexisb> natefinch, katco, ericsnow, wwitzel3: loving the demo videos!
<natefinch> alexisb: :)
<ericsnow> alexisb: :)
<wwitzel3> alexisb: thanks
<katco> alexisb: :)
<natefinch> ericsnow: btw, this is what I mean by function-scoped variables in python:  https://repl.it/pIP
<natefinch> ericsnow: as opposed to block scoped, like Go: http://play.golang.org/p/ADdQIO9GaP
<ericsnow> natefinch: oh, you mean that basically only blocks (functions, classes, and modules) in Python define scopes?  and that indentation does not (necessarily)
<natefinch> ericsnow: yes
<ericsnow> natefinch: I could see how that would be confusing at first
<natefinch> ericsnow: when you're used to block scope (where any branch counts as a block), function-scope is really confusing.
<mgz> ehe, but go lets you do all kinds of ugly shadowing that messes up with their scoping rules...
<ericsnow> natefinch: yeah, just think of it as Python only has scope against function bodies (and modules/class definitions are just special case functions)
<mgz> the practice of using 'err' as the name everywhere can create some fun bugs
<natefinch> mgz: pretty much any language let you shadow variables from a higher scope, the reason it's different for go is the confusion between = and :=
<natefinch> mgz: for example, you can shadow trivially in python: https://repl.it/pIW
<natefinch> python just doesn't have block scope
<natefinch> mgz: sorry, better example: https://repl.it/pIW/1
<mgz> natefinch: sure, in practice you get less of it with fewer scopes existing
<mgz> people do still confuse themselves
<natefinch> and in languages where you can't shadow, you get crap like err1 err2 err3, etc
<mgz> err, inner_err, inner_inner_err...
<natefinch> if Go didn't have := and you had to declare every variable as var foo = "bar"  .... it would be much more obvious where the shadowing was occurring. But it would also be annoyingly more verbose.  It's a tradeoff.
<voidspace> right, happy weekend everyone
<voidspace> see you on Tuesday!
<natefinch> voidspace: see ya
<katco> natefinch: heyo, back yet?
<natefinch> mgz, sinzui, abentley: https://github.com/juju/juju/wiki/Testing
<sinzui> natefinch, cd juju-ci-tools; make install-deps
<sinzui> that will provide the real deps need by the files
#juju-dev 2015-05-23
<mup> Bug #1457031 changed: Juju cannot deploy to any substrate <blocker> <bootstrap> <ci> <regression> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.23:Fix Released by ericsnowcurrently> <https://launchpad.net/bugs/1457031>
<mup> Bug #1414369 changed: manual machines should not automatically be removed <juju-core:Expired> <https://launchpad.net/bugs/1414369>
<mup> Bug #1414369 was opened: manual machines should not automatically be removed <juju-core:Expired> <https://launchpad.net/bugs/1414369>
<mup> Bug #1414369 changed: manual machines should not automatically be removed <juju-core:Expired> <https://launchpad.net/bugs/1414369>
<thumper> lazyPower: not sure if you are online, but I just proposed a massive refactor for django-python charm
<thumper> lazyPower: the last merge was very broken
<thumper> lazyPower: this fixes it and adds tests
<lazyPower> ok. i'm not at home, but i can take a look later
<thumper> lazyPower: awesome
<lazyPower> whats weird is it passed the existing tests
<thumper> lazyPower: I will be checking email later
<lazyPower> i guess it had some less than stellar tests
<thumper> and I want to continue to add the celery workers to it
<lazyPower> thumper: i'm thinking like Tuesday - when i get back on the clock.
<thumper> lazyPower: fair enough
<thumper> I'll continue building on it
 * lazyPower nods
<thumper> it is much more understandable and testable now
<thumper> at least, I think so
<lazyPower> Thats awesome to hear, glad you're on the case thumper
<thumper> I need to make sure I can upgrade from my current local charm to this one
<thumper> then I'll be adding celery workers
<thumper> probably today too
<thumper> maybe tomorrow night
<thumper> we'll see
<thumper> laters dude
#juju-dev 2015-05-24
<mup> Bug #1326904 changed: juju sync-tools fails with proxy. <addressability> <network> <proxy> <sync-tools> <juju-core:Expired> <https://launchpad.net/bugs/1326904>
<mwhudson> good morning #juju
<davechen1y> thumper: doh, too slow, http://www.mtviggy.com/wp-content/uploads/2011/05/pid_17864.jpg
<davechen1y> ping http://reviews.vapour.ws/r/1764/
<davechen1y> ping http://reviews.vapour.ws/r/1764/
<thumper> davechen1y: I'll take a look now
<thumper> davechen1y: done, shipit
<thumper> davechen1y: cmd/juju/bootstrap_test.go:246 - tests non-supported architecture failure using arm64
<thumper> davechen1y: this is why your branch won't land
<thumper> davechen1y: we need a different arch there.
<mwhudson> thumper: lol what
<thumper> mwhudson: yeah...
<davechen1y> thumper: facepalm, i'll fix that
#juju-dev 2016-05-23
<davecheney> oh this fslock class
<davecheney> func (r *Reboot) checkForRebootState() error {
<davecheney>         if r.machineLock.IsLocked() == false {
<davecheney>                 return nil
<davecheney>         }
<davecheney> another fun part is the way the reboot worked lets you provide it with a lock ...
<natefinch> yeah, the fslock code is only half the problem, the other half is how the rest of the code is abusing it
<natefinch> is there a trick to bootstrapping maas in 2.0?
<natefinch> $ ./juju bootstrap maas maas --upload-tools
<natefinch> ERROR detecting credentials for "maas" cloud provider: credentials not found
<natefinch> $ ./juju add-credential maas
<natefinch> ERROR cloud maas not valid
<rick_h_> natefinch: I think it's the whole $maas/$IP thing that got added in
<natefinch> rick_h_: I don't even see maas in juju list-clouds... is there help for this somewhere?
<rick_h_> natefinch: in the release notes
<natefinch> rick_h_: people who commit new functionality without proper help in the CLI make hulk angry
<anastasiamac_> natefinch: i think u ned to add cloud of type maas first, then bootstrap it \o/
<natefinch> ug, this is a poor UX
<natefinch> "MAAS works out of the box"   "Bootstrapping models using ... MAAS ... providers also Just Work"  ....lol no
<natefinch> unless we have drastically different ideas of what "Just works" means.
<natefinch> a little ways down "For MAAS and OpenStack clouds, it's necessary to edit a clouds.yaml file."... ahh, ok.  Maybe we should remove the "just works" BS from mentioning MAAS?
<davecheney> great, the worker/uniter test _require_ that BreakLock _always_ breaks the lock
<natefinch> :wq
<natefinch> heh
<davecheney> hmm, or maybe they don't pass at all
<davecheney> nope, they just don't pass at all
<natefinch> nice
<natefinch> davecheney: I do have a new fslock library mostly written.  I got stuck with an error message on windows having to do with overlapped IO, which is required to do the timeout in a non-terrible way... but since the linux code is already doing the timeout in a semi-terrible way, I could easily do windows the same way and have it read to go pretty quickly (or I could spend some time figuring out the windows error)
<natefinch> davecheney: I didn't look too carefully at what it would take to rip out all the garbage from juju's code that requires breaklock etc, however
<mup> Bug #1584565 opened: Documentation regarding user management is out of date <juju-core:New> <https://launchpad.net/bugs/1584565>
<natefinch> davecheney: there also appeared to be some BS About storing a message in the file and then reading the message to determine who had the lock, which I also assumed was just a Bad Idea, and therefore would probably not be hurt by just being ripped out.
<davecheney> that is a monsterous idea
<davecheney> it's not concurrency safe
<davecheney> lock.ReadMessage() => "stale message"
<natefinch> yawp
<davecheney> if you make a decision based on that data, that's a mistake
<natefinch> it's possible it was only something as innocuous as a log message, still bad, but not fatal
<davecheney> maybe that's a way to approach it
<davecheney> remove the Message() method
<davecheney> that might give a more tractable hold on removing BreakLock
<natefinch> davecheney: https://github.com/juju/juju/blob/master/worker/uniter/uniter.go#L394
<davecheney> stop, you'll make me cry
<natefinch> and actually... anywhere at all that calls lock or lock with timeout or lock with func.... https://github.com/juju/utils/blob/master/fslock/fslock.go#L279
<natefinch> which at least is apparently just for logging
<natefinch> gah, what I wouldn't give for a dedicated maas environment that I could test on.
<davecheney> great, c.Logf, doesn't log anything
<natefinch> don't you need either a failure or -checl.vv or something to get those to show up?
<natefinch> this is why I just use fmt.printf, because that just works
<natefinch> (for debugging, obv)
<natefinch> bedtime for me
<davecheney> thanks, that worked
<davecheney> that's stupid
<davecheney> i want log output
<davecheney> i don't want to debug gocheck
<davecheney> which is what -vv does
<davecheney> http://reviews.vapour.ws/r/4881/
<mup> Bug #1584616 opened: mtu configuration should not be moved to bridge interface <juju-core:New> <https://launchpad.net/bugs/1584616>
<mup> Bug #1584616 changed: mtu configuration should not be moved to bridge interface <juju-core:New> <https://launchpad.net/bugs/1584616>
<mup> Bug #1584626 opened: cmd/jujud/agent: MachineSuite.TestDyingModelCleanup is flaky and unreliable <intermittent-failure> <test-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1584626>
<dimitern> babbageclunk: morning :) can you have a second look on http://reviews.vapour.ws/r/4871/ please?
<babbageclunk> dimitern: sure! Looking now.
<dimitern> babbageclunk: ta!
<voidspace> dimitern: http://reviews.vapour.ws/r/4874/
<frobware> voidspace: did you want to chat. 1:1?
<voidspace> frobware: I need coffee!
<voidspace> frobware: anything you want to chat about?
<frobware> voidspace: ok. top of the houor?
<frobware> hour
<voidspace> sure
<dimitern> babbageclunk: the whole point of having "preferred" addresses is that once set, they cannot change (unless they get removed, then another "sticky" preferred will be picked)
<dimitern> babbageclunk: PrivateAddress() and PublicAddress() prefer to return IPv4 addresses if available, falling back to IPv6 otherwise (if any), finally returning NoAddressError
<babbageclunk> dimitern: It seems like those are at odds - if a machine gets an IPv6 address first, and then later gets an IPv4 address, won't its preferred address change?
<dimitern> babbageclunk: AIUI your suggestion, it's about adding a PreferredPrivateAddress []address on machineDoc containing up to 2 addresses (1 for each type)
<babbageclunk> dimitern: Yes, that's what I meant.
<dimitern> babbageclunk: well, the preferred addresses will settle long before the machine agent actually starts (or state transitions to started from pending), so no workloads will be running
<babbageclunk> dimitern: ok, fair enough.
<dimitern> babbageclunk: that specific issue deals with racy DHCPv4 vs DHCPv6 in the lxd provider (either one can come first, but eventually we'd like to use IPv4 as preferred private/public)
<dimitern> babbageclunk: however, your suggestions make a lot of sense now that I'm looking at the implementation..
<babbageclunk> dimitern: Sure, but I still don't understand why you can't implement the fallback logic on top of the addresses being stored in []address
<babbageclunk> dimitern: Ok
<babbageclunk> dimitern: :)
<dimitern> babbageclunk: so I'd try to use a slice of addresses or perhaps a map[type]address
<dimitern> babbageclunk: it might even work better at the end :)
<babbageclunk> dimitern: yeah, I could see wanting a map instead - I wasn't sure about that.
<babbageclunk> dimitern: The reason I didn't suggest that was that it's probably worse space and time wise with only two entries.
<babbageclunk> dimitern: But it also makes the code a bit simpler, so maybe still worth it.
<dimitern> babbageclunk: I doubt it makes much difference - one is a nested doc the other is a nested array
<dimitern> babbageclunk: yeah
<mup> Bug #1584805 opened: Timeout in github.com/juju/juju/apiserver/service on windows <ci> <regression> <test-failure> <timeout> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1584805>
<mup> Bug #1584815 opened: SSHSuite.TestSSHCommand fails on windows <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1584815>
<hoenir> who broke the upstream?
<hoenir> http://paste.ubuntu.com/16633838/
<katco> dooferlad: hey, are you feeling better/back yet?
<dooferlad> katco: not really
<dooferlad> katco: just catching up on email between sleeps
 * dooferlad had the flu
<katco> dooferlad: ouch, i'm sorry to hear that :( i hope you feel better :(
 * dooferlad has a body that still doesn't work
 * katco understands
<mup> Bug #1576313 changed: windows: uniter tests fail because logs get dumped to stderr <ci> <regression> <test-failure> <windows> <juju-ci-tools:Triaged> <juju-core:Fix Released by bteleaga> <https://launchpad.net/bugs/1576313>
<mup> Bug #1584832 opened: payloads are not cleaned up when units are destroyed <juju-core:Triaged> <https://launchpad.net/bugs/1584832>
<babbageclunk> dimitern, voidspace: sorry guys, I can't make the networking knowledge share this week - I have to get home to help with the baby.
<voidspace> babbageclunk: ok
<mup> Bug #1582798 changed: After bootstrapping, juju gui isn't automatically deployed <cpe-sa> <juju-core:Invalid> <https://launchpad.net/bugs/1582798>
<dooferlad> dimitern, voidspace, babbageclunk, frobware: is the networking call on?
<dimitern> dooferlad, voidspace, frobware: omw, sorry - 2m
<voidspace> alexisb: ping
<alexisb> voidspace, ping
<alexisb> pong
<perrito666> ah solo pingpong game
<lazyPower> Thats when you know you're dealing with a pro
<natefinch> lazyPower: http://i.imgur.com/A6VxL6d.gif
<lazyPower> it always comes back ot this for me https://www.youtube.com/watch?v=qHe6vhexm6g
<natefinch> lazyPower: holy crap, is that real?
<lazyPower> very much so
<natefinch> hmm... comments say it's an ad for a nokia phone
<lazyPower> dont take this away from me natefinch
<lazyPower> ;_;
<natefinch> lol sorry
<katco> natefinch: hey don't forget to flag that bug as incomplete if you can't repro
<mup> Bug # opened: 1584896, 1584897, 1584899, 1584900, 1584901, 1584903, 1584905, 1584906
 * fwereade could use a review of http://reviews.vapour.ws/r/4873/ if anyone has some time
<perrito666> 4 pages?
<rick_h_> katco: ping, did we end up with a download % indicator in the resources output?
<katco> rick_h_: no i believe we deferred that. let me 2x check
<rick_h_> katco: ty
<perrito666> fwereade: I +1 but I want another set of eyes there
<katco> rick_h_: yeah, we deferred. see your comment on the resource spec... last comment in the first thread in the doc
<katco> rick_h_: (if that's not too confusing)
<rick_h_> katco: rgr ty
<natefinch> ericsnow: why only alphanumerics and dashes for payload class?  Why only ascii?
<ericsnow> natefinch: starting as minimal as possible; we can broaden it later
<perrito666> I feel discriminated, I want "El PailoÃ¡d"
<fwereade> perrito666, thanks, appreciated
<perrito666> fwereade: in principle, as you said, these seem like almost mechanical changes
<fwereade> perrito666, you have no idea how hard it was to keep it mechanical, but I think I really did
<fwereade> perrito666, the one change was to TestDyingModelCleanedUp and I think I called it out explicitly
<perrito666> you did
<fwereade> perrito666, ok, cool -- and you're happy with the sanity of the high-level changes? that's the really important thing ;)
<perrito666> fwereade: I am, I also celebrate that we have one thing less called test*
<fwereade> perrito666, cool, thanks :)
<cherylj> natefinch: are you still looking into bug 1537585?
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged by natefinch> <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1537585>
<katco> cherylj: yes he is
<cherylj> cool beans
<cherylj> thanks, katco
<katco> natefinch: you never responded to me... this ^^^ is why i asked you to mark the bug incomplete :)
<katco> cherylj: np... fyi he's working with the landscape team because he can't repro
<natefinch> katco: crud, sorry, missed that somehow
<katco> natefinch: no worries, i knew cherylj would come poking around ;)
<cherylj> heh
<katco> natefinch: maybe add a comment to that affect too
 * cherylj grabs her poking stick
<cherylj> yeah, comments good
 * katco puts on her nerd mask
<natefinch> katco: that's no mask! ;)
<katco> lol
<natefinch> ericsnow: left a comment on the names.ValidatePayload review... I don't really see why we need to limit people to such a limited set of characters.  If there's a technical reason, then sure, but I would hope there's not, other than maybe some delimiter somewhere.
<ericsnow> natefinch: why do we do it for all the other tags?
 * perrito666 sees tags a people in the same sentence and smells something weird
<natefinch> ericsnow: no idea
<natefinch> ericsnow: misplaced paternalistic instinct?
<natefinch> ericsnow: eurocentric standards of beauty? :)  Maybe ask fwereade.  Sounds like something someone Decidedâ¢ at some point.
<perrito666> natefinch: is that limitation for tags?
<natefinch> perrito666: yes
<perrito666> tags hasve nothing to do with users, users talk in names
<natefinch> perrito666: I hate our naming system :/
<perrito666> a user will never ever see a tag
<perrito666> if you check names for units there is a whole conversion of names to tags removing invalid chars
<natefinch> perrito666: double checked the code, this is the
<natefinch> perrito666: the  user-facing name, I think
<ericsnow> natefinch: correct; it's the "ID"
<perrito666> then there is no reason as long as you can encode whatever you have into a tag name
<perrito666> (that might be the origin of the limitation btw)
<ericsnow> natefinch: currently the only rationale I have for restricting the ID is that making it more restrictive later has a substantially higher cost that making it less restrictive later
<natefinch> ericsnow: are the tags for payloads new?  I don't see payloadtag in master
<ericsnow> natefinch: no, I believe you added them last year
<perrito666> natefinch: I have them inmaster
<natefinch> ericsnow: my reading skills just suck, evidently :)
<perrito666> bbl, since I need to talk to some downunders if they show up today :)
<natefinch> perrito666: anyone at the sprint will not be here today
<natefinch> perrito666: even thought today is tomorrow for them
<perrito666> you are so underestimating the workoholism of some people
<natefinch> hmm, good point, Ian may be one
<natefinch> on
<natefinch> ericsnow: I don't know... the id-tag-payloadclass thing still confuses the hell out of me.
<natefinch> ericsnow: as you said, it's easy to expand later
<perrito666> beware of expanding names later
<perrito666> you might end up having clashes after sanitization
<natefinch> have I mentioned I hate our naming system?  We have a real "let's encode way too much information into an ID" problem
<ericsnow> natefinch: how strongly do you feel about that "payloadClass" const?  (I'd rather leave it)
<redir> there's only 2 things that are hard in computer science
<redir> cache invalidation, naming things, and off by 1 errors
<natefinch> ericsnow: meh... seems extraneous... but why not put ^ $ in the constant?
<ericsnow> natefinch: habit I guess; often snippets like that are re-usable if you leave the ^ $ off
<natefinch> ericsnow: heh, I try to avoid using regexes in the first place ;)  Sure, that's fine.
<ericsnow> natefinch: k, thanks :)
<thumper> fwereade: sorry I missed our call this morning
<perrito666> Thumper hey dunno if you are working today, but if so, could you see my answers In the acl doc to make sure stuff hasn't changed since the creation of that spec?
<thumper> perrito666: I am, but I have quite a list ...
<perrito666> I can put up a good impression of shreck's cat look :p
<fwereade> thumper, haha, sorry so did I
<fwereade> thumper, didn't quite expect you back and recovered, but I guess it's tuesday for you
<alexisb> thumper, ping
<anastasiamac_> is there something wrong with laning bot? I've asked it to merge my stuff 17mins ago and it did not respond :D
<anastasiamac_> landing*
<davecheney> launchpad is having problems
<davecheney> i just had a build fail because the launchpad.com/tomb package was 503
<katco> davecheney: we should look to migrate to the new version which i believe is hosted on github
<alexisb> thumper, I am on our 1x1 HO when you are ready to join
<wallyworld> anastasiamac_: axw: perrito666: otp, running a bit late
<anastasiamac_> wallyworld: perfect.. just spilled hot drink so love this xtra time to clean up.. ping when ready
<mup> Bug #1584979 opened: better container networking <juju-core:New> <https://launchpad.net/bugs/1584979>
<wallyworld> axw: anastasiamac_: in standup now
<davecheney> katco: yup, v1 and v2 are verys imilar
<davecheney> v2 has one extra method which we can ignore
#juju-dev 2016-05-24
<davecheney> here is a small review for someone, http://reviews.vapour.ws/r/4883/
<thumper> damn it, no closer to starting this email
 * thumper out to walk the dog while the sun is shining
 * redir eod
<mup> Bug #1585005 opened: list-* commands should be aliases for what they're listing <usability> <juju-core:Triaged by macgreagoir> <https://launchpad.net/bugs/1585005>
<mup> Bug #1585005 changed: list-* commands should be aliases for what they're listing <usability> <juju-core:Triaged by macgreagoir> <https://launchpad.net/bugs/1585005>
<mup> Bug #1585005 opened: list-* commands should be aliases for what they're listing <usability> <juju-core:Triaged by macgreagoir> <https://launchpad.net/bugs/1585005>
<mup> Bug #1585015 opened: worker/terminationworker: test timeout during CI <juju-core:New> <https://launchpad.net/bugs/1585015>
<cherylj> thumper, wallyworld - can one of you comment on the proposal in bug 1568944?  (last comment)
<mup> Bug #1568944: Failure when deploying on lxd models and model name contains space characters <juju-core:Triaged by reedobrien> <https://launchpad.net/bugs/1568944>
<wallyworld> ok
<wallyworld> cherylj: want me to create that 2.0 kanban?
<cherylj> gracias
<cherylj> wallyworld: anastasiamac was kind enough to create one for me
<cherylj> I'm working on populating it now
<wallyworld> cherylj: ok, great, i have stuff also
<cherylj> wallyworld: do you know how to add people into the leankit site?  I noticed that christian and mick need accounts
<cherylj> wallyworld: https://canonical.leankit.com/boards/view/122133644#workflow-view
<wallyworld> cherylj: i think so, i'll see if i have perms
<cherylj> thanks!
<thumper> if wallyworld doesn't, I do
<wallyworld> cherylj: christian is already a user on the site
<cherylj> wallyworld: must've missed him :/
 * cherylj checks again
<wallyworld> adding mick now
<wallyworld> cherylj: mick added
<cherylj> yarp, there he is
<cherylj> thanks, wallyworld!
<anastasiamac> cherylj: i have also added christian to our "other" board :-P
<cherylj> thanks, anastasiamac  :)
 * thumper heads away for a bit to write emails
<menn0> windows batch is a nightmare... bad decisions all the way down
<natefinch> menn0: why are you messing with batch files?
<menn0> natefinch: we have tests which patch out executables with fake shell scripts/batch files (depending on platform). i'm fixing one. just about done but it's been painful.
<natefinch> menn0: omg, don't use those
<natefinch> menn0: https://godoc.org/github.com/juju/testing#PatchExecHelper
<natefinch> menn0: lightyears bettter.
<natefinch> menn0: 100% go..
<natefinch> menn0: I sent an email about it a while back.  might require a tiny refactor to the production code, but the testing is SO much better
<natefinch> menn0: just forwarded the email to you
<menn0> natefinch: that looks pretty great but I'm not sure I can use it in this case
<menn0> natefinch: I need to know the contents of a temporary file which gets passed to the executable
<menn0> and gets deleted
<menn0> right now the patched in replacement executable print out the contents of the file they're given
<natefinch> menn0: you could do the same thing that helper does and just code it by hand
<natefinch> menn0: in other words, write your own special test function that does whatever you're trying to do in batch
<menn0> natefinch: hmm ok. i'll take a look
<menn0> ironically, I just got the change working
<natefinch> heh
<natefinch> menn0: I should add another helper that does this rigamarole but takes the name of a custom test function, so you only need to write the test function, and not the exec hacking code.
<bradm> I just hit bug 1537585 if anyone knows how to fix it
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged by natefinch> <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1537585>
<bradm> seems fairly intermittant, I've deployed a number of times without seeing it
<mup> Bug #1585059 opened: juju status tabular format doesn't show controller member status <canonical-bootstack> <juju-core:Confirmed> <https://launchpad.net/bugs/1585059>
<mup> Bug #1585059 changed: juju status tabular format doesn't show controller member status <canonical-bootstack> <juju-core:Confirmed> <https://launchpad.net/bugs/1585059>
<mup> Bug #1585059 opened: juju status tabular format doesn't show controller member status <canonical-bootstack> <juju-core:Confirmed> <https://launchpad.net/bugs/1585059>
<mup> Bug #1583934 changed: revision specified in the charm, how? <juju-core:Invalid> <https://launchpad.net/bugs/1583934>
<babbageclunk> voidspace: around?
<babbageclunk> fwereade: ping?
<babbageclunk> menn0: hey
 * babbageclunk tumbleweeds.
<voidspace> babbageclunk: yep, sorry
<thumper> voidspace, babbageclunk: morning
<thumper> frobware: alive?
<thumper> dooferlad: feeling better?
<voidspace> thumper: morning
<frobware> thumper: yep. thankfully. :-D
<babbageclunk> voidspace: no worries, worked it out.
<voidspace> babbageclunk: I figure that left to your own devices you'll usually figure stuff out
<babbageclunk> morning thumper
<voidspace> babbageclunk: so I never worry about replying to you
<voidspace> babbageclunk: you're probably better off if I don't...
<voidspace> ;-)
<babbageclunk> voidspace: generally with a bit more swearing
<voidspace> babbageclunk: :-)
<mup> Bug #1568079 changed: github.com/juju/juju/apiserver/client unit tests fail if xenial is the LTS <xenial> <juju-core:Fix Released by reedobrien> <juju-core 1.25:Fix Released by reedobrien> <https://launchpad.net/bugs/1568079>
<mup> Bug #1568079 opened: github.com/juju/juju/apiserver/client unit tests fail if xenial is the LTS <xenial> <juju-core:Fix Released by reedobrien> <juju-core 1.25:Fix Released by reedobrien> <https://launchpad.net/bugs/1568079>
<mup> Bug #1568079 changed: github.com/juju/juju/apiserver/client unit tests fail if xenial is the LTS <xenial> <juju-core:Fix Released by reedobrien> <juju-core 1.25:Fix Released by reedobrien> <https://launchpad.net/bugs/1568079>
<frobware> alexisb: ping - audio/HO trouble... brt
<alexisb> frobware, np
<alexisb> frobware, I am going to take the opportunity to get coffee, brb
<alexisb> frobware, heh
<alexisb> frobware, did you want to reschedule?
<alexisb> while you work out audio?
<frobware> frobware: one min
<alexisb> k
<natefinch> dooferlad: you around?
<katco> fwereade: ping
<fwereade> katco, pong
<katco> fwereade: hey, re. http://reviews.vapour.ws/r/4882/
<katco> fwereade: eric is trying to mechanically move this code over to state to enable cmr for tim
<katco> fwereade: can we please land as is and address your comments as bugs. i think eric has opened
<fwereade> katco, what are the odds that my concerns will actually be addressed this year?
<katco> fwereade: lol, i don't know the answer to that. but the priority comes from the top-down
<katco> fwereade: we don't have resources to address these comments right now
<katco> fwereade: we're just trying to move the code because tim wants it over in state
<fwereade> katco, because it needs to be part of a solid migration process... which, there is every likelihood, will be breakable by components that skip referential integrity, because AIUI the migration actually validates the model it sends over
<katco> cherylj: alexisb: i need to pull nate off of bug 1537585 so he can ramp up on what ericsnow is working on b/c ericsnow will be out for pycon
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged by natefinch> <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1537585>
<alexisb> katco, ok
<fwereade> katco, I very much appreciate the structural work ericsnow has done, but it *actually does* need both referential integrity and detailed race testing
<mup> Bug #1585059 changed: juju status tabular format doesn't show controller member status <canonical-bootstack> <juju-core:Confirmed> <https://launchpad.net/bugs/1585059>
<katco> fwereade: i understand, but we need to address those as bugs, separately. eric needs to land this so he can help hand off to nate so he can go to pycon. this is a PM thing, not a code quality thing
<katco> ericsnow: can you demonstrate to fwereade that you have bugs open for the issues he's bringing up?
<fwereade> katco, this is not code quality, this is level 0 "does it actually work"
<fwereade> katco, I cannot convince myself that it works correctly on the basis of the code, and AFAICS there is no test that actually exercises the functionality I am worried about
<katco> fwereade: OK, can we please address those concerns separate from moving the code? it will unblock tim, eric, and nate
<fwereade> katco, I have yet to be convinced that they will be addressed
<fwereade> katco, IIRC, I expressed these concerns a *very long time ago* when this code was first written, and reminded you on several occasions, and they have remained untouched
<fwereade> katco, and if it goes into state like this people will use it as a model and spread the badness
<katco> fwereade: ok, that is a valid concern. we'll have to drop this patch for the moment then. ericsnow needs to be working with natefinch before he leaves
<fwereade> katco, ack, thank you
<katco> fwereade: i'll send an email to tim and cc you explaining the reasoning
<mup> Bug #1585289 opened: Juju2: websockets API should alert, error or warn about ignored parameters <landscape> <usability> <juju-core:New> <https://launchpad.net/bugs/1585289>
<mup> Bug #1585300 opened: environSuite invalid character \"\\\\\" in host name <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1585300>
<redir> fwereade: yst?
<perrito666> hello, sorry all I got a small relapse last night, I am well and alive now, anyone needs a review?
<katco> perrito666: welcome back to the land of the living
<katco> perrito666: you're working on acl stuff, correct?
<perrito666> katco: I am
<perrito666> I have been in the world of the living for a few hours but was working without looking at the chat
<katco> perrito666: cool.
<natefinch> ericsnow: do you want to pair?
<ericsnow> natefinch: sure
<ericsnow> natefinch: give me a couple minutes
<natefinch> ericsnow: np
<jcastro> katco: heya
<katco> jcastro: o/
<jcastro> I noticed the last few releases the controller is now much more robust inbetween reboots, etc.
<jcastro> it hasn't failed on me at all
<katco> jcastro: i'm sorry, we'll try to introduce more bugs.
<jcastro> I just wanted to tempt fate
<alexisb> :)
<katco> jcastro: seriously, that's good to hear :)
 * perrito666 ads a call to ohYouHadToSayIt(jcastro)
<natefinch> lol
<ericsnow> natefinch: sorry about that
<ericsnow> natefinch: ready?
<natefinch> ericsnow: ye[
<natefinch> yep
<perrito666> bbl
<redir> is the mergebot alive?
<redir> yes it is
<thumper> morning
<lazyPower> o/ thumper
<mup> Bug #1585359 opened: juju debug-log prints some log messages, but quickly stops streaming <juju-core:New> <https://launchpad.net/bugs/1585359>
<mup> Bug #1585361 opened: megawatcher delta is missing data (service & relation) <juju-core:New> <https://launchpad.net/bugs/1585361>
<thumper> hmm..
<alexisb> thumper, hmm... ??
<thumper> I was 2 minutes late for the standup and no one was there...
<thumper> then michael turned up
<wallyworld> alexisb: sts call? you should attend this one
<alexisb> wallyworld, be there soon
<sinzui> thumper: bug 1585388 is the new form of broken ssh to a container. the sxample shows --proxy was used.
<mup> Bug #1585388: Container networking cannot ssh after machine is ready <ci> <lxc> <lxd> <network> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585388>
<mup> Bug #1585388 opened: Container networking cannot ssh after machine is ready <ci> <lxc> <lxd> <network> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585388>
<mup> Bug #1585388 changed: Container networking cannot ssh after machine is ready <ci> <lxc> <lxd> <network> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585388>
<perrito666> mmm, we error out mentioning a model by its uuid, that is a ux faux pas
<mup> Bug #1585388 opened: Container networking cannot ssh after machine is ready <ci> <lxc> <lxd> <network> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585388>
<alexisb> ug sorry wallyworld, cherylj I missed that call
<wallyworld> alexisb: am sending an email
<alexisb> thanks
<wallyworld> anastasiamac: perrito666 axw be right there
<natefinch> davecheney: online but encumbered by a sleeping toddler, if you want to talk fslock
#juju-dev 2016-05-25
<davecheney> natefinch: see tim's email thread from last year about the proposed design
<davecheney> sorry i cannot link to it
<davecheney> i dunno how to link to messages on that ML
<natefinch> davecheney: search for diaf is surprisingly useful
<davecheney> :)
<mup> Bug #1582841 changed: remove check for lxdbr0 <juju-core:Invalid> <https://launchpad.net/bugs/1582841>
<natefinch> davecheney: I don't see anything in that thread that says that flock won't work.  It seems to say flock won't release on process end, but that's incorrect from my reading and experience.  go closes the file handle and the lock is released on process exit.
<natefinch> er lock is released when the file descriptor is closed, which happens automatically on exit
<natefinch> davecheney: if you just want to avoid futzing with files on disk, that's valid, too
<redir> anastasiamac: FWIW I only found these 3 https://goo.gl/JG8NXL none of which would have been fixed by the updates to aggregator AFAICT.
<davecheney> natefinch: please, no flock, use unix domain sockets as we discussed
<natefinch> davecheney: ok
<davecheney> flock requires a file on disk
<davecheney> if I delete that file I can effeictly release that lock
<davecheney> and we're back to square one
<perrito666> can you do that in windows? (without a signifficant effort?)
<natefinch> nope
<natefinch> windows is a real lock
<natefinch> I know flock is only advisdory
<davecheney> it's not that flock is advisory
<davecheney> but the thing that is being locked can be moved or deleted which breaks the lock invariant
<davecheney> flock also leaves with the 'oh i crashed and there is a file on disk' problem
<natefinch> man page says "Apply or remove an advisory lock on the open file specified by fd."  maybe the docs are misleading, but regardless.. yes, you can break it trivially
<natefinch> the file on disk doesn't matter
<davecheney> yes it does
<davecheney> you need to have a file to apply a lock to
<natefinch> the existence of the file has no meaning
<davecheney> you cannot flock a non existant file
<davecheney> so you have to create that file
<natefinch> well, yes
<davecheney> so now you have two lockers racing to create the file
<davecheney> if you fail to create the file because someone else was racing with does taht mean you cannot lock it, etc
<perrito666> k people, EOD, see you all on thu
<davecheney> please just use unix domain sockets
<davecheney> they solve 100% of these problems
<natefinch> it's really not a race, but that's fine
<davecheney> net.Dial("unix", "@/juju/$SOMEHASH")
<natefinch> I open with O_create... one proc will create first, the other will open
<davecheney> good point
<davecheney> but please
<davecheney> no files on disk
<natefinch> I agree that files on disk are a liability.  I was just doing it that way to minimize changes to current code.  but we can do socket on linux and mutex on windows
<bradm> fwiw I seem to be hitting bug 1537585 a lot more with juju 2 and maas 2
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1537585>
<davechen1y> thumper: what tha ?
<davechen1y> lucky(~/src/github.com/juju/juju/cmd/jujud/agent) % go test -i -v . && go test .
<davechen1y> ok      github.com/juju/juju/cmd/jujud/agent    122.841s
<davechen1y> lucky(~/src/github.com/juju/juju/cmd/jujud/agent) % go test -race -i -v . && go test -race .
<davechen1y> ok      github.com/juju/juju/cmd/jujud/agent    1.168s
<davechen1y> ^ this test runs 100x faster under the race detector ...
<natefinch> lol, we should use the race detector during production, evidently
<davechen1y> uh oh
<davechen1y> i have a terrible feeling about this
<natefinch> I have a great feeling that this is going to be an awesome bug
<davechen1y> lucky(~/src/github.com/juju/juju/cmd/jujud/agent) % pt -i race
<davechen1y> package_test.go:
<davechen1y> 15:     if testing.RaceEnabled {
<davechen1y> 16:             t.Skip("skipping package under -race, see LP 1519133, 1519097")
<natefinch> lol
<natefinch> nice
<natefinch> bug 1519133
<mup> Bug #1519133: cmd/jujud/agent: data race <2.0-count> <juju-core:Triaged> <https://launchpad.net/bugs/1519133>
<natefinch> bug 1519097
<mup> Bug #1519097: juju/utils/fslock: data race caused by createAliveFile running twice <2.0-count> <race-condition> <tech-debt> <juju-core:Fix Released by dooferlad> <https://launchpad.net/bugs/1519097>
<davechen1y> OH MY FUCK
<davechen1y> this god damn fslock nonsense
<natefinch> what, it's fix released
<natefinch> no more race
<davechen1y> let's remove that check and see what breaks
<natefinch> well, the fslock one we are working on
<mup> Bug #1585424 opened: all: data race's in tests are surpressed <juju-core:New> <https://launchpad.net/bugs/1585424>
<davechen1y> thumper: https://bugs.launchpad.net/juju-core/+bug/1585424
<davechen1y> please see email
<mup> Bug #1585424: all: data race's in tests are surpressed <juju-core:New> <https://launchpad.net/bugs/1585424>
<davechen1y> and let me know your decision
<mup> Bug #1585424 changed: all: data race's in tests are surpressed <juju-core:New> <https://launchpad.net/bugs/1585424>
<davechen1y> bug 1519095
<mup> Bug #1519095: state: tests to not pass with -race under Go 1.2 <2.0-count> <juju-core:Triaged> <https://launchpad.net/bugs/1519095>
<davechen1y> oh wow
<davechen1y> welp that's what happens when a bug isn't a blocker
<davechen1y> it goes straight to the bottom of the pool
<mup> Bug #1585424 opened: all: data races in tests are surpressed <juju-core:New> <https://launchpad.net/bugs/1585424>
<natefinch> davechen1y: IIRC the idea was that at least we wouldn't get any *new* data races in core if we're running the race detector.  The problem being that we never went back and actually fixed the ones that got skipped
<davechen1y> yeah, that's what I remembmer now
<davechen1y> i remember it was an uphill battle to stop new races being added
<davechen1y> it's good to know that we can prdict the future
<davechen1y> http://reviews.vapour.ws/r/3204/
<davechen1y> :(
<natefinch> *sad trombone*
 * davechen1y considers replacing paul the octopus as a fortune teller
<mup> Bug #1585430 opened: Cloud-init failed on windows <ci> <cloud-init> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1585430>
<davechen1y> if you thought the state tests were slow without -race, just wait til you see them with -race
<natefinch> heh yeah, -race and -cover are always interesting additions
<davechen1y> thumper: cherylj https://github.com/juju/juju/pull/5452
<davechen1y> please read the description closely
<thumper> I think our timeout for race is like 60 minutes
 * thumper double checks
<davechen1y> that's probably _just_ enough
<thumper> 30 minutes
<thumper> go test -race -test.timeout=1800s ./...
<davechen1y> it'll be tight
<davechen1y> this passed in 1500s on my machine
<davechen1y> without any other tests running
<davechen1y> it'll probably be ok
<davechen1y> at least we know which change to revert
<thumper> right
<thumper> I say push it in
<davechen1y> jolly good
<mup> Bug #1583893 changed: 1.25.5: goroutine panic launching container on xenial <landscape> <juju-core:Fix Released> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1583893>
<mup> Bug #1583893 opened: 1.25.5: goroutine panic launching container on xenial <landscape> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1583893>
<davechen1y> cherylj: anastasiamac https://bugs.launchpad.net/juju-core/+bug/1518806
<mup> Bug #1518806: apiserver: tests to not pass with -race under Go 1.2 <2.0-count> <juju-core:Triaged by dave-cheney> <https://launchpad.net/bugs/1518806>
<davechen1y> i think there is a still a race here
<davechen1y> what testing did you do and why do you think the race is not present ?
<davechen1y> anastasiamac: cherylj for reference the race I see is https://bugs.launchpad.net/juju-core/+bug/1519133/comments/2
<mup> Bug #1519133: cmd/jujud/agent: data race <2.0-count> <juju-core:Triaged by dave-cheney> <https://launchpad.net/bugs/1519133>
<cherylj> davechen1y: I ran the race test locally
<cherylj> specifically for apiserver/
<anastasiamac> davechen1y: did u ran only apiserver package with race detector (in isolation) or started at the root, i.e. from top of juju/juju?
<davechen1y> anastasiamac: did you remove the skip ?
<davechen1y> that package knows when it's run under the race detector and skips all the tests
<davechen1y> it does log it
<anastasiamac> davechen1y: i have not run race detector here as I have not been in this package yet
<davechen1y> but the log is not printed anywhere
<davechen1y> :(
<davechen1y> righto, i guess it's not fixed then
<anastasiamac> if u r seen it, then it does not sound fixed ;-P
<anastasiamac> it could have also been re-introduced?..
<cherylj> I ran the test locally removing the skip and didn't see a race in apiserver/, but that doesn't mean that there isn't one
<anastasiamac> at some stage, we should really unskip tests that r related to bugs that are considered fixed...
<davechen1y> cherylj: i think it's not well exercised by the race
<davechen1y> the tests in cmd/jujud/agent agetate the cert updater so that's why it shows up there
<davechen1y> but ht race is definitely in the apiserver code
<davechen1y> i'll try to write a test that agrevates the problem
<natefinch> davechen1y: when you said use unix domain sockets and net.Dial(....) did you mean net.listen?  can you dial a domain socket without anyone listening?
<davechen1y> yes, i'm sorry, i meand net.ListenUnix
<natefinch> EOD++ for me.  night all.
<anastasiamac> axw: wallyworld: this is permissions check that i was talking about at standup - http://reviews.vapour.ws/r/4895/
<axw> ok, taking a look
<wallyworld> anastasiamac: but
<wallyworld> 		// password, so that we don't allow unauthenticated users to find
<wallyworld> 		// information about existing entities.
<wallyworld> that information hiding is now lost
<wallyworld> it's like when you enter user/pass into a banking website
<wallyworld> it just says bad credentials or something
<wallyworld> it doesn't tell you what was invalid
<anastasiamac> it's not lost, we return permission denied without saying that entity does not exist
<anastasiamac> when u go to the bank website, they say the same thing
<wallyworld> right, and then when the correct entity is passed, we then return BadCreds right?
<wallyworld> so 2 different errors depending if the user is valid or not
<anastasiamac> we have a reaction to bad credentials error - we restart stuff... so we need to ensure that we only return bad credentials when we intent to resat agent, for eg..
<anastasiamac> yes, if user is valid but does not have permission - deny acces
<wallyworld> but the error code becomes different so we leak info
<anastasiamac> if user/pwd is not valid, bad credentials - do whatever needs to be done like restart...
<anastasiamac> we do not leak info - we do not say 'not found"
<wallyworld> the point is - we need to return the same error if user/pass together are not valid, regardless of cause
<anastasiamac> we say - u have no permissions to do what u r trying to do
<wallyworld> you can infer the not found
<wallyworld> by looking at the return error
<anastasiamac> if we return bad credentials error, for example, we terminate api... we do not want to that when the login entity has valid creds but not permissions
<wallyworld> sure, that's an upstream problem though
<wallyworld> we cannot solve that issue by leaking information
<anastasiamac> ?
<wallyworld> the comment that was deleted in the PR explained why it was done that way
<anastasiamac> this is the reason we keep restarting agents
<anastasiamac> permission error doe s not imply something is not found...
<wallyworld> it does in this context as it is different to what's returned when the entity is found
<wallyworld> the question to ask is - why is the api being called with a bad user?
<wallyworld> isn't that the root cause issue?
<anastasiamac> the root cause is in several bug, the prime one is refered to in PR.
<anastasiamac> essentially, we (juju) reacts differently to bad creds
<anastasiamac> so we  need to be careful about when we throw it
<wallyworld> sure, but we can't fix the issue by leaking information
<anastasiamac> otherwsie, we;'ll have a reaction that is too drastic, is uncalled for and is not helping
<anastasiamac> \o/ i have difficulty seeing info leak
<wallyworld> if i'm a hacker, i throw different user/pass at the api. if i get an errperm vs an badcreds, i can deduce if the user exists or not
<wallyworld> that's an information leak
<wallyworld> once i know a user exists vs not exists, the cracking problem becomes much easier
<wallyworld> i now just need to guess the passowrd
<anastasiamac> sure, but if "entity" is not found, u really do not want to restart everything either
<wallyworld> correct
<wallyworld> so we need a different solution
<wallyworld> that doesn't leak info
<anastasiamac> wel, we need an error that is not bad creds (coz we have a reaction to that)
<anastasiamac> the only other one that we have is PermErr
<wallyworld> we need to understand the upstream root cause
<anastasiamac> whcih is actually what we are after - permission is denied \o/
<wallyworld> why is the caller passing a bad entity name?
<anastasiamac> this func is not just used by users, if u look for dependent calls
<anastasiamac> also i think that this conversation should be taken off-line and have Will and Andrew involved :D
<anastasiamac> m happy to have it anytime and m happy with any solution that involves not to restart for every failure
<wallyworld> sure, we can discuss. but it's an information leak, and there was a comment that explained why. if it is decided we don't care about the information leak, then fine. but it's risky to leak information without a proper discussion
<anastasiamac> true information leask is a concern - m not convinced this si the case here.... another concern is the restart that is caused by bad creds... solution needs to ensure that neither happen :D
<wallyworld> disclosure of whether an user protected by a password exists or not is an attack vector
<wallyworld> the information is whether the user exists and it is leaked
<wallyworld> if different errors are returned. it's why banks for example always just return a generic "invalid user password combination" error
<wallyworld> they don't say *which* of the user or password is wrong
<anastasiamac> in this instance, find entity determines if the call is made by machine/unit/user/service/etc... r u saying that if an unknown to us machine is making a call, it is the correct call for us to restart the api? :D
<axw> anastasiamac wallyworld: what has the server's behaviour got to do with the client's behaviour around restarting the API? that's a client policy. you should not be leaking information (yes there *is* leakage) from the server to control the client's behaviour
<wallyworld> +1
<wallyworld> that's my point
<axw> anastasiamac: like wallyworld says, if you can infer that the username is valid because "not found", then you're leaking info, and htat's a security concern
<axw> that reduces the cost of a brute foce attack to focusing mostly on the password
<axw> force*
<wallyworld> yep
<anastasiamac> sure and i agree
<anastasiamac> but throwing bad creds here if the entity is not a user, is what is causing issues
<axw> anastasiamac: so the issue is related to the unhandled tag kind?
<anastasiamac> axw: the issue is thatwe use given tag to find entity. if entity is not found we throw bad creds which we react to.. i think we should throw something else here
<anastasiamac> m happy to revert this badcreds instance for now and only keep the other instances (that are more cleanly defined) in this PR
<davechen1y> axw: this fix is surprisingly simple, can I get a sanity check https://github.com/juju/juju/pull/5455
<davechen1y> thnks!
<axw> anastasiamac: I don't understand how can say "sure and i agree" and then what you just said. either we return ErrBadCreds, or we leak information
<axw> davechen1y: looking
<axw> davechen1y: LGTM
<anastasiamac> axw: I agree about obfuscating and not leaking info :D i am not convinced that badcreds is always the right answer here.. maybe only for user tags...either way, i'll come back to this instance later but will adress other comments on PR after school/kids pick-up
<davechen1y> who has the power to unblock a message to juju-dev ?
<wallyworld> axw: "application-wordpress" or "app-wordpress" ?
<axw> wallyworld: "app-" please
<wallyworld> ok, i kinda like application- :-)
<wallyworld> app- is ok i guess
<axw> wallyworld: feel free to get a second opinion, I just think application- is tedious
<wallyworld> yeah it is, but app- sorta grates for some reason. maybe just me
<wallyworld> axw: if you have a moment, first of a few sigh http://reviews.vapour.ws/r/4897/
<axw> wallyworld: hrm, app- looks grating to me too now. maybe bring it up in the meeting later
<wallyworld> axw: i do think application works better
<wallyworld> it's not much longer than service
<wallyworld> i have a charm.v6 change queued up, so i'll change to application-
<axw> wallyworld: did you intend to change the package paths to gopkg.in?
<wallyworld> yeah
<wallyworld> so we can rev up the version
<wallyworld> to names.v2
<axw> right, ok
<wallyworld> so we don't buuger up 1.25 etc
<wallyworld> there'll be a lot of churn in all this sadly
<mup> Bug #1584059 changed: Deployment of swift-storage charms fails with Juju 2.0 - swift-storage-relation-joined KeyError: 'JUJU_ENV_UUID' <oil> <juju-core:Triaged> <swift-storage (Juju Charms Collection):New> <https://launchpad.net/bugs/1584059>
<wallyworld> axw: next one https://github.com/juju/charm/pull/210
<wallyworld> gotta duck out for school pickup, will look at yuors in a bit
<wallyworld> axw: the yaml thing - we had the relations data unmarshalled with the v2 Unmarshall, and I think the service data with GetYaml() from v1, it was a bit of a mess
<wallyworld> i had a quick look at the v2 release notes and didn't see anything that jumped out
<wallyworld> multi-line strings have been improved, but i don't think that affects charms
<axw> wallyworld: non capisco. is anything feeding charm metadata into yaml.v1? not sure which things use it TBH. charmstore possibly? new charm tool?
<axw> no argument that it's a mess, just not sure if we can fix it without breaking others
<wallyworld> given we had a mix of v1 and v2 before, it seems unlikely we'd be relying on v2 specific behaviour
<wallyworld> the charm metadata is fairly simple yaml i guess?
<wallyworld> nothing too esoteric
<axw> wallyworld: my point is that (AFAIK) the MarshalYAML methods won't be called if the structs are passed to yaml.v1
<axw> wallyworld: so the custom behaviour that was in GetYAML won't be triggered
<wallyworld> yeah, that's correct
<wallyworld> i can't fathom why we'd need both
<axw> wallyworld: only because some clients might not have been updated
<wallyworld> now is the time ot make this change i guess
<wallyworld> it is charm.v6-unstable, clients out there would be using .v5
<axw> ok
<wallyworld> that's my self justification
<wallyworld> if i say it often enough it makes sense
<axw> wallyworld: :)
<axw> wallyworld: seems fair, I'm just not sure what the impact is
<wallyworld> yeah, i'm not 1000% sure
<wallyworld> we can fix anything that comes up easily enough
<axw> wallyworld: yep ok, consider the question dropped
<wallyworld> ok
<fwereade> wallyworld, in hangout
<bradm> anyone know what the deal with br-bond1 is?  we have 2 bonds on these machines, and juju creates br-bondx for both of them, but only one of them is put in the interfaces file
<bradm> this is with juju-2
<wallyworld> bradm: dimitern or frobware would know
<frobware> bradm: can you PB the original /e/n/i file and Juju's morphed version.
<bradm> frobware: thats the point, there is no details about br-bond1 in /e/n/i
<frobware> bradm: MAAS?
<bradm> frobware: yup
<bradm> frobware: in this case, maas 2.0 with juju 2.0
<frobware> bradm: ah. haven't really tried MAAS 2.0 that hard. hmm.
<wallyworld> bradm: frobware: maybe voidspace then
<bradm> I have to run to take the boy to cubs, will check back in later
<frobware> bradm: but there should be the original /e/n/i available
<dimitern> frobware, bradm: hey, what's the issue?
<bradm> frobware: ayup, there is - it adds br-bond0 to /e/n/i, but not br-bond1
<frobware> bradm: it has the extension -before-juju-add-bridge.py
<bradm> got to run, didn't realise the time
<frobware> bradm: ack
<dimitern> perhaps is br-bond1 unconfigured and has no address, but it has configured 'children'?
<davechen1y> can someone with mod rights on juju-dev please let my message out of moderation
<davechen1y> ta!
<rogpeppe> anyone know anything about the reasoning behind https://github.com/juju/juju/pull/5100 ?
<rogpeppe> for example, was it deliberate to remove "juju help plugins" ?
<rogpeppe> fwereade, davechen1y, axw: ^
<bradm> frobware, dimitern: yes, our bond1 is unconfigured, we use it for neutron-gateway lxcs
<frobware> bradm: so juju should not create a bridge for it, but it should still be present in /e/n/i
<dimitern> bradm: so you have bond1 unconfigured but up and e.g. bond1.42 also unconfigured and up?
<bradm> frobware: right, that would do us - we create a bridge ourselves in the preseed
<dimitern> oh..
<bradm> dimitern: in maas we create teh bond and leave it unconfigured, we add it to the lxcs as an extra interface, so we do need a bridge for it
<bradm> which is fine if you guys do it, we can use whichever bond is there
<bradm> er, bridge
<bradm> just having 2 bridges on the interface doesn't work so well :)
<dimitern> bradm: well, as long as the bridge you create in the preseed is called 'br-bond1' and e.g. 'br-bond1.42' for the neutron-ext vlan, the bridge script shouldn't mess with it
<frobware> bradm: can you PB the original /e/n/i file - it should be /etc/network/interfaces-before-juju-add-bridge.py
<frobware> bradm: are there any VLANs configured on the bond?
<bradm> frobware: https://pastebin.canonical.com/157211/ - nope, very plain
<bradm> I can try creating it as br-bond1 tomorrow to see if that fixes it
<frobware> bradm: your dns-nameserver entry (x.y.z.5) - is that right?
<bradm> yeah, I just hid our ip range, out of habit
<bradm> frobware: thats the IP of the maas node.
<frobware> bradm: ok, just checking... :)
<frobware> bradm: so I don't see juju messing with bond1 at all
<bradm> trying to do as much as we can in maas, and not hacking up preseeds for it
 * frobware tries to recall what the issue was/is ...
<bradm> frobware: right, yet there's a br-bond1 up
<bradm> frobware: there's commands in the cloud-init-output.log that configure it
<bradm> frobware: https://pastebin.canonical.com/157212/
<frobware> bradm: so juju will create br-bond0 based on your original eni - http://pastebin.ubuntu.com/16676634/
<bradm> actually, there's a br-ethx for all of the eth devices too
<frobware> bradm: ah... let me check something...
<bradm> but only br-bond0 gets written out to /e/n/i
<bradm> maybe because none of the others have IPs?  dunno
<frobware> bradm: br-bond0 get written to eni because its parent device is 'static'
<frobware> bradm: bond1 is manual.
<frobware> bradm: so does not get bridged
<bradm> frobware: ah, but it does!  it creates a br-bond1
<bradm> frobware: its just not in /e/n/i
<frobware> bradm: right. that's my "aha" - just checking something...
<bradm> frobware: like I said, all the interfaces have a bridge
<bradm> well, except for lo
<frobware> bradm: I would say, based on the input, the /e/n/i as written looks correct, just checking on something that has changed in the bridge script quite recently.
<bradm> frobware: yup, /e/n/i is what we want.  its all the extra interfaces that are up we don't want :)
<frobware> bradm: agreed. state of interfaces should reflect what's in /e/n/i
<bradm> frobware: https://pastebin.canonical.com/157215/ <- those are the extra interfaces
<bradm> we just create our br1 in the maas preseed, which is fine
<frobware> dooferlad: ^^
<bradm> I can see in the cloud-init-output log where it does it all
<frobware> dooferlad: I have a sneaky suspicion your recent changes to the bridge script may be bringing interfaces up where it shouldn't...
<bradm> honestly, either way works - if it wants to create a br-bond1 because the interface is there, thats fine, we can use it
<bradm> just need it in /e/n/i
<anastasiamac> bradm: plz clarify - r u using juju2.beta7 or master tip?
<bradm> anastasiamac: juju2 beta 7 with maas 2.0 beta 5
<dooferlad> frobware: that is horrid. Yea, maybe that is happening.
<frobware> dooferlad: bridge_now() should consider is_active - we are creating bridges where we should not.
<frobware> dooferlad: see https://pastebin.canonical.com/157215/
<frobware> bradm: this is a recent regression
<frobware> bradm: I'm guessing that if you were to reboot the machine it would come up as you expect
<bradm> frobware: ayup
<bradm> frobware: well, except the lxcs are cranky because they couldn't access anything in our public IP range, so neutron is a little upset
<bradm> but, cool, happy to see its not me totally misunderstanding something about the new stuff, these things happen.
<frobware> bradm: I think we just need to get a fix and a binary to you. Would that work?
<bradm> frobware: sure, thats fine, this is for an internal stack right now
<bradm> frobware: and I'm not at work for another 12 hours anyway. :)
<frobware> bradm: you happy with some unofficial build from some bloke off the net?
<bradm> frobware: ideally we want it fixed in upstream :)
<frobware> bradm: of, of course.
<frobware> bradm: I was merely trying to find a way to unblock you.
<frobware>  dooferlad, bradm: https://bugs.launchpad.net/juju-core/+bug/1585582
<mup> Bug #1585582: MAAS bridge script bridges inactive interfaces <network> <juju-core:New for dooferlad> <https://launchpad.net/bugs/1585582>
<mup> Bug #1585582 opened: MAAS bridge script bridges inactive interfaces <network> <juju-core:New for dooferlad> <https://launchpad.net/bugs/1585582>
<bradm> frobware: perfect.
<fwereade> ericsnow, katco: I can add a payload with ID "xyz" (and class type/name docker/payloadA), but when I list the unit payloads it's reported as a Result with ID "payloadA"
<fwereade> ericsnow, katco: I presume this is expected behaviour, but I'm not sure what it's for
<fwereade> ericsnow, katco: can you enlighten me?
<voidspace> dimitern: ping
<dimitern> voidspace: pong
<voidspace> dimitern: subnets in model migration
<voidspace> dimitern: would you consider spacename and availabilityzone fields of the subnet to be "optional"?
<voidspace> dimitern: (i.e. should they be omitempty in the yaml)
<voidspace> dimitern: what do you think?
<dimitern> voidspace: I think we should make best effort to set them, but failing that it's better to have them stored empty (i.e. not omitempty)
<voidspace> dimitern: ok, cool - thanks
<voidspace> no default on the import schema either then
<dimitern> voidspace: yeah, I think so
<dimitern> voidspace: on a side-note: the subnetDoc should support storing multiple AZs per subnet
<voidspace> dimitern: yeah, it should
<voidspace> fwereade: ping
<voidspace> dimitern: ping
<dimitern> voidspace: pong
<voidspace> dimitern: fwereade: the State.AddSubnet method has a checkModelActive in the transaction
<dimitern> voidspace: yeah?
<voidspace> this causes adding a subnet during a model migration import to fail because the model is not active
<voidspace> the obvious solution is to remove the check...
<voidspace> is that ok, or is there a "right way" to do this
<dimitern> voidspace: I don't think that's ok
<dimitern> voidspace: removing the assert I mean
<dimitern> voidspace: why won't the model be alive ?
<voidspace> this test fails: https://github.com/juju/juju/compare/master...voidspace:model-migrations-subnets#diff-62dc0c2f793cbec94ae9fcbdf9c9ab84R507
<voidspace> on that assert
<voidspace> it fails with: github.com/juju/juju/state/model.go:878: model "new" is being migrated
<voidspace> dimitern: a model that is being migrated is not active
<voidspace> dimitern: the checkModelActive assert actively checks for this
<voidspace> dimitern: see the definition of checkModelActive
<dimitern> voidspace: well, the checkModelActive is pretty common - how is that working for other types?
<voidspace> dimitern: AddSpace doesn't do the check
<dimitern> voidspace: it should btw; but how about e.g. machines, units, ..
<dimitern> voidspace: maybe it's ok to skip the checkModelActive check *only* during migration?
<babbageclunk> Weird - if I cd '$GOROOT/src/github.com/juju/juju/mongo; go test -v -check.v' the tests all run twice. Does anyone know why that is?
<babbageclunk> Does anyone else see that?
<dimitern> babbageclunk: perhaps due to how '-v' and '-check.v' are handled?
<babbageclunk> Oops, meant $GOPATH
<dimitern> babbageclunk: check `go help testflag`
<voidspace> dimitern: machine import has it's own transaction and manually inserts the machine into state
<babbageclunk> Well, without -v you don't see the output from -check.v
<dimitern> voidspace: so then it sounds like AddSubnet's ops cannot be used literally for import
<voidspace> dimitern: copying and pasting them sounds worse
<dimitern> babbageclunk: try '-check.v -test.v' in that order?
<dimitern> voidspace: they are likely to be quite different anyway, aren't they?
<voidspace> dimitern: identical minus that check
<voidspace> I think, let me confirm
<dimitern> voidspace: assuming they are called in a certain order - e.g. if spaces weren't yet imported..
<babbageclunk> dimitern: oh no - taking off the -v I still see the (duplicate) output
<voidspace> dimitern: we only store SpaceName on subnet
<voidspace> dimitern: so order doesn't matter
<voidspace> AddSubnet is quite short - I can copy it into migration_import and remove the assert
<dimitern> babbageclunk: usually, mixing '-check.v' and '-test.v' is not a good idea - as -check.v works for the current package, whereas '-test.v' can work on a root import path recursively - e.g. go test -v github.com/juju/juju/...
<voidspace> not so terrible I guess
<babbageclunk> dimitern: Could you try 'cd $GOPATH/src/github.com/juju/juju/mongo; go test -check.v -check.f oplogSuite.TestRestartsOnError' for me and see if you see the test run twice?
<dimitern> voidspace: or perhaps better, make a helper that returns addSubnetOps which does not add checkModelActive and use it for both import and AddSubnet, but in the last case append the check before running?
<voidspace> dimitern: heh, yeah - better
<dimitern> babbageclunk: just a sec..
<voidspace> dimitern: ta
<babbageclunk> dimitern: Thanks! I think the -check.v flag is a red-herring - I'm trying to understand why the tests run twice.
<dimitern> babbageclunk: yeah, it does run the test twice
<dimitern> babbageclunk: so not related to -check.v / -test.v
<babbageclunk> dimitern: What's up with that?
<dimitern> babbageclunk: usually it's because a 'baseSuite' or something is embedded in 2 other, separately registered suites (i.e. var _ = gc.Suite(&SuiteEmbeddingBase{})
<dimitern> babbageclunk: due to the embedding, any baseSuite.TestFoo() will be called once for each SuiteEmbeddingBase like that
<dimitern> as TestFoo is 'inherited'
<babbageclunk> dimitern: I looked to see if there was a suite embedding this one, but no dice. Also this one just embeds BaseSuite.
<dimitern> babbageclunk: indeed, might be something else..
<dimitern> babbageclunk: always try '-check.vv' in such cases to see which setup/teardown methods where called
<alexisb> cherylj, are you joining us this morning
<alexisb> fwereade, ping
<babbageclunk> dimitern: ok, I'll try that too, thanks.
<dimitern> babbageclunk: ah! got it.. nasty indeed - internal_test.go:31
<dimitern> babbageclunk: calling gc.TestingT twice in the same package is *infinitely* more sinister than embedding a suite
<babbageclunk> dimitern: Ooooh. Nasty!
<dimitern> babbageclunk: (it's already called in package_test.go as well)
<dimitern> babbageclunk: and gc.TestingT is the entry point gc uses to hook into 'go test' machinery
<babbageclunk> dimitern: Right. So is there any reason not to just delete the internal_test one?
<dimitern> while gc.Suite is registering a suite (not fixture - it needs TestXX() methods to be a suite) into gc to run
<dimitern> babbageclunk: please do, and I'm sure fwereade might like to add this to the guidelines :) never run gc.TestingT more than once per package
<babbageclunk> dimitern: Awesome - thanks for the help!
<dimitern> np :)
<hoenir> could anyone review my PR?
<hoenir> http://reviews.vapour.ws/r/4900/
<voidspace> dimitern: babbageclunk: http://reviews.vapour.ws/r/4901/
<dimitern> voidspace: looking in a sec
 * voidspace lunches
<sinzui> abentley: can you review https://code.launchpad.net/~sinzui/juju-ci-tools/controller-model/+merge/295657
<abentley> sinzui: sure.
<babbageclunk> perrito666: ping?
<babbageclunk> He's probably off doing fun birthday things.
<hoenir> could someone review my PR http://reviews.vapour.ws/r/4900/
<hoenir> ?
<hoenir> my PR fixes this https://bugs.launchpad.net/juju-core/+bug/1585430
<mup> Bug #1585430: Cloud-init failed on windows <ci> <cloud-init> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1585430>
<dimitern> voidspace: reviewed
<natefinch> katco: +1 on the comment about aliases. Feels like two people disagreed so we decided to just do both
<katco> natefinch: yeah... death by options
<frobware> dimitern: please could you take a look at http://reviews.vapour.ws/r/4903/
<frobware> dimitern: we looked at this a while ago, was just getting back to it
<dimitern> frobware: sure, looking
<natefinch> fwereade: when you change a model config in a running model, what executes those changes?
<fwereade> natefinch, depends -- what changes are you thinking of? substrate config will be watched for and applied by several things that have their own environs
<fwereade> natefinch, different bits of config will be watched for by other things, e.g. logging-config
<natefinch> fwereade: working on the syslog forwarding stuff... didn't know if there was a generic watcher for it, or if I should write a new one for the syslog stuff explicitly
<fwereade> natefinch, you need to watch model config under the hood whatever you do
<fwereade> natefinch, so at least it's easy to create the watcher
<fwereade> natefinch, just don't let on what it's really watching, call it WatchRsyslogConfig or something
<natefinch> fwereade: yep, ok
<fwereade> natefinch, (note: do not attempt to filter out changes that don't apply to your fields: it is not possible to do so reliably)
<natefinch> fwereade:  good to know.
<natefinch> ahh, so there's ModelWatcher.WatchForModelConfigChanges
<fwereade> natefinch, yeah, that's the one
<fwereade> natefinch, ...although, looking at it, there's a bit of WTF there
<mup> Bug #1585674 opened: "Waiting for agent initialization to finish" needs to be more detailed. <juju-core:New> <https://launchpad.net/bugs/1585674>
<frobware> dooferlad, voidspace: would appreciate a review of http://reviews.vapour.ws/r/4903/
<fwereade> natefinch, non-bulk, doesn't identify model, returns an error indicating OMG INFRASTRUCTURE BORKED instead of "this request didn't work"
<mup> Bug #1585674 changed: "Waiting for agent initialization to finish" needs to be more detailed. <juju-core:New> <https://launchpad.net/bugs/1585674>
<mup> Bug #1585674 opened: "Waiting for agent initialization to finish" needs to be more detailed. <juju-core:New> <https://launchpad.net/bugs/1585674>
<mup> Bug #1585674 changed: "Waiting for agent initialization to finish" needs to be more detailed. <juju-core:New> <https://launchpad.net/bugs/1585674>
<mup> Bug #1585674 opened: "Waiting for agent initialization to finish" needs to be more detailed. <juju-core:New> <https://launchpad.net/bugs/1585674>
<voidspace> dimitern: ping
<katco> natefinch: lmk when you're done with the 1-pager; need to send an email out about the bug
<natefinch> katco: ok, working on it now
<dimitern> voidspace: pong
<katco> natefinch: ta
<voidspace> dimitern: you suggest abstracting newSubnetFromArgs out from addSubnetOps
<voidspace> dimitern: but addSubnetOps still needs to construct a subnetDoc - so that at least would be duplicated
<voidspace> dimitern: not a big deal, but that's why I didn't do it
<voidspace> dimitern: if you still think it's worth it I'll make the change
<dimitern> voidspace: sorry, otp - will get back to you shortly
<dimitern> voidspace: well, addSubnetOps can take a prepopulated subnetDoc
<dimitern> voidspace: it doesn't have to also create it
<voidspace> dimitern: still leaving creation and validation of the Subnet to be duplicated, plus the subnetDoc is a bit of an "internal type" so it feels a bit yucky
<voidspace> I'll duplicate creation of the subnetDoc
<dimitern> voidspace: I'll leave it up to you
<bogdanteleaga> natefinch, you could look at the retrystrategy stack, it's essentially watching a model config variable and acting on changes
<alexisb> can someone please review this commit: https://github.com/juju/juju/pull/5457
<alexisb> katco, ^^
<katco> alexisb: tal
<alexisb> btw perrito666, happy bday!
<katco> perrito666: happy birthday :) hope you're feeling better
<alexisb> thanks katco
<natefinch> bogdanteleaga: awesome, thanks
<katco> alexisb: looks like that already has a review: http://reviews.vapour.ws/r/4900/
<alexisb> does the code author use review board?
<katco> alexisb: i don't know, but the project does :) the link is automatically injected into the PR. i'll leave a note asking them to review the review
<alexisb> katco, thank you
<bogdanteleaga> alexisb, katco, yes he does, he recently joined our juju team :)
<alexisb> bogdanteleaga, aaaah, ok that makes sense
<katco> bogdanteleaga: grats on the add :)
<mbruzek> katco: https://bugs.launchpad.net/juju-core/+bug/1585701
<mup> Bug #1585701: juju resources needs to support extensions other than ".tgz" <juju-core:New> <https://launchpad.net/bugs/1585701>
<katco> mbruzek: tal
<katco> mbruzek: replied on bug
<babbageclunk> perrito666: ping?
<bogdanteleaga> katco, cheers :)
<babbageclunk> bogdanteleaga: I've reviewed it as well (although I'm technically only a junior reviewer).
<mbruzek> katco: Thanks for pointing that out. My bad.
<katco> mbruzek: no worries; new feature
<bogdanteleaga> babbageclunk, you've basically expressed my exact thoughts on it :P
<katco> mbruzek: fwiw we opposed checking the file extension because it's just a blob of data
<bogdanteleaga> I guess we should start reviewing on rbt as well
<katco> bogdanteleaga: i didn't see the need at first, but after using it for awhile it's hard when i have to review something on GH
<babbageclunk> bogdanteleaga: ok great - I thought maybe I was missing something.
<natefinch> sinzui: what's with the multiple posts from the bot on this PR? https://github.com/juju/juju/pull/5419
<natefinch> sinzui: (at the bottom it says merge request accepted 4 times in response to a single $$merge$$_
<sinzui> natefinch: I have never seen that
<sinzui> and I am looking at it now
<natefinch> sinzui: k, just figured I'd bring it to your attention in case it indicates a problem
<katco> natefinch: ding! your hour on the 1-pager is up :) are you about wrapped up?
<natefinch> katco: forgot and got lunch in the middle, but still almost done. It's actually fairly simple.  5 minutes.
<katco> natefinch: kk
<voidspace> dimitern: why does the subnetDoc have an IsPublic field?
<natefinch> katco: last two pages here: https://docs.google.com/document/d/1x60GL9zckzXNfHw_yyY_syxF1WSL5JkcWoWKJtwT_Ww/edit#
<dimitern> voidspace: because subnets can be public or private (i.e. with and without shadow addresses support)
<dimitern> (by spec, not as implemented currently except by mocking)
<katco> natefinch: ta; can you send an email to ian and cc eric, me stating as such?
<natefinch> katco: sure thing
<katco> natefinch: and then you're rolling back onto the bug now?
<natefinch> katco: yep
<katco> natefinch: cool, reassign yourself plx :)
<voidspace> dimitern: then why is that field not *used*
<voidspace> dimitern: it's not on SubnetInfo
<voidspace> dimitern: and it's not exposed on Subnet
<voidspace> dimitern: so it's never set or changed
<dimitern> voidspace: because we haven't quite got there yet
<voidspace> dimitern: unused code for imaginery future use cases... :-p
<dimitern> voidspace: not imaginary, just not done yet
<voidspace> dimitern: I've added it as an ignored field in the migration_internal_test - when we do start using it we'll have to remember to migrate it
<dimitern> voidspace: if you read the network model spec, private/public subnets and spaces are part of it
<dimitern> voidspace: sounds good, +1
<voidspace> dimitern: that doesn't change my opinion - specs are always imaginery until they're actually implemented
<dimitern> voidspace: of course - I'd leave a TODO on the field at least - to keep it in mind; there's even a bug for it
<voidspace> dimitern: good idea, adding comment
<dimitern> voidspace: thanks!
<rogpeppe> a simple addition to the juju/cmd package. anyone up for a little review? https://github.com/juju/cmd/pull/37
<rogpeppe> voidspace, dimitern: ^
<rogpeppe> natefinch: ^
<rogpeppe> fwereade: ^
<dimitern> rogpeppe: LGTM
<rogpeppe> dimitern: you angel, thanks!
<natefinch> rogpeppe: I'm sort of surprised that this wouldn't just be something we add to the help command itself
<rogpeppe> natefinch: how would we do that?
<rogpeppe> natefinch: the help command is built into SuperCommand
<rogpeppe> natefinch: FWIW we considered about 3 dozen options here :)
<rogpeppe> natefinch: this was the simplest and best tailored to what we actually needed
<natefinch> rogpeppe: oh yeah, I forgot the help command is built in....
<rogpeppe> natefinch: this is what it enabled: https://github.com/juju/charmstore-client/pull/66
<natefinch> rogpeppe: very nice
<rogpeppe> natefinch: yup, always happy to delete code :)
<redir> reboot brb
<mup> Bug #1585750 opened: Destroying a service in error gives no feedback <juju-core:New> <https://launchpad.net/bugs/1585750>
<katco> natefinch-afk: you never reassigned yourself to bug 1537585 :(
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1537585>
<axw> rogpeppe: not sure why help plugins was removed.
<axw> nor the rest really
<cherylj> axw: the help topics were removed to avoid having to keep two places up-to-date: (online docs and cli help)
<axw> cherylj: ok, fair enough. plugins tho? that might have been accidental?
<cherylj> oooh, yeah I forgot that help plugins wasn't just a topic
<cherylj> yeah, that might've been an oversight
<wallyworld> axw: anastasiamac: have standup without me today, in another meeting
<axw> ok
<axw> anastasiamac: standup?
<axw> perrito666: ^
<mup> Bug #1585825 opened: Takes too long to download a resource from a controller to unit <ci> <resources> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1585825>
#juju-dev 2016-05-26
<davechen1y> ping alexisb
<alexisb> heya davechen1y
<alexisb> whats up?
<davechen1y> alexisb: i just sent you an email that requires acknowledgement
<davechen1y> it's very short, could you please read and respond
<davechen1y> thanks
<alexisb> davechen1y, will do
<davechen1y> thanks
<alexisb> davechen1y, responded
<davechen1y> alexisb: ta
<davechen1y> ok, next question
<davechen1y> JFDI or not JFDI
<davechen1y> i have a PR that is at the bottom of a lot of blocked bugs for beta8
<davechen1y> can I JFDI ?
<anastasiamac_> davechen1y: is it blocker? u can't just use $$fixes-blah$$?
<davechen1y> anastasiamac_: i don't know
<davechen1y> the issue i am assigned is not the one that is blocking landing
<anastasiamac_> ic. is it the race fix?
<davechen1y> yes
<davechen1y> well
<davechen1y> maybe
<davechen1y> this is not the issue that CI is blocked o
<davechen1y> on
<anastasiamac_> so u r right, plz do not JFDI unless u have an explicit "please land" from alexisb or cherylj or sinzui
<davechen1y> ok, i've marked my PR as blcoked on the landing bot and i'll work on something else
<anastasiamac_> \o/
<davechen1y> how can I find out when this build was run http://reports.vapour.ws/releases/3993/job/run-unit-tests-race/attempt/1495
<davechen1y> this is the last race build that was run
<davechen1y> i require the CI rae infrastructure to validate the fixes I am landing
<davechen1y> is the ci race build running currently ?
<sinzui> davechen1y: follow the link to the build number in the top nav. http://reports.vapour.ws/releases/3993  Wed May 25 08:59:46 2016
<davechen1y> i get a 404
<davechen1y> when I follow that link
<davechen1y> sorry, i get a 404 when I follow the link to the jenkins job
<sinzui> davechen1y: are you no longer a member oc canonical?
<davechen1y> sinzui: one can only hope
<sinzui> davechen1y: jenkins jobs only last a few days. All the data is on the reports site
<davechen1y> sinzui: so does that mean the race job has not run for at least a few days ?
<davechen1y> run-unit-tests-race1495Bug #1579051
<mup> Bug #1579051: Race in juju/controller/destroy and TestDestroyCommandConfirmation <blocker> <ci> <intermittent-failure> <race-condition> <regression> <juju-core:Fix Released by thumper> <https://launchpad.net/bugs/1579051>
<davechen1y> ^ this bug that the race build is complaining about is marked fixed release
<sinzui> davechen1y: no. the job is run with every revision
<davechen1y> sinzui: that doesn't make sense
<davechen1y> if run-unit-tests-race 1495 was run recently, as in since last night then the jenkins job would be 404
<davechen1y> wouldn't be 404
<sinzui> davechen1y: if you cannot seen any the job at all, then you need to login. 90% of the site is not public because juju likes to send my credentials to logs.
<davechen1y> sinzui: i dont' have a login for that site
<davechen1y> or chrome has forgotten them :(
<mup> Bug #1585836 opened: Race in github.com/juju/juju/provider/azure <azure-provider> <ci> <race-condition> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1585836>
<mup> Bug #1585836 changed: Race in github.com/juju/juju/provider/azure <azure-provider> <ci> <race-condition> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1585836>
<mup> Bug #1582408 changed: System id is listed during bootstrap and deploy of charms instead of system name <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1582408>
<mup> Bug #1585836 opened: Race in github.com/juju/juju/provider/azure <azure-provider> <ci> <race-condition> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1585836>
<wallyworld> axw: a small one when you are free http://reviews.vapour.ws/r/4905/
<mup> Bug #1582408 opened: System id is listed during bootstrap and deploy of charms instead of system name <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1582408>
<axw> wallyworld: what changed the error message change?
<wallyworld> axw: new dep i suspect
<wallyworld> httprequest maybe
<axw> wallyworld: that function just uses net/http AFAICS
<wallyworld> axw: yeah, i have no idea why that message changed then, i can look into it i guess, i hust assumed it was due to one of the new deps
<axw> wallyworld: please do, doesn't look related to me
<wallyworld> ok, otp will do soon
 * redir goes eod
<mup> Bug #1582408 changed: System id is listed during bootstrap and deploy of charms instead of system name <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1582408>
<axw> wallyworld: I see you reverted, did you get to the bottom of it?
<wallyworld> axw: nope, master failed in the same way, nfi
<mup> Bug #1585846 opened: worker/terminationworker: test timeout after 20 minutes <juju-core:New> <https://launchpad.net/bugs/1585846>
<mup> Bug #1585847 opened: LXD creates all the interfaces as the physical machine has when using MAAS. <juju-core:New> <https://launchpad.net/bugs/1585847>
<wallyworld> axw: you happy to +1 with that error message reverted?
<axw> wallyworld: yep, will do
<wallyworld> ta
<axw> wallyworld: done
<wallyworld> ty
<mup> Bug #1585849 opened: Azure native bundle 404 creating resources <azure-provider> <bundle> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585849>
<mup> Bug #1585851 opened: Cannot restore-backup of controller model <backup-restore> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585851>
<mup> Bug #1585856 opened: worker/txnpruner: test failure during ci <juju-core:New> <https://launchpad.net/bugs/1585856>
<mup> Bug #1585857 opened: Unable to bootstrap to maas 2.0 cloud with juju 2.0 beta 7 <juju-core:New> <https://launchpad.net/bugs/1585857>
<wallyworld> axw: as per roger's suggestion, i added extra tests and improved the marshalling https://github.com/juju/charm/pull/210
<axw> wallyworld: LGTM
<wallyworld> ty
<wallyworld> axw: small one, https://github.com/juju/idmclient/pull/17
<wallyworld> there's sooooo many :-(
<wallyworld> axw: idmclient just uses UserTag so v1 and v2 are the same in that respect
<axw> wallyworld: I was thinking more that names types might exposed in the idmclient API, but it seems not
<axw> wallyworld: shipit
<wallyworld> ta
<mup> Bug #1585878 opened: It does not remove the DHCP and DNS configurations on MAAS after juju removes LXD container(machine). <juju-core:New> <https://launchpad.net/bugs/1585878>
<mup> Bug #1585878 changed: It does not remove the DHCP and DNS configurations on MAAS after juju removes LXD container(machine). <MAAS:New> <https://launchpad.net/bugs/1585878>
<mup> Bug #1585878 opened: It does not remove the DHCP and DNS configurations on MAAS after juju removes LXD container(machine). <MAAS:New> <https://launchpad.net/bugs/1585878>
<wallyworld> axw: do you have time to talk?
<axw> wallyworld: yes, but gotta go in ~15m to pick up charlotte
<wallyworld> ok, 1:1
<wallyworld> axw: google hates me, be there in a sec
<mup> Bug #1585878 changed: It does not remove the DHCP and DNS configurations on MAAS after juju removes LXD container(machine). <MAAS:New> <https://launchpad.net/bugs/1585878>
<mup> Bug #1585878 opened: Removing a container does not remove the underlying MAAS device representing the container unless the host is also removed. <juju-core:Triaged> <https://launchpad.net/bugs/1585878>
<babbageclunk> frobware: 1:1?
<frobware> babbageclunk: yep, sent link via PM
<frobware> dimitern, dooferlad: standup
<dooferlad> frobware: just booking some passport stuff. There soon.
<frobware> dooferlad: k
<hoenir> http://reviews.vapour.ws/r/4900/ can anyone have another look on this PR? it fixes this https://bugs.launchpad.net/juju-core/+bug/1585430
<mup> Bug #1585430: Cloud-init failed on windows <ci> <cloudbase-init> <format> <regression> <windows> <juju-core:In Progress by sgiulitti> <https://launchpad.net/bugs/1585430>
<voidspace> fwereade: ping
<fwereade> voidspace, pong
<mup> Bug #1585857 changed: Unable to add credential to maas cloud with juju 2.0 beta 7 <hs-arm64> <juju-core:New> <https://launchpad.net/bugs/1585857>
<mup> Bug #1586007 opened: cannot ssh to LXD container on non MAAS systems using juju ssh --proxy 0/lxd/0 <juju-core:New> <https://launchpad.net/bugs/1586007>
<voidspace> fwereade: philosophy question about model migration
<voidspace> fwereade: will we assume data integrity - i.e. referred to entities exist - or should we import entities in the right order (any circular references will be a problem) and verify during import
<voidspace> fwereade: i.e. addresses reference devices (amongst other things) - must we import the devices first and check when we import the addresses
<voidspace> fwereade: or is it ok to assume data integrity (the device must have existed for the address to be created)
<voidspace> fwereade: without checks the new model will only be as screwed as the old model
<voidspace> fwereade: with checks we may refuse to import at all
<fwereade> voidspace, my understanding is that we validate the model both source-side and dest-side
<voidspace> fwereade: so we need to ensure we do the imports in the right order
<fwereade> voidspace, if either side fails we can't migrate but we can at least unfreeze the model in the source controller and keep it going
<voidspace> fwereade: and to do the checks the imports need to be implemented in the right order
<fwereade> voidspace, I'm not sure we do? the interface seems to me like it's tuned for just setting lots of things and then validating the whole lot at the end
<fwereade> voidspace, so source validates before sending; target validates before applying; failure at any stage, including application, should roll back the whole thing
<fwereade> voidspace, we only drop the model from the source when everything's confirmed it's happily talking to the target controller
<voidspace> fwereade: certainly a validation step could be added
<voidspace> fwereade: I don't *see* it here though: https://docs.google.com/document/d/1HVWgLxV3CJWI-IAzb2NvFOxhkx4Q3qsePxQ9-F2ZYIE/edit#
<fwereade> voidspace, core/description.Model has a Validate method
<voidspace> fwereade: ah, indeed it does
<fwereade> voidspace, I don't think of validation as a separate step so much as something you Just Do when data crosses a boundary
<fwereade> voidspace, definitely on the way in, plausibly on the way out as well at times
<voidspace> fwereade: the examples I was following for implementation didn't need any validation so didn't include anything here
<voidspace> fwereade: ok, so some additional work needed here
<fwereade> voidspace, I'm not really familiar with the nuts and bolts of the validation, I'm afraid
<voidspace> fwereade: cool, you answered my question anyway
<voidspace> fwereade: appreciated
<frobware> cherylj: ok to merge http://reviews.vapour.ws/r/4903/ ?
<frobware> dooferlad: ping
<dooferlad> frobware: pong
<frobware> dooferlad: having trouble with your bridge script changes - can we HO?
<dooferlad> frobware: sure
<dooferlad> frobware: juju-sapphire?
<frobware> yep
<babbageclunk> voidspace: ping?
<mup> Bug #1518806 changed: apiserver: tests to not pass with -race under Go 1.2 <2.0-count> <juju-core:Fix Released by dave-cheney> <https://launchpad.net/bugs/1518806>
<mup> Bug #1519133 changed: cmd/jujud/agent: data race <2.0-count> <juju-core:Fix Released by dave-cheney> <https://launchpad.net/bugs/1519133>
<mup> Bug #1519191 changed: worker/addresser: FAIL: worker_test.go:260: workerEnabledSuite.TestWorkerAcceptsBrokenRelease <2.0-count> <juju-core:Fix Released by dave-cheney> <https://launchpad.net/bugs/1519191>
<cherylj> frobware: yes, go ahead and merge
<babbageclunk> dimitern: ping?
<dimitern> babbageclunk: pong
<babbageclunk> dimitern: I'm trying to debug a flaky test I'm hitting.
<babbageclunk> It's this: https://github.com/juju/juju/blob/master/state/subnets_test.go#L424
<babbageclunk> dimitern: (asking you because voidspace didn't respond)
<dimitern> babbageclunk: let me have a look..
<dimitern> babbageclunk: I don't think it's flaky
<babbageclunk> dimitern: It looks like the patch there isn't working - the code doesn't use the patched name, so it's calling the original function and just happening to pass.
<babbageclunk> dimitern: It is on 3.2 with my changes ;)
<dimitern> babbageclunk: yeah, the way PatchValue is used might cause it to be though..
<babbageclunk> dimitern: which might be my fault, but I'm trying to understand what's happening.
<dimitern> babbageclunk: have a look at the internals of PatchValue - there are 2 slices of patched stuff - suite-level and test-level
<babbageclunk> dimitern: But in the code https://github.com/juju/juju/blob/master/state/subnets.go#L330 it uses the lowercase name, so it looks like it always uses the unpatched function
<babbageclunk> (Also I put in some logging and verified that.)
<dimitern> babbageclunk: oh ..
<dimitern> babbageclunk: the real pickAddress is non-deterministic
<dimitern> babbageclunk: as it picks random addresses from the subnet range
<babbageclunk> dimitern: So I think the test has been passing by accident.
<dimitern> babbageclunk: I suspect the way it's patched is wrong
<dimitern> babbageclunk: e.g. I'd expect to see something like s.PatchValue(&state.Foo, func(matching-args-decls) { ... })
<babbageclunk> dimitern: Well, the test is in a different package, so I don't think it can patch the lowercase name.
<dimitern> babbageclunk: i.e. &mockPickAddress looks wrong
<dimitern> babbageclunk: depends how state.PickAddress is exported
<babbageclunk> Ok - so you think it's the & that's wrong, rather than the naming?
<dimitern> babbageclunk: for funcs, it's quite common to use var Foo = &foo in export_test, then s.PatchValue(pkg.Foo, func (...) { .. })
<dimitern> babbageclunk: naming is fine - it looks weird, but because of export_test.go (btw such things cannot be followed with godef as well)
<dimitern> babbageclunk: OTOH, you have var Foo = foo in export_test of package pkg, &pkg.Foo in pkg_test points to the pkg.Foo var in export_test, NOT to the pkg.foo
<lazyPower> ping natefinch
<babbageclunk> dimitern: Ok, I think I follow - I'll try changing export_test to make it work.
<dimitern> babbageclunk: that is important for patching functions, not so much for simple vars (e.g. exposing struct foo{unexp. fields..})
<dimitern> babbageclunk: +1
<mup> Bug #1580186 changed: Race in github.com/juju/juju/worker/instancepoller/aggregate_test.go <ci> <intermittent-failure> <race-condition> <regression> <unit-tests> <juju-core:Fix Released by reedobrien> <https://launchpad.net/bugs/1580186>
<mup> Bug #1585430 changed: Cloud-init failed on windows <blocker> <ci> <cloudbase-init> <format> <regression> <windows> <juju-core:Fix Released by sgiulitti> <https://launchpad.net/bugs/1585430>
<mup> Bug #1585856 changed: worker/txnpruner: test failure during ci <juju-core:New> <https://launchpad.net/bugs/1585856>
<mup> Bug #1586057 opened: Race in github.com/juju/juju/container/lxc <ci> <lxc> <race-condition> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1586057>
<frobware> dimitern, jam: were we going to talk about fan/networking today?
<dimitern> frobware: we still can :)
<voidspace> babbageclunk: sorry, was on lunch
<babbageclunk> voidspace: No worries. Picked dimitern's brains instead, although I'm still a bit stuck
<voidspace> babbageclunk: yeah, read the backscroll
<babbageclunk> voidspace: can you see what's wrong with the patching? Is it possible to do that from outside the package if the code's calling the lowercase name?
<voidspace> babbageclunk: no, the code should call the name that's being patched
<babbageclunk> So maybe I should change subnets.go to assign PickAddress (and call it) and  remove the alias from export_test?
<voidspace> babbageclunk: the alias is needed to allow patching
<voidspace> babbageclunk: without an alias it can't be patched
<voidspace> babbageclunk: are you suggesting removing the patching (which doesn't do anything) as well?
<babbageclunk> voidspace: I thought the alias was to make it visible from outside the package.
<voidspace> babbageclunk: you can't patch a function directly - however if you create a pointer alias you can patch *that*
<babbageclunk> voidspace: no - the test needs the patch, otherwise it randomly fails.
<voidspace> babbageclunk: so just change the code to call the upper case version (the patched one)
<babbageclunk> voidspace: pickAddress is defined as var pickAddress = func in subnets.go.
<babbageclunk> voidspace: But the uppercase one only exists in exports_test.go
<voidspace> babbageclunk: hmm...
<babbageclunk> voidspace: (sorry, export_test.go)
<voidspace> babbageclunk: so an uppercase alias in export_test is usually for testing an unexported function
<voidspace> babbageclunk: and a function created as var *can* be patched, but if it's lower case then only from within the package - not from a black box test
<mup> Bug #1585430 opened: Cloud-init failed on windows <blocker> <ci> <cloudbase-init> <format> <regression> <windows> <juju-core:Fix Committed by sgiulitti> <https://launchpad.net/bugs/1585430>
<babbageclunk> voidspace: So I couldn't rebind a function defined as an uppercase var from outside the package? (As I type that it sounds like a bad idea even if possible.)
<babbageclunk> voidspace: turns out that *does* work - the patched function now gets called. But it still sometimes fail for reasons I've not yet worked out.
<voidspace> babbageclunk: ok, that sounds like the right thing to do for this test
<voidspace> babbageclunk: now work out why it fails and fix it :-)
<babbageclunk> voidspace: Maybe I should have done that in the first place. :)
<voidspace> babbageclunk: hah
<babbageclunk> voidspace: The first thing I did was get confused about why my added logging wasn't showing up.
<voidspace> babbageclunk: why wasn't it?
<babbageclunk> voidspace: because I added it to mockPickAddress, which wasn't being called.
<voidspace> ah!
<cmars> hey, is there a mongodb 3.2 in the ubuntu archives? i noticed that we're using it for the controller now.. is there a ppa where i can apt install it from too?
<frobware> cherylj: fyi - https://bugs.launchpad.net/maas/+bug/1586075
<mup> Bug #1586075: Deploying a trusty node on MAAS 1.9.3 gives us 2 entries for eth0 in /e/n/i <juju> <MAAS:New> <https://launchpad.net/bugs/1586075>
 * katco needs to reboot
<alexisb> frobware, dimitern, just sent a note, take a look and let me know
<frobware> alexisb: looking
<dimitern> alexisb: sure
<dimitern> alexisb: replied
<alexisb> dimitern, thanks
<natefinch> fwereade: you around?
<mup> Bug #1586089 opened: azure arm instance-ids are not ids, cannot find machine in azure <azure-provider> <ci> <ha> <jujuqa> <testing> <juju-core:Triaged> <https://launchpad.net/bugs/1586089>
<babbageclunk> perrito666: ping?
<perrito666> babbageclunk: pong
<babbageclunk> perrito666: I'm just trying to finish up these mongo3.2 changes...
<babbageclunk> perrito666: but I've hit a couple of places where 3.2 behaves a bit oddly
<perrito666> babbageclunk: ok, tell me, how can I help you?
<babbageclunk> perrito666: (Happy birthday for yesterday by the way!)
<perrito666> babbageclunk: tx :)
<babbageclunk> perrito666: one of them is in OplogTailer
<perrito666> mh, I am not familiar with it
<babbageclunk> perrito666: Doh, I thought I saw your name on it, sorry!
<perrito666> babbageclunk: I might have passed by changing something
<perrito666> but I am pretty sure its one of menn0's
<babbageclunk> perrito666: yup, you're right
<perrito666> sorry that was not helpful :(
<babbageclunk> perrito666: Well, I'll bug him about this instead! Sorry!
<babbageclunk> perrito666: :)
<frobware> voidspace, dimitern, babbageclunk, dooferlad: anybody done $(juju add-machine lxd:0) on trusty recently, bootstrapping on MAAS?
<dimitern> frobware: not in a few days
<babbageclunk> frobware: nope, just mongo bashing
<frobware> dimitern: I don't seem to have backports enabled in trusty any more
<dimitern> frobware: did you update the images?
<frobware> https://bugs.launchpad.net/juju-core/+bug/1568895
<mup> Bug #1568895: Cannot add MAAS-based LXD containers in 2.0beta4 on trusty <ci> <jujuqa> <lxd> <maas-provider> <cloud-images:Fix Released> <juju-core:Fix Released> <https://launchpad.net/bugs/1568895>
<frobware> dimitern: my maas setup says "last update:	Thu, 26 May. 2016 17:45:03"
<frobware> dimitern: are you in a position to run a lxd container at all?
<frobware> babbageclunk, dooferlad, voidspace ^^
<dimitern> frobware: let me do a quick test locally..
<dimitern> frobware: uh oh.. it failed to deploy at all.. I was messing with LVM groups and volumes to try to simulate having 2 disks to deploy OS
<frobware> dimitern: so unrelated failures?
<dimitern> frobware: I'm trying again to verify..
<frobware> dimitern: ok, also trying on my laptop which still has maas-1.9.2
<frobware> dimitern: though I don't think that's the issue at all
<dimitern> frobware: ok, now it got deployed, adding lxd:0
<frobware> dimitern: in theory you shouldn't need juju
<frobware> dimitern: just deploy a node
<frobware> dimitern: you only need to verify the content of /e/apt/sources.list
<dimitern> frobware: no backports here
<frobware> dimitern: I think this is bust again. Let me update the bug. Could you do so too.
<dimitern> E: The value 'trusty-backports' is invalid for APT::Default-Release as such a release is not available in the sources
<frobware> dimitern: sigh. that was a waste of a few hours.
<frobware> dimitern: sometimes it feels hard to make progress. :(
<frobware> dimitern: I'm surprisd this is not showing up in CI tests
<frobware> sinzui: ^^
<dimitern> frobware: updated the bug (I assume you meant 1569985?)
<dimitern> oops not that - 1568895
<frobware> dimitern: yep
<sinzui> frobware. I think we have seen it., but is is not a current issue
<sinzui> frobware: bug 1556137 mentions it
<mup> Bug #1556137: Juju 2.0: cannot connect to LXC container when deploying bundle with LXC container placement with maas 1.9.1 as provider <2.0-count> <oil> <juju-core:Triaged> <MAAS:New> <https://launchpad.net/bugs/1556137>
<frobware> sinzui: ooh, I now why that one is broken
<mup> Bug #1585430 changed: Cloud-init failed on windows <blocker> <ci> <cloudbase-init> <format> <regression> <windows> <juju-core:Fix Released by sgiulitti> <https://launchpad.net/bugs/1585430>
<mup> Bug #1586116 opened: GCE bootstraps but fails to provision <gce-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1586116>
<frobware> sinzui: ah, sorry, nope. I was actually thinking of: https://bugs.launchpad.net/juju-core/+bug/1576674/  Sorry for the noise
<mup> Bug #1576674: 2.0 beta6: only able to access LXD containers (on maas deployed host) from the maas network <lxd> <maas-provider> <oil> <ssh> <juju-core:Triaged> <https://launchpad.net/bugs/1576674>
<dimitern> frobware: if you're still about for a short while, I'm finally proposing a fix for the IPv6 bug
<frobware> dimitern: I can be
<dimitern> (take 3, totally rethought and as simple as possible)
<frobware> dimitern: let's stop at 11. :)
<dimitern> frobware: good plan :)
<dimitern> frobware:http://reviews.vapour.ws/r/4915/
<frobware> dimitern: looking but will mostly try first.
<dimitern> frobware: +1
<frobware> dimitern:  for a repro, just deploy 20 units?
<dimitern> frobware: yeah - less than 20 should still work
<dimitern> frobware: ah! important: make sure your lxdbr0 is configured to give *both* ipv4 and ipv6 addresses
<frobware> sinzui: what's the correct status for reopening a bug? mark as "confirmed" again? Just want to ensure bug #1568895 get some other eyes verifying current status
<mup> Bug #1568895: Cannot add MAAS-based LXD containers in 2.0beta4 on trusty <ci> <jujuqa> <lxd> <maas-provider> <cloud-images:Fix Released> <juju-core:Fix Released> <https://launchpad.net/bugs/1568895>
<frobware> dimitern: does it matter which series I should use?
<sinzui> frobware: if you have the power, mark it triaged, otherwise confirmed
<sinzui> frobware: it does matter if the issue is not in master
<frobware> sinzui: seems more likely to be outside of juju
<dimitern> frobware: I used xenial, but shouldn't matter really
<dimitern> frobware: actually a couple of days I also tested with trusty (the earlier fix)
<frobware> dimitern: but not with containers
<sinzui> frobware: about trusty-backports. Old images do not have them enabled. backports were enabled by default in xenial images. trusty imagages were updated and released to the clouds a few months ago
<frobware> sinzui: but I did go through a stage where a trusty MAAS node had backports enabled.
<sinzui> frobware: I expect newish trusty images to have trusty-backports enabled. The maas images are made by the team that make cloud-images
<frobware> sinzui: but wasn't this previously failing in CI, then confirmed as working once the images had been updated.
<perrito666> bbl
 * natefinch is a team of one, evidently.
<redir> go team nate
<redir> go team natefinch even
<natefinch> :)
<redir> rah rah ree kick em in the knee
<mup> Bug #1568895 opened: Cannot add MAAS-based LXD containers in 2.0beta4 on trusty <ci> <jujuqa> <lxd> <maas-provider> <cloud-images:Confirmed> <juju-core:Confirmed> <https://launchpad.net/bugs/1568895>
<natefinch> I guess we're not having the team meeting today?  Now that we only have managers that are asleep at this time, having a team meeting seems silly
<redir> bbiab
<natefinch> alexisb:
<mup> Bug #1586150 opened: destroy-model does not remove the model <cdo-qa> <destroy-model> <juju-core:New> <https://launchpad.net/bugs/1586150>
<mup> Bug #1586150 changed: destroy-model does not remove the model <cdo-qa> <destroy-model> <juju-core:New> <https://launchpad.net/bugs/1586150>
<mup> Bug #1586150 opened: destroy-model does not remove the model <cdo-qa> <destroy-model> <juju-core:New> <https://launchpad.net/bugs/1586150>
<mup> Bug #1586150 changed: destroy-model does not remove the model <cdo-qa> <destroy-model> <juju-core:New> <https://launchpad.net/bugs/1586150>
<arosales> natefinch: katco: jsut as a heads up, if you have some time could you take a look at https://github.com/juju/docs/pull/1122
<arosales> we need to get terms docs to users to so they can start prepping their charms for GA
<arosales> 2.0 ga, that is
<arosales> mbruzek: ^
<rick_h_> arosales: ty, wil try to hook up cmars and his team with the docs folks as they did terms
<cmars> rick_h_, arosales https://github.com/juju/docs/blob/master/src/en/developer-terms.md
<rick_h_> cmars: <3
<arosales> cmars: :-) so those land and I already referenced them on the list :-)
<arosales> s/so/saw/
<arosales> sorry earlier I meant to say resources we need a review on
<arosales> per https://github.com/juju/docs/pull/1122
<arosales> I think mbruzek was looking for a final ack from katco and natefinch
<arosales> for resources on terms
<cherylj> alexisb: release standup?
<cherylj> tych0: ping?
<mup> Bug #1585851 changed: Cannot restore-backup of controller model <backup-restore> <blocker> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585851>
<babbageclunk> hey menn0!
<menn0> babbageclunk: hey hey!
<babbageclunk> menn0: happy birthday for yesterday!
<babbageclunk> menn0: Have you got a moment for a quick q?
<menn0> babbageclunk: cheers
<menn0> babbageclunk: sure
<babbageclunk> menn0: I'm trying to get the full test suite passing with Mongo 3.2...
<babbageclunk> menn0: But there are a couple of places where we get different errors now than we used to.
<babbageclunk> menn0: *sometimes*
<babbageclunk> menn0: So they make the tests flaky.
<menn0> babbageclunk: that sounds fun :-/
<babbageclunk> menn0: Well, it's a bit suboptimal.
<menn0> babbageclunk: do you mean that for the same root cause mongodb returns different errors at times?
<babbageclunk> menn0: Yeah - for example:
<babbageclunk> https://github.com/babbageclunk/juju/commit/61da79e198d15263579f717da21e0e5925da0f3b
<babbageclunk> menn0: Instead of a nice mgo.ErrCursor, we sometimes get a generic mgo.QueryError
<menn0> babbageclunk: ah right... I even know about that one. https://bugs.launchpad.net/juju-core/+bug/1573286
<mup> Bug #1573286: juju/mongo: oplog tests fail with mongod 3.2 <mongodb> <unit-tests> <juju-core:Triaged by menno.smits> <https://launchpad.net/bugs/1573286>
<babbageclunk> menn0: Ha!
<babbageclunk> menn0: Ok, hadn't seen that.
<menn0> babbageclunk: mwhudson created that ticket when he first ran into this
<menn0> and told me about it
<alexisb> wallyworld, menn0, cherylj, sinzui, rick_h_, I need some of your time i fyou are available
<menn0> babbageclunk: I haven't put any thought into it yet
<menn0> alexisb: i'm around
<wallyworld> alexisb: ok
<alexisb> https://plus.google.com/hangouts/_/canonical.com/juju-release
<mwhudson> babbageclunk: wow, that general task is one hell of a welcome to the job
<alexisb> mwhudson, yes yes it is
<menn0> babbageclunk: let me have a think about that particular issue ... have to jump on this call
<menn0> alexisb, mwhudson: luckily babbageclunk is a smart cookie :)
<babbageclunk> menn0: Yeah - I'm not sure whether my fix is good - it's essentially only happpening during the tests, so it seems odd to handle it in the prod code.
<babbageclunk> menn0: Ok, talk later.
<mwhudson> menn0, alexisb: that's ok then :-)
 * babbageclunk blushes.
<babbageclunk> mwhudson: It seemed ok at first!
<mwhudson> babbageclunk: that's a good sign, i guess :)
<babbageclunk> mwhudson: I feel like it's *just this last bit*, but I've thought that for about 4 days now. :)
<mwhudson> babbageclunk: heh
<mwhudson> babbageclunk: did you beat the state tests into submission?
<mwhudson> that seemed like the worst bit
<babbageclunk> mwhudson: Yeah, that 100x slowdown is crazy!
<babbageclunk> mwhudson: I mean, I've hacked around it, mostly (although they still run slower than with 2.4).
<mwhudson> babbageclunk: oh yes, i saw something about that, emptying the db rather than recreating it?
<babbageclunk> mwhudson: yuo
<babbageclunk> uh, yup
<mwhudson> babbageclunk: i also saw something running out of file descriptors sometimes, you've not hit that?
<babbageclunk> mwhudson: No...
<mwhudson> well good, i guess :)
<babbageclunk> mwhudson: I saw your mention of it, but it's not happened to me.
<babbageclunk> mwhudson: good unless it starts happening in builds I guess.
<mwhudson> heh yes
<babbageclunk> mwhudson: Right, I have to go sleep. Nice to meet you!
<mwhudson> babbageclunk: good night!
<mup> Bug #1585851 opened: Cannot restore-backup of controller model <backup-restore> <blocker> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1585851>
<davechen1y> ci builds are failing because gopkg.in is unwell
<anastasiamac_> davechen1y: see #juju
<anastasiamac_> davechen1y: sinzui is onto it \o/ but landing is paused for now...
<menn0> babbageclunk: i'm off the call now... what was your fix for the oplog test problem?
<babbageclunk> menn0: I just in some code to check for the specific QueryError I was seeing, which is hacky but ok (I think).
<babbageclunk> menn0: It's really the other one that's giving me pause - sending you an email 'cause I really need to go to bed.
<babbageclunk> menn0: It turns out toddlers don't stay in bed later just because you got another baby!
<menn0> babbageclunk: ok no problems. sleep is good :) i'll take a look at this today
<babbageclunk> menn0: Who knew?
<menn0> babbageclunk: yep :) Sander gets up and into our bed about 5:30am every day. Amelia sleeps in most days at least
<babbageclunk> menn0: Say hi to the family! Night.
<menn0> babbageclunk: goodnight!
<mup> Bug #1586197 opened: Upgrade to 1.25.5 not possible, due to cannot store tools metadata: invalid binary version \"1.25.5--amd64\" <sts-needs-review> <juju-core:New> <https://launchpad.net/bugs/1586197>
#juju-dev 2016-05-27
<axw> wallyworld: https://github.com/juju/juju/pull/5462 doesn't actually have a bug, should I JFDI or leave it?
<wallyworld> axw: um, create a bug i think
<axw> ok
<mup> Bug #1586217 opened: azure: bootstrapping prints out scary, spurious ERROR messages <blocker> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1586217>
<wallyworld> axw: when you are free, a small one, then i can propose the charmrepo one https://github.com/juju/charm/pull/211
<redir> is there prior art for parsing cmd line flags? e.g. timestamps?
<wallyworld> axw: i tried a high level test but couldn't get the bson marshalling behaviour to work. i couldn't figure out the magic to get a nil result from unmarshal instead of an empty struct (or a marshalling error) so figured a test is better than nothing
<wallyworld> i'll take another look
<axw> wallyworld: marshal a struct with a field of type BundleData, with omitempty on the field?
<wallyworld> yeah, that may work
<mup> Bug #1581748 changed: [2.0b7] add-credential no longer supports maas as a cloud type <cdo-qa> <credentials> <maas> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1581748>
<axw> davechen1y: please see my reply on http://reviews.vapour.ws/r/4911/
 * davechen1y looks
<axw> davechen1y: the race was between multiple writers, not writers&readers
<axw> the lock was added to stop the multiple writers from stepping on each other
<davechen1y> axw: that doesn't matter, all readers and writers have to agree on the same mutex, so there is a happens before relationship
<davechen1y> it's the same as if you use atomic.StoreUint32 you must always use atomic.LoadUint32
<davechen1y> once one path is covered by a lock, all the other paths have to be covered by the same lock
<axw> davechen1y: https://golang.org/ref/mem -- "Channel communication" disagrees with you
<davechen1y> you're not using a channel
<davechen1y> you're using a mutex
<davechen1y> this is what I meant by "somethign else must be enforcing the happens before"
<davechen1y> ie, all the gorutines are going through another lock (or channel send/receive) and that's what's enforcing the memory barrier
<axw> davechen1y: and like I said, we always write to the requests from the same goroutine that made the API call. there's two possibilities: they're in the same goroutine as the test, so natural happens-before. or they're made in a goroutine that we wait for with a wait group
<davechen1y> if the former was happening, then there wouldn't be a race
<davechen1y> so it's probably not always happening
<axw> davechen1y: in the particular case of the race that was reported, they were goroutines spawned and then waited for with a wait group. so the readers would wait for writers, but the writers were racing with each otehr
<axw> davechen1y: the writers racing with each other is fixed by the mutex. the readers and writers were never racing.
<davechen1y> maybe you should get someone else to review this
<davechen1y> i'm probaby wrong
<axw> davechen1y: you're one of the few people who would know about memory orderings, but I'll get a second opinion :)
<redir> is github.com/juju/juju/audit used anywhere?
<natefinch> redir: I think katco just started working on that, and so it may well not be used anywhere
<redir> natefinch: code from 2014, mostly looks like davechen1y's work
<redir> unrelated to the other audit stuff
<natefinch> oh huh
<natefinch> no clue
<redir> nothing imports it
<natefinch> redir: then it probably should be deleted.
<redir> I did, but then realize I should ask first, so I undeleted it.
<natefinch> redir: kill it with fire, and if anyone complains, that's what VCS is for :)
 * redir gets out his big pink eraser
<redir> gone
<redir> and so am I
<menn0> fix for one of the criticals: http://reviews.vapour.ws/r/4916/
<menn0> wallyworld or axw: ^?
<axw> menn0: looking
<menn0> axw: thanks
<mup> Bug #1577988 changed: Revert destroy service when machine is off <juju-core:Invalid> <https://launchpad.net/bugs/1577988>
<mup> Bug #1583109 changed: error: private-address/public-address not set (1.25.5) <sts> <juju-core:New> <https://launchpad.net/bugs/1583109>
<mup> Bug #1586007 changed: cannot ssh to LXD container on non MAAS systems using juju ssh --proxy 0/lxd/0 <lxc> <lxd> <ssh> <juju-core:In Progress by menno.smits> <https://launchpad.net/bugs/1586007>
<axw> menn0: looks good, just doing QA
<axw> tho pretty trivial, probably doesn't need it
<menn0> axw: you'd just be doing exactly what I just did
<menn0> axw: I just added a step by the way (you need to set a feature flag)
<axw> menn0: yeah ok, I'll skip. LGTM
<menn0> axw: ta
<davechen1y> so, who wants to know what the second top cpu consumer in the agent tests ?
<davechen1y> you'll never guess
<davechen1y> (the first is the gc, but that isn't a surprise)
<davechen1y> +   6.34%       agent.test  agent.test                          [.] runtime.scanobject                                                                                                                                                        â
<davechen1y> -   4.11%       agent.test  agent.test                          [.] math/big.addMulVVW                                                                                                                                                        â
<davechen1y>    - math/big.addMulVVW                                                                                                                                                                                                                       â
<davechen1y>       - 97.78% math/big.nat.expNNMontgomery                                                                                                                                                                                                   â
<davechen1y>          - 99.91% math/big.nat.expNN                                                                                                                                                                                                          â
<davechen1y>             - 86.43% math/big.nat.probablyPrime                                                                                                                                                                                               â
<davechen1y>                - 99.98% math/big.(*Int).ProbablyPrime                                                                                                                                                                                         â
<davechen1y>                   - 99.62% crypto/rand.Prime                                                                                                                                                                                                  â
<davechen1y>                      - crypto/rsa.GenerateMultiPrimeKey                                                                                                                                                                                       â
<davechen1y>                         - 96.14% crypto/rsa.GenerateKey                                                                                                                                                                                       â
<davechen1y>                            - 99.77% github.com/juju/juju/cert.newLeaf                                                                                                                                                                         â
<davechen1y>                               - 88.83% github.com/juju/juju/cert.NewDefaultServer                                                                                                                                                             â
<davechen1y>                                  - 53.01% github.com/juju/juju/environs/config.(*Config).GenerateControllerCertAndKey                                                                                                                         â
<davechen1y>                                     + github.com/juju/juju/worker/certupdater.(*CertificateUpdater).updateCertificate                                                                                                                         â
<davechen1y>                                  + 46.99% github.com/juju/juju/cmd/jujud/agent.(*MachineAgent).upgradeCertificateDNSNames                                                                                                                     â
<davechen1y>                               + 11.00% runtime.stackBarrier
<davechen1y> that's right, TLS negotiation :)
<davechen1y> oh, no
<davechen1y> it's generate key
<davechen1y> it's generating all those faux tls certificates
<davechen1y>   // XXX Do bound checking against totalLen.
<davechen1y> ^ shit i read
<natefinch> well that's a waste
<davechen1y> % go test -v -race -timeout=9001s
<davechen1y> this timeout is over 9000!
<davechen1y> https://bugs.launchpad.net/juju-core/+bug/1586244
<mup> Bug #1586244: state: DATA RACE in watcher <2.0-count> <blocker> <race-condition> <juju-core:New> <https://launchpad.net/bugs/1586244>
<davechen1y> ^ cherylj this one just crept in, the state tests were passing less than 48 hours ago when I re-enable them
<natefinch> wallyworld: you said in the other channel that there aren't any checks in the buildtxn loop that match the assert... is that something we require?  I've never done that.  I assumed the asserts were there specifically to ensure the state we expect.  Doing a check before just seems like it's adding a race condition.
<wallyworld> natefinch: if there's a build txn loop with logic to compose txn slices and retry, unless the logic matches things will blow up
<wallyworld> it's not a race because the txn asserts will reject changes where someone else got in fiest - that's why tests need to use the txn hooks to check that stuff
<wallyworld> but people don't always write the necessary tests
<natefinch> wallyworld: I guess I don't understand why it'll blow up.  I mean, yes, the transaction will fail... but how is that different than if the initial checks fail?
<wallyworld> because the loop that generates the txn slice will never generate a slice that will succeed
<wallyworld> and so it will give up with failing asserts
<natefinch> wallyworld: oh, ok, I see.  so you're saying the logic can be wrong and so the asserts could be effectively contradictory
<wallyworld> yes
<wallyworld> it may not be the case here - i haven't groked the code
<wallyworld> but it's a likely cause
<wallyworld> there's a loop, txns, and that error
<wallyworld> the asserts may not match the code which created them
<natefinch> so, I checked on nil matching nonexistent fields, and it generally does (if you have a sparse collection, it won't, but we don't use those).  All the asserts for this particular transaction are against different fields entirely, so they can't really be contradictory, as far as I can tell. The only thing that seems suspicious is that the assert I pointed you  to assumes that either the address being set either was never set before, or is not being
<natefinch> changed.  So I assume this is a case where the address actually is being changed.   But I'm not sure why we're asserting that setAddress isn't changing the address.
<mup> Bug #1586244 opened: state: DATA RACE in watcher <2.0-count> <blocker> <race-condition> <juju-core:New> <https://launchpad.net/bugs/1586244>
<natefinch> wallyworld: and actually... this code explicitly only calls setaddress if the address *has* changed: https://github.com/juju/juju/blob/master/state/machine.go#L1261
<wallyworld> natefinch: it's not the asserts that are contradictory, it's the asserts and the logic
<wallyworld> ie if might be the asserts and the logic, that's what causes the changing too quickly error
<natefinch> wallyworld: that's my point... we're adding an assert that says "assert that the address hasn't changed" but we're only adding that assert if the address *has* changed.  Granted, if it changes from nothing to something, then the assert will pass.  but all the rest of the code seems to be assuming that it's ok to change from something to something else... except the pesky assert
<wallyworld> perhaps, i haven't deeply read the code. i think martin may have done this logic. the issue if i recall was that we were setting the address all the time even if it hadn't changed and that was casuing watchers to fire etc etc
<wallyworld> i think this code is an attempt to only hit the db if we really need to
<wallyworld> it would be useful to know what the inputs are, write a failing test, and fix the test
<natefinch> wallyworld: yeah, I'll see if I can do that.
<wallyworld> natefinch: also ask martin (if git says so) to confirm the intent of the logic
<davechen1y> axw: got a sec ?
<davechen1y> https://bugs.launchpad.net/juju-core/+bug/1586244/comments/1
<mup> Bug #1586244: state: DATA RACE in watcher <2.0-count> <race-condition> <juju-core:New for dave-cheney> <https://launchpad.net/bugs/1586244>
<davechen1y> i'm not sure which way to go
<davechen1y> i could try to make multiwatcherstore.Get return a copy, but that might be tricky
<davechen1y> the alternative might be to make the test not screw with the data being returned, or try to do the copy then
<axw> davechen1y: looking
<natefinch> wallyworld: looks like it was michael foord.  I'll ping him in the morning... but it seems like this logic is just backwards.  The whole point of the PR is "unit address sometimes changes" and this assert is asserting that we're only setting the value if it *hasn't* changed.
<natefinch> wallyworld: https://github.com/juju/juju/pull/3215/files#diff-287434ac7f7449f3a0c18912a8650f6aR1108
<wallyworld> natefinch: that does seem to be the case doesn't it
<axw> davechen1y: it would be nice for the info thingies to be copied on the way out of allwatcher, but I think that's too much for right now. if it's possible without too much trouble, it'd be good to mock out time so we have predictable timestamps to compare against. otherwise, copy the info objects in the test
<natefinch> wallyworld: oh, I think I get it.  I'm reading it wrong.  It's asserting that the previous address that we think we're going to change it *from* hasn't changed
<wallyworld> that makes sense
<natefinch> wallyworld: so if we think we're changing it from A->B and we run the transaction, and it's actually C, we bail
<wallyworld> sounds right
<natefinch> and when we retry the transaction, we get a new copy of the machine from state, which should now have C
<wallyworld> and if that changes we try again. maybe there's something else setting the value
<natefinch> wallyworld: I'll talk to michael in the morning, maybe he'll have some insight.  I gotta get to bed, it's late.
<wallyworld> np, ty
<wallyworld> natefinch: recording input will be good
<wallyworld> so we cn see what's coming in to trigger it
<natefinch> wallyworld: yep... I wish we had a simpler repro case, but I can make a binary with increased logging and see if we can trigger it and look at the values and/or stack traces.  Wonder if it's competing workers somehow.
<wallyworld> yeah, log what's in db, what we receive etc
<wallyworld> if they can repor then great
<natefinch> yeah, landscape seems to be able to repro by running a big deploy script.
<natefinch> ok, I'm out.
<davechen1y> axw: thanks for the advice
<davechen1y> i'll see what I can do
<davechen1y> the multiwatcher is such a garbage fire
<anastasiamac_> cmars: ping
<anastasiamac_> rogpeppe1: ping
<rogpeppe1> anastasiamac_: hiya
<frobware> dimitern: ping
<dimitern> frobware: pong
<frobware> dimitern: I played with your patch a bit - no IPv6 addresses turned up. Also see on #juju - stokachu tried it too
<dimitern> frobware: yeah, but my fix was apparently to restrictive
<frobware> dimitern: "frobware: so I did a new deploy with the default ipv6 enabled, and juju pulled the ipv4 like it was supposed to"
<frobware> dimitern: how so?
<dimitern> frobware: I still need to allow IPv6-only tests and setups to work
<dimitern> frobware: that's why I initially wanted to store preferred addrs per type
<dimitern> frobware: e.g. there's single test in cmd/juju/commands/scp_unix_test.go that fails for a good reason "test 11: scp works with IPv6 addresses"
<dimitern> frobware: it feels wrong to pretend there's a single private/public address of any machine anyway
<dimitern> frobware: rather than fixing the charms like rabbitmq-server..
<mup> Bug #1482226 changed: juju status with 'prefer-ipv6' shows address, not DNS name. <amd64> <apport-bug> <ipv6> <network> <status> <trusty> <uec-images> <juju-core:Won't Fix> <https://launchpad.net/bugs/1482226>
<Alex_____> kwmonroe cory_fu kjackal admcleod stub, guys could anybody please point me to the docs on using spot instances with juju on AWS?
<dimitern> frobware: I think I got it
<kjackal> Hi Alex_____ , I guess I am the only one here at this time
<dimitern> frobware: so if we still return the preferred public|private by default when asked, but when they're not set we try to select a new one by scope from all the available addrs seems to work ok
<frobware> dimitern: why would by scope return IPv4 in favour of IPv6?
<Alex_____> kjackal: great! Could you help me pointing to the way of using spot instances plz?
<kjackal> <Alex_____ I am not using AWS but let me search on this. It seems to be a question for Juju in general
<dimitern> (ssh/scp should really try to use all addresses, rather rely on a single public or private)
<dimitern> frobware: it won't
<kjackal> Alex_____: Let me do a quick search
<dimitern> frobware: but if you have any IPv4 addrs already, one will be preferred anyway
<Alex_____> kjackal: thanks, I appreciate! can move to #juju
<frobware> dimitern: is this "desired" for a mixed IPv4/6 setup?
<dimitern> frobware: and that extra lookup will allow ipv6-only setups to still work (and woe to whoever tries to deploy rabbitmq-server there :D)
<dimitern> frobware: our "desired" state should be "don't discriminate addresses by type/scope/etc. just try all in parallel"
<dimitern> well for legacy / badly written charms we need to pretend still there is 1 private and 1 public addr only..
<dimitern> but well written charms can ask for all addrs (opt. filtering them by type etc.) via network-get
 * dimitern dreams about the day when 'unit-get private-address' will be a 'charm proof' *ERROR*
<dimitern> jamespage, gnuoy: do you know why rabbitmq-server charm does not work well with IPv6 addresses?
<jamespage> dimitern, it should do
<dimitern> jamespage, gnuoy: it seems the upstream rabbit supports fine.. or is it due to magic hacluster VIP behavior?
<jamespage> dimitern, you don't need to use the hacluster charm with rabbitmq
<jamespage> infact I'm going to take out that support
<dimitern> jamespage: well, I've been struggling to fix juju to let rabbit work - see bug 1574844
<mup> Bug #1574844: juju2 gives ipv6 address for one lxd, rabbit doesn't appreciate it. <conjure> <juju-release-support> <landscape> <lxd-provider> <juju-core:In Progress by dimitern> <rabbitmq-server (Juju Charms Collection):New> <https://launchpad.net/bugs/1574844>
<jamespage> dimitern, ok you might want to try my inflight fixes for rmq
<dimitern> jamespage: simply hiding ipv6 addresses from charms (unit-get private|public-address) sounds like the wrong kind of fix
<dimitern> jamespage: sure, I'd like to try
<jamespage> dimitern, try with cs:~james-page/xenial/rabbitmq-server-bug1584902
<jamespage> I've simplied how units cluster together alot and removed the requirement for DNS forward/reverse lookup althogether
<dimitern> jamespage: ta! trying now
<dimitern> jamespage: awesome! so this might actually solve both that bug ^^ and the one about rmq failing to work when the node's hostname is not resolvable..
<jamespage> dimitern, yes the fix is horrible but it should work
<jamespage> dimitern, the charm already writes <ip> <hostname> into /etc/hosts for all peers in the cluster
<jamespage> so even if that's an IPv6 one it should just dtrt
<dimitern> jamespage: sweet! tyvm
<dimitern> that should let my os-lxd on maas 2.0 setup with the nucs to work without manual steps / tweaks!
<jamespage> dimitern, if you could feedback on your testing on the associated bug that would be great
<jamespage> dimitern, I want to backport that to the stable charm, so the more testing the better...
<voidspace> babbageclunk: morning
<dimitern> jamespage: will do
<babbageclunk> voidspace: o/
<dimitern> jamespage: so just deploying rmq with your fix, then adding 1 or 2 units should cover the simplest test?
<dimitern> (also looking for errors, etc. ofc)
<jamespage> yah
<dimitern> ok
<axw> wallyworld: I'm thinking the cloud/credential stuff in mongo would probably fit well with other controller config, like ca-cert. so maybe "two birds"
<dimitern> jamespage: your fix works like a charm! :) - successfully tested deploying 3 units on maas 2 and 8 units on lxd (one of them with IPv6 address); eventually all reported "Unit is ready and clustered" and no hook errors in sight!
<jamespage> dimitern, love it when ripping out code results in better solutions...
<dimitern> jamespage: tell me about it.. :D
<mup> Bug #1586298 opened: Cannot run upgrade-juju as upload-tools refers to "admin" model post the s/admin/controller/ change <juju-core:New> <https://launchpad.net/bugs/1586298>
<dimitern> jamespage: ah, actually the lxd machine with IPv6 address had a hook error hook failed: "cluster-relation-joined"
<dimitern> jamespage: I'll add logs to the bug comment
<dimitern> jamespage: bug 1584902 updated; it looks like the hook error was due getting NXDOMAIN
<mup> Bug #1584902: Setting RabbitMQ NODENAME to non-FQDN breaks on MaaS 2.0 <canonical-bootstack> <cpec> <juju2> <maas2> <sts> <rabbitmq-server (Juju Charms Collection):Fix Committed by james-page> <https://launchpad.net/bugs/1584902>
<dimitern> jamespage: do you think I can provide anything else useful before I tear it down?
<jamespage> dimitern, content of /etc/hosts on all units please
<dimitern> jamespage: sure, that's all?
<jamespage> dimitern, OK i think I see the problem
<jamespage> I missed a get_host_ip call
<dimitern> jamespage: yeah - there's something fishy about that machine-7 - /e/hosts does not look like on the other machines: http://paste.ubuntu.com/16727753/
<dimitern> added to the bug
<jamespage> dimitern, ta
<jamespage> dimitern, can you output unit-get private-address on all units as well pls
<dimitern> jamespage: it's already part of the status output paste
<voidspace> dimitern: ping
<jamespage> oh yes of course..
<jamespage> sorry
<dimitern> np
<dimitern> voidspace: pong
<voidspace> dimitern: core/description/interfaces.go:IPAddress has a ConfigMethod - is it ok for that return a state.AddressConfigMethod do you think
<voidspace> dimitern: or should it return a string
<voidspace> dimitern: nothing else in core/description/interfaces.go returns state types
<dimitern> voidspace: yeah, we need to apply the same pattern as for api params and state docs
<dimitern> voidspace: i.e. duplicate the type in core/desc/ (however unfortunate)
<dimitern> voidspace: there's actually a test somewhere there that verifies description (or core?) does not import extra pkgs
<voidspace> dimitern: right
<voidspace> dimitern: I can see elsewhere places using string
<voidspace> dimitern: e.g. Address.Scope (which is a legacy IPAddress)
<dimitern> voidspace: we should not migrate the legacyipaddressesC stuff
<voidspace> dimitern: actually probably a network address
<dimitern> frobware, dooferlad, fwereade, jam: standup?
<dooferlad> dimitern: I am out of the office today getting a new passport.
<dimitern> dooferlad: what are you doing here then? :D
<dooferlad> dimitern: irccloud + laptop <--> phone. Welcome to the future :-)
<frobware> dimitern: OoO too
<dimitern> frobware: yeah, I forgot, sorry
<jamespage> dimitern, ok so I see the issue
<jamespage> the is_ip function in charm-helpers fails to support IPv6 addresses
<jamespage> fixing that now
<dimitern> jamespage: awesome! \o/
<voidspace> dimitern: why does IPAddress have DNSServers/DNSSearchDomains/GatewayAddress?
<voidspace> dimitern: surely they belong to Subnet
<dimitern> voidspace: they can apply per device as well
<voidspace> dimitern: ok, but not *per address*, surely?
<dimitern> voidspace: well, dooferlad insisted to keep layers (2 and 3) separate, which is good
<jamespage> dimitern, https://code.launchpad.net/~james-page/charm-helpers/is-ip-ipv6/+merge/295920
<dimitern> voidspace: so they do apply per address
<voidspace> dimitern: hmm
<dimitern> voidspace: as they mirror what you can do on NICs
<dimitern> jamespage: cheers, looking
<mup> Bug #1574844 changed: juju2 gives ipv6 address for one lxd, rabbit doesn't appreciate it. <conjure> <juju-release-support> <landscape> <lxd-provider> <juju-core:Won't Fix> <rabbitmq-server (Juju Charms Collection):In Progress by james-page> <https://launchpad.net/bugs/1574844>
<dimitern> jamespage: nice and short fix :) could I test it with rmq?
<jamespage> dimitern, cs:~james-page/xenial/rabbitmq-server-bug1574844
<dimitern> jamespage: ta! testing now on lxd
<dooferlad> dimitern: I agree that we should keep physical separate from IP, separate from higher layer protocols. We should have collections per layer of the TCP/IP stack. That would give us NetPhysical and NetIP collections.
<dooferlad> dimitern I agree with voidspace that IPAddress is a confusing name for something with DNS and routing information in it
<dimitern> dooferlad: well I suggested LinkLayerDeviceAttachment..
<dimitern> :)
<dimitern> or assignment
<dooferlad> dimitern: too long :-)
<dimitern> dooferlad: indeed
 * dimitern wishes we had a good way to namespace stuff in state 
<dimitern> then we could've used e.g. statenetwork.Device, statenetwork.Attachment, ...
<dimitern> fwereade: how about that? ^^
<fwereade> dimitern, perhaps: +1 on namespacing, indeed
<dimitern> fwereade: cheers, I'll think about it
<fwereade> dimitern, but, well, I feel like namespacing of the implementation details is a more pressing issue?
<mup> Bug #1574844 opened: juju2 gives ipv6 address for one lxd, rabbit doesn't appreciate it. <conjure> <juju-release-support> <landscape> <lxd-provider> <juju-core:Won't Fix> <rabbitmq-server (Juju Charms Collection):In Progress by james-page> <https://launchpad.net/bugs/1574844>
<dimitern> fwereade: example?
<dimitern> jamespage: it works! \o/ http://paste.ubuntu.com/16728135/
<voidspace> dooferlad: :-)
<voidspace> cool
<jamespage> dimitern, can I see the /etc/hosts again pls?
<dimitern> jamespage: all of them look consistent now: http://paste.ubuntu.com/16728155/
<jamespage> dimitern, good-oh
<jamespage> dimitern, change is up for review
<fwereade> dimitern, well, let me reorient myself: is your problem that state exports too many types?
<dimitern> jamespage: great, thanks for the quick fix!
<jamespage> dimitern, do we understand why one unit gets an IPv6 address?
<jamespage> that does not seem particularly deterministic to me
<fwereade> dimitern, my perspective is that the private state namespace is utterly stuffed with bits that we plausibly *could* carve into a few /internal/ packages
<fwereade> dimitern, but that it will be much harder to separate useful exported types because import cycles
<dimitern> jamespage: yes, it's not - just so happens LXD provider reports the IPv6 first (and that's the only one at that moment), but the it's taken as preferred private address and sticks
<dimitern> fwereade: well, not quite - the problem I have is naming
<dimitern> fwereade: and SoR principle
<fwereade> dimitern, naming objecty things with methods, right?
<fwereade> dimitern, (rather than in/out data types?)
<dimitern> fwereade: yeah, but some names are much too overloaded to make sense in a global namespace; so we need to use longer, more descriptive, eventually awkward at times..
<mup> Bug #1574844 changed: juju2 gives ipv6 address for one lxd, rabbit doesn't appreciate it. <conjure> <juju-release-support> <landscape> <lxd-provider> <juju-core:Won't Fix> <rabbitmq-server (Juju Charms Collection):In Progress by james-page> <https://launchpad.net/bugs/1574844>
<dimitern> fwereade: e.g. a machine can have a bunch of things in different 'namespaces', which are separate concepts and can use the same term (disk device vs network device)
<babbageclunk> voidspace, dimitern - If I want to try juju with mgo.v2-unstable to see if it fixes the txn problem I'm seeing with 3.2, is there any way to do it without changing all of the 286 files that import gopkg.in/mgo.v2 to have -unstable?
<dimitern> babbageclunk: there is - govers
<babbageclunk> Ooh
<dimitern> babbageclunk: go get -u -v github.com/rogpeppe/govers/...
<babbageclunk> dimitern: thanks. Ok, but I still need to rewrite all those files.
<dimitern> babbageclunk: don't you need it only for juju/juju/ though?
<babbageclunk> babbageclunk: Maybe? I guess - it depends if juju/juju calls anything in a library that then uses mgo.
<babbageclunk> dimitern: Duh, replied to myself. ^
<dimitern> babbageclunk: well, you can try godeps -n -t github.com/juju/juju/...
<babbageclunk> dimitern: ?
<dimitern> babbageclunk: and see if you end up with more than one mgo import after running govers to update imports?
<babbageclunk> dimitern: Oh, I see - ok, I'll commit my current stuff and try that. Thanks!
<babbageclunk> dimitern: It turns out govers does that check itself, yay! Unfortunately: http://pastebin.ubuntu.com/16729162/
<dimitern> babbageclunk: hmm.. well, try with e.g. gopkg.in/mgo.v2-unstable/bson ?
<babbageclunk> dimitern: I just did it at the top level, seems to have worked! Building now.
<dimitern> babbageclunk: \o/
<hoenir> can anyone look in this pr? http://reviews.vapour.ws/r/4924/
<babbageclunk> hoenir: I already did! Good stuff, thanks for revisiting it.
<hoenir> babbageclunk, thanks again !
<babbageclunk> dimitern: Ah well, it was worth checking. Same problem still.
<babbageclunk> dimitern: Now to revert all those changes!
<dimitern> babbageclunk: so you managed to build with the unstable mgo?
<dimitern> babbageclunk: but it didn't help?
<babbageclunk> dimitern: yup
<babbageclunk> dimitern: nope
<dimitern> babbageclunk: I see - well, as you said it was worth at least checking :)
<babbageclunk> dimitern: :) Yeah, better that than report a bug that's already fixed in the unstable branch.
<dimitern> babbageclunk: +1, indeed
 * dimitern finally managed to add & commission a kvm node to the hw maas 2.0.. after breaking *everything* for a short while
<hoenir> so babbageclunk no +1 and $$merge$$ or you forgot ?
<babbageclunk> hoenir: Oh, I'm not a fully-fledged reviewer yet (new myself) - you should get someone else to look at it as well. Sorry!
<dimitern> hoenir: I'll have a look
<hoenir> dimitern, thanks
<dimitern> hoenir: how could I test this?
<dimitern> hoenir: e.g. trying to add a windows machine to an controller should trigger it I guess?
<hoenir> dimitern, you should boostrap + deploy a windows machine and it should be fine, also when deploing the windows machine try sudo su in the machine where you have the maas installed
<hoenir> and execute this commands
<hoenir> maas-region-admin shell
<hoenir> from metadataserver.models.nodeuserdata import NodeUserData
<hoenir> nodeobj = NodeUserData.objects.get(node__hostname="<name_of_the_windows_machine_in_maas")
<hoenir> nodeobj.data
<dimitern> hoenir: awesome, will try that then
<hoenir> and you should check if the nodeobj.data dosen't contain "\n\n" in the begining..
<hoenir> and also when the windows machine was deployed you could log into it and check the cloudbase-init logs. if in the logs tails if it installed the jujud and all other stuff it should be fine
<dimitern> hoenir: remind me how was I supposed to import a windows image into maas 2?
<hoenir> if you see "[WARNING] unsupported format,blablbala than the nodeobj.data var that's holding the string it's bad
<hoenir> I have one maas windows image win2kr12r2 from cloudbase , i think in order to make one yourself you must exec some custom python code. There is a script in the cloudbase git repo that will do so, but from what I know it will take 6-7 hours to complete it.
<hoenir> https://github.com/cloudbase/windows-openstack-imaging-tools/tree/experimental
<hoenir> powershell* , excuse me for saying it's python.
<hoenir> dimitern, I tested myself before submitting the patch to upstream so I think this will not be a problem, but feel free to test it yourself if you want to.
<dimitern> hoenir: it's not that I don't trust the patch, but so close to the release it was decided to verify all fixes more carefully
<mup> Bug #945862 opened: Support for AWS "spot" instances <adoption> <juju-core:New> <https://launchpad.net/bugs/945862>
<hoenir> I was disconnected, so dimitern , you said something more?
<dimitern> <dimitern> hoenir: it's not that I don't trust the patch, but so close to the
<dimitern>            release it was decided to verify all fixes more carefully  [15:15]
<natefinch> voidspace: you around?
<voidspace> natefinch: yep
<natefinch> voidspace: I'm looking at this: https://bugs.launchpad.net/juju-core/+bug/1537585
<mup> Bug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged by natefinch> <juju-core 1.25:Triaged by natefinch> <https://launchpad.net/bugs/1537585>
<natefinch> voidspace: which I think is failing because this assert is failing: https://github.com/juju/juju/blob/master/state/machine.go#L1238
<natefinch> voidspace: which git seems to say was something you worked on like 9 months ago :)
<natefinch> voidspace: so I'm sure it's still fresh in your mind
<voidspace> natefinch: :-)
<voidspace> dimitern: ^^^^
<voidspace> natefinch: preferred address logic has changed recently
<natefinch> voidspace: my first question is - if we're just changing the address anyway, why do we care if the old address is the same as the one we expected it to be?
<voidspace> natefinch: the point of the assert is that we shouldn't change the preferred address once set
<voidspace> natefinch: we're not changing it - we should either be setting it for the first time (nil) or just setting it to the same
<voidspace> natefinch: that was the point of the assert
<voidspace> natefinch: a better way would be to just assert it's nil
<natefinch> voidspace: but the function one step up the stack has already checked if we're changing it: https://github.com/juju/juju/blob/master/state/machine.go#L1262
<voidspace> natefinch: race condition - two concurrent operations changing it at the same time
<voidspace> (set on first access - very likely to get concurrent ops here)
<voidspace> that's why we have the assert as well as the check
<natefinch> voidspace: right, so last one in wins... that's what is going to happen anyway, since we're just going to retry the transaction until it works
<natefinch> voidspace: or in this case, it doesn't ever work and the machine gets borked... still not sure why that is
<voidspace> natefinch: but that doesn't matter as it will have to be the same one - it will always fail if it tries to set a different one
<natefinch> voidspace: but then what is that code doing that is checking if it's changed?
<natefinch> voidspace: on line 1261... if changing is bad, shouldn't we just bail there?
<voidspace> ah, no - I'm slightly mistaken
<voidspace> we can change if we have a better match on scope
<voidspace> let me look again at that assert
<voidspace> natefinch: hmmm... so actually - if two operations try to concurrently set a new address theen the first one will succeed and the second will fail because the address is now not "current"
<voidspace> natefinch: that would better be done as a buildTxn function
<voidspace> refreshing the doc on each attempt
<natefinch> voidspace: well, so, this is called from inside a buildtxn and we do refresh the doc each time
<voidspace> ah yes, I see that now in setAddresses
<voidspace> the intent of the assert is that the doc hasn't changed (i.e. current address is unset or as we expect)
<voidspace> if that fails the first time - the address has changed, why does it fail *again*
<natefinch> yeah... that's the question.  I think I may need to toss in a bunch of extra logging so we can see what it's being called with. That might make it more obvious what's going on.  If it's ping ponging between two or if one is just oddly failing over and over
<natefinch> voidspace: it's hard to repro... the landscape guys need to deploy a semi-large environment from a script to trigger it, and even then it doesn't always happen, so it's probably timing related
<voidspace> but it shouldn't ping pong as the address seen shouldn't actually change until the transaction is applied
<voidspace> so from my reading of the code even if two concurrent changes come in, one will work, the second will fail once and then either come back with no ops or succeed
<natefinch> yeah, that's what I would expect from reading the code
<voidspace> so I don't understand I'm afraid
<natefinch> voidspace: it's ok. I'll see if I can get some better logs
<voidspace> natefinch: :-/ good luck
<natefinch> voidspace: thanks for looking at it with me though.
<dimitern> natefinch, voidspace: I might have some insight about that issue with setting addresses
<dimitern> natefinch, voidspace: I've been changing that code while trying to fix a related bug avoiding IPv6 preferred addresses
<dimitern> natefinch, voidspace: and I suspect a few possible causes, mainly the way the buildTxn in setAddresses is handling subsequent attempts
<voidspace> dimitern: by the way - I have thoughts about your change to preferred addresses for ipv6
<voidspace> dimitern: I'm back to thinking we don't need to track seperate ipv4 and ipv6 preferred addresses
<dimitern> natefinch, voidspace: there's also maybeGetNewAddress overriding the origin and not checking whether it was selected or not (i.e. it can try setting an empty address)
<voidspace> dimitern: you said that the intent of that field on the doc was not to change - so we needed separate ipv4 & ipv6 addresses
<dimitern> voidspace: told you so :)
<voidspace> dimitern: actually the intent was that the public api (PreferredAddress) doesn't change
<voidspace> dimitern: and your change *does* change that
<voidspace> dimitern: so under the hood changing preferred address from ipv6 to ipv4 is *fine*
<voidspace> dimitern: and better than adding new fields and methods
<dimitern> voidspace: but fortunately, it's unnecessary now as jamespage fixed the real issue with the rabbitmq charm and charm-tools (helpers?) not handling IPv6 addrs properly
<voidspace> dimitern: hah
<voidspace> dimitern: ok
<natefinch> dimitern: I can get more logs to get a better idea of what we're actually setting... but it sounds like you've found some problems in the logic anyway.
<dimitern> natefinch: yeah, I also tried *really* hard to repro it.. no luck.. well, except for poking into mongodb
<natefinch> dimitern: yeah, the only reliable repro I can find is landscape's deployment script that makes like 15 machines/containers at once
<dimitern> natefinch: e.g. if you manage to get to the mongo client shell of a running model, updating an existing machineDoc's preferredprivateaddress field to a non-nil, serialized version of state.address{}
<dimitern> and I suspect that's what actually happens only occasionally, under heavy load like with the landscape scenario
<natefinch> yep
<dimitern> natefinch: also, the upgrade step (see state.AddPreferredAddressesToMachines or whatsit) might be causing it (didn't we start doing auto-upgrading across patch-versions of the same minor version at some point?)
<natefinch> dimitern: hmm, good question
<dooferlad> frobware: ping?
<fwereade> katco, natefinch: do you know where the surprise introduced at payload/context/register.go:83 is corrected?
<katco> fwereade: tal
<fwereade> katco, ty
<natefinch> fwereade: yikes
<fwereade> katco, natefinch: fwiw, all I can see is payload/state/unit.go:58, which is commented out and may just be a casualty of the recent changes
<fwereade> katco, natefinch: it certainly seems to get all the way to state without anything checking it makes sense
<katco> fwereade: natefinch: payload/api/helpers.go:72 looks like it overwrites it
<fwereade> katco, that's just the id parsed from whatever unit was sent
<katco> fwereade: hm you are correct
<katco> fwereade: we have standup and then we'll continue investigating
<fwereade> katco, thanks
<cmars> is there a feature branch where service is being renamed to application?
<alexisb> cmars, yes
<cmars> alexisb, service-to-application, that's gotta be it :)
<cmars> thanks
<alexisb> :)
<alexisb> cmars, Ian sent me a summary of happenings as well
<alexisb> if it is relavent to you I can forward it along
<cmars> alexisb, sure, that'd be great. i'm fixing up the romulus change atm, just wanted to test against juju before proposing
<alexisb> cmars, awesome
<alexisb> dooferlad, you still around?
<katco> fwereade: natefinch did a test, and it's definitely getting set somewhere. i think it should be passed in here (config.UnitName) and then utilized instead of a placeholder
<katco> fwereade: oops, there was intended to be a link there: https://github.com/juju/juju/blob/master/component/all/payload.go#L112-L113
<natefinch> is the standard way to log into mongo different in 2.0?  I'm using mongo --ssl -u admin -p <oldpassword from agent.conf> localhost:37017/admin
<massIVE> In juju 2.0 mass is not listed in list-clounds, any ideas to why?
<natefinch> massIVE: you have to add-cloud
<natefinch> massIVE: it's a little cumbersome right now.  You have to make a .yaml file with the maas cloud defined in it, thusly: http://pastebin.ubuntu.com/16733874/
<natefinch> massIVE: then juju add-cloud mymaas myclouds.yaml
<dooferlad> alexisb: sorry, yes
<dooferlad> alexisb: for some reason my PC thought USB was a silly idea for a few minutes *confused*. I need that for typing!
<natefinch> massIVE: then you can juju bootstrap controllername mymaas   .. I think it'll prompt for the oauth cred at that point
<massIVE> natefinch: i did that, but must have done something wrong last time, works now ;)
<natefinch> massIVE: cool.. yeah, it's not the most user friendly experience right now.  I think we're working on simplifying it soon
 * dooferlad reboots again. For the fun.
<massIVE> natefinch: um, ERROR no registered provider for "mass"
<natefinch> massIVE: maas or mass?
<massIVE> lol  :) k
<natefinch> :D
 * dooferlad has discovered that a USB extension cable not plugged into anything can be a bad thing
<alexisb> dooferlad, heya, was just looking for an udpate on: https://bugs.launchpad.net/juju-core/+bug/1577945
<mup> Bug #1577945: Bootstrap failed: DNS/routing misconfigured on maas <blocker> <bootstrap> <ci> <maas-provider> <network> <regression> <juju-core:In Progress by dooferlad> <juju-core 1.25:Triaged by dooferlad> <https://launchpad.net/bugs/1577945>
<dooferlad> alexisb: I am waiting for a +1 on http://reviews.vapour.ws/r/4902
<alexisb> dooferlad, cool, can you put the PR in the bug with an update please
<dooferlad> alexisb: sure
<alexisb> voidspace, if you are around looks like dooferlad could use a review :)
<dooferlad> voidspace, frobware, dimitern: http://reviews.vapour.ws/r/4902/ *poke*
<katco> fwereade: i found it... this is a bug that was introduced very recently (~10 days). trying to provide you with a walkthrough
<dimitern> dooferlad: looking
<katco> fwereade: nate is checking whether tip works correctly, but to me it looks like a bug was introduced here: https://github.com/juju/juju/commit/b3fb5cbc9c31#diff-c04416a1afc4fb32911bcaafcfdb48a1L29
<katco> fwereade: (in that last block)
<hoenir> could anyone check this PR and $$merge$$ it? http://reviews.vapour.ws/r/4924/
<fwereade> katco, you don't consider "a-service/0" making it safely through... context, api client, api server, state... 4 layers, before being silently overwritten in the persistence layer, to be a bug?
<fwereade> katco, that is pretty much just working by accident
<natefinch> fwereade: I believe eric's intent was to remove unit from the payload struct entirely. This may have been a step in that direction, that just never got completed
<natefinch> fwereade: tip is broken btw
<fwereade> natefinch, I don't really see how introducing nonsense data at runtime is anything other than... introducing nonsense data at runtime
<natefinch> fwereade: well, certainly, I don't think this should have ever made it into master as-is
<katco> fwereade: just trying to help you figure out where it stopped working. looks like that's it.
<fwereade> natefinch, my branch fixes it in state, fwiw, if I finish it today it'll be late I'm afraid
<dimitern> dooferlad: lgtm; however, I'd appreciate steps how to verify this locally
<natefinch> fwereade: seems like a refactor that was not completed, and was accidentally committed to master.
<fwereade> katco, ok, sure, but... the *persistence* layer silently overwriting one of the fields specified by state? I thought that was where you were putting the business rules -- surely that *should* be the layer that decides *what* gets persisted?
<katco> dimitern: can you +1 this? http://reviews.vapour.ws/r/4924/
<dooferlad> dimitern: just start a node that uses DHCP as its address allocation method. dhclient will be running against the bridged interface rather than the parent. Very simple.
<dimitern> dooferlad: and you mean DHCP not Auto Assign?
<katco> fwereade: yes i agree. finding the commit where it broke does not imply complicit acceptance of an approach.
<dooferlad> dimitern: yes
<fwereade> katco, well, this situation does seem to be a direct consequence of the loose/flexible style you were advocating earlier
<katco> fwereade: what? no, not at all
<fwereade> katco, heh, it struck me as well-timed and instructive
<katco> fwereade: those two concepts aren't even connected
<fwereade> katco, bad data making it through 4 layers *not* connected with trusting your clients because it lets you go "fast"?
<katco> fwereade: you are misconstruing my comment. it's not at all about not doing data validation at your layer boundaries.
<katco> fwereade: going to go do some work, enjoy your evening.
<natefinch> fwereade: FWIW, the only way to verify the unit info would be to hit the DB.... should we really do that at every boundary?  IMO, the only real problem is that we let it get into the DB that way.  That's the only time you really need to ensure the unit is valid.  There should have been an assert that the unit exists.
<fwereade> natefinch, uh, no: in the apiserver layer, you are supplied with an authoriser that tells you what entity you're connecting on behalf of. if you don't check that the connected entity is allowed to make the changes it asks for, you are just broken
<natefinch> fwereade: yes, also a bug
<natefinch> fwereade: sorry, gotta run to pick up my daughter from preschool... back in am hour
<fwereade> katco, sorry, but: (1) you don't validate, and (2) bad data gets through; ISTM that (1) => (2). do you have some alternative explanation for (2)?
<fwereade> katco, or: you are disagreeing with some aspect of "don't trust your clients" but not actually advocating the approach taken in payloads, despite "trusting its clients" apparently being a design principle and a source of evident problems?
<dimitern> dooferlad: still there?
<dooferlad> dimitern: not really
<rogpeppe1> haven't done this for a while, friday fun: here's a stab at a complete auto-generated representation of the Juju API (RPC calls only) http://paste.ubuntu.com/16739140/
<natefinch> rogpeppe1: call it FacadeName so it prints out above Methods :)
<natefinch> rogpeppe1: very cool
<rogpeppe1> natefinch: i should change rjson so that it preserves ordering
<natefinch> rogpeppe1: even better
<natefinch> rogpeppe1: also, man, that's huge.
<natefinch> rogpeppe1: I guess it includes definitions of even stdlib types, though
<rogpeppe1> natefinch: yeah
<rogpeppe1> natefinch: here's the code i used to generate it: http://paste.ubuntu.com/16739827/
<rogpeppe1> natefinch: anyway, early days yet. at some point, i'll add doc comments too, and produce linked HTML output :)
<rogpeppe1> natefinch: then we might actually have something like an API doc...
<natefinch> rogpeppe1: that would be amazing
<rogpeppe1> natefinch: i don't think it would be more than half a day's work
<rogpeppe1> natefinch: anyway, gotta go and frolic
<rogpeppe1> natefinch: cheerio, and thanks for being interested :)
<natefinch> rogpeppe1: definitely. happy weekend!
<alexisb> rogpeppe1, this is awesome
<alexisb> rogpeppe1, you would be my hero!
<mup> Bug #1584815 changed: SSHSuite.TestSSHCommand fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1584815>
<mup> Bug #1585388 changed: Container networking cannot ssh after machine is ready <blocker> <ci> <lxc> <lxd> <network> <regression> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1585388>
<mup> Bug #1584815 opened: SSHSuite.TestSSHCommand fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1584815>
<mup> Bug #1585388 opened: Container networking cannot ssh after machine is ready <blocker> <ci> <lxc> <lxd> <network> <regression> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1585388>
<hoenir> So does this build not failed? https://github.com/juju/juju/pull/5449
<alexisb> hoenir, it looks like htat did merge, there were some issues with the bot yesterday, so it is a bit clear to me what happened there
<hoenir> aha, so everything is ok then..
<mup> Bug #1584815 changed: SSHSuite.TestSSHCommand fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1584815>
<mup> Bug #1585388 changed: Container networking cannot ssh after machine is ready <blocker> <ci> <lxc> <lxd> <network> <regression> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1585388>
<mup> Bug #1586512 opened: juju2 websocket api response consistency APIHostPorts versus Login response <landscape> <usability> <juju-core:New> <https://launchpad.net/bugs/1586512>
#juju-dev 2016-05-29
<mup> Bug # changed: 1539785, 1558232, 1558239, 1560191
<thumper> ugh
<thumper> provider/lxd tests don't pass if you aren't in the lxd groupd
<thumper> ffs
<thumper> test isolation FTW
<davecheney> thumper: protip: uninstall lxd and they pass
<davecheney> (i'm not joking)
#juju-dev 2017-05-22
<babbageclunk> wallyworld: The subordinate bug fix PR - can you take a look please? https://github.com/juju/juju/pull/7369
<wallyworld> sure
<babbageclunk> wallyworld: Thanks - checking about that other related bug now.
<wallyworld> sgtm
<babbageclunk> wallyworld: do you remember who was looking at the "state changing too quickly" error when removing and redeploying an application?
<wallyworld> babbageclunk: eric was late last week
<babbageclunk> I just got it too - I guess he isn't having any trouble reproducing it anyway.
<babbageclunk> Huh, I can't put ntp on mysql, even though I've deployed mysql to trusty?
<babbageclunk> gah, having lots of trouble trying to get subordinates on real applications that also connect together - series mismatches
<blahdeblah> babbageclunk: I'm pretty good at deploying the ntp charm - yell if you need help. :-)
<babbageclunk> blahdeblah: Thanks! I'm trying again now, I think I needed to specify series for ntp as well as for mysql.
<blahdeblah> Yeah - if you've got a multi-series model, you need separate versions for each subordinate
<blahdeblah> here's an example from one of my test envs: http://pastebin.ubuntu.com/24621417/
<babbageclunk> blahdeblah: Ah right - thanks. For the thing I'm testing I need it to be the same ntp application related to both other things. All seems to work ok now.
<blahdeblah> babbageclunk: You can associate two ntp subordinates with each other and they will peer
<babbageclunk> blahdeblah: oh, neat
 * babbageclunk goes for a run
<axw> wallyworld: https://github.com/juju/juju/pull/7371 fixes a windows test failure
<wallyworld> looking
<wallyworld> axw: lgtm, thanks
<wallyworld> axw: here's a small one when you get a moment later https://github.com/juju/juju/pull/7370
<axw> wallyworld: what's the point of that extra executing status? it's going to be replaced immediately by "running install hook"
<wallyworld> axw: probably a good point; i ported across some code from an old PR which wasn't quite right as it didn't do the set status check. i should just remove that
<axw> wallyworld: I think so. LGTM otherwise
<wallyworld> ta, will remove that bit
<wallyworld> axw: here's a another small one https://github.com/juju/juju/pull/7372
<wallyworld> axw: for the list-clouds one, maybe the word "customised" instead of "bespoke" ?
<axw> wallyworld: yeah I guess so. maybe "private/custom"?
<wallyworld> ok, will use that
<wallyworld> i'll push the change
<wallyworld> axw: thanks!
<axw> wallyworld: np
<urulama_> gsamfira: ping
<gsamfira> urulama: pong
<bdx> heyyyy, does JUJU_UNIT_NAME still get set in the charm env?
<bdx> I'm hitting this http://paste.ubuntu.com/24626263/
<cmars> bdx, grepping through the source code, it looks like it should still be
<cmars> bdx, if the hook is executed by the unit agent (jujud)
<cmars> bdx, did you try running this with debug-hooks?
<cmars> i never use debug-hooks, usually debug with print statements in lxd .. there might be a regression in that feature
<bdx> cmars: thanks ... I don't see it in my env when in debug-hooks
<babbageclunk> wallyworld: ugh, there is some crazy stuff going on with Uniter API versioning. We're in a weird situation now, I think,
<wallyworld> babbageclunk: otp with heather, can we catch up soon?
<babbageclunk> yup yup
<wallyworld> babbageclunk: free now if you want to chat
<babbageclunk> wallyworld: yup yup - just off the phone with thumper
<babbageclunk> in stand
<babbageclunk> up?
<thumper> still seeing this... machine-2: 11:10:38 ERROR juju.state database.go:273 using unknown collection "remoteApplications"
<thumper> from a recent develop
<thumper> when was this fixed?
<thumper> wow...
<thumper> goroutine profile: total 123958
<babbageclunk> thumper: sorry, was otp - fix landed on develop 6 days ago, but maybe there's another place that also does it?
<babbageclunk> wallyworld: sweet, SLA stuff isn't on 2.1 branch, so I can simplify everything down to the two versions.
<wallyworld> babbageclunk: awesome
#juju-dev 2017-05-23
<axw> wallyworld: was doing dropoff (same goes for every tuesday), hence not at standup. yesterday I worked on taking instance AZ from volume attachments
<wallyworld> axw: no worries, figured that's where you were
<axw> I think I've got it working with AWS
<wallyworld> awesome
<wallyworld> axw: we also need to look at bug 1692729
<mup> Bug #1692729: juju add-storage doesn't always grab ebs volumes on aws <juju:New> <https://launchpad.net/bugs/1692729>
<axw> weird
<axw> ok
<hml> wallyworld: can you take a look at the updates to: https://github.com/juju/juju/pull/7354
<wallyworld> hml: yep
<hml> wallyworld: ty
<axw> wallyworld: is https://bugs.launchpad.net/juju/+bug/1692729 a release blocker?
<mup> Bug #1692729: juju add-storage doesn't always grab ebs volumes on aws <juju:Triaged by axwalk> <https://launchpad.net/bugs/1692729>
<wallyworld> axw: i think so, given a stakeholder found it and it affects conjure-up
<axw> ok
<axw> I'll finish up what I'm doing and look at that next
<wallyworld> hml: LGTM, but with some tweaks to the test structure
<hml> wallyworld: looking
<thumper> babbageclunk: I need a rubberduck... or teddybear
<thumper> babbageclunk: can I use you?
<babbageclunk> thumper: yay, of course!
<thumper> babbageclunk: stuff it is
<wallyworld> babbageclunk: here's a PR with an upstream patch if you have a moment later https://github.com/juju/juju/pull/7375
<babbageclunk> wallyworld: lgtm's
<wallyworld> ty
<babbageclunk> oops - lgtm'd
<wallyworld> babbageclunk: let's hope it fixes one of the source of races
<babbageclunk> <fingers crossed emoji>
 * thumper gets on filing bugs
<thumper> jam: got 5 minutes for an update
<thumper> ?
<jam> thumper: sure
<thumper> jam: 1:1
<jam> joining
<mup> Bug #1629113 changed: Juju deploy wordpress fails with MaaS <juju-core:Expired> <https://launchpad.net/bugs/1629113>
<wallyworld> axw: when you get a moment, here's a small PR to fix some CI test races https://github.com/juju/juju/pull/7376
<axw> wallyworld: left some comments
<wallyworld> ok, ta
<wallyworld> axw: i replied, see what you think
<axw> wallyworld: "The race is because a test patches the Period value. This is done outside the watcher and is normally ok, except when race test are run."  <- if it's outside the watcher, how is there a race?
<wallyworld> axw: because other tests use the watcher
<wallyworld> so one test patches the global var
<wallyworld> and another test is using a watcher which tries to access that var
<wallyworld> that's my theory. i could repro easily. the PR fixes it
<axw> wallyworld: the tests within a package are run sequentially. it sounds an awful lot like the test is doing dodgy things, and we should fix that too. the code change is fine though.
<wallyworld> even with race?
<wallyworld> it only shows up with race testing
<axw> wallyworld: it won't ever do anything particularly bad, just maybe affect the timing of unrelated tests
<wallyworld> the test doesn't do anything too bad on the surface. the Test() itself runs s.PatchValue(&watcher.Period, 200*time.Millisecond)
<wallyworld> and then it starts up stuff
<axw> which test is it?
<wallyworld> TestDowngradeOnMasterWhenOtherControllerDoesntStartUpgrade
<wallyworld> core issue looks like a problem with clean up
<wallyworld> maybe watcher isn't being stopped
<wallyworld> or stops when the cleanup runs to reset the patched value
<axw> wallyworld: that test is totally doing the wrong thing. once the test has started, State (thus the watcher) will already have been created
<axw> wallyworld: I think the PatchValue should be moved to SetUpSuite
<wallyworld> hmmm, i think that's a good point. so it is patching the value after the watcher starts and tries to rely on that
<wallyworld> yeah, moving it would be better
<axw> wallyworld: changing the code to pass the period in is a step in the right direction though IMO
<wallyworld> yeah, agreed
<axw> wallyworld: i.e. away from using globals
<wallyworld> yup
<wallyworld> i'll move the patch code also
<axw> wallyworld: thanks. just wanted to make sure the test wasn't doing anything too heinous
<wallyworld> np, thanks for being thorough
<wallyworld> thumper: can you attach relevant log snippets to the txnlog bug?
<thumper> sure
<babbageclunk> wallyworld: Can you take another look at https://github.com/juju/juju/pull/7369? It's changed a bit...
<wallyworld> sure
<babbageclunk> thanks
<wallyworld> babbageclunk: reviewed, thanks. i'm still quibbling about a set and a slice in the watcher. i like the uniter facade rework. oops, one thing i forgot - we should really catch and handle any CodeNotImplemented errors in the api uniter client. there's not much that can be done except to log a nice message and error out so the uniter restarts with the expectaction that the server will eventually start running the right version (I think)
<wallyworld> or maybe it's not an issue since the controller always gets upgraded first
<wpk> a one-liner: https://github.com/juju/juju/pull/7377
<babbageclunk> wallyworld: yeah - I think you're right about the slice+set. Doesn't make sense on the scale of relations we're talking about.
<mup> Bug #1692866 opened: /snap/bin not in path for juju run/juju ssh <juju-core:New> <https://launchpad.net/bugs/1692866>
<babbageclunk> wpk: around? I've changed the Uniter API, and needed to change some of the versioning there. https://github.com/juju/juju/pull/7369
<babbageclunk> wpk: Could you take a look and confirm that I've handled your NetworkConfig/NetworkInfo change right?
<babbageclunk> wpk: I've got NetworkConfig only on the v4 API and NetworkInfo only on v5. Make sense?
<wpk> Is that the right PR? I don't see any Network{Info,Config} related things there
<wpk> Ah, OK, NOw I see
<babbageclunk> wpk: Well, none of it's about those, but if you look at the changes to apiserver/uniter/uniter.go
<babbageclunk> wpk :)
<wpk> Ah, the diff wasn't loaded
<wpk> IMHO it's cleaner to have a 'clean' object on top (like UniterAPI) with everything implemented and make all the workarounds in 'older' stuff, so that if we want to get rid of old one we can simply, well, get rid of it
<wpk> so UniterAPI is complete V5, and UniterAPIv{somethingolder} has the newer functions e.g. masked out
<wpk> But that's a political decision :)
<wpk> (we can even have the old functions in separate file, for cleaniness sake)
<babbageclunk> wpk: Well, there's no way to mask out functions properly if you embed the pristine one, so the only way to do that would be not to embed but to explicitly delegate all the ones we want on each version.
<wpk> Why there's no way?
<wpk> From API standpoint if you create same-named function with 'wrong' signature it won't get published
<babbageclunk> wpk: ? I'm not sure that's right.
<wpk> func (u *UniterAPIV3) NetworkInfo(_, _ struct{}) {}
<wpk> this one
<wpk> it masks NetworkInfo from UniterAPIV3
<wpk> since we don't publish functions with 2 arguments
<babbageclunk> wpk: Hmm, didn't know about the 2-argument thing.
<wpk> the rules are in rpc/rpcreflect/type.go, newMethod
<babbageclunk> wpk: Ok, I'll check that out in the morning. Thanks.
<wpk> function can have 0 or 1 argument, and 0 or 1 return values + optional error
<wpk> (so (), (error), (x), (x, error), but not (x,y) )
<babbageclunk> wpk: Yeah, makes sense. Ok, I'll do that, thanks!
<wpk> Good night :)
<babbageclunk> wpk: 'night!
<mup> Bug #1692866 changed: /snap/bin not in path for juju run/juju ssh <juju:Incomplete> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1692866>
<thumper> I'm at that phase of bug diagnostics where I'm wondering: how can this possibly be happening
<babbageclunk> thumper: oh I love that bit
<thumper> babbageclunk: I'm hitting the same code path from two different directions, one succeeds and one fails
<thumper> WTF
<thumper> babbageclunk: got 5 minutes to be a teddy bear?
<babbageclunk> thumper: sure!
#juju-dev 2017-05-24
<axw> thumper: when you have a minute can you please email me those sprint dates?
<axw> got another wedding coming up...
<balloons> burton-aus, veebers, is this ready to land? https://github.com/juju/juju/pull/7373
<thumper> axw: sure
<veebers> balloons: should be, haven't checked update yet will do shortly
 * thumper has a bad feeling that resource migration hasn't ever worked fully
<babbageclunk> hey thumper, I forgot to mention - rogpeppe was trying to find something out from reviewboard the other day and was pretty bummed that it was gone.
<thumper> ok
<babbageclunk> thumper: Would it be hard to bring back? It's definitely useful to try to understand some design decisions.
<thumper> yes
<thumper> it is gone gone
<thumper> AFAICT
<babbageclunk> thumper: :( He'll be very unhappy. (He said to say that.)
<thumper> ok
<thumper> I'm not even sure who to ask about any form of backup
<babbageclunk> thumper: What was the rationale for stopping it? Cost?
<thumper> maintaining yet another service
<thumper> veebers: do you know of a simple charm that has a resource in the charmstore?
<axw> babbageclunk: nice work on the subordinates bug :)
<babbageclunk> axw: Thanks!
<axw> gnarly one
<veebers> thumper: not that I'm aware off, partly why we created the simple resource charm for testing
<babbageclunk> axw: it was definitely fiddlier than I'd expected.
<wallyworld> axw: sorry about delay. lgtm with a suggestion to improve an error message
<axw> wallyworld: thank you
<axw> wallyworld: how's this? "cannot create instance with placement %q, as this will prevent attaching EBS volumes in zone %q"
<wallyworld> sgtm
<thumper> ugh
<thumper> resources are so fucked
<anastasiamac> thumper: yes :)
<anastasiamac> thumper: r u geenrally looking? do u want bug 1627127 too?
<mup> Bug #1627127: resource-get gets hung on charm store <cdo-qa> <cdo-qa-blocker> <juju:In Progress by anastasia-macmood> <https://launchpad.net/bugs/1627127>
<thumper> no
<thumper> I'm looking at the migration bugs
<anastasiamac> thumper: ack
<anastasiamac> thumper: altho we r both essentially lloking at placeholder resources vs download failures (atm both cases are identified by size 0 which kind of sucks...)
<thumper> um...
<thumper> no
<thumper> that's not what I'm looking at
<thumper> I'm completely avoiding sending placeholders
<anastasiamac> thumper: \o/
 * babbageclunk goes for a run
<thumper> well poo
<thumper> forgot to make install
 * thumper offline for abit
<wpk> https://github.com/juju/juju/pull/7383 a large one, so it'll probably need two reviewers
<wpk> (add functionality, remove code, http://www.quickmeme.com/img/13/1324dfd733535e58dba70264e6d05c9b70346204d2cacef65abef9c702746d1c.jpg )
<wallyworld_> babbageclunk: the recent subord PR introduced a race http://reports.vapour.ws/releases/5275/job/run-unit-tests-race/attempt/2764
<wallyworld_> could you take a look at fixing?
<babbageclunk> wallyworld_: ah, d'oh - yeah fixing now
 * thumper again types juju instead of git
<thumper> alexisb: oh hai
<alexisb> heya thumper!
<thumper> what brings you back to your old stomping grounds?
 * thumper wonders if this was an auto-join...
<alexisb> :) yes it was an auto join
<babbageclunk> wallyworld_: chasing that race - the only way I can see it happening is if the lifecycleWatcher returns nil from Err() even though Changes() is closed.
<thumper> alexisb: you are always welcome :)
<wallyworld_> babbageclunk: yeah, could be, not looked too closely
<alexisb> I dont know some shady folks hang about around here
<thumper> :D
<alexisb> I even hear some of them are a kangaroo short of a top paddock ;)
<thumper> haha
<alexisb> <cough> wallyworld_ <cough>
<wallyworld_> alexisb: ha de ha ha
 * babbageclunk snickers
<thumper> oh ffs
<thumper> babbageclunk: didn't you have some tests recently where you had to create some resources?
 * thumper headdesks
<babbageclunk> thumper: no, subordinates
<thumper> ah
<thumper> nm, I've got it sorted now
<thumper> just more fucking hoops
<babbageclunk> yay!
#juju-dev 2017-05-25
 * thumper headdesks some more
<thumper> why...
<thumper> WHY???
<thumper> testing resources written to the db are invalid when read back out
<thumper> using the resourcetesting package
<thumper> FFS
 * thumper backs up before he gets more angry
<wallyworld_> babbageclunk: free if you still want to talk
<babbageclunk> wallyworld_: sorry, was just grabbing some lunch. now good?
<wallyworld_> sure
<babbageclunk> wallyworld_: in standup?
<thumper> someone seriously needs to rewrite resources in the standard way
<thumper> I've spent over two hours trying to get some test data in
<thumper> and still no closer
<anastasiamac> thumper: totally agree \o/ who and when?
<babbageclunk> thumper: fell like an easy review to take your mind off resources? https://github.com/juju/juju/pull/7385
 * thumper takes a look to see if it will make me feel better
<thumper> babbageclunk: wallyworld_ has approved it already
<babbageclunk> gah, missed that!
<thumper> phew
<thumper> finally got tests for this
<wallyworld_> thumper: i've pushed changes to https://github.com/juju/juju/pull/7382 once you get a chance
<wallyworld_> i commented on the upgrade issue in the pr comments
<thumper> ok, I'll trade you once I've finished writing up the PR for mine
<wallyworld_> ok
<wallyworld_> also need to look at CI failures
<axw> wallyworld_: I've added another commit to my PR, wouldy ou please take a look at https://github.com/juju/juju/pull/7381/commits/e1972615d535b01efd43151acafe50ed4af0f84a
<wallyworld_> sure
<thumper> wallyworld_: https://github.com/juju/juju/pull/7387
<wallyworld_> ok
<wallyworld_> axw: can we factor out a common function to fill in missing storage constraints, shared between importer and upgrader
<axw> wallyworld_: it would be non-trivial (one's entirely in-memory, one's fetchign from mongo; one iterates over all volumes, one queries specific ones, one normalises info/params, one doesn't...)
<axw> wallyworld_: so I don't think it'd be a net win
<wallyworld_> righto, just thought i'd check, didn't look at the upgrade code, just recalled from yesterday
<wallyworld_> axw: lgtm then
<axw> wallyworld_: I'm not happy about having to write it twice though. I wonder if we could do upgrades by doing an export+import when the schema changes, rather than what we do now
<axw> I was going to add a tech board itemm
<wallyworld_> yeah, let's discuss
<wallyworld_> thumper: lgtm, nice testing notes
<wpk> thumper: ping?
<thumper> wpk: pong... thought I closed this
<thumper> whazzup?
<wpk> thumper: About the discoverspaces API - as I understand (and it's very probable that I'm wrong) it's only used by controller so it'd be impossible to have discoverspaces worker (from old controller) to connect to discoverspacesAPI-less controller
<thumper> wpk: I was wondering that. double check with jam. if it is the case that it is only used by the controller itself, then we can remove it
<wpk> -	if !authorizer.AuthController() {		
<wpk> ï¿¼ -		return nil, common.ErrPerm		
<wpk> ï¿¼ -	}
 * thumper is closing out for the day now
<wpk> bye
<thumper> ok
<thumper> night all
<wpk> https://github.com/juju/juju/pull/7392
<wpk> jam: ^^^
<thumper> veebers: any idea why the merge bot is failing with  Error: retrieving gpg key timed out. ?
<veebers> thumper: not fully yet, we'll have a look
<thumper> that's ok, I know you are busy
<veebers> thumper: odd, babbageclunk was able to land something
<veebers> (just after you)
<thumper> weird, must be some timing issue somewhere
<wallyworld> thumper: standup?
<wallyworld> babbageclunk: could you also review that cmr feature branch pr today?
<babbageclunk> wallyworld: oh yes! Sorry, completely forgot about it.
<wallyworld> np
#juju-dev 2017-05-26
<anastasiamac> thumper: axw: application name bug 1693588
<mup> Bug #1693588: Application name requires alphanumeric character after a hyphen <sts> <juju:New> <https://launchpad.net/bugs/1693588>
<wallyworld> axw: if you did want to discuss the uuid / folder length issue, let me know. my personal preference is to preserve model names, truncate the uuid a bit, and modify the filtering to transform the uuid filter terms before matching
<wallyworld> we still store the full uuid as tags IIANM
<wallyworld> but those tags aren't user visible
<wallyworld> so it's just the folder names
<wallyworld> and uuids mean far less to the user than midel names
<axw> wallyworld: doing that will make the provider more complicated for negligible gain, since this only happens with really long model names. can talk more when I'm back from school
<wallyworld> ok
<babbageclunk> Does anyone know why we say this in the docs? https://jujucharms.com/docs/2.0/authors-subordinate-services#caveats
<babbageclunk> It's true that you can't remove a subordinate unit directly, but I'd have thought that's because we don't have any way to say in the model "this principal unit shouldn't have an associated subordinate".
<babbageclunk> Is there also some problem with stop hooks?
<axw> wallyworld: back, do you want to chat in 1:1?
<wallyworld> axw: ok
<thumper> babbageclunk: I think the docs are to work around the bug you jsut fixed
<babbageclunk> thumper: weird. What's the process to get the docs fixed? Just make a PR on the docs repo?
<thumper> yep
<thumper> wallyworld: OMFG that thing that isn't set by default, is only ever set to one value and never changed
<thumper> FFS
<wallyworld> fark
<wallyworld> kill it with fire
<thumper> I have a horrible feeling that doing what I'm about to do will create an import loop
<thumper> may be more drastic
 * thumper fetches flamethrower
 * thumper dons fireproof gear
<thumper> FFS, almost ready to back away from this clusterfuck
 * thumper rants and raves
 * thumper burns some more code
 * thumper has finished burning source files, now to the tests
<axw> wpk: thanks for the review
<anastasiamac> wpk: our friday standup conflicts with my school pickup. could we move it forward by an hr? jam fyi
<rogpeppe> in case anyone is around for a review, here's some refactoring of the API connection logic: https://github.com/juju/juju/pull/7407
<redir> There was a fire on the next blick here yesterday.
 * redir blames thumper
#juju-dev 2017-05-28
<thumper> babbageclunk: ping
<thumper> babbageclunk: reping
<thumper> veebers: also ping
<veebers> thumper: hey o/ what's up?
<thumper> veebers: was looking at the failing CI runs for 2.0 and 2.1 branches, got any insight?
<veebers> thumper: haven't looked indepth since Friday (due to releases). We where having some odd failures (some runs seems to be changing the ssh on disc? odd). I can take a more indepth this morning
<thumper> veebers: although to be honest, develop isn't looking so hot either
<thumper> veebers: prioritise develop checking over the others...
<thumper> 2.0 and 2.1 are interesting, but develop is vital
<veebers> thumper:ack
<babbageclunk> thumper: sorry, on the phone
<babbageclunk> thumper: ok off now
<veebers> thumper: FYI so far the ones that are obviously infra issues: Maas 2-1, test needs updated. We're trying to exclude a space that doesn't exist (not sure if we should just create the space, or remove that constraint from test.) s390 and arm64 machines can't access streams, this is fallout from moving away from canonistack swift.
<veebers> thumper: I aim to have a fix for s390 today, and maybe amd64 today too
<veebers> thumper: those failures will be affecting 2.0, 2.1 and devel
<veebers> thumper: oh, and prodstack jobs are failing due to resource quota issues, I'm on that as well (sorry for spamming you ^_^ )
<thumper> veebers: ack, all good
<thumper> babbageclunk: quick hangout?
<babbageclunk> thumper: sure
<veebers> thumper, babbageclunk: For juju to use a http proxy when curling the agent (during bootstrap) how is that set in the config. It looks like we're doing it wrong in the test
<thumper> not sure, would have to look it up
<babbageclunk> thumper: I suddenly got a mental image of a yak shaved like a poodle.
<wallyworld> babbageclunk: small review? https://github.com/juju/juju/pull/7410
<babbageclunk> wallyworld: sure
<babbageclunk> wallyworld: LGTM!
<wallyworld> tyvm
#juju-dev 2018-05-25
<kjackal> Hello  do we know who updates these: https://streams.canonical.com/juju/images/releases/streams/v1/com.ubuntu.cloud.released-aws.json
<rick_h_> kjackal: so those would be between the juju team and the cloud teams. What's up?
<kjackal> thanks rick_h_ the AMI centos images seem to be outdated. Kudos to Dmitrii-Sh who traced this
<rick_h_> kjackal: k, can you file a bug and I'll see if we can figure out how those get updated.
<kjackal> is this a bug against juju on lp?
<rick_h_> kjackal: yes please
<kjackal> will do thanks
<kjackal> rick_h_: Thank you https://bugs.launchpad.net/juju/+bug/1773391
<mup> Bug #1773391: Pointing to outdated CentOS images on AWS <juju:New> <https://launchpad.net/bugs/1773391>
<rick_h_> kjackal: <3 ty will see how to get things updated.
