#juju-dev 2013-08-12
<davecheney> thumper: ping
<davecheney> oh dear, no thumper
<davecheney> has anyone tried runnig multiple local providers on the same machine, under _different_ user accounts ?
<axw> davecheney: no, but it's *meant* to work according to various comments int he code... have you tried it and it doesn't?
<davecheney> it does not
<davecheney> it may work for multiple local environments all owned by a single user
<davecheney> but does not work when multiple users try to each have a local environment
<axw> davecheney: I would've thought hte opposite :)   it namespaces things on the username
<davecheney> axw: there is only one mongodb
<davecheney> it binds to 0.0.0.0
<davecheney> and always binds to the same port
<davecheney> that is not configurable
<axw> true - there's probably a TODO in there someone
<davecheney> don't forget, we're using 1.12
<davecheney> did somethig change in the last 2 weeks ?
<axw> sorry, don't know
<axw> uhh actually, the port is configurable
<axw> state-port
<axw> un moment
<axw> at least on the service registraiton side anyway
<davecheney> ahh, it's in config/config
<davecheney> axw: it doens't do shit
<davecheney> state-port doens't do anything
<axw> davecheney: you bootstrapped anew? I just did, and it worked. I'm on trunk tho
<davecheney> it is checked, validated then ignored
<davecheney> axw: as two different users ?
<axw> davecheney: well no, but it picked it up, created the service with that port, and connected to it
<axw> davecheney: I'll create a new user and try. Should be able to set state-port, storage-port and shared-storage-port to unique values and have it work tho
<davecheney> yeah, that is what I am trying
<davecheney> YES
<davecheney> it worked!
<davecheney> thank heavens for that
<axw> :D
<davecheney> shitballs that was a close one
<davecheney> we only have one machine to demo on
<bigjools> davecheney has been learning swearing from Debra Morgan
<jam> hi all
<davecheney> SUCK BAG!
<axw> jam: hiya
<jam> davecheney: I'm so sorry :)
<jam> oh wait, ECONTEXT :)
<davecheney> lagging lag is laggy
<jam> wallyworld: poke. Just checking if you're feeling up to a 1:1 today
<davecheney> what does 'no CA certificat in environment configuration' mean ?
<davecheney> never mind
<davecheney> ok, next problem
<davecheney> how to I change the api server address
<davecheney> ?
<davecheney> api-port I guess
<wallyworld_> jam: not sure if you saw my last message, i got disconnected, i'm ok for 1:1
<jam> wallyworld_: I didn't. Thanks
<davechen1y> is there a way to disable the automatic apt-get update behavior ?
<davechen1y> ie, when we start a machine, it always does apt-get update -y
<jam> davechen1y: it is one of the flags we past to cloud-init
<jam> davechen1y: cloudinit/option.go apt_update: true
<jam> sorry, apt_update, yes, yes
<jam> wallyworld_: I'm there whenever you're ready
<wallyworld_> ok
<davechen1y> probably can't change it without doing a release
<wallyworld_> jam: one sec, i need to install the plugin, since it was lost when my system died
<jam> davechen1y: I'm pretty sure it was requested (and intentional) behavior. Given we even had a work around when the apt_update flag was broken.
<jam> wallyworld_: np
<davechen1y> jam: fair enough
<davechen1y> thanks
<davechen1y> does anyone know if it is possible to remove a single subordinate unit
<jam> davecheney: "remove a single subordinate unit", you mean I want X on all units of Y, except for machine-Z ?
<jam> wallyworld_: did you see bug #1211147
<_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages <juju-core:Triaged> <https://launchpad.net/bugs/1211147>
<rogpeppe> mornin' all
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: - | Bugs: 4 Critical, 85 High - https://bugs.launchpad.net/juju-core/
<dimitern> mgz: hey
<dimitern> https://codereview.appspot.com/12698043/ ?
<mgz> is that one of the ones I have a tab open for already...
<mgz> yup.
<dimitern> well, I'm waiting from friday :)
<rogpeppe> dimitern: looking
<dimitern> rogpeppe: thanks
<noodles775> I'm just doing a small refactor for a bug, but am looking at environs/state.go:BootstrapState.StateInstances - why is that a slice? (Are there multiple state instances for some providers, or for the future?)
<noodles775> And more specifically, is it OK/intended that StateInfo will return as soon as it has the hostname of one state instance?
<mgz> dimitern: a couple of queries
<dimitern> mgz: well I could've called it canModifyOrRead - because it's the same check, but once it's for reading, once for writing
<mgz> I don't follow
<dimitern> mgz: what's not clear?
<mgz> the function that returns is actually a can-anything-happen, despite being called getCanRead?
<mgz> the other functions call the returnvalue canRead but otherwise use it identically
<dimitern> some functions, like SetPublicAddress are "write" requests, while others like SubordinateNames are "read" requests
<mgz> right.
<dimitern> both have the same check - "are you the owner of that entity?"
<mgz> but we call the function getter for that check "getCanRead"?
<dimitern> I agree the name should probably be canAccess or something
<mgz> having the return value sometimes named canRead and sometimes named canModify isn't really helpful
<dimitern> I named the returned function after it's actual role in the method - read or modify
<dimitern> its*
<dimitern> will it be better if I call it getCanAccess and canAccess respectively?
<mgz> seems like an improvement to me
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<rogpeppe> dimitern: +1 to canAccess
<rogpeppe> wallyworld_: ping
<jam> mgz: /wave
<mgz> hey jam!
<rogpeppe> jam: welcome back!
<jam> rogpeppe: hey
<dimitern> mgz, rogpeppe: https://codereview.appspot.com/12748043
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe, mgz: thanks!
<dimitern> rogpeppe: I don't really get your comment here https://codereview.appspot.com/12748043/diff/1/state/api/params/internal.go#newcode35
<rogpeppe> dimitern: oh, what doesn't make sense?
<dimitern> rogpeppe: aren't we going to introduce a backwards-incompatible change that way?
<rogpeppe> dimitern: i don't think so
<dimitern> rogpeppe: changing all results with Error to be omitempty
<rogpeppe> dimitern: how would it be backwardly incompatible?
<rogpeppe> dimitern: you'll get exactly the same result, won't you?
<dimitern> rogpeppe: not in javascript
<rogpeppe> dimitern: nothing is using any of these calls from javascript
<dimitern> rogpeppe: having {Result: x, Error: null} is different from {Result: x}
<rogpeppe> dimitern: i don't see your point. none of the agent API calls is used by anything except Go.
<dimitern> rogpeppe: or anything that's relying on it to be JSON
<dimitern> rogpeppe: how about third-party agents?
<rogpeppe> dimitern: these calls are exclusively for our own use (currently anyway)
<rogpeppe> dimitern: we don't support third-party agents.
<dimitern> rogpeppe: and can you guarantee nobody else is using them?
<dimitern> rogpeppe: there were already some python helpers using the api
<rogpeppe> dimitern: yes, the client API is our only public API
<rogpeppe> dimitern: if anyone is using the agent API, they'd have to be parsing our agent config files, getting the password, and pretending to be a machine agent or whatever
<dimitern> rogpeppe: and that's going to be like that forever?
<rogpeppe> dimitern: not necessarily
<rogpeppe> dimitern: but it is now
<dimitern> rogpeppe: hmm.. ok, but istm this is a change that's not for this CL
<rogpeppe> dimitern: and so we don't care about non-Go backward compatibility in the agent API at the moment
<rogpeppe> dimitern: sure
<rogpeppe> dimitern: but you can at least do it for StringsResult
<rogpeppe> dimitern: well actually
<rogpeppe> dimitern: you don't even need to do it for StringsResult
<rogpeppe> dimitern: you can add the Error field
<dimitern> rogpeppe: ah.. ok I got you
<rogpeppe> dimitern: and even without omitempty it will still be backwardly compatible
<dimitern> rogpeppe: you're saying rename StringsErrorResults to StringsResults and add an Error field to StringsResult
<rogpeppe> dimitern: yes
<dimitern> rogpeppe: ok then, will do
<rogpeppe> dimitern: thanks
<dimitern> rogpeppe: https://codereview.appspot.com/12748043/ - looks good?
<rogpeppe> dimitern: looking
<dimitern> rogpeppe: just noticed a typo in StringsResults doc comment
<rogpeppe> dimitern: +1
<rogpeppe> dimitern: looks great, thanks
<dimitern> rogpeppe: cheers
<rogpeppe> dimitern: we could actually benefit in a few places from a way to induce an error talking to the underlying mongo
<dimitern> rogpeppe: oh, yeah
<rogpeppe> dimitern: one way to do it might be to make all the tests actually connect to mongo through an intermediate relay that the tests could close when desired
<rogpeppe> dimitern: but that would probably disrupt lots of other test machinery which expects to be able to talk to state after a test has completed
<dimitern> rogpeppe: or introduce hooks into the mgo connections?
<rogpeppe> dimitern: that only helps if the code you're trying to test is using the same *State that the test has access to
<rogpeppe> dimitern: the fact that State is using mgo is pretty much hidden outside of the state package.
<rogpeppe> dimitern: i guess we could have a global function, state.SetQueryError(err error) that sets the error returned when anything makes a mongo query.
<rogpeppe> dimitern: and similarly for transactions
<rogpeppe> dimitern: anyway, something to think on
<dimitern> rogpeppe: sounds plausable
<rogpeppe> dimitern: standup?
<dimitern> jam: ^^
<davecheney> reported on #juju, http://pastebin.com/FUri1dx3
<davecheney> any ideas ?
<rogpeppe> wallyworld_: any chance of a chat at some point?
<wallyworld_> rogpeppe: sorry i wasn't around before, was at soccer. did you want to catch up? give me 5?
<rogpeppe> wallyworld_: np
<rogpeppe> wallyworld_: i'll stay in this call for a bit; probably 10 minutes or so?
<davecheney> sounds like the OP has deployed to his maas nodes
<davecheney> then turned them off
<davecheney> ...
<wallyworld_> rogpeppe: start a new hangout if the other one is busy? https://plus.google.com/hangouts/_/e4864938922c332329d0d696d9cb67f50b0d04f4?authuser=1
<dimitern> rogpeppe, natefinch, mgz: next in line https://codereview.appspot.com/12756043/
<noodles775> Hi people, I've got 3 branches needing reviews if anyone has time - the first just needs feedback from rogpeppe, but the other two haven't been looked at yet:
<noodles775> https://codereview.appspot.com/12659043/ https://codereview.appspot.com/12755043/ https://codereview.appspot.com/12603047/
<rogpeppe> dimitern, noodles775: sorry, was in a call
<davecheney> question from the field
<davecheney> does anyone have an objection to me adding config keys to expose the AptProxy and AptUpdate/AptUpgrade items in cloudinit
<davecheney> ?
<davecheney> i can explain why we need such things, but not in this channel
<dimitern> davecheney: i think sidnei was working on something similar
<davecheney> dimitern: or'ly
<davecheney> i'll talk to im
<davecheney> him
<davecheney> we sort of need it for the thing that marco and I are doing this week
<davecheney> dimitern: what hours does sidney work ?
<davecheney> sidnei:
<dimitern> davecheney: he's usually around at this time or couple of hours later
<dimitern> davecheney: utc-3
<davecheney> dimitern: right o
<davecheney> i might hack something in anyway, we need it tomorrow
<davecheney> but i'll check with S
<dimitern> davecheney: i believe that's what i was referring to https://codereview.appspot.com/12143043/
<davecheney> fuck yeah
<davecheney> someone has done all the work for me
<davecheney> sweet
<dimitern> :D
 * davecheney goes to try it 
<davecheney> ohhh, interesting, he's changed the lxc provider as well ...
<davecheney> https://codereview.appspot.com/12143043/diff/34001/cloudinit/options.go?column_width=80
<davecheney> did this already land ?
<davecheney> it
<davecheney> it's there now
<dimitern> yes that one did land last week
 * davecheney smacks head
<davecheney> of course, we don't see (closed) on our CLs when they land
<dimitern> noodles775: looking the last two of yours
<dimitern> davecheney: yep, I only know it because I reviewed it and I can see it in my list
<dimitern> I have the habit of manually closing them when then land
<dimitern> noodles775: there was never more than one state instance
<dimitern> noodles775: there are plans for ha support, but nothing done yet
 * davecheney wonders if we should backport that to 1.12.1
<davecheney> naaah
<davecheney> it's a lot of work
<davecheney> and it isn't a *bugfix*
<dimitern> there's a linked bug, but not about that - about the local provider/lxc
<davecheney> kk
<dimitern> noodles775: and about hp cloud account, you can ask mgz or wallyworld_ - we have one you can use perhaps
<noodles775> dimitern: k, so BootstrapState.StateInstances is a slice just for the future plans? Do you want me to then s/WaitForFirstDNSName/WaitForDNSNames/ then (since there is only ever one state server, it'll leave the helper more general)
<noodles775> dimitern: Re the hp cloud account, that'd be great to test - but no problems if not (if someone can verify the branch, that'd be enough for now).
<dimitern> noodles775: still looking, will get back to you
<dimitern> noodles775: so, not sure why you need WaitForFirstDNSName at all?
<dimitern> noodles775: can you expand a bit?
<noodles775> dimitern: because that's what the existing code in trunk was doing - environs/state.go:StateInfo (or am I misreading)?
<noodles775> It polls until it gets the hostname of a (well, the really) state server.
<natefinch> mgz: gotta go for a bit, I'll be back around 15-20 past the hour
<dimitern> noodles775: yeah, it's written to accept more than one hostname, but in reality it's always one
<mgz> noodles775: which branch? ...eh, I'll just look at all three
<mgz> natefinch: sure, poke me when you're back
<noodles775> Thanks mgz (the last two, dimitern is looking now too)
<dimitern> noodles775: so WaitDNSNames is used by the providers internally
<dimitern> noodles775: to implement StateInfo
<noodles775> dimitern: Yeah, the fact that BootstrapState.StateInstances is a slice makes it hard to code otherwise (I mean, I could just take the 0th index, but...)
<rogpeppe> noodles775: i'm not convinced by https://codereview.appspot.com/12755043/
<mgz> ick, I'm not a fan of the dummy provider
<rogpeppe> noodles775: it sounds like a bug in the provider that HP cloud uses (openstack ?)
<noodles775> rogpeppe: Maybe, see jam's comment on the related bug too.
<noodles775> rogpeppe: but do you mean you wouldn't want to keep polling when the provider returns ErrNoInstances?
<dimitern> noodles775: yeah, the more I read it, the more I think the approach is not right
<dimitern> noodles775: ErrNoInstances is expected - they take time to start
<rogpeppe> noodles775: yeah. in fact, i wouldn't poll at all when calling Instances
<rogpeppe> noodles775: i don't think jam's comment is relevant
<rogpeppe> noodles775: as we're not calling AllInstances here
<rogpeppe> noodles775: (unless you mean a different comment
<rogpeppe> )
<noodles775> No, I meant that one - I hadn't checked the change referred to (only saw the comment this morning).
<rogpeppe> noodles775: i think the original code looks ok
<noodles775> So I'm a bit confused - dimitern, you're saying that ErrNoInstances is expected because they take time to start, but that means we'd want to keep polling there...
<noodles775> rogpeppe: It could be - without an HP account I didn't first reproduce the reported bug.
<dimitern> noodles775: yes, that's what we do
<mgz> noodles775: I think the key is this used to work, it's just a regression
<rogpeppe> noodles775: dimitern isn't quite right there
<mgz> we don't want new polling code to resolve the issue, just to work out what broke and undo it
<dimitern> rogpeppe: ok, care to explain better please?
<rogpeppe> dimitern: Environ.Instances is expected to poll when it can't find the instances
<rogpeppe> dimitern: and, at least in ec2, that's what it does
<rogpeppe> dimitern: so you'll only get ErrNoInstances or ErrPartialInstances once it's already polled for long enough to be certain
<dimitern> rogpeppe: hmm.. ok
<dimitern> rogpeppe: and the openstack provider does the same, no?
<rogpeppe> i do think that environs.LongAttempt is a smell
 * rogpeppe checks
 * noodles775 looks for recent changes adding ErrNoInstances
<rogpeppe> noodles775: it's quite possible that shortAttempt is too short in HP openstack
<dimitern> and that's a recent change
<noodles775> mgz: it's not new polling code - I'm just extending the errors that are allowed while polling in StateInfo.
<mgz> noodles775: I see new public functions in environs/polling.go
<noodles775> mgz: yep, but they're used in environs.StateInfo with the same (intended) functionality as was already there (I was just trying to make StateInfo both explicit and simpler, but may have failed).
<rogpeppe> i think there should probably be a separate package (environs/utils?) that exposes functions for the providers themselves to use. i think that having, for example, environs.Bootstrap (intended to be called by clients of environs) and environs.StateInfo (intended to be called back by providers) is confusing
<rogpeppe> noodles775: i don't see anything wrong with the current implementation of environs.StateInfo
<dimitern> that's a whole different talk I think rogpeppe
<rogpeppe> dimitern: i agree
<rogpeppe> dimitern: just an aside
<rogpeppe> noodles775: one other thing: what was the motivation behind the changes to the dummy provider?
<noodles775> rogpeppe: Yep, nothing wrong with it - I just thought that that was where we'd need to also allow ErrNoInstances to avoid this bug, but as you said, a better place may be in openstack/provider.go (the shortAttempt).
<noodles775> rogpeppe: The broken-error? To be able to test the behaviour of StateInfo (ie. that it still timed out if ErrNoInstances was raised, rather than passing the error up straight away).
<rogpeppe> noodles775: ah, i see.
<noodles775> (which ended up being encapsulated by WaitForInstances and so tested there)
<rogpeppe> noodles775: probably not necessary if we end up just raising openstack.shortAttempt
<dimitern> rogpeppe: perhaps not all of it, but just for that test?
<noodles775> rogpeppe: yep - I think the whole branch won't be necessary, but was a good learning exercise :-)
<rogpeppe> noodles775: :-)
<rogpeppe> dimitern: ?
<dimitern> rogpeppe: openstack.shortAttepmt
<rogpeppe> dimitern: the bug was encountered for real, not in a test
<rogpeppe> noodles775: do you remember just how long it *did* take before the command actually succeeded?
<dimitern> rogpeppe: hmm.. so the shortAttempt used by the code you mean, not the one in tests? aren't these two linked?
<rogpeppe> noodles775: the bug report shows that it took at least 17 seconds
<rogpeppe> dimitern: yes
<rogpeppe> dimitern: they're not really linked
<rogpeppe> dimitern: except in the live tests, i guess
<noodles775> rogpeppe: Yes - I've not seen the bug in action (don't have HP creds), so am goin on the 17seconds there (the LongAttempt is 3 minutes)
<rogpeppe> noodles775: i'd go for 20 seconds at least
<rogpeppe> noodles775: looks like HP's consistency is really very eventual :-)
<noodles775> lol
<noodles775> rogpeppe: OK, so I'll do a second (much simpler) branch which just updates the shortAttempt there :-)
<rogpeppe> noodles775: thanks. it seems a pity (if you've got a genuine error you'll have to wait for 20 seconds before it tells you) but i suppose that's the price we pay for distributed databases
<rogpeppe> i wonder if it might be useful to allow the timeout to be adjusted for different openstack providers
<dimitern> we need more live testing instead of making obscure tweakable settings like that
<sidnei> davecheney: there's a couple bugs i've uncovered about the apt-proxy stuff IRL mainly when the host is precise, i have a bug open and a plan to fix it.
<davecheney> sidnei: right
<davecheney> so we are using a raring host for precise machines
<davecheney> is that the bug ?>
<davecheney> rogpeppe: on a long enough timeline, HP will be eventually consistent
<sidnei> davecheney: nope, the one bug i found is taht if the host is precise, apt-config dump dumps the whole config without filtering, which is then shoved into /etc/apt/apt.conf.d/99proxy in the container
<rogpeppe> davecheney: is HP more tardy in general than other providers?
<dimitern> rogpeppe, mgz, natefinch: poke https://codereview.appspot.com/12756043/
<sidnei> where as in raring/saucy (havent checked others) it is filtered down to the args you provide after 'dump'
<davecheney> sidnei: ewww
<rogpeppe> dimitern: i will get there :-)
<dimitern> rogpeppe: thanks :)
<sidnei> davecheney: a separate issue that hazmat brought up is that it requires configuring the host to use apt proxy so it's not automatic. pyjuju was simply checking if port 3142 (apt-cacher-ng) is up and listening and then adding that as the apt proxy without any extra config.
<davecheney> sidnei: yeah, i wanted to add a config property to environs/config.go
<davecheney> err
<davecheney> environs/config/config.go
<davecheney> which exposed the cloud-init apt-proxy key
<davecheney> crude
<davecheney> but reliable
<sidnei> davecheney: that was one of the options i brought up on the list, but there were conflicting replies.
<davecheney> sidnei: well, this may not get merged, but I need something simple
<rogpeppe> noodles775: i think your fix in https://codereview.appspot.com/12603047 is a reasonable one, but i don't think we can do it
<noodles775> rogpeppe: Yeah, as per the description there, I wasn't happy with it. Do you have a better way forward, or what other issues do you see?
<rogpeppe> noodles775: currently it's valid to call "juju get myservice --debug"
<rogpeppe> noodles775: your change will break that usage AFAICS
<rogpeppe> noodles775: one possibility is to change the Command interface so that individual commands can request that flags are not allowed to be interspersed.
<sidnei> davecheney: im definitely +1 on an explicit option, be it in the environ config or as i've implemented and landed, by changing the host's apt config. while the automatic option like in pyjuju is better from 'i just want to get started using juju quickly' pov, it's also a bit... surprising.
<davecheney> sidnei: i'll try your patch tomorrow
<davecheney> i think we can change the host so it passes throught the right values
<noodles775> rogpeppe: yeah - that sounds simpler than the other suggestion I had there on the description.
<sidnei> tks, i'll put up a bug fix for precise shortly.
<rogpeppe> noodles775: for example, add an AllowInterspersedFlags() bool method
<rogpeppe> noodles775: to the Command interface
<noodles775> rogpeppe: great - I'll try that out. Thanks for the feedback.
<rogpeppe> noodles775: there are a couple of other possibilities too
<rogpeppe> noodles775: i haven't decided which i dislike least :-)
<rogpeppe> noodles775: if the suggestion above is 1)
<rogpeppe> noodles775: then:
<rogpeppe> noodles775: 2) create another interface: type NonInterspersedFlagsCommand interface {NonInterspersedFlags()} and make SuperCommand.Init test dynamically if the command implements that.
<rogpeppe> noodles775: 3) add another argument to cmd.Register
<rogpeppe> noodles775: the down side with 1) is that, though simple, it requires that method to be added to every single command.
<rogpeppe> noodles775: although...
<rogpeppe> noodles775: hmm, one mo
<rogpeppe> noodles775: ha, we've already got CommandBase! perfect.
<noodles775> yeah - I thought BaseCommand could do som...
<noodles775> :)
<noodles775> Excellent, that makes 1) the winner then right?
<rogpeppe> noodles775: yup
 * rogpeppe starts looking at dimitern's review
<natefinch> mgz: sorry, back finally. Kids make everything take twice as long.
<mgz> :)
<sidnei> trivial-ish review: https://codereview.appspot.com/12758044
<davecheney> sidnei: your apt-config based proxy is working well
<davecheney> it's makeing the hotel wifi much more apporachable
<sidnei> davecheney: good to know. see cl above for the precise fix.
 * davecheney looks
<dimitern> sidnei: looking
<davecheney> oh, i see what's happehned
<rogpeppe> dimitern: how about something like this for factoring out the endless duplication? http://paste.ubuntu.com/5977395/
<rogpeppe> dimitern: i think i can implement that in an hour or so
<rogpeppe> dimitern: and i think it applies to pretty much all our API calls
<dimitern> rogpeppe: and the implementation will basically be a lot of hardly readable reflection magic?
<rogpeppe> dimitern: well, at least it can be well tested and it will save us 1000s of lines of code and days of implementation time
<dimitern> rogpeppe: how about getCanX? it just seems too generic to be useful
<rogpeppe> dimitern: one mo, i'll paste an example of how i envisage it would be used
<dimitern> rogpeppe: we need at least one for Get and one for Set - they'll do different things
<dimitern> rogpeppe: maybe more that 2 will be needed
<rogpeppe> dimitern: really?
<rogpeppe> dimitern: why the difference?
<dimitern> rogpeppe: one returns something other than an error + optional error or bool to something, and the other, just an error and takes a bunch of args
<rogpeppe> dimitern: i don't see the difference. they both take some parameters and return some results
<dimitern> rogpeppe: how about providing a bunch of helpers + callbacks and do the rest inline?
<dimitern> rogpeppe: like prepareResult, checkAuth, processEntities
<rogpeppe> dimitern: here's how it might look for CharmURL, for example: http://paste.ubuntu.com/5977431/
<dimitern> rogpeppe: and customizing each such helper to work with different results/params
<dimitern> rogpeppe: that's sounds like forcing us to use reflection with it's performance issues
<rogpeppe> dimitern: i'd be interested to see a concrete proposal
<rogpeppe> dimitern: do you know that performance is a problem here?
<dimitern> rogpeppe: where's the auth in the example?
<rogpeppe> dimitern: in the func returned by getUnitFunc
<dimitern> rogpeppe: I'll need a more concrete example then, sorry
<rogpeppe> dimitern: ok, hold on
<rogpeppe> dimitern: here's a sample implementation of getUnitFunc in terms of the existing UniterAPI primitives: http://paste.ubuntu.com/5977456/
<dimitern> rogpeppe: also, fwiw I don't think this approach is any faster than what we have now, it'll just reduce code duplication at the expense of having to look at several places to see what's going on
<dimitern> rogpeppe: that'll work except for NotFoundErr, which needs to be converted to ErrPerm, but I begin to see your point
<rogpeppe> dimitern: reducing the number of lines of code from 26 to 7 seems like a pretty good win to me
<rogpeppe> dimitern: over however many API calls
<rogpeppe> dimitern: plus the fact that we can potentially reduce the amount of test code
<rogpeppe> dimitern: because we can be sure that the details of the bulk call are dealt with correctly
<dimitern> rogpeppe: how can we do that?
<rogpeppe> dimitern: at the moment there's a significant amount of logic in each bulk API call that needs to be tested
<rogpeppe> dimitern: if each bulk API call contains a single call to BulkCall, surely we can make our tests rely on that and test only that the
<rogpeppe> dimitern: call is made with the expected parameters.
<dimitern> rogpeppe: are you getting at not writing individual tests for each call?
<rogpeppe> dimitern: no, but at the moment each test needs to pass in a vector with several arguments in, each designed to test an aspect of the logic
<rogpeppe> dimitern: we'd just need to pass a single-element vector containing one argument, and check that the single expected result is returned
<dimitern> rogpeppe: yeah, and I'm sure we can do something about reducing the size of tests, without losing code coverage
<rogpeppe> dimitern: exactly
<natefinch> rogpeppe, dimitern: for what it's worth, I think code generation would make the code less error prone than it is now, and more readable than some really generic thing like the BulkCall Roger posted
<dimitern> natefinch: I was thinking of a way to declaratively generate both code and tests for the api calls
<rogpeppe> natefinch: i'm a bit reluctant to go down that road
<rogpeppe> natefinch: although if we've factored things out to use reflection, it would not be hard to change things to go that route in the future if desired.
<natefinch> rogpeppe: I'm reluctant to go down the road of writing a function that takes 4 empty interface arguments and needs 5 paragraphs of comments to explain it :)
<dimitern> rogpeppe: surely, generated code can be optimized to run faster than a generic all-in-one code relying on reflection magic
<rogpeppe> dimitern: performance isn't the issue here
<rogpeppe> dimitern: we haven't even started trying to optimise stuff yet
<rogpeppe> dimitern: beyond hand-wavy "bulk API calls will be faster" remarks
<dimitern> rogpeppe: using reflection several times for each call will surely degrade performance, don't you agree?
<rogpeppe> dimitern: sure
<rogpeppe> dimitern: so will using many features that we're using all the time
<rogpeppe> dimitern: currently for each API call we're making several round trips to mongo
<rogpeppe> dimitern: *that*'s the place to start optimising stuff
<rogpeppe> dimitern: a few reflection calls are neglible compared to a network round trip
<dimitern> rogpeppe: sure, but adding more less performant code on top of it won't improve things much in the long run
<rogpeppe> dimitern: first rule of optimisation: measure first
<dimitern> rogpeppe: exactly, that's why we need a proof of concept for both
<rogpeppe> natefinch: i agree with your reservations about the reflect call. but automatically generated code has its own down sides.
<rogpeppe> dimitern: i'm not convinced that macros are better than reflection in general
<dimitern> rogpeppe: macros?
<rogpeppe> dimitern: code that writes code
<dimitern> rogpeppe: well, code that's generated and compiled, rather than dynamically reflected at run-time
<natefinch> rogpeppe: definitely. Neither is ideal. But I think generated code has better tradeoffs.  the code it creates is a lot easier to read, understand and debug than reflection-heavy generic code
<dimitern> natefinch: oh, that's +1
<mgz> rogpeppe: does goamz use lbox workflow for landing things?
<rogpeppe> mgz: yes
<mgz> ta
<rogpeppe> natefinch: i'm not sure. i think that 50 lines of reflection magic to reduce code by 75% elsewhere can be worth it (think fmt.Printf for example)
<rogpeppe> natefinch: i'd be interested to see how the code might look using gocog, for example, though
 * dimitern has to step out for a bit
<natefinch> rogpeppe: I'll whip something up, so we'll at least have some real code to look at and complain about :)
<sidnei> rogpeppe: if you could review https://codereview.appspot.com/12758044/
<rogpeppe> sidnei: looking
<rogpeppe> sidnei: looking
<rogpeppe> sidnei: i mean, reviewed!
<dimitern> rogpeppe: hey
<rogpeppe> dimitern: yo
<dimitern> rogpeppe: how about to offer you a deal? :) let's keep that duplicated way until we finish the uniter client and server side, so we can actually refactor the uniter to use it
<dimitern> rogpeppe: then let's discuss how to minimize the duplication - reflection or code generation, i don't mind, let's discuss it then
<rogpeppe> dimitern: my aim was to reduce the amount of work you need to do now
<dimitern> rogpeppe: because now I have certain momentum and would like to keep it up
<rogpeppe> dimitern: keep on going
<dimitern> rogpeppe: it's no trouble really
<dimitern> rogpeppe: thanks!
<dimitern> natefinch, mgz, rogpeppe: I still need 2 reviews on https://codereview.appspot.com/12756043/
<rogpeppe> dimitern: oh yes, sorry, i got some way through, then distracted
<dimitern> no worries
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> mgz: ?
<mgz> dimitern: looking
<sidnei> rogpeppe: fixed, thanks for the review. shall i land?
<rogpeppe> sidnei: i'll just take a quick last look
<rogpeppe> sidnei: LGTM, thanks. sorry, i had an unpublished comment suggesting the use of the m flag; hope you didn't have to look too hard for it!
<sidnei> rogpeppe: no, was a google + two clicks away :)
<sidnei> spent more time figuring out why \s? didn't work :)
<rogpeppe> sidnei: FWIW golang.org/pkg/regexp/syntax is an easy quick ref
<rogpeppe> sidnei: why didn't it work?
<mgz> dimitern: reviewed, favour extraction to review the branch I have up please :0
<sidnei> because it's the wrong syntax, it should have been \s*
<rogpeppe> sidnei: ah, i hadn't noticed that
<dimitern> mgz: thanks - sure, can I have a link?
<sidnei> rogpeppe: although, hum the page at http://golang.org/pkg/regexp/syntax/ say it should work
<mgz> dimitern: 12752043 :)
<sidnei> "x?             zero or one x, prefer one" "x*             zero or more x, prefer more"
<rogpeppe> sidnei: what were you trying to match?
<dimitern> mgz: sneaky :)
<sidnei> rogpeppe: zero or more spaces, but i actually had two in the test i did.
<rogpeppe> sidnei: ah, so it's right that it didn't work
<sidnei> indeed
 * sidnei blames on timezone
<dimitern> mgz: reviewed
<rogpeppe> dimitern, natefinch: FWIW, here's the magic: http://play.golang.org/p/BzS1Kn8peV
<rogpeppe> dimitern: all the type inspection code can be factored out and done only once for each combination of types
<dimitern> rogpeppe: will look more carefully a bit later
<sidnei> rogpeppe: reminder that im waiting on your ack.
<rogpeppe> sidnei: no need to wait - I already gave you my LGTM
<rogpeppe> sidnei: thanks anyway :-)
<dimitern> rogpeppe, mgz: last of the unit ops https://codereview.appspot.com/12776043
<sidnei> rogpeppe: sorry i was looking for one in the cl and missed the one on irc. :)
<rogpeppe> sidnei: np
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> mgz, natefinch: second review?
<dimitern> please..
<mgz> dimitern: sec
<mgz> dimitern: done, alos responded to you comments on review
<dimitern> mgz: thanks!
<natefinch> mgz: back
<mgz> natefinch: hey, with you in a sec
<natefinch> mgz: no rush
<natefinch> arg... ubuntu, why do you hate me so?  I can get either the microphone or the headphones of my headset to work, but not both at the same time.  It's like whatever tab of the sound settings I'm on is what ubuntu decides will work, when I plug in the USB cord. :/
 * rogpeppe feels for natefinch
<rogpeppe> time to go
<rogpeppe> g'night all
<natefinch> rogpeppe: g'night
<rogpeppe> natefinch: see ya!
#juju-dev 2013-08-13
<marcoceppi> Where is the cloud-init file that juju pushes to machines placed?
<axw> marcoceppi: for lxc, /var/lib/juju/containers/<container>/cloud-init
<marcoceppi> axw: I found them in /var/lib/cloud/seed/nocloud-net, let me check that path too
<axw> marcoceppi: sorry, I'm looking on the host here
<marcoceppi> axw: ack, np. Thank for the info!
<axw> nps
<davecheney> can anyone help with a tmux session ?
<davecheney> question
<davecheney> using tera term
<davecheney> the tmux bottom line causes the terminal to scroll one line every second
<davecheney> any suggestions ?
<jam> davecheney: futz with $COLUMNS ?
<davecheney> jam: inside the debug-hook ?
<jam> davecheney: I don't really know, I would have said trying it in the container before starting the tmux session. It sounds like tera disagrees with 'write to the end of the line' as off-by-one
<davecheney> jam: yeah
<jam> which is why some people say you should code to line 79 because sometimes the terminal wraps the final \n. :)
<davecheney> i don'
<davecheney> i don't know if its fixable right now
<davecheney> i might have to twiddle the tmux conf
<jam> davecheney: can you just use xterm, etc ?
<davecheney> jam: we are using xterm
<davecheney> well, tera says it's xterm
<jam> (10:09:03) davecheney: using tera term
<jam> davecheney: right, tera is lying and claiming its xterm, but then treating screen width differently (from what you've said at least).
<davecheney> i've tried vt100
<davecheney> vt32
<davecheney> vt52
<davecheney> etc
<jam> have you tried a terminal other than tera ?
<davecheney> jam: not an option sorry, we're in an SOE
<davecheney> standrard operating environment
<davecheney> which lacks freedom
<davecheney> jam: tmux thinks the window is one character larger than it claims
<davecheney> wider
<jam> davecheney: my guess is that tmux writes " "*39 + "\r", and tera thinks that the "\r" needs to wrap.
<jam> I haven't found ways to tweak either tera or tmux in my googling.
<davecheney> jam: thanks
<davecheney> let me try one thing
<jam> Windows shells tend to do the same thing wrong (which is why in our bzr code, we always write to COLUMNS-1, though some terminals let you write all the way to COLUMNS)
<davecheney> jam: any idea where the default ubuntu branded tmux conf lives ?
<jam> davecheney: http://askubuntu.com/questions/192401/where-is-the-default-tmux-conf-file-located
<jam> says there isn't one
<jam> but there are examples
<jam> in /usr/share/doc/tmux/examples
<davecheney> jam: where does debug hooks get it's magic one from ?
<davecheney> is this byobu ?
<davecheney> ah, it is
<axw> davecheney: it'll use ~/.tmux.conf if it exists
<axw> on first execution, that'll get populated with "source /usr/share/byobu/profiles/tmux"
<axw> source-file..
<davecheney> fuck
<rogpeppe> mornin' all
<rvba> jam: mgz:  Hiâ¦ would any of you be available to update gwacl on the landing bot's machine?
<rogpeppe> fwereade: yo!
<fwereade> rogpeppe, heyhey
<rogpeppe> fwereade: how's tricks?
<fwereade> rogpeppe, not bad, had a lovely peaceful weekend
<fwereade> rogpeppe, and yourself?
<rogpeppe> fwereade: i had a lovely weekend full of frenetic outdoor activity :-)
<rogpeppe> fwereade: and in general things are ok
<wallyworld_> jtv: are you working on code to call getImageBaseURLs() on the azure provider?
<rogpeppe> fwereade: current area of discussion is how we might make the bulk API call code a little less... bulky
<jtv> wallyworld_: not working on anything in the provider ATM, why?
<rogpeppe> fwereade: one possible approach is this: https://codereview.appspot.com/12845043
<rogpeppe> fwereade: natefinch has another thought, which you might have seen (automatic code generation)
<jtv> wallyworld_: the simplestreams code is already working for Azure, apart from a few limitations (images for any location only available in West US; only daily saucy images actually functional yet)
<wallyworld_> jtv: i'm doing some refactoring to extrude into a common function the url gathering which augments the base url with provider specific ones (if the provider supports such an operation)
<wallyworld_> so the azure function can be deleted
<jtv> wallyworld_: that sounds nice... will it grab both the daily and the released images?
<wallyworld_> jtv: the current function just returns the DefaultBaseURL - i wasn't going to change that behaviour
<wallyworld_> how is simplestreams working if nothing calls that function?
<wallyworld_> for azure
<jtv> wallyworld_: argh, another piece of dead code then!  We just use the same global variable that that function returns.
<fwereade> rogpeppe, interesting -- it's one of those situations that makes me weep for the lack of generics really
<jtv> Sometimes I find myself wishing for a tool to warn about dead code in Go.  Some kind of lint checker.
<wallyworld_> the code should have been calling that function :-)
<jtv> wallyworld_: feel  free to change it!
<wallyworld_> ok, i'll work on it, thanks :-)
<rogpeppe> jtv: there might have been something like that added to go vet recently
<jtv> rogpeppe: thanks!  Worth reading up on then.
<rvba> wallyworld_: sorry to bother you with thatâ¦ but do you have access to the landing bot?  I need gwacl to be updated there.
<wallyworld_> rvba: yes, sure
<rvba> wallyworld_: thanks a lot.
<rogpeppe> fwereade: it's this kind of repetition which was the reason i structured the API the way I did in the first place (so that the magic was all in one place, in the rpc package)
<jtv> rogpeppe: not seeing anything in the godoc for "go vet."  There is some stuff that might be nice to make more routinely accessible though.
<rogpeppe> jtv: on tip?
<jtv> The PPA.
<wallyworld_> rvba: now on rev 217. correct?
<rogpeppe> jtv: ah, it hasn't even landed on tip yet: https://codereview.appspot.com/12507043/
<rvba> wallyworld_: perfect, thanks again.
<wallyworld_> np, anytime
<fwereade> rogpeppe, yeah, I can see that limiting the spread of magic is generally a good thing
<rogpeppe> jtv: it only works with unexported identifiers though, so i'm not sure if it would flag your case above (what's the dead code you're referring to?)
<dimitern> fwereade, rogpeppe: fwiw we agreed yesterday to continue with the current approach until we have the uniter talking to the API first, and then proceed to refactor the server-side stuff - as we agree to do it - code generation or reflection
<jtv> rogpeppe: unused unexported method.  But if it's not landed yet, my individual case is sort of moot.  :)
<fwereade> dimitern, that sgtm, thanks
<fwereade> dimitern, I'm kinda suspicious of both approaches ;p
<rogpeppe> jtv: you could always apply the CL locally
<dimitern> fwereade: I have my doubts as well, but the most important thing for me now is not to lose momentum with the implementation
<fwereade> dimitern, +100
<jtv> rogpeppe: to detect a dead method we've already detected?  Thanks.  :-)
<rogpeppe> jtv: lol
<rogpeppe> jtv: but what about all the others we haven't? :-)
<jtv> Their time will come.
<fwereade> rogpeppe, does https://bugs.launchpad.net/juju-core/+bug/1205286 ring any bells?
<_mup_> Bug #1205286: charm directory permissions now more restrictive <juju-core:New> <postgresql (Juju Charms Collection):Triaged> <postgresql-psql (Juju Charms Collection):Triaged> <https://launchpad.net/bugs/1205286>
<rogpeppe> fwereade: not really, but i regularly campaign against the gratuitous use of non-0777 permissions when creating directories, usually to no avail
<noodles775> fwereade: I've updated the Makefile to add stable (now that the mongodb deps are there). If you can mark the MP to approved it should land: https://code.launchpad.net/~michael.nelson/juju-core/update-readme/+merge/179165
<fwereade> noodles775, thanks, will do
<noodles775> ta
<rogpeppe> fwereade: hmm, a naive grep doesn't show that as a problem here though.
<fwereade> noodles775, done
<fwereade> rogpeppe, yeah, I was just poking at it and wondering what the deal was
 * rogpeppe should really try out the local provider :-)
<fwereade> rogpeppe, that said, my instincts are leaning towards the "everything should keep working if everything jujey is removed"
<fwereade> rogpeppe, which would imply the "feature" interpretation
<rogpeppe> fwereade: by "everything", you mean the charm?
<fwereade> rogpeppe, yeah
<rogpeppe> fwereade: i think it's important that the charm directory be definitively available
<rogpeppe> fwereade: apart from anything, lots of charms use it to put config file templates in etc
<fwereade> rogpeppe, templates, fine... they'll only be used by hooks, surely
<fwereade> rogpeppe, but I'm pretty sure that the charm dir should be considered off-limits to anything not running as a hook
<rogpeppe> fwereade: hmm, it's certainly arguable
<fwereade> rogpeppe, because nothing else has any reason to believe that arbitrary changes are not happening at any given time
<rogpeppe> fwereade: yeah - assuming we overwrite charm dirs rather than duplicate them
<fwereade> rogpeppe, I think all it takes is assuming that hooks might do anything to the charm dir, and that v1 of a charm can't know what subsequent versions may choose to do instead
<rogpeppe> fwereade: but i'd like to at least know why we're seeing this behaviour - it doesn't look as if we're deliberately using 700 perms
<dimitern> rogpeppe, fwereade: https://codereview.appspot.com/12849043 - service-related uniter API calls
<fwereade> rogpeppe, I agree there
<rogpeppe> fwereade: personally i think the charm directory should be considered immutable
<rogpeppe> fwereade: but i realise that's probably a lost battle now
<fwereade> rogpeppe, one day we'll need some charm format changes, we can force it then, I hope
<fwereade> rogpeppe, all it really takes is *providing* a directory that charms are expected to write to
<fwereade> rogpeppe, and then rolling it up with some more changes that are compelling enough to make people want to switch ;p
<rogpeppe> fwereade: tbh i don't think we'll ever do it - we'd be giving up the main value of juju, which is the large base of charms
<rogpeppe> fwereade: yeah, we should definitely provide a charm scratch dir
<rogpeppe> fwereade: we could do that now actually
<fwereade> rogpeppe, yeah, that sounds smart actually
<fwereade> rogpeppe, then it's just a matter of communicating it to the charmers, and phasing in requiring its use
<rogpeppe> fwereade: yeah
<fwereade> rogpeppe, hard to check for automatically but we can certainly put it on the checklist
<fwereade> rogpeppe, on the other hand, we do lose the rollbackability
<fwereade> rogpeppe, however I'm not sure anyone's really using that
<fwereade> rogpeppe, ...but we don't know
<rogpeppe> fwereade: rollbackability assumes that *all* state is in the charm dir
 * rogpeppe likes that word
<fwereade> rogpeppe, true
<fwereade> rogpeppe, or at least it assumes that external state is regenerated from the contents of the charm dir, but yeah
<rogpeppe> fwereade: anyway, we could theoretically roll back the contents of the scratch dir
<dimitern> mgz, jam: https://codereview.appspot.com/12849043/
<fwereade> rogpeppe, yeah
<rogpeppe> fwereade: (assuming there are only files in there - what do we do if people are putting unix-domain sockets in there? :-))
<rogpeppe> dimitern: i'm looking
<dimitern> rogpeppe: thanks
<fwereade> rogpeppe, hunt them for sport? ;p
<rogpeppe> fwereade: lol
<jam> fwereade: so did you have a response for bug #1205286 so that stub can get unblocked?
<_mup_> Bug #1205286: charm directory permissions now more restrictive <juju-core:New> <postgresql (Juju Charms Collection):Triaged> <postgresql-psql (Juju Charms Collection):Triaged> <https://launchpad.net/bugs/1205286>
<jam> fwereade: also, you never commented on Placement Directives and I wanted to send that to the list: https://docs.google.com/a/canonical.com/document/d/1Gq8RKyI4uLSYeamz7C2F0tgXdHhTj-d8thKhyX6ae4c/edit?usp=sharing
<fwereade> jam, I'm still thinking/trying to figure out what's happened with the bug
<jam> k
<jam> I don't have a particular stake in the matter, just care that stub has a way forward
<jam> and no particular rush on placement directives, but sort of a "lets get docs from the sprint out to everyone lese"
<fwereade> jam, I've just made the only comment I think I want to on placement directives
<jam> fwereade: well, if you want to barf on the overlaps, you'd need a way to disable the overlap for the current request (i think)
<jam> and the MaaS case for tags that might be service level, but might be unit level
<jam> are a bit less obviously "broken" than zone
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> rogpeppe: LifeGetter works on units, how can it work on services as well?
<rogpeppe> dimitern: with no problem?
<dimitern> rogpeppe: with different entities?
<dimitern> rogpeppe: and we need both Watch for units and services
<rogpeppe> dimitern: LifeGetter works on any entity that has a Life method
<dimitern> rogpeppe: we need different methods for watching units and services
<rogpeppe> dimitern: all you'd need to do is make the auth func return ok for both units and services
<rogpeppe> dimitern: really?
<dimitern> rogpeppe: yes
<rogpeppe> dimitern: don't we watch both units and services with a Watch method?
<rogpeppe> dimitern: which has the same signature for each
<dimitern> rogpeppe: why should the same method operate differently?
<dimitern> rogpeppe: that's mixing responsibilities
<rogpeppe> dimitern: it's not operating differently - we're passing in entity names to watch, and it's doing that
<dimitern> rogpeppe: it should work either on units or on services, you can't mix them
<rogpeppe> dimitern: why not?
<rogpeppe> dimitern: we're saying "watch these things"
<rogpeppe> dimitern: where a thing is uniquely identified by its tag
<dimitern> rogpeppe: how about configsettings then?
<dimitern> rogpeppe: by your logic we shouldn't need another method
<rogpeppe> dimitern: we still need an entry point for that
<rogpeppe> dimitern: because we don't have an entity for config settings
<rogpeppe> dimitern: we don't consider having a Lifer interface to be mixing responsibilities
<dimitern> rogpeppe: so you propose to create a different authFunc just for LifeGetter and Watch works on either only units or only services
<dimitern> rogpeppe: I don't think it should accept mixed units and services tags in one call
<rogpeppe> dimitern: why not?
<rogpeppe> dimitern: it's a bulk call
<dimitern> rogpeppe: because it complicates the tests - we need to test all variations
<rogpeppe> dimitern: it takes a load of entities
<rogpeppe> dimitern: it's no more complicated than having extra entry points
<rogpeppe> dimitern: probably less so
<rogpeppe> dimitern: and the API calls make more sense that way, i think
<dimitern> rogpeppe: no, the simplification for having a single entry point will be transfered to increasing the test size 4-fold
<rogpeppe> dimitern: why should the API provide both UnitLife and ServiceLife when a single Life entry point is sufficient?
<rogpeppe> dimitern: really? isn't it a single extra entry in the argument slice?
<dimitern> rogpeppe: and we're talking about both Watch and Life - so AgentEntityWatcher and LifeGetter both need to work on units and/or services
<rogpeppe> dimitern: isn't that trivial? we just make an auth function that allows both the unit or the unit's service
<dimitern> rogpeppe: no, each tests tests at least 3 cases: valid tag, invalid tag, valid but unauthorized tag
<rogpeppe> dimitern: so that's 2 more entries in the arg slice, no?
<dimitern> rogpeppe: when we take into account services and units can be passed, that comes to 6 cases
<dimitern> rogpeppe: and in addition, we need to add a case when the tag is neither service nor unit
<dimitern> rogpeppe: so that's 7 cases to test
<dimitern> rogpeppe: i agree they can be done in a single call
<dimitern> rogpeppe: just the testing logic will be a bit more complicated after we get the results and check the resources
<rogpeppe> dimitern: yes, you'll have some extra cases to check, but you'd need to check all those cases if you had a different API call, no?
<dimitern> rogpeppe: and when relations come into play, I suppose their Life will be using the same entry point, so that's 3 more cases
<rogpeppe> dimitern: sure - but isn't it great that we can use the same API call (and code) for getting the Life of any entity?
<rogpeppe> dimitern: doesn't that make it particularly nice that we are actually sending tags over the wire, rather than just ids?
<dimitern> rogpeppe: I'll think about it a bit
<rogpeppe> dimitern: we already have the LifeGetter implementation, which can be used for all of this
<jam> rogpeppe: so I think they may end up in different facades, and the different facade may need to restrict what they are allowed to look at. But as long as the Auth is valid (unit agents can't look at machines, for example) having a unified call sounds good.
<rogpeppe> dimitern: (which, FWIW, would be wrong to use if you continue in the same direction - why should we have Life and ServiceLife not UnitLife and ServiceLife?)
<rogpeppe> jam: yeah, LifeGetter allows arbitrary restrictions on what you can look at
<dimitern> rogpeppe: so the auth func for LifeGetter should check AuthOwner for units and 1) get auth'ed entity, 2) compare it's service tag with the given tag - for services
<dimitern> rogpeppe: and basically the same for AgentEntityWatcher as well
<rogpeppe> dimitern: well, it could be simpler than that
<rogpeppe> dimitern: it sounds reasonable. i'll wait to see the code
<rogpeppe> dimitern: it basically comes down to: return tag == unit.Tag() || tag == names.ServiceTag(unit.ServiceName)
<jam> rogpeppe: what is '.tsv' stand for? tab-separated-value (like csv, just with a \t ?)
<dimitern> rogpeppe: I though you suggested having a way to get the auth'ed entity from the authorizier
<rogpeppe> jam: yeah
<dimitern> rogpeppe: like GetAuthEntity() interface{}
<rogpeppe> dimitern: yes - the above is assuming you've got unit from there
<rogpeppe> jam: i thought it was better than .txt and it does seem to have precedent
<davecheney> good evening gentlemen
<davecheney> i have a request from the far east
<davecheney> i need to build a version of juju that does not attempt to create security groups on openstack/hp cloud
<davecheney> is anyone able to help me ?
<rogpeppe> davecheney: do you need tests to pass too?
<davecheney> rogpeppe: no
<rogpeppe> davecheney: have you tried just commenting out the code that creates the security groups?
<davecheney> rogpeppe: yes, that is what I wanted to do
<davecheney> i was hoping for some advice
<davecheney> maybe I should just turn off the firewaller
<rogpeppe> davecheney: hmm, without security groups, how can you open any ports at all?
<rogpeppe> davecheney: 'cos presumably by default all ports are closed
<davecheney> rogpeppe: according to marcoceppi they don't actually firewall anything
<davecheney> just prevent most people from being able to deploy more than 10 services in total on their hp cloud account
<jam> rogpeppe: I know in ec2, the default is that all machines in the same security group can talk to eachother on any port. I'm not sure about hp/openstack in general.
<rogpeppe> davecheney: ah, so it should be irrelevant
<rogpeppe> jam: presumably these machines won't be in the same security group (because we can't create one)
<davecheney> jam: yeah, security groups in ec2 work
<davecheney> they don't in hp cloud
<jam> rogpeppe: I don't know that dave is asking for 0, just not 1-per-machine.
<davecheney> so the creation of the groups just presents an annoying limit
<rogpeppe> davecheney: ah, so you can create one group for the whole environment?
<davecheney> rogpeppe: preferably zero
<davecheney> groups
<davecheney> wallyworld_: poing
<davecheney> ping
<rogpeppe> davecheney: in that case, i'd just see what happens when you change environ.setUpGroups to return nil, nil
<davecheney> rogpeppe: ok, will do
<wallyworld_> davecheney: wat
<fwereade> davecheney, otherwise firewall-mode global will create just one, if that helps?
<marcoceppi> wallyworld_: we're going to be demoing on az1 and az2 "tomorrow", however last I checked az2 and az3 don't have simple stream data
<rogpeppe> fwereade, davecheney: yes, that was going to be my other suggestion
<wallyworld_> marcoceppi: maybe not, i haven't checked. i did some initial data for az1 using the juju tool. if we need it for the other regions it will need to be hand crafted. when is "tomorrow"?
<davecheney> fwereade: that might be better
<jam> marcoceppi: there won't be simplestreams data in cloud-images, since hp isn't (yet) a CPC
<davecheney> i'll try that
<marcoceppi> wallyworld_: 12 hours from now
<jam> wallyworld_: can we generate stuff and put it with the tools we copied over there ?
<wallyworld_> marcoceppi: ok, i'll see if i can whip something up tonight. i may not be around tomorrow as it's a holiday here. how long will you e around for now?
<marcoceppi> jam: az1 data exits already, it'd be nice if az2 and 3 were also in there
<marcoceppi> wallyworld_: I'll be around for 2-3 hours
<wallyworld_> jam: what he said - the existing nmetadata nwds to be added to
<wallyworld_> marcoceppi: ok, i'll see what i can do. maybe you could test once i do it
<marcoceppi> wallyworld_: I'd be happy to test!
<wallyworld_> ok, leave it with me
<wallyworld_> marcoceppi: what image ids do we want?
<wallyworld_> there's a bug i think with those listed
<marcoceppi> wallyworld_: thanks for taking a look! If not I can dig through the metadata plugin if you run out of time and put it in a seperate bucket
<wallyworld_> i'll see if i can find it
<marcoceppi> wallyworld_: I'll double check in the console
<marcoceppi> they may have changed
<wallyworld_> ok, if you tell me the images ids for az2 and az3 that would be great
<wallyworld_> marcoceppi: what series also
<wallyworld_> precise?
<jam> wallyworld_: I see endpoint data but no content: https://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:hpcloud.sjson
<wallyworld_> jam: the data isn't there yet. it's in the hp cloud public bucket
<jam> ah, ok.
 * wallyworld_ wishes it was there
<davecheney> sorry, marcoceppi has lagged out
<davecheney> he'll be back in a sec
<davecheney> marcoceppi: firewall-mode: global
<davecheney> fwereade: firewall-mode: global might be the trick for us
<fwereade> davecheney, cool
<fwereade> davecheney, glad it's good for something, I was worried about it for a while ;p
<davecheney> fwereade: that means we're down to 2 sec groups per student
<davecheney> that'll do
<davecheney> we can squeak by
<davecheney> thanks
<marcoceppi> wallyworld_: yes, precise
<wallyworld_> ok
<marcoceppi> wallyworld_: would it be possible to put the data in the same bucket as the one davecheney sends out during ANN for juju-core testing on hpcloud?
<wallyworld_> marcoceppi: yeah,that's what i'm doing
<marcoceppi> wallyworld_: <3 awesome, thanks!
<wallyworld_> updating existing metadata
<wallyworld_> just need the images ids :-)
<davecheney> wallyworld_: thanks man
<wallyworld_> anytime
<davecheney> so far we've been able to avoid spinning custom tools
<davecheney> there is one more problem which I'm not sure I can fix without changing the tools
<wallyworld_> yes?
<davecheney> byobu doesn't interact well with tera term
<marcoceppi> wallyworld_: AZ2: 68481 AZ3: 49850
<wallyworld_> ta
<davecheney> so I need to put `apt-get remove byobu` in the cloud-init so I can remove byobu and fall back to tmux
<jam> wallyworld_: don't you need the uuids or HP being Diablo just has the short names
<jam> ?
<natefinch> morning all
<wallyworld_> jam: we support both uuids and ints transparently
<natefinch> (and afternoon and evening as the case may be)
<jam> hi natefinch
<marcoceppi> wallyworld_: I created simplestream data a while a go for az3, if it helps
<davecheney> natefinch: ãã¯ãããããã¾ã
<wallyworld_> marcoceppi: i've done the new data, just using the validation tool to smoke test it. but i have no perms to re-upload it turns out
<marcoceppi> wallyworld_: ack
<davecheney> wallyworld_: sad trombone
<wallyworld_> marcoceppi: do you have permission to write to the public bucket?
<natefinch> davecheney: ha.  Sorta surprised google knew what to do with that.
<marcoceppi> wallyworld_: no that I know of
<wallyworld_> marcoceppi: is arosales around? he can
<davecheney> wallyworld_: arosales has been up for about a day
<davecheney> he's been sorting out all our shit today
<marcoceppi> wallyworld_: he may be around, but I'm not sure
<davecheney> i think he's turned in
<wallyworld_> ok
<marcoceppi> slacker ;)
<noodles775> rogpeppe: RE the HP provider returning ErrNoInstances, I just went to make the change to shortAttempt that you mentioned, but afaics from the bug, it's not related? That said, I *think* I found another potential cause... details in the description: https://codereview.appspot.com/12795045
<natefinch> only one day?
<wallyworld_> so what you can do is create a new container, make it globally readable, and put tools and new metadata in it
<marcoceppi> wallyworld_: I might be able to actually, let me check
<wallyworld_> and set the public-bucket-url accordingly
<rogpeppe> noodles775: looking
<wallyworld_> marcoceppi: davecheney: i have the new metadata if only we could write it
<wallyworld_> you guys need to give me more advance notice
<davecheney> wallyworld_: i'll thank you personally for that helpful comment next week
<wallyworld_> davecheney: just saying
<jam> wallyworld_: I have perms to the hp public bucket, but I thought you did, too.
<davecheney> yeah, sorry, we're really hulk smashing this whole training
<wallyworld_> davecheney: just being a prick, sorry
<davecheney> s'ok
<davecheney> if that was the worst thing that had happende this week, i'd be upset
<davecheney> but it isn't
<davecheney> not by a long shot
<wallyworld_> jam: i logged in and can only see us-west
<wallyworld_> and cannot see the public bucket
<marcoceppi> wallyworld_: I have a bukkit
<wallyworld_> marcoceppi: we may be able to write the files
<rogpeppe> noodles775: good catch!
<marcoceppi> \o/
<jam> wallyworld_: there is only 1 storage
<jam> there are 3 computes
<jam> they all use the same storage
<rogpeppe> noodles775: looks like a good fix. it would be nice to be able to test it though.
<jam> az3.geo-1, vs just geo-1
<jam> IIRC
<wallyworld_> jam: i can only see 3 containers, none of which is the public bucket when i log in
<jam> wallyworld_: I see roughly the same thing, but we have the tools somewhere in that account :)
<jam> wallyworld_: "control-bucket: juju-dist"
<wallyworld_> jam: ah, maybe i am a dickhead, there is a drop down with projects which i didnt see
<wallyworld_> let me try selecting the right project
<jam> wallyworld_: I didn't see it either
<jam> wallyworld_: https://console.hpcloud.com/object_store/region-a_geo-1/containers/juju-dist
<wallyworld_> yep, uploading now
<wallyworld_> marcoceppi: ready for testing
<wallyworld_> davecheney: ^^^
<jam> dimitern, rogpeppe, standup? https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471
<davecheney> wallyworld_: ok, will test
<davecheney> wallyworld_: thank you
<jam> axw: ^^
<wallyworld_> davecheney: np at all.  if it doesn't work, i'll fix, let me know
<davecheney> wallyworld_: http://paste.ubuntu.com/5980804/
<davecheney> ^ it's copying tools, is this expected ?
<jam> davecheney: it wasn't expected for me
<jam> do you have public-bucket-url set for HP?
<davecheney> jam: sorr, i might have screwed up
<wallyworld_> yes, looks like public bucket is not set right if it can't see tools
<davecheney>     public-bucket-url: https://console.hpcloud.com/object_store/region-a_geo-1/containers/juju-dist
<wallyworld_> davecheney: https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910
<davecheney> ok, that was what I had before
<davecheney> my bad
<wallyworld_> davecheney: soon, public bucket will not be necessary at all \o/
<davecheney> 2013-08-13 11:37:34 INFO juju provider.go:734 environs/openstack: started instance "1670895"
<davecheney> 2013-08-13 11:37:35 INFO juju supercommand.go:276 command finished
<davecheney> got an instance in az3
<davecheney> rocking
<wallyworld_> yay \o/
<marcoceppi> ditto over here for az2
 * wallyworld_ is happy he wrote the image validation tool to check the metadata prior to copying to the bucket
<marcoceppi> just waiting for juju status to come back
<davecheney> thanks so much for your help
<wallyworld_> my pleasre. good luck with the demo :-)
<marcoceppi> wallyworld_: def, thank you sir!
<wallyworld_> anytime
<noodles775> rogpeppe: Thanks! And yeah - that was my next question. Any relevant examples you could point me at for tests? The ones in the respective provider_test.go are just for one particular function. Otherwise I'll hunt around.
<rogpeppe> noodles775: in a call. be with you in a bit.
<mgz> davecheney: I'm going to poke the release tarball script to use the revs from our new dependencies.tsv file
<davecheney> mgz: sounds good to me
<rvba> Anyone up for a one-liner? https://codereview.appspot.com/12852043
<davecheney> rvba: worst pickup line, ever
<jam> wallyworld_: did you see: https://bugs.launchpad.net/juju-core/+bug/1211147
<_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages <juju-core:Triaged> <https://launchpad.net/bugs/1211147>
<jam> You might want to give it a poke, since you have the most rsyslogd knowledge on the team.
<davecheney> jam: yeah, jonathon totally blew up his environment with that one
<wallyworld_> jam: didn't see it, will look
<marcoceppi> fwereade: I've got a question about optional relations in juju, if you set a relation to be optional: False, what happens?
<fwereade> marcoceppi, nothing at all
<davecheney> fwereade: is it purely decorative ?
<fwereade> davecheney, essentially yes
<marcoceppi> fwereade: will it ever do anything?
<davecheney> fwereade: for ceremonial occasions ?
<fwereade> marcoceppi, it is not currently a high priority
<marcoceppi> I must admit, it's really pretty :)
<wallyworld_> jam: do we even support putting a service on the bootstrap node?
<marcoceppi> fwereade: what's the idea of what it would do in the future when you guys are done with priority things and bored and want to implement it?
<davecheney> wallyworld_: sure, didn't you write that ?
<davecheney> or are you asking, as an optoin we'd be proud to tell customers about ?
<wallyworld_> i didn't write it
<wallyworld_> i really didn't think we supported it
<fwereade> davecheney, marcoceppi: there was talk, a long time ago, that we might automatically create non-optional relations between matching services, but that always seemed kinda crazy to me
<fwereade> davecheney, marcoceppi: there *may* be some mileage in the idea that we just flag non-optional relations that are not satisfied
<fwereade> davecheney, marcoceppi: but doing anything automatic in response seems like a really bad idea to me, because even if we think of something useful to do with it I can't see how it wouldn't be a terrifying behaviour change
<marcoceppi> fwereade: My thought was, if a relation is not optional, it wouldn't "deploy" the charm until juju was aware of the relation via add-relation, must like the subordinate mechanism
<davecheney> fwereade: some background, our students are interesting optional: false relations because they are thinking it will help them know when a service is properly deployed
<davecheney> ie, all it's non optoinal relationships are fulfilled
<fwereade> wallyworld_, rogpeppe, I can hear you
<fwereade> wallyworld_, rogpeppe, but can't seem to say anything you can hear... brb
<jam> fwereade: one more thing for you to glance at quickly: https://code.launchpad.net/~rogpeppe/juju-core/359-no-lax/+merge/178357  (As you were the one that implemented the StartSync vs Sync schism, you should at least be aware that it has all been migrated to Sync)
<dimitern> rogpeppe: updated https://codereview.appspot.com/12849043
<dimitern> jam: take a look as well? ^^
<rogpeppe> jam: i'd like your thoughts on that too
<rogpeppe> jam: 359-no-lax that is
<wallyworld_> fwereade: we lost you again?
<fwereade> wallyworld_, I can hear you, but I'll rejoin
<fwereade> wallyworld_, yeah, it doesn't want to let me rejoin :/
<axw> jam: sorry, was having dinner earlier
<jam> axw: np. I don't know that it will work in your Timezone, but we do a whole group get-together at UTC 11:30
<jam> I think Tim was thinking of doing another one earlier in the day
<jam> for those in the high UTC offsets.
<jam> axw: I added you to the calendar entry, but it is optional for you, since it is roughly dinner/family/anything-that-isn't-work time for you.
<axw> jam: thanks for that. I hit "maybe" because of that, but I'll try to make it sometimes. If there's something important, just let me know and I'll be there
<jam> axw: it is just our daily "catch up with eachother on what we're working on". You should have a weekly 1:1 with either Mark or Tim already for the "important" things.
<dimitern> rogpeppe: does it look better now?
<rogpeppe> dimitern: sorry, still in a call
<rogpeppe> fwereade: you've frozen again...
<fwereade> wallyworld_, rogpeppe, did I get cut off again?
<rogpeppe> fwereade: yes
<wallyworld_> yep
<dimitern> rogpeppe: np
<dimitern> natefinch, mgz: second review on https://codereview.appspot.com/12849043 please?
<mramm> hey all
<mgz> dimitern: sure
<dimitern> mgz: cheers
<dimitern> mgz: just note I forgot to update the description - ServiceLife, ServiceCharmURL and WatchService are gone, replaced (as suggested by rog) with Life, Watch amd CharmURL handling both units and services
<mramm> is it an lxc prereq issue that causes this error:
<mramm> 08:41 danwest: $ juju -v bootstrap
<mramm> 08:41 danwest: 2013-08-13 12:40:06 INFO juju.environs.local environprovider.go:32 opening environment "local"
<mramm> 08:41 danwest: 2013-08-13 12:40:06 ERROR juju.environs.local net.go:18 cannot find network interface "lxcbr0": net: no such interface
<mramm> 08:41 danwest: 2013-08-13 12:40:06 ERROR juju.environs.local environprovider.go:44 failure setting config: net: no such interface
<mramm> 08:41 danwest: 2013-08-13 12:40:06 ERROR juju supercommand.go:235 command failed: net: no such interface
<mramm> 08:41 danwest: error: net: no such interface
<dimitern> mramm: you need sudo juju bootstrap with the local provider
<mramm> ahh
<dimitern> mramm: but apart from that it might be something else
<dimitern> (prereq issue)
<mramm> ok
<mramm> I will tell him to try that
<mramm> also we should give a better error message there too
<mramm> sudo did not fix it
<dimitern> so it's a different issue
<dimitern> we recently updated the README I think wrt the local provider
<dimitern> nope, it's not in yet
<mramm> also, installing lxc was not the fix
<mramm> I am having him file a bug
<dimitern> mramm: I found this thread http://comments.gmane.org/gmane.linux.kernel.containers.lxc.general/3819 - it says to check /var/log/upstart/lxc* for errors
<danwest> dnsmasq: failed to create listening socket for 10.0.3.1: Address already in use
<danwest> hmm, if true should handle that a bit more gracefully
<danwest> looking into it now
<mramm> agreed
<dimitern> danwest: yup, that's what was mentioned in that thread as well
<dimitern> mramm, danwest: it seems an lxc issue though, not a juju issue
<mramm> juju should handle that if nessisary
<mramm> I mean we should talk to serge and whatnot, but juju needs to just work or at the very least give good error messages
<dimitern> mramm: agreed, that specific error means gonet wasn't able to open the interface lxcbr0
<danwest> same issue after killing dnsmasq
<dimitern> danwest: take a look what's listening on 10.0.3.1
<danwest> tftp
<danwest> nothing
<dimitern> maybe it starts dnsmasq twice or something? anything else useful in the log?
<danwest> nothing in the log, actually the entry must have been old as I moved the log aside and re-ran - no new lxc log
<danwest> entry was not timestamped
<dimitern> danwest: sorry, perhaps hallyn_ or someone else with more in-depth knowledge of lxc can help
<danwest> dimitern: thx
<hallyn_> danwest: i need to deal with some movers for the next hour.  addres already in use is weird.  can you reproduce at will?  can you email or pastebin a reproducer?
<danwest> hallyn_: will try, thx
<hallyn_> (or just a launchpad bug against lxc :)
<hallyn_> danwest: oh wait - are you starting containers nested, inside lxc?
<danwest> hallyn_: nope
<hallyn_> danwest: if so then you may be running an older lxc - it shoudl fall back to 10.0.4.x if 10.0.3.x is taken by the
<hallyn_> ok
<hallyn_> thx, ttyl :)
<dimitern> jam: so, suggestions about what error should we return when len(entities) == 0 ?
<dimitern> jam: thanks for the review btw
<mgz> hm, I find that latest branch a little hard to follow dimitern
<dimitern> mgz: oh? what's not clear?
<mgz> various changes made so CharmURl also support services? then I'm not clear on the new getCanAccessUnitOrService func
<dimitern> mgz: yes, as rogpeppe suggested
<dimitern> mgz: the new authFunc checks if you're allowed to access the given tag, which is either a unit or service
<dimitern> mgz: in the latter case, you can access it only if it matches your authentication entity (unit)'s service
<mgz> okay, and it works, because a unit tag will never match a service tag,
<dimitern> mgz: it's used in CharmURL to allow both unit and service tags as args
<mgz> so the extra code will always get passed over in the unit case...
<dimitern> no
<dimitern> the code is the same for a unit or a service - both support CharmURL (string, bool)
<mgz> right, but passing a unit to CharmURL now uses a different auth check
<dimitern> yes
<mgz> but... if a unit is passed, it will still end up calling AuthOwner(tag) the same as before
<dimitern> no actually, it's still the same
<dimitern> it's just extended
<dimitern> yeah
<dimitern> in case it's not an auth'ed service tag
<dimitern> AuthOwner will never work for anything else than a unit tag
<mgz> those two funcs don't actually need closures, right?
<dimitern> and services cannot log in anyway
<mgz> could you put them at top level with doc comments before assigning into UniterAPI?
<dimitern> we need to pass them to LifeGetter and AgentEntityWatcher as well
<dimitern> why? they need the authorizer that was passed to NewUniterAPI
<dimitern> and they're used only there
<dimitern> using closures like this is nice, because we're currying the authorizer as an implict arg
<mgz> ah, they do need closures, hadn't notices authorizer was a param not a package
<dimitern> yep
<mgz> well, lgtmed.
<dimitern> cheers!
<dimitern> rogpeppe: I'll wait for you to take a look before landing
<rogpeppe> dimitern: i'm still looking at it
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: thanks
<dimitern> rogpeppe: i don't want to return Entity there - I want the real thing
<dimitern> rogpeppe: in GetAuthEntity
<rogpeppe> dimitern: what?
<rogpeppe> dimitern: state.Entity is more real than interface{}
<dimitern> rogpeppe: but does it have CharmURL and any other method I might need?
<rogpeppe> dimitern: no, but anything is better than bare interface{}
<rogpeppe> dimitern: at least state.Entity suggests the type that it might be returning
<dimitern> rogpeppe: so what, for each method we need we have to change state.Entity to add that method?
<rogpeppe> dimitern: the set of types
<rogpeppe> dimitern: ???
<rogpeppe> dimitern: interface{} has no methods at all.
<dimitern> rogpeppe: no, but I can do entity.(*state.Unit).CharmURL() on it
<rogpeppe> dimitern: just as you can if it's state.Entity
<rogpeppe> dimitern: there's no difference in that respect
<dimitern> rogpeppe: so you can cast any type, no just an interface{} ?
<rogpeppe> dimitern: um, yes?
<dimitern> rogpeppe: c-style?
<rogpeppe> dimitern: any interface type
<rogpeppe> dimitern: and state.Entity is an interface
<dimitern> rogpeppe: hmm
<dimitern> rogpeppe: ok then, I'll try it
<rogpeppe> dimitern: thanks
 * rogpeppe gets some lunch
<dimitern> anyone: next one in line https://codereview.appspot.com/12850044
<jam> natefinch: did you see: https://code.launchpad.net/~michael.nelson/juju-core/1208504-post-bootstrap-hp-no-instances-found-try2/+merge/179899 I think noodles775 has touched the same code as you, but I might be thinking of the wrong patch (I've gone through a lot of patches today :)
<jam> dimitern: len(entities) == 0 can just be kept the same way. If we do decide to change it, then "ErrNoArgs" or something along those lines. But it would be a policy change across the board so it should be thought about and discussed I think.
 * jam I'm off for now
<noodles775> natefinch, jam: happy to reject mine if it's already fixed :)
<dimitern> jam: I decided to remove it
<dimitern> jam: when we introduce backward-incompatible changes to params we'll have to take that into account, like for SetStatus
<jam> dimitern: well we need it in place *before* we make an incompatible change, otherwise the "old servers don't error on new requests" is still valid.
<jam> anyway, it is a separate change from what you're doing.
<dimitern> jam: yep
<natefinch> jam, noodles775:  looking now
<natefinch> jam, noodles775: looks very different from my changes.  Mine is more focused on when you really haven't bootstrapped yet, rather than bootstrapping not being finished
<natefinch> jam, noodles775: I think they're complimentary changes, and shouldn't affect one another
<noodles775> natefinch: Cool - thanks for checking.
<jam> ah, both noodles775 patches. https://codereview.appspot.com/12755043/ vs https://codereview.appspot.com/12795045/
<jam> noodles775: is https://codereview.appspot.com/12795045/ supposed to supersede the other?
<noodles775> jam: yeah it is. I'd not realised you'd commented on the former - it should be closed.
 * noodles775 closes.
<jam> noodles775: I tend to drive reviews from my email folder, so that doesn't always help :)
<dimitern> mgz, jam, rogpeppe: when you have some time PTAL https://codereview.appspot.com/12850044
<fwereade> rogpeppe, so, https://codereview.appspot.com/12352044/
<fwereade> rogpeppe, your analysis is I think correct
 * rogpeppe breathes a sigh of relief
<fwereade> rogpeppe, but you removed a great big pile of test coverage with the "fix"
<dimitern> rogpeppe: also updated https://codereview.appspot.com/12849043/
<fwereade> rogpeppe, and AFAICT it could have been resolved with a LaxStringsWatcher at the end of the unit and machine tests
<rogpeppe> fwereade: really?
<fwereade> er LaxNotifyWatcher
<rogpeppe> fwereade: what's no longer tested?
<fwereade> rogpeppe, well, unless you removed the coalescence behaviour and verified that the tests failed
<rogpeppe> fwereade: i didn't understand why StartSync tests coalescence that Sync doesn't
<fwereade> rogpeppe, which they wouldn't have done, because the Sync version essentially forces coalescence irrespective of watcher implementation
<fwereade> rogpeppe, because the Sync tests force all events to have been read before we read from the watcher
<fwereade> rogpeppe, the StartSync ones were explicitly there to test what happened when we read from the out channel before the in channel was exhausted
<rogpeppe> fwereade: i still don't quite see it
<rogpeppe> fwereade: the difference between StartSync and Sync is just one or two scheduling decisions. I don't see how that can be the basis of a reliable test.
<fwereade> rogpeppe, the point is that Sync will always pass, while StartSync will sometimes fail if the behaviour's wrong
<rogpeppe> fwereade: perhaps you could describe to me a possible failure mode when using StartSync
<fwereade> rogpeppe, more reliability is good, I would be happier if the tests could reliably pass/fail in all circumstances
<rogpeppe> fwereade: so I can understand what we're trying to guard against
<sidnei> fwereade: https://codereview.appspot.com/12859043 and https://bugs.launchpad.net/juju-core/+bug/1201503 's comments for some investigation
<_mup_> Bug #1201503: Add os disk constraint <constraints> <juju-core:Triaged by sidnei> <https://launchpad.net/bugs/1201503>
 * arosales reading backscroll, looks like wallyworld_ jam figured out the public bucket issue
<fwereade> sidnei, cheers
 * fwereade continues to try to marshal a clear argument for rogpeppe
 * rogpeppe continues to wear a slightly puzzled expression
 * fwereade might have come up with an illustrative question
<rogpeppe> fwereade: are you talking specifically about testing the behaviour of the collect function?
<fwereade> rogpeppe, how is it possible for us to test event coalescence when Sync is the only tool in play?
<fwereade> rogpeppe, yeah
<fwereade> rogpeppe, whenever we Sync we force the watcher to finish handling all events before we even try to read from Shanges
<fwereade> rogpeppe, er Changes
<rogpeppe> fwereade: how's that?
<rogpeppe> oh, i see
<fwereade> rogpeppe, so you are perfectly correct about what was going on
<fwereade> rogpeppe, it's somewhat interesting that a case could be made that the tests were failing due to a hard-to-detect bug in the watcher implementation
<fwereade> rogpeppe, but changing that stopped us from being able to detect similar bugs that would I think unarguably be bugs
<rogpeppe> fwereade: it depends what semantics you expect from StartSync really
<fwereade> rogpeppe, as opposed to this behaviour which is, er, arguable
<dimitern> rogpeppe: should I land https://codereview.appspot.com/12849043/ now? I think I did all you asked
<fwereade> rogpeppe, I just expect it to return as soon as possible, and for that to leave a moderate chance of detecting collect bugs
<dimitern> mgz: hey
<rogpeppe> fwereade: i *think* the tests could still break when coalescence is disabled and using Sync
<rogpeppe> dimitern: looking
<rogpeppe> fwereade: in fact, i might try that out
<dimitern> mgz: https://codereview.appspot.com/12850044 quick review?
<fwereade> rogpeppe, I think that NotifyWatcher and StringsWatcher are safe from that, and I couldn't repro it just now, but I didn't try very hard
<fwereade> rogpeppe, any more complex watcher where there's another layer in play is vulnerable
<fwereade> rogpeppe, but NW and SW only have one select loop, and as soon as the last event's been delivered the watcher finishes processing it before going round again and potentially sending on Changes
<rogpeppe> fwereade: i think a better way of testing coalescence is to make the test start a goroutine to transfer all events into a buffered channel, then call Sync
<rogpeppe> fwereade: as a specific coalescence test
<fwereade> rogpeppe, yeah, that sounds nice
<fwereade> rogpeppe, but that sort of test does get rather tricky to follow... iirc you argued against that sort of approach when I first proposed it
<rogpeppe> fwereade: istm that's a better approach than spraying StartSync about the place as if it's testing something specific
<fwereade> rogpeppe, well, hmm
<rogpeppe> fwereade: i'd have only one test like that for each watcher
<fwereade> rogpeppe, I think that Sync is very hard to justify
<rogpeppe> fwereade: i think that StartSync is even harder to justify :-)
<fwereade> rogpeppe, expand please, Sync STM to be much further divorced from anything resembling a real situation
<fwereade> rogpeppe, you know what
<rogpeppe> fwereade: StartSync gives you no guarantees at all, which makes it hard to write reliable tests
<fwereade> rogpeppe, fuck both sync and startsync, we should just patch out the watcher period
<fwereade> rogpeppe, but Sync is completely fake and makes it very easy to write completely useless tests
<dimitern> fwereade: actually that sgtm
<mgz> dimitern: looking
<dimitern> fwereade: we call Sync to make sure the events have arrived, if we patch the watcher period, we can guarantee that
<fwereade> dimitern, hmm, though, I think we still need *some* sort of machinery somewhere, or to make *very very* timing-dependent tests, which ain't great
<rogpeppe> dimitern: how can we guarantee that?
<rogpeppe> dimitern: timing isn't guaranteed, as fwereade says above
<rogpeppe> dimitern: i still don't quite understand why you need getUnitOrService
<dimitern> rogpeppe: for cases like Life, Watch and CharmURL
<rogpeppe> dimitern: why not just call State.Entity directly in those cases?
<dimitern> rogpeppe: there's no state.Entity()
<dimitern> rogpeppe: there's FindEntity and what's wrong with my approach?
<rogpeppe> dimitern: oh yeah, FindEntity, sorry!
<rogpeppe> dimitern: what does getUnitOrService give you that State.FindEntity doesn't?
<dimitern> rogpeppe: better validation of expected tag kinds?
<rogpeppe> dimitern: that's already validated by canAccess, no?
<dimitern> rogpeppe: no
<dimitern> rogpeppe: it only checks if you can access it
<dimitern> rogpeppe: it's not validating tag kinds
<dimitern> rogpeppe: a bit of extra sanity checking at the expense of 3-4 lines seems like a good idea
<dimitern> rogpeppe: and it's only done once in getUnitorService
<rogpeppe> dimitern: how could canAccessUnitOrService return true for a tag that wasn't a service or a unit?
<dimitern> rogpeppe: when state is flawed somehow?
<rogpeppe> dimitern: i think it's worth having clear security checking boundaries, rather than just piling extra checks on like they might help
<dimitern> rogpeppe: you have a unit that has an invalid service attached to it?
<rogpeppe> dimitern: there are hundreds of ways we could go wrong if our code is flawed
<rogpeppe> dimitern: you're *already* assuming that canAccessUnitOrService is operating as expected
<dimitern> rogpeppe: I didn't say our code is flawed
<dimitern> rogpeppe: I said the state in mongo might be invalid
<rogpeppe> [16:01:58] <dimitern> rogpeppe: when state is flawed somehow?
<dimitern> rogpeppe: ^^
<dimitern> (to many things called "state")
<rogpeppe> dimitern: even if the state in mongo is invalid, your checks would still not fail
<noodles775> rogpeppe: for later/tomorrow/next week, I've also updated the interspersed flags based on our conv. yesterday: https://codereview.appspot.com/12603047
<rogpeppe> dimitern: there's no way we can return a *Unit from a service- tag
<rogpeppe> noodles775: thanks
<dimitern> rogpeppe: ok, I give up
<rogpeppe> dimitern: i'm only trying to get us to write less, simpler, code
<dimitern> rogpeppe: I'm getting rid of getServiceOrUnit and replacing it with FindEntity
<rogpeppe> dimitern: cool, thanks
<dimitern> rogpeppe: I want to land this already
<dimitern> rogpeppe: but will keep the getUnit and getService helpers
<rogpeppe> dimitern: yeah, i thought they were worth it for the static type conversion
<rogpeppe> s/conversion/return
<dimitern> rogpeppe: reproposing with that change
<dimitern> rogpeppe: done
<dimitern> rogpeppe: was that the only issue that needed fixing?
<dimitern> mgz: thanks
<rvba> rogpeppe: I think that review is for you :).  Nothing urgent though, it's just a clean-up. https://codereview.appspot.com/12861043/
<rogpeppe> dimitern: reviewed. thanks for bearing with me!
<fwereade> noodles775, just looking at https://codereview.appspot.com/12603047/patch/11001/12004
<fwereade> noodles775, what extra args were you expecting for debug-log
<fwereade> ?
<dimitern> rogpeppe: thanks
<rogpeppe> rvba: LGTM
<rogpeppe> fwereade: well, debug-log does say that it accepts ssh args
<rogpeppe> fwereade: which seems weird to me, but there y'go
<fwereade> rogpeppe, seems weird to me too
<noodles775> fwereade: I wasn't expecting any... but debug-log apparently doesn't embed CommandBase (at least, I got a compile error saying it wasn't implementing the iface due to the missing method).
<rogpeppe> fwereade: do we know what people might use that feature for?
<noodles775> fwereade: so I needed to add the method - happy to update it to instead return CommandBase.AllowInterspersedFlags which defaults to true).
<fwereade> rogpeppe, afraid not... wallyworld_ implemented it iirc
<rogpeppe> fwereade: does the python version allow arbitrary ssh args there, d'ya know?
<fwereade> rogpeppe, the only case I can think of is needing to pass args down to ssh to cause that to work, but that seems surprising... an indicator of a different bug maybe?
<fwereade> rogpeppe, looks like it doesn't
<fwereade> rogpeppe, OTOH it *does* accept a bunch of other args that we don;t
<rogpeppe> fwereade: ha ha
<fwereade> rogpeppe, including, it seems, "--replay" equivalent to your requested "--all"
<fwereade> rogpeppe, although I think I still disapprove ;p
<rogpeppe> fwereade: i will definitely use --all most of the time, despite the bandwidth
<fwereade> noodles775, cool, I think if we just make it match CommandBase I'll be happy
<rogpeppe> fwereade: it would be nice to have a "last x minutes" flag though
<fwereade> rogpeppe, not to mention filtering
<rogpeppe> fwereade: yeah, that too
<fwereade> noodles775, reviewed, LGTM with that change
<fwereade> noodles775, thanks
<noodles775> Thanks fwereade, updating now.
<rogpeppe> noodles775: reviewed
<rogpeppe> fwereade: cmd.ParseArgs can go, yay!
<rogpeppe> noodles775: i'm very happy how that turned out
<noodles775> rogpeppe: yeah, much nicer - thanks for the suggestions.
<noodles775> rogpeppe: erm, you meant s/false/true when you said "I'd just return false here" didn't you? (as per fwereade, we just want the same behavior as CommandBase)
<rogpeppe> noodles775: oh, possibly, yes, assuming we *don't* want to allow arbitrary ssh args to be passed through
<noodles775> Right.
<noodles775> rogpeppe: sorry, another question - when you say that ParseArgs can now be removed, are you meaning I should add SuperCommand.AllowInterspersedFlags() returning false, or is there another reason why it'll just work that I'm missing?
<noodles775> rogpeppe: nm - I see you said that later.
<rogpeppe> noodles775: that's the main reason i'm particularly happy with this change
<rogpeppe> noodles775: the non-interspersing parsing of supercommand was a hack, but now it just falls out naturally.
<noodles775> Great :-)
<rogpeppe> noodles775: it also means that supercommands can now nest nicely if we want them to
<natefinch> trying to land a branch, but bzr is saying it can't get the lock on the branch.... from natefinch@bazaar.canonical.com.  That seems bad.
<natefinch> Maybe I'm doing something wrong
<natefinch> rogpeppe, dimitern, mgz:  any thoughts on ^
<rogpeppe> natefinch: can you paste the message you're getting?
<natefinch> nate@Asgard:~/code/src/launchpad.net/juju-core$ bzr push lp:~natefinch/juju-core/juju-core
<natefinch> Unable to obtain lock  held by natefinch@bazaar.launchpad.net on taotie (process #25639), acquired 9 minutes, 56 seconds ago.
<natefinch> See "bzr help break-lock" for more.
<natefinch> bzr: ERROR: Could not acquire lock "(remote lock)": bzr+ssh://bazaar.launchpad.net/~natefinch/juju-core/juju-core/
<_mup_> Bug #25639: gs-gpl - .getdeviceparams gets called with broken types <gs-gpl (Ubuntu):Invalid> <gs-gpl (Debian):Fix Released> <https://launchpad.net/bugs/25639>
<rogpeppe> natefinch: why are you naming your branch "juju-core"?
<natefinch> I was attempting to follow a document Frank gave me - https://docs.google.com/document/d/1G74QKbJqPLooEBVPOeB5qgJSQ5fb7KGpmUTTjIK0NnQ/edit
<natefinch> rogpeppe: I may have done something wrong :)
<rogpeppe> natefinch: ah, i think those instructions are misleading
<rogpeppe> natefinch: we usually make a unique (to the user) name for each new branch
<rogpeppe> natefinch: (personally, i use branch names of the form 999-description, so i can remember the order i created them in)
<natefinch> I tried that... at least, I thought so.  I had named it 002-bootstrap or something like that.  Obviously not quite the right thing
<rogpeppe> natefinch: ah, actually that is suggested in that document: "bzr push --remember lp:~themue/juju-core/001-my-useful-change"
<rogpeppe> natefinch: when you write "bzr push lp:~natefinch/juju-core/juju-core" you're trying to push it to a branch named "juju-core" in your lp space
<rogpeppe> natefinch: which seems to exist already
<dimitern> natefinch: I saw that yesterday
<dimitern> natefinch: note - it's better never to interrupt lbox (^C) once started
<natefinch> dimitern: I may have done that once or twice :/
<dimitern> natefinch: just do bzr break-lock bzr+ssh://bazaar.launchpad.net/... (the branch it told you)
<dimitern> natefinch: and then repropose, should work
<rogpeppe> natefinch: i've also got a lock held
<rogpeppe> Unable to obtain lock  held by rogpeppe@bazaar.launchpad.net on taotie (process #30432), acquired 1205 hours, 16 minutes ago.
<_mup_> Bug #30432: skim is hung on with "sudo kate" <skim (Ubuntu):Invalid> <https://launchpad.net/bugs/30432>
<rogpeppe> See "bzr help break-lock" for more.
<dimitern> natefinch: and yes, it's better to pick some names for your branches
<rogpeppe> natefinch: :-)
<rogpeppe> natefinch: it's actually quite useful, because it's on lp:~rogpeppe/juju-core/trunk
<natefinch> dimitern: I tried to... I think I just misread something somewhere
<rogpeppe> natefinch: and the lock message warns me that i've forgotten to rename my branch again
<rogpeppe> natefinch: are you using cobzr?
<natefinch> rogpeppe: no, but only because I don't know what it is :)
<rogpeppe> natefinch: http://labix.org/cobzr
<rogpeppe> natefinch: some people hate it
<rogpeppe> natefinch: i find it works pretty well for me
<natefinch> rogpeppe: man that guy is prolific :)
<rogpeppe> natefinch: ain't it so
<rogpeppe> natefinch: i define bzr as cobzr (i use a shell function)
<natefinch> rogpeppe: ahh, this was what mgz was using yesterday... I was wondering how he did that switching between branches
<rogpeppe> natefinch: i think you can use a bzr feature too, but i don't know how to do that
<rogpeppe> natefinch: i usually do: cobzr branch lp:juju-core/trunk
<rogpeppe> natefinch: then: cobzr switch trunk
<rogpeppe> natefinch: then: cobzr checkout -b 123-my-new-branch
<rogpeppe> natefinch: then edit the branch, commit, and lbox propose
<natefinch> rogpeppe: that looks pretty good
<rogpeppe> natefinch: i can switch between branches with cobzr switch branch-name
<rogpeppe> natefinch: and i can keep the same GOPATH for most development purposes
<rogpeppe> natefinch: (i actually keep another one for testing against different go versions)
<natefinch> rogpeppe: yeah, that was immediately my thought... having multiple branches in the same directory is a big help.  I've been renaming ones I'm not actively working on, but that's kind of a headache
<rogpeppe> natefinch: i've got 307 branches in my juju-core tree currently :-)
 * rogpeppe should probably do some garbage collection some time
<natefinch> rogpeppe: disk space is cheap :)
<rogpeppe> natefinch: yeah, it's only 54MB
<mgz> natefinch: no, I was using native colo
<mgz> it's got a few quirks, but is far less buggy
<natefinch> mgz: I saw it was an upcoming feature.. what version of bzr are you on?
<mgz> natefinch: the one in the current distro/ppa for older versions
<natefinch> mgz: ahh yeah, it looks like my version (2.6) has it.  Cool.
<mgz> so, `apt-add-repository ppa:bzr/ppa` if you're on precise or something
<mgz> right, 2.6 is what you need
<natefinch-lunch> back in a bit... thanks for the help
<natefinch> mgz, rogpeppe, dimitern: can one of you (or anyone else) help me detangle my bzr mess and get my branch landed?
<rogpeppe> natefinch: sure
<natefinch> rogpeppe: thanks
<ahasenack> hm, since when does juju bootstrap perform a tool sync from amazon?
<natefinch> rogpeppe: so, I think this is probably one problem:
<natefinch> Repository tree (format: 2a)
<natefinch> Location:
<natefinch>   shared repository: /home/nate/code
<natefinch>   repository branch: .
<natefinch> Related branches:
<ahasenack> http://pastebin.ubuntu.com/5981915/
<natefinch>     push branch: bzr+ssh://bazaar.launchpad.net/~natefinch/juju-core/002-notbootstrapped/
<natefinch>   parent branch: /home/nate/code/src/launchpad.net/juju-core-trunk
<natefinch>   submit branch: bzr+ssh://bazaar.launchpad.net/+branch/juju-core/
<rogpeppe> ahasenack: since quite recently, but i don't think that should happen when using --upload-tools
<mgz> natefinch: what's the mess?
<ahasenack> rogpeppe: yeah, and it's syncing the wrong tools even (1.12)
<natefinch> mgz: see the paste above... that looks wrong, but I don't really know what it's supposed to look like, so maybe that's ok
<rogpeppe> ahasenack: looks odd to me
<mgz> natefinch: apart from submit branch (which doesn't really matter) it's fine?
<rogpeppe> ahasenack: it should find the tools it's uploaded
<natefinch> mgz: oh, ok.  I thought the submit branch was what was tripping me up.
<ahasenack> rogpeppe: fwiw I'm on raring, and requested a saucy bootstrap node, but it did say it was uploading a saucy tarball
<ahasenack> I'm filing a bug
<rogpeppe> ahasenack: +1
<natefinch> mgz: so, how do I land this thing? I merged from trunk, so I should be up to date
<mgz> natefinch: see my "Landing bot active again" post to juju-dev for an easy way
<mgz> in the archives, july 29th
<natefinch> mgz: where's the archive?  I wasn't on the mailing list at that point
<mgz> lists.ubuntu.com
<natefinch> mgz: nice, thanks
<fwereade> sidnei, reviewed
<rogpeppe> dimitern, fwereade, natefinch: fairly trivial change to utils/set: https://codereview.appspot.com/12872043
<sidnei> fwereade: uhm, not sure i get the comment about instance types without instance-storage? what are those?
<rogpeppe> right, environs/ec2 tests pass. time to stop for the day.
<rogpeppe> g'night all
<natefinch> rogpeppe: LGTM'd
#juju-dev 2013-08-14
<davechen1y> axw: ping
<axw> davechen1y: pong
<davechen1y> axw: i'll find the bug
<davechen1y> but we have a prolem when we use deploy --to 0
<davechen1y> which causes debug-log to go crazy
<axw> okey dokey
<davechen1y> i'd really like to be able to fix it
<davechen1y> because we're leaning heavily on the gui deployed to machine 0
<davechen1y> the workaround is don't use --to
<davechen1y> axw: sorry, i don't know why I'm making this your problem
<davechen1y> you didn't write --to
<axw> heh that's okay, I can look into it
<axw> there's no consolidated log on local, so may take me a bit to get set up
<davechen1y> yeah
<davechen1y> that is why we didn't hit it til now
<davechen1y> bug 1211147
<_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages <juju-core:Triaged> <https://launchpad.net/bugs/1211147>
<davechen1y> if it's too hard to fix easily I can stop useing --to
<davechen1y> but we're struggling to keep the number of machines deployed down to a reasonable level
<davecheney> hello
<davecheney> question from the field
<davecheney> does juju upgrade-charm --switch work
<davecheney> to switch from cs: to local: ?
<davecheney> hazmat: ping
<axw> davecheney: don't know that, but to fix your logging problem temporarily...
<axw> delete /etc/rsyslog.d/26-juju*, restart rsyslogd
<axw> (I think)
<davecheney> axw: thanks, i'll work up a one line juju ssh fix
<axw> on the state server that is
<axw> davecheney: I haven't tested it.
<axw> I don't have canonistack set up atm
<davecheney> axw: testing now
<davecheney> hmm, close, but there is still something else going on
<davecheney> axw: on positive news
<davecheney> debug hooks is working like a chapm
<davecheney> champ
<axw> davecheney: sorry went to get a sandwich
<axw> glad to know debug hooks is working. sorry about the logging... I'll keep looking
<davecheney> axw: it should work, maybe rsyslog is holding the config somewhere else ...
<davecheney> axw: ahh, it could be that debug-log is brokwn
<davecheney> totally
<axw> davecheney: how's that?
<davecheney> made a new environment without using --to
<davecheney> debug log still spews output
<axw> hmmkay
<davecheney> which is a bit of letdown
<davecheney> oh
<davecheney> i think the problem is all the \n's have been stripped from the debug log file
<davecheney> sorry
<davecheney> all-machines.log
<davecheney> so tail looks insane
<jam> morning axw and davecheney, how are things in AU?
<axw> jam: morning! not too shabby thank you
<jam> davecheney: is your debug-log problem bug #1211147?
<_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages <juju-core:Triaged> <https://launchpad.net/bugs/1211147>
<davecheney> jam: yes, and it is actually our log format
<axw> struggling to get back onto canonistack tho..
<davecheney> axw: canonistack is fail
<davecheney> just get ec2 or hp cloud credentials
<davecheney> jam: the problem with debug-log is the log format
<davecheney> it's stripping the \n
<davecheney> so wc -l on all-machines says there are 0 lines
<davecheney> and tail prints everything fromthe beginnning of time
<jam> davecheney: so "juju debug-log" is just doing "juju ssh 0 tail -nX /var/log/juju/all-machines.log" but you're saying that something in how we are doing rsyslog is causing us to strip all \n chars?
<davecheney> jam: that is my diagnosise so far
<jam> davecheney: that certainly sounds like a new problem
<davecheney> jam: try it
<axw> davecheney: I guess there are company accounts for EC2/HPCloud? who do I need to ask?
<davecheney> axw: no
<davecheney> you shold obtain your own credentials and expense them
<axw> ah ok
<davecheney> i'm sorry nobody explained this too you
<davecheney> this really should be day one induction stuff
<davecheney> the company has resources, canonistack
<axw> actually I think rog mentioned we could expense EC2..
<davecheney> but it's frequently not up to the task
<jam> davecheney: actually, we do have a company account with arosales for hp
<axw> that's a bit sad :(
<davecheney> axw: yeah, there is even a line item on the expense system for those type of expenses
<axw> ok
<axw> thanks
<marcoceppi> jam: I'm not sure it's a company account, more like an account that we have for certain testing
<marcoceppi> and now the occasional emergency charm school
<jam> marcoceppi: right. For "testing" which is what axw is doing. vs hosting your own project.
 * marcoceppi doesn't know what anyone does. Not even meself
<jam> axw: which Canonical will pay for you to host your own project (equivalent of up to 3 m1.tiny (?)) as long as you manage it with Juju to give us feedback on the tool.
<axw> cool.
<axw> I better think up some interesting projects then
<davecheney> jam: it is quite strange, the output format of the rsyslog config hasn't changed since wallyworld_ landed the fix
<davecheney> sorry the original support
<davecheney> i wonder if it is a hp cloud thing
 * davecheney tries ec2
<davecheney> indeed it is
<davecheney> it works perfectly in ec2 ...
<davecheney> i wonder if it is the hostname
<davecheney> jam: i think there are two problems
<davecheney> hulk smash causes rsyslog to feed back to itself
<davecheney> but also the output format + hp cloud does something strange
<jam> davecheney: right, hulk-smash is something I feel we have an understanding of the bug, we just need to fix it. hp cloud stripping '\n' is something new we haven't seen before.
<davecheney> jam: understood
<davecheney> will treat as a second issue
<davecheney> just trying axw's fix for hulk smash
<davecheney> jam: axw confirm that reminging /etc/rsyslog/26-juju* and restarting rsyslog works
<axw> goodo
<axw> thanks
<jam> davecheney: so that fixes the \n thing as well? or just the syslog feeding into itself?
<davecheney> jam: only 1211147
<davecheney> i'll dig into the log format thing on hp cloud
<davecheney> and open a new issue
<davecheney> axw: is it possible to do
<davecheney> juju status 1
<davecheney> and show only machine 1
<axw> davecheney: nope
<axw> only unit or service
<davecheney> axw: roger
<davecheney> question, what is the presence ping delay ?
<davecheney> 60 seconds ?
<axw> there's a var in presence/presence.go that says 30s, tho I don't claim to understand the workings of that code
<axw> ^ davecheney
<marcoceppi> So, juju deploy mysql mysql-1 (using mysql-1 as an alias) throws an error, is this expected
<davecheney> axw: thanks
<davecheney> i was able to demo that
<davecheney> does anyone know whre the missing and unknown states show up
<davecheney> is that only for machine records ?
<axw> erk
<axw> jam: I reproduced that line-ending bug on lcy02
<jam> I'm happy it was reproducible, less so that it exists :)
<axw> I'll look into it after fixing the original problem
<noodles775> Is StoreSuite known to fail sometimes? http://paste.ubuntu.com/5984006/
<axw> noodles775: yeah I think I've seen TestBlitzKey fail before, randomly
<jam> noodles775: we are supposed to be rigorous about not having casually failing tests in the test suite, though it seems being away for 3 weeks caused this rigor to diminish a bit.
<jam> (to land code, all tests must pass on the bot, for example)
<rogpeppe> axw, fwereade_, jam, davechen1y, mgz, wallyworld_: morning!
<noodles775> jam: Yep, although it can be v. hard to fix them when they're hard to reproduce.
<noodles775> axw: ack, txs.
<axw> rogpeppe: good morning
<rogpeppe> axw: fancy coming to the standup today?
<rogpeppe> axw: i don't know what time it is for you then, mind
<axw> rogpeppe: about 7:30 (dinner time ;)) - but I'll come today
<rogpeppe> axw: don't feel obliged - i just thought it might be nice if you could make it
<axw> rogpeppe: no I would like to come, and I haven't been to one yet
<rogpeppe> axw: cool
<rogpeppe> axw: how's it going anyway?
<axw> rogpeppe: not too shabby. working on manual deployment, and fixing a debug-log bug atm
<rogpeppe> axw: cool
<rogpeppe> axw: some time i hope we can have a way of changing the debug level in an environment dynamically
<axw> rogpeppe: I believe thumper has a branch in the works
<axw> it would be nice
<rogpeppe> axw: ah, that's good to know
<axw> jam1: if it interests you: https://bugs.launchpad.net/juju-core/+bug/1212148
<_mup_> Bug #1212148: rsyslog accumulator strips newlines on openstack <juju-core:New> <https://launchpad.net/bugs/1212148>
<dimitern> rogpeppe: hey
<rogpeppe> dimitern: yo!
<dimitern> rogpeppe: a relation is not an entity, right? we cannot reuse the LifeGetter for it
<rogpeppe> dimitern: hmm, yeah
<dimitern> rogpeppe: ok
<rogpeppe> dimitern: we could make it an entity, i guess
<dimitern> rogpeppe: I don't think this is a good idea
<dimitern> rogpeppe: it's not used as other entities are
<rogpeppe> dimitern: the AllWatcher reports on changes to it, not too differently from other entities
<rogpeppe> dimitern: entities are not all used the same
<dimitern> rogpeppe: yeah, but there's no agent or client that uses it as its main entity
<rogpeppe> dimitern: that's not true of Service either
<rogpeppe> dimitern: or Environment
<noodles775> I'm trying to reproduce bug 1208504, but get "Cannot find requested image 81078" when trying to bootstrap on HP Cloud. If anyone has any experience with HP Clound and knows what I'm doing wrong, please let me know :) http://paste.ubuntu.com/5984281/
<_mup_> Bug #1208504: "error: no instances found" Post bootstrap on HP Cloud  <papercut> <juju-core:In Progress by michael.nelson> <https://launchpad.net/bugs/1208504>
<dimitern> rogpeppe: hmm
<dimitern> rogpeppe: we can do that later, if we decide to do it anyway
<dimitern> rogpeppe: and what will be its key? there are 2 keys that are used - id int and a key string (created from the endpoints)
<dimitern> rogpeppe: the string key is _id in the doc, but is some places the id is also used
<rogpeppe> dimitern: i'd presume the latter, but there might well be reasons why that's a bad idea
<dimitern> fwereade_: you around?
<dimitern> and it's even less clear about endpoints
<dimitern> relationUnit seems it can be implemented at client-side with only a few methods on the server-side
<dimitern> fwereade_: when you have some time, we need to have a chat about how best to implement certain uniter objects in the api
<rogpeppe> does anyone else see this test failure on trunk? http://paste.ubuntu.com/5984415/
<dimitern> rogpeppe: you need to update goamz
<rogpeppe> dimitern: ah
<rogpeppe> dimitern: wow, that's an unexpected way for it to fail (updating goamz did fix it)
<rogpeppe> dimitern: do you know what the issue was?
<dimitern> rogpeppe: yes, it stems from the changes to support instance-state in juju status
<axw> rogpeppe: sorry, I should have sent an email
<rogpeppe> axw: did you update the dependencies file?
<axw> rogpeppe: uhmm. I forget. I think so...
<rogpeppe> axw: cool
<axw> rogpeppe: the issue fixed in goamz was in the testing server, to pass through the instance state
<axw> previously it was always blank, hence the test failure
<axw> back later for the standup
 * rogpeppe has just discovered how to speed up ec2 local tests from 3 minutes to 6 seconds
<dimitern> rogpeppe: how?
<rogpeppe> dimitern: change s3.attempts
<dimitern> rogpeppe: and still all pass?
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: cool!
<dimitern> rogpeppe: is that in juju or goamz?
<rogpeppe> dimitern: goamz
<dimitern> rogpeppe: ah
<mgz> yeah, the ec2test local bits are... fun in many ways
<rogpeppe> mgz: apologies for that :-)
<mgz> :)
<rogpeppe> mgz: i did the bare minimum required to get something working.
<rogpeppe> mgz: what issues are you having, BTW?
<mgz> general lack of flexibility mostly
<rogpeppe> mgz: right; it wasn't designed to be flexible
<mgz> tests in juju-core rely on hardcoded values that are in goamz, and aren't otherwise exposed or chnageable from inside a test, for instance
<rogpeppe> mgz: the instance address == instance id thing?
<mgz> like that, yes
<rogpeppe> mgz: i don't think it should be too hard to interject flexibility in some places
<rogpeppe> dimitern, mgz: https://codereview.appspot.com/12918044
<mgz> looking
<dimitern> me too
<rogpeppe> i'd like to have a separate project containing AttemptStrategy, so that both juju-core and goamz can use it, but i'm not sure how gustavo would feel about that
<dimitern> rogpeppe: lgtm
<rogpeppe> dimitern: thanks
<mgz> also. I still wonder if this is really the right way of dealing with eventual consistency things. we poll-wait in a bunch of cases where actually it doesn't matter and operations could just continue
<mgz> I guess it's simpler like this at least
<rogpeppe> mgz: eventual consistency is very hard to deal with
<rogpeppe> mgz: can you think of a better plan for dealing with it?
<jam1> rogpeppe: I don't know if you remember me mentioning the s3 thing about 2 months ago, and getting pushback from exposing it as a knob from goamz. I think there was some buy-in that we could have a flag for "disable retries entirely"
<rogpeppe> jam1: ah, ok
<rogpeppe> jam1: i wondered about that
<mgz> well, there are a bunch of places where pushing the handling up a few layers would mean we could make better choices
<rogpeppe> jam1: environs/ec2 is ridiculously slow to test
<jam> rogpeppe: yep
<mgz> but again, I think it's pretty set that it needs to be handled inside goamz, which limits things
<jam> It triggers because we check to see if you have any local tools/images/etc that we should be using, and the s3 eventual-consistency code kicks in saying "I don't have anything in your private bucket yet, but I might in 3 seconds"
<rogpeppe> mgz: what places are you thinking of that we are poll-waiting when we don't need to?
<rogpeppe> jam: yeah. that sounds reasonable - that may well happen for real when we're uploading tools
<jam> rogpeppe: so it is fine that it triggers in real life, adding 3-5s on every bootstrap isn't a big deal. In the case of the test suite, it is detrimental.
<rogpeppe> jam: absolutely
<jam> goamz's double *isn't* eventually consistent. :)
<rogpeppe> jam: indeed
<jam> I was also quite surprised that we were spending 3+min in time.Sleep()
<rogpeppe> jam: almost exactly three minutes in the case of environs/ec2
<noodles775> Could someone please set the following MP to Approved so it lands? https://code.launchpad.net/~michael.nelson/juju-core/1183159-ssh-options/+merge/179422
<dimitern> noodles775: done
<noodles775> ta
<rogpeppe> jam, mgz, dimitern: alternative formulation that might be more niemeyer-acceptable
<rogpeppe> https://codereview.appspot.com/12740044
<dimitern> rogpeppe: reviewed
<rogpeppe> dimitern: the other one is just as unclean and should have the same warning really
<rogpeppe> dimitern: well, it's a bit less clean, i suppose as it's less direct
<rogpeppe> dimitern: (the new CL that is)
<jam> rogpeppe: what about putting the Retry boolean into the s3 constructor?
<jam> AutoRetry perhaps?
<rogpeppe> jam: that's ok if the tests have access to where the S3 instance is being constructed
<jam> rogpeppe: well, at least it would be exposed from codebase A into codebase B, and codebase B can expose it up the stack as necesasry.
<rogpeppe> jam: there are many possibilities here
<jam> I'm a *big* fan of having as little mutable global state as possible
<jam> dimitern: IIRC the specific objective Gustavo had for the attempts logic was that "real" code shouldn't ever have to tweak this setting because we should make sure it Just Works.
<jam> I'm not sure that I agree 100%, because site-specific tweaking is always a possilibity (what if your ping time means it takes longer to see a consistent state?)
<jam> For our particular use case, the boolean fits better what we actually need
<jam> and is nice an minimalist
<dimitern> well, we can have "just works" and better tests in the same time
<rogpeppe> jam: the sense of the boolean should be the other way if it's exposed in S3
<rogpeppe> jam: although
<rogpeppe> jam: i hope nobody is initialising the S3 struct themselves
<jam> fwereade_, rogpeppe, dimitern, axw, mgz, wallyworld_, natefinch: standup time:
<jam> https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471
<hazmat> jamespage, re mulitple networks and juju (bug 1188126) , is there any order to how quantum attaches the nics, or is quantum api usage the only way to resolve the ambiguity?
<_mup_> Bug #1188126: Juju unable to interact consistently with an openstack deployment where tenant has multiple networks configured <canonistack> <serverstack> <juju:New> <juju-core:Triaged> <https://launchpad.net/bugs/1188126>
<jamespage> hazmat, the order quantum attaches nics is unhelpful
<jamespage> hazmat, its attaches then in alpha numeric order for the uuid of the networks
<jamespage> hazmat, i'd prefer to see some sort of option to specific which network to use in juju - then the instances can be connected to the quantum networks in a deterministic order - i.e. a preferred admin network type arrangement
<jamespage> hazmat, that might also fit well with using LXC containers when adding extra network interfaces for each container
<jam> axw: I did say you could get back to work, not go back to your personal life :)
<jam> have a great evening
<mgz> natefinch: I'll catch up with you after lunch here to do some review/code things
<natefinch> mgz: cool. probably going to have to be in a few hours, due to my daughter's doctor's appointment.  Not ideal for your schedule, I know. I'll ping you when I'm back.
<mgz> sure!
<hazmat> jamespage, ala vpc_id/quantum_net_id for an environment?
<hazmat> jamespage is this primarily a hosts issue when building a cloud, ie the baremetal gets networks for each of the guest tenants its hosting?
<jamespage> hazmat, no - I hit this purely with openvswitch overlay networking
<jamespage> where I can have any number of L2 networks attached to each instance
<hazmat> jamespage, does any of that mapping of nic to net id get exposed  in instance metadata, or is quantum the only way to resolve?
<hazmat> jamespage, you'd want to spec a net id for private net usage via an env setting? or have juju resolve the entire net topology via quantum
<jamespage> hazmat, not sure about metadata (it might - I'll check)
<jamespage> its all exposed in quantum of course
<jamespage> hazmat, specing a net-id for the private network as an env setting works for me
<axw> jam: hah, thanks :)
<sidnei> can i get a pair of eyes on https://codereview.appspot.com/12859043/ ?
<sidnei> noodles775: did you get the tests to run to completion?
<rogpeppe> time for lunch
<noodles775> sidnei: which tests?
<sidnei> noodles775: all of them *wink*
<noodles775> sidnei: sorry - you've lost me. Are you talking about a juju-core test run for which branch?
<sidnei> noodles775: for any branch, iirc you were having trouble getting a clean test run of the whole suite, and i told you i was running only tests for specific packages.
<noodles775> sidnei: yes, complete test run works fine. I don't remember the context last week where I was having issues. Perhaps before I had the correct mongedb version?
<sidnei> noodles775: could be. and you're in raring not saucy right?
<noodles775> sidnei: saucy locally, but all my dev is on a precise canonistack instance.
<sidnei> noodles775: ok, so you run the tests on precise.
<noodles775> yep.
 * sidnei tries on a precise lxc
<dimitern> rogpeppe, jam, mgz: https://codereview.appspot.com/12906044 - endpoints refactoring as we discussed
<dimitern> rogpeppe, jam: thanks!
<dimitern> rogpeppe, jam, mgz: next one https://codereview.appspot.com/12748044 - relations as entities
<dimitern> rogpeppe: I might have missed something important there
<rogpeppe> dimitern: looking
<sidnei> rogpeppe: if you get a chance, https://codereview.appspot.com/12859043/
 * sidnei lunches
<cruejones> anyone ever see this on juju bootstrap ... 'juju open.go:89 state: connection failed, will retry: dial tcp 127.0.0.1:37017: connection refused'
<cruejones> full verbose details @ http://pastebin.ubuntu.com/5985297/
<dimitern> cruejones: why sudo ? is that the local provider?
<cruejones> yup
<dimitern> cruejones: do you have mongo running on 127.0.0.1:37017 ?
<dimitern> mgz: ping
<cruejones> dimitern: does not look like anything is running at that port
<dimitern> cruejones: does it run on something like 10.0.3.x ?
<dimitern> cruejones: that's the lxc container address where it should run
<dimitern> rogpeppe: thanks for the review - updated with your suggestions
<cruejones> dimitern: nope
<dimitern> cruejones: are you using the tarball version of mongo, as described in the README?
<dimitern> cruejones: ah, sorry you probably followed the procedure there and did make install-dependencies ?
<mgz> dimitern: hey
<cruejones> dimitern: ha, no
<dimitern> mgz: https://codereview.appspot.com/12748044/ please?
<dimitern> cruejones: did you get juju-core from source?
<cruejones> dimitern: just updated to latest juju-core, setup my environment.yaml and ran 'sudo juju bootstrap'
<mgz> dimitern: okay
<dimitern> cruejones: if you can run the tests successfully, that's the first step: "go build ./... && go test ./..." in juju-core/
<cruejones> dimitern: ok, having to go there is not so user friendly though
<cruejones> mramm: ^
<dimitern> cruejones: well, I haven't seen this error, sorry, but then again I haven't used the local provider yet
<dimitern> cruejones: most likely it's an issue with the correct version of mongo and/or lxc prerequisities - that's why we've updated the README to streamline the process
<mramm> cruejones: we are doing two things to make this better 1) adding error messages when the packages aren't installed correctly
<mramm> and 2) adding a juju-local-provider package that installs both juju and the local provider dependencies
<cruejones> dimitern: sorry, don't get me wrong I really appreciate the help - trying to point out where things could and do go wrong with first impressions
<cruejones> or I am just clueless and/or bad luck ... probably a both ;)
<dimitern> cruejones: you're right, it's not user friendly, but we're trying to make it better and appreciate sharing the problems you encounter
<mramm> cruejones: definitely, we have to fix this so the process feels seamless and we very much value anybody pointing out where it is not
<dimitern> cruejones: can you try getting juju-core from source and retry the same, possibly following the steps in the README to see if it helps? 1.12 was some time ago and some changes were already made for the better
<cruejones> dimitern: will do after some mid-morning meetings, let you know how it goes .. thx
<henninge> fwereade_: ping
<dimitern> henninge: he's out today
<henninge> Ah, that's why. ;-)
<henninge> dimitern: Will he be back tomorrow?
<dimitern> henninge: I think so
<henninge> cheers
<dimitern> mgz: does it look ok?
<rogpeppe> sidnei: you've got a review
<sidnei> cool, not as bad as i expected ;)
<sidnei> rogpeppe: do the changes to instanceData doc need some sort of schema migration between versions?
<mgz> dimitern: fine I think, though I'm not sure I get all the implications
<rogpeppe> sidnei: i don't *think* so
<dimitern> mgz: since nothing is using relation as entities yet, there are no implications I can see
<mgz> fair enough
<dimitern> mgz: the only issue might be that now you can attach annotations to relations as well
<dimitern> rogpeppe, mgz: which might be a good thing
<dimitern> but there's no api for that anyway
<sidnei> rogpeppe: on a related topic, i think this is wrong: http://paste.ubuntu.com/5985486/
<sidnei> rogpeppe: i think the append should be inside the if s != ""?
<rogpeppe> sidnei: i don't *think* so
<rogpeppe> sidnei: i think you're allowed to say "mem= disk=10M"
<rogpeppe> sidnei: to say "no constraint on memory, but i want to specify that"
<rogpeppe> sidnei: i think that's true for any constraint attribute
<sidnei> rogpeppe: but that's on HardwareCharacteristics.String()
<rogpeppe> dimitern: i don't think you can now attach annotations to relations
<sidnei> rogpeppe: so it ends up as 'mem= ' in juju status
<rogpeppe> sidnei: ha, yes
<dimitern> rogpeppe: while looking at the code it seemed so, but I might be wrong
<rogpeppe> sidnei: only if you've got zero memory of course
<rogpeppe> dimitern: i think you can only set annotations on entities that embed annotator
<sidnei> rogpeppe: right, which is never the case. the one i hit was one case with OsDisk being nil, but it should never be nil anyway.
<rogpeppe> sidnei: i think the hardward characteristics stuff was probably just copied naively from constraints
<sidnei> it definitely looks so
<rogpeppe> sidnei: they're so similar (and they share some identical code) that i think they could both sit usefully in the same package
<dimitern> rogpeppe: ah, right!
<rogpeppe> sidnei: i don't think the h/w characteristics stuff should use uintStr at all, which is just there for that constraint logic
<rogpeppe> if hc.Mem != nil {strs = append(strs, fmt.Sprintf("mem=%M", hc.Mem))}
<sidnei> rogpeppe: %M? is there an extra char missing there?
<rogpeppe> sidnei: ha ha, %d i meant
<sidnei> k, will make that change.
<rogpeppe> %dM
<rogpeppe> in fact
<sidnei> rogpeppe: in the comment 'Just "return" should be do ok here.' that's meant for the 'return nil' line?
<rogpeppe> sidnei: it's meant for the if and the return
<natefinch> mgz: back finally
<mgz> natefinch: ace
<sidnei> rogpeppe: oh, i guess i could just 'return err' without the if you mean? since err will be nil anyway?
<sidnei> ah, not even that since it assigns to err
<sidnei> got it now
<rogpeppe> sidnei: yeah, and since you've already declared the err return variable, you can use a bare return
<natefinch> rogpeppe: what's our stance on bare returns? In general I try to avoid them, since they sometimes can return things you didn't mean to return
<rogpeppe> natefinch: i think in very short functions where it's easy to see everything at a glance, they're ok
<rogpeppe> natefinch: that said, i try to avoid them on most occasions too
<rogpeppe> natefinch: in this case, the function's only 3 lines long - it seems reasonable there
<natefinch> rogpeppe: yeah, for short functions it doesn't matter much... I just usually get to the return and think "really? I'm trying to save 3 (or whatever) characters?" so in practice I never actually use bare returns even though I'm not fundamentally against them
<rogpeppe> natefinch: yeah, i see that pov
<natefinch> rogpeppe: I sorta wish I didn't even have the option, then I wouldn't have to argue with myself about it ;)
<rogpeppe> natefinch: yeah. adg shares your view there, i think.
<sidnei> allenap: around?
<sidnei> or jtv
<allenap> sidnei: Hi there.
<sidnei> allenap: i assume you are somewhat knowledgeable about azure things. im looking at this table http://msdn.microsoft.com/en-us/library/windowsazure/dn197896.aspx and wondering which of the two columns for 'os disk-space' i should use to fill in some values
<sidnei> allenap: or maybe that already comes from roleSize?
<sidnei> oh, it seems to be defined there already, just need to pick one of the two
<allenap> sidnei: Yeah, that's right, but let me check in GWACL.
<sidnei> allenap: yeah, im looking in gwacl/rolesizes.go, i just don't know which of the two values to use
<allenap> sidnei: NewRole's first argument can be "ExtraSmall", for example.
<allenap> sidnei: That seems to be what Azure expects.
<sidnei> allenap: sorry, let me be more explicit. when i do 'juju deploy' in azure, do i get a 'cloud service' or a 'virtual machine', more explicitly even, the instance that comes up has OSDiskSpaceCloud MBs or OSDiskSpaceVirt MBs in it's root disk?
<sidnei> allenap: this is for exposing the os disk size in HardwareCharacteristics and constraints as part of bug #1201503
<_mup_> Bug #1201503: Add os disk constraint <constraints> <juju-core:Triaged by sidnei> <https://launchpad.net/bugs/1201503>
<allenap> sidnei: You get the VM sizes.
<sidnei> allenap: awesome, thanks!
<rogpeppe> does anyone know anything about Config.SSLHostnameVerification ?
<rogpeppe> it's only mentioned in one place, and that's just to say it can't be used
<sidnei> rogpeppe: addressed comments + added azure info per discussion with allenap above.
<sidnei> is fwereade_ not around today?
<dimitern> sidnei: nope, he's off today
<sidnei> dimitern: feel like doing a review? https://codereview.appspot.com/12859043/
<dimitern> sidnei: sure
<mgz> sidnei: I have a few quibbles on that branch
<mgz> will post comments ina sec
<rogpeppe> sidnei: do you think we ever want to accept blank values for h/w specification attributes?
<sidnei> thanks mgz
<sidnei> rogpeppe: afaict those are mainly filled from instance type, so no?
<rogpeppe> sidnei: i suspect that we don't want the 'if str != ""' parse logic in instance.go
<rogpeppe> sidnei: (which makes the code simpler, and usefully different from that in constraints, i think)
<sidnei> indeed
<dimitern> sidnei: you've got a review
<dimitern> g'night all
<sidnei> mgz: waiting for your comments
<sidnei> jam: iiuc you're the one looking into migrations between versions?
<sidnei> ah, beat me to it
<sidnei> mgz: i like the root disk suggestion, i was afraid someone would take 'os-' as openstack as well.
<rogpeppe> sidnei, mgz: i nearly suggested root-disk too
<rogpeppe> i like that as a name
<rogpeppe> although to be accurate it should probably be root-disk-size, though that's probably more verbose than we want
 * rogpeppe smells dal cooking downstairs. time to stop for the day.
<rogpeppe> g'night all
<sidnei> natefinch: i renamed to root-disk per rog/mgz suggestion.
<natefinch> sidnei: cool :)
<sidnei> natefinch: applied one fix, replied to the rest. seems like it's mostly stylistic issues which i have no opinion about, but neither rog or dimitern catched, so my assumption is that they are fine with them. i'll wait until tomorrow to get a second pass.
<natefinch> sidnei: yeah, nothing terribly important.
<natefinch> arosales: you around?
<arosales> natefinch, hello
<natefinch> arosales: hey, was talking to John Meinel, looking for something to work on in juju, and he suggested the HP no instances bug you'd looked at
<natefinch> arosales: https://bugs.launchpad.net/juju-core/+bug/1208504
<_mup_> Bug #1208504: "error: no instances found" Post bootstrap on HP Cloud  <papercut> <juju-core:In Progress by michael.nelson> <https://launchpad.net/bugs/1208504>
<arosales> natefinch, I think michael, noodles775, was working on it
<arosales> natefinch, you may want to touch base with him
<sidnei> natefinch: yeah, noodles775 has a branch started but seems like it was not on the right track
<natefinch> arosales: has there been anything done on it since the comments on the bug?
<natefinch> arosales: ok, thanks
<natefinch> Yeah, I saw the comment from John about fixing the providers. Luckily, I'm not averse to diving in and changing a bunch of stuff if it makes the code better.  But I'll touch base with Noodles first.
<arosales> natefinch, I am not 100% sure. Best bet would be to confirm with noodles775
<natefinch> arosales: cool, ok, thanks.
<arosales> natefinch, I am guessing the notes are up to date, but I can't say for certain.
<sidnei> natefinch: what about https://bugs.launchpad.net/juju-core/+bug/1130372 ? seems like an interesting one
<arosales> natefinch, have you searched papercut bugs?
 * arosales gets the link
<natefinch> arosales: yeah, I have. I fixed a couple, the HP bug was just what John suggested looking at
<arosales> https://bugs.launchpad.net/juju-core/+bugs?field.tag=papercut
<arosales> ah ok
<natefinch> sidnei: that definitely looks like something we need to address
<sidnei> https://bugs.launchpad.net/juju-core/+bug/1187959 is kinda papercut-ery but might be a bit more involved
<_mup_> Bug #1187959: juju does not detect instance launch error, waits forever? <juju-core:Triaged> <https://launchpad.net/bugs/1187959>
<natefinch> sidnei: it says it's assigned to Danilo, though that was as of 6/15 and nothing's been posted since then
<sidnei> i don't think danilo is working on it
<arosales> natefinch, some low hanging, wishlist bugs:
<arosales> https://bugs.launchpad.net/juju-core/+bug/1201628
<_mup_> Bug #1201628: HP Cloud Boiler-plate (juju init) template has AWS info <bitesize> <juju-core:Triaged> <https://launchpad.net/bugs/1201628>
<natefinch> sidnei, arosales: We have a team meeting in about 12 hours and a standup a few hours after that, I'll talk to people then. Either of those look appropriate, but the pyjuju seems more important, since we don't want to dissuade people from upgrading from  pyjuju
<arosales> https://bugs.launchpad.net/juju-core/+bug/1201628
<_mup_> Bug #1201628: HP Cloud Boiler-plate (juju init) template has AWS info <bitesize> <juju-core:Triaged> <https://launchpad.net/bugs/1201628>
<arosales> https://bugs.launchpad.net/juju-core/+bug/1210593
<_mup_> Bug #1210593: Show the user all current running environments <juju-core:Triaged> <https://launchpad.net/bugs/1210593>
<sidnei> https://bugs.launchpad.net/juju-core/+bug/1190985 is another favorite of mine
<arosales> given I am partial as I submitted those bugs ;-)
<natefinch> haha
<arosales> natefinch, sorry pyjuju?
<natefinch> arosales: https://bugs.launchpad.net/juju-core/+bug/1130372
<arosales> natefinch, ah ok
<natefinch> arosales, sidnei: I'll keep those on my shortlist if John doesn't have something specific he wants me to work on.
<arosales> natefinch, https://bugs.launchpad.net/juju-core/+bug/1201628 is my vote
<_mup_> Bug #1201628: HP Cloud Boiler-plate (juju init) template has AWS info <bitesize> <juju-core:Triaged> <https://launchpad.net/bugs/1201628>
<arosales> but like I said I am partial
<arosales> ;-)
<natefinch> arosales: that one *is* pretty embarassing... and I presume pretty trivial to fix
<arosales> sorry I meant to paste, https://bugs.launchpad.net/juju-core/+bug/1201628
<_mup_> Bug #1201628: HP Cloud Boiler-plate (juju init) template has AWS info <bitesize> <juju-core:Triaged> <https://launchpad.net/bugs/1201628>
<arosales> ugh
<arosales> this one
<arosales> https://bugs.launchpad.net/juju-core/+bug/1210576
<_mup_> Bug #1210576: Show exposed URL after exposing and on command 'show' <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1210576>
<natefinch> haha
<natefinch> arosales: ahh, neat idea, definitely
<natefinch> arosales: definitely makes for a really good first impression (not that the rest of juju isn't pretty impressive as-is).
<arosales> yup, but john may have some other priorities too, so I think your thought process on checking with his is the correct approach.
<natefinch> arosales: yep
<natefinch> well, it's EOD for me here.
 * thumper looks around for others working
 * thumper remembers that he is on the other side of the planet
 * davecheney waves
<thumper> gym time
<davecheney> thumper: ping
#juju-dev 2013-08-15
<davecheney> hey upgrade-charm --switch works
<davecheney> that is awesome
<davecheney> wallyworld_: was there a command to print the current enviroment ?
<davecheney> nm, juju switch without a name
<wallyworld_> yeah
<wallyworld_> davecheney: how did the hp cloud demo go? well i hope
<davecheney> wallyworld_: yeah, it's working well
<davecheney> marco is demoing juju deployer
<wallyworld_> great
<davecheney> and about to wow them with an import of a whole openstack HA setup
<davecheney> entirely hands off
<wallyworld_> good luck :-)
<davecheney> wallyworld_: thanks, we've tested this
<davecheney> well, at least once
<wallyworld_> how many machines do you need for openstack HA?
<davecheney> dozen at least
<wallyworld_> can't wait till containers are cooked a bit more so we can reduce that
<davecheney> man, the lack of the auto layout in the gui is killing us
<wallyworld_> i thought they were going to work on that?
<wallyworld_> something was said in oakland anyway
<davecheney> wallyworld_: yes, i wonder why it hasn't landed
<davecheney> wallyworld_: have you used upgrade-charm --switch much ?
<wallyworld_> no, not at all sorry
<davecheney> fair enough
<wallyworld_> is there an issue?
<davecheney> i guess that is thumper/rog/fwereade
<davecheney> wallyworld_: no, it looks like it works
<wallyworld_> great
<davecheney> we didn't relalised it was done
<davecheney> which is good
<davecheney> because it hepls when you need to fix a charm locally
<sidnei> thumper: just as a heads up, i hope you haven't started on the lxc-clone thing discussed at iom, i'm planning to dive down into it tomorrow as soon as my other branch lands.
<wallyworld_> arosales: not sure if you are around - do you have a preference for juju tools location? perhaps "http://juju.canonical.com/tools" afaiui, we want to avoid using ubuntu in the url? i need to ask IS if they can set up that location
<axw> wallyworld_: when you have a moment, can you please review this (regarding syslog)? https://codereview.appspot.com/12909044/
<axw> rsyslog even - apparently you wrote that
<wallyworld_> sure
<axw> ta
<wallyworld_> axw: it was written with the understanding that we would not deploy services to the bootstrap node unless in a container. clearly now that we have the --to option, that is no longer true :-/
<axw> wallyworld_: yeah I figured that was the case
<axw> my "solution" is still not perfect even
<axw> if you have multiple state servers... they're not going to see each others' logs
<davecheney> wallyworld_: that url looks good
<wallyworld_> davecheney: ta. afaik, juju.canonical.com doesn't exist yet, so i'll see if IS can set everything up for me
<axw> davecheney: heya. I reproduced that line-ending issue on lcy02 yesterday
<davecheney> axw: and I just ran into it on ec2
<davecheney> it doesn't happen every time .... ?!?
<axw> davecheney: seems the rsyslog conf file got mangled, and the "\n" got interpreted
<axw> ah
<axw> wtf :(
<axw> davecheney: if yo utake a look at /etc/rsyslog.d/25-juju.conf, then you'll see a " on its own line
<axw> it should be up a line, with a literal \n before it
<axw> I better update the bug that says it works fine on EC2 then
<wallyworld_> axw: sorry, i marked the merge proposal as not lgtm since there's a cleaner was we can check if a machine is a state server, and we can abstract it behind a method for possible subsequent refactoring when we go to ha
<davecheney> urgh, ec2 speed is killing me
<axw> wallyworld_: nps, thanks for the review
<davecheney> just bootstrapped a machine that had so little io bandwidth it could not unpack the tools
<wallyworld_> let me know if you have questions
<axw> wallyworld_: will do, ta
<wallyworld_> axw: see state/machine.go - it defines the jobs a machine can run and the db document for machine containing the Jobs attr
<davecheney> axw: nice diagnosis
<thumper> sidnei: it is all yours :-)
<axw> wallyworld_: I would've done that actually, but Machine isn't exposed to SimpleContext at the moment. I see now there's only one place that NewSimpleContext is called tho, so it's easy enough to pass one through
<wallyworld_> axw: cool, i didn't really look too much at the surrounding code. i assumed a deployer or whatever would have had access to the machine it was operating on
<axw> wallyworld_: nope, but it will soon :)  thanks for the comments
<wallyworld_> np
<thumper> hi wallyworld_
<thumper> wallyworld_: how's the world?
<wallyworld_> g'day
<wallyworld_> busy
<wallyworld_> lots of simplestreams stuff swirling around
<wallyworld_> how was iom? sounds like it was good
 * thumper nods
<thumper> wallyworld_: yeah pretty good
<thumper> didn't catch as much flak for the api lateness as i thought we might
<wallyworld_> :-)
<davecheney> thumper: thanks for implementing update-charm --switch
<wallyworld_> thumper: i'm on an ease of use mission, so hopefully soon juju will Just Work better
<thumper> davecheney: ah... I didn't
<thumper> wallyworld_: awesome
<wallyworld_> lots to do though
<thumper> wallyworld_: got general agreement between me, jam and fwereade__ around -v meaning verbose, not showlog
<wallyworld_> \o/
<thumper> also, useful output by default
<thumper> with -q meaning quiet
<wallyworld_> thumper: what about recognising that log != cli
<thumper> yeah, we are going to log into $JUJU_HOME/juju.log (or something) for every command
<wallyworld_> log should go to file, with different stuff going to stdout for cli feedback
<thumper> kinda like .bzrlog
<thumper> wallyworld_: ack
<wallyworld_> cool, can't wait
<davecheney> wallyworld_: +1
<davecheney> if it means we don't have to run the command twice to debug it
<davecheney> actually +1 million yen
<wallyworld_> i would have done it myself if i had the time
<thumper> well, we wanted to make sure that we had agreement before starting
<thumper> so we wouldn't get rebuffed in reviews
<wallyworld_> yeah, that too
<davecheney> thumper: wallyworld_ anything that gets most of hte debug shit out of hte logs
<wallyworld_> my initial email didn;t get much love way back when
<davecheney> it's really hard to demo juju and explain what is happening when 1/2 the line is full of shit like file:line etc
<thumper> davecheney: ack
<wallyworld_> yep
<davecheney> same goes for the agents
<wallyworld_> thumper: the ability for a user to define a log config and have that used would be great too
<davecheney> which tend to log most things as debug then at info
<davecheney> wallyworld_: -1, overengineering
<thumper> wallyworld_: I have a branch for that
<davecheney> there should be one log level
<davecheney> silent or not silent
<wallyworld_> davecheney: disagree
<davecheney> anything more than that goes into this log file
<thumper> davecheney: there are two users here
<thumper> developers and users
<thumper> developers need configurable logging
<thumper> users shouldn't have to look at it
<thumper> ever!
<wallyworld_> what he said
<wallyworld_> also, as a developer, i might want to configure logging to there is a stdout sink
<thumper> which is why we want useful output by default
<thumper> and verbose meaning "show me more" not "show me the developer log messages"
<davecheney> thumper: sounds good to me
<davecheney> i never want to know about it
<axw> davecheney: "10mbit ethernet is considered high speed in australia"  ;)  I was thinking the same thing, but then remembered there will be other guests
<davecheney> axw: honestly, this is the best we get
<axw> yeah I know
<davecheney> the bigger problem will be their shitful captive wifi portal
<davecheney> axw: wallyworld_ thumper is there a way to show the default constraints ?
<davecheney> similar to juju get $SERVICE
<wallyworld_> davecheney: you mean the constraints specified on bootstrap i assume, which become the so-called environment constraints
<davecheney> wallyworld_: yes
<wallyworld_> i think there may be a get command of some sort, but i've not used it. i'll see if i can find what it may be
<davecheney> juju get-constraints blocks
<davecheney> juju get-constraints $SERVICE returns nothing
<wallyworld_> get-env constraints  ??
<thumper> davecheney: I don't know actuall
<thumper> y
<wallyworld_> i looked at the code, get-constraints should be the thing to use
<wallyworld_> not sure why it is blocking
 * thumper tries to remember which nights in oakland were non-claimable
<thumper> putting in expense reports
<davecheney> thumper: none of them :)
<thumper> davecheney: well, we did have a team dinner didn't we?
 * thumper is trying to remember
<thumper> I remember the go meetup
<thumper> but having trouble thinking of other evenings out
<thumper> except for that horrible vegan place
<davecheney> thumper: the steak house on thursday ?
<thumper> thursday was the go meetup
<davecheney> tuesday ?
<thumper> sounds right, but I'm having a lot of difficulty remembering it
<thumper> ah
<thumper> kincaids
<thumper> yes, I remember now
<davecheney> thumper: i don't think it matters
<davecheney> just make sure the number of days add up to the right number
<thumper> davecheney: it matters to me :)
<davecheney> thumper: it was tuesday
<thumper> ta
<axw> that was quick, thanks thumper
<thumper> np
<thumper> I saw it come up
<thumper> axw: trivial ones means only one review needed
<axw> thumper: yup, thank you
<thumper> actually we have an experiment coming up where we are moving to a "one lgtm needed" for a month
<thumper> to see how it goes
<axw> hmmk
<thumper> axw: there have been a number of times where good work is blocked on noone around to do a second review
<axw> I had a change last week where I got a "LGTM, this looks solid" and then a "NOT LGTM this is going to break everything catastrophically" ;)
<thumper> the idea is that we'll be doing weekly reviews of modules
<thumper> axw: which was that on?
<axw> thumper: a change to handling of Dying in the uniter/modes code
<axw> so that a uniter would die if it hadn't started yet
<axw> I can get the number if you're interested
<thumper> no, that's fine
<thumper> axw: also, technically I should be on-call reviewer today
<thumper> this morning was catching up on all the emails
<thumper> now expenses etc.
<axw> thumper: ah yes, what does that mean exactly?
<thumper> generally, that your primary focus should be reviewing others code
<thumper> and some bug work when not reviewing
<thumper> if all that is done
<thumper> then you can code
<axw> gotcha
<thumper> otherwise people tend to prioritise their own work
<thumper> and no one looks at bugs :)
<axw> thumper: doesn't mean people are going to call you up out of hours for reviews? :)
<axw> heh
<thumper> no
<thumper> :)
<axw> thumper: speaking of reviews, did you see my manual provisioner stuff?
<axw> I'd like to know if I'm on the right track...
<thumper> axw: I haven't but I want to look at it
<axw> thumper: thanks. I have other stuff to carry on with for now
<thumper> I need to get my wallet for the IOM expenses, but the dog is sleeping in front of the door and I really don't want to wake her up
 * thumper nods
<axw> hehe
<axw> oops, that change was noise.. state isn't used directly in the code creating the deployer
<axw> thumper: turns out I should've added that method to state/api/agent/Entity, if anywhere at all
<axw> that code seems much more  minimalist than state
<axw> should I be leaving it that way, or go ahead and add it in there too?
<thumper> why do you feel it should be moved?
<axw> no I made a mistake in the first instance; the code I need to update uses api, not state
<axw> I'm updating jujud to check if it's a state server when creating a deployer context
<axw> and jujud is working with api/agent Entity objects, rather than state.Machines
<thumper> ah..
<thumper> makes sense
<thumper> why does jujud care if it is a state server for the deployer context?
<axw> thumper: because the deployer needs to not install rsyslog forwarding config if it's on a state server, or we get a feedback loop
<axw> bug 1211147
<_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1211147>
<axw> thumper: thanks for the review comments. I realise it's ugly/messy, was more interested in making sure I wasn't way off with how I understood things should work
 * thumper nods
<thumper> axw: I think you are kinda there
<axw> so what's the "manual" provider going to do?
<axw> nothing?
<thumper> right
<thumper> a manual provider isn't really a provider at all
<axw> it exists just to be referred to?
 * thumper nods
<axw> ok
<thumper> more importantly, so we can distinguish between environmentally created machines
<thumper> and ones you have added manually
<thumper> the whole "destroy-machine" thing needs thought
<axw> yeah didn't even touch that yet
<thumper> it seems that there needs to be a hook inside the machine agent so it can nuke itself
<thumper> as in, remove the upstart job
<thumper> and die
<thumper> right now
<thumper> without any way to discriminate between top level machines
<thumper> the environ provider will raise an error
<thumper> when it notices that a "manual" machine needs to die
<thumper> because it can't find it
<thumper> at least the process will die and restart
<arosales> wallyworld_, +1 on juju.canonical.com/tools
<wallyworld_> arosales: great thanks, will get the ball rolling
<arosales> wallyworld_, thanks, are you going to file an RT with IS?
<wallyworld_> yes
<wallyworld_> will do it today
<arosales> wallyworld_, thank you
<wallyworld_> np, i'm looking forward to getting this all working
<arosales> wallyworld_, ben also reports simple streams for HP and Azure are all set with the SRU for Azure inprogress to enable 12.04 on Azure
<arosales> +1 on "it just works"
<arosales> :-)
<wallyworld_> arosales: so we can theoretically delete the hack up simplestream data i did for hp cloud
<wallyworld_> i might test that locally
<wallyworld_> s/might/will
<arosales> +1 on testing it :-)
<arosales> wallyworld_, I think you have access to the sub hp account
<wallyworld_> yeah, i uploaded new metadata a day or so ago for the demo
<arosales> wallyworld_, cool, and thank you
<wallyworld_> np
<wallyworld_> hope things are going well for you guys over there
<arosales> its marco and dave that are in the trenches in japan
<arosales> I am just assisting remotely
<wallyworld_> ah ok
<arosales> but I hear things are going better now
<wallyworld_> \o/
<arosales> is tim around
<arosales> thumper, ?
<arosales> thumper, need to confirm your plan/travel for the Brisbane sprint
<arosales> email is also out on that subject
<bigjools> o/ arosales
<arosales> bigjools, hello
<arosales> davecheney, all going well with the charm shool?
<thumper> arosales: here
<thumper> arosales: was having a coffee and some fud
<arosales> thumper, umm coffee . . .
<thumper> arosales: oh yes, as much as davecheney disagrees, I think they'll be fine without me
<arosales> thumper, hey I was just going to check if you need to book any travel for the brisbane sprint
<thumper> arosales: no, no I don't
<arosales> thumper, ah, ok
<thumper> :)
<thumper> arosales: I should go edit some docs I guess
<thumper> if the flights were better, it perhaps could have been an option
<thumper> but it is another five days away
<arosales> ah
<thumper> which doesn't gel with the family
<thumper> I had it calculated for me that i have been away for a total around 8 months since I started with canonical
<arosales> wow
<arosales> thumper, I updated the ss
<davecheney> thumper: arosales yup
<davecheney> whatever
<davecheney> after this week, I don't think there is anything you four can throw at me
<thumper> davecheney: don't worry, you get to see me in October
<davecheney> thumper: :heart
<thumper> davecheney: really, whazzup?
<davecheney> thumper: this week has been ... trying
<arosales> davecheney, battle hardened
<axw> wallyworld_: second attempt: https://codereview.appspot.com/12852044
<wallyworld_> axw: will look soon, just finishing a bit of stuff
<axw> nps, thanks
<bigjools> thumper, damn, I was looking forward to insulting you in person
<thumper> bigjools: sorry dude
<arosales> bigjools, lol
<thumper> bigjools: you'll have to wait until next year, or when you're up to travelling again
<arosales> bigjools, I don't know thumper seems like a dude not to mess with in person
<arosales> probably safer over IRC
<thumper> family has a trip planned for gold coast next winter
<bigjools> arosales: yes he's good at body checking
<thumper> bigjools: and foot stomping ;-)
<bigjools> ah yes :)
<bigjools> thumper: cool, we could head there too and ruin it for you :)
<thumper> arosales: just realised that SS is spread sheet, not secret service
<arosales> ah yes, sorry about that
<bigjools> arosales: I just realised I am going to miss the Monday of the sprint
<thumper> arosales: did have me wondering why they cared...
<bigjools> but I was only planning on being there one day anyway
<arosales> thumper, lol
<arosales> bigjools, well at least you will be able to make it for a few days.
<bigjools> thumper: so anyway yes, I am unlikely to be travelling anywhere for the forseeable future :/
<arosales> ok fellas I am going to get some sleep.  Have a good day!
<axw> good night
<arosales> davecheney, keep rocking the charm school, and may the juju force be with you.
<bigjools> nn arosales
<thumper> I was considering leaving the office and coming back later for the meeting
<thumper> but I can hear the kids not cleaning the kitchen
<thumper> perhaps I'll just stay here for a bit
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1212538
<_mup_> Bug #1212538: cmd/juju: deploy --to a non existent machine fails too late in the process <papercut> <juju-core:New> <https://launchpad.net/bugs/1212538>
<davechen2y> what the heck is --format=constaints ?
<davechen2y> what the heck is --format=constraints ?
<davechen2y> mmm, love that legacy smell
<rogpeppe> mornin' all
<axw> morning rogpeppe
<mgz> rogpeppe: can you review the trivial https://codereview.appspot.com/12768044/ please?
<rogpeppe> mgz: LGTM
<mgz> how go-idomatic would it be to do something like... range over an anon struct, in real code, rather than just for table tests?
<mgz> it's the closest thing I have to the python idiom of say `for key in ('thing', 'other'): val = getattr(obj, key, None); if val is not None: ...`
<fwereade> wallyworld_, ping
<wallyworld_> heya
<fwereade> wallyworld_, wondering about the deployer syslog change
<wallyworld_> ah, i did an initial review but forgot to take another look
<fwereade> wallyworld_, it's not immediately apparent how the unit logs will get to all-machines.log if we just make forwarding contingent on non-state-serveriness
<fwereade> wallyworld_, but you probably know if/how it will work without me digging ;p
<wallyworld_> you mean unit logs on state server machines?
<fwereade> wallyworld_, yeah
<wallyworld_> i must admit i didn't think that bit through as i was more concerned with stopping the loop
<wallyworld_> i think it will just be a tweak to the syslog conf when deploying on state server
<wallyworld_> but i'd have to look at the syslog docs
<wallyworld_> to see the specs for a "append to file" module
<wallyworld_> as opposed to a "forward to this port" module that we use now
 * wallyworld_ still hates that we support deploying directly to state servers
 * fwereade sympathises but doesn't think we can kill it yet
<fwereade> wallyworld_, hmm, when we get onto the HA work we'll probably have to consider job changes
<wallyworld_> yeah, likely
<fwereade> wallyworld_, and we could perhaps then default to creating state server machines without JobHostUnits
<fwereade> wallyworld_, but then I'm not totally wild about directly exposing machine jobs to the user
<fwereade> worth bearing in mid though
<wallyworld_> when do we expose machine jibs to the usrr?
<fwereade> wallyworld_, we'd have to if we wanted to keep a path to allow it
<wallyworld_> sorry, i must have missed something
<fwereade> wallyworld_, ah sorry
<fwereade> wallyworld_, I think we have to allow that path, evil and crackful though it may be, to enable cheap operation even without containers
<fwereade> wallyworld_, if we get containers everywhere the conflict disappears
<fwereade> wallyworld_, but if we have containers in *most* places I think it'd be ok to disallow units on 0 by default
<wallyworld_> fwereade: "allow that path" - not quite getting the context
<fwereade> wallyworld_, allow JHU on machine 0
<wallyworld_> right
<fwereade> wallyworld_, I think we might be able to gradually restrict it
<fwereade> wallyworld_, but I suspect it will remain a valid (if distasteful) use case for the forseeable future
<wallyworld_> well if we can deploy containers to machine 0 easily, then problem solved
<fwereade> wallyworld_, yeah, but that depends on provider capabilities
<wallyworld_> yeah, i realise :-(
<wallyworld_> it was more wishful thinking
<rogpeppe> mgz: i'd need to see the actual code
<fwereade> wallyworld_, but if we have "enough" container addresability support we might be able to switch off JHU on 0 by default -- but we'd need some way to reenable it in annoying contexts... hence some level of user exposure to machine jobs
<wallyworld_> well maybe not Jobs per se - just allow users the ability to toggle the capability to host units from a semantic/logical perspective
<fwereade> wallyworld_, yeah, indeed, I'm just worried that we'll end up with full control of machine jobs in a really ad hoc way
<fwereade> wallyworld_, I'm just fretting uselessly really )
<wallyworld_> i think you are - we would never expose jobs directly :-) just the abstract notion of "capability"
<mgz> rogpeppe: in this case, I think I should actually just reuse an existing struct, and just not append if its lacking a value
<rogpeppe> mgz: i'm afraid i can't provide any useful input when i don't know the context :-)
<rogpeppe> fwereade: chat?
<fwereade> rogpeppe, I'm starting to think that this connection just straight-up hates hangouts, can we do it by irc perhaps?
<rogpeppe> fwereade: it's just possible that if you *start* a hangout, it might be hosted somewhere more sensible for you
<rogpeppe> fwereade: although that's possibly not how they work at all though
<mgz> rogpeppe: `bzr diff -c1646 lp:~gz/juju-core/ec2_addresses` then `bzr diff -c1647 lp:~gz/juju-core/ec2_addresses`
<mgz> does gocheck have any composable matchery things
<mgz> ?
<mgz> to do something like... c.Assert(aStruct, DeepEquals, ....actually, this doesn't make any sense in a go context, the types wouldn't match
<mgz> that makes checking a list of structs where you don't care about one field more annoying
<mgz> all: will miss standup again as it's Thursday in town day
<fwereade> so, I'm going to try to join the standup hangout in a mo, but this connection seems to throw a fit whenever I do so
<fwereade> so if I'm not there don't wait for me
<natefinch> rogpeppe, jam: standup?
<rogpeppe> natefinch: good point
<dimitern> fwereade, rogpeppe: relation ops https://codereview.appspot.com/12990043
<dimitern> fwereade, rogpeppe: I forgot KeyRelation() - i'll be adding it
<dimitern> fwereade: shouldn't I do that?
<dimitern> rogpeppe, fwereade: poke :)
<rogpeppe> dimitern: i'm on it
<dimitern> rogpeppe: thanks
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> rogpeppe: didn't want to use the struct by value to avoid copying in stateEndpointToParams
<dimitern> rogpeppe: does that apply?
<rogpeppe> dimitern: no
<dimitern> rogpeppe: ok
<rogpeppe> dimitern: i mean, it will be copied, but the cost of that is irrelevant
<rogpeppe> dimitern: there's more cost from the allocation
<dimitern> rogpeppe: :)
<dimitern> rogpeppe: noted
<noodles775> natefinch, sidnei: RE bug 1208504, I've got a fix (now that I've got an HP account to test) - it's pretty straight forward. Though unit-testing it is proving a little more difficult (the test double for openstack doesn't seem to have something similar to the ec2 double's SetInitialInstanceState).
<_mup_> Bug #1208504: "error: no instances found" Post bootstrap on HP Cloud  <papercut> <juju-core:In Progress by michael.nelson> <https://launchpad.net/bugs/1208504>
<natefinch> noodles775: ahh, good.
<dimitern> noodles775: there is something more powerful in goose: ControlHooks
<dimitern> noodles775: take a look at ProcessFunctionHook and how it's used
<dimitern> noodles775: it allows you to return anything from a test double's method
<noodles775> dimitern: hrm, I was looking at RegisterControlPoint but couldn't see how it can be used to change the actual state.
<noodles775> dimitern: Ah? OK - I could only see how I could return an *error* for any method. Let me look again.
<dimitern> noodles775: what do you need? change the state of a started instance?
<noodles775> dimitern: yep
<noodles775> dimitern: I'm happy to look again though - I thought the ServiceControl was the only option.
 * noodles775 looks.
<dimitern> noodles775: in addition to returning the error, you have access to the nova object n
<noodles775> dimitern: I tried that too (ie. doing sc.(novaservice.Nova) but got impossible type assertion: novaservice.Nova does not implement hook.ServiceControl (RegisterControlPoint method requires pointer receiver)
<dimitern> noodles775: try sc.(*novaservice.Nova) perhaps?
 * dimitern gtg - be back in a couple of hours
<noodles775> Thanks dimitern, I'll try that.
<dimitern> fwereade: one last poke for https://codereview.appspot.com/12990043/
<noodles775> dimitern: so the conversion works, but there's not much exposed on Nova (like .servers :/). I'll look more closely.
<mgz> noodles775: what's your current diff, out of curiosity?
<rogpeppe> fwereade: https://docs.google.com/document/d/1NRkTkZiVXcOL7wPQJ8GGxnW3YtPiwRQThTGNNT92HW4/edit?usp=sharing
<sidnei> fwereade: good morning! got a few reviews on https://codereview.appspot.com/12859043/ already, but waiting on your ack before merging
<noodles775> mgz: https://code.launchpad.net/~michael.nelson/juju-core/1208504-post-bootstrap-hp-no-instances-found-try2/+merge/179899
<noodles775> mgz: It is also possible to fix it without increasing the shortAttempt, but it would rely on us checking for HP specific status codes (ie. "BUILD(spawning)")
<noodles775> mgz: For the various HP BUILD states, see http://docs.hpcloud.com/api/compute#ServerStates
<rogpeppe> fwereade: one thing i didn't mention: i was thinking of moving environs/{all,azure,dummy,ec2,local,maas,openstack} to environs/provider as the environs name space is getting really cluttered these days
<mgz> noodles775: either of those approaches seems fine to me
<noodles775> mgz: cool, thanks.
<fwereade> rogpeppe, +1 to that
<cruejones> seems there was a juju upgrade in juju/stable today which removed 'juju' cmdline symlink?  anyone else experience this?
<fwereade> sidnei, hmm, root-disk would make sense if we did make sure to allocate a suitably sized ebs volume for the 0 instance-store cases
<fwereade> sidnei, but adding that seems likely to be non-trivial
<sidnei> fwereade: i don't understand what you're referring to there? we're using ebs instances and those all have an 8G root disk which can be resized on boot by passing the proper block device mapping.
<sidnei> i man, i don't understand the '0 instance-store' cases
<sidnei> s/man/mean
<fwereade> sidnei, ah, I see -- but it seems a bit strange to be claiming 8G for every ec2 instance type
<fwereade> sidnei, the implication is that you can't get anything on ec2 with more than 8G
<sidnei> fwereade: that's currently the case until we change goamz to allow passing a custom block device mapping yes.
<fwereade> sidnei, ...and while I see your point that that's what you get for the root disk in each case, it rather seems that that is actually nothing to do with instance type
<fwereade> sidnei, I may be too ec2 focused, but it seems very strange to be presenting (say) an m1.xlarge as having 8G rather than >1T
<sidnei> fwereade: as mentioned in my first reply, this is only for the root disk, not the additional ephemeral disks
<fwereade> sidnei, OTOH the instance storage available is maybe somewhat irrelevant given that you need to know where it is to make use of it and the average charm does not
<fwereade> sidnei, I don't suppose you know offhand whether it's possible to map root-disk cleanly to maas? dimitern, mgz, jam?
<fwereade> or azure? jtv, rvba?
<rvba> Hi fwereade.
<sidnei> fwereade: i already added the azure mapping
<fwereade> sidnei, <3
<sidnei> fwereade: since the ebs root device (/dev/sda1) is resizable on boot, a proper follow-up branch would be to set the RootDisk to nil on the instance types and pass through the requested constraint via block device mapping to resize it to the requested size
<sidnei> so for ec2 all instance types would match, and you would always get the size you requested on the constraint for the root disk
<fwereade> sidnei, that would be *fantastic* and would basically negate my quibbles; how realistic is it in the near future?
<fwereade> rvba, in fact, you know maas too, right?
<sidnei> fwereade: very realistic, the required changes in goamz seem to be fairly simple to implement.
<rvba> fwereade: I do, but I confess I'm not entirely sure what you're talking about right now :)
<fwereade> rvba, how trivial is it to ask for a machine with a particular amount of root disk space?
<rvba> fwereade: the machines is installed using d-i so it's only a matter of passing the right config to d-i.
<rvba> I think it's something that smoser talked about a few months ago.
<fwereade> sidnei, ok, I'll do a quick pass, but I'll be happy on the condition we do get followups addressing that issue
<fwereade> rvba, I don't recall that -- from a juju-core perspective, what would need to be done to hook up a rot-disk constraint for maas?
<fwereade> er root-disk
<sidnei> fwereade: ack. my priority list is landing this to unblock webops on openstack, then look at quick lxc-clone in local provider, then return and implement the block device mapping.
<fwereade> sidnei, that sounds awesome -- would you add a juju-core bug and assign yourself please, so it doesn't get lost?
<sidnei> definitely
<rvba> fwereade: hum, actually i was talking about a slightly different thing: I was talking about configuring d-i so that the partitions follow a certain plan.
<fwereade> rvba, I thought you might be, I didn't quite follow what you said at first :)
<rvba> fwereade: okay, so the disk size is something that MAAS collects from the lshw output (same as RAM capacity) so it's probably only a matter of exposing that as another available constraint.
<fwereade> rvba, cool, so we can basically make it a straight passthrough
<rvba> Yes.
<sidnei> https://bugs.launchpad.net/juju-core/+bug/1212688
<_mup_> Bug #1212688: ec2: pass root-disk constraint via block device mapping <juju-core:Triaged by sidnei> <https://launchpad.net/bugs/1212688>
<fwereade> rvba, fantastic, tyvm
<rvba> welcome
<fwereade> sidnei, great, thanks
<sidnei> i'll file one about maas too
<fwereade> sidnei, you read my mind :)
<fwereade> sidnei, LGTM
<fwereade> sidnei, thanks very much
<sidnei> i'll sneak in a change to ignore and log root-disk on maas for now, just like it does for cpu-power
<sidnei> i wonder what to do for local provider
<mgz> sidnei: ignoring there seems sensible as well
<sidnei> ok
<sidnei> seems local just passes through all constraints without even looking at them
<fwereade> sidnei, yeah, there'll be more to do wrt containers in general too but I don;t think that needs to be on your plate
<sidnei> fwereade: natefinch had a couple interesting comments on his review, which are mostly stylistic issues, i wonder what is your opinion.
<sidnei> https://codereview.appspot.com/12859043/#msg11
<mgz> sidnei: one final rename in state/machine.go and lgtm.
<fwereade> sidnei, I'm not too bothered by the overwriting of results on error, IMO any error renders the whole object trash anyway
<sidnei> mgz: ha! good catch.
<fwereade> sidnei, but, whoops, I completely missed the "" case, and I need to reload state to remember
<fwereade> just a mo
<mgz> fwereade is currently creating a multi-gig swapfile
<fwereade> sidnei, I think that ""->0 is correct -- the implication is "I don't care" rather than "fall back to environment constraints"
<fwereade> sidnei, since 0 matches everything it's fine -- in arch, for example, it needs to be handled a little differently but again "" implies "I don't care", overriding whatever env defaults may exist
<fwereade> sidnei, to fall back you just don;t specify
<mgz> the old constraint case was "blah=" meaning "use the default" and "blah=any" meaning "unset the constraint"
<mgz> so, "mem=" would ask for a machine with 512mb or whatever, and "mem=any" wouldn't check ram at all
<fwereade> mgz, yeah; but IIRC there was general agreement that juju-level defaults were madness and horror, and use of a special string was just too unpleasant
<fwereade> mgz, so essentially the "default" decision is now just left up to the providers
<mgz> fwereade: indeed
<sidnei> there were missing tests for rootdisk in findCleanMachineQuery, added those and verified working too.
<natefinch> fwereade, mgz, sidnei:  maybe change the code to have "" return early, and make it more obvious that it's purposeful?
<fwereade> natefinch, mgz, sidnei: not sure it helps a great deal tbh, but happy to follow consensus
<natefinch> possibly even with a comment? :)
<fwereade> natefinch, mgz, sidnei: a named "doesntMatter" const might be even better
<fwereade> natefinch, mgz, sidnei: not that that's a *good* name
<fwereade> natefinch, the trouble with a comment vs a name is the necessity of repeating it
<natefinch> fwereade: yes, a name is as good as a comment. Mostly I just wanted it to be clear that this is the intended effect, and not just happenstance from the way it was written
<fwereade> natefinch, +1 indeed
<natefinch> fwereade: especially because once it's out there, there's going to be 1000 people who have discovered the happenstance and now rely on that functionality
<fwereade> natefinch, quite so
<natefinch> or just one really loud person
<natefinch> fwereade: do you have a suggestion for a bug for me to work on. The one John mentioned to me yesterday seems taken care of.  I got a bunch of suggestions from arosales, but I'm not really sure on the relative priority of them
<fwereade> natefinch, I can *certainly* find you a bug if there's nothing addressability-related you can do with mgz
<natefinch> fwereade: I'd be happy to help mgz, but it seems it's a little difficult to split off pieces
<mgz> natefinch: so, we still need to land the remaining ec2 bits
<mgz> then do basically the same for azure/maas, which are even simpler
<natefinch> mgz: I like simple
<mgz> so, azure is nice and easy, no gwacl changes needed
<mgz> the structs built from the xml have some address details in them, that just wants exposing under the Addresses() method as we did for ec2
<fwereade> natefinch, mgz: excellent
<fwereade> natefinch, I have a couple that aren't so simple if mg thinks it's likely that he'll be unable to use yu for a day or 2 at any stage
<fwereade> natefinch, but I'd be interested to know arosales' requests, because he should have a good handle on the ones that are really upsetting charmers, and they are an important audience
<natefinch> fwereade: https://bugs.launchpad.net/juju-core/+bug/1210576
<_mup_> Bug #1210576: Show exposed URL after exposing and on command 'show' <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1210576>
<natefinch> fwereade: https://bugs.launchpad.net/juju-core/+bug/1201628
<natefinch> fwereade: https://bugs.launchpad.net/juju-core/+bug/1210593
<_mup_> Bug #1210593: Show the user all current running environments <juju-core:Triaged> <https://launchpad.net/bugs/1210593>
<arosales> fwereade, that is the one we discussed at the sprint
<mgz> natefinch: see RoleInstance in xmlobjects.go in gwacl for starts
<fwereade> arosales, I think I can split a simple and useful one out of the first one
<natefinch> mgz: cool.. what branch are you working on, should I just check it out and work on that one as well?
<fwereade> arosales, `juju <verb-tbd> wordpress/0` => open browser if service exposed/complain if not
<fwereade> arosales, the wait-for-some-unit-to-be-exposed functionality is a little tricky
<mgz> niemeyer: thanks for the review
<arosales> fwereade, ya the first "juju show" I think has more utility
<mgz> natefinch: a fresh branch should be the right thing, shouldn't be any conflicts
<natefinch> mgz: cool
<mgz> natefinch: I'm going to wrap up the ec2 bits and get them landed
<fwereade> arosales, listing running environments is a good one, but I think it'll have to wait a bit until rog/ian's work on environments has progressed a little
<natefinch> mgz: how do I do that branch switching you were doing?
<natefinch> mgz: bzr help switch was not very helpful :)
<mgz> `bzr switch trunk` to get you onto trunk again (or create trunk) `bzr pull` for latest code, `bzr switch -b new_feature` to switch to a new feature branch
<arosales> fwereade, ack. The thought there was I am a new juju user I tried to deploy a few times in AWS not I am not certina if I have instances spending my money
<mgz> where 'new_feature' is an appropriate name of some kind
<fwereade> arosales, I think that if I was mistrustful of juju I'd be checking the console directly -- and that environments can be removed from environments.yaml anyway, rendering the functionality incomplete at best -- so I'm not 100% sure of the utility
<fwereade> arosales, I understand the intent
<natefinch> mgz: bzr switch trunk says bzr: ERROR: Not a branch: "/home/nate/code/src/launchpad.net/trunk/"   when I do it from code/src/launchpad.net/juju-core (which is targetted at trunk)
<fwereade> arosales, and hopefully a clear way of listing will land soon
<arosales> cool, and I saw you mentioned the "show" command may be a bit tricky, but I have _full_ confidence in juju core devs :-)
<mgz> natefinch: you need a new branch is you're changing the workflow you're using probably
<mgz> *if
<mgz> or use -b if you've not switched to that mode yet
<mgz> `bzr branches` should list several things
<mgz> if it just says (default) you're not using colocation yet
<natefinch> mgz: ok, cool, I get it
<arosales> mramm, per the cross team meeting seems a lot of folks are hitting https://bugs.launchpad.net/juju-core/+bug/1188126
<_mup_> Bug #1188126: Juju unable to interact consistently with an openstack deployment where tenant has multiple networks configured <canonistack> <openstack> <serverstack> <juju:New> <juju-core:Triaged> <https://launchpad.net/bugs/1188126>
<arosales> fwereade ^
<fwereade> arosales, hmm, thanks for the heads up
<arosales> fwereade, issue hit by IS, Server, and Lanscape
<fwereade> arosales, yeah, there's no way that's wishlist
<fwereade> arosales, tyvm
<arosales> fwereade, could you update the importance?
<fwereade> arosales, just made it high
<arosales> fwereade, thank you
<mgz> that bug is too much of a hydra
<fwereade> mgz, I suspect this interacts with what you're doing
<mgz> everyone commenting means something else by it
<mgz> james' would be fixed by using the neutron api, or fixing openstack to be less insane about how it selects networks, or making cloud-init understand multiple networks
<mgz> elmo's they worked around by not doing that in the end
<mgz> and I don't even know what the issue the landscape guys have is really
<natefinch> the fix is obviously to have juju disable all networks it isn't using ;)
<fwereade> haha
<arosales> other juju core bugs that were surfaced in the cross team meeting, fwiw
<arosales> https://bugs.launchpad.net/juju-core/+bug/1170337
<_mup_> Bug #1170337: maas provider: missing support for maas-specific constraints <openstack> <juju-core:Triaged> <https://launchpad.net/bugs/1170337>
<arosales> and
<arosales> https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1200878
<_mup_> Bug #1200878: Upgrade breaks existing pyjuju deployment <apport-collected> <papercut> <regression-release> <saucy> <juju-core:New> <juju-core (Ubuntu):Triaged> <https://launchpad.net/bugs/1200878>
<mgz> if cloud-init doesn't bring up all networks, we're kinda screwed. with the current changes, we can try multiple addresses, or be smarter at selecting which one to use, but giving they're selected by netron UUID it's a little painfule (and fragile) to do 'correctly' in juju
<mgz> we'd need to annotate the addresses in state with their internal id (after querying that over the netron api), and sort the list in the same fashion openstack does when selecting a preferred address
<fwereade> mgz, ouch, that sounds fragile indeed
<fwereade> mgz, how bad would it be to try all of them? it looks like there's a TODO in the code for that already
<mgz> we can try all of them, the fun part is then what do we tell charms via the legacy api, which expect one canonical address?
<arosales> mgz, (brainstrom) would it be helpful to pass cloud-init a param to init all detected networks?
<mgz> we'd need to predetermine which work, and mark some as broken
<mgz> yeah, cloud-init should learn how to bring up multiple networks
<mgz> then you can just select any
 * arosales asks smoser in #juju
<fwereade> mgz, or just not record the ones that don;t work?
 * dimitern is bacl
<dimitern> back*
<dimitern> fwereade: https://codereview.appspot.com/12990043/ please?
<fwereade> arosales, lp:1200878 is indeed awful
<fwereade> dimitern, sorry; on that now
<arosales> fwereade, ya mramm said there would be a story there. juju-core just needs to communicate what that store is with folks doing packaging (james page) and in general. When the story is ready.
<fwereade> rogpeppe, dimitern: re the bug above: shouldn't LoadState be detecting pyjuju environments?
<rogpeppe> fwereade: it won't get that far
<rogpeppe> fwereade: it's failing to create the config
<dimitern> fwereade: yeah, it's more than what danilos did it seems
<fwereade> rogpeppe, search for "no CA cert"
<fwereade> rogpeppe, checks in LoadState would let us catch that problem
<rogpeppe> fwereade: hmm, yes, you're absolutely right
<fwereade> rogpeppe, knowing exactly how to handle it is maybe trickier -- is it reasonable to assume that someone in this situation *must* have juju 0.7 installed
<rogpeppe> fwereade: pyjuju 0.7?
<mramm> fwereade: arosales: here's the release schedule: https://wiki.ubuntu.com/SaucySalamander/ReleaseSchedule
<rogpeppe> fwereade: is there anything in the py juju state info file that marks it out as py juju ?
<mramm> we need to think about what we do for it.   We will have a new cloud archive tools pocket, so I'm not sure exactly how critical it is -- but what we put into Saucy should be pretty stable if we can manage it
<hazmat> rogpeppe, state info  file?
<fwereade> rogpeppe, danilos did work to detect exactly this situation
<hazmat> rogpeppe, the unit state  files are completely different between the two
<fwereade> hazmat, this is the file in provider storage pointing to the state instance
<hazmat> fwereade, ah
<fwereade> rogpeppe, LoadState having to read two things to determine what sort of env it's talking to is fine by me
<mramm> anyway, thie release stuff is something for us to talk through next week
<fwereade> rogpeppe, alternatively, just sticking another key in that file would allow us to differentiate going forward
<rogpeppe> hazmat: they're both in the same provider-state file aren't they?
<hazmat> rogpeppe, they are but they have different keys for state servers
<rogpeppe> hazmat: cool
<mgz> rogpeppe: can I have a stamp on <https://codereview.appspot.com/12765043/> please? also, I'm assuming the landing process is just lbox submit still?
<hazmat> rogpeppe, pyjuju uses 'zookeeper-servers', juju-core uses 'state-servers'
<rogpeppe> fwereade: i think it could just look for len(StateInstances) == 0
<rogpeppe> fwereade: or, better, StateInstances==nil
<rogpeppe> fwereade: and assume that if that's the case, it's a pyjuju-created file
<hazmat> er... 'state-instances'
<rogpeppe> hazmat: yeah
<fwereade> rogpeppe, better yet
<rogpeppe> mgz: looking
<arosales> mramm, ack. I'll follow up with david c. when he returns.
<rogpeppe> mgz: reviewed
<mgz> rogpeppe: I also updated the other goamz proposal again, sorry for the bother :)
<rogpeppe> mgz: np
<hazmat> mgz, its not really about cloud-init understanding the networks afaics, its about juju understanding the right network to use
<hazmat> afaics
<mgz> okay, let's go in here :)
<mgz> hazmat: in james' case, that's true. but the intention of the IS migration was that both addresses would work for a time
<mgz> there's nothing on the juju side that I can see we can do to enable that.
<hazmat> hmm.. so there's a net device attached to the instance, with an ip allocated from openstack, but nothing for it on the host since its not configured/up there?
<hazmat> s/host/instance
<hazmat> yeah. cloud-init doing the net dance sounds sane, but then we come back to juju handling the multiple ip addresses/net devs.. and using the right one for its own usage.
<mgz> hazmat: yes, that's my understanding from agy's postmortem email to canonistack-announce
<fwereade> dimitern, reviewed
<hazmat> re merging to juju-core have the instructions changed from just $ lbox submit
<hazmat> oh.. tarmac
<mgz> hazmat: yup, you can use lp:rvsubmit bzr plugin if you like the submitty workflow
<rogpeppe> mgz: reviewed
<mramm> arosales: we should also add curtis from orange squad to the release planning discussion
<arosales> mramm, ack
<dimitern> fwereade: thanks!
<rogpeppe> go doc is going :-( https://codereview.appspot.com/12974043/
<rogpeppe> fwereade: do you have any idea why, in the local provider, environProvider.Open sets AgentVersion in the config?
<smoser> mgz, i'm interested in knowing what the question and solutoin was for networking things.,
<fwereade> rogpeppe, I *suspect* that it's part of thumper's hackery wrt upload-tools vs sync-tools, but I'm not directly familiar with the code in question
<arosales> mgz, smoser is referencing the question I surfaced in regards to bug 1188126 that you and hazmat were discussing ealier
<_mup_> Bug #1188126: Juju unable to interact consistently with an openstack deployment where tenant has multiple networks configured <canonistack> <openstack> <serverstack> <juju:New> <juju-core:Triaged> <https://launchpad.net/bugs/1188126>
<rogpeppe> fwereade: ah, i thought you were the principal reviewer
<rogpeppe> fwereade: what's the specific hackery you're talking about?
<mgz> smoser: the solution for one of those cases, was IS just not doing that for the lcy02 network migration
<mgz> right, I have to run now I'm afraid
<fwereade> rogpeppe, he abuses upload-tools pretty badly to mimic sync-tools -- there's some layer breaking around juju bootstrap
<fwereade> rogpeppe, I managed to convince him it was crazy at iom
 * rogpeppe should have spent more time looking at that code
<fwereade> rogpeppe, but a fix has as usual been put off until it actively hurts us -- it may be you're in that situation now
<fwereade> rogpeppe, fwiw the motivation for landing it as was was, well, we needed the local provider stat, and the effects seemed to be localized
<rogpeppe> fwereade: yeah, istr that
<rogpeppe> fwereade: i've just realised that if Prepare does all the work, then Bootstrapper is unnecessary. we can just have Prepare(cfg *config.Config) (Environ, error)
<fwereade> rogpeppe, awesome, ++simplicity
 * rogpeppe undoes a load of stuff
 * fwereade sympathises
 * rogpeppe is actually very happy about that
<rogpeppe> fwereade: how about this as a suggestion: the environment file contains *all* attributes that are not explicitly mentioned in the environments.yaml file (or later, the juju.conf file)?
<rogpeppe> fwereade: hmm, that's simple to implement (and results in a nicely predictable user model - stuff in the home directory is only consulted the first time an environment is prepared) but it will result in more attributes than we strictly need.
<dimitern> fwereade: updated https://codereview.appspot.com/12990043 - PTAL
<fwereade> rogpeppe, I'm not immediately keen tbh, I feel like every attribute we put in there that's not strictly necessary is likely to end up a dependency of some sort
<fwereade> dimitern, looking
<rogpeppe> fwereade: the way i'm seeing it is that the environment file binds the attributes, so that subsequent operations have a consistent view of them, rather than looking around in the home directory for authorized-keys, for example
<fwereade> rogpeppe, will we ever want to look up authkeys except at bootstrap time?
<fwereade> rogpeppe, those should be packaged up, sent to the server, and forgotten, I think
<rogpeppe> fwereade: no - quite a few attributes are only useful in the interval between preparation and bootstrap
<fwereade> rogpeppe, authkeys in particular is not useful *until* bootstrap, is it?
<rogpeppe> fwereade: yeah, but it seems a bit odd to have keys that are bound at different times. perhaps that's just me though. i'm still thinking it over.
<fwereade> rogpeppe, authkeys is crack anyway
<fwereade> rogpeppe, if a key represents anything it's a user
<fwereade> rogpeppe, the only reason it was ever in the environment is because we were trying to get away without a user model :/
<rogpeppe> fwereade: users are often represented by principals a.k.a. private keys
<rogpeppe> fwereade: so authkeys doesn't seem too bad to me
<fwereade> dimitern, wrt RelationInfo I was just asking whether RelationResult{RelationInfo, Error} might be cleaner -- it's probably not, if it doesn't make you immediately say "YEAH!" then I wouldn't bother
<fwereade> rogpeppe, I'm not saying we shouldn't have authorized keys -- just that they shouldn't be on the environment
<dimitern> fwereade: I didn't get what were you referring to as RelationInfo?
<fwereade> rogpeppe, the env should have users, and those users should themselves have authkeys
<rogpeppe> fwereade: i guess we'll still want to allow ssh access to the bootstrap node
<fwereade> dimitern, just struct {Key, Id, Endpoints}
<fwereade> rogpeppe, I can well imagine cases in which you don't want any ssh access enabled anywhere by default
<rogpeppe> fwereade: do you mean "the *state* should have users" ?
<dimitern> fwereade: ah, there is a RelationInfo in params already, and it's just Key and []Endpoint
<dimitern> fwereade: used by the allwatcher
<rogpeppe> fwereade: well, that's easy enough to arrange - just provide an invalid authorized key :-)
<fwereade> rogpeppe, ha, yeah
<fwereade> dimitern, hmm
<fwereade> dimitern, I reckon that one ought to have id in there as well, really
<dimitern> fwereade: I can add it, but I'll need to change a bunch of watcher tests
<fwereade> dimitern, let's not actually
<dimitern> fwereade: I'm trying it now
<fwereade> dimitern, I'm not willing to say "these two things in these two contexts are actually the same" because I think they're probably not -- the uniter has no reason to know the remote endpoint
<fwereade> dimitern, and while the allwatcher surely should include the relation id, that's not what we're concerned with here
<fwereade> dimitern, my point about LocalEndpoint is just that there's no reason for the uniter to know what service it's having relations with
<fwereade> dimitern, so why send that information?
<fwereade> dimitern, when we call Relation we can trivially find out the service the connected unit is a member of, and get that endpoint directly and send that down alone
<fwereade> dimitern, am I making any sense there?
<dimitern> fwereade: sorry, not really
<dimitern> fwereade: are you saying we don't need a "all endpoints" field at all?
<fwereade> dimitern, yeah
<dimitern> fwereade: then, we'll need an Endpoint() call for each service name, right?
<fwereade> dimitern, then the Relation type we expose to the uniter literally just has an Endpoint(no args) method
<fwereade> dimitern, because any uniter.Relation only needs to know one, and that's the one the API will be guaranteed to send anyway
<dimitern> fwereade: how about that case when we call rel.Endpoint(u.unit.ServiceName()) ?
<fwereade> dimitern, it's always u.unit.ServiceName -- we always know what that is ahead of time
<fwereade> dimitern, so we just call Endpoint(), and that gives us the only endpoint we have a right to know about
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers
<dimitern> fwereade: so what do we return on Relation() API call? only endpoints for our unit-tag?
<dimitern> fwereade: so we need both rel-tag and unit-tag to call the API Relation() methods
<fwereade> dimitern, that would be reasonable, I think, I had been imagining we could figure it out from knowing the connected entity, but that's probably sleazy/lazy/short-sighted
<dimitern> fwereade: ok, so I'll define a params.Relations, []params.Relation which has Relation and Unit string fields, expected to be the respective tags
<dimitern> fwereade: and then I can use the same params type for Enter and LeaveScope, Settings, etc.
<fwereade> dimitern, that sgtm
<dimitern> fwereade: ok then
<fwereade> dimitern, thanks, sorry for the hassle
 * fwereade bbl
<dimitern> rogpeppe, fwereade: updated https://codereview.appspot.com/12990043/ again - I hope this time I managed to do all suggestions
<dimitern> rogpeppe, fwereade: if it looks ok, I'll land it
<sidnei> uhm, anyone around for some go advice? niemeyer?
<niemeyer> sidnei: Here.. just a sec.. finishing a meeting
<sidnei> niemeyer: golxc has a Container interface with Clone(name), now i need to change that to Clone(name string, snapshot boolean, backingStore BackingStore, templateArgs ...string), but i guess to not break bw-compat i would need to create a new interface?
<niemeyer> sidnei: Hmm
<niemeyer> sidnei: There are lots of details around such a change.. hard to give good advice without much info
<niemeyer> sidnei: In general, yes, if you drop a method that satisfies an interface you don't satisfy th einterface anymore
<sidnei> niemeyer: and i can't add a new method to the same interface as anyone that should be implementing that interface will suddenly not be implementing it anymore right?
<niemeyer> sidnei: Exactly
<niemeyer> sidnei: But that's easy to solve
<niemeyer> sidnei: You can just define another interface that contains the new method
<sidnei> niemeyer: and embed the old interface?
<niemeyer> sidnei: Not necessarily
<niemeyer> sidnei: If they are alternatives to each other, they can be independent
<sidnei> uhm, they are not alternatives, the only difference is the new method, all the other existing methods are still required
<hallyn_> hm, 'juju bootstrap' claims my ec2 environment is already bootstrapped.  (it's not)
<thumper> hallyn_: it could be that the "special bootstrap file" has been writted to the storage
<thumper> hallyn_: try destroy-environment then try again?
<thumper> hallyn_: if it still says it is bootstrapped, then it is probably a bug
<thumper> could well be a bug anyway
<hallyn_> thumper: aaah!  that worked.  where is that special file?
<thumper> hmm...
 * thumper looks at the code
<hallyn_> thumper: thanks!
<niemeyer> sidnei: Sorry, system crashed badly
<hallyn_> thumper: no nm, don't waste your time on that right nwo :)
<niemeyer> sidnei: So the old method is required even for those implementing the new method?
<sidnei> niemeyer: nope, but all the other old methods in the interface are.
<thumper> hallyn_: "provider-state"
<thumper> hallyn_: it writes the state instance id there I think
<hallyn_> thumper: in what dir?  that didn't exist in my ~/.juju
<thumper> hallyn_: in the root of the private storage I think
<hallyn_> 'the private storage'?
<thumper> in ec2 speak, the private bucket
<thumper> we use "storage" internally
<thumper> anyone know where we are re ARM versions?
<sidnei> hallyn_: around?
<hallyn_> sidnei: yup
<sidnei> hallyn_: heya, getting the faster lxc-clone into juju, hitting a small roadblock
<sidnei> hallyn_: http://paste.ubuntu.com/5990680/
<hallyn_> sidnei: does /var/log/juju exist in the container?
<sidnei> hallyn_: if that's meant to be /var/lib/lxc/sidnei-local-machine-1/delta0/var/log/juju, then no
<sidnei> i guess i should create that in the base template anyway
<hallyn_> sidnei: in delta0, or in the underlying rootfs (whichever container that comes from)
<hallyn_> sidnei: what does 'grep rootfs /var/lib/lxc/sidnei-local-machine-1/config' give?
<sidnei> lxc.rootfs = overlayfs:/var/lib/lxc/sidnei-local-precise-template/rootfs:/var/lib/lxc/sidnei-local-machine-1/delta0
<hallyn_> sidnei: ok, then /var/lib/lxc/sidnei-local-precise-template/rootfs/var/log/juju existing would also suffice
<hallyn_> but yeah you can't mount to a nonexisting dir
<sidnei> hallyn_: ok, trying again.
<hallyn_> \o
<hallyn_> haven't had time today, but can't wait to try out the owncloud charm :)
<hallyn_> (using lxc)
<sidnei> thumper: did you see my question to gustavo above? do you have any ideas?
<sidnei> thumper: need to change the signature of golxc Clone() to pass extra args, but i dont want to break bw compat
<thumper> sidnei: which one?
<thumper> sidnei: well, we don't use the Clone method
<thumper> sidnei: so you won't break juju-core
<thumper> and I don't know of anyone else using it
<thumper> so go for it
<sidnei> lol wut
<sidnei> ok
<sidnei> thumper: https://code.launchpad.net/~sidnei/juju-core/lxc-clone-with-overlayfs/ and https://code.launchpad.net/~sidnei/golxc/clone-with-backing-store/ is what i have so far, it's getting pretty far then blowing up in cloud-init: http://paste.ubuntu.com/5990730/
<thumper> sidnei: what are the bits leading up to that?
<sidnei> thumper: following the steps from smoser, create a base template with no userdata, then lxc-clone with userdata, then lxc-start
<thumper> sidnei: so not running a juju-command
<sidnei> thumper: yes, as a juju command with the above two branches
<sidnei> so bootstrap then juju deploy ubuntu
<thumper> sidnei: well, it looks like the upstart job isn't being created in the userdata
<sidnei> it should be, i just moved userdata from lxc-start to lxc-clone, without changing anything
<sidnei> thumper: here's the generated userdata: http://paste.ubuntu.com/5990758/
<sidnei> oddly /var/lib/lxc/sidnei-local-machine-1/delta0/etc/init/jujud-machine-1.conf exists
<thumper> sidnei: hmm, sorry, no idea right now
<thumper> hmm...
<thumper> overlay problem??
<sidnei> mebbe
<sidnei> going to try again shortly
<sidnei> odd, the file is there inside the container after ssh in
<sidnei> but upstart doesn't know about it
<sidnei> ha, kill -HUP 1 to the rescue
<bigjools> word up
 * sidnei waves to bigjools
<bigjools> hey sidnei
<bigjools> how's the twins?
<sidnei> bigjools: they're doing great. though staying at home this week because it's freezing cold outside and they were coughing heavily over the weekend, after 2 weeks of antibiotics. :/
<sidnei> so a little more fun than usual on my afternoons
<sidnei> bigjools: still at the hospital?
<thumper> sidnei: IIRC I overheard some talk about overlayfs not working well with inotify
<bigjools> sidnei: thankfully no
<sidnei> thumper: indeed, smoser on that email thread that you started. this is bound to make things very interesting. i replied.
<thumper> bigjools: can I get you to boot up your maas?
<bigjools> thumper: sure.  /me needs to get ipmi working on the server node
 * thumper pretends to know what bigjools is talking about
<sidnei> bigjools: great to hear you're out
<bigjools> thumper: it would mean I could turn it on without getting out my seat :)
<bigjools> sidnei: thanks man
<sidnei> thumper: so kill -HUP 1 does the trick, booted a couple more units without problem
<bigjools> thumper: I need to dist-upgrade my saucy maas server if you're ok with that?
<thumper> bigjools: yep
<bigjools> ok - it will probably take down the appserver briefly
<thumper> bigjools: if you can tell me when it is up again, that be swell :)
<sidnei> thumper: https://code.launchpad.net/~sidnei/golxc/clone-with-backing-store/+merge/180444 https://code.launchpad.net/~sidnei/juju-core/lxc-clone-with-overlayfs/+merge/180445 both wip, will do more tests tomorrow
<thumper> kk
<bigjools> thumper: ok
<bigjools> thumper: try in ~15 minutes
<bigjools> or more to the point, ping me if I've not poked you by then
<thumper> kk
<thumper> bigjools: says 102 packages can be updated
<thumper> bigjools: want me to update them?
<bigjools> thumper: it's upgrading already
<thumper> oh
<thumper> ok
<bigjools> run top and you will see the disk getting hammered
<bigjools> ah the joys of a development release, package upgrade failures
<thumper> \o/
<thumper> system restart required
<thumper> wanna bounce it?
<thumper> bigjools: ^?
<bigjools> thumper: not yet
<bigjools> mysql is stuck
<thumper> kk
<thumper> so I should wait before I boot some maas nodes?
<bigjools> thumper: yes, I need to reboot in a moment
<thumper> kk
 * thumper afk for lunch and haircut
<bigjools> thumper: it's up
<arosales> any folks know here if juju does a log rotate in 1.13 >
<arosales> or should I file a bug
<davecheney> arosales: i think it *sohuld* log rotate
<davecheney> certainly for all-machines.log
<davecheney> wallyworld_: committed that
<wallyworld_> i didn't do the rotation bit
<arosales> okay that came up in #juju not sure if it was address in 1.x juju
<bigjools> hopefully it's just re-opening the log on a sighup?
<davecheney> bigjools: doubt it
<davecheney> dunno what loggo does
#juju-dev 2013-08-16
<bigjools> apps should not be rotating their own logs
<bigjools> twisted tried that and it was miserably shit
<davecheney> bigjools: was talking about reopening on HUP
<thumper> wallyworld_: aren't you off today?
<wallyworld_> thumper: yeah, about to go to in
<axw> thumper: if someone "not lgtm"s something, I need that same person to lgtm to go ahead - is that right?
<thumper> axw: it depends :)
<thumper> axw: what is it?
<axw> just a sec
<axw> thumper: https://codereview.appspot.com/12924043/
 * thumper looks
<davecheney> axw: depends, could you take them in a straight fight ?
<axw> heh :)
<thumper> axw: if you can address my comments, I can give you an LGTM
<thumper> axw: william's opposition was due to no test, since you've added one, I think you'll be fine
 * thumper goes to check on the pizza in the oven
<axw> thumper: thanks
<axw> thumper: that LC_ALL thing is existing. do you want me to update those things as I go?
<thumper> :) do you feel like making it more up to date?
<axw> ... yes
<axw> :)
<axw> I think I'll write a tool to check import groups
<axw> prevent myself from submitting crap
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1212903
<_mup_> Bug #1212903: cmd/juju: debug hooks should set a better PS1 <papercut> <juju-core:New> <https://launchpad.net/bugs/1212903>
<thumper> axw: That bug ^^^ seems pretty trivial to do right?
 * axw looks
<axw> thumper: yes should be pretty simple
<axw> I'll take a look in a moment
<davecheney> thumper: axw yeah
<davecheney> it would be super useful to have for our upcoming engagements
<davecheney> we're always zooming the screen
<davecheney> then the tmux output and the path become a real issue
<davecheney> for a charm the standard path consumes almost all the screen width
<thumper> davecheney: I fully agree with the bug description
<thumper> I think it is sound
<davecheney> sweet
<davecheney> i didn't think it would be a technical issue to fix
<davecheney> more ... philosophical
<thumper> axw: you aren't looking at breaking up the agent config are you?
<thumper> axw: because if you aren't, I'll take that on
<axw> thumper: not right now
<axw> okey dokey
<thumper> I kinda need it for some other work
<axw> thumper: nps. I'll work on bugs and continue with the manual provisioner
<thumper> as after talking with william, he convinced me that shoving env vars into the upstart scripts isn't very good
<axw> heh ok :)
<thumper> rewriting upstart scripts on upgrade is a little crazy
<thumper> but adding stuff to agent config isn't too bad
<axw> right
 * thumper recalls the need to write an email about juju containers
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1212916
<_mup_> Bug #1212916: uniter/worker: unit agent MUST remove stale unix socket <papercut> <juju-core:New> <https://launchpad.net/bugs/1212916>
<thumper> davecheney: if you put an importance on the bug, please mark it triaged
<davecheney> thumper: will do
<davecheney> sorry
<davecheney> will fix
<thumper> davecheney: I've done those two
<thumper> but for future reference
<davecheney> ok, i understand now
<axw> thumper: thanks, I've updated checkers -> jc. will merge
<thumper> axw: ack
<thumper> axw: if you run the trunk tests, do your ec2 ones pass?
 * thumper wonders if goamz has been updated
<axw> thumper: goamz has indeed been updated
<axw> are you getting a TestInstanceState failure?
<thumper> yes
 * thumper pulls goamz and tries again
<axw> I'll send an email now. Sorry, should have done it last week
<thumper> I am prepared to buy drinks for whoever makes the ec2 tests run faster (without skipping or producing incorrect results)
<axw> thumper: rogpeppe said he sped it down to 6s or something like that
<thumper> but decided not to commit it?
<axw> IIRC it required modifying goamz itself
<axw> I think there's some global variable that would affect the normal operation
<thumper> well, if he fixes it, I'll buy him some drinks :)
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1212936
<_mup_> Bug #1212936: cmd/juju: debug-hooks without hook fired should place the user into a simple hook environment <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1212936>
<davecheney> relation question
<davecheney> can you run the relation-get hook command during the relation-departed hook ?
<rogpeppe> mornin' all
<axw> morning rogpeppe
<rogpeppe> axw: hiya
<noodles775> dimitern: Hi! If you have a few minutes at some point, can you read what I tried RE testing with RegisterControlPoint etc on the decscription of https://codereview.appspot.com/12795045/ , and let me know what I have missed, or whether it's something that needs to be added to goose?
<dimitern> noodles775: I think an update to goose is in order, so that you can tweak the instance state in the test double
<dimitern> noodles775: sorry, I was wrong that you can tweak it without code changes
<noodles775> dimitern: k - do you think I should do that before landing that branch (ie. so it doesn't land without tests), or it's OK as is?
<dimitern> noodles775: I don't think increasing the timeout fixes anything
<dimitern> noodles775: it's still fragile
<dimitern> noodles775: if anything, we should fix the nova server filter to be smarter and return stuff not only in ACTIVE or BUILD states, but also hp-cloud specific BUILD(spawning)
<noodles775> dimitern: Yeah, I did try that, but it's still fragile (takes about 8-9 seconds to get to BUILD(spawning) from my testing)
<dimitern> noodles775: and to test it, you'll need to change goose to allow you to change the instance state when you start a server with the test double
<dimitern> noodles775: ok, let me rephrase - timeout by itself is not enough to fix it, you'll need the above things as well
<dimitern> noodles775: fixing the filter to use HasPrefix instead of == should be trivial
<noodles775> dimitern: That doesn't work - I tried that (see the description).
<dimitern> noodles775: it doesn't work, because it returns BUILD(networking) and at that state the instance is not yet accessible?
<noodles775> dimitern: It means that it returns as soon as HP sets BUILD(networking), but juju-core then never connects (I didn't dig deeper, but I assume we make some assumptions)...
<noodles775> Yes.
<dimitern> well, it seems obvious that the network is not yet up at that point
<noodles775> Right - but it continues to not connect. Let me paste you output, it may make more sense.
<noodles775> dimitern: http://paste.ubuntu.com/5991914/
<dimitern> noodles775: my point is, instead of HasPrefix, you could use server.Status == nova.StatusBuild || serverStatus == "BUILD(spawning)" (perhaps even define this as const HPStatusBuildSpawning)
<dimitern> noodles775: then run it 10 times or more to see if it passes every time, if sometimes it doesn't, then increase the timeout
<noodles775> dimitern: Right, I did do that and it then takes about 8-9 seconds - so it is better than 15s that it takes for ACTIVE. If you think it's better to use that way, I'll do that.
<dimitern> noodles775: the hp-specific build const should be definied in goose, like other nova.Status* consts I think, and a change to enable setting it in the double is in order, so you can test it
<noodles775> dimitern: yep, will do.
<dimitern> noodles775: thanks!
<noodles775> dimitern: np, thanks for the feedback and direction.
<dimitern> rogpeppe: hey
<rogpeppe> dimitern: hiya
<dimitern> noodles775: no worries, glad I can help
<dimitern> rogpeppe: updated https://codereview.appspot.com/12990043/
<rogpeppe> dimitern: looking
<dimitern> rogpeppe: cheers
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: thanks
<dimitern> fwereade: ping
<dimitern> blast.. william is out today
<dimitern> rogpeppe: I have a question
 * rogpeppe is all ears
<dimitern> rogpeppe: now that I changed Relation() to return only the endpoint of the auth'ed unit, should it return []params.Endpoint at all or just a single one?
<rogpeppe> dimitern: hmm, since there can only be one local endpoint, i think it should just return a single one
<dimitern> rogpeppe: then I shouldn't really use params.RelationInfo, but a copy of it without the Endpoints slice, just a single field
<rogpeppe> dimitern: i suppose so, but i can't get too worried about it.
<rogpeppe> dimitern: the comment in UniterAPI.Relation should definitely be updated though - i missed that
<dimitern> rogpeppe: ok
<dimitern> rogpeppe: did you have time to check the charm store for possible relation names?
<rogpeppe> dimitern: no, i'm going to do that this morning
<dimitern> rogpeppe: great
<rogpeppe> dimitern: i've also been trying to make some sort of sense of the config mess this morning. this is as far as i've got: https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AghcHASXWzXmdHdOX2htQVJITkdaYVh4VnZ2Z0RrQWc#gid=0
<dimitern> rogpeppe: looking
<rvba> Anyone up for a tiny review? https://codereview.appspot.com/12850045/
<rogpeppe> rvba: looking
<rogpeppe> rvba: LGTM
<rvba> rogpeppe: ta
<dimitern> rogpeppe: for openstack, the username, password and tenant-name are not generated, only the password is secret; auth-url and auth-mode are cloud-specific and not generated, access-key and secret-key are used with auth-keypair only (otherwise username/password are used) - both secret, region is cloud-specific, not generated or secret, both buckets can be generated and are not secret
<rogpeppe> dimitern: feel free to change the spreadsheet!
<dimitern> rogpeppe: ah, I wasn't sure I can
<rogpeppe> dimitern: if you're logged into your canonical account, you should be able to
<dimitern> rogpeppe: yeah, I'm already half way through
<rogpeppe> dimitern: yeah, i saw that - very helpful, thanks!
<rogpeppe> rvba: if you have a moment at some point, could you perhaps fill out some of the azure-specific entries in this spreadsheet? https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AghcHASXWzXmdHdOX2htQVJITkdaYVh4VnZ2Z0RrQWc#gid=0
<rogpeppe> rvba: i'm trying to start rationalising our config attribute twistiness
<rvba> rogpeppe: sure, I'll have a look.
<rogpeppe> does anyone know how to use the new charm browser to list all available charms?
<rogpeppe> the old one was easy enough to screen scrape, but this one isn't so easy.
<dimitern> rogpeppe: not sure about the last 4 columns though
<rogpeppe> dimitern: "needed for sync tools" is whether an attribute is needed when pushing to the private storage
<rogpeppe> dimitern: "needed for bootstrap" is whether the attribute needs to be passed into the bootstrap node
<rogpeppe> dimitern: "needed for client operations after bootstrap" is whether you need to use the attribute when connecting to the state later.
<dimitern> rogpeppe: I have to think about all of these - all in all they seem to be the same as for ec2, but have to check
<rogpeppe> dimitern: i would expect them to be similar
<rvba> rogpeppe: what does "OK to specify in environments.yaml?" mean exactly?
<rogpeppe> rvba: do you want a user to be able to specify that attribute in the environments.yaml file?
<rogpeppe> rvba: in your case, the answer is likely to be yes for all
<rvba> Okay.
<rogpeppe> rvba: we have some weird attributes like agent-version, which don't really make sense for a user to be able to specify there
<rogpeppe> rvba: that's part of the twistiness
<rvba> rogpeppe: I see.
<rvba> rogpeppe: I'm a bit confused by "Cloud account information?".  I interpret this as "The value part of one's cloud account information" but I see ec2's 'region' is checked (with a question mark though).
<rogpeppe> rvba: it's whether the attribute should be considered part of the cloud account information (we're planning to move towards specifying cloud accounts as a separate entry in the juju conf file)
<rogpeppe> rvba: just leave it blank if you're not sure
<rvba> rogpeppe: 'region' does not seem to be part of the cloud account information thenâ¦ it's more linked to the deployment isn't it?
<rogpeppe> rvba: yes, you're probably right.
<rogpeppe> rvba: i'm not quite sure of fwereade's conception of accounts though
<rogpeppe> fwereade: you should be interested in this too BTW: https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AghcHASXWzXmdHdOX2htQVJITkdaYVh4VnZ2Z0RrQWc#gid=0
<rvba> rogpeppe: okay, ta.  I'll put X? then.
<rogpeppe> rvba: sgtm
<fwereade> rogpeppe, rvba: really briefly, because I'm out today: an account is essentially <reference to cloud info> + <auth info> + <possible overrides to cloud info>
<fwereade> rogpeppe, oh, nice spreadsheet
<rogpeppe> fwereade: so would the ec2 region be part of an account?
<fwereade> rogpeppe, might be, if someone almost always wants to use (say) eu-west-1
<fwereade> rogpeppe, that's one of those ugly things that really needs to be available for override at env and account level
<rogpeppe> fwereade: is the idea to be able to specify *any* attribute at account level?
<fwereade> rogpeppe, I don't think we can have a coherent model without allowing that, upsetting though it may be
<fwereade> rogpeppe, but I don't think it's too painful to just pass unknown keys up levels until they're found to be comprehensible
<rogpeppe> fwereade: hmm, so it's not really "account" - it's more like "environment superclass"
<fwereade> rogpeppe, I don't personally see that as a particularly useful conception of it, but I expect mileage legitimately varies
<rogpeppe> fwereade: or, i guess "environment template"
<fwereade> rogpeppe, no, I don't think so...
<rogpeppe> fwereade: i don't really see anything particularly account-like about it, that's all
<fwereade> rogpeppe, the env may contribute attrs to the account, not vice versa
<fwereade> rogpeppe, the auth info is the critical bit
<fwereade> rogpeppe, that's the stuff that has to be included at that level
<rogpeppe> fwereade: erm, surely we're saying that the account can contribute attrs to the env too, no?
<fwereade> rogpeppe, how is region anything to do with the environment?
<fwereade> rogpeppe, it's all about the cloud
<fwereade> rogpeppe, but it can be specified differently at more specific levels
<rogpeppe> fwereade: ah
<rogpeppe> fwereade: so we've really got three levels, "account", "cloud" and "environment"
<fwereade> rogpeppe, it's definitely a surprising model from our existing perspective but it really holds together pretty well when you think about it enough, I think
<rogpeppe> fwereade: but... are you conflating "account" and "cloud"?
<rogpeppe> fwereade: i find it surprising that i can specify a region in something called an "account", which sounds like it's just got user credentials in it
<rogpeppe> fwereade: it feels a bit like the responsibilities are mixed up there
<fwereade> rogpeppe, the names indicate what must be specified, the other bits that *can* be specified are about convenience for users (and, yeah, some degree of irritating conceptual bleed for us developers)
<rogpeppe> fwereade: i'm no longer sure how many levels of inheritance we're aiming for here
<rogpeppe> fwereade: i'm not sure it's just a conceptual difficulty for developers - i'd find it awkward to explain to a user too.
<fwereade> rogpeppe, it's not the environment inheriting from the account inheriting from the cloud -- it's the environment overwriting values on the account overwriting values on the cloud
<rogpeppe> fwereade: i'm not sure i see the difference there
<fwereade> rogpeppe, inheritance implies an env is an account is a cloud
<fwereade> rogpeppe, it's rather more like embedding, I think
<rogpeppe> fwereade: if i can specify all the env attrs on an account or a cloud, i'm not sure i see there is a difference
 * fwereade is neglecting his family
<rogpeppe> fwereade: right, go away! :-)
<rogpeppe> fwereade: sorry for distracting you
<fwereade> rogpeppe, (when did I say you could specify an env attr on an account? I don't *think* it needs to flow that way too)
<rogpeppe> fwereade: hmm, i'd thought the env ended up with all the attributes from the account
<fwereade> rogpeppe, I think that the env ends up with an account (which may have had overrides applied to it), and the *env* only has a few juju-level things set -- one of which is the account
<rogpeppe> fwereade: but are we allowing users to specify a "cloud" thing as well as an environment and an account?
<fwereade> rogpeppe, because of private clouds essentially
<fwereade> rogpeppe, most of the time all that tedious cloud stuff comes from simplestreams, so account specification is trivial
<rogpeppe> fwereade: an account specifies a cloud too?
<fwereade> rogpeppe, sometimes it needs to be hand-specified, and we (1) don't want to dirty up the account specification and (2) may well have a couple of accounts on the same cloud for say testing/staging/production
<fwereade> rogpeppe, yes, account = <cloud reference> + <auth info> + <possible cloud overrides for convenience>
<fwereade> rogpeppe, env  = <account reference> + <juju info> + <possible account overrides for convenience>
 * fwereade is really off now, and hopes the model makes some sort of sense
 * rogpeppe thinks about it
<rogpeppe> fwereade: thanks
<rogpeppe> hmm, that's a first for me - laptop just spontaneously rebooted. i wasn't doing anything significant at the time
 * rogpeppe tries to recollect all the threads
<dimitern> rogpeppe: it happens to me sometimes since I upgraded to raring - all due to video drivers
<rogpeppe> dimitern: yeah, i reckon that's likely to be the problem
<dimitern> rogpeppe: btw, should we have an EnvironConfig() call in the uniter API? the uniter uses it, but I remember there were some concerns about it
<rogpeppe> dimitern: i also get a weird thing occasionally where everything goes ultra slowly
<rogpeppe> dimitern: what is the uniter using it for?
 * dimitern checks
<dimitern> rogpeppe: to get the provider type and from then the private address
<dimitern> rogpeppe: and public
<rogpeppe> dimitern: i think you should just have a ProviderType API call
<dimitern> rogpeppe: ok
<natefinch> morning all
<rogpeppe> dimitern: here's the list of all the relation names used in the 193 charms i could find in the charm store, most frequent first: http://paste.ubuntu.com/5992327/
<rogpeppe> natefinch: hiya
<rogpeppe> dimitern: looks like we can be quite restrictive without it being a problem
<dimitern> rogpeppe: so it looks like ([a-z-]+)+ should be enough?
<dimitern> rogpeppe: or more like ([a-z]+-?)+, except it shouldn't end with -
<natefinch> mgz: I was looking at maas and azure for addressability. I have some code for azure, but it's pretty ugly... I couldn't find where maas was exposing IPs, though
<rogpeppe> dimitern: i think i'd go for [a-z][a-z0-9]*(-[a-z0-9]+)*
<dimitern> rogpeppe: sgtm
<rogpeppe> dimitern: in case you're interested, i downloaded all the charms with this: http://paste.ubuntu.com/5992345/ (unfortunately obtaining the list of charms was a bit more hassle)
<rogpeppe> dimitern: and i got it to spew out the relation names with this: http://paste.ubuntu.com/5992349/
<rogpeppe> dimitern: (i ran it with this command line: http://paste.ubuntu.com/5992351/)
<rogpeppe> dimitern: so that command is also potentially useful for other charm metadata analysis we might want to do
<dimitern> rogpeppe: cool! so you didn't miss anything
<mgz> natefinch: for maas, you might want to ask one of the red squad guys
<rogpeppe> dimitern: well, i might well have missed some charms
<mgz> we need a different mechanism for the local provider currently, we might need one for maas too, but I think when I looked at it there seemed to be a reasonable way
<mgz> standup guys?
<geme_> juju-core 1.13.1 won't bootstrap on private openstack. /var/log/cloud-init-output.log reports : ERROR juju open.go:89 state: connection failed, will retry: dial tcp 127.0.0.1:37017: connection refused. There's nothing running on this port.mongod is running on 27017 and 28017 though
<mgz> geme_: you need to ssh into the state server machine and look at the juju logs
<mgz> I had a similar issue a while back, but I'm failing to find or recall what the root cause was
<geme_> which log ?
<mgz> look at things under /var/log/juju
<mgz> there's also a mongo log that I remember looking at...
<mgz> gah, I remember talking to jam about it, but can't find it in the chat logs anywhere
<mgz> geme_: any luck? the first few lines of /var/log/juju/machine-0.log should be about mongo state connection
<geme_> mgz, there is no machine-0.log - just an empty all-machines.log
<mgz> okay, so can you pastebin your whole cloud-init please? seems the agent didn't get started at all
<geme_> cloud-init.log and cloud-init-output.log ?
<mgz> -output please
<geme_> mgz, jusr seen : 2013-08-16 10:36:05,970 - cc_apt_update_upgrade.py[WARNING]: Source Error: ppa:juju/experimental:add-apt-repository failed
<mgz> what do you get if you sudo run thant command from the box now?
<geme_> mgz, http://paste.ubuntu.com/5992536/
<mgz> oh god, that statecert logging
<mgz> geme_: okay, so it installs the wrong version of mongodb (2.0.4-1ubuntu2.1) because the repo add failed
<geme_> adding the repo works from the cmdline
<geme_> though it needs a return response
<mgz> meh, and it's a bare except clause in cloudinit/config/cc_apt_update_upgrade.py that throws away the actual error when logging
<geme_> mgz, any idea why adding the repo failed ?
<mgz> my best guess is somehow the networking isn't up yet so it fails trying to talk to launchpad
<mgz> but could be anything, and after all the install works fine after that
<mgz> one thing you could try is reproducing the error with `nova boot --user-data CLOUDCONFIGFILE`
<geme_> mgz, when cloud-init runs can it access envars like HTTP_PROXY ?
<mgz> where cloud config is just something like a file containing "#cloud-config\napt-sources:\n - source: ppa:juju/experimental" (with real newlines etc)
<mgz> geme_: set where? if you have some funky proxy setup for external net access, that is likely the issue
<geme_> mgz, /etc/bash.bashrc
<mgz> doubt very much cloud-init will pick up that, except perhaps as a side-effect to some subcommands
<geme_> mgz, Is there another way to make these proxy setting available to cloud-init ?
<mgz> I'm not even sure that's sufficient, not everything respects HTTP_PROXY
<geme_> mgz, so firewalls are a blocker then - sorry for the pun :)
<mgz> not really, we've got a bunch of firewalled clouds that work fine
<mgz> I don't think any try to do it using HTTP_PROXY though
<mgz> which is an application level setting, if you're trying to inject that into your vms it doesn't seem a reliable way of doing things
<mgz> you need to configure openstack to understand your network setup
<geme_> mgr, the proxy settings work in the shell
<mgz> sure, because it's a shell
<mgz> lots of things in a vm will not be spun up from bash
<mgz> I think what you probably want is just a local apt proxy, but I'm still not sure how apt-add-reposistory interacts with that
<mgz> you might want to try asking in #juju how people have set up juju on openstack clouds without outgoing access to the wider internet
<mgz> because a bunch of guys in there have done that
<geme_> mgr, ok I'll ask. Thanks
<rogpeppe> dimitern: i realised that i'd missed most charms in my download, but rick_h in juju-gui pointed me to the charmworld api, so here's a program that downloads *all* the charms: http://paste.ubuntu.com/5992628/
<rogpeppe> dimitern: 663 successfully acquired at last count
<dimitern> rogpeppe: nice
<rogpeppe> dimitern: and our relation regexp is still fine
<dimitern> rogpeppe: great!
<dimitern> rogpeppe: we should have some repo for such useful tools
<rogpeppe> dimitern: yeah, they might well be useful to others
<dimitern> rogpeppe: so if you have a peer relation, are the remote and local units of it the same?
<rogpeppe> dimitern: no
<dimitern> rogpeppe: why did you resolve all my comments on that document?
<rogpeppe> dimitern: i turned then all into notes
<rogpeppe> dimitern: which seemed a much more appropriate way of doing things
<rogpeppe> dimitern: (i didn't know about notes when i started adding comments)
<dimitern> rogpeppe: ha! I didn't know there is such a feature
<rogpeppe> dimitern: thank rvba
<dimitern> rvba: thanks :)
<dimitern> rogpeppe: so I'm pondering how to get the remote unit from a relationunit
<dimitern> rogpeppe: I can get the relation of the RU, call RelatedEndpoints on it, passing the local unit's service name
<dimitern> rogpeppe: I'll a slice with length 1 (peer) or 2 (provider-requirer) with the endpoints
<dimitern> rogpeppe: and I can get the remote service name from eps[0].ServiceName or eps[1].ServiceName respectively
<dimitern> rogpeppe: but how to get which unit is on the other end?
<dimitern> rogpeppe: /I'll a/I'll get a/
<rogpeppe> dimitern: an endpoint doesn't imply a unit
<dimitern> rogpeppe: so there's no way to know which is the remote unit
<rogpeppe> dimitern: the client will need to name the unit, i think
<dimitern> rogpeppe: ok, so ReadSettings cannot be made to enforce "use remote unit only"
<dimitern> rogpeppe: in fact I can see in test that it gets called for the local unit as well
<rogpeppe> dimitern: well, there are many possible remote units
<dimitern> rogpeppe: right..
<rogpeppe> dimitern: and we need to be able to read our own settings too
<dimitern> rogpeppe: for that, you have another call
<dimitern> rogpeppe: Settings()
<rogpeppe> dimitern: what's the difference between the two?
<dimitern> rogpeppe: Settings() returns a mutable *Settings, whereas ReadSettings() returns a plain map[string]interface{}
<rogpeppe> dimitern: but they'll use the same underlying API call, right?
<dimitern> rogpeppe: no
<dimitern> rogpeppe: but perhaps they should actually
<rogpeppe> dimitern: ah, what's the difference?
<dimitern> rogpeppe: ReadSettings only
<rogpeppe> dimitern: i think they're both doing the same thing
<rogpeppe> dimitern: the difference is only at the client side
<dimitern> rogpeppe: and it to simulate Settings() you need to call it with "relation-x", "unit-x", "unit-x"
<dimitern> rogpeppe: I have defined a RelationUnitPair with Relation, LocalUnit and RemoteUnit string
<dimitern> rogpeppe: I can make RemoteUnit to be optional
<rogpeppe> dimitern: hmm, can't you just omit LocalUnit?
<rogpeppe> dimitern: because it's implied by the client
<dimitern> rogpeppe: no, let's not do that
<dimitern> rogpeppe: if we're going to start omitting stuff because it's implied by the authenticated entity, that's a whole new can of worms
<rogpeppe> dimitern: ok, fair enough
<dimitern> rogpeppe: possibly making bulk calls a moot in general
<rogpeppe> dimitern: ahem, yes
<rogpeppe> dimitern: [don't get me started :-)]
<dimitern> rogpeppe: so, until and if we decide to do that, I'll prefer to be explicit
<rogpeppe> dimitern: so LocalUnit must always be the authenticated unit, right?
<dimitern> rogpeppe: yes
<rogpeppe> dimitern: so i think there's no need to make RemoteUnit optional
<rogpeppe> dimitern: if you want to get settings of the remote unit, just have RemoteUnit==LocalUnit
<dimitern> rogpeppe: so pass both and make them the same to request local settings
<dimitern> rogpeppe: ok
<dimitern> rogpeppe: no, it's exactly the opposite :)
<rogpeppe> dimitern: then RemoteUnit is always the tag of the unit you're asking for settings of
<dimitern> rogpeppe: Local!=Remote => remote settings
<rogpeppe> dimitern: ha, yes, typo
<rogpeppe> dimitern: if you want to get settings of the *local* unit... !
<dimitern> rogpeppe: ok then, almost done - will propose soon
<rogpeppe> dimitern: cool
<dimitern> rogpeppe: remind me again why you removed the unitTagToName conversion from names?
<rogpeppe> dimitern: you've got ParseTag now
<dimitern> rogpeppe: ah, right
<rogpeppe> dimitern: when you've a moment, try running "go get launchpad.net/juju-utils/cmd/getallcharms"
<rogpeppe> dimitern: there's also launchpad.net/juju-utils/cmd/charmmeta
<geme_> mgz, added repo to the image. Now bootstraps successfully and juju status respond. Likely the problem will come back and bite me.
<dimitern> rogpeppe: done both
<rogpeppe> dimitern: you could try and see if it works
<rogpeppe> dimitern: e.g. getallcharms ~/allcharms
<dimitern> rogpeppe: trying now
<rogpeppe> dimitern: it should fetch all the charms in the charm store
<dimitern> rogpeppe: is there a -v option?
<dimitern> rogpeppe: ah, it dumps them
<rogpeppe> dimitern: yeah. it's actually verbose by default
<dimitern> rogpeppe: for some it reports 2013/08/16 15:40:58 get "cs:~philipballew/precise/dyndns-0" failed: charm "noip2" relation "dynamic-ip-reporting" using a reserved interface: "juju-info"
<rogpeppe> dimitern: yup, that's the most common issue
<dimitern> rogpeppe: done, 361M fetched
<rogpeppe> dimitern: cool
<dimitern> rogpeppe: and charmmeta ?
<dimitern> rogpeppe: it's not recursive
<rogpeppe> dimitern: now you can run charmmeta over all of them by doing something like: charmmeta sometemplatetext $(cat $HOME/allcharms/urls.txt)
<rogpeppe> dimitern: no - it probably should recurse looking for metadata.yaml files
<rogpeppe> dimitern: but i've spent too much time on it already :-)
<dimitern> rogpeppe: np, with some tweaking it works
<rogpeppe> dimitern: what tweaking?
<dimitern> rogpeppe: charmmeta '..template..' `cat ~/allcharms/urls.txt` fails
<dimitern> rogpeppe: it reports Dir==nil on each line
<rogpeppe> dimitern: ah!
<rogpeppe> dimitern: that's because you need to be inside the allcharms directory
<dimitern> rogpeppe: yeah, I figured :)
<dimitern> man.. testing Enter and LeaveScope is a bitch
<dimitern> I have to use a scope watcher..
<rogpeppe> dimitern: charmmeta is now recursive. couldn't quite resist doing it now :-)
<dimitern> rogpeppe: cool, will test it
 * rogpeppe must get some lunch now
<rogpeppe> trivial review, anyone? https://codereview.appspot.com/13053043
<rogpeppe> just some package renaming, mechanical changes
<natefinch> rogpeppe: I can review it
<rogpeppe> natefinch: thanks
<natefinch> rogpeppe: reviewed. LGTM, bonus points for fixing a few of the import sorts
<rogpeppe> natefinch: good catch
<rogpeppe> dimitern, mgz, rvba, jtv, fwereade: another review appreciated please. trying to avoid a prereq. https://codereview.appspot.com/13053043
<dimitern> rogpeppe: looking
<rogpeppe> dimitern: ta
<rogpeppe> dimitern: i okayed this with fwereade previously, BTW
<dimitern> rogpeppe: while I'm at it, would you look at https://codereview.appspot.com/13055043
<rogpeppe> dimitern: looking
<dimitern> rogpeppe: thanks
<dimitern> rogpeppe: didn't we discuss having the providers at top-level?
<rogpeppe> dimitern: they're very much to do with environs
<rogpeppe> dimitern: (they implement an interface defined in environs)
<rogpeppe> dimitern: so i strongly feel they below below environs
<rogpeppe> dimitern: (as does instance, but that's another story)
<dimitern> rogpeppe: that shouldn't stops us - there are cases where other packages implement interfaces defined outside their parent package
<rogpeppe> dimitern: yes, but really environs is all about providers
<rogpeppe> dimitern: hmm, well, i can see your p.o.v. too
<dimitern> rogpeppe: providers, like charms are things that need an environment
<rogpeppe> dimitern: not really: providers *provide* an environment
<dimitern> rogpeppe: but really the environment needs both of them
<rogpeppe> dimitern: well, they provide an Environ anyway
<dimitern> rogpeppe: the environment is much more that what a provider provides
<dimitern> rogpeppe: it provides means to access the environment, but not the only means
<rogpeppe> dimitern: a provider provides an Environ
<dimitern> rogpeppe: state and API are other means to access it
<rogpeppe> dimitern: and that's what environs is all about
<dimitern> rogpeppe: well, Environ is hardly the best name for it
<rogpeppe> dimitern: state and API don't really have anything to do with Environ
<rogpeppe> dimitern: that may be true
<dimitern> rogpeppe: i think we're conflating the Environ type and what a juju environment is all about
<dimitern> rogpeppe: the Environ type is *not* a juju environment
<rogpeppe> dimitern: i didn't say it was
<rogpeppe> dimitern: well,
<rogpeppe> dimitern: it depends what you mean by a juju environment
<dimitern> rogpeppe: I mean both local and remote stuff
<rogpeppe> dimitern: but regardless, the provider types are there entirely to implement the interface defined in environs/interface.go
<dimitern> rogpeppe: settings, instances, cloud account info
<rogpeppe> dimitern: they aren't accessed in any way accept through environs
<rogpeppe> dimitern: so i think it's reasonable to put them under that directory
<dimitern> rogpeppe: they are - through the CLI and the API
<rogpeppe> dimitern: not really. whenever the CLI gets an Environ, it gets it via the environs package
<dimitern> rogpeppe: should we put the API in environs as well, because all it does is to provide access to a running environment's parts?
<rogpeppe> dimitern: no, an Environ (whatever it should really be called) is all about encapsulating provider-specific functionality
<dimitern> rogpeppe: I really liked the way pyjuju has providers at top level
<rogpeppe> s/whatever/or whatever/
<dimitern> rogpeppe: and the argument that providers need to implement an interface in environs/interface.go is not a strong point to keep them together IMO
<dimitern> rogpeppe: we have plenty of cases where that's not true
<rogpeppe> dimitern: if they provided independent functionality too, i might agree
<rogpeppe> dimitern: but given that nothing else in the system imports them directly (except for the side effect of having them there, and excepting dummy which is special), i think it's reasonable to hide them under environs
<dimitern> rogpeppe: how about the null provider?
<dimitern> rogpeppe: it's not a complete provider - also possibly the ssh one
<rogpeppe> dimitern: i don't know what the null provider is. have we got one of those?
<dimitern> rogpeppe: we will soon
<dimitern> rogpeppe: read the specs
<dimitern> rogpeppe: it's an example of a "partial" provider
<dimitern> rogpeppe: you cannot start or stop instances for example
<dimitern> rogpeppe: or xenv rels?
<rogpeppe> dimitern: it will still need to implement all the methods defined in environs.EnvironProvider
<rogpeppe> dimitern: even if it returns errors for some of them
<dimitern> rogpeppe: I'm saying this might change
<rogpeppe> dimitern: i'm not sure it needs to
<dimitern> rogpeppe: we might need to separate that huge interface into smaller ones
<rogpeppe> dimitern: and we can change things again when that happens
<rogpeppe> dimitern: i plan to do just that
<dimitern> rogpeppe: well..
<rogpeppe> dimitern: but i'm still happy to have them all under environs
<dimitern> rogpeppe: ok, at least havingthem in environs/provider/ will clean up some of the mess already in there
<dimitern> rogpeppe: the same should apply to cloudinit imo
<rogpeppe> dimitern: where should environs/cloudinit go?
<dimitern> rogpeppe: in cloudinit/
<rogpeppe> dimitern: cloudinit/cloudinit ?
<rogpeppe> dimitern: (there's already a juju-core/cloudinit package)
<dimitern> rogpeppe: I know
<dimitern> rogpeppe: why shouldn't it be in the same top-level package?
<dimitern> rogpeppe: why the separation?
<rogpeppe> dimitern: because the top level package is entirely juju-agnostic
<rogpeppe> dimitern: whereas environs/cloudinit is intimately bound up with juju environment-related stuff
<rogpeppe> dimitern: and that's kinda where my reasoning for when we should have a top level package or namespace comes from - it should be largely independent functionality.
<dimitern> rogpeppe: anyway, that's entirely another topic in its own right
<rogpeppe> dimitern: the cloudinit package works well as something standalone. environs/cloudinit builds neatly on top of it.
<dimitern> rogpeppe: if it's independent it shouldn't be in juju-core - it should be an outside package
<rogpeppe> dimitern: perhaps. but if we don't care about other people using them, it's more convenient to keep it in juju-core
<rogpeppe> dimitern: we can still have decent modularity, even inside juju-core
<dimitern> rogpeppe: if it's useful independently, it shouldn't be inside
<rogpeppe> dimitern: there are lots of pieces that are useful independently. having them outside is a burden we don't want to have to bear for all of them
<rogpeppe> dimitern: schema and rpc are two examples
<rogpeppe> dimitern: anyway, anyone outside can still use these things
<rogpeppe> dimitern: they don't need to be in an independent package to enable external reuse
<dimitern> rogpeppe: I don't quite agree the burden is that big
<dimitern> rogpeppe: most of this things are stable and change rarely
<rogpeppe> dimitern: i don't think there's that much in the way of advantage from having them in separate projects
<dimitern> rogpeppe: actually that could be a reason to move it out
<dimitern> rogpeppe: of course there is - smaller code base, easier to maintain
<rogpeppe> dimitern: i don't think the maintenance burden is any smaller if the code is separated across several projects
<rogpeppe> dimitern: the opposite if anything
<dimitern> rogpeppe: there's only a burden if we have to update them often - the same thing as with gwacl, goose, etc.
<rogpeppe> dimitern: if we don't need to update them, where's the burden from having them in the same code base?
<rogpeppe> dimitern: i think it's great to build a project from many small indpendent parts
<rogpeppe> dimitern: and if those parts were principally built for the main project, i don't really see there's an advantage to moving them out
<rogpeppe> dimitern: the main advantage of doing that is if we want to promote external users of those packages
<dimitern> rogpeppe: it's not a burden as such to have them in the same codebase, it just inflates it, and if they rarely change, it's actually better and cleaner to have them outside
<rogpeppe> dimitern: i guess i don't see that. the more external dependencies, the more hassle.
<rogpeppe> dimitern: otherwise we'd have one project for each package
<dimitern> rogpeppe: what's the hassle exactly?
<rogpeppe> dimitern: moving the code base to different versions doesn't automatically update the dependencies
<rogpeppe> dimitern: i can see the history of those dependencies in with the main log
<rogpeppe> dimitern: if they're inside the code base
<dimitern> rogpeppe: aren't we moving towards handling dependencies better in general?
<rogpeppe> dimitern: yeah, but it's still not cost-free
<dimitern> rogpeppe: it depends on how easy we make it
<dimitern> rogpeppe: nothing is cost free, there are hidden costs you have to consider
<rogpeppe> dimitern: back to the environs/provider thing: for me, it's about strict encapsulation boundaries. the environs package should entirely encapsulate the different providers.
<rogpeppe> dimitern: hidden costs like?
<dimitern> rogpeppe: reviewed
<rogpeppe> dimitern: i don't want to do any import renaming in that branch if that's ok. i can go through in another branch doing all the gocheck renames if you'd like
<dimitern> rogpeppe: like having a bunch of code which is not directly juju-related, only slightly and could be reused by external projects
<rogpeppe> dimitern: external projects can still reuse that code
<dimitern> rogpeppe: Ok, at least fix the imports separation then - they were just a few
<rogpeppe> dimitern: (and does - for instance that charmmeta command)
<rogpeppe> dimitern: will do, thanks
<dimitern> rogpeppe: i don't want to argue anymore about this - it deserves a separate discussion in good time :)
<rogpeppe> dimitern: indeed :-)
<natefinch> dimitern, rogpeppe:  FWIW I agree with both of you... both ways have advantages and disadvantages. It's a tale as old as time (or at least as old as modern programming)
<dimitern> natefinch: nah.. c'mon you can't agree like that :)
<natefinch> dimitern: what, I have to take a side?  :)
<dimitern> natefinch: hehe
<dimitern> natefinch: you can have a look at this perhaps? https://codereview.appspot.com/13055043/
<natefinch> dimitern: sure
<dimitern> natefinch: ta
<natefinch> dimitern: just curious, why do we do "type RelationUnits struct { RelationUnits []RelationUnit}"  rather than "type RelationUnits []RelationUnit" ?
<dimitern> natefinch: because the rpc layer of the API needs a struct as args and results
<natefinch> dimitern: ahh, ok
<dimitern> natefinch: take a look at doc/api.txt and rpc/server.go -> Serve()'s doc comment you're interested in the internals
<dimitern> fwereade_: hey
<dimitern> fwereade_: just in time for a review? :)
<dimitern> natefinch: thanks
<dimitern> natefinch: btw - I'd like to leave the checkers.IsTrue etc. for a follow-up, if you don't mind. The diff is already big enough
<dimitern> rogpeppe: does it look roughly ok?
<rogpeppe> dimitern: i'm working on doing all the gc.etc stuff in about 4 giant branches
<rogpeppe> dimitern: i think i can do most of the changes automatically
<dimitern> rogpeppe: good to hear
<dimitern> rogpeppe: but please don't forget my branch as well ;)
<rogpeppe> dimitern: i'm looking now
<dimitern> rogpeppe: cheers
<natefinch> rogpeppe: I was looking at how to do the . to gc change automatically... did you find a good way to do it?
<rogpeppe> natefinch: i've got a command that may help - we'll see if it's up to the task
<rogpeppe> natefinch: it's pretty much a failed experiment, but still sometimes useful
<natefinch> rogpeppe: yeah, I looked around for something that might already do it, gofix or something, but nothing seemed to be able to do it, so I was looking into writting a command to do it by parsing the AST... hopefully yours will work so I don't have to do that :)
<rogpeppe> natefinch:code.google.com/p/rog-go/exp/cmd/gosym
<rogpeppe> natefinch: gofmt isn't far off being able to do it
<natefinch> rogpeppe: yeah, I looked at gofmt too... wasn't sure how to get it to figure out what is getting imported by . and fix it automatically.... at least in the general case
<rogpeppe> natefinch: it can't do that
<rogpeppe> natefinch: it needs to resolve type symbols
<rogpeppe> natefinch: gosym resolves symbols but it doesn't cope with import to . well
<natefinch> rogpeppe: I'm honestly surprised import . even exists in Go.  Seems like the kind of "almost always a bad idea" thing that it would not support
<rogpeppe> natefinch: it's only there for a particular edge case
<rogpeppe> natefinch: but it's been abused
<rogpeppe> dimitern: can settings attributes be deleted?
<dimitern> rogpeppe: they are replaced
<dimitern> rogpeppe: so yeah
<rogpeppe> dimitern: hmm, it looks like relation-set calls settings.Delete
<dimitern> rogpeppe: perhaps an additional test is needed
<rogpeppe> dimitern: but it looks as your API doesn't cater for deletions
<dimitern> rogpeppe: settings.Delete is only local
<rogpeppe> dimitern: it's possible that's not a problem though
<dimitern> rogpeppe: any changes are always written with Settings.Write()
<rogpeppe> dimitern: but if you've called Delete, Settings.Write will actually delete the attribute
<rogpeppe> dimitern: it doesn't just overwrite it with nil
<dimitern> rogpeppe: the settings are one blob, right?
<rogpeppe> dimitern: no
<rogpeppe> dimitern: the settings are held as mongo fields
<rogpeppe> dimitern: each settings gets its own field
<rogpeppe> s/settings/setting/
<rogpeppe> dimitern: look in state/settings.go
<dimitern> rogpeppe: so what happens when you overwrite it with some different map?
<rogpeppe> dimitern: it changes only the ones that have locally changed since you last read the settings
<rogpeppe> dimitern: it unsets the ones that have been deleted and sets the others
<rogpeppe> dimitern: it might be reasonable to say that setting a value to nil will be treated as a deletion
<dimitern> rogpeppe: ok, will add more tests to make sure what I can read from the unit after calling WriteSettings is exactly as passed in the args
<rogpeppe> dimitern: how will attributes get deleted?
<dimitern> rogpeppe: by calling Update before Write?
<rogpeppe> dimitern: Update never deletes an attribute
<dimitern> rogpeppe: so you're saying the operation has to be more sophisticated
<rogpeppe> dimitern: i think so... assuming we care about deleting attributes
<dimitern> rogpeppe: reading old settings, finding what to delete, updating the rest, finally writing the result
<rogpeppe> dimitern: no
<dimitern> rogpeppe: no?
<rogpeppe> dimitern: the API call has to be able to specify which settings are to be deleted
<dimitern> rogpeppe: delete just does delete(c.core, key)
<dimitern> rogpeppe: it doesn't seem it keeps the deleted ones separately
<rogpeppe> dimitern: it infers which ones are deleted by looking at the difference between c.core and c.disk
<dimitern> rogpeppe: assuming we start with an empty map, update with the new settings, then write should take care of deletions as well
<rogpeppe> dimitern: what happens if we do: Update({"x": "y"}) then Update({}) ?
<dimitern> rogpeppe: will delete everything
<rogpeppe> dimitern: nope
<rogpeppe> dimitern: "x" will still be around
<dimitern> rogpeppe: no, I mean we can't do that really
<dimitern> rogpeppe: Update is client-side only
<dimitern> rogpeppe: on the API level we have read and write only
<rogpeppe> dimitern: it would be possible to make Update overwrite all attributes
<rogpeppe> dimitern: but i don't think that's a great idea
<dimitern> rogpeppe: how?
<rogpeppe> dimitern: because it means potentially quite a bit more network traffic
<dimitern> rogpeppe: sorry, I don't get you
<dimitern> rogpeppe: we have old settings, we want to write new ones we got over the wire
<rogpeppe> dimitern: if a hook just does a single "relation-set x=y", we don't really want to send all the other settings too
<dimitern> rogpeppe: calling Read() will reset the disk cache, then we reset all core values, update the new ones, then write
<rogpeppe> dimitern: are you talking client- or server-side here?
<dimitern> rogpeppe: server-side
<dimitern> rogpeppe: client-side we implement a Settings proxy object with a subset of the interface that state.Settings has, but only Write calls the API
<dimitern> rogpeppe: the overhead is not significant
<rogpeppe> dimitern: if we replace all core values, the client must send all those values up with the Write call
<dimitern> rogpeppe: yes it will
<dimitern> rogpeppe: the client calls Write eventually
<dimitern> rogpeppe: in context, if I'm not mistaken, after the hook is comitted
<dimitern> committed
<dimitern> rogpeppe: yep, context.go:290
<rogpeppe> dimitern: are you happy that at the end of the hook, all settings will be sent across the network, even if they have not changed?
<dimitern> rogpeppe: I see no problem with that
<dimitern> rogpeppe: the settings are usually just a few
<rogpeppe> dimitern: i don't know - have you looked at how charms use settings?
<dimitern> rogpeppe: a few yes
<rogpeppe> dimitern: i think it's quite possible that people have some large values in there sometimes
<dimitern> rogpeppe: haven't look through all of them
<rogpeppe> dimitern: we're not talking about config settings
<dimitern> rogpeppe: can we check that reliably?
<dimitern> rogpeppe: find all relation-set calls in all charms?
<rogpeppe> dimitern: no - i just think it's not hard to be a little more efficient
<dimitern> rogpeppe: like how?
<dimitern> rogpeppe: have UpdateSettings and DeleteSettings calls?
<dimitern> rogpeppe: pass every client-side call over the wire?
<rogpeppe> dimitern: we could treat a nil value as a delete
<rogpeppe> dimitern: so to delete an attribute you'd do: WriteSettings({"x": nil}) for example
<dimitern> rogpeppe: deletes are not a problem
<dimitern> rogpeppe: you're talking about something different
<dimitern> rogpeppe: having incremental updates
<rogpeppe> dimitern: afaics, deletes are the only problem with the current scheme
<rogpeppe> dimitern: the current scheme will work fine to do incremental updates
<rogpeppe> dimitern: the only exception being that you won't be able to delete an attribute
<dimitern> rogpeppe: the current scheme will change as described and deletes won't be a problem anymore - it'll overwrite all stuff on write
<dimitern> rogpeppe: not really
<dimitern> rogpeppe: we're removed from the source when using the API
<dimitern> rogpeppe: we can't rely on Update to do things smartly
<dimitern> rogpeppe: we read the settings every time when we're about to write them
<dimitern> rogpeppe: so there's no cache, etc.
<rogpeppe> dimitern: i thought the idea was to replicate some similar Settings logic client side
<dimitern> rogpeppe: yeah, except for Write
<dimitern> rogpeppe: but Write will still overwrite what's currently in state
<rogpeppe> dimitern: so that the client would read settings, make some modifications, then call Write, which could intelligently decide which have changed and issue an appropriate Update API call
<dimitern> rogpeppe: I'm not keen on the idea of having an Update API call
<rogpeppe> dimitern: why not?
<dimitern> rogpeppe: it seems too much complexity for very marginal gain
<rogpeppe> dimitern: in that case, there's no point in duplicating any of the Settings logic in the client side.
<rogpeppe> dimitern: the uniter may as well just use map[string]interface{} directly
<dimitern> rogpeppe: it being "in the odd case some charm writes huge settings keys once and changes a few smaller ones often"
<dimitern> rogpeppe: well, it doesn't and changing the uniter is not necessary
<dimitern> rogpeppe: the client-side will replicate the same logic with a proxy Settings
<dimitern> rogpeppe: but on Write it'll overwrite current settings
<dimitern> rogpeppe: and it will still work the same
<dimitern> rogpeppe: update and delete will operate on the proxy object's internal map
<dimitern> rogpeppe: write will send the final, merged map for saving
<rogpeppe> dimitern: you'd only need one map, of course.
<dimitern> rogpeppe: yeah
<dimitern> rogpeppe: I can add a TODO in WriteSettings saying something like "if this proves to be inefficient we can change it to allow incremental updates" ?
<rogpeppe> dimitern: how are you going to reset all attributes, BTW?
<dimitern> rogpeppe: in fact, having huge values in settings keys is a possible DoS attack on mongo from a malicious charm
<dimitern> rogpeppe: get the old ones, delete everything, write, update, write again is the easiest way that comes to mind
<dimitern> rogpeppe: but perhaps there's a more efficient way, have to experiment a bit, to do it with a single write
<rogpeppe> dimitern: i'm not sure there's a "delete everything" mongo operation
<dimitern> rogpeppe: there is $unset deletions
<dimitern> rogpeppe: that's what Write does
<rogpeppe> dimitern: yes, but you have to know what attributes to unset
<rogpeppe> dimitern: and the way you're using Settings, unset will never happen
<dimitern> rogpeppe: all of them, which do not conflict with the new ones, which are simply updated
<rogpeppe> dimitern: how do you know the "all"?
<dimitern> rogpeppe: yes, we already discussed that, I change what's being done on WriteSettings
<dimitern> rogpeppe: there's Keys()
<dimitern> rogpeppe: or, perhaps simpler - get and delete each key not in the new map
<dimitern> rogpeppe: just delete actually should do it
<rogpeppe> dimitern: ISTM like the state.Settings object is actually getting in the way here.
<dimitern> rogpeppe: after doing Read to populate the disk cache
<dimitern> rogpeppe: how so?
<rogpeppe> dimitern: because you'll be duplicating the same kind of logic that's already inside Settings
<dimitern> rogpeppe: not much - just a for loop for deletions, a read before it, and update + write after it
<rogpeppe> dimitern: part of the reason i'm slightly concerned here is that i suspect that these settings writes may be a good proportion of the ongoing mongo traffic in a large juju installation
<dimitern> rogpeppe: yeah, and how is my approach different from what's already happening?
<dimitern> rogpeppe: we always read them to create a settings object, do some ops on it, and call write finally - only the first and last operations are actually mongo ops
<rogpeppe> dimitern: currently for each hook execution, we only have two mongo ops: read then write
<rogpeppe> dimitern: we're going to change that so that it reads twice
<rogpeppe> dimitern: and also currently, we read the settings, but we only write the data for the ones that have changed
<dimitern> rogpeppe: not really, we can change the settings object to provide ClearCache method
<rogpeppe> dimitern: how will that work?
<dimitern> rogpeppe: that way we'll still have 1 read and 1 write
<rogpeppe> dimitern: i don't know how if there's a mongo operation to delete all attributes on a doc without knowing what those attributes are.
<dimitern> rogpeppe: it doesn't matter how many keys we set or unset - it's still a single update operation
<dimitern> rogpeppe: I'm not talking about a mongo operation
<dimitern> rogpeppe: I'm talking about ClearCache method which clears the settings object's internal disk cache
<dimitern> rogpeppe: the same one write uses to determine which keys to unset
<rogpeppe> dimitern: uniter.go:725  reads all the settings; uniter.go:728 writes them
<rogpeppe> dimitern: every hook execution will call ReadSettings (which reads all the settings), then WriteSettings (which reads all the settings then writes them)
<dimitern> rogpeppe: can we please stop discussing what's there now and focus on what we want to have there?
<dimitern> rogpeppe: that's what I'm trying to explain
<rogpeppe> dimitern: ok, so i think we still want to have ReadSettings followed by WriteSettings, right?
<dimitern> rogpeppe: yes
<dimitern> rogpeppe: it's the same as now
<rogpeppe> dimitern: so the question is: how does WriteSettings delete any attributes which aren't in the map we're writing?
<dimitern> rogpeppe: RU.Settings() does readSettings, Settings.Write() does the write
<rogpeppe> dimitern: right - so we'll still have exactly the situation i described above, no?
<dimitern> rogpeppe: by deleting them from the map, unless they are to be overwritten by keys in the new map
<rogpeppe> dimitern: i.e. we'll read all the settings twice, then write them
<dimitern> rogpeppe: ah, right, but in 2 separate API calls
<rogpeppe> dimitern: yes
<rogpeppe> dimitern: but the increase in mongo traffic is real
<dimitern> rogpeppe: read does read, write does read+delete+update+write
<dimitern> rogpeppe: how can you measure that?
<rogpeppe> dimitern: write *should* just do a single mongo update transaction, i think
<rogpeppe> dimitern: and i don't think it's at all hard to arrange that
<dimitern> rogpeppe: we can't avoid the read in WriteSettings
<rogpeppe> dimitern: yeah, i think we can
<dimitern> rogpeppe: we don't have the orinal settings object around anymore
<rogpeppe> dimitern: we'll just have to bypass the Settings object entirely.
<rogpeppe> dimitern: it's not really appropriate for the server side
<rogpeppe> dimitern: however....
<dimitern> rogpeppe: ah, so you're saying implment state.WriteSettings that does all that in one go?
<rogpeppe> dimitern: yeah. or probably RU.WriteSettings
<dimitern> rogpeppe: ok, that sounds reasonable
<rogpeppe> dimitern: however... perhaps we could leave this as a TODO
<dimitern> rogpeppe: and how will that work internally, without knowing what keys are already there?
<rogpeppe> dimitern: we treat nil elements in the map passed to WriteSettings as deletions
<rogpeppe> dimitern: the client side keeps track of deletions and passes them through as nils
<dimitern> rogpeppe: that seems worse traffic-wise
<rogpeppe> dimitern: only if deletions are common
<dimitern> rogpeppe: now, instead of sending only stuff we actually want to store, we'll send also deleted things
<rogpeppe> dimitern: we'll be sending deltas
<rogpeppe> dimitern: if there are more than a few attributes that aren't changed, then it'll definitely be a win
<dimitern> rogpeppe: yeah, that's one of those questions with no easy answers without having to go through every charm hook
<rogpeppe> dimitern: yeah
<rogpeppe> dimitern: it would be great to have some operational data to inspect
<rogpeppe> dimitern: i really have to go, i'm afraid
<dimitern> rogpeppe: ok, so: 1) treat key->nil as deletions, 2) update the rest
<rogpeppe> dimitern: for the time being, i think you could leave it as it is
<dimitern> rogpeppe: 3) add a todo to possibly replace all that with a RU.WriteSettings call
<rogpeppe> dimitern: we won't be able to delete anything, but i don't think that's a big problem for the time being
<rogpeppe> dimitern: actually, yes, that will work better
<dimitern> rogpeppe: ok, let's continue on monday then
<rogpeppe> dimitern: then deletions will actually work, and we can be more efficient later
<rogpeppe> dimitern: cool, thanks for going along with me
<dimitern> rogpeppe: no worries, it was useful
<dimitern> rogpeppe: have a good weekend! :)
<rogpeppe> dimitern: i will - going to a folk festival, yay!
<rogpeppe> dimitern: and you
<rogpeppe> dimitern: ttfn
<rogpeppe> g'night all
<dimitern> yay :)
<dimitern> fiddles then
<rogpeppe> oooh yes
 * dimitern @ eod as well
<dimitern> see y'all on monday!
#juju-dev 2013-08-18
<hazmat> i'm trying to connect to the mongodb directly using the admin credential in environments.yaml but i always get auth fail errors. is the admin password for the juju db just the admin-secret in environments.yaml
<thumper> morning
<weblife> can you not bootstrap to a t1.micro instance?
<weblife> on aws
<thumper> no
<thumper> :)
<thumper> weblife: but adding in the provider specific constraints is on the short term roadmap
<weblife> thumper:  hurah,  you mean in the environments.yaml file right?
<thumper> weblife: well, no, we are changing the config file
<weblife> :x
<thumper> but you'll be able to set it somewhere
<weblife> lol
<thumper> I'm just not 100% sure where yet
<weblife> Reading the 2.0 milestone blueprint for ssh.  I like the idea.  I could use that feature now :x
#juju-dev 2014-08-11
<axw> waigani: #1270466 can be closed now?
<axw> hm, mup doesn't recognise LP bug IDs anymore? :(
<waigani> axw: closed
<axw> cool
<waigani> axw: so getting api endpoints at end of bootstrap. I can get machine 0's public address. But what about HA? I see a machine has a DistributionGroup. Should I use that to see if we are in HA and get the endpoints of the others? I think api.Client.APIHostPorts() would get them all, but that func is not allowed while 'in upgrade'
<axw> waigani: there's only one machine during bootstrap. bootstrap already grabs that machine's address so it can SSH in
<axw> waigani: see waitSSH in provider/common/bootstrap.go
<waigani> axw: will do, cheers
<axw> waigani: once Bootstrap has returned successfully, you should be able to assume that the instance has addresses, so you don't need the complicated logic from there
<axw> you may just want to return the addresses from there tho
<waigani> axw: ah, perfect. I assumed something like that was going on, as bootstrap finished but common.ProviderStateInstances was not finding any instances. I'll use waitSSH to wait
<axw> waigani: just to be clear, waitSSH is already used by bootstrap. So you should just be able to bubble up the addresses it calculates
<waigani> axw: I have not been able to do that so far
<waigani> axw: but I'll have a look at how waitSSH is used and see what I'm missing
<axw> waigani: provider/common.Bootstrap should return addresses. it should get them from FinishBootstrap, which should get them from waitSSH
<waigani> axw: ah, the dummy bootstrap does not use common.Bootstrap
<axw> waigani: you'll need to change Environ.Bootstrap's signature to return hostports, and dummy/manual/local will need different logic as they don't use common.Bootstrap
<axw> not that straightforward; maybe you do want to make an API call for the first cut
<axw> waigani: if you make an API connection, *any* method (assuming Login passes validation), then you
<axw> 'll get back the hostports
<axw> cached by juju.NewAPIConn
<waigani> axw: sorry otp, reading now
<waigani> axw: yeah, so I that is where I've ended up, dummy and local don't use common.Bootstrap, so I'm doing an API call to get endpoints (EnvCommandBase.NewAPIClient()). But client.APIHostPorts() is not allowed, because we are in 'upgrade' at the end of bootstrap.
<waigani> axw: so I can only (so far) get the public address of machine 0
<axw> waigani: (you could just return localhost for local, and any old thing for dummy...)    -- but you don't actually need to call APIHostPorts, you could just call, say, client.Status
<axw> that is allowed during upgrade
<axw> the act of making a method call will cache the addresses
<axw> you don't need to explicitly do anything with that
<axw> waigani: actually, I don't think you even need to make a method call. just open and close the API connection
<waigani> axw: that was my first thought but thumper was adamant that just calling status does not work, he said he tried. I'll try myself (to be sure)
<axw> waigani: you can see how it works in juju/api.go
<axw> at the bottom of newAPIFromStore
<waigani> axw: grrrr, just calling state bloody works. I'll try just connecting to the api
<axw> waigani: just calling status you mean?
<waigani> axw: yes sorry
<axw> sure, just checking I understand
<waigani> yep, that works - sigh
<waigani> thanks axw
<axw> waigani: cool, nps
<bigjools> how much use do you guys make of syslog?  I got the impression from the ML that you're thinking of stopping using it?
<axw> bigjools: we don't use syslog directly, we write our logs to a file and have rsyslog tail it
<axw> then they all send to a centralised rsyslog to aggregate
<bigjools> ok
<bigjools> In maas we're moving towards just sending everything to syslog and letting it do the heavy lifting
<axw> bigjools: does that get sent anywhere remote? aggregated?
<bigjools> the plan is to aggregate somewhere, yes, not sure where yet
<axw> mmk
<bigjools> well we don't have a "central" place
<bigjools> so the decision is not easy :)
<axw> when I said centralised, I actually meant to all of the juju state servers
<axw> so it's still HA
 * axw doesn't know MAAS well enough to draw parallels
<bigjools> right
<bigjools> we might do something similar and sync to all appservers
<dimitern> morning all
<voidspace> yay, online
<voidspace> morning everyone
<TheMue> morning
<tasdomas> could someone take a look at https://github.com/juju/juju/pull/464 ?
<TheMue> tasdomas: yep, will do.
<tasdomas> TheMue, thanks!
<fwereade> tasdomas, LGTM with a followup, I think the problems may run a little deeper
<TheMue> fwereade: just seen youâve LGTMed the PR
<voidspace> hmm... when I try to submit a "national holiday" to hr.canonical.com it asks me to attach supporting evidence to the request...
<mattyw> rogpeppe, would you be able to take another look at https://github.com/juju/juju/pull/469
<rogpeppe> mattyw: looking
<mattyw> voidspace, https://www.gov.uk/bank-holidays
<mattyw> voidspace, I think Auntie Liz has to approve bank holiday's each year so I did a quick google to see if I could find a pdf of her signature
<rogpeppe> mattyw: i'm mostly +1 on fwereade's suggestion
<rogpeppe> mattyw: except that i'd use Reset rather than creating a new timer each time
<mattyw> rogpeppe, that is what's i'm doing in the latest code
<rogpeppe> mattyw: oh, weird. i never quite get how github comments work
<mattyw> rogpeppe, oh right - fwereade commented 4 minutes ago
<mattyw> rogpeppe, so you were refering to a comment I hadn't seen yet :s - concurrency on a monday morning
<rogpeppe> mattyw: his code is pretty much functionally equivalent, but slightly neater and fits with a common worker pattern
<voidspace> mattyw: hehe,
 * fwereade bbiab
<voidspace> TheMue: dimitern: internet has recovered, so I may make standup after all
<voidspace> but first, coffee...
<dimitern> voidspace, great :) see you there
<perrito666> morning
<tasdomas> could somebody take a look at https://github.com/juju/juju/pull/455 ?
<wallyworld> katco: mgz: standup?
<mgz> no katco yet alas.
<tasdomas> I'm seeing a failure on master in the test suite for utils/ssh : http://paste.ubuntu.com/8016636/ - is anybody else seeing something like this?
<mgz> tasdomas: nope, what does `ssh -V` say for you?
<mgz> ...not that that should matter actualyl
<tasdomas> mgz, ssh: Could not resolve hostname hostname: Name or service not known
<mgz> tasdomas: your ssh looks broken then, check yuor network config?
<tasdomas> mgz, that's the error I get when running the ssh line in terminal
<tasdomas> mgz, ssh -V : OpenSSH_6.6.1p1 Ubuntu-2ubuntu2, OpenSSL 1.0.1f 6 Jan 2014
<mgz> oh, I was right, the ssh should in fact not be used
<mgz> tasdomas: try `BASH_ENV="" go test`?
<tasdomas> mgz, seems like it was a godeps issue - forgot to run godeps -u
<mgz> fair enough.
<tasdomas> mgz, and I need to update my vm
<tasdomas> mgz, thanks
<natefinch> fwereade: you around?
<fwereade> natefinch, heyhey
<fwereade> natefinch, how's it going?
<natefinch> fwereade: not bad.  Well rested from my week of not traveling ;)
<wwitzel3> lol, too soon
<natefinch> fwereade:  I'm a bit stuck on what to do about log rotation.  There's a few different bad options, and I'm not sure how to choose between them
<natefinch> wwitzel3: heh
<fwereade> natefinch, haha
<wwitzel3> natefinch: I'm working on tht now, with rsyslog, should I not be, or you talking about Windows?
<fwereade> natefinch, wwitzel3: windows is also interesting, gsamfira certainly has relevant input there
<natefinch> wwitzel3: you're doing all-machines, right?
<fwereade> natefinch, wwitzel3: I would hope that log rotation would encompass *all* our logs, even if the work came in a few stages
<natefinch> fwereade: yes of course.  There's really only a couple spots to hit - all-machines is a different beast than the machine-n and unit-n logs
<fwereade> natefinch, cool
<wwitzel3> natefinch: yeah, all-machine
<wwitzel3> s
<natefinch> wwitzel3: cool, yes, there's no real option for anything but logrotate / rsyslog rotation there.
<gsamfira> not even if we log directly to rsyslog from juju? in the case of all-machines.log I mean
<gsamfira> ?
<natefinch> gsamfira: the rotation has to be on the state machine, which is listening using rsyslog
<gsamfira> that's fine. That should be easily doable using standard logrotate
<gsamfira> and for all other juju logs, lumberjack can take care of the rotation
<gsamfira> no?
<natefinch> gsamfira: yes, sorry if I wasn't clear. That's what wwitzel3 is doing
<gsamfira> ahh
<wwitzel3> yep :)
<gsamfira> my mistake. Read diagonally
<natefinch> the thing that I'm talking about is rotating the unit-n and machine-n logs.  Some people have questioned why we don't just use logrotate for those as well, and only make windows use something else.
<natefinch> my preference would be that if we can do something the same everywhere, to do it the same, that way we don't need to maintain different codepaths depending on OS
<gsamfira> well, as you pointed out on the mailing list, its better to have one config that works across platforms then 2-3 separate ones :)
<natefinch> yes
<gsamfira> yep
<gsamfira> :)
<gsamfira> the more os specific  dependencies you can remove, the better
<natefinch> I believe that as well
<natefinch> fwereade: the question is - how far do we go?  Do we write directly to syslog from inside juju, thereby bypassing rsyslog and not needing to worry about whether it misses log messages due to rotation?  Or do we continue with rsyslog and figure out how to make that work with rotation?  Or do you disagree with me. and think we should use logrotate on linux?
<gsamfira> implementing syslog using Go, should be simple enough. I will cleanup the fork I made of the standard syslog package, and I can have a shot at implementing it in Juju by tomorrow.
<fwereade> natefinch, so, if we can write directly to syslog, I would like that very much
<fwereade> natefinch, I think thumper was going to reply to your post explaining the problems he'd found with go syslog
<natefinch> fwereade: ahh, that would be very good to hear about.
<fwereade> natefinch, apart from handwaving about how it outputs stuff is in some way inconvenient I can't actually remember the issue with it myself
<gsamfira> hmm. Syslog implements io.Writer. In theory, it should be like writting to a file. The only thing is that you need to set a prefix, which you would normally do via the rsyslog config
<gsamfira> but nay formatting should be done via loggo
<natefinch> gsamfira: as long as loggo does it's output in a single write, that's ok.  If it batches them or breaks a single line into multiple writes, it can be a problem.  the syslog package expects each write to be a single log message.
<gsamfira> I would like to try and tackle any issues that might have stopped us from using this. Until better log aggregation is found at least :)
<natefinch> gsamfira: I think that would be great
<wwitzel3> natefinch: standup :)
<gsamfira> natefinch: Hopefully loggo adheres to that expectation as well :). I'll have a look
<katco> good morning all
<katco> mgz: wallyworld: sorry about missing the standup
<perrito666> natefinch: wwitzel3 ericsnow suddenly the page died on my and I can no longer connect sorry
<mgz> katco: no problem, ian will catch us up on last week tomorrow
<katco> mgz: ok cool
<natefinch> perrito666: np
<mgz> natefinch: have you seen bug 1355219?
<bodie_> morning all
<natefinch> mgz: go get gopkg.in/natefinch/npipe.v2 works for me
<natefinch> mgz: It looks like we assume juju-core_1.21-alpha1.tar.gz has it in there.... we should look at how that's generated and why it's not getting juju's dependencies.
<natefinch> mgz: my guess is that it solely uses godeps and someone either messed up godeps or forgot to update it.
<natefinch> brb
<dimitern> anybody willing to review a quick fix https://github.com/juju/juju/pull/491 for bug https://bugs.launchpad.net/juju-core/+bug/1353443 ?
<dimitern> TheMue, rogpeppe, voidspace, fwereade ^^
<rogpeppe> dimitern: looking
<dimitern> rogpeppe, ta!
<rogpeppe> dimitern: what if we're running inside kvm inside lxc? should that trigger the same logic?
<dimitern> rogpeppe, so the kvm is the host, and the lxc is where the MA+networker runs?
<rogpeppe> dimitern: i was thinking the other way around actually
<dimitern> rogpeppe, i don't think you can run kvm inside lxc
<rogpeppe> dimitern: you sure about that?
<dimitern> rogpeppe, 99%
<dimitern> rogpeppe, it doesn't make sense, as kvm needs full access to hardware virtualization, which an lxc host cannot provide
<rogpeppe> dimitern: i'm still a bit uncomfortable about assuming that there's no container type that can run inside lxc
<rogpeppe> dimitern: now and for the indefinite future
<dimitern> rogpeppe, lxc can run inside lxc
<rogpeppe> dimitern: ok, no non-lxc container type :-)
<rogpeppe> dimitern: i'd be happier if the logic looked at the last container type
<rogpeppe> dimitern: because that's what it actually cares about, no?
<dimitern> rogpeppe, you mean the logic about parsing the machine id ?
<rogpeppe> dimitern: yes
<rogpeppe> dimitern: specifically the fact you use strings.Contains
<natefinch> sinzui: you around?
<dimitern> rogpeppe, well, my idea there is that if lxc is involved at any level of nesting, we shouldn't modprobe
<natefinch> mgz: I presume that 1.21 build is running off of trunk?
<mgz> natefinch: yup
<voidspace> dimitern: that's a big test for a little bit of code
<natefinch> mgz: k thanks
<dimitern> voidspace, it's a fairly involved worker for all it does :)
<voidspace> heh, right
<dimitern> voidspace, we need to simplify it
<voidspace> "we"
<voidspace> :-p
<voidspace> I'm going afk for a while (two hours)
<rogpeppe> dimitern: i have difficulty working out if that's a valid decision or not
<rogpeppe> dimitern: because it's not clear to me that you might not be able to nest some container type that *does* allow modprobe
<dimitern> rogpeppe, i'm chatting with jamespage and hallyn at #server (@c) about this actually
<jamespage> dimitern, rogpeppe: lets talk here
<dimitern> jamespage, right, sorry about the context switching :)
<jamespage> dimitern, np
<dimitern> jamespage, so, is my idea about not allowing modprobe if lxc is involved at any level of nesting, not only the last level?
<jamespage> dimitern, rogpeppe: its possible to hack an LXC container sufficiently so it can run a KVM instance - but this still runs within the host kernel
<dimitern> is it a correct approach to the problem, i mean
<jamespage> dimitern, a KVM instance will always support modprobe as it has its own kernel etc...
<jamespage> dimitern, LXC will not and should not
<jamespage> dimitern, this is why its possible to run the nova-compute charm on KVM, but not LXC
<dimitern> jamespage, so you can modprobe 8021q for example inside the kvm, hosted inside an lxc without 8021q on its respective host?
<jamespage> dimitern, yes
<jamespage> dimitern, in the very hyperthetical situation that is true
<dimitern> jamespage, ok, so to be absolutely correct, we should check what rogpeppe suggests - only the last-level-of-nesting container, which runs the agent should not be lxc
<rogpeppe> i think we should allow that case, even if hypothetical
<jamespage> dimitern, yes
<dimitern> jamespage, rogpeppe, ok, fair enough, thanks for the feedback
<rogpeppe> dimitern: another approach might be just to ignore that error from modprobe
<rogpeppe> dimitern: that way if someone fixes lxc, the code will just work again
<dimitern> rogpeppe, I don't really like this
<dimitern> rogpeppe, because it's hard enough to deal with the output lsmod and modprobe give anyway
<rogpeppe> dimitern: yeah
<rogpeppe> dimitern: another thought is to put the "isRunningInLXC" logic inside (or called from) ensureVLANModule
<rogpeppe> dimitern: because at the moment it's really not clear why we  call isRunningInLXC before calling ensureVLANModule
<rogpeppe> dimitern: but if we passed the machine id into ensureVLANModule and did the check in there, the connection is obvious (and the comments would read nicely too)
<dimitern> rogpeppe, ensureVLANModule should not be called like this, but this is another issue - we should only call it, if there are any VLANs to setup
<jcw4> Fix for LP-1355319 : https://github.com/juju/juju/pull/493
<dimitern> rogpeppe, updated isRunningInLXC a bit to check the last level container type
<rogpeppe> dimitern: looking
<rogpeppe> dimitern: i don't think that will actually work, will it?
<natefinch> I hate it when I know I'm doing something stupidly wrong.... but still can't figure out what it is
<dimitern> rogpeppe, why?
<rogpeppe> dimitern: don't you want something like: if len(parts) > 2 && parts[len(parts)-2] == "lxc" { ?
<rogpeppe> dimitern: because AFAICS the logic you've got now checks whether *any* of the splits holds the string "lxc"
<dimitern> rogpeppe, ah, because it could be "0", yes ofc
<dimitern> rogpeppe, it checks the last split first going forward
<rogpeppe> dimitern: better test case required :-)
<rogpeppe> dimitern: i'd write a test case for isRunningInLXC directly
<dimitern> rogpeppe, you mean add a test for isRunningInLXC specifically?
<dimitern> ok
<rogpeppe> dimitern: it would seem worth it, yes
<rogpeppe> dimitern: your current logic doesn't seem to test the last split moving forward...
<jcw4> wallyworld, perrito666 since you're on call reviewers :)  PTAL https://github.com/juju/juju/pull/493
<jcw4> trivial dependencies.tsv fix
<perrito666> jcw4: certainly
<jcw4> perrito666: tx
<jcw4> natefinch: shall I just close https://github.com/juju/juju/pull/493 ? (perrito666)
<natefinch> jcw4: leave it open for now, if I can't get my fix in before yours, we can just use yours.  But I'd like to see if I can get mine in so godeps produces the correct result
<jcw4> natefinch: +1 :)
<perrito666> jcw4: I was really expecting a larger patch
<jcw4> perrito666: :)
<rogpeppe> dimitern: what calls networker.SetUp ?
<rogpeppe> dimitern: ah, it's StringsWorker
<rogpeppe> dimitern: why are privateInterface and priviateBridge global variables?
<rogpeppe> dimitern: and unguarded by a mutex
<rogpeppe> dimitern: that gets my trouble whiskers twitching
<dimitern> rogpeppe, that's all vlad's code, and needs some work
<rogpeppe> dimitern: but it's landed in core? hmm.
<dimitern> rogpeppe, yes, but it's hardly ever used for now - only with maas and only when there are networks defined
<rogpeppe> dimitern: yeah but... sigh
<dimitern> rogpeppe, I agree it needs refactoring, there are tech dept bugs for some things already
<dimitern> rogpeppe, thanks for bearing with me :)
<dimitern> rogpeppe, updated yet again - isrunninginlxc fixed + added a test
<dimitern> rogpeppe, all comments addressed or replied to, can I get LGTM? :)
<katco> is there a way to do an in-place branch in bazaar? creating a branch for some code changes is messing up the imports since it creates a new directory.
<rogpeppe> dimitern: i'm still trying to grok the test
<dimitern> rogpeppe, so there's a fair amount of boilerplate setup code in there
<dimitern> rogpeppe, the main thing is patching the executeCommands and monitoring modprobe does not happen before ifup commands
<dimitern> rogpeppe, I tested if fails when I comment out the && !isRunningInLXC part
<jcw4> natefinch: the more I think about it the more I think I should just close my PR and let you do the full fix.  If we're going to fix this bug we should fix it right and your plan sounds right :)
<natefinch> jcw4: I have mine ready to propose anyway
<jcw4> natefinch: perfect; thx
<rogpeppe> dimitern: review done
<dimitern> rogpeppe, thanks!
<natefinch> rogpeppe: quick review? https://github.com/juju/juju/pull/494
<natefinch> jcw4: ^^
<rogpeppe> natefinch: i hope you mean "godeps -t ./..."
<natefinch> oh oops
<natefinch> rogpeppe: no, but they apparently produce the same results on github.com/juju/juju
<natefinch> (currently)
<rogpeppe> natefinch: i guess that's not actually too surprising
<rogpeppe> natefinch: because of the testing packages
<dimitern> wtf is this?! "Build failed: Does not match ['fixes-1355219', 'fixes-1347715']\n        build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/261\n"
<natefinch> rogpeppe: yep, was thinking that
<dimitern> neither of those were mentioned in the PR at all
<rogpeppe> dimitern: you *need* to mention those in the PR
<rogpeppe> dimitern: because they're the critical bugs that currently need fixing
<natefinch> dimitern: master is locked
<dimitern> natefinch, rogpeppe, I see, ok
<natefinch> dimitern: I have this labeled as "CI blockers" in my bookmarks bar: https://bugs.launchpad.net/juju-core/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.importance%3Alist=CRITICAL&field.tag=ci+regression+&field.tags_combinator=ALL
<dimitern> nothing to worry about then, it will get picked up by the bot when trunk is unblocked?
<natefinch> that I'm not sure about
<natefinch> you might need to repropose
 * rogpeppe adds that to his bookmarks bar too
<rogpeppe> i actually want a status constantly showing whether juju-core is currently blocked
 * dimitern as well, thanks!
<natefinch> (that link is from curtis)
<natefinch> rogpeppe: challenge: add it to your shell prompt ;)
<rogpeppe> natefinch: i don't believe in dynamic shell prompts
<natefinch> rogpeppe: ha ok
<rogpeppe> natefinch: too much noise
<rogpeppe> natefinch: i just have a single character for my prompt
<natefinch> rogpeppe: I find it handy for seeing the current git branch, but yeah, I get that it can be noisy
<natefinch> rogpeppe: where would you want to see the notification that master is blocked?
<rogpeppe> natefinch: in the (largely blank) notification bar at the top of my screen
<dimitern> rogpeppe, you won't like my shell prompt then: [maas-trusty] (lp-1353443-dont-modprobe-in-lxc) dimitern@kubrik:~/work/juju-core$
<dimitern> :)
<natefinch> rogpeppe: fair enough.  I hhave no idea how to do that.
<rogpeppe> dimitern: i'm surprised you've got any room to type a command there! :-)
<dimitern> rogpeppe, like most problems in the world, this as well can be solved with a bigger screen :D
<rogpeppe> for many of the shell commands i execute, i don't have a command prompt at all :-)
<rogpeppe> dimitern: yeah, i could do with a 6 foot by 6 foot screen :-)
<rogpeppe> natefinch: why does your PR update the mgo dependency too?
<dimitern> rogpeppe, but then again... who couldn't!
 * dimitern needs to stop for today
<dimitern> DmC hack&slash awaits!
<natefinch> rogpeppe: it's the same commit, it was just in the wrong (not alphabetical) spot
<rogpeppe> natefinch: ha, good point :-)
<rogpeppe> dimitern: enjoy!
<natefinch> rogpeppe: but thanks for checking... I realized I hadn't actually double checked that the commit numbers were the same.
<rogpeppe> natefinch: reviewed
<natefinch> rogpeppe: what portable API could it export?
<rogpeppe> natefinch: a named pipe implementation that works on all platforms?
<perrito666> rogpeppe: that is a bit overkill
<gsamfira> named pipes on windows are a bit different then on linux. They are more akin to unix sockets.
<natefinch> rogpeppe: it's just an implementation for net.Listener and net.Conn
<natefinch> (etc)
<rogpeppe> natefinch: the portable API could be almost exactly that exported by the juju/sockets package
<rogpeppe> natefinch: (speaking of which, why is that package under juju at all?)
<natefinch> rogpeppe: that's where a bunch of our OS-specific abstractions live... probably just by accident, because thats where osenv lived first
<rogpeppe> natefinch: osenv *is* very juju specific
<rogpeppe> natefinch: sockets is not at all juju specific
<rogpeppe> natefinch: FWIW, i'm not really sure why we're using named pipes at all under windows. we could presumably just use localhost sockets instead (the current port could be stored in the unit agent directory)
<rogpeppe> natefinch: hmm, just thought: does juju-run mean that any process on the host can get root privileges?
<sinzui> hi natefinch
<natefinch> sinzui: hey
<gsamfira> rogpeppe: that's how it was implemented in the PoC (tcp sockets bound to localhost). The run.socket actually held 127.0.0.1:<PORT> :)
<rogpeppe> gsamfira: PoC ?
<natefinch> proof of concept
<gsamfira> Proof of Concept
<rogpeppe> gsamfira: ah. that seems fine to me. apart from you can't hide the socket inside a root-accessible-only directory, i guess. which i *hope* we're doing currently.
<gsamfira> TCP sockets are slower though. It was noticeable when running juju commands.
<gsamfira> well, on windows at least, named pipes are not represented on disk
<rogpeppe> gsamfira: *really*? the sockets are that high-bandwidth?
<gsamfira> its not bandwidth...but the whole protocol stack init process and round trip
<gsamfira> named pipes are recommended for Inter-process communication on windows. Chrome/Firefox use them
<gsamfira> overhead is minimal
<gsamfira> and they behave like unix sockets
<gsamfira> but with a file API
<gsamfira> so you treat them as you would any file
<rogpeppe> gsamfira: i guess that a hook will make quite a lot of these connections
<gsamfira> but it allows bidirectional communication
<gsamfira> yes
<gsamfira> most do
<gsamfira> even juju-run was slower then with named pipes
<gsamfira> the biggest issue was allocating ports though
<rogpeppe> gsamfira: that was slow?
<gsamfira> because there is no database locally, you needed to find a free one
<gsamfira> and each time you deploy a new charm, you need to make sure you don't overlap
<gsamfira> a simple directory listing via juju-run would take about a second
<gsamfira> :)
<gsamfira> so in terms of allocating a named pipe, its simple. you have `\\.\pipe\unit-tag-run` and `\\.\pipe\unit-tag-agent`
<gsamfira> ohh, and it was easier for an installed application (by a charm for example) to take over the port you allocated. So when a juju command would try to run, it would fail to bind on that port.
<rogpeppe> gsamfira: couldn't you just let the OS allocate one for you?
<rogpeppe> gsamfira: i guess that last is a potential problem though
<gsamfira> you could in theory allocate a different port each time.
<gsamfira> but in the end it was simpler just to use nate's named pipes module :)
<rogpeppe> gsamfira: :-)
<gsamfira> named pipes is an unfortunate name
<rogpeppe> gsamfira: yeah.
<gsamfira> especially considering that on unix it has such different behavior
<natefinch> namespaces are important :)
<rogpeppe> gsamfira: a better name might be fsnet, i suppose (for "file-system network")
<natefinch> windows.NamedPipe != linux.NamedPipe
<rogpeppe> natefinch: you couldn't possibly know how much i agree with you there :-)
<rogpeppe> natefinch: (about namespaces)
<gsamfira> yep :)
<rogpeppe> natefinch: i still think that npipe should export a portable interface
<gsamfira> well, its kind of hard to do
<rogpeppe> natefinch: so that anyone that wants a filesystem based listener can use it, and their code will work regardless of whether it's on windows or not
<gsamfira> named pipes on windows allows bi-directional communication, whereas on linux it does not
<natefinch> rogpeppe: it's specifically a windows implementation... I think it would be a good idea to provide a different package that uses it to produce a platform-agnostic API
<natefinch> rogpeppe: maybe the problem is just that it should be called winnpipe or something
<rogpeppe> natefinch: what is in the npipe API that's windows-specific?
<gsamfira> whatever it ends up being called, I would love to see it in the standard "net" package :)
<perrito666> natefinch: why dont you call the thing winPipes :p
<rogpeppe> natefinch: i know you've written it as a window-specific package, but i don't see why it has to be. by making it portable, you make it more useful.
<rogpeppe> s/window/windows/
<rogpeppe> natefinch: i guess the name space argument looks quite different
<gsamfira> rogpeppe: I'm not sure named pipes on Linux can be used in the same way
<rogpeppe> gsamfira: it wouldn't use named pipes on linux
<rogpeppe> gsamfira: it would use unix-domain sockets
<gsamfira> hmm, but the net package already does that
<rogpeppe> gsamfira: sure. but not for windows.
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs 1355320, 1347715, 1355324
<gsamfira> if nate's implementation makes it into the stdlib, we should be all set :)
<rogpeppe> gsamfira: so this would be a portable way of getting the same functionality using either net directly or the windows named pipe stuff
<gsamfira> my hope is that at some point we could do a rpc.Dial("local", path) and have the rpc package use named pipes on windows
<gsamfira> and unix sockets in linux
<gsamfira> without any 3rd party package
<gsamfira> but I see what you are saying
<gsamfira> a helper package would be ok until that happens
<rogpeppe> gsamfira: rpc doesn't have a Dial function :-)
<rogpeppe> gsamfira: well, the juju one anyway
<natefinch> rogpeppe: yeah, the real problem is that net.Listen("namedpipe", "\\.\foo") doesn't call my code on windows :)
<natefinch> rogpeppe: that's the cross-platform API
<rogpeppe> natefinch: we should always try to make cross-platform APIs when possible, in my view
<natefinch> rogpeppe, gsamfira: I'd love to get the package into the stdlib... I don't know how likely that is, though.
<rogpeppe> natefinch: platoform-specific code should be pushed down to as low a level as possible
<natefinch> rogpeppe: I agree completely
<natefinch> rogpeppe: like I said, it should live in platform-specific code inside net, if that were possible
<rogpeppe> natefinch: sure, but given that it can't why not provide an interface similar to what you'd hope the net package *should* provide
<rogpeppe> ?
<rogpeppe> natefinch: so that folks can use the abstraction your package provides without needing to write windows-specific shims like our juju/sockets package
<gsamfira> I guess it should be a simple addition. something like fsnet.Listen(path) and fsnet.Dial(path). Basically what the sockets package does :)
<gsamfira> but if you plan on proposing it for stdlib, I can see why you would not want this
<natefinch> rogpeppe: I dunno, I prefer just having a package that is compatible with the net package.  I don't think it's really that bad, and it keeps my package more simple
<rogpeppe> natefinch: for instance, i think it's really crappy that the highest level code in juju (cmd/jujud) needs to define RunCommand.winSockPath
<natefinch> rogpeppe: that could live somewhere less prominent
<rogpeppe> natefinch: it's only compatible with the net package on windows.
<natefinch> rogpeppe: it only need to be compatible with the net package on windows. It can't run on linux.
<rogpeppe> natefinch: if the abstraction was right, there would be no need for two separate paths
<rogpeppe> natefinch: the abstraction can surely be implemented under linux
<gsamfira> that's my fault. I'll have a go at cleaning that up as soon as the final bits of the windows support is in. apologies
<natefinch> rogpeppe: both sides have to agree to use named pipes.  You can't have both sides randomly picking some transport and assume they'll line up.
<natefinch> rogpeppe: so, at some point, you have to say "hey, used named pipes", and that's only possible on windows
<rogpeppe> natefinch: sure. you'd need to document that it use unix sockets under linux and named pipes under windows.
<natefinch> rogpeppe: named pipes work across the network too.  You can't connect to a named pipe on another machine from a unix socket on this machine
<rogpeppe> natefinch: and you'd need to document how the mapping is made between dial/listen addresses and the underlying named pipe/socket addresses
<rogpeppe> natefinch: that's fine - it's an implementation restriction.
<natefinch> rogpeppe: I don't like letting someone do a listen on one machine and a dial on the other using the same code and then have it not work because one is windows and one is linux.
<rogpeppe> natefinch: obviously you can't accomplish the impossible.
<rogpeppe> natefinch: if you defined things right, it would be obvious when you were dialling or listening in an OS-specific name space
<natefinch> rogpeppe: I think that sometimes making it ugly and obvious is better than making it pretty and obscured.  The fact that both the client and server need to be on the same OS makes me want to make sure it's super obvious this is windows-only, and you can't just listen from linux and have it work.
<rogpeppe> natefinch: if i was doing it, i might make a package interface something like this: http://paste.ubuntu.com/8019246/
<rogpeppe> natefinch: i think the fact that the paths have backslashes in is a pretty good indication of that
<rogpeppe> natefinch: i suspect that most people will be using named pipes for intra-machine communication
<rogpeppe> natefinch: (we certainly are)
<natefinch> rogpeppe: we are, and that's why we can safely just pick whichever transport and know the other side will work.  But I don't want my very specific package to encourage generic use without people thinking about it.
<rogpeppe> natefinch: why would anyone use your package unless they knew what they needed it for?
<rogpeppe> natefinch: i just hate to see OS-specific stuff spread throughout juju-core unnecessarily
<rogpeppe> anyway, gotta go
<rogpeppe> natefinch: ttfn
<natefinch> rogpeppe: see ya
<natefinch> rogpeppe: FWIW, I don't disagree.  an OS-agnostic wrapper around npipe is certainly welcome, I just don't think it belongs *in* npipe :)
<natefinch> CI blockers are like a hydra... fix one and two more appear
<perrito666> natefinch: what broke now?
<natefinch> perrito666: HA, what else?
<perrito666> ...
<perrito666> could you pass me the search link again?
<natefinch> perrito666: https://bugs.launchpad.net/juju-core/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.importance%3Alist=CRITICAL&field.tag=ci+regression+&field.tags_combinator=ALL
<natefinch> perrito666: you should bookmark it  :)
<natefinch> perrito666: oh yeah, and restore :/
<perrito666> natefinch: I bookmarked it on the other computer sorry
<natefinch> perrito666: heh it's ok :)
<natefinch> perrito666: chrome sync would fix that if you were using it :)
<perrito666> natefinch: I am using chrome, which is odd, I guess when the computer froze during the hangout it caused it not to sync
<natefinch> ah, huh.  Wonder if it was an in-memory cache that didn't get written out
<perrito666> natefinch: dunno, should be ok now
<perrito666> mmm, restore breakage makes no sense
<sinzui> perrito666, I am going to try changing the restore command in the test to use --show-log to get more info
<perrito666> sinzui: thank you
<natefinch> heh, git diff <commit> HEAD  makes a lot more sense than git diff HEAD <commit>  .... the later makes it look like we removed a whole bunch of functionality, stopped using gopkg.in, etc
<sinzui> perrito666, --show-log isn't giving more information. I will add an attempt to get logs from the machine that added
<perrito666> sinzui: --debug perhaps?
<perrito666> if not I can reproduce this test in a moment
<perrito666> I have a doctor appointment but will be back later
<dpb1> Hi -- testing 1.20.4 -- I'm having trouble getting this openstack cloud to bootstrap: http://paste.ubuntu.com/8020244/ -- I suspect that the cloud api endpoint is not getting written out correctly somewhere.  Anyone seen something like this?
<sinzui> perrito666, Somone need to ensure --debug  stops leaking credentials. really! no person can post a log with --debug because the whole work gets the keys to use their account
 * perrito666 agrees with sinzui
<perrito666> sinzui: what are you running the tests with, jenkins or your box?
<perrito666> if its your own box you could just mail me the results, I do have access to these credentials anyway, I think
<perrito666> now fine people if you excuse me, I really, really need to the doc
<perrito666> might be back online from the waiting room
<sinzui> perrito666, I may do. I am trying to create a special log when a failure is detected and the env is still up
<perrito666> I usually do that with the restore test by getting the logs before the destroy in finally, even when the tests dond fail
<sinzui> perrito666, I am trying to capture the new ip of the machine. It is in the string of the error message. I hope to get the log a the moment before destroy is called
<dpb1> n/m, I've sorted it
<dpb1> sinzui: https://bugs.launchpad.net/juju-deployer/+bug/1243827  -- are the juju tools compiled against an old goyaml by any chance?
<dpb1> sinzui: I'm getting something very similar when the tools jujud reads in a yaml with an underscore in the openstack password.
<sinzui> dpb1, The tools are from the debs we build. the debs used the tarball. The tarball pinned the deps to the version specified by juju-core
<dpb1> sinzui: ok.  I guess I'll file a new bug then?  That same symptom seems to be happening with jujud from 1.20.4.  jujud reads in the environment on the bootstrap node, and if the password contains an '_', it gets stripped.
<sinzui> dpb1, both devel and stable are 1 version behind
<sinzui> dpb1, I see the fix was made...but the engineer never updated juju to use it :(
<dpb1> oh dear
<dpb1> :(
<sinzui> dpb1, https://bugs.launchpad.net/juju-deployer/+bug/1243827
<sinzui> ^ the bug wrongly says it is invalid for juju-core.
<dpb1> ohhh
<sinzui> I will correct the bug and target it to 1.20 now
<dpb1> sinzui: thanks!  I'll add our tags so we can track.  appreciate it
<menn0> sinzui: have you had a chance to look at those CI log archiving changes I sent you?
<menn0> sinzui: they could be useful in figuring out the current blockers
<sinzui> menn0, I was looking for them and couldn't find them. I thought you forgot to request a merge
<menn0> sinzui: I don't think I'm allowed to request merges for that project or can I?
<menn0> I sent you the changes in an email
<menn0> as a bzr merge directive
<sinzui> menn0, https://code.launchpad.net/juju-ci-tools is a public project
<menn0> you can bzr pull the changes straight out
<menn0> sinzui: ok. let me try again.
<sinzui> thank you menn0
<menn0> sinzui: https://code.launchpad.net/~menno.smits/juju-ci-tools/log-dumping-improvements/+merge/230398
<menn0> sinzui: I blame my complete inexperience with Launchpad
<sinzui> menn0, hmm, at first glance, this wont help the current bug because the bootstrap host may not exist after the restore was attempted
<menn0> sinzui: yep you're right. dump_env_logs assumes the bootstrap node is available.
<menn0> it should help with the HA blockers though
<sinzui> menn0, okay that means my current branch is still valid. I am searching for the  address of the new instance. I will base my branch on yours though.
<menn0> sinzui: where are you searching for the address?
<sinzui> menn0, the address might be in the output of error that is raised. I can see the "attempting to connect (.*)" lines, so I could use the last one as an address to ssh to
<menn0> sinzui: that could work
<sinzui> menn0, does dump_euca_console work with openstack. that is where the tests can be run
<sinzui> and joyent
<sinzui> and azure
<menn0> sinzui: apart from being in a new function, that code hasn't changed
<menn0> dump_euca_console doesn't do anything unless the host_id is set
<sinzui> menn0, I see. That code is very naive
<menn0> waigani: could the failures at the bottom of bug #1347715 be related to your recent work?
<sinzui> menn0, I merged your branch. I ponder one more change though
<sinzui> menn0, We really shouldn't try to get bootstrap_host (ip) after is was destroyed. that undermines getting all the logs from all the machines.
<sinzui> menn0, I think we should set bootstrap_host to None to encourage your new functions to get everything it can from what ever exists in the env
<waigani> menn0: I've read through bug and back through my possibly related change sets, nothing jumps out. were you thinking of anything in particular?
<menn0> waigani: just that it appears to be SSH related
<menn0> waigani: it might be a completely unrelated problem
<menn0> waigani: so feel free to ignore me
<menn0> sinzui: the idea behind passing in the bootstrap host address is that it allows retrieval of at least some logs even if "juju status" is no longer functioning
<menn0> sinzui: generally tests seem to be remembering the bootstrap host address before doing something drastic
<menn0> sinzui: thinking out loud... perhaps a better approach might be to ask the underlying provider infrastructure for machine addresses if "juju status" fails.
<menn0> sinzui: we could then remove the bootstrap_host arg to dump_env_logs
#juju-dev 2014-08-12
<perrito666> sinzui: news?
<perrito666> ok, something is very very broken with aws
<waigani> axw: do you mean just setting state-server: true in tests where it is set to false?
<axw> waigani: yes, the ones where it matters anyway
<axw> leaking details of the dummy provider into the command is not great
<axw> AFAIK, the state-server thing is just there to speed up the tests
<axw> but if we need it, then the tests should change
<waigani> axw: yeah for sure, it's ugly. I assumed the state-sever was set to false in tests for a reason
<axw> waigani: there's lots of wrong in the code, don't be afraid to question it ;)
<waigani> axw: so you're saying the state-server setting should be deprecated?
<axw> no, just change it to true for the tests where we run the bootstrap command
<axw> alternatively, do the TODO now... not sure which one is less work really
<waigani> sigh
<waigani> kludges are less work ;)
<waigani> I had to look that up
<axw> yes, often less work in the short term
<waigani> and found this image: http://en.wikipedia.org/wiki/Kludge#mediaviewer/File:Miles_Glacier_Bridge,_damage_and_kludge,_1984.jpg
<axw> lol
<waigani> sums it up pretty nicely haha
<waigani> I should put that on my coding profile ;)
 * wwitzel3 flips his desk
<axw> wwitzel3: what'cha doing?
<wwitzel3> trying to get rsyslog rotation for all-machines.log working
<axw> ah, fun times
<wwitzel3> since all day
<wwitzel3> :/
<wwitzel3> I was in bed attempting to go to sleep and I was have weird dreams about rsyslog in human for bullying and taunting me.
<wwitzel3> s/for/form
<wwitzel3> lol
<axw> well, that's not cool...  I don't recall having personified a software package before
<wwitzel3> haha, happens to me all the time
<axw> did it look like this guy? https://plus.google.com/+RainerGerhards
<axw> :)
<wwitzel3> though my favorite dream was I was travel through code and I kept hitting this unhandled exception (which was the same one that was happening in production). And when I woke up I had instant thought of how to fix it and got it resolves in 5 minutes.
<menn0_> axw: bad news... the "State remove setmongopassword" change is what's causing this CI blocker: bug #1355320
<axw> god damnit
<menn0_> axw: I've been bisecting my way through recent changes and that's the one
<axw> thanks menn0_
<axw> I'll take it over if you like
<menn0_> sure
<menn0_> I'll update the ticket with the best repro details I have
<menn0_> give me a few minutes to see if I can simplify them a bit further
<axw> menn0_: thanks
<axw> weird, I'm certain I tested this...
<waigani> menn0_: how do you bisect your way through the changes?
<menn0_> axw: after issuing ensure-availability did you wait for the new state servers to hit "started"?
<axw> menn0_: pretty sure I did, but it was a little while ago and my memory isn't great
<menn0_> I'm seeing the new machine agents getting stuck in pending because they can't connect to the API
<axw> yeah, I see the same thing now
<menn0_> axw: I've updated the ticket and assigned it to you. All yours!
<axw> menn0_: thanks
<menn0_> waigani: the process I used is:
<menn0_> reproduce the problem with master
<menn0_> use git log to see the changes between 1.20.1 and master
<menn0_> git checkout <some rev in between>
<menn0_> git install ./..
<menn0_> try to reproduce the problem
<menn0_> does it exist?
<menn0_> no: the problem revision is after this on
<menn0_> yes: the problem revision is before this one (or it IS this one)
<menn0_> you can do a straight binary search
<menn0_> but if you have some idea of where the problem lies, like in this case, you can be a bit smarter about picking revs to narrow it down more quickly.
<menn0_> keeping good notes on what you've done helps a lot too
<waigani> menn0_: ah right, thanks for the explanation
<menn0_> if the reproduction steps were completely automated (which they can be but I didn't bother) then you could get "git bisect" to do all the work for you.
<menn0_> what i've described is exactly what it does
<menn0_> actually, looking at the docs "git bisect" can also be used to assist with manual searches. I should have used it.
<menn0_> previously I've only ever used "git bisect run" which does automated searching
<waigani> menn0_: yeah I was thinking that should be able to be automated
<menn0_> waigani: the fiddly part in this case is writing some code to wait at the right times and parse the status output
<menn0_> waigani: all doable though and I'm sure the helper functionality in juju-ci-tools would help too.
<waigani> menn0_: setting up a process that could automatically find the offending commit for a CI bug would save us a lot of time
<waigani> potentially reverting it too
<menn0_> I think a semi-automatic process is probably best
<menn0_> the CI tests often fail for infrastructure related reasons
<waigani> right, well it could add suspicious commits to the bug comments
<menn0_> you wouldn't want these to trigger reverts or long running investigations
<menn0_> yes
<stokachu> is there any examples where juju actions are being used in practice?
<menn0_> axw: you associated bug 1347715 with the "don't enter upgrade mode unnecessarily" PR
<axw> menn0_: yep
<menn0_> that PR wasn't intended to solve that bug :)
<axw> menn0_: see the latest error message at the bottom of the bug
<axw> that LP bug has morphed
<axw> the latest one was due to "upgrade in progress"
<menn0_> all that PR does is allow the machine agent workers to start up a little faster, and avoid some unnecessary log noise
<menn0_> I can see how it might help with that bug
<menn0_> but I wonder if that's the whole story
<menn0_> if it is then, great
<stokachu> there doesnt seem to be an action-set either is this feature not complete yet?
<menn0_> but I'm not sure
<axw> stokachu: still a work in progress AFAIK
<stokachu> axw, ok cool
<menn0_> axw: is it worth reverting 3b6da1d429bff627a636ca4512c1ae8230f26539 or do you think you'll have this sorted quickly
 * menn0_ would like to get CI unblocked
<axw> menn0_: I guess it's no big deal to revert it in the mean time
<axw> I'll do that...
<jcw4> stokachu: yes, it's still in-progress
<jcw4> stokachu: bodie_ is primarily working on action-set and action-get
<axw> menn0: reverted
<menn0> axw: yep, saw that. cheers
<menn0> axw: does the bug get marked as "Fix committed" now?
<axw> menn0: yeah I've done that
<axw> I'll keep looking into it tho
<menn0> cool
<menn0> of course
<axw> this is one time where I really wish I hadn't rebased
<axw> pretty sure I tested it and it all worked, then I rebased and it's broken since then
<menn0> axw: so you think it's some interaction with this change and some other change that got rebased in?
<axw> I think so
<menn0> axw: in other news, that manual provider problem is still there despite the "don't enter upgrade mode unnecessarily" change
<axw> sigh
<axw> all good news today :)
 * menn0 reopens the bug
<axw> hold up
<axw> it's still publishing?
<axw> menn0: ^^
<axw> or did you test it manually?
<axw> eh, never mind
<axw> hadn't refreshed
<menn0> http://juju-ci.vapour.ws:8080/job/manual-deploy-trusty-ppc64/523/
<menn0> axw: shall I take 1347715 for a bit?
<axw> menn0: that'd be great, thank you
<menn0> axw: cool
<menn0> I need to go out and pick up some furniture but I'll keep going with it once I'm back
<menn0> axw: I'm back but need to deal with the kids for a bit
<axw> menn0: no worries
<menn0> I'll continue with this manual provider issue later this evening
 * axw hopes it isn't something else he's broken
<axw> menn0: I'm getting "auth fails" even without my change
<axw> :/
<axw> hrm, 2nd shot worked
 * axw scratches head
<mattyw> morning all
<TheMue> heya
<dimitern> morning
<voidspace> TheMue: morning
<voidspace> dimitern: morning
<menn0> axw: I'm back again (kids were a nightmare to get to bed)
<axw> menn0: heya. fun :(
<axw> menn0: so I've reworked my previous PR and got something working now
<axw> ignore my comment from before, I think I hadn't uploaded the right tools
<menn0> axw: ok that's good to hear
<menn0> axw: let me know if you need a review
<axw> the problem with the old one was that mongo.SetAdminMongoPassword was also doing Login straight after
<dimitern> so is anyone working on the blocker bug https://bugs.launchpad.net/juju-core/+bug/1355324 ?
<axw> menn0: it can wait, it's well after EOD there...
<axw> not I
<menn0> dimitern: no I don't think so
<menn0> other blockers have been keeping us busy
<dimitern> menn0, how about the other one assigned to you?
<voidspace> launchpad won't even show me the bug
<dimitern> voidspace, oh, which one?
<voidspace> I don't think it's my internet  being rubbish this time
<voidspace> 1355324
<voidspace> page won't load
<dimitern> voidspace, :( aw
<voidspace> just launchpad being slow I think
<menn0> voidspace: weird. it's working for me
<voidspace> odd
<menn0> I'm going to look at #1347715 for a bit longer
<menn0> but someone else might have to take over depending how far I get with it
<mattyw> voidspace, I'm having trouble with lp, canonical.com and ubuntu.com this morning
<mattyw> voidspace, you on bt?
<voidspace> mattyw: over here in Romania? I doubt it :-)
<voidspace> mattyw: don't know who the isp is to be honest
<voidspace> but today most of the internet works well, which is better than previous days, *except* for launchpad
<mattyw> voidspace, cool - how long you there for?
<voidspace> I'll try canonical.com and ubuntu.com as well
<voidspace> mattyw: till the end of August
<voidspace> mattyw: visiting wife's family
<mattyw> voidspace, nice, hope you have a great time
<voidspace> canonical.com loads fine
<voidspace> mattyw: thanks
<voidspace> mattyw: I'm mostly working
<voidspace> mattyw: we're taking a week at the end of August to visit seaside and mountains though
<voidspace> should be good
<voidspace> yep, launchpad not loading for me at all at the moment
<voidspace> tried two browsers
<voidspace> I wonder if it's a dns issue, I can't ping it either
<mattyw> voidspace, I was able to get to canonical.com by changing dns servers but everything else is down for me
<menn0> axw: well this looks like a problem. using the manual provider:
<menn0> 2014-08-12 09:36:54 INFO juju.mongo open.go:104 dialled mongo successfully on address "127.0.0.1:37017"
<menn0> 2014-08-12 09:36:54 DEBUG juju.worker.logger logger.go:45 reconfiguring logging from "<root>=DEBUG" to "<root>=WARNING;unit=DEBUG"
<menn0> 2014-08-12 09:36:55 ERROR juju.worker runner.go:219 exited "machiner": machine-0 failed to set status started: cannot get machine 0: EOF
<axw> menn0: yep, but what would be causing the EOF?
<menn0> that's the last thing in the machine-0 log following bootstrap
<axw> I saw it, but it's not a very enlightening error message ;)
<menn0> I'm going to crank up the log level during bootstrap and see what I can see
<axw> cool
<menn0> the error happens after the root logger goes to WARNING
<axw> ah right
<axw> yeah, I just set my environments' logging-config="<root>=DEBUG"
<menn0> and I might have to do the bisect thing again to see if I can track down the rev
<menn0> this one has been broken for a while so there could be a lot of revs...
<mattyw> folks - is someone able to help me out with a couple of things around the unit type?
<mattyw> ^^ it appears (at least in some tests) that the CharmURL in a unit doesn't have a value (== nil)
<menn0> axw: alright
<menn0> now I have a nice long panic
<axw> menn0: cool
<axw> I have to run
<perrito666> morning
<menn0> axw: no problems
<menn0> I need to stop soon too
<axw> menn0: if you get sick of looking at that, can you please attach the panic to the bug and I'll take a look tomorrow
<menn0> axw: I'll try to get someone else to look in the mean time too
<menn0> but I will add everything I have to the bug either way
<axw> cool
<TheMue> dimitern: a cable technician just came, could be that Iâm a bit later at our hangout
<dimitern> TheMue, ok, np
<voidspace> dimitern: are we postponing?
<dimitern> voidspace, let's wait for TheMue a few minutes
<voidspace> dimitern: sure
<TheMue> dimitern: Iâm there, are you done?
<voidspace> TheMue: we didn't start
<menn0> right... I'm done
<menn0> who wants to take over this CI blocker: 1347715
<menn0> I've done a lot of the legwork. There's repro instructions and other findings attached to the ticket.
<menn0> I'm pretty close but I haven't quite nailed the actual bug yet.
<menn0> no takers regarding the above?
<tasdomas> could somebody take a look at https://github.com/juju/juju/pull/490 ?
<mgz> menn0: I read through the bug, did you narrow down when it was introduced, or not get that far yet?
<menn0> mgz: well as per my last comment, it's definitely since 1.20.2
<menn0> but I haven't narrowed it further than that
<menn0> given that we have a fair simple repro, git bisect could help?
<mgz> yeah, just a bit slow, right, as it involves reprovisioning a machine?
<mgz> or can you do it on a dirty one?
<menn0> focussing on changes that involved the manual provider might also help narrow things down more quickly
<menn0> no you don't need to reprovision
<menn0> because it's the manual provider
<menn0> just make sure you do a juju destroy-environment --force --yes after each run
<menn0> that's pretty good at getting rid of most of the previous run
<menn0> it's not exactly fast (the bootstrap still takes a while) but it's manageable
<menn0> limiting the upload series to just trusty also helps a bit
<menn0> it's midnight. I really need to go to bed
<mgz> menn0: okay, good night
<menn0> have a good one
<fwereade> tasdomas, LGTM
<tasdomas> fwereade, thanks
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs 1347715, 1355324
<fwereade> natefinch, ping -- when you have a moment I'd like to chat about transactions
<natefinch> fwereade: how about now? :)
<fwereade> natefinch, perfect :)
<fwereade> natefinch, in irc for a bit in case anyone else sees something ineresting go past?
<natefinch> fwereade:  sure
<fwereade> natefinch, basically I'm worried about interactions between the two systems as we switch
<natefinch> fwereade: yep
<fwereade> natefinch, (oh wait, it might actually not be a problem, let's keep going for a bit)
<fwereade> natefinch, in particular, assertions
<fwereade> natefinch, now I think that we will write to every affected document --including those we just assert on -- as part of a transaction
<fwereade> natefinch, if that's the case I think we're good
<fwereade> natefinch, the problem isn't at crossover time
<fwereade> natefinch, it's with using toku on its own
<fwereade> natefinch, in particular, assertions about particular document states
<fwereade> natefinch, I think that within any transaction we need to write to any document on whose state we depend
<natefinch> fwereade: I haven't actually done a lot of research into how toku transactions work.... do you not get a consistent view of the database once you enter a transaction?  Or only the ones that you write to?
<fwereade> natefinch, and that, best-case, this is going to lead to frequent document deadlocks
<fwereade> natefinch, consistent state doesn't help though, I think
<natefinch> ahh right, if someone else changes the doc out from under you
<fwereade> natefinch, it's about needing all that state to still satisfy certain conditions when the txn lands
<natefinch> right
<fwereade> natefinch, writing to the document will help there, it pulls it into the transaction if you like
<fwereade> natefinch, but I think it *seriously* increases the already-present risks of deadlock in the DB
<fwereade> natefinch, I read docs X and Y, you read Y and X
<natefinch> right
<natefinch> classic deadlock
<natefinch> fwereade: have you read up on their transactions?  I don't feel qualified to talk about them without doing some reading.  I'd just been doing some testing to make sure it was even possible to use their DB.... but hadn't actually researched the way their transactions worked.  hazmat had done the work there, I believe.
<fwereade> natefinch, I've read like 1 small white paper
 * natefinch is reading the white paper now
<fwereade> natefinch, so anyway -- I think it will work fine with mgo txns underneath, for a given value of fine, if not actually very performant
<fwereade> natefinch, although
<hazmat> fwereade, its mvcc fwiw
<hazmat> consistent reads, so a simple query satisifes mgo conditions
<hazmat> fwereade, more of concern is the use of write hotspots in the current codebase
<fwereade> hazmat, are you thinking more of "presence" or of things like service documents?
<hazmat> fwereade, service docs
<fwereade> hazmat, right, I think those things are a nexus of fuckery in either txn system
<hazmat> fwereade, they exist due to async gc ? or..
<hazmat> fwereade, latent gc can use simple queries afaics.. i tended to regard them as offshots of the extant txn sys in conjuction w/  lifecycle management
<fwereade> hazmat, well service documents get cleaned up with the last unit to be removed from a dying service
<fwereade> hazmat, but that's managed by a refcount on the service document
<hazmat> fwereade, why not just a async gc cleaner job in the state server?
<Beret> wallyworld, any update on that flag to prevent a failed bootstrap from tearing itself down?
<hazmat> fwereade, doesn't that mean machine loss of last unit leaves to stuck service otherwise?
<hazmat> minus using destroy-machine --force to try and post clean up after the fact from the api server
<fwereade> hazmat, no? when a unit gets removed either by the machine agent or by forced removal of the machine, the last one cleans up anything that depends on it
<fwereade> hazmat, or indeed "yes" depending on perspective
<natefinch> Beret: we've been fighting some critical bugs lately, and most of the team leads were on a sprint last week, so it hasn't gotten done yet, AFAIK.  But I think it could be done this week if we have someone free to work on it.
<Beret> natefinch, ok, thanks
<natefinch> Beret: we'll try to get it in... it's something we want for ourselves, too.
<Beret> natefinch, it's not more important than real bugs, I just wanted to make sure it hadn't gotten in and we just didn't know it
<fwereade> hazmat, to step back a moment
 * hazmat nods
<hazmat> meeting.. bbiab
<fwereade> hazmat, in either txn system, our performance is largely dependent on the areas of document space touched by a given txn -- thus designing db operations that minimise overlap is important either way
<hazmat> fwereade, agreed
<fwereade> hazmat, we have occasionally done a really poor job of designing these things in the past
<fwereade> hazmat, but now we have schema changes we have a path to mitigate these, and we can expect to see benefits from doing this in *either* system
<fwereade> hazmat, however, we won't be able to eliminate overlaps
<fwereade> hazmat, our job is structuring things such that overlaps are (1) rare and (2) not too disruptive
<fwereade> hazmat, and I am at the moment mainly thinking about the impact on how we actually have to write code
<fwereade> hazmat, natefinch: so if we wrap a mgo txn in a toku txn...
<fwereade> hazmat, natefinch: let's pretend for a moment we get all our info from a consistent snapshot, and craft a txn assuming the truth of that stuff, and then we execute that txn
<fwereade> hazmat, natefinch: only when we execute that txn do we hit the docs and grab the locks, and if we deadlock at that point we may have trouble
<fwereade> hazmat, natefinch: if the txn runner fails out for this reason, what exactly is the impact on the code trying to craft the txn and retry in the face of contention?
<fwereade> hazmat, go back to the beginning of the whole set of operations and start crafting txns again from scratch?
<fwereade> hazmat, in particular I fear *that* is going to be a major source of gray goo transactions that end up locking large chunks of the DB, deadlocking, failing out, and doing the same thing over and over again
<fwereade> natefinch, sorry, I am also interested in your thoughts on the above
<natefinch> hah ok
<natefinch> I was going to give my thoughts anyway ;)
<fwereade> natefinch, just realised I'd stopped badging you at some point
<fwereade> (but that doesn't mean I'm going to stop badgering you, ho ho)
 * fwereade looks shamefaced
<natefinch> haha
<natefinch> fwereade: I gotta run in a couple minutes unfortunately..... but... do we need mgo transactions if we have toku transactions?  Can we just stop using mgo transactions?
<fwereade> natefinch, yes, *but* that's a lot of stuff to rewrite rather than wrap
<natefinch> right
<fwereade> natefinch, there will certainly be things that are easier to do in a pure-toku world
<natefinch> yep
<fwereade> natefinch, I'm just worried that mixing the two will actually have a pathologically awful impact
<natefinch> I gotta run, sorry, I'll keep thinking and will read the channel history.. Back in an hour-ish
<fwereade> natefinch, cheers, take care
<hazmat> fwereade, per the acid doc, we could divert mgo transactions to toku
<fwereade> hazmat, I can sort of see it but it's a bit fuzzy in my mind
<fwereade> hazmat, we still have to do what I said, I think?
<fwereade> hazmat, we can be a bit more proactive about writing to docs to take their locks a bit earlier than we could otherwise
<fwereade> hazmat, and I daresay we can convert failure to do so into ErrRefresh, and make all the transactions handle it?
<fwereade> hazmat, but wouldn't we have to actually start a fresh toku txn *anyway*?
<fwereade> hazmat, so in a multi-step toku operation, if the Nth step fails we back *everything* out and start again?
<fwereade> hazmat, heh, maybe even track and try to grab locks on everything we thought we needed last time through?
<fwereade> hazmat, it all feels potentially *really* yucky
<fwereade> natefinch, more blithering above ^^
<hazmat> fwereade, sorry still in meeting, bbiab
<fwereade> hazmat, np, I'm around for a bit I think
<natefinch> fwereade: back
<fwereade> natefinch, I need to be off soon, but read back and see if anything resonates or inspires a response
<fwereade> natefinch, just type at me in here if I'm gone, I'll see it soon
<hazmat> fwereade, sorry my meeting ran over time.. and into another meeting.. i'll capture notes for discussion tomorrow if your not around in a bit
<hazmat> fwereade, nutshell locks are implicit with mods to a doc in a txn.. basically we just take a op runner and do it in toku txn (begin, end txn) with catch on lock and retry behavior.
<natefinch> fwereade: no problem.  Trying to do a few things at once here.   I don't know that grabbing locks early has any effect in toku... if two threads are both trying to grab locks early, it doesn't really buy us much.  My preference would be to just make the code straightforward and do what it's supposed to do, and if something else gets into the DB first, the TX will have to retry.
<hazmat> fwereade, re backout, its implicit at that commit, failed lock is rollback, and error to app to handle, at which point we retry similiar to runner loop now
<hazmat> there is no grabbing a lock, its implied by writing to a doc during a txn.
<fwereade> hazmat, natefinch: ok, this sounds broadly sane, I am worried that we have some funky layering issues to work around wrt actually managing rollbacks
<natefinch> that's certainly possible
<fwereade> hazmat, natefinch: well I am contending that we do need document locks for those docs that in mgo/txn we would assert on
<fwereade> hazmat, natefinch: but that we will have to lock them ourselves by, as you say, writing to them in a transaction
<hazmat> fwereade, we don't because we're operating in a mvcc world with implicit write level locks.
<fwereade> hazmat, right
<hazmat> ie read committed, or serialized if preferred (both options avail)
<fwereade> hazmat, which is great for ensuring that conditions hold for the documents we're writing
<hazmat> fwereade, right.. toku should be a nice runner for the extant mgo transactions since conditions are explicitly specified and re-runnable
<natefinch> fwereade: right... my reading of MVCC is... you don't have to worry about basing your behavior on reads that get out of date, because if they get out of date, one of the two transactions aborts and retries
<fwereade> hazmat, natefinch: so once you start a transaction, toku tracks everything you read? and aborts your txn if someone writes to one of those docs?
<fwereade> hazmat, natefinch: that's the only way I could see it aborting a transaction in that situation
<hazmat> fwereade, again depends on isolation level.. read serialized you'll read the copy that was current at the time of txn begin
<hazmat> fwereade, read committed, means you read other docs current as of the time of read
<hazmat> as opposed to our read dirty model now.. ie. read partially committed state
<fwereade> hazmat, natefinch: I don't see how either read serialized *or* read committed actually helps us -- I'm not interested in either of those two states
<hazmat> rephrasing .. current == latest commmitted revision of the doc
<hazmat> fwereade, their rather important .. but perhaps we should take a step back.. what's the concern?
<fwereade> hazmat, if I have a txn that should only go ahead if a property continues to hold for some doc not in the txn
<fwereade> hazmat, then it's philosophically impossible for me to read a document in either of those situations and be certain that it can't change under me
<fwereade> hazmat, but if I *write* to that document I can
<fwereade> hazmat, because I take a lock
<fwereade> hazmat, and guarantee the failure of either myself or the other txn that wanted to write it
<fwereade> hazmat, (sucks if that one just wanted to read it too, but anyway)
<fwereade> hazmat, making sense?
<hazmat> fwereade, serializable would give you that semantic
<fwereade> hazmat, ok, so that does take locks on read?
<fwereade> hazmat, (hopefully happy smart read locks not nasty actually-triggering-serialization write locks?)
<hazmat> fwereade, it does.. but it may not cover every use case.. ie inserting new doc in collection.. since your asserting a negative read
<fwereade> hazmat, yeah, we depend on that a bit
 * hazmat attention is gripped by tosca meeting
<fwereade> hazmat, natefinch: ok, modulo d- asserts, we sound like we'll be fine with the serializable level
 * fwereade needs to go out anyway
<fwereade> thanks for clearing that up, I didn't get that impression from the descriptions
<fwereade> I guess we need to be careful about what we *do* read when we're in a txn then..?
<natefinch> fwereade: well, generally you don't just read stuff for no reason.  You read stuff because the logic depends on the contents
<fwereade> natefinch, ok, but sometimes you read an easy-to-express superset of what you need and extract in code
<hazmat> fwereade, it maybe we need to do separate collection level locks for insert
<natefinch> it's funny, Toku has an office in Lexington.. I could like, drop down there and say hi
<fwereade> natefinch, if that's going to quietly take a bunch of locks we should be aware of it
<natefinch> fwereade: that's true
<hazmat> fwereade, i'd like to define the common scenarios and patterns we have
<fwereade> hazmat, I can live with that, I'm just fretting because I want to be sure we've got answers for all these things ;)
<hazmat> and then verify solutions.. common idioms for them with tokumx
<fwereade> hazmat, that would probably be the smart thing to do, indeed -- natefinch, are you ok to try to build those up?
<fwereade> natefinch, would be happy to discuss in more detail
<fwereade> natefinch, ...but not now
 * fwereade disappears in a puff of smoke
<hazmat> fwereade, natefinch should i setup a call for tomorrow?
<natefinch> fwereade: see ya
<natefinch> fwereade: yeah
<hazmat> fwereade, cheers
<hazmat> done
<jcw4> mgz, cmars fix for lp-1355521 https://github.com/juju/juju/pull/500 , ptal
<mgz> jcw4: you can jfdi that through if you want
<ericsnow> natefinch: one-on-one?
<jcw4> mgz: tx
<jcw4> mgz tx again :)
<mgz> :)
<natefinch> ericsnow: sorry, coming
<wwitzel3> woohoo! I'm down to permission issues
<wwitzel3> victory is mine!
<natefinch> wwitzel3: nice!
<perrito666> sinzui: I would prett much guess that the cause for restore to no longer finish is https://github.com/juju/juju/commit/55a9507924dea63658598361797ec864b9879e84#diff-d41d8cd98f00b204e9800998ecf8427e
<wwitzel3> natefinch: yeah, the outchannel command cannot take arguments, so you have to do it as a script. also if outchannel encounters a non-zero exit code from the script, it assumes the channel is bad and stops sending to it. Until you restart/reload.
<natefinch> huh ok, so your script wasn't returning 0 I guess?
<wwitzel3> natefinch: and lastly, the logrotate conf itself, must have the right set of permissions.
<wwitzel3> natefinch: right, it wasn't because of a permission issue, which was just being thrown away
<natefinch> ahh
<sinzui> perrito666, I just got permission to merge my log recovery changes. The next runs of the recovery tests will try to get logs from that machine that restore created
<sinzui> perrito666, yes, the commit does look like it relates to the vague error in the jenkins console
<wwitzel3> natefinch: so it had 3 issues, but I just had to discover them in the right order.
<wwitzel3> natefinch: for example, at first I was calling logrotate as my command and passing it arguments ... logrotate was being called and returning 0 so logging continued, but the actual conf wasn't being passed.
<perrito666> voidspace: care to shed some light on the commit?
<perrito666> it has your name
<wwitzel3> natefinch: then finally burried in the documentation for outchannel I found an small italic note about the fact outchannel command can't take any arguments.
<wwitzel3> oh and the issue with logrotates default state file path, that was easy though once I was actually able to start getting debug output from the rsyslog outchannel executing the command.
<natefinch> yeesh
<natefinch> I think you should write all that up and put it on juju-dev, so that someone else has a chance of understanding everything later.... and actually, a big comment in the code about it wouldn't hurt either.
<wwitzel3> natefinch: so the rsyslog cert and key, live in the log folder even though they aren't logs.
<natefinch> or in the docs somewhere... something
<natefinch> wwitzel3: nice
<wwitzel3> natefinch: so I assume it is ok for the logrotate.conf to live there too?
<wwitzel3> natefinch: also the logrotate helper script that runs logrotate with the juju conf.
<wwitzel3> natefinch: yeah I was planning to document the behavior of outchannel and ref the docs as well as the permission requirements for the logrotate.conf and state file.
<wwitzel3> natefinch: since I also need to document the existence of all-machine.log.1 and the max size, etc..
<natefinch> ahh yep, definitely
<natefinch> and yeah, we can put more stuff in there if there's already non-logs in there
<tych0> h
<hazmat> wwitzel3, you get your rsyslog issues resolved?
<wwitzel3> hazmat: yep, thanks :)
 * sinzui kicks CI to test ha and restore now
<natefinch> sinzui: I have a possible fix for the other bug too
<natefinch> sinzui: https://github.com/juju/juju/pull/501
<natefinch> perrito666, ericsnow, wwitzel3: one of you want to review? ^^  really simple change with a test and everything.  I don't actually know why the code worked before and then suddenly stopped working, but at the very least this is a change that won't break anything and has a test to make sure it continues working.
<natefinch> super simple change
<ericsnow> natefinch: sure
<natefinch> test verified to panic on the old code, just like in comment #18 here: https://bugs.launchpad.net/juju-core/+bug/1347715
<natefinch> (and test verified not to panic with the new code)
<natefinch> aside:  I wish the comments on launchpad bugs were anchors, so I could do <bug-url>#comment-18 and have it jump directly to the comment. Instead the comment numbers just open up a separate window with no context which is completely useless.
<thumper> morning folks
<thumper> sinzui: you around?
<sinzui> I am
<thumper> sinzui: what's the tl;dr on CI status?
<sinzui> thedac, 2 blockers, natefinch's fix for 1 is queue to play in the next hour
<natefinch> thumper: here's the fix... though I admittedly don't know why it worked before: https://github.com/juju/juju/pull/501
<sinzui> thumper, my effort to get more logs from the failed restore tests also failed. no new data
 * thumper takes a quick look
 * sinzui wishes --debug didn't leak certs, keys, users, and passwords
<natefinch> sinzui: me too
<natefinch> I gotta run
<natefinch> good luck everyone, and good night
<thumper> sinzui: so the remaining CI failure is a restore one?
<sinzui> thumper, yes https://bugs.launchpad.net/juju-core/+bug/1355324
<sinzui> thumper, the test is playing right now. I hope to ssh in at the right moment and cat the logs
<thumper> kk
<waigani> thumper: welcome back!
<menn0> thumper, waigani: good morning
<thumper> o/
<waigani> morning :)
 * thumper headdesks
<sinzui> perrito666, I added logs to bug 1355324. Tomorrow I am going to investigate redirecting all stderr to a private location on disk to try --debug
<perrito666> sinzui: the rev I pointed is the culprit, it appears that we no longer have mongo listening on StatePort and therefore restore can not connect to It I need to get voidspace or fwereade to find out more about it
<thumper> perrito666: why isn't mongo listening?
<sinzui> perrito666, okay. that right, I have too much going on to keep my work list clear
<perrito666> thumper: I dont think its not listening, we are no long publicizing the port I believe
<perrito666> thumper: https://github.com/juju/juju/pull/449/files
<sinzui> thumper, we agreed some months ago that nothing is allowed to talk to mongo directly...but restore does
<ericsnow> perrito666: from the PR it sounds like it's not even listening on external ports
<perrito666> ericsnow: I really need to dig more into this otherwise I am talking in educated guesses
<thumper> ah...
<thumper> didn't I see a change where mongo only listened internally?
<ericsnow> perrito666: https://github.com/juju/juju/pull/449#issuecomment-50825367
<thumper> like only on localhost?
<ericsnow> perrito666: "I've also confirmed that with current master I can telnet to port 37017, and that with this branch I can't."
<perrito666> not even a day running ubuntu on this machine and I already have unity behaving oddly.. that must be a record for a fresh install
<thumper> ericsnow: that's the one
<perrito666> thumper: sounds to me that not even, which puzzles me a bit unlessss
<perrito666> api is using direct connection
<perrito666> I dont know if direct is the right name
<ericsnow> perrito666: yeah, that's what I find weird
<ericsnow> thumper: yep, that's the PR for the changeset that broke restore
<perrito666> thinkpads ability to swap fn/ctrl on the bios is marvelous
<thumper> ericsnow, perrito666: simplest thing IMO is to revert PR 449
<thumper> and take a fresh look later at closing the port
<thumper> better than trying to work out now how to selectively open it
<perrito666> I guess there is no other option, also by doing this I can get the original author to grep again for the use of StatePort :p
<ericsnow> perrito666: yeah, tell him about that grep thing ;)
<ericsnow> thumper, perrito666: +1 on reverting
<ericsnow> (too bad another blocker will show up before I have a chance to merge anything tomorrow <wink>)
<perrito666> ok, let me have a late evening snack and Ill propose the revert
<perrito666> btw, did I mention I have the eating habits of a hobbit?
<perrito666> so, to revert we do a reverse pr or use the "sorry I screwed up, please undo" feature from github
<thumper> perrito666: um... there is a github feature?
<thumper> perrito666: I'd have just done a reverse PR
 * thumper goes to make a coffee
<perrito666> thumper: I saw github offering me to undo with a button the other day
<thumper> waigani: standup hangout time
<perrito666> how do I reference a ticket/pr on a pr description
<perrito666> ?
<perrito666> sinzui: or anyone
<perrito666> the $$fixes tag is to be added into the $$merge or in the pr description?
<sinzui> perrito666, fixes-nnnnn
<sinzui> perrito666, in a comment by itself or with other text
<perrito666> sinzui: yup, but, that is to be added into the pr body or into the merge comment?
<perrito666> my question is, will that trigger the merge?
<sinzui> merge comment perrito666
<sinzui> perrito666, it doesn't trigger a merge.
<sinzui> perrito666, $$merge$$ to trigger the merge and fixes-nnnnnn ti explain why the merge is valid
<perrito666> tx sinzui
<perrito666> ok anyone ptal https://github.com/juju/juju/pull/503
<ericsnow> I'm signing off, but if anyone is able, ptal @ https://github.com/juju/utils/pull/16 https://github.com/juju/utils/pull/19 https://github.com/juju/juju/pull/462 https://github.com/juju/juju/pull/453
<sinzui> perrito666, I just discovered the first comment is not a comment according to github api. I can see ericsnow is the first comment, and my test comment is the second
<perrito666> sinzui: that is why I did not add the $$ fixes part, I will add it in the merge comment
<sinzui> perrito666, okay. I am a little disappoint because I think it is nice to state in the first comment why the branch should be merged :/
<menn0> testing... please ignore: bug 1347715
<mup> Bug #1347715: Manual provider does not respond after bootstrap <bootstrap> <ci> <regression> <juju-core:In Progress by natefinch> <https://launchpad.net/bugs/1347715>
<menn0> \o/ (fixed!)
<perrito666> thumper: that is not going to work
<perrito666> you did not add the $$fixes-###$$
#juju-dev 2014-08-13
<axw> menn0: hey, you're not looking at the manual provider bug atm are you?
<axw> I can see the problem now - just added a comment to the bug
<axw> oh, I see natefinch has a PR up...
<waigani> axw: I was going to see if reverting my 'manual provision with custom ssh key' branch fixes it. Should I keep on or have you got it?
<axw> waigani: I can repro in my env now, so leave it with me for now
<axw> waigani: I'm pretty sure it was failing before your change went in
<axw> will see tho
<waigani> axw: okey dokey
<axw> waigani: nothing to do with your change, it's related to removal of storage
<waigani> axw: okay, thanks
 * thumper stares at this code trying to work out what is different
<menn0> axw: nate's change went in
<menn0> with it I can happily bootstrap using the manual provider
<menn0> the CI test is still failing, but in a different way
<menn0> the SSH key error that waigani was referring to
<menn0> axw: to answer your original question, no I'm not looking at this bug any more
<axw> menn0: thanks, no problems; I found the issue
<axw> https://github.com/juju/juju/pull/504  <- review please someone
<axw> fixes CI blocking bug
<menn0> axw: looking
<axw> menn0: agreed about using use-sshstorage. do you have any ideas of what else I can use there?
<menn0> axw: not off the top of my head
<menn0> axw: I'm concerned that if change how SSH storage is used, this code is going to break
<menn0> (e.g. if we stop using it all together, or start using it for the bootstrap node)
<axw> menn0: I understand and agree, but at this point I don't think there's an alternative
<axw> probably we should have a provider independent way of determining whether you're running from inside the env
<menn0> axw: I'll trust your judgment on that
<axw> something like use-sshstorage, but for this purpose
<menn0> could we have some tests that ensure that SSH storage is used in the current way?
<axw> there are I'm pretty sure
<axw> I just added one, too
<axw> i.e. that verification is elided if use-sshstorage=false
<menn0> I'm thinking something right next to these tests that emit a message if they fail to remind us that this code needs to be updated
<menn0> I saw your test and that's obviously required
<menn0> but I'm also wondering if it's possible to have something that checks useSSHStorage on a bootstrap node, and not and ensures it's what we expect here
<menn0> if it fails then it should error with something like: u"seSSHStorage semantics have changed. Please update manualEnviron.StateServerInstances"
<menn0> maybe that's overkill
<menn0> or too hard
<menn0> but that's the kind of thing I'd aim for
<axw> menn0: there's also tests in provider_test.go that check that Prepare sets use-sshstorage, and Open doesn't. there should be one for Bootstrap too, I'll add one
<menn0> ok sounds good
<axw> tho testing Bootstrap may be a PITA, will see...
<menn0> axw: if it's going to be too hard then leave it
<menn0> it's probably more important to get this fix in at this point
<axw> menn0: shouldn't take long I think, I'll see how I go
<axw> won't waste too much time on it
<menn0> sweet
<menn0> well you have my LGTM anyway
<menn0> axw: just remembered... not sure if you need someone else's too. I'm a "junior reviewer". thumper?
 * thumper sighs...
<menn0> :)
<thumper> I should sort that shit out
 * thumper looks
<menn0> thumper: at least this one is a small change
 * thumper needs to take kid to hockey
<thumper> bbl
<jcw4> thanks axw
<axw> jcw4: nps
<jcw4> is there a publicly accessible repo with the funcitonal tests used by jenkins?
<jcw4> s/funcitonal/functional/
<jcw4> ah, I'm guessing it's https://code.launchpad.net/juju-ci-tools
<ericsnow> if anyone has some time, I'd really appreciate a review:  https://github.com/juju/utils/pull/16 https://github.com/juju/utils/pull/19 https://github.com/juju/juju/pull/462 https://github.com/juju/juju/pull/453
<voidspace> morning all
<ericsnow> :)
<voidspace> ericsnow: a little collection there!
<voidspace> ericsnow: morning
<ericsnow> voidspace: noone wants to review them :(
<voidspace> ericsnow: hehe, let me get coffee and I'll take a look
<ericsnow> voidspace: FYI, 55a9507 (drop direct mongo access) got reverted because it broken restore
<voidspace> ericsnow: restore needs direct mongo access?
<voidspace> that's horrible
<ericsnow> voidspace: apparently
<ericsnow> voidspace: for now (the new restore won't)
<ericsnow> and with that, I'm going to bed!
<voidspace> ericsnow: goodnight!
<dimitern> morning
<voidspace> dimitern: morning
<voidspace> dimitern: so "shutting off direct db access" got reverted
<voidspace> dimitern: as it was this change that broke restore :-(
<voidspace> dimitern: I thought restore used ssh rather than direct mongo access, but it seems I'm wrong
<dimitern> voidspace, oh, bugger :(
<dimitern> voidspace, I think restore needs to be smarter
<dimitern> voidspace, and use ssh to run mongo commands remotely
<voidspace> dimitern: right
<voidspace> dimitern: but restore is being changed anyway, so the "new restore" will be smarter
<voidspace> but until then...
<dimitern> yeah..
<TheMue> morning
<dimitern> morning TheMue
<voidspac_> TheMue: morning
<voidspac_> does anyone know the lxc-create magic invocation to get it to share home directory with the host?
<dimitern> voidspac_, why do you need this?
<voidspac_> dimitern: especially for nested lxc containers it makes experimenting simpler
<voidspac_> dimitern: shared access to scripts / ssh keys etc
<voidspac_> dimitern: only for experimentation
<dimitern> voidspac_, you can take a look at man lxc.container.conf - there is a way to specify additional mount points there; or just ask stgraber or hallyn in #server (@can)
<voidspac_> dimitern: there's a u1 development wiki page that explains it somewhere, I'm looking now
<voidspac_> it's how we used to do dev (inside an lxc container)
<dimitern> voidspac_, ah, nice
<voidspac_> very useful, if you screw up your dev environment just blow it away and create a new one
<dimitern> voidspac_, take a look at https://wiki.debian.org/LXC - "Bind mounts inside the container" section
<voidspac_> dimitern: thanks
<voidspac_> dimitern: https://wiki.canonical.com/UbuntuOne/Developer/LXC
<voidspac_> dimitern: sudo lxc-create -t ubuntu -n u1-precise -- -r precise -a i386 -b $USER
<voidspac_> obviously modifying appropriately for trusty / amd64
<voidspac_> but it's the -b $USER
<dimitern> voidspac_, ah, even nicer, thanks! :)
<voidspac_> then start the container as a daemon, ssh in and do your dev work there
<dimitern> voidspac_, so you can still mess up your /home from inside the container, but nothing else?
<voidspac_> dimitern: right
<voidspac_> dimitern: and you can have separate ppas and packages installed
 * TheMue just wondered where his blocked PR is and then recognized that the bot merged it half an hour ago :)
<fwereade> dammit, I thought it was quiet -- irc client wasn't actually running :/
<TheMue> *rofl*
<TheMue> morning fwereade
<axw> fwereade: I thought you were just hiding :)
<fwereade> axw, if I'd realised I was I would have been coding ;p
<menn0> hello peoples
<axw> :p
 * menn0 is back for more
<fwereade> menn0, everybody, heyhey :)
<voidspac_> fwereade: morning :-)
<voidspac_> menn0: morning
<rogpeppe> wallyworld: just checking: have you seen the issues that i raised on juju/blobstore? i was wondering what your thoughts were there.
<wallyworld> rogpeppe: no, haven't seen them, have been focused on 1.20 issues and the sprint last week. i'm back at work tomorrow. do you have bug numbers?
<rogpeppe> wallyworld: https://github.com/juju/blobstore/issues
<wallyworld> rogpeppe: oh, ok. we should be using launchpad for rasing bugs
<wallyworld> otherwise i don't see them
<rogpeppe> wallyworld: ah
<wallyworld> and we can't track them into milestones
<rogpeppe> wallyworld: i thought it was more appropriate to raise the issues against the repo itself
<rogpeppe> wallyworld: but i see the milestone issue too
<wallyworld> for sub repos, that may be a good point
<wallyworld> but, yeah, it's sort of messed up using two tools
<wallyworld> 9 issues :-(
<wallyworld> i may not get to look in detail till a bit later this week
<rogpeppe> wallyworld: (there's no way of tracking external bugs in lp?)
<rogpeppe> wallyworld: if you starred juju/blobstore, i think you might get email messages about issues etc with it. (but maybe that's another setting)
<rogpeppe> wallyworld: some are harder to fix than others
<wallyworld> rogpeppe: there is a way of importing bugs yes; it would need to be set up. but since we use lp for juju-core, i'm conflicted about introducing a separate  tools for other things
<wallyworld> i may have got emails buried in  my in box, will need to check - i didn't have filters set up
<rogpeppe> wallyworld: the other side of that coin is that if you're looking at a sub-repo, it makes sense to be able to trivially see all the bugs associated with it
<rogpeppe> wallyworld: but i'm happy to move the bugs to lp if you think that's better.
<wallyworld> rogpeppe: yeah, i would prefer other affected people help make that call, not just me. i suspect the answer will be juju-core stays in lp and the sub repos are in github since they are just libraries and don't have a release schedule as such
<wallyworld> Beret: hi, sorry i only just saw your ping in the back scroll (I've been off on leave for a few days). that work hasn't started yet but i hope to have something done by end of week for Juju 1.21 alpha
<gsamfira> hello folks. Anyone care to review: https://github.com/juju/juju/pull/499 ? :)
<mattyw> fwereade_, ping?
<fwereade_> mattyw, pong
 * dimitern lunches
<perrito666> voidspac_: r u there?
<hazmat> fwereade_, if you can't make the txn meeting.. i'd prefer we just push it one day or alternatively move it to earlier today (+ natefinch)
<natefinch> hazmat: with the toasca thing at 10, we'd have to meet like right now to make it in earlier today.
<hazmat> natefinch, yup
<mattyw> dimitern, ping?
<dimitern> mattyw, pong
<fwereade_> hazmat, I'm about to go out now and *hope* I will be back by 5
<voidspac_> perrito666: yep
<fwereade_> hazmat, and I haven't taken my swap day yet and was going to take it tomorrow
<fwereade_> hazmat, natefinch: I would hope you can be somewhat productive without me?
<perrito666> voidspac_: I reverted a PR from you last night
<voidspac_> perrito666: I saw
<voidspac_> *grrr*
<voidspac_> :-(
<voidspac_> perrito666: restore still requires direct db access
<voidspac_> so we can't close it off just yet
<perrito666> voidspac_: new restore does too, but I do accept ideas to change that
<voidspac_> perrito666: ssh
<voidspac_> perrito666: we want to close off direct db access
<perrito666> voidspac_: I am a bit curious, how is db access going to be done now?
<voidspac_> perrito666: not externally
<voidspac_> perrito666: db access externally should never be needed - all calls should go through the api
<voidspac_> perrito666: and the state server wouldn't need the port to be open to connect to it on the same machine
<voidspac_> perrito666: so if access to the db is needed an api endpoint should be created - or ssh used
<perrito666> voidspac_: wait, it means that the db would be listening locally?
<perrito666> as in localhost:FORMERSTATEPORT ?
<voidspac_> perrito666: yes
<voidspac_> perrito666: it already is
<voidspac_> I believe
<hazmat> fwereade_, k, hopefully we'll see you at 5 then, have fun.
<perrito666> voidspac_: well your patch changes that
<voidspac_> perrito666: I didn't believe so
<perrito666> voidspac_: since this stopped working:
<perrito666> mongo --ssl -u admin -p {{.AgentConfig.Credentials.OldPassword | shquote}} localhost:{{.AgentConfig.StatePort}}/admin --eval "$1"
<perrito666> :)
<perrito666> you might want to add a test for that
<perrito666> and that is run via ssh
<voidspac_> perrito666: show me where my patch changes that?
<perrito666> into the machine
<voidspac_> perrito666: in the code
<voidspac_> ...
<voidspac_> perrito666: it may just be that the template doesn't work now
<perrito666> voidspac_: possible
<voidspac_> the port (and port binding) didn't change
<perrito666> I also thought of that
<voidspac_> in which case that is much easier to fix I think
<perrito666> voidspac_: sorry I did not try to fix it more in depth, we really needed CI back
<voidspac_> perrito666: no problem - so long as new restore is implemented not needing external db access
<perrito666> voidspac_: well if you can guarantee that localhost:StatePort works should be no problem
<voidspac_> perrito666: cool
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None
<wwitzel3> woohoo .. None
<ericsnow> perrito666, voidspac_: If StatePort is gone then it definitely makes sense that {{.AgentConfig.StatePort}} no longer works in the template.
<voidspac_> ericsnow: it's not gone, it's just not opened externally
<perrito666> ericsnow: I am taking state port from the agent.config and it still is there
<voidspac_> I don't believe I actually removed it from AgentConfig
<voidspac_> trying to get back to the original PR as it's now closed
<perrito666> voidspac_: I wonder if its a problem derivedfrom the permision groups in ec2
<ericsnow> voidspac_, perrito666: ah
<perrito666> which would not make much sense but hey, you never know
<perrito666> voidspac_: https://github.com/juju/juju/pull/449/files
<ericsnow> perrito666: weren't the failures on the HP cloud?
<perrito666> ericsnow: ah not sure actually
<perrito666> sinzui: ?
<sinzui> perrito666, at this hour there are no critical bugs affecting juju devel or stable. This is the first time in months
<perrito666> sinzui: dont jinx it
<perrito666> sinzui: was the error happening in hp too?
<ericsnow> rogpeppe: could you take another look at https://github.com/juju/utils/pull/16?
<ericsnow> perrito666: from http://juju-ci.vapour.ws:8080/job/functional-backup-restore/1309/console: "https://region-a.geo-1.objects.hpcloudsvc.com/v1/..."
<sinzui> perrito666, I am not sure what the question is. restore has failed on both aws and hpcloud. it is currently testing on hpcloud. I changed it last week to see if the recent bug was different on Hp
<perrito666> sinzui: this could be so much easier if we could all read each other thoughts
<perrito666> voidspac_: ok, its not permissions I have no clue then, I guess Ill have to find out
<voidspac_> I'm trying to look at it as well
<perrito666> voidspac_: I think the issue is restore line 287
<voidspac_> perrito666: it's using the external address
<voidspac_> perrito666: it's pinging mongo to wait for it to come up
<perrito666> that will never work
<voidspac_> well, it used to work
<perrito666> yes I know
<voidspac_> hehe
<voidspac_> perrito666: so the restore probably succeeds - but then can't connect to mongo and thinks it has failed
<perrito666> voidspac_: sortof
<perrito666> the restore of state machine succeeds
<perrito666> but it fails when trying to update the other agents
<perrito666> bc it needs st.AllMachines
<voidspac_> right, before updateAllMachines
<perrito666> exactly
<voidspac_> does that need a new API endpoint then
<voidspac_> which can be used instead of directly connecting to mongo
<voidspac_> and the strategy can make repeated calls to that instead
<voidspac_> don't we already know AllMachines?
<voidspac_> or it could run that code on the state server or execute a mongo query
<perrito666> voidspac_: we dont, we try to work that out from the recently restored db
<voidspac_> perrito666: so restoreBootstrapMachine could run an extra command to get the info
<voidspac_> using runViaSsh
<voidspac_> and return the extra information
<voidspac_> or we could add a new endpoint and use apiState
<voidspac_> perrito666: which do you think would be better?
<perrito666> for the case of current restore implementation we can go with runViaSsh, for the new one I can do something prettier
<voidspac_> perrito666: shall I do this - I have some spare cycles
<sinzui> We are one unittest run away from having a passing devel. The anxiety is too much
<voidspac_> sinzui: :-)
<wwitzel3> :)
<katco> wallyworld: still there by chance?
<perrito666> voidspac_: please do, If I keep context switching I will never in my life finish the new restore implementation
<voidspac_> perrito666: ok
<voidspac_> perrito666: and for new restore you will take this into account?
<perrito666> I will
<voidspac_> so I'm working in restore.go still
<voidspac_> not the plugin
<perrito666> voidspac_: the plugin
<voidspac_> (just checking)
<voidspac_> ah...
<voidspac_> no wait
<voidspac_> restore.go is the plugin...
<perrito666> voidspac_: cmd/plugins/juju-restore/restore.go
<perrito666> voidspac_: makesit clearer?
<perrito666> :)
<voidspac_> yep, that's where I've been looking
<perrito666> yup
<voidspac_> thanks
<marcoceppi> OMG
<perrito666> marcoceppi: ?
<marcoceppi> wrong room, though it still applies, I got excited because the buildbot is unblocked
<rogpeppe> ericsnow: looking
<ericsnow> rogpeppe: thanks!
<rogpeppe> ericsnow: reviewed
<ericsnow> rogpeppe: much appreciated
<katco> when updating a library, should i specify the specific commit that fixes the issue, or the latest commit in a stable release?
<katco> sorry, in dependencies.tsv
<natefinch> it depends
<natefinch> generally... latest seems like a reasonable choice, as long as it doesn't break anything else.
<katco> yeah
<natefinch> on the assumption that other bugs may have been fixed in the meantime, and no sense waiting until we hit them to include them in our build
<katco> this is for goyaml, so i am assuming the v1 branch is _relatively_ stable
<natefinch> yeah
<katco> ok cool, i'll grab latest :)
<katco> ty nate! :)
<katco> er one more question
<katco> i switched goyaml over to gopakg.in... i noticed logger isn't in the dependencies.tsv yet, but some gopakg.in packages are
<katco> wasn't gopakg.in designed to obviate godeps?
<natefinch> not exactly
<natefinch> they're somewhat orthogonal,  though related
<katco> ah ok, i misunderstood its purpose then
<natefinch> you need godeps to ensure that you have a repeatable build.  Even supposedly non-breaking changes on a stable branch by definition change behavior. For a release you need to make sure it's possible to recreate the exact same binary multiple times.  Godeps does that
<katco> yeah
<katco> somewhat misunderstood what gopakg.in was designed to solve
<katco> can someone land this for me? it's already been reviewed/approved, just needs to be landed into trunk: https://code.launchpad.net/~cox-katherine-e/goamz/lp1319475.v4.signature.support
<katco> (i don't have permissions, or obviously i would do it myself :p)
<natefinch> katco: what happened to getting goamz moved to github?
<katco> natefinch: we haven't found a home for it yet.
<katco> and didn't want it to impede development any further. we have a customer waiting on this functionality.
<natefinch> katco: I guess we don't control http://github.com/go-amz huh?
<katco> natefinch: no idea, but i would have thought wallyworld or niemeyer would have mentioned it :p
<katco> i am still not clear on why we don't have some sort of canonical repo that all code goes into
<katco> big C canonical
<katco> on github
<natefinch> katco: 'cause we don't control http://github.com/canonical either :)
<katco> is that preventing us from registering canonical-ltd, or canonical-code, or canonical-* lol
<TheMue> dimitern: did you meant the SupportNetworks capability? (which btw the should be renamed to SupportsNetworks)
<niemeyer> natefinch: Define "we"? :)
<natefinch> niemeyer: "Gustavo" :)
<niemeyer> natefinch: Well, I registered go-amz, IIRC
<natefinch> I hoped so
<natefinch> but you never know in the wild west of internet name squatting
<katco> mgz: you around yet?
<mgz> katco: just back from lunch now
<katco> mgz: ah ok, see pm's pls :)
<mgz> hm, on irc? I may be being blind, don't see any
<katco> mgz: hrm yeah
<katco> mgz: sorry my mistake... window wasn't connected
<hazmat> natefinch, fwereade_ meeting time..
<voidspac_> perrito666: I don't have access either
<voidspac_> anyone know *where* the October sprint is?
<voidspac_> I don't think I can make it
<perrito666> voidspac_: I think only people on the cc list of the mail by sarah can see it
<voidspac_> I already have that time booked off...
<perrito666> so natefinch would you tellus where it is?
<natefinch> I asked yesterday, she said they don't know yet
<ericsnow> voidspac_: if you can't be there we should reschedule :)
<voidspac_> ericsnow: definitely
<voidspac_> I land back in the UK on the Sunday 5th October
<voidspac_> ericsnow: actually, I could just fly from India to the sprint
<voidspac_> I'd be away for two weeks then, but ah well
<ericsnow> voidspac_: you'd already be packed ;)
<voidspac_> ericsnow: well yes, but I'd need to pack for two weeks instead of one
<voidspac_> but so be it I guess
<ericsnow> voidspac_: conference T-shirts FTW
<voidspac_> heh
<ericsnow> voidspac_: the price of being a Python luminary :)
<voidspac_> ericsnow: get them, wear them, throw them
<dimitern> TheMue, yes that one
<ericsnow> voidspac_: :)
<voidspac_> ericsnow: you should be with me...
<perrito666> voidspac_: when that kind of think happens to me I pack two bags and let one at home and swap upon arrival
<dimitern> TheMue, it'll go away soon anyway, as the new model kicks in
<voidspac_> perrito666: I don't think I can do "land in uk then immediately fly out again"
<voidspac_> perrito666: but fly straight from India might be doable
<perrito666> voidspac_: if someone is waiting for you in the airport with the spare bag you might
<voidspac_> perrito666: heh, possible depending on flight times I guess :-)
<perrito666> natefinch: wwitzel3 standup?
<TheMue> dimitern: had been in a meeting, so answering now
<dimitern> TheMue, no worries, I was just replying to your earlier questions :)
<fwereade_> natefinch, hazmat: here now if there's still worthwhile time?
<TheMue> dimitern: currently I also have no more questions, only wanted a confirmation ;)
<natefinch> fwereade_: yep, 1 minute
<dimitern> TheMue, :) cheers
<natefinch> ericsnow: btw, I recommend "starring" docs you want to be able to find, so they're under the "Starred docs" in google drive
<ericsnow> natefinch: good tip :)
<natefinch> ericsnow: took me a while to figure that out, after having trouble finding docs again.... it's like google drive specific bookmarks :)
<perrito666> yup, seems that people behind google docs never used a filesystem on their lives
<TheMue> itâs also no problem to move docs to own folders as they are only virtual (like creating a symlink)
<TheMue> so it shoud be possible to access them via a google drive client on a phone or pc too
<katco> it looks like i might have to make some updates to some of our repositories under github.com/juju/* that are not under github.com/juju/juju... what's the workflow for that? fork/pr?
<natefinch> katco: yeah, same as juju/juju   fork & pr
<katco> natefinch: k thanks
<natefinch> katco: not sure about the state of botness on those other repos, though
<katco> natefinch: this should be loads of fun. updating imports of goyaml, touches like 3 sub-repos
<natefinch> katco: at least there's no interdependence... no repo is passing an object from the yaml package to code from another repo... so they can be updated non-simultaneously
<katco> natefinch: at least there's that
<katco> ugh i have to backport all of these too
<katco> this is going to eat up my entire day :(
<katco> well... actually. maybe i should defer the switch to gopkg.in, since it looks like these libraries will just use whatever juju-core specifies in dependencies.tsv
<katco> and save that change for a non-backporting commit
<mattyw> fwereade_, are you around or busy?
<natefinch> katco: yeah, if there's nothing we need in the new yaml package for the old branches, I wouldn't bother backporting
<fwereade_> mattyw, bit of both, am I behind on your reviews?
<mattyw> fwereade_, not at all I landed the periodic worker one as I added the test you asked for
<katco> natefinch: no, i need to backport, i'm just not going to switch to gopkg.in for this commit
<mattyw> fwereade_, but my metrics one I have a question
<fwereade_> mattyw, sweet
<fwereade_> mattyw, ah go on
<katco> natefinch: that way the sub repos should keep using launchpad.net/goyaml which juju-core should drive to the correct version
<katco> errr no wait, b/c that change is not in the launchpad version is it, so i'm looking at an import change regardless
<natefinch> katco: what's the change that you need in yaml?  I thought the move to gopkg.in didn't have any signficant functionality changes
<katco> natefinch: https://bugs.launchpad.net/juju-deployer/+bug/1243827
<mup> Bug #1243827: juju is stripping underscore from options <canonical-webops> <cloud-installer> <config> <landscape> <goyaml:Fix Released by adeuring> <juju-core:In
<mup> Progress by cox-katherine-e> <juju-core 1.20:Triaged by cox-katherine-e> <juju-deployer:Invalid by hazmat> <https://launchpad.net/bugs/1243827>
<katco> natefinch: the move to gopkg.in was a side-effect of having to change the code already
<sinzui> Ladies and Gentlemen, CI has blessed 	Blessed: gitbranch:master:github.com/juju/juju 36fe5868 (Build #1699). Devel is regressions free after 49 days
<katco> woo!
<natefinch> woo hoo!
<natefinch> katco: ahh I see.  Interesting.
<alexisb> sinzui, wow
<natefinch> wait, isn't 49 days about the length of time katco and ericsnow have been on the team.... ? ;)
<katco> lol
<ericsnow> squirrel!
<natefinch> lol
<alexisb> sinzui, now the flood gates will be opened
<sinzui> alexisb, yes, I am prepared to new kinds of hate mail from CI tomorrow
<katco> i have PRs to juju/(cmd|utils|charm) that need reviewing. 1-line import change. blocking cut if anyone wants to have a quick look.
<natefinch> cmars: can you try https://github.com/juju/juju/pull/495 with the new code?   It gives much nicer error messages now.
<katco> can anyone review the aforementioned changes?
<natefinch> katco: link me?
 * natefinch is lazy
<katco> hey np, gladly :)
<katco> https://github.com/juju/utils/pull/21
<katco> https://github.com/juju/charm/pull/39
<katco> https://github.com/juju/cmd/pull/5
<natefinch> katco: did you compile these?  the package name changed from "goyaml" to "yaml"
<natefinch> katco: I know only because I just made the same change in another codebase, and realized it's not a one line change (unfortunately)
<katco> lol you are right, sorry. i did all these through scripting, so i forgot about that
<katco> sigh ok well good review haha
<natefinch> katco: heh np :)
 * cmars takes another look
<natefinch> cmars: it'll only fail at the first line of differences, but it should be clear what's different, at least.
<katco> natefinch: have another look? all building now. still 1 line change :)
<natefinch> katco: ha. wondered if you'd go that route
<natefinch> katco: I vaguely disapprove of  renaming the import just to avoid changing more lines of text, but I don't think it's a huge deal.
<katco> natefinch: actually, i don't think i know this: how do you utilize a package imported via gopakg?
<katco> would it be yaml.v1.Foo()?
<natefinch> katco: the import path and the package name are actually totally unrelated.  by convention they are the same... but a package name cannot include punctuation (I believe the actual restriction is something like like unicode letter followed by any number of unicode letters, numbers, or undescore)
<natefinch> katco: the convention for gopgk.in is that the version is not part of the actual package name, so "yaml.v1" is package yaml
<katco> natefinch: ah so you just do import yaml "gopkg.in/yaml.v1"?
<natefinch> katco: import "gopkg.in/yaml.v1"   and then use it as yaml.Foo()
<natefinch> katco: you don't have to name the import, it gets named by what "package foo" says in the code, which in this case is "package yaml"
<katco> natefinch: huh? how does that resolve? it elides the .v1?
<katco> ohhh i see
<natefinch> katco: https://github.com/go-yaml/yaml/blob/v1/yaml.go#L7
<natefinch> katco: that's what I mean by the import path and the package name not being related.  You can put that code at https://github.com/natefinch/ABCD and import it as import "github.com/natefinch/ABCD" and you'd still refer to the package as yaml.Foo()
<katco> natefinch: gotcha, thanks
<natefinch> katco: this was actually one of the biggest complaints about the way gopkg.in does versioning - the last part of the URL is not the same as the package name
<katco> natefinch: yeah, i wonder if like gopkg.in/v1/yaml would have worked
<natefinch> katco: there's a couple problems with that - 1.) it sorts badly in the list of imports... so gopkg.in/v1/yaml might be far away from an import of gopkg.in/v2/yaml   (the .v1 .v2 imports would sort to be right next to each other)
<natefinch> katco: 2.) it puts a /v2/ directory in your filesystem with a bunch of unrelated code in it, and again, the v1 code is far from the v2 code
<katco> natefinch: ah
<katco> natefinch: anyway, does this all look ok?
<natefinch> katco: sorry, tangent :)
<katco> natefinch: not a problem :) just trying to get this in for sinzui
<natefinch> katco: LGTM'd.
<katco> natefinch: thanks for your help today
<natefinch> katco: welcome
<thumper> morning
<alexisb> morning thumper
<thumper> alexisb: morning
<katco> morning thumper
<thumper> o/ katco
<katco> (not intended just for thumper) i'm running into a strange kind of circular dependency b/c of gopkg.in. i'm trying to update v3 of juju/charm, which utilizes gopkg.in/juju/charm.v3 to reference itself. so it's referencing the wrong version of itself... if that makes sense?
<thumper> huh?
<katco> am i doing something wrong? or should i hack this to get around it
<thumper> what exactly are you doing?
<katco> alright, so i'm working with github.com/juju/charm
<katco> and all i'm trying to do is update some imports
<thumper> AFAICT, if you have packages that use gopkg.in, then you need to be in that dir
<thumper> yeah...
<thumper> so work in the dir gopkg.in/juju/charm.v3
<katco> so i should be making these changes w/in gopakg.in on my machine?
 * thumper nods
<thumper> I think so
<katco> that's what i was doing wrong then
<fwereade_> thumper, heyhey
<thumper> hi fwereade_
<fwereade_> thumper, how's the time difference?
<thumper> terrible
<thumper> you mean now?
<thumper> or from germany?
<fwereade_> thumper, from germany
<fwereade_> thumper, I have a notion that we may disagree on the yuckiness of the Factory varargs
<thumper> heh, yeah
<fwereade_> thumper, I'm interested in counterarguments
<thumper> I'd rather have ickyness in one place, the factory, than at every call site
<thumper> I agree it is a little icky
<thumper> but working around golang limitiations
<thumper> it was dave's idea
<thumper> originally I had two methods
<fwereade_> thumper, I think it was the *repeated* ickiness inside Factory that really put me off
<thumper> for each type
<thumper> but the ickiness there is limited in scope, and contained
<thumper> vs. spreading it around all the places the factory is used
<fwereade_> thumper, just to be clear, it's the nil that's yucky?
<thumper> mostly, and the c
<thumper> what I *want* is: factory.MakeUnit()
<thumper> however
<thumper> due to bugs in gocheck
<thumper> we need the c
<fwereade_> thumper, indicating "I don't care" in place of a set of explicit instructions
<thumper> yes
<thumper> I had earlier...
<thumper> factory.makeAnyUser()
<thumper> and factory.MakeUser()
<thumper> damn capitals
<thumper> we joined those methods together
<thumper> to avoid the nil
<thumper> having: factory.MakeUser(c, nil) isn't obvious
<thumper> factory.MakeUser(c) is slightly more so IMO
 * thumper misses python
<fwereade_> thumper, I know the feeling
<fwereade_> thumper, but I'm not sure that even python's varargs aren't more trouble than they're worth
<fwereade_> thumper, explicit is better than implicit
<thumper> python has the advantage of explicit default args
<thumper> fwereade_: IMO, nil isn't explicitly stating what you want, you have to go look up what nil is
<thumper> whereas not having nil is being explicit :)
<fwereade_> thumper, (python has default args with some really entertaining behaviour, but anyway)
<thumper> sure...
<fwereade_> thumper, I can read it just as easily as nil=>no preference, and that as a stronger statement than no statement at all
<thumper> I have a gut reaction to blind params, especially with nil
<thumper> I don't care strongly enough to fight for long
<thumper> speaking of which,
<thumper> the whole user sub(super)command is in question in the spec
<thumper> which I'm losing the will to fight as well
<thumper> suppose I should write something up
<fwereade_> thumper, oh really? I do think that we do ourselves no favours by polluting the command namespace
 * thumper nod, I'll CC you on the email about it
<fwereade_> thumper, cheers
<katco> these are always funny. just received a panic from:// can't happen, these values have been validated a number of times
<katco> 		panic(err)
<thumper> katco: for some value of funny (i.e. not funny)
<thumper> :)
<katco> thumper: haha
<wallyworld> katco: hi, how'd you get on with the deps update?
<katco> wallyworld: still working... ran into a bunch of panics using the head of the yaml lib
<wallyworld> oh :-(
<katco> i had to update 3 sub repos to use the latest version of goyaml
<katco> that's what took the longest
<wallyworld> np, sounds like you poked a hornet's nest
<katco> i think i'm going to try and sit on the commit that fixed the reported issue and see what that does
<katco> wallyworld: i did get my ~20 day old change landed :)
<katco> that made katco very happy
<katco> this run seems to be going better with commit 1b9791953ba4027efaeb728c7355e542a203be5e
<katco> yeah almost done. i'm going to stick with this one and submit a PR after tests have passed
<ericsnow> fwereade_: (on the off chance you're in a reasonable timezone for now) you still around?
<davecheney> moin
<katco> wallyworld: well, now i have test failures because of goamz. i'm guessing it's because we're mocking environments and not specifying a signer. can this wait until tomorrow?
<wallyworld> katco: be with you soon, just in a meeting
<waigani> thumper: standup?
<thumper> sorry, on my way
<katco> wallyworld: ok
<ericsnow> davecheney: you one of the reviewers today?
<wallyworld> katco: yeah, it can wait. sorry that it turned out to be more problematic than first thought
<katco> wallyworld: no worries... we're almost there
<wallyworld> yep :-)
<katco> wallyworld: just have to find out where these mocked regions are
<wallyworld> ok
<katco> wallyworld: alright, way past my EOD. going to spend some time with my daughter before she has to go to bed :)
<katco> talk to you tomorrow!
<wallyworld> will do, thanks for taking the extra time :-)
<davecheney> ericsnow: ok
<davecheney> i have calls for the next 2 hours
<davecheney> i'll take a look after that
<ericsnow> davecheney: cool, thanks
<ericsnow> davecheney: https://github.com/juju/utils/pull/19 https://github.com/juju/juju/pull/462 https://github.com/juju/juju/pull/453
#juju-dev 2014-08-14
<davecheney> thumper: ping, i'm free to talk whenever you are
<katco> wallyworld: https://github.com/juju/juju/pull/513 :)
<katco> wallyworld: going to eat dinner, i'll check for a LGTM in a bit and hopefully land it!
<wallyworld> katco: awesome. for the 1.20 update, you will need to ensure you use the 1.20 branches of the relevant sub repos ie you will need to update their imports and commit the changes and then use the relevant rev shas
<axw> o/ wallyworld
<axw> how was nuermberg?
<axw> nuern nurem... equal edit distance
<wallyworld> axw: hey ya. good but busy. we gots a lot to do. got time for a chat?
<axw> wallyworld: sure
<wallyworld> see you in standup meeting
<thumper> davecheney: around?
<davecheney> yeah
<davecheney> thumper: sup
<thumper> chat?
<davecheney> thumper: sure, lets meet in the 1-1 hangout
<thumper> kk
 * thumper is there waiting for davecheney
<wallyworld> thumper: did you want to delay our 1:1
<thumper> wallyworld: yes
<wallyworld> ok, just ping me
<thumper> wallyworld: now?
<wallyworld> sure
<davecheney> lucky(~/src/github.com/juju/juju) % gb ./...                                                                                                          âÂ·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·Â·
<davecheney> juju/sockets/sockets.go:6:2: no buildable Go source files in /home/dfc/src/gopkg.in/natefinch/npipe.v2
<davecheney> anyone else getting this error ?
<davecheney> wallyworld: thumper axw https://github.com/juju/juju/pull/514
<davecheney> review please
<davecheney> this is blocking my build
<wallyworld> davecheney: otp, will look soon
<axw> eh, I thought it was updated to add a file for linux
 * axw check
<axw> s
<davecheney> i ran godeps and it didn't complain
<davecheney> i belive I have the right rev
<davecheney> seriously
<axw> davecheney: I think you just need to merge trunk again, I think there's a new rev
<davecheney> this isn't the right way to fix the problem
<davecheney> axw: trunk of which repo ?
<axw> npipe
<axw> um
<axw> merge trunk of juju
<axw> there's a new rev of npipe in dependencies.tsv
<davecheney> gopkg.in/natefinch/npipe.v2     git     e562d4ae5c2f838f9e7e406f7d9890d5b02467a9
<davecheney> that is what it says
<davecheney> and that is the rev I have
<davecheney> I FUCKING HATE GOPKG>IN AND THE INTERACTION WITH GODEPS
<davecheney> I HATE IT SO MUCH
<davecheney> I MAKES ME MISERABLE
 * thumper chuckles 
<axw> davecheney: what's the problem exactly? I'm not having any problems with godeps
<davecheney> we have two tools that are trying to do the same thing, pin a dependency of a go package
<davecheney> i think it is a serious error to combine those two tools
<thumper> davecheney: I don't think it is too terrible, just annoying
<thumper> one seems to pin (vaguely) the api, the other pins to commits
<wallyworld> davecheney: did you see all the syslog stuff being done? are you happy with it? i thought at one point you may have had an opinion on it? maybe i am misremembering?
<davecheney> wallyworld: as in moving away from rsyslog and doing file rolling inside the application ?
<wallyworld> davecheney: that and there was talk of using Go's syslog, not sure if that went ahead or not
<davecheney> don't use that package
<davecheney> it's fucked
<davecheney> we can't remove it
<davecheney> but it's still fucked
<wallyworld> yup
<davecheney> i also am not in favor of doing file rolling inside the applicatoin
<davecheney> from a decade of sysadmin experience
<wallyworld> yep
<davecheney> that is a sure fire way to loose logs
<wallyworld> davecheney: can you replay to the thread on juju-dev, or email nate directly with your concerns?
<wallyworld> reply
<davecheney> wallyworld: will my comments be given serious consideration
<davecheney> i've not seen a lot of that in the past
<davecheney> i know that is a snotty thing to say
<davecheney> but i'm not going to kick up a stink unless there is actual possiblity of chance
<davecheney> change
<davecheney> otherwise i'll save everyone the discomfort of hearing me rant
<wallyworld> they will be taken seriously by me, and whomelese else i talk to i'll tell them :-)
<wallyworld> don't rant as such, just provide facts
<wallyworld> and i'll make sure nate and whoever else takes note
<davecheney> here are the facts
<davecheney> 1. the state/api/database/mongo whatever runs on linux
<davecheney> it will continuye to run on linux for the forseable future even if we have windows workloads
<davecheney> we should continuye to use rsyslog for log collection
<davecheney> as it decouples the application from log rolling
<davecheney> applications write to stdout
<davecheney> somethign else takes care of where that stdout goes
<davecheney> if you make the application responsible for rolling it's own logs
<davecheney> you'll generally find that you end up writing to a log file that has been unlinked
<axw> I did comment on the PR about the approach to rolling being a bit broken, but I don't see any change in nate's new PR
<wallyworld> i agree. can you put that in an email to the list? feel free to mention your previous sysadmin experience to add weight to your arguments
<davecheney> wallyworld: http://12factor.net/logs
<davecheney> ^ this is what we shold be aiming for
<davecheney> wallyworld: i honestly don't think i can be objective at thispoint
<davecheney> i've given you what I can
<davecheney> i don't want to take on the heartache of another loosing battle
<wallyworld> hmmm, ok. i would prefer you provided the raw material (based on yuor experience) - we can run with it from there
<wallyworld> ie fight the battle
<wallyworld> but i can do it also
<davecheney> wallyworld: which thread ?
 * wallyworld looks
<davecheney> i can't find it
<davecheney> if it's on juju-team
<davecheney> i'm not subscribed to that list
<davecheney> or i should say, my subscription is still pending
<wallyworld> juju-dev - getting rid of all-machines.log
<wallyworld> but we could do a new post referencing that thread
<axw> also: https://github.com/juju/juju/pull/375, superseded by https://github.com/juju/juju/pull/512
<axw> wallyworld: that big test was just moved. I'd rather not change existing things too much at this point
<axw> ah you noticed :)
<wallyworld> axw: yeah, just made a followup comment :-)
 * axw tests it once more for good luck
<axw> wallyworld: can you please review https://github.com/juju/testing/pull/28 too?
<wallyworld> sure
<axw> wallyworld: btw, to test with mongo 2.6 you can just do "JUJU_MONGOD=path/to/mongod go test ./..."
<axw> will not work in CI, though; that would require additional changes
<wallyworld> axw: np, thanks. there was a test CI job I started a while back
<axw> cool
<wallyworld> but i never finished it
<TheMue> morning
<dimitern> morning TheMue
<dimitern> I thought jam is here today..
<TheMue> dimitern: IMHO he is on holiday the whole week
<TheMue> dimitern: oh, no, just took a look into the calendar
<dimitern> TheMue, yeah, maybe he'll pop in at some point today
<TheMue> dimitern: yep
<TheMue> dimitern: so comming back to my change of SupportsNetwork and its usage
<dimitern> TheMue, how's that capability branch for the networker going?
<dimitern> TheMue, :)
<TheMue> dimitern: bingo
<waigani_> axw: https://github.com/juju/juju/pull/518
<TheMue> dimitern: the capability is currently already implemented, imho only with the wrong naming
<TheMue> dimitern: but is this enough
<TheMue> dimitern: and then use it where today we test if the providerType is MAAS?
<axw> waigani_: looking
<axw> waigani_:  :x a much simpler solution has just dawned on me
<axw> waigani_: we could have just got the bootstrap instance and used its Addresses method...
<dimitern> TheMue, there's a new capability needed, let's call it AllowOnlySafeNetworker(machineId string) bool
 * axw continues looking anyway
<dimitern> TheMue, which will return false for all providers, except local
<dimitern> TheMue, and for local it will check internally if the machineId == bootstrapMachineId (or however it's called) and return true (i.e. do not allow unsafe networker on the bootstrap node) or false (for all other machines)
<TheMue> dimitern: ah, already wondered. this description better matches what Iâve seen in the discussion, thanks.
<TheMue> dimitern: as we call them mostly Supportsâ¦ (reads better) I would call it SupportsUnsafeNetworker(machineId)
<dimitern> TheMue, I'm fine with that (in this case the meaning is inverted - true for most, false for local bootstrap node)
<TheMue> dimitern: Iâm also not happy with Networker and SafeNetworker, which implies the first one is âunsafeâ ;)
<axw> waigani_: commented on the PR
<dimitern> TheMue, well it is kinda unsafe :)
<dimitern> waigani_, the bootstrap node can have and does have more than one address usually
<waigani> axw: you horrible man
<axw> waigani: :(
<axw> waigani: on the plus side, the alternative is really quick to implement :p
<waigani> axw: lol, no it's a good solution - I raised an eyebrow when I came across Addresses, but assumed we could not use it for all providers
<waigani> axw: will the bootstrap instance have  Id() == "0" ?
<axw> waigani: no
<axw> that's machine ID
<axw> instance ID you cannot predict
<axw> waigani: just call AllInstances() and get the one and only element out of it
<waigani> axw: okay, so we only bootstrap one instance? When to the HA instances get bootstrapped?
<axw> waigani: bootstrap = start the first instance
<axw> HA comes later, when you do "juju ensure-availability"
<waigani> right
<voidspac_> dimitern: TheMue: just noticed your conversation
<voidspac_> dimitern: TheMue: shouldn't "safe networking" be true for manual provider too
<dimitern> voidspac_, yes
<voidspac_> dimitern: TheMue: and I agree "safe" is not a brilliant name
<voidspac_> but I dislike burning too much energy on name bikeshedding
<voidspac_> pick something and use it
<dimitern> voidspac_, for all machines with the manual provider, and only for the bootstrap node with local
<voidspac_> yep
<axw> don't forget machines can be manually added to a non-manual provider environment
<axw> in which case provider != "manual"
<voidspac_> axw: oh, ouch
<axw> probably don't want to muck with their networking either
<voidspac_> dimitern: the unit test failures are all uniter tests
<voidspac_> dimitern: are you / is anyone looking at them?
<dimitern> voidspac_, which failures?
<voidspac_> dimitern: the ones jam emailed us about yesterday
<voidspac_> dimitern: CI failures
<TheMue> axw, dimitern: so manual || local bootstrap || (non-manual && manually added)?
<voidspac_> dimitern: you said you would look at them :-)
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1459/console
<dimitern> axw, good point; TheMue you should check for manually provisioned machines and not allow unsafe networker there as well
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1458/console
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1457/console
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1456/console
<dimitern> axw, what's the best way to tell if a machine is manually provisioned by its id?
<dimitern> voidspac_, aw, sorry, I'm struggling with networker refactoring, will have a look before the standup
<axw> dimitern: from what context? you can tell from the state.Machine
<voidspac_> dimitern: I can have a look, although maybe not before standup
<dimitern> axw, ah, so the machine id is not special
<TheMue> axw: idea is to have an environ capability able to say if unsafe networkers are supported by passing the machine id
<axw> dimitern: nope. the nonce is special
<dimitern> axw, we'll need to lookup the machine in state via the api to be able to tell
<axw> dimitern: that would probably be best
<dimitern> axw, is the nonce constant?
<axw> dimitern: yup
<axw> set by the instance broker
<axw> well, in this case it's set by the manual provisioning code
<dimitern> axw, alright, thanks
<axw> dimitern: TheMue: state.Machine.IsManual()
<TheMue> axw: nice
 * TheMue takes some notes
<TheMue> Hmm, just found âSetHasVote()â. Looks strange, would have named it âGrantVote()â.
<waigani> axw: I'm hitting the same problem that I hit when I connected to the api after bootstrap: something is hanging. I've set state-server to true on tests
<waigani> axw: here's what I've done: http://pastebin.ubuntu.com/8043738/
<axw> waigani: *shouldn't* need to do that... there should be no interaction with the state server to obtain the instances or their provider addresses
<waigani> axw: I wonder if it is this: c.ConnectionEndpoint(false)
<axw> waigani: I have nfi what ConnectionEndpoint is
<mattyw> fwereade_, ping?
<mattyw> fwereade_, ^^ I'm going to rebase the metric branch before landing - I think most of the review was us working around what should be implemented so I'm going to cleanup the history to make it easier to understand
<perrito666> morning
<perrito666> fwereade_: ping me when you arearound plz
<waigani> axw: okay I got it working. Do we also only want to grab the first address of the first instance?
<axw> waigani: I think we should just get them all, like the existing api address caching does
<waigani> axw: cool, that is how I have it now
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1356806
<mup> Bug #1356806: cmd/juju: juju takes 3 seconds to do nothing <juju-core:New> <https://launchpad.net/bugs/1356806>
<rogpeppe> davecheney: it's a pity that parsing rsa keys is so slow. ISTR that it's because it does a "probably prime" check when it parses the key.
<rogpeppe> davecheney: although i thought that issue was fixed. hmm.
<davecheney> rogpeppe: this is on ppc64
<davecheney> where there is no asm math library
<davecheney> juju is just doing something dumb
<davecheney> it doesn't _always_ need to read the private key
<rogpeppe> davecheney: i agree it's dumb, but it should not take 3s to parse an RSA key on any platform
<davecheney> i don't know how optimised the ppc port is at the moment
<davecheney> one would suggest, not at all
<rogpeppe> davecheney: even if not optimised
<rogpeppe> davecheney: parsing an RSA key should not require many math operations
<rogpeppe> davecheney: it's essentially just reading a couple of big numbers
<davecheney> rogpeppe: i think we're in agreement here
<davecheney> yet here we are, waiting 3 seconds for nothing to happen
<rogpeppe> davecheney: i'd like to fix both places
<rogpeppe> davecheney: both the Go library (to make parsing RSA keys more efficient) and juju-core (to avoid doing that when it doesn't need to)
<dimitern> alexisb, ping
<dimitern> voidspac_, standup?
<voidspac_> dimitern: I'm there...
<voidspac_> dimitern: connection lost :-(
<voidspac_> dimitern: overlapping subnets are not allowed in the new model - we have one network
<dimitern> voidspac_, yes, but where have you seen the overlapping subnets in the model?
<voidspac_> dimitern: but isn't it possible that subnets on different nics will look like they overlap
<voidspac_> but don't because they're on separate networks
<voidspac_> but we can't represent that in the new model
<voidspac_> cidr is the primary key
<dimitern> voidspac_, I'm not sure I understand
<voidspac_> or should the different nics be bound to different subnets anyway
<dimitern> voidspac_, cidr is the pk for a subnet, yes
<dimitern> voidspac_, nics can be bound to one or more subnets
<voidspac_> dimitern: two different networks can have the same subnets allocated to different machines though, right?
<voidspac_> dimitern: so the same cidr can be duplicated on different networks
<dimitern> voidspac_, the confusion I think comes from the term "network"
<voidspac_> heh
<dimitern> voidspac_, in juju, per the new model, all subnets are in the same (virtual/logical?) network
<dimitern> voidspac_, in provider-space, if networks and subnets are supported, there is a possibility to have 2 networks with overlapping subnets
<voidspac_> dimitern: right
<voidspac_> dimitern: how do we cope with that in the juju model?
<dimitern> voidspac_, but I don't think this is a valid case
<voidspac_> dimitern: ok
<voidspac_> dimitern: because they won't be routable to each other and so you can't use them for anything useful
<dimitern> voidspac_, it a weird setup, and we should not allow that, aiui
<dimitern> voidspac_, yes
<voidspac_> dimitern: but we currently don't "disallow it", we pretend it can't happen
<dimitern> voidspac_, but good point about bringing this up
<voidspac_> you can still have unroutable subnets, and just whichever one is added first wins
<voidspac_> dimitern: I think in summary, networking is hard
<dimitern> voidspac_, another possible case is having overlapping vlan subnets with different tags
<voidspac_> right
<dimitern> :) oh yeah
<dimitern> voidspac_, you can't do this in maas I think
<voidspac_> dimitern: TheMue:  also I spent a while talking to perrito666 yesterday - current restore does not *need* direct mongo access
<dimitern> voidspac_, so juju shouldn't take it either
<voidspac_> dimitern: TheMue: it just uses it after doing the restore to update some data, and can be fixed to not do that
<voidspac_> dimitern: right
<dimitern> voidspac_, great!
<voidspac_> dimitern: fair enough
<voidspac_> dimitern: TheMue: so that's what I've been on, I was going to swap and look at the bugs
<voidspac_> dimitern: I have a precise VM, should I try and repro these bugs?
<dimitern> voidspac_, yes please, I'll do the same; hopefully different setups can lead to reproduction of at least some bugs
<voidspac_> dimitern: cool, coffee first then I'll spin up my VM
<natefinch> fwereade_: team leads?
<sinzui> wallyworld, ppc64el unittests are failing in many different ways.
<wallyworld> oh joy
<sinzui> wallyworld, I am going to increase the number of retries, and also look for cruft left on the machine
<sinzui> I am not sure juju is really bad on ppc
<wallyworld> sinzui: could it be new tests with ordering issues, or is it existing tests that have started failing? i'll have to look at the logs. currently in standup
<dimitern> voidspac_, managed to reproduce some of the ci test failures on a precise vm using rev 9b2d2106476bad0ac528256db23bad073257d4bf
<voidspac_> dimitern: cool
<voidspac_> dimitern: any ideas on the cause?
<voidspac_> dimitern: I took a break and am now still updating the vm
<dimitern> voidspac_, actually I'm having issues with mongo now
<voidspac_> juju won't build for me
<voidspac_> think i need to update go
<dimitern> voidspac_, I installed mongodb-server from the ppa:juju/experimental
<voidspac_> godeps completed but still build errors
<voidspac_> ah
<voidspac_> ls
<dimitern> voidspac_, you need to install mongodb-server and golang from there
<voidspac_> dimitern: I can't just build golang from source?
<natefinch> where Go comes from is certainly not the problem
<dimitern> voidspac_, you can ofc, but to mimic the test environment as close as possible, I'd use the ppa
<voidspac_> ok
<voidspac_> just added the ppa and doing an update
<voidspac_> go is building anyway
<voidspac_> natefinch: actually I think it was the problem (which Go I was using)
<voidspac_> natefinch: I think we're no longer compatible with the golang in precise
<voidspac_> which I was using before
<voidspac_> but building go from source works fine
<natefinch> voidspac_: oh, yeah, sorry, I thought you were using something reasonable
<natefinch> :D
<voidspac_> although I now have golang from the experimental ppa as well
<dimitern> voidspac_, interestingly, I can run (most of) state/ tests without any mongo issues, but for agent/ or worker/uniter/ tests I get the same failures "cannot set admin password: need to login"} ("cannot set admin password: need to login"
<voidspac_> natefinch: I'd moved it out of the way to try something a while ago and not moved it back
<natefinch> voidspac_: I think precise has like 1.1 or something, and we need ~1.2.1
<voidspac_> dimitern: running worker/uniter now
<voidspac_> natefinch: right
<natefinch> voidspac_: newer versions of Go always work with older code.  But older versions of Go don't always work with newer code
<voidspac_> natefinch: the actual build error was something nice and helpful like
<voidspac_> ../../../code.google.com/p/go.crypto/openpgp/packet/encrypted_key.go:8: import /home/michael/canonical/pkg/linux_amd64/code.google.com/p/go.crypto/openpgp/elgamal.a: object is [linux amd64 go1.2.2 X:none] expected [linux amd64 go1.1.2 X:none]
<voidspac_> which looked like a version error to me...
<dimitern> does anybody have any clue about that error? "cannot set admin password: need to login"
<voidspac_> natefinch: right, in theory anyway
<voidspac_> dimitern: axw has done a lot of work in that area recently
<natefinch> ahh yeah.... that's actually just old binaries in your pkg directory
<dimitern> the "need to login" part happens in several unrelated tests, but not all of them fail
<dimitern> voidspac_, this is actually on trunk HEAD
<natefinch> voidspac_: they're actually really really really careful to make sure they maintain backwards compatibility.
<dimitern> voidspac_, I had too many issues with rev 9b2d2106476bad0ac528256db23bad073257d4bf so decided to try trunk first
<voidspac_> dimitern: right
<voidspac_> *sigh*
<voidspac_> adding upstream repo and actually pulling HEAD
<voidspac_> now re-running
<voidspac_> I had some failures, but I wasn't on HEAD
<voidspac_> although I was on master, so failures still od
<voidspac_> *odd
<voidspac_> but different
<voidspac_> filter_test.go:305 c.Fatalf("unexpected config event")
<dimitern> voidspac_, no failures with worker/uniter on HEAD; now re-running on 9b2d2106476bad0ac528256db23bad073257d4bf, after I wiped out $GOPATH/pkg/, just in case
<voidspac_> dimitern: I get one failure on HEAD
<voidspac_> the same as above
<dimitern> voidspac_, lots of failures, all of them stemming from "need to login"
<voidspac_> dimitern: so it looks to you like that issue has been fixed
<voidspac_> if it's on 9b2d2106476bad0ac528256db23bad073257d4bf but not on head
<dimitern> voidspac_, I think something's wrong with my mongo there, I want to fix this first, so I can have a reliable setup
<voidspac_> ok
<voidspac_> I'm trying worker/uniter with 9b2d2106476bad0ac528256db23bad073257d4bf
<voidspac_> dimitern: I get no worker/uniter failures on 9b2d2106476bad0ac528256db23bad073257d4bf
<dimitern> voidspac_, hmm how about for agent/ ?
<voidspac_> will try
<voidspac_> dimitern: I re-ran the worker / uniter tests and got the failure I got previously
<voidspac_> ran agent/... tests fine
<voidspac_> no failures
<voidspac_> but I didn't have this failure on worker/uniter before
<voidspac_> so looks like it's flaky rather than consistent
<voidspac_> but it's still a different failure to the one on CI
<voidspac_> or the ones you have
<dimitern> right
<voidspac_> so, fark
<voidspac_> not helpful at all
<hazmat> hmm.. recent yaml changes seem to cause some compat issues.. $ juju status -> error unmarshalling "/opt/juju/environments/ocean.jenv": YAML error: resolveTable item not yet handled: < (with <root>=DEBUG;unit=DEBUG)
<dimitern> voidspac_, what mongodb are you using?
<dimitern> voidspac_, $ `which mongod` --version ?
<dimitern> voidspac_, sorry, I meant $ which mongod && mongod --version
<perrito666> natefinch: wwitzel3 standup
<dimitern> voidspac_, the version from the ppa is 2.2.4, but this is too old for what we're doing around state.Initialize(), more specifically, SetAdminMongoPassword uses admin.UpsertUser(), which is available in 2.4+
<dimitern> sinzui, ping
<sinzui> hi dimitern
<dimitern> hey, I think I discovered a potential regression re precise + mongodb
<sinzui> :(
<dimitern> sinzui, but it's funny how it doesn't show on the CI tests - is the mongod there 2.2.4 from ppa:juju/experimental ?
<dimitern> sinzui, because I can reproduce this issue at will with both HEAD of trunk and rev 9b2d2106476bad0ac528256db23bad073257d4bf (from http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1456/consoleFull)
<sinzui> dimitern, CI uses 2.4.6 to match cloud-tools
<dimitern> sinzui, from where?
<sinzui> http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/m/mongodb/
<sinzui> dimitern, ^ looks like I should switch to 2.4.9 to match
<dimitern> sinzui, how can I install mongodb-server from there? add-apt-repo ...?
<dimitern> sinzui, is it know that we no longer support mongodb 2.2.x, even on precise?
<dimitern> known*
<hazmat> sinzui, is manual provider in our functional tests?
<hazmat> i'm encountering quite a few regressions.. just trying to determine sanity
<sinzui> hazmat, yes, deploys an upgrades to instance in aws and real metal
<dimitern> ah, found it - ppa:juju/stable
<sinzui> sudo add-apt-repository cloud-archive:tools
<hazmat> sinzui, hmm.. ic. okay. i'll keep filing bugs then
<sinzui> dimitern, https://wiki.ubuntu.com/ServerTeam/CloudToolsArchive
<dimitern> thanks sinzui
<sinzui> dimitern, I copy packages to juju stable because many people fail to install that archive when using local precise
<dimitern> sinzui, right, I'm trying the same tests now with 2.4.6 from the archive
<dimitern> sinzui, I can't reproduce any of the failures now :(
<hazmat> sinzui, looks like manual provider add-machine but oddly not bootstrap is failing if there isn't an ubuntu user in the base image.
<sinzui> ah
<dimitern> sinzui, but I have a suggestion how to add debug logging to mgo, so we can see what's going on in more detail
<sinzui> hazmat, we have a bug for a cases where ubuntu was removed from a server image and lxc fails
<hazmat> sinzui, also wierdly bootstrap but not add-machine uses ssh-agent
<voidspac_> dimitern: /usr/bin/mongod
<voidspac_> db version v2.4.6
<voidspac_> Thu Aug 14 17:19:47.712 git version: nogitversion
<voidspac_> dimitern: I added the ppa and installed mongodb-server
<dimitern> voidspac_, right I suspected that much - it'll never work with mongodb-server 2.2.4 from ppa:juju/experimental
<voidspac_> dimitern: but it must be preferring a newer version from elsewhere
<voidspac_> dimitern: but that's not the same error that CI has, right?
<dimitern> voidspac_, with mongodb-server 2.4.6 from cloud-archive:tools - no failures
<voidspac_> they just have uniter test failures
<bac> jcastro: cross-team call today?
<voidspac_> ah
<voidspac_> dimitern: what about CI with HEAD?
<voidspac_> I guess I can check...
<dimitern> voidspac_, I'll try HEAD next, just running uniter tests once more to be sure
<sinzui> hazmat, I re added ubuntu user to the kvm/maas machine I am provisioning. And I provisioned it with manual provider
<voidspac_> dimitern: 1470 passed!
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1470/
<dimitern> voidspac_, sweet!
<voidspac_> REVISION_ID=abfd7625309d31ecddd8fa799d64c8d4fa41977c
<voidspac_> is that head?
<voidspac_> I believe so
<hazmat> sinzui, ok.. its a regression though.. used to work in 1.18.. glad to hear it works if the bug isn't triggered but ideally it could be part of the regression check.
<dimitern> voidspac_, so I think the failures were either bogus or they got fixes in subsequent branches
<voidspac_> dimitern: I think so
<voidspac_> dimitern: there are some canonistack-deploy failures
<voidspac_> http://juju-ci.vapour.ws:8080/job/canonistack-deploy-precise-amd64-devel/
<voidspac_> dimitern: but they're time-outs, so not sure how seriously to take those
<voidspac_> I get that with canonistack fairly often
<dimitern> voidspac_, perhaps the previous failed test didn't clean up properly?
<voidspac_> dimitern: well, the very latest one is a bootstrap failure - the two before that are timeouts
<voidspac_> so yes, looks like the test environment is now screwed :-)
<dimitern> voidspac_, yes, my thoughts exactly :)
<dimitern> sinzui, are you seeing this ^^
<sinzui> voidspac_, dimitern . I have seen it. I have not completed investigation. Don't panic. The tests are -devel because canonistack is not production grade. when swift gets slow for example, everything fails
<voidspac_> sinzui: but the unit test failures have been fixed
<voidspac_> http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1470/
<voidspac_> current HEAD
<sinzui> voidspac_, yes. I should admit that ppc64el is doing worse. I increased retesting to 4 to improve the chances to pass
<voidspac_> sinzui: that's showing latest build passed too
<voidspac_> Same revision.
<voidspac_> Looks like the increased retesting did the trick...
<voidspac_> There was a replicaset failure for that revision.
<Beret> alexisb, sinzui - I forgot to mention this on the call
<Beret> alexisb, sinzui - sparkiegeek has a branch up to fix an issue we found, would appreciate it getting through whatever process - https://github.com/juju/utils/pull/22
<Beret> that hits us quite a lot - it would be good to get it into 1.20
<alexisb> Beret, ack
<Beret> thanks
<alexisb> anyone know who is the oncall reviewer today?
<alexisb> natefinch, ^^
<natefinch> alexisb: tim and dave: https://docs.google.com/a/canonical.com/spreadsheets/d/1iQLLOWrjzxddm5VhYWYi0-2k3xI6wTMlpkvnVNJCYGY/edit
<sinzui> Beret, does that address bug 1354685
<mup> Bug #1354685: installation of packages for containers should be retried in face of lock errors <cloud-installer> <landscape> <lxc> <juju-core:In Progress by adam-collard> <https://launchpad.net/bugs/1354685>
<sparkiegeek> sinzui: yes :)
<natefinch> alexisb: I know that's not helpful
<sinzui> sparkiegeek, excellent...CI experiences this issue often.
<Beret> sinzui, yes
<dimitern> Beret, sparkiegeek, alexisb, https://github.com/juju/utils/pull/22 reviewed
<alexisb> dimitern, you rock, thanks!
<sparkiegeek> alexisb: hear hear!
<sparkiegeek> dimitern: thanks, first reaction: awesome review :)
<hazmat> sinzui, is there any trick to building 1.20 branch.. its been broken for me for a while.. here's my latest attempt.. http://paste.ubuntu.com/8046050/
<dimitern> sparkiegeek, cheers :)
<natefinch> hazmat: bzr branches have broken on me a few times over the past few months.  if you delete /home/kapil/src/launchpad.net/gnuflag and go get it again, it should be fine
<natefinch> (and then rerun godeps)
<hazmat> natefinch, that did the trick
<hazmat> thanks
<sinzui> hazmat, natefinch speaks the truth. I have also changed to each branch and pulled the lastest. godeps doen't pull, so if the rev isn't in the repo, it fails
<hazmat> sinzui, yeah.. i had been manually doing bzr pull but it kept outputting no changes.. too late to debug more now.. but good to know the workaround.
<dimitern> sinzui, hazmat, I have a couple of nice scripts to run godeps and update automatically what's needed - I'm sending a mail to juju-dev in case you might be interested
<hazmat> dimitern, sounds like a great merge request for a makefile ;-)
<natefinch> sinzui, dimitern, hazmat: current godeps -u *will* fetch new revisions from the remote repo
<hazmat> natefinch, *should*
<natefinch> sinzui, dimitern, hazmat: sometimes you have to do run godeps more than once
<dimitern> natefinch, oh really?
<dimitern> natefinch, yep, my script takes care of that
<hazmat> natefinch, didn't work for me.. and running multiple times .. with intermittent failures.. ick
<natefinch> hazmat: one time can't possibly always work
<hazmat> natefinch, pull before update seems sane to me
<natefinch> hazmat: actually... strike that
<natefinch> hazmat: yeah, the new code does that with -u
<natefinch> hazmat: where "new" = a month or so ago
<hazmat> hmm.. mine is pretty old (~march).. does the new version support comments
 * hazmat updates
<natefinch> hazmat: that should fix the problem of not pulling down updates before trying to set the current commit
<rogpeppe> when i use juju switch, i see:
<rogpeppe> ERROR couldn't read the environment
<rogpeppe> this seems to have broken some time relatively recently
<rogpeppe> ha, it should really print the actual error
<rogpeppe> ah, that's better:
<rogpeppe> ERROR couldn't read the environment: cannot parse "/home/rog/.juju/environments.yaml": YAML error: found character that cannot start any token
<hazmat> rogpeppe, yup.. recent yaml changes
<ericsnow> rogpeppe: when you have a minute could you take a look at https://github.com/juju/utils/pull/19?
<hazmat> rogpeppe, i had the same issue earlier on trunk. needed to quote some strings
<hazmat> in my jenv file
<rogpeppe> hazmat: actually, it seems i accidentally had a tab at the start of my environments.yaml
<hazmat> rogpeppe, oh. i had this one .. $ juju status -> error unmarshalling "/opt/juju/environments/ocean.jenv": YAML error: resolveTable item not yet handled: < (with <root>=DEBUG;unit=DEBUG)
<rogpeppe> hazmat: (i think it was probably me experimenting with whether yaml is fully JSON-compatible)
<katco> rogpeppe: sorry about that; yesterday i put us on a newer version of goyaml.
<rogpeppe> katco: it's not your fault
<rogpeppe> katco: it's the fault of whoever printed that error without including the actual cause...
<katco> hazmat: i received a _lot_ of those errors on goyaml head, so i used an old commit that targetted a fix we wanted. didn't look into it too much
 * rogpeppe does not run git blame.
<rogpeppe> katco: oh, interesting.
<rogpeppe> katco: we're not using gopkg.in/yaml.v1 tip?
<hazmat> katco, fair enough.. i'm  glad for the underscore stripping fix.. just worried about users in the field who are going to encouter this when they upgrade
<katco> rogpeppe: we are not. lots of tests failed with hazmat's error
<katco> hazmat: did you figure out what it was?
<rogpeppe> hazmat, katco: i'm guessing it's a quoting error on output
<rogpeppe> perhaps it was not adding the correct quotes for strings containing "<"
<hazmat> katco, we're writing out jenv with    logging-config: <root>=DEBUG;unit=DEBUG
<rogpeppe> hazmat: yup
<hazmat> katco, but it couldn't be read without quoting it "<root>=DEBUG;unit=DEBUG"
<dimitern> hazmat, sinzui, mail sent (Running godeps -u dependencies.tsv easily)
<hazmat> what rogpeppe said ;-)
<katco> hazmat: rogpeppe: ah i see... so we're causing our own failure with automatied output? is that going to be an issue for 1.20.4
<katco> ?
<hazmat> katco, very likely
<katco> hazmat: frown. so we need to fix that before i backport?
<natefinch> gah, this is why yaml is a PITA
<hazmat> katco, ie. if a user has existing an envs, and upgrades, they can't use juju without manually editing their jenvs..
<rogpeppe> hazmat: interesting pyyaml also prints that string unquoted
<rogpeppe> hazmat: (actually i think it might be the same original C code base)
<hazmat> rogpeppe, yeah.. but it parses it fine.
<katco> hazmat: yeah that's not a good experience at all.
<hazmat> hmm.. i should verify that with trunk using godeps.. its possible i accidentally just ran with the binary produced from go get.. since it actually worked and i wanted trunk of juju.
<rogpeppe> hazmat, katco, niemeyer: yes, it's definitely a bug in gopkg.in/yaml.v1 tip
<rogpeppe> this fails: http://paste.ubuntu.com/8046246/
<rogpeppe> and it should not
<rogpeppe> as it's producing output that it cannot parse
<katco> rogpeppe: sounds like it might be a bug further back in history as well?
 * rogpeppe bisects
<katco> https://github.com/go-yaml/yaml/commit/1418a9bc452f9cf4efa70307cafcb10743e64a56#diff-d41d8cd98f00b204e9800998ecf8427e
<katco> is the version trunk is using (and what we're wanting to backport to 1.20)
<hazmat> katco, yeah.. with the pinned version in dependencies.tsv things work as expected
<hazmat> so no issue there
<katco> hazmat: ahh great!
<katco> hazmat: sounds like i made the correct decision ;)
<hazmat> indeed :-)
<rogpeppe> katco: you definitely did
<rogpeppe> looks like the problem was introduced with 72c33f6840f49f9ed7d1faef7562b3266640fdf4
<rogpeppe> which is not surprising, as https://github.com/go-yaml/yaml/issues/1 shows a < being used as a special char
<rogpeppe> once again, yaml is too complex for its own good
<rogpeppe> hazmat, katco, niemeyer: https://github.com/go-yaml/yaml/issues/24
<katco> rogpeppe: thank you kindly rogpeppe
<rogpeppe> katco: np
 * perrito666 suddenly notices he cannot ssh from state server into agents bc so straightforward as he thought
<bodie_> fwereade_, if/when you're available, could you comment on https://github.com/juju/juju/pull/415 ? I think it's good to go but it still has the overwrite behavior, which seems simpler to me and more like what is probably desired
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1356899
<sinzui> hi natefinch hazmat identified a regression between 1.20 and 1.21. This isn't a blocking bug because CI didn't find it. Can you help direct an engineer to look at bug 1356899
<mup> Bug #1356899: manual provider add-machine fails if no ubuntu user on machine <manual-provider> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1356899>
<natefinch> sinzui: interesting, ok
<niemeyer> rogpeppe: Thanks, I'll have a look
<gsamfira> perrito666: try adding: "ForwardAgent    yes" on your machine in ~/.ssh/config
<gsamfira> helps alot
<alexisb> perrito666, I am running a few minutes late
<perrito666> no worries, I am trying to kick my camera back into life
<alexisb> perrito666, now I am waiting on my hangout
 * alexisb tries firefox
<mattyw> fwereade_, I wonder if the periodicworker should have a mechanism for working out how many times the workerFunc has been called - this could just built into whatever implements it I guess
<fwereade_> mattyw, ehh, we'll build it when we need it
<fwereade_> bodie_, tab opened -- and overwriting in general sgtm, I don't think I was arguing against that, I'll take a look
<mattyw> fwereade_, I *might* find it useful in the cleanup worker test I'm writing so that I know that a cleanup has been run
<mattyw> fwereade_, I'll type some things ans see how it *feels*
<fwereade_> mattyw, sgtm
<fwereade_> mattyw, it'll need to be goroutine-safe
<bodie_> fwereade_, cool, thanks.  action-fail should be ready in a few minutes too
<bodie_> fwereade_, PR 520 up
<alexisb> ericsnow, team meeting?
<alexisb> fwereade_, team meeting?
<mattyw> I'm calling it a night then folks, see you all tomorrow
<alexisb> bye mattyw !
<waigani> natefinch: found your comment - usually I get notified by email. Github must not notify for closed PRs...
<natefinch> waigani: ahh, dang.  I'll remember that for the future
<sinzui> :(
<sinzui> we have a utopic regression.
<natefinch> *sad trombone*
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1356899 1357033
<sinzui> natefinch, Can you pass this on to an engineer: https://bugs.launchpad.net/juju-core/+bug/1357033
<mup> Bug #1357033: sourcesSuite.TestGetFilesToBackup consistently fails on utopic <ci> <regression> <test-failure> <utopic> <juju-core:Triaged> <https://launchpad.net/bugs/1357033>
<natefinch> sinzui: yep
<alexisb> wwitzel3, ready when you are for our 1x1, if you would like to meet early
<wwitzel3> alexisb: sure :)
<jrwren> how do I get relation data from outside a relation hook?
<jrwren> relation-ids says error: no relation name specified
<jrwren> relation-list says error: no relation id specified
<jrwren> nevermind. consider ^^ that my rubber duck session.
<jrwren> relation-ids <relation name> works fine. I typoed :(
<waigani> natefinch: https://github.com/juju/juju/pull/521
<waigani> natefinch: just reverted for now, I'll look at fixing the bug today and propose a new branch
<waigani> natefinch: oh and notification of your comment did come through, my mail client is just slow
<katco> https://github.com/juju/juju/pull/522 ready for review
<katco> one of the last blockers for 1.20.4 (though i saw we found something new :(  )
<natefinch> katco: lgtm'd
<katco> natefinch: ty, sir
<natefinch> katco:  welcome :)
<ericsnow> natefinch: FYI, I'm planning on talking to Ian and Martin tonight about reviewboard.
<natefinch> ericsnow: awesome, thanks for spending time after hours to do so
<ericsnow> natefinch: no worries.  I really want to get this moving. :)
<natefinch> ericsnow: awesome
<natefinch> EOD for me
<waigani> thumper: so instance.Instance  has a Ports function, asks for a machine ID. I could call that from the bootstrap instance, pass in "0" for id.
<thumper> nuh
<thumper> not what you want
 * thumper gets back to reviewing...
<waigani> thumper: environ.Ports() then?
<thumper> hang on
<waigani> thumper: another question: should I add a new address for each port returned i.e. ["localhost:1234","locahost:5678"]
 * thumper ignores waigani for a minute
<waigani> hehe
<thumper> waigani: if you read the comment on the environ.Ports method, it is pretty clear that it isn't what you want...
#juju-dev 2014-08-15
<hazmat> wallyworld, ping
<wallyworld> hazmat: otp, one sec
<thumper> time to wrap up warm and take the dog for a walk
<davecheney> thumper: still snowing ?
 * hazmat wonders if the dog gets wrapped
<davecheney> http://www.nancysdogpawsboutique.com/images/products/Italian%20Greyhound%20English%20Style%20Fleece%20Dog%20Coat%20brown%20001.jpg
<davecheney> sinzui: is wolfe-01 you box ?
<thumper> hazmat: no, not that cold, dog is fine
<thumper> davecheney: not snowing, just cold
<davecheney> thumper: this one is for you, https://github.com/juju/loggo/pull/3
 * thumper looks
<thumper> davecheney: what is that?
<thumper> godoc badge
<axw> just a link to the godoc page for loggo
<thumper> ah...
<thumper> ok
<davecheney> waigani: are you working on https://bugs.launchpad.net/juju-core/+bug/1356899 ?
<mup> Bug #1356899: manual provider add-machine fails if no ubuntu user on machine <manual-provider> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1356899>
<waigani> davecheney: the revert branch has merged
<thumper> waigani: did it? I thought it was rejected
<waigani> Pull request successfully merged and closed
<waigani> You're all setâthe waigani:revert-manual-prov  branch can be safely deleted.
<thumper> ok, good
<waigani> davecheney: ^ from github
<davecheney> waigani: please assign yourself to the issue and mark it in progress
 * thumper scrolls up
<thumper> oh, there it goes
<davecheney> if it's fixed, mark it as fix-comitted to unblock the build
<thumper> actually, mark it fix committed
<waigani> done
<thumper> waigani: are you going to re-propose without the deleted line?
<waigani> thumper: yep
<wallyworld> axw: finished
<thumper> coffee needed
<hazmat> coffee finished
<hazmat> waigani, btw.. if you want a functional test case  for that regression try... using juju digitalocean plugin
<waigani> thumper: axw: https://github.com/juju/juju/pull/524
<thumper> hazmat: do DO machines not have the ubuntu user?
<hazmat> thumper, no ubuntu image thats not cpc cloud does
<thumper> EPARSE
<thumper> really?
<thumper> so, since DO are not CPC, they have no ubuntu user?
<thumper> why is that?
<hazmat> thumper, you've installed ubuntu desktop
<thumper> what do they have instead?
<hazmat> they have a sane version of ubuntu server
<thumper> they need some user right?
<hazmat> thumper, yeah... its called root
<axw> waigani: LGTM, but please test it live
<thumper> really?
<thumper> ew
<waigani> axw: okay, will do now.
<thumper> :-)
<hazmat> thumper, ew.. like a user who can admin stuff without typing sudo..
<thumper> yeah, yeah
<davecheney> scott/tiger
 * hazmat bows to the oracle 
 * thumper missed that reference
<hazmat> thumper, default/doc'd/example'd password on oracle
<davecheney> thumper: http://xforce.iss.net/xforce/xfdb/970
<thumper> ?!
<thumper> heh
<menn0> davecheney: do you have time to cast your eyes over this again? https://github.com/juju/juju/pull/508
 * davecheney looks
<waigani> axw: $ juju add-machine ssh:scott@10.0.3.244
<waigani> scott@10.0.3.244's password:
<waigani> [sudo] password for scott:
<waigani> install: cannot create directory /home/ubuntu: Permission denied
<waigani> ERROR subprocess encountered error code 1 (Connection to 10.0.3.244 closed.)
<waigani> axw: scott is in the sudoers file
<waigani> axw: I just ran the same with trunk, I get same result
<axw> waigani: sorry was afk having lunch
<waigani> axw: np, I've commented on PR
<axw> waigani: can you confirm you can "sudo mkdir /home/ubuntu" on the machine when logged in as scott?
<waigani> axw: yep I could
<waigani> axw: but it asks for scott's password
<axw> waigani: add-machine should prompt you for that, is it not doing so?
<waigani> axw: yep it did
<waigani> axw: same behaviour on trunk btw
<axw> waigani: I'll try to repro, sounds odd
<waigani> axw: that would be great, thanks
<axw> waigani: I cannot reproduce on master. did you remove /home/ubuntu? userdel does not do that by default
<ericsnow> Here's a patch that fixes the blocking bug if anyone has a minute: https://github.com/juju/juju/pull/525
<ericsnow> mgz_: you have a minute?
<thumper> ericsnow: done
<ericsnow> thumper: thanks
<ericsnow> wallyworld: ping
<wallyworld> ericsnow: hey
<ericsnow> you have a minute to talk about reviewboard
<wallyworld> ericsnow: sure, hangout?
<ericsnow> sure
<wallyworld> ericsnow: https://plus.google.com/hangouts/_/canonical.com/tanzanite-stand
<waigani> axw: does it work for you?
<axw> axw> waigani: I cannot reproduce on master. did you remove /home/ubuntu? userdel does not do that by default
<waigani> axw: yes
<waigani> strange
<axw> *shrug*, just worked for me
<axw> please look into what's going on there, because it doesn't sound right
<waigani> yep
<waigani> axw: just to confirm, 'scott' just needs to be in the sudoer's file
<axw> waigani: well, I just s/ubuntu/scott/ in the juju file in /etc/sudoers.d
<waigani> axw: did you delete or rename home dir?
<axw> waigani: I deleted /home/ubuntu
<waigani> axw: I did sudo adduser scott sudo
<axw> waigani: I'll try again with that method.
<waigani> axw: and I'll try swith your
<waigani> axw: ah
<waigani> axw: it's still ubuntu in sudoers.d
<axw> waigani: ?
<axw> waigani: it should be. if you just did "adduser scott sudo" that's just adding it to the group
<waigani> axw: right, I thought that was enough
<axw> waigani: what that doesn't do is enable passwordless login
<axw> but if you have a password that should be fine
<waigani> axw: scott has a password and I gave it to juju add-machine
<axw> waigani: works for me using that approach too.
<axw> waigani: my /home is 0755
<waigani> axw: and *not* changing /etc/sudoers.d ?
<axw> waigani: correct, I just did "sudo adduser scott sudo" and used the password
<waigani> hmph
<waigani> axw: I can't reproduce it now, everything seems to be working as expected.
<axw> hrm ok.
<axw> well, I say land it. CI and/or hazmat will complain if it's still broken ;)
<waigani> okay, I'll jfdi it
<voidspace> morning all
<dimitern> morning voidspace
<voidspace> o/
<TheMue> morning
<voidspace> TheMue: morning
<stub> Reading the pain points spreadsheet, and I was thinking a manifest file in the charm, listing one or more URIs and the path to install or unpack them, could solve several items. It that the sort of thing already being talked about?
<dimitern> morning TheMue
<TheMue> dimitern: heya, seen my mail?
<dimitern> TheMue, yep, I haven't had time to reply yet, but I'll get back to you soon (I have a call with Mark S in 15m)
<TheMue> dimitern: ok, np, and thanks already
<TheMue> Hmmmpf, shitty net, currently it dislikes me.
 * fwereade_ is on a public holiday and not working
 * fwereade_ would still like to express his frustration at not being able to express a `chan foo` as a `chan interface{}`
<dimitern> TheMue, I think we need a branch to expose all capabilities on the apiserver/agent facade
<TheMue> dimitern: I found a way
<dimitern> TheMue, including the new one; no need to expose the environ config, just the part of the interface defined as state.EnvironCapabilities
<TheMue> dimitern: Iâll paste the part, one moment
<dimitern> TheMue, the provisioner and upgrader api facades have access to the environ config, but that's a hacky approach
<TheMue> dimitern: http://paste.ubuntu.com/8052638/
<TheMue> dimitern: Iâm using the config too
<dimitern> TheMue, ah, this looks nice
<dimitern> TheMue, cheers
<TheMue> dimitern: so right now the providers only return constants, now the second part letting them return real values
<dimitern> TheMue, what constants?
<TheMue> dimitern: as I understand the local bootstrap as well as manual and manually provisioned machines on other providers need the safe networker
<dimitern> TheMue, yes, all the other cases (maas, ec2, etc.) will use the normal networker
<TheMue> dimitern: true, but nothing Iâll push before the change descibed above. only wanted to see it compiling
<TheMue> dimitern: do manually provisioned machines exist on maas too? today this isnât covered
<dimitern> TheMue, to make it reusable, I'd add a method to the agent that returns state.EnvironCapability, and internally uses the st.Environment().EnvironConfig() + config.New, etc.
<dimitern> TheMue, manually provisioned machines can exist regardless of the provider the environment uses
<TheMue> dimitern: ok, sounds good
<dimitern> TheMue, cheers
<dimitern> voidspac_, standup?
<voidspac_> dimitern: sorry!
<voidspac_> dimitern: TheMue: omw
<dimitern> mgz_, hey
<mgz_> hey
<dimitern> mgz_, so you're switching to the juju ci team?
<mgz_> yup
<dimitern> mgz_, congrats! :)
<dimitern> mgz_, that was kinda sneaky hehe
<katco> good morning, team :)
<mgz_> hey katco
<katco> congratulations on the move, mgz_ :)
<wallyworld> katco: hey, i'm back from soccer if you wanted to chat
<katco> wallyworld: you're pretty much up to speed w/ what i'm doing. but we can definitely chat if you'd like :)
<wallyworld> katco: only if you need to, happy to let you keep going... :-)
<katco> wallyworld: $$greatful it landed$$
<katco> err *grateful
<wallyworld> :-)
<katco> i do have one question: did you win? ;)
<wallyworld> katco: nope, our stikers kept missing the net :-(
<katco> aw bummer
<katco> what position do you play?
<wallyworld> left right out :-)
<wallyworld> centre back
<katco> ah! that was my old position :) sweeper/left fullback
<katco> <3 defense
<wallyworld> i like that position, less running
<katco> haha
<wallyworld> plus i read the game well, so it suits me
<katco> yeah it always struck me as an analytical position
 * wallyworld wonders if fwereade_ is around
<TheMue> wallyworld: public holiday, but has been around this morning once
<wallyworld> TheMue: tanks
<wallyworld> thanks
<TheMue> yw
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None
<mattyw> fwereade_, I'm working on that api to get a list of metrics if you fancy doing some bikeshedding with me ;)
<jrwren> who are the eco folks?
<ericsnow> mgz_: ping
<mgz_> ericsnow: hey
<ericsnow> mgz_: just following up on my reviewboard email :)
<mgz_> yeah, 's in my queue of many emails to respond to...
<ericsnow> mgz_: no worries (wanted to be sure it didn't get lost in the ether)
<mgz_> ericsnow: short answer is past the charm and some minor changes, most of the work is in just deploying the thing
<ericsnow> mgz_: awesome!
<mgz_> so, ther's not *much* to pass over, one thing to sort out is where to host it
<ericsnow> mgz_: I was considering pinging Curtis about that (after I'd gotten info from you)
<alexisb> ericsnow, on and ready when you are
<ericsnow> alexisb: brt
<katco> alexisb: hey!
<sinzui> hi katco ericsnow do either you have a minute to review https://github.com/juju/juju/pull/526
 * katco looking
<ericsnow> sinzui: sure
<katco> sinzui: looks benign to me, although i'm not sure exactly what
<katco> hangs off of version.Version
<ericsnow> sinzui, katco: yeah, same here
<sinzui> ericsnow, katco Both version changes define the internal version juju or the installer reports. It is also the version that packaging will adopt
<sinzui> I make this change after ever release, though since I haven't really released, I am a little disgruntled I need to make this change
<ericsnow> sinzui: why don't we use some tagging mechanism in our versions (e.g. 1.20.0-a1, 1.20.0-rc3)?
<katco> ericsnow: +1
<natefinch> don't we do that?  we have like 1.20.0-alpha
<katco> ericsnow: i mean presumably versions are free, but there's a cognitive overhead people have when you jump from 1.20.2 to 1.20.4 or 1.20.5
<ericsnow> natefinch: not if version/version.go has anything to say about it
<sinzui> natefinch, we cannot go back in time to tell 1.18 not to panic when it sees an alpha in the version
<ericsnow> sinzui: yuck
<natefinch> sinzui: oh, I see... so for 1.20.x we can't use our fancy new system for alphas, but for the release after, we can, correct?
<katco> sinzui: so if we wanted to make that change, we need a major point release?
<sinzui> natefinch, yep, all this pain of internal testing wouldn't be happening if we were already on 1.21
<natefinch> sinzui: can we just call the next release 1.21? :)
<ericsnow> sinzui: ah, it's just a 1.20 issue
<sinzui> natefinch, well, we could all agree to abandon 1.20.x dev becomes 1.22-alpha1 and stable becomes 1.21-rc1...because that is how we are actually treating these branches/releases
<natefinch> sinzui: version numbers aren't important to developers... but I presume there's someone else who would raise a stink if we did that
<sinzui> natefinch, they are important to debian/ubuntu. Since 1.20.x is not in either, I don't think anyone would notice
 * sinzui is not sure how to announce a major version that doesn't offer new features
<natefinch> sinzui: "hey guys, this makes my life 100x easier and will mean we get releases out faster"  ... I think everyone will understand.
<katco> sinzui: it supports v4 AWS signing i.e. China region :)
<natefinch> there, see?
<katco> not that i have enough information to have a horse in this race, but there's that
<katco> _although_ we did not have a china instance to test against
<katco> so i would hate to call that out as a major value-add and then have it not work (even though all signs say that it will)
<natefinch> sinzui: the new feature is that dev revisions are now -alpha :)
<katco> haha
<sinzui> natefinch, the dev features really are alpha. Given the problems getting them to pass, I think we can say we are a month away from a  beta
<sinzui> i even have plans to release an alpha until we have sorted out the utopic stable release
<katco> sinzui: hey is there a doc that describes our release strategy? versions, process, etc.?
<sinzui> katco, not versions since we just switched the process.
<katco> sinzui: ah ok. i'd love to grok that at some point (like i don't already have enough to learn) :p
<sinzui> katco, CI tests the release process so this doc and all the test scripts document what must happen to be a blessed https://docs.google.com/a/canonical.com/document/d/1o0au1ZNPpA8sGBNj2yNgOV3PH7Oxpyqw3hQNuIFaS-w/edit
<sinzui> katco, This template illustrates the automated and manual steps of a release https://docs.google.com/a/canonical.com/document/d/1aPOHa_JNKjKd5lAcOlFahjbNl8V_KgtMx6rgYdC2f7k/edit
<katco> sinzui: cool, thank you!
<sinzui> katco, this is the log of the 1.20.1 release which is a little easier to read https://docs.google.com/a/canonical.com/document/d/1BOWKU4Iqy363pp4w8IWBwabDT2TdA9fb4futQVjtuVQ/edit
<natefinch> katco: odd minor versions used to be dev releases, even minor versions were stable.  But luckily we realized that was dumb and non-obvious, so we switched to "anything N.N.N is stable, anything N.N.N-<some_words> is a dev release"
<katco> natefinch: ah ok
<natefinch> katco: and stable releases only upgraded to new stable releases, and dev->dev.  so 1.20 is the cross-over, where it knows about the new naming scheme, but 1.18. which wants to upgrade to 1.20, doesn't.
<katco> natefinch: gotcha. thus we can't fiddle with versions in 1.20 b/c 1.18 wouldn't know what to do?
<natefinch> katco: correct
<katco> makes sense
<jcw4> TheMue, axw https://github.com/juju/juju/pull/527 PTAL :)
<katco> fun friday fact: i just realized i have 3 8 core machines at my disposal here at home. that's quite nice :)
<katco> i wonder if i could set up my own juju ci to continually run tests as i commit so it's not a big bang at the end :p
#juju-dev 2014-08-16
<natefinch> morning fwereade_
<fwereade_> natefinch, heyhey
<natefinch> fwereade_: so, do we have a mechanism for charms to say what version of juju they're compatible with?  My guess is the answer is "no" and that some people will have a heart attack at the thought.
<fwereade_> natefinch, so, I wrote this a while back: https://docs.google.com/a/canonical.com/document/d/1lV6Vlqj3VlOXPLZD1T4vV865Ab16YgIJivyWnZXdhCo/edit
<fwereade_> natefinch, it seemed pretty uncontroversial -- but equally we've, uh, ocmpletely failed to explicitly schedule it
<fwereade_> natefinch, and, yeah, it's important for a lot of things
<natefinch> fwereade_: seems like it
<fwereade_> natefinch, so, dammit, we need to get it done somehow, and probably backported to older versions
<natefinch> fwereade_: well, my team is doing the charmers pain points, seems like it could go to us.  Doesn't seem terribly difficult from a 10,000 foot view.
<fwereade_> natefinch, I don't think it should be
<fwereade_> natefinch, there will probably be some devils in the details
<natefinch> fwereade_: there usually are.
<fwereade_> natefinch, in particular it onvolves charm store work
<fwereade_> natefinch, which will need some extra coordination
<natefinch> fwereade_: hmm... that's unfortunate.  I had hoped we could get away with keeping the changes only in core.... but yeah, I guess not.
<fwereade_> natefinch, ok, so, I think this is a prerequisite for a bunch of things your team will want to be involved with
<fwereade_> natefinch, when do you think you can get it scheduled? and would you start a conversation with rick_h__ (?) about how to coordinate the charm store stuff?
<natefinch> fwereade_: I'm going to start on it ASAP.  Since it's required for the default-hook stuff, I might as well get going on it right now.
<natefinch> fwereade_: I'll coordinate with Rick on it.
<fwereade_> natefinch, cool
<fwereade_> natefinch, do a first pass over the spec looking for places I get it completely hilariously wrong
<fwereade_> natefinch, there will almost certainly be at least one
<fwereade_> natefinch, you've aleady got edit on that doc I think
<natefinch> fwereade_: yep, looks like it.  Thanks.
<natefinch> fwereade_: gotta run. Will keep you up to date, though.
<rick_h__> fwereade_: cool on the feature flags, will comment on the doc some
#juju-dev 2014-08-17
<thumper> morning folks
<mwhudson> thumper: you haven't turned into a zombie then?
<thumper> no... survived with all my lives
<hazmat> thumper, fully encrypted email to public list?
<thumper> hazmat: yeah, no idea what happened there
 * thumper is trying to work it out
<thumper> I can't even open the item in my sent mail :-(
<thumper> I wrote heaps
<thumper> damn
<hazmat> fwereade_, i'm not convinced that the something-changed hook is actually a step forward compared to the default-hook impl.. the former requires rewriting charms with an entirely different frame of mind to utilize, while the later is a simple optimization that removes tedium.
<hazmat> although both need a feature flag
<rick_h__> thumper: defeated by the gpg :P
<thumper> rick_h__: yeah...
 * thumper goes to make another coffee
<thumper> lol - Panic: Invalid user tag "not touched" (PC=0x414676)
<thumper> panic: Invalid user tag ""
<thumper> gnn,,,
<thumper> hmm... even
<davecheney> thumper: waigani menn0 https://github.com/juju/juju/pull/531
<menn0> davecheney, thumper, waigani: and here's mine https://github.com/juju/juju/pull/508
<menn0> davecheney: looking at yours now
<thumper> hmm...
<wallyworld> thumper: interesting lxc issue in bug 1357552 - some but not all containers can't see the http server on port 8040 which is used to provide tools into the containers via curl. sigh
<mup> Bug #1357552: lxc containers created, juju can't seen or communicate with them <landscape> <juju-core:Triaged> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1357552>
<thumper> I'm wanting the git equivalent of 'bzr diff -r -2..-1'
<thumper> I want the diff of the last merge
<thumper> git diff HEAD^ HEAD doesn't quite do it
<thumper> as it shows the last commit, not the last merge diff
<thumper> any git gurus?
<thumper> wallyworld: that's weird
<thumper> wallyworld: sounds like an lxc networking problem
<wallyworld> we have a few lxc issues at the moment
 * thumper nods
<wallyworld> :-(
<wallyworld> i could understand if all containers failed
<wallyworld> but not some
<wallyworld> thumper: i'm thinking we should just mount a host dir into the container and grab the tools from there rather than a http server
<davecheney> wallyworld: why not use wget -c
<wallyworld> i wonder if that will work for nested
<menn0> thumper: does "git show -m <sha>" give you what you want?
<davecheney> then it'll loop automagically
<thumper> wallyworld: I'd really prefer if we didn't treat lxc too specially
<thumper> but I can see the desire
<wallyworld> davecheney: i have no idea, someone else used curl
<thumper> menn0: shows the last commit
<thumper> not the last merge
<wallyworld> thumper: lxc is already special - there's a big hack in machine agent to start the http server :-(
<thumper> wallyworld: yeah, but we are changing that aren't we?
<thumper> we want machine 0 to be a container
<thumper> any change like this just makes that move harder
<wallyworld> thumper: sure, but we do need a way to get stuff passed in
<thumper> agreed
<thumper> but lets not make it hard for us to move where we want to go
<wallyworld> how is mounting a host dir making it hard? don't we want to do that anyway for charm develppment?
<menn0> thumper: does this help? http://stackoverflow.com/a/7335580
<thumper> wallyworld: yes we do (ish)
<thumper> hah... found out why we are now panicing making user tags
<thumper> in the past, when we created a user tag from a string, there was no validation
<thumper> now there is
<davecheney> thumper: BOMM!
<davecheney> names.NewXXXTag is only for testing
<davecheney> or when you are very very sure that what you are passing it is a valid tag
<davecheney> in which case, you should probably just use names.ParseXXXTag
<thumper> should still be valid though
<davecheney> what is the line ?
<thumper> environs/cloudinit_test.go line 49 ish
<thumper> although I have another
<thumper> Tag:        names.NewUserTag(info.APICredentials().User),
<thumper> from juju/api.go
<thumper> seems that the user is blank
<thumper> blank isn't a valid tag
<thumper> s/tag/user/
<thumper> grr
<davecheney> bingo
<davecheney> thumper: why do the apiserver tests embed another test suite ?
<davecheney> type rsyslogSuite struct { testing.JujuConnSuite *commontesting.EnvironWatcherTest
<thumper> NFI
<davecheney> the embedding with a pointer means that the EnvironWEatcherTets are run when the rsyslogSuite tests are rung
<davecheney> run
<thumper> ah...
<thumper> yes, that is by design
<davecheney> continue
 * thumper thinks back
<davecheney>         s.EnvironWatcherTest = commontesting.NewEnvironWatcherTest(
<davecheney>                 api, s.State, s.resources, commontesting.NoSecrets)
<davecheney> oh
<thumper> there were situations where each other test had a copy of the common test functions
<davecheney> this is somet facade thing
<thumper> I consolidated them
<thumper> yeah
<davecheney> hmm
<davecheney> i still don't see what it does
<thumper> it isn't great
<thumper> it repeats a lot of stuff
<thumper> but it is better than it was
<davecheney> well
<davecheney> i sort of do
<thumper> I agree it isn't correct yet
<thumper> just 'less shit'
<davecheney> 'cos it is this logic that does the canAuth("") logic
<davecheney> lucky(~/src/github.com/juju/juju/state/apiserver) % grep -e 'can.*(\"' -r *
<davecheney> common/environwatcher.go:       if !canWatch("") {
<davecheney> common/environwatcher.go:       if !canReadSecrets("") {
<davecheney> common/environmachineswatcher.go:       if !canWatch("") {
<davecheney> provisioner/provisioner.go:     if !canWatch("") {
<davecheney> the best i can make from this logic is
<davecheney> if !anyone is allowed to watch ...
<menn0> davecheney: review done
<davecheney> ta
<wallyworld> thumper: you are assigned to bug 1320543, are you working on it?
<mup> Bug #1320543: debug-log uses internal names for filtering <debug-log> <usability> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1320543>
<thumper> wallyworld: no
<wallyworld> ok
 * wallyworld unassigns
#juju-dev 2015-08-10
<axw> thumper: can you please review https://github.com/juju/juju/pull/2888? apparently a RB review never got created
<thumper> ack
<thumper> axw: on some calls for a while, but I'll get to it
<axw> thumper: no rush, thank you
<mup> Bug #1483082 opened: provider/maas: volume source won't work for physical machines/disks <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1483082>
<mup> Bug #1483083 opened: worker/storageprovisioner: unnecessary Attach just after a Create that creates attachment <juju-core:Triaged> <https://launchpad.net/bugs/1483083>
<mup> Bug #1483082 changed: provider/maas: volume source won't work for physical machines/disks <maas-provider> <juju-core:In Progress by axwalk> <juju-core 1.24:Triaged by axwalk> <https://launchpad.net/bugs/1483082>
<mup> Bug #1483083 changed: worker/storageprovisioner: unnecessary Attach just after a Create that creates attachment <juju-core:Triaged> <https://launchpad.net/bugs/1483083>
<mup> Bug #1483082 opened: provider/maas: volume source won't work for physical machines/disks <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1483082>
<mup> Bug #1483083 opened: worker/storageprovisioner: unnecessary Attach just after a Create that creates attachment <juju-core:Triaged> <https://launchpad.net/bugs/1483083>
<mup> Bug #1483086 opened: storage/provider: managedfs fails to create fs on full disks <juju-core:In Progress by axwalk> <juju-core 1.24:Triaged by axwalk> <https://launchpad.net/bugs/1483086>
<davecheney> thumper: ping
<axw> wallyworld: can you please review this fairly trivial PR: https://github.com/juju/utils/pull/147
<wallyworld> sure
<axw> wallyworld: I bit the bullet and added a common clock package/interface
<wallyworld> yay
<axw> wallyworld: btw I got disks working on azure, but I need to spend some more time figuring out the best way to represent volume IDs so we can attach/detach. as usual, the representation of resources is not like other providers... :)
<wallyworld> sigh, love azure
<axw> also need to make some changes to prevent disks being destroyed when machines are destroyed
<wallyworld> good news on the progress though :-)
<axw> wallyworld: when you have time, I've updated http://reviews.vapour.ws/r/2298/ to address your comments
<wallyworld> looking
<wallyworld> axw: lgtm, ty for adding clock implementation
<axw> wallyworld: cool, thanks
<menn0> thumper: here's the majority of the state changes for the cross env watcher: http://reviews.vapour.ws/r/2328/
<menn0> thumper: there's still a few little bits coming but it's worth getting this reviewed and merged at this point
<axw> wallyworld: would it be reasonable to only support non-persistent volumes in the current version of the azure provider?
<axw> wallyworld: it'll avoid some complication around deleting VMs and possibly leaking disks
<wallyworld> axw: i think so, we can deliver that and then see where we stnd wrt the new apis etc
<axw> cool, saves a bunch of work
<wallyworld> although if we were to be able to list disks that have been allocated, we can then clean them up
<wallyworld> we can do that as a stretch goal perhaps
<axw> wallyworld: we can, but the problem is more to do with OS disks. when you delete a VM you either delete *all* the attached disks or *none* of them. we can do it, but it's just a bit messy... so yes, stretch goal
<wallyworld> so long as limitations are documented...
 * thumper out
<dimitern> axw, (if still around), voidspace, mgz, can you please review this https://github.com/go-amz/amz/pull/60 ?
<voidspace> dimitern: wow, that's big
<voidspace> dimitern: I can take a look - but it will take a while
<voidspace> dimitern: the stuff in .gitignore - can't you get your git to globally ignore those
<voidspace> dimitern: rather than polluting the project with emacs
<dimitern> voidspace, thanks
<voidspace> :-)
<dimitern> voidspace, perhaps I can but don't know how :)
<dimitern> voidspace, that .gitignore is more or less the same as juju/juju/.gitignore
<voidspace> dimitern: heh, ok
<voidspace> dimitern: https://help.github.com/articles/ignoring-files/
<voidspace> git config --global core.excludesfile ~/.gitignore_global
<dimitern> voidspace, good to know, thanks! :)
<dimitern> voidspace, if it makes you feel better there are both vim and emacs rules in there :P
<voidspace> dimitern: :-p
<dimitern> jam, hey, 1:1?
<jam> dimitern: omw
<voidspace> dimitern: I think I found a bug in srv.SetAvailabilityZones
<voidspace> dimitern: see the first comment
<voidspace> dimitern: still reading
<dimitern> voidspace, will do, in a call now
<voidspace> kk
<axw> wallyworld: https://code.launchpad.net/~axwalk/gwacl/gwacl-disks/+merge/267477 when you can, please
<dimitern> voidspace, responded to your comments so far
<voidspace> dimitern: the comment about extra methods makes sense
<voidspace> dimitern: I should have checked!
<dimitern> voidspace, np, the ec2test code is only slightly more familiar to me, but it's still confusing in some places
<voidspace> making coffee
<voidspace> brb
<dimitern> voidspace, i'm ready
<dimitern> voidspace, dooferlad, we need to sync up net-cli with master today I think
<dimitern> it's been a whil
<dimitern> while
<voidspace> yep
<voidspace> dimitern: I'm in the hangout
<dimitern> voidspace, me too  - which one are you in?
<voidspace> dimitern: hah
<voidspace> dimitern: I'm in the wrong one
<voidspace> :-)
<dimitern> :)
<voidspace> dimitern: sorry, omw
<voidspace> dimitern: so the admin space, and the routing we need for that, *is* part of the MVP I think
<voidspace> dimitern: for MAAS I think the user will need  to create a real space and setup routing themselves
<voidspace> dimitern: as we don't control routing on MAAS
<voidspace> dimitern: for EC2 we can put the routing in place
<TheMue> dimitern: you like the explicit interface more?  fine, it indeed "feels" better ;)
<dimitern> TheMue, yeah, I think it's better
<TheMue> dimitern: fine, and I found the error you talked about and will now introduce it
<dimitern> TheMue, cheers
<voidspace> dimitern: did you run the tests against live ec2?
<voidspace> dimitern: a couple more trivial comments added, nearing the end now
<voidspace> dimitern: structurally all sound, good work getting through all that
<dimitern> voidspace, yes, numerous times
<dimitern> voidspace, thanks, will respond soon
<voidspace> dimitern: right, done - no further comments
<voidspace> dimitern: LGTM, modulo those minor points
<dooferlad> dimitern: hangout?
<dimitern> TheMue, RB somehow stopped working half way through the review, so I've sent my comments so far, but I'm still on it
<dimitern> dooferlad, omw
<dimitern> voidspace, thanks!
<TheMue> dimitern: ok, good to know
<voidspace> great work
<dimitern> voidspace, thanks :)
<dimitern> TheMue, reviewed
<TheMue> dimitern: thanks
<dimitern> mgz, are you about?
<dimitern> jam, voidspace, dooferlad, I've updated the model with comments and a few text changes. Most important new bits are on the comment about Discovering VPC support at bootstrap
<mgz> dimitern: hey
<dimitern> mgz, hey, can you some time for a second review on https://github.com/go-amz/amz/pull/60 please?
<mgz> ah, I saw the mail with that over the weekend, looks interesting
<mgz> dimitern: v3 is not yet released? I'm not sure where we are on goamz versioning
<dimitern> mgz, it is released
<dimitern> mgz, that's building on top of it, v4-unstable is not released
<mgz> dimitern: isn't renaming methods a compat break then?
<dimitern> mgz, I knew you'll say that :)
<dimitern> mgz, ok, I'll add an alias SetInitialAttributes for SetAccountAttributes
<mgz> dimitern: I mean, it may well not be something we care about, do we know if anyone except juju uses ec2test?
<mgz> I'd hope they would, but most other uses of goamz I've seen are pretty limited
<dimitern> mgz, not that I know of, but the compatibility promise should be respected when possible
<voidspace> dimitern: the space definition in state is the space name plus the list of associated subnet Ids
<voidspace> dimitern: the Space type needs a Subnets method that returns a slice of state.Subnet
<voidspace> dimitern: should I fetch those Subnets at the same time the Space is fetched
<voidspace> dimitern: or ok to fetch on demand when space.Subnets is called?
<voidspace> in terms of data integrity, it makes sense to fetch them at the same time
<voidspace> but then it's extra work that has to be done when a Space is constructed - and it may not be needed
<voidspace> so from that point of view it makes sense to only fetch them when needed
<dimitern> voidspace, well, during creation we should have all of the ids already, so it should be simple
<voidspace> dimitern: it is simple - it's the same either way
<voidspace> it's just where it's done
<voidspace> dimitern: so you're suggesting at creation time
<dimitern> voidspace, and since the subnet ids of a space cannot change underneath us, fetching the along with the space is ok I think
<voidspace> dimitern: why can't they change underneath us?
<dimitern> voidspace, but if you do that, and the list of ids is not updated on Refresh(), it should
<voidspace> dimitern: isn't there a race condition between fetching the space and adding / removing a subnet somewhere else?
<voidspace> dimitern: in which case doing at creation time is better as we're less likely to hit that
<voidspace> (that's what I meant by data integrity)
<dimitern> voidspace, spaces are a juju concept, unlike the instance status
<voidspace> dimitern: ah, right
<dimitern> voidspace, so the only way to change a space is to create or modify it
<voidspace> dimitern: but two clients could do that simultaneously :-)
<voidspace> these state objects should be ephemeral though
<dimitern> voidspace, that's true :)
<dimitern> voidspace, hmm what a asec
<dimitern> voidspace, wait even
<dimitern> voidspace, subnets have space name field, right? why should we store subnet ids in the space doc?
<voidspace> dimitern: so, Space.validate is already fetching them all
<voidspace> dimitern: well, true enough
<voidspace> dimitern: denormalisation?
<voidspace> dimitern: it's nice not to have to scan all subnets to find the ones associated with a space
<voidspace> dimitern: it means adding a subnet changes the Space definition too
<dimitern> voidspace, you're not scanning them - just Find({{space_name, "foo"}}).All()
<voidspace> dimitern: how do you think mongo finds them ;-)
<dimitern> voidspace, we'll add an index
<dimitern> voidspace, it should even be unique I think
<voidspace> dimitern: I can kill the subnetIds part of the Space doc
<dimitern> voidspace, yes, if it was added for convenience, it's not correct
<dimitern> voidspace, it should be a method returning ([]*state.Subnet, error)
<dimitern> voidspace, thanks for bringing this up btw!
<dimitern> voidspace, I have to look at what's our state model so far for spaces and subnets
<voidspace> np, of course
<voidspace> dimitern: go for it
<dimitern> voidspace, I remember adding something like BackingSpace.SubnetIds, but I fully intended in *state.Space to use a method for that, not a slice on the doc
<voidspace> dimitern: yep, I'm writing the method now
<voidspace> dimitern: dammit
<voidspace> dimitern: subnet doesn't *yet* have a SpaceName
<voidspace> dimitern: I guess I have to add it
<voidspace> dimitern: there's another card for subnet changes, so I'll just add the field for now
<voidspace> dimitern: *sigh*, and because there is no SpaceName field on Subnet, the Space.SubnetIds is currently *the canonical* list of subnets for the Space
<voidspace> AddSpace takes a list of subnets
<voidspace> dimitern: so all these changes can be done as part of the subnet ticket
<voidspace> dimitern: otherwise this one balloons
<dimitern> voidspace, yes, please - add SpaceName on subnetDoc
<dimitern> voidspace, hmm
<voidspace> dimitern: I can't just do that
<dimitern> voidspace, why not?
<voidspace> well, I can
<voidspace> it's just work that I think belongs to the subnets card we already have
<voidspace> AddSpace takes a slice of subnet ids
<voidspace> so AddSpace would then have to update the subnets too
<voidspace> so it becomes a lot more work that I think we already have a card for
<dimitern> voidspace, that's correct, but it seems that and your card are closely related
<voidspace> well, they are
<voidspace> let me go read the model doc again for how subnets are added
<perrito666> morning
<voidspace> perrito666: 'ning
<dimitern> voidspace, yes, and AddSpace should update the subnets docs, in a transaction, but it's tricky as it can change in the mean time, so think about using txn-revno
<dimitern> voidspace, also, as ref counting lands, we should not change the space of a subnet with refcount > 0
<voidspace> dimitern: that definitely is in the other ticket
<voidspace> dimitern: as ref counting belongs there
<voidspace> dimitern: so, "juju create subnet <CIDR>" doesn't *require* a space name does it?
<voidspace> dimitern: we have a chicken and egg problem
<voidspace> dimitern: creating a space requires a subnet
<voidspace> dimitern: but if you create a subnet you either specify a space, or it get puts in the default one
<voidspace> dimitern: so I assume it must be possible to move a subnet from default space to another space, right?
<voidspace> (unless refcount > 0)
<dimitern> voidspace, well, what's clear is that you shouldn't be able to use a subnet without a space for deployments
<dimitern> voidspace, so creating it with an empty space should be fine
<dimitern> voidspace, what a sec
<dimitern> voidspace, it's "$ juju subnet create <CIDR> <space> <zone1>" actually
<dimitern> voidspace, so both the space and the zone are required when creating (or adding) a subnet
<voidspace> dimitern: ok - but you need a subnet (at least one) to create a space
<voidspace> dimitern: so you need a space to create a subnet, but you need a subnet to create a space
<dimitern> voidspace, this is a better point
<dimitern> voidspace, but there should always be a pre-created "default" space
<voidspace> dimitern: in which case you have to be able to move a subnet
<voidspace> dimitern: which is the cli command to move a subnet?
<voidspace> well, not quite
<voidspace> creating the new space with the existing subnet moves it
<voidspace> dimitern: I added a couple of comments to the model doc clarifying (slightly) about the moving of subnets
<dimitern> voidspace, we have juju space update CIDRs..
<voidspace> dimitern: that replaces *the whole list*
<voidspace> the normal use case would be to add or remove a single one I expect
<dimitern> voidspace, and there's the new space creation moving (unused) subnets to it, yes
<voidspace> having to specify all of them is a nuisance and an invitation for errors
<voidspace> but ah well]
<voidspace> *well
<dimitern> voidspace, the update is intended only when you want to change all of them
<dimitern> voidspace, thanks for your feedback and comments on the doc
<voidspace> dimitern: so subnet moving is an implicit part of space creation
<dimitern> voidspace, I think so - when safe, that is unused
<dimitern> jam, how do you feel about allowing the creation of "empty" spaces, but not allowing them as deployment targets until they have subnets?
<TheMue> dimitern: instead of creating them with at least one initial subnet?
<dimitern> TheMue, yes, because it's convenient to create a few spaces first, then either create or add existing subnets to them; also not allowing this *force* you to create or add subnets to the "default" space first, then move them
<TheMue> dimitern: from UX perspective I agree, yes
<TheMue> dimitern: it allows me to setup my network environment step by step
<dimitern> TheMue, exactly
<voidspace> dimitern: that sounds good to me
<dimitern> dooferlad, voidspace, let's do it then, we could change it later, if at all needed
<dimitern> I'll update the docs to reflect this and add comments
<voidspace> dimitern: deploying to a space with no subnets will raise an error (needs adding to the doc)
<dimitern> voidspace, yeah - it will come naturally as it happens on StartInstance, and there we need all subnets anyway to do AZ distribution
<TheMue> voidspace: and in case of only one subnet this is the default? or does it always has to be specified explicitely?
<dimitern> voidspace, it's still OK though, I think to allow empty spaces in constraints, until they're actually used
<dimitern> voidspace, as constraints are eval'ed each time
<voidspace> dimitern: sounds good
<voidspace> TheMue: you don't have to specify a subnet, and if there is only one then it will be used
<voidspace> TheMue: in fact you don't specify a subnet usually, you specify the space
<voidspace> I gotta go
<voidspace> see you all tomorrow
<TheMue> voidspace: ok, thx for info
<TheMue> voidspace: enjoy your evening
<dooferlad> mgz: hi, could you take a look at https://code.launchpad.net/~dooferlad/juju-ci-tools/juju-ci-tools-addressable-containers/+merge/267494 please?
<mgz> dooferlad: sure
<dooferlad> dimitern, TheMue: EOD for me. I have a couple of review requests up that are both small (the merge trunk one looks big, but in reality I only changed a couple of lines)
<dooferlad> http://reviews.vapour.ws/r/2333/
<TheMue> dooferlad: will have a look
<dooferlad> http://reviews.vapour.ws/r/2335/
<dooferlad> TheMue: thanks!
<TheMue> dooferlad: enjoy your evening
<dooferlad> and you
<TheMue> thx
<dimitern> dooferlad, cheers, will have a look
<cherylj> Can I get a couple quick reviews?  They're both just cherry-picks of already approved PRs:  http://reviews.vapour.ws/r/2336/      http://reviews.vapour.ws/r/2334/
<mgz> dooferlad: are you still working?
<perrito666> hey wwitzel3 ericsnow is there a spec for gce provider?
<ericsnow> perrito666: not really
<perrito666> :(
<wwitzel3> like an internal spec? we have a compatability spreadsheet
<ericsnow> perrito666: what's up?
<perrito666> wwitzel3: ericsnow I was thinking even developer notes
<perrito666> ericsnow: wwitzel3 working on gce storage
<ericsnow> perrito666: ah
<perrito666> ericsnow: wwitzel3 just looking for some reference, ill read the code and gce docs
<ericsnow> perrito666: we didn't document the implementation in great detail, if that's what you mean
<ericsnow> perrito666: that's you best bet (and ping either of us with questions)
<perrito666> i will, thank you :)
<ericsnow> perrito666: we tried to write it in an obvious way :)
<wwitzel3> perrito666: yeah, if you run in to any sticking points don't hesitate
<perrito666> ericsnow: I do recall that yea :) and it has been praised for clarity so I am not worried
<ericsnow> perrito666: haha
<sinzui> ericsnow: do you have a moment to review http://reviews.vapour.ws/r/2339/
<sinzui> katco: Can you review ^
<katco> sinzui: you have 2 ship-its
<katco> sinzui: now 3
<katco> :)
<sinzui> thank you for using the cluex4
<perrito666> bbl
<wallyworld> alexisb: you able to make release standup?
<alexisb> yes will be there soon
<thumper> waigani: back in the standup?
<waigani> thumper: yep
<mup> Bug #1483421 opened: Can not install juju-local on precise Ubuntu <juju-core:New> <https://launchpad.net/bugs/1483421>
<mup> Bug #1483421 changed: Can not install juju-local on precise Ubuntu <juju-core:New> <https://launchpad.net/bugs/1483421>
<mup> Bug #1483421 opened: Can not install juju-local on precise Ubuntu <juju-core:New> <https://launchpad.net/bugs/1483421>
<perrito666> wallyworld: axw our talks sound a lot like the restaurant at the end of universe
<axw> perrito666: I'm a bad nerd, I've never read hitchhiker's guide (I just know references...)
<perrito666> axw: well by that part the good parts are almost over, but the conversations inside it look a lot like speaking to australians being in south america
<perrito666> by all, see you tomorrow
<davechen1y> thumper: https://github.com/golang/go/issues/10628
<davechen1y> upstream issue for ppc
<thumper> cheers
<thumper> that is the sort of problem that I'm pleased others are looking into
 * thumper runs to physio appt
 * thumper actually walks to the car...
<davechen1y> thumper: eh ?
#juju-dev 2015-08-11
<mwhudson> davechen1y: just a peanut gallery comment but does that ppc64 crash happen with GOGC=off?
<davechen1y> mwhudson: let me check
<davechen1y> i already checked stack copying
<davechen1y> and adjusted the min stack size to be higher
<davechen1y> what it looks like to me is the receiver of the method is not being carried over in the case that there are no arguments
<davechen1y> which is weird
<davechen1y> but that'ts the case
<mwhudson> does it happen all the time?
<mwhudson> i could guess that autogenerated methods might screw up SP manipulation but that seems like the sort of thing that ought to break hard all the time
<davechen1y> mwhudson: 100% of the time
<mwhudson> oh
<mwhudson> well that shooooooooouuuuuuuuuuuuulllllllllllllld make it easier to fix
<davechen1y> github.com/juju/juju/cmd/envcmd.(*SysCommandBase).ConnectionInfo(0x478420, 0x0, 0x0, 0x0, 0x0) /home/ubuntu/src/github.com/juju/juju/cmd/envcmd/systemcommand.go:130 +0xec fp=0xc820484c98 sp=0xc820484c20
<davechen1y> github.com/juju/juju/cmd/envcmd.(*SysCommandBase).ConnectionEndpoint(0x478420, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /home/ubuntu/src/github.com/juju/juju/cmd/envcmd/systemcommand.go:116 +0x5c fp=0xc820484df8 sp=0xc820484c98
<davechen1y> receiver is 0x0 in the autogenerated method
<davechen1y> then bonkers in the next frame down
<davechen1y> opps
<davechen1y> github.com/juju/juju/cmd/envcmd.(*SysCommandBase).ConnectionEndpoint(0x478420, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /home/ubuntu/src/github.com/juju/juju/cmd/envcmd/systemcommand.go:116 +0x5c fp=0xc820484df8 sp=0xc820484c98
<davechen1y> github.com/juju/juju/cmd/juju/user.(*AddCommand).ConnectionEndpoint(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) <autogenerated>:17 +0x30 fp=0xc820484e00 sp=0xc820484df8
<davechen1y> this, not the other one
<mwhudson> is a nil receiver plausible?
<davechen1y> theorerically yes
<davechen1y> calling through an interface, no
<davechen1y> and in this case
<davechen1y> the receiver was not nil
<davechen1y> the weird thing is genwrapper is only called from two places in the compiler
<davechen1y> it is called a _lot_
<mwhudson> right, and you haven't been able to reduce it?
<davechen1y> it takes minutes every run
<davechen1y> i am trying to reduce it
<davechen1y> but everything on ppc takes so long
<davechen1y> also, you cannot breakpoint an autogenerated wrapper
<davechen1y> which is fucked
<davechen1y> if you could
<davechen1y> this would be easy
<mwhudson> eh?
<mwhudson> gdb and go, two great tastes etc
<mwhudson> davechen1y: you can break point on an address btw
<mwhudson> br *0xdeadbeef
<davechen1y> github.com/juju/juju/cmd/envcmd.(*sysCommandWrapper).Run(0xc82023e580, 0xc82040b590, 0x0, 0x0) <autogenerated>:61 +0x9c fp=0xc820485300 sp=0xc8204852b8
<davechen1y> what's the address ?
<mwhudson> does "p 'github.com/juju/juju/cmd/envcmd.(*sysCommandWrapper).Run'" show anything?
<mwhudson> davechen1y: lolz
<mwhudson> (vivid-ppc64el)mwh@kelsey01:~$ ~/go/bin/go get github.com/juju/juju/...
<mwhudson> # cd /home/mwh/gopath/src/github.com/juju/juju; git show-ref
<mwhudson> package github.com/juju/juju/...: signal: segmentation fault
<davechen1y> /away
<davechen1y> mwhudson: it prints something
<davechen1y> but it's not that useful
<davechen1y> i have some ideas for a repro now
<davechen1y> i'll keep trying
<wallyworld> axw: did you have 5 minutes? in standup hangout?
<axw> wallyworld: be there in a sec
<mup> Bug #1483492 opened: worker/storageprovisioner: machine agents attempting to attach environ-scoped volumes <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1483492>
<aisrael> axw: building with your latest pull request
<davechen1y> mwhudson: Thuper: got it!
<davechen1y> i have a repro
<davechen1y> which doesn't involve any juju code
<davechen1y> https://github.com/golang/go/issues/12108
<mup> Bug #1483525 opened: Document action-set quoting <juju-core:New> <https://launchpad.net/bugs/1483525>
<axw> aisrael: thanks for the update. the other things are known issues. to destroy with persistent storage, you currently have to use --force, or destroy the machines that the volumes are attached to. I'll send a PR soon that makes this unnecessary
<davechen1y> mwhudson: i know what it is
<mup> Bug #1236859 changed: errors reported in agent-state can contain too much information (traceback, etc) <logging> <traceback> <juju-core:Fix Released> <https://launchpad.net/bugs/1236859>
<voidspace> dimitern: ping
<voidspace> dimitern: unping
<dimitern> voidspace, (un)pong? :)
<voidspace> dimitern: hah
<voidspace> dimitern: I had a question, but I figured it out...
<dimitern> voidspace, awesome!
<voidspace> hah :-)
<voidspace> not sure that counts as awesome, but thanks
<voidspace> you must have had your coffee already this morning
<mup> Bug #1298819 changed: Mechanism for retrieving relation names <feature> <relations> <juju-core:Won't Fix> <https://launchpad.net/bugs/1298819>
<dimitern> voidspace, well, you know at least it's a great start of the morning
<dimitern> voidspace, :)
<voidspace> :-)
<voidspace> apparently giving your smilies noses, the - in :-) , is a sign that you're old
<voidspace> I just can't get used to :) though
<voidspace> his eyes are too close to his mouth
<dimitern> hehe
<dimitern> I like :o) sometimes or just ;] when lazy
<voidspace> :o) looks like a clown
<anastasiamac> voidspace: really? it has something pig-ish to my eyes .. ")
<voidspace> anastasiamac: heh, maybe
<voidspace> clown pig
<voidspace> dimitern: soooo....
<voidspace> dimitern: we have an issue with address allocation for containers
<voidspace> dimitern: we get the subnet from the provider
<voidspace> dimitern: and create it in state if necessary
<voidspace> dimitern: and now subnets in state have space information
<voidspace> dimitern: but address allocation for containers cares nothing for spaces yet
<voidspace> dimitern: so if the subnet it discovers from the provider doesn't exist in state it will create it in the default space
<voidspace> dimitern: I might just ignore this for my PR and I think we need a new card for it
<voidspace> dimitern: because fixing that is quite a bit of thinking
<voidspace> anastasiamac: good evening
<voidspace> dimitern: what it ought to do is check the space constraints for the container, iterate over all subnets until it finds one that matches and use that
<mup> Bug #1298819 opened: Mechanism for retrieving relation names <feature> <relations> <juju-core:Won't Fix> <https://launchpad.net/bugs/1298819>
<anastasiamac> voidspace: evening :D
<dimitern> voidspace, it does need a new card, yes
<mup> Bug #1298819 changed: Mechanism for retrieving relation names <feature> <relations> <juju-core:Won't Fix> <https://launchpad.net/bugs/1298819>
<dimitern> voidspace, but it needs spaces-in-constraints support first
<voidspace> dimitern: fair enough
<voidspace> dimitern: maybe it's time to start telling containers which subnet(s) to pick an address from instead of just picking an arbitrary one
<voidspace> dimitern: at the moment we discover subnets and create them in state when we use them
<voidspace> anyway
<voidspace> I'll add a card for us to estimate
<voidspace> dimitern: I added a card for updating AddSpace/CreateSpace to allow spaces with 0 subnets
<voidspace> dimitern: not much work, but some work
<dimitern> voidspace, cheers
<dimitern> voidspace, as soon as we finish the mvp stuff, we should make sure the address allocation still works and fix it as needed
<voidspace> dimitern: why does subnetsAPI define AllSpaces?
<voidspace> and define it differently to the spacesAPI
<dimitern> voidspace, to verify the spaces
<dimitern> voidspace, is this the backing interface or?
<voidspace> dimitern: apiserver/subnets/subnets.go
<voidspace> dimitern: and it returns a list of space tags, whereas AllSpaces as used by "space list" returns full space information
<dimitern> voidspace, well, we only need the tags for the CLI, but I guess there's no harm do get all the information if it's easier
<voidspace> dimitern: ah, it's called ListSpaces on the spacesAPI
<voidspace> dimitern: but the backing call is AllSpaces
<voidspace> dimitern: slightly confusing but not actually wrong
<dimitern> voidspace, these are one of the earliest attempts - it was expected for the various cli, api, apiserver, and backing interfaces to change, but get some sort of convergence at the backing side eventually
<voidspace> ok
<mwhudson> davechen1y: nice test case reduction
<mwhudson> davechen1y: ooooh i think i know
<davechen1y> mwhudson: do tell
<davechen1y> it's something about the setup for duff copy
<davechen1y> i'll see what russ says overnight
<mwhudson> davechen1y: well duffzero is entered by bl
<davechen1y> OH
<mwhudson> yeah
<davechen1y> that' won't help
<mwhudson> and the function exits by b to the method
<davechen1y> yup
<davechen1y> i've hit that before in risc compilers
<mwhudson> which blr's out...
<davechen1y> and back to the callers caller
<davechen1y> no
<davechen1y> hang on
<mwhudson> back to the generated method i think
<davechen1y> hmm
<davechen1y> i don't think that is it
<mwhudson> i was clued in by a breakpoint being hit an impossible number of times
<davechen1y> oh
<davechen1y> so it's going around and around until it rolls off the end of the stack ?
<mwhudson> er not sure
<davechen1y> i think you're close
<mwhudson> i think the stack pointer is wrong
<davechen1y> but that isn't the whole story
<davechen1y> the reciver is zeroed out
<davechen1y> anyway
<mwhudson> so the load from the stack at the end of the function is not reading the right bit of stack
<mwhudson> this is all very familiar feeling, did we fix this bug in arm64 already or something?
<davechen1y> i think you're right with the tail call thing
<davechen1y> wrapper methods are designed to return to their caller
<davechen1y> anyway, i know how to fix juju
<davechen1y> even if it means a small patch
<mwhudson> disable duffzero?
<mwhudson> oh, or a juju change?
<davechen1y> just disable duffzero
<davechen1y> we'll live
<mwhudson> i wonder if this fixes the GOARCH=amd64 go build runtime thing
<mwhudson> that's probably a bit optimistic
<davechen1y> probaby a long shot
<davechen1y> mwhudson: anyway, not looking at it any more tonight
<mwhudson> davechen1y: fair enough
<davechen1y> thanks for the sugestions, i'll have a look tomorrow
<mwhudson> i'll add a quick comment
<dimitern> dooferlad, go fmt is sad: cmd/juju/commands/main.go
<davechen1y> mwhudson: thanks for that comment
<davechen1y> i think you're right
<davechen1y> there was something similar a while back with somethign in the arm32 runtime
<davechen1y> i don't rememver exactly
<davechen1y> it was something like the kernel atomic helper
<davechen1y> or something like that
<davechen1y> NO
<davechen1y> it was the gettls helper
<davechen1y> which was sort of the same thing
<davechen1y> needs a kernel helper to get the tls offset
<mwhudson> ah yes
<davechen1y> blah blah
<davechen1y> tail call didn't call no more
<mwhudson> that tickles faint bells
<davechen1y> mwhudson: b377c9c6a9b720d0897d298652bebd3887ceeb46
<davechen1y> for your ref
<davechen1y> i think
<davechen1y> i might have fixed a similar problem
<davechen1y> but i cannot find the ref
<davechen1y> anyway
<davechen1y> it's the same smell
<davechen1y> arch's with link registers get tripped up by these shennaigans
<mwhudson> i can't for the life of me find the code that compiles the tail call
<davechen1y> genwrapper
<mwhudson> i mean the liblink bit
<davechen1y> i don't think it's there
<mwhudson> sorry, i mean the bit that creats the b $foo instruction
<mwhudson> i found it, then realized i was looking at arm64
<davechen1y> i think it's
<davechen1y> compile/internal/gc/subr.go
<davechen1y> generapper
<mwhudson> case 11 in asm9.go
<davechen1y> oh god
<davechen1y> please don't make me look at asmN.go
<mwhudson> heh ok
<davechen1y> that's the worst
<davechen1y> hmm, this doesn't look to magic
<davechen1y> i don't think that's the right place to look
<davechen1y> ok
<davechen1y> i see, ADUFFCOPY is implemened in case 11
<davechen1y> so
<davechen1y> arm and arm64 will be in the same boat
<davechen1y> what are they doing I wonder
<mwhudson> yeah i wonder how come they don't break
<mwhudson> arm64 reloads the link register from the stack before the tail call
<davechen1y> are you looking at zerorange in arm64 ?
<davechen1y> or is that in asm7 ?
<davechen1y> mwhudson: here's some math for ya
<davechen1y> 8+frame+lo-8
<mwhudson> davechen1y: seems like that could do with some constant folding
<mwhudson> davechen1y: i think i've found the ppc64le/arm64 point of difference
<mwhudson> the ARET case in preprocess
<davechen1y> i think it could be soethign else
<davechen1y> have a look at zerorange in arm64
<davechen1y> and zerorange in ppc64
<davechen1y> arm64 laosd the SP into rt1 then adds the size to zero to it
<davechen1y> ppc64 does not load the sp
<davechen1y> oh
<davechen1y> sorry
<davechen1y> yes it does
<davechen1y> it sets the source reg to be regsp
<davechen1y> and the dest reg to rt1 with an add
<davechen1y> we cannot do the same for arm64 because you cannot use Sp as the source or something
<davechen1y> anyway
<davechen1y> that's not it
<davechen1y> i'm sure you're right about the tail call not reloading sp
<mwhudson> davechen1y: disabling duffzero seems like the right fix
<mwhudson> for the short term
<mwhudson> and to a general lack of surprise it doesn't fix the "can't compile amd64 code" thing
<voidspace> dimitern: we used to have space validation that would check all specified subnets have a valid CIDR and exist
<voidspace> dimitern: I replaced the validation logic with a check that they exist in the transaction
<voidspace> dimitern: so an invalid subnet still fails (because it doesn't exist in state)
<voidspace> dimitern: the place to check for an invalid cidr would seem to be when adding a subnet
<voidspace> dimitern: so I'm going to remove the failing test I have as a consequence
<voidspace> dimitern: specifying an invalid cidr when adding a space still fails
<voidspace> dimitern: but now it fails with subnet not found
<voidspace> and there's a test for that already
<dimitern> voidspace, hmm.. I have to take a look to remind myself..
<voidspace> dimitern: TestAddSpaceInvalidCIDR is gone
<voidspace> dimitern: see https://github.com/juju/juju/compare/net-cli...voidspace:spaces-list-state?expand=1
<voidspace> dimitern: due to the removal of Space.validate and changes to AddSpace
<voidspace> dimitern: so the only difference is that for an invalid CIDR in AddSpace we don't validate first, but we do ensure the subnet exists (it won't) and fail with a sensible error message (subnet not found)
<dimitern> voidspace, hmm..
<dimitern> voidspace, so invalid CIDR I think should be validated before looking it up
<voidspace> dimitern: why?
<voidspace> dimitern: that's only sensible if we will be putting invalid CIDRs in state
<voidspace> dimitern: which we really shouldn't!
<dimitern> voidspace, well, it how about ";drop tables foo" :)
<voidspace> dimitern: you're saying we have an injection vulnerability...
<voidspace> dimitern: I really don't think so
<dimitern> voidspace, but seriously, we shouldn't put invalid cidrs in state indeed
<dimitern> voidspace, jk
<voidspace> :-p
<dimitern> voidspace, so you're saying adding a space with an empty list of subnets is now allowed?
<voidspace> dimitern: yes
<dimitern> voidspace, but ISTM non-empty list with possibly invalid cidrs leads to erraborted on addspace
<voidspace> dimitern: yep
<voidspace> dimitern: and there's a check for that
<voidspace> dimitern: if we get ErrAborted that isn't due to the space already existing we check to see which subnet is missing
<voidspace> dimitern: so we can report the error correctly
<voidspace> dimitern: and there's a test for it, which passes
<voidspace> well, missing subnet
<voidspace> which is exactly the same thing (as an invalid cidr will always be missing)
<dimitern> voidspace, indeed, sounds reasonable
<voidspace> cool, thanks
<dimitern> TheMue, you have review - sorry it took so long
<TheMue> dimitern: great, thanks.
<perrito666> morning people
<mup> Bug #1483672 opened: Allow charms to associate structured data with status <cloud-installer> <landscape> <juju-core:Triaged> <https://launchpad.net/bugs/1483672>
<TheMue> dimitern: the mentioned trick is, because State.DeadIPAddresses() returns a []*state.IPAddress, but in the mock I want a []StateIPAddress. a direct type cast of a slice isn't possible, only in a loop one by one
<TheMue> dimitern: any more intelligent way is welcome
<dimitern> TheMue, I see, ok
<TheMue> dimitern: heyCompilerImNotDumbSoCast([]myType, mySubInterfaceSlice)
<dimitern> TheMue, :) I'd suggest adding a comment like // Convert to addresses to []StateIPAddress
<TheMue> dimitern: yeah, makes it more clear.
<dimitern> TheMue, uhm.. s/to[1]//
<dimitern> just Convert addresses..
<TheMue> dimitern: already thought so
<frankban> dpb1: ping
<TheMue> dimitern: voidspace: do you have a moment to review http://reviews.vapour.ws/r/2324/ ?
<voidspace> TheMue: after the MAAS call maybe
<dimitern> TheMue, yeap
<TheMue> great, thx
<bogdanteleaga> alexisb, katco, perrito666: there's also this as a dependency for the centos PR, it doesn't seem to show up on rbt because the request didn't get automatically created
<voidspace> dooferlad: dimitern: if you have time http://reviews.vapour.ws/r/2344/
<voidspace> dimitern: hmmm... "space add" won't initially be supported by ec2 will it. So I'm not sure why we've implemented it (with the same semantics as create it seems)
<voidspace> ah well
<voidspace> I'm not even entirely convinced we need both anyway...
<voidspace> we should just have add, if it already exists on the provider that one should be used - if it doesn't (and creating is supported) then it should be created
<voidspace> possibly the same for subnets
<dpb1> hi frankban
<voidspace> added as a comment to the model doc
<frankban> dpb1: hi, could you please either give me write access to lp:juju-deployer or take a look, merge and release https://code.launchpad.net/~frankban/juju-deployer/guiserver-bundle-v4/+merge/267540 ?
<ericsnow> dooferlad: would you mind adding a link to the PR to bug #1468349?
<mup> Bug #1468349: discoverySuite.TestDiscoverServiceLocalHost: invalid series for wily and vivid <centos> <test-failure> <unit-tests> <vivid> <wily> <juju-core:Fix Committed by dooferlad> <https://launchpad.net/bugs/1468349>
<dooferlad> ericsnow: done
<ericsnow> dooferlad: thanks!
<dimitern> voidspace, well, I think technically speaking, juju space add ... is a great way to "import" spaces from previous deployments
<voidspace> dimitern: sure, and it would remain so
<voidspace> dimitern: however, if there is no previous deployment then it would create them
<voidspace> dimitern: I'm not suggesting any change in functionality, merely merging add and create
<dimitern> voidspace, because it allows you to discover tagged subnets as part of a space, and .. hmm.. although overwriting the list of subnets seems dangerous
<voidspace> dimitern: doesn't add already have this problem
<voidspace> dimitern: if you want to add an existing space then don't specify any subnets
<voidspace> dimitern: the user will always want to either add or create, but they only need a single entry point to do it
<voidspace> dimitern: whichever we do under the hood, from the user perspective they are adding a space to the current environment
<dimitern> voidspace, and, in case of AWS if you pre-tag subnets you effectively create spaces you can immediately use for bootstrapping
<dimitern> voidspace, so "add" only makes sense to take a name, yeah
<dimitern> voidspace, perhaps it can optionally take CIDRs and verify if they match, or if not - create it..
<TheMue> voidspace: thx for review. "failled" is no typo, the more l, the more it failed ;)
<voidspace> TheMue: hehe
<voidspace> failllllllled
<voidspace> dimitern: we don't need to take CIDRs - let's not
<voidspace> dimitern: I've added a comment to the model doc
<voidspace> dimitern: I'd prefer we just merged add and create
<dimitern> voidspace, or if they match (or are a subset of) the actual tagged subnets just ignores them and adds all subnets
<voidspace> dimitern: all existing subnets should be discovered
<voidspace> dimitern: in fact - we probably don't want add at all
<voidspace> dimitern: we probably want "space discover"
<dimitern> voidspace, not really
<voidspace> dimitern: find and add all subnets and spaces
<dimitern> voidspace, I think the original idea was that "create" is an admin command
<voidspace> dimitern: in case you only want a subset of available spaces and subnets?
<voidspace> dimitern: it's not in the doc...
<dimitern> voidspace, while "add" can be used to "subclass" a bigger space
<voidspace> dimitern: MVP is ec2 only, which doesn't support add at all
<dimitern> voidspace, :) it's in one of the other docs or numerous notes, I'm sure
<voidspace> and "space discover" would still be useful - find and add all spaces and subnets
<dimitern> voidspace, or space list --all
<voidspace> dimitern: and I bet that would be all most users would ever use... with manually adding a subset being an esoteric and rarely used option
<dimitern> voidspace, or you mean more like "space import --existing"
<voidspace> "space import"
<voidspace> the --existing flag is redundant
<voidspace> "space list --all" would be cool too (later)
<dimitern> yeah, it could instead be --from file.yaml
<voidspace> bbiab
<voidspace> hah
<dimitern> voidspace, agreed, I'll fix this to be reflected by the doc
<dimitern> voidspace, it should be easy to drop the parsing/tests around add with CIDRs
<voidspace> we don't need it at all for ec2, but yes
<dimitern> voidspace, and we could then just omit it from the MVP
<dimitern> voidspace, I'd like to keep it in net-cli, as soon we should be able to discover and add maas spaces
<TheMue> dimitern: will you find a moment for a hopefully final review too?
<dimitern> TheMue, did you see my comment about the EnvironWatcher ?
<TheMue> dimitern: yeah, removed it
<TheMue> dimitern: now everything is even smaller and more simple
<dimitern> TheMue, hmm well I still see it in api/
<TheMue> dimitern: oh, will take a look, thought I got them all
<perrito666> ericsnow: wwitzel3 any of you around?
<ericsnow> perrito666: sure
<wwitzel3> perrito666: heyo
<perrito666> ericsnow: a quick question (hopefully) I see you guys implemented a basic disk abstraction
<perrito666> and that you have only types for persistent and scratch
<TheMue> dimitern: sh*t, yes, killed it on server side but missed client, one moment
<ericsnow> perrito666: right
<ericsnow> perrito666: those are needed during provisioning
<perrito666> what do I get with persistent, standard persistent or ssd persistent?
<ericsnow> perrito666: I don't really remember (the GCE docs can tell you though)
<perrito666> mmm I see that goes for an attached disk, Ill look more into creating un attached disks
<perrito666> tx guys
<TheMue> dimitern: I don't know which EnvironWatcher you mean. *cough* *cough*
<TheMue> :D
<ericsnow> perrito666: np
<wwitzel3> perrito666: pd-standard
<wwitzel3> perrito666: we don't explictly set the diskType in the initializeParams for the attachDisk call, so it defaults to pd-standard
<wwitzel3> perrito666: if that was something you wanted to specify, you'd do so in the initializeParams on line 92 in gce/google/disk.go
<wwitzel3> perrito666: you would just extend the DiskSpec to capture that information
<voidspace> dooferlad: I replied to your review comment
<voidspace> dooferlad: and that change was discussed with dimitern :-)
<voidspace> wwitzel3: o/
<voidspace> wwitzel3: I'm just off to exercise, so just waving as I go...
<dooferlad> voidspace: cool. Fine by me!
<voidspace> great
<voidspace> dooferlad: and thanks
<wwitzel3> voidspace: o/
<perrito666> wwitzel3: thanks man :) I am seeing how to code it properly to match the other volume implementation yet :)
<dimitern> TheMue, you have a review
<TheMue> dimitern: thx
<voidspace> right, bye all
<TheMue> dimitern: in case of a remove error we currently bail out directly with the according error, no ErrTryAgain
<TheMue> dimitern: that's why I removed the "retry" in the log
<TheMue> dimitern: could change it to handle it like release errors before
<dimitern> TheMue, why kill the worker if it can't remove an address?
<TheMue> dimitern: good question, but then err = errors.Annotatef(err, "removing IP address %q", ipAddress.Value()) of last review is wrong
<TheMue> dimitern: logging and returning ErrTryAgain is better then
<dimitern> TheMue, although.. killing it with a some sort of ErrTryAgain with decreasing number of retries, finally killing it; with that on both release or remove errors, the runner can restart the worker, re-trigger the watcher and we have automatic-retry
<TheMue> dimitern: eh, today the worker isn't killed when the returned error is an ErrTryAgain
<dimitern> TheMue, there was something rogpeppe did about giving a worker and/or runner a callback to decide if an error is more important or not (i.e. ErrTryAgain can be less important than ErrTryAgain{retries-1} or {retries=0}
<TheMue> dimitern: it simply waits until the next trigger and tries again. in case of other errors the worker will be stopped and restarted
<dimitern> TheMue, yeah, it isn't, but doesn't that sound like a good way to simplify things like retrying?
<TheMue> dimitern: it sounds to me more like complicating
<dimitern> TheMue, so, if you have this you don't need to use a timer to trigger it
<TheMue> dimitern: imho there's no advantage when restarting this worker
<dimitern> TheMue, when restarted, it will call watch again and get all dead addresses
<TheMue> dimitern: you mean because restarting always starts to clean?
<dimitern> TheMue, yes, it's as good, if not better than calling Handle() yourself out-of-order to the watcher
<TheMue> dimitern: ok, so in this case we don't need ErrTryAgain. we simply die when releasing or removing fails
<TheMue> dimitern: at least we don't have to separate on worker side.
<TheMue> dimitern: if cleanup fails we restart the worker
<dimitern> TheMue, that's the simplest thing, but remember - if we don't manage to run handle() without an error for a long time it's just sitting there eating resources and log space
<TheMue> dimitern: yes, optimally cleanup returns no error and e'thing is fine
<dimitern> TheMue, that's why I'm suggesting to add a field or method on ErrTryAgain or ErrLimitedRetry if you prefer
<dimitern> TheMue, which "grants you" a retry when called (e.g. decrementing the num retries at construction) - like a semaphore lock, but doesn't have to be
<dimitern> TheMue, if retires run out it returns the last error instead of no error or an "acquired" ErrLimitedRetry
<dimitern> TheMue, have you looked at moreImportant (or was it errMoreImportant)..
<TheMue> dimitern: hmmm, so the worker manages this number? because the api is stateless
<dimitern> TheMue, ok, to let's keep it simpler
<dimitern> TheMue, I've looked at the more important stuff in detail and, although it can be useful, it's purely an optimization of the runner
<TheMue> dimitern: ok
<dimitern> TheMue, so, assuming Remove() doesn't return NotFound error or we ignore it (possibly logging it), it's fine like this I think
<TheMue> dimitern: I would like to change it, that in case of a not found it's fine and in case of any other error we continue like at release errors (logging and setting the retry flag). so we at least try all and not bail out at the first error
<dimitern> TheMue, I saw a few places gc.IsNil checks for errors - please, use jc.ErrorIsNil - the only places where it might not work is if something like [*]params.Error{} is returned instead of *params.Error(nil)
<dimitern> especially true for client-side apis
<TheMue> dimitern: that should be the only place where I used it
<TheMue> dimitern: at least I tried so, will check again
<dimitern> TheMue, sure, I had issues with some cases like this - that's why we have things like .OneError() on ErrorResults
<TheMue> dimitern: sound like an even better change then
<dimitern> TheMue, cheers
<TheMue> dimitern: your comment regarding waitForReleaseOp(), here I lost you.
<natefinch> ericsnow: you around?
<ericsnow> natefinch: yep
<dimitern> TheMue, which one? :)
<TheMue> dimitern: for worker/addresser/worker_test.go line 107
<dimitern> TheMue, there are two cases - waiting to happen and waiting not to happen
<dimitern> TheMue, without creating a chan to listen for it, you can't guarantee it happened or not
<TheMue> dimitern: not sure if we talk about the same. the comment is "There should be at least one check in the tests below where we do a similar loop, but for a shortAttempt, asserting it wasn't called."
<dimitern> TheMue, yes, me too - the loop there waits for a ReleaseAddressOp on the dummy channel
<TheMue> dimitern: it has no loop, it's right now a select with timeout
<dimitern> TheMue, for the places where we know ReleaseAddress should not be called (or it should fail), we need a similar attempt, waiting for an op to make sure it wasn't called
<dimitern> TheMue, oh, I see
<dimitern> TheMue, that's due to RB's somewhat confusing folding diffs
<TheMue> dimitern: ah, getting closer, maybe only had my troubles with the location of the comment.
<TheMue> dimitern: hehe, exactly
<dimitern> TheMue, this comment is actually mean for the loop below
<dimitern> TheMue, in TestWorkerReleasesAlreadyDead
<TheMue> dimitern: ah, and I see it here at the function declaration
<TheMue> dimitern: that's what made me wonder
<perrito666> mm, I wish my editor had a "make this struct satisfy that interface" shortcut
<TheMue> perrito666: oh, cool idea for my plugin
<perrito666> TheMue: tell me your plugin is for vi
<TheMue> perrito666: I've once built one for  vi but right now it's for BBEdit
<perrito666> ah osx, that explains
<perrito666> aghh I think Ill have to write it myself and propose it
<TheMue> perrito666: hehe
<perrito666> is there a go tool that returns the methods for a given interface?
<dimitern> TheMue, yeah it's a bit surprising
 * dimitern at eod
<TheMue> dimitern: what? os x?
<natefinch> query katco
<natefinch> lol
<katco> ?
<natefinch> katco: I had the wrong day for my daughter's doctor's appointment... it's on thursday overlapping with our 1:1... can we move the 1:1?
<katco> natefinch: sure, let me find a time
<katco> natefinch: would you mind reviewing http://reviews.vapour.ws/r/2199/ and http://reviews.vapour.ws/r/2318/ ?
<katco> natefinch: they're both windows/powershell specific, so you seem like the right person :)
<natefinch> katco: will do
<katco> natefinch: ty
<katco> natefinch: also, how's the unregister method coming?
<natefinch> katco: written but needs tests. I'm going to be working late tonight due to the late start this morning, too.  So, hopefully wrapped up today.
<katco> natefinch: cool beans, thanks for the update
<natefinch> katco: I may be the best person on the Juju team to review those patches, but that doesn't mean I'm qualified.  I've never used powershell.  I'll do what I can, but I could easily miss simple things in the powershell code.
<katco> natefinch: no worries, just do your best. PS is in many ways like C#
<katco> natefinch: give a +1 instead of a +2 if you have to
<natefinch> katco: yeah, definitely reading it like it's bizzaro C# helps
<mup> Bug #1392786 changed: charm has no way to report error state <charms> <feature> <hooks> <juju-core:Fix Released> <https://launchpad.net/bugs/1392786>
<mup> Bug #1483879 opened: MAAS provider: terminate-machine --force or destroy-environment (without --force) don't DHCP release container IPs <landscape> <juju-core:New> <https://launchpad.net/bugs/1483879>
<mup> Bug #1483889 opened: OSX elcapitan support <juju-core:New> <https://launchpad.net/bugs/1483889>
<perrito666> ericsnow: still around?
<ericsnow> perrito666: yep
<perrito666> ericsnow: I have read https://godoc.org/google.golang.org/api/compute/v1 but I cannot find a way to create a disk not attached to an instance, did you by any chance saw something like that?
<perrito666> its like it is possible to create the attached one, and to use one already created
<perrito666> but not to create a volume independent of an instance
<ericsnow> perrito666: check out https://cloud.google.com/compute/docs/reference/latest/
<ericsnow> perrito666: the "disks" section probably
<perrito666> oh I see they seem to lack a nice implementation for that, how convenient ...not
<thumper> mramm2: around?
<thumper> or mramm
 * thumper suspects auto-reconnection
<davecheney> mwhudson: i found the prolog stuff
<davecheney> it's quite hairy
<davecheney> so I sent https://go-review.googlesource.com/13570 instead
<mup> Bug #1324318 changed: build and host osx binaries  and update docs to point to them <feature> <osx> <juju-core:Fix Released by sinzui> <https://launchpad.net/bugs/1324318>
<mup> Bug #1483889 changed: OSX elcapitan support <osx> <juju-core:New> <https://launchpad.net/bugs/1483889>
<mup> Bug #1324318 opened: build and host osx binaries  and update docs to point to them <feature> <osx> <juju-core:Fix Released by sinzui> <https://launchpad.net/bugs/1324318>
<mup> Bug #1483889 opened: OSX elcapitan support <osx> <juju-core:New> <https://launchpad.net/bugs/1483889>
<mup> Bug #1324318 changed: build and host osx binaries  and update docs to point to them <feature> <osx> <juju-core:Fix Released by sinzui> <https://launchpad.net/bugs/1324318>
<mup> Bug #1483889 changed: OSX elcapitan support <osx> <juju-core:New> <https://launchpad.net/bugs/1483889>
<davecheney> suddenly -- netspilts!
<perrito666> odd, I just saw the disconnects
<perrito666> not the netsplit warning
<perrito666> axw: let me know if you prefer to have the talk before or after
<perrito666> since it is just you and I
<axw> perrito666: hey, want to get started now?
<perrito666> as you wish
<menn0> thumper: can you take another look at http://reviews.vapour.ws/r/2328/ please?
<menn0> thumper: http://reviews.vapour.ws/r/2328/diff/1-2/ is probably what you want
<thumper> ack, otp
<mwhudson_> davecheney: yeah, v hairy
<mwhudson_> i was looking at it because it needs to change for shared libraries on power (i think)
<mwhudson_> mucho mucho hair
<perrito666> thumper: tx for the reviews and follow up
<perrito666> bbl
#juju-dev 2015-08-12
<mwhudson_> davecheney: so i talked to steve and we're going to go for uploading go1.5rc1 within a week
<mwhudson_> davecheney: enabling arm64 and ppc64el (with your patch!)
<mwhudson_> davecheney: i wanted to give you a chance to yell and say "don't enable it for ppc64el" or similar
<mwhudson_> davecheney: is there anyone else I should talk to about this? (qa team?)
<davecheney> mwhudson_: i'm testing ppc64 right now
<davecheney> ./all.bash passes
<davecheney> and i'm doing juju stress tests today
<davecheney> so, yes, please include that patch in rc1
<davecheney> there _will_ be a 1.5rc2
<davecheney> probably at the end of the week
<davecheney> i dunno if my patch will make it in
<mup> Bug #1483932 opened: unexpected public-address when lxc container has 2 NICs and IP addresses <juju-core:New> <https://launchpad.net/bugs/1483932>
<davecheney> mwhudson_: russ +2'd my patch to disable duffzero
<mwhudson_> davecheney: oh ok, maybe we'll upload 1.5rc2 then, depends on timings
<davecheney> i'll commit it in a few hours once' i've done some more testing
<mwhudson_> davecheney: yay!
 * mwhudson_ disappears for a bit
<axw> waigani cherylj: I'd really appreciate reviews of http://reviews.vapour.ws/r/2332 before any of my other branches. this needs to land before the freeze next week; preferably this week, since I'm sprinting next week
<waigani> axw: I'm half way through 2331, should be done soon. Then I'll get onto that one
<axw> waigani: awesome, thanks
<menn0> thumper: forgot to say. a number people have been hitting bootstrap problems where it appears mongodb isn't starting
<menn0> thumper: it's happening on both upstart and systemd systems
<menn0> thumper: https://launchpadlibrarian.net/213796443/juju_debug_output.txt
<menn0> thumper: bug 1468579
<mup> Bug #1468579: juju bootstrap failed - cannot dial mongo to initiate replicaset: no reachable servers <bootstrap> <mongodb> <oil> <juju-core:Triaged by menno.smits> <juju-core 1.24:Triaged by menno.smits> <https://launchpad.net/bugs/1468579>
<menn0> thumper: I need to give that some attention soon as well
<thumper> hmm...
<thumper> menn0: do you know what the cause is?
<menn0> thumper: no idea... I was kinda thinking a systemd integration issue, but james page has just reported the issue on trusty
<menn0> thumper: i'm going to ask people to provide /var/log/syslog for when the problem happens. that might tell us if/why mongod isn't starting
 * thumper nods
<lazyPower> thumper: o/
<thumper> lazyPower: 'sup?
<lazyPower> Just checking in on ya, i haven't pulse checked django in a while
<thumper> hmm...
<thumper> I've been a bit lax
<thumper> although likely to roll out some personal changes soon
<thumper> so no doubt, a few pushes upstream
<lazyPower> no stress, the last time we spoke you were working on workers
<thumper> lazyPower: what is best practise for dealing with charm tools?
<thumper> should we pip install?
<lazyPower> fill me in on the context
<lazyPower> ah
<lazyPower> If you need bleeding edge, thats the way to go
<lazyPower> i keep a branch around in a venv for bleeding edge, pypi release, and system defaults to debs
<lazyPower> context switching is fairly painless then
<thumper> more just don't want it included in the charm tree
<lazyPower> i bzrignore .venv and keep all the project deps around in a virtualenv.
<thumper> problem becomes the main hook .py file uses it
<thumper> so needs it to install  :)
<lazyPower> hmm, what are you pulling in from charm tools?
 * thumper shrugs
<thumper> normal stuff I guess
<thumper> not looked in detail
<lazyPower> are you talking about charm-helpers then?
<thumper> uh... yeah
<thumper> that
<lazyPower> ah ok :)  Common mistake
<lazyPower> yeah, I personally use the pypi delivery mechanism for charm-helpers
<thumper> and install?
<lazyPower> on the other hand, our openstack team is very much steeped in embedding charm-helpers in the charm with the sync scripts
<lazyPower> that way they can independently version/control the version of charm helpers shipping with the charms.
<thumper> also, what is the best practice for storing state in a charm?
<thumper> does the charm helpers help there?
<lazyPower> i use the kv helpers in charmhelpers
<lazyPower> indeed
<lazyPower> https://pythonhosted.org/charmhelpers/api/charmhelpers.core.unitdata.html
<lazyPower> its kv storage for charms backed by sqlite (disclaimer) - i'm using it in the ETCD charm if you'd like to see a usage example in practice
<menn0> waigani: i just replied to your review comments
<thumper> lazyPower: yeah, should look
<lazyPower> thumper: https://github.com/whitmo/etcd-charm/blob/master/hooks/hooks.py#L127 - is a method body thats using it. its pretty straight forward
<lazyPower> getters/setters
<thumper> cheers
<davecheney> mwhudson_: https://go-review.googlesource.com/#/c/13570/2
<davecheney> i commented out the wrong duffzero the first time around
<davecheney> PTAL
<mwhudson_> ah
<lazyPower> davecheney: sorry to sideline this topic - are yinz no longer using reviewboard?
<davecheney> this one does actually fix the bug in juju
<davecheney> lazyPower: this isn't a juju patch
<davecheney> it's for go upstream
<lazyPower> ooo
<davecheney> we're trying to land some last minute fixes for ppc64
<lazyPower> ok, i've heard great things about gerrit so thought I'd ask :)
<davecheney> i think gerrit is ok
<davecheney> but out of the box is blows
<lazyPower> duly noted
<davecheney> to go one has a full time person tending to it's ministrations
<davecheney> custom skin
<davecheney> custom email notifcation
<davecheney> a host of bots covering it's various failings
<lazyPower> needs some juju loving is what you're telling me. get a stellar gerrit out of the box  preconfigured for joyous ruptures of usage goodness
<davecheney> that's sort of like saying "want a fantastic career, then get a super duper diploma from the university of new mexico. apply online today!"
<lazyPower> Exactly, marketing speak to make it sound greater than it really is
<davecheney> caution: results may not represent real life. Please consult a health care professional before deciding if online education is right for your family
<thumper> :)
<lazyPower> davecheney: its really funny that you mention that ;) Thats exactly the sector of marketing my last jobs clients were from. So i've got a long history of working in that bubble you used as an example :D
<davecheney> as we said at the market maker
<davecheney> past performance is not a predictor of future profit
<lazyPower> davecheney: "at the market maker" i dont follow
<davecheney> share trading firms call themselves market makers
<lazyPower> oh nice. #TIL
<davecheney> for the same reason people that like to break into other peoples shit call themselves security researchers
<waigani> menn0: thanks, just replied to yours :)
<menn0> waigani: next one: http://reviews.vapour.ws/r/2350/ (much smaller)
<waigani> menn0: looking
<waigani> menn0: done
<menn0> waigani: thank you
<mup> Bug #1483987 opened: [LXC caching] deletion of images does not work <juju-core:New> <https://launchpad.net/bugs/1483987>
<mup> Bug #1483987 changed: [LXC caching] deletion of images does not work <juju-core:New> <https://launchpad.net/bugs/1483987>
<mup> Bug #1483987 opened: [LXC caching] deletion of images does not work <juju-core:New> <https://launchpad.net/bugs/1483987>
<voidspace> dimitern: hey, morning
<voidspace> dimitern: I saw your review on my AllSpaces branch after I already had a "Ship It" from dooferlad
<TheMue> voidspace: good morning and happy birthday
<voidspace> dimitern: so I've addressed your comments in a follow-up branch that I'll submit now
<voidspace> TheMue: bah, humbug :-)
<voidspace> TheMue: morning
<dimitern> voidspace, hey, yes I saw you landed it too late
<voidspace> dimitern: there were some good suggestions in your comments though
<dimitern> voidspace, oh happy birthday indeed! :)
<voidspace> dimitern: hah, bah
<voidspace> as if I want to be reminded that I'm getting old...
<dimitern> voidspace, hehe
<dimitern> voidspace, well, if you like them you can do a follow-up ;)
<voidspace> dimitern: done
<voidspace> dimitern: just needs me to create the pull request
<voidspace> dimitern: just completing a final test run before I do that
<TheMue> voidspace: you're getting old? I'm just laughing. *lol*
<voidspace> TheMue: :-)
<dimitern> voidspace, cheers!
<dimitern> I had some bad headache yesterday night
<voidspace> too much vodka?
<dimitern> so I might be taking it slower today
<voidspace> ok
<voidspace> dimitern: did you see our EuroPython video went live?
<dimitern> no, 36 degrees during the day and 28 at night I guess
<voidspace> dimitern: I posted a link on twitter and google+
<voidspace> dimitern: :-(
<dimitern> voidspace, in g+ ?
<voidspace> dimitern: yep
<dimitern> voidspace, I didn't realize it was the video as well - cool!
<voidspace> dimitern: http://reviews.vapour.ws/r/2352/
<dimitern> voidspace, looking
<voidspace> dimitern: there's a DeferredAnnotatef - which is why I removed the errors.Trace(err)
<voidspace> dimitern: before you ask
<dimitern> voidspace, ok, as long as we're not ignoring an error
<dimitern> voidspace, LGTM
<voidspace> dimitern: thanks
<voidspace> dimitern: I don't assertSpace on them because there's already a test that does that
<voidspace> dimitern: this isn't a test for AddSpace
<voidspace> dimitern: it's a test for AllSpaces
<voidspace> dimitern: if AddSpace is broken there's another test that will fail
<voidspace> dimitern: once a unit of code is tested it's reasonable to assume that unit works for a test of a different unit
<voidspace> so long as you have *a test* that will fail if it breaks
<voidspace> dimitern: so I'll pick up subnets CLI next
<voidspace> dimitern: and work CLI -> api -> apiserver -> state again
<dimitern> voidspace, agreed, sounds good
<voidspace> dimitern: hmm... weird. We have CLI of subnet list, apiserver of space create and state of subnet add
<voidspace> dimitern: still left to do
<voidspace> dimitern: still, I can do them in CLI -> apiserver -> state order, they're just all different pieces of functionality...
<dimitern> voidspace, for state "add" means both create and add in api/cli terms
<dimitern> voidspace, create just first creates the subnet
<dimitern> (with the provider api)
<voidspace> dimitern: add ought to check with the provider
<voidspace> dimitern: right? as it's semantically "add a space that already exists with the provider"
<voidspace> dimitern: or subnet
<dimitern> voidspace, yes, but not at state level, at apiserver level
<dimitern> voidspace, once we get the info from the provider, we still eventually call state add
<voidspace> dimitern: yep
<voidspace> dimitern: dooferlad: TheMue: I'll be a few minutes late to stdup, sorry
<dimitern> np
<TheMue> ok
<dooferlad> dimitern: it has been pointed out that my CI test has something trying to ping google, but we want to have the test pass on closed networks. Was thinking about pinging the default gateway instead, if it is defined.
<dooferlad> of course not having a default gateway when there should be one is a problem, but it isn't related to what Juju does.
<dimitern> dooferlad, yeah, sounds good - perhaps instead of google try archive.ubuntu.com
<dimitern> voidspace, ;)
<voidspace> dimitern: omw
<mattyw> axw, ping?
<mattyw> morning, afternoon all. Review request 2332 has been issued some kind of award by reviewboard. Can anyone work out the reference? http://reviews.vapour.ws/r/2332/
<perrito666> Mattyw no idea but the code for rb is open
<axw> mattyw: pong, what's up?
<mattyw> axw, review #2332 got some kind of trophy for being 2332
<mattyw> axw, but I don't understand the reference, thought you might like to know
<axw> mattyw: heh yeah, I looked at the code a while ago, it's for palindromic numbers
<axw> mattyw: but why.. I don't know
<mattyw> axw, hmm, I'm glad they took the time to do that
<axw> mattyw: :)
<perrito666> lool
<perrito666> they also did it for review 1000 or 1024
<mattyw> right
<mattyw> so if reviewNum in someListOfNumbersWeLikeForWhateverReason { printTrophy() }
<perrito666> yup
<mattyw> I'd have probably fixed being able to display files that have moved before tackling that feature
<mattyw> but that's just me
<perrito666> mattyw: I am pretty sure that if number in list of numbers is a rather easy feature compared to the other thing
<perrito666> also it is free software, you can always fix being able to display files that have moved
<TheMue> dimitern: pushed the PR again
<dimitern> TheMue, ok, will have a look shortly
<TheMue> dimitern: thx
<dimitern> TheMue, hey
<dimitern> TheMue, take a look at this: http://play.golang.org/p/QHyS4l9JvB
<TheMue> dimitern: yep
<dimitern> TheMue, it shows clearly that returning nil if result.Error == nil is better than returning it (even with Trace) if result.Error != nil
<TheMue> dimitern: yep
<dimitern> TheMue, that way when a nil error is returned, it will always be an untyped nil and the ErrorIsNil will work
<dimitern> TheMue, as I was about to add a comment about a few places with IsNil wanted to see what was the way to fix it
<TheMue> dimitern: makes sense, yes. will change the two API functions as well as their tests
<TheMue> dimitern: ugly, those nil but not nil errors
<mup> Bug #1484105 opened: juju upgrade-charm returns ERROR state changing too quickly; try again soon <juju-core:New> <https://launchpad.net/bugs/1484105>
<voidspace> dimitern: ping
<TheMue> dimitern: feels good, and tests with ErrorNotNil now pass as wanted
<dimitern> voidspace, pong
<dimitern> TheMue, indeed; almost ready with your review
<voidspace> dimitern: quick check - the card I'm working on says to implement "AddSubnets" to the API (client side API call)
<voidspace> dimitern: the apiserver call is bulk, AddSubnets
<voidspace> dimitern: but the API call should be AddSubnet (singular), right?
<voidspace> dimitern: I'm pretty sure I answered this anyway by looking at cmd/juju/subnets/subnets.go
<voidspace> and it calls AddSubnet
<voidspace> dimitern: *plus*, I should do Create *and* Add - even though the card only says AddSubnet and AllSubnets
<voidspace> dimitern: cmd/juju/subnets/create.go exists alredy
<voidspace> *already
<dimitern> voidspace, AddSubnet should be the api and state method I think
<dimitern> voidspace, and the api should have a CreateSubnet as well
<dimitern> (also calling state.AddSubnet eventually at the apiserver side, after the subnet is created)
<voidspace> dimitern: yep, that matches my expectation
<voidspace> dimitern: thanks
<voidspace> dimitern: this card is just the api side stuff, so easy enough
<voidspace> no messing around with providers here
<dimitern> voidspace, so we'll need a method on environs.Networking I guess - CreateSubnet (eventually, for now it can be a method on an interface like in apiserver/common)
<voidspace> dimitern: yep
<voidspace> dimitern: and a "FindSubnet" or similar to verify an existing one
<perrito666> wwitzel3: hi?
<katco> perrito666: he is not in atm
<perrito666> oh that is unfair, he has an answering person (which I somehow suspect is a emacs plugin)
<perrito666> voidspace: hb
<katco> perrito666: (ERROR unknown defun 'oh that is unfair, he has an answering person (which I somehow suspect is a emacs plugin))
<perrito666> lol
<voidspace> perrito666: hb
<voidspace> perrito666: :-)
<mup> Bug #1483421 changed: Can not install juju-local on precise Ubuntu <precise> <juju-core:New> <https://launchpad.net/bugs/1483421>
<katco> perrito666: wtf are you trying to hack my emacs
<perrito666> lol
<natefinch> katco: gonna be ~5 min late to meeting
<dimitern> TheMue, if the release fails, do you try to remove the address?
<katco> natefinch: ok thx for the heads-up
<dimitern> TheMue, ah, sorry, that was after release was unbroken
<voidspace> dimitern: CreateSubnet doesn't exist on the API server and there's no card for it
<voidspace> dimitern: it was probably "implicit" in another card, but not done
<voidspace> dimitern: I'll add a card
<voidspace> dimitern: it takes slightly different parameters to AddSubnet
<voidspace> dimitern: (takes an additional public bool parameter)
<voidspace> dimitern: and the current apiserver AddSubnets does nothing with the provider - merely validates and adds to state
<voidspace> I think that's wrong (or at least "a bug" that we can fix later)
<voidspace> unless I'm mistaken
<lazyPower> natefinch: wwitzel3 - is there anything special i need to pass to --upload-tools to build the juju-process-docker bin or have i run into the end of the sidewalk on the branch?
<natefinch> lazyPower: sidewalk error
<TheMue> dimitern: yep, addresses w/o releasing aren't removed, they will be retried later. that what I've tested by break and unbreak it
<natefinch> lazyPower: currently it needs to be bundled in your charm
<lazyPower> oh?
<natefinch> lazyPower: check out https://github.com/natefinch/whalesay  and https://github.com/natefinch/whalesay/blob/master/hooks/install
<lazyPower> natefinch: ok, i can work around that. Any guess as to how long until thats shipping w/ the tools? (i'm assuming feature branch landing in master?)
<dimitern> TheMue, reviewed
<dimitern> TheMue, I'm fine if you fix just those that look more serious and the rest in a follow-up PR, if you like
<TheMue> dimitern: some are already done and pushed
<natefinch> lazyPower: it'll land after 1.25... it's going to be integrated into the normal tools, so there won't be a separate binary deployed
<lazyPower> ok, thats what i suspected. Thanks natefinch
<TheMue> dimitern: using ipAddress instead of ipAddress.Value() returns the string representation of network.Address() instead of the vallue (which is e.g. "public:0.1.2.3")
<TheMue> dimitern: for log output that imho is ok
<dimitern> TheMue, I see, ok then if you can, use a local var if the same ipAddress.Value() is used in more logs
<TheMue> dimitern: ok
<voidspace> g'night all
<mup> Bug #1484177 opened: Upgrade charm local juju failed <juju-core:New> <https://launchpad.net/bugs/1484177>
<natefinch> ericsnow: you around?
<ericsnow> natefinch: yep
<natefinch> ericsnow: it occurs to me... you're caching all these updates in the context...  I was just going to cache the untrack as a delete.... but what if someone does untrack and then track with the same id? Should the later track removed the cache untrack?  I presume so....
<ericsnow> natefinch: that should work given that tracking events are based strictly on the ID
<natefinch> ericsnow: yeah, just makes this all a lot more complicated.  It's unfortunate we have to do our own caching layer before flushing out to the real data storage
<ericsnow> natefinch: I guess the plugin type must factor in too
<natefinch> ericsnow: hmm.... yeah.  Damn, that's annoying.
<ericsnow> natefinch: it's definitely an unexpected corner case, but the first tracked proc may have the same ID as the replacement, but a different type
<ericsnow> natefinch: that little facet certainly complicates the matter a little
<natefinch> ericsnow: yeah.... I wonder if that's actually a problem with all our code... we currently only use the id for identifying the process
<ericsnow> natefinch: it's only in the hook context that it's a problem an only in the untrack...track case (without a flush in between)
<ericsnow> s/an/and
<ericsnow> natefinch: basically it's an artifact of caching operations between flushes
<natefinch> ericsnow: api get just takes ids.  If more than one process has the same id (but different type), you'd get back more than one process
<ericsnow> natefinch: presently it isn't an issue because basically we flush after every operation (instead of waiting until the conclusion of the hook context)
<ericsnow> natefinch: ID is the key (per unit) so there is only one
<natefinch> ericsnow: gotta run, but I'll talk to you when I get back.
<ericsnow> natefinch: only one hook can run at a time for a unit so we won't collide there...but at the point that we have workers touching state we'll have to introduce some sort of refresh in the hook context component
<ericsnow> k
<thumper> mramm: ping
<mramm> pong
<thumper> mramm: can we have a chat in about 30 minutes?
<mramm> probably
<thumper> master of committment there
<mramm> thumper: we can meet in 30-40 min for sure
<mramm> just not sure how long this danwest meeting is going to go
<thumper> kk
<menn0> thumper: regarding that mongod not starting issue... one of the reporters of the bug just attached the syslog output and the problem is the shared-secret file isn't being written out
<menn0> https://launchpadlibrarian.net/214260999/syslog.txt
<menn0> thumper: should be fixable now at least
 * thumper nods
<thumper> is it a race?
<menn0> thumper: i've only just seen this so haven't checked but that seems likely
<menn0> thumper: given that it doesn't happen for most people
 * thumper nods
<lazyPower> heyo o/ can i tap someone for info about https://bugs.launchpad.net/juju-core/+bug/1456265
<mup> Bug #1456265: Openstack provider should work without object-store <openstack-provider> <juju-core:Fix Committed by axwalk> <https://launchpad.net/bugs/1456265>
<lazyPower> such as if its currently in the juju-devel ppa for test driving?
<mup> Bug #1484303 opened: keyManagerSuite teardown fails  <ci> <intermittent-failure> <test-failure> <juju-core:New> <https://launchpad.net/bugs/1484303>
<davechen1y> mwhudson: https://groups.google.com/forum/#!topic/golang-dev/vNboccLL95c
#juju-dev 2015-08-13
<mup> Bug #1484308 opened: MachineSuite fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1484308>
<mwhudson> davechen1y: yeah, i should reply to that
<perrito666> Hey guys I am going to sleep I won't make it awake to the team meeting
<natefinch> perrito666: heh, good night.
<natefinch> ericsnow: you around?
<wwitzel3> natefinch: I am if that helps
<natefinch> wwitzel3: there's a line in the hook context's flush that looks dangerous/bad
<natefinch> wwitzel3: https://github.com/juju/juju/blob/feature-proc-mgmt/process/context/context.go#L206
<natefinch> wwitzel3: when we flush, we set updates to nil, but the rest of the code expects that updates will be non-nil
<wwitzel3> natefinch: I thought we explictly checked for len(c.updates) in all those places?
<wwitzel3> natefinch: or we have a !ok check for the key accessor
<wwitzel3> natefinch: if not, we should
<natefinch> wwitzel3: we don't, so we should.  Or just not set it to nil, just set it to an empty map.
<wwitzel3> natefinch: yeah, might as well just set it to an empty map
<axw> wallyworld: re your comment in http://reviews.vapour.ws/r/2330/. I would rather not add that assertion. it's done in other tests already. these tests are big and unwieldy already, I'm trying to make them a bit more targeted
<wallyworld> axw: sure, np
<axw> wallyworld: did you want to review http://reviews.vapour.ws/r/2332/ (azure volume source), or should I go ahead and land it?
<wallyworld> axw: looks like it's had 2 others, so go for it i reckon
<axw> I do so like an unblocked master
<axw> wallyworld: 2nd point in follow-ups in description: "- update storage provisioner to set status when detaching or destroying"
<axw> wallyworld: I can do it in this branch if you prefer
<wallyworld> ah
<wallyworld> tis ok, sorry i missed that
<huwshimi> thumper: Quick question about jujulib, I'm trying to find the package for python-websocket. I couldn't see anything on pypi.
<thumper> :)
<thumper> huwshimi: websocket_client
<huwshimi> thumper: Ah hah!
<thumper> huwshimi: quick hangout?
<huwshimi> thumper: Sure
<thumper> huwshimi: https://plus.google.com/hangouts/_/canonical.com/onyx-standup
<menn0> davechen1y: https://github.com/juju/juju/pull/2977 please
<voidspace> TheMue: morning
<mup> Bug #1484403 opened: provider/openstack: default block source not set to "cinder" <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1484403>
<mup> Bug #1484419 opened: Local provider: fail to download charm from state server when using an isolated network <landscape> <juju-core:New> <https://launchpad.net/bugs/1484419>
<voidspace> dooferlad: ping
<TheMue> yeeeehaw, mini provider mock and envrion work for test
<mup> Bug #1307445 changed: openstack: instances with multiple nics on the same network don't have deterministic private-addresses <addressability> <amd64> <apport-bug>
<mup> <ec2-images> <network> <openstack-provider> <trusty> <juju-core:Triaged> <juju-core (Ubuntu):Triaged> <https://launchpad.net/bugs/1307445>
<perrito666> marcoceppi: ping?
<marcoceppi> perrito666: yooooo
<perrito666> liberal icmp protocol
<marcoceppi> icmp for the 21st centry
<perrito666> marcoceppi: can you take a look at this review comment and give me your opinion on how we should display the new available version on tabular format? http://reviews.vapour.ws/r/2314/#comment15670
<perrito666> ill give some context
<perrito666> I have added a new tools version availability check
<marcoceppi> context is for chumps, I dive right in and make uninformed decisions on everything I do ;)
<perrito666> that will display, in status, whenever a new version is available for upgrade
<marcoceppi> so, that will only display when you have a newer version of tools available?
<perrito666> yep
<marcoceppi> perrito666: aesthetically I'd prefer it if had a header, something like [Juju] and then just a Running: and Available: key, but that's really just personal opinion to keep everything inline style wise. No one is going to be parsing the tabular output so I'm fine whatever people choose. My only real concern is the amount of realestate it takes, if it's going to consume 4 lines at the top it better be damn important because in deployments of >
<marcoceppi> 5 services and 10 units info on normal displays start to truncate while running watch
<marcoceppi> perrito666: also, how does this look in yaml and json? is it parsable there?
<perrito666> marcoceppi: in yaml and json is parsable
<perrito666> it is at the same level as environment
<marcoceppi> cool
<marcoceppi> perrito666: so, tl;dr; I'd prefer a heading and simple output (just the keys) but I don't have a strong opinion on the subject, I'd just find it pleasing to the eye
<perrito666> marcoceppi: well you are more of a user than we are
<perrito666> and position, top?
<marcoceppi> perrito666: hum, that's a good question. Considering most people won't be upgrading the moment a new release comes out (IS, for example) having that nag as the top item seems wasteful. I'd probably possition it between machines and units but that feels weird too
<marcoceppi> perrito666: having it at the bottom is fine too. I think it's a bit foolish to assume juju is the most important thing to display in the output. To us it may be, to the user I think they're more concerned about their workload
<perrito666> well shame on them
 * perrito666 is about to go fix mramm's connection
<perrito666> marcoceppi: thanks for your imput
<voidspace> dimitern: TheMue: if you have time http://reviews.vapour.ws/r/2360/
<voidspace> relatively straightforward
<voidspace> dimitern: TheMue: dammit, it's not ready yet - belay that
<voidspace> dimitern: TheMue: ok, it's ready now...
<voidspace> (again...)
<marcoceppi> why do we hardcode series in Juju?
<natefinch> marcoceppi: just to piss everyone off
<natefinch> marcoceppi: including ourselves
<marcoceppi> natefinch: but like, I'm serious. What's the use for it in juju? I'm trying to understand the justification
<marcoceppi> (without reading a bunch of go code)
<natefinch> marcoceppi: ostensibly it's so we can know if someone is using a series that's too old.  Though I would think the lack of tools would be a perfectly acceptable way to stop them from deploying to a machine where it wouldn't work.
 * marcoceppi considers ripping all this out and just submitting a patch
<jcastro> yeah so, we're broken on OSX and Windows because of this
<jcastro> natefinch: what do you need from us to help?
<natefinch> honestly, what we should do is hardcode the series that are too old, since those are static.
 * marcoceppi hates the idea of hard coding anything
<natefinch> jcastro: what problem are you seeing?
<marcoceppi> Simplestreams seems to be the best means
<jcastro> doesn't work on OSX El Capitan or Windows 10
<natefinch> le sigh
<jcastro> like, we don't work at all currently
<natefinch> for the client... we shouldn't look at series at all. If the code can run enough to look at series, it's probably fine.
<jcastro> +50
<jcastro> natefinch: in the meantime: https://github.com/AdamIsrael/juju/commit/909bf8e3074c0bae25579e84f41a71996fe61a3f
<natefinch> jcastro: gah, yeah.  Send a PR
<marcoceppi> natefinch: https://github.com/juju/juju/pull/2969
<jcastro> natefinch: we'll need a fix for Windows 10 as well, which is actually released and in the wild.
<jcastro> natefinch: incoming from marco in a few minutes
<natefinch> jcastro: cool.  I told CI to merge the OSX one
<marcoceppi> OK
<marcoceppi> I understand why we have series matching for client, it appears to be able to map caveats for things?
<marcoceppi> like on Linux do this on Windows do this and on OSX do this?
<natefinch> marcoceppi: we don't need series for that, just OS, and Go gives us OS detected (i.e. darwin vs linux vs windows).  What it doesn't give us is, for example, trusty vs Wily or centos vs ubuntu
<marcoceppi> natefinch: I'm just trying to figure out how to remove hard coding versions all together
<marcoceppi> like, why do we need to know what version of osx or windows you're running at that point?
<natefinch> marcoceppi: I am all for that
<marcoceppi> natefinch: well, you'
<marcoceppi> ll love this patch
<natefinch> marcoceppi: one trick is local provider
<marcoceppi> igaf about local provider, since it's only Ubuntu and we should be able to detect ubuntu fairly well given we created the OS
<marcoceppi> idgaf*
 * katco sitting in the standup all by my lonesome, wwitzel3 natefinch ericsnow 
<perrito666> katco: introspective standup :p
<bogdanteleaga> anybody in for a fast one? http://reviews.vapour.ws/r/2361/
<aisrael> natefinch: Sorry, I fixed the failing test for that osx patch
<natefinch> aisrael: cool
<TheMue> dimitern: I'm postponing the test of isNetworkingEnvironment() in cmd/jujud/agent/manchine.go. as we talked about this morning SupportsNetworking() does not rely on the feature flag but on a type assertion of the environment
<TheMue> dimitern: will add a card for it
<natefinch> aisrael: can you add back a test for version 16 producing series unknown etc?
<aisrael> natefinch: done
<ericsnow> cmars: I'd like to have a chat about dependency engine at some point today if you have some time
<katco> wwitzel3: you going to be around for our 1:1?
<wwitzel3> katco: hey, sorry, just woke up
<katco> wwitzel3: no worries... in a meeting atm
<dimitern> TheMue, sounds good
<katco> wwitzel3: k sorry... are you up for a 1:1? or is your throat too sore?
<mup> Bug #1484606 opened: bootstrap fails if control bucket not specified and exists <juju-core:New> <https://launchpad.net/bugs/1484606>
<mup> Bug #1484617 opened: Juju doesn't honour agent-stream: released in the config <blocker> <ci> <regression> <streams> <juju-core:Triaged> <https://launchpad.net/bugs/1484617>
<perrito666> Bbl
<mup> Bug #1484688 opened: [Juju HA] Past logs do not get synced to new slave state servers <juju-core:New> <https://launchpad.net/bugs/1484688>
<alexisb> cherylj, you around?
<alexisb> o you are in standup
<cherylj> alexisb: in standup
<alexisb> nm, ping me post standup please
<cherylj> alexisb: will do
<mup> Bug #1478232 changed: juju 1.24 poor performance <cpec> <sts> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1478232>
<thumper> mramm: ping?
<mup> Bug #1450092 changed: juju fails to bootstrap local provider with vivid and upstart <systemd> <upstart> <vivid> <juju-core:Won't Fix> <juju-core 1.23:Won't Fix> <juju-core 1.24:Won't Fix> <https://launchpad.net/bugs/1450092>
<mup> Bug #1460087 changed: quickstart deployment fails to add relations when bootstrap goes "down" <deploy> <quickstart> <juju-deployer:Triaged> <https://launchpad.net/bugs/1460087>
<mup> Bug #1484718 opened: String config values can't be set as all integers <juju-core:New> <https://launchpad.net/bugs/1484718>
<mup> Bug #1410876 changed: Error executing lxc-clone: lxc_container: utils.c: mkdir_p 220 Not a directory - Could not destroy  snapshot %s - failed to allocate a pty; Insufficent privileges to control  juju-trusty-lxc-template <lxc> <oil> <openstack-provider> <stakeholder-critical> <trusty> <uosci>
<mup> <juju-core:Invalid> <lxc:New> <https://launchpad.net/bugs/1410876>
<mup> Bug #1473450 changed: upgrade-juju from 1.18 to 1.20.14 leaves some agents down <sts> <upgrade-juju> <juju-core:Fix Released> <https://launchpad.net/bugs/1473450>
<perrito666> mm, gce as a provider works really well, cheers wwitzel3 and eric
#juju-dev 2015-08-14
<wwitzel3> perrito666: thanks :) glad to hear it
<wallyworld> axw: a minute late, just finishing with tim
<axw> wallyworld: sure, just ping when you're ready
<axw> I'll go make some tea
<wallyworld> axw: bring your tea
<menn0> thumper: can you have a look at this please? http://reviews.vapour.ws/r/2366/diff/
 * thumper looks
<menn0> thumper: i've realised that there's yet one more PR required: for the api package
<thumper> k
<menn0> thumper: even though Juju won't call WatchAllEnvs itself we should probably have it implemented (WatchAll is)
<menn0> everything takes too long...
<thumper> yeah
<mup> Bug #1339967 changed: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>
<mup> Bug #1339967 opened: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>
 * thumper heads out to drop down two laptops to the repair person
<mup> Bug #1235217 changed: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>
<mup> Bug #1339967 changed: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>
<mup> Bug #1427505 changed: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>
<mup> Bug #1235217 opened: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>
<mup> Bug #1427505 opened: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>
<mup> Bug #1235217 changed: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>
<mup> Bug #1427505 changed: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>
<mup> Bug #1484789 opened: add-machine fails with hosted environments <juju-core:New> <https://launchpad.net/bugs/1484789>
<voidspace> TheMue: if you get a chance, could you look at http://reviews.vapour.ws/r/2360/
<TheMue> voidspace: morning, will do now
<voidspace> TheMue: morning
<voidspace> thanks
<TheMue> voidspace: you've got a review
<mup> Bug #1484833 opened: juju sync-tools fails to complete as environment is not bootstrapped (catch 22 situation!) <juju-core:New> <https://launchpad.net/bugs/1484833>
<dimitern> any takers for a trivial review? http://reviews.vapour.ws/r/2363/
<dimitern> voidspace, dooferlad, TheMue, axw, wallyworld ^^
<TheMue> dimitern: *click*
<dimitern> TheMue, ta!
<TheMue> dimitern: reviewed
<voidspace> TheMue: thanks for the review
<TheMue> voidspace: yw
<voidspace> dimitern: TheMue: dooferlad: be with you in a few minutes
<dimitern> voidspace, ok
<dimitern> TheMue, thanks!
<TheMue> oh, yes, we're starting now, have to fetch my headset
<dimitern> voidspace, ping
<voidspace> dimitern: ome
<voidspace> *omw
<dimitern> voidspace, ping
<dimitern> voidspace, looking at what was scheduled and agreed to deliver for the MVP, subnet add (rather than create) is expected to land, FWIW
<dimitern> voidspace, and just space create, and both of subnet|space list - that's it
<dimitern> (along with constraints ofc)
<dooferlad> dimitern, voidspace, TheMue: I am back this afternoon. Will take a look at the CI test reviews I have got and the board to see the new plan. Anything urgent that should take priority?
<TheMue> dooferlad: ah, you're back, fine. how do you feel?
<mup> Bug #1484930 opened: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>
<dimitern> dooferlad, awesome!
<dooferlad> TheMue: a bit foggy, but mostly fine
<dimitern> dooferlad, if you don't feel well though, better not though
<dimitern> dooferlad, nothing is *that* important ;)
<TheMue> dimitern: +1
<dooferlad> dimitern: I will stop if I think I need to. I can at least get on with low brain effort tasks.
<dimitern> dooferlad, only you can know that :)
<dimitern> dooferlad, I'll leave it to your judgment
<dooferlad> dimitern: sure, but once you have run out of paid sick days, it sure feels like the company wants you back :-|
<mup> Bug #1484930 changed: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>
<dimitern> dooferlad, it happens sometimes, but I'm sure we've not gone there yet
<dooferlad> dimitern: oh, I have run out.
<dimitern> dooferlad, don't worry about it for now
<mup> Bug #1484930 opened: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>
<dooferlad> dimitern: why do we want a reference count for state.subnet? We can just perform a simple mgo query to see which spaces are using a subnet (if that is what it is for)
<dimitern> dooferlad, it's about which subnets have running (etc.) instances
<dooferlad> dimitern: still sounds like a DB query...
<voidspace> dooferlad: in which case the relationship between instances and subnets needs to be stored somewhere
<voidspace> dooferlad: in fact that's true anyway
<dimitern> voidspace, dooferlad, otp, sorry
<dooferlad> voidspace: That is what I thought. Seems a shame to add what seems to be a workaround to something that could easily be managed by our single source of truth.
<voidspace> dooferlad: well, we don't yet have a single source of truth - that's a big part of that card
<voidspace> dooferlad: references could actually be "list of machine ids" I guess
<dooferlad> voidspace: that is backwards. Each machine is part of 1+ spaces, so you get the machine document to store the spaces it is in.
<voidspace> dooferlad: it needs to know subnet, not just space
<dooferlad> voidspace: then you can query spaces that contain subnets, then machines in those spaces
<dooferlad> Actually, you are right, you want to store subnets that a machine is in
<voidspace> dooferlad: that doesn't tell you which machine is using any specific subnet
<voidspace> right
<voidspace> I suggested putting it on the machine record, dimitern wasn't so kee
<dooferlad> then if you move a subnet between spaces you don't need to change anything
<voidspace> *keen
<voidspace> he'd have to explain why
<voidspace> part of it maybe transactional
<voidspace> dooferlad: our transaction simulation doesn't actually work across collections
<voidspace> dooferlad: so keeping things like reference count in their own collection avoids some problems
<voidspace> dooferlad: for IPAddresses we store machineId (and instanceId) on the IPAddress
<dooferlad> http://blog.labix.org/2012/08/22/multi-doc-transactions-for-mongodb - "Operations may span multiple collections"
<voidspace> dooferlad: operations can, the transaction guarantees aren't the same I believe
<voidspace> dooferlad: that's what I was told, although that documentation doesn't seem to agree with it :-)
<dooferlad> I am going with the documentation.
<voidspace> dooferlad: good luck :-)
<voidspace> dooferlad: there maybe another (better) reason to prefer reference counts in their own collection
<voidspace> dooferlad: and it may or may not apply to storing subnet <-> machine relationships
<voidspace> not amazingly helpful I realise... but I know fwereade prefers we do that for reference counts at least
<dooferlad> My problem is this: we have a database. You don't need reference counts if you have the right relations. If you have reference counts you have to work hard to make sure they are right.
<voidspace> dooferlad: ah, see doc/hacking-state.txt
<voidspace> dooferlad: the reason for a separate doc is for the sake of watchers
<voidspace> watchers don't need to be notified when reference count changes and *watching* only works at the whole doc level
<voidspace> I believe...
<dimitern> voidspace, dooferlad, instance state != machine life
<voidspace> dimitern: what does that have to do with it?
<TheMue> dimitern: quick review of http://reviews.vapour.ws/r/2369/ ? isn't complex
<dimitern> voidspace, dooferlad, that's why we need a refcount + txn.Ops on it to avoid removing subnets in use
<dimitern> (even if no instance are yet/anymore running on it)
<voidspace> dimitern: we're not proposing anything to do with instance state
<voidspace> dimitern: we're saying that we need to store machine ids using subnets *anyway*
<dimitern> TheMue, sure, looking in a moment
<voidspace> dimitern: in which case a subnet is in use if the number of alive machines using it > 0
<voidspace> dimitern: so you don't need a reference count, you count alive machines
<voidspace> dimitern: using the db
<dimitern> voidspace, you can't store machine ids that don't exist yet though
<voidspace> dimitern: isn't the machine id created straight away?
<voidspace> dimitern: it's instance id that is created later
<dimitern> voidspace, i.e. as a side-effect or provisioning with space constraints
<voidspace> dimitern: as soon as you provision with contraints a machine will be created, right? and I don't know what you mean by "side-effects"
<voidspace> dimitern: picking a subnet is part of provisioning - i.e. part of machine record creation
<dimitern> voidspace, I mean a service's space requirements indirectly should keep its subnets in state
<voidspace> dimitern: a service constraint for undeployed services won't keep any specific subnets in use
<dimitern> voidspace, and by requirements I realize constraints are a bad example (except maybe in environment constraints)
<katco> natefinch: standup
<dimitern> voidspace, but for bindings of a service
<dimitern> voidspace, once we have them (that was the intent behind --networks)
<voidspace> dimitern: in which case we add the machine id to all subnets they are "using"
<dimitern> voidspace, so even if we avoid adding refcounts and use machineIds-to-subnets, we'll need it pretty soon
<voidspace>  I don't think so
<voidspace> we *still* need to do that, even if we use reference counting - so that we could decrement
<dimitern> (and not having it will keep the subnet up *until* the dead machine doc is in state)
<voidspace> (by that I mean - store machine id)
<voidspace> dimitern: in order to decrement properly you have to store somewhere the machine id referencing subnets
<voidspace> dimitern: agreed?
<dimitern> voidspace, ok, first, I absolutely agree we need to be able to link machines to subnets (many-to-many)
<voidspace> even if it's an "implicit" reference as you describe
<voidspace> dimitern: and if we have that link then the reference count *must* equal the number of links
<voidspace> so the reference count is always redundant
<dimitern> voidspace, and the original and perhaps not great model was to use machine's NICs (which have subnets) for that
<voidspace> dimitern: ok, that sounds good - we can do that
<voidspace> any reason not to?
<voidspace> we have those in state already
<dimitern> voidspace, nope - refcount shouldn't stop you from removing a subnet referenced only by non-alive machines
<dimitern> voidspace, the only reason not too (not a good one) is we'll need to update (add) NICs too
<voidspace> right
<voidspace> NICs are already added by container address allocation
<voidspace> yeah, adding NICs on machine creation is non-zero amount of work :-/
<dimitern> complexity, which can be avoided by a machineSubnetsC for the time being
<dooferlad> why do we need subnets --> machines? Again, this is a query.
<voidspace> dooferlad: in order to determine if the subnet is in use
<dimitern> dooferlad, that's what we're discussing
<alexisb> ericsnow, I moved our 1x1 to monday so that you can focus on moonstone planing
<voidspace> dooferlad: so you can tell if it is safe to move the subnet to a different space or remove it altogether (we have a subnet remove command)
<dimitern> dooferlad, to be able to do that query, you need less data than to ensure it stays like this during an multi-op insert
<dooferlad> dimitern: ah, interesting. *thinks*
<voidspace> dooferlad: or are you saying "subnets --> machines" is a query. In which case a query of *what*, that information doesn't exist unless we store it.
<ericsnow> alexisb: thanks!
<dooferlad> voidspace: I get that, but it feels like you should store in the machine document, what subnets it is part of.
<dimitern> dooferlad, yeah, mongo makes trivial design discussions so much spicier :D
<dooferlad> then everything flows from that
<dimitern> dooferlad, preferably not on the machine document
<dooferlad> well, wherever the machines network information is. I don't have that in my head yet and need to look it up.
<voidspace> dimitern: why not? (out of interest)
<dimitern> dooferlad, voidspace, having it there means any change to it potentially triggers a lot of watchers and workers
<dooferlad> I think my argument is that if you have x is a member of y, you should be able to determine that by x storing that it is a member of y. If you store that y has x, y, z as members then if x goes away you need to update that membership list.
<voidspace> dimitern: but it doesn't change after machine creation
<dimitern> dooferlad, voidspace, it changes rarely, but always significantly
<voidspace> dimitern: when does it change?
<dimitern> dooferlad, voidspace, e.g. when space membership changes or subsequent deployments
<voidspace> dimitern: space membership of a *machine*?
<voidspace> dimitern: how is that possible
<dimitern> dooferlad, voidspace, you deploy another service from a different space on the same machine
<voidspace> dimitern: and if it's significant *shouldn't* watchers be triggered...
<voidspace> dimitern: which would require adding an extra NIC, which we won't support initially
<voidspace> dimitern: I believe
<voidspace> or can you bind multiple subnets to the default NIC on an EC2 instance?
<dimitern> dooferlad, voidspace, that brings a new NIC on a subnet in that other space, triggers a lot of address and other changes already
<voidspace> the *right* thing to do is to use NICs to model this
<voidspace> so maybe we should do that
<dooferlad> voidspace: you don't need physical NICs to have >1 IP address (or addressable containers wouldn't work). I wouldn't worry about physical interfaces (or anything that looks like them)
<voidspace> dooferlad: EC2 has a concept of a NIC and different instance types have a maximum number of supported NICs
<dimitern> voidspace, it's not a change of the machine itself (as juju sees it) - exactly like and why we're not storing what services we deploy to a machine
<voidspace> dooferlad: and a max number of IP per NIC
<dooferlad> voidspace: yes, I know
<dimitern> (only indirectly from their units)
<voidspace> dooferlad: I think only one subnet per NIC, I may be wrong
<voidspace> dooferlad: I'm not worrying about physical NICs :-)
<dimitern> dooferlad, I know - that's yet to be decided (the model expects 1-nic-per-space on machines)
<voidspace> dooferlad: I'm worrying about modelling machines on multiple spaces in EC2
<dooferlad> voidspace: That EC2 specific logic should be in one tidy place. It is a restriction on what we can do, not an addition.
<dimitern> with one-nic-multiple-ips some things will be simpler, but impossible on EC2
<voidspace> dooferlad: well, sure
<voidspace> we seem to have sidelined from the main discussion
<voidspace> we have a model of NICs in state already
<voidspace> NICs hold the relationship between subnets and machines
<dooferlad> indeed. back to the point...
<voidspace> so they seem like the right level of abstraction for what we need
<voidspace> and they conveniently match what we're trying to model anyway...
<dooferlad> sounds good :-)
<dimitern> dooferlad, voidspace, it's not just EC2 specific, but AWS is an important target
<dooferlad> dimitern: I completely agree. We just need to make sure that EC2 is a subset, not some blob on the side.
<voidspace> yeah, our model needs to *work* on EC2 - just not necessarily be entirely bound by it's limitations
<dimitern> and NIC in juju terms is more like a linux device than a Elastic Network Interface
<dimitern> dooferlad, of course, we started from the free-for-all openstack model and AWS restrictions constrained the model in a few ways, but made it better in others
<dooferlad> so, we have IPAddress. That contains a SubnetId and a MachineId. So, we can find all the IPAddress documents that reference a SubnetId before we delete it and use that as the reference count, right?
<voidspace> dooferlad: IPAddress is only used for "container address allocation"
<voidspace> dooferlad: not (yet) as a general IPAddress store
<dooferlad> erk
<dooferlad> well, that is horrid.
<voidspace> because it's only for containers that we specifically allocate an address (if the feature flag is on)
<voidspace> we get them automatically for machines
<dooferlad> what do we use for machines?
<voidspace> so no need to request or model them
<dooferlad> to store them that is
<voidspace> well, to work out which is the current public/private address for a machine we spit on our finger and stick it in the wind
<voidspace> which is why we have all sorts of bugs around the private/public address of machines appearing to change...
<dooferlad> machineDoc has two slices of IP addresses.
<voidspace> (we ask the machine OS and the provider what addresses it thinks it has and report those - holding them on the machineDoc)
<voidspace> right
<dooferlad> but no CIDRs
<dooferlad> great
<dooferlad> oh, hang on...
<voidspace> we have a bug we have to fix reqarding machine IP addresses apparently changing
<voidspace> have to fix  *soon*
<voidspace> so moving to a model that actually works would be better
<dooferlad> the address struct that machine uses could become ipaddressDoc.
<dooferlad> that would make life easier...
<dooferlad> then we use that plus a list of machines not yet provisioned that have been allocated a subnet
<dooferlad> which I assume is generated from another query looking at constraints
<dooferlad> (assuming we can't just look at constraints)
<natefinch> katco: dang.... somehow my copy of the link failed, can you re-paste?
<TheMue> dooferlad: dimitern: thx for reviews. the whitebox test does not rely on BaseSuite, so no PatchValue() this time.
<mup> Bug #1484993 opened: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>
<mup> Bug #1484993 changed: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>
<mup> Bug #1484993 opened: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>
<dimitern> TheMue, you can still call it from juju/testing
<dimitern> TheMue, at least it does the same thing
<TheMue> dimitern: will have a look, thought it's a method of an embedded type, which isn't used here
<TheMue> dimitern: in the moment the patched attempt strategie like you said by PatchAttemptStrategies isn't working
<dimitern> TheMue, it is a method of BaseSuite via CleanupSuite, which calls PatchValue inside
<TheMue> dimitern: so I would have to embed the CleanupSuite into the whole suite only for this one patching of a variable I directly have access to
<dimitern> TheMue, what a moment
<TheMue> dimitern: will see how the side effects are
<dimitern> TheMue, s/wait/
<TheMue> yep
<dimitern> TheMue, environSuite (in whitebox test) uses PatchValue already
<TheMue> the whitebox tests are a bit ... different ;)
<dimitern> TheMue, the other environSuite (non-whitebox one) directly embeds BaseSuite
<dimitern> looking provider/maas on master
<TheMue> so, back, just had phone, sorry
<TheMue> dimitern: yeah, for the non-wb I've seen, but missed it here
<dimitern> TheMue, whitebox tests effectively use providerSuite, which already uses PatchValue as it embeds CleanupSuite via Fake(Juju)HomeSuite
<TheMue> dimitern: yes, deep nested embedding, almost like the "good" old OOP
<dimitern> TheMue, luckily rogpeppe's godef is amazing for navigating deeply nested types
<TheMue> dimitern: have to see how I can integrate it into my editor macros
<dimitern> TheMue, +1
<voidspace> right EOW
<voidspace> happy weekend everyone
<TheMue> dimitern: hehe, this bad foo == 5, gc.Equals, true is a leftover from some former approaches
<TheMue> dimitern: PatchValue works fine now
<TheMue> dimitern: even optimised it using variables for retries and enoughRetries to patch it only once, final problem is now the strategies patching
<dimitern> TheMue, great!
<dimitern> TheMue, just use PatchValue for the strategies
<TheMue> dimitern: yep, thought the same
<dimitern> TheMue, patching it once and using a retries chan for reporting back
<mup> Bug #1485013 opened: MgoSuite tests must be run with MgoTestPackage <ci> <intermittent-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1485013>
<mup> Bug #1485017 opened: add the ability to open terminal in hook context <juju-core:New> <https://launchpad.net/bugs/1485017>
<TheMue> dimitern: hmm, think it's more simple now, will push it so that you can have a final look
<TheMue> dimitern: so, updated
<dimitern> TheMue, looking
<TheMue> dimitern: thx for review, will now merge it
<dimitern> TheMue, go for it
<TheMue> dimitern: I added the cards for dummy, maas and ec2. the original one is still there, I grouped them at the end of the lane, so you'll easier find them. I also change the "master card" to a user story, the points are still in there. IMHO these have to be removed due to the points on the individual cards now.
 * TheMue grumbles, has branch cannot land due to a fix blocker *sigh*
<alexisb> TheMue, you could fix it ;)
<TheMue> alexisb: hehe, sure, but then I have to fix my wife too
<alexisb> I assigned it to Ian because it is his PR that caused the regression but really it is up for anyone to fix that is affected by the block
<alexisb> heh understood
<alexisb> it is friday evening :)
<TheMue> alexisb: yep, but sill will have a look
<dimitern> TheMue, got it, thanks!
<TheMue> dimitern: yw
<mup> Bug #1485071 opened: juju deploy and add-machine not installing agent <juju-core:New> <https://launchpad.net/bugs/1485071>
<katco> ericsnow: wwitzel3: sorry, also forgot we have to take care of this bug: https://canonical.leankit.com/Boards/View/114568542/116652767
<wwitzel3> oh right
<natefinch> ericsnow: what do you want to do about the problem with my untrack code and ids that could conflict if they're for different workload types?
#juju-dev 2015-08-15
<mup> Bug #1468349 changed: discoverySuite.TestDiscoverServiceLocalHost: invalid series for wily and vivid <centos> <test-failure> <unit-tests> <vivid> <wily> <juju-core:Fix Released by dooferlad> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1468349>
<mup> Bug #1485145 opened: metadata generate-tools ignores product path in streams <ci> <metadata> <streams> <juju-core:Triaged> <https://launchpad.net/bugs/1485145>
#juju-dev 2016-08-15
<wallyworld> anastasiamac: why did you tag 1612645 as a blocker?
<anastasiamac> wallyworld: for 1.25 not master
<wallyworld> ok
<anastasiamac> wallyworld: it's not a critical on master so should not block it
<wallyworld> we haven't really considered adding a new region being a blocker before
<wallyworld> there's plenty of other regions to choose from in the interim
<wallyworld> i guess it's 1.25 so it doesn't matter so much
<anastasiamac> wallyworld: i've added based on comments
<anastasiamac> wallyworld: and precedence is not a great driver when we strive to improve :D
<axw> menn0: which HO?
<wallyworld> i guess my definition of what a blocker is is different
<menn0> axw: good question. let's use the standup one
<menn0> wallyworld: standup hangout?
<anastasiamac> wallyworld: plz get a chance to review https://github.com/juju/juju/pull/5869
<thumper-dogwalk> menn0: all done chatting?
<menn0> thumper: yep
<thumper> anything problematic?
<wallyworld> ok
<menn0> thumper: nope all good actually - I'm removing work from the MM completion doc as the way the config stuff was done works well with MM
<thumper> coolio
<wallyworld> anastasiamac: one thing to add to tech debt sprint - finish complete removal of env storage, including http://reviews.vapour.ws/r/4303/
<anastasiamac> wallyworld: pleaae bug it up and add `tech-debt` tag. the plan is to tackle that list
<axw> wallyworld: FYI I've just verified that destroy-controller ends up cleaning juju from a workload machine
<axw> with my change
<wallyworld> axw: awesome thanks
<wallyworld> anastasiamac: there may be a bug already, i'll try and find it
<anastasiamac> wallyworld: \o/ even better
<axw> wallyworld anastasiamac: if there's any chance that the MAAS provider could be updated to use tags instead of storage at the same time that would be great
<wallyworld> axw: i think maas 2.0 can do that but not 1.9 :-(
<axw> probably out of scope though
<anastasiamac> axw: ^^ would b awesome to bug it up and tag as 'tech-debt' :D altho, I m not sure what our stance is with maas 1.9.. probably criticals only too?..
<axw> wallyworld: we can keep it for just 1.9
<wallyworld> that is true. in that case, we should do it
<axw> anastasiamac: yeah not sure if there's a bug, will have a look
<wallyworld> now that we have 2 provider implementations
<axw> anastasiamac: covered by this I think: https://bugs.launchpad.net/juju-core/+bug/1611267
<mup> Bug #1611267: maas:  kill-controller should take down known hosted models <juju-core:Triaged> <https://launchpad.net/bugs/1611267>
<anastasiamac> axw: k. i'll a tech-debt card
<anastasiamac> tag*
<axw> anastasiamac: I've  just tagged it
<anastasiamac> axw: u r awesome
<anastasiamac> axw: of course, u'd b even more awesomer (!) if u joined the sprint that would fix it :D
<axw> anastasiamac: is it going to be in Perth? ;p
<anastasiamac> axw: :) dunno yet. i need to talk to HR/sprint org admins... unlikely tho :)
<anastasiamac> axw: if it was, do i have ur word, u'd join?
<axw> anastasiamac: tongue is in cheek, nobody wants to go to perth
<anastasiamac> axw: i can think of at least couple of ppl who'd b keen and several others are not attached to a location :) so everything is possible...
<axw> oh true, forgot about andy :p
<anastasiamac> axw: :) that's one of these ppl \o/
<axw> wallyworld: the more I dive into azure auth, I don't think we're going to be able to automatically generate creds. AFAICS, authentication with username/password must be interactive
<wallyworld> awesome :-(
<axw> wallyworld: that would be OK for bootstrap, but not for adding creds afterwards
<axw> because you might not have custom azure code in the client
<axw> wallyworld: I think the best we're going to be able to do for now is a separate plugin/tool
<wallyworld> we'll need to identify the gaps and talk to the ms guys maybe
<axw> yep
<wallyworld> stop gap would be good, that can probably happen next week
<blahdeblah> quick Q: juju 1.x has "api-endpoint" and "api-info" - juju 2.x doesn't.  How should a client program work out the correct API endpoint of the controller under 2.x?
<blahdeblah> anastasiamac: ^ That's the root cause of the issue we were talking about earlier
<anastasiamac> blahdeblah: tvm! thumper:wallyworld:axw:menn0: ^^
<blahdeblah> anastasiamac: My issue with juju-deployer on 2.x, that is.  I don't think this is going to help you prioritise or fix any bugs. :-)
<wallyworld> juju list-controllers
<axw> blahdeblah: "juju show-controller" contains the list of addresses
<axw> the first one in the list is the most recently used
<anastasiamac> blahdeblah: i figured ;)
<blahdeblah> axw: where "most recently used" == the one that juju switch will say is selected?
<wallyworld> juju switch pertains to controller/model. most recently used refers to the api endpoints for a given controller
<wallyworld> controller cluster
<wallyworld> in an HA setup, there's more than one endpoint to choose from, they are rotated
<blahdeblah> What about if you've got multiple client processes locally hitting multiple juju envs on different controllers?  Is there a way to reliably determine from the CLI which API endpoint you should hit for env X?
<wallyworld> juju show-model will get the controller uuid
<wallyworld> from there you can get the endpoints
<wallyworld> not sure if there's a better way, would need to look into it
<blahdeblah> wallyworld: that will do for now
<blahdeblah> show-controller gives the uuid, so they can be compared
<wallyworld> blahdeblah: it could be with the 2.x rework stuff has got harder in some areas, so would be good to know those pain points
<wallyworld> raise a bug if you want it tracked etc
<blahdeblah> wallyworld: yeah - that's the point of what I'm doing today; try to find where the gaps are in getting us ready for 2.x
<wallyworld> hope you don't find too many :-)
<blahdeblah> wallyworld: well, the big one is juju-deployer
<wallyworld> i thought folks had been using that ok with the recent api changes to it
<blahdeblah> well, I could be doing it wrong
<wallyworld> i'm only going on hearsay, i've not used it
<blahdeblah> I've just pulled down the trunk version of that + python-jujuclient, and it's definitely not working out of the box
<wallyworld> there was an email to the list recently i think about it all being updated
<wallyworld> not sure now though
 * blahdeblah goes to find it
<blahdeblah> Anyway, context for all of this is that in the conversations anastasiamac and I have been having, it has become evident that we'll need to have a 2.x story if we want our non-critical bugs fixed; so that's what I'm deep-diving into today.
<menn0> thumper: https://github.com/juju/juju/pull/5992 pls
<mup> Bug #1613150 opened: migrationmaster continually restarts once a model has migrated back to a controller <juju-core:New for menno.smits> <https://launchpad.net/bugs/1613150>
<mup> Bug #1613150 changed: migrationmaster continually restarts once a model has migrated back to a controller <juju-core:New for menno.smits> <https://launchpad.net/bugs/1613150>
<mup> Bug #1613150 opened: migrationmaster continually restarts once a model has migrated back to a controller <juju-core:New for menno.smits> <https://launchpad.net/bugs/1613150>
<menn0> thumper: I was surprised to find the user access constants in core/description too
<thumper> :(
<blahdeblah> Do you folks have any API-level doc for juju2?  I'm waist-deep in the python-jujuclient code, and all I can work out is that its getting refusals from the API server based on the parameters its constructing.  But I have no idea what those should be.
<menn0> thumper: i think someone is misunderstanding what core/description is for
<thumper> yeah
<thumper> blahdeblah: which call?
<blahdeblah> thumper: login
<thumper> which params do you have questions about?
<thumper> I'm not clear on the macaroon stuff
<blahdeblah> thumper: All of them?  I suppose I should just read the source and find out where the error is originating and infer back from there, but go is almost as unreadable as lisp to me...
<blahdeblah> thumper: Here's what's happening: https://pastebin.canonical.com/163111/
<blahdeblah> Line 8 is the params its sending on the RPC call
<blahdeblah> I hacked it to remove the password, because the Canonistack controller doesn't seem to want one.
<blahdeblah> But it gives the same response with or without a password
<thumper> blahdeblah: you need to prefix "user-" to the auth-tag
<thumper> it is expecting a stringified tag,
<blahdeblah> hang on - I just realised that
<thumper> and user tags all need "user-"
<blahdeblah> and I had fixed it, but not pushed the code to my live copy
 * blahdeblah tries again
<blahdeblah> OK - here's the result this time: https://pastebin.canonical.com/163112/
<blahdeblah> thumper: and if I include 'credentials' (which is None), I get the same result
<blahdeblah> I really wonder whether this is due to the Canonistack controller running a hacked authentication package. <-- wallyworld, any thoughts?
<thumper> blahdeblah: it'll be expecting some creds now
<thumper> not sure about the macaroon bits...
<blahdeblah> What are "the macaroon bits"?
<thumper> password?
<thumper> opaque credentials with magic HMAC chaining
<blahdeblah> there's no password in ~/.local/share/juju/accounts.yaml
<wallyworld> blahdeblah: i think you need to juju login
<blahdeblah> So all of this was just another missing step? :-(
<blahdeblah> wallyworld: where username == launchpad username?  Or launchpad email address?
<blahdeblah> ERROR already logged in
<wallyworld> hmmm, not sure in that case
<wallyworld> uros will be online in  a couple of hours
<blahdeblah> hmmm - pretty sure there was a login step actually, and it worked, because when I log out it removes the accounts.yaml entry for that controller
<wallyworld> that seems correct
<blahdeblah> OK, I might wait for Uros
<blahdeblah> thanks
<wallyworld> yeah, sorry, he'll know straight away what's wrong
<veebers> Hmm, If I've added a user with permission 'write' and attempt to add a model and get this error, what might I be doing wrong? cmd supercommand.go:458 permission denied (unauthorized access)
<veebers> I"m confident that it's passing --config authorized-keys="..."
<veebers> menn0: Would you have any insight? ^^ user with write perms should be able to add a model right?
<menn0> veebers: writer perms on the controller model?
<veebers> menn0: hmm, I believe so, let me check the test code.
<wallyworld> veebers: write only pertains to models
<wallyworld> you need addmodel permission on controller to add models
<wallyworld> controller permissions are login, addmodel, superuser
<wallyworld> if that's ot in the release notes we'll need to fix them
<veebers> wallyworld: ah ok, seems I'm going off old information (I'm resurrecting some old code of mine)
<wallyworld> axw: if you had time, here's a small tweak to fix upgrade-juju http://reviews.vapour.ws/r/5433/ (main change was a few lines, but then tests...)
<veebers> wallyworld: so this makes sense? juju add-user myusername --models testmode,controller --acl 'addmodel' -c controllername
<veebers> err, testmode -> testmodel. Or does addmodel only pertain to controller and that will error?
<wallyworld> veebers: i thin --models have been removed
<wallyworld> and -acl is replaced by a positional arg
<wallyworld> check the release notes
<veebers> wallyworld: ack, will do. juju add-user --help lists --models as an arg btw
<wallyworld> looks like another case of cli help out of date
<wallyworld> assuming my memory has not failed me
<veebers> wallyworld: which is the most up to date release notes? https://jujucharms.com/docs/devel/temp-release-notes or https://docs.google.com/document/d/1ID-r22-UIjl00UY_URXQo_vJNdRPqmSNv7vP8HI_E5U/edit# (or some third option)?
<wallyworld> https://docs.google.com/document/d/1ID-r22-UIjl00UY_URXQo_vJNdRPqmSNv7vP8HI_E5U/edit
<veebers> sweet cheers
<wallyworld> i don't know about the jujucharms ones, not looked at them
<axw> wallyworld: reviewed
<wallyworld> ta
<veebers> ugh, I'm really confused now. Trying to add-user with addmodel gives me: error: invalid model access permission "addmodel"
 * veebers re-reads release notes incase he missed something
<veebers> wallyworld: I can't see mention of any of "login, addmodel, superuser" in the release notes, perhaps like you say they need updating. Or perhaps that part hasn't landed yet (as per error I'm seeing)
<wallyworld> veebers: it's in master, are you running from source?
<veebers> wallyworld: I am, I pulled and rebuilt this morning. I'll re-pull now to confirm
<wallyworld> veebers: i tested last week and it seemd to work ok
<wallyworld> i used grant
<wallyworld> after adding a user
<veebers> wallyworld: hmm right, might be what I'm doing wrong (I'm trying to use add-user). Thanks I'll try that
<wallyworld> veebers: i juest checked the code - adduser still has the --models but it shouldn't i don't think, i 'll need to check the spec. and you can only specify model permissions, not controller (by design). i think the model permssions are going away too, but i'll need to check to be sure
<wallyworld> so add-user becomes just that - adds a user
<wallyworld> then use grant afterwards
<veebers> wallyworld: makes sense. When you do check can you confirm re: add-user changes (and using grant) as that'll need a change in the CI tests etc.
<wallyworld> using grant will not change - that's the supported use case, give me a few minutes to check for the other bits
<wallyworld> veebers: https://docs.google.com/document/d/15zwHWctS7WhNIzLNIeG_ly6udIVt_8wRlZTLFqJF-zY
<wallyworld> axw: those 2 things fixed, ok to ship?
<axw> wallyworld: ship it
<wallyworld> ta
<wallyworld> veebers: i added a card to our board to fix the add-user help text etc
<veebers> wallyworld: sweet thanks
<wallyworld> veebers: everything is now positional arguments, and if you leave off model, it assumes a controller permission
<veebers> wallyworld: ack. I'll touchbase with my team to make sure this is known. It's possible it's just me thats out of date
<rogpeppe1> menn0-afk: thanks for the review comments
<rogpeppe> wallyworld: hiya
<wallyworld> hi
<rogpeppe> wallyworld: apparently i need approval from the tech board for this PR to land. any idea what the procedure is for that? http://reviews.vapour.ws/r/5428
<wallyworld> rogpeppe: the PR seems like a reasonable goal to me. pop an email to john, menno, andrew, william just to highlight you want them to check it
<rogpeppe> wallyworld: thanks.
<wallyworld> rogpeppe: maybe cc juju-dev so everyone can see the discussion?
<rogpeppe> wallyworld: ok
<rogpeppe> wallyworld: but that means i have to write a much longer email... :)
<rogpeppe> wallyworld: and not quite so frivolous :)
<wallyworld> whatever works for you then :-) don't want to block
<wallyworld> axw: we'll never want to make an api call to get more than just the defaut cloud tag? seems reasonable to return relevant details. maybe a CLI command may want that info to display?
<axw> wallyworld: maybe, but we don't have a need for it now, so I don't want to guess how it should look any more. as it is, we can get hte default cloud name, and then the details of that cloud
<axw> wallyworld: really I think we should be shuffling the cloud's Regions so that the default one (i.e. the controller's region) is at the front
<axw> then we'd have all we need
<rogpeppe> wallyworld: tbh it's kinda trivial; i think it's probably not worth the noise on juju-dev
<axw> rogpeppe: probably should've emailed just you and frankban about that one
<wallyworld> axw: agree with regions i think. i worry though about designing a chatty diestibuted api. gotta run to school picku, bbiab
<rogpeppe> axw: have you actually done DestroyModels or are you waiting for me to do it?
<axw> rogpeppe: I've done it, merging now
<rogpeppe> axw: awesome, thanks!
<axw> rogpeppe: https://github.com/juju/juju/pull/5991
<axw> rogpeppe: I'm looking at creds now
<rogpeppe> axw: if you have a moment or three, it would be great if you could take a look at http://reviews.vapour.ws/r/5428
<axw> rogpeppe: ok
<axw> rogpeppe: LGTM, please answer katco's issues - QA in particular is the big one
<rogpeppe> axw: yeah
<rogpeppe> axw: and i know for a fact that QA in one instance will fail - just working on fixing that nowq
<rogpeppe> axw: 'cos the command-line tests don't actually test against the real API
<rogpeppe> axw: for DestroyModel
<rogpeppe> axw: the fix will require your DestroyModels that's being tested
<axw> rogpeppe: indeed, we should probably have a featuretest
<rogpeppe> axw: that's better than just making the command-line test use the actual API?
<axw> rogpeppe: that's the way we're going, yeah. at least one "happy path" test in featuretest, the rest are unit tests
<rogpeppe> axw: ok
<rogpeppe> axw: i worry that that's not really enough (is one "happy path" good enough to actually determine that all the variations work?), but fair enough
<axw> rogpeppe: I think your message before indicates you are, but can you confirm that you're happy with this? http://reviews.vapour.ws/r/5435/diff/#
<rogpeppe> axw: looking
<rogpeppe> axw: looks fine to me
<axw> rogpeppe: ta
<rogpeppe> axw: glad to see the back of a bulk API call :)
<axw> shh :)
 * rogpeppe still thinks that the bulk API call thing has been the biggest waste of time ever
<axw> rogpeppe: for the creds issue, I'm thinking of doing this: introducing a credential tag which includes owner, cloud, and cred name. then I'll have the Cloud.Credentials method return the set of credential tags that a user has access to (possibly still with a cloud filter)
<axw> rogpeppe: then I'll update CreateModel to take a credential tag
<rogpeppe> axw: that seems reasonable to me i think
<rogpeppe> axw: CreateModel already takes a CloudCredential field
<rogpeppe> axw: i'm wondering what the best ordering of owner, cloud and cred name is
<rogpeppe> axw: maybe it should be cloud/owner/cred-name
<axw> rogpeppe: I'm not sure. I had started on owner/cloud/cred, but *shrug*
<axw> rogpeppe: bbs, school pickup
<rogpeppe> axw: k
<rogpeppe> axw: are there any existing precedents
<rogpeppe> ?
<blahdeblah> So is there any way to get juju 2.x to tell me where the problem is in my bundle that it's saying is invalid?
<frankban|afk> in
<frankban> morning al
<frankban> all
<rogpeppe> axw: good point about ControllerTag - i didn't know about that
<rogpeppe> axw: i think probably the API should be changed to use that
<menn0> rogpeppe: no worries
<menn0> wallyworld: I'm seeing uploadSuite.TestMockBuildTools failing consistently on my machine but only under Go 1.7 (and in CI it seems)
<menn0> wallyworld: looks to be related to your recent changes
<wallyworld> menn0: axw mentioned there could be a checksum different between 1.6 and 1.7
<wallyworld> so maybe if the algorithm is different our test values need to be changed
<menn0> wallyworld: the checksum and length are different
<wallyworld> awesome, so we have a 1.6 vs 1.7 issue
<menn0> the tools size is 132 under Go 1.7 and 127 under 1.6
<rogpeppe> menn0: do you know if the controller tag UUID is different from the controller model tag's UUID, by any chance?
<menn0> wallyworld: which probably makes sense if it's being built with a different Go version right?
<wallyworld> yeah. can you send me the test failure?
<wallyworld> rogpeppe: they are supposed to be but not yet IIRC
<menn0> rogpeppe: AFAIK, they're always the same, but I think there was talk of making them different. wallyworld or axw might remember?
<wallyworld> we need to fix that
<wallyworld> on the todo list
<wallyworld> test changes will suck
<rogpeppe> menn0: ah, ok, that's good to know
<rogpeppe> menn0: i guess the login API should return a controller tag not a model tag for the controller then
<menn0> rogpeppe: yes, that seems right
<menn0> wallyworld: here's the test failure: http://paste.ubuntu.com/23057634/
<menn0> wallyworld: it also appears to have failed for me during a CI merge run ... although there's no details of the actual failure.
<menn0> http://juju-ci.vapour.ws:8080/job/github-merge-juju/8759/artifact/artifacts/trusty-out.log
<menn0> this is why I started looking
<menn0> that code has zero to do with the change I'm making
<wallyworld> menn0: i see that one intermittently even on 1.6, NFI
<wallyworld> you can go to environs/sync and no errors
<menn0> wallyworld: perhaps the compiler output isn't completely predictable?
<wallyworld> run tests from a level higher using ./... and intermittent
<menn0> wallyworld: well it's failing for me consistently under 1.7. not sure if it's the same issue or not though.
<wallyworld> menn0: i think that mock build tools test is bogus anywat
<wallyworld> i have no idea why we are relying on specific compiler behaviour
<wallyworld> surely the test can be rewritten
<menn0> wallyworld: yeah, seems dumb
<wallyworld> i'll try and look tomorrow
<menn0> wallyworld: so that test doesn't actually compile anything. it just creates a tarball with a fake jujud in it.
<menn0> wallyworld: so it's not compiler specific
<menn0> wallyworld: I bet the tar or gzip implementation has changed in 1.7 though
<wallyworld> yeah, something has changed
<menn0> wallyworld: the test shouldn't be comparing the size or hash ... that's bogus
<wallyworld> +100000
<axw> rogpeppe: no precedent that's relevant AFAIK
<menn0> wallyworld: what it /could/ do is extract the contents and make sure it's as expected. that is 100% predictable.
<wallyworld> agreed
<wallyworld> that's what it should have done
<rogpeppe> axw: i guess my inclination is to keep user and name together as they're together in other places too
<menn0> wallyworld: I'll tackle it now if you like.
<menn0> wallyworld:  easy fix
<rogpeppe> axw: so user/name/cloud or cloud/user/name but not user/cloud/name
<axw> rogpeppe: sure, sounds fine to me
<wallyworld> menn0: if you want, ok, thanks, i'm just tidying up some other stuff, sorry
<menn0> wallyworld: np, I've got it
<axw> rogpeppe: I think I'll go for cloud/user/name. on the CLI, I'll make it so you can refer to credentials when adding a model with either "name" (short hand for current-user/name), or user/name
<axw> rogpeppe: (and the code will tack on the cloud name when crafting the tag)
<rogpeppe> axw: is that in the API or just on the command line?
<mup> Bug #1613200 opened: TestMockBuildTools fails intermittently under Go 1.6 and always under 1.7 <juju-core:New for menno.smits> <https://launchpad.net/bugs/1613200>
<rogpeppe> wallyworld, axw: speaking of tests failing under different Go versions, apiserver/provisioner tests fail under Go tip
<wallyworld> yeah, that's been the case for a while :-(
<rogpeppe> wallyworld: *sigh*
 * rogpeppe goes off to fix it
<wallyworld> hasn't been a huge issue since most are still running go 1.6
<rogpeppe> wallyworld: i think it's very useful to be able to run tip
<wallyworld> those running tip can scratch their own itch :-)
<rogpeppe> wallyworld: because it means we can proactively get fixes into Go before they actually break us later
<wallyworld> true
<rogpeppe> wallyworld: and usually the test failures under tip point to smelly tests rather than bad Go changes
<rogpeppe> wallyworld: ha, in apiserver/provisioner, one of the failing tests (TestProvisioningInfoWithStorage) passes in Go 1.6 when run as part of the whole test suite but not when run on its own
<rogpeppe> wallyworld: the failure is because of image metadata - do you think there might be a mock failure or something?
<rogpeppe> wallyworld: it looks like it's actually managing to get real image metadata from the network
<anastasiamac> rogpeppe: it should b mocked out \o/ please open a bug and tag it with `tech-debt`
<menn0> wallyworld, anastasiamac: fix for TestMockBuildTools - http://reviews.vapour.ws/r/5436/
<menn0> wallyworld, anastasiamac: no rush. I'm about to EOD. feel free to merge if you're happy with it.
<axw> wallyworld rogpeppe: https://github.com/juju/names/pull/70 -- PTAL if you have a moment
<rogpeppe> axw: reviewed
<axw> rogpeppe: ta
<babbageclunk> dimitern: no fwereade still :(
<rogpeppe> wallyworld: yeah, that test is definitely dialing out to 91.189.88.141
<rogpeppe> anyone know what mechanism is supposed to stop test code from dialing out to the ubuntu cloud images server?
<wallyworld> rogpeppe: sorry, was having dinner. john found te same root cause a while back. i think tge issue is that patching the DefaultImaheURL to "" fails
<rogpeppe> wallyworld: ok, thanks
<rogpeppe> wallyworld: DefaultImageURL doesn't seem to exist as a word in the juju sources
<wallyworld> it's something like that, i can look
<rogpeppe> wallyworld: do you mean DefaultUbuntuBaseURL ?
<wallyworld> probs, let me check
<wallyworld> that and also DefaultJujuBaseURL
<wallyworld> rogpeppe: there's also this function PatchOfficialDataSources()
<rogpeppe> wallyworld: well, it looks like withoutControllerSuite doesn't call that function
<wallyworld> joy
<rogpeppe> wallyworld: i don't understand why it only fails sometimes though
<wallyworld> me either, but i haven't looked too deeply into it
<frobware> babbageclunk: ping - can we sync/HO?
<babbageclunk> frobware: sure
<babbageclunk> frobware: where?
<babbageclunk> frobware: just in core?
<frobware> ok
<rogpeppe> wallyworld: ok, i've uncovered the root of the issue - our "prevent outgoing calls" hack doesn't work under Go 1.7: https://play.golang.org/p/P4L5nQTsBp
<rogpeppe> wallyworld: that program prints "ok" which it shouldn't
<wallyworld> actually, that rings a bell from when john looked at it
<wallyworld> obviously no one has looked into how to fix
<rogpeppe> wallyworld: it's quite easy to fix
<wallyworld> awesome
<rogpeppe> wallyworld: in Go 1.7 and later, DialContext overrides Dial
<rogpeppe> wallyworld: and DialContext is set in DefaultTransport by defailt
<rogpeppe> wallyworld: so we're setting Dial but DialContext is overriding it
<wallyworld> ah, i see, makes sense
<rogpeppe> wallyworld: so we need to set DialContext too
<wallyworld> maybe i should try Go 1.7, i guess there's a PPA somewhere
<frobware> dimitern: ping - can we sync?
<dimitern> frobware: hey, welcome back :) sure - standup HO?
<frobware> yep
<mwhudson> wallyworld: ppa:gophers/archive!
<rogpeppe> wallyworld, axw, dimitern: here's a fix for the broken-in-go1.7 outgoing-connection-checking logic: https://github.com/juju/utils/pull/233
<rogpeppe> i'm thinking that it might be better to make that logic panic when trying to access an external address
<rogpeppe> rather than just returning an error
<rogpeppe> mwhudson: fancy a little review that might even be a tad enlightening (i found out something i didn't know) https://github.com/juju/utils/pull/233
<rogpeppe> ?
<rogpeppe> mgz, katco, mattyw, dooferlad: ^
<mgz> rogpeppe: sure
<rogpeppe> mgz: ta!
<mattyw> rogpeppe, can't see anything wrong with that, lgtm
<rogpeppe> mattyw: tyvm
<mgz> rogpeppe: change makes sense to me, my only real complaint is neither the implementions of the install... functions nor their call sites actually say they're about preventing test outbound network access
<rogpeppe> mgz: yeah, i've just added a comment
<mgz> with that then go ahead
<dimitern> rogpeppe: ship it!
<rogpeppe> mgz, dimitern, wallyworld: thanks
<wallyworld> rogpeppe: np. ensure you add testing notes as per new review policy
<wallyworld> thanks fir biting the bullet and fixing
<rogpeppe> wallyworld: for such a simple low level fix, i think that requiring a QA from the top level of Juju is overkill tbh
<wallyworld> rogpeppe: really? changing how http works could break everything. and it is the new policy for core
<rogpeppe> wallyworld: i'll certainly QA when this version of utils gets incorporated as a dependency into juju-core
<wallyworld> oh right
<wallyworld> it's utils, sorry
<wallyworld> yeah, when core deps get updated, test then
<wallyworld> jumped the gun sorry
<rogpeppe> wallyworld: np
<mup> Bug #1613262 opened: simplestreams: error message can hide real error <juju-core:New> <https://launchpad.net/bugs/1613262>
<mup> Bug #1613262 changed: simplestreams: error message can hide real error <juju-core:New> <https://launchpad.net/bugs/1613262>
<mup> Bug #1613262 opened: simplestreams: error message can hide real error <juju-core:New> <https://launchpad.net/bugs/1613262>
<dimitern> rick_h_: ping
<rick_h_> dimitern: pong
<dimitern> rick_h_: hey, frobware and me would like to have a chat re future work/planning/breakdown - when will be a good time for you? it shouldn't take more than ~20m
<rick_h_> dimitern: any time, welcome back frobware
<rick_h_> dimitern: frobware let me know when works for you and will join in the standup room?
<dimitern> rick_h_, frobware: top of the hour? i.e. in ~30m?
<rick_h_> dimitern: wfm
<dimitern> rick_h_: ok, cheers
<dimitern> frobware: ^^
<frobware> dimitern, rick_h_: yep good for me
<rogpeppe> i've updated http://reviews.vapour.ws/r/5428/ with QA instructions (wasn't quite sure where I should have put them). if someone could QA it for me, that would be great.
<frobware> rick_h_: if you time to jump back in before standup that would be useful
<rick_h_> frobware: sure thing, I'm back sorry
<frobware> rick_h_, dimitern: the google part of the net has disappeared for me
<rick_h_> frobware: google knows all, you were thinking bad thoughts :P
<rick_h_> voidspace: ping for standup
<voidspace> rick_h_: omw
<natefinch> sinzui, mgz: seems like we still have a windows error in CI.  I think I need to do a run with --keep-env, but the scripts don't support that yet.  I tried looking for where I should put support for that, but it's a little twisty. Give me a hint where to look?
<mgz> natefinch: the failing test seems like just a standard deploy job? so we can add --keep-env there fine?
<sinzui> natefinch: all CI scripts support --keep-env
<mgz> natefinch: do you want us to kick off a run with that?
<natefinch> mgz, sinzui: oh, sorry, looking at notes it was constraints, but I don't need that anymore
<sinzui> natefinch: yeah, contraints remain a gap. Which do you need?
<natefinch> sinzui: it's ok. it was only as a workaround for the azure thing that andrew already fixed.  So I should be all set now
<rick_h_> frobware: one other thing for your "welcome back" is that dimitern looked at this lxd spec but would be good if you also gave it a look over this week: https://canonical.leankit.com/Boards/View/122969419/123761889
<mup> Bug #1613300 opened: github.com/juju/juju/environs/sync fails with little info <centos> <ci> <jujuqa> <test-failure> <juju-core:Triaged by alexis-bruemmer> <https://launchpad.net/bugs/1613300>
<rick_h_> dimitern: what came of: https://bugs.launchpad.net/juju-core/+bug/1612624 last week?
<mup> Bug #1612624: Bootstrap fail on MAAS if ipv6 is disabled <juju-core:Triaged> <https://launchpad.net/bugs/1612624>
<dimitern> rick_h_: I haven't had time to look into that
<rick_h_> dimitern: ok
<dimitern> rick_h_: I could try later today
<rick_h_> dimitern: all good, I was just reviewing bugs and recalled we had the conversation of simplifying that work
<rick_h_> dimitern: so wanted to see what came after that discussoin we had.
<rick_h_> dimitern: hold off and wrap up WIP before we move forward there.
<dimitern> rick_h_: frobware and I discussed a few ways to detect ipv6 being enabled or not and if not - don't pass --ipv6 to mongod
<dimitern> rick_h_: but couldn't get as far as a wip fix for it
<rick_h_> dimitern: all good, ty for the update
<mup> Bug #1613311 opened: show-log and list-models disagree after migration <ci> <intermittent-failure> <jujuqa> <model-migration> <juju-core:Triaged> <https://launchpad.net/bugs/1613311>
<frobware> rick_h_: apologies for dropping out - have plumber here and needed somebody to stop the flow...
<rick_h_> frobware: :) all good
<frobware> rick_h_: there's less water now :)
<rick_h_> frobware: as long as it's less out of the pipes, and more in the pipes, seems like a net win to me
<frobware> rick_h_: houses - just like s/w
<frobware> rick_h_: I took a look over LXD spec - looks good
<redir> morning
<natefinch> if someone could review https://github.com/juju/jsonschema/pull/1  it would be a big help
<frobware> voidspace: ping - scripts, what were the issues?
<voidspace> frobware: the scripts have various dependencies - python libraries and system libraries
<voidspace> frobware: and they're not detailed anywhere
<voidspace> frobware: so I had to work them out
<frobware> voidspace: that's outrageous! :-D
<voidspace> frobware: e.g. xmlstarlet
<voidspace> frobware: so run, fail, work out why it failed and what package provides the missing bit, repeat...
<voidspace> frobware: :-)
<frobware> voidspace: let me add a prerequisites.sh
<voidspace> frobware: cool
<voidspace> frobware: I have an empty vm I was just about to re-start the procedure so I could work out what they were again
<frobware> voidspace: did this not work?
<frobware> type -p xmlstarlet > /dev/null || die "please install xmlstarlet(1)"
<voidspace> frobware: I didn't see that message
<frobware> hmm
<voidspace> frobware: I saw one telling me to *reinstall* xmlstarlet
<voidspace> frobware: that was just the first missing dependecy - there were about four I think
<frobware> voidspace: must come from something else
<voidspace> frobware: it was just add-node I was using
<frobware> voidspace: that 'type -p' is in add-node
<voidspace> frobware: *really* useful
<voidspace> frobware: so thanks
<voidspace> frobware: it failed at commissioning, but probably because I don't have power types set up
<frobware> voidspace: we need (MUST!) do more of this
<voidspace> frobware: but I can do that manually
<voidspace> frobware: rick_h_ said move it to github.com/juju so we can maintain it collectively
<voidspace> frobware: and I agree
<frobware> voidspace: sometimes I do see commisionning fail, but I believe that's a maas issue
<voidspace> frobware: I don't care about that anyway - creating the vm and enlisting is cool enough
<mup> Bug #1613311 opened: show-status and list-models disagree after migration <ci> <intermittent-failure> <jujuqa> <model-migration> <juju-core:Triaged> <https://launchpad.net/bugs/1613311>
<frobware> voidspace: I need to fix a small issue with remove-node, so will look at this now
<voidspace> frobware: ok
<voidspace> frobware: past 6pm here, dinner time
<voidspace> frobware: I'll be on later though
<voidspace> frobware: want to finish this actions stuff
<frobware> voidspace: sure
<natefinch> mgz, sinzui: you guys seen this before, when destroying an azure env? ERROR listing resource groups: azure.ServicePrincipalToken#Refresh: Failure sending request for Service Principal 10f015aa-8519-47cf-bd76-d60b1b06ba7d: StatusCode=0 -- Original Error: http: nil Request.URL
 * rick_h_ gets his taco on
<dimitern> frobware: I'll need to skip the networking call btw
<rogpeppe> katco: any chance of a "shipit" on http://reviews.vapour.ws/r/5428/ ?
<rogpeppe> katco: i added some QA steps, got it approved by tech board, QA'd it myself
<rogpeppe> katco: and it's in the way of a couple of other branches now
<rogpeppe> ... or anyone else... voidspace? mgz? dooferlad?
<mgz> rogpeppe: are the open issues actually all addressed/responded to now?
<rogpeppe> mgz: all except the ones i replied to to say i didn't think they were worth doing, yes
<mgz> rogpeppe: code changes look good to me, I don't fully get the implications of changing the return signature of ModelTag or adding a new /api alias
<mgz> presumably both are safe/sane enough?
<rogpeppe> mgz: thanks
<mgz> you have a +1 from me, and a minor comment shipit from the other martin
<rogpeppe> mgz: ta!
<natefinch> mgz, sinzui: --keep-env doesn't seem to actually keep the environment?  does that just keep the machines around?
<sinzui> natefinch: --keep-env means don't call kill-controller at the end of the script. It exits after collecting logs
<natefinch> sinzui: hmm weird
<natefinch> sinzui: so the controller should show up if I do JUJU_DATE=./cloud-city juju controllers, right?
<natefinch> (obv JUJU_DATA)
<mgz> natefinch: that's the bit I'm not sure is correct right now
<mgz> I think our poking of code around config/creds might have confused somethig
<rogpeppe> mgz: when a 'bot merge fails now, where do we go to find the test failure output?
<natefinch> mgz: ok... I don't seem to have a controllers.yaml in cloud-city
<rogpeppe> mgz: (it used to be in the console output but no longer)
<mgz> because last time I wanted to do something similar I couldn't see the controller of a running env outside the script
<mgz> rogpeppe: you need to look at the artifacts, linked from one level up on the job page
<natefinch> mgz: to be fair, it did throw an exception during log collections because ./logs didn't exist :/
<rogpeppe> mgz: ah!
<mgz> natefinch: it probably should not have actually called kill-controller though? that should be apparent from the log
<sinzui> natefinch: not exactly the controller is JUJU_DATA=cloud-city/jes-homes/<name-from-commandline>
<mgz> rogpeppe: generally trusty-out.log for test failures
<natefinch> sinzui: ahh, that's the problem
<mgz> rogpeppe: well, that's interesting: juju/juju/environs/sync a test failed, but there's no output for the failure
<rogpeppe> mgz: yes
<rogpeppe> mgz: i was just writing that to you
<natefinch> mgz, sinzui: works like a charm, thanks
<rick_h_> mgz: rogpeppe there's a bug on that filed this morning
<mgz> can you repo locally by just running that package?
<mgz> rick_h_: ah
<rick_h_> mgz: rogpeppe https://bugs.launchpad.net/juju-core/+bug/1613300
<mup> Bug #1613300: github.com/juju/juju/environs/sync fails with little info <centos> <ci> <jujuqa> <test-failure> <juju-core:Triaged by wallyworld> <https://launchpad.net/bugs/1613300>
<mgz> rogpeppe: so, it's master, not new with you
<rogpeppe> mgz: i guess i should just $$merge$$ again
<mgz> rogpeppe: well, ideally not, but I guess seeing how intermittent it is caan also be interesting
<rogpeppe> mgz: i can't get it to fail on my machine
<natefinch> ug... user admin@local is really confusing when it refers to a remote controller
<rogpeppe> natefinch: ha ha
<rogpeppe> natefinch: maybe the command line clients should rewrite admin@local to admin@$controllername
<natefinch> rogpeppe: that would be helpful I think....  I saw this:
<natefinch>  juju switch controller
<natefinch> nate-win-debug:admin@local/nate-win-debug -> nate-win-debug:admin@local/controller
<natefinch> and I thought for a minute I'd switched to a different controller, since it say "admin@local/controller at the end
<rogpeppe> natefinch: well that's the problem with the name of the controller model
<rogpeppe> natefinch: i still think that name is an endless source of problems
<natefinch> rogpeppe: I agree... using the same name for two different things is asking for trouble
<natefinch> but the same goes for "local" :)
<rogpeppe> natefinch: if it had been nate-win-debug:admin@local/admin, it wouldn't have been as much of a problem, i think
<rogpeppe> natefinch: well, @local names are always relative to the controller they're in
<natefinch> rogpeppe: local means on my machine
<natefinch> rogpeppe: we may not be using it that way, but that's what it means to most people
<rogpeppe> natefinch: and mhilton points out to me that we really can't add arbitrary @domains
<rogpeppe> natefinch: maybe it should be admin@3ecffebe-c248-41a2-893c-f67df84ec6bf :)
<natefinch> rogpeppe: I don't even really understand why there's an @ at all.  Aren't all users defined in the namespace of the controller they're on?
<rogpeppe> natefinch: no
<rogpeppe> natefinch: external users are not
<rogpeppe> natefinch: the @ is optional
<rogpeppe> natefinch: tbh, as per a discussion in juju-dev some time ago, i'd really like to get rid of the whole special-case name "local" completely
<natefinch> me too... I'd love to drop the @ entirely for users local to the controller
<rogpeppe> natefinch: then any name without an @ is treated relative to whereever it's used
<rick_h_> rogpeppe: yes, we've agreed that it needs to go away, just don't think it's been removed through and through yet
<rogpeppe> natefinch: and that simplifies the names logic too
<rogpeppe> rick_h_: really? cool!
<rogpeppe> rick_h_: that would simplify a bunch of stuff on lots of places
<rogpeppe> s/on/in/
<natefinch> rick_h_: that helps a lot... optional input is nice, but if we always still output it, it's even more confusing, because now I've input "nate" and juju replies with "nate@local"
<rick_h_> rogpeppe: yea, and it's not a case of 'by default users shouldn't need to mentally grok wtf is local' since to them there's no other option.
<rick_h_> natefinch: right
<natefinch> well, awesome :)
<natefinch>  sinzui, mgz: I keep getting "no matching tools" when trying to run the deploy CI test on windows... what am I doing wrong?
<natefinch> I'm running this command: JUJU_HOME=./cloud-city JUJU_REPOSITORY=./repository ./juju-ci-tools/deploy_job.py parallel-azure-arm $GOPATH/bin/juju ./logs nate-win-debug --series win2012r2 --keep-env
<sinzui> natefinch: you might is in a bad position
<natefinch> am I getting screwed by the new auto-upload-tools?
<sinzui> natefinch: you need streams to provide two different agents
<mgz> natefinch: do you need to be testing your exact binary?
<sinzui> natefinch: I don't think so in this case since --upload-tools will fail the same ways
<mgz> if not, you can probably use our streams
<mgz> otherwise, you're going to need to build some windows tools and make streams
<rick_h_> natefinch: --upload-tools is still there today?
<sinzui> natefinch: If you can use the streams that CI is building then use the current revision add this to your command: --agent-stream=revision-build-4254
<natefinch> sinzui: is there a way to say "just use the latest stream"?
<natefinch> sinzui: I think I can use streams, if they've been built since andrew made his azure storage fix
<sinzui> natefinch: no, there is no way to detect the latest testing stream
<sinzui> natefinch: but you can see a stream for *weeks*
<sinzui> natefinch: I tend to use the version that matches master at the start of my day: http://reports.vapour.ws/releases
<natefinch> sinzui: but what I want 99% of the time is whatever is the latest stream, since that may have fixes I need
<sinzui> natefinch: right. You can check that page to see which version is available
<rogpeppe> here's a small PR that removes some API server cruft from the code: https://github.com/juju/juju/pull/5997
<natefinch> mgz: how hard is it to make my own stream?  making my own jujud.exe is trivial, at least
<mgz> natefinch: it's pretty trivial, I believe abentley has some tools to make it less painful as well.
<natefinch> mgz: those two statements seem to be at odds with each other ;)
<mgz> it's easy to do and easy to do wrong :)
<natefinch> can I just do sync tools or something?  I just want to tell juju "here's a couple different tools, please use them" :/
<natefinch> except not sync tools, obv
<mgz> yerp nope.
<natefinch> we really seem to work hard to make things difficult for ourselves
<mgz> juju metadata generate-tools does get you most of the way though.
<sinzui> natefinch: I sent uyou a scipt that makes streams from you local jujuds
<sinzui> natefinch: pull lp:juju-ci-tools and look at streams-from-local.bash which you can pass jujus and jujud.exe to
<natefinch> sinzui, mgz: should I be using series win2012hvr2?  I've been using win2012r2
<sinzui> natefinch: does azure have that? We use it in our maas, but but in azure because I could find tht image
<natefinch> sinzui: I have no idea what azure has
<natefinch> sinzui: I honestly can't even tell what azure has when looking at their list of VMs to create
<sinzui> natefinch: no, you can't :(. We installed the azure python sdk to write a script to query that was available
<natefinch> not sure what "Windows Server 2012 R2 Datacenter" maps to in our list of series names
<natefinch> well I'm glad it's not just me
<sinzui> natefinch: win2012r2 did with with juju om july 6
<sinzui> natefinch: I am still looking for some sample streams that show the available series
<natefinch> sinzui: that streams-from-local.bash gives me line 68: unexpected EOF while looking for matching `"'
<sinzui> :(
 * sinzui looks
<natefinch> sinzui: WIN_JUJUD=${3"-}
<sinzui> thank you name
<natefinch> I presume that's wrong, although it may be a bashism I don't understand
<sinzui> thank you natefinch
 * natefinch gets the feeling he's being addressed by a bot
<sinzui> natefinch: bash is not pleasant
<sinzui> natefinch: oh, we need to also add the other win series too
<sinzui> natefinch: I used this with the maas. I will fix the quote and add the other win series
<sinzui> natefinch: pull lp:juju-ci-tools. I added win2012r2 and replaced " with :
<natefinch> sinzui: thanks
<natefinch> sinzui:  I've never made my own streams before.  after running the streams-from-local script, do I just bootstrap with --metadata-source, or is there more I need to do?
<sinzui> natefinch: --metadata-source worked last time I used (when I wrote the script).
<natefinch> gah, of course --metadata-source doesn't work with the ci script :/    I'm sorta surprised we don't just pass any unrecognized flags to bootstrap... seems like that would be the right thing to do most of the time
<natefinch> mgz, sinzui: to add support for a new arg, do I just add another add_argument call to deploy_job_parse_args?
<sinzui> natefinch: probably not. Most args ar needed by all scripts. utility.py add_basic_testing_arguments() fif the arge is for a general juju or test case. If the arg is specific to a single test, then the script can get it.
<natefinch> sinzui: ok, so how do I get the flag to actually be passed to bootstrap?  I added it to add_basic_testing_arguments ... but not exactly sure what to put as the action
<sinzui> natefinch: you will need to update BootstrapManager in deploy_stack.py. You probaly need the arg passed down to booted_contex()
<sinzui> natefinch: which arg to you need
<natefinch> sinzui: metadata-source
<sinzui> abentley:  we need the new juju bugs reported to discuss them on the release call
<sinzui> :/
 * rick_h_ goes to get the boy from summer camp
<abentley> sinzui: I reported bug #1613300 and bug #1613311.  Is that what you're asking?
<mup> Bug #1613300: github.com/juju/juju/environs/sync fails with little info <centos> <ci> <intermittent-failure> <jujuqa> <test-failure> <juju-core:Triaged by wallyworld> <https://launchpad.net/bugs/1613300>
<mup> Bug #1613311: show-status and list-models disagree after migration <ci> <intermittent-failure> <jujuqa> <model-migration> <juju-core:Triaged> <https://launchpad.net/bugs/1613311>
<sinzui> natefinch: I would consider a cheat to to get moving. you could change EnvJujuClient.bootstrap() to append the extra arg. You need to do that to add full support. Just added enough to get yourself unblocked. You cna push your change up for yourself to complete or maybe the qa team. I am sure we will want comprehensive support in the future
<natefinch> man I miss static typing
<natefinch> is args a string or a list?
<natefinch> answer: neither, it's a tuple :/
<natefinch> sigh
<katco> rick_h_: ping
<rick_h_> katco: pong
<katco> rick_h_: got a sec? i have a usability error message q
<rick_h_> katco: otp atm, will ping when off
<katco> rick_h_: np, ta
<rick_h_> katco: meet you in standup room when you're free
<katco> rick_h_: omw
<mup> Bug #1613429 opened: introspectionSuite.SetUpTest failure <intermittent-failure> <test-failure> <testing> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1613429>
<natefinch> sinzui: I can't get metadata-source to work... I just always get no tools available, even though I have win2012hvr2 in my local streams stuff
<sinzui> :/
<sinzui> natefinch: is metadata-source broken?
<sinzui> natefinch: you can push the streams to people.canonica.com then set image-metadata-url. I did this earlier this year (http://people.canonical.com/~curtis/images)
<natefinch> sinzui: I'm not sure.  I don't know enough about how it is supposed to work.  Definitely there seems to be a lack of logging around it.  Just saying "no tools available" is kind of useless for debugging.
<sinzui> natefinch: yeah. that's below average.
<veebers> natefinch, sinzui: is it possible that might be related to the changes wallyworld did for the upload-tools/build-agent stuff?
<natefinch> I think streams data is a separate path, but it's possible this is an unintended side effect
<sinzui> veebers: I don't think so since CI isn't seeing failures yet. Juju is very bad at letting developers test mixed os/arch. the new feature is actively ignoring that we must support windows, centos, and 3 other archs
<veebers> sinzui: ack. I ask because he is double checking something for me based on his recent changes :-)
<sinzui> veebers: I removed --upload-tools when I woke so we know deployments without streams continue to work.
<sinzui> natefinch: You hit bug 1591488
<mup> Bug #1591488: Unable to bootstrap with --metadata-source <cpe-sa> <juju-core:In Progress by anastasia-macmood> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1591488>
<natefinch> fffffffff
<veebers> sinzui: huh I should have checked trunk changes more closely :-\ You already fixed this bug before I came across it: https://bugs.launchpad.net/juju-ci-tools/+bug/1613192
<mup> Bug #1613192: EnvJujuClient.grant() passes invalid argument <juju-ci-tools:New> <https://launchpad.net/bugs/1613192>
<natefinch> sinzui: well, I guess I have to do the image-metadata-url thing
<sinzui> natefinch: just add image-metadata-url to the env in environments.yaml. It will be added to config.yaml for bootstrap
<natefinch> sinzui: cool... do you have a link to instructions on how to get stuff up to people.canonical.com?
<natefinch> gotta run, back later
<rick_h_> natefinch-afk: you can just use dropbox and get a url for it, or use the s3 file web UX to upload and get a generated url for it.
<rick_h_> natefinch-afk: don't let people.canonical.com hold you up, unblock
<rick_h_> natefinch-afk: shoot, upload the file into a pastebin and direct link to the raw url path
<anastasiamac> wallyworld: thumper: could u joun bug scrub?
<anastasiamac> join*
<wallyworld> it's not on my calendar, url?
<mup> Bug #1613444 opened: Remove-user doesn't remove user from list-shares <juju-core:New> <https://launchpad.net/bugs/1613444>
<thumper> nor mine
<anastasiamac> wallyworld: thumper: https://hangouts.google.com/hangouts/_/canonical.com/bug-scrub?authuser=0
<voidspace> axw: ping
<voidspace> thumper: ping
<thumper> voidspace: hey, in a call right now
<thumper> voidspace: is this about the actions review?
<voidspace> thumper: hey, cool
<thumper> because I didn't finish it, sorry
<voidspace> thumper: sort of - I'm working on QA steps and failing to do a basic migration
<thumper> ah...
<voidspace> thumper: I'll email you what I'm trying to do and you can correct my bad syntax
<voidspace> because...
<thumper> hmm... where's menno?
<voidspace> $ juju help migrate
<voidspace> ERROR unknown command or topic for migrate
<thumper> ah
<thumper> you need the feature flag
<voidspace> ah, I did for the migrate command - but not for the help
<voidspace> thumper: I'll bother menn0
<thumper> menn0: hey, can you help voidspace with a migration?
<thumper> cheers
<voidspace> ta
<menn0> voidspace: yep, what's up?
<voidspace> menn0: hang on, I'll create a pastebin
<voidspace> menn0: my thoughts for QA steps for action migration was that it would look something like this:
<voidspace> menn0: http://pastebin.ubuntu.com/23059562/
<voidspace> menn0: however the migration step of this seems to not (visibly) do anything
<voidspace> menn0: so I've obviously got something wrong
<menn0> voidspace: the QA steps look sensible to me
<voidspace> menn0: after running the migration there is no git unit in B
<voidspace> menn0: nor any visible sign of anything happening
<menn0> voidspace: it might be worth checking `JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-model` on the model before and after the migration to see if the action data is being included
<voidspace> menn0: ah, ok
<menn0> voidspace: did the model actually make it across?
<voidspace> menn0: however, I'm not yet convinced a migration is actually happening
<voidspace> menn0: ah, of course
<voidspace> I'm in B:default
<voidspace> thanks
<voidspace> not looking at the new model
<menn0> voidspace: I see the problem
<menn0> voidspace: you're trying to migrate "default" across but there's already a default on the other side
<menn0> voidspace: the migration will have aborted because of that
<voidspace> menn0: ok, will fix this
<menn0> voidspace: to see try something like: juju debug-log -m A:controller --replay -T | grep migrationmaster
<menn0> voidspace: if you remove "default" from B and then retry the migration you should have better luck
<voidspace> menn0: yep, exactly that
<voidspace> menn0: or just add a new model, deploy to that and migrate that
<menn0> voidspace: that's what I normally would do
<menn0> voidspace: but given you already have it set up as default you could just delete default first
<voidspace> menn0: ok, so I can confirm that actions are not migrated on master - now to try it with my branch :-D
<voidspace> menn0: thanks for the help
<menn0> voidspace: no problems
<menn0> voidspace: migrations will support renaming of the model during migrations at some point
<voidspace> cool
<menn0> voidspace: it'll also fail in a more obvious way for these kinds of early checks (i.e. before the `juju migrate` command returns)
<voidspace> menn0: that will really help
<voidspace> menn0: and after much futzing (required merging master in) I can confirm that actions are migrated by my branch
<voidspace> menn0: will add the successful QA steps to RB
<voidspace> and bedtime
<voidspace> g'night all o/
<menn0> voidspace: good night
<mup> Bug #1613459 opened: MainSuite.TestFirstRun2xFrom1xNotUbuntu forbidden (on closed network) <ci> <regression> <test-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1613459>
 * rick_h_ goes to make the fam dinner, night all
<thumper> wallyworld: what was the final thoughts on the blocker?
<wallyworld> thumper: i'm going to remove it
<thumper> menn0: ^^
<wallyworld> done
<menn0> wallyworld: ty, this didn't seem like blocker material to me either
<wallyworld> agreed
<menn0> redir: can you show me how you added the config to the cloud? I just tried that (via a new lxd cloud called "local") and none of the config made it the the bootstrapped host
<redir> menn0: so far I've been doing it with aws since i am working on something that requires regions. but lemme make sure it works with lxd too
<redir> menn0: so I have this file stored where I won't delete it: http://paste.ubuntu.com/23059827 (you only care about lxdtest)
<redir> and I add it with juju add-cloud lxdtest ~/Sync/juju/2.0/development-config.yaml
<redir> menn0: and I can see that the lxd controller is using squid-deb-proxy
<menn0> redir: that's essentially what I did but none of those options made it into the bootstrapped models
<menn0> redir: I must have missed something
<menn0> redir: I see the problem -- "juju add-cloud" ignored all the extra options
<menn0> redir: looks like I have to edit them into clouds.yaml directly
<redir> menn0: I just imported them
<redir> if you already have a config then you need --replace maybe
<redir> also maybe local is overloaded
<redir> menn0: ^
<menn0> redir: it was definitely a new cloud
<menn0> redir: I'll try a different name
<menn0> redir: nope
 * redir blinks
<menn0> redir: add-cloud just ignores the extra bits
<menn0> I end up with a cloud with just one field "type: lxd"
<redir> menn0: juju model-defaults no-proxy
<redir> ATTRIBUTE  DEFAULT  CONTROLLER
<redir> no-proxy   -        https://lxd-no-regions
<menn0> redir: very strange ... so you used "juju add-cloud" as well?
<redir> strange
<redir> yes
<redir> whenever I edit I edit in the non-volatile location and --replace
<redir> do you have only one cloud?
<redir> menn0: ^
<menn0> redir: yes... there's one cloud in clouds.yaml
<redir> menn0: very strange, indeed
<redir> menn0: FWIW, I am at ed24813 with a couple commits on top.
<menn0> redir: I'm a little behind you then (last updated late yesterday)
<menn0> redir: I'll fiddle with this a bit more later
<menn0> redir: thanks for your help
<redir> some help:)
<menn0> redir: I certainly think having this config in the cloud config is the way to go
<redir> I'll update the wiki next week.
<redir> wfm in AWS too (sans apt proxy)
 * redir looks to see if wallyworld is in meetings
<wallyworld> not right now
<redir> wallyworld: looks ripe for interruption
<wallyworld> almost, give me a minute
<redir> got a few minutes for some model-config ?s
<redir> k
<katco> wallyworld: change pushed
<wallyworld> ta
<katco> wallyworld: ty
<mup> Bug #1605756 changed: [ juju2 beta11 ] system show up in juju status as pending but there is no attempt to deploy in maas <oil> <oil-2.0> <juju-core:Invalid> <MAAS:Invalid> <https://launchpad.net/bugs/1605756>
<menn0> wallyworld: so this test I fixed last night isn't the same thing that's causing the mysterious failures in environs/sync
<menn0> wallyworld: I was just bitten by it again
<menn0> wallyworld: it's happening a lot
<menn0> wallyworld: any ideas?
<wallyworld> menn0: otp, give me a sec
<mwhudson> er are you all in some different standup to me?
<mup> Bug #1613476 opened: juju cli commands should be logged by juju <oil> <oil-2.0> <juju-core:New> <https://launchpad.net/bugs/1613476>
<wallyworld> redir: did you want to talk some more?
<wallyworld> or you all good?
<redir> wallyworld: yeah was mid thought
<redir> still in standup
<wallyworld> redir: gogle crashed, one sec
<redir> np
#juju-dev 2016-08-16
<mup> Bug #1613487 opened: Fails to build tools when no matching tools found. <bootstrap> <ci> <regression> <juju-core:New> <https://launchpad.net/bugs/1613487>
<mup> Bug #1613491 opened: juju service names limited to 66 characters <juju-core:New> <https://launchpad.net/bugs/1613491>
<menn0> wallyworld: i've spent a bit of time looking into this environs/sync issue and nothing jumps out
<menn0> wallyworld: I don't see any cases of a gc.C being stored inappropriately or even goroutines being used
<wallyworld> yeah, i had a brief look earlier and am in the same boat
<wallyworld> we really need there to be more info logged
<menn0> wallyworld: ha! I had it running with dave's stress script while I was having lunch and managed to have it fail like that once out of ~100 times
<menn0> wallyworld: so it is somewhat reproducible
<wallyworld> and yet CI fails more regularly
<wallyworld> hard to know where to target though with no errors
<menn0> wallyworld: i'm going to try running it again with -check.vv
<menn0> wallyworld: ok it failed quite quickly that time
<menn0> just checking the output
<menn0> wallyworld: got it
 * menn0 pastebins
<menn0> wallyworld: here's the 3 failures I just got: http://paste.ubuntu.com/23060118/
<wallyworld> menn0: intersting, wtf isn't that deterministic
<wallyworld> i can look into it
<menn0> wallyworld: I see it
<menn0> wallyworld: GetMockBundleTools is patched in during SetupTest and it keeps a reference to the gc.C that was passed to SetUpTest
<menn0> wallyworld: when it fails during the test that gc.C is no longer the current one
<menn0> (i.e. the one that gets passed to the test)
<menn0> that's what causes these tests that both pass and fail
<menn0> wallyworld: moving the GetMockBundleTools patching into the tests themselves makes them fail consistently (due to typed nil)
<axw> wallyworld: if you don't mind, PTAL at http://reviews.vapour.ws/r/5441/
<wallyworld> menn0: ah, i hate that multiple gc.C thing
<wallyworld> menn0: i have a branch i need to start, i can fix in that
<menn0> wallyworld: I'll push what i've done so far
<wallyworld> menn0: ok, np. i haven't had any time so far today to dive in, keep getting pulled left and right
<menn0> wallyworld: understood. this one got my attention because a) it's odd and 2) it's bitten me several times in the last day
<wallyworld> yeah, it will be fixed... today!
<wallyworld> axw: lgtm. we'll need to address the @local suffic at some point because I think it was going to be removed
<axw> wallyworld: okey dokey. I'd love it if we just had one canonical form always
<wallyworld> agreed, i think that's where the push is coming from, need to touch base on that again
<menn0> wallyworld: pushing now. all the tests pass again except one and I'm not sure if the test is wrong or the implementation.
<menn0> wallyworld: the tests were only passing by luck before it seems.
<menn0> (due to the typed nil comparison
<wallyworld> menn0: ok, ta, will merge into mine
<menn0> wallyworld: https://github.com/mjs/juju/tree/environ-sync-tests
<wallyworld> ty
<natefinch> could really use a review: http://reviews.vapour.ws/r/5440/  axw or anyone else
<axw> wallyworld: FYI, possibly not going to have these changes done today. I'm going to have to change a bunch of code in state, API, etc. to use credential tags instead of names
<wallyworld> not surprised
<axw> so probably won't get to LXD till tomorrow
<natefinch> wallyworld: the auto-upload-tools thing still seems dangerous to me. What if you have 2.0 deployed, have a 2.0.1.1 client/server built locally, but 2.2 is available in streams.  What does juju upgrade-juju do?
<wallyworld> upgrade to 2.2
<wallyworld> well, actually,i need to check the urgrade rules
<wallyworld> if you have a 2.0.1.1, then you have built it and so are expected to know what you are doing
<natefinch> wallyworld: expected to know what you're doing is a lot different than understand exactly what fine details of the code are going to do
<wallyworld> i can't recall off hand how we select the allowed upgarde version when searching
<natefinch> wallyworld: it seems like any time both streams and your local are valid upgrade paths, there'
<wallyworld> upgrades have always been like that
<natefinch> there's no way to know which one the user intends to use
<wallyworld> it searches for packaged tools first
<wallyworld> if it finds them, it will use them in preference
<natefinch> ok
<wallyworld> so whatever upgrade rules we have for selecting will be used
<wallyworld> i think we look for major.minor
<wallyworld> it only uploads automatically if no packaged tools are there, and only if what you have is different and only if jujud matches the client being used
<natefinch> I guess my point is - I want to make sure we never automatically upgrade someone from a released version to a development version with some explicit flag or confirmation
<natefinch> s/with/without/
<wallyworld> right now, there's not confirmation - it will use a jujud mactching juju
<natefinch> that seems undesirable
<wallyworld> so if juju is 2.1-beta1 it will upgrade a 2.0 (if the upgrade rules select such a version)
<wallyworld> i'l need to check that
<natefinch> I just don't want someone to get screwed who happens to build juju from source to test out the latest and greatest... forgets for a week, then upgrades their production server and boom, now they're on devel
<mup> Bug #1611097 changed: cannot add/destroy a model if done too fast <oil> <oil-2.0> <juju-core:Invalid> <https://launchpad.net/bugs/1611097>
<wallyworld> menn0-afk: when you are back, here's a PR to fix that test and a CI issue http://reviews.vapour.ws/r/5442/
<menn0> wallyworld: I am back.... forgot to update nick
<menn0> looking
<wallyworld> menn0: ah, i see a mistake, a wrong test suite name, fixing
<wallyworld> hmm, scrap that, it can stay as is
<anastasiamac> axw: wallyworld: bug 1582021 was addressed, right?
<mup> Bug #1582021: Juju loses track of current controller when destroying model <2.0> <bitesize> <destroy-model> <juju-release-support> <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1582021>
<wallyworld> think so?
<axw> don't recall exactly
<anastasiamac> i'll mark as fixed committed \o/ would have been sooo very lovely to have a functional test that would just tell us :) mayb even have a link to a passing ci test in the bug :)
<anastasiamac> veebers: ^^
<menn0> wallyworld: done!
<wallyworld> ta
<menn0> wallyworld, thumper: do you think I moved the notification about an available upgrade into the new NOTES status column too?
<thumper> axw: why do we store the charm url on the storage instance when you can get it from the owner?
<thumper> menn0: +1
<menn0> thumper: that's what I was thinking too
<thumper> menn0: showing the most relevant info
<thumper> migration > upgrade available > nothing
<wallyworld> menn0: yeah, good idea
<axw> thumper: it's not necessarily the same. it's the charm URL that was in use when the storage instance was created, not the current one
<menn0> thumper: yep that's exactly what I was thinking
<thumper> axw: but why do we care?
<axw> thumper: I forget why we have it at all. we don't use it AFAIK
<axw> wallyworld: do you recall?
<wallyworld> um
<thumper> we could just drop it, yeah?
<axw> thumper: if we haven't needed it thus far, probably safe to drop it
<wallyworld> thumper: axw: i can't even see where it's used
<thumper> axw: you ok if I do that as part of the migration branch I'm doing?
<axw> thumper: SGTM
<thumper> ok, cool
<axw> thanks
 * thumper is looking how to add storage to the model to test exports
<wallyworld> menn0: thumper: it seems to me that in the multi-model world, stuff is not getting removed when a model is destroyed. it never used to matter in the single model case. but now for example, i can see where we are creating model settings and constraints in modelSetupOps but i can't see where this data is ever removed via calls to removeConstraintsOps etc. I can see where constraints and what not are removed for applications, units,
<wallyworld> machines, but not models. am i on crack?
<menn0> wallyworld: I didn't write this stuff but RemoveAllModelDocs catches everything
<menn0> wallyworld: very brute force
 * wallyworld looks at that method
<menn0> wallyworld: not sure when/how it gets called when models are destroyed normally
<menn0> wallyworld: until yesterday it was missing user perms but I fixed that
<menn0> wallyworld: looks like the undertaker worker ends up calling it, so everything should be getting removed
<wallyworld> menn0: awesome thanks
 * natefinch tries for like the millionth time to use custom streams with the CI tests
<mup> Bug #1474291 changed: juju called unexpected config-change hooks after read tcp 127.0.0.1:37017: i/o timeout <bug-squad> <docs> <hooks> <openstack> <uosci> <juju-core:Invalid> <https://launchpad.net/bugs/1474291>
<mup> Bug #1613516 opened: AWS instance types should be detected <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1613516>
<natefinch> it's really a shame we can't just upload tools to the controller and tell it what series they're for.
<thumper> OMG, state.AddApplication is 230 lines...
<natefinch> at my first job out of college, the main entry function was 1000 lines long
<thumper> over 100 lines of that is arg validation
<thumper> natefinch: I worked at a bank where the CreateInstrument function was 10000 lines
<natefinch> OMG
<thumper> it was horrible
<thumper> and noone wanted to fix it
<natefinch> that's beyond horrible
<thumper> because "what if it broke"
<natefinch> yes.. what IF it broke? :/
<thumper> :)
<thumper> no unit tests, just overnight regression tests
<natefinch> my last job, we barely had tests.  I'm sure it was like 5% coverage.... and many developers openly criticized the idea of writing tests.
<natefinch> it's a damn miracle anything worked.  We did have like 8 full time QA people doing manual tests, I guess.
<menn0> wallyworld: so can you show me your clouds.yaml?
<wallyworld> only if you show me yours :-D
<wallyworld> i'll pastrebin
<wallyworld> menn0: http://paste.ubuntu.com/23060299/
<thumper> axw: able to save me some time and point me at the simplest test code that adds a storage instance to the db?
<menn0> wallyworld: I see the problem. I had my config at the wrong level... no "config" section.
<thumper> axw: factory.MakeApplication?
<menn0> wallyworld: I wonder if juju add-cloud should complain if you mess this up?
<menn0> (like I did)
<wallyworld> probably
<veebers> wallyworld, axw: re: a test for 1582021 as mentioned by anastasiamac, does this sound reasonable for a testplan? test would bootstrap, add a new model, switch to it, (ensure status works) destroy that model then ensure that it could both "list-models" and "status" successfully?
<wallyworld> veebers: if you just destroy a model and don't switch to another one, you will get an error
<wallyworld> it will be interesting to repro the race condiiton
<veebers> wallyworld: is that the error that is shown in the bug? Perhaps I'm not sure what the test behaviour should be
<wallyworld> it depends if the bug has been fixed. the bug refers to a not found which is the original symptom. if the current model is cleared out, then there will be a different error refering to no current model selected
<anastasiamac> wallyworld: veebers is writting the test to detemrine whether the bug has been fixed. he just needs to confirm if his scenario will veriffy the fix
<wallyworld> right, understood. as i said, there's 2 aspects. the resetting of current model and the race
<wallyworld> the error wrt the resetting of current model will depend on if the bug has been fixed. and you need to switch models if you want to avoid an error of either kind
<wallyworld> if the bug has been fixed, you get the not found, if not you'll get the no current model as stated above
<wallyworld> assuming you don't switch models
<anastasiamac> wallyworld: the bug specifically deals with no-explicit-switch situation.. it's covered in bug description
<anastasiamac> wallyworld: mark is not happy with "not found"... did u read the bug? :D
<wallyworld> right, but the test also needs to cover the switch case
<wallyworld> yes
<veebers> wallyworld: Ok, so I want to confirm the mention bug is fixed, so I destroy and don't switch, doing a list-models or status should give me . . .something? better asked what should the expected result be
<wallyworld> and as i said, if the bug has been fixed (ie current model reset), you will not get the not found
<wallyworld> but you will get a different error
<wallyworld> until you switch to a different model
<wallyworld> i thought i explain it! you will get a different error
<wallyworld> not a not found error but an error saying there is not current model
<wallyworld> and if you switch everything will be ok again
<veebers> ok cool, I'll run with that. Cheers
<wallyworld> if you get stuck, let me know; sorry if i was unclear
<axw> thumper: sorry, was having lunch. if you just add a unit whose charm has non-optional storage, it'll add a storage instance automatically
<thumper> axw: is there an example of that anywhere?
<thumper> do we have a testing charm that has non-optional storage?
<axw> thumper: state/storage_test.go, StorageStateSuiteBase.setupSingleStorage (actually that one's non-optional, but that code sets up the storage constraints)
<axw> thumper: err sorry, that one's *optional*
<thumper> that'll do...
<thumper> I'll crib from that
<veebers> wallyworld: fyi I get the error message "..no model in focus" which seems like the bugfix is in place for that part at least
<wallyworld> veebers: awesome, that means that it is fixed, yeah
<wallyworld> so now the test should switch
<wallyworld> and confirm that everything is ok
<veebers> sweet
<wallyworld> i would have thought list-models could still work though
<wallyworld> even with no model in focus
<wallyworld> if it doesn't that's a bug IMO
<wallyworld> how do you know what to switch to
<wallyworld> if you can't list models
<anastasiamac> veebers: would be awesome if commented on the bug with your findings. we can change status of the bug based on ur test \o/
<veebers> wallyworld: 'models' and 'list-models' seem to work, although switching to a model doesn't seem to work, although I could be doing something off in my test code. I'll re-pro manually
<wallyworld> list-models works even without a current model? cool
<wallyworld> i think we set current model with a *
<wallyworld> so list-models should not have the * without a current model selected
<natefinch> wallyworld: I'm trying to use my own streams to run a test.  if I bootstrap with --metadata-source ... where is that stored?  I don't see that config val in get-model-config or get-controller config
<wallyworld> it's not stored
<wallyworld> what streams? tools or images?
<natefinch> wallyworld: tools
<wallyworld> tools are stored in blobstore
<wallyworld> metadata-source is only used to tell bootstrap where to find tools to upload and store
<natefinch> wallyworld: I'm trying to test something on windows, so I need a secondary machine to be able to get the windows binary that I have built. how do I do that?
<wallyworld> it's not trivial
<wallyworld> one way is to use agent-metadata-url pointing to an apache http server
<natefinch> well, so I have tools & streams generated and sitting in S3
<wallyworld> if it's a public bucket you could use that url as agent-metadata-url
<natefinch> wallyworld: ok, I thought metadata-source would have done that, but I can set metadata-url, that's fine
<natefinch> er agent-metadata-url
<wallyworld> metadataa-source -s just fpr bootstrap
<wallyworld> to tell it where to find a local (as in dir) source of tools/image metadata
<wallyworld> generated using the plugins
<wallyworld> which is then sent to controller suring bootstrap and stored in state
<natefinch> ahh... then maybe it should complain if I give it a URL :/
<wallyworld> file:// is a url
<wallyworld> which it uses
<wallyworld> i guess it could complain if not file://
<wallyworld> or a dir
<natefinch> but --metadata-source https://s3.amazonaws.com/24b99da448314945835933a0c8edffb3 just fails silently
<natefinch> not fails... is ignored
<wallyworld> yeah, there's not been much polish on dev only tools
<wallyworld> i didn't think it failed, that must be a new bug
<natefinch> bootstrap doesn't fail, which is kind of the problem
<menn0> thumper or wallyworld or anastasiamac: MM status in tabular output http://reviews.vapour.ws/r/5444/
<wallyworld> ok
<wallyworld> menn0: i asked for one more test
<menn0> wallyworld: I thought of that one but was being lazy. done now :)_
<wallyworld> :-)
<wallyworld> we've all been there
<wallyworld> you have a shipit anyway
<mup> Bug #1613537 opened: metadata-source doesn't sanity check the value you give it <juju-core:New> <https://launchpad.net/bugs/1613537>
<wallyworld> axw: if you had a moment at any time, this is a mechanical change to the model defaults api to accommodate cloud/region. not used yet but will be when reed's stuff gets further along http://reviews.vapour.ws/r/5445/
<axw> wallyworld: ok, in a little while
<wallyworld> sure, no hurry
<axw> wallyworld: one fundamental thing: we have cloud tags now, should be using that instead of just a cloud name
<wallyworld> ah, yeah
<wallyworld> doh
<wallyworld> fixing
<wallyworld> axw: updated for when you get to look later
<wallyworld> thans axw
<axw> np
<mup> Bug #1551959 changed: juju-tools 1.25.0 with juju-agent 1.25.3 has networking issues <juju-core:Won't Fix> <https://launchpad.net/bugs/1551959>
<anastasiamac> wallyworld: axw: just to confirm - i believe this bug has been fixed with new model naming convention that includes user name, non? :D bug 1576359
<mup> Bug #1576359: Cannot talk to a new model with a reused name <juju-release-support> <switch> <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1576359>
<wallyworld> not sure off hand
<wallyworld> i wasn't familar with that bug
<anastasiamac> axw:  ? :D
<axw> anastasiamac: not sure that that one is fixed
<axw> anastasiamac: I don't think I did anything specifically that would have fixed that
<anastasiamac> axw: wallyworld: k. thnx
<axw> wallyworld: https://github.com/juju/names/pull/72  - sorry, third time's the charm perhaps?
<mup> Bug #1100560 changed: Support method of removing a subordinate charm <canonical-is> <canonical-webops-juju> <remove-relation> <subordinate> <pyjuju:Won't Fix> <juju-core:Invalid> <https://launchpad.net/bugs/1100560>
<mup> Bug #1432759 changed: Transient error on status while running deployments via quickstart <api> <quickstart> <race-condition> <status> <juju-core:Invalid> <https://launchpad.net/bugs/1432759>
<wallyworld> axw: just gotta race out to school pickup, will look when i get back
<axw> np
<wallyworld> am running really late
<miken> anastasiamac: Hey there. Just to be clear, bug 1613491 is not critical for me - we're working around it easily enough.
<mup> Bug #1613491: juju service names limited to 66 characters <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1613491>
<anastasiamac> miken: awesome \o/ thank you - i've updated the bug
<mup> Bug # changed: 862422, 1228355, 1301255, 1340756, 1468223
<mup> Bug # changed: 1217814, 1283866, 1398589, 1401358, 1403460, 1417149, 1420996
<axw> wallyworld rogpeppe: this PR should address concerns about the cloud API, and also updates other bits to use the new cloud credential tag: http://reviews.vapour.ws/r/5447/
<rogpeppe> axw: i'll swap ya :) http://reviews.vapour.ws/r/5439/
<axw> rogpeppe: sure, looking
<axw> rogpeppe: hmm, I think it was intentional to leave them there, so old clients can get a useful error message
<rogpeppe> axw: won't they still get the same error message?>
<rogpeppe> axw: i think i tried to leave the behaviour unchanged. perhaps i didn't though...
<axw> rogpeppe: maybe? I just saw a test that removed a check for the error message, so assumed it didn't any more
 * axw looks deeper
<rogpeppe> axw: ah, yes, maybe i removed that test when i shouldn't have
<wallyworld> the idea is that older 1.25 clients should get a nice message
<wallyworld> not sure of the details though
<rogpeppe> wallyworld: yes, they still should, but yeah, i removed some of the infrastructure that supported the test
<wallyworld> if it works with the proposed changes, then that's ok
<wallyworld> +1 to deleting code if it makes sense :-)
<rogpeppe> wallyworld: i think we could keep the test but make it manually call Login with version 2; no need for OpenWithVersion, I think.
<wallyworld> sgtm, keep as little as possible
<mup> Bug #1494936 changed: imageSuite.TestDownloadEnvironmentPath <bug-squad> <ci> <intermittent-failure> <panic> <simplestreams> <unit-tests> <wily> <juju-core:Invalid> <https://launchpad.net/bugs/1494936>
<mup> Bug #1494936 opened: imageSuite.TestDownloadEnvironmentPath <bug-squad> <ci> <intermittent-failure> <panic> <simplestreams> <unit-tests> <wily> <juju-core:Invalid> <https://launchpad.net/bugs/1494936>
<mup> Bug #1494936 changed: imageSuite.TestDownloadEnvironmentPath <bug-squad> <ci> <intermittent-failure> <panic> <simplestreams> <unit-tests> <wily> <juju-core:Invalid> <https://launchpad.net/bugs/1494936>
<rogpeppe> axw: reviewed
<axw> rogpeppe: ta
<axw> rogpeppe: I'd prefer to leave mention of user/cred syntax in the help docs until we have a means to share
<rogpeppe> axw: "a means to share" ?
<axw> rogpeppe: share credentials - ACLs
<axw> rogpeppe: i.e. you may be able to use the syntax user/cred, but it won't work unless user == you
<rogpeppe> axw: the problem is that historically that means we'll forget to add it to the help docs
<axw> rogpeppe: I'd rather that than confuse people by having them think they can do something they can't
<rogpeppe> axw: i'd add it to the help docs and add a caveat that currently the user must be the same as the model user
<rogpeppe> axw: or, tbh, since it'll just fail, that's probably fine
<rogpeppe> axw: it'll fail with "permission denied" presumably
<rogpeppe> axw: which is exactly how it would fail if the permissions hadn't been shared
<axw> rogpeppe: "not found" at the moment I think
<rogpeppe> axw: even if the other user's creds actually exist?
<rogpeppe> axw: oh yes, because that's what we always do for perm denied i guess
<axw> rogpeppe: yeah. generally we do that so you can't sniff out info
<rogpeppe> axw: so, i think it's fine to document the syntacx
<rogpeppe> axw: then when we implement sharing, people will find a use for it
<axw> rogpeppe: ok
<rogpeppe> axw: also, describing it in the help docs is often a good indicator as to whether it's a decent feature or not
<rogpeppe> axw: i.e. can it be described nicely
<rogpeppe> ?
<rogpeppe> axw: tell me about CreateLocalLoginMacaroon
<axw> rogpeppe: it's how you get a macaroon that you can log into a controller as a local user
<axw> rogpeppe: what do you want to know about it in particular...?
<rogpeppe> axw: i'm just wondering if it should be part of the controller-only API or not
<axw> rogpeppe: yes, it is a controller level thing
<rogpeppe> axw: i'm getting a test failure where login fails
<rogpeppe> axw: because it's using the model-level connection (that it's using to log in) but then uses that to create a local login macaroon
<rogpeppe> axw: and that fails because UserManager is controller-only
<axw> rogpeppe: what's the test?
<rogpeppe> axw: featuretests.cmdLoginSuite.TestLoginCommand
<rogpeppe> axw: it seems like a legit failure to me
<axw> rogpeppe: ah, it's actually being used in change-user-password
<axw> which should be controller-only..
<rogpeppe> axw: it is
<rogpeppe> axw: changePasswordCommand embeds ControllerCommandBase
<rogpeppe> axw: hmm, so is login actually
<rogpeppe> axw: so now i'm confused :)
<axw> rogpeppe: that should mean they're getting controller API connections
<rogpeppe> axw: yup
<rogpeppe> axw: so it should work, but i'm getting
<rogpeppe> [LOG] 0:00.242 ERROR cmd failed to create a temporary credential: permission denied (unauthorized access)
<axw> rogpeppe: sorry, nothing's jumping out from the code... login is passing "" for the model name to get the API connection
<rogpeppe> axw: yeah, there's something odd going on. thanks.
<rogpeppe> axw: now i just have to work out how to get the featuretests to show trace-level debug logs of the api server
<axw> rogpeppe: loggo.GetLogger("juju.apiserver").SetLogLevel(loggo.TRACE)
<axw> I think
<rogpeppe> axw: i'll try that
<rogpeppe> axw: still not seeing any api server traffic (i also tried GetLogger with "" and "<root>")
<rogpeppe> axw: ISTR working out a way to do this before, but it's gone from my memory now unfortunately
<babbageclunk> rogpeppe: I've never managed to set the root logger level using GetLogger("<root>") - try using ConfigureLoggers instead.
<rogpeppe> babbageclunk: ha, thanks
<rogpeppe> babbageclunk: the loggo code thinks that it should be ""
<babbageclunk> rogpeppe: ah, that might be what I got wrong
<rogpeppe> babbageclunk: but i suspect that all the logging is being lost
<rogpeppe> babbageclunk: GetLogger should really have a comment
<rogpeppe> babbageclunk: or better, all loggers should start with "." so you wouldn't need a special case for <root>; "." would always be the root logger.
<rogpeppe> babbageclunk: too late for that now though
<babbageclunk> dimitern: do you know if Will's out all week?
<dimitern> babbageclunk: I don't know - yesterday was a public holiday in Malta IIRC
<babbageclunk> dimitern: Ah, ok.
<babbageclunk> dimitern: Although he might have piggybacked a holiday on that.
<babbageclunk> dimitern: Do you have any more comments other than the model tag checking stuff in http://reviews.vapour.ws/r/5366/?
<dimitern> babbageclunk: let me have a look..
<babbageclunk> dimitern: Thanks! I made the other changes you suggested.
<babbageclunk> dimitern: I've finished (for some value of finished :) the api client as well.
<rogpeppe> axw: ok, i just about succeeded. the problem is that juju/cmd.Log.Start tries to make the logs go to ctx.Stderr; it tries to replace the default loggo writer, but there isn't one. we should make it so you can just see the logs when desired, i think. maybe add another writer or something.
<macgreagoir> wallyworld axw: I have that AddController change merged, much thanks.
<dimitern> babbageclunk: machineundertaker will be running once per model, right?
<axw> macgreagoir: thank you :)
<dimitern> babbageclunk: i.e. it will be configured inside cmd/jujud/agent/model/manifolds.go ?
<dimitern> like the space importer, etc.
<babbageclunk> dimitern: I think so? Where's environ-tracker?
<babbageclunk> dimitern: Yes, in cmd/jujud/agent/model/manifolds.go
<dimitern> babbageclunk: yeah, it's there as well
<babbageclunk> dimitern: I think you're right that the model tag is unnecessary.
<dimitern> babbageclunk: yeah, that's what I'm getting at :)
<rogpeppe> axw: ah! i've discovered what the problem is. CreateLocalLoginMacaroon isn't considered a read-only call
<babbageclunk> dimitern: But I'm reluctant to pull it out and then have Will explain a sensible reason why it's needed and then have to put it back in.
<dimitern> babbageclunk: as long as you're a model manager, there's no need to check or take a model tag (uuid) to work on - you'll only every work on one model anyway
<axw> rogpeppe: why is that a problem?
<dimitern> babbageclunk: I'm interested to learn Will's reasoning for having ConnectedModel() ?
<rogpeppe> axw: because in that test i'm guessing the user is read-only
<rogpeppe> axw: (i haven't checked)
<babbageclunk> dimitern: But you mentioned in your review that there were some other things you were thinking about as well?
<rogpeppe> axw: but in general, i think it's probably OK for even read-only users to be able to create a login macaroon
<babbageclunk> dimitern: ConnectedModel is only needed for the model checking.
<axw> rogpeppe: yeah probably. it's not writing anything to the model
<rogpeppe> axw: yup
<rogpeppe> axw: i haven't yet worked out why this has changed in my branch though
<dimitern> babbageclunk: so AuthModelManager() being true means you've logged in as a machine agent with jobmanagermodel
<dimitern> babbageclunk: it's true authorizer.GetAuthTag() does not give you the model uuid, just that machine tag you logged in as
<dimitern> babbageclunk: but then again, the model manager engine runs per model
<dimitern> babbageclunk: in startModelWorkers (cmd/jujud/agent/machine.go)
<babbageclunk> dimitern: ok (reading)
<dimitern> babbageclunk: so, if checking whether the model uuid matches was an issue, it should've been done for at least some of the other workers+facades part of the model manager manifold
<dimitern> unless I'm missing something..
<babbageclunk> dimitern: Yeah, in which case we'd have needed ConnectedModel ages ago.
<babbageclunk> Ok, I'm convinced. It's easy to rip it out, and I'll keep the commit around so that it should be easy to put it back in if Will convinces us the other way.
<dimitern> babbageclunk: only if we need to test impossible cases :)
<dimitern> babbageclunk: but Will might have another good reason
<dimitern> babbageclunk: i.e. being explicit about which uuid you should allow in the facade
<dimitern> babbageclunk: but if that's the case, then it's much easier to take modelTag as argument to machineundertaker.NewAPI(), store it and check it internally
<babbageclunk> dimitern: I think it might have been something like that. Hang on, I'll find the conversation in the logs.
<dimitern> babbageclunk: I'm happy to be proven wrong :)
<dimitern> babbageclunk: perhaps the intent is to call AllMachineRemovals() with multiple model uuids as args, say if we're tearing down more than 1 model
<dimitern> babbageclunk: but this means we're designing the facade with the intent to make it global, not per-model, and to allow 1 instance of machineundertaker to run for all models - then it makes sense
<babbageclunk> dimitern: No, I don't think so. As you say, we'll be connected to one model.
<dimitern> babbageclunk: I'll add some comments
<dimitern> if we had JobManageController, implying "can manage all hosted models", then fine..
<babbageclunk> dimitern: For some more background (but nothing conclusive as far as I can tell) search for ConnectedModel in here: https://irclogs.ubuntu.com/2016/08/09/%23juju-dev.txt
<dimitern> babbageclunk: cheers, will do
<babbageclunk> dimitern: But yeah, if you've got any other suggestions for changes, I'll make those before ripping out the model checking so that it should still be easy to put it back.
<dimitern> babbageclunk: do you have a link to Will's previous review about AllMachineRemovals() taking a list of model tags?
<babbageclunk> dimitern: It's in comments on katco's review - not much detail I'm afraid.
<dimitern> ah ok
<babbageclunk> fwereade: Hey! Speak of the devil!
<fwereade> babbageclunk, hey, sorry
<fwereade> babbageclunk, was signed into canonical but not here for some reason
<fwereade> babbageclunk, what can I do for you?
<babbageclunk> dimitern and I are trying to understand why AllMachineRemovals needs to take the model uuid?
<babbageclunk> fwereade: (as seen in http://reviews.vapour.ws/r/5366/)
<fwereade> babbageclunk, the api method?
<babbageclunk> fwereade: yeah
<rogpeppe> axw: ok, i've found out what's happening. prior to my change, all the controller-only facades were accessible to read-only users, but now the read-only layer always applies
<fwereade> babbageclunk, because depending on the connection state is generally unhelpful, and silently ties the implementation down so it can't be moved to other contexts
<rogpeppe> axw: i have a problem with apiserver.admin.doLogin - it's too big and hard to understand for such a security-crucial method
<babbageclunk> dimitern: ^^
<fwereade> babbageclunk, we have multi-model controllers -- even if all our connections right now have an implicit model, that's not fundamental to the problem space
<fwereade> babbageclunk, when you're talking to a controller directly, for example, there are going to be at least some things you can rationally do to all-your-known-models
<fwereade> babbageclunk, and so just making sure we name our targets from the beginning makes it trivial to extend that code out, differing only in auth concerns, as we grow
<fwereade> babbageclunk, sane?
<babbageclunk> fwereade: yeah, that makes sense to me. dimitern?
<dimitern> babbageclunk, fwereade: catching up now
<fwereade> babbageclunk, note, tangentially, that controllers make an additional api connection per hosted model, and this is kinda silly -- the fact that implicit-model is baked in all over the place makes it necessary for the moment, but it's not a Good Thing, and writing our code *without* assuming that context will help us if/when we actually address that
<dimitern> fwereade: so you're saying AllMachineRemovals(modelTags ...) is a good method to have on a facade that's only going to work against a single model?
<fwereade> dimitern, yes
<fwereade> dimitern, this shoudl not even be a question in your mind
<fwereade> dimitern, because I don't want it to be a question in consumers' minds either
<fwereade> dimitern, facade methods accept bulk args, full stop :)
<dimitern> fwereade: I'm trying to see why - if in the future that same facade is used by a controllerManager worker of sorts, that can deal with multiple hosted models, that's fine
<dimitern> fwereade: but which model you're working on seems to align well with the SRP
<fwereade> dimitern, well, no, because you can't share the implementation without a rewrriate that takes implicit state into account
<fwereade> dimitern, it's really just another special case of "explicit is better than implicit"
<dimitern> fwereade: are we going to change all the other modelmanager manifold workers to use that approach?
<fwereade> dimitern, which ones are not?
<fwereade> dimitern, this is not even remotely in question
<dimitern> fwereade: I see how this is good, but it irked me as inconsistent w.r.t current code examples
<fwereade> dimitern, it has been How You Write An API from the very beginning
<dimitern> fwereade: well, not so much for models
<dimitern> fwereade: but I guess why not
<fwereade> dimitern, we didn't even have models
<fwereade> dimitern, name your targets
<dimitern> fwereade: well, different models are likely to have different cloud creds
<fwereade> dimitern, and?
<dimitern> fwereade: it seems to me we need a new JobManageController or something
<fwereade> dimitern, we just need a JobFlag in the machine agent, I think
<dimitern> fwereade: I'm not that familiar with the flags yet
<dimitern> fwereade: but fair enough
<fwereade> dimitern, the whole flag implementation is in jujud/agent/engine, it's pretty comprehensible
<dimitern> fwereade: how about machineundertaker.NewAPI() taking a modelUUID / tag to work on
<fwereade> dimitern, the two workers that implement engine.Flag are pretty trivial
<fwereade> dimitern, is that apiserver-side?
<fwereade> dimitern, then, no
<dimitern> fwereade: yeah
<dimitern> fwereade: isn't that more explicit than going through the authorizer each time?
<fwereade> dimitern, what you are saying is "I have a special case that I can imagine a different implementation, please may I throw consistency to the winds and take on secret technical debt?"
<fwereade> dimitern, just accept your Entities like every other method should
<dimitern> fwereade: I'm just saying we can change the NewAPI constructor if we need later, keep the AllMachineRemovals() as-is (taking model tags), and have a slightly simpler initial implementation
<fwereade> dimitern, api-side, sure, create the type with a single ModelUUID and call .AllMachineRemovals() -- but we want the data we send over the wire to make sense in isolation
<fwereade> dimitern, how do we change the NewAPI constructor? isn't that essentially constrained by the facade package?
<fwereade> dimitern, as it is we have the implicit model known because we have the *State, and yes, unpicking that will be tedious
<dimitern> fwereade: it is, but it's not visible to api clients, and can change without breaking the wire format of the facede
<rogpeppe> [11:21:24] <fwereade> babbageclunk, when you're talking to a controller directly, for example, there are going to be at least some things you can rationally do to all-your-known-models
<rogpeppe> fwereade: FWIW controller-only APIs exist in a separate space
<rogpeppe> fwereade: so it will of necessity be a different facade in that case
<dimitern> fwereade: ok, I just wanted to clarify your reasons for asking to add ConnectedModel()
<fwereade> rogpeppe, ofc
<dimitern> fwereade: I don't mind it otherwise
<fwereade> rogpeppe, facades should basically be thin and just contain auth handling, and hand over to shared backend implementations
<rogpeppe> fwereade: so (assuming I've understood the gist correctly) I have no objection to a connection on a model-only API not taking the model tag as parameters
<fwereade> rogpeppe, right, but I do
<rogpeppe> fwereade: i don't get it
<rogpeppe> fwereade: we don't pass the model tag when, say, adding a service
<fwereade> rogpeppe, right, and that is also unhelpful
<rogpeppe> fwereade: i really don't understand. what's the purpose in passing redundant information over the API other than to add more possible error paths and code bloat generally?
<fwereade> rogpeppe, because it expands the scope of places you need to know and understand to comprehend the meaning and effect of a given api call
<fwereade> rogpeppe, tell me more about how extra magic reduces error paths
<rogpeppe> fwereade: i don't consider denormalisation "less magic". it's just more code.
<rogpeppe> fwereade: understanding that an API connection is per-model is fundamental currently
<rogpeppe> fwereade: i think it actually makes the code *harder* to understand
<fwereade> rogpeppe, except for the connection ones, which are a bit more interesting in how they interact with one-or-maybe-many models
<rogpeppe> fwereade: because someone that starts to understand it will think "but why is this necessary?" and there's no good answer
<rogpeppe> fwereade: "the connection ones"?
<fwereade> rogpeppe, s/connection/controller/ sorry
<rogpeppe> fwereade: sure, the controller ones are indeed not model-specific, and that's also fundamental currently.
<rogpeppe> fwereade: and to work on the API you need to have a good understanding of that
<rogpeppe> fwereade: and adding unnecessary parameters just confuses matters
<fwereade> rogpeppe, it's not fundamental at all, it's a consequence of trying to enforce conn<->model in the first place
<rogpeppe> fwereade: we already have hideous amounts of code bloat and it seems like you *like* that
<rogpeppe> fwereade: the conn<->model thing is fundamental to our architecture
<fwereade> rogpeppe, I am surprised that you would pick such a fight on the battleground of "the one approach that actually gets us reused code in apiserver"
<fwereade> rogpeppe, no, it's just a lazy mistake
<rogpeppe> fwereade: it's actually not
<rogpeppe> fwereade: anyway, as i've said far too many times before, bulk calls are an expensive red herring
<rogpeppe> fwereade: not that that specifically applies in this case other than the requirement to watch several models at once.
<rick_h_> dimitern: ping for sync?
<dimitern> rick_h_: oops, omw
<rick_h_> fwereade: ping when you get a sec please. Would like to sync up in https://hangouts.google.com/hangouts/_/canonical.com/rick?authuser=1
<mup> Bug #1613672 opened: Nothing prevents ModelManager.DestroyModels from destroying the controller model <juju-core:New> <https://launchpad.net/bugs/1613672>
<mup> Bug #1613672 changed: Nothing prevents ModelManager.DestroyModels from destroying the controller model <juju-core:New> <https://launchpad.net/bugs/1613672>
<rogpeppe> it's always nice when you start to create a new bug and find that not only is there an existing bug report but the fix went in a couple of hours ago :) https://bugs.launchpad.net/juju-core/+bug/1593350
<mup> Bug #1593350: Juju register allows users to register alternatively named controllers of the same UUID <juju-core:Fix Committed by macgreagoir> <https://launchpad.net/bugs/1593350>
<mup> Bug #1613672 opened: Nothing prevents ModelManager.DestroyModels from destroying the controller model <juju-core:New> <https://launchpad.net/bugs/1613672>
<mup> Bug #1613688 opened: adding a controller user gives user read-write access to all models <juju-core:New> <https://launchpad.net/bugs/1613688>
<wallyworld> dimitern: please has review of small test fix? http://reviews.vapour.ws/r/5448/
<wallyworld> or perrito666 ? ^^^^
<perrito666> wallyworld: checking
<perrito666> wallyworld: shipit
<wallyworld> yay
<wallyworld> ty
<perrito666> now go to sleep
<anastasiamac> wallyworld: perrito666 sadi it ^^
<anastasiamac> said*
<wallyworld> soon
 * rick_h_ goes for a coffee run
<voidspace> frobware: ping
<frobware> voidspace: pong
<voidspace> frobware: what's the best ppa for maas 1.9?
<voidspace> frobware: for trusty
<rick_h_> natefinch: dimitern ping for standup
<frobware> voidspace: pretty sure that ppa:maas/stable
<voidspace> frobware: sounds good - thanks
<dimitern> omw
<perrito666> dimitern: around?
<dimitern> perrito666: o/
<natefinch> mgz: was it juju.environs.simplestreams that I needed at TRACE?  Don't want to run an 8 minute bootstrap to figure it out :/
<mgz> natefinch: yeah, but let me just double check
<mgz> natefinch: yeah, that's what I did last time, probably doesn't hurt to do juju.environs as a whole, as some things have likely changed since then
<natefinch> ahh yes, the trickle down effect
<mgz> natefinch: remember to do --show-log or --log-file global options as well
<natefinch> there's tools-metadata-url and agent-metadata-url.... what's the difference?
<mgz> tools- is the old name
<mgz> they end up at the same place
<natefinch> ok
<voidspace> frobware: does maas 1.9 have vlans?
<frobware> voidspace: yes
<frobware> voidspace: but much easier to set them up in /e/n/i before installation of maas
<voidspace> frobware: ah yes, they need to be in /e/n/i of the rack controller
<voidspace> I forgot, thanks
<voidspace> I've already installed maas unfortunately
<voidspace> I'll add it and restart maas and see if it picks it up
<frobware> voidspace: it won't :(
<voidspace> frobware: bl**dy fecking bunghole
<voidspace> frobware: thanks
<katco> /names
<katco> [12:35]       <katco> rick_h_: ok
<katco>             <rick_h_> katco: ty
<katco> [12:38]         <mup> Bug #1612744 opened: MachinerSuite.TestMachinerMachineAssignedUnits agent should be terminated <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612744>
<katco>                 <mup> Bug #1612745 opened: rackerSuite.TestGainLeadership number of ClaimLeadership is wrong <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612745>
<mup> Bug #1612744: MachinerSuite.TestMachinerMachineAssignedUnits agent should be terminated <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612744>
<mup> Bug #1612745: rackerSuite.TestGainLeadership number of ClaimLeadership is wrong <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612745>
<katco>                 <mup> Bug #1612747 opened: DebugHooksServerSuite.TestRunHook signal: killed <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612747>
<mup> Bug #1612747: DebugHooksServerSuite.TestRunHook signal: killed <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1612747>
<katco> [12:46]             * rick_h_ gets some lunch with that done
<katco> [13:09]    <kwmonroe> ejat: i use juju/azure
<katco> [13:10]        <ejat> kwmonroe: okie .. its working now .. thanks
<katco> [14:11]       <redir> bbiab
<katco> [14:19]             * rick_h_ runs to get my truck and the boy from summer camp. have a good weekend folks
<katco> [14:30]         <mup> Bug #1612775 opened: resource get fails, resource already staged <juju-core:New> <https://launchpad.net/bugs/1612775>
<katco> [15:02]  <perrito666> it would be extremely nice of us to name the same thing in the same way across facades
<mup> Bug #1612775: resource get fails, resource already staged <juju-core:Triaged> <https://launchpad.net/bugs/1612775>
<katco>               <redir> perrito666: novel idea
<katco> [15:03]  <perrito666> authorizer and state are called something different across all facades
<katco> [15:04]  <perrito666> and just found one without authorizer
<katco> [15:13]   <natefinch> sinzui: are you still having problems with that windows deploy CI test?  I seem to be able to deploy a windows charm just fine on azure
<katco>              <sinzui> natefinch: I think not. I am still wating for one to complete to say all is well
<katco>              <sinzui> natefinch: once juju broke the reate limit. azure locked our for many hours
<katco>           <natefinch> sinzui: great :)
<katco> [15:14]   <natefinch> oops
<katco> [15:19]      <sinzui> natefinch: I see some azure successes. most failed though.I suspect the failure were at the tail end of the lockout. I will retest.
<katco> [15:27]       <katco> sinzui: hey does qa have any plans to release the qa tools as a snap?
<katco>              <sinzui> katco: we expect to do dailies
<katco>               <katco> sinzui: daily releases of the qa tools?
<rick_h_> katco: ummm, howdy?
<katco> [15:28]      <sinzui> katco: oh, no. no plans to snape jujuc-ci-tools or juju-release-tools
<katco>               <katco> sinzui: sorry, jujuc-ci-tools
<perrito666> gotta love emacs
<katco>               <katco> sinzui: having just gone through the setup process, it would be awesome to just download a snap
<katco> [15:29]      <sinzui> maybe balloons will want try it. Since snap doesn't work with windows, centos, or osx, we are dedicated to working with a common solution.
<katco> [15:30]       <katco> sinzui: ah good point
<katco>                 <mup> Bug #1612793 opened: bootstrap azure/southcentralus is failing in 2.0-betaX <juju-core:New> <https://launchpad.net/bugs/1612793>
<mup> Bug #1612793: bootstrap azure/southcentralus is failing in 2.0-betaX <azure-provider> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1612793>
<rick_h_> perrito666: heh, is this the equiv of a emacs core-dump?
<katco> [15:39]  <perrito666> ... obtained int = 2
<perrito666> mgz:
<katco>          <perrito666> ... expected int = 2
<katco>          <perrito666> evidently
<natefinch> lol
<katco> [15:44]       <katco> yay my recator compiles for the 1st time
<katco>               <katco> refactor
<katco> [16:39]         <mup> Bug #1612335 changed: Azure rate limit leads to catastrophic failure for subscription <azure-provider> <bootstrap> <ci> <destroy-controller> <kill-controller> <regression> <juju-core:Fix Released by axwalk> <https://launchpad.net/bugs/1612335>
<mup> Bug #1612335: Azure rate limit leads to catastrophic failure for subscription <azure-provider> <bootstrap> <ci> <destroy-controller> <kill-controller> <regression> <juju-core:Fix Released by axwalk> <https://launchpad.net/bugs/1612335>
<natefinch> someone pasted the wrong thing into their IRC window
<katco>                 <mup> Bug #1612722 changed: Juju cannot destroy models/machine, all testing substrates are exhausted <blocker> <ci> <destroy-controller> <kill-controller> <regression> <juju-core:Fix Released by rharding> <https://launchpad.net/bugs/1612722>
<katco> [16:48]         <mup> Bug #1612836 opened: Cannot bring up hosted  model machines in azure <azure-provider> <ci> <deploy> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1612836>
<mup> Bug #1612722: Juju cannot destroy models/machine, all testing substrates are exhausted <blocker> <ci> <destroy-controller> <kill-controller> <regression> <juju-core:Fix Released by rharding> <https://launchpad.net/bugs/1612722>
<mup> Bug #1612836: Cannot bring up hosted  model machines in azure <azure-provider> <ci> <deploy> <regression> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1612836>
<katco> [17:14]       <katco> rick_h_: ping, don't suppose you're on
<katco> [17:34]  <perrito666> mmm, ppas dont work nicely with my deb proxy
<katco> [20:15]       <redir> EoW. LALters
<katco>               <redir> laters even
<katco> ERC>
<natefinch> last time on juju-dev....
<perrito666> looool
 * balloons hears natefinch in a cool annoucer's voice
<perrito666> I read him with the bsg font
<sinzui> katco: Since the azure provider is broken by bug 1612836 we cannot see if the windows deployment is still broken
<mup> Bug #1612836: Cannot bring up hosted  model machines in azure <azure-provider> <ci> <deploy> <regression> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1612836>
<perrito666> sinzui: katco left the channel
 * perrito666 pastes the last 100 lines to show sinzui 
<natefinch> lol
<natefinch> she must have hit the wrong thing in emacs
<perrito666> it all started with a /names so she most likely hit the combination for paste instead of enter
 * redir has flashback and blinks
<mgz> it wasn't even a flood kick...
<perrito666> katco: are you ok?
<katco> wow i have no clue what happened
<perrito666> need an exsorcism?
<perrito666> lol, I have no clue how to write that in english I just noticed
<katco> i don't even know what it ended up posting to the channel... did i spam everyone or something?
<rick_h_> katco: :) a bit
<perrito666> katco: it was a nice recap of friday
<katco> ...
<rick_h_> katco: we had a rerun episode of #juju-dev, not a bad one though.
<katco> did it send the buffer from friday?
<perrito666> katco: you did
<katco> weird...
<perrito666> katco: where you trying to send the buffer from a different day?
<katco> i was trying to ask mgz and sinzui a question lol... let me try to paste this error message again
 * perrito666 cringes
<katco> mgz: sinzui: have you seen this error before? value *errors.Err = &errors.Err{message:"", cause:(*errors.Err)(0xc82395c5f0), previous:(*errors.Err)(0xc82395c640), file:"github.com/juju/juju/cmd/juju/application/deploy.go", line:691} ("the provided bundle has the following errors:\ncharm path in application \"dummy\" does not exist: dummy")
<katco> (sorry for the weird spam earlier all)
<katco> mgz: sinzui: trying to figure out if this is a spurious error, i.e. a timing issue in CI where the dummy path doesn't exist before the bundle is deployed, or...?
<mgz> katco: new to me. looks like more local charm/deploy syntax confusion?
<sinzui> katco: no. do you have juju-ci-tools/repository checked out and do you gave JUJU_REPOSITORY seto the path so that the path to the charm is correct
<katco> mgz: hm, ok. i didn't get this locally when running through the test suite. i'll try the ci suite next
<dimitern> rick_h_: whew.. bug 1603473 updated
<mup> Bug #1603473: Relation fails as units/machines are on different subnet on a multi NIC setup, juju 2.0 beta11 <11> <2.0> <beta> <juju> <juju-core:Incomplete by dimitern> <https://launchpad.net/bugs/1603473>
<katco> sinzui: this is from an official ci run, not me running ci locally
<rick_h_> dimitern: ty, let's see how that goes
<dimitern> rick_h_: can you have a look and perhaps comment, if needed?
<rick_h_> dimitern: /me goes to look
<sinzui> katco: good, my thoughts remain the same.
<mgz> katco: github-merge-juju/8788 ? looks like a real issue with your branch I think
<dimitern> rick_h_: cheers!
<natefinch> mgz: thanks for the help with the logging... that helped get me past my problem (the code was expecting to be pointed one directory deeper than I realized.... if only we had documentation for this sort of thing.
<mgz> natefinch: ace, glad it helped
<katco> mgz: i was looking at 8785, but i'm sure that one is the same
<mup> Bug #1613476 changed: juju cli commands should be logged by juju <oil> <oil-2.0> <juju-core:Invalid> <https://launchpad.net/bugs/1613476>
<natefinch> mgz: it's not *working* but at least I get better output and fixed that one problem
<admcleod> babbageclunk: hello
<babbageclunk> admcleod: hey!
 * natefinch lunches
<admcleod> babbageclunk: hows it going?
<babbageclunk> admcleod: yeah, good thanks - how are you? Still in Spain?
<admcleod> babbageclunk: not bad, yeah im still in spain but im in barcelona now...
 * rick_h_ grabs lunchables
<admcleod> babbageclunk: i think i said i was going to live in valencia. im not sure now :)
<babbageclunk> admcleod: who can be sure in this mad world?
<beisner> hi all.  ? re: manual provider.  by default, lxc units get placed behind nat on the bridge, meaning lxc units are in a bit of a bubble in terms of directly communicating with lxc units on another manual unit.  is there a juju-ism to specify transparent bridge?
<admcleod> babbageclunk: i just dont know. which is a great segue into my question about juju lxc networking with manual provider
<admcleod> babbageclunk: which is also beisers question
 * beisner yields the floor :)
<katco> mgz: sinzui: thanks, i can reproduce locally
<admcleod> beisner: was it something i said?
<beisner> admcleod, ha!
<rick_h_> admcleod: beisner i'm not aware of one atm. file a bug and i'll see if the network experts have any thoughts. it might requure some juju-run-fu
<rick_h_> admcleod: beisner more complicated setups require the provider worm (such as maas has gotten) so the manual provider hasn't gotten complex network love
<rick_h_> admcleod: beisner there's a todo for using the fan on public clouds for container coms, worth adding manual to that list it seems.
<admcleod> rick_h_: ah right, thanks
<beisner> rick_h_, thanks, we'll push on with a few ideas and raise a bug if it looks like something outside the expected behavior.
<frobware> voidspace: ping - do you still have problems at the commissioning stage with my scripts?
<frobware> so it turns out my first snap is never going to work... :(
 * frobware one square to back...
<mup> Bug #1613782 opened: windows cannot bootstrap trusty <bootstrap> <ci> <jujuqa> <regression> <windows> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613782>
 * frobware has exactly this problem, but for accessing .maascli.db: https://www.mail-archive.com/snapcraft@lists.ubuntu.com/msg00345.html
<natefinch> anyone know what this means? sinzui, mgz: 2016-08-16 15:25:49 DEBUG juju.environs.simplestreams simplestreams.go:454 skipping index "https://s3.amazonaws.com/24b99da448314945835933a0c8edffb3/tools/streams/v1/index2.json" because of missing information: "content-download" data not found
<mup> Bug #1613804 opened: mongod sigabrt <intermittent-failure> <test-failure> <windows> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613804>
<mgz> abentley: ^any ideas on natefinch's message?
<natefinch> I see  "datatype": "content-download" under index": {  "com.ubuntu.juju:released:tools": {
<abentley> natefinch: I can tell you that LXD and MAAS use "content-download" and AWS uses "image-id".
<natefinch> abentley: what does azure use?  Because that's where I'm trying to deploy
<natefinch> note this is streams for tools, not images
<abentley> natefinch: Oh, sorry.  For tools, everything uses content-download.
<abentley> natefinch: I see that this index has only the released streams.  Are you testing a beta?
<natefinch> abentley: I'm testing my own code
<sinzui> natefinch: abentley is suggesting that you need to decale the agents to be devel
<natefinch> man, I really wish I could just juju upload-tools win2012hvr=/home/nate/bin/jujud.exe
<abentley> natefinch: What sinzui said.  You'll need devel streams or to set agent-stream=released for this to work.
<sinzui> natefinch: I think the call to juju metdata generate-agents needs an arg to make agent stanzas
 * sinzui though juju refused to honour agent-stream=released when the version has letters in it
<natefinch> lol
 * natefinch is gonna have to document the crap outta this
<sinzui> natefinch: I think the command is
<sinzui> uju metadata generate-tools -d <workingdir> --stream
<sinzui> natefinch: and we can detect a letter in the version to switch from released to devel. I can fix that now
<mup> Bug #1613823 opened: Google App Engine IP is ephemeral by default  <juju-core:New> <https://launchpad.net/bugs/1613823>
<mup> Bug #1613824 opened: cmdSubnetSuite teardown fails: unexpected message <ci> <intermittent-failure> <jujuqa> <unit-tests> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613824>
<natefinch> sinzui: I was just using the streams-from-local script
<sinzui> natefinch: yeah, that is the script I ma going to change
<natefinch> sinzui: ok.  yes, it would be nice if what it output worked by default with the ci scripts that it is right next to.
<sinzui> natefinch: the script is checking for alpha or beta in the version and setting devel. What version is yoru juju?
<natefinch> sinzui: beta.. whatever master is
<sinzui> natefinch: in your stream directory, do you have a sir named "tools/evel"
<sinzui> natefinch: in your stream directory, do you have a sir named "tools/devel"
<natefinch> sinzui: no, just released.  I'm currently rerunning with agent-stream=released
<sinzui> natefinch: is your system juju 1.25.6?
<sinzui> natefinch: well that wouldn't matter becuase the script runs jujud --version
<sinzui> natefinch: found the problem.
<mup> Bug #1613829 opened: TargetDiskBlobAlreadyExists on deploy (Azure) <azure-provider> <ci> <deploy> <intermittent-failure> <jujuqa> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613829>
<mgz> rick_h_: got a slot for a pre-imp call some point in the next few hours?
<rick_h_> mgz: sure thing, getting off the phone shortly
<rick_h_> mgz: free when you are
<dimitern> anybody up for a short review ? http://reviews.vapour.ws/r/5449/ - fixes bug 1612624
<mup> Bug #1612624: Bootstrap fail on MAAS if ipv6 is disabled <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1612624>
<mup> Bug #1613829 changed: TargetDiskBlobAlreadyExists on deploy (Azure) <azure-provider> <ci> <deploy> <intermittent-failure> <jujuqa> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613829>
 * dimitern eod-s
<natefinch> sinzui: does --agent-stream work?  It seems like it's being ignored: http://pastebin.ubuntu.com/23062301/
<sinzui> natefinch: that is what I muttered about above. wallyworld told me he added a rule that a devel version would never use  released streams. in CI tests, we create devel streams
<natefinch> sinzui: I guess I can just edit my tools to s/released/devel
<mup> Bug #1613829 opened: TargetDiskBlobAlreadyExists on deploy (Azure) <azure-provider> <ci> <deploy> <intermittent-failure> <jujuqa> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613829>
<sinzui> natefinch: you. can. my change to parse the "juju version" does pass the riight args, but juju then flips out because it seems to alway require a released stream too
<sinzui> natefinch: generate-tools refuse to make devel streams :( you only need to change "released" ot "devel" in index2.json to move forward. I am going to explore this approach for the script
<natefinch> sinzui: this is like some greek tragedy
<natefinch> sinzui: I can edit easily enough, will retry once I fix it in s3
<natefinch> gah, this stuff is a wall of text
<natefinch> http://pastebin.com/raw/RAYY12i8
<mup> Bug #1613838 opened: TestBootstrap on arm/ppc64el/s390 cannot build agent binary <arm64> <bootstrap> <ci> <jujuqa> <ppc64el> <regression> <s390x> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613838>
<mup> Bug #1613838 changed: TestBootstrap on arm/ppc64el/s390 cannot build agent binary <arm64> <bootstrap> <ci> <jujuqa> <ppc64el> <regression> <s390x> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613838>
<natefinch> sinzui: do you understand what's wrong here? ^  I don't know what the streams logging is supposed to look like, so it's hard for me to understand what's wrong
<natefinch> it looks like one of the products is com.ubuntu.juju:win2012r2:amd64  and one of the candidates is the same
<natefinch> unfortunately someone printed out with %v not %#v so it's hard to map the textual output to the fields in the struct that it's printing
<sinzui> natefinch: streams/v1/com.ubuntu.juju-devel-tools.json is listing agents we didn't even include in streams. this is crazy
<natefinch> what's weird is that this log line is after the code has supposedly stripped out all non-matching candidates, yet clearly there are still non-matching ones, like com.ubuntu.juju:12.04:amd64
<natefinch> and in fact, the next line just chooses the first matching one, which means it is choosing com.ubuntu.juju:12.04:amd64
<natefinch> not that it tells us that...
<natefinch> in the logs
<mup> Bug #1613838 opened: TestBootstrap on arm/ppc64el/s390 cannot build agent binary <arm64> <bootstrap> <ci> <jujuqa> <ppc64el> <regression> <s390x> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1613838>
<mup> Bug #1613839 opened: bundle deployment does not accept "1.0" as valid float <juju-core:New> <https://launchpad.net/bugs/1613839>
<sinzui> natefinch: generate-tools is completely busted. It never includes the agents we asked to be added. this proabbly because 1.x is not allowed to know about 2.x. I amd going to switch system jujus and see if 2.x can make streams for 2.x
<natefinch> sinzui: k
<bleepbloop> I did a reboot of my juju state nodes and when they restarted I lost the ability to run `juju --debug status` and now get an error `health ping failed: connection is shut down` and in my `juju debug-log` I am getting ` cannot publish API server addresses: cannot set API addresses: state changing too quickly; try again soon`. I installed the version that came
<bleepbloop> with xenial which is 2.0 beta12, what is weird is everything else seems to still be working, it is just freaking out when juju status is run
<bleepbloop> Hopefully this is the right place to ask since the issue is with the beta version
<mup> Bug #1613839 changed: bundle deployment does not accept "1.0" as valid float <juju-core:New> <https://launchpad.net/bugs/1613839>
<natefinch> bleepbloop: we encountered something like this.. I think it's been fixed since that beta
<natefinch> bleepbloop: hmm.. the one I'm thinking of was fixed before beta 12
<mup> Bug #1613839 opened: bundle deployment does not accept "1.0" as valid float <juju-core:New> <https://launchpad.net/bugs/1613839>
<bleepbloop> @natefinch Hmm okay, I'm also seeing a `pinger failed: E11000 duplicate key error collection: presence.presence.pings`, could that be related to the `health ping failed` error?
<meetingology> bleepbloop: Error: "natefinch" is not a valid command.
<bleepbloop> Oops meant natefinch:, been using slack too much :P
<natefinch> haha
<natefinch> bleepbloop: I think it's very similar to the other bug that we fixed, maybe this is another instance of it nearby in the code.  It's an error from juju's mongo driver.  Basically, mongo is changing stuff out from under us because Â¯\_(ã)_/Â¯ it's mongo
<mup> Bug #1613842 opened: [Feature] Juju Resources should also support fingerprint checking <juju-core:New> <https://launchpad.net/bugs/1613842>
<natefinch> bleepbloop: the pinger thing I'm guessing is a different problem... but not 100% sure.
<bleepbloop> Hmm okay, is there a good way to workaround it for now? I just tried to upgrade to beta15 to see if that would help and ended up with a "ERROR unknown object type "ModelConfig" (not implemented)" so that seems to not be the way to go :/
<natefinch> sinzui: we don't officially support upgrading from beta to beta, do we?
<sinzui> natefinch: no we don't
<natefinch> bleepbloop: I understand this is the version that comes with Xenial... it's unfortunately still in beta due to, well, a bunch of reasons.
<natefinch> bleepbloop: rebooting may help.  It generally only happens then the machine is under load
<natefinch> bleepbloop: it may help to simply not reboot and leave it alone for a while, too.  Upgrade sometimes can work and sometimes doesn't... seems like you're hitting one of the "it doesn't"
<natefinch> bleepbloop: I filed a bug for it - https://bugs.launchpad.net/juju-core/+bug/1613855  It'll get adressed soon, but that won't help fix your current environment, unfortunately.
<mup> Bug #1613855: SetAPIHostPorts runs afoul of "state changing too quickly" <juju-core:New> <https://launchpad.net/bugs/1613855>
<bleepbloop> natefinch: Okay, no problem I rolled back from the ppa, it seems to have not made things any worse so thats good at least, would it help if I  included the full logs for the error?
<mup> Bug #1613855 opened: SetAPIHostPorts runs afoul of "state changing too quickly" <juju-core:New> <https://launchpad.net/bugs/1613855>
<bleepbloop> Also just found some more fun `panic: runtime error: index out of range` seems to be happening on all of the nodes (servers are running in HA)
<natefinch> bleepbloop: ew
<natefinch> bleepbloop:  you may be in a bad state if you tried to upgrade and it failed halfway through.  Still, I wouldn't expect to see a panic like that.
<bleepbloop> natefinch: I don't think the panics are due to my upgrade attempt, the panics all happened 2-3 hours ago where all I'd done is rebooted the machines
<natefinch> ahh hmm
<natefinch> can you pastebin me one of them?
<bleepbloop> sure no problem, one second
<bleepbloop> natefinch: http://pastebin.com/UKeD4msg it happens 3 times in the included log
<bleepbloop> natefinch: if you search for "panic: runtime error" you'll find it
<mup> Bug #1613858 opened: generate-tools ignores 2.x agents and creates bad path <jujuqa> <metadata> <regression> <streams> <juju-core:Triaged> <https://launchpad.net/bugs/1613858>
<natefinch> bleepbloop: looks like you stumbled on another bug
<natefinch> bleepbloop: yet another notch on your belt - https://bugs.launchpad.net/juju-core/+bug/1613866  Sorry you're running into these problems.  Not really sure why we haven't seen this one more often... maybe something special in the timing in your system.
<mup> Bug #1613866: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
<mup> Bug #1613864 opened: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1613866 opened: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
<natefinch> rick_h_: this whole streams debacle is just crazy
<mup> Bug #1613864 changed: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1613866 changed: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
<natefinch> rick_h_: it's very obvious that we need much better UX around setting custom streams URLs if we actually expect anyone to ever do it, not to mention much better facilities for debugging problems with it.  And that's not to mention the fact that we have a showstopped bug with it.
<mup> Bug #1613864 opened: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1613866 opened: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
<mup> Bug #1613864 changed: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1613866 changed: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
<natefinch> thumper: Let me know what you think of my assessment of this bug: https://launchpad.net/bugs/1613855
<mup> Bug #1613855: SetAPIHostPorts runs afoul of "state changing too quickly" <juju-core:New> <https://launchpad.net/bugs/1613855>
<natefinch> (sorry to jump on you :)
<thumper> morning
<natefinch> morning :D
<thumper> natefinch: yep, agree with your analysis
<natefinch> yay
<thumper> needs to be fixed asap
<thumper> natefinch: I've marked it critical
<rick_h_> natefinch: don't disagree. It's why wallyworld's spent time and things trying to find ways to work w/o streams
<mup> Bug #1613864 opened: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1613866 opened: server panic during juju status if no status history for a unit <juju-core:New> <https://launchpad.net/bugs/1613866>
 * natefinch writes his own streams data by hand
<bleepbloop> So as a last ditch effort and to curb my own curiousity, are there more up to date instructions with exactly how to connect to the mongodb database directly?
<rick_h_> natefinch: what happened with kicking off a CI run, pausing it, and investigating that? I know you were trying to do that the other day and didn't hear why that fell over?
<rick_h_> bleepbloop: hmm, someone had a tool for helping with that. katco?
<katco> rick_h_: a tool for pausing ci?
<natefinch> rick_h_: there was some problem where the new code was not available yet.  but maybe that's no longer the case.  sinzui?
<rick_h_> katco: no, for connecting to mongodb
<rick_h_> katco: maybe I was wrong it wasn't you. /me is trying to search email for it
<katco> rick_h_: oh, right... i sent a script out to the email list
<katco> natefinch: rick_h_: check juju-dev
<rick_h_> katco: right, that's what I was thinking of. bleepbloop is asking about that and maybe the list email would be a useful start for bleepbloop
<katco> bleepbloop: ok, one sec
<rick_h_> katco: july 27th, found the email
<katco> bleepbloop: https://lists.ubuntu.com/archives/juju-dev/2016-July/005772.html
<katco> bleepbloop: give that a try. fairly easy to understand
<rick_h_> ty katco
<katco> rick_h_: hth
<bleepbloop> Thanks!
<natefinch> sinzui: is CI able to run that windows test I've been trying to run, and if so, could we run it with --keep-env so I can investigate it?
<cmars> are there docs, on how to test with clocks ?
<katco> what's the testing suite that allows me to create temporary files/directories?
<natefinch> katco: c.Mkdir()
<natefinch> katco: not a suite
<katco> natefinch: ta
<mup> Bug #1587644 changed: jujud and mongo cpu/ram usage spike <canonical-bootstack> <canonical-is> <juju-core:Fix Released> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1587644>
<mup> Bug #1612653 changed: lxd fails in beta15 when installing via snap <juju-core:Triaged> <https://launchpad.net/bugs/1612653>
<mup> Bug #1613882 opened: stateSuite.SetUpTest mongo is unreachable <ci> <intermittent-failure> <regression> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1613882>
<bleepbloop> Thank you all for your help! Last question, how well does 1.25 work on xenial? Are there any compatibility issues or should it be fully compatible?
<perrito666> bleepbloop: I don't think 1.25 works on xenial
<rick_h_> natefinch-afk: this has your name on it, can you update/create a card/etc please? https://bugs.launchpad.net/juju-core/+bug/1552274
<mup> Bug #1552274: juju list-credentials inconsistencies between format output <2.0-count> <bitesize> <conjure> <juju-release-support> <rc1> <usability> <juju-core:In Progress by natefinch> <https://launchpad.net/bugs/1552274>
<bleepbloop> I see there are xenial packages for 1.25, is there anything specific that doesn't work?
<perrito666> bleepbloop: it was an educated guess, I am not sure the right mongo package is built for xenial
<perrito666> but then again, I might be wrong
<perrito666> rick_h_: do you know?
<rick_h_> bleepbloop: perrito666 nothing that I'm aware of? I mean it works?
<perrito666> rick_h_: it means it works
<mup> Bug #1613888 opened: Cannot destroy application (contention) shortly after unblocking it <ci> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1613888>
 * rick_h_ runs for the night
 * redir hopes rick_h_ catches the night.
<mup> Bug #1613892 opened: juju "show" flags are inconsistent  <juju-core:New> <https://launchpad.net/bugs/1613892>
<perrito666> Will be 5 late to standup
<axw> wallyworld: this needs another review if you would later: http://reviews.vapour.ws/r/5447/
<perrito666> wallyworld: you just dropped
#juju-dev 2016-08-17
<wallyworld> rick_h_: there are two tech debt bugs you moved to beta16, they are really not urgent, i'd like to move them back eg 1577589
<mup> Bug #1442149 changed: UniterSuite.TestUniterUpgradeConflicts fails <ci> <intermittent-failure> <test-failure> <juju-core:Invalid> <juju-core 1.24:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1442149>
<rick_h_> wallyworld: ok i think you meam ones marked as adsigned and were theybin progress?
<rick_h_> wallyworld: ok with moving them, just wanted to checknon reality vsnbug info
<wallyworld> vsn?
<rick_h_> vs bug info
<rick_h_> darn phone typing ftw
<wallyworld> np :-) yeah, will be nice to get the affect methods sorted, but not this week
<anastasiamac> wallyworld: so moving to beta17 and re-assigning from Horacio :D
<wallyworld> no,
<wallyworld> horatio still needs to fix at some point
<wallyworld> and not necessarily for beta17
<wallyworld> moved back to 2.0.0 for now
<anastasiamac> tyvm
<redir> I need a hobby to do while waiting for test runs
<anastasiamac> redir: picking up and fixing a bug could b a very relaxing hobby :D
<redir> anastasiamac: but I can't run the tests while working on a different branch.
<redir> and my other computer is running gostress on that race,  unsuccsessfully
<thumper> Unable to fetch Juju GUI info: error fetching simplestreams metadata: cannot read product data, invalid URL "https://streams.canonical.com/juju/gui/streams/v1/com.canonical.streams-released-gui.sjson" not found
<thumper> wat?
<thumper> do we care?
<thumper> I feel we should
<anastasiamac> thumper: we should as we can browse to it..
<anastasiamac> thumper: what are u running? how are u hitting it?
<thumper> http://pastebin.ubuntu.com/23063199/
 * thumper is tapping fingers while reproducing issue
<thumper> well, hopefully QAing that I've fixed it
<anastasiamac> thumper: if there is no bug, I'd say it ^^ should be
<anastasiamac> thumper: and u can navigate to this URL, right? it's not like u do not have internet access...
<redir> what's internet access
<redir> ?
<thumper> wallyworld: ping
<wallyworld> wot
<thumper> chat?
<wallyworld> sure
<redir> wallyworld: this bud's for you http://reviews.vapour.ws/r/5454/
<wallyworld> redir: thanks, will look soon
<redir> wallyworld: ^ it's the dummy provider region stuff. If you approve I'll come back and merge for a fresh start tomorrow.
<redir> bbiab
<wallyworld> ok
<natefinch> wallyworld: can you help me debug my custom simplestreams setup for agent tools?  I'm trying to deploy a windows binary on azure but it keeps saying no tools found even though it really should be finding them, and I can't tell from the logs why it's not finding them.
<wallyworld> i'm sorta tied up trying to fix a CI issue, debug output lists the steps it goes through along the way, does that not help?
<wallyworld> you can also check what gets into state as that iis where the tools come from
<wallyworld> when you use metadata-source
<wallyworld> using metadata-source is supposed to grab the tools and save in state
<wallyworld> and from there, no simplestreams is used
<wallyworld> as the controller will search state and see the tools in there
<natefinch> wallyworld: not enough..  it finds the index2.json in my s3 bucket, finds the list of candidates... one of which looks like it should match, and then the next thing it logs is trying the default simplestreams source
<natefinch> wallyworld: I think there's a bug filed about metadata source not working, unless that's been fixed
<wallyworld> you can add debug to the match function
<wallyworld> see why there's no match
<anastasiamac> natefinch: metadata-source is not fixed.. in progress
<natefinch> anastasiamac: cool
<wallyworld> probs the easiest is to add extra debug to the match function to print for each candidate why the match failed
<wallyworld> it's probably related to dev vs non-dev
<natefinch> wallyworld: yeah, I'll do that. Thanks
<wallyworld> that stuff really needs to die (the old dev paradigm)
<wallyworld> odd version = dev stuff
<wallyworld> it's all obsolete now
<natefinch> I'm certainly more than happy to never have to deal with simplestreams again :)  generate metadata is *also* broken, so I had to actually hack the metadata by hand.
<wallyworld> natefinch: and if the debug is useful, leave it in there for the next poor f*cker who has to deal with this
<natefinch> wallyworld: haha yeah
<wallyworld> what's broken with egenrate?
<natefinch> wallyworld: only generates for 1.x versions or something... lemme find the bug
<natefinch> https://launchpad.net/bugs/1613858
<wallyworld> that tools hasn't been updated in so long
<natefinch> heh, I guess mup doesn't like that format - https://bugs.launchpad.net/juju-core/+bug/1613858
<wallyworld> not surprised it's out of date
<natefinch> or mup's broken
<natefinch> I was just lamenting that all I really want is juju upload-tools win2012r2 amd64 /path/to/jujud.exe
<natefinch> we always get so fancy, and what I really want is the simplest thing that can possibly work
<wallyworld> natefinch: we can add that command now that tools are stored in state, it was never possible before. but no one has gotten around to it
<wallyworld> it's been one of those friday afternoon things
<wallyworld> maybe you have an itch you'd like to scratch :-)
<natefinch> wallyworld: like I've been rolling in poison ivy
<wallyworld> lol
<axw> thumper: is migration supposed to handle unprovisioned machines?
<thumper> no
<thumper> the prechecks will fail
<thumper> at this stage
<axw> okey dokey
 * thumper taking kiddo to hockey
<thumper> back later
<natefinch> wallyworld: if you use upload-tools, we don't look in simplestreams at all, do we?
<wallyworld> correct, but upload tools now deeted
<wallyworld> deleted
<wallyworld> and so we do look
<natefinch> heh
<natefinch> I should pull master evidently
<wallyworld> and if none fund, we upload
<natefinch> I really miss being able to just tell juju what to, rather than hoping it'll do what I want it to do :/
<natefinch> It still seems like making the same mistake we made with ensure-HA, where we'd do the "right thing" but then users couldn't tell what the command would do before they ran it.
<natefinch> wallyworld: so... if I build from current master... it always uses upload-tools?
<natefinch> (looks like yes, from the logs)
<natefinch> still does the nasty "add a build number of .1" though, I see.  ick.
<wallyworld> that's expected and required
<wallyworld> so we can tell what has been custom uploaded
<wallyworld> so when people say they have an issue we know if it is stock or not
<natefinch> there's a lot of ways to record that we've custom uploaded without changing the version number itself.  Maybe the version number is the most visible... just annoyed I have to go modify my streams metadata to make the version .1 (I presume juju will only accept exact matches to what is on the controller)
<natefinch> wallyworld: do you know why this if statement is there? https://github.com/juju/juju/blob/master/environs/simplestreams/simplestreams.go#L457   seems to prevent us from logging the simplestreams error unless it's one specific kind
<natefinch> wallyworld: for example, it prevented me from seeing the json unmarshal error from my simplestreams data
<wallyworld> not sure anymore, i think it was because there we all sorts of reasons why tools search could fail (external to juju) and we didn't care about the specifics - if one path failed, the next would be tried
<wallyworld> anything from unauthorised http to url not being there etc etc
<wallyworld> also stale streams data that wasn't cleaned up
<natefinch> all of that seems like useful information... if the user has set a streams url, they probably expect it to be used.
<wallyworld> sometimes, but not always
<wallyworld> they could only have a subset of data there and expect fallback to official sources
<wallyworld> the output was waaaay noisy with all the "expected" errors
<wallyworld> and so i think people wanted it to be quiet
<natefinch> maybe we can put back the general error logging in trace?
<wallyworld> sure
<wallyworld> if there's a need for it, i have no opinion either way
<wallyworld> i was happy enough with lots of output
<wallyworld> but when people bootsrapped with --debug they complained
<wallyworld> especially since we had up to 4 datasources wach with json and sjson
<wallyworld> and index and index2
<natefinch> yeah, I agreed with them at the time, but when things break, it's nice having the output.  It's hard to know when it's an expected error and when it's not
<wallyworld> yeah, i guess it hasn't been an issue because setting up the streams has all been upstream from us
<wallyworld> done by the Qa guys
<natefinch> yep
<wallyworld> as you say, not till you roll around in the poison ivy that you start to itch :-D
<natefinch> haha yep
<wallyworld> did you finf your issue?
<wallyworld> in thw matching?
<natefinch> yeah, some invisible unicode character in the end of my tools.json was making it fail unmarshaling
<wallyworld> ah
<natefinch> we'll see what happens when I try again with it fixwed
<natefinch> this is like the 8th problem I've fixed
<natefinch> I think always logging that error message will help a lot though.
<natefinch> holy crap I think it worked
<redir> trumpets blare
<redir> angels sing
<redir> first growth bordeaux rains from the sky
<natefinch> I've been trying to get ci to run with my windows tools for literally 16 hours of active work time
 * redir fails to generate a good relativity joke
<redir> wallyworld: http://reviews.vapour.ws/r/5454/ has a shipit button that is calling your name:)
<wallyworld> ooh, let me look
<wallyworld> redir: +1 from me, we can debate the name next pr
<redir> wallyworld: works for me.
<wallyworld> natefinch: you would be a good person to review the doc page for adding your own tools then :-)
<wallyworld> https://jujucharms.com/docs/1.25/howto-privatecloud
<wallyworld> axw: care to do a very small review to fix a CI regression http://reviews.vapour.ws/r/5456/ ?
<axw> sure
<axw> natefinch: what are you doing with windows agents? I wrote a plugin for cross-compiling/building and uploading agents, might be useful...
<veebers> wallyworld, axw: Who's best to ask about grant revoke? I'm debugging the user grant/revoke test. I see a user removed but the user still remains in the list-shares output. (but is gone from list-users output)
<veebers> That might be a bug?
<wallyworld> sounds like it, i think there's a bug already raised
<wallyworld> https://bugs.launchpad.net/juju-core/+bug/1613444
<mup> Bug #1613444: Remove-user doesn't remove user from list-shares <ci> <list-shares> <regression> <revoke> <juju-core:Triaged by hduran-8> <https://launchpad.net/bugs/1613444>
<axw> wallyworld: reviewed
<wallyworld> ta
<veebers> wallyworld: ah thanks missed that bug
<wallyworld> no wuckers
 * veebers googles wuckers
<veebers> hah, love it
<veebers> wallyworld: ok one more concern, if I do something like: "juju add-model new-model2 && juju destroy-model new-model2 -y" new-model2 stays around in list-models output. I would say this is a bug even if it's pretty odd to create then immediately destroy the model
<veebers> oh also, I cannot destroy the model now :-\ it's not found when I try
<wallyworld> veebers: surely as a kiwi you'd know what "wuckers" means? :-D
<wallyworld> it takes a finite time for the model to be cleaned up
<veebers> wallyworld: hah never heard the shorthand wuckers, def heard the long form
<wallyworld> perhaps we can improve the experience there
<wallyworld> there may be a bug already, not sure tbh
<veebers> wallyworld: it's been 4+ minutes and the model still appears in list-models
<veebers> (this is local lxd)
<wallyworld> i'd need to check - i think list-models just looks at local yaml file
<wallyworld> well, it used to
<wallyworld> worth a bug so we can investigate
<wallyworld> if there's not already one there
<veebers> I'll have a look see if I can find one
<veebers> axw: as an aside I think you can this one now right? https://bugs.launchpad.net/juju-core/+bug/1605710
<mup> Bug #1605710: Fix and reland axw/cli-model-owner <ci> <grant> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1605710>
<axw> veebers: yup, thanks
<veebers> wallyworld: fyi I filed 1613960
<wallyworld> ok, ta
 * thumper-afk headdesks
<thumper-afk> oh
<thumper-afk> didn't change back
<thumper> wallyworld: bumping up against romulus dependencies
<thumper> with the command formatter branch
<wallyworld> \o/
<wallyworld> details?
<thumper> it specifies command formatters
<thumper> and I've changed them :)
<wallyworld> ah
<wallyworld> patches accepted :-D
<thumper> bah humbug
<mup> Bug #1605710 changed: Fix and reland axw/cli-model-owner <ci> <grant> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1605710>
<mup> Bug #1613960 opened: list-models can show a model that was supposed to have been deleted <juju-core:New> <https://launchpad.net/bugs/1613960>
<wallyworld> thumper: the default yaml marshaller does append a new line, but weren't we stripping it off for a reason, so as not to append an extra one in our output?
<thumper> no, no good reason at all
<wallyworld> why change the existing output behaviour just to add colour?
<thumper> this isn't
<thumper> this is standardising on the formatters
<wallyworld> ok
<thumper> why not output what it is?
<thumper> if it is new lines
<thumper> output the damn new lines
<thumper> it won't change any of our output
<thumper> because we don't marshall strings by themselves
<thumper> because that's dumb
<thumper> it is attempting to be smart of the hell of it
<thumper> when there is no good reason
<wallyworld> and yet a newline is appended to json
<thumper> because it doesn't append a new line to the end
<thumper> the formatters make sure it finishes with a new lien
<thumper> lien
 * thumper headdesks
<thumper> line
<wallyworld> <thumper> why not output what it is?
<wallyworld> :-P
 * thumper slaps wallyworld
<wallyworld> lol
<wallyworld> don't you hate it when your own words come back and slap you on the arse
<wallyworld> i gave it a ship it, but just make sure you don't f*ck up anything
<thumper> we add a new line to the end of the json so your next prompt doesn't end up at the end of the json line
<wallyworld> exactly
<wallyworld> but
<wallyworld> wouldn't stripping the yaml \n have cuased the same issue?
<wallyworld> so maybe we are adding a \n somewhere else
<wallyworld> and now for json we'll get 2
<wallyworld> i may be wrong, just a guess based on what the code used to do
<wallyworld> it would have been stripping the \n for a reason
<wallyworld> i bet we add a \n in juju
<wallyworld> i reckon you need to try it out before landing to confirm one way or the other
<thumper> wallyworld: for you :) http://reviews.vapour.ws/r/5457/
<thumper> no
<thumper> you don't get it
<thumper> if you want to chat we can
<thumper> but you are missing something
<wallyworld> it's ok, i trust you
<wallyworld> almost
<thumper> :)
<thumper> let's just say "I won't fuck it up" (probably)
<thumper> but now, it's dinner time
<thumper> so I'll look tomorrow morning
<thumper> laters
<axw> TIL: there's (fairly minimal) Go bindings for OpenStack in their tree: http://git.openstack.org/cgit/openstack/golang-client/
<wallyworld> axw: maybe we can hope to kill off goose for xmas dinner at some point :-)
<axw> wallyworld: that would be nice
<ejat> jamespage: are u there
<mup> Bug #1613992 opened: 1.25.6 "ERROR juju.worker.uniter.filter filter.go:137 tomb: dying" <juju-core:New> <https://launchpad.net/bugs/1613992>
<wallyworld> axw: did you get a chance to look at the manual provider issue?
<axw> wallyworld: nope, been looking at the azure one all day
<wallyworld> np
<voidspace> as usual uninstalling maas and trying to reinstall completely fecked the VM
<voidspace> starting again from scratch
<babbageclunk> voidspace: :(
<voidspace> babbageclunk: :-)
<voidspace> babbageclunk: hey, hi
<babbageclunk> voidspace: Maybe snapshot before installing?
<voidspace> babbageclunk: yeah, good plan
<voidspace> babbageclunk: wotcha working on?
<babbageclunk> voidspace: the worker to release container addresses. Hopefully the last bit!
<voidspace> babbageclunk: ah, cool
<voidspace> babbageclunk: I'm on a 1.25 backport of network stuff (the bridge scripts), so installing maas 1.9
<voidspace> babbageclunk: I had to go back and read up how to configure a maas 1.25 environment too, it's that long since I've done it...
<babbageclunk> voidspace: Oh, I just got pointed at this too: https://bugs.launchpad.net/juju-core/+bug/1605653
<mup> Bug #1605653: backup-restore failed creating collection EOF <backup-restore> <ci> <regression> <xenial> <juju-core:Triaged by 2-xtian> <https://launchpad.net/bugs/1605653>
<babbageclunk> voidspace: do you know whether failures are automatically added to these pages? http://reports.vapour.ws/releases/issue/57922732749a5624aac9f7b8
<voidspace> babbageclunk: anything to do with backup and restore is horrible
<voidspace> babbageclunk: I believe so, if the error matcher finds them
<babbageclunk> voidspace: 'cause if it's automatic then it looks like this failure hasn't happened for 3 weeks.
<voidspace> babbageclunk: right
<voidspace> babbageclunk: maybe it got fixed by accident :-)
<babbageclunk> voidspace: yeah, that's what I'm thinking. Kind of annoying.
<babbageclunk> voidspace: but good I guess!
<voidspace> :-)
<mup> Bug #1614010 opened: juju register: cannot register a user when controller already exists <juju-core:New> <https://launchpad.net/bugs/1614010>
<voidspace> right, cloned trusty vm with correct /e/n/i and just before maas install
<voidspace> frobware: ping
<frankban> wallyworld: hey, with current tip, Login no longer includes ModelManager in the facade list
<wallyworld> hmmm, ok
<wallyworld> frankban: could that be rogpeppe's latest change to separate controller and model logins?
<rogpeppe> frankban: are you logging in to a model?
<frankban> rogpeppe: yes
<rogpeppe> frankban: in that case, ModelManager is not available
<rogpeppe> frankban: ModelManager is now only available to a controller-only API connection
<rogpeppe> frankban: along with the other controller-only facades
<rogpeppe> frankban: i sent an email to the group about this
<frankban> rogpeppe: so, in order to get current model info (like default series, name, provider type) the GUI now needs to use the connection to the controller, right?
<rogpeppe> frankban: that's right
<frankban> rogpeppe: it's a bit weird that you cannot get info on the currently connected model from the model connection itself
<rogpeppe> frankban: because the model info contains things like the model name which can be different between the controller and in the sub-controller
<rogpeppe> frankban: tbh it might only be the name that's problematic
<rogpeppe> frankban: would it be OK if ModelInfo was provided on another (non-controller-specific) facade and didn't provide the model name?
<rogpeppe> frankban: i mean, presumably we already know the name, right?
<frankban> rogpeppe: why should the GUI know the name? anyway, that should be ok
<rogpeppe> frankban: because how else does it know how to connect to the model?
<frankban> rogpeppe: a model is a websocket URL
<rogpeppe> frankban: how do you get that URL?
<frankban> rogpeppe: it is provided by either the charm or juju itself in gijoe
<frankban> rogpeppe: in the dynamically generated GUI config.js
<frankban> rogpeppe: are there any other facades no longer available on the model?
<rogpeppe> frankban: so how do you know where to get config.js from?
<rogpeppe> frankban: the group email mentioned them:  AllModelWatcher Cloud Controller MigrationTarget ModelManager UserManager
<frankban> rogpeppe: ok so the GUI only uses ModelManager in this list
<frankban> rogpeppe: would it be doable to reintroduce that in the models, just to give the GUI time to implement the double connection?
<rogpeppe> frankban: could you just avoid using the latest juju-core for the time being?
<frankban> rogpeppe: I guess those changes will be released friday on the new beta, so I am worried we don;t have enough time
<rogpeppe> frankban: i'd prefer to implement ModelInfo on a different facade instead
<rogpeppe> frankban: at least then we'd be going in the right direction
<frankban> rogpeppe: ModelInfo on another facade solves some problems (even if we need the model name as well). not sure it gives time sufficient time to have a working GUI on next beta (no ListModels, CreateModel etc.). let's discuss this later with Jeff and Uros
<dimitern> menn0, katco, babbageclunk: I'd appreciate a review on http://reviews.vapour.ws/r/5449/, it's quite short
<frankban> rogpeppe: anyway, ModelInfo on a model facade sounds good to me, as it means we don't have to share model data between the two websocket connections
<dimitern> frobware: ^^
<rogpeppe> frankban: yeah
<babbageclunk> dimitern: looking
<dimitern> babbageclunk: ta!
<babbageclunk> dimitern: LGTM!
<dimitern> babbageclunk: thanks!
<dimitern> voidspace: hey, did you manage to get your maas setup going?
<dimitern> voidspace: if so, could I ask you to QA my fix on your maas?
 * frobware is dismayed that all the individal QA efforts for commits don't get captured into tests that can be repeated ad nauseum.
<rogpeppe> frobware: i think the point is that QA efforts are a bit random
<rogpeppe> frobware: you play with the new thing
<rogpeppe> frobware: if all the individual QA efforts were captured into tests, our test suite would run for weeks
<frobware> rogpeppe: but after you do the same thing for a while you have to question why your're doing it again.
<frobware> rogpeppe: that's fine. they don't have to run all the time. you could have one job a week that runs them all.
<rogpeppe> frobware: that's what our CI tests are essentially
<frobware> rogpeppe: it's the endless setup/teardown cost for humans that I think is a problem
<rogpeppe> frobware: if you find yourself repeating the same thing, there's nothing stopping us scripting something
<rogpeppe> frobware: the reason for the QA thing is that everyone was always just relying on the test suite to find problems, and loads of problems slipped through the net because noone thought to actually use the thing in practice
<frobware> rogpeppe: agreed. but in reality it seems more like... "i'll just stop what I'm doing, quickly (ha!) test this thing, then move on"
<frobware> rogpeppe: 90%+ of what I do is manual testing. this seems wrong.
<rogpeppe> frobware: that seems a bit surprising. the test suite takes 30 minutes by itself to run, right?
<frobware> rogpeppe: the test suite doesn't test real networking.
<frobware> rogpeppe: it's too perfectly fictious.
<rogpeppe> frobware: so how would you suggest that the tests *did* test real networking?
<rogpeppe> frobware: QA is not intended to be a substitute for automated tests
<rogpeppe> frobware: it's just a smoke-test thing
<frobware> rogpeppe: we turn /some/ of the QA tests into running those manual steps. Sure, there's a boatload of infra to get to that point but until we address that we'll continue with the manual effort.
<rogpeppe> frobware: if you're spending 90% of your time doing manual QA, there's something wrong
<frobware> rogpeppe: ^^ :)
<rogpeppe> frobware: i can't parse the sentence "we turn /some/ of the QA tests into running those manual steps"
<frobware> rogpeppe: so I initially said "all" whereas that's not necessarily useful or practical. The "some" is me being more pragmatic.
<rogpeppe> frobware: i still don't understand
<rogpeppe> you want to turn some of the QA tests into manual steps?
<rogpeppe> frobware: they already are manual steps, right?
<frobware> rogpeppe: ah, I see.... my fault.
<frobware> rogpeppe: how do we capture those manual steps into an automated test/script/whatever?
<rogpeppe> frobware: right
<rogpeppe> frobware: i think there's a lot to be said for coming up with a set of tools that (say) makes QAing (and possibly automated testing) networking stuff easier
<rogpeppe> frobware: i think this is probably a particular issue for networking tests
<frobware> rogpeppe: yep. I want to automatically provision three nodes, where two nodes have one bonded nic.... and so on.
<rogpeppe> frobware: exactly
<frobware> rogpeppe: this is where my 90% time comes from. by the time I've done that the next thing I move onto is a variation on that, hence the setup/tear down/setup cost.
<rogpeppe> frobware: do the usual thing: imagine how you'd like to specify your usual range of QA scenarios in an ideal world, then think how that might be done
<rogpeppe> frobware: i.e. think devops :)
<frobware> rogpeppe: you mean like this? https://circleci.com/blog/its-the-future/ :-D
<rogpeppe> frobware: precisely :)
<frobware> rogpeppe: made me chuckle
<rogpeppe> anyone want a straightforward review? this allows Ping to work on controller-only API connections: http://reviews.vapour.ws/r/5458/)
<voidspace> dimitern: yes, I have maas 1.9 and 2.0 setup
<voidspace> dimitern: I'm reluctant to screw with my maas 1.9 setup (only one node on it currently) until I've finished this testing though
<voidspace> dimitern: send me the branch and the QA steps
<dimitern> voidspace: it's easy to revert
<voidspace> dimitern: I'm not worried about juju
<dimitern> voidspace: it's all in the PR desc: http://reviews.vapour.ws/r/5449/ and the branch is lp-1612624-ipv6-mongod
<voidspace> dimitern: I'm taking a lunch break and can look after that
<voidspace> dimitern: ok
<dimitern> voidspace: thanks!
<voidspace> dimitern: ok to test on maas 2?
<dimitern> voidspace: shouldn't matter, 1.9 or 2.0
<voidspace> dimitern: ok, cool
<dimitern> strictly speaking, maas is not even needed - there should be a way to test it on lxd, assuming the kernel args can somehow be passed to the container via cloud-init
<mup> Bug #1614065 opened: Unable to attach storage, storage not found <juju-core:New> <https://launchpad.net/bugs/1614065>
<dimitern> another tiny review anyone? http://reviews.vapour.ws/r/5459/
<mgz> is it really better to do
<mgz> var (
<mgz>   a uint64
<mgz>   b error
<mgz> )
<mgz> rather than just have two var lines?
<dimitern> it's a matter of taste I guess
<mgz> err at least could be one scope in
<dimitern> var ( longVarHere evenLongerType\nandAnotherLongVar map[string]interface{}\n )
<mgz> dimitern: my only realy question is if 0 is the right no-backing-inode value
<mgz> I guess it's safely invalid?
<dimitern> mgz: it's the zero value of the underlying type we're parsing the value into
<dimitern> mgz: also this: http://stackoverflow.com/questions/2099121/why-do-inode-numbers-start-from-1-and-not-0 :)
<mgz> dimitern: that's what I wanted to know, thanks
<dimitern> mgz: but good question ;)
<mgz> dimitern: shipit
<dimitern> mgz: awesome! did you try QAing it?
<mgz> dimitern: nope, but I certainly can now
<dimitern> it took me longer to write the QA steps than the fix :D
<dimitern> mgz: if you can, even better!
<mgz> hm, I lost my remote dimitern alias
<dimitern> (I suspect the steps should work, but haven't actually tried them myself - with the exact commands I mean)
<anastasiamac> mgz: have u seen bug 1613864? I'd really appreciate if you could comment there :)
<mup> Bug #1613864: Missing "juju-2"/"juju2" command. <landscape> <juju-core:New> <https://launchpad.net/bugs/1613864>
<mup> Bug #1614065 changed: Unable to attach storage, storage not found <juju-core:New> <https://launchpad.net/bugs/1614065>
<dimitern> anastasiamac: only as a title - will have a look in a bit
<anastasiamac> dimitern: i think it may b packaging related and was hoping mgz would know \o/ I've seen his name on a couple similar ones :D
<mgz> anastasiamac: that bug does describe reality
<anastasiamac> mgz: all bugs do :D
<mgz> anastasiamac: it's what was decided back in the discussions before xenial release - I think my first version had all the aliases?
<mgz> I'd need to look at email threads again to remember the reasoning, but generally we don't like too many names for the same thing
<mup> Bug #1614065 opened: Unable to attach storage, storage not found <juju-core:New> <https://launchpad.net/bugs/1614065>
<mup> Bug #1614072 opened: storage minimum size and default size are conflated <canonical-is> <juju-core:New> <https://launchpad.net/bugs/1614072>
<anastasiamac> mgz: i agree.. plz comment on it (when u get chance) that the behavior & observation is expected. it'll give us grounds to mark as 'won't fix'.. unless we need to fix?...
<mgz> it's not an unreasonable request
<mgz> dimitern: git checkout HEAD doesn't do what you want
<mgz> otherwise the steps are okay
<mgz> dimitern: qa okay, http://paste.ubuntu.com/23064328
<dimitern> mgz: thanks! I'll update the second git checkout step
<rogpeppe> i'm needing a second review of this please - does anyone have a moment to have a look? http://reviews.vapour.ws/r/5458/
<dimitern> anastasiamac: bug 1613864 - seems like a packaging issue, but natefinch might know if to fix it we need to change cmd/juju's main funcs
<anastasiamac> dimitern: thnx. i have also asked mgz :)
<dimitern> rogpeppe: I was about to, but the QA steps were unclear
<dimitern> rogpeppe: how to make a controller-only api connection
<anastasiamac> dimitern: i'll reach out to nate during my day tomorrow -he seems to be doing a few late nights :)
<dimitern> anastasiamac: +1
<rogpeppe> dimitern: well, you could review it and leave the QA to someone else
<rogpeppe> dimitern: you make a websocket connection to the juju API at the path /api
<rogpeppe> dimitern: i'm afraid i can't think of any other way to QA this
<dimitern> rogpeppe: with what? curl? :)
<rogpeppe> dimitern: probably by writing a custom Go program to do it
<rogpeppe> dimitern: i think that frankban can QA it more easily
<dimitern> rogpeppe: see - if you can't describe with a few steps, who could QA it apart from you? :)
<rogpeppe> dimitern: as the GUI makes long-lived connections to the controller-only API
<rogpeppe> dimitern: i did describe it :)
<dimitern> rogpeppe: ah, well - if that works, it should be easy
<rogpeppe> dimitern: i just didn't write the code to do it
<rogpeppe> dimitern: it's a pity that it's hard to open an API connection from Go. lots of boilerplate required. i think i might write a little package to make it easier.
<dimitern> rogpeppe: is a controller long identified by the controller uuid as username/tag?
<rogpeppe> dimitern: "controller long" ?
<dimitern> rogpeppe: sorry, login
<rogpeppe> dimitern: no, it's identified by the websocket URL being /api rather than /model/:modeluuid/api
<dimitern> rogpeppe: ok, I'll leave it to frankban to QA then (ideally with a comment how he did it)
<rogpeppe> dimitern: thanks
<frankban> rogpeppe: I can QA it very quickly with a python script
<rogpeppe> frankban: cool
<frankban> rogpeppe: a comment on how I did it would be just "I wrote a Python script"
<rogpeppe> frankban: i guess you could include the script :)
<frankban> rogpeppe: well, I have a little library that allows me to run arbitrary ws calls, so it's not really a single file script
<voidspace> dimitern: testing your branch now
<voidspace> dimitern: well, starting to
<frankban> rogpeppe: I'll try to make it a standalone
<dimitern> rogpeppe: reviewed
<dimitern> frankban: well, pasting the script will help anyone that has to repro it later ;)
<dimitern> voidspace: great!
<voidspace> dimitern: frobware: with my backport of the bridge script fixes to 1.25, the following is the /e/n/i generated on the bootstrap node
<voidspace> dimitern: frobware: http://pastebin.ubuntu.com/23063945/
<dimitern> voidspace: looking
<voidspace> dimitern: frobware: this is a machine with two bonded NICs and a vlan
<voidspace> dimitern: frobware: it looks good to me
<voidspace> dimitern: frobware: maas 1.9 didn't pick up the vlan though - I had to add it as an interface to the controller, so it appears as "untagged"
<voidspace> dimitern: frobware: which is concerning :-/
<dimitern> voidspace: the backport of https://github.com/juju/juju/pull/5791 ?
<voidspace> dimitern: yes
<voidspace> dimitern: although basically I pulled in *all* the changes
<dimitern> voidspace: 1.25 only bridges the default route interface
<rogpeppe> dimitern: i'm not sure that it's worth defining a type for those constants - they're only used in an error message
<dimitern> voidspace: so that's as I'd expect it to look I think
<voidspace> dimitern: cool
<voidspace> dimitern: once I've done your tests (bootstrapping now) I'll add a  new node and do a deploy as a sanity check
<rogpeppe> dimitern: restrictedRoot could be used for other purposes too - there's no particular need to enumerate the possible strings, i think
<natefinch> morning all
<voidspace> natefinch: o/
<natefinch> dimitern, anastasiamac, mgz: seems like you guys have that juju2 bug under control?
<rogpeppe> dimitern: it's not like we actually check against the values of connType or anything
<dimitern> rogpeppe: would it be too much trouble though? :)
<rogpeppe> dimitern: i think it gives the wrong impression
<dimitern> voidspace: that's ok, but if you managed to bootstrap with ip6.disabled=1 you've QA-ed it already :)
<rogpeppe> dimitern: like those names are important somehow
<rogpeppe> dimitern: but we don't define constants for all the strings we pass to fmt.Println
<voidspace> dimitern: I'm bootstrapping now
<dimitern> natefinch: o/ well kinda - it's worth double checking no code changes will be needed
<voidspace> dimitern: ah, complete
<rogpeppe> dimitern: so i don't really see why it's worth doing here
<dimitern> rogpeppe: well.. ok
<rogpeppe> dimitern: let alone exporting this trivial implementation detail
<voidspace> dimitern: to be fair I want to see it fail with master too - to ensure it's actually making a difference
<dimitern> rogpeppe: they don't have to be exported
<dimitern> rogpeppe: but I don't want to argue :) I'll drop the issue
<rogpeppe> dimitern: would you prefer it if we passed in the whole message to print?
<dimitern> voidspace: ah, right
<rogpeppe> s/print/use in the error/
<voidspace> dimitern: just reverted to master and bootstrapping with the setting still in place
<dimitern> rogpeppe: not really
<dimitern> voidspace: +1
<rogpeppe> dimitern: i think it's worth defining constants when the values actually matter to something
<dimitern> rogpeppe: it my mind having a const you can jump to or autocomplete is worth the time you'll otherwise spend grepping for a string literal, and sifting through lots of possibly unrelated hits
<rogpeppe> dimitern: i don't understand. these values are used in exactly one place.
<dimitern> rogpeppe: ok, forget about it :)
<frankban> rogpeppe: is the Pinger included in controller facades?
<rogpeppe> frankban: that's what http://reviews.vapour.ws/r/5458 is fixing
<dimitern> rogpeppe: in this case, I tend to agree with you - if it was used in more than one place, then it'll be different
<rogpeppe> dimitern: thanks
<frankban> rogpeppe: it's not listed in the facades after login
<rogpeppe> frankban: hmm, it should be
<frankban> rogpeppe: http://pastebin.ubuntu.com/23064485/
<rogpeppe> frankban: that's having bootstrapped with this branch?
<frankban> rogpeppe: yes, I can try again
<rogpeppe> frankban: let me just check the code again
<rogpeppe> frankban: ah!
<rogpeppe> frankban: ok, my fault
<frankban> useful QA for the win
<rogpeppe> frankban: it allows the call but i didn't change the isControllerFacade function
<frankban> yes, that was my impression
<rogpeppe> frankban: i thought i was doing a better thing there :)
<rogpeppe> frankban: +1
<frankban> rogpeppe: seems updated now right
<frankban> ?
<voidspace> dimitern: I can confirm that I can bootstrap with your branch, but not with master, with the disable ipv6 flag in place
<rogpeppe> frankban: no, not yet - i'm just running some tests
<dimitern> voidspace: thanks for confirming!
<frankban> rogpeppe: ok ping me
<voidspace> rick_h_: 1:1 ?
<rick_h_> voidspace: sorry, omw
<rogpeppe> frankban: updated now
<rogpeppe> frankban: please ping me when you've QA'd and I'll start the branch landing
<frankban> rogpeppe: sure on call now
<rogpeppe> frankban: ok
<rogpeppe> dimitern: FWIW i've just pushed up a "convenience API" package to hopefully make it easier to make ad-hoc connections to the juju API: github.com/rogpeppe/misc/jujuconn
<rogpeppe> dimitern: i haven't actually run the code yet though :)
<rogpeppe> dimitern: as i'm on a train so can't bootstrap a juju instance to play with
<dimitern> rogpeppe: awesome! thank you, I'll have a look and try it
<rick_h_> voidspace natefinch  ping for standup
<rick_h_> dimitern: ^
<voidspace> rick_h_: sorry
<frankban> rogpeppe: bootstrapping again
<rogpeppe> frankban: ta
<natefinch> sinzui, mgz: how do I rdp to an azure machine started by CI? There's a "connect" button that gives me RDP info, but my rdp client doesn't ever connect... not sure if I need to open a port on the vm or vpn in somewhere or something
<dimitern> voidspace: please link your PR to the card
<sinzui> natefinch: no idea. we have never done that
<voidspace> dimitern: kk
<mgz> natefinch: I've never done it with azure, only our maas where we can supply our rdp cert to maas
<dimitern> voidspace: in the "Add External Link" section, first field, please
<frankban> rogpeppe, dimitern: QA ok, published instructions at http://reviews.vapour.ws/r/5458/
<voidspace> dimitern: ah, ok
<dimitern> :) it'll become second nature very soon, not trying to be a kanban nazi
<voidspace> dimitern: done
<dimitern> frankban: awesome! thanks for sharing
<rogpeppe> frankban: ta!
<rogpeppe> frankban: have you destroyed the controller already?
<frankban> rogpeppe: no
<rogpeppe> frankban: perhaps you could test this program for me (one mo as i write it :-])
<dimitern> voidspace: ta!
<rogpeppe> frankban: http://paste.ubuntu.com/23064622/
<rogpeppe> frankban: (you'll need to go get the jujuconn package)
<rogpeppe> frankban: it assumes that it's the current controller
<frankban> rogpeppe: http://paste.ubuntu.com/23064626/
<rogpeppe> frankban: awesome!
<rogpeppe> frankban: thanks
<rogpeppe> frankban: always nice when code works first time
<rogpeppe> frankban: i wonder if you've got an old controller going that you can check to see if it fails...
<dimitern> voidspace: I'm QA-ing your fix now on maas 1.9 and a node with a bond and 3 vlans
<babbageclunk> dimitern: stupid question - how do I bootstrap with trace logging?
<dimitern> babbageclunk: I don't think you can easily do that
<babbageclunk> dimitern: :(
<dimitern> babbageclunk: passing --debug to bootstrap gives you debug logging, while setting logging-config='<root>=TRACE' inside the bootstrap config will give you trace logging *later*
<dimitern> babbageclunk: there was a way though.. let me try to remember
<babbageclunk> dimitern: Hmm - actually logging-config='<root>=TRACE' might do it.
<babbageclunk> dimitern: I mean, might do what I want
<dimitern> babbageclunk: try exporting both JUJU_LOGGING_CONFIG and JUJU_STARTUP_LOGGING_CONFIG (to '<root>=TRACE') before running bootstrap
<dimitern> babbageclunk: that's what I found gives you the most verbose initial logging
<dimitern> lazyPower has a nice gist about it actually :) https://gist.github.com/chuckbutler/753ff6b88e2220b7a10a
<babbageclunk> dimitern: my problem was happening late enough that just the logging-config was enough, thanks! I just couldn't find or remember the exact syntax.
<dimitern> voidspace: LGTM + QA OK
<dimitern> babbageclunk: nice ;)
<babbageclunk> dimitern: The API requests from my worker are failing with `unknown object type "MachineUpdater"`. Is there another place I need to register my facade? I've added it to allfacades and have a RegisterStandardFacade call.
<dimitern> babbageclunk: yes, let me have a look where it was
<dimitern> babbageclunk: api/facadeversions.go and apiserver/allfacades.go need updating
<natefinch> two line log output change anyone? http://reviews.vapour.ws/r/5461/
<babbageclunk> dimitern: facadeversions!
<dimitern> natefinch: LGTM, thanks for this - I remember doing something similar some time ago to debug simplestreams
<babbageclunk> dimitern: Thanks
<natefinch> dimitern: yeah... I realized when I saw that if statement, that we were throwing away everything *except* the expected errors.  Would have saved me hours of debugging if I'd seen the json unmarshal error for my simlpestreams data
<natefinch> dimitern: thanks for the review
<dimitern> :)
 * rick_h_ goes for lunchables
<redir> morning
<mup> Bug #1614161 opened: Juju does not remember controller after register <ci> <jujuqa> <register> <regression> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614161>
<perrito666> redir: morning
<redir> is it morning there too perrito666 ?
<redir> I think you're like an hour off from me
<perrito666> its 13:45
<babbageclunk> Is it legit to add a method to an api facade without bumping the version number?
<perrito666> no more
<babbageclunk> perrito666: :( So I guess I need to learn how to bump versions tomorrow!
<redir> or like 4
<natefinch> anyone seen this error? 2016-08-17 15:00:25 ERROR juju.worker.diskmanager lsblk.go:116 error checking if "fd0" is in use: open /dev/fd0: no such device or address
<perrito666> talk about user friendly: error: The requested backend 'zfs' isn't available on your system (missing tools).
<perrito666> natefinch: something is reading your floppy disk
<perrito666> :p
<natefinch> perrito666: ahh, a less well known feature of azure VMs, I guess.
<natefinch> sinzui: do we run the windows CI tests on maas/windows?  or just azure/windows?
<sinzui> natefinch: we do have the tests for maas 1.9. I think they are only running for 1.25 since juju 2 supports azure with window
<natefinch> sinzui: ok, just checking.  I can't for the life of me get RDP to work to an azure VM
<sinzui> :(
<natefinch> to be fair, I can't even ping the IP address
<perrito666> natefinch: I am not sure windows will answer ping
<perrito666> in a default config
<natefinch> windows does answer ping... at least by default. no idea how azure's vms might handle it though
<perrito666> natefinch: windows server?
<natefinch> feh, dunno.  I think they'd be silly to turn it off.
<rick_h_> natefinch: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-connect-logon/ anyone have portal access for the instance?
<rick_h_> natefinch: that states you need to get a generated .rdp file with the ports/etc needed to talk to a specific instance: Clicking Connect creates and downloads a Remote Desktop Protocol file (.rdp file). Click Open to use this file.
<natefinch> rick_h_: yeah, done that. The rdp file is just an IP address... which doesn't work for our VMs.  I'm trying to start a VM manually on azure (using the portal UI) to see if maybe something we're doing with the network is disabling RDP.
<natefinch> rick_h_: yeah, I can connect to RDP from a manually created instance no problem
 * rick_h_ grabs the boy from summer camp, biab
<natefinch> mup: help
<mup> natefinch: Run "help <cmdname>" for details on: bug, contrib, echo, help, infer, issue, login, poke, register, run, sendraw, sms
<natefinch> bug 1614230
<mup> Bug #1614230: can't remote desktop into windows machines created by juju  <windows> <juju-core:New> <https://launchpad.net/bugs/1614230>
<natefinch> ahh good, mup was borken last night, glad it's back
<natefinch> when in doubt... reboot
<natefinch> perrito666: for the record, I can't ping the machine I brought up manually, even though RDP works (obviously ping != rdp, but still, good to know)
<mup> Bug #1614230 opened: can't remote desktop into windows machines created by juju  <windows> <juju-core:New> <https://launchpad.net/bugs/1614230>
<katco> does anyone have an opinion on whether it's reasonable for functions to open new API roots? shouldn't that be something that's passed in?
<mup> Bug #1614239 opened: bootstrap-timeout ignored in --config <landscape> <juju-core:New> <https://launchpad.net/bugs/1614239>
<natefinch> katco: I don't really know what you mean by opening an API root
<katco> natefinch: https://github.com/juju/juju/blob/master/cmd/juju/application/deploy.go#L435
<katco> natefinch: this, but like... at all levels of the call-stack
<katco> natefinch: i.e. deploying bundles creates a new api root, deploying charms, resources and then anything else, etc.
<katco> natefinch: it seems like we should just be opening 1 root at the top of the call graph and passing it through
<natefinch> katco: I have no idea what that function actually does
<katco> natefinch: but i'm not sure if we need to keep reopening it for some reason
<natefinch> katco: probably copy pasta
<mup> Bug #1614239 changed: bootstrap-timeout ignored in --config <landscape> <juju-core:New> <https://launchpad.net/bugs/1614239>
<katco> natefinch: it's basically this: https://github.com/juju/juju/blob/master/api/interface.go#L156
<katco> natefinch: returning that, i mean
<natefinch> it seems like passing in an api.Connection is purely superior....
<katco> natefinch: that's what i thought, but... the juju codebase does weird stuff sometimes and it's not always clear why
<natefinch> katco: my guess is that probably most things that currently call NewAPIRoot, should be taking some subset of api.Connection, since that interface is clearly way too big
<katco> natefinch: yeah, that's exactly where i'm headed
<natefinch> lol // This first block of methods is pretty close to a sane Connection interface.
<katco> natefinch: but first, need to change everything so it's not constructing new connections all over the place
<katco> natefinch: all of this so i can write an actual unit test
<natefinch> heh
<natefinch> I just had a brilliant idea... I can cross compile test binaries from my linux desktop and fling them onto a windows machine to run.
<natefinch> so I don't have to set up a dev environment on a windows machine just to run some tests
<katco> +1
<mup> Bug #1614239 opened: bootstrap-timeout ignored in --config <landscape> <juju-core:New> <https://launchpad.net/bugs/1614239>
<thumper> cmars: ping
<cmars> thumper, pong
<thumper> cmars: I have a branch that modifies the signature of cmd Formatters, and I need to tweak romulus
<thumper> but I notice that juju master is behind romulus master by 32 commits
<thumper> is there anything that would break if we moved to latest?
<cmars> thumper, only one way to find out
<thumper> heh
<thumper> ok
<thumper> I'll just keep sloging forwards
<cmars> thumper, i'd be happy to move juju/romulus/cmd/... commands over to cmd/juju/romulus or something
<cmars> thumper, the coupling across these projects on juju/juju/cmd has been thorny
<cmars> thumper, thoughts?
<thumper> sounds good
<thumper> but please wait until I'm done :)
<cmars> thumper, ok!
<cmars> thumper, in some cases, because of this coupling, it's difficult to land changes into juju/romulus with the bot
<thumper> ah...
<thumper> poos
<cmars> thumper, because romulus lands based on tests against a revision of juju
<thumper> yeah
<thumper> fark
<cmars> thumper, in those cases, ping me and i'll run the tests and just push teh green button
<thumper> cmars: are you ok if I move the commands ?
<cmars> thumper, sure, definitely
<thumper> because I'll be messing with those
<thumper> ok
 * thumper adds to the list
<mup> Bug #1614256 opened: TestClaimExpire fails with "leadership claim denied" <ci> <jujuqa> <leadership> <test-failure> <unit-tests> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614256>
<natefinch> rick_h_: FYI, figured out how to get in via RDP, needed to edit the security group the VM was a part of
<rick_h_> natefinch: gotcha
 * natefinch sups
<mup> Bug #1346597 changed: cannot get replset config: not authorized for query on local.system.replset <cloud-installer> <landscape> <oil> <juju-core:Invalid> <juju-core 1.24:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1346597>
<mup> Bug #1575895 changed: juju loses apt-http/s-proxy information if a model is deleted and a new one created <add-model> <juju-release-support> <landscape> <rc1> <usability> <juju-core:Invalid> <https://launchpad.net/bugs/1575895>
<mup> Bug #1574963 changed: juju2 lxd launch hostname reverse lookup inconsistent <landscape> <cloud-images:New> <juju-core:Triaged> <cloud-init (Ubuntu):Confirmed> <https://launchpad.net/bugs/1574963>
<redir> wallyworld: available for badgering?
<wallyworld> ferreting perhaps?
<redir> general rodentry
<wallyworld> now you're talking
<redir> standup HO?
<wallyworld> standup ho?
<redir> hangout
<redir> in stand-up hangout?
<thumper> cmars: https://github.com/juju/romulus/pull/61
<cmars> thumper, ok, cool. do you have the corresponding juju/juju branch for this yet?
<thumper> working on it
<thumper> I'll land in order
<thumper> ffs
<thumper> I don't want to say this
<thumper> but wallyworld was right
<wallyworld> yay!!!!!!!
<anastasiamac> now u've done it, thumper \o/
<wallyworld> must. not. say. I TOLD YOU SO :-D :-D
<thumper> cmars: updated romulus branch for test depds
<cmars> thumper, thx
<thumper> damn it
 * thumper headdesks
<thumper> cmars: I'm making a dedicated branch for the romulus commands
<cmars> thumper, sounds good
<thumper> cmars: can you give a +1 to the romulus branch?
<thumper> I'll need to refer to the new hash
<cmars> thumper, ah, right. sure thing
<cmars> thumper, go ahead and try $$merge$$
<thumper> ta
<thumper> cmars: https://github.com/juju/juju/pull/6017
<thumper> cmars: it is very boring :)
<cmars> thumper, looks good.. i'd like to do some QA on this branch, shouldn't take too long
<menn0> thumper: happy fun times!!! http://reviews.vapour.ws/r/5465/
<thumper> cmars: ok
<redir> mmm so bootstrapping a controller in aws/us-west-1 then adding a model in us-east-1 should there be anything alive in us-east-1?
<thumper> menn0: shpiit
<thumper> or shipit
<natefinch> anastasiamac: invalidating my bugs, eh?
<thumper> cmars: are you just testing that the commands are still there?
<thumper> cmars: what exactly are you QAing?
<redir> wallyworld: doesn't seem that creating a new model in another region actually creates anything in that region.
<redir> wallyworld: so asking the model for it's current region might not do what we want.
<wallyworld> i haven't tried myself yet
<anastasiamac> natefinch: according to axw, there is a config u need to do on ur environment ;)
<natefinch> anastasiamac: he said "use juju run" which is not really valid, since that means if juju is broken, you can't access the machine to figure out why juju is broken
<axw> natefinch: that was how you can do it now, not what I'm saying is the ideal way
<anastasiamac> natefinch: feel free to triage it differently.. first step is juju-independent tho :)
<axw> it's not invalid tho
<axw> I think we should make it easier/better
<axw> not sure what to do about the password
<natefinch> axw: good point about the password.  not sure.  wasn't there a spec somewhere about keeping secrets in juju?
<axw> natefinch: jam replied to a thread on the list about secrets, I don't know about a spec
<axw> redir: I need to go help with kids, if there's an issue with --region could you let me know how to repro and I'll look into it?
<menn0> wallyworld, thumper, axw: I'm thinking about proposing this: http://paste.ubuntu.com/23065898/
<menn0> wallyworld, thumper, axw: is this ok or would you prefer these were kept as debug/trace
<axw> menn0: perhaps tracef, errorf is dumb tho
<thumper> menn0: is it ever useful?
<thumper> tracef
<thumper> I agree with axw
<menn0> thumper: the only one that could be useful to keep is the on in destroyHostOps
<menn0> IMO
<menn0> i'll just make them trace
#juju-dev 2016-08-18
<mup> Bug #1346597 opened: cannot get replset config: not authorized for query on local.system.replset <cloud-installer> <landscape> <oil> <juju-core:Incomplete> <juju-core 1.24:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1346597>
<mup> Bug #1575895 opened: juju loses apt-http/s-proxy information if a model is deleted and a new one created <add-model> <juju-release-support> <landscape> <rc1> <usability> <juju-core:Incomplete> <https://launchpad.net/bugs/1575895>
<redir> axw mostly just a question, when I `juju add-model --region us-east-1 foo` from a controller boostrapped in `us-west-1`, does anything actually happen in `us-east-1`? or is that all stored on the controller and nothing happens in us-east-1 until I `juju deploy something`?
<redir> axw: I was under the impression from my convo with wallyworld that I should see something come up when I add a model to another region, but it makes sense that it wouldn't, too.
<menn0> axw: rubber stamp pls http://reviews.vapour.ws/r/5467/
<thumper> menn0, axw, wallyworld: thoughts ?  http://reviews.vapour.ws/r/5468/
<wallyworld> output looks nice
<menn0> thumper: +1 the output and args are nice
<menn0> thumper: ship it
<axw> redir: atm, nothing happens. we may change it soon so that the model's security group gets created when you add the model
<redir> axw tx
<redir> gtg. Until tomorrow, #juju-dev
<wallyworld> cmars: yo
<wallyworld> cmars: you and roger really need to talk more, or you need to read his email :-D
<natefinch> axw or anyone else.... trying to run juju on windows, but it seems to die mysteriously right away? http://pastebin.ubuntu.com/23066001/
<axw> natefinch: probably https://bugs.launchpad.net/juju-core/+bug/1612836
<mup> Bug #1612836: Cannot bring up hosted  model machines in azure <azure-provider> <ci> <deploy> <regression> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1612836>
<axw> natefinch: one mo, I have a small patch you can apply to test
<axw> natefinch: http://paste.ubuntu.com/23066004/
<natefinch> axw: will give it a go
<natefinch> axw: thanks
<axw> natefinch: if you have a moment to review, the fix is in a PR now: http://reviews.vapour.ws/r/5469/
<natefinch> axw: sure
<natefinch> axw: doh - axw/juju-tools/main.go:45: cannot use buildToolsCommand literal (type *buildToolsCommand) as type modelcmd.ModelCommand
<natefinch> axw: bah,. it's because gnuflag moved
<axw> natefinch: ah, I'll update it in a moment
<axw> natefinch: pushed
<natefinch> axw: cool
<natefinch> I really love that go install has no middle man between git push and me getting and installing the code.
<natefinch> like, no npm repository to update or anything.  Just get the code from git and build it.
<natefinch> I wish kill controller had a --jfdi that would just kill the machines via the provider... waiting for azure to nicely close everything down is so slow...
<mup> Bug #1614329 opened: Cannot deploy charm to new lxd container on machine: permissions error <lxd> <juju-core:New> <https://launchpad.net/bugs/1614329>
<mup> Bug #1614330 opened: agree command uses 'less' for a pager, fails on windows <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1614330>
<axw> natefinch: I often just "azure group delete -q <resource group> --nowait", then "juju unregister -y <controller>"
<natefinch> axw: nice
<axw> natefinch: groups are quite neat - only one thing to delete, and your model or controller is cleaned out
<axw> the time they take to delete is just a bit painful
<anastasiamac> pretty awesome suffix git put on commit 0806e96e6e79b530161cbb40f54bd8b338e0bad1
<thumper> axw: where will I find the code that stores storage-pool settings?
<thumper> think I've found it
<axw> thumper: sorry went afk, sorted?
<thumper> yeah
<axw> wallyworld: if we start making API calls in list-controllers, then one bad controller is going to hang the whole command. can we isolate the access to show-controller at least?
<wallyworld> that becomes rather inconvenient for a user to see at a glance what they have access to
<wallyworld> but you are right about a bad controller
<wallyworld> maybe it has to be show controller
<thumper> axw: will there ever be storage pools for pools that aren't defined on volumes or filesystems?
<axw> thumper: non capisco. do you mean "will there ever be storage pools that aren't referred to by volumes or filesystems"?
<thumper> kinda
<thumper> will there be pool settings for pools not currently refered to
<axw> thumper: if so, yes. e.g. when you bootstrap ec2, you get a storage pool out of the box called "ebs-ssd"
<thumper> how do I know all the pool names?
<axw> thumper: storage/poolmanager.PoolManager.List
<axw> name is part of the config returned
<thumper> actually, I could just iterate through all settings looking for "pool#" prefix
<thumper> I have all the settings read
<axw> or that
<thumper> perhaps that :)
<mup> Bug #1614345 opened: add-subnet subcommand references absent create-subnet command <usability> <juju-core:New> <https://launchpad.net/bugs/1614345>
<wallyworld> axw: i did it for show controllers, but also added a tabular option to show controllers to make the output nice if needed
<axw> wallyworld: sounds good
<wallyworld> i'll now move on to list-models and list-users
<veebers> wallyworld: re: --build-agent, can it use just a binary in path (i.e. for jujud) or does it need to build from source?
<wallyworld> veebers: it uses a binary in the path if build-agent is not specified
<wallyworld> build-agent means build
<veebers> wallyworld: hmm ok. Is it still possible to pass something in to upgrade-juju to get the build number bumped (as I was trying with --upload-tools). Does that require --build-agent?
<wallyworld> no need, the build number is bumped regardless
<veebers> wallyworld: ok cool, thanks :-)
<wallyworld> if you find an issue, let me know
<veebers> will do :-)
<thumper> menn0: http://reviews.vapour.ws/r/5472/
<menn0> thumper: about to do a kid pickup. will review after.
<thumper> np
<mup> Bug #1614364 opened: manual provider lxc units are behind NAT, fail by default <manual-provider> <uosci> <juju-core:New> <https://launchpad.net/bugs/1614364>
<axw> wallyworld: I have reviewed http://reviews.vapour.ws/r/5471/, would appreciate if you would also take a look later, after anastasiamac has responded to comments
<wallyworld> axw: np, just popping out to get kid from camp, bbiab
<axw> wallyworld: there's lots of changes, but there's a bunch of discrete changes in separate commits on github
<axw> thanks, later
<mup> Bug #1604474 changed: Juju 2.0-beta12  userdata execution fails on Windows <azure-provider> <ci> <juju2.0> <oil> <oil-2.0> <regression> <vpil> <windows> <juju-core:Fix Released by natefinch> <https://launchpad.net/bugs/1604474>
<thumper> axw: hmm... part of the reason of using the database implementation to get at the storage pools is that is exactly what the migration import and export code does, it does low level database stuff, which is why the entire setting collection is read at once
<thumper> all the migration code has deep knowledge of implementation details
<thumper> that is how it works
<thumper> I'm happy enough changing the core description definition
<axw> thumper: I guess that's fine, changing core description would be enough then
<thumper> to make it slightly less hacky
<axw> thumper: though I thought importing didn't involve inserting docs?
<axw> thumper: but using AddMachine, etc.
<thumper> haha, no
<axw> no?
<axw> ok
<thumper> not at all
<thumper> much hacky doc op foo
<thumper> fu
<axw> thumper: yeah that wouldn't work, not sure what I was thinking :)  very well then
<thumper> most of the AddMachine etc high level function try to apply checks we don't want
 * axw nods
<mup> Bug #1604474 opened: Juju 2.0-beta12  userdata execution fails on Windows <azure-provider> <ci> <juju2.0> <oil> <oil-2.0> <regression> <vpil> <windows> <juju-core:Fix Released by natefinch> <https://launchpad.net/bugs/1604474>
<thumper> I'll tweak the description :)
<thumper> hmm.. why has review board stopped updating with some changes?
<thumper> https://github.com/juju/juju/pull/6025
<thumper> phew... romulas command branch finally landed
<thumper> weird, because it found http://reviews.vapour.ws/r/5474/
<mup> Bug #1604474 changed: Juju 2.0-beta12  userdata execution fails on Windows <azure-provider> <ci> <juju2.0> <oil> <oil-2.0> <regression> <vpil> <windows> <juju-core:Fix Released by natefinch> <https://launchpad.net/bugs/1604474>
<menn0> thumper: ship it on the loggo one
<thumper> menn0: I'm just updating the storage pools
<thumper> which loggo one?
<menn0> thumper: the small one which changes the default writer
<thumper> the cmd one?
<thumper> I see it :)
<thumper> I hope it lands due to deps
<thumper> if not, I'll poke veebers :)
<thumper> menn0: https://github.com/juju/juju/pull/6025
<menn0> thumper: RB now working?
<thumper> menn0: didn't pick up that one
<thumper> no idea why
<menn0> thumper: LGTM, with one query
 * menn0 wants this stuff to land so he can see what it looks like
<thumper> menn0: it is different due to loggo.Level vs. string key
<thumper> debug-log sends but string representations of the level
<menn0> thumper: ah right. makes sense.
<menn0> thumper: all good.
<veebers> thumper: Hey I'm about to EOD, did you want me to take a look at something?
<thumper> perhaps
<thumper> gimmie two ticks
<thumper> veebers: nope, all good
<veebers> thumper: sweet :-)
<menn0> thumper: VALIDATION phase done... pushing now
<thumper> sweet
<thumper> heh
<thumper> hmm
<thumper> I have two branches
<thumper> and was getting confused
<thumper> color-debug-log
<thumper> and debug-log-color
<thumper> was editing wrong one
<blahdeblah> Hi folks; anyone know if there has ever been any attempt to deploy juju environments in a fully disconnected environment?  i.e. no streams available, no charm store, no external repos
<thumper> not something I do personally
<thumper> blahdeblah: some of the folk with orange boxes do I think
<blahdeblah> thumper: OK, interesting; any suggestion who to talk to?
<thumper> try scott
 * thumper looks for nick
<thumper> txspud
<thumper> Scott Croft
<thumper> I think he has
<blahdeblah> thanks thumper
<voidspace> frobware: ping
<frobware> voidspace: hi
<voidspace> frobware: hey, hi
<voidspace> frobware: I'm looking at a 1.25 port of the fix  for bug 1602716
<mup> Bug #1602716: MAAS provider bridge script doesn't handle alias interfaces IP <2.0> <maas-provider> <network> <sts> <juju-core:Fix Released by freyes> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1602716>
<voidspace> frobware: even with the new code in place, 1.25 doesn't bridge the alias
<voidspace> frobware: is this because 1.25 does a lot less bridging than 2.0?
<frobware> voidspace: yes
<frobware> voidspace: we only brindge the default route
<voidspace> frobware: so I'll leave the code in place, but mark the specific bug as wontfix for 1.25 (again)
<frobware> voidspace: is there harm it being there? in general I think the bridge script should be identical between 1.25 and 2.0
<voidspace> frobware: I've put the code in there - but there was a card to handle it linked to the LP bug
<voidspace> frobware: so I'm landing the code but it doesn't actually fix the bug... I was just checking this was expected.
<voidspace> frobware: which you've confirmed. So thanks.
<frobware> voidspace: this is not expected. why doesn't it fix the bug?
<voidspace> frobware: on 1.25 it doesn't fix the bug because we don't bridge the alias (which was the bug)
<voidspace> frobware: surely?
<frobware> voidspace: hmm. can you create an alias for the default route (MAAS UI) and validate that we bridge the alias
<frobware> voidspace: you'll need an alias for this to make sense
<voidspace> voidspace: I have an alias for eth0, eth0 is bridged but eth0:1 isn't - as expected
<voidspace> frobware: I have an alias for eth0, eth0 is bridged but eth0:1 isn't - as expected
<voidspace> frobware: if I now disable eth0 should that make the alias the default route, or should I do something else?
<frobware> voidspace: can you PB your original ENI
<voidspace> frobware: http://pastebin.ubuntu.com/23066815/
<voidspace> frobware: that's before and after
<voidspace> frobware: the gateway is specified on the bond, should I create an alias for that?
<frobware> voidspace: looking at the PB
<voidspace> re-bootstrapping with an alias to the bond as well and grabbing coffee
<frobware> voidspace: the script seems broken
<frobware> voidspace: I take that back. apologies.
<frobware> voidspace: you can do this without rebooting - just run the script on the original eni, observing the output depending on what you pass to --interface-to-bridge
<voidspace> frobware: so what should I test for 1.25?
<frobware> voidspace: still looking and understanding...
<voidspace> frobware: ok
<frobware> voidspace: do you have an ENI where bond0 has an alias?
<voidspace> frobware: this is the branch https://github.com/juju/juju/pull/6013
<voidspace> frobware: I just did that - bond0 is still active though
<voidspace> frobware: juju-br0 is created but the bond alias is left in place, no additional bridge is created
<voidspace> frobware: if we only bridge the interface with the gateway (which I assume is the default route) then that is still expected I think
<frobware> voidspace: if you're running the script by hand, you can ignore the gateway bit. the arguments --interface-to-bridge and --bridge-name control what gets bridged
<frobware> voidspace: on 2.0 we don't use those arguments so everything (appropriate) gets bridged
<voidspace> frobware: so it won't fix the specific bug for 1.25 (unless the alias is the default route) but it's nice to have the code the same in both 1.25 / 2.0 for maintenance
<frobware> voidspace: yes, for the latter. but you can have an alias so we should check what it would do
<frobware> voidspace: just a little confused about what its doing for the 2.0 case atm
<voidspace> frobware: ok
<voidspace> frobware: dimitern and I have both tested this branch with deploys and it seems to work fine
<voidspace> frobware: http://reviews.vapour.ws/r/5460/
<frobware> voidspace: deploys that involve an alias on 1.25?
<voidspace> frobware: you can see what dimitern used in his comment on the review
<frobware> voidspace: and an alias on the interface that is the default route?
<voidspace> I'm just looking now
<voidspace> ah no, not an alias
<dimitern> frobware: no aliases
<frobware> voidspace: isn't that a different PR?
<dimitern> is it broken with them?
<frobware> voidspace: that PR is about the MTU
<voidspace> frobware: yeah, but I rolled the code into that
<voidspace> frobware: so no, that's the right PR
<frobware> ???
<frobware> why
<voidspace> frobware: I did one PR with all the new code
<frobware> voidspace: why not separate PRs?
<frobware> voidspace: that makes no sense to me. if we ever had to revert the MTU for example we just added/created more work
<voidspace> frobware: because the patches didn't apply cleanly - so it would have been a lot more work to do as separate patches
<frobware> voidspace: why not separate patches?
<voidspace> frobware: we created "possibly more work" - doing them separately now would be "definitely more work"
<frobware> voidspace: disagree. we should do discrete things.
<voidspace> frobware: because the patches don't apply cleanly as separate patches
<frobware> voidspace: but we can fix the patches per PR
<voidspace> frobware: I understand your disagreement. This patch is really "bring 1.25 up to using the same code for bridging as 2.0" which is a discrete thing.
<frobware> voidspace: fwiw, I think this is the wrong approach. patches should be discrete so that they can be reverted if necessary. seems like we have conflated things.
<voidspace> frobware: I understand and I don't think it is worth the extra effort. If this patch introduces regressions it can be reverted.
<frobware> voidspace: so i think the script is broken for the single use case, where single use means we explicitly pass the interface name to be bridged
 * frobware was trying to look at yakkety... and wonders how much more of this fun and games is to come in netplan
<frobware> voidspace, dimitern: https://bugs.launchpad.net/juju-core/+bug/1614471
<mup> Bug #1614471: MAAS bridge script doesn't bridge aliases correctly where --interface-to-bridge is specified <juju-core:New> <https://launchpad.net/bugs/1614471>
<voidspace> frobware: ok
<voidspace> frobware: what do you think the right fix is for 1.25, or will that need some thinking about?
<frobware> voidspace: just going to try it
<frobware> voidspace: not a fix, just to understand the current behaviour
<voidspace> frobware: I can look at breaking the patch into pieces and talk to Rick about it. I have a critical bug I'm assigned to in the meantime (although that should be easy).
<voidspace> frobware: sure
<frobware> voidspace: take a look at the critical patch - was going to try the alias stuff
<voidspace> frobware: I can't create an alias that is the default route on my maas setup, but it can be done by directly calling the script
<frobware> voidspace: even if that means I'm just doing some additional QA
<voidspace> frobware: ok
<frobware> voidspace: I'm wondering if the alias stuff would have worked if we had landed 395cd8d812c004e8c9a8783c47fcfc22ded9de2e some time earlier.
<frobware> voidspace: or more conveniently https://github.com/juju/juju/pull/5792
<frankban> axw: could you please take a look at http://reviews.vapour.ws/r/5475/ ?
<babbageclunk> My ssd died. :(
<mup> Bug #1614471 opened: MAAS bridge script doesn't bridge aliases correctly where --interface-to-bridge is specified <juju-core:New> <https://launchpad.net/bugs/1614471>
<babbageclunk> dimitern, voidspace, frobware: my ssd died. Setting up on a loaner machine now (my ansible script has mostly worked!).
 * dimitern is back; catching up
<dimitern> babbageclunk: oh bugger :/ any notable losses?
<voidspace> babbageclunk: :-( but glad you're getting set back up
<dimitern> frobware: re bug 1614471
<mup> Bug #1614471: MAAS bridge script doesn't bridge aliases correctly where --interface-to-bridge is specified <juju-core:New> <https://launchpad.net/bugs/1614471>
<dimitern> frobware: I'm not sure we should create juju-br0:1 in that case
<frobware> dimitern: because?
<dimitern> frobware: how will that be useful?
<dimitern> frobware: juju-br0 will work with its primary IP coming from eth0, which will be on the default route as well
<frobware> dimitern: so... this is what I'm trying to understand. why was it broken and would the presence of an auto stanza have fixed the initial bug report
<dimitern> frobware: eth0:1 will also work, if you need for some reason not to go through juju-br0
<dimitern> or perhaps it won't actually..
 * dimitern gives it a quick test
<babbageclunk> dimitern: Lost a couple of days work. :( Really wish I'd pushed yesterday afternoon.
<dimitern> babbageclunk: oh man :( sorry!
<dimitern> yeah, pushing at EOD is a good rule of thumb
<dimitern> frobware: so if we're bridging aliases as well, what happens if the alias is on a different subnet?
<dimitern> frobware: that would imply a separate bridge, not just moving the alias address over to the bridge
<fwereade> so, I am without power where I'm staying, and am rather stuck with cafes if I want internet
<fwereade> I'll be catching up on mail and reviews and stuff while I'm here, but if anyone wants my advice/company/whatever, they should talk to me while I'm here
<dimitern> fwereade: are you in the uk?
<fwereade> dimitern, yeah
<dimitern> fwereade: so brexit reached the power companies? :)
<fwereade> dimitern, haha
<fwereade> dimitern, not yet, that I'm aware of ;p
<dimitern> fwereade: ;) too early
<fwereade> dimitern, trouble in the building, people are working on it
<dimitern> fwereade: is your bandwidth wide enough for a HO, if you have some time as well?
<fwereade> dimitern, HO probably not great, there's background noise
<dimitern> fwereade: I'd like to chat about the ipam story and how it fits in core
<dimitern> fwereade: right, ok - irc then :)
<voidspace> frobware: I think you're right - with auto the interface would have come up and the fact that it wasn't bridged would only have mattered for containers
<voidspace> frobware: so for 1.25 just ensuring the auto is there is enough
<voidspace> frobware: *unless* it's an ipv6 specific thing
<voidspace> frobware: can you effectively have two default routes, one for ipv6 and one for ipv4?
<voidspace> frobware: the alias in the original bug report was ipv6
<dimitern> frobware, voidspace: I've tested with eth0 bridged to juju-br0 and eth0:1 not, deployed 3 lxc containers - works fine
<voidspace> frobware: ah no, in the original bug report the alias does have auto in the generated /e/n/i
<voidspace> dimitern: cool
<frobware> voidspace: yeah, I don't know how ipv6 creeped into that
<frobware> voidspace: I think you raise a good point about containers - should aliases be bridged for containers?
<frobware> dimitern: ^^
<frobware> voidspace, dimitern: can we HO re: aliases?
<voidspace> frobware: ok with me
<voidspace> frobware: in 1.25 we only alias the default route, we don't provide multi-nic support - which probably means no ipv6  support if that's on a separate interface/alias
<dimitern> frobware: sure
<voidspace> frobware: which is probably fine as we're doing it in 2.0
<voidspace> frobware: HO channel?
<frobware> voidspace: standup
<voidspace> kk
<voidspace> omw
<dimitern> me2
<babbageclunk> mgz: ping?
<voidspace> mgz: ping
<babbageclunk> voidspace: dibs
<voidspace> mgz: do you still want a review from me? you were going to create some PRs
<niedbalski> fwereade, ping
<voidspace> babbageclunk: haha, good luck with that...
<fwereade> niedbalski, pong
<fwereade> niedbalski, what can I do for you?
<babbageclunk> voidspace: I'm pretty sure etiquette dictates that he talk to me first.
<mgz> babbageclunk: yo
<mgz> voidspace: alyo
<niedbalski> fwereade, hey William :) , Could you check this https://pastebin.canonical.com/163030/? this is preventing to start the units. I think is related to LP: #1534103
<mup> Bug #1534103: "unknown operation kind run-action" (1.26alpha3) <2.0-count> <actions> <sts> <juju-core:Triaged by rharding> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1534103>
<babbageclunk> mgz: Hey, the mgo fix is in upstream, so we no longer need to apply the nasty patch in our build.
<niedbalski> fwereade, here is the current action list on the database (I've a full mongo dump) http://pastebin.ubuntu.com/23067364/
<babbageclunk> mgz: If I land a change to remove the .diff file, juju/juju/patches is probably going to disappear (since there's nothing else in it).
<fwereade> niedbalski, that just looks like... utterly broken uniter code
<babbageclunk> mgz: Is that going to break your script that applies the patch?
<fwereade> niedbalski, looking further
<babbageclunk> mgz: Or, I could leave a readme file in patches to keep the dir alive in the repo. Would your script cope with that?
<mgz> babbageclunk: a readme sounds good to me
<mgz> babbageclunk: it does not at present but that should change I think
<mgz> voidspace: I shall link you my prs, onesec
<voidspace> mgz: cool, thanks
<babbageclunk> mgz: I guess the alternative is a null patch, but if you're happy to change the script that's probably less obtuse.
<voidspace> babbageclunk: pthbthpthbthpt
<mgz> babbageclunk: so, options, delete the dir, add note to general hacking docs somewhere
<mgz> babbageclunk: or add dummy file for git in patches dir, make sure that lands after I update the script
<babbageclunk> voidspace: don't care, he responded to me first. I'll allow multiplexing.
<voidspace> babbageclunk: I refer the honourable gentleman to my previous comment (pthbthpthbthpt)
<mgz> voidspace: https://code.launchpad.net/~gz/juju-ci-tools/jujupy_ssh_keys
<mgz> voidspace: https://code.launchpad.net/~gz/juju-ci-tools/assess_ssh_keys
<voidspace> mgz: looking
<mgz> voidspace: I'll bug a qa guy as well, so don't worry too much about juju-ci-tools specifics, but feedback on approach/code welcome
<fwereade> niedbalski, and, yeah, that bug looks like completely broken uniter code as well
<babbageclunk> mgz: I'm ok with the latter option - having the note in the dir is probably more useful than in the docs somewhere.
<mgz> babbageclunk: so, we think patcher should only apply things named *.patch in the dir given?
<mgz> or some other rule would be more suitable?
<fwereade> niedbalski, I don't have any immediate insight, except that it's 99.99% certainly a uniter problem and mongo isn't relevant
<voidspace> mgz: ok, one of those branches has conflicts (the assess one)
<babbageclunk> mgz: The existing one is called *.diff, so should probably use that.
<mgz> voidspace: yeah, I did the merge for it's prereq but having propogated that merge yet
<mgz> -'
<babbageclunk> mgz: but otherwise yes.
<mgz> hm...
<babbageclunk> mgz: Although now you say it *.patch is better.
<niedbalski> fwereade, OK, I tried restarting the agent, then it enters in the agent.go:17 [AGENT-STATUS] failed: resolver loop error, state.
<niedbalski> fwereade, can you think on any way to manually workaround this?
<mgz> lets allow both, I will just not be able to glob
<voidspace> mgz: first one is fine, nice and straightforward
<voidspace> mgz: I hate __metaclass__ = type
<voidspace> mgz: :-(
<voidspace> mgz: but I'm sure we've had this discussion before
<voidspace> and future imports generally
<voidspace> magic stuff at the start of a file that changes the semantics of code later in the file
<mgz> voidspace: I am also not a fan, but am following style
<voidspace> mgz: yeah
<fwereade> niedbalski, not without hand-hacking the uniter state file and I am very uncertain about what consequences would follow from that
<voidspace> mgz: hardcoding the key in the script?
<voidspace> mgz: VALID_KEY
<voidspace> mgz: ah, is that a test
<voidspace> fair enough then
<fwereade> niedbalski, my immediate judgement is more "omg that bug is clearly critical, how has it been hanging around for 8 months" :(
<mgz> voidspace: yeah, could use ssh-keygen to make one up on the fly, but just using a known valid thing is fine I think?
<niedbalski> fwereade, agree :)
<voidspace> mgz: if it's just for a test fine
<voidspace> mgz: that all looks fine to me
<mgz> voidspace: thanks!
<voidspace> some interesting patterns in the testing there
<voidspace> looks good
<niedbalski> anastasiamac, fwereade Could be possible to flag LP: #1534103 as critical for the next beta release?
<mup> Bug #1534103: "unknown operation kind run-action" (1.26alpha3) <2.0-count> <actions> <sts> <juju-core:Triaged by rharding> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1534103>
<anastasiamac> niedbalski: fwereade: sure :D William, shall I assign it to u?Happy to tackle it?
<fwereade> anastasiamac, niedbalski: just did so
<fwereade> anastasiamac, mmmmmmmight be better targeted to axw/wallyworld, I am having real trouble with the charm-deletion bits
<fwereade> anastasiamac, and they're likely more immediately familiar with that code
<anastasiamac> fwereade: k.. we'll assign at release call, unless rick_h_ has a better plan
<fwereade> anastasiamac, cheers
<anastasiamac> fwereade: niedbalski: \o/
 * rick_h_ 's ears tingle 
<niedbalski> fwereade, anastasiamac thank you!
<anastasiamac> niedbalski: nps -it's not fixed yet :D
<rick_h_> niedbalski: do we have somehting to replicate with?
<niedbalski> rick_h_, unfortunately I am not able to replicate that bug locally; It's happening on a beta7 installation on which we have access/logs/dumps.
<rick_h_> niedbalski: right, so the question for us will be how to replicate/verify it's fixed
<mgz> babbageclunk: please review https://code.launchpad.net/~gz/juju-release-tools/non_patch_ext/+merge/303263
<rick_h_> it looks like it's happened with a few differnt actions on different workloads, benchmarking, casey's stuff, and openstack in the bug
<fwereade> rick_h_, niedbalski: it looks like it should be reproable by unit test
<niedbalski> rick_h_, yep, we need to work on getting consistent reproducer, so far, no luck for me.
<rick_h_> fwereade: can you brain dump any helpful info for someone to chase down into the bug please?
<babbageclunk> mgz: LGTM! Spelling error - should be PATCH_EXTENSIONS
<fwereade> rick_h_, it'll be about 1 vague sentence but I will try to compose a helpful one
<mup> Bug #1614471 changed: MAAS bridge script doesn't bridge aliases correctly where --interface-to-bridge is specified <juju-core:Invalid> <https://launchpad.net/bugs/1614471>
<mgz> babbageclunk: ehe, whoops
<mgz> did I tyop that twice, or copy or, or can I just not spell? I am not sure.
<natefinch> that feeling when you typo the word typo
<dimitern> frobware: so with the eni produced by the script, it boots ok and here's the routes and addresses: http://paste.ubuntu.com/23067462/
<frobware> dimitern: and presumably an ipv6 route
<frobware> dimitern: curious that it doesn't fail on 'post-up ifup eth0:1'
<dimitern> frobware: yeah - both ipv6 addrs ended up on br-eth1 though
<rick_h_> dimitern: fwereade voidspace macgreagoir ping for standup
<dimitern> omw T-10s
<voidspace> rick_h_: omw
<mup> Bug #1614559 opened: Juju rejects lxd 2.1 <blocker> <bootstrap> <ci> <jujuqa> <lxd> <regression> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614559>
<natefinch> rick_h_:  you want me to fix that security bug?
<rick_h_> natefinch: yes please
<rick_h_> natefinch: hopefully more a packport than anything
<natefinch> rick_h_: it's tricky, because the code is in the juju/utils repo, so updating 1.25 with a new version of that may break other things.  BUt maybe I'll get lucky and it won't be a big deal
<natefinch> fwereade: I really wish juju controllers listed the juju version
<mup> Bug #1614571 opened: Data race: TestLogOutput <ci> <jujuqa> <race-condition> <unit-tests> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614571>
<katco> rick_h_: ta for the response; updated 1614329. lmk what you think
<katco> bug 1614329 that is
<mup> Bug #1614329: Cannot deploy charm to new lxd container on machine: permissions error <lxd> <juju-core:New> <https://launchpad.net/bugs/1614329>
<rick_h_> katco: looking ty
<katco> rick_h_: another update with a simpler/alt repro. sorry if i'm nto being clear
<redir> morning juju-dev
<katco> redir: morning
<redir> :)
 * rick_h_ lunches
<mup> Bug #1614599 opened: i/o timeout downloading resource <ci> <intermittent-failure> <jujuqa> <resources> <juju-core:Incomplete> <https://launchpad.net/bugs/1614599>
<bleepbloop> I have a bundle I exported from the juju gui in the current beta version and it fails to import and complains about `cannot unmarshal !!str `xenial` into charm.legacyBundleData`, is this a known issue?
<perrito666> Pseudo morning
<rick_h_> bleepbloop: not triggering any bells at the moment. can you file a bug over at the gui tracker https://github.com/juju/juju-gui please?
<bleepbloop> Sure will do
<redir> pseudo morning, perrito666
<redir> or maybe sudo morning
<bleepbloop> redir is not in the sudoers file.  This incident will be reported.
<redir> hah
<redir> that's awesome
<redir> rick_h_: can we mark bug 1608533 as invalid or incomplete or something?
<mup> Bug #1614622 opened: Cannot list-resources <ci> <intermittent-failure> <jujuqa> <list-resources> <resources> <juju-core:Incomplete> <https://launchpad.net/bugs/1614622>
<mup> Bug #1608533: Race in github.com/juju/juju/apiserver/tools_test <ci> <race-condition> <regression> <unit-tests> <juju-core:In Progress by reedobrien> <https://launchpad.net/bugs/1608533>
<rick_h_> redir: looking
<redir> tx
<redir> also are you working on bug 1614571?
<mup> Bug #1614571: Data race: TestLogOutput <ci> <jujuqa> <race-condition> <unit-tests> <juju-core:In Progress by rharding> <https://launchpad.net/bugs/1614571>
<rick_h_> redir: rgr, looks like it's not occurred since June so definitely ok putting it to bed unless it shows up again
<redir> rick_h_: ^^^
<rick_h_> redir: I am the target to assign bugs so I can gets folks assigned to work on
<rick_h_> redir: so no, I'm not working on it but need to find someone to work on it.
<redir> OK I'm happy to start trying to repro that in the bg while working... and switch to fixing when it triggers and I am otherwise waiting for a build  or bootstrap
<redir> rick_h_: ^
<rick_h_> redir: rgr ty much. Feel free to take it
<redir> ack
<mup> Bug #1613487 changed: Fails to build tools when no matching tools found. <bootstrap> <ci> <regression> <juju-core:Fix Released by wallyworld> <https://launchpad.net/bugs/1613487>
<natefinch> mgz. perrito666:  isee you both have committed stuff to juju/utils 1.25 branch lately.  What's the policy? Do we cherry-pick from master?
<mgz> natefinch: I landed the same code on both branches
<mgz> basically in parallel
<mgz> but whatever work
<natefinch> mgz: ok
<natefinch> sinzui, mgz: 1.25 still builds with go 1.2, right?
<mgz> natefinch: it needs to, yes
<mgz> it will build with 1.6 but shouldn't require it
<natefinch> mgz: right
<natefinch> mgz: ok
<sinzui> natefinch: mgz: no, we switched the same say we switched master
<mup> Bug #1614633 opened: A unit with a failed storage-detaching hook cannot be destroyed <juju-core:New> <https://launchpad.net/bugs/1614633>
<sinzui> mgz: natefinch there is a bug in Juju's Makefile that installs the wrong jgolang
<mgz> sinzui: all the backported versions are also going to be 1.6?
<sinzui> mgz: natefinch: Ci is testing with Go 1.6 install and the agents and clients are built with go 1.6
<mgz> I guess they can be, though I'm not crazy about changing compiler in sru minor updates
<sinzui> mgz: golang-1.6 is in trusty universe. Of course the people preparing the ubuntu packages need to update controll an rules to match the ~juju packages
<natefinch> wait... so we are building 1.25 with go 1.6?
<mgz> natefinch: we are in CI
<mgz> and curtis is saying we should be for the distro as well (but we aren't atm)
<sinzui> natefinch: I keep saying that on IRC and emails since April 10 2-16
<sinzui> oh look, I pointed out the Makefile deps were wrong on April 1- too
<mup> Bug #1614633 changed: A unit with a failed storage-detaching hook cannot be destroyed <juju-core:New> <https://launchpad.net/bugs/1614633>
<natefinch> I would fix the makefile except I think it's an abomination against nature
<natefinch> well good, I'll stop worrying about why I can't build the go tool for go 1.2 then
 * natefinch goes back to go 1.7
<perrito666> natefinch: lately?
<natefinch> perrito666: top two PRs. More lately than anyone else :)
<perrito666> natefinch: wow, those must be at least 3 months old
<mup> Bug #1614633 opened: A unit with a failed storage-detaching hook cannot be destroyed <juju-core:New> <https://launchpad.net/bugs/1614633>
<mup> Bug #1614635 opened: Bundle or app deploy fails behind a proxy <kanban-cross-team> <landscape> <juju-core:New> <https://launchpad.net/bugs/1614635>
<rick_h_> cmars: filed 8min ago: https://bugs.launchpad.net/juju-core/+bug/1614635
<mup> Bug #1614635: Bundle or app deploy fails behind a proxy <kanban-cross-team> <landscape> <juju-core:New> <https://launchpad.net/bugs/1614635>
<cmars> rick_h_, thanks, i'll keep an eye on it
<natefinch> for anyone with a little time to kill and who has ever played D&D: http://whothefuckismydndcharacter.com/
 * perrito666 uses his +10 in social life to repell that link
<natefinch> katco: ^^
<katco> lol
<natefinch> omg, it's so good
<natefinch> quick backport review anyone? http://reviews.vapour.ws/r/5479/
<natefinch> literally copy and paste from master, all new code, so... anyone want to rubber stamp me?
<mgz> natefinch: lgtm
<natefinch> mgz: thanks
<redir> what version of go are we using with juju on AWS currently?
<natefinch> another backport using the previously mentioned utils backport.  This one was manually created because for some reason cherry-pick was getting mad.... but it's just a mechanical change: http://reviews.vapour.ws/r/5480/
<natefinch> redir: 1.6 for everything
<redir> 1.6.?
<redir> natefinch: ?
<natefinch> redir: 1.6.0 IIRC
<natefinch> redir: why?
<redir> natefinch: but if I bootstrap from my machine it runs the agent built with my version, yes?
<natefinch> redir: if you have a self-built client, it'll use the self-built server, yes.
<redir> natefinch: trouble reproducing races recently, and the only difference I can think of is the version of go that CI is building juju with
<natefinch> races might be load-dependent. I've definitely had ones that only happen if I'm deploying a large bundle etc.
<bleepbloop> Could someone help me with respect to completely nuking the configuration of juju and restarting from scratch? I had juju 2 installed and I uninstalled it but it seems there are still some artifacts hanging around and I want to start completely fresh
<rick_h_> bleepbloop: sudo updatedb && locate juju ?
<natefinch> for juju2, everything is stored in ~/.local/share/juju
<natefinch> if you're switching to juju 1.x, you don't need to worry about it, they can run side by side
<natefinch> (they use different storage paths, different environment variables, everything)
<redir> natefinch: yeah also have process loading the CPU to help drive contention to trigger the race -- hopefully
 * natefinch goes to see who is on call reviewer, sees that it is himself.
<bleepbloop> Thanks Nate and Rick, the leftover files that were messing things up were located in ~/.local/share/juju
 * redir lunches
<mup> Bug #1614689 opened: azure-arm provider is very slow <azure-provider> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1614689>
<voidspace> rick_h_: did you get an ok on that 1.25 JFDI for network backports?
<rick_h_> voidspace: sorry, I never looked for it tbh
<voidspace> rick_h_: ok, no problem
<voidspace> rick_h_: I'm hanging around to see if thumper appears so I thought I'd ask
<rick_h_> voidspace: yea, I can ask folks now
<natefinch> anyone for a rubber stamp on a backport?  http://reviews.vapour.ws/r/5480/
<rick_h_> redir: can you peek at ^ please?
<redir> rick_h_: ack
<rick_h_> redir: <3 ty
<redir> natefinch-afk: LGTM
<redir> rick_h_: I followed the link in that race bug. It led me to http://goo.gl/R8J8zJ
<redir> which is linked to a fixed race, rick_h_. So sinzui is creating a new bug for it and updating.
<redir> and which I've repro'd and will fix before the one assigned I guess.
<rick_h_> redir: I didn't follow that but ... ok?
<mup> Bug #1614724 opened: Juju isn't protected from new versions of LXD <juju-core:New> <juju-release-tools:New> <https://launchpad.net/bugs/1614724>
<thumper> hmm... forgot to open this...
<rick_h_> thumper: voidspace was going to look for you
<rick_h_> thumper: not sure if he found you before this or not then
<thumper> ah... nope
<thumper> epic review up for grabs:  86 files changed, 775 insertions(+), 719 deletions(-)
<thumper> perhaps I'll save it for menno
<thumper> he loves those
<perrito666> Thumper i owe you one of these
<thumper> :)
<thumper> just found one failing test...
<thumper> two failing tests...
 * thumper gets to fixing
<voidspace> thumper: hey, hi
<thumper> voidspace: hey
<voidspace> thumper: so we have another bug related to comparing structs in asserts
<thumper> yep
<voidspace> thumper: however this one is actually comparing a slice of slices of structs, so building it into a collection of asserts is annoying (double loop to build)
<voidspace> thumper: fwereade suggested just comparing txn-revno instead
<voidspace> thumper: I wondered if you had any insight before I went ahead and did that
<thumper> nope
<thumper> :)
<voidspace> hah, ok
<thumper> sounds reasonable
<voidspace> thanks, thought I'd ask
<voidspace> yeah, if I have to I can build the asserts with the double loop - but I'd rather not
<mup> Bug #1614732 opened: Race in github.com/juju/juju/api/state.go <ci> <intermittent-failure> <race-condition> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1614732>
<thumper> sinzui: in the bug ^^^ title says race in state, but text says catacomb
<sinzui> thumper: I typo/paste error. I will fix
<thumper> ta
 * rick_h_ runs away for the day
 * thumper sighs
<thumper> conflicts with master
 * thumper pulls, merges, and resolves
<mup> Bug #1614749 opened: juju ssh fails on azure <azure-provider> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1614749>
<thumper> I guarantee that this branch will be less painful to review than it was to write
<thumper> perrito666, menn0: http://reviews.vapour.ws/r/5481/
<menn0> thumper: 1 min
<thumper> no worries
 * thumper goes back to reviewing menno's branch
<menn0> thumper: must .... stay .... awake... :)
<thumper> heh
<perrito666> thumper: QA steps
<thumper> heh... test everything?
<perrito666> man, you fail the checklist at step 3
<perrito666> shame
 * thumper builds and tests
<thumper> again
<thumper> wallyworld: why would 'juju list-models' hang?
<thumper> surely it shouldn't
<thumper> oh... I have a cunning idea...
<thumper> for later
<thumper> perrito666: can you check `juju models`?
<thumper> wallyworld: fyi http://pastebin.ubuntu.com/23068729/
<menn0> thumper: while I'm still working through it, QA steps for that PR? (especially that the smart formatter still works with the feature flag)
<thumper> lots of the jujuc tests show smart working with the feature falg
<menn0> thumper: ok
<thumper> but I can manually check
<thumper> I should have done that before deploying elastic search
<thumper> ...
<thumper> well they all failed due to bad squid proxy
<thumper> so I'll create controller
<thumper> wallyworld: I think it may have been a more up to date client and older controller
<thumper> will check
<menn0> thumper: done
<perrito666> thumper: what should I check about juju models?
<thumper> perrito666: nm
<thumper> menn0: I'm just doing some manual QA
<thumper> and yes, for application get if you ask for JSON and give a single name, then you don't get json
<thumper> this is the existing behaviour
<thumper> however dumb
<menn0> thumper: worth fixing before 2.0 final?
<thumper> maybe
<thumper> weird
<thumper> apt.cache.FetchFailedException: W:Failed to fetch http://packages.elastic.co/elasticsearch/2.x/debian/dists/stable/main/binary-amd64/Packages  403  Forbidden
<thumper> from the install hook
<thumper> in lxd
<mup> Bug #993557 changed: Charm store should delete charms that have been removed. <store> <pyjuju:Won't Fix> <juju-core:Invalid> <https://launchpad.net/bugs/993557>
<mup> Bug #1178497 changed: Provide an API for listing all charms that are present in the store <charmbrowser> <store> <juju-core:Fix Released> <https://launchpad.net/bugs/1178497>
<wallyworld> thumper: sorry, was out buying dog food
<thumper> that's fine
<wallyworld> i also got a 403 when using squid deb proxy
<thumper> I've restarted my lxd with no proxy
<mup> Bug #1602572 changed: Handler function is not being called even after changed one of the config values of the config.yaml <juju-core:Invalid by johnsca> <https://launchpad.net/bugs/1602572>
<perrito666> oh man wallyworld now you have me humming ready for the 80s
<wallyworld> lol
<redir> I was getting a lot of 403s with squid deb proxy until I allowed world.
<thumper> wallyworld: w00t, looks like the big command branch will land, all tests look like they passed
<wallyworld> yay
<wallyworld> that is most excellent
<katco> thumper: i'm afraid to ask, but: does this affect the deploy command?
#juju-dev 2016-08-19
<menn0> thumper: easy one: http://reviews.vapour.ws/r/5482/
<menn0> axw: when you're around can we talk about bug 1569632 please?
<mup> Bug #1569632: status: decide on statuses for migration <juju-core:Triaged by menno.smits> <https://launchpad.net/bugs/1569632>
<perrito666> thumper: well that is awkwards, menn0 gave you a nice ship it and I just added a bunch of issues
<axw> menn0: I'm around now
<redir> wallyworld: I think I'm ready for a consult
<redir> if you're available
<wallyworld> ok
<wallyworld> standup ho?
<perrito666> wallyworld: that is an awful thing to call redir
<wallyworld> perrito666: you funny
<wallyworld> btw
<wallyworld> i am having acl issues
<wallyworld> there's some code left in the admin_root which checks for read only calls
<wallyworld> and the list of read only calls in incorrect
<wallyworld> and we don't even need that check anymore
<perrito666> wallyworld: well we can nuke that
<wallyworld> yep, doing it
<perrito666> I decided to nuke it post-implement acls
<perrito666> just as a failsafe
<wallyworld> sure, i need to remove it for my branch to work
<wallyworld> users can't list models
<wallyworld> admins can, but not others
<redir> brt
<perrito666> wallyworld:mm, after reading the list models method I assumed it would show to each user their own models
<perrito666> therefore it was safe to make it RO
<perrito666> since RO users will just not get results
<wallyworld> it is, but that hard coded list omitted the ListModels call
<perrito666> ahh I see
<wallyworld> RO users need to be able to see their own models
<wallyworld> anyways, all fixed now
<wallyworld> i just wanted to check that you agreed we should remove that code
<thumper> katco: no
<thumper> perrito666: hmm... we often don't check the error returns of fmt.Fprintf as well
<thumper> perrito666: given the number of places we call flush without checking, I think this is ok
<perrito666> thumper: I assumed so, It tickled me because we where checking it in just one place and you removed it, but I believe its ok too
<perrito666> thumper: the other ones are valid though :p
<thumper> you expect me to file bugs for dfc's old comment?
<thumper> that's a bit rough
<thumper> perrito666: and changing other fmt.errorf calls...
<perrito666> thumper: you are the one that said that we should .... meh forgot the words but something about leaving stuff neater than we found, also you passed by that code and dfc was your teammate :p at least re-own it
<thumper> :P
<perrito666> anyway, eod, cheers all
<wallyworld> thumper: running a minute late
<katco> thumper: yay
<wallyworld> thumper: with colour, axw had a good point - you need to keep the * still, for the current controller etc even when it is a terminal and colour is used
<thumper> wallyworld: let me show you something
<axw> (for colour blind people; unless the contrast between green and white is enough?)
<thumper> ah...
<thumper> good point
<thumper> it was so good though :)
<wallyworld> having an extra * is not big deal, it's still green
<thumper> wallyworld: intermittent failure :(
<wallyworld> faaaaaaaarrrrkkkk
 * thumper needs foot
 * thumper needs food
 * thumper has two feet 
<axw> wallyworld: RB bot is asleep, can you please review https://github.com/juju/juju/pull/6034
<wallyworld> sure
<axw> wallyworld: I decided not to make the change to the machiner worker after all, if you force destroy a machine you shouldn't expect it to uninstall cleanly
<axw> wallyworld: and there's another scenario where it could happen, with a note that we explicitly don't handle it because state could just be borked
<wallyworld> that's the change to react to not found?
<axw> wallyworld: and uninstalling because of a bug would be bad
<axw> wallyworld: yes. so I just changed the cleanup to not force destroy on destroy-model
<wallyworld> ok
<wallyworld> axw: lgtm, thanks
<axw> ta
<thumper> wallyworld: landed!!!!
<thumper> \o/
<wallyworld> awesome
<katco> thumper: grats
<wallyworld> will propose my branch now
<wallyworld> thumper: http://reviews.vapour.ws/r/5483/
<thumper> wallyworld: why have you removed read only calls?
<wallyworld> 1. out of date, 2. did you read horatio's email?
<thumper> no...
<thumper> also
<wallyworld> no longer needed since permission checks are on each moethod now
<thumper> what would the mechanism be to block all modifying calls for things like migration?
<wallyworld> the methods do permission checks
<wallyworld> sigh, so many issues this week with people not reading their emails
<thumper> well, been a bit busy
<wallyworld> all facade methods are expected to have a permission check now
<wallyworld> as a code review checklist step, that must be enforced
<wallyworld> sometimes the check is done for the entire facade at construction time
<wallyworld> if all methods require write for example
<wallyworld> that hard coded list was also out of date - it omitted legitimate read only methods
<wallyworld> so people were not maintaining it
<anastasiamac> wallyworld: great idea ^^ could u plz add it to our review checklist? https://github.com/juju/juju/wiki/Code-Review-Checklists
<wallyworld> i thought it had been, i'll need to double check it
<anastasiamac> it's there :D "Do facades and methods have the required authorization checks? "
<veebers> thumper: I'm late to the game but the colour output is pretty damn sweet
<thumper> :)
<thumper> ta
<mup> Bug #1614809 opened: api-caller remains post migration <juju-core:New for menno.smits> <https://launchpad.net/bugs/1614809>
<wallyworld> thumper: thanks for review, i was also wondering about MODEL UUID
<wallyworld> i started with CONTROLLER but thought it looked a bit wrong tbh
<wallyworld> i can go with that and we'll get feedback i'm sure
<wallyworld> i went with Controller because it's not a table header
<anastasiamac> wallyworld: axw: menn0:thumper: can u get to RB?
<natefinch> is reviews.vapour.ws broken for anyone else?
<axw> anastasiamac: nope
<axw> AccessDenied
<wallyworld> just broke right now
<wallyworld> ffs
<anastasiamac> yep. the same :(
<menn0> broken for me
<katco> this test has defeated me. /EOD
<wallyworld> katco: wow, you're on late
<anastasiamac> axw: wallyworld: I've cleaned up PR, PTAL https://github.com/juju/juju/pull/6023
<wallyworld> real soon
<thumper> axw: my cunning plan to use the poolmanager has been poleaxed by an import cycle
<thumper> stateenvirons
<axw> thumper: what about stateenvirons?
<thumper> axw: I was trying to use it in migration_import and export
<thumper> which are in state
<axw> thumper: yeah, you can't/shouldn't use stateenvirons from state. you also shouldn't need it
<thumper> to get a NewStorageProviderRegistry
<axw> un moment
<axw> thumper: func (st *State) storageProviderRegistry() (storage.ProviderRegistry, error)
<thumper> ah ha
<thumper> axw: think I figured out a way to not expose the pool global key malarky
<axw> thumper: move the poolmanager code into state? :)
<axw> thumper: how?
<thumper> you'll see
 * thumper does more fixes
<axw> okey dokey
 * thumper crosses fingers while tests run
<thumper> axw: I think this last attempt will may you much happier
<thumper> w00t, tests pass
<thumper> now a bit of cleanup
<thumper> axw: https://github.com/juju/juju/pull/6024
<axw> thumper: yes, much nicer :)
<thumper> wallyworld, axw, menn0: https://github.com/juju/juju/pull/6036
<thumper> not sure about the green for current actually
<thumper> I was thinking something that "pops" a bit more
<menn0> thumper: light blue?
 * wallyworld is looking at another pr
<thumper> menn0: not sure
<menn0> thumper: LGTM to me anyway
<menn0> gah...
 * menn0 is tired
<thumper> perhaps we land and bikeshed over color?
<thumper> or make the color definable with an env var?
<thumper> heh
 * thumper out
<natefinch> so we're bikeshedding on what to bikeshed on now?
<natefinch> every time I look at gojsonschema it looks worse
<axw> wallyworld: pretty sure the ssh issue is another latent bug, woohoo
<wallyworld> yay
<axw> wallyworld: the instance poller backs off when it sees a machine that isn't provisioned, doesn't start querying again immediately when it is provisioned
<axw> wallyworld: so it takes a while for it to kick back in, since the machines are taking around a minute to come up
<wallyworld> i can beliebe that - the poller is independent of the provisioner
<wallyworld> if only we had pubsub
<axw> wallyworld: nah it's just crappy implementation, it *can* react to the machine change
<axw> and I will fix it to do that now
<wallyworld> ok
<menn0> thumper-afk: I probably shouldn't tell you this but "less -R" will let you use less with colors ... could be interesting to integrate into debug-log
<wallyworld> anastasiamac: latest pr is much smaller change set from the first one which is good; for some reason i find it easier to review on rb compared to github
<mup> Bug #1614835 opened: Unittest guiVersionSuite.TearDownSuite fails <ci> <unittest> <juju-core:Triaged> <https://launchpad.net/bugs/1614835>
<anastasiamac> wallyworld: \o/
<anastasiamac> wallyworld: also hosted arch is added so that we can validate upload tools..
<wallyworld> anastasiamac: but it's not an arch supported by the substrate
<wallyworld> if you bootstrap from amd64 and try and run an arm64 controller that should break
<wallyworld> when uploading tools, ther question to ask is - does the cloud and image i will be using to bootstrap on match the arch of my bootstrap client
<wallyworld> adding host arch to the list of supported arches subverts that
<anastasiamac> wallyworld: let me clean up this pr and if this is the question to ask then validateuploadallowed is not asking it
<anastasiamac> wallyworld: i really need to land it, u can fix as an iteration. this fix fixes my bug
<wallyworld> but introduces a new one
<anastasiamac> wallyworld: i dont see how it introduces new one - currnet behavior
<wallyworld> it's not current behavour, that's the issue
<wallyworld> if you try and upload tools from an amd64 client to a ppc64 cloud it will fail
<wallyworld> or more so, from a s390 client to aws for example
<anastasiamac> if there is a test, then i will not b able to land my change. at this stage = all tests pass
<axw> wallyworld: PTAL, https://github.com/juju/juju/pull/6037
<axw> wallyworld: BTW I was confused, we don't get updates on the machine doc in general; only lifecycle changes
<axw> so I've left it as polling for now, but changed it to only back off after it's been provisioned
<wallyworld> axw: yeah, that's why i was wishing for pubsub - could easily notify of such changes, looking now
<wallyworld> axw: i think i'm reading the test wrong - it seems that prior to m.setInstanceId("inst-ance") we *want* case <-polled and thereafter not, but it seems the test is the other way around?
<axw> wallyworld: before setInstanceId, the test fails if <-polled. after, it fails if it times out waiting for <-polled
<wallyworld> yeah, that's what's confusing me
<wallyworld> before set instace id, machine is not provisioned so we want regular polling
<wallyworld> so we'd expect to see,_polled, no?
<axw> wallyworld: perhaps the var name is throwing you off. <-polled means that it queried the provider for instance info
<wallyworld> i was thinking polled was for querying instance addresses
<wallyworld> which we want to do regularly until provisoned
<axw> wallyworld: and that hasn't changed, the only change is that there's no backoff. that's proven by the clock advancing exactly the "short poll" amount of time
<axw> wallyworld: nope, how would you do that without an instance ID?
<wallyworld> good point
<wallyworld> i haven't context switched in the code properly
<wallyworld> lgtm though, i was was confused
<mattyw> hey folks, I'm getting an XML AccessDenied from reviewboard, anyone else?
<wallyworld> mattyw: yeah, it's borked
<mattyw> wallyworld, awesome, hello by the way!
<wallyworld> hey!
<voidspace> fwereade_: ping
<fwereade_> voidspace, pong
<voidspace> fwereade_: txn-revno is maintained by the txn layer, so all I need to use it is to declare it in the doc. Right? Like this: http://pastebin.ubuntu.com/23069609/
<fwereade_> voidspace, so long as you *only* read that doc and don't write it, yes
<fwereade_> voidspace, iirc accidentally writing a txn-revno can mess everything up
<voidspace> fwereade_: sounds like fun
<voidspace> fwereade_: http://pastebin.ubuntu.com/23069620/
<voidspace> fwereade_: although I think that code should return no-ops if the hostports are equal (it doesn't currently do that either)
<fwereade_> voidspace, +1 to that
<voidspace> kk
<fwereade_> voidspace, and if you just declare that doc inside that func then you shoudl be safe from anyone accidentally writing it in future
<voidspace> yep
<voidspace> fwereade_: I don't think this particular contention problem is possible to test directly - other fixes we've committed didn't have new tests
<voidspace> fwereade_: so if the existing tests pass (checking that they're good tests of course) I'll call it good
<voidspace> currently they pass - just trying again with the NoOp path
<fwereade_> voidspace, hm, surely you can test SetAPIHostPorts racing with itself at least?
<voidspace> fwereade_: I'm hoping there is already a race test :-)
<voidspace> maybe optimistic
<fwereade_> voidspace, fwiw I can think of two: (1) BeforeHook change to a new value, check 2 txn attempts; (2) BeforeHook change to desired value, check 1 attempt only
<voidspace> fwereade_: understood
<voidspace> fwereade_: there's at least one concurrent test already
<fwereade_> voidspace, cool
<voidspace> two - ConcurrentSame and ConcurrentDifferent
<fwereade_> huh, access denied from reviews.vapour.ws
<voidspace> fwereade_: reading them
<fwereade_> anyone else
<fwereade_> voidspace, awesome
<fwereade_> voidspace, the test names seem good, anyway ;)
<voidspace> fwereade_: yep, me too
<fwereade_> mgz, I think of you as the person who effortlessly fixes everything, is that something you can address? ^^
<voidspace> All current tests pass, that's a good sign
<hoenir> how to know if juju is using the default tools or the tools that I specified ?
<hoenir> this means it's using the deafult?
<hoenir> https://paste.ubuntu.com/23069672/
<hoenir> cmd is juju bootstrap maas maas --debug --show-log --config agent-metadata-url='192.168.122.157/toolsdir/tools/' --bootstrap-series='trusty'
<hoenir> anyone?
<perrito666> hoenir: looks a lot like its using the default
<hoenir> perrito666, how can I use my own tools?
<mup> Bug #1614929 opened: Panic when destroying controller <juju-core:New> <https://launchpad.net/bugs/1614929>
<fwereade_> voidspace, if you have a moment, https://github.com/juju/juju/pull/6040
<fwereade_> voidspace, the critical thing is really to QA it, and come to a conclusion re the JustRemoveOp(..., -1); everything else has been reviewed in detail by many people
<voidspace> fwereade_: about to go on lunch - can look after that
<fwereade_> voidspace, cheers
<mup> Bug #1614929 changed: Panic when destroying controller <juju-core:Invalid> <https://launchpad.net/bugs/1614929>
<mup> Bug #1614948 opened: cmd/juju: "error" capitalisation is inconsistent <usability> <juju-core:Triaged> <https://launchpad.net/bugs/1614948>
<jam> natefinch-afk: ping if you're around
<natefinch> jam: pong
<perrito666> tha lag
<perrito666> mgz: could you kick reviewboard server?
<mup> Bug #1614961 opened: juju ssh to azure hosts fails <azure-provider> <ci> <jujuqa> <regression> <ssh> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1614961>
<mgz> perrito666: I don't think I can
<perrito666> mgz: I think you do :p
<perrito666> that is one of CI machines, I am sure of it
<mgz> well, I guess I do have ssh access, just not the management creds
<perrito666> or you can provide me access+sudo and Ill gladly dig into the pile of... code
<mgz> so i can poke around at least
<mgz> perrito666: you have it
<perrito666> mgz: oh
<mgz> cloud-city and ssh with the staging-juju-rsa key
<mgz> hm, proxy error spam
<mgz> why is it even trying to access /releases/unknown-outcome
<mup> Bug #1614961 changed: juju ssh to azure hosts fails <azure-provider> <ci> <jujuqa> <regression> <ssh> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1614961>
<voidspace> hmmm... reviewboard is borked again
<mgz> yeap, not sure what the fix is
<perrito666> voidspace: looking into it
<voidspace> perrito666: thanks
<rick_h_> natefinch voidspace ping for standup
<voidspace> rick_h_: omw
<sinzui> perrito666: voidspace I am going to reboot review board. the services are healthy. I think the host or the proxy have thrown a wobbly
<voidspace> sinzui: much appreciated
<babbageclunk> voidspace, frobware: Could you take a look at this please? (github link since there's no RB link) https://github.com/juju/juju/pull/6042
<voidspace> babbageclunk: I think my review plate is full until EOD I'm afraid
<babbageclunk> voidspace: :(
<voidspace> babbageclunk: I have two to do and I want to leave and go to mountains, sorry
<babbageclunk> voidspace: Well, at least it's not just on my hard drvie now!
<voidspace> babbageclunk: haha, yeah - ice
<voidspace> I mean *nie
<voidspace> *nice dammit
<voidspace> fat fingers
<babbageclunk> voidspace: That's wjat I meant too.
<voidspace> babbageclunk: new SSD or repaired computer yet?
<voidspace> ERROR invalid config: lxd version 2.1, juju needs at least 2.0.0
<voidspace> anyone seen that - is it fixed on master?
<babbageclunk> voidspace: using a loaner that Rodney gave me - it's a beast! 8 cores and 32Gb
<voidspace> I guess I can check master...
<voidspace> babbageclunk: wow
<babbageclunk> voidspace: test runs are nice and quick. Not much fun biking home with it though.
<voidspace> babbageclunk: yow
<voidspace> yeah, I bet
<katco> with juju/errors, how can i retrieve the underlying error if it has been Wrap(..)ed
 * katco crosses fingers and does a full test run of cmd/juju/application
 * rick_h_ looks for a chicken to sacrifice to the test run gods, but finds none
<natefinch> katco: if you want the original error, you can get Cause(err) ... if there's N wraps, that gives you the first one, if you want the N-1th one, I'm not sure you can
<katco> natefinch: hm, that didn't seem to work
<katco> natefinch: i'll sanity check and try again after this run is done. didn't get much sleep
<natefinch> katco: oh, wrap, yeah...
<natefinch> wrap changes the cause to the new error
<natefinch> you want Underlying()
<katco> hm, i thought i tried that too
<katco> i think it just gave me back the wrapped error or something. i probably screwed it up
<natefinch> it might that if it's wrapped and then gets annotated, you lose access to the underlying error
<katco> ahhh that is almost definitely what's going on
<natefinch> in theory you're not supposed to need to know what the underlying error is
<katco> so this was inflicted by my own hand. a very techy error was making it out to the cli, so i wrapped it. mask and annotate still leak the techy error
<rick_h_> katco: can you also look at natefinch's PR please. In the bug is a direct link to the .sh script to QA with and get the security report which should work against a lxd deploy I htink.
<katco> natefinch: actually, no, the full history is still in the error because i can gc.Matches on errors.Details(err)
<rick_h_> natefinch: I'm assuming you've done this as well to make sure it works
<katco> rick_h_: ok
<katco> not bad: OOPS: 204 passed, 1 PANICKED, 1 FIXTURE-PANICKED, 1 MISSED
<katco> that's down from hundred failing/panicing
<rick_h_> katco: <3
<natefinch> rick_h_: I haven't, actually. I should do that.
<natefinch> I'm sure it'll pass, since I hand-wrote the list of ciphers we support, but it would be good to verify with the tool they're using
<rick_h_> natefinch: rgr, it's what the bug is filed with so if we don't pass the tool, the bug reportor will come back at us.
<mup> Bug #1614724 changed: Juju isn't protected from new versions of LXD <juju-release-tools:New> <juju-core (Ubuntu):New> <https://launchpad.net/bugs/1614724>
<mup> Bug #1614992 opened: Cannot assign unit: unable to connect <add-unit> <ci> <intermittent-failure> <jujuqa> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614992>
<natefinch> lol, I've forgotten how to bootstrap 1.25
<katco> not sure i've done that myself in awhile
<natefinch> you have to switch environments first and then juju bootstrap without specifying the environment
<natefinch> but now it's saying no registered provider for lxd, which is weird
<katco> lxd is a 2.0 thing haha
<katco> believe it or not. it's been that long
<natefinch> lol
<natefinch> wow
<katco> have to use local i think? or lxc
<natefinch> lcoal yeah
<natefinch> lulz
<natefinch> and gce panics during bootstrap with a nil pointer
<natefinch> oy
<katco> wtf? on latest in 1.2x?
<natefinch> yep.  I'm guessing it's this line:
<natefinch> 		// The missing credentials will be caught later.
<natefinch> 		return nil, nil
<natefinch> oh, they were caught all right
<katco> sigh... please raise a critical bug. gce flat out not working is a huge deal i imagine. rick_h_ ^^^
<natefinch> it's an edge-ish case, I think.  my config specifies a credentials file that apparently isn't there
<natefinch> lemme double check if that's the problem
<katco> ah
<katco> that would make me feel loads better
<natefinch> no, that's not it.  Sorry, let me figure out exactly what's causing it
<katco> i hope it's an edge case
<natefinch> yeah, my config is all messed up. wrong property names and stuff. Possibly from very very early in gce lifetime
<natefinch> Let me attempt to fix it and double check gce works, but I'm pretty sure that's it
<mup> Bug #1615008 opened: jujud uses max cpu cores 100% overnight <juju-core:New> <https://launchpad.net/bugs/1615008>
<natefinch> ahhh..... this is not a helpful error message: ERROR there was an issue examining the environment: invalid config: key "project_id" not supported
<natefinch> what environment? what config? where?
<katco> natefinch: like ~/.juju/environments.yaml under "local"
<katco> likely
<natefinch> yeabut.. that key doesn't exist there
<natefinch> and I don't have JUJU_HOME set
 * rick_h_ runs to long family lunch today biab
<natefinch> or any JUJU_* variable, actually
<katco> grep -r project_id ${HOME}
<natefinch> yeah, just started running that.  gonna be a while
<natefinch> funny seeing it pick up javascript in my browser cache
<natefinch> Skills: 5+ years of Agile, Scrum
<natefinch> oh recruiters
<natefinch> MUST have long projects and long time in US
<natefinch> ...what does that even mean?
<perrito666> natefinch: wrong channel?
<natefinch> nope
<natefinch> recruiter sent me an email for a .Net job.
<perrito666> probably means they wont do the h1 for you
<natefinch> We are looking to fill a contract opportunity with one of our direct customers and like to check if you have any resources available for the same. Please submit the resume with rate, contact details, and availability.  Kindly note that each resource submitted must be on your W2 payroll.
<katco> i once had a recruiter attempt to recruit me to my old company
<mgz> what's a w2 payroll?
<mgz> "must have long projects, must not have long hair"
<natefinch> W2 is the form a business uses to report your income to the government (and to you)
<natefinch> it sounds like this was half rewritten from something sent to a contracting company
<natefinch> Job Title : Superstar .Net Technical Lead/Architect (strong SQL) (Chi)-multi-year contract (long projects)
<mgz> yeah, they seem to think you are an owner of resources
<perrito666> I thought owning such resources was sort of illegal
<perrito666> for at least a couple of hundred years?
<natefinch> haha
<natefinch> you don't own them, you just rent them out
<redir> morning
<katco> hey redir
<natefinch> katco: I figured out the gce problem... I'm using a file to hold my gce credentials, and evidently that file format does not work for 1.25.  I'll test in a minute if it works for 2.0
<katco> ah cool
<perrito666> btw, reviewboard is back, you can thank sinzui for that
<natefinch> 2016-08-19 15:57:27 ERROR juju.cmd supercommand.go:429 cannot initiate replica set: cannot dial mongo to initiate replicaset: no reachable servers
<natefinch> ERROR failed to bootstrap environment: subprocess encountered error code 1
<natefinch> ffffffffffffff....
<natefinch> me testing this bug fix: https://www.youtube.com/watch?v=d1CYncXkCv4
 * natefinch abandons gce for now and bootstraps amazon
<natefinch> uh hmm
<natefinch> is 1.25  supposed to work on xenial?
<redir> natefinch: does that mean you'll be breaking bad soon?
<natefinch> redir: gotta break sumethin'
<redir> natefinch: lemme know when it happens, I'll make popcorn:)
<mup> Bug #1615051 opened: Dubious hook failures deploying trivial charm <ci> <deploy> <hooks> <jujuqa> <regression> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1615051>
<rick_h_> natefinch: 1.25 is packed ajd in universe for xenial...so yes? 1.25 and local...not so much
<cmars> fix for LP:#1614330, http://reviews.vapour.ws/r/5484/
<mup> Bug #1614330: agree command uses 'less' for a pager, fails on windows <windows> <juju-core:In Progress by cmars> <https://launchpad.net/bugs/1614330>
<rick_h_> perrito666: can you review ^ please?
<perrito666> rick_h_: certainly
<rick_h_> perrito666: ty much
<natefinch> hm... I seem to remember there being some extra tweak I had to do to TLS for mongo because mongo was being a turd
<katco> really quick review for someone who has a few seconds: http://reviews.vapour.ws/r/5485/
<natefinch> yes yes, I have it
<perrito666> cmars: did you see thumper's patch that now takes stuff and writes them into a writer?
<cmars> perrito666, no
<perrito666> cmars: landed last night iirc
<perrito666> cmars: you tried this with latest masteR?
<cmars> perrito666, implemented & tested this on top of 3785445 which is pretty recent
<perrito666> cmars: please try with current master and if so ill ship it
<cmars> perrito666, it's up to date with current master, a fetch & a rebase show it's still up to date w/master. i tested this maybe 30 min ago
<perrito666> k, ship it
<cmars> perrito666, thanks
<perrito666> np
<natefinch> katco: I updated http://reviews.vapour.ws/r/5480/ with steps to test using the script from the bug (after verifying it myself).
<katco> natefinch: ta. i will finally be getting around to this in just a bit
<natefinch> good timing then :)
<natefinch> katco: welp... - https://bugs.launchpad.net/juju-core/+bug/1477712
<mup> Bug #1477712: GCE provider dumps stacktrace when missing a config option/value <config> <gce-provider> <panic> <juju-core:Triaged> <https://launchpad.net/bugs/1477712>
<natefinch>  reported by Charles Butler on 2015-07-23 importance: medium
<katco> wow this was reported over a year ago
<natefinch> also this one: https://bugs.launchpad.net/juju-core/+bug/1533790
<mup> Bug #1533790: GCE provider is very restrictive when using a config-file (json) <docteam> <juju-core:Triaged> <https://launchpad.net/bugs/1533790>
<natefinch> both are 1.25 only
<natefinch> it behaves nicely in 2.0
<redir> EAsy review anyone http://reviews.vapour.ws/r/5486/
<rick_h_> natefinch: katco good to know, I'm glad to hear we cleaned it up in 2.0.
<rick_h_> natefinch: with that in can you look at redir's pr please?
<natefinch> rick_h_: yep
<rick_h_> katco: I'll QA yours here in a sec just need someone to do the code review
<katco> rick_h_: cool ty
<katco> natefinch: qa checks out
<rick_h_> katco: QA in, it wasn't clear what changes to expect in this branch vs not expect so feel free to tell me to be patient for future branches
<katco> rick_h_: it only corrects the bugs mentioned in the commit message
<rick_h_> katco: right, but in my QA it only fixes the local case in that bug not the full bug
<katco> rick_h_: which now that i type that i remember you had listed multiple issues
<katco> rick_h_: let me fix the rest. that bit is trivial now that i can just write unit tests
<rick_h_> katco: rgr, it was a wrapper for several "deploy" scenarios around the exposure of "add charm" which shows up in the QA several cases still
<rick_h_> katco: k
<rick_h_> katco: <3 on the reduction of api connections and such
<katco> rick_h_: yeah that was kind of rediculous
<katco> rick_h_: but that's what i mean when i say cruft builds up. people are understandably just trying to get their change in, and they just cargo-cult bad practice
<redir> aka tech debt
<natefinch> rick_h_: can I jfdi this backport of the security fix?  the changes are very localized and highly unlikely to break anything unrelated.
<natefinch> (for 1.25)
<rick_h_>  natefinch shoulnd't need to since you can fixes-xxxxx
<rick_h_> natefinch: unless I'm missing something?
<natefinch> rick_h_: not sure how that works for private bugs
<natefinch> rick_h_: I can try :)
<rick_h_> natefinch: well this all came about because the bot told voidspace he couldn't land because it wasn't that fix so I'd expect it to work
<natefinch> ha, ok
<rick_h_> natefinch: obviously, if that doesn't you're fixing a blocking critical so jfdi seems the only way around it
<natefinch> rick_h_: yeah, just didn't realize it was a blocker since I usually use juju.fail for that, which I believe you mentioned has that blind spot for private bugs
<rick_h_> natefinch: rgr
<natefinch> redir: your PR fixes https://bugs.launchpad.net/juju-core/+bug/1614732 - so that's the thing bug should be testing, right?
<mup> Bug #1614732: Race in github.com/juju/juju/api/state.go <ci> <intermittent-failure> <race-condition> <regression> <unit-tests> <juju-core:Fix Committed by reedobrien> <https://launchpad.net/bugs/1614732>
<redir> natefinch: I  don't follow
<natefinch> redir: sorry, changed thoughts mid-sentence
<redir> natefinch: understood
<natefinch> redir: the QA steps are a little unclear.
<redir> ahh
<redir> I'll update
<natefinch> redir: I should apply the mgo patch and then run the api test suite with -race on a slow AWS machine, and it should trigger the race?  And then apply your patch and it should no longer trigger the race?
<redir> natefinch: to clarify when running the fix on the aws instance it was no longer triggering hte race that it fixes, but the other race which has a fix htat is waiting for upstream to incormprate. So I applied that patch too, to make sure it doesn't hide the race that this PR is addressing.
<redir> I do'nt know if htat is more clear but Le t me know if not
<redir> and I'll update the PR desc on RB
<natefinch> redir: I get it.  Just wanted to make sure I was understanding it correctly
<redir> OK
<natefinch> f yeah, tests that rely on files in the git, thanks!
<katco> ......
<natefinch> panic: cannot read repository found at "/home/nate/src/github.com/juju/juju/testcharms/charm-repo"
<natefinch> I geuss it was foolish to think we had isolated tests.  I was trying to avoid having to set up a whole dev environment on an aws server in order to catch a race condition
<natefinch> maybe if I just run hangouts and compile juju while I run the tests, that'll be slow enough to trigger the race
<katco> we have to chip away at this stuff as we see it. it's the only way forward
<rick_h_> natefinch: 1hr limit, if you can't QA it in that time then just review the code please and let the CI runs figure out if the race appears again or not
<natefinch> rick_h_: ok, thanks
<natefinch> ls
<redir> natefinch: just symlink /home/ubuntu to home/nate...
<redir> natefinch: also. Uh, I'll update the wiki
<natefinch> redir: I was trying to run a test binary on a machine without any of our code or anything... it probably would have failed due to not having mongo there anyway
<natefinch> sinzui: is the landing bot for 1.25 using go 1.2?  It seems to be missing TLS ciphers that are in 1.6 but not 1.2
<sinzui> natefinch: check the Makefile
<natefinch> sinzui: I don't know where the makefile is
<redir> natefinch: true. I installed mongo on the instance and rsynced the env up... whatevs
<sinzui> natefinch: the landing bot is go 1.6. the tests come from the juju branch. If the Makefile in the branch installs the wrong go, then you found your problem
<perrito666> aghh this is ridiculous, I need to fix a bug to be able to reproduce another
<sinzui> natefinch: in my experience, the Makefile is rarely updted to match the deps
<natefinch> sinzui: oh, THAT makefile. Sorry
<sinzui> perrito666: get in line. natefinch is about 4 bugs deep trying to fix  azure windows deploy.
<perrito666> sinzui: I am starting to be suspicious of one of the reports as it sounds odd that they could actually find that issue with the other in place
<natefinch> sinzui: I thought axw was working on the azure bug
<mup> Bug #1615095 opened: storage: volumes not supported <landscape> <juju-core:New> <https://launchpad.net/bugs/1615095>
<sinzui> natefinch: He is. you were working in the winows machines do not come up when we deoploy a windows charm. That is difficult to fix when the azure provider is broken and metadata generate-tools is broken
<natefinch> sinzui: for what it's worth, the bug with TLS is not the problem with that. the machine downloads the tools from the controller just fine.  Andrew submitted a fix for some other bug that I saw on the machine later.  Ihoped that would fix the last of the azure bugs.
<natefinch> sinzui: and I *think* the makefile is installing go 1.6?  I can't tell if that's actually what it's using to build, though.  And given that it's failing to find tls ciphersuites that exist in 1.6 but not in 1.2, I'd say it's probably not using 1.6 to compile.
<natefinch> I should make a canary file to tell us when someone tries to compile with an old version of go
<sinzui> natefinch: I wish you had said that earlier. I agree with you
<sinzui> natefinch: http://reports.vapour.ws/releases/4278/job/azure-arm-deploy-windows-amd64/attempt/100
<sinzui> natefinch: ^ I see the windows charms were deployed and they did exchange tokens
<sinzui> natefinch: The failue is just azure is slow. I bet this test would pass if I added 10 minutes
<natefinch> oh yeah, I saw Ian say that azure slowed even further with their latest update because everything is serialized now
<natefinch> redir: gave you a ship it... didn't ultimately have time to run the tests, but I trust you did, and the code is simple enough.
<natefinch> I gotta run early today, prepping for daughter's birthday party tomorrow
<redir> natefinch: yeah and still running, if it doesn't trigger by my EoD I'll merge. I was able to repro pretty easily -- and repeatedly before fixing.
<natefinch> cool
<cmars> i'm confused why this didn't land: https://github.com/juju/juju/pull/6043
<cmars> specifically http://juju-ci.vapour.ws:8080/job/github-merge-juju/8854/
<mup> Bug #1615106 opened: actionSuite.SetUpTest: forcibly closed <ci> <intermittent-failure> <mongodb> <test-failure> <unit-tests> <windows> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1615106>
<mup> Bug #1615108 opened: ConstraintsMerge arch problems <arm64> <ci> <jujuqa> <ppc64el> <s390x> <test-failure> <unit-tests> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1615108>
<mup> Bug #1615112 opened: EnableHASuite.SetUpTest: forcibly closed <intermittent-failure> <jujuqa> <mongodb> <test-failure> <unit-tests> <windows> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1615112>
<perrito666> cmars: flaky test
<cmars> perrito666, should i wait for it to be fixed? or just keep trying?
<perrito666> cmars: retry, it has been like that for a while
 * perrito666 runs slow tests
<perrito666> cmars: if its a known flaky test (as is the case with that agent one) I believe that your magic number is 3, if same failure hits you 3 times you have broken it a bit more than it already was
<cmars> perrito666, uh-oh, this is the third try.. D:
<perrito666> same failure?
<cmars> perrito666, running now
<perrito666> cmars: you might want to run that isolated test on your machine
 * rick_h_ runs, have a good weekend folks
<mup> Bug #1615118 opened: clientSuite.TearDownTest: mongodb has exited without being killed <ci> <intermittent-failure> <jujuqa> <mongodb> <test-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.25:Triaged by rharding> <https://launchpad.net/bugs/1615118>
<mup> Bug #1615121 opened: SettingsSuite.SetUpSuite "no reachable servers" <ci> <intermittent-failure> <jujuqa> <test-failure> <unit-tests> <juju-core:Incomplete> <juju-core kpi-metrics:Triaged> <https://launchpad.net/bugs/1615121>
<redir> taking a break, bbiab
#juju-dev 2016-08-20
 * redir goes EoW
<babbageclunk> Anyone around? I tried fixing bug 1614559, but once the version is parsed correctly I still can't bootstrap to lxd.
<mup> Bug #1614559: Juju rejects lxd 2.1 <bootstrap> <ci> <jujuqa> <lxd> <juju-core:Triaged by rharding> <https://launchpad.net/bugs/1614559>
<babbageclunk> I get an error saying "ERROR failed to bootstrap model: cannot start bootstrap instance: unable to get LXD image for ubuntu-xenial: The requested image couldn't be found."
<babbageclunk> Which is weird, because I can lxc launch an ubuntu xenial instance fine.
#juju-dev 2016-08-21
<menn0> thumper: tiny fix that took half a day to debug: http://reviews.vapour.ws/r/5488/
<menn0> thumper: adding the QA steps that were done now
 * thumper looks
<veebers> menn0: Follow on from Friday, I have the migration test with a user done. I'm still seeing the error message "ERROR cmd supercommand.go:458 empty target password not valid" when the migrate command fails
<menn0> veebers: let me try and replicate
<veebers> menn0: coolio
<thumper> menn0, wallyworld: where I am right now isn't conducive to hangouts
<wallyworld> ok
<thumper> wallyworld: report for standup: landed migrating storage pools, landed color tabwriter with list-connections as example, working on migrating storage constraints
<wallyworld> rightio
<thumper> veebers: once the storage constraints has landed, we should be good for a CI test for model with storage
<wallyworld> at least i won't have to lsiten to your gloating about the rugby
<thumper> heh
<thumper> I so enjoyed watching that game
<wallyworld> yeah, was awesome
<anastasiamac> thumper: is ur location conductive to reviews? I'd love to get a stamp on http://reviews.vapour.ws/r/5489/ :D
 * thumper looks
<anastasiamac> \o/
<veebers> thumper: ok, will poke later for further details
<thumper> anastasiamac: does that work on windows>
<thumper> ?
<thumper> anastasiamac: what about ppc64le
<thumper> ?
<anastasiamac> thumper: the bug is not related to windows :D and I do not have windows machine.. will have to rely on CI
<thumper> don't think ec2 supports that
<anastasiamac> thumper: the call is made to utils/arch/hostArch - whatever it supports, the test will support too
<anastasiamac> thumper: i cannot fix the whoole world \o/
<thumper> ok
<axw> anastasiamac: please see my reply, your change smells very coincidental
<mwhudson> can someone review https://github.com/juju/juju/pull/6031#issuecomment-241203007 pls?
<thumper> mwhudson: done
<menn0> axw: regarding bug 1569632
<mup> Bug #1569632: status: decide on statuses for migration <juju-core:Triaged by menno.smits> <https://launchpad.net/bugs/1569632>
<menn0> axw: is there still something to do there?
#juju-dev 2017-08-14
<wallyworld> thumper: thanks for looking. i pushed some changes and replied to comments
<axw_> burton-aus: standup?
<babbageclunk> wallyworld, axw_ - quick API design question? This API method returns a params.ErrorResults with any collected mismatches it finds. Should it return (params.ErrorResults, error)? Or just jam any incidental errors it hits into the ErrorResults?
<axw_> thumper: the machiner updates observed network config, but that's as observed on the machine. the details won't include provider IDs. so I think there's still work to be done
<axw_> babbageclunk: if it's an overall failure unrelated to a specific parameter, it should return an error besides the ErrorResult IMO
<wallyworld> babbageclunk: if the mechanics of getting the error results struct fails, you return a normal error for that
<babbageclunk> Ok
<wallyworld> the error results struct is just another api call result
<wallyworld> all api calls should return an error along with their results
<babbageclunk> Cool - that's what I have now, it just felt slightly odd to return (list of errors, error). I guess the migrationmaster worker will do different things for the different values thought.
<babbageclunk> Kind of like expected-errors, unexpected error.
<babbageclunk> Thanks
<babbageclunk> wallyworld, axw_: And the client method, should that return ([]params.Error, error) or ([]error, error)? I guess the former, since that includes more information.
<wallyworld> babbageclunk: ideally params should never escape the api layers, but they do :-(
<axw_> babbageclunk: does it? a *params.Error is an error
<wallyworld> often OneError() is used to collasce
<wallyworld> or CombineErrors() if more then one
<babbageclunk> wallyworld: Hmm - CombineErrors sounds like what I want
<wallyworld> could be
<wallyworld> axw_: here's a trivial juju/names PR to add "_" to cloud tags https://github.com/juju/names/pull/81
<axw_> wallyworld: lgtm
<wallyworld> ta
<wallyworld> veebers: looks like a CI script breakage? http://juju-ci.vapour.ws:8080/blue/organizations/jenkins/github-merge-juju-names/detail/github-merge-juju-names/46/pipeline
 * veebers looks
<veebers> wallyworld: hah, that job hasn't run for 3 months before now. Looks like things have changed from underneath it. I'm looking now
<wallyworld> ta
<veebers> wallyworld: fixed and re-ran
<wallyworld> great ty
<wallyworld> axw_: here's a trivial PR to pull in the names dep https://github.com/juju/juju/pull/7736
<blahdeblah> wallyworld, axw_, anastasiamac: any of you folks still around?  I'm trying to work out how juju decides what image id to use when I tell it "juju model-config image-stream=daily"; it's currently picking the wrong one.
<axw_> blahdeblah: yes, I'm around. which provider?
<blahdeblah> openstack
<blahdeblah> axw_: It's giving me this fun status output: https://pastebin.canonical.com/195832/ and the image id doesn't exist.
<axw_> blahdeblah: AFAICT, image-stream only affects the public cloud image metadata
<axw_> blahdeblah: I think you're supposed to change image-metadata-url for private clouds, if you want different images
<blahdeblah> axw_: I've never had to do that previously
<blahdeblah> i.e. this has worked before on this cloud with this image-stream
<blahdeblah> which suggests my juju controller is failing to get the correct image list for some reason
<axw_> blahdeblah: sorry, I don't know then, it'll require a deep dive. please file a bug and so forth. anastasiamac may be able to provide assistance more readily - she's been in that area more recently
<blahdeblah> No worries - thanks
<blahdeblah> I'll keep digging and file a bug if I can't bend it to my will
<wallyworld> blahdeblah: the image look up code has not changed in a long time. i would suspect that the json files on the canonistack swift container have been changed or are aout of date
<blahdeblah> wallyworld: How can I tell which URL juju is trying to hit for that?
<blahdeblah> (nothing in machine-0 log that I can see)
<wallyworld> use --debug when bootstrapping should print it i think
<wallyworld> you need debug to see it
<blahdeblah> but I'm not bootstrapping, I'm just adding a model
<wallyworld> turn on debug then
<blahdeblah> on the add-model, or something else?
<blahdeblah> actually, the problem doesn't appear until I deploy
<wallyworld> no, on the controller. juju model-config logging-config="<root>=DEBUG;"
<wallyworld> or something like that
<blahdeblah> OK - thankss
<wallyworld> on the controller model
<wallyworld> i *think* that will produce the output you need
<wallyworld> it should print the swift url it looks at
<wallyworld> you can also list the keystone catalog using the openstack CLI
<wallyworld> that will also have the swift container used
<menn0> wallyworld, axw_ : when you have a chance (no rush): https://github.com/juju/juju/pull/7738
<wallyworld> ok
<wallyworld> menn0: i'll do a proper look first thing tomorrow
<menn0> wallyworld: thanks. I wasn't expecting a review tonight :)
<wallyworld> :-) a bit tired tonight
<natefinch> rogpeppe1: is there support for private charms in the charm store?
<rogpeppe1> natefinch: yes
<natefinch> awesome
<rogpeppe1> natefinch: well, of course they're not *totally* private 'cos the charmstore gets to see them unencrypted
<rogpeppe1> natefinch: but i suspect that's good enough for what you need
<natefinch> I think that's probably fine, as long as no one outside of charm store admins can see them
<natefinch> We're looking into using Juju at Mattel
<natefinch> 'cause Terraform and nomad just turn into thousands of lines of bash masquerading as declarative config
<natefinch> rogpeppe1: is there a good local development story on Mac?
<rogpeppe1> natefinch: development of what?
<natefinch> rogpeppe1: local deploy, a la lxd on linux
<rogpeppe1> natefinch: good question. i'm not sure, as i don't have a mac. might need to use a local vm, i guess.
<natefinch> hmm
<rogpeppe1> natefinch: give it a go
#juju-dev 2017-08-15
<menn0> wallyworld: thanks for the review
<wallyworld> menn0: no worries, might be worth an eyball from andrew also?
<menn0> axw_: ^^ https://github.com/juju/juju/pull/7738 pls
<menn0> wallyworld: I've got a monster ready for review once this one lands :)
<wallyworld> i bet
<axw_> menn0: looking
<axw_> menn0: LGTM, thanks
<axw> babbageclunk: in case you've not seen them, I have a couple of PRs up for review on 1.25-upgrade
<babbageclunk> axw: oh sorry - looking at them now
<axw> babbageclunk: I have some more changes in the works, which ~require me to make use of the juju2 tree. to build some of the juju2 packages, I have to kill some of the (unused) code as it doesn't all build with the dependencies required by juju1
<babbageclunk> axw: Yeah, I had to make some small changes in juju2 to resolve version conflicts before too.
<anastasiamac> axw: could u PTAL @ https://github.com/juju/juju/pull/7740 (state pwd reset in prep for token re-issuing)
<axw> anastasiamac: okey dokey
<anastasiamac> axw: \o/
<babbageclunk> axw: both approved
<axw> babbageclunk: thanks
<axw> babbageclunk: by "to the description", you mean in the output displayed to the user?
<babbageclunk> axw: oh, no - I meant on the PR. But actually, that might be good call too.
<axw> babbageclunk: ah right, ok
<thumper> wallyworld: https://github.com/juju/juju/pull/7742
<wallyworld> looking
<wallyworld> thumper: lgtm with a suggestion
<thumper> wallyworld: ta
<wallyworld> axw: small one. for some reason, the cloud city creds didn't work for me, with any azure region, not just the new ones https://github.com/juju/juju/pull/7743
<axw> wallyworld: LGTM, thanks
<wallyworld> ta
 * babbageclunk goes for a run
<thumper> wallyworld: are long running actions landed?
<thumper> and are they 2.3?
<wallyworld> thumper: no, there was disagreement about the spec/solution
<wallyworld> so sort of put on hold
<thumper> hmm...
<thumper> disagreement between whom?
<wallyworld> next thing was going to be action file i/o
<wallyworld> disagreement internally - what stakeholders wanted was quite loosely defined
<axw> babbageclunk: I know it doesn't matter too much, but the command name "verify-source" doesn't sit well with me. I'm thinking of renaming it, perhaps to "export-source". thoughts?
<axw> babbageclunk: and I'm thinking of making it responsible for installing LXD on the LXC hosts...
<wallyworld> i have 2 PRs that need reviewing if anyone is bored... https://github.com/juju/juju/pull/7741 https://github.com/juju/juju/pull/7744
<menn0> babbageclunk, wallyworld: here's the big GetModel removal: https://github.com/juju/juju/pull/7745
<menn0> +353, -473
<babbageclunk> axw: I'm fine with that - the command predates me. Although I wasn't envisaging that we'd be exporting, saving the output, then importing - there are things like charms to think about too.
<babbageclunk> menn0: ok, looking now
<wallyworld> menn0: looking
<menn0> thanks guys
<axw> babbageclunk: ah ok. maybe we should just stop writing out the model description then
<axw> babbageclunk: I'll leave it for now
<babbageclunk> axw: yeah, it's useful for now, but I don't think we'd want it to be dumped for users. Maybe make it an option.
<axw> babbageclunk: so I've put up https://github.com/juju/1.25-upgrade/pull/16, which augments the model description written by verify-source. the meat is in a function, so it should be reusable in the command used to import into the target controller
<thumper> poo
<thumper> I have an initialization loop
 * thumper things how to unloop it
<thumper> axw: I don't suppose you have 5 minutes?
<axw> thumper: for you I might even have 6
<thumper> :)
<thumper> 1:1?
<axw> thumper: ok, brt
<mup> Bug #1710886 opened: juju 2.1 issues certificates with wrong DNS alternative names on juju upgrade. <juju-core:New> <https://launchpad.net/bugs/1710886>
#juju-dev 2017-08-16
<axw> burton-aus: standup?
<axw> jam thumper: tech board
<axw> menn0: ^
<jam> omw
<jam> went to load the page and foudn the proxy had died, almost there
<babbageclunk> axw: all approved
<axw> babbageclunk: thanks!
<babbageclunk> axw: sorry for the delay!
<mup> Bug #1710886 changed: juju 2.1 issues certificates with wrong DNS alternative names on juju upgrade. <juju:Triaged by jose-pekkarinen> <juju 2.2:Triaged by koalinux> <https://launchpad.net/bugs/1710886>
<seyeongkim> just quick question.. is there ETA for juju 2.3 alpha?
<rick_h> seyeongkim: good question, will have to bug the team on that.
<rick_h> wallyworld: ping when you're in
<thumper> morning
<wallyworld> rick_h: hi, just about to have short meeting, can talk after
<rick_h> wallyworld: rgr
<wallyworld> rick_h: hey, free now
<rick_h> wallyworld: k meet you in our 1-1 room
<rick_h> wallyworld: k, email sent to the list. Hopefully drives up some interest
<wallyworld> rick_h: awesome, tyvm
#juju-dev 2017-08-17
<wallyworld> axw: https://hangouts.google.com/hangouts/_/canonical.com/ian-tim?authuser=1
<wallyworld> can you jump in
<axw> ok
<wallyworld> axw: did you see this one? https://bugs.launchpad.net/juju/+bug/1711125
<mup> Bug #1711125: juju destroy-model asks for --destroy-storage even if no storage is in use <storage> <usabil> <juju:Triaged> <https://launchpad.net/bugs/1711125>
<axw> wallyworld: nope, thanks
<thumper> wallyworld: oh, I had a thought about the i/o timeouts
<wallyworld> do tell
<thumper> otp with jam
<thumper> but this is a reminder
<wallyworld> righto
<anastasiamac> wallyworld: looks like thumper is not using post-its on the screen :D
<wallyworld> epostits
<thumper> wallyworld: 1:1
<wallyworld> ok
#juju-dev 2018-08-14
<mup> Bug #1786965 opened: juju deploy bundle model comparison doesn't take into account still-deploying units <canonical-bootstack> <juju-core:New> <https://launchpad.net/bugs/1786965>
<mup> Bug #1786967 opened: Cannot target zone constraints in juju deploy bundle with MAAS provider <juju-core:New> <https://launchpad.net/bugs/1786967>
<mup> Bug #1786965 changed: juju deploy bundle model comparison doesn't take into account still-deploying units <bundles> <canonical-bootstack> <juju:Triaged> <https://launchpad.net/bugs/1786965>
#juju-dev 2018-08-16
<mup> Bug #1786967 changed: Cannot target zone constraints in juju deploy bundle with MAAS provider <juju:Incomplete> <https://launchpad.net/bugs/1786967>
#juju-dev 2018-08-17
<mup> Bug #1749272 changed: Adding existing subnets to space fails <juju:Triaged> <https://launchpad.net/bugs/1749272>
<mup> Bug #1787672 opened: snap-http-proxy and snap-https-proxy not honored in juju 2.4.1 <canonical-bootstack> <juju-core:New> <https://launchpad.net/bugs/1787672>
