#juju-dev 2013-01-28
<fwereade> jam, ping
<jam> hi fwereade
<fwereade> jam, I need to talk with you about constraints at some point
<fwereade> jam, I'm probably still a bit too tired to do so usefully today
<jam> fwereade: np. You're in EU time now, right?
<fwereade> jam, but I would be most grateful if you would think briefly about what was painful so we can have a productive chat tomorrow
<fwereade> jam, yeah
<fwereade> jam, my rough recollection is that juju-supplied defaults are bad, and the provider should be supplying those itself; this sounds sane, but if there's anything else I'll want to know :)
<fwereade> rogpeppe, ping
<fwereade> TheMue, heyhey
<TheMue> fwereade: Good morning, William. Had a good travel back?
<fwereade> TheMue, yeah, not bad
<fwereade> TheMue, frankfurt airport seems to have become nicer somehow
<fwereade> TheMue, maybe it's just that I was there at 5am on sunday and it seemed like nobody else was :)
<TheMue> fwereade: Hehe, yeah, a special time.
<TheMue> fwereade: A few days agon, on Monday, FRA was alost closed due to snow.
<TheMue> almost
<fwereade> TheMue, yeah, LHR and JFK were also both snowy, I was pretty lucky with all my travel tbh
<rogpeppe> fwereade: pong
<fwereade> rogpeppe, heyhey
<rogpeppe> morning all!
<fwereade> later guys, might be around off and on but don't count on it, basically calling today a swap day
<TheMue> re, back from dentist.
<TheMue> Hmm, somehow my editor dislikes me this morning. Package management for extensions like e.g. for Go fails.
<jam> dimitern: if you see mgz around, let him know I might be a little bit late for our 1:1
<dimitern> jam: sure
<dimitern> mgz: jam asked me to tell you he'll be a bit late for the 1:1
<mgz> dimitern: thanks
<jam> hi mramm, I'm chatting with mgz right now, but when we're done, I'd love to finally catch up with you
<mramm> jam: sounds good.
<jam> mramm: poke. I'm on mumble if you are
<mramm> I am
<TheMue> lunchtime
<niemeyer> Greetings
<TheMue> so, back
<TheMue> niemeyer: Hiya
<TheMue> niemeyer: Nice idea, the statistics for baby feeding. ;)
<niemeyer> TheMue: Cheers :)
<mgz> core guys, can someone look at my one-line change and confirm it's correct? <https://codereview.appspot.com/7233046/>
<TheMue> mgz: You've got a +1.
<fwereade> mgz, hell, sorry about that... we should probably have a trivial test on every package to stop things like this from happening
<fwereade> morning niemeyer
<fwereade> (not actually working today)
<mgz> fwereade: yeah, I'd like commands to have some "does --help" work thing, but haven't worked out a neat version of the python idiom for that yet
<niemeyer> fwereade: yo
<fwereade> mgz, good idea
<fwereade> niemeyer, how's it going?
<niemeyer> fwereade: Good, will start the week having a look at goamz
<fwereade> niemeyer, excellent
<rogpeppe>  fwereade: i've got a patch for gotest which complains if this happens
<fwereade> rogpeppe, sweet
<rogpeppe> fwereade: unfortunately i seem to remember it wasn't acceptable
<fwereade> rogpeppe, haha, ok
<fwereade> rogpeppe, go build ./... && go test ./... :-/
<rogpeppe> fwereade: isn't it working?
<fwereade> rogpeppe, I expect it is now, but it wasn't
<rogpeppe> fwereade: BTW i've got a workaround script for the fact that you can't do  go test -c ./...
<TheMue> fwereade: How about the still open idea of having a CI server?
<rogpeppe> fwereade: ah, that must've been your Conn changes. don't you always run go test ./... before submitting?
<fwereade> rogpeppe, ofc I do
<fwereade> rogpeppe, that doesn't help with the packages without tests ;p
<rogpeppe> fwereade: ha
<TheMue> fwereade: hehe
<rogpeppe> fwereade: i think go test should at least try to compile the package even if has no tests
<fwereade> TheMue, +1 on that, not sure when I'm going to get around to it though
<fwereade> rogpeppe, agreed
<rogpeppe> fwereade: i'll raise an issue
<fwereade> TheMue, wasn't davecheney working on that a while ago?
<fwereade> rogpeppe, cool
<TheMue> fwereade: dunno if already working, but imho thinking about it.
 * niemeyer => lunch.. biab
<niemeyer> mramm: Do you know if the consistency call is rolling today?
<mramm> niemeyer: I believe that it is not
<niemeyer> mramm: Cool
<rogpeppe> just run first tentative benchmark of rpc server, just for sanity check, and i get about 4000 requests a second (or 2000 rpc when round-tripping to mongo). i *think* that should be ok.
<rogpeppe> s/rpc/rps/
<rogpeppe> s/2000 rpc/2000 rps/ :-)
<TheMue> rogpeppe: Sounds nice.
<rogpeppe> TheMue: that's not fantastic really - it's an upper bound on how many requests we can expect to serve per second from a single api server instance.
<rogpeppe> TheMue: but i think we probably have some possibilities for optimisation
<TheMue> rogpeppe: I think it's ok as it is your first approach and e'thing runs on only one machine (calling and processing).
<rogpeppe> TheMue: yeah, but that means it's actually faster than it will be in real life, i think (we're doing almost no processing). we'll see.
<TheMue> rogpeppe: I'm optimistic that you may tune it a but.
<TheMue> bit
<rogpeppe> TheMue: we actually have the possibility of using a different protocol for agents (we could maybe use gob, for example)
<rogpeppe> TheMue: and that could be quite a bit more efficient than websockets+json, i think
<rogpeppe> TheMue: so we do have leeway
<TheMue> rogpeppe: gob may be nice as the volume is smaller. but how about clients in different langs then?
<rogpeppe> TheMue: all our agents are written in the same language :-)
<TheMue> rogpeppe: thankfully you don't use xml. :P
<TheMue> rogpeppe: the api is only inteded between the agents and the server? thought of the html client too.
<niemeyer> "test/plain"
<niemeyer> Interesting content-type to find in a test case..
<rogpeppe> TheMue: we could use a different protocol for agents vs html clients
<rogpeppe> niemeyer: ha
<TheMue> rogpeppe: ah, ic, that's in that case pretty nice.
<niemeyer> I almost don't want to fix it :-)
<rogpeppe> niemeyer: lol
<TheMue> niemeyer: :)
<rogpeppe> right, all tests passing, time to call it a night
<rogpeppe> g'night all
<rogpeppe> niemeyer: happy goamz sprinting!
<niemeyer> rogpeppe: Cheers
<TheMue> bye roger
<niemeyer> rogpeppe: Have a good time there
<rogpeppe> niemeyer: and you
#juju-dev 2013-01-29
<niemeyer> Night all
<wallyworld> jam: hi, you around?
<jam> hey wallyworld
<wallyworld> hi, stupid question, i cannot get a go install or build etc to include any of the openstack stuff
<wallyworld> it seems like the openstack package is ignored for me
<jam> wallyworld: what command are you running, and what result are you expecting? You mean being able to run 'juju foo' with openstack as the environ?
<wallyworld> yes
<jam> or you expect something in $GOPATH/bin/...
<wallyworld> so i "go install launchpad.net/juu-core/..."
<wallyworld> and my go bin dire is updatred
<wallyworld> with the latest juju
<wallyworld> but onloy if i change somthing in ec2 package
<wallyworld> if i tweak something in openstack package the build/install process seems to ignore it
<wallyworld> so that implies the openstack stuff is not being included
<jam> wallyworld: right, I'm guessing we are missing the registration of the openstack provider
<wallyworld> and matches what i see cause the openstack init() is not being run
<wallyworld> the registration is done inside init()
<wallyworld> and init() is not being run for me since the go install seems to be ignoring my openstack package
<jam> wallyworld: look at launchpad.net/juju-core/cmd/main.go
<jam> it does "import _ "launchpadb.net/juju-core/environs/ec2"
<jam> we probably just need to add "openstack" there.
<wallyworld> ah ok, that makes sense
<fwereade> jam, wallyworld: that's right :)
<fwereade> jam, wallyworld: good mornings
<wallyworld> thanks, i didn't know about that - was driving me crazy
<jam> good early morning to you as well, fwereade
<wallyworld> morninh
<wallyworld> so till now it seems no one has tried to test go juju with openstack apart from running hte tests
<fwereade> wallyworld, that's weird, I could swear dimitern told me you had it bootstrapping -- maybe that's via the live tests though
<wallyworld> yeah, i suspect so
<wallyworld> and i thought i'd take a moment today to try stuff out so to speak
<fwereade> wallyworld, +1
<wallyworld> with no luck till now :-)
<jam> wallyworld: I thought he had, but I thought he had just done a local patch for it. It is how I would have tested it.
<wallyworld> jam: maybe it's time to plug it in then?
<fwereade> wallyworld, jam, I think so, definitely
<fwereade> wallyworld, jam, I would be +1 on some very basic tests that just verify that we will accept a trivial config for each of the providers that "ought" to be registered
<wallyworld> fwereade: so with the juju go, running "juju bootstrap" does not create a default environments.yaml like pyjuju, right?
<fwereade> wallyworld, that's right; I think there's a bug related t it
<wallyworld> ok, np.
<wallyworld> +1 to adding a few simple tests also
<fwereade> wallyworld, it's https://bugs.launchpad.net/juju-core/+bug/993616
<jam> wallyworld: so if we are actually at a point where 'juju bootstrap' with a openstack config doesn't barf/cause corruption, I'm happy to expose it.
<_mup_> Bug #993616: We should ship a boilerplate environments.yaml. <juju:Confirmed> <juju-core:Triaged> < https://launchpad.net/bugs/993616 >
<fwereade> wallyworld, it looks like it has some good samples in there
<jam> fwereade: what is the standard for testing actual binaries?
<jam> I don't think I've seen much testing of it in the code.
<fwereade> jam, we have thus far got away with testing main() in-package
<fwereade> jam, I would also be +1 on smart improvements to this... there's at least one test which actually builds go code
<fwereade> jam, I think that's the test for --upload-tools, perhaps
<wallyworld> jam: the main issue i see with exposing openstack by default is perhaps the lack of a suitable public bucket
<jam> wallyworld: we have a shared account now, I think I sent you the access keys
<wallyworld> but if we ship a yaml that has stuff commented out, neither openstack nor ec2 will work out of the box anyway
<jam> we can turn that into the public buckte for now.
<wallyworld> jam: right, but we wouldn't want to ship those keys
<jam> wallyworld: "public bucket"
<jam> we use the shared account to create a readable bucket
<jam> which other people can use
<wallyworld> ok, sorry, brain fade
<jam> wallyworld: so we should be able to do something like set up a test double, override the appropriate env variables, run 'juju bootstrap', and inspect the double afterwards
<jam> if you want a bit of an end-to-end (but with doubles) test.
<jam> that (IMO) makes a pretty decent smoke test
<wallyworld> yeah, agree
<wallyworld> i also want to try it on an openstack instance (ie mine) to see how it goes
<jam> wallyworld: certainly. I just wouldn't put your account details into the test suite :)
<wallyworld> never fear, i was just wanting to run juju locally
<wallyworld> not the tests, the actual tools themselves
<jam> certainly
<wallyworld> bbiab, have to take son to physio
<fwereade> jam, if you have a moment, would you take a look at https://codereview.appspot.com/7227045/ please? and consider in particular the rambling about InferEndpoints, which I am starting to believe is mildly crackful
<jam> fwereade: looking at the test, it does feel like there should be a call like AddRelation that does the infer first and then adds them. Given you are doing retries in case the service definition changes, it does seem like a combined op.
<fwereade> jam, cool, thanks
<fwereade> niemeyer, good morning
<fwereade> niemeyer, midnight feeding? :)
<jam> niemeyer: is this good morning? or good evening? :)
<fwereade> TheMue, morning
<fwereade> dimitern, morning
<fwereade> TheMue, free for a quick call?
<TheMue> fwereade: could we do it a bit later? i've to step out to the post office in a few moments, just packing a package i have to sent out today. ;)
<dimitern> fwereade, TheMue: morning
<fwereade> TheMue, I expect so -- I'll need to pop out for a bit in an hour or so myself
<fwereade> TheMue, when will you be back?
<TheMue> fwereade: i think in 30 minutes, but let's talk now
<fwereade> TheMue, ok, cool, should be quick
<rogpeppe> fwereade, TheMue, jam, dimitern, wallyworld: morning!
<dimitern> rogpeppe: morning!
<niemeyer> fwereade: Heya
<niemeyer> jam: Yo
<niemeyer> fwereade, jam: Sorry, missed the messages earlier
<niemeyer> fwereade: Yeah, kind of :)
<niemeyer> Making use of the quiet time, though
<fwereade> niemeyer, enjoy it while you can :)
<fwereade> gents, popping out for a bit
<fwereade> rogpeppe, are you free for a call about the API etc?
<rogpeppe> fwereade: definitely
<rogpeppe> fwereade: one mo, i'll paste you an initial CL
<fwereade> rogpeppe, cool
<rogpeppe> fwereade: https://codereview.appspot.com/6878052
<dimitern> jam, mgz, wallyworld: PTAL https://codereview.appspot.com/7228058/
 * wallyworld looks
<dimitern> fwereade, rogpeppe: also, this one is closely related: https://codereview.appspot.com/7241045/ to ^^
<wallyworld> dimitern: done
<dimitern> wallyworld: thanks!
<wallyworld> np. 'see' you at the standup
<fwereade> dimitern, cheers, on it shortly
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe: cheers!
<fwereade> dimitern, I think I'm in agreement with rogpeppe -- why is it that GetPrivateAddress needs to be exported?
<dimitern> fwereade: because I need to call it in the tests, which are in a separate module
<fwereade> dimitern, that sounds like a job for export_test.go
<dimitern> fwereade: yes, but would it be usable from openstack then? tests are in openstack_test and putting it in export_test will make it available for tests, but not for the openstack module I think
<fwereade> dimitern, but shouldn't it really be tests against the openstack Instance type anyway?
<dimitern> fwereade: well, initially it was a method of instance (lower case one - in provider.go), but then creating an empty instance only so I can call it on it seemed wrong
<fwereade> dimitern, also, shouldn't Instance.DNSName be returning the public address? I suspect I'm hopelessly confused here ;)
<dimitern> fwereade: no, the same logic is ported from py-juju - private address is used, the public one is not available (at least not always), until you set a floating IP to the instance
<fwereade> dimitern, ah, ok, sorry, you told me that the other day
<dimitern> fwereade: yeah, but anyway we don't need the public address there or dns name - just an address to access the instance - private will do
<fwereade> dimitern, so, wait, don't we need some public address so we can talk to the "bootstrap" node from outside? isn't that done via DNSName()?
<dimitern> fwereade: I'm not sure, but I think it's not used from outside
<dimitern> fwereade: maybe we should have a short call to discuss this with whoever knows best about how is this supposed to work
<fwereade> dimitern, ISTM that we have to talk to the bootstrap node from outside
<fwereade> rogpeppe, ping
<dimitern> fwereade: well, how did it work in py-juju then?
<rogpeppe> fwereade: pong
<fwereade> rogpeppe, what's your understanding of DNSName() re public/private address?
<fwereade> dimitern, I have no idea :)
<rogpeppe> fwereade: the only important thing AFAIR is that it should return a publicly usable address
<fwereade> rogpeppe, that was what I thought :)
<rogpeppe> fwereade: because that's what we use to connect to a unit from outside
<rogpeppe> fwereade: nothing in juju connects to machines, i think
<fwereade> rogpeppe, define "nothing in juju" please :)
<rogpeppe> fwereade: bah. ykwim :-)
<rogpeppe> dimitern: when you say "would it be usable from openstack", what openstack package are you referring to there?
<dimitern> rogpeppe: environs/openstack
<rogpeppe> dimitern: any function defined in openstack is callable from openstack
<dimitern> rogpeppe: the problem is the tests are in openstack_test
<rogpeppe> dimitern: then you can use export_test
<dimitern> rogpeppe: and still use it in both openstack and openstack_test ?
<rogpeppe> dimitern: or define some tests in openstack instead, if that's more appropriate
<rogpeppe> dimitern: sure
<fwereade> dimitern, hence export_test.go -- that will be in package openstack, but thanks to the _test.go name will only be built with the tests
<rogpeppe> dimitern: in export_test:
<fwereade> dimitern, you won't be able to use export_test.go stuff in openstack
<rogpeppe> func GetPrivateAddress(addresses map[string][]nova.IPAddress) (string, error) {return getPrivateAddress(addresses) }
<fwereade> dimitern, so you have an exported name that calls the unexported function
<fwereade> dimitern, what rogpeppe said :)
<dimitern> rogpeppe: ok, I'll that then
<dimitern> fwereade: still not sure about the DNSName semantics though
<rogpeppe> dimitern: generally we try not to put significant code in export_test - it's used to make things available for the external tests only.
<fwereade> dimitern, it kinda looks like you're always returning a private address there
<fwereade> dimitern, I think I'd be ok with a private address if and only if no public one is available
<rogpeppe> dimitern: private addresses are pretty much useless for the purposes juju uses DNSName for
<fwereade> dimitern, but DNSName() should usually return a public one
<dimitern> fwereade: I still think we better discuss this - with mgz as well, he has better view on the py openstack provider
<rogpeppe> fwereade: i tihnk i disagree - it's misleading if juju ssh tries to connect to an address that doesn't work
<dimitern> I need to wrap my mind around it a bit
<fwereade> dimitern, ISTM that the python uses a private one only if it can't get a public one
<fwereade> rogpeppe, ^
<dimitern> fwereade: where do you see this?
<rogpeppe> fwereade: what's the use of doing that?
<fwereade> rogpeppe, dimitern, mgz: my unlettered reading of this is that some openstacks have nothing public, but that the client is connecting from the same network, so it's "ok"
<rogpeppe> fwereade: hmm, if that's really true, then it might be ok
<fwereade> dimitern, providers/openstack/machine.py:62
<dimitern> fwereade: yeah, I see this - so it's fine to do it like this?
<fwereade> dimitern, I think that falling back to a private address when no public one exists is *probably* ok, assuming we get mgz to confirm we're not smoking crack
<dimitern> mgz: ping
<fwereade> dimitern, but the current approach of always returning private is not ok, I think
<dimitern> fwereade: well, if we need definitely a public IP always it changes things quite a bit
<dimitern> fwereade: we i'll then need to provision a floating IP to each machine we're starting in the environment
<fwereade> dimitern, I think I'm failing to communicate
<fwereade> dimitern, if a public address is available, we should definitely use that
<rogpeppe> fwereade: maybe we should have an environ parameter, say private-addresses, saying whether we want to use private or public addresses
<fwereade> dimitern, if not, ISTM that falling back to a private address is roughly sane
<dimitern> fwereade: it's never available for canonistack and I doubt it will be anything but rare in other cases
<rogpeppe> fwereade: when it's false, we could try to allocate floating ip addresses, or whatever's necessary to make dns names public
<fwereade> dimitern, assuming my reasoning holds -- I was not privy to the thinking behind this behaviour in python, and I would like some input from someone who's done this before ;)
<dimitern> rogpeppe: dns name allocation is also supported somewhat, but it's a different API call and likely to be similar to how floating IPs are assigned
<fwereade> rogpeppe, yeah, why does it have to be DNSName()? is it not SomeStringThatWillGetYouToTheInstance()?
<rogpeppe> fwereade: well, it's SomeStringThatWillGetYouTotheInstanceFromSomewhere() :-)
<fwereade> rogpeppe, ha, yeah
<rogpeppe> fwereade: and it's the "somewhere" that we can't easily second guess
<dimitern> rogpeppe: I was thinking about that :)
<fwereade> dimitern, what time's your standup again?
<dimitern> fwereade: now
<fwereade> dimitern, please ask mgz then, and we shall reconvene shortly?
<dimitern> fwereade: sure, once he comes round
<fwereade> dimitern, cool
<mgz> dimitern: hey
 * mgz reads discussion
<jam> mramm: we're mumbling if you want to unmute
<mramm> cool be on in 1 min
<jam> mramm: you were echoing
<mgz> so, for now returning whatever you can (by the logic from Python) for DNSName will get things working
<rogpeppe> fwereade: ping
<fwereade> rogpeppe, pong
<rogpeppe> fwereade: i'm looking at auth stuff
<rogpeppe> fwereade: and wondering about the "admin" of SetAdminPassword
<rogpeppe> fwereade: i'm wondering if it might be more appropriate if the "admin" was actually "client"
<rogpeppe> fwereade: because Admin sounds like it has superuser properties, but i don't think that's what we want to give it
<fwereade> rogpeppe, -1, I think we will before too long have to worry about other users with fewer privileges
<rogpeppe> fwereade: indeed, and i think that "client" can more easily mutate into other usernames
<fwereade> rogpeppe, who should have those privileges if not whoever set up the environment?
<rogpeppe> fwereade: i'm not sure that anyone has all the privileges
<fwereade> rogpeppe, this may be something we need a call to cover properly
<rogpeppe> fwereade: for instance, i don't think we want to enable a dodgy admin to act as a uniter
<fwereade> rogpeppe, ok -- but I think whoever set up the environment should have all applicable privileges on that environment -- and "admin" seems to fit that better than just "client" which is IMO way too generi
<fwereade> rogpeppe, at least we're not calling them "root" ;0
<fwereade> rogpeppe, btw, we don't have any ssh proxy support in state.Open, do we..?
<rogpeppe> fwereade: i'm also looking at it from the perspective of the actual mechanics of logging in. here's the code i currently have: http://paste.ubuntu.com/1585690/
<rogpeppe> fwereade: no. do we want it?
<fwereade> rogpeppe, given what mgz said above, yes, probably, if we want canonistack to work
<rogpeppe> fwereade: where would the proxy live?
<fwereade> rogpeppe, you ssh into canonistack instances via chinstrap AIUI
<rogpeppe> fwereade: the question wrt the above code is: what underlying type does State.Entity return when EntityName=="admin" ?
 * rogpeppe hasn't heard of chinstrap
<fwereade> rogpeppe, IMO we need a state.User
<rogpeppe> fwereade: i think setting passwords directly on the applicable entities is fine. but we may need state.User too
<fwereade> rogpeppe, I'm 90% sure we need a state.User, but I need to talk to you about it
<fwereade> rogpeppe, I imagine it would have worked transparently when we were using SSH directly, because everybody would have had their proxies set up
<rogpeppe> fwereade: what is chinstrap?
<rogpeppe> fwereade: ah, it's a machine!
<rogpeppe> fwereade: we don't need ssh proxying, just a straightforward tcp tunnel
<rogpeppe> s/tunnel/forwarder/
<rogpeppe> fwereade: but maybe sshd can provide that.
<fwereade> rogpeppe, just realised I need to eat before team meeting
<rogpeppe> fwereade: ok, enjoy
<fwereade> bbiab :)
<wallyworld_> jam: i'm trying to lbox submit those branches and the very first one is failing with a sad gofmt, complaining about a file which doesn't even exist. have you seen that before?
<rogpeppe> wallyworld_: that sounds a bit weird. if gofmt is talking about a file, i'm pretty sure it must be there.
<rogpeppe> wallyworld_: have you tried go fmt ./... from the project root?
<wallyworld_> yeah, tell me about it. but 'ls' says otherwise
<wallyworld_> yep, no issues
<rogpeppe> wallyworld_: try running the same command as lbox does: find * -name '*.go' | xargs gofmt -l
<rogpeppe> wallyworld_: and see if it prints anything
<wallyworld_> yeah, did that too
<wallyworld_> let me see what it said
<wallyworld_> no output, so happy
<wallyworld_> yet lbox submit complains
<wallyworld_> i did wipe my local go bin dir and did a go install launchpad/net/lbox earlier
<rogpeppe> wallyworld_: try putting a debug statement in .lbox.check to see if it's actually running; and include the value of $BADFMT too.
<rogpeppe> s/a debug/an echo/
<wallyworld_> yep, will do
<wallyworld_> ah, complains branch is not clean, i'll have to comment out that bit
<rogpeppe> wallyworld_: just commit it...
<rogpeppe> wallyworld_: (then revert it later)
<wallyworld_> ok
<wallyworld_> hmmm. my echo wasn't printed
<wallyworld_> now i'm confused
<rogpeppe> wallyworld_: sanity check failed :-)
<wallyworld_> yeah, but that's not something i expected to fail, it's just worked previously
<wallyworld_> and yet something is running go fmt
<rogpeppe> wallyworld_: that's what sanity checks are all about :-)
<rogpeppe> wallyworld_: more: something is saying "gofmt is sad", right?
<wallyworld_> yes
<rogpeppe> wallyworld_: which means *a* lbox.check script is running, from somewhere...
<wallyworld_> for a non existant file
<wallyworld_> yeah, i'll have to try and track it down
<rogpeppe> wallyworld_: how about running lbox --versose --debug ?
<rogpeppe> s/versose/verbose/
<jam> poke for our hangout?
<wallyworld_> good idea, didn't realise those optins were there
<rogpeppe> on my way
 * fwereade joins
<TheMue> hangout link?
<jam> mgz: poke for the hangout
<jam> https://plus.google.com/hangouts/_/33acfc2c8af8792568274fa110371db956f9fde7
<jam> w7z: https://plus.google.com/hangouts/_/33acfc2c8af8792568274fa110371db956f9fde7
<rogpeppe> niemeyer: meeting?
<niemeyer> rogpeppe: Sorry, yep
<wallyworld_> rogpeppe: figured it out - jam's fault :-P
<wallyworld_> sad go fmt file in trunk which was not there when i started my branch
<jam> wallyworld_: I'm willing to take the blame, I accidentally committed something directly that I was trying to submit via botd
<jam> sorry about that.
<wallyworld_> np
<wallyworld_> jam: i've submitted my stuff, needing to merge trunk (and hence your changes) first. there were some conflicts which i think i've done correctly. but it would be cool if you could check
<rogpeppe> fwereade: i'm just considering adding a user api to the state. we'd use it for admin only to start with, but potentially expandable in several directions. what do think of this? http://paste.ubuntu.com/1585936/
<fwereade> rogpeppe, that looks roughly sane, I'll want to have a call again in a bit but my brain is currently a touch overloaded
<rogpeppe> fwereade: np!
<dimitern> rogpeppe, wallyworld: follow-up https://codereview.appspot.com/7228058
<mgz> rogpeppe: what was your trick for doing an initial codereview thing? I can just propose current trunk against an empty (or r1) branch for the current state to come up in diff, for instance
 * niemeyer => lunch
<dimitern> rogpeppe, mgz: follow-up on https://codereview.appspot.com/7241045/
<mgz> dimitern: looking
<mgz> ...I still don't like this struct-list-test+loop thing
<dimitern> mgz: well, I like it more each time :) so compact and expressive
<mgz> it's longer than the original!
<dimitern> mgz: it's not yes, I changed it as we discussed - use public if available, otherwise use the private - as a compromise
<dimitern> mgz: aah longer
<mgz> each test in python is 5-7 lines, with pretty formatting
<mgz> and that's the *complete* test
<mgz> not just a test definition that then needs overly complicated testing code that lives somewhere off the page
<dimitern> mgz: anyway it does what's supposed and it's easy to extend, etc.
<mgz> each of these definitions is 8 lines, and that's without any test code, or the struct defintion
<mgz> it's harder to do particular customisations for special cases though
<mgz> you'd end up having to add a new field to your struct as a marker
<dimitern> mgz: it's definitely shorter than individual test case methods + comments for each
<mgz> add that to the each struct constructor (or used the named param idiom), and then special case that in the testing codde
<mgz> yeah, because there are no comments
<mgz> so, you've no idea what the expected result is...
<mgz> anyway, looks fine apart from me disliking the style (which is idiomatic here, so correct for you to do...)
<dimitern> mgz: ok :) is it LGTM then?
<mgz> sec, hitting m shortly
<fwereade> ok gents, I've been on since 6am and I'm a bit fried now; I'll probably swing by later, but I'm off to decompress for a bit
<mgz> dimitern: okay, get landing :)
<mgz> have a nice rest fwereade :)
<dimitern> mgz: cheers
<dimitern> rogpeppe: sorry I missed your last comments, will do a follow-up CL
<rogpeppe> dimitern: np
<rogpeppe> dimitern: just makes the docs look a bit more professional, i think
<dimitern> rogpeppe: yeah, I agree - wasn't sure if using func-style comments for struct fields is ok
<rogpeppe> dimitern: it's not always done, but i think it looks better when it is. particularly when some are longer.
<dimitern> rogpeppe: yes, if it's a whole line, rather than a short one at the end of the line
<rogpeppe> dimitern: yeah.
<dimitern> rogpeppe: and here it is: https://codereview.appspot.com/7235058
<rogpeppe> dimitern: LGTM with one typo
<dimitern> rogpeppe: cheers
<TheMue> Yeah, build of Juju on OS X works. Will test it later.
<rogpeppe> a CL to review, if anyone fancies, nothing too onerous: https://codereview.appspot.com/7226056/
<rogpeppe> that's me for the day
<rogpeppe> g'night all!
#juju-dev 2013-01-30
<jam> wallyworld: /wave
<wallyworld> g'day
<wallyworld> jam: i've put up a couple of mp's - bootstrapping a real openstack works :-)
<jam> wallyworld: sounds great
<wallyworld> but canonistack is responding now with a resource limit exceeded error, but it was working, o swear
<wallyworld> i ran it against my user id for everything (public bucket etc), didn't try out shared tenant
<jam> wallyworld: you could certainly have started to many instances, etc in a time frame.
<wallyworld> yes, i did a lot of debugging/testing
<rogpeppe> morning all!
<dimitern> rogpeppe: morning
<TheMue> rogpeppe, dimitern: morning
<dimitern> TheMue: hiya
<rogpeppe> i'd appreciate some comments on https://codereview.appspot.com/6878052 if anyone fancies having a look
<TheMue> Aaaargh, heavy rain showers here. OK, much warmer than a week ago, but so wet.
<TheMue> Thankfully no need to step out.
<TheMue> rogpeppe: *click* before continuing my OS X tests (build went w/o probs).
 * dimitern looking as well
<dimitern> rogpeppe: done
<rogpeppe> dimitern: that was quick, thanks!
<dimitern> rogpeppe: np
<dimitern> wallyworld: ping
<jam> rogpeppe: I'm trying to use 'lbox propose' but I'm still getting "local error: protocol version not supported".
<jam> Is there any way to get more context in the error messages?
<rogpeppe> jam: lbox --verbose ?
<TheMue> rogpeppe: You've got a review.
<rogpeppe> TheMue: tyvm
<jam> rogpeppe: it looks like it might be a username failure, but where are the cached auth details saved?
<TheMue> rogpeppe: Missed to enter the LGTM, so after your answers I'll do it. ;)
<rogpeppe> jam: hmm, i've never looked, sorry
<rogpeppe> jam: i'd appreciate your comments on this CL too, if you have some time. https://codereview.appspot.com/6878052
<jam> rogpeppe: ... it is telling me that it successfully read from a file that doesn't exist....
<rogpeppe> jam: hmm.
<rogpeppe> jam: lbox is?
<jam> rogpeppe: I hacked rietveld to tell me where it was loading the credentials, and it logs "Successfully loaded from $HOME/.goetveld_codereview.appspot.com" but there is no such file.
<rogpeppe> jam: weird. more printfs required, i think :-)
<rogpeppe> jam: i could have a look, but i'd only be duplicating your work, and i can't reproduce the problem...
<jam> yeah, working on it
<jam> lots of prints now
<jam> rogpeppe: step 1: failed to read file, step 2: successfully read credentials... WTF?
<rogpeppe> lol
<jam> rogpeppe: there is a func "filterNotFound()" which turns not-found errors into nil
<jam> so it says it loaded when the credentials aren't actually there.
<rogpeppe> jam: so it's deliberate. perhaps this isn't the reason after all.
<jam> rogpeppe: sure, but the log message saying it successfully loaded credentials is confusing
<jam> And it isn't asking me to log in.
<jam> It did the very first time I ran it with -v
<rogpeppe> jam: indeed. and that seems odd.
<jam> and now it just dies
<jam> It seems to get a 401
<jam> and then... protocol not supported tends to happen.
<jam> but I'm not able to get my creds into the system, because it doesn't get to the point of actually asking me to log in.
<rogpeppe> jam: if you run it with --debug, you'll see the traffic to and from the server too, which might help possibly
<jtv> Hi rogpeppe (and thanks for all your helpful comments by the  way!), hi jam.
<jam> h
<jam> hi jtv
<rogpeppe> jtv: hiya
<rogpeppe> jtv: on the go gotchas doc?
<jtv> The Cultural Learnings For Make Benefit.
<rogpeppe> jtv: a great title
<jtv> Julian's.  Stroke of genius.
<rogpeppe> jtv: i think that at least some of it might be good to publish more widely
<jtv> Good point.  I'll mention it in our standup call.
<jtv> Meanwhile, another thing you gents may be able to help with:
<jtv> when a juju provider needs to know things like the base URL it should talk to get its virtual machines etc., that all goes into the config and is parsed out by Provider.SetConfig(), right?
<rogpeppe> jtv: yup
<rogpeppe> jtv: or at least it should be derivable from the config
<jtv> Thanks.  And in general, is the config meant to be a single level deep, or nest structures, or be very very deep so the whole world can fit in if needed..?
<rogpeppe> jtv: generally it's a single level
<rogpeppe> jtv: i'm not sure if we allow deeper nesting - it might be useful at some point (i've considered a situation where we might want nested provider configs...)
<jtv> For gomaasapi we built a little object layer for dealing with dynamic JSON structures.  Not great, but it gets us past the static/dynamic problem.
<rogpeppe> jtv: have you seen juju's schema package?
<jam> rogpeppe: well, eventually after running the command probably 50 times now, it finally asked for my password and I could get it entered. I still don't know what was wrong with the protocol, but it apparently blocked me actually logging in.
<jtv> No..?
<jam> jtv: you can use a map[string]interface{} for dynamic structures, (I think)
<rogpeppe> jtv: it makes it easier to coerce dynamic JSON structures into a known form.
<jtv> jam: yes we do use that, but our code hides the casting etc. underneath.
<jtv> rogpeppe: that was part of our problem: we couldn't guarantee much about the known form.
<jam> jtv: how dynamic is the actual structure? We went with more of a known structs which allows the json marshall/unmarshall to handle casting to types rather than writing a lot of functions
<jam> (function calls)
<rogpeppe> jtv: that can be awkward
<rogpeppe> jtv: i'd be interested to see the range of forms you have to deal with.
<jam> (the argument is that if you don't actually know the structure, you don't really know how to use it, do you?)
<jtv> Well in our case we had the problem of a boundary of responsibilities between the code that uses the objects (and thus has to know /something/ about their structure) and the code that serves them up.
<rogpeppe> jam: yeah, but json structures can be alarmingly content-dependent sometimes
<jam> jtv: you can pass in the expected structure (via interface{}) and have it get populated and returned
<jtv> Yes.  We had a few slightly different requirements though.  The one thing we know is that "MAAS objects" are JSON objects with a resource_uri entry.
<jtv> (Although conversely of course, a JSON object with a resource_uri entry need not be a MAAS object)
<jtv> Was a bit shocked to find that numbers can parse to float64 *or* int, depending.
<jam> rogpeppe: should 'rpc' be in juju-core ? it feels like something that should be an external lib.
<jam> (with all the dep issues that entails, maybe not worth it yet, I guess)
<rogpeppe> jam: quite possibly. i think i'm in favour of leaving it in juju for the time being, as it may become more juju-specific.
<rogpeppe> jam: we can easily factor it out later (and i'm in favour of doing that when it stabilises)
<rogpeppe> TheMue: "
<rogpeppe> Just to get it right, the coverage of the API is defined by the types and their
<rogpeppe> methods here in the package, but those are using the generic way interact with
<rogpeppe> state?
<rogpeppe> "
<rogpeppe> TheMue: i'm not sure i understand your question there
<TheMue> rogpeppe: Just wanted to know if the type and methods the API exposed have to be defined in the API package like you've done it for Machine.InstanceId().
<rogpeppe> TheMue: yes. that's how all our Go-implemented clients and agents will interact with the state
<TheMue> rogpeppe: Fine.
<TheMue> rogpeppe: So more control than exposing all public state types and methods.
<rogpeppe> TheMue: yes
<TheMue> rogpeppe: +1
<TheMue> rogpeppe: Like it.
<TheMue> rogpeppe: Btw, our client runs fine on OS X. :D
<rogpeppe> TheMue: of course, external clients will speak JSON directly to the API server.
<rogpeppe> TheMue: cool.
<rogpeppe> TheMue: you ran the live tests?
<TheMue> rogpeppe: First step has been just to bootstrap. Next step are the live tests.
<rogpeppe> TheMue: ah, you bootstrapped ok - that's good enough for me :-)
<TheMue> rogpeppe: *lol*
<rogpeppe> TheMue: and juju status worked ok, presumably?
<dimitern> we may have a problem with openstack bootstrap, it seems the user_data we're passing is limited to 65K base64 encoded string
<TheMue> rogpeppe: Exactly
<dimitern> and ours is slightly over 80K
<rogpeppe> dimitern: compressed?
<dimitern> rogpeppe: no, it's just b64-ed
<rogpeppe> dimitern: that's huge, BTW. why is it so big?
<dimitern> rogpeppe: I don't know but I intend to find out shortly
<rogpeppe> dimitern: ec2 sends the user data compressed, BTW
<rogpeppe> dimitern: i mean the ec2 provider sends...
<dimitern> rogpeppe: nova does not support this, looking at the source
<rogpeppe> dimitern: the limit for ec2 user data is 16K and we fit in that alright. 80K is huge!
<dimitern> rogpeppe: it's either improperly encoded or it's repeated or something
<rogpeppe> dimitern: does nova need to know about the compression? i thought it was a cloud-init thing.
<dimitern> rogpeppe: not that I can see - compression is neither mentioned in the API/docs nor I can see it being handled at the nova source level
<rogpeppe> dimitern: https://help.ubuntu.com/community/CloudInit
<rogpeppe> dimitern: "content found to be gzip compressed will be uncompressed. The uncompressed data will then be used as if it were not compressed. "
<dimitern> rogpeppe: that's what I have http://paste.ubuntu.com/1589183/
<dimitern> rogpeppe: I see - so nova doesn't have to handle the decompression - cloud init will do that itself?
<rogpeppe> dimitern: yeah
<dimitern> rogpeppe: so that paste looks not right? it should be gz-ed
<dimitern> rogpeppe: which module should I use? compress/zlib?
<rogpeppe> dimitern: check out the code in the ec2 provider
<jam> dimitern: zlib != gz, though they are similar
<rogpeppe> dimitern: there's already a convenience function to do it
<jam> (both are DEFLATE, but gz has different headers)
<dimitern> rogpeppe: can you point me where?
<rogpeppe> dimitern: (in trivial)
<jam> not to be confused with mgz, who has no headers :)
<dimitern> :D
<rogpeppe> dimitern: in the userData function in ec2.go
<dimitern> rogpeppe: ah, ok
<rogpeppe> dimitern: 	cdata := trivial.Gzip(data)
<mgz> I have a head, though...
<dimitern> rogpeppe: yeah, this is removed in OS when porting the ec2 stuff .. weird
<jam> mgz: sometimes at least. :)
<dimitern> so, with the gzip part done, now it seems we need to set a public IP as well, otherwise you cannot connect to mongo, even with the sshebang VPN stuff for canonistack, and nova ssh machine-0 complains no public IPs are set
<dimitern> and I cannot ssh directly, because the authorized keys expect my key pair to be on chinstrap as well
<dimitern> and I cannot connect to the mongo on the bootstrap node at all - with either private canonistack IP or with an attached floating IP (public), even though I can see the secgroup allows port 37017
<dimitern> should I be able?
<jam> dimitern: so is it trying to ssh to the machine, or is it trying to connect directly to mongo?
<jam> if it is a direct connect to mongo, then you would need an ssh tunnel to it, or as you mention we need a public ip set for the machine.
<dimitern> jam: directly on port 37017
<dimitern> jam: I have a public IP set and through it it doesn't work (drops with conn refused, rather than blocking forever, when using a private IP)
<dimitern> jam: and I cannot seem to find a way to ssh into the machine to see what's running there (public key auth fails - used several ways to connect - with the sshebang, directly with -i ~/.ssh/mykey, etc.)
<mgz> you can also just ssh to chinstrap, then use a 10. address from there to test things sometimes
<mgz> it's likely the cloud-init was bugged though
<dimitern> mgz: tried that as well - still pubkey auth failed
<mgz> dimitern: what you need to do is look at the boottime console output
<dimitern> mgz: how? juju bootstrap is doing all
<mgz> which will tell you if injecting your public key failed somehow
<jam> dimitern: I'm pretty sure you can use a different ssh key to chinstrap than you use to the actual bootstrap node, but it might involve a bit of ssh config magic.
<dimitern> mgz: if I use lpsetup to launch a machine - it all works and at the end I'm in a ssh session
<mgz> dimitern: eg by `euca-describe-instances` to get the i-000000 id form
<mgz> then `euca-get-console-outp... something like that, don't remember the form
<mgz> anyway, pipe that to file and pastebin it
<mgz> novaclient should actually expose the console output stuff now as well
<dimitern> mgz: ahaa! 10x
<jam> mgz: nova list works for me
<jam> it reports the ip address of the target machine
<dimitern> mgz: http://paste.ubuntu.com/1589253/
<dimitern> nova list works for me as well, but nova ssh says "no public address set", which is wrong - there is a fip set
<mgz> jam: as in, what dimitern actually needs is the bootlog
<mgz> the eucatools list was just to get the other id form to use with the console output command
<jam> dimitern: well, I don't use a public IP to ssh to
<dimitern> mgz: that's troublesome - 2013-01-30 09:54:04,352 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'H4sIAAAJbogA/+y9+ZOq2rIn...'
<mgz> dimitern: ^right, that
<jam> dimitern: I use "nova list" which gives: in the last column "Networks" which has "canonistack:10.55.60.XX" and I just "ssh 10.55.60.XX"
<mgz> I'm *so* glad I added that log line to cloud-init :)
<dimitern> mgz: yay :)
<mgz> you appear to have done base64 then gzip?
<mgz> it should be gzip then base64
<dimitern> jam: using ssh 10.55.xx does not work either (pubkey auth error)
<jam> dimitern: um, I'm able to get to: 10.55.60.246
<dimitern> mgz: thought just that :) I'm looking
<jam> which is the one listed in your log
<mgz> once you ungzip, the first bit of literal text must be #cloud-config
<dimitern> jam: me too
<jam> sorry, bad paste
<jam> I get perm denied going to .246
<dimitern> jam: exactly
<mgz> jam: when cloud-init doesn't like the user data you pass (which includes the key to inject), you can't ssh to the machine in the normal way
<dimitern> jam: used to work before - in fact for things lpsetup created it works ok
<jam> mgz: sure, I imagine the key to use is in the cloud-init, and if that isn't there, bad news bears :)
<jam> mgz: any way to get gzip to decompress even when the data isn't complete?
<mgz> it's there, but can't read the user-data dimitern passed, because it comes out base64ed still
<mgz> jam: not really.
<jam> mgz: http://www.urbanophile.com/arenn/coding/gzrt/gzrt.html :)
<jam> but that might still be block-sized
<mgz> well, it can be done, but cloud-init has no facilities to do it
<mgz> and you can't get to the box to fiddle yourself
<jam> mgz: oh, I just want to see the decompression of the base64 stuff, to see if it makes any sense.
<jam> dimitern: btw, you might want to use pastebin.canonical.com for stuff that might contain things like private keys
<jam> p.u.c is fully public
<mgz> cloud-init dump is generally safe, but yeah
<dimitern> jam: ah, ok, will do
<dimitern> jam: luckily, because of the bug no keys were actually of importance there
<jam> mgz: well, the 24 base-64 bytes is not enough to get any data out of gz
<mgz> yeah, is a gzip stream though
<jam> mgz: well gzip("#cloud-config").encode('base64") => H8sIAAAA... which is the same as  the paste bin. However I imagine that is the gzip header, and not the actual compressed contenct.
<jam> mgz: well, you can't take out arbitrary bytes in the middle, but if you get enough of the prefix, you should be able to get something out, I imagine.
<jam> H4s
<mgz> well, we have the length
<jam> dimitern: one option is to dump the content (log it) in the client, to see what it thinks it is sending. And you can at least check that the prefix matches.
<jam> mgz: the error message sounds like it wants multipart sections
<mgz> ah, not we don't
<mgz> jam: I'm just checking with cloud-init, but I suspect over-endoing
<jam> mgz: but what dimitern posted before: http://paste.ubuntu.com/1589183/
<mgz> *encoding
<dimitern> jam, mgz: so I did bootstrap and logged, took the console log as well
<jam> mgz: I wonder if it was "base64(gzip(base64(content)))" or something like that
<dimitern> it seems the client is sending b64(gzip(cloud_init)) and it's exactly as it is - correctly
<dimitern> but yeah - I did the gzip(header) -> base64 and it's H4S.. something, so indeed another b64 is happening
<jam> dimitern: gzip("").encode('base64') == H4sIAAAA
<jam> so that is just the "I'm a gzip stream" bit
<jam> not the "and the content looks like #cloud-init" stuff.
<dimitern> it seems the data is not decoded on the other side
<jam> dimitern: so I'm wondering about the base64 nature
<jam> mgz: is there a reasonable way to get the cloud-init content out of pyjuju?
<jam> to compare?
<mgz> yeah, but it's quite different
<mgz> I don't think they even gzip it
<dimitern> where's the cloud-init decoding the stuff?
<dimitern> mgz: for ec2 rogpeppe said they gzip it, because the limit is 16K (in Os is 65K, ours uncompressed was 80K so bootstrap was failing with 500, and I added the gzip step as in ec2)
<jam> mgz: help.ubuntu.com indicates that we likely need to because of space limitations.
<jam> dimitern: I wonder if it could even be a "#cloud-config" with "\r\n" or some sort of line-ending/whitespace issue.
<mgz> yeah, it's not gziped.
<dimitern> jam: the code to produce it is in userData() method in environ - cloudinit.Render()
<dimitern> append([]byte("#cloud-config\n"), data...)
<dimitern> so it's \n and whatever's there
<mgz> so, wrapping it in mime junk would probably work, though it should function like this too, just trying to understand the handler code in cloud-init again
<mgz> this... must be base64 over-encoding
<jam> mgz: well, "Content-Type: text/cloud-config"
<mgz> what cloud-init is seeing is base64 encoded
<jam> https://help.ubuntu.com/community/CloudInit
<dimitern> mgz: exactly
<mgz> generally, what you do is base64 to pass in json over the apu
<jam> doesn't say anything about base64 that I've found
<dimitern> mgz: what's in the cloudinit is yaml
<mgz> then when you request the user-data from the metadata service, it's not base64 encoded
<mgz> so, it's that stage, rather than anything gzip or cloud-config related
<mgz> it never gets that far
<mgz> dimitern: what's the branch with this code?
<mgz> we're just base64ing too many times
<dimitern> mgz: cloud-init in juju-core - that's what I'm looking at
<dimitern> mgz: openstack api requires user_data to be base64 encoded
<mgz> dimitern: goose also base64s
<dimitern> mgz: only goose does, juju-core doesn't touch the gzipped yaml
<rogpeppe> fwereade: https://codereview.appspot.com/6878052 :-)
<fwereade> rogpeppe, cheers ;)
<mgz> wow, cloudinit.verifyConfig is the stupidest function I've seen in a while
<dimitern> mgz: but somehow the cloudinit doesn't expect base64-er userdata (with gzip inside)
<mgz> it shouldn't come off the metadata service encoded
<mgz> and doesn't
<dimitern> how is it taken from the metadata?
<mgz> we could reproduce more closely what juju-core is doing, and in a safe way where ssh will still work, with lpsetup
<jam> dimitern: according to http://docs.openstack.org/trunk/openstack-compute/admin/content/user-data.html it is a web request
<mgz> but I'm pretty sure it's not openstack at fault
<jam> mgz: is it possible to manually make a metadata request to verify what the instance got?
<mgz> yeah, curl the magic url
<mgz> but you need ssh to do that :)
<jam> mgz: what is the magic url?
<jam> openstack/date/user_data doesn't quite make sense
<jam> or is the 169.254 actually the magic IP?
<mgz> it is.
<rogpeppe> mgz: cloudinit.verifyConfig pretty closely from the pyjuju code.
<rogpeppe> s/pretty/came pretty/
<jam> mgz: so you have to do it from the instance? (it uses source address to determine what content to serve?)
<mgz> yup.
<jam> mgz: booo
<dimitern> i'm starting a node now with lpsetup to see what the metadata can be
<jam> dimitern: interestingly, openstack's metadata is in yaml, but ec2's is in json
<dimitern> jam: yeah, because juju-core uses yaml for both
<jam> dimitern: *might* be a problem
<mgz> dimitern: we can also hack around the ssh problem in juju-core
<mgz> if you pass key_name=dimitern and have some keys installed
<mgz> it won't matter if cloud-init gets borked data
<dimitern> mgz: keys installed where?
<mgz> `nova keypair-list`
<mgz> I'll double check the exact param as well
<rogpeppe> jam: i didn't think that ec2 used json at all
<jam> mgz: from what I can find, pyjuju does not gzip the content.
<mgz> jam: indeed, I said that ^up there :)
<jam> rogpeppe: http://docs.openstack.org/trunk/openstack-compute/admin/content/metadata-service.html seems to indicate that if you get data from the metadata api in ec2 compat mode, it comes back as json
<rogpeppe> jam: we only need to gzip the content now because we're passing around big certificates and keys
<jam> ah, there is 'metadata' and 'metadata.json' perhaps
<jam> rogpeppe: sure, but for sanity checking, I might recommend dimitern not gzip until we get something working in case the decode step isn't working.
<rogpeppe> jam: ah, i thought you were referring to goamz/ec2 or environs/ec2, neither of which use json AFAIK
<rogpeppe> jam: perhaps he could try a little standalone program
<jam> rogpeppe: it is a bit unclear what the metadata service actually does, as I haven't studied the document and it goes back and forth between JSON and yaml.
<jam> It might depend on what specific URL is accessed.
<jam> dimitern: so my immediate hack would be to take out gzip (either everywhere or just for openstack), and then take out the certs, just to see if the instance node comes up
<jam> also, if you push your code, we can probably try to reproduce your findings.
<mgz> dimitern: okay, add the following param server dict in goose: 'key_name': $OS_USERNAME
<mgz> then make sure you have a key under that name (you should anyway)
<rogpeppe> dimitern: you could try reducing KeyBits in juju-core/cert to a small number, say 128
<mgz> then is you bootstrap with the borked cloudinit
<rogpeppe> dimitern: then the cloudinit will be significantly smaller
<mgz> you'll be able to ssh in anyway, and then curl down the exact data the metadata service has
<rogpeppe> dimitern: it would be interesting to see if it still failed then
<dimitern> ok, one by one :)
<dimitern> rogpeppe: I'll try reducing the keybits first
<dimitern> jam: not sure how to remove the keys, otherwise removing gzip causes a failure on bootstrap every time
<TheMue> rogpeppe: live tests fail, looking for interesting tools: 1.9.8-unknown-amd64.
<rogpeppe> TheMue: ah of course
<TheMue> rogpeppe: not all fail, but 19
<rogpeppe> TheMue: no ubuntu series on macos
<TheMue> rogpeppe: yep, exactly
<rogpeppe> TheMue: perhaps we can't expect live tests to pass under macos
<TheMue> rogpeppe: seems so
<rogpeppe> TheMue: unless we have an option to disable tools upload for the live tests, which might be a reasonable way to go
<niemeyer> Hello!
<mgz> dimitern: http://paste.ubuntu.com/1589367/
<dimitern> mgz: I'll try that
<TheMue> niemeyer: Hiya
<dimitern> otherwise, reducing the keybits to 128 reduced the cloudinit almost in half, but still 48K uncompressed, so removing gzip doesn't help unless I somehow strip the keys
<dimitern> mgz: adding the key name didn't do the trick - still cannot connect with my key
<wallyworld> mgz: dimitern: jam: standup?
<mgz> mumble has decided it likes hanging...
<niemeyer> mgz: I'd be more surprised if Hangout decided it likes to mumble
<jam> niemeyer: I hear mumbling all the time on hangouts :)
<niemeyer> jam: :)
<dimitern> mgz: did it work for you with key_name?
<rogpeppe> dimitern: why don't you try making a little standalone program that starts an instance using nova and uses a custom-created cloudinit (you could still use juju-core/cloudinit) ?
<rogpeppe> dimitern: then it'll be easier to check without all the juju baggage
<dimitern> rogpeppe: i'm using lpsetup to start an instance with nova client and it's all ok
<dimitern> rogpeppe: the problem is I cannot find any way to connect to the borked cloudinit instance
<rogpeppe> dimitern: in that case, try it with gzipped user data
<dimitern> rogpeppe: without gzip bootstrap fails with 500 (too long)
<rogpeppe> dimitern: i mean try your standalone thingy with gzipped user data
<dimitern> rogpeppe: can I somehow strip the ca certs from the cloud init?
<rogpeppe> dimitern: sure, just hack the cloudinit package, it's trivial
<dimitern> rogpeppe: it's just marshaling into yaml the cfg
<rogpeppe> dimitern: make it so it doesn't use the same certs
<dimitern> rogpeppe: i'm not even sure which keys are the biggest
<rogpeppe> dimitern: just zero 'em all out
<dimitern> rogpeppe: which keys?
<rogpeppe> dimitern: something like this should do the job (untested): http://paste.ubuntu.com/1589462/
<dimitern> rogpeppe: will state work with after this?
<rogpeppe> dimitern: no, but the instance will be started ok
<rogpeppe> dimitern: so you should be able to ssh to it
<dimitern> rogpeppe: ok, I'll try it
<dimitern> mgz: http://paste.ubuntu.com/1589468/
<mgz> so, cuter magic on metadata service folsome(?) or later: http://169.254.169.254/openstack/2012-08-10/meta_data.json
<dimitern> rogpeppe: when I set these to nil it says error: cannot start bootstrap instance: cannot make user data: invalid machine configuration: missing CA certificate
<rogpeppe> dimitern: did you use the snippet i suggested?
<mgz> dimitern: so, this is double-base64 encoded
<rogpeppe> dimitern: are you base64 encoding inside environs/openstack ?
<mgz> rogpeppe: yes.
<mgz> well, no
<mgz> but... it's happening *somewhere* :)
<rogpeppe> ha
<dimitern> rogpeppe: only goose encodes to base64
<dimitern> wait a minute - there are 2 cloudinits - in juju-core/ and in juju-core/environs/ - the provider uses the latter
<dimitern> ...which uses the former actually
<dimitern> and there's this (line 151 in environs/cloudinit.go): 				" --env-config "+shquote(base64yaml(cfg.Config))+
<mgz> ...which is fine
<mgz> dimitern: are you using Ian's RunServer-userData-fix branch?
<dimitern> mgz: don't think so
<mgz> because this looks like something daft like the go json code being helpful by base64ing byte[] behind the scenes, after it's already been done
<dimitern> mgz: looks like if I remove the b64 encoding in runserver it works ok
<dimitern> mgz: could be! yeah - []byte to "64"
<mgz> right, so the right fix is... remove 4 lines...
<dimitern> mgz: i did, but still not sure if it's right and we should fix something else instead
<mgz> jam, dimitern: http://paste.ubuntu.com/1589539/
<dimitern> yay! ssh now works!
<dimitern> exactly
<dimitern> I'm doing that in the CL then
<dimitern> a separate one for goose
<mgz> so, Ian's mp which switches the type to string likely works as well
<dimitern> where's that branch?
<mgz> dimitern: lp:~wallyworld/goose/RunServer-userData-fix
<mgz> it is documented behaviour of the json go implementation though, so the use []byte and remove those lines versions is probably nicer
<dimitern> mgz: from go docs: Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON object.
<dimitern> :) yeah
<dimitern> mgz: actually, without these 4 lines ian's branch is not needed
<mgz> right. but Ian wasn't wrong when he said he'd got it working
<dimitern> yes
<mgz> we then just ignored him, so it broke for us :)
<dimitern> :) maybe I should NOT LGTM and provide that above reasons?
<dimitern> if in addition he did the gzip thing, his change would work
<mgz> well, I think I'll put up a trivial branch that actually adds a test and removes those lines
<dimitern> cool, if you can land it before mine it'll be perfect
<mgz> then we can reject the one that changes the RunServer interface, and update the enable openstack branch
<dimitern> yes, I also need that change
<dimitern> shouldn't mongo be running on the bootstrap node?
<dimitern> rogpeppe: ^^ ?
<rogpeppe> dimitern: yes, mongo should run on the bootstrap node, but it won't if you didn't have the server cert and key
<dimitern> I have this in /var/log/juju/machine-0.log: /bin/sh: 1: exec: /var/lib/juju/tools/machine-0/jujud: not found
<dimitern> (repeated muliple times)
<dimitern> rogpeppe: no, it's ok now - the userdata with all keys is passed ok and cloud init seems to work
<dimitern> rogpeppe: I'd sshed into the machine and looking - no mongo though
<rogpeppe> dimitern: could you paste the cloud-init-output.log file from /var/log ?
<dimitern> rogpeppe: https://pastebin.canonical.com/83392/
<rogpeppe> dimitern: the key is probably the "authorization failed" error
<dimitern> rogpeppe: it seems the tools tgz is somehow broken?
<rogpeppe> dimitern: the url tools download is failing
<dimitern> rogpeppe: yeah, looking at that
<rogpeppe> dimitern: when you're on the machine, try downloading that url in the same way
<dimitern> ok
<rogpeppe> dimitern: that is, try curl http://juju-dist.s3.amazonaws.com/tools/mongo-2.2.0-quantal-amd64.tgz or whatever
<dimitern> rogpeppe: it gets a bunch of binary back
<rogpeppe> dimitern: hmm. try putting it into /tmp and run tar tzf on it
<rogpeppe> dimitern: to see if it actually *does* download ok
<dimitern> rogpeppe: I got it with wget (41M), untarred in /tmp and it seems ok
<dimitern> the problem seems before that - tools is not extracted properly and jujud is missing
<rogpeppe> dimitern: strange. i'd add some echo statements in the cloudinit (including the args passed to wget); and maybe make it put the archive in a temp file before unarchiving it, so you can see what was fetched
<rogpeppe> i've got a call for lunch
<rogpeppe> dimitern: back shortly
<dimitern> rogpeppe: I got it working - the problem was ACLs on the bucket - the machine was not able to get the tools
<rogpeppe> dimitern: cool
<dimitern> rogpeppe: but mongo is still not running
<rogpeppe> dimitern: anything useful in the log?
<dimitern> /var/lib/juju/tools/1.9.8-quantal-amd64/jujud: 1: /var/lib/juju/tools/1.9.8-quantal-amd64/jujud: Syntax error: "(" unexpected
<dimitern> and trying to run root@machine-0:/var/lib/juju/tools/machine-0# ./jujud gives "-su: ./jujud: cannot execute binary file"
<dimitern> rogpeppe: ^^
<rogpeppe> dimitern: what does "file /var/lib/juju/tools/machine-0/*" say?
<dimitern> rogpeppe: the same line (first one) - multiple times
<rogpeppe> dimitern: which line?
<dimitern> /var/lib/juju/tools/1.9.8-quantal-amd64/jujud: 1: /var/lib/juju/tools/1.9.8-quantal-amd64/jujud: Syntax error: "(" unexpected
<dimitern> rogpeppe: and I still get "-su: /var/lib/juju/tools/machine-0/jujud: cannot execute binary file" when I try to run jujud in the tools dir
<rogpeppe> dimitern: surely you didn't get the above line when running the file command?
<dimitern> rogpeppe: what file command? I did sudo su - before to get root
<rogpeppe> dimitern: the command i suggested you run: "file /var/lib/juju/tools/machine-0/*"
<dimitern> rogpeppe: with sudo bash the error on ./jujud is a bit different: bash: ./jujud: cannot execute binary file
<dimitern> rogpeppe: aah file
<dimitern> jujud:              ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), stripped
<dimitern> rogpeppe: how can I check what arch is running? 32/64?
<dimitern> rogpeppe: ah, I did ldd `which bash` and it seems this is a 32bit machine, not 64 bit
<rogpeppe>  dimitern: uname -a
<rogpeppe> dimitern: that could possibly have something to do with it :-)
<rogpeppe> dimitern: i'm surprised you don't get a better error message though
<dimitern> rogpeppe: uname -a does not mention 64 bit
<rogpeppe> dimitern: althugh i suppose it's the usual unix errno limitations
<dimitern> rogpeppe: could be - so the tools are wrong.. or the image possibly
<rogpeppe> dimitern: the image is wrong
<rogpeppe> dimitern: we don't currently push 32 bit tools
<rogpeppe> dimitern: although... have you tried bootstrap --upload-tools ?
<dimitern> rogpeppe: yes, nailed it then - I'll set the correct default image id
<dimitern> dimitern: yes, that's how I do it - my machine is also 64bit
<dimitern> so I should use "smoser-cloud-images/ubuntu-quantal-12.10-amd64-server-20121218" instead of "smoser-cloud-images/ubuntu-quantal-12.10-i386-server-20121017"
<mgz> haiup.
<dimitern> now with the correct image - eeeeverything works :) bootstrap, status, mongo, ssh - it's a charm
<dimitern> still cannot connect to the state from the private ip though
<mgz> you got a charm working too? :)
<mgz> charming!
<dimitern> mgz: not yet :)
<mgz> charmation...
<dimitern> actually, it doesn't work after connecting to state - it seems it's using AMI-like instance ids, rather than openstack ones
<mgz> right, that's the bug I mentioned earlier
<mgz> in the provider:
<mgz> InstanceIdAccessor: "$(curl http://169.254.169.254/1.0/meta-data/instance-id)",
<mgz> wrong.
<mgz> compare with the pyjuju code.
<mgz> for folsom, you have a shortcut (sort of, still have icky shell goop to deal with)
<mgz> using the other metadata service url I pasted ^up there
<mgz> shell isn't great for parsing json :)
<mgz> though I'm sure smoser would manage it somehow
<dimitern> mgz: so?
<mgz> the id is wrong
<dimitern> yes I see that :)
<mgz> we're doing openstack things, we need the UUID, not the ec2-style
<dimitern> but is there a workaround to get the instance right
<mgz> if you use the url^^ up there and parse the json, yes
<mgz> or the old hack (which we should put off worrying about till later, now canonistack has been upgraded)
<mgz> http://169.254.169.254/openstack/2012-08-10/meta_data.json
<dimitern> I'm still confused - how does it get to use that ami-comp. id?
<dimitern> it's not in the provider
<mgz> ...you want the long explaination?
<mgz> so, the bootstrap node (or whatever we should be calling it these days) wants to know it's own id
<dimitern> ok..
<mgz> when we create the user-data, which is the main way of doing customisation on a new instance
<mgz> we don't know the id yet, because we've not created the instance
<dimitern> yes
<mgz> so, instead what gets passed in is a snippet of shell, that when run during machine setup, resolves to the id
<mgz> this is icky
<dimitern> yes, but it seems the only way to go
<mgz> but basically works, because you can get the info from the metadata service
<dimitern> unless we change the metadata right after we have the id again
<mgz> you can't
<fwereade> mgz, dimitern: so long as the metadata service is working at that moment, anyway
<mgz> you could do something else, not involving the metadata service though
<dimitern> fwereade: isn't the metadata svc external to the machine?
<fwereade> dimitern: I have seen it fail
<mgz> because it needed to work on essex, what pyjuju does for openstack
<mgz> is stick the id of the bootstrap node in a well-known location on swift
<mgz> (or nova-storage, for openstack_s3)
<mgz> the machine could also, really, ask the api for the id itself
<fwereade> dimitern: very rarely, and it may be rare enough that we don't care, but it would be nice to be more robust if there's an opportunityt
<mgz> but the way this stuff is layed out currently, that's not in the right order
<dimitern> fwereade: no, the svc is working now
<mgz> anyway, for folsom or later, you can get the openstack-style server uuid from the metadata service, from a different url that returns json with useful stuff
<dimitern> mgz: so we should use some other metadata key to get the OS inst Id
<mgz> right, which is the link I pasted
<dimitern> mgz: but it'll not work for essex
<mgz> indeed, but we don't care about that for now.
<dimitern> mgz: ok then, I'll look into it
<mgz> and it would be nicer to fix the mess of shell at the same time, later
<dimitern> mgz: and that should happen in userData() ?
<mgz> dimitern: this is violently horrible, but bascially all you need:
<mgz> -               InstanceIdAccessor: "$(curl http://169.254.169.254/1.0/meta-data/instance-id)",
<mgz> +               InstanceIdAccessor: "$(curl http://169.254.169.254/openstack/2012-08-10/meta_data.json|python -c \"import json,sys;print json.loads(sys.stdin.read())['uuid'])\"",
<mgz> fixing the model would be nice, but as a just-get-things-working step...
 * niemeyer => lunch
<dimitern> mgz: it works with the fix!
<dimitern> mgz, rogpeppe, jam: it's ready https://codereview.appspot.com/7223059
<TheRealMue> dimitern: You've got a review.
<dimitern> TheRealMue: thanks
<mgz> douple
<dimitern> also this https://codereview.appspot.com/7229060
<dimitern> mgz: cheers
<dimitern> mgz: care to take a look again at the comments? https://codereview.appspot.com/7223059/
<dimitern> TheMue: 10x
<mgz> dimitern: what does %q do when you give it a bunch of binary junk?
<dimitern> mgz: what %s does, quoted
<mgz> print the whole []byte contents in some form?
<mgz> I'd lose that much at least, even if you keep the log statement for the length
<dimitern> mgz: in string form, just like %s -> string([]byte) does
<mgz> we don't want the cloud-init stuff appearing in the logs in any form, as it contains certs and such like
<dimitern> mgz: ok, I'll remove the %q there (it's printed with --debug anyway)
<dimitern> mgz: it won't appear in any logs, just on the console, and anyway - the certs are encoded
<mgz> my first step when people have juju issues is ask them to run with as much logging as possible then pastebinit :)
<dimitern> also, we seem to need floating ip attached to be able to connect to the state/0 node, even with the sshebang stuff for canonistack, simple connections won't work unless tunneled
<mgz> right, that's from rog's changes to how shizzle works, which I didn't really follow
<dimitern> mgz: done that
<dimitern> mgz: and how about the goose CL ? is it LGTM?
<mgz> ...I said land, but does need to go with the goose change...
<mgz> and really that does need a test
<dimitern> mgz: ok :)
<mgz> which I did say I would write, but then got distracted with other things
<dimitern> mgz: even without the goose change, juju won't break - at least until you try to bootstrap an OS env
<mgz> and the tests pass?
<mgz> or there aren't enabled bootstrap tests that care yet?
<dimitern> mgz: the tests pass
<mgz> in which case, go ahead and land
<dimitern> mgz: he problem come after launching the machine
<mgz> ...and then work on a test that would actually have failed :D
<dimitern> which the doubles do not do
<dimitern> ..really
<dimitern> what should that test be? check the output of RunServerOpts serialization?
<mgz> I mean on the juju-core side
<mgz> there are some more live ones we have skipped, that actually depend on a running bootstrap node, right?
<dimitern> yes, some of them - these which require provisioner and state connection
<mgz> and yeah, we don't have any testing a the right level for this in goose, which is painful, because pyjuju has a ton
<dimitern> mgz: so should I land the goose CL then?
<mgz> I'd suggest working on whatever remaining things are borked with them then, if nessersary put up a mp that auto-assigns a floating ip for the bootstrap node if that's a dependency
<dimitern> mgz: that will be needed yes - the floating ip for node -
<mgz> dimitern: I guess, and reject Ian's one that does the same thing
<dimitern> s/node/machine-0/
<mgz> I hate the fact our testing has worse coverage and isn't as robust as the (notably flakey) python juju testing, but we'll get there... I hope...
<dimitern> I think he should reject it - I'm not sure I can
<dimitern> :) yeah, we'll get there.. in 2 months at most :)
<mgz> you can antilgtm and note we've landed an alternative in the comment at least
<dimitern> mgz: I did that
<dimitern> rogpeppe: can you reject and delete this branch pls - I remember it was just a test.. lp:~rogpeppe/goose/client-using-identity
<TheMue> so, have to step out for dinner. will be back later.
<rogpeppe> fwereade: i've updated the rpc CL taking account of your comments about Machine: https://codereview.appspot.com/6878052
<rogpeppe> time to stop for the day
<fwereade> rogpeppe, sweet, tyvm, I will try to look at those when laura is settled... this is not imminent I'm afraid
<rogpeppe> fwereade: :-)
<fwereade> rogpeppe, enjoy your evening :)
<rogpeppe> fwereade: and you.
<rogpeppe> g'night all
<niemeyer> rogpeppe: Night!
<niemeyer> and I think I'm done with multipart uploads
#juju-dev 2013-01-31
<arbit30719> can some body help me solve juju and MAAS problem : I have also posted on ask ubuntu  http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas
<arbit30719> hello ?
<arbit30719> can some body help me solve juju and MAAS problem : I have also posted on ask ubuntu  http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas
<bigjools> arbit30719: most of the devs are in Europe and USA
<arbit30719> I see  :P
<arbit30719> hi all
<jtv> jam: Why do we create one global instance of each provider, but *register* a different instance?
<jam> jtv: well for ec2 'environProvider' has no state (struct{}) so the concept of an instance of it is a bit weak.
<jam> I think the important part is to register that the instance exists.
<jam> I see your point, though.
<jam> It does look like
<jam> environs.RegisterProvider("ec2", providerInstance)
<jam> would make more sense.
<jam> It works because there is no state, and all the functions just exist to create the actual environ where work is done.
<jtv> Thanks!  I was wondering if there might be externalized data associated with pointers-to-providers too.
<jam> jtv: I imagine the openstack code just copied the ec2 code.
<jtv> Yeah...  I'm not reading too much into the "everyone does it" for that reason.  :)
<jam> jtv: yep. We inherited a fair amount of stuff I suspect from copying it from ec2
<jam> It would make sense to me to have a patch that registered the global variable
<jam> otherwise, why have that var?
<jtv> Maybe as a prototype..?
<jam> well it is used in functions
<jam> SetConfig() calls providerInstance.newConfig() etc.
<jtv> Yeah.  It looks a lot as if the thing has neither state nor identity...
<jam> so *if we get state* I think we have to use the global variable in the registration.
<jtv> That does sound sane, yes.
<jam> jtv: I imagine there is a discrete instance, (you can probably take the address of it)
<jam> &providerInstance
<jam> but it is not actually used
<jam> I think the type is there to collect methods
<jam> and you can't just call Type.method()
<jam> so you have to somewhere generate an instance of it
 * jam always doubts if global state is correct, though.
<jtv> Well if there's no state...  :)
<jtv> It looks as if a singleton was intended.
<jam> jtv: then you should be using a local var
<jam> ?
<jam> to make it clear
<jam> jtv: yeah, it does look like someone was trying for a singleton pattern.
<arbit30719> hi all
<jtv> A local var would be nice, but puts it out of reach of the code that needs the singleton.
<jtv> Hello arbit30719
<arbit30719> can some body help me solve juju and MAAS problem : I have also posted on ask ubuntu  http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas
<arbit30719> =)
<jam> arbit30719: from everything I can see "agent-state: not started" means that the juju control isn't established on your machine 1, and from the traceback, it appears it will never start.
<jam> I would say kill it (juju destroy-service mysql, juju delete 1)
<jam> and maybe try again.
<jam> I'm not sure why it is having trouble getting a zookeeper connection, though.
<arbit30719> I tried several times and failed
<jam> give "juju status" indicating the root machine is working.
<arbit30719> I am pretty sure root machine works fine between machine 0 and maas server
<jam> jtv: where is the state stored in maas? Is it reasonable to try and query that for what the saved 'zookeeper' information is, to try a manual connection
<jam> ?
<jtv> bigjools worked on that code â he may know
<arbit30719> ?
<bigjools> this is a juju bug as I recall
<bigjools> something to do with ZK
<bigjools> but I can't remember what causes it or what the resolution is
<bigjools> perhaps update to the latest version from the PPA
<jam> bigjools: does maas do firewalling? Is it possible juju is trying to connect to ZK on a port that isn't open?
<bigjools> it doesn't
<arbit30719> I also tried disable firewall and failed...
<bigjools> I remember this bug pretty well, just not well enough to know what the problem was :)
<bigjools> I encountered it on my own systems
<bigjools> arbit30719: what version of juju are you using?
<arbit30719> 0.6
<bigjools> is that from precise/quantal?
<arbit30719> quantal
<bigjools> can you try the official PPA
<arbit30719> you mean ?
<bigjools> also put juju-origin: ppa in the config
<arbit30719> I think MAAS in quantal is much easier to setup
<bigjools> it will get even better with the SRU that's coming
<arbit30719> yes I do put pap in environment.yaml
<arbit30719> SRU ?
<arbit30719> typo   pap  -> ppa
<bigjools> ok
<bigjools> not sure where else to go
<bigjools> I expect one of the core devs will remember when they start (in ~2.5 hours)
<arbit30719> um...
<arbit30719> ok
<bigjools> arbit30719: ah I have an idea
<bigjools> can machine 1 resolve the machine 0's DNS address?
<bigjools> the name of the machine that you use in MAAS needs to be resolvable
<arbit30719> ha I also checked the name server (MAAS-dns)
<arbit30719> They  can resolve both 0's and controller's name
<bigjools> so ssh to machine 1
<bigjools> and do:
<bigjools> host node-4487fc70b037
<arbit30719> 192.168.1.5
<bigjools> can you do:
<bigjools> telnet node-4487fc70b037 2181
<bigjools> and see if it connects to ZK
<jam> jtv: one small thing about your patch, I'm pretty sure you want to just use "providerInstance" not "&providerInstance"
<jtv> Where exactly?
<jam> +func init() { +	environs.RegisterProvider("maas", &providerInstance) +}
<jam> I would be wrong, since that actually creates a copy of the instance
<jam> (but again, no actual state :)
<jam> but the code you are replacing was:
<jam> +func init() { +	environs.RegisterProvider("maas", environProvider{}) +}
<jam> which isn't an address.
<jtv> Yeah.  Don't we have a type system for this?
<jam> jtv: it is an interface
<jam> I imagine that both providerInstance and &providerInstance support the methods you need to be part of the interface.
<jam> EnvironProvider
<jtv> No...  only providerInstance does.
<jam> (interfaces are based on what methods are available)
<jam> jtv: if you define a method func (e environProvider) Foo()
<jam> is that also exposed on *environProvider ?
<jam> If it is, then both provide the interface
<jam> if the funcs themselves are func (e environProvider) than using the address has no effect
<jam> because a new copy will be created for each function call.
<jtv> AFAIK it is only exposed on the object itself, not the pointer.
<jtv> I think I just took the address to satisfy the type system.  Let me try.
<jam> jtv: I'm actually pretty sure it is both
<jam> there is a cast earlier that hints at it
<jtv> That would explain.  And also freak me out.
<jtv> Grr.  It works either way.
<jam> jtv: http://bazaar.launchpad.net/~gophers/juju-core/trunk/view/head:/environs/openstack/provider.go#L38
<jam> so we know that a pointer to a provider provides the interface
<jam> because we have an explicit cast
<jtv> That sort of destroys the last vestige of sense in the ec2 code, doesn't it?  Why create an instance and register its address, when registering an instance will do the same thing?
<jam> however: http://bazaar.launchpad.net/~gophers/juju-core/trunk/view/head:/environs/openstack/provider.go#L61
<jam> all of the actual methods are done based on the object itself
<jam> not its pointre
<jam> so if you define funcs on the pointer, then you must have a pointer to call the object
<jam> but if you define funcs on an object, you can have the object or a pointer to it.
<jtv> But not the other way around.  Shudder.
<jam> jtv: and when calling a func-on-object, a new actual object is created during the function call
<jam> so you cannot mutate state in one of those calls
<jam> so you must either have 0 state, or at least 0 mutable state
<jtv> Yes.
<jam> jtv: so IMO, funcs on objects don't make a lot of sense
<jam> which would then put you in a case where you have to pass the address.
<jtv> They sort of do...  but only for small, immutable objects.
<jam> but, that is the way the code exists.
<jam> jtv: without state changes, which means why is it on an object rather than a plain func.
<jam> I guess interfaces mean you have a way to group funcs.
<jtv> That's exactly what it is.
<jam> I want *this* set of vpointers
<jtv> They call them "method sets," I think.
<jtv> AFAICS the only form of dynamic dispatch in the language is that an interface is basically a tuple (vtable, object)
<jtv> An interface *value*, that is.
<jam> jtv: right
<jtv> Not an interface *definition*, obviously.
<jtv> Anyway, I wrote all that up in our Cultural Learnings document last week:
<jtv> https://docs.google.com/a/canonical.com/document/d/1GQ3u7keE3hJH2oaweAm28K78hQZAbqeQqR9xwD9OR4I/edit#
<jtv> I'll update it with the new knowledge that a pointer to an implementing object is also a valid interface value.
<jam> jtv: I'm using my canonical creds with lbox (though I do have some difficulties) it does seem to work.
<jam> (reading through your docs)
<jtv> Yeah, Gavin had the same experience.  Don't know what I did wrong.
<jam> I got a bunch of "protocol not supported" failures until it happened to work
<jam> and then it seems to generally work afterwards, though I just had propose fail again
<jtv> Weird
<jtv> Thanks for going through the document BTW.
<jam> I'm pretty sure I read it the first time you mentioned it as well.
<jam> but it is good to be reminded from time to time.
<jtv> jam: I updated my branch to register (a copy of) the provider instance, rather than its address.  For what it's worth!
<jam> jtv: I see the point either way. Having thought about it, I probably prefer the address method, but that is probably just my 'why isn't it actually an instance'
<jam> It is a method set, and instance doesn't really mean anything, but we have no way to talk about it without an instance.
<jtv> Okay, I'll undo the change you asked for.  :)
<jtv> Or do you mean you would like the *methods* implemented on the pointer?
 * jtv relocates
<rogpeppe> mornin' all
<jtv> Hi rogpeppe
<rogpeppe> jtv: hiysa
<jtv> jam: did you mean earlier that you would prefer the provider to implement its interface on the pointer only?
<fwereade> rogpeppe, heyhey
<fwereade> everyone, heyhey
<rogpeppe> jtv, jam: that looks like a bug in Register method
<rogpeppe> although it won't make any difference.
<jtv> rogpeppe: I wasn't aware, but if class T implements interface N, then *T also implements N.
<rogpeppe> jtv: yeah
<rogpeppe> jtv, jam: Environ.Provider was a latecomer to the party
<rogpeppe> jtv, jam: and if you don't have that, there's you don't need providerInstance.
<rogpeppe> in fact, you don't need providerInstance at all anyway
<rogpeppe> environs.Provider could just return environProvider{}
<jam> jtv: I would say it is "method set" vs "singleton". If you are going to have a singleton factory, then you should have the methods implemented of "*Foo" so that they all use the same instance.
<jam> If you just have a method set, then there isn't an instance to really think about
<jam> as it just gets created all the tiem.
<jam> time
<jtv> And now we're discussing how to implement a singleton.  I miss python.  :)
<jtv> If it really doesn't matter at all, I'll just go with Environ.Provider() and forget about the singleton.
<jtv> Thanks for pointing that out, rogpeppe  â it makes a few missing pieces fall into place for me.
<jam> jtv: I would probably push towards one or the other for consistency.
<jam> At this point, I don't really care which.
<jam> But we shouldn't have both.
<jtv> Makes sense.
<jam> Because it makes you say: oh we have a singleton, but wait, these are all pass by value, but wait, all the methods are also pass by value so it doesn't matter.
<jam> The last one especially requires you to actually inspect all the methods.
<jam> Rather than just being obvious from the pattern.
<rogpeppe> jtv: the reason only reason that the global var is useful is to make it more convenient to do environProvider.newConfig(environProvider{})...
<jtv> Oh
<jam> rogpeppe: doesn't that hint that 'newConfig' should just be a local func?
<rogpeppe> jam: yeah, it probably should be
<rogpeppe> jam: if it was calling one of the environProvider's exported functions, it would be right
<jam> but you can poke at private methods as long as you are in the same package, right?
<TheMue> Morning
<mgz> mornin'
<jam> morning mgz
<mgz> hey!
<rogpeppe> jam: sure, but there's no real reason for newConfig to be defined on environProvider
<dimitern> jam: hey, I didn't add a test for checking userdata encoding/etc. because I don't yet see how I can test it
<jam> dimitern: have a test that creates an object, serializes it to json, and asserts the content is base64 encoded? (and decodes to the right content?)
<jam> that would at least be something.
<jam> the bug was that we were encoding to base64, and then it was getting encoded *again* by JSON
<dimitern> jam: you think that's enough? done on RunServerOpts only?
<jam> so a test that it is encoded just 1 time would fit the bill
<jam> possibly we mighht want to refactor the code so that the part that fills out the object can be tested separately from actually making an RPC
<dimitern> yeah, but we were wrong because didn't bother to read carefully what the json module does to []byte fields..
<jam> dimitern: so I realize what the bug was, and the fix is correct. But lets add a test so that in 6 months we don't forget and wonder "why aren't we encoding this field"
<dimitern> is it useful to test that go does something it says it does?
<jam> dimitern: clearly it was non-obvious
<jam> or we wouldn't have made the mistake in the first place.
<dimitern> jam: I added a comment there in RunServer
<jam> dimitern: the fact that it confused us for roughly 2 days
<jam> means that we could have some help
<jam> I still think it is clearly untested code
<jam> or it wouldn't have broken
<dimitern> jam: ok, so should it in juju or in goose?
<jam> dimitern: seems like a goose level test to me
<dimitern> jam: ok and now with the bot - we follow the steps you outlined from now on?
<jam> dimitern: yeah, if it doesn't work, just poke me
<jam> but things should land in ~2min from when you mark it approved.
<jam> depending on how slow the juju-core test suite is.
<dimitern> jam, ok then, I'll think of a test then
<dimitern> and in juju I still have to set a FIP to the state machine when bootstrapping
<jam> dimitern: FIP ?
<jam> floating ip
<dimitern> floating ip
<fwereade> blast: I have a doctor's appointment, it might take a little while: I'll be back when I can be :/
<jam> rogpeppe: arbit30719 was asking about http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas earlier
<jam> any ideas about problems connecting to ZK when using juju w/ maas?
<jam> or other people to point him to?
<rogpeppe> jam: hmm
<rogpeppe> jam: it seems as if machine 1 can't connect to zk, even though the juju client can.
<jam> rogpeppe: right, I think bigjools said he had seen something similar in the past, but we didn't have much  more state than that.
<rogpeppe> jam: it would be nice if that exception provided more info - perhaps it got a connection refused or something
<rogpeppe> jam: and it's weird that he can't use juju ssh to connect to machine 1
<rogpeppe> jam: any of the original juju team would be better at answering this question than me :-)
<rogpeppe> jam: i think bigjools' answer is a good one
<mgz> ...who are the original juju team?
<rogpeppe> mgz: kapil, jim baker, ben howard, william, gustavo
<mgz> ah, jim was too? I don't know ben.
<rogpeppe> mgz: oops not ben howard. ben... surname escapes me
<rogpeppe> mgz: ben saller
<rogpeppe> doh
<arbit> hello i am the previos arbit30719
<jam> mgz: btw, I think your public containers stuff has landed, we should update the kanban board
<mgz> jam: good point
<mgz> I'll put up some tasks in 'coding' that aren't coding as such but could do with tracking there
<jtv> Hi there mgz â did you have any notes about the gomaasapi code?
<mgz> jtv: not yet, was hoping to look at it with Gavin today
<jtv> Ah OK.  Thanks again!
<mgz> mentioned it in the standup the other day, and rogpeppe noted it would be nice to have the whole thing up on codereview for those (gojuju people) who like to use that when reviewing things
<jtv> And jam, are you comfortable enough with my "provider singleton" branch that I can land it on our feature branch?  This one: https://code.launchpad.net/~jtv/juju-core/mpv-singleton/+merge/145777
<mgz> so, will probably put up a r1..-1 dummy merge for that
<jtv> On Rietveld?
<mgz> yup
<rogpeppe> jtv: yeah. it's really nice to be able to spray comments around in context :-)
<jtv> Looking forward to learning more.
<jtv> I imagine it is, yes.
<jtv> Twitter Review API!
<jtv> Sorry, just a weird thought.
<mgz> rogpeppe: how did we do that last time? would probably make sense for jtv to put it up rather than me
<rogpeppe> mgz: you didn't do that last time :-)
<rogpeppe> mgz: perhaps a change that changes each source file in a trivial way (e.g. adding a "// to review" comment at the start)
<rogpeppe> mgz: then propose that
<mgz> I'd do something like `bzr push -r1 lp:~/gomaasapi/base` then `~/go/bin/lbox propose -cr -v -for=lp:~/gomaasapi/base`
<mgz> which would give a diff of all the bits that matter
<rogpeppe> mgz: that's a good plan
<rogpeppe> mgz: sounds good
<rogpeppe> mgz: except that the target isn't right, but i guess that doesn't matter.
<mgz> well, the target is backwards, but it gives the right diff
<arbit> hi all, does anybody have some suggestions about the issue I pasted on ask ubuntu http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas
<arbit> It seems that I cannot connect to zk server of machine 1
<mgz> arbit: #juju is the channel for general juju questions, but you might need to wait for US timezone peeps to awaken
<mgz> or #maas in this case
<rogpeppe> arbit: have you tried bigjools' suggestion?
<arbit> yes I tried and failed to connect
<arbit> igjools told me to paste here
<rogpeppe> arbit: what did the connection failure say?
<arbit> uh I might need to reply this to you later.
<arbit> I have difficulty connect to the server right now
<mgz> jtv, rogpeppe: https://codereview.appspot.com/7228069
<jtv> Thanks
<rogpeppe> mgz: cool, thanks
<rvba> Thanks for doing this mgz.
<TheMue> lunchtime, biab
<dimitern> rogpeppe, mgz: here is another bootstrap-related fix https://codereview.appspot.com/7226068
<rogpeppe> dimitern: will have a look in a bit, thanks.
<jam> dimitern: wallyworld_: poke for the mumbling
<dimitern> jam: right there
<dimitern> jam: my mumble decided to start crashing :(
<mgz> it's like my yesterday
<dimitern> dimitern@kubrik:~$ mumble
<dimitern> Segmentation fault (core dumped)
<niemeyer> Hey all
<dimitern> niemeyer: hiya
<rogpeppe> niemeyer: yo!
<dimitern> rogpeppe: did you have time to look at the CL?
<rogpeppe> dimitern: sorry, been involved elsewhere. will look now.
<rogpeppe> dimitern: mgoPortSuffix = ":" + string(mgoPort)
<rogpeppe> lol
<rogpeppe> entirely understandable!
<dimitern> rogpeppe: :) yeah, learning everyday
<rogpeppe> dimitern: i should've caught it!
<dimitern> har har :)
<rogpeppe> dimitern: when would FloatingIP.InstanceId not be a string?
<rogpeppe> dimitern: oh, i see
<rogpeppe> dimitern: i think it should be defined as *string, not interface{}
<dimitern> rogpeppe: it's defined as interface{} and it might be either string or null
<rogpeppe> dimitern: in nova, that is
<dimitern> rogpeppe: well, there was something about it, which prevented *string somehow, can't remember now
<rogpeppe> dimitern: what's the difference between null and the empty string?
<dimitern> rogpeppe: we don't care for anything but non-empty string
<dimitern> rogpeppe: that's just how OS returns it - null, "" or "someip"
<rogpeppe> dimitern: BTW i'd be interested to know what prevented using *string
<rogpeppe> dimitern: 'cos AFAIK that's the canonical way to do null-or-string
<dimitern> rogpeppe: well, if it's missing from the response?
<rogpeppe> dimitern: it turns into nil
<dimitern> yeah
<rogpeppe> dimitern: (assuming the field value was nil originally)
<rogpeppe> dimitern: missing fields are unchanged.
<dimitern> rogpeppe: I suppose  I should change that then in goose as a follow-up
<rogpeppe> dimitern: i think it's worth using static types where possible, yeah
<dimitern> rogpeppe: that was the very beginning - still leaning go and how stuff works
<rogpeppe> dimitern: :-) i know the feeling.
<dimitern> not that I know it all now, but slightly better than ten :)
<dimitern> then*
<dimitern> rogpeppe: what worries me a bit is there is no obvious way to test this locally..
<dimitern> rogpeppe: otherwise from cmd line it works - tried multiple times
<rogpeppe> dimitern: why not?
<rogpeppe> dimitern: can't you implement similar functionality in the double?
<dimitern> rogpeppe: yeah, it starts to shape up a bit in my head how I could actually
<dimitern> rogpeppe: have 2 cases (or more probably) - with free ip and without (to alloc new), then some errors, etc.
<dimitern> rogpeppe: and in juju i can then call bootstrap on the double and ensure it passes
<rogpeppe> dimitern: sounds plausible
<dimitern> rogpeppe: cannot see how to live test it (on canonistack)
<rogpeppe> dimitern: ah, because canonistack always allocates an IP?
<dimitern> rogpeppe: it's very flaky and unreliable with floating ips
<rogpeppe> dimitern: ha
<dimitern> rogpeppe: because there are rarely any free to alloc
<rogpeppe> dimitern: in which case, you're stuffed :-)
<dimitern> rogpeppe: and the live test will fail randomly (but very often)
<rogpeppe> dimitern: yeah. maybe you should add a live test, make sure it passes at least once, and then disable it until we've got a decent live testbed.
<dimitern> rogpeppe: or even check if FIPs are available first, otherwise skip it in the suite
<rogpeppe> dimitern: that sounds like a good plan
<dimitern> rogpeppe: yeah, I'll try it out next
<rogpeppe> dimitern: you've got a review
<dimitern> rogpeppe: tyvm
<fwereade> rogpeppe, btw, I was wondering about having PasswordHash directly in the document and the PasswordValid method pn AuthEntity
<fwereade> s/pn/on/
<rogpeppe> fwereade: yes? (that's what i'm proposing, right?)
<fwereade> rogpeppe, I'm wondering what will use PasswordValid
<rogpeppe> fwereade: the api server
<fwereade> rogpeppe, ok, and it's never actually going to be sent out to anything else?
<rogpeppe> fwereade: indeed
<rogpeppe> fwereade: it's specifically so that the api server can authenticate users
<dimitern> mgz: ping
<rogpeppe> fwereade: (and agents, of course)
<fwereade> rogpeppe, ok, I guess that's reasonable... I have something knocking around in my mind that a private key is maybe a good approach here as well -- we're not restricted to mongo's name/password scheme any more
<fwereade> rogpeppe, this is mainly because the authorized-keys in an environment should probably be thought of as belonging to the admin user now
<fwereade> rogpeppe, and I was wondering if it might be profitable to think around that idea a little
<rogpeppe> fwereade: passwords over https are a fairly conventional way of doing authentication over the web. i like the idea of private keys too, but passwords are more straightforward and not too bad.
<fwereade> rogpeppe, and, yeah, I'm not sure we can expect such a thing from the gui
<rogpeppe> fwereade: yup
<rogpeppe> fwereade: well... a private key is just a very large password from the client's point of view
<fwereade> rogpeppe, well, it never gets sent
<rogpeppe> fwereade: but it's quite useful to have a password you can type
<fwereade> rogpeppe, yeah, indeed
<rogpeppe> fwereade: true. it would be nice if the client used their private key to establish the tls connection
<fwereade> rogpeppe, not quite sure the tradeoff is the same from the agent POV
<fwereade> rogpeppe, but yeah, I'm not blocking anything, just raising it as something to keep in the back of your mind
<rogpeppe> fwereade: yeah. i'm just going with the architecture previously agreed with niemeyer
<fwereade> rogpeppe, the one case to keep in mind even if you don't address it right away is the one in which we use ubuntu SSO
<rogpeppe> fwereade: i'm not sure how the SSO protocol works
<rogpeppe> fwereade: but presumably the SSO key goes straight to the web server, so we can treat it as a password with an additional level of indirection to validate the user.
<rogpeppe> fwereade: s/SSO key/generated SSO token/
<fwereade> rogpeppe, nor am I, I'm afraid -- but please would you look into it briefly, enough that you're reasonably sure that it'll integrate cleanly with your auth plans?
<rogpeppe> fwereade: i saw it did kerberos, and thought "hmm, that might be a lot of work" but .... http://godoc.org/github.com/jmckaskill/gokerb
<rogpeppe> s/did/used/
<fwereade> rogpeppe, cool
<rogpeppe> fwereade: although maybe we just need to speak openid
<dimitern> fwereade: can you take a look at this pls: https://codereview.appspot.com/7226068/
<fwereade> dimitern, sure
<fwereade> dimitern, I have a couple up on https://code.launchpad.net/juju-core/+activereviews as well :)
<dimitern> fwereade: cheers, I'll take a look
<fwereade> dimitern, what's the situation in which a freshly-started instance has a fip allocated?
<dimitern> fwereade: there is no way this can happen
<dimitern> fwereade: started instances do not have fips attached by default
<dimitern> fwereade: but even if it does, I have the check fip.instanceid == serverid and exit
<fwereade> dimitern, ok -- I was wondering then whetherit makes sense to try to get the floating ip before we start the instance, then -- ie separate allocate and assign
<fwereade> dimitern, at bootstrap, if that fails, we can know not to even launch the instance
<dimitern> fwereade: launching the instance and having fips available are not related - startup can fail, but then it may succeed and fip allocation can fail
<dimitern> fwereade: and even checking before startup, nothing guarantees a positive check will not fail after startup (fips may be exhausted in the mean time)
<dimitern> fwereade: at least most cases of errors are handled sanely and a shutdown is performed
<mgz> dimitern: hey, still need me for owt? shall I just check for things in need of review? :)
<fwereade> dimitern, so you can allocate the IP and have someone else take it away before you assign it to the instance? that seems surprising
<dimitern> mgz: yeah, if you want - take a look pls https://codereview.appspot.com/7226068
<mgz> if by someone else you mean "another process using your tenant and credentials", yes
<dimitern> fwereade: no, if you allocate it (or already have at least 1 avail) then assignment will always succeed (unless nova itself does not respond sanely)
<fwereade> dimitern, in that case, if the environment requires a floating IP, it seems best to start by allocating that and then, if you get one, provisioning a machine to put behind it
<fwereade> dimitern, (and finally assigning the ip to the machine)
<dimitern> fwereade: yes, you're right
<dimitern> haven't thought of that
<dimitern> it makes sense, better logic, and possible earlier failure
<fwereade> dimitern, TLDR; I think we want `allocatePublicIP()` and `assignPublicIP(serverName string)`
<fwereade> dimitern, cool, I'll note it
<dimitern> fwereade: great, 10x
<dimitern> mgz: of course, you're right as well
<dimitern> mgz: if you mess up your own account from another location, then it'll break
<mgz> well, it's more of an issue with shared accounts like the one we have for goosebot
<mgz> then you might have multiple copies of juju, even, trying to do things at the same time :)
<dimitern> yes
<dimitern> and why did we need a shared account in the first place? because of the tools?
<mgz> well, we wanted shared resources regardless, the reason we have a shared account rather than a shared tenant is just IS hysterical raisins it seemed
<dimitern> I see :)
<mgz> but the point is we want to manage a shared resource (a juju deployment of tarmac) between multiple people. there's no particular reason we'd be doing conflicting things at the same time, but it's possible
<rogpeppe> "
<rogpeppe> mt.Sprintf(":%d", apiPort)
<rogpeppe> Why did these change?
<rogpeppe> "
<rogpeppe> fwereade: because the previous version was just wrong :-)
<rogpeppe> fwereade: string(1234) != "1234"
<fwereade> rogpeppe, glaargh
<fwereade> rogpeppe, good point indeed, I was focusing on var/const :/
<mgz> and there's no concept of pure functions in go...
<mgz> so, Sprintf can't promise that, given const input, output is const-suitable
<rogpeppe> mgz: indeed. no function is really pure though, even in haskell :-)
<fwereade> mgz, rogpeppe: thanks :)
<rogpeppe> in limbo, a close ancestor of Go, string(1234) did give "1234" (so it was const-able) but the only way of generating a string containing a single constant rune was sprintf("%c", x). i think i prefer Go's decision, but they're both useful
<rogpeppe> jam, fwereade: i'm interested in your reactions to my comments on the boilerplate environments branch: https://codereview.appspot.com/7223063/
<rogpeppe> mramm, fwereade, TheMue: meeting?
<TheMue> rogpeppe: I'm prepared, just waiting for the invitation.
<mramm> ahh
<mramm> joining
<mramm> hangout is in the meeting invite
<rogpeppe> TheMue: https://plus.google.com/hangouts/_/539f4239bf2fd8f454b789d64cd7307166bc9083
 * niemeyer => lunch
<rogpeppe> fwereade: here's the api model code i was playing with, in case you missed the link i pasted in the hangout: http://paste.ubuntu.com/1593298/
<fwereade> rogpeppe, sorry, I did; thanks
<rogpeppe> fwereade: it just models a single client reading a single changing value
<rogpeppe> fwereade: (in the naive way i'd originally envisaged
<fwereade> rogpeppe, I hope I will get to playing with it tonight - I'll be looking after laura as well but will probably be back later on :)
<fwereade> rogpeppe, (I'm still around until just after 6)
<rogpeppe> fwereade: i'm just heading out for some lunch and a stomp outside - will be back before then but not much before.
<fwereade> rogpeppe, then enjoy your stomp, and also your evening
<fwereade> rogpeppe, I will see you if I see you :)
<rogpeppe> fwereade_: stomp enjoyed
<rogpeppe> fwereade_: found about a dozen broken plastic sledges at the bottom of the village "run"
<fwereade_> rogpeppe, haha
<rogpeppe> fwereade_: all the snow's gone now though
<fwereade_> rogpeppe, I realised I was old when my instinctive "yay, it's snowing" was almost instantly overwritten by "grrmbl I have to walk to work through that"
<fwereade_> rogpeppe, and then I ran away to the med to try to forget it ;p
<rogpeppe> fwereade_: i'm still in the yay! phase
<TheMue> fwereade_: both branches are in, will also add a bug for the sudo-test-change
<fwereade_> TheMue, awesome, tyvm
<TheMue> fwereade_: are your review cards still active? if so, could you copy the link to the reviews into the cards? makes it simpler.
<fwereade_> TheMue, ah, sorry, sure -- I use +activereviews to keep up with them... er, when I do keep up with them :/
<TheMue> fwereade_: hehe
<TheMue> fwereade_: it's nice to see the cards in review state and directly follow the links
<fwereade_> TheMue, agreed
<fwereade_> TheMue, added
<fwereade_> TheMue, remind me when I forget
<TheMue> fwereade_: cheers
<TheMue> fwereade_: So, bug is entered at https://bugs.launchpad.net/golxc/+bug/1111640 and assigned to me.
<_mup_> Bug #1111640: Extend tests to run without sudo <golxc:New for themue> < https://launchpad.net/bugs/1111640 >
<fwereade_> TheMue, lovely, thanks
<TheMue> fwereade_: yw
<TheMue> fwereade_: You've got two reviews.
 * rogpeppe is done for the day
<rogpeppe> g'night all
<TheMue> rogpeppe: good night, I'm leaving too.
#juju-dev 2013-02-01
<arbit30719> hi all
<arbit> ls
<arbit> is anybody home ?
<arbit> itsZero: ha
<arbit> hello all
<arbit> Can any body help me solve juju/mass issue
<arbit> I have posted in ask ubunut : http://askubuntu.com/questions/249350/juju-cannot-deploy-services-with-maas/250102#250102
<_mup_> Bug #250102: rhythmbox crashed with SIGSEGV in gst_marshal_BOOLEAN__POINTER() <apport-crash> <apport-failed-retrace> <GStreamer:Expired> <Rhythmbox:Invalid> <gstreamer0.10 (Ubuntu):Incomplete> < https://launchpad.net/bugs/250102 >
<niemeyer> arbit: Would you mind to write a message to the mailing lits pointing your problem out?
<niemeyer> arbit: May be just a link to the ask ubuntu page
 * niemeyer => bed!
<arbit> hi all
<arbit> does any one use jitsu ?
<arbit> Can jitsu 3 or more services on a single machine ?
<fwereade__> arbit, my understanding is that jitsu will co-locate as many units as you ask it to
<fwereade__> arbit, but #juju is probably a better place for that sort of question
<arbit> ok thanks
<rogpeppe> fwereade__, jam, wallyworld_: morning!
<fwereade__> rogpeppe, everyone, heyhey
<fwereade__> bbiab, coffee
<TheMue> lunchtime
<jtv> Anyone free to review a patch?  https://codereview.appspot.com/7225082
<dimitern> jtv: i'm on it
<dimitern> jtv: done
<jtv> Thanks dimitern!
<jtv> Oh, I need one more, right?
<dimitern> jtv: yeah, unless it's trivial
<jtv> I'll start begging then.  Me, dignity?  Hah!
<jtv> rogpeppe, can I  trouble you for a second review perhaps?  â https://codereview.appspot.com/7225082
<dimitern> :)
<rogpeppe> jtv: i've got a couple of other reviews in progress, but i will have a look after that.
<jtv> Thanks :)
<TheMue> jtv: You've got a 2nd review.
<jtv> Thanks TheMue!
<jtv> Hmm...  I don't think I have permission to land my  branch.  I'm getting a bzr error out of lbox:
<jtv> bzr: ERROR: Cannot lock LockDir(chroot-91275088:///%2Bbranch/juju-core/.bzr/branch/lock): Transport operation not possible: readonly transport
<dimitern> jtv: it seems you're not a member of go language gophers
<jtv> Nope.  Not surprising really, given my short experience coding in the language.  :)
<dimitern> jtv: :) - ask someone to add you and your team perhaps? https://launchpad.net/~gophers/+members#active
<jtv> I could give it a whirl, thanks
<rogpeppe> jtv: i'm just reviewing your branch
<rogpeppe> jtv: i think there are a few possible improvements.
<rogpeppe> jtv: but i've got lunch now. back in an hour.
<fwereade__> rogpeppe, TheMue, mramm: hangout?
<TheMue> fwereade__: I'm here
<jtv> Hi rogpeppe â thanks.  As you see I have a second vote now, so no pressure.  :)
<rogpeppe> jtv: i'd prefer it if you waited until i've send my review
<jtv> Of course.
<niemeyer> Yo
<niemeyer> Heading for a late lunch.. biab
<dimitern> fwereade__, rogpeppe: i'm ready with this and i'd really like to land it today :) https://codereview.appspot.com/7226068/
<rogpeppe> dimitern: i'm just finishing the gomaas review, then will finish jtv's then i might hopefully have time to look at yours. 35 minutes to go :-)
<dimitern> rogpeppe: cheers
<dimitern> and I have one for goose as well - https://codereview.appspot.com/7265043/ (related)
<dimitern> fwereade__: ping
<fwereade__> dimitern, pong
<dimitern> fwereade__: I just saw your msg
<fwereade__> dimitern, no worries
<rogpeppe> jtv: reviewed
<rogpeppe> dimitern: first one reviewed
<dimitern> rogpeppe: tyvm
<rogpeppe> dimitern: presumably you need the goose branch to land before you can submit the other one?
<dimitern> rogpeppe: yes
<rogpeppe> dimitern: i'm having a look now
<dimitern> rogpeppe: ty, I'm done with your suggestions, will wait fwereade__ and repropose
<rogpeppe> dimitern: goose branch reviewed
<rogpeppe> right, time to go
<rogpeppe> g'night all
<dimitern> rogpeppe: thanks and good night :)
<rogpeppe> dimitern: BTW anyone on red squad here?
<rogpeppe> anyway, anyone on red squad: i made some comments on the gomaasapi CL:  https://codereview.appspot.com/7228069/
<dimitern> rogpeppe: yes, reviewed it - very nice and short, will ping jtv or someone if I see them
<fwereade__> dimitern, reviewed -- I'm sorry to say I think it needs more tests
<fwereade__> dimitern, convince me otherwise over beer?
<dimitern> :)
<dimitern> fwereade__: could do probably :)
<dimitern> i'm off
<dimitern> happy weekend all
<fwereade__> happy weekends everybody
<niemeyer> fwereade__: Have a great weekend
<niemeyer> Anyone up for a review on goamz?
<niemeyer> Oh the irony
<niemeyer> While working on a branch to retry operations against S3:
<niemeyer> Please try again | Sorry, there was a problem connecting to the Launchpad server. | Try reloading this page in a minute or two.
<niemeyer> OMG... a full functional live S3 test run in multiple regions without any error
#juju-dev 2014-01-27
<jam> dimitern: Good morinng! I'm in the hangout whenever you want to join
<dimitern> jam, morning, i'm just about to open it :)
<rogpeppe> mornin' all
<dimitern> morning ;)
<drj11> morning morty
<jam> dimitern: you might want to check your Spam folder. BTSTravel successfully booked the flight, but the email with itinerary got flagged as spam.
<dimitern> hmm
<jam> And google, at least, thinks emails with attachments from bts are spam
<dimitern> jam, thanks, I'll have a look
<jam> " We've found that lots of messages from btstravel.be are spam.Â "
<dimitern> jam, nope, i've just got a reply
<dimitern> jam, she'll book it soon
<jam> dimitern: I get direct emails from them, but not emails including the itinerary. You may need to check once its booked :)
<dimitern> jam, will do :)
 * fwereade is having further happy fun electricity times
<TheMue> fwereade: heya
<TheMue> fwereade: have you been able to talk to rogpeppe about his approach?
<fwereade> TheMue, yeah, he convinced me, please confer with him
<rogpeppe> TheMue: i'm in a sprint this week, but i should be able to get a few moments here and there to help
<dimitern> fwereade, if you have a moment, https://codereview.appspot.com/53210044 fixes bug 1067979
<_mup_> Bug #1067979: race: concurrent charm deployments corrupts deployments <deploy> <race-condition> <test-needed> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1067979>
<fwereade> dimitern, awesome, I will try
<fwereade> everyone, however, we've got to open up a wall and figure out if there's something that'll burn the house down in there
<fwereade> so my presence this morning at least is likely to be random and fleeting
 * dimitern is afk for a while
 * fwereade grumbles
<TheMue> rogpeppe: ok, much is already in your branch, so I can use it, especially with the noted todos. in case of questions I simply try to reach you
<TheMue> rogpeppe: where is the sprint?
<rogpeppe> TheMue: london
<TheMue> rogpeppe: nice, would like to go there too. only been once in london, in heathrow for 3h :D
<rogpeppe> TheMue: i'm not that keen on london :-)
<TheMue> rogpeppe: would like to do some sightseeing but surely not live
<cgz> prerecorded sightseeing would be boring...
<cgz> all the canned laughter...
<TheMue> cgz: what is "prerecorded sightseeing"?
<cgz> TheMue: I was poking you a little about the idea of 'live' being as in live television, rather than as in to reside :P
<cgz> jam: 1:1 on g+?
<jam> hey cgz, sure
<jam> cgz: in the weekly one, or in the 1:1 ?
<jam> sorry, in the weekly calendar 1:1 or in the daily standup hangout?
<cgz> the daily will do
<jam> cgz: I'm there
<TheMue> cgz: hehe, ok, fought with my lack of knowledge of the english language
<jam> TheMue: they're pronounced differently, but spelled the same
<jam> short 'i' vs long 'i'
<TheMue> jam: maybe we should write phonetics here ;)
<jam> TheMue: english would be much better if it was phonetical
<TheMue> jam: same problem here. and my younger daughter fights with it even more while learning japanese
<jam> TheMue: Chinese is just as bad: http://forum.koohii.com/viewtopic.php?pid=105992
<TheMue> jam: *iirks*
<jam> TheMue: listen to the audio, it is pretty amazing
<rogpeppe> anyone got an opinion on the best way to fix https://bugs.launchpad.net/juju-core/+bug/1273169 ?
<_mup_> Bug #1273169: bootstrap: error from apt-get update is uninformative <juju-core:New> <https://launchpad.net/bugs/1273169>
<rogpeppe> two immediate possibilities spring to mind, but perhaps there's a better idea
 * fwereade waves at meebey
<jam> fwereade: rogpeppe: standup?
<jam> dimitern_: a quick thought. if you change how charm archives are named in storage, it would be good to keep the charm name in there. (so charm-123-UUID, or somethnig like that) otherwise pure UUID/SHA256 hashes are really meaningless if you ever just listed your storage
<vds> anyone can help me in understanding what's happening: https://pastebin.canonical.com/103609/ ?  http://paste.ubuntu.com/6825878/
<dimitern_> jam, that's a fair point
<dimitern_> vds, if you're using 1.16, this is a known issue with the simple streams format
<jam> vds: if you're using --upload-tools, it expects to find a 'jujud' in the same directory as the 'juju'
<jam> dimitern_: that would be 1.17, actually (I'm pretty sure)
<dimitern_> jam, ah, ok
<jam> 1.16 didn't go for streams..../juju
<dimitern_> right!
<vds> dimitern_,  hello, for what I can see, it's there: /home/vds/canonical/bin/juju /home/vds/canonical/bin/jujud
<vds> dimitern_, is it ok that https://streams.canonical.com/juju returns a 404?
<vds> if I hit it with a browser?
<dimitern_> vds, actually, I'm not that into simplestreams stuff myself
<dimitern_> vds, but if you're trying to bootstrap from source with --upload-tools it should work, as long as your GOPATH and other Go stuff is in the right place
<jam> vds: it won't "real soon now' but it is expected right now
<jam> well, the code didn't expect it, but we're fixing ithat
<vds> dimitern_, once everything is compile, does GOPATH still matters?
<vds> in any case, the GOPATH is fine, the same command used to work on Thu and it is broken since Fri.
<vds> jam, I was running an older revno, now I'm on trunk and I still get the same issue, I was wondering if it's just me or that bootstrap is broken.
<jam> vds: if you're running from source, you probably ran "go build" in the source tree, you need to run "go install" and run it from where jujud is next to juju (I believe
<dimitern_> vds, have you tried not giving --series=... ?
<vds> jam, yep, did that
<dimitern_> vds, I have this handy little script to rebuild the binaries
<dimitern_> vds, http://paste.ubuntu.com/6825937/
<vds> dimitern_, yes, it works, but I need trusty, how else do I do it?
<dimitern_> vds, default-series: trusty?
<dimitern_> in env.yaml
<dimitern_> vds, if you're on trusty, --upload-tools automagically uploads the tools in both "default-series" and your current series
<vds> dimitern_, I'm on saucy, but the charm I'm trying to test requires trusty, I've changed the default in env.yaml, let's see if it works :)
 * dimitern_ needs to step away for and hour or so
<fwereade> vds, I think you need a fix for https://bugs.launchpad.net/juju-core/+bug/1217397
<_mup_> Bug #1217397: image-stream configuration option for openstack provider <config> <openstack-provider> <juju-core:In Progress by wallyworld> <https://launchpad.net/bugs/1217397>
<fwereade> vds, ian's working on it
<fwereade> vds, (or, you don't necessarily need it, but you'd need to generate your own image metadata and set image-metadata-url)
<vds>  fwereade thanks for the info
<vds> any known issues with juju debug-log, I'm getting this: http://paste.ubuntu.com/6826178/
<TheMue> vds: local provider? in that case debug-log doesn't work (yet)
<vds> TheMue, OpenStack
<TheMue> vds: in that case it's strange, hmm
<TheMue> vds: current debug-log makes a ssh connect to the state server and tails the mentioned file
<vds> TheMue, http://paste.ubuntu.com/6826243/
<TheMue> vds: hmm, I'm stunning
<TheMue> vds: it is set in the syslog config
<TheMue> vds: it seems this is missing. troubles during bootstrapping?
<vds> TheMue, not that I can see, and it happens all the time, I'm using juju-core trunk
<TheMue> vds: what do you see in /etc/rsyslog.d on the state server?
<vds> TheMue, /etc/rsyslog.d/25-juju.conf http://paste.ubuntu.com/6826312/
<TheMue> vds: looks right *headshaking*
<TheMue> vds: and rsyslogd is running?
<vds> TheMue, yep
<TheMue> vds: could be interesting would happens if you restart it, but that's not the idea behind it
<vds> TheMue, not much
<TheMue> vds: sadly I'm reaching my limits now
<vds> TheMue, thanks :)
<vds> TheMue, I wonder if that happens just to me, I'm bootstrapping a trusty machine, that's it, nothing strange
<TheMue> vds: never tested it on trusty so far
<jamespage> fwereade, hey - just repro'ed that issue I was taking about last week when tearing down environments
<jamespage> http://paste.ubuntu.com/6826470/
<fwereade> jamespage, cool, thanks; remind me, was there already a bug for that?
 * jamespage digs
<jamespage> fwereade, https://bugs.launchpad.net/juju-core/+bug/1244807
<_mup_> Bug #1244807: juju-deployer teardown fails due to hook errors <destroy-service> <theme-oil> <juju-core:Triaged> <https://launchpad.net/bugs/1244807>
<fwereade> jamespage, gaah, found the bug
<fwereade> jamespage, thanks
<vds> TheMue, btw: when I bootstrap I get a lot of lines like: ERROR TLS handshake failed: EOF
<TheMue> vds: only when bootstrapping trusty?
<vds> thedac, well, so far that's what I'm bootstrapping.
<fwereade> rogpeppe, natefinch, dimitern_: anyone recognise http://paste.ubuntu.com/6826597/ -- I seem to be seeing it 100% of the time on trunk
<natefinch> fwereade: sorry, was afk.  I haven't seen that one.
<fwereade> natefinch, can you repro? it's possible I'm doing something dense
<natefinch> fwereade: checking
<natefinch> gah, I wish bzr switch would warn me that I have uncommitted changes that'll get merged into the new branch and screw everything up
<fwereade> natefinch, ouch, sorry :(
<natefinch> it's ok, luckily fairly segregated changes.
<natefinch> fwereade: go test ./... from worker/uniter passes for me
<natefinch> fwereade: on trunk
<natefinch> fwereade: oops, noticed that's not where you were testing
<natefinch> fwereade: launchpad.net/juju-core/state/apiserver/uniter  also passes for me
<fwereade> natefinch, thanks for confirming
<sinzui> nate_finch, CI consistently sees an error that I think is in the machine, not the test or juju
<sinzui> ERROR juju.cmd supercommand.go:294 cannot use 37019 as state port, already in use
<sinzui> ^ How do I smack mongodb to be in a pristine state
<nate_finch> sinzui: you should remove the default mongo upstart script.... though that should be on 27019
<sinzui> nate_finch, 37019 is the port juju says it wants to use
<nate_finch> sinzui: oh wait, mongo is 27017  ... hmm
<nate_finch> sinzui: remnants from an old test that died?
<sinzui> possibly
<nate_finch> check for instances of mongo and kill them mercilessly
<nate_finch> also some of the tests create folders called /tmp/mongo-test*    (typing from memory, but something like that)
<nate_finch> mgo-test I think
<nate_finch> those should get blown away, if only to maintain disk space
<sinzui> I killed mongodb, but I juju still complains about the port in use
<sinzui> WTF, netstat says it is there, but ps says there is no mongodb
<sinzui> I see some progress and killing the proc that was restarted
<nate_finch> sinzui: it's almost certainly a port getting stuck from mongo, but not sure why it wouldn't be freed up when you killed mongo
<sinzui> I have bootstrapped with 1.17.1 \o/
<sinzui> I cannot destroy-environment though. Looks like the command no longer honors JUJU_HOME
<sinzui> there will never be a .juju dir
<nate_finch> huh weird
<vds> could you please tell me where am I supposed to find the logs of a subordinate charm? they don't appear in machine-XXX.log nor in unit-MAINCHARM-0.log
<cgz> https://codereview.appspot.com/56560043 is ready, really I want Andrew to look at it when he comes on, because I don't understand why localStorage is so complex in the first place
<cgz> I'm off now, later!
 * thumper tries to remember details around structure equality in golang
<nate_finch> thumper: gotta be careful, since it depends on whats in the structure (i.e., if it has a map, you can't do it)
<thumper> nate_finch: it is a struct with three strings
<thumper> nate_finch: http://play.golang.org/p/e3O_2dSfgC
<thumper> nate_finch: different values, but same contents
<thumper> yes you need to be careful...
 * thumper thinks he has this...
<nate_finch> yeah, comparing pointers is rarely what you want
<thumper> natefinch: I was thinking back to how equality works in C++
<thumper> member-wise comparison by default
<thumper> which is kinda like this
<thumper> obviously two maps are going to be two different maps
<thumper> and it isn't going to check content
<thumper> is that right?
<natefinch> thumper: right.... well, they could be the same map, but they probably aren't
 * thumper nods
<thumper> if they are the same map, is it equal?
<thumper> yes
<natefinch> thumper: I think go just smacks you if you try
<thumper> haha
<thumper> return errors.New("don't do that"
<thumper> )
<thumper> natefinch: I have a new go project
<thumper> natefinch: which I've not started yet
<thumper> natefinch: got a few minutes for a hangout?
<natefinch> thumper: sure
<thumper> natefinch: https://plus.google.com/hangouts/_/76cpi448v94dmmna9rmctckokk?hl=en
<natefinch> thumper: prog.go:14: invalid operation: c == b (struct containing map[string]string cannot be compared)
<natefinch> thumper: http://play.golang.org/p/DfJsb1AIMT
<thumper> natefinch: ta
<thumper> natefinch: similar in some idea...
<thumper> natefinch: I'll see if I have some time to hack some stuff later
<natefinch> thumper: cool
<wallyworld> thumper: hiya, do you know that status of the 1.17 release?
<thumper> wallyworld: nope
<thumper> stuck I think
<wallyworld> i saw a bug that was assigned to 1.17.1 moved to 1.17.2
<wallyworld> i had that work committed last last week
<wallyworld> maybe a 1.17.1 is being rolled out taking a rev prior to the local provider changes
 * thumper shrugs
<wallyworld> thumper: at the moment, i'm working on allowing generic support for non-released images in simplestreams - previously azure supported a hard coded approach. it's a little messy due to inconsistent url path conventions. sigh. more work than i thought, and more testing also
<wallyworld> william wants this for thursday
<thumper> wallyworld: I thought you weren't working today
<wallyworld> well, i am somewhat
<wallyworld> i may have to be afk a little tomorrow morning for lachie's first day of school
 * thumper nods
<wallyworld> plus i want to get this stuff done. it's on my mind
 * thumper wants an event bus inside juju-core
 * thumper signs
<thumper> perhaps one day
 * thumper resists the temptation to write one for this test
<davecheney> yo dog, i put an event bus inside juju, so you could wait for events, while waiting for events
<davecheney> #pleasemakeitstop
<thumper> :)
<wallyworld> thumper: i think someone said there was a go pubsub project somewhere? in our copious free time, let's plug it in
<hazmat> wallyworld, nsq
<hazmat> different operational semantics than what we want for juju, at least once instead of at most
<wallyworld> hazmat: i wasn't going to do it :-)
<wallyworld> but i do think our event substrate is broken somewhat
<hazmat> wallyworld,  how so?
<hazmat> wallyworld, i agree fwiw, but curious your reasons
<wallyworld> hazmat: hard to explain in detail over irc. but a couple of things. 1. we notify of changes but then make the recipients all poll back to get the changes. so stampeding herd problem and race condition
<wallyworld> 2. we don't mux/demux notifications to recipients
<hazmat> wallyworld,  yup that's  the same nutshell. moving the pub/sub to the api layer would help.. non composable db transactions as observation feed aren't ideal.
<wallyworld> so send a lot more round trips than we need
<hazmat> actually that's a little different but also good
<wallyworld> i agree somewhat with backend issues you mention. but think we can model around that
<wallyworld> generally, any polling based system doesn't scale well
<hazmat> wallyworld, the notify of change, and fetch change actually helps solve some races as well, its fairly tried and true for state based changes as opposed to event queue work
<wallyworld> depends. i've also seen the opposite. i guess it related to ovrall system architecture
<hazmat> the herd is a bit worse than that, since its baked into the charm model.
<wallyworld> oh :-(
<hazmat> svc a 1k units related to svc b's 1 unit.. .. svc b changes.
<wallyworld> having arrived late to juju-core, i do get the feeling scalability wasn't up front in the design considerations
<wallyworld> or as important as perhaps it might have been
<wallyworld> cause "we'll solve it later wen needed" sorta can lead to design that's hard to change
<hazmat> there was a lot resistance to event mq models in favor of state observation models
<wallyworld> they don't have to be mutually exclusive
<hazmat> true
<wallyworld> eg events can be used to communicate the model state at the time the event was published
#juju-dev 2014-01-28
<wallyworld> and a degree of latency can be used to smooth out spikes in state change
<wallyworld> based on knowledge of the model interactions
<hazmat> the issue is unless its a work event, a state change event can be stale with network partitions or transient disconnects, if things end up pushiing forward an invalid/old state into the application tier.. ie the current fetch is probably still needed for the app / charm model
<hazmat> on design choices and scalability i actually had to do some work recently to have juju support provisioning  many machines with a single provider api call..
<wallyworld> you mean to add a bulk call to allow more than one machine to be provisioned with a single api call, not one per machine?
<hazmat> yes
<wallyworld> i've been pushing for bulk api calls since the api was first mooted
<hazmat> wallyworld, i'm not entirely clear things have changed with the move to core.. take ec2 for example..  standard cloud best practice would be multi-zone .. not something we can actually do with core... or take our instance type hardcodes which are old (we promote more expensive less powerful instances than what's current best practice).
<hazmat> and overlapping, and we can't specify instance-type
<hazmat> wallyworld, for the bulk provisioning i ended up, seeding cloud-init data with something that dialed back home, and got the actual machine specific provisioning script.
<wallyworld> agreed. we could and should change all that. i know people want to
<wallyworld> ah that would work
<hazmat> yeah.. works well, this work was all manual provider based though not in core per se. but the pattern works nicely
<wallyworld> the whole hard coded insance type thing - that's just for ec2 because of what the lib provided. with openstack, we don't have that problem
<wallyworld> we do need to fix ec2 but have had too many other things taking a higher priority
<hazmat> wallyworld, we should ideally host that on cloud-images, not internal to the src.
<wallyworld> oh yes, you think?
<wallyworld> :-)
<hazmat> :-)
<wallyworld> so there's a lot of us that want to get this stuff sorted out badly
<wallyworld> i'm still hopeful we can do it
<davecheney> ubuntu@ip-10-248-60-212:~/src/launchpad.net/juju-core$ rm -rf ~/.juju
<davecheney> ubuntu@ip-10-248-60-212:~/src/launchpad.net/juju-core$ ~/bin/juju init
<davecheney> A boilerplate environment configuration file has been written to /home/ubuntu/.juju/environments.yaml.
<davecheney> Edit the file to configure your juju environment and run bootstrap.
<davecheney> ubuntu@ip-10-248-60-212:~/src/launchpad.net/juju-core$ ~/bin/juju status
<davecheney> ERROR Unable to connect to environment "".
<davecheney> Please check your credentials or use 'juju bootstrap' to create a new environment.
<davecheney> Error details:
<davecheney> control-bucket: expected string, got nothing
<davecheney> this sounds wrong
<davecheney> juju init defaults to amazon
<davecheney> why does it say the environment is ""
<thumper> nfi
<thumper> but definitely a bug
<davecheney> this is a fresh install
<davecheney> i
<davecheney> i'll poke around some more
<davecheney> i smell JUJU_HOME in there somewhere
 * davecheney throws a chair
<davecheney> no, JUJU_EV
<davecheney> ubuntu@ip-10-248-60-212:~/src/launchpad.net/juju-core$ export JUJU_ENV="amazon"
<davecheney> ubuntu@ip-10-248-60-212:~/src/launchpad.net/juju-core$ ~/bin/juju status
<davecheney> ERROR Unable to connect to environment "amazon".
<davecheney> bingo
<davecheney> thumper: ~/.juju/current-environemnt
<davecheney> where did that come from
<davecheney> that is being consulted always
<thumper> switch
<davecheney> but I never used switch
<thumper> then it probably shouldn't be there
<davecheney> but if it's not there
<davecheney> the env comes up as ""
<thumper> that's a bug
<thumper> it whould fall back to the default environment
 * thumper thinks
<thumper> ah...
<thumper> yes
<thumper> what happens
<thumper> is that if the string is empty
<thumper> which you would have got if you didn't specify -e or have an env var
<thumper> it is treated "specially"
<davecheney> hmm,
<thumper> we should remove the special
<thumper> and have a sane fallback
<thumper> always been like that
<davecheney> the correct behavior is to fall back to the order in the yaml file
<thumper> now we are just outputting it
<thumper> no actually
<thumper> fall back to the default if specified
<thumper> if no default specified
<davecheney> there is a default
<thumper> then use the one if only one specified
<davecheney> this is from juju init
<thumper> otherwise error
<davecheney> all this bullshit is too complicated
<thumper> but it is using the default specified
<davecheney> juju switch was a bad idea
<thumper> but it is getting it by the special case of ""
<thumper> no
<thumper> this isn't about switch
<thumper> this is how it worked before
<thumper> I just added an extra thing in
<thumper> we weren't outputting it before
<davecheney> ok
<thumper> whoever added the outputting should have changed the behaviour
<thumper> but didn't
<davecheney> lemmie see if I can put this in an issue
<thumper> having "" special cased is dumb
<davecheney> ok
<davecheney> thakns
<davecheney> i'll log a bug after lunhc
<sinzui> thumper, I have a branch that I can release as 1.17.1. It is the last r2248 + mgz's openstack and mgo fixes
<sinzui> thumper, It doesn't pass on canonistack though.
<sinzui> I cannot get stable juju to work on canonistack today, so maybe I should release this mashup as 1.17.1 anyway
<wallyworld> sinzui: what's the issue on canonistack?
<wallyworld> that has been flakey on occasion
<sinzui> After a successful bootstrap, the client can never talk the the env. That is true for 1.16.5 and 1.17.1 from inside canonistack, outside with public ips and outside with sshuttle vpns
<wallyworld> hmmm
<sinzui> wallyworld, The cloud health tests show that canonistack has been barely usable for several days
<rick_h_> sinzui: did the sshuttle connect?
<sinzui> yes it did
<wallyworld> if everything else works, probably good to release then. i assume hp cloud works
<sinzui> but status and deploy always timeout
<sinzui> Hp is indeed very health with my branch
<jam> rick_h_: sinzui: sounds like a firewall issue, they might be blocking everything but port 22 even for the public IPs or something
<jam> or, public IPs aren't routable, but sshuttle is using the chinstrap bounce to connect
<sinzui> jam, from inside canonistack?
<jam> sinzui: so when you have sshuttle connected, do you *also* have a public IP assigned?
<wallyworld> sinzui: 2248 is an older revision from last week?
<jam> because we won't end up using shuttle if we think we have an address outside the 10.* space
<jam> I'm just theorizing, though.
<sinzui> jam the health check is an hourly deployment using stable on each cloud. canonistack is ill http://162.213.35.54:8080/job/test-cloud-canonistack/ ...
<wallyworld> latest health check works, quick release :-)
<sinzui> ...but since we are not seeing resources deleted properly from trunk tests, we know that some of the failures can be cause by the cruft left behind
<wallyworld> sinzui: i see 2248 was before local provider sudo changes
<sinzui> I have been manually deleting instances, security groups, and networks all day to give CI a chance to pass
<wallyworld> it would be good if 2249 could be the rev we use
<sinzui> It is
<wallyworld> unless that is broken
<sinzui> wallyworld, FUCK NO
<wallyworld> 2249 eliminates polling for status
<sinzui> wallyworld, 2249+ does not pass
<wallyworld> oh
 * wallyworld is sad
<wallyworld> i'll have to take a look then
<wallyworld> i tested on ec2
<sinzui> wallyworld, I know. I really wanted to realeas tip. local is not healthy in trunk since CI was tainted by the destroy-environment problems, I ask other people to test. Local doesn't just work, for anyone who has ever used local before
<wallyworld> my change in 2249 was related to juju status only
<wallyworld> i'll check that it is ok though
<sinzui> wallyworld, I have been testing for 3 days. People want a release... and the want they favourite features in it
<wallyworld> oh, sorry, didn't intend to push for 2249 to be included. i was just concenred i may have broken somethong
<sinzui> I am burned out trying to make a release when CI gave an answer last week
<sinzui> wallyworld, I want to release every week, which will help get the good work out into the wild
<wallyworld> yeah. but we need to stop breaking stuff
<sinzui> wallyworld, Juju trunk was very good last week. On the last day a lot of branches landed just broken tests every where
<wallyworld> i think they were mainly the local provider changes
<wallyworld> if i understand correctly, it seems like the newer code doesn't like a previous dirty disk
<sinzui> wallyworld, Yeah, but since it cannot destroy itself (possibly a bug in the very version I propose releasing) the disk will always be dirty
<wallyworld> so the clean up this week will need to be able to deal with that
<wallyworld> i think the issue may be understood, so if 2248 is released and the final polish applied this week to trunk, 1.17.2 at the end of the week hopefully :-)
<wallyworld> or next week
<sinzui> wallyworld, this bug causes subsequent runs of tests to fail. https://bugs.launchpad.net/juju-core/+bug/1272558
<_mup_> Bug #1272558: destroy-environment shutdown machines instead <ci> <destroy-machine> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1272558>
 * wallyworld looks
<sinzui> We keep hitting resource limits, or instance already exists failures from machines that were shutdown instead of destroyed.
<sinzui> oh, and it didn't help that our trust amis expired.
<wallyworld> sinzui: that is an interesting bug. i had a quick look at the Openstack provider, and StopInstances does seem to call "delete server". so more investigation required
<sinzui> ah, juju-test-cloud-canonistack-machine-0 got shutdown instead of destroyed. I expect the next health check to fail
<sinzui> wallyworld, the bug also affects aws and azure though.
<sinzui> very odd
<wallyworld> yeah, i just thought i'd see what openstack did out of interest
<sinzui> Azure can take hours to delete a network when I do it from the console too
<wallyworld> azure seems to take a looooong time to do *anything*
<wallyworld> sinzui: so i assume that for example you could do a "nova list" and the old machines would be shown still and have a status "shutdown"
<sinzui> The durations shown here are consistent with previous weeks: http://162.213.35.54:8080/
<sinzui> we do run azure tests in parallel to keep all CI test to about 30 minutes, but that also puts us at risk to exceeding our 20 cpu limit
<sinzui> wallyworld, exactly what I see
<sinzui> wallyworld, Hp is the only cloud/substrate not affected.
<wallyworld> hmmm. i'm not intimately familiar with destroy-env code. i wonder if maybe something changed recently
<wallyworld> we have some investigating to do
<wallyworld> sinzui: i'll make sure the issue is known at the next core standup and we'll ensure someone is assigned to fix as a matter of priority
<sinzui> wallyworld, That is appreciated.
<wallyworld> we really need to get some more closed loop feedback from CI -> devs
<wallyworld> cause i reckon not may devs even know the address of the CI dashboard
<wallyworld> and/or pay attention to the status
<wallyworld> so you poor folks cop stuff we break without timely action to fix it
<wallyworld> and then have to push shit uphill to get a release out
<wallyworld> i'll offer my opinion and hopefully it will be shared and we can implement some workflow to improve the situation
<wallyworld> sinzui: i'll make sure you get feedback to let you know the outcome of the above
<sinzui> wallyworld, Jenkins has a bad UI. We are creating a report site that explains what was tested. http://162.213.35.69:6543/
<sinzui> wallyworld, You do need to log in. to see the report of the revision I create
<sinzui> d
<wallyworld> so i do
<thumper> WTF is going on!
<thumper> damnit
<thumper> this was working before
<wallyworld> sinzui: E403 after logging in
<sinzui> The overall PASS status is there because I manually tested local, then hacked the PASS status. Canonistack should have damned the who rev though
<sinzui> wallyworld, did you check all the boxes?
<wallyworld> thumper: do you have any knowledge of the destroy-env issue sinzui  mentions in the scrollback?
<wallyworld> sinzui: ah, no :-)
<wallyworld> i thought they were for information
<sinzui> oh, arosales reported the same thing. control-reload to force the pages I think
<thumper> wallyworld: where is the scrollback, it is long
<wallyworld> thumper: https://bugs.launchpad.net/juju-core/+bug/1272558
<_mup_> Bug #1272558: destroy-environment shutdown machines instead <ci> <destroy-machine> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1272558>
<wallyworld> sinzui: ah it works now
<thumper> sinzui: try with --force
<wallyworld> thumper: i'd need to read the code. i wonder why not having --force shuts down instances instead of deleting them
<sinzui> thumper, destroy-environment didn't return an error when it failed. It didn't tell us it needed to use use force
<thumper> hmm...
<thumper> this was axw's area, not sure
<thumper> I think it had to do with moving to the api, but can't confirm
<thumper> wallyworld: destroy-environment tries to be nice first
<thumper> I think
<sinzui> thumper, I suspect the issue is older than last week, but something made it more visible.
<wallyworld> sinzui: is it just the bootstrap machine left behind?
<sinzui> well, no, I have never seen a machine SHUTDOWN before last week
<wallyworld> i was thinking perhaps that if the logic was moved behind the api, the code that does the destroy is running on the bootstrap machine itself and there may be an issue destroying it
<sinzui> wallyworld, most of the time, but we have see the service machines shutdown too.
<wallyworld> and that the other nodes could still be destroyed
<wallyworld> ok, was just a guess :-)
<thumper> something seems all fucked up
<thumper> local provider in trunk is giving weird arse lxc errors
<thumper> machine-0: 2014-01-28 03:31:00 ERROR juju.container.lxc lxc.go:102 lxc container creation failed: error executing "lxc-create": + '[' amd64 == i686 ']'; + '[' amd64 '!=' i386 -a amd64 '!=' amd64 -a amd64 '!=' armhf -a amd64 '!=' armel ']'; + '[' amd64 '!=' i386 -a amd64 '!=' amd64 -a amd64 '!=' armhf -a amd64 '!=' armel ']'; + '[' amd64 = amd64 -a amd64 '!=' amd64 -a amd64 '!=' i386 ']'; + '[' amd64 = i386 -a amd64 '!=' i386 ']'; + '[' amd64
<thumper>  = armhf -o amd64 = armel ']'; + '[' released '!=' daily -a released '!=' released ']'; + '[' -z /var/lib/lxc/tim-testlocal-machine-1 ']'; ++ id -u; + '[' 0 '!=' 0 ']'; + config=/var/lib/lxc/tim-testlocal-machine-1/config; + '[' -z /usr/lib/x86_64-linux-gnu/lxc ']'; + type ubuntu-cloudimg-query; ubuntu-cloudimg-query is /usr/bin/ubuntu-cloudimg-query; + type wget; wget is /usr/bin/wget; + cache=/var/cache/lxc/cloud-precise; + mkdir -p
<thumper> /var/cache/lxc/cloud-precise; + '[' -n '' ']'; ++ ubuntu-cloudimg-query precise released amd64 --format '%{url}\n'; failed to get https://cloud-images.ubuntu.com/query/precise/server/released-dl.current.txt; + url1=; container creation template for tim-testlocal-machine-1 failed; Error creating container tim-testlocal-machine-1
<thumper> WTH
 * thumper moves from current work back to trunk
<wallyworld> still on precise?
<thumper> saucy
<thumper> wallyworld: having issues with trusty?
<wallyworld> thumper: i tried local on trusy last thing friday and it didn't work
<wallyworld> but didn't look into it
<wallyworld> andrew and i had a quick look
<wallyworld> nothing jumped out as being wrong but we didn't deep dive
<thumper> hang on...
<thumper> I moved back to trunk and now it is working
<thumper> ...
<wallyworld> trusty as host, precise containers
<thumper> I didn't touch this area though
<thumper> so pretty confused right now
<wallyworld> isn't that always the way
<thumper> ah fark
<thumper> I think I know what it is
<thumper> I have a fake https-proxy set
<thumper> to "rubbish"
<thumper> and I bet lxc is trying to download the latest server using the proxy
<wallyworld> ha ha ha
<thumper> that is kinda funny
<thumper> in a terrible way
<wallyworld> the proxy stuff works :-)
<thumper> hehe, that's it
<thumper> I guess the proxy works
 * thumper proposes
<thumper> actually
<thumper> I may break this up as I broke something before
<thumper> wallyworld: https://codereview.appspot.com/57590043/  simple fix for my fubar
 * wallyworld looks
<wallyworld> thumper: environs/config/config.go - are those new methods related to this mp? am i missing something?
<thumper> wallyworld: ah, they may be used in the next
<thumper> but I thought they were for that one
<thumper> sorry
<thumper> I'm just proposing the next
<wallyworld> np, thought i was being dumb
<thumper> wallyworld: and if you feel like it: Rietveld: https://codereview.appspot.com/57600043
<wallyworld> looking
<wallyworld> thumper: so we could later on move existing clients to use the new common fasÃ§ard?
<thumper> could
<thumper> and faÃ§ade
<wallyworld> bah, can't spell
<wallyworld> thumper: not sure if you agree, i find !a || !b easier to read than !(a && b), especially if the latter is split over two lines, where by !a i mean a != foo etc
<thumper> wallyworld: I don't care that much
<wallyworld> yeah, me either was just a thought
<wallyworld> took me a couple of scans to grok it
<wallyworld> cause of the line break and (
 * thumper nods
<thumper> happy to change if you think it'll make a difference
<wallyworld> thumper: i didn't lgtm because we're missing a test for jujud
 * thumper sighs
<wallyworld> sorry
<thumper> and how do you suggest we test it?
<wallyworld> yeah
<wallyworld> there's some existing examples in machine_test
<wallyworld> basically start a jujud and check for an expected result
<wallyworld> similar code to the worker test itself
<thumper> hmm...
<thumper> ok
<wallyworld> but a cut down version
<wallyworld> just to test the wiring up of it all
<wallyworld> not a blovker, but it would be good not to have the "first" bool required
<wallyworld> or maybe it is essential here, not sure. but other workers don't seem to need it, but i could be mis remembering
<wallyworld> it just complicates things
<wallyworld> oh balls, gotta run - Belinda's car door is stuck and i have to drive to help her out
<wallyworld> i'll check back a bit later
<davecheney> oh bollocks
<davecheney> ... obtained []charm.CharmRevision = []charm.CharmRevision{charm.CharmRevision{Revision:23, Sha256:"6645c56965290fc0097ea9962a926e04b8c5b1483f2871dce9e33e9613e36dbd", Err:error(nil)}, charm.CharmRevision{Revision:23, Sha256:"6645c56965290fc0097ea9962a926e04b8c5b1483f2871dce9e33e9613e36dbd", Err:error(nil)}, charm.CharmRevision{Revision:23, Sha256:"6645c56965290fc0097ea9962a926e04b8c5b1483f2871dce9e33e9613e36dbd", Err:error(nil)}}
<davecheney> ... expected []charm.CharmRevision = []charm.CharmRevision{charm.CharmRevision{Revision:23, Sha256:"2c9f01a53a73c221d5360207e7bb2f887ff83c32b04e58aca76c4d99fd071ec7", Err:error(nil)}, charm.CharmRevision{Revision:23, Sha256:"2c9f01a53a73c221d5360207e7bb2f887ff83c32b04e58aca76c4d99fd071ec7", Err:error(nil)}, charm.CharmRevision{Revision:23, Sha256:"2c9f01a53a73c221d5360207e7bb2f887ff83c32b04e58aca76c4d99fd071ec7", Err:error(nil)}}
<davecheney> #gccgo
<dimitern> fwereade, hey, i've updated https://codereview.appspot.com/53210044/, can you take a look whether it's good to land?
<fwereade> dimitern, sure, thanks
<fwereade> dimitern, reviewed,bbs
<dimitern> fwereade, ta
<davecheney> lucky(~) % juju destroy-environment ap-southeast-2 -y
<davecheney> ERROR state/api: websocket.Dial wss://ec2-54-206-142-42.ap-southeast-2.compute.amazonaws.com:17070/: dial tcp 54.206.142.42:17070: connection refused
<davecheney> but it did destroy the environment
<jam> davecheney: I believe it tries to contact the environment to check if there are any manually registered machines before nuking it all from the client side, but even if it fails, it still nukes from client side
<davecheney> but why did it fail ?
<davecheney> was the bootstrap machine already nuked >
<jam> dimitern: standup ?
<dimitern> fwereade, updated https://codereview.appspot.com/53210044/
<natefinch> rogpeppe: want to talk now?
<rogpeppe> natefinch: good plan, yes
<rogpeppe> natefinch: i just went back into the hangout
<natefinch> rogpeppe: cool brt
<fwereade> rogpeppe, btw, re SOAP, I remember enjoying http://wanderingbarque.com/nonintersecting/2006/11/15/the-s-stands-for-simple/
<jam> fwereade: https://codereview.appspot.com/56020043 you had asked for a deprecation warning (when supplying -e to destroy-environment), care to check the spelling and see if you like how I worded it?
<fwereade> dimitern, re {placeholder:false}, might a {$ne {placeholder:true}} work as a general replacement?
<fwereade> jam, ack
<fwereade> jam, LGTM
<dimitern> fwereade, and pendingupload as well
<dimitern> fwereade, i'll try it on my local mongo and if it works I could change it
<jam> fwereade: thanks
<fwereade> dimitern, reviewed
<dimitern> fwereade, tyvm
<fwereade> oh *fuck* these jujud tests
<fwereade> and also that provisioner one
 * fwereade will sort out the provisioner one but needs a volunteer for the jujud one
<natefinch> fwereade: delete them.  Tests are for inferior programmers anyway.
<fwereade> natefinch, well volunteered!
<natefinch> fwereade: which jujud one?
<fwereade> natefinch, there's a class of machine agent test failure
<fwereade> natefinch, happens in a few different ways now I think
<fwereade> natefinch, we try to test that a machine agent works by testing the side-effects of particular jobs
<fwereade> natefinch, but the MA isn't set up quite right, and so the api job barfs, and kills all the others
<natefinch> fwereade: sounds like fun
<fwereade> natefinch, and it's a matter of luck whether they managed to express their side effect in time
<fwereade> natefinch, btw, you don't actually have to volunteer, HA is at least as important
<rogpeppe> natefinch: http://play.golang.org/p/WwGvP5RUbM
<rogpeppe> natefinch: it could probably go in its own package somewhere
<rogpeppe> natefinch: perhaps in utils
<natefinch> fwereade: if I was less behind in the HA stuff I'd be happy to volunteer.... but if someone else can do it, that would probably be better  for the schedule
<rogpeppe> natefinch: does that make sense as a primitive?
 * rogpeppe needs to lunch
<natefinch> rogpeppe: looks good though I'm not sure I'm entirely happy about genericizing it with the interface{} ...
<rick_h_> TheMue: morning, I wanted to check on the status of the debug-log work. I know rogpeppe brought up some ideas for potentially more effecient ways to handle communication.
<rick_h_> TheMue: anything I can/should note on our tracking card for this during out standup?
<TheMue> rick_h_: yep, roger and william agreed on this new approach and I'm now finishing the outline (there are some todos left)
<rick_h_> TheMue: cool, thanks for the update.
<TheMue> rick_h_: most looks good so far because this new approach avoids much of the problems of the old one
<rick_h_> TheMue: always a good thing, glad to hear it
<jam> fwereade: did you get a chance to talk with Tim about planning for capetown?
<fwereade> jam, he got a message from me about it, but only while I was briefly midnightly awake and he was at the gym
<jam> fwereade: so its all sorted out, then :)
<rogpeppe> natefinch: i know what you mean, but it's general enough that it might be useful for other things. it's a bit more mechanism than i'd like to see in cmd/jujud directly, and as a separate package i wouldn't really want it to depend on the agent package
<fwereade> jam, mu -- I have pushed mramm to figure out what else we may need, but at least thumper is aware of where he needs to add things that he thinks of
<rogpeppe> natefinch: you could always write a thin type-safe wrapper on top if the interface{} thing gets you down
<fwereade> rogpeppe, can you remember what emitter of NotProvisionedError justifies returning true from isFatal in the machine agent?
<fwereade> rogpeppe, if *that particular* MA is not provisioned, sure, that's a problem
 * rogpeppe looks to see what emits NotProvisionedError
<fwereade> rogpeppe, and the fact that other workers are emitting it is I think *also* a problem
<fwereade> rogpeppe, but they should surely be restricting their problems to themselves
<rogpeppe>  fwereade: yeah, that may well be problematic. I think we should probably use a more specific error, probably defined in cmd/jujud, and transform from NotProvisionerError into that at the appropriate place only
<rogpeppe> fwereade: and take NotProvisionedError out of the list of fatal errors
<rogpeppe> fwereade: i *think* that the specific case that we were thinking about there is the one returned by apiRootForEntity
<fwereade> rogpeppe, excellent, that matches my rough analysis
<fwereade> rogpeppe, thanks
<rogpeppe> fwereade: cool
<fwereade> would appreciate a quick look at https://codereview.appspot.com/57740043 -- it's the flaky provisioner tests
<fwereade> the jujud ones demand a bit more obsessive/paranoid care
 * fwereade succumbs to rage against the machine agent, goes for walk
<mgz> the battle of los agents
<rogpeppe2> anyone here know about lxc?
<rogpeppe2> we're seeing this error from lxc-start on a machine:
<rogpeppe2> 2014-01-28 13:35:27 ERROR juju.provisioner provisioner_task.go:399 cannot start instance for machine "3/lxc/5": error executing "lxc-start": command get_init_pid failed to receive response
<fwereade> natefinch, btw, a thought: it's *vital* that we start (non-bootstrap) state DBs only in response to info from the API -- it's only the API conn that does the nonce check and is therefore safe from edge case failures in the provisioner accidentally starting two instances for one machine
<fwereade> natefinch, I know that's what we're doing anyway, but it's another reason not to mess around with cloudinit
<natefinch> fwereade: ahh, yeah, good point
<natefinch> rogpeppe2: you may be right about the interface there, but I still am hesitant to add a bunch of code and some complexity for a feature we don't even support right now.  It's like, either do this whole watching thing, or just call a function inline.
<sinzui> fwereade, jam, I created a branch from the last rev that CI blessed then merged mgz openstack/mgo fixes.
<rogpeppe2> natefinch: i'm not sure what you mean by "just call a function inline" there
<sinzui> fwereade, jam. since I created a new branch, I created series 1.18 and moved 1.17.1 to that series. https://launchpad.net/juju-core/1.18
<natefinch> rogpeppe2: given what fwereade said above... are we not just going to be calling the API to see if we should be a state server when MachineAgent.Run is called
<sinzui> fwereade, jam, I am not happy with this situation. If either of you are not happy, we should talk about our options for a 1.17.1 release
<rogpeppe2> natefinch: we can't do that if there's only one state server machine agent
<rogpeppe2> natefinch: that's kind of the whole point of the design i've suggested
<rogpeppe2> natefinch: i don't think the SharedValue stuff is unreasonable complexity (it's been well tested in the past, under another guise)
<rogpeppe2> natefinch: i'd be happy to commit it with tests if you don't fancy doing that
<natefinch> rogpeppe2: that's fine, it just seems like we're doing all that instead of writing func SetUpStateServer()
<rogpeppe2> natefinch: we're moving towards a design that enables future stuff, rather than cluttering the existing design with more stuff.
<natefinch> rogpeppe2: it doesn't seem like clutter when it's code we'll need either way. The future stuff we may never get to.
<natefinch> rogpeppe2: can you explain this a bit more: we can't do that if there's only one state server machine agent
<rogpeppe2> natefinch: perhaps you could sketch some pseudocode for what you think SetUpStateServer should do?
<natefinch> rogpeppe2: that's probably what is tripping me up
<rogpeppe2> natefinch: if there's only one state server machine agent, there's no API to connect to
<natefinch> rogpeppe2: if there's no API to connect to, mongo can't sync its data from anywhere either
<natefinch> rogpeppe2: I don't think this code applies to the bootstrap node
<rogpeppe2> natefinch: there is no such thing as the "bootstrap node" in an HA environment
<rogpeppe2> natefinch: well, that's not entirely true
<mgz> sinzui: thanks for that
<rogpeppe2> natefinch: but the bootstrap node is only important at bootstrap time
<mgz> I don't think I have any cunning solutions either unfortunately
<natefinch> rogpeppe2: other than the bootstrap node, there must already be a state server in existence when new state servers come up
<rogpeppe2> natefinch: not necessarily
<rogpeppe2> natefinch: i want to allow for the possibility of going from 3 servers to 1.
<natefinch> rogpeppe2: then we're boned anyway, because they can't sync the mongo data
<rogpeppe2> natefinch: not necessarily
<TheMue> rogpeppe2: btw HA, where does the all-machines.log reside then?
<rogpeppe2> natefinch: the peer group logic i've written should allow it
<rogpeppe2> TheMue: that's a good question and one we haven't resolved yet
<TheMue> rogpeppe2: hehe, ok
<rogpeppe2> TheMue: we perhaps sync it to all state server nodes, but we need to think about it
<natefinch> rogpeppe2: afk a sec, sorry, brb
<TheMue> rogpeppe2: yes, may be the right solution. also we still have no logrotate, don't we?
<natefinch> rogpeppe2: going from 3 to 1 and putting manage environ on existing machines are not things we need to deliver right now, and I don't think there's any throw away code we'd need to write to deliver what is required without adding this code.  I guess if you want to commit the shared valuestuff with tests, that's fine with me... I just don't really want to take on *any* additional code burden right now.
<natefinch> TheMue: yes, there is no logrotate
<rogpeppe2> natefinch: perhaps you could paste some pseudocode with your idea for your suggested solution
<fwereade> rogpeppe2, TheMue: unless there's a serious problem with the approach I think we should just fix the rsyslog conf to push to all state servers
<fwereade> TheMue, yes, there is no logrotate and we really need it
<rogpeppe2> fwereade: +1000
<fwereade> TheMue, don't suppose you're currently bored
<fwereade> ;p
<rogpeppe2> fwereade: what happens when a new state server comes up?
<rogpeppe2> fwereade: (do we lose all the previous log on that state server?)
<fwereade> rogpeppe2, I think it's ok that new state servers won't have logs from before they existed, if that's what you mean?
<rogpeppe2> fwereade: it is
<rogpeppe2> fwereade: i'm not sure that's really acceptable, tbh
<rogpeppe2> fwereade: but if you think it is, i'll go with it
<natefinch> rogpeppe2: in machineagent.Run:  open API, check this machine's jobs.  If manageEnviron, install & run mongo upstart script.  (if we can't assume mongo is installed, throw in an apt-get install in there).
<natefinch> rogpeppe2: I know it's naive, but I'm not understanding under what circumstances it'll fail in the stuff we need to sup[port for 14.04
<fwereade> rogpeppe2, https://codereview.appspot.com/54950046 -- I don't think this is necessarily the *best* solution, but it involves no API changes and AFAICT it resolves the jujud flakiness -- opinions?
 * fwereade needs to eat something, would be most grateful to return and see a review of https://codereview.appspot.com/57740043 as well
<natefinch> rogpeppe2: sorry if I'm asking the same thing over and over.  There's obviously something I keep misunderstanding.
<rogpeppe2> natefinch: one mo, i'm just doing a sketch, so you can see how my suggestion actually simplifies the existing code
<natefinch> rogpeppe2: that's fine, and thank you for helping me understand.
<TheMue> fwereade: I've got the slight feeling the the topic debug logging will accompany me for some time. ;)
<fwereade> mgz, is the bot wedged? https://code.launchpad.net/~waigani/juju-core/remove-local-shared-storage/+merge/202789 was approved yesterday -- but I'm sure I saw it doing something earlier today
<rogpeppe2> natefinch: here's the kind of thing i'm thinking of: http://paste.ubuntu.com/6832569/
<mgz> fwereade: I'll have a look
<rogpeppe2> natefinch: (try a diff against cmd/jujud/machine.go)
<natefinch> rogpeppe2: looking
<sinzui> rogpeppe2, mgz, natefinch  Do either of you have time to review my branch to inc juju to 1.17.2?  https://codereview.appspot.com/57750043
<fwereade> sinzui, LGTM
<sinzui> thank you fwereade
<mgz> fwereade: the bot is cycling through and looking for proposals fine
<mgz> fwereade: waigani just didn't set a commit message
<mgz> we can do that and it will land
<rogpeppe2> fwereade: i'd be interested in what you think about my suggestion to nate above, machine agent changes (http://paste.ubuntu.com/6832569/)
<fwereade> mgz, doh, sorry
<rogpeppe2> natefinch: does it make some sort of sense? it's somewhat more code, but i think it separates concerns better, and there are no special-case hacks
<fwereade> rogpeppe2, I think that looks nice
<natefinch> rogpeppe2: I still don't understand the *why*.  When we first start up in MachineAgent.Run, we can call the API right then and determine if we need to run mongo, and do so right then.  I'm not sure what we get by making a continuous watcher thingy, since we don't currently support changing a machine from a non-state server to a state server.
<rogpeppe2> natefinch: we can't open the API if we're supposed to be running the API
<rogpeppe2> natefinch: unless you add special case hacks for machine 9
<fwereade> rogpeppe2, surely we *can*, though
<rogpeppe2> machine 0 even
<rogpeppe2> fwereade: well, we *can*, and that's what my suggestion does
<natefinch> rogpeppe2: yes, one special case hack for THE special case in the system
<fwereade> natefinch, every special case I see for machine 0 makes me sad
<natefinch> fwereade: I agree
<fwereade> natefinch, this reduces that special case to setting up the agent conf so that machine 0 alone already knows it's meant to run the state worker
<fwereade> natefinch, everything else gets it via the api
<fwereade> natefinch, (ultimately via the api)
<rogpeppe2> natefinch: in your case you have to have a completely separate path for opening the API in MachineAgent.Run, and then you'll start the APIWorker which then needs to open it again
<natefinch> fwereade: maybe I'm missing that part because of all the magic watcheryness.
<fwereade> natefinch, rogpeppe2: yeah, my opinions are predicated on the watching all being sane, and the agent conf all being properly goroutine safe, etc
<rogpeppe2> fwereade: of course
<fwereade> natefinch, rogpeppe2: but I think it's a suitable channel for this sort of information
<natefinch> rogpeppe2: it seems like you're arguing about the contents of needsStateWorker, which doesn't seem to be in the code you're talking about
<rogpeppe2> natefinch: needsStateWorker is just a "does machine jobs contain JobStateWorker?"
<rogpeppe2> natefinch: (or however that info is stored in the config)
<natefinch> rogpeppe2: right, fine.  Ok.  Why do we need 150 lines of magic watcherness rather than just checking the jobs in machineAgent.run?
<rogpeppe2> natefinch: because we need to watch that stuff *anyway*
<natefinch> rogpeppe2: aren't we already doing that?
<rogpeppe2> natefinch: because we need to save the addresses
<rogpeppe2> natefinch: i don't think so
<natefinch> fwereade: aren't the addresses already in the config?
<natefinch> fwereade: ahh, I guess if the config changes
<natefinch> fwereade: who changes the config and how?
<fwereade> natefinch, :179
<fwereade> natefinch, I think it depends on infrastructure that isn't written yet ( rogpeppe2 ?) but the shape of it looks sane to me
<rogpeppe2> fwereade: the infrastructure that's not written yet is outlined in newConfigWatcher
<rogpeppe2> fwereade, natefinch: it's pretty bog standard stuff - just watch that stuff and change the config appropriately
<fwereade> rogpeppe2, indeed, I was just checking there wasn't something I'd completely missed :)
<rogpeppe2> natefinch: so, the config changes because that watcher changes it, because something that it's watching that needs to go into the config has changed
<rogpeppe2> natefinch: FWIW i've been wanting to move towards this kind of structure in the machine agent for ages
<rogpeppe2> natefinch: and i'd much prefer to do it now rather than twist the structure more
<natefinch> rogpeppe2, fwereade: I think what was confusing me was that the worker functions look like they're just called once, but they're runners/workers so they keep getting called over and over.  I'm not entirely sure why I couldn't put ensureMongoServer inside StateWorker or something
<rogpeppe2> natefinch: the other problem with "just" connecting to the API in MachineAgent.Run is that you have to be careful to allow the agent to be stopped, and all that logic rapidly becomes quite complex (and duplicates logic that's already there elsewhere)
<rogpeppe2> natefinch: you definitely could do that
<rogpeppe2> natefinch: but not in the current code
<rogpeppe2> natefinch: well, actually, it would probably work
<rogpeppe2> natefinch: but you'd still need the config watching stuff
<rogpeppe2> natefinch: and tbh i don't really like the current twistiness with ensureStateWorker
<fwereade> would someone please review https://codereview.appspot.com/57740043 so I can deflake the bot a bit more?
<natefinch> rogpeppe2: I'm definitely not a fan of passing around an anonymous function that passes through another anonymous function
<natefinch> rogpeppe2: I believe that you and William know what the heck you're talking about, and ignore whatever last 2% I'm missing.
<natefinch> rogpeppe2: er rather I'm going to have to ignore what I'm missing.
<natefinch> rogpeppe2: my sleep's been pretty terrible the last few days which isn't helping anything
<rogpeppe2> natefinch: np at all
<rogpeppe2> fwereade: LGTM
<fwereade> rogpeppe2, cheers
<rogpeppe2> fwereade: according to the Go oracle, there are only three places that call agent.Config.Write - BootstrapCommand.Run, MachineAgent.APIWorker and UnitAgent.APIWorkers
<rogpeppe2> fwereade: this corresponds to my intuition
<rogpeppe2> fwereade: and means that the correct place to put the config writing code *is* in the APIWorker
<dimitern> fwereade, final look at https://codereview.appspot.com/53210044/ ? updated as suggested
<dimitern> bbiab
<natefinch> thumper: got that errgo thing all written yet? :)
<thumper> natefinch: no, but will do a little today :)
<thumper> fwereade: around?
<thumper> natefinch: what is the simplest way to insert a value at the start of a slice?
<natefinch> thumper: s = append([]type{ val }, s...)
<thumper> hmm...
<thumper> I was hoping there was a nicer way, but that is what I'll do
<natefinch> thumper: yeah, it's not great, but there's no real magic you can do with it
 * thumper nods
<natefinch> thumper: It occurs to me, if prepending is something you're doing a lot of, it's probably better to just append, and treat the back as the front, if you know what I mean
<thumper> natefinch: I do, but most of the rest of the operations are starting from the most recent
<thumper> I want the equivalent of push_front
<natefinch> yeah, well, push_front is always ugly, so, there you go :)
<natefinch> thumper: unless you use a linked list.... and you should never use a linked list :)
<thumper> :)
<hazmat> do we have any size  recommendations on state-server size ?
<hazmat> ie 4 core / 16gb for 500 nodes env?
 * hazmat rereads notes from jam scale test in nov
<wallyworld> thumper: 2 things. 1. you got the loggo user name? 2. i hate all the code churn just to relocate a friggin project 3. don't forget to update the bot 4. i can't count
<thumper> wallyworld: yes I got the loggo name on github, agree on the churn
<thumper> wallyworld: hai by the way
<wallyworld> hi :-)
<wallyworld> i about to take lachie to his first day of high school, will be bbiab
<wallyworld> i'll look at your worker code review once you add the test :-)
<hazmat> there's a couple of tools out there for package name rewriting
<wallyworld> hazmat: sure, but it's sad it actually *changes the code*
<wallyworld> all that code churn
<wallyworld> sucks
<wallyworld> as opposed to just updating a depedencies file
<wallyworld> like in python
<sinzui> natefinch, Did you say I should remove the mongodb upstart script from /etc/init ?
<natefinch> sinzui: yeah... I don't think it actually breaks anything, but I think we removed it from the servers we deploy, IIRC.
<natefinch> sinzui: frees up some disk space and memory etc.
<fwereade> thumper, heyhey
<thumper> fwereade: hey, how are you doing?
<fwereade> thumper, not bad
<fwereade> thumper, landed some actual code today, would you believe?
<sinzui> since we are seeing the port taken in CI. I like the thought that there is no reason for it to be up if there is no test running
<thumper> fwereade: wow
<davecheney> sinzui: ping
<sinzui> hi davecheney
<davecheney> sinzui: are we doing a hangout now ?
<davecheney> oh hey
#juju-dev 2014-01-29
<axw> thumper: you got loggo released I take it?
<axw> on github
<thumper> the github name?
<thumper> yes
<axw> cool
<axw> thumper: about the configstore/chown thing. I was going make it so we couldn't use sudo/root at all; now if you use sudo you'll end up with ~/.juju/environments owned by root. do you think it's worth the trouble, or people just have to learn to not do that?
<thumper> I've been thinking a little about that
<thumper> what if it is root?
<thumper> or
<thumper> what testing was doing:
<axw> as in, not using sudo at all?
<thumper> sudo su other
<thumper> then that shell does "juju bootstrap"
<thumper> I'm not sure we should be that opinionated
<axw> well I think we'd have to check the current uid, and only do something if uid == 0
<axw> but yeah... I'm starting to think it's just overly complicating things
<thumper> I think it is safer to just leave it how it is now
<thumper> and not add anti-features
<axw> ok
<axw> do you think I should take out the bit in local that prevents root then?
<axw> I think I may as well, if we're not doing it elsewhere
<axw> thumper: by which I mean, the calls to ensureNotRoot in provider/local/environ.go
<thumper> that one I think may be worth keeping for now
<thumper> to train people to do it right
<thumper> otherwise people doing what they have always done will create broken systems
<wallyworld> thumper: axw: do we need to discuss that critical destroy-env bug?
<axw> yeah ok. if they were doing it before, then they probably have existing dirs anyway
<axw> wallyworld: the what now? :)
 * wallyworld digs up the bug number
<thumper> wallyworld: can you just chat with axw about it? I'm in the middle of something
<wallyworld> sure
<wallyworld> https://bugs.launchpad.net/juju-core/+bug/1272558
<_mup_> Bug #1272558: destroy-environment shutdown machines instead <ci> <destroy-machine> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1272558>
<axw> looking
<wallyworld> this one has been causing the CI folks a lot of grief
<wallyworld> i haven't looked closely just yet
<wallyworld> trying to get stuff done for tomorrow
<axw> huh
<wallyworld> they were suspecting recent destroy env work
<axw> probably
<wallyworld> but that's just heresay
<axw> I will see if I can reproduce
<wallyworld> ok ta :-)
<thumper> axw, wallyworld: now that we are four (when waigani is around), how do you feel about a daily standup around this time?
<wallyworld> sure
<thumper> aim is for 5-10min max
<wallyworld> \o/
<thumper> but gives us a way to touch base and know if anyone is blocked
<axw> thumper: sgtm
<axw> where is waigani anyway?
<wallyworld> his sheep died
<thumper> shall we do a quick stand up now?
<axw> sure
<wallyworld> ok
<thumper> https://plus.google.com/hangouts/_/7ecpjir6j9gl1e7ttn38o7hq40?hl=en
<axw> thumper: in errgo, did you consider having a Annotator type and DefaultAnnotator var, rather than RecordLocationWithAnnotations/FilePathElements globals?
<thumper> briefly
<axw> that is more "idiomatic Go"
<thumper> but I thought this could be something that you want enabled for debug builds
<thumper> and disabled for production
<thumper> how do you do that in idiomatic Go?
<axw> so, IMHO, you'd do that in the user. i.e. you have juju-core/errgo or something, which uses errgo/errgo but has its own configuration
<axw> juju-core/errors would be a good place for it actually.
 * thumper sucks in his cheeks
<thumper> however then the runime.Caller is all fucked up
<thumper> I'll finish this up then we can bikeshed :-)
<axw> no worries
<thumper> go has short circuit boolean evaluation right?
<axw> yes
<thumper> wallyworld: are the trees I need to update on the gobot in /home/tarmac/trees?
<wallyworld> i think so yes
<wallyworld> there was another trees dir also but i can't recall without looking what it was for
<axw> thumper: it just occurred to me that local provider no longer prevents units on machine 0, since it's using the common state init code
<axw> I'll put a bug in for that...
<thumper> yeah... I did mean to ask about that :)
<axw> wallyworld: why can't people just set image-metadata-url to cloud-images.ubuntu.com/daily or whatever? rather than another attribute for the stream
<wallyworld> because we don't want them having to know about cloud-images.u.com per se
<wallyworld> and
<wallyworld> the stream name is used to form the product id
<axw> ah yeah
<wallyworld> it's a bit messy
<waigani> axw: hello :)
<axw> waigani: heya
<waigani> Can I ask you a Go question?
<axw> yes go for it
<waigani> http://play.golang.org/p/W1UxbGdERJ
<axw> looking
<waigani> My question is in the comment, just playing with channels to understand them better
<waigani> I get that the for loop blocks and waits for a new val to be added to the channel
<axw> waigani: the program is exiting before the 4th goroutine is executed
<waigani> yes but why?
<axw> you're not waiting for the goroutines to complete, so the main func just falls off the end
<axw> one sec
<waigani> sure
<axw> so "c" is an unbuffered chan, which means that the sender blocks until the receiver is ready. the background goroutine gets the value "4", but there's no guarantee it does anything with that value before the scheduler switches back to the main function and then exits
<axw> does that make sense?
<waigani> I thought <-c within the for loop blocks and waits for any new vals added to the channel?
<axw> yes it does. so the value was transferred. the goroutine has the value 4, it just didn't get as far as printing it
<axw> waigani: http://play.golang.org/p/9GYN3InLIH
<rogpeppe2> axw: ping
<axw> rogpeppe2: pong
<rogpeppe2> axw: i've been looking at making the bootstrap stuff a bit better when things go wrong, and i'm wondering what you think a good approach is here
<rogpeppe2> axw: in particular, when something goes wrong, we just see "Exit status 1" and no output from the failed commands
<rogpeppe2> axw: (and then of course the instance is instantly shut down, so there's no way to find out what the issue might be)
<axw> rogpeppe2: yeah I was planning to do something about that at some stage... ;)
<axw> umm
<rogpeppe2> axw: i'm thinking of saving all Stdout and Stderr and spitting it out when things fail
<axw> rogpeppe2: probably tail cloud-init-output.log
<axw> hmm or that I guess
<axw> rogpeppe2: all stdout/stderr goes into /var/log/cloud-init-output.log, IIRC
<axw> so you could just tail/cat that I think
<rogpeppe2> axw: ah, that's useful to know - i thought that might not be happening any more
<rogpeppe2> axw: but how many lines to tail?
<axw> rogpeppe2: yeah... maybe just cat it on failure?
<axw> hmm
<rogpeppe2> axw: the whole thing's going to be quite big, isn't it?
<axw> yes I think it will get pretty big, with all the apt-get update output, and so on
<axw> alternatively we could grab the file and write to disk, but that's only useful when calling from the CLI
<rogpeppe2> axw: one possibility might be to try to work out which command failed and print the output from only that
<rogpeppe2> axw: do we send commands one-by-one, or as a big bunch?
<axw> rogpeppe2: no, as a single script
<axw> that would be a good default
<axw> there may be cases where command failure needs more context, but I can't think of any
 * axw thinks how to get there
<rogpeppe2> axw: so, it would presumably be possible to print some distinctive text before each command, then print output from the last occurrence
<axw> yes, that would work
<rogpeppe2> axw: ok, i'll perhaps do that today
<axw> cool
<rogpeppe2> axw: (we've seen that problem quite a bit here)
<fwereade> axw, if you're talking about the CA cert in https://codereview.appspot.com/56560043/ I think everyone *does* know it
<axw> fwereade: oh? I thought it was a secret. it gets stripped out before we bootstrap...?
<fwereade> axw, the private key does,for sure
<rogpeppe2> axw: it shouldn't involve changing anything outside cloudinit/sshinit, right?
<fwereade> axw, but agent conf always has the CA cert for the environment, and I'm pretty sure it's available over the api as well
<axw> rogpeppe2: right, I think that's the most sensible place to do it, and it should be contained
<rogpeppe2> axw: the CA cert is public info
<axw> fwereade: makes sense.
<axw> I had it in my mind we didn't publish either, but of course we have to
<rogpeppe2> it would be good if the CA cert contained the environment UUID but that's for another day
<fwereade> rogpeppe2: yeah, +1000
<thumper> hi rogpeppe2
<fwereade> tomorrow? ;p
<thumper> rogpeppe2: are you in bluefin?
<rogpeppe2> thumper: hi. yeah.
<thumper> rogpeppe: have you seen mramm around yet today?
<rogpeppe> thumper: i have
<thumper> rogpeppe: could you tell him that I have a few questions for him and can he get on irc?
<thumper> :)
<rogpeppe> thumper: ok; i think he's in the scrum-of-scrums meeting right now, but will collar him after that
<thumper> scrum of scrums?
<rogpeppe> thumper: his term
<thumper> sounds dangerous
<rogpeppe> thumper: meeting of the team leads
<axw> fwereade: re the ssh keys change, do you just mean you'd like to see an explicit "-i" for each of the default keys in ~/.ssh?
<fwereade> axw, I'm wondering whether we should use explicit -i a bit more than we do, yeah
<axw> fwereade: it's not necessary for the default keys, which is why I don't add it
<axw> they just get tried anyway
<fwereade> axw, ok, yeah, if they have a .ssh then they probably do have ssh installed :)
<fwereade> axw, cheers
<axw> fwereade: ah you meant for using in go.crypto?
<fwereade> axw, yeah, I was just wondering if there was any gap in our coverage as it were
<axw> actually I did that at one point, but then realised that it wouldn't work when keys are encrypted (require a keyphrase)
<axw> which is (I hope!) most of the time
<rogpeppe> thumper: i saw what seemed like a local provisioner bug yesterday BTW
<fwereade> axw, so one would indeed hope ;)
<axw> people can drop/symlink whatever keys they want to use into ~/.juju/ssh though
<thumper> rogpeppe: oh?
<rogpeppe> thumper: perhaps you might have an idea of what might be going on
<thumper> I'm currently upgrading to trusty
<thumper> to make sure they get ironed out quick smart
<rogpeppe> thumper: so, lxc-start was failing (for some as-yet-unknown reason)
<thumper> hmm..
<thumper> I've seen a few weird ones
<rogpeppe> thumper: so we could see the instances, and each one with a state holding the lxc-start status
<rogpeppe> thumper: but we couldn't get the instances out of that state
<thumper> ?!?
<thumper> weird
<rogpeppe> thumper: one mo, i'll try to find the bug number
<rogpeppe> thumper: no bug reported yet, oh well
 * thumper shrugs
<thumper> if you see it again, please file one
<thumper> and assign it to me
<thumper> I'll see it then
<rogpeppe> thumper: the output status is in agent-state and somnething like "lxc-start: cannot get init-pid-status"; the problem seems to be something to do with network bridge naming
<rogpeppe> thumper: but that wasn't the particular provisioner issue i was thinking about
<thumper> hmm...
<thumper> ok
<rogpeppe> thumper: here's the error that we saw:
<rogpeppe> machine-3: 2014-01-28 15:21:12 ERROR juju.container.lxc lxc.go:129 container failed to start: error executing "lxc-start": command get_init_pid failed to receive response
<thumper> rogpeppe: this is on maas?
<thumper> is it still using precise?
<rogpeppe> thumper: might be trusty actually
<thumper> do we have trusty charms?
<rogpeppe> thumper: no, bootstrap node is trusty, unit node is precise
<thumper> rogpeppe: is this local provider?
<rogpeppe> thumper: no
<thumper> maas?
<rogpeppe> thumper: yes
<thumper> ok
<thumper> I'll look tomorrow
<thumper> and see if I can find anything
<rogpeppe> thumper: the particular issue i'm concerned about isn't actually that one
<thumper> rogpeppe: stick it in an email to juju-dev or just me if you prefer
<rogpeppe> thumper: will do
<rogpeppe> there's another bug i've just seen
<rogpeppe> juju bootstrap doesn't appear to work with --upload-tools in 1.17.1
<rogpeppe> anyone got an idea of what might be happening here? https://pastebin.canonical.com/103759/
<thumper> rogpeppe: mramm mentioned that it looks like the network bridge isn't coming up
<thumper> rogpeppe: I wonder if the behaviour has changed
<thumper> we are kinda abusive in the maas provider
<thumper> where we restart networking
<mramm> the other thing that we should work out
<thumper> robie basak suggested a different approach
<thumper> where we go "ifdown eth0", make the mods, bridge eth0 with br0, then go "ifup br0"
<thumper> and that would be better
<thumper> but I never got time to try it
<thumper> rogpeppe: is that enough info?
<rogpeppe> thumper: not really. i had difficulty following the lxc container code, i'm afraid
<rogpeppe> thumper: a bit of a twisty maze
<thumper> rogpeppe: this is not in the container code
<thumper> but inside the maas provider
<thumper> where we create the cloud-init scripts
<rogpeppe> ah ok
<thumper> as we create a different network bridge for the host
<thumper> so the containers inside can get to the maas dhcp server
 * thumper is being called to head to the lounge
<thumper> good luck
<rogpeppe> thumper: ok
<rogpeppe> thumper: the ifup change was how they fixed it
<rogpeppe> thumper: (manually, of course)
<rogpeppe> fwereade: it looks like the simplestreams stuff is borked in 1.17.1. or *something* is anyway.
<rogpeppe> does anyone know if streams.canonical.com is supposed to have useful data these days?
<fwereade> rogpeppe, worst-case sinzui and utlemming and I will sort it out when we're together this weekend, but I was hoping something would land sooner than that
<fwereade> rogpeppe, looks to me like it still just completely lacks content
<rogpeppe> fwereade: currently we're unable to bootstrap at all
<fwereade> rogpeppe, --source?
<rogpeppe> fwereade: hmm, i don't know about --source
<fwereade> rogpeppe, it's in the 1.17.1 release notes
<mramm> ok, so we *need* to get this streams.canonical.com thing sorted out quickly
<mramm> as in before we end up getting hammered for it in Cape Town :/
<mramm> perhaps need is strong.  But it is bad for our users, bad for us, and will result in a bit of ungentle poking in SA.
<rogpeppe> fwereade: you mean --metadata-source, presumably?
<fwereade> rogpeppe, ha, yes, sorry
<fwereade> mramm, so, the issue AIUI is that utlemming is currently crushed under the weight of paid public cloud work
<rogpeppe> fwereade: can you just provide a directory with the tools in?
<fwereade> mramm, I will follow up with arosales when he's online, it seemed on monday that we might be able to steal an hour's work for a quick solution butthat hasn'thappened
<rogpeppe> fwereade: or do you need the metadata there too?
<mramm> ok
<mramm> the alternative is to make juju not broken if it isn't there... :/
<mramm> I know the right thing to to to move us forward
<mramm> but...
<fwereade> rogpeppe, you need metadata
<fwereade> rogpeppe,  If your workflow previously was to download the juju tools to a local
<fwereade> directory, then bootstrap with the --source option to upload the tools to your
<fwereade> environment, you need to call âjuju metadata generate-toolsâ per the previous
<fwereade> example. See âjuju help bootstrapâ for more information.
<mramm> fwereade: juju help bootstrap is nice!
<mramm> fwereade: I also understand the pressure everybody is under
<mramm> fwereade: so it's not a problem, just trying to troubleshoot next week
<fwereade> mramm, yeah, quite so
<rogpeppe> fwereade: juju help bootstrap doesn't say anything about generate-tools
<rogpeppe> fwereade:  and "juju metadata generate-tools --help" doesn't say anything about what's expected
<rogpeppe> fwereade: (i tried it on a directory containing a jujud binary, but it failed to find tools
<rogpeppe> )
<fwereade> rogpeppe, it's the same expected directory structure as juju-dist has always had
<fwereade> rogpeppe, but yeah we should be more explicit about that
<rogpeppe> fwereade: i also tried with --metadata-source, with a directory containing jujud
<rogpeppe> fwereade: ahhh
 * rogpeppe tries to remember what format the directory was in
<fwereade> rogpeppe, just put them in the tools subdir iirc
<rogpeppe> fwereade: no arch/series info?
<rogpeppe> fwereade: just tools/jujud ?
<hazmat> rogpeppe, please capture some notes on what your doing.. and attach to https://bugs.launchpad.net/juju-core/+bug/1267795
<rogpeppe> it aaaallll broookeen
<fwereade> rogpeppe, it's exactly how it's always been -- tgzs with names in the usual format
<rogpeppe> fwereade: ah, ok
<rogpeppe> fwereade: that's not very user friendly
<fwereade> rogpeppe, sorry, tools/releases
<rogpeppe> fwereade: --upload-tools *should* work, right?
<hazmat> rogpeppe,  i know.. that's why we all use upload-tools..  check the contents here (from that bug report) which has the generate-tools structure http://pastebin.ubuntu.com/6726391/
<hazmat> yeah.. tools/releases/tarball.tgz should do it for generate-tools
<fwereade> rogpeppe, you can just download the bits you need from juju-dist -- the fuckedness of streams.c.c is a distinct issue
<fwereade> rogpeppe, and, yes, I thought yu *could* still use upload-tools, I thought this was about using the released tools, sorry
<fwereade> rogpeppe, is *that* broken?
<rogpeppe> fwereade: it seems to be
<rogpeppe> fwereade: we can't bootstrap at *all*
<rogpeppe> fwereade: which is a major blocker currently
<rogpeppe> fwereade: see https://pastebin.canonical.com/103759/
<mramm> shang says that he believes it was all working last night
<hazmat> rogpeppe, this is on trunk?
<mramm> just not working today
<jamespage> sinzui, please can you ensure that any PPA builds are using ~XXX suffixes otherwise the versions will conflict with distro
<rogpeppe> hazmat: this is on 1.17.1 as released
<fwereade> rogpeppe, that is deeply bizarre, it seems to work here, let me dig
<mramm> we are not using EC2, and are on trusty
<hazmat> rogpeppe, k.. testing.. with 1.17.1 tarball from https://launchpad.net/juju-core/+download
<mramm> so there are some important differences
<fwereade> rogpeppe, that "47927a59-4867-442f-8239-ae2d657f4475-tools/releases/juju-" bitis very weird
<rogpeppe> fwereade: yes
<hazmat> fwiw.. bootstrap --upload-tools from (13.10) works with 1.17.1 tarball
<rogpeppe> fwereade: is it possible to use metadata generate-tools to generate metadata about tools in a local directory?
<hazmat> rogpeppe, yes see that pastebin link i sent.. re private clouds  http://pastebin.ubuntu.com/6726391/
<fwereade> rogpeppe, yes: grab the subset of the juju-dist bucket that you want/need, and point generate-tools at the base dir, then use --metadata-source
<rogpeppe> fwereade: i *think* i did that
<hazmat> with -d tools_dir
<rogpeppe> hazmat, fwereade: so what am i doing wrong here? http://paste.ubuntu.com/6836995/
<rogpeppe> juju metadata generate-tools is currently hung on "finding tools"
<rogpeppe> sigh
<hazmat> rogpeppe, can you run it with --debug
<hazmat> and pastebin
<rogpeppe> hazmat: it's fetching all tools from somewhere
<rogpeppe> hazmat: http://paste.ubuntu.com/6837008/
<hazmat> rogpeppe, its hitting streams.cannonical.com
<rogpeppe> hazmat: so is there a way i can tell it to generate metadata from my local directory?
<rogpeppe> hazmat: (from the tools in there)
<mramm> hazmat: fwereade: thanks for helping with this -- it is a blocker for a couple of teams here in london
<hazmat> i thought  so.. but.. looking at it.. it looks like it just wants to store it..
<hazmat> on the local dir, instead of using the local dir for tools, the pastebin docs are sent are confusing in that regard.
<rogpeppe> hazmat: that's how i interpret it too
<hazmat> rogpeppe, i'm a bit more concerned with why --upload-tools is failing
<rogpeppe> hazmat: well, i want to know why normal bootstrap is failing too
<rogpeppe> hazmat: if generate-tools can get stuff from streams.canonical.com, why can't normal bootstrap?
<rogpeppe> wallyworld_: ping
<cgz> thanks for the review axw
<fwereade> rogpeppe, nobody can get anything from streams.c.c at the moment because there is no content :/
<rogpeppe> fwereade: so where's generate-tools getting all this from? http://paste.ubuntu.com/6837008/
<rogpeppe> fwereade: i don't *think* it's juju-dist, because juju-dist doesn't appear to have 1.17.1 in
<hazmat> rogpeppe, 1.17.1 is  there in the testing/tools   @ https://juju-dist.s3.amazonaws.com/
<rogpeppe> ah, it does
<rogpeppe> i was looking in tools not tools/releases
<wallyworld_> rogpeppe: hi
 * hazmat mramm don't think i'm helping.. but trying..
<hazmat> whoops
<rogpeppe> wallyworld_: do you know what might be happening here? https://pastebin.canonical.com/103759/
 * wallyworld_ looks
 * wallyworld_ hates 2fa
<rogpeppe> wallyworld_: oh, sorry, one mo
<wallyworld_> ok
<hazmat> rogpeppe, so it can do tool upload.. but you have to specify tools-metadata-url for the env to a local dir
<rogpeppe> wallyworld_: http://paste.ubuntu.com/6837045/
<hazmat> for the environ
<rogpeppe> hazmat: "it" ?
<rogpeppe> hazmat: juju metadata generate-tools ?
<hazmat> rogpeppe, wallyworld_ re generating tools metadata for an environ with juju metadata generate-tools  from a local dir
<hazmat> going through the src @ juju-core/cmd/plugins/juju-metadata/toolsmetadata.go
<wallyworld_> rogpeppe: it looks like it didn't upload any tools in that pastebin
<wallyworld_> rogpeppe: it should say "Uploading xxx kbytes"
<wallyworld_> i also notice it found a jujud. i've only ever seen it have to compile jujud from source
<wallyworld_> so maybe that other codepath is broken, not sure
<hazmat> i've always seen it use a pre-existing jujud binary..
<rogpeppe> wallyworld_: currently we can't bootstrap at all because of this issue
<wallyworld_> i bootstrapped just on friday no worries
<wallyworld_> on ec2
<hazmat> wallyworld_, i depend on that behavior fwiw.. re use jujud binary
<wallyworld_> i don't think anything has changed in trunk since then?
<hazmat> wallyworld_, their having issues on trusty
<rogpeppe> wallyworld_: "found existing jujud" should mean that it can use that
<wallyworld_> i was on trusty i think
<hazmat> wallyworld_, i also just did a bootstrap today with the 1.17.1 tarball on ec2 .. worked okay (i'm not on trusty though))
<wallyworld_> rogpeppe: sure. but i didn't really write that code so am not familair with it. i can look though
<rogpeppe> wallyworld_: oh, sorry, i thought you did most of the simplestreams stuff
<wallyworld_> rogpeppe:  i did
<hazmat> wallyworld_, can juju metadata generate-tools  use a local directory to find tools?
<wallyworld_> but the jujud vs compile from source is in a separate bit of code
<wallyworld_> it's sort of independent
<wallyworld_> hazmat: yes
<wallyworld_> let me check the details
<wallyworld_> hazmat: generate tools is just to produce the simplestreams metadata as you probably know
<hazmat> wallyworld_, from the src it looks like tools-metadata-url for the env pointing to a local directory file:// ...
<hazmat> wallyworld_, yes.. separate issues.. --upload-tools fail.. and no simplestream md makes normal bootstrap fail.
<wallyworld_> once you have the tarballs and metadata locally, you can point bootstrap at that and it will upload that stuff to the cloud
<wallyworld_> hazmat: lwt me check source - the tools-metadata-url should not be involved in generate-tools from a local dir
<wallyworld_> hazmat: it should just be with the -d option
<wallyworld_> juju metadata generate-tools -d blah
<wallyworld_> where blah contains a releases dir with tarballs
<wallyworld_> nd the result is a streams dir with metadata next to releases dir
<hazmat> wallyworld_, that looks like where to place the md  per its help description, not where to find the tools.. and it looks like it still goes to streams.canonical it looks like.
<hazmat> wallyworld_, ah.. so -d for both output  and input
<rogpeppe> oh, tim's broken dependencies.tsv
<wallyworld_> hazmat: when you say "find the tools" above, do you mean for bootstrap?
<wallyworld_> cause that's different to generate-tools
<wallyworld_> generate-tools is to take tarballs as input and produce the metadata so that the tarballs and metadata can be used for bootstrap
<wallyworld_> bootstrap will take those and upload to cloud storage
<wallyworld_> that is different and separate to using tools-metadata-url
<hazmat> right.. that's what --metadata-source on bootstrap is for
<wallyworld_> tools-metatdata-url is for when the tools and metadata have already been uploaded or exposed somewhere via a public url
<wallyworld_> yes
<wallyworld_> and yes, the generate-tools output is insitu
<wallyworld_> cause then you have a dir tree that bootstrap can consume
<wallyworld_> does that make sense?
<hazmat> wallyworld_, i was ref'ing tools-metadata-url based on the src for juju-core/cmd/plugins/juju-metadata/toolsmetadata.go which does this odd check  envtools.DefaultBaseURL of that appending file://
<wallyworld_> hazmat: that is to allow people to just specify a dir name ans it turns it into a url. it also allows people to forget to put the /tools suffix and it will deal with that robustly
<wallyworld_> the envtools.DefaultBaseURL is overridden from the --metadats-source from bootstrap
<wallyworld_> the --metadata-source hides streams.c.com so that bootstrap gets its data from the local dir
<hazmat> wallyworld_, it makes sense.. except the part where its not working :-)
<wallyworld_> which provider?
<hazmat> wallyworld_, rog is using maas, i'm trying with ec2
<wallyworld_> ok. i used ec2 on friday
<hazmat> i think everyone in london this week is using maas
<wallyworld_> i haven't tested maas. let me look at the CI dashboard
<hazmat> wallyworld_,  just to be clear.. i mean generate-tools re ec2.. bootstraping with --upload-tools works okay for me (not on trusty).
<wallyworld_> when i tested with ec2 onfriday, i used --upload-tools
<wallyworld_> so generate-tools fails?
<wallyworld_> i'm on trusty now, can't recall if i tested when i was still on precise
<hazmat> wallyworld_, for maas.. rogpeppe is getting a hang.. see bottom of this http://paste.ubuntu.com/6837008/
<rogpeppe> apparently they can't bootstrap with 1.17.0 either now
<wallyworld_> bugger. CI doesn't test maas it seems http://162.213.35.54:8080/
<wallyworld_> i'm confused by that pastebin
<wallyworld_> i didn't think juju-dist.s3.amazonaws.com was in the codebase anymore
<hazmat> wallyworld_, i'm getting a different error.. with generate-tools http://paste.ubuntu.com/6837127/
<wallyworld_> i'll check
<hazmat> actually my error is just pointing to tools instead of .
<wallyworld_> yep
<wallyworld_> the dir for -d is the dir containing tools
<wallyworld_> since it's tehn consistent with what's used for images
<wallyworld_> so you can have a dir with tools and images metadata
<rogpeppe> oh bugger, of course. godeps doesn't support git.
<wallyworld_> \o/
<hazmat> wallyworld_, so  can the output of generate-tools be used to seed a private cloud for all users, if its placed in a 'magic' bucket?
<cgz> rogpeppe: doesn't support how? just the update bit?
<wallyworld_> that is the idea
<rogpeppe> cgz: no at all - i'm just doing it
<wallyworld_> hazmat: with openstack, you can even put the url in keystone
<wallyworld_> like we do for canonistack
<hazmat> wallyworld_, and that bucket name varies?.. i'm wondering about maas atm.
<hazmat> wallyworld_,
<hazmat> its just juju-dist everywhere ?
<wallyworld_> hmmm. maas. i'm not 100% familair with the conventions there as i've never run juju on maas
<wallyworld_> um, not sure about juju-dist
<wallyworld_> let me have a quick look
<jam> wallyworld_, dimitern, rogpeppe, fwereade: standup?
<wallyworld_> jam: sec, just helping on a support issue
<wallyworld_> hazmat: the latest code looks like it just uses the same conventions as for other clouds - a tools dir in private storage for the cloud
<jam> hazmat: unfortunately, AIUI MaaS doesn't have a shared "bucket/container" concept
<jam> so you only have user-local stuff
<wallyworld_> by private storage, i mean whatever env.Storage() returns
<jam> the Simplestreams stuff *does* support any-old HTTP location, which means you could host them somewhere to be shared.
<wallyworld_> i think we require people using maas to --upload-tools right?
<wallyworld_> if no public tools are available?
<wallyworld_> hazmat: we're in a standup now. did you want to join after to ask questions?
<wallyworld_> where the whole dev team is available?
<fwereade> rogpeppe, any luck with tools-metadata-url or is that also problematic?
<hazmat> wallyworld_, rogpeppe is the one whose blocked.. i'm mostly asking to help there and for future ref on others.
<rogpeppe> fwereade: i haven't tried that
<rogpeppe> fwereade: what should it be?
<rogpeppe> fwereade: some URL for juju-dist?
<wallyworld_> hazmat: ok. rogpeppe can pop into the standup for help :-)
<rogpeppe> wallyworld_: will do
<wallyworld_> hazmat: if you have further questions, let me know,maybe email me or whatever
<hazmat> wallyworld_, will do, thanks
<rogpeppe> natefinch: ha, sorry, there are no tests :-)
<natefinch> rogpeppe: that's ok :)  I can write tests :)
<TheMue> rogpeppe: heya, do you know the error "missing or bad WebSocket-Origin"?
<rogpeppe> TheMue: sounds like you're not adding a websocket origin to the request
<rogpeppe> TheMue: where is the error coming from?
<TheMue> rogpeppe: it tells websocket.Dial, so should be the client
<rogpeppe> TheMue: have you grepped for it in the source?
<TheMue> rogpeppe: it's not in there, the whole line is "ERROR websocket.Dial wss://ec2-54-197-191-161.compute-1.amazonaws.com:17070/log?lines=10&entity=environment-3740322e-a899-4849-8fd2-a21c7e3b8d3a: missing or bad WebSocket-Origin"
<rogpeppe> TheMue: have you grepped in the go source too?
<rogpeppe> TheMue: and in the websocket source
<TheMue> rogpeppe: not yet, will do
<rogpeppe> TheMue: it's usually a good first step
<TheMue> rogpeppe: origin killed, no stumbling over a certificate error *sigh*
<fwereade> mramm, so, how can we make noise about getting some damn resources so we can do CI on MAAS?
<bigjools> fwereade: I suggested to Curtis that you share the maas lab
<natefinch> fwereade: yeah, it's pretty pathetic that we aren't even doing CI with our own product
<mramm> fwereade: I am making noise right now
<mramm> fwereade: and will make some more this afternoon
<fwereade> mramm, tyvm
<mramm> I also wan to do CI on the orange box -- just because that's where we plan to do demos
<mramm> if we are doing all our demos on a specific maas configuration with specific hardware -- that damn well better work
<mramm> every time
<mramm> and the bundles we demo should get tested regularly
<natefinch> yup
<rogpeppe> wallyworld_, fwereade: that doesn't seem to work: https://pastebin.canonical.com/103776/
<rogpeppe> wallyworld_: it's looking for index.json, which doesn't seem to be there
<rogpeppe> wallyworld_, fwereade: 2014-01-29 12:09:25 DEBUG juju.environs.simplestreams simplestreams.go:482 fetchData failed for "http://juju-dist.s3.amazonaws.com/streams/v1/index.sjson": cannot find URL "http://juju-dist.s3.amazonaws.com/streams/v1/index.sjson" not found
<fwereade> rogpeppe, fuck, tools-url needs trailing tools
<rogpeppe> ah!
<rogpeppe> fwereade: yes, i just realised that
<fwereade> rogpeppe, cool
<rogpeppe> fwereade, wallyworld_: that worked, thanks
<wallyworld_> \o/
<rogpeppe> wallyworld_: time is thereby bought
<wallyworld_> whooohooo
<cgz> morning niemeyer
<niemeyer> cgz: Heya
<gary_poster> hey jamespage.  if you get a chance today, I'd like to get your thoughts about https://bugs.launchpad.net/ubuntu/+bug/1273865 (juju quickstart inclusion in Trusty).  Please let me know if you might have some availability.
<dimitern> fwereade, https://codereview.appspot.com/53210044/  again? :)
<hazmat> dimitern, fwereade how was the networking discussion?
<hazmat> dimitern, i'm going through your network doc and adding some additional comments.
<dimitern> hazmat, you mean about goamz?
<hazmat> dimitern, yeah
<hazmat> there's like 20 forks adding various features on github..
<sinzui> r2268 broke dependencies.tsv
<sinzui> http://162.213.35.54:8080/job/prepare-new-version/946/console
<dimitern> hazmat, we decided to extend goamz with vpc/networking support and put all new apis in the EC2 type, keeping the public API intact
<cgz> sinzui: rogpeppe is aware, apparently godeps lacks git support, but is sprinting so don't think he's fixed it yet
<hazmat> dimitern, there's one significant item that feels like it missing, the current sec group per machine need disappears.
<hazmat> dimitern, since groups can be attached at runtime to machines
<vds> if I go install juju-core from LP trunk, from where do I get commands like relation-set?
<dimitern> hazmat, that's not entirely related is it?
<dimitern> hazmat, it's to do with the firewaller mostly
<hazmat> dimitern, its an additional api for the set.. also why the EIP manipulation?
<jamespage> gary_poster, looking
<sinzui> I would suggest reverting the change then. The project is not releasable or testable
<dimitern> hazmat, i don't think we'll change how we handle security groups much, just add vpc support for them, at least for now
<rogpeppe> cgz: godeps does actually have git support,
<rogpeppe> cgz: but thumper broke dependencies.tsv unfortunately
<dimitern> hazmat, the EIPs are needed to allocate/ssign more than one public IP to a machine, and subsequently assign them to containers on that machine
<rogpeppe> sinzui: do you want to propose a fix, or shall i?
<hazmat> dimitern, there seems to be some fundamental misunderstanding
<cgz> rogpeppe: I thought it did... that's why I was confused earlier :)
<dimitern> hazmat, oh? what is it?
<cgz> if it's just thumper's normal spaces, I can fix that again
<sinzui> rogpeppe, cgz, it has no revno
<dimitern> sorry, bbiab
<hazmat> dimitern, you don't need eips for a public ip on a vpc instance, they are a precious resource with small limits, that juju shouldn't be touching (ie like 5 per vpc) so also entirely inappropriate for that use. and the important aspect for containers is private ip addressing via additional veths and ips which isn't covered
<sinzui> rogpeppe, cgz, The file is tab separated, but there is no revno for github.com/loggo/loggo
<rogpeppe> sinzui: it doesn't need a revno
<rogpeppe> sinzui: but it does need a trailing tab
<sinzui> ah
<rogpeppe> sinzui: git doesn't have revnos
<sinzui> I have the file open, I will test and propose now. Thank your rogpeppe
<rogpeppe> sinzui: it should probably be a bit more lenient about the number of fields actually
<hazmat> dimitern, nevermind i do see private addresses in the api changes, veth are also useful (esp cross subnet traffic) but perhaps not required
<hazmat> dimitern, oh. and that's 5 vpc EIPs for an account not per vpc.
<sinzui> rogpeppe, godeps is not happy once it can read the line:
<sinzui> $ godeps -u dependencies.tsv
<sinzui> godeps: cannot parse "dependencies.tsv": cannot parse "github.com/loggo/loggo\tgit\t89458b4dc99692bc24efe9c2252d7587f8dc247b\t": unknown VCS kind "git"
<rogpeppe> sinzui: try go get -u launchpad.net/godeps
<jamespage> gary_poster, do you publish releases of quickstart anywhere?
<rick_h_> hazmat: what's the format for adding a kvm machine via the cli? I'm huning for some docs but not finding any. Design is asking what to call creating a kvm instance and getting confused a bit in 'machine' 'bare metal' etc
<gary_poster> jamespage: on call, in ppa, details here: https://bugs.launchpad.net/ubuntu/+bug/1273865
<jamespage> gary_poster, yes - I read that
<hazmat> rick_h_, sub kvm for lxc.. ie juju deploy --to=kvm:1  ... its in juju deploy/add-unit --help
<sinzui> sorry rogpeppe , godeps still reports the same error
<gary_poster> so jamespage only ppa then
<jamespage> gary_poster, how about releasing versions to pypi like hazmat does for juju-deployer?
<rick_h_> hazmat: awesome thanks
<hazmat> rick_h_, not he kvm, but the container syntax with lxc.. and then its just s/lxc/kvm
<rogpeppe> sinzui: just to sanity check, what revision of godeps are you on?
<sinzui> 10, rogpeppe
<rogpeppe> sinzui: ah, 14 is the latest
<sinzui> whoa, your on 14
<rogpeppe> sinzui: try bzr pull
<sinzui> I did
<rogpeppe> sinzui: try removing the godeps directory and doing go get again
<sinzui> rogpeppe, this my problem, not CI, it always gets a fresh checkout of godeps
<dimitern> hazmat, yeah, these limits will hit us at some point
<hazmat> dimitern, some point? even a trivial environment would hit those.
<rogpeppe> sinzui: if CI gets godeps for fresh, it should work ok
<hazmat> its not a good solution for the use case you gave of density.
<rogpeppe> sinzui: i just tried it and it seems to work (with the fixed .tsv file)
<dimitern> hazmat, not necessarily - most services will happily work with private addresses, just the ones that need exposing will be an issue I think (after the limit is hit)
<sinzui> rogpeppe, yep, It does. My branch is stuck because I was fighting with autobuilding in September.
<cgz> if I bootstrap, it fails, I update the environments.yaml, and bootstrap again, it seems the ENV.jenv doesn't get updated
<cgz> that's not deliberate, right?
<hazmat> dimitern, there's seems to be a misunderstanding.. you don't need an eip for a vpc instance to attach the public net
<rogpeppe> cgz: yes, you'll need to remove iyt
<cgz> rogpeppe: that kinda sucks
<rogpeppe> cgz: we really need to fix that behaviour
<rogpeppe> cgz: agreed
<rogpeppe> cgz: if bootstrap fails, it should remove the .jenv file if it just created it
<cgz> okay, bug 1247152 it seems
<dimitern> hazmat, you need an EIP for each instance in a non-default VPC
<hazmat> dimitern, no you don't
<dimitern> hazmat, from the aws docs: We assign each instance in a nondefault VPC only a private IP address, unless you specifically request a public IP address during launch. To ensure that an instance in a nondefault VPC that has not been assigned a public IP address can communicate with the Internet, you must allocate an Elastic IP address for use with a VPC, and then associate that EIP with the elastic network interface (ENI) attached to the instance.
<hazmat> dimitern, why do you think yoou need it?
<hazmat> see the unless part?
<dimitern> hazmat, yes
<dimitern> hazmat, but that won't be the case until you actually need a public address
<hazmat> so if you launch with a public addr why do you need it?
<hazmat> dimitern, you can't communicate to the internet from the instance without it
<sinzui> rogpeppe, cgz : do eith of you have a minute to review https://codereview.appspot.com/58260044
<hazmat> er. without at least a public address attached to the instance, eip, or nat isntance setup
<rogpeppe> sinzui: will do
<cgz> it'sarace
<dimitern> hazmat, with a default vpc, you can - they use nat to relay private ip outgoing traffic to a public ip from the ec2-vpc pool, not your account
<hazmat> dimitern, basically depending on more than one pub address per instance is basically broken, due to the low limits on eips (which are meant primarily not for pub addresses, but for static addresses)
<hazmat> dimitern, default vpc is the same as launching with a public address, hence option 1 of the 3 i listed.
<hazmat> and no it doesn't use nat.
<hazmat> nats are for private subnets outbound traffic, default vpc, is basically public subnet with every instance getting a public address at launch by default.
<dimitern> hazmat, "We assign each instance in a default VPC two IP addresses at launch: a private IP address and a public IP address that is mapped to the private IP address through network address translation (NAT)."
<hazmat> which is transparent to the user, its effectively ec2 internal impl.. the traditional nat in ec2 vpc is something very different
<dimitern> hazmat, so for the default vpc we can still assign additional EIPs
<hazmat> dimitern, ie.. this is nat for vpc.. http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html
<hazmat> hmm.. bad link.
<hazmat> dimitern, yes.. up to 5 max for a user's entire account, and then things break.
<dimitern> hazmat, so what can we do? there's a form you can fill in to raise your limits I saw somewhere
<hazmat> dimitern, yes.. and justify use.. i'm not sure.. juju needs it.. is going to be a good response.
<dimitern> hazmat, "juju needs it" won't do, because it's per account
<dimitern> hazmat, so users that need to deploy more than 5 machines/containers within a non-default vpc and all of them need EIPs, can be advised to file that form
<gary_poster> jamespage: sorry, off call.  would putting it on PyPI help Ubuntu packaging?  To be clear, the story we want to advertise is something to the effect of "To get started with Juju, run `sudo apt-get install juju-quickstart && juju-quickstart [bundle name]` to immediately be walked through installation, configuration, and deploying your first full working solution on top of Juju`
<jamespage> gary_poster, thats fine - but I'd prefer that we have something from juju-quickstart as an upstream that is 'released'
<jamespage> rather than some arbitary snapshot that we agree on
<hazmat> dimitern, and juju is going to handle that partial failure mode well.. and its less than 5 if their doing standard eip jobs (bastion hosts, nat instances, etc) designing something that breaks with real world use feels quite odd. amz is pretty stingy with eip allocations. but maybe its okay.. this use of eips wouldn't fly with any aws vpc user i know, but their all using elbs (public or private) for the entry points / expose endpoints.
<gary_poster> jamespage: ah, ok.  makes sense.  we'll get that done ASAP and I'll update the bug?
<jamespage> gary_poster, sounds good - I'll cut a package for the archive and upload sometime this week
<gary_poster> aqwaome thanks!
<jamespage> your packaging branch general looks good btw
<gary_poster> great
<jamespage> just needs a few extras for the archive
<hazmat> dimitern, the reality check is no public cloud hands out public addresses like candy.. neutron is exposed in rackspace and hpcloud, but you can't get additional pub ips there.
<dimitern> hazmat, are you saying we can alleviate some of the issues by using elb?
<hazmat> dimitern, i'm saying real world usage of aws, doesn't expose services directly, they front with elb.
<dimitern> hazmat, right
<dimitern> hazmat, how about the dense containerization story?
<dimitern> hazmat, it seems such deployments are more suited for private clouds on hyperscale hardware + openstack more than on ec2
<hazmat> dimitern, even there what does expose mean?
<hazmat> dimitern, a) firewaller not imiplemented.. b) a generic iptable firewall would be generic and useful across all of them
<hazmat> c) its not a public ip addr
<hazmat> dimitern, the story for private clouds is basically the same as public .. if you subtract the delta on expose.. its all about private networking
<dimitern> hazmat, well, you can still have a large deployment with 100s of hadoop nodes and only a handful of web servers that take tasks or display results
<hazmat> dimitern, dense containerization isn't really about lots of public endpoints.. its about efficiency
<dimitern> hazmat, and in that case, you'll need EIPs only for the web servers and a nat instance
<hazmat> dimitern, but you don't need it for the web servers.. you already have a public ip there
<hazmat> and you wouldn't have  nat instance created by juju..
<rogpeppe> axw: ping
<rogpeppe> axw: (i can be hopeful :-])
<cgz> rogpeppe: he's almost certainly asleep :)
<axw> am not :)
<dimitern> hazmat, it seems juju needs to manage a nat instance anyway
<hazmat> dimitern, why?
<axw> rogpeppe: what's up?
<cgz> waaaat
<dimitern> hazmat, and only assign EIPs to exposed machines
<hazmat> dimitern, nat instances are for a very particular use case
<rogpeppe> axw: looking at sshinit.ConfigureScript
<dimitern> hazmat, because all the other nodes will need net access
<hazmat> dimitern, did you see my nat instance vpc link?
<dimitern> hazmat, yes
<hazmat> dimitern, that's why you can give them public ips when launching.
<hazmat> dimitern, maintaining an ec2 nat instance is  pita.. and its huge spof without a bunch of hacky scripts.
<hazmat> well. its not a pita. but eliminating the spof is
<dimitern> hazmat, non-default vpc, only private addresses unless added at launch (which we need not do unless we know the instance will need it), a nat instance for all the other nodes + EIP for it
<hazmat> dimitern, you just always create with one
<rogpeppe> axw: i'm looking at sshinit.ConfigureScript
<rogpeppe> axw: and wondering how the stderr logic could possibly work
<dimitern> hazmat, and you hit the wall after the 5th instance
<hazmat> dimitern, those are not EIPs
<rogpeppe> axw: first i wondered why there were two nested ()s
<hazmat> there random public ips.. eips are static addresses associated to the account
<dimitern> hazmat, which are those?
<rogpeppe> axw: then i saw that AFAICS there's nothing to separate stdout from stderr
<hazmat> dimitern, if you launch a vpc instance with a pub addr its .not an eip allocation
<dimitern> hazmat, in a non default vpc eips are the only way to have a public address, which is most likely our case
<hazmat> dimitern, that's not true
<dimitern> hazmat, it is
<axw> rogpeppe: just a moment, context switching
<dimitern> hazmat, according to the docs
<rogpeppe> axw: np
<hazmat> dimitern, your reading them wrong
 * hazmat sighs
<dimitern> hazmat, "We don't automatically assign a public IP address to an instance that you launch in a nondefault subnet. Therefore, if you want an instance in a nondefault subnet to communicate with the Internet, you must either enable the public IP addressing feature during launch, or associate an Elastic IP address with the primary or any secondary private IP address assigned to the network interface for the instance."
<hazmat> dimitern, OR
<hazmat> either you spec pub at launch.. which is not an eip allocation
<hazmat> or you runtime attach an eip
<hazmat> so launch it with a pub
<axw> rogpeppe: stdout and stderr are combined in our usage of cloud-init
<rogpeppe> axw: yeah
<rogpeppe> axw: so that logic is unlikely to have been tested
<dimitern> hazmat, ok, I think I see now, sorry
<rogpeppe> axw: AFAICS if you specify stderr, it will be lost.
<rogpeppe> axw: or rather, not redirected anywhere
<axw> rogpeppe: it won't be lost, it'll go into /var/log/cloud-init-output.log
<axw> sorry just a sec
<rogpeppe> axw: i don't think so
<rogpeppe> axw: because if stderr is specified, there appears to be nothing that actually redirects stderr
<dimitern> hazmat, ah, unless there are more than one network interface attached to the instance
<rogpeppe> axw: i *think* what you want is (commands) >stdoutfile 2>stderrfile
<hazmat> dimitern, none of this changes imo, that eip usage like this is a bad idea, trying to break one pub addr per instance is not going to be portable to any other public cloud, its entailing alot of work for a minute gain that is inherently limited in it scale, and won't work with any other cloud.
<axw> rogpeppe: true, the SetOutput value would need to be mangled to get it to do the right thing
<rogpeppe> axw: whereas currently it's ((commands) > stdoutfile) > stderrfile)
<axw> rogpeppe: well, the "> ..." bit is specified in cloudinit.Config.SetOutput
<axw> but it'd be awkward
<dimitern> hazmat, from juju's perspective, we need to be able to assign multiple public addresses to an instance, regardless of cloud
<axw> and you're right, we don't do it and it's not tested
<dimitern> hazmat, in order to use these public addresses for containers running on that instance
<axw> rogpeppe: do we need to separate stdout/stderr?
<rogpeppe> axw: not too awkward, except that i don't know if bash allows redirecting stderr to a pipe. maybe you can do: foo 2| bar
<rogpeppe> axw: not really
<rogpeppe> axw: we could just remove all the related logic
<dimitern> hazmat, it seems the only way to achieve that on EC2 is using VPC+EIPs
<dimitern> hazmat, do we agree so far?
<axw> rogpeppe: if we did that, I'd prefer if we just removed it all the way up to and including juju-core/cloudinit
<hazmat> dimitern, there are other techniques.. and how do you achieve that on hp cloud or rackspace, or google, or joyent or azure.
<hazmat> quite simply... you don't
<axw> to prevent someone using something that's not supported all the way down
<rogpeppe> axw: yeah, that's what i was thinking
<hazmat> dimitern, fwereade would be nice to schedule a followup meeting.
<dimitern> hazmat, on other clouds that support assigning multiple public ips to an instance, it will work just as well
<dimitern> hazmat, with the relevant changes to the libraries for openstack, joyent, etc.
<dimitern> hazmat, yes, a meeting will be nice
<axw> rogpeppe: you can do "2>&1 |", but I don't know about piping without stdout...
<dimitern> hazmat, but all this doesn't really change anything re goamz changes - we need these extra API calls anyway, no matter how we decide juju should use them
<hazmat> dimitern, other clouds? there are no other public clouds that allow that.. i just listed a bunch, and none of them do. and amz does it at a very limited scale (and those addresses have primary uses for other purposes) and for private openstack, its not a public address.
<hazmat> dimitern, fair enough
<hazmat> dimitern, pls, pls move goamz to github
<hazmat> dimitern, there are so many forks out there because its not on github
<dimitern> hazmat, it's not up to me as you know :)
<hazmat> niemeyer, , pls, pls move goamz to github
<axw> rogpeppe: see bottom of http://www.tldp.org/LDP/abs/html/io-redirection.html
<rogpeppe> axw: you can do it AFAIR
<rogpeppe> axw: yeah, that's similar to the approach i'd use
<dimitern> hazmat, ok, so by "public ip" I mean "accessible from outside the cloud somehow", regardless public or private
<rogpeppe> axw: but i think we can just forget it all
<axw> rogpeppe: yep, +1 to deleting code
<niemeyer> hazmat: With fwereade on a call.. biab
<hazmat> dimitern, as an another example of where that's moot.. one use case people do with vpc is to extend their org ip address space into the public cloud via directconnect or vpn connectivity, so the subnets ip ranges are from the org and traffic gets routed back through the org as egress. the whole org gets access though because its all part of their network.
<axw> sleepy time. good night folks
<dimitern> hazmat, that's the vpn story - gateways on both sides of the vpc subnet
<hazmat> right but what does expose mean there... vpns make every 'accessible outside of the cloud somehow'
<hazmat> within the org
<arosales> fwereade, good afternoon
<arosales> fwereade, current status on simple stream is sinzui is making the jenkins workflow that he will hand off to utlemming. utlemming has adjusted priorities to make this happen asap.
<fwereade> arosales, <3
<sinzui> arosales, fwereade I am testing the simplstreams job now
<dimitern> hazmat, yes, and in this case private ips are in fact "public" (using the above definition) - but that's a good point - one we have networks in juju that vpn network that spans from the org into the cloud will be set as a public network
<arosales> sinzui, thanks I know you have been hitting it hard lately on QA, tools, reporting, bugs, and releases
<arosales> sinzui, utlemming should be available once your testing is complete and you need to add the workflow to the build server
<arosales> mramm, ^
<mramm> arosales: perfect
<mramm> arosales: sinzui: thanks for jumping on this
 * arosales ows sinzui a drink of his choose next week :-)
<dimitern> fwereade, review poke if you have 5m?
<niemeyer> Wow
<niemeyer> mramm, fwereade: It actually crashed
<niemeyer> :)
<niemeyer> Good timing
<fwereade> haha
<fwereade> dimitern, I have 5m now, on it
<dimitern> fwereade, cheers!
<niemeyer> Maybe they have speech detection for "take care"
<fwereade> lol
<niemeyer> hazmat: So, using github would be good, but as we discussed in the call, the way goamz is organized also encourages people to fork away
<niemeyer> I'd like to solve that at some point
<gary_poster> jamespage: https://pypi.python.org/pypi/juju-quickstart has a 1.0 and I added a comment to bug with link, FWIW.  Thanks again, and please let me know if we should do anything else.
<hazmat> niemeyer, i think on github that people will push pull requests.. its a much more common etiquette... and helps build into a central core for all amz svc usage when people expand usage.. but understood re goamz structure and svc mapping.. i just don't think that's the underlying issue
<hazmat> most of the forks are adding features to extant services (esp big ones like ec2), a few do new services (dynamodb) but that's not as common.. i think having a 'canonical' repo on github will become a magnet for both.
<niemeyer> hazmat: I think it is, actually.. the API is gigantic, and people have to fork for every single field we didn't care to wrap
<hazmat> true
<hazmat> niemeyer, so that's more fundamental.. i thought you meant more of a service split of the pkg.. but yeah.. addressing the fundamental of the api service would be good.
<hazmat> i know your not a fan.. but i still think auto-gen from the api json would be a win.
<mramm> hazmat: that does feel like a reasonable possibility, definitely a theory that is worth testing
<niemeyer> hazmat: There are better ways to do it
<hazmat> and could cover a param passing style that is extensible
<niemeyer> hazmat: Auto-generation leads to a crappy and undocumented API
<mramm> hazmat: niemeyer: I mean the github bit, not nessisarily the auto-generation -- I have not thought that through at all.
<niemeyer> hazmat: We can have an extensible and dynamic interface without that
<hazmat> true
<niemeyer> hazmat: As we do in lpad, for example
<hazmat> niemeyer, the aws autogen and gce autogen do include docs though.. agree though its not always the most idiomatic interface
<niemeyer> hazmat: And they're usually poor docs, often making no sense for the wrapped interface
<hazmat> its more about just getting solid coverage of the huge api in a single step
<niemeyer> hazmat: If we were to auto-generate, we can as well just build a dynamic layer and point people to the upstream docs
<niemeyer> Which is what other libraries do
<natefinch> hazmat, niemeyer:  if people are forking to add more to the API.... could we not just then encourage them to issue a pull request to put them back in the base repo?  Isn't that the whole point of open source collaboration?
<hazmat> niemeyer, so would you be able to move goamz to github.com/juju/goamz ?
<niemeyer> natefinch: Kind of.. I don't want to be the one in the front line of such an infinite number of pull requests adding tiny bits each
<hazmat> niemeyer, or do you want to explore the extensible/dyn interface more first
<hazmat> niemeyer, yeah.. boto had the same issue.. hence botocore based on autogen
<hazmat> re infinite pull requests
<niemeyer> hazmat: I shouldn't be the one moving that, if a decision is made
<niemeyer> hazmat: I haven't been working on it, nor depend on it for anything I'm working on
<natefinch> hazmat: pretty much anyone in core can move it
<hazmat> niemeyer, gotcha. ok.
<niemeyer> hazmat: It'd be wonderful if we could solve the API extensibility issue.. I have a path, but haven't had much of an incentive to stop what I'm doing to fix it
<niemeyer> hazmat: Rather than being immediately helpful, this will actually create more work for other people
<hazmat> natefinch, i had asked dimitern about doing it as part of the networking work, and he wanted niemeyer's ok first i think
<niemeyer> hazmat: We can talk next week
<hazmat> niemeyer, sounds good
<niemeyer> hazmat: Regarding moving, I'm happy either way
<niemeyer> hazmat: We should have a clear understanding of why we're doing it, though
<niemeyer> hazmat: "because we'll have pull requests" doesn't look like a good answer
<niemeyer> hazmat: We have had pull requests now, and seldom people step up to care
<natefinch> niemeyer: because sabdfl said so? ;)
<niemeyer> natefinch: What?
<hazmat> niemeyer, they do on gh and its part of the social culture, and their forking there, its a trivial process for them to push it back.
<niemeyer> natefinch: I have no idea about what you mean by that
<niemeyer> hazmat: I mean we *have* pull requests
<niemeyer> hazmat: and we *have* people in the mailing list asking questions
<niemeyer> hazmat: and we don't have IMO a good maintainership story
<hazmat> niemeyer, understood, i'm saying we'll likely get more.. and decrease the long lived extant forks that i see out there.
<niemeyer> hazmat: Having *more* requests won't solve that issue :)
<hazmat> the maintainer story is an issue
<hazmat> probably the biggest, and a good informal topic for next week
<rogpeppe> anyone know what revno 1.17.0 corresponded to?
<natefinch> rogpeppe: Any reason not to use a RWMutex in SharedValue instead of a regular mutex?
<hazmat> rogpeppe, 2173
<hazmat> rogpeppe,  $ bzr tags
<rogpeppe> natefinch: could do perhaps, but it's an optimisation that's not really worth it
<rogpeppe> natefinch: making it an RWMutex would make the code more complex
<rogpeppe> natefinch: we don't hold the lock for any length of time, so the chance of contention are extremely slight, particularly in our use case
<natefinch> rogpeppe: it barely makes the code more complex.  It adds zero lines, just instead of calling Lock in Get you call RLock.
<natefinch> rogpeppe: I'm sure it doesn't matter for our purposes, just seemed like the right thing to use.
<rogpeppe> natefinch: i generally think of RWMutex as an optimisation - it makes the code a little harder to reason about, and the gain in this case is zero
<rogpeppe> natefinch: if clients were calling Get very frequently, it might be worth it, but most clients will call Getter
<natefinch> rogpeppe: well, either way they're locking to check the value.  In fact... it might be better, because when the value changes and the Broadcast fires, every watcher is going to "simultaneously" try to relock and check the value
<natefinch> rogpeppe: with an RWMutex they can all do it at the same time
<natefinch> rogpeppe: I don't know at what level of watchers you'd get any perceptible benefit, but I also don't see the added complexity as being terribly large either.
<rogpeppe> natefinch: fair enough; do it if you like. (you'll want to use RWLock.Locker)
<natefinch> rogpeppe: yep
<rogpeppe> natefinch: i can't get excited about it unless the frequency of changes is in the millions per second and there's more than one watcher, neither of which is true for us.
<natefinch> rogpeppe: "for us"  :)  I'm hoping at some point we'll move all this generic code outside the juju walled garden and make them independent open source repos.
<rogpeppe> natefinch: yeah. you know i have mixed feelings about that :-)
<natefinch> rogpeppe: I know. :)
<natefinch> rogpeppe: btw, that Cond.Wait magic sauce is pretty awesome
<rogpeppe> natefinch: i took a little while to arrive at the particular idiom you see there, but it does work nicely, doesn't it?
<natefinch> rogpeppe: very cool
<TheMue> rogpeppe: got my breakthrough from command line to logging. especially filtering for multiple entities is nice.
<rogpeppe> TheMue: yay!
<TheMue> rogpeppe: only used the whole afternoon finding a self-produced bug *grmblfxÃ
<rogpeppe> trivial (one line, but critical) review anyone? https://codereview.appspot.com/56070044/
<rogpeppe> fwereade, fwereade, dimitern: ^
<natefinch> rogpeppe: reviewed
<rogpeppe> natefinch: thanks
<natefinch> thumper: o/
<dimitern> small review anyone? https://codereview.appspot.com/58170045
<natefinch> dimitern: sure
<dimitern> natefinch, thanks!
<natefinch> dimitern: could bootstrap-ssh-timeout be simply called bootstrap-timeout?  the fact that we're connecting with SSH doesn't really matter, right? This is just a generic bootstrap timeout, right?
<thumper> morning natefinch, dimitern
<dimitern> natefinch, well, all of these 3 are only used for waitSSH
<dimitern> thumper, hey
<dimitern> natefinch, but I guess ssh is a inside detail
<natefinch> dimitern: exactly my point. To a user, this is just the timeout on juju bootstrap.  Putting ssh on there is confusing.
<dimitern> natefinch, i'm not against renaming it :) just comment on it pls
<natefinch> dimitern: I am :)  Just wanted to discuss live first, to make sure I was understanding it correctly.
<natefinch> dimitern: there you go :)
<natefinch> dimitern: nice to get that in.
<dimitern> natefinch, tyvm
<natefinch> dimitern: welcome
<thumper> WTH
<thumper> natefinch: you're on trusty, right?
<thumper> natefinch: why do I get this...
<thumper> tim@jake:~/go/src/code.google.com/p/go.tools/cmd/vet$ go install
<thumper> go install code.google.com/p/go.tools/cmd/vet: open /usr/lib/go/pkg/tool/linux_amd64/vet: permission denied
<thumper> why is it trying to put it in /usr/lib?
<thumper> doesn't for juju
<thumper> when I go make install there
<natefinch> thumper: that happened to me
<natefinch> thumper: I forget what I did to fix it though.....
<natefinch> thumper: have you pulled and updated the code?
<natefinch> thumper: I had to do that first.... now it go installs just fine.
<thumper> hmm
<thumper> no changes
<natefinch> thumper: I did upgrade to go 1.2... not sure if that affects anything... I wouldn't think it would change where things are installed.
<thumper> do I need to set another GOPATH type var?
<natefinch> thumper: just gopath is all you need
<thumper> seems not
<thumper> ffs
<thumper> can't use lbox
<thumper> because it wants go vet
<thumper> can't install go vet because it is dumb
 * thumper will poke davecheney about it later, perhaps he knows
 * thumper goes to beat something up
<hatch> 1.17.1 on 12.04 after trying to bootstrap local without sudo I get the following error ERROR Get http://10.0.3.1:8040/provider-state: dial tcp 10.0.3.1:8040: connection refused
<hatch> I thought that 1.17.1 removed the sudo requirement from local deploys?
<davecheney> thumper-afk: might be simpler to remove that requirement from lbox.check
<davecheney> the debain packaing for 1.2 is AFU
<davecheney> combined with some poor decisions from upstream means you probably won't be able to make that work without building go from source
<mbruzek> Hi juju-dev.  I have an up to date trusty system and I can not use the local environment.
<mbruzek> When I bootstrap local that runs but I can't get a juju status afterward
<wallyworld_> mbruzek: thumper-afk is your best bet for local provider issues. i know there may have been some trusty issues, but am not across the detail
<mbruzek> thanks wallyworld_  afk means away from keyboard right?
<wallyworld_> yeah, he's not too far away
<wallyworld_> he should be back soon
<mbruzek> thanks
<wallyworld_> we're still ironing out any remaining trusty issues. lxc changed between precise and trusty so we have some work to do
<mbruzek> I understand, and have evaluated .deb packages for sinzui before
<mbruzek> I just can't make any progress on my local and I would *really* like to.
 * mbruzek had it working 2 days ago, but decided to upgrade 
<davecheney> http://paste.ubuntu.com/6840848/
<davecheney> juju is not happy this morning
<davecheney> is the environment bootstrapped ? or not
<wallyworld_> davecheney: looks like something killed your instances? and now juju is confused
<wallyworld_> cause it has the .jenv file and thinks it should be bootstrapped
<wallyworld_> does the hp console show anything running?
<hazmat> davecheney, juju destroy-env..
<hazmat> i thought this bug got listed as fixed
<davecheney> hazmat: wallyworld_ yeah, removed the .jevn file and everything was fine
<davecheney> i guess the bug isn't fixed
<hazmat> davecheney,  https://bugs.launchpad.net/juju-core/+bug/1176961
<hazmat> nope.. its not.. it got marked low.
<wallyworld_> well that is less than optimal
<hazmat> its the one where you can't bootstrap or destroy.. and if you don't know to jenv remove..  it kinda sucks.
<wallyworld_> i did hear mumblings that we should fix that issue
 * wallyworld_ thinks it should be High
<wallyworld_> i'll follow up with some folks and see if we can get that one sorted out
<wallyworld_> it is a pretty poor user experience
<hazmat> cool, thanks wallyworld_
<wallyworld_> np. i'll even have a go myself once i clear my current work items if i can't get traction to get it sorted
<davecheney> sweet
<davecheney> thanks
<davecheney> http://paste.ubuntu.com/6840900/
<davecheney> not the best error message
#juju-dev 2014-01-30
<davecheney> http://paste.ubuntu.com/6840930/
<davecheney> great, how am I supposed to shut down this environment ?
<wallyworld_> i would kill the machines by hand and delete the control container, then delete the jenv
<thumper> davecheney: really, WTF?
<thumper> how do I remove the govet check?
<thumper> wallyworld_: I upgraded to trusty last night
<thumper> trunk starts the local provider
<wallyworld_> hmmm
<thumper> although my http-proxy wasn't on the right port
<thumper> trunk mind, not release
<thumper> release is all fubared
<wallyworld_> when i tried on friday, the agent ran, but no lxc container would start
<thumper> for some reason I've lost the all-machines.log
<wallyworld_> yeah, me too
<thumper> need to figure that out
<thumper> I have a branch to submit for destroy
<wallyworld_> \o/
<thumper> if I can get lbox shite working
<wallyworld_> thumper: with go vet, i installed from source and got it to work that way
<thumper> wallyworld_: you installed go from source?
 * wallyworld_ tries to remember
<wallyworld_> go get -u ...something
<wallyworld_> let me see if my history shows it
<wallyworld_> i think the error message gave a hint
<thumper> it tries to open the /usr/lib/go pkg dir
<thumper> for some reason it isn't honouring the GOPATH
<thumper> davecheney: can I just change an environment variable?
<wallyworld_> not sure if this was the attempt that worked
<wallyworld_> sudo bash -c "export GOPATH=/usr/lib/go; go get code.google.com/p/go.tools/cmd/vet"
<wallyworld_> something like that was necessary
<wallyworld_> i just assumed it was my fucked up set up
<wallyworld_> didn't realise it was a general issue for trusty
<thumper> hmm...
<thumper> yeah, that looks weird to the extent I don't want to do that
<wallyworld_> i may have had to do it with a GOPATH pointing to a dir off my home, not usr/lib
<wallyworld_> but i recall sudo and messing with GOPATH was necessary
<thumper> a fuck it...
<thumper> nah, that doesn't work
<wallyworld_> hmmm sorry. i can't recall exactly what worked. i *think* i had to point GOPATH to a local src dir, but use sudo. not sure thpugh
 * thumper stabs .lbox.check
<thumper> FARK
<thumper> can't propose with a dirty branch
<thumper> so either I commit the lack of vet
<thumper> or don't use lbox
 * thumper decides to skip lbox
<wallyworld_> \o/
<thumper> wallyworld_: https://code.launchpad.net/~thumper/juju-core/fix-lxc-destroy/+merge/203861
<wallyworld_> looking
<wallyworld_> thumper: done. btw did you see google offloaded motorola to lenovo. those lenovo guys are really going on a shopping spree
<thumper> no I didn't know that
<thumper> wallyworld_: did google keep all the patents?
<wallyworld_> yep :-)
<wallyworld_> of course
<thumper> heh
<wallyworld_> they licenced them to lenovo
<thumper> wallyworld_: I can't land it yet, it didn't fix the problem
<wallyworld_> so they paied $12bil, sold for $3bil
<thumper> ah...
<thumper> hang on
<thumper> I see what's happened now
<thumper> I have a running juju machine agent for zero
<thumper> rebuilding a local juju doesn't help destroy environment
<thumper> because the environment is already running
<thumper> and using an old jujud
<thumper> which doesn't have my fix
<thumper> FFS
<thumper> manual hackery FTW
<wallyworld_> i've done the same thing before too
<thumper> ok, at least that is the local provider starting and stopping in trusty
<thumper> now to poke and see why we have no all-machines.log
<thumper> hmm... I think I may see
<axw> thumper: I looked into that the other day, it was permissions
<thumper> yeah, I saw that too
<thumper> axw: have you a fix ready?
<thumper> if not, I'll do it
<axw> thumper: I think we may have to do something yucky, like symlinking to /var/log/juju
<axw> no
<axw> it passed out of my memory on the flight back
<thumper> wallyworld_: I'm witting in our weekly hangout...
<wallyworld_> oh yeah
<hazmat> thumper, how's proxy support coming?
<thumper> hazmat: trunk is almost feature complete
<thumper> there are a few things outstanding
<thumper> - remove old apt proxy cruft in container setup
<thumper> - make sure containers get the proxy info for the container providers
<thumper> - some upgrade code
<hazmat> hmm.. that sounds pretty good... does that include things like the cli respecting std http_proxy env vars for tool lookup?
<thumper> hazmat: in a hangout just now
<thumper> hazmat: how long have you got?
<hazmat> thumper, i'll make some time.. i'm pushing through on a late night
<thumper> waigani: hangout? standup
<waigani> yep
<davecheney> cap
<davecheney> crap
<davecheney> now destroy-environment cannot save me
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1274355
<thumper> school run
<thumper> bbs
 * axw rejoices and promptly stops thinking about windows cli
<axw> davecheney: try destroy-environment --force
<axw> davecheney: actually... what version are you on?
<davecheney> trunk
<davecheney> axw: yup
<davecheney> that worked
<axw> davecheney: destroy-environment now goes and talks to the API server to destroy the environment "cleanly"
<axw> if it's borked, --force is required
<axw> we just had a chat on our scrum, and figured we should probably tell that to the user if it fails and --force wasn't specified
<davecheney> couldn't hurt
 * thumper sighs
 * thumper created a broken local provider
 * thumper deletes things
<thumper> fark!!!
<thumper> I think it is apparmour
<axw> thumper: doing what?
<thumper> stopping writing to ~/.juju/log/all-machines.log
<axw> hm ok
 * thumper pokes and tweaks
 * thumper does something icky
 * axw goes out for lunch
<axw> bbl
<axw> if I don't return, the heat has claimed me
<thumper> wallyworld_: ping
<thumper> wallyworld_: who owns your /var/log/syslog file?
<axw> thumper: mine is syslog:adm
<thumper> hmm...
<thumper> mine is messagebus:adm
<thumper> no idea where that came from
<wallyworld_> thumper: um, let me check
<thumper> wallyworld_: that's ok
<wallyworld_> i don't have one right now
<thumper> all these permission problems are due to rsyslog apparmor profiles
<wallyworld_> ah
<thumper> I've got a work-around
<thumper> but I need to go make dinner
<thumper> so it'll have to wait
<wallyworld_> f*cking apparmor
<wallyworld_> :-)
<thumper> waigani: is there a bug for the "no all-machines.log for the local provider"?
<waigani> thumper: I have not created on
<waigani> *one
<axw> wallyworld_: when you're not busy, can you please poke the bot?
<wallyworld_> sure
<wallyworld_> will do it now
<wallyworld_> axw: poked
<axw> thanks
<axw> wallyworld_: I thought there was a reason we needed FullPath for the manual bootstrap - the reason eludes me though
<wallyworld_> axw: FullPath is filled in when the particular tools are read from simplestreams. but is was unnecessary to fill in that value when writing metadata files caused it is ignored then
<wallyworld_> we just store ralative path in the json
<wallyworld_> and construct fullpath when needed as the metadata is loaded
<axw> yeah, I thought it was used in-memory... my memory may just be faulty though
<axw> welp, seems to work anyway
<wallyworld_> yay
<axw> wallyworld_: in provider/maas/storage.go, there's talk about 404 and IsNotFoundError
<axw> does this not work?
<wallyworld_> um
<wallyworld_> that works
<wallyworld_> but the issues is URL()
<axw> ah sorry
<axw> URL, not Get
<wallyworld_> yeah, maas is the only one that can't give a url for a non existent fle
<wallyworld_> so i changed dummy storagr to match that
<axw> wallyworld_: ahh, so MAAS 404s even for URL
<axw> got it
<wallyworld_> yeah :-(
<wallyworld_> sadly yes
<axw> wallyworld_: why do we no longer set imagemetadata.DefaultBaseURL in bootstrap? and does tools.DefaultBaseURL continue to be updated?
<axw> (sorry for dumb questions, just trying to be thorough...)
<wallyworld_> axw: np. the image metadata is uploaded from a local dir to cloud storage
<wallyworld_> it was a mistake to set th default url
<wallyworld_> in the cloud storage, the uploaded metadata is in the search path
<wallyworld_> and keeping default url unchanged means we fall back
<wallyworld_> if no images found in user metadata
<axw> ok
<axw> whereas if we change it, it'll try to look in cloud storage and in the local dir
<axw> right?
<wallyworld_> yeah, but it will error out later in the boot process
<wallyworld_> i didn't look into why
<axw> mk
<wallyworld_> i decided it was wrong to change the default url
<wallyworld_> tools url is changed cause that's what sync tools needs
<wallyworld_> end result - tools and image metadata is uploaded to cloud storage if local dir is used
<axw> thanks
<wallyworld_> and cloud storage is in simplestreams search path, behind config urs and ahead of default locations
<axw> got it
<axw> I remember why FullPath/URL was required before: we used to do a URL fetch, rather than go through storage
<axw> so we previously had to keep the URL in memory to pass along to the merging bit
<wallyworld_> axw: just to check - i'm pretty sure that we don't need full url now right
<axw> wallyworld_: right
<wallyworld_> cool, that matches my understanding too
<axw> since we use the path & storage now
<wallyworld_> yep
<wallyworld_> just wanted to be 1000% sure :-)
<axw> :)
<wallyworld_> axw: with the abs(), i thought it better to report the whole dir to the user to avoid confusion
<axw> okey dokey
<jam> axw: do we actually have a solution for people trying to use juju-core with Canonistack and setting up 'sshuttle' while they are trying to bootstrap?
<jam> or does bootstrap itself complete because it uses the SSH redirection, (I thought one of the steps was to directly connect to the API to know that it is up, so now that step will fail)
<axw> jam: bootstrap works because it goes through SSH, sshuttle is still required as it was before post-bootstrap
<axw> there's no API interaction during bootstrap
<jam> axw: so "before" you could use sshuttle to the bootstrap node once it came up, and then everything else would go direct to the API, that stil works?
<axw> we did talk about that during SFO - juju status at the end - but it hasn't been implemented
<jam> ah, k
<axw> jam: yep
<axw> sshuttle -r ubuntu@bootstrap-node subnet/cidr
<axw> I would kinda of like it if API connections did some magical tunnelling as a fallback, but it's probably too messy
<jam> axw: It would be helpful for people at Canonical, I'm not sure that it is a general solution.
<jam> Saying "you need public access to the cloud machines you are using" doesn't seem like a particularly bad thing to me.
<axw> yeah I guess so. I don't really know how many of our private cloud users will have the same issues
<axw> probably not that many
<mgz> morning
<axw> morning mgz
<jam> axw: I think you'll generally need some-sort-of-VPN but we actually have 2 different ones inside Canonical. (the Server Stack needs OpenVPN to access, and doesn't work via SSH at all)
<axw> true, though our bootstrap won't even work without ssh access
<axw> I was just thinking it'd be nice to have guaranteed API access if bootstrap succeeds
<axw> it's not a big deal to have to sshuttle though, really
<mgz> I may be a min or two late for meeting later
<rogpeppe> wallyworld_: ping
<rogpeppe> axw: you reviewed this branch, i think: https://codereview.appspot.com/58510044
<rogpeppe> axw: i'm having difficulty understanding how it works - did you work it out?
<axw> rogpeppe: sorry, which part?
<axw> doh, good catch on the dummy storage boolean...
<axw> rogpeppe: re "How does the URL get into the metadata now?" -- simplestreams metadata doesn't encode the URL
<axw> we used to store it in the in-memory object when tools metadat was fetched by URL
<axw> but now we use the path, and go through storage
<axw> rogpeppe: and the fact taht we were calling the environment storage'URL
<axw> oops
<axw> storage's URL method, caused an error wtih MAAS
<thumper> where's wallyworld_
<wallyworld_> here
<wallyworld_> on the juju call. yay. will look in a bit. i'm also 3/4 drunk :-D
<bigjools> wallyworld_: ROFL
<thumper> wallyworld_: 3/4 drunk and working on maas fixes \o/
<wallyworld_> no, i'm ok now :-)
<bigjools> haha
<wallyworld_> it's been an hour or more since my last wine, i was exagerating :-)
<wallyworld_> a man can drink a glass of wine with dinner :-)
<bigjools> digging more like
<bigjools> a single glass has been known to get you blotto
<fwereade> right, I'm not interviewing horacio right now because we need to turn the power off here to investigate scary burning smells
<fwereade> hopefully I will be back shortly
<natefinch> fwereade: good luck
<fwereade> cheers
<jam> mgz: are you available to just hang out?
<rogpeppe> axw, wallyworld_: what's the purpose of ToolsMetadata.FullPath? are we actually setting it now?
<rogpeppe> ah! it's ignored
<wallyworld_> rogpeppe: it's used when wget fetches the tools
<wallyworld_> it's ignore writing out
<rogpeppe> wallyworld_: ok
<wallyworld_> it is composed from the relative path stored in the json
<wallyworld_> composed when needed
<wallyworld_> rogpeppe: is trunk working?
<rogpeppe> wallyworld_: i'm going to try it in a mo
 * wallyworld_ crosses his fingers
<mgz> jam: sure
<jam> mgz: I'm just going to grab a snack, mumble or g+?
<mgz> lets try mumble first, I'll be on there
<jam> yeah, I can't understand you at all, which means it is messed up, I'll reconnect 1 more time
<mgz> jam: we'll have to just fall back to hangout
<mgz> use the daily standup  one?
<rogpeppe> wallyworld_: sorry, it appears that there are no spare boxes currently, so testing it will have to wait for a little bit
<wallyworld_> ok, np
<jam> mgz: now ff is freaking out again... just a sec
<mgz> :)
<wallyworld_> rogpeppe: i'm quietly confident it will work :-)
<jam> mgz: I'm in the "juju core team" one from earlier today
<rogpeppe> wallyworld_: did you see my remarks on your CL?
<wallyworld_> no, i'll look
<wallyworld_> rogpeppe: tomorrow i'll fix the remaining issues, thanks
<rogpeppe> wallyworld_: thanks
<wallyworld_> rogpeppe: it seems that we're still not quite universally consistent with converting relative paths to absolute when running commands
<rogpeppe> wallyworld_: yeah, we should probably use Context.AbsPath throughout
<wallyworld_> yeah, i didn't realise we had that method
<dimitern> rogpeppe, i'm trying to debug this missing GOPATH issue that cause uniter tests to fail in an install hook
<dimitern> rogpeppe, so i'm using your debug.Callers() thing to dump the stack when I detect GOPATH is empty in testing/charms.go:init()
<rogpeppe> dimitern: ok
<dimitern> rogpeppe, but what I'm getting is a bunch of files, and the line number for each one is always the last line in the file
<dimitern> rogpeppe, what does that mean?
<rogpeppe> dimitern: paste?
<jam> rogpeppe: so if you *don't* supply -test.timeout, then when it times out it just says "test took to long" if you *do* supply a value (like 10s in my testing) then you get a panic
<dimitern> rogpeppe, http://paste.ubuntu.com/6843438/
<dimitern> rogpeppe, and my changes to init(): http://paste.ubuntu.com/6843442/
<dimitern> rogpeppe, for some reason during the install hook the environment gets reset or something and GOPATH is lost
<jam> rogpeppe: it looks like '-test.timeout' is for each individual test triggering a panic, vs the global timeout being run by the meta-process when running multiple packages
<rogpeppe> jam: interesting
<rogpeppe> dimitern: sorry, i got waylaid by insanity. i have to go to lunch now, sorry, otherwise i won't make it to the discussion in 40 mins. will look when i can.
<dimitern> rogpeppe, ok
<dimitern> rogpeppe, I found a solution: http://paste.ubuntu.com/6843674/ adding GOPATH to the list of env vars a hook gets
<rogpeppe> dimitern: yeah, i thought that might be the problem
<dimitern> rogpeppe, what i don't get is why it wasn't a problem before?
<rogpeppe> axw: are you around?
<rogpeppe> dimitern: agreed - i'm not sure either
<rogpeppe> dimitern: there's a perhaps-better solution
<dimitern> rogpeppe, what's that?
<rogpeppe> dimitern: just make it so init doesn't panic
<dimitern> rogpeppe, but it won't solve the issue - it won't find the repo path still
<rogpeppe> dimitern: it doesn't need the repo path
<rogpeppe> dimitern: it's just re-running the test binary as a hook, right?
<dimitern> rogpeppe, why don't remove init() altogether then?
<dimitern> rogpeppe, it seems better to get the absolute path of the charms.go in testing and infer the path from there
<dimitern> rogpeppe, is that possible?
<rogpeppe> dimitern: how would you do that?
<rogpeppe> dimitern: i think that removing the init altogether is reasonable though
<dimitern> rogpeppe, well, in python i'll do it with os.path.realpath(".") or something
<rogpeppe> dimitern: the current directory isn't related to where charms.go is
<dimitern> rogpeppe, no, sorry - i'll use the __file__ magic const that contains the full path to the current file
<dimitern> rogpeppe, kind of like $0 in bash
<rogpeppe> dimitern: define testing.Charms as a function, say func() *charms.Repo
<rogpeppe> dimitern: no such thing in Go
<jam> there is some 'build' runtime magic stuff
<rogpeppe> dimitern: (and probably a good thing too, as we'd be tempted to do this, and it would be a bad idea :-])
<rogpeppe> jam: we're already doing that
<jam> we use it in testing/charms.go ? something ilke that
<rogpeppe> jam: but GOPATH isn't set if we re-exec the binary
<rogpeppe> dimitern: something like this perhaps? http://paste.ubuntu.com/6843741/
<rogpeppe> dimitern: that actually makes tests that don't use testing charms slightly faster
<rogpeppe> s/tests/suites/
<axw> rogpeppe: sort of around, what's up?
<rogpeppe> axw: just wondering whether it's currently possible to manually bootstrap on a node, but not use the manual/null provider
<axw> rogpeppe: bootstrap or "provision"?
<axw> add-machine?
<rogpeppe> axw: bootstrap
<axw> no
<TheMue> rogpeppe: interested in reviewing https://codereview.appspot.com/58510045/? it still misses tests (have to think about it and how to do best), but it works live.
<rogpeppe> axw: right, i thought so
<rogpeppe> axw: but needed to check
<axw> rogpeppe: nps. who wants that and why?
<rogpeppe> axw: there are currently some people who have duplicated the entire bootstrap logic in shell (and it works... kinda)
<rogpeppe> axw: just to get this behaviour
<rogpeppe> axw: the specific reason is that they want to bootstrap juju onto a maas controller node
<dimitern> rogpeppe, it doesn't work
<axw> oh, the installer thing
<dimitern> rogpeppe, it still fails if GOPATH is not set
<rogpeppe> dimitern: doesn't matter
<rogpeppe> dimitern: why would that hook code be calling testing.Repo ?
<axw> rogpeppe: why do they not just use the manual provider...?
<axw> no ssh?
<rogpeppe> axw: because they want the environment to be using the maas provider
<axw> oh yeah, right
<dimitern> rogpeppe, it doesn't, it just imports stuff from testing
<rogpeppe> dimitern: right, so it's ok then
<dimitern> rogpeppe, adding GOPATH to that list of vars fixes the issue, i don't want to make it any more intrusive, and i want to land this already
<rogpeppe> dimitern: that actually makes it *more* intrusive, no?
<rogpeppe> dimitern: because that list of vars is in production code, no?
<dimitern> rogpeppe, so what - in production it will be empty if you don't have go installed, and even if you do, it won't affect anything
<rogpeppe> dimitern: we should not be changing production code for this kind of reason - it's a testing bug and it should be fixed in the testing code
<rogpeppe> dimitern: there might even be a less intrusive way, one mo
<dimitern> rogpeppe, what you proposed with making Repo a func doesn't work, because it's a type and it's exported
<dimitern> rogpeppe, I have to refactor the whole thing
<rogpeppe> dimitern: sorry, i meant Charms
<rogpeppe> dimitern: but, i'm pretty sure there's a much less intrusive way
<rogpeppe> dimitern: just checking
<dimitern> rogpeppe, in every test that uses testing.Charms.xxx I have to go and change it so it reads testing.Charms().xxx
<rogpeppe> dimitern: sure, but that's one trivial global change: gofmt -r -w 'testing.Charms -> testing.Charms()'
<rogpeppe> dimitern: but, again, i think there's a better way
<dimitern> rogpeppe, I'm listening
<rogpeppe> dimitern: i'm still testing whether it's viable
<rogpeppe> dimitern: something like this: http://paste.ubuntu.com/6843805/
<rogpeppe> dimitern: those are the only changes necessary, i think
<dimitern> rogpeppe, I'll try it
<TheMue> rogpeppe: your last hint to me works fine and can be reviewed ;)
<rogpeppe> TheMue: i'll have a look
<TheMue> rogpeppe: thx. unit tests are still missing, have to think how to do it best
<rogpeppe> TheMue: what's the backward compatibility issue you mention w.r.t. environtag/
<rogpeppe> ?
<rogpeppe> TheMue: oh, i see
<rogpeppe> TheMue: why not just do that behaviour server side, i.e. no tags means everything?
<TheMue> rogpeppe: ah, fine. yes, today the command doesn't need an argument. william wanted, that always a tag has to be passed
<rogpeppe> TheMue: then you won't need any special casing for the environment tag at all
<rogpeppe> TheMue: really?
<TheMue> rogpeppe: but i have to add an error replay from server-side w/o a tag
<rogpeppe> TheMue: seems odd to me
<TheMue> rogpeppe: yes, really ;)
<rogpeppe> TheMue: i don't understand the "error replay" thing
<TheMue> rogpeppe: I could live with a default choosing the environment too, would make it simpler
<TheMue> rogpeppe: eh, reply
<rogpeppe> TheMue: yeah
<TheMue> rogpeppe: if you initially call w/o entities or later set empty entities
<rogpeppe> TheMue: it simplifies things
<dimitern> rogpeppe, this works yes, tyvm
<rogpeppe> dimitern: np
<TheMue> rogpeppe: definitely, please add a comment regarding this point
<rogpeppe> dimitern: sorry for the pushback, but the GOPATH thing sounded like creeping dependencies, fixing things far away from where the actual problem was
<rogpeppe> TheMue: will do
<dimitern> rogpeppe, yeah, I agree - sorry, but I was trying to land this for a day now and getting really frustrated
<TheMue> rogpeppe: but filtering is really nice ;) tested it with one and multiple tags
<rogpeppe> TheMue: cool
<rogpeppe> dimitern: i know the feeling
<jam> rogpeppe: we had a gecko show up inside, which my wife made quite clear that it had to go :)
<rogpeppe> jam: lol
<rogpeppe> jam: from a position firmly on top of a table?
<jam> they're a bit harder to deal with than mice, as mice don't climb 2m up the wall
<jam> rogpeppe: not quite so bad, but certainly back off a ways and quite vocal
<mgz> geckos sound fun
<rogpeppe> mgz: +1
<jam> mgz: they're actually really quite cute (IMO) but I can understand not wanting to catch unexpected motion out of the corner of your eye. It can be a bit disturbing.
<mgz> okay, lunch for me, hae proposed a much simplified https://codereview.appspot.com/56560043 along the lines you suggested jam (previous rev has the real cert stuff in)
<mramm> so anybody got an update for me on the 1.17.2 world
<mramm> as in, are we able to do a 1.17.2 release anytime soon (as measured in hours)
<mramm> sinzui: ^^^?
<marcoceppi> mgz: are you still going to be able to make it to cfgmgmt camp?
<mgz> marcoceppi: yeah, I'll give you and rbasak my travel details
<marcoceppi> mgz: cool, robbie will be at fossdem but not cfgmgmt camp, that'll just be us
<mramm> arosales: you around?
<sinzui> mramm, local provider is very broken in trunk. It is not releasable.
<sinzui> mramm, but when we have a blessed revision, a release takes 4 hours
<mramm> juju dev team, is there a way to get just 1.17.1 plus the MAAS bootstrap fix as a branch that we could release as 1.17.2?
<mgz> sinzui: so, in the meeting this morning our antipodes were under the impression the only remaining broken things were misunderstandings/script things to fix
<mgz> sinzui: from the mailing list thread, I'm not clear what's still borked
<sinzui> mgz, maybe one thing needs to be fixed. local deploys of 1.17.1/2 have not worked since last Friday, so I have nothing to give me confidence that there is only one thing blocking the fix
<sinzui> mgz, though We can see that changes to trunk are fixing things. Since yesterday, local deploy fails because wordpress fails to start: http://162.213.35.54:8080/job/local-deploy/
<sinzui> We also see this in some of the failing azure tests
<rogpeppe> TheMue: you have a review
<mramm> mgz: sinzui: who do we need to get into a hangout to sort out these issues?
<mgz> mramm: I think ideally the US/AUS overlap time
<mramm> well, that is quite a bit from now...
<mramm> but if that's where we need a meeting, we can do it
<arosales> mramm, hello
<mramm> but I'd like to have everything we can sorted out by then
<mramm> arosales: saw your message regarding MAAS for the charmers -- I have one priority over that which is MAAS for CI
<mramm> I think we can use the MAAS testing lab for that
<mramm> james page and I were just talking about it
<arosales> sinzui, are you available for the juju cross team?
<mramm> is that now?
<mramm> joining
<mramm> sorry lost track of time here
<sinzui> arosales, now? me in standup
<arosales> sinzui, ah ok. I think the cross team meeting is an hour early than it was in 2013
<arosales> mramm, ^
<TheMue> rogpeppe: thank you
<mramm> arosales: we can move it later
<arosales> sinzui, and simple stream workflow is still in progress or waiting on utlemming
<TheMue> rogpeppe: I like the idea with the regex, it is more powerful. will change it.
<TheMue> rogpeppe: and also to no-filter-means-show-everything
<rogpeppe> TheMue: thanks
<sinzui> mgz, I consistently see a mysql error in local deploy http://162.213.35.54:8080/job/local-deploy/767/console
<sinzui> mgz, We are not getting logs though...
<sinzui> mgz, do I dare increase the memory constraints as we did for HP
 * sinzui accidentally deployed juju on 512m this week and was surprised to see the tests pass...juju's memory requirements probably shrank
<mgz> yeah, juju itself is generally good
<hatch> I am trying to determine which version of juju-core actually removes the requirement for sudo on local deployments? 1.17.1 seems to require it but I'm hearing reports that 1.17.2 does not
<mgz> mysql still does that "allocate 80% of machine's memory" thing though
<mgz> which is probably not good if there's anything else on the box at all
<sinzui> hatch, trusty + 1.17.2 (which I have not released
<hatch> sinzui what about precise?
<hatch> do you guys consider the 1.17.x release to be a non-stable release? Should quickstart use sudo up to 1.18 ?
<sinzui> hatch, precise version will still ask for sudo on demand
<hatch> even in 1.18?
<sinzui> 1.18 will request sudo passwords on demand
<hatch> ok so trusty with 1.17.2 or greater will not require sudo, everything else does
<hatch> sinzui: sorry that was a question :)
<sinzui> hatch, yes. In general, juju will will ask for input as needed. That means your scripts need to provide sudo passwords after they call bootstrap, or do a trivial sudo op before bootstrapping so that the op is not paused for a password
<hatch> sinzui right, but we also don't want to run it as sudo when it's not required to so I was hoping for something we could programatically check for to see if sudo will be required
<mgz> hatch: basically it depends on juju version and lxc version
<sinzui> okay, hatch, I understand. "juju version" will return 1.XX.X. if the dotted numbers are greater than 1.17.1, don't use sudo
<hatch> mgz, well, apparently also the ubuntu series
<hatch> sinzui even on precise?
<mgz> that's only an indication oflxc version
<sinzui> hatch lxc version is implicit with series
<sinzui> hatch, yes, even precise that I test
<sinzui> This is was ran 10 minutes ago
<sinzui> juju --show-log bootstrap -e local --constraints mem=2G
<sinzui> on precise local
<hatch> ok perfect, haha wow that was confusing for a bit :)
<hatch> version > 1.17.1 ! require sudo :D
<sinzui> bingo
<hatch> ok on it!
<hatch> sinzui will 1.17.x be released to the stable ppa or is it just a development version?
<sinzui> hatch, odd numbers are always unstable and not subtle for production
<sinzui> hatch, when we think the 17 changes are complete, we will call them 1.18.0
<hatch> ahh ok gotcha
<hatch> doing the node.js like version releases :)
<frankban> sinzui: can we assume odd numbers are never published on the stable ppa or on main/universe?
<sinzui> frankban, The answer should be yes and yes...
<frankban> sinzui: cool thanks
<sinzui> frankban, BUT
<frankban> sinzui: I feel like there is a BUT
<sinzui> frankban, I had no intention of letting odd numbers into the ubuntu devel (trusty), but jamespage wanted it in trusty to test gccgo
<bac> rogpeppe: i'm getting a mongo error from juju 1.17.1 when trying to deploy a charm.  was wondering if you've seen it before.  https://pastebin.canonical.com/103823/plain/
<bac> rogpeppe: on trusty with local provider
<frankban> sinzui: so 1.17.1 is in trusty?
<rogpeppe> bac: 1.17.1 is broken
<rogpeppe> bac: i can't see your paste, but i know what it's gonna look like
<bac> rogpeppe: good parts: E11000 duplicate key error index: juju.charms.$_id_ dup key:
<frankban> sinzui: yes it is, thank you
<mramm> frankban: yes it is in trusty
<mramm> 1.17 is the last time we will push a dev release into trusty though
<mramm> we need to get better at releasing stable versions on a regular schedule
<mramm> that will eliminate the felt need to push devel releases into the release
<rogpeppe> bac: ah, that's a different issue
<mramm> but we needed to get that process running, due to the main inclusion process that we need to go through this cycle
<rogpeppe> bac: dimitern knows about that, i believe
<rogpeppe> bac: it's something to do with charm uploads - were you running deploy twice, or something?
<rogpeppe> bac: (twice concurrently, i mean)
<bac> rogpeppe: no.  tried one deploy from juju-gui and got the error
<rogpeppe> bac: hmm.
<dimitern> bac, I'm trying to land a fix for bug 1067979 since yesterday - hopefully not long now
<_mup_> Bug #1067979: race: concurrent charm deployments corrupts deployments <deploy> <race-condition> <test-needed> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1067979>
<rogpeppe> bac: can you reproduce?
<bac> rogpeppe: yes, could yesterday via the gui.  was going to spin up an env by hand and see if i can reproduce
<bac> rogpeppe: was just checking first to see if it was a known issue that could be waved off.  glad to help with reproducing if needed
<rogpeppe> bac: once dimitern has landed his fix, perhaps you could try to see if you could reproduce the problem from trunk tip
<bac> will do, rogpeppe
<TheMue> rogpeppe1: take a look: http://paste.ubuntu.com/6844672/
<TheMue> rogpeppe1: ;)
<dimitern> bac, the fix for bug 1067979 landed, btw
<_mup_> Bug #1067979: race: concurrent charm deployments corrupts deployments <deploy> <race-condition> <test-needed> <juju-core:Fix Committed by dimitern> <https://launchpad.net/bugs/1067979>
<natefinch-afk> rogpeppe1: When you get a chance, I'd like to talk about the machine agent stuff again.  The API code has me mystified as to where the info is that I'm supposed to be watching
<rogpeppe1> natefinch-afk: ok, perhaps soon?
<natefinch> rogpeppe1: sure thing
<natefinch> rogpeppe1: just wasn't sure how late you'd be there today
<dimitern> natefinch, a quick review? https://codereview.appspot.com/54680045
<dimitern> rogpeppe1, fwereade, https://codereview.appspot.com/54680045 ?
 * dimitern bbiab
<rogpeppe1> natefinch: i'm here for another hour, but currently in a discussion
<arosales> mgz, fwereade using the conf call today for the joyent provider sync
<natefinch> rogpeppe1: ok, I know that takes precedence
<natefinch> dimitern: I can review
<mgz> I'll need to read the wiki about conf call things
<mgz> arosales: I actually don't have a pin, if I need one, and getting one seems to involve an rt
<dimitern> natefinch, thanks
<arosales> mgz sent you the callin details and updated the invite
<mgz> dstroppa: so, first thing to try with the amulet tests is adding sentries=False when constructing amulet.Deployment()
<mgz> then just remove the d.sentry.wait() call below, it's not strictly needed
<mgz> it gives you fewer hooks to actually inspect your charms and hooks, but can be changed back later when you need them
<dstroppa> mgz: tried that, this is what I'm getting now http://paste.ubuntu.com/6844982/
<mgz> dstroppa: much better
<mgz> dstroppa: get lp:juju-deployer and put the script on your path
<dimitern> natefinch, review poke?
<natefinch> dimitern: yeah, sorry, had to do one thing in the meantime, but working on it now
<dimitern> natefinch, cheers
<dstroppa> mgz: I'm getting syntax errors when installing juju-deployer
<rogpeppe1> natefinch: hangout?
<natefinch> rogpeppe1: yeah, standup one?
<rogpeppe1> natefinch: ok
<rogpeppe1> natefinch: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=1
<rogpeppe1> TheMue: cool
<natefinch> rogpeppe1: you're frozen in the hangout?
<dpb1> Hi, is this a bug?  I'm trying to do juju terminate-machine, and I'm in this weird state.  I think that this instance launched in an 'ERROR' state, fwiw:  http://paste.ubuntu.com/6845201/
<fwereade> dpb1, you should be able to --force it
<dpb1> fwereade: nice.  I suppose it will not clean up the instance in the "ERROR" state, though?
<dpb1> --force appears to at least removed it from juju's brain, which is what I needed.
<fwereade> dpb1, afraid not, it's just a way to get it out of juju's hair
<dpb1> fwereade: k, thx
<rogpeppe1> g'night all
<natefinch> dimitern: reviewed, sorry for the delay
<marcoceppi> got a question about api stuff?
<marcoceppi> that was a astatement
<natefinch> marcoceppi: I'll answer if I can
<marcoceppi> natefinch: nvm, sorted as maas sadness
<natefinch> marcoceppi: heh ok
 * thumper has a few errands in town to prep for the trip
<thumper> will be back soonish
<bac> dimitern: you still around?
 * thumper wants to stab something
<thumper> ffs
<thumper> natefinch: you around?
<natefinch> thumper: yep
<natefinch> thumper: please don't stab me
<thumper> natefinch: have you tried the ec2 provider recently
<thumper> mine won't start
<natefinch> thumper: nope, I can try it
<thumper> ERROR cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again.
<thumper> I'm trying to try out my all-machines fix for the local provider
<thumper> and want to confirm it doesn't break others
<thumper> hmm... seems to be working now
<thumper> after failing twice
<thumper> wonder if it was an amazon glitch
<natefinch> thumper: yeah weird
<thumper> natefinch: lbox still hates me (no go vet), can you review on LP? https://code.launchpad.net/~thumper/juju-core/all-machines-trusty/+merge/204106
 * thumper also wants to get waigani to test it
<thumper> as I know he is on saucy still and had a similar problem with no all-machines.log
<thumper> I'm pretty sure this will fix it, but always good to get confirmation.
<natefinch> thumper: at least it's a small change, so not so bad w/o side by side diffs
<thumper> yeah
<thumper> uh oh
 * thumper has a bad feeling
<thumper> poo
<thumper> doesn't work on precise
 * thumper hangs head
<thumper> FARRRRKKK!!!!!
<natefinch> thumper: my wife knows the guy that owns fark, actually.
 * thumper resists the urge to have series specific rsyslog rendering
<natefinch> thumper: still want a review on that code, or...
<thumper> not just yet
<thumper> stabby stabby stabby
<thumper> natefinch: care to take a look now?
<thumper> just confirming on ec2
<thumper> works for local
 * thumper looks for someone not on trusty
<natefinch> thumper: looking
<thumper> hoo fucking ray
<thumper> seems to work on precise
<natefinch> thumper: maybe other people more familiar with rsyslog and bash would understand it, but personally, I'd like a comment about why you have to set and reset FileCreateMode like you do.
<thumper> natefinch: where do you want the comment?
<thumper> I thought I had one?
<thumper> ah, it is in the description of the merge
<thumper> can put it in the code somewhere if you like
<natefinch> Yeah, just the first time you do it.
<thumper> I'll have it in the block comment for the template itself
<thumper> I don't want it rendered in the actual config file
<natefinch> right
<natefinch> ok, time for me to go.  Have fun in cape town
<hazmat> thumper, so i'm using latest trunk, and juju get-env doesn't show proxy config as an option.. how does one go about setting it?
<thumper> probably because the default is to Omit it if not set
<thumper> juju set-env http-proxy=http://foo
<thumper> ya know
<thumper> I'm tempted to change it from Omit to default to ""
<hazmat> got it.. revno 2224
<thumper> what do you think?
<thumper> because if you have one set, you can't unset it
<thumper> as it won't allow ""
<thumper> stupid juju
<hazmat> hmm.
<hazmat> thumper, a doc merge proposal would do :-)
<hazmat> its pretty hard to discover otherwise
<thumper> oh for sure
<thumper> I want it feature complete before we bang on about it though :)
<hazmat> the diff on revno 2224 shows all the options. i needed to find.
<hazmat> thumper, i'll have users testing it tonight :-)
<thumper> cool
<thumper> let me know if there are problems or surprises
<thumper> I probably won't see the email until I'm in cape town
<thumper> but I will see it
<hazmat> thumper, will do.. careful what you wish for :-)
<thumper> you going to be there?
<hazmat> thumper, yup.. i should be in saturday late
 * thumper nods
<thumper> I get in 9am ish sunday morning
 * thumper has been futzing around with rsyslog configs
<hazmat> there's some other fire i'm parachuting in to work on.. which will take up most of the freetime during the sprint week, but this project with proxies will probably still keep kicking and screaming.
<sinzui> thumper, lxc is looking better today, upgrade tests work. deploys always fail because of a wordpress agent error. DO you have any ideas about r2282 that would break the agent
 * thumper looks at that revno
<sinzui> trusting on aws is broken too, but I think that is aws since canonistack's trusty passed
<thumper> sinzui: I just used aws with trusty and trunk ok
<thumper> I hit a glitch with s3 when I first tried
<thumper> but that resolved itself
<sinzui> thumper, lucky you. I have many tests and 4 personal runs that cannot bootstrap
<thumper> sinzui: no idea
<thumper> sinzui: try now
<thumper> it was working for me
<thumper> sinzui: we should get a real test that looks for the all-machines.log for the local provider
<thumper> I have just fixed the bug and merge should be working through tarmc
<sinzui> thumper, I add recovery of that log today for lxc
<thumper> lxc or local?
<thumper> we have containers in other providers remember :)
<sinzui> thumper, lxc. since we had no logs and lots of failures, I wrote a hack to find the log in local/log before the machine is torn down.
<thumper> I have a horrible feeling that I am missing something important that I should be doing today to prepare for the trip tomorrow
<sinzui> http://162.213.35.54:8080/job/local-deploy/780/ is the last failure
<thumper> sinzui: oh, you'll love my last change then
<sinzui> thumper, CI scp's the log from the bootstrapped machine, but it has never been able to do that with lxc
<thumper> I keep the log file around until you re-bootstrap
<thumper> will be in /var/log/juju-<user>-<envname>/all-machines.log
<sinzui> thumper, thank you
<sinzui> thumper, aws trusty still dies on apt-get upgrade
<thumper> hmm
<thumper> are you deploying to trusty?
<thumper> I'm not doing that
<sinzui> thumper, I am just boostrapping on trusty and failing.
<sinzui> it worked for CI yesterday
<thumper> hmm...
<thumper> sorry, but worked for me just an hour ago
<sinzui> thumper, I think it is aws.
<thumper> waigani_: can I get you to pull trunk, and try a local provider?
<thumper> waigani_: see if you get all-machines.log now
<thumper> I expect that you should
<waigani_> okay
<sinzui> the unittests failed when they removed the ami we were using, so I expect to tweak the tests to keep them current with trusty in aws
<bigjools> morning wallyworld_ how's the head? :)
<wallyworld_> fine
<wallyworld_> jeez
<sinzui> wallyworld_, I am seeing azure header issues.
<wallyworld_> oh
<sinzui> wallyworld_, I just started a manual test of stable to reassure myself
<wallyworld_> you are seeing issues running trunk?
<sinzui> wallyworld_, we are see this with recent revs: http://162.213.35.54:8080/job/azure-deploy/758/console
<sinzui> looking good for the manual bootstrap
<wallyworld_> sinzui: i can't think of anything that has changed in juju wrt that error
<sinzui> I fear that azure has changed.
<wallyworld_> oh joy
<sinzui> We find the machines are often left running or stopped in azure after a test.
<wallyworld_> in azure but not ec2 or other clouds?
<sinzui> wallyworld_, azure and canonistack are leaving machines behind. but in the case of azure, I am seeing the header errors and a lot of 307 temporary redirect errors that state the delete failed
<wallyworld_> hmmmm. off hand, i can't offer any useful insight
<sinzui> wallyworld_, good news. 1.16.5 on azure is good. I will try to collate the trunk errors into something meaningful
<wallyworld_> good or worked when run but might be flakey next time?
<wallyworld_> trunk could well be to blame. there have been no azure specific changes to my knowledge but destroy for example has changed and maybe that is indirectly causing issues on azure
#juju-dev 2014-01-31
<sinzui> thumper-afk, wallyworld_ CI cannot make a tarball for r2283. The error here implies Lp or goyaml project is borked. http://162.213.35.54:8080/job/prepare-new-version/960/console
<sinzui> ^ I cannot make sense of it
<wallyworld_> let me look
<wallyworld_> i blame lp
<wallyworld_> the last commit to trunk was back in nov i think
<sinzui> I agree
 * thumper is back
<sinzui> I can branch that though
<wallyworld_> sinzui: maybe just retry? could be transient?
<sinzui> I did two reties, but I am doing a 3rd since I can see the branch
<sinzui> http://162.213.35.54:8080/job/prepare-new-version/961/console
<wallyworld_> this time tomb is borked
<sinzui> I just branched tomb as well
<sinzui> what!
<sinzui> wallyworld_, This is my rapid rebuild: http://162.213.35.54:8080/job/prepare-new-version/962/console
 * sinzui finds an old-fashioned losa
<wallyworld_> wtf
<sinzui> I have have several rebuilds now and the failures are always a random Lp branch
<sinzui> wallyworld_, I am going to let CI rest. Lp is have bazaar issues. random 503s for branches that are fine
<wallyworld_> ok
<wallyworld_> sinzui: did someone ask you about a 1.17.2 release?
<sinzui> Yeah
<wallyworld_> would be easier if lp worked
<thumper> sinzui: y u break lp?
<sinzui> Local deploy hasn't passed in 7 days though, and and azure is doesn't work either
<thumper> sinzui: well that is fucked up
<thumper> I wish I knew what was going on there
<thumper> I deployed to local on trusty today :-(
<wallyworld_> it worked?
<thumper> yes
 * thumper think
<wallyworld_> i must retry that
 * thumper tries
<thumper> wallyworld_:sinzui, yes just deployed the ubuntu charm locally successfully
<hazmat> do addresses need to be defined up front when adding machines to state, or can they be detected at runtime by the machine agent?
<axw> thumper: does rsyslogd create /var/log/juju-${namespace} if it doesn't exist...?
<axw> I can't see a mkdir for it
<thumper> axw: i believe it should
<axw> mk
<hazmat> thumper, can proxy values be set at runtime?
<thumper> hazmat: yes
<hazmat> axw, do addresses need to be defined up front when adding machines to state via inject machine, or can they be detected at runtime by the machine agent?
<hazmat> cool
<axw> hazmat: they can be detected, but if you do that there's no guarantee about which one will be "public"
<axw> so dns-name in status may show up some interface you don't want
<axw> s/can/will/
<axw> hazmat: I made a change just yesterday to the manual provider to record the bootstrap-host as public
<axw> for that reason
<axw> thumper: is there a reason not to change MachineConfig.LogDir, and not hard-code the /var/log/juju${namespace} in log/syslog?
<hazmat> axw, hmm. ic. thanks. the notion of public and private in a private cloud is a bit wonky.
<thumper> axw: yes there is a reason
<axw> hazmat: indeed
<thumper> axw: but on a call right now with waigani
<axw> ok
<davecheney> hey, did I hear that deploying to hp cloud was fixed ?
<sinzui> I added a jackhammer to CI. It will try harder to get the branches from Lp. r2283 is being tested now
<hazmat> davecheney, got a minute to help out with a golang cross compile setup?
<hazmat> davecheney, also does go support aarm64?
<hazmat> er t242-servergroup0002s:
<hazmat> er.. aarch64
<davecheney> hazmat: ack
<davecheney> golang-go does not support aarch64
<davecheney> if you are using trusty
<davecheney> install gccgo-go
<sinzui> I have more info on bug 1274780. postfix cannot be setup on the unit because the unit's host name doesn't appear to be sane
<_mup_> Bug #1274780: Wordpress charm no longer works with local deploy. <charms> <ci> <local-provider> <lxc> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1274780>
<hazmat> davecheney, hmm
<thumper> sinzui: I commented on that bug
<thumper> according to the RFC, dash is a valid hostname character
<sinzui> A leading dash is valid?
<davecheney> hazmat: yeah, mwd is working on the aarch64 support in gccgo
<davecheney> thumper: i think it cannot be the first character
<davecheney> experts-exchange.com << all good
<davecheney> -expertsexchange.com << not good
<thumper> we don't have leading dashes though do we?
<thumper> ah... -local-machine-1
<thumper> the local provider grabs $USER I think
<thumper> let me look
<davecheney> ah
<davecheney> $USER-local-machine-1
<thumper> namespace = fmt.Sprintf("%s-%s", os.Getenv("USER"), cfg.Name())
<thumper> sinzui: how are you running this?
<thumper> why is there no USER env var?
<sinzui> thumper, CI runs it
<thumper> sinzui: sure... how?
<thumper> CI isn't running like a user would
<thumper> so we need to fix CI
<thumper> to behave like a user
<thumper> that also means having a valid environement
<sinzui> thumper, This script is calling juju during the test of the env named local http://bazaar.launchpad.net/~juju-qa/juju-core/ci-cd-scripts2/view/head:/deploy_stack.py
<thumper> ok, so what is the environment the script is running under?
<davecheney> thumper: if juju requires a $USER to be set
<davecheney> then it should abort
<davecheney> bigjools: see what I did there
<thumper> davecheney: agreed
<davecheney> thumper: i think this might also be an issue during cloud init bootstrap
<sinzui> thumper, jenkins run a bash script that sets JUJU_HOME (which is not entirely honored) then destroys anything left behind by a previous test, then calls ^ that script to bootstrap deploy and status
<davecheney> sinzui: i'm having problems making saucy environments in HP cloud
<davecheney> i heard that there was an known issue
<davecheney> is this true ?
<thumper> sinzui: ok, we can get around this by adding "USER=jenkins-ci" to the environment
<sinzui> I have heard that, I have not test it
<thumper> in the same place you set JUJU_HOME
<thumper> but I do agree with davecheney
<thumper> juju should abort if it isn't set
<sinzui> I think it is set
<sinzui> jenkins@juju-juju-ci-machine-1:~$ echo $USER
<sinzui> jenkins
<sinzui> thumper, the charm only fails in local-deploy. 1.16.5 deploys it locally. So to other clouds, and the charm has not changed in a month
<thumper> sinzui: but CI has changed
<thumper> it works fine for me
<thumper> I can tell you that the process that is running juju doesn't have $USER set
<sinzui> You are a developer with taited setup
<thumper> that is why the name starts with a -
<thumper> it may be because the environment is being overridden when it shells out
<thumper> I don't know
<thumper> what I do know
<thumper> is that it isn't set
<davecheney> sinzui: I can confirm as of 12 hours ago I could not make saucy environments in HP cloud
<sinzui> I am forcing a built with an exported USER
<thumper> is there any way we can hook into the bit that calls juju
<thumper> to see what $USER is?
<sinzui> . o (sudo was set to pass the env, sudo is not used anymore)
<axw> thumper: how did this work before? we used $USER before too...?
<sinzui> davecheney, WTF, where is https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910/juju-dist/tools
<davecheney> i have no idea
<thumper> axw: you may make me go look, but check blame
<davecheney> i was using --upload-tools so I didn't notice
<thumper> I think it looked at SUDO_USER
<sinzui> davecheney, typo. I have got to get sleep
<axw> thumper: pretty sure it used $USER, but I'll double check
<thumper> axw: well, it worked before :)
<axw> thumper: ah, it used $USER and then $SUDO_USER if $USER wasn't set...
<sinzui> davecheney, I see the tools are in place, but have you lost os images?
<axw> or rather, if it was root
 * sinzui sees that trusty is not an option on hp
<thumper> sinzui: and now there is no fallback to check on the SUDO_USER so it is empty
<sinzui> thumper I award you a star â¶
<sinzui> Alas jenkins was so excited by the success it fell over and is reseting the tests, damn it
<axw> I think we can just fall back to using os/user
<axw> seems odd not to have $USER set though.
<davecheney> sinzui: we don't have trusty support anywhere
<davecheney> but if you can get a saucy macine
<davecheney> you can upgrade it
<sinzui> davecheney, You have a brilliant suggestion. We test trusty on aws and caonistack. azure has it too,
<davecheney> sinzui: to make trusty work we need two things
<davecheney> 1. juju has to understand trusty as a series
<davecheney> i'm looking for the magic constant now
<davecheney> 2. we have to have trusty images avaialble in simple streams
<axw> thumper: would you like me to fix this one, or are you doing it already?
<davecheney> i'm guessing that happens at alpah 1
<sinzui> dave 2. is true alpha two
<thumper> axw: fix what?
<axw> thumper: things not working if $USER not set
<thumper> axw: can you?
<axw> yes, I think so
<davecheney> cloudinit/sshinit/configure_test.go
<davecheney> 74:var allSeries = [...]string{"precise", "quantal", "raring", "saucy"}
<davecheney> ^ we have a lot of these magic constants
 * davecheney fixes
<sinzui> davecheney, I think the simplestreams module also defines that, then overrides it with the data from /usr/share/distro-info/ubuntu.csv
<davecheney> simplestreams does define trusty, i wonder where other places do not
<davecheney> sinzui: to clarify, suacy on HP appears broken, should I raise a bug ?
<sinzui> yes
<thumper> sinzui: are you coming to cape town?
<sinzui> Yes, thumper
<davecheney> sinzui: ok
<sinzui> http://www.youtube.com/watch?v=P3-f0k620Vg
<thumper> sinzui: I can't remember, do you drink alcohol?
<thumper> sinzui: if so, I'll get you a drink for all the CI work :)
<thumper> if not, then geez
<thumper> water
<sinzui> thumper, Most when trying to use windows, otherwise a beer does me well
 * thumper fixes two of the remaining proxy issues
<thumper> man my triceps hurt today
<thumper> I thought that boxing gave them a good workout
<thumper> but yesterday I took my first Gracie Jiu-Jitsu class
<thumper> very different muscles uses
<thumper> used
<thumper> axw, wallyworld_: https://code.launchpad.net/~thumper/juju-core/remove-container-special-proxy/+merge/204152
<axw> heading out for lunch in a few minutes, will take a look when I get back
<wallyworld_> looking
 * thumper EOWs and goes to pack
<dimitern> bac, hey, are you still there?
<axw> wallyworld_: would you mind taking a look at https://codereview.appspot.com/58140044/?
<wallyworld_> sure
<wallyworld_> i did see that one and meant to do it but forgot
<axw> nps
<axw> I forgot about it myself :)
<wallyworld_> fwereade: hi, i have a fix for the instance type selection. now selects 1024MB instance by default on canonistack (not 512MB) and constraints which cannot be satisfied eg cpu-cores=9000 are rejected https://codereview.appspot.com/58950043/
<fwereade> wallyworld_, *sweet*
<fwereade> wallyworld_, I will try to take a look today but am uncomfortably aware of a bunch of SA prep I am behind on
<wallyworld_> i added extra tests which illustrate the behaviour, seems to ll work ok
<wallyworld_> yeah. np. i won't land till you look, can wait till next week
<TheMue> fwereade: this is the current debug-log after the reimplementation, review by roger and changes based on that
<TheMue> fwereade: we decided to filter now with regexp, and it's really nice
<fwereade> TheMue, I direct the honourable gentleman to my previous response
<TheMue> fwereade: already played with it yesterday
<fwereade> TheMue, cool
<TheMue> fwereade: eh, that link was missing https://codereview.appspot.com/58510045/ ;)
<TheMue> fwereade: SA means South Africa?
<fwereade> TheMue, yeah
<TheMue> fwereade: will surely be an interesting trip. but also exhausting. mine next week is shorter, only flying to munich to give talks about juju and golang.
<fwereade> dimitern, rogpeppe1: standup?
<rogpeppe1> fwereade: 3 mins
<TheMue> fwereade, rogpeppe1: could it be that the Environment of state stores the name of the env, but doesn't provide an accessor?
<hazmat> what's provider-safe-mode do?
<hazmat> oh.. its the  don't kill instances
<TheMue> fwereade or rogpeppe1: when using the local provider, nothing is written into its all-machines.log. is this a known problem or only a local problem with rsyslog?
 * TheMue looks around in the channel. no one here?
<TheMue> fwereade: ping
<TheMue> rogpeppe1: ping
<rogpeppe1> TheMue: we're in the standup
<TheMue> rogpeppe1: still? ok
<TheMue> rogpeppe1: thx for info, i'll be patient. ;)
<bac> hi dimitern
<dimitern> hey bac
<dimitern> bac, i commented on the bug
<bac> hi dimitern, i've just read your bug report.
<bac> i'll try to isolate the problem and update the bug if i discover anything new.  thanks for looking at it.
<dimitern> sure
<bac> dimitern: your testing was on trusty?
<dimitern> bac, i'm on saucy and I've been deploying on precise
<bac> dimitern: thanks
<natefinch> fwereade, rogpeppe1, mgz, dimitern:  Morning guys, sorry I missed the standup.  Overslept.  ZoÃ« was up every hour last night :/
<dimitern> natefinch, morning ;)
 * TheMue needs help with local provider
<TheMue> rogpeppe1: ended standup?
<dimitern> rogpeppe1, review poke, when you can pls
<dimitern> TheMue, what's the issue?
<TheMue> dimitern: the bootstrap environment creates its all-machines.log but the data of machine-0.log isn't written into it
<TheMue> dimitern: the rsyslog config looks good so far (as I can tell)
<dimitern> TheMue, maybe syslog is not running?
<dimitern> TheMue, there's a syslog-port in the env.yaml you can set (and a default is used otherwise)
<TheMue> dimitern: thx, will try. rsyslogd is running
<fwereade> dimitern, mgz, rogpeppe1, hazmat: sorry, I need to go out for a bit -- is it better for me to be late, or for us all to push the meeting back 30 mins?
<fwereade> please confer amongst yourselves
<dimitern> fwereade, i'm ok pushing it 30m
<hazmat> sounds good
<rogpeppe1> fwereade: sgtm
<dimitern> natefinch, updated https://codereview.appspot.com/54680045/
<ahasenack> guys, any idea what's up with this error while bootstrapping on aws?
<ahasenack> 2014-01-31 13:42:36 WARNING juju.environs open.go:258 failed to write bootstrap-verify file: cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again.
<ahasenack> 2014-01-31 13:42:36 ERROR juju supercommand.go:282 provider storage is not writable
<rogpeppe1> TheMue: pong
<rogpeppe1> TheMue: (finally! sorry)
<natefinch> dimitern: looking
<TheMue> rogpeppe1: do you no any reason why no data is written into all-machines.log at the local provider? it is created by syslog
<rogpeppe1> TheMue: all-machines.log is at a different file path in the local provider
<rogpeppe1> TheMue: hence my comment on your review
<TheMue> rogpeppe1: yeah, sure, I know
<TheMue> rogpeppe1: I talk about the all-machines.log of the local provider
<TheMue> rogpeppe1: it is created at its right place below ~/.juju/<env>/log/
<TheMue> rogpeppe1: but empty
<TheMue> rogpeppe1: while e.g. machine-0.log contains data (which should be aggregated then)
 * TheMue is happy that at least testing with local provider is by far faster than with EC2 :)
<natefinch> dimitern: if the charm packaging code you put into the apiserver is basically a copy of the code in the charm package.... could we not just reference the charm package code here?
<dimitern> natefinch, no, fwereade wants them separate
<dimitern> natefinch, most of these are unexported helpers
<dimitern> natefinch, and exporting them is not an option
<rogpeppe1> TheMue: i'm afraid i don't know why - i'd have to look into the code
<rogpeppe1> TheMue: it might be something to do with the address that containers are putting into the syslog configs
<TheMue> rogpeppe1: ok, thx, only asked if somebody may made the same experience
<rogpeppe1> TheMue: i've not looked actually
<rogpeppe1> TheMue: one mo, i'll see if it happens on my machine
<dimitern> ahasenack, did you retry successfully?
<TheMue> rogpeppe1: I'll continue investigating
<TheMue> rogpeppe1: ta
<ahasenack> dimitern: no
<rogpeppe1> TheMue: i don't see any all-machines.log in my local env/log file
<rogpeppe1> s/file/dir/
<rogpeppe1> TheMue: but that may be an old juju version
<rogpeppe1> TheMue: let me try with trunk
<dimitern> ahasenack, if retrying doesn't help, file a bug please?
<rogpeppe1> anyone remember how to fix the gobot when there's an exclusive lock that it's blocked on?
<dimitern> rogpeppe1, yep
<ahasenack> dimitern: ok
<dimitern> rogpeppe1, kill the mongo and delete its sock file from tmp
<rogpeppe1> it's been moribund since 7am this morning
<rogpeppe1> dimitern: ok, thanks
<TheMue> rogpeppe1: I'm using one near trunk. location is $HOME/.juju/$JUJU_ENV/log/all-machines.log
<rogpeppe1> ha, i wonder what happens if you name your local environment "environments"
<hazmat> dimitern, hangout for that meeting?
<dimitern> hazmat, none created, we can reuse the standup one - https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig
<hazmat> argh.. not allowed to join call.
<dimitern> fwereade, rogpeppe1, mgz, ^^ (it's in 20ish minutes)
<dimitern> hazmat, can you do now? i invited you
<hazmat> dimitern, yup that works thanks
<hazmat> nice empty room :-)
<dimitern> hazmat, it's not on for 20 more minutes
<ahasenack> dimitern: turns out a bug about it already exists: https://bugs.launchpad.net/juju-core/+bug/1183571
<_mup_> Bug #1183571: A conflicting conditional operation... <cmdline> <hours> <juju-core:Triaged> <https://launchpad.net/bugs/1183571>
<dimitern> ahasenack, hmm it seems intermittent and region-specific
<rogpeppe1> hazmat: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=1
<rogpeppe1> fqw
<mramm> sinzui: any news on the possibility of 1.17.2 release with the lxc bridge fix and the MAAS metadata fix?
<sinzui> mramm, I selected r2285 this hour and am writing the release notes
<sinzui> mramm, Azure is very brittle and I may note that in the note for the release
<sinzui> If r2286 passes this hour, I will use it instead
<mramm> nice
<mramm> thanks
<mramm> sinzui: you rock
<TheMue> sinzui: he, found a solution for the local debug-log
<TheMue> sinzui: s/he/ha
<TheMue> sinzui: if all-machines.log now would contain data too it would be fine
<dimitern> fwereade, are any of metadata.yaml, config.yaml and revision optional for a charm?
<sinzui> TheMue, https://bugs.launchpad.net/juju-core/+bug/1270671 says it is fix committed
<_mup_> Bug #1270671: local provider all-machines.log is empty  <juju-core:Fix Committed by thumper> <https://launchpad.net/bugs/1270671>
<fwereade> dimitern, pretty sure config is optional
<sinzui> TheMue, It is in today's release
<fwereade> dimitern, and in theory revision is, sort of
<TheMue> sinzui: fantastic
<dimitern> fwereade, ok, so only metadata is mandatory
<fwereade> dimitern, although that's something we should fix as it comes in, and Ithink the charm package handles that anyway
<fwereade> dimitern, yeah, missing config is fine, missing revision is sorta fine but we should fix it aswe repack
<dimitern> fwereade, ok
<TheMue> sinzui: hmm, it changed into a symlink in /var/log/..., but is still empty *sigh*
<TheMue> sinzui: just merged trunk
<sinzui> :(
<TheMue> sinzui: oh, stop, strange, it's only missing machine-0. machine-1 and unit-... are now added
<sinzui> That is odd
<TheMue> sinzui: and my debug-log works locally *yippieh*
<dimitern> rogpeppe1, btw I'm still getting godeps: cannot parse "dependencies.tsv": cannot parse "github.com/loggo/loggo\tgit\t89458b4dc99692bc24efe9c2252d7587f8dc247b\t": unknown VCS kind "git" after pulling the latest godeps
<rogpeppe1> dimitern: go get -u launchpad.net/godeps
<dimitern> rogpeppe1, I did
<dimitern> rogpeppe1, and then I went in $GOPATH/src/...godeps and did bzr pull - same issue
<dimitern> (it said no revisions to pull)
<rogpeppe1> dimitern: what revno do you get in the godeps dir?
<rogpeppe1> dimitern: try rm -r the godeps dir, then go get it again
<dimitern> rogpeppe1, rev 10
<dimitern> rogpeppe1, ha! it appears to have moved from ~rogpeppe/godeps/trunk
<rogpeppe1> dimitern: ah
<rogpeppe1> dimitern: rev 14 is the latest
<dimitern> rogpeppe1, yep, now it's fine, thanks
<rogpeppe1> dimitern: cool
<sinzui> rogpeppe1, dimitern Do ewither of you have time to review this branch because I am releasing 1.17.2 https://codereview.appspot.com/59050044
<rogpeppe1> sinzui: looking
<rogpeppe1> sinzui: LGTM
<sinzui> thank you rogpeppe1
 * fwereade needs to be out again for a bit, will be around again most of the evening
<rogpeppe1> TheMue: you have another review
<TheMue> rogpeppe1: yep, thx, just answering
<gary_poster> jamespage, hi.  do you still think you'll have time for quickstart packaging?  if you are going to capetown I suspect that might take some of your time
 * gary_poster stepping out for lunch
<dimitern> TheMue, you have yet another review
<jamespage> gary_poster, just did the packaging work and uploaded to trusty for archive admin review
<TheMue> dimitern, rogpeppe1: thanks for your reviews. I added some comments and one for the whole stuff. will continue on monday.
<dimitern> TheMue, thanks
<gary_poster> awesome, thanks jamespage!
<hatch> hey why does -e not work in destroy-environment but it's required everywhere else?
<hatch> I'm trying to figure out the proper message to tell users of quickstart
<hatch> but now the -e is also dependent on the version too?
 * hatch hopes it's a bug and he can leave the -e in there
<hazmat> hatch, it used to be.. it was abruptly changed without notice.. its been restored on trunk w/ a deprecation warning
<hazmat> 1.17.2 should have it
<hatch> hazmat so now everything else will not have -e either eventually?
<hazmat> hatch, just destroy-env for hysterical raisons
<hazmat> its so people don't accidentally destroy something
<hazmat> because env to act upon  can be looked in one of four ways
<hatch> ok thanks for clarifying
<natefinch> hazmat, hatch: that was me, sorry.  I wanted the environment name to be mandatory, to make it less likely you'd accidentally destroy-environment on a production environment... however, I didn't think to make -e optional, and should have.
 * hatch shakes finger at natefinch 
<hatch> :P
<natefinch> :D
<natefinch> really, I just wanted to see if anyone was paying attention ;)
<hatch> it's fine either way really, I just need to know what to tell the user in quickstart
<hatch> haha
<hazmat> .
#juju-dev 2015-01-26
<mattyw> morning folks
<mattyw> juju status
<mattyw> hmmm
<mattyw> anyone seen this before? http://paste.ubuntu.com/9875907/
<axw> mattyw: looks like the critical bug in the channel subject
<mattyw> axw, sorry - yeah just saw, looks like the fix is having trouble landing as well
<mattyw> axw, https://github.com/juju/juju/pull/1480
<axw> mattyw: yeah, not sure what that is all about
<mattyw> axw, I've managed to recover lxc - which is what I needed for now
<jam1> just checking if this message gets through. I'm trying to set up an IRC proxy because my normal connection is being terrible now.
<jam> menn0: ping (just in case you are on, I'm just checking that everything is working)
<menn0> jam: pong
<jam> thanks menn0
<menn0> jam: np
<anastasiamac> dimitern: thnx u for committing the fix for blocker :D do we have an eta for a free for all CI?
<dimitern> anastasiamac, well it's up to the CI jobs passing cleanly first IIRC
<dimitern> anastasiamac, but I'll follow closely those jobs and make sure to unblock it ASAP
 * anastasiamac signs and waitd
<anastasiamac> dimitern: thnx for keeping an eye out
<anastasiamac> dimitern: it's the end of public holiday here
<anastasiamac> dimitern: and one of my son's is starting another school year tomorrow
<anastasiamac> dimitern: so m on and off
<anastasiamac> dimitern: but really want to land couple of things :D
<anastasiamac> dimitern: so i'll keep my eyes open for a while :
<dimitern> anastasiamac, sure, I know how frustrating it can be to try to land stuff with a block on :/
<axw> dimitern: what caused that weird error about extant dir?
<dimitern> axw, no idea - but I suspect it's a lander script issue
<axw> dimitern: so you just retried and it worked?
<anastasiamac> dimitern: m not frustrated... yet :D
<anastasiamac> dimitern: ask me in a few months time...
<dimitern> axw, retrying didn't help yesterday :) so I suppose somebody fixed the lander between yesterday and today
<dimitern> anastasiamac, :)
<axw> righto
<dimitern> mgz, hey, are you around?
<dimitern> it seems publish-revision job fails with a surprising error - cp: cannot stat `/var/lib/jenkins/streams/juju-dist/devel/tools/releases/juju-*.tgz': No such file or directory
<perrito666> morning
<mgz> dimitern: hey, checking it out
<dimitern> mgz, thanks!
<dimitern> mgz, any update on the publish-revision job failing?
<mgz> dimitern: we haven't changed the code or the job as far as I see, looking at the machine currently
<dimitern> mgz, ok, cheers
<dimitern> mgz, it seems to be looking for a devel tarball, but looking at the whole log I can't see any getting generated - can this be related with 1.21's release?
<mgz> everything pulled down seems to be in testing, not devel
<mgz> it's seems possible a release process thing has left the script broken, but I'm failing to see how currently
<dimitern> mgz, what's the difference between testing and develop?
<mgz> actually, maybe it is a change curtis made to the assemble streams script over the weekend
<dimitern> that's more likely :)
<mgz> I'm a bit loath to just revert and see as I'm not sure if other changes were made elsewhere
<mgz> but can probably fix
<rogpeppe> anyone know about juju-core plans for cli 2.0 ?
<rogpeppe> dimitern: ^
<rogpeppe> fwereade: ^
<dimitern> rogpeppe, I think thumper is in charge of this IIRC
<rogpeppe> dimitern: yeah, but i'm never in at the same time as him unfortunately
<dimitern> rogpeppe, o'rly? :)
<dimitern> well, you can always send him a mail I guess
<rogpeppe> dimitern: yeah. i was hoping for a very quick clarification though.
<dimitern> rogpeppe, fwereade might know more; sorry I can't help you there :/
<rogpeppe> dimitern: np, thanks!
<mgz> dimitern: trying a rebuild with older code.
 * dimitern crosses fingers and hopes for the sweet unblock
<sinzui> dimitern, we are waiting for ci to bless the build before letting developers merge. Ci is actually broken at the moment so the QA team are meeting to fix the issue
<dimitern> thanks sinzui
<fwereade> rogpeppe, I don't have anything newer than ~brussels there I'm afraid
<rogpeppe> fwereade: ok, thanks
<rogpeppe> fwereade: just wanted to confirm that we weren't planning on doing "juju <id> operation"
<fwereade> rogpeppe, confirmed :)
<rogpeppe> fwereade: thanks :)
<axw> rogpeppe: "juju add-machine" is now "juju machine add" with an alias. new commands follow that style, but I don't think any of the other existing commands have been updated
<rogpeppe> axw: ok cool
<perrito666> ericsnow: wwitzel3 brt
<cherylj> Is there someone around who can help me with a merge question?
<perrito666> cherylj: i might and also congratulations
<cherylj> thanks, perrito666 :)
<cherylj> I submitted a merge request, and got this error:  Build failed: Does not match ['fixes-1414016']
<cherylj> perrito666: do I need to rebase?
<perrito666> cherylj: does your pr fix that bug?
<cherylj> no
<perrito666> cherylj: then you cannot merge :)
<perrito666> you see, whenever there is a critical bug/regression CI becomes blocked so we dont pile on top of already existing brokenness
<cherylj> aahhh
<perrito666> so until someone submits the fix to said bug[s] nothing goes in
<cherylj> perrito666, that would explain it :)
<perrito666> I believe dimitern was trying to fix that
<cherylj> perrito666, thanks!
<perrito666> when you do submit that fix, instead of merge you use $$fixes-1414016$$ instead of whatever else inside the $ signs
<perrito666> cherylj: there is abit more to it but I am sure in the induction sprint someone will explain the whole process
<cherylj> perrito666, good, because being new to git, this is all voodoo to me :)
<perrito666> heh, that is all our own doing, for once git is not the culprit
<dimitern> perrito666, the fix *is* committed, but CI is broken right now apparently
<aznashwan> ericsnow: you there?
<ericsnow> yeah
<dimitern> voidspace, https://github.com/dimitern/juju/tree/wip-ec2-addressable-containers
<aznashwan> so I updated the PR but there's some things left hanging I want to get back on
<aznashwan> ericsnow: http://reviews.vapour.ws/r/671/
<ericsnow> aznashwan: yeah, saw that :)
<aznashwan> ericsnow: got a minute?
<ericsnow> aznashwan: OTP
<ericsnow> aznashwan: I'll ping you when I'm done
<aznashwan> ericsnow: sadly I have to log off right now, be back in a couple of hours in the vain hope you'll be here :D
<ericsnow> aznashwan: I'll be here
<ericsnow> aznashwan: sorry about missing you :(
<voidspace> dimitern: https://github.com/voidspace/juju/compare/wip-networkinterfaces-ec2
<mgz> dimitern: I'm curious, your branch to fix bug 1414016 did you actually change anything for the build dep error on merging or did it just work in the end?
<mup> Bug #1414016: Local-provider lxc Failed to create lxc_container <ci> <local-provider> <lxc> <regression> <juju-core:Fix Committed by dimitern> <https://launchpad.net/bugs/1414016>
<dimitern> mgz, no changes to deps - just modified how we handle lxc config on createContainer / clone
<mgz> I'm wondering what changed between when the merge job rejected it and when it landed, Ian asked Curtis about it
<dimitern> mgz,that *indeed* is curious :)
<mgz> (I have a change to the script I wanted anyway which I'll land shortly)
<mgz> just not sure exactly what triggered the error if you don't either :)
<dimitern> mgz, it's not something I did though - could it be one of the deps getting updated
<dimitern> the actual subrepos, not the .tsv file
<mgz> that does sound possible, a rev in another dep - but I don't see it from the go get log clearly
<mgz> anyway, will fix (I'll add you to the mp for your edification :)
<jam> heya mgz, how goes? haven't said hi in a while
<mgz> hey jam! I've seen the good news
<perrito666> hi jamcongrats
<jam> thanks perrito666 and mgz
<perrito666> now imagine thatif I had pressed the space bar
<jam> perrito666: I'll just go by jamcongrats :)
<mgz> you guys are sprinting currently?
<jam> mgz: I'm not, dimiter is with his group
<perrito666> jam: you are in dubai?
<jam> perrito666: yes
<perrito666> jam: isn't like the middle of the night for you?
<jam> its late in my evening (21:30), but this is after my son sleeps and I have a chance to do meetings with the US
<perrito666> you know since I work here i suck at calculating your current time :p
<mgz> ah, so you've already made dimiter take all the responsibility? :)
<jam> mgz: :)
<jam> perrito666: I find it much more useful to just go to UTC offsets
<jam> I'm +4
<perrito666> Im -3
<jam> its 2:30 for you and 9:30 for me
<perrito666> my first interview with you was at a time that was both inconvenient for you and for me because we both did that calculation wrong :p
<perrito666> we use utc+- tzs here its you guys that use the smaller only us set, which btw, I still dont know how to use
 * perrito666 adds us tzs to his clock
<perrito666> well, clock in vivid is broken
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs:  None
<rogpeppe> i've updated the gnuflag package to merge in changes from Go core, in case anyone fancies having a look: https://codereview.appspot.com/199070043/
<aznashwan> ericsnow: ping
<ericsnow> aznashwan: hi
<aznashwan> ericsnow: hey, sorry for the delay, you got a minute? :D
<ericsnow> aznashwan: sure
<aznashwan> ericsnow: so, about the PR, i don't know if you got round to looking through the changes, which, although very few, looked absolutely horrendous...
<ericsnow> aznashwan: haven't looked yet
<aznashwan> ericsnow: well, let me just say that the library was made for the low-level stuff the CoreOS guys are doing with it and was really ugly to use...
<ericsnow> aznashwan: that's a shame
<aznashwan> ericsnow: apart from that, everything looks decent, did a test run (with two echo lines, but still) and everything worked
<aznashwan> ericsnow: just need a go-ahead from you guys and i'll get to adapting the tests
<ericsnow> aznashwan: cool, I'll take a look at some point today
<hazmat> sounds like part of systemd assimilation
<aznashwan> ericsnow: also, fair warning, I could not figure out how to do get the enabled/disabled status for a service with the library, so our options are either to look for the symlinks in systemd's paths, or do an exec call for it (which is how I have left it for now)
<ericsnow> aznashwan: yeah, I was wondering about that
<aznashwan> ericsnow: you judge for yourself whether it's acceptable or not, and please let me know on the PR if there's any other issues
<ericsnow> aznashwan: will do
<aznashwan> much obliged :D
<thumper> o/
<natefinch> howdy thumper
<sinzui> thumper, can you or a member of your squad help triage bug 1413752 ?
<mup> Bug #1413752: Bootstrapping to Openstack Environment fails with "no instances found" <hyperscale> <openstack-provider> <juju-core:New> <https://launchpad.net/bugs/1413752>
<thumper> :-(
<thumper> goose collectInstances swallows errors from nova
<thumper> doesn't log nor return them
 * thumper files a bug
<thumper> oh... that isn't goose, that is juju
<thumper> sinzui: looked at that bug
<thumper> sinzui: and we just aren't logging enough information to work out what is going on
<sinzui> thumper, :(
<thumper> I've commented on the bug, and filed another
<thumper> in the openstack provider, where we are looking for instances
<thumper> we are ignoring the errors from nova
<thumper> so if there were issues, we can't tell what they are
<thumper> since the provider has obviously started a machine, and we are trying to bootstrap it
<thumper> there must be a problem with the call to nova to describe the instance
<thumper> but since we are throwing away the error...
<thumper> not much we can do
<sinzui> thumper, natefinch , do you have a minute to review http://reviews.vapour.ws/r/807/
<thumper> shipit
<sinzui> thank you thumper
#juju-dev 2015-01-27
<katco> wallyworld_: fyi: https://github.com/wallyworld/juju/pull/4
<wallyworld_> katco: ta, looking, was talking to mr horatio
<katco> wallyworld_: no rush
<wallyworld_> axw: also ^^^^^^
<axw> cool, will look after standup
<axw> wallyworld_: your latest push to storage-get appears to have lots of unrelated changes
<axw> ehh.. but it's submitted
<wallyworld_> axw: i rebased and pushed and now the diff shows all the commits prior
<wallyworld_> the commits it shows were those to master from the time i branched to now
<axw> wallyworld_: are you reusing that because you're doing a PR to the feature branch?
<wallyworld_> yeah, i wanted to commit to both master and feature branch. but i had to rebase because of the uniter facade changes
<wallyworld_> both master and featuure branch now have storage get
<axw> mk
<axw> cool
<wallyworld_> the diff is noisy, but the code is good
<wallyworld_> without the rebase, there were conflicts
<wallyworld_> and i wanted to resolves those now, early
<axw> wallyworld_: can you get quassel to highlight when there's unread notifications?
<axw> I mean, when you have quassel minimised
<wallyworld_> um
<wallyworld_> the little white arrow goes blue
<axw> xchat overlays on the icon with the number of unread notifications
<wallyworld_> on the unity launcher
<axw> huh, I don't see anything like that
<wallyworld_> i don't *think* quassel has unity integration ootb
<wallyworld_> so on my unity launcher icon
<wallyworld_> there's a little white triangle to the left, representing how many open instances there are
<wallyworld_> this is for any programme
<wallyworld_> for quassell, that triangle goes blue
<wallyworld_> when there's a notification
<axw> wallyworld_: can you please mention my nick in 30s?
<wallyworld_> maybe there's a unity plugin, not sure
<wallyworld_> sure
<wallyworld_> axw: you are awesome
<axw> wallyworld_: ;p
<axw> thanks
<wallyworld_> did it work?
<axw> it's very difficult to see the difference on my monitor :(
<wallyworld_> ah
<axw> teeny weeny little arrow
<wallyworld_> maybe you canuse unity tweak tool to change the colour
<axw> will take a look, thanks
<wallyworld_> i would also love better integration
<anastasiamac> axw: wallyworld_: u r both awesome :D
<axw> wallyworld_: https://github.com/wallyworld/juju/pull/7
<axw> wallyworld_: all tests pass locally
<wallyworld_> looking
<axw> wallyworld_: I think we should change storage-get to be a bit more like relation-get, and pass the storage instance ID in as a flag
<axw> wallyworld_: which defaults to the hook.Info's StorageId
<katco> axw: hey i have to get to bed pretty quick, but i wanted to get some clarification while you're on. got a sec?
<wallyworld_> axw: sure, sounds reasonable
<axw> katco: certainly
<katco> axw: ty :) if the devices aren't necessarily created in the same session, where should the volume id's be persisted?
<axw> katco: volume IDs will be stored in state. we need to map them back to the volumes without relying on in-process state. if you had to you could create a local file to persist state (since it's all bound to the one machine anyway), but I think we might be able to just get away with encoding information in the volume ID
<axw> katco: namely, the loop device name, /dev/loopN
<katco> axw: ah i see. is that considered a temp. work-around, or can that really supplant storage in state?
<katco> (i.e. what should i work towards?)
<wallyworld_> axw: the  WatchStorageInstances api can be moved to the v2 uniter facade
<wallyworld_> s/can/should
<axw> katco: whatever you can do to make it stateless (process-wise) is fine. I'd prefer if we encode info in the volume ID if possible
<wallyworld_> now that we have that facade
<axw> wallyworld_: can I just do that when it comes to merging with trunk? there's going to be lots of changes anyway
<katco> axw: okie doke, i'll try that. gives me something to go on for tomorrow
<axw> katco: cheers
<katco> axw: wallyworld_ anastasiamac goodnight tanzanite. have a great rest of the day
<axw> good night
<katco> and good night to everyone else as well :)
<anastasiamac> katco: u 2! c u soon :D
<wallyworld_> axw: + StorageIds set.Strings `yaml:"storage-ids,omitempty"`  <- we were going with a single storage id per hook right?
<wallyworld> axw: ah ffs, my connection dropped, did you see any of my stuff recently?
<axw> wallyworld: I saw your message about storage-ids
<axw> wallyworld: was about to reply :)
<wallyworld> ah good
<axw> wallyworld: that is state for recording which storage instances have changed; we cycle through them firing off storage-X hooks for them individually
<wallyworld> ah right ok
<wallyworld> my uniter foo is not very good
<wallyworld> axw: pr looks ok to me (but william should really comment). question save me checking code - is the storage watcher implementation based on our standard patterns for watchers used previously?
<axw> wallyworld: I think so? that's also going to change
<axw> wallyworld: it's currently watching storage instances, it should be watching attachments
<wallyworld> agreed
<axw> wallyworld: this is not meant to be quality, just enough to work
<wallyworld> yep :-)
<wallyworld> merge it i reckon
<axw> wallyworld: thanks. you'll need to, I don't have write access
<wallyworld> ah balls, i'll see if i can change that
<axw> wallyworld: it's your user, you probably don't want to do that
<wallyworld> well, i was going to trust you guys :-)
<wallyworld> i thouh i could enable write acess to a branch only
<axw> I wouldn't trust me not to break it :)
<wallyworld> but merged now anyway
<axw> thanks
<wallyworld> i almost expect breakages atm
<menn0> waigani: here's the apiserver change that is required avoid the panics I was seeing with multi-envs
<menn0> http://reviews.vapour.ws/r/808/
<waigani> menn0: swap you: http://reviews.vapour.ws/r/805/
<menn0> waigani: looking
<menn0> waigani: Ship It (just one non-issue comment)
<waigani> menn0: thanks, reading through yours. I haven't worked with connecting/closing an API connection before - so I may be learning more than reviewing
<menn0> waigani: leave it if you were planning on finishing up, or if the review is too much of a distraction from what you're supposed to be doing
<menn0> waigani: I'm about to finish up now anyway
<waigani> menn0: okay, can we talk over the presence watcher tomorrow?
<menn0> waigani: and Tim's best placed to review that PR. it's changing some stuff he implemented.
<menn0> waigani: sure
<waigani> menn0: I've started reading it, so I might just finish for my own benefit if no one elses
<menn0> waigani: k thanks
<menn0> waigani: well I'm off. see you tomorrow
<waigani> menn0: night
<wallyworld> axw: free for a chat?
<axw> wallyworld: sorry, eating
<axw> soon
<wallyworld> sure, no hurry
<axw> wallyworld: see you in 1:1
<anastasiamac> of course - feel very special :D
<wallyworld> lol, anastasiamac typed into the wrong window :-)
<axw> wallyworld_: so I'm quite close to getting the hooks working... then it just dawned on me that using the subordinate isn't going to work until we have dynamic storage working
<axw> wallyworld_: subordinates don't get deployed until after the machine is started
<wallyworld_> ah balls, yes
<wallyworld_> axw: i just put up a pr https://github.com/wallyworld/juju/pull/8
<wallyworld_> we'll have to think of a different way to demo
<axw> wallyworld_: looking
<axw> wallyworld_: yep. I'll take a look at the cassandra charm, it looked pretty easily to modify too
<wallyworld_> \o/
<axw> tho I think it might've wanted filesystems not block devices ..
<wallyworld_> hmmm
<wallyworld_> once this current work lands, we can hook up again to agree what needs doing next
<axw> wallyworld_: reviewed
<axw> you've been busy :)
<wallyworld_> axw: yeah, wish i was just focussed on storage
<wallyworld_> thanks fr review
<axw> wallyworld_: cassandra wants fs. I'm going to look how much is involved in changing ceph-osd, and how demonstrable it is
<wallyworld_> ok
<jam> fwereade: how goes?
<fwereade> jam, hmm, not bad, I'm a bit blocked on How To Do Something Right at the moment, but I'll get there :) how about you?
<jam> wallyworld_: are you seeing a lot of messages about be leaving and rejoining?
<jam> I set up a proxy server, and I'm trying to figure out if it is spamming you guys, or just me. (my internet seems to kill SSL connections every 5-10 minutes for some reason.)
<axw> jam: nope
<jam> axw: thanks. So the proxy is working as I had hoped.
<wallyworld_> jam: there's a kernel bug that keeps killing my network card :-(
<jam> wallyworld_: ouch
<wallyworld_> only solution is to reboot
<wallyworld_> happens randomly and sometimes a lot, sometime a little
<wallyworld_> atm, it's a lot :-(
<jam> apparently it is scaring waigani and dimitern as well :)
<dimitern> jam, wow what's that? :)
<jam> dimitern: just that alot of people are bouncing on IRC
<dimitern> jam, ah, well I haven't noticed affecting me as much
<dimitern> wallyworld_, there's no useful info about bug 1413910 btw
<mup> Bug #1413910: 1.21-rc1: Ambiguous resolution of private-address <network> <openstack-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1413910>
<wallyworld_> oh, that's sad. i didn't look too closely
<dimitern> wallyworld_, no logs etc. who should I ask for more details?
<dimitern> :)
<dimitern> jamespage, can you provide more info about that bug please? ^^
<wallyworld_> ask on the bug, or ping james directly
<dimitern> wallyworld_, cheers, sorry for the grumpiness - i've just had my coffee
<wallyworld_> dimitern: what? i didn't notice anything wrong :-)
<wallyworld_> i just appreciated that look looked at it
<dimitern> wallyworld_, :) ok, np - will try to figure out what's wrong
<wallyworld_> thanks :-) i can get involved a bit tomorrow or next day if needed
<dimitern> ok, cool
<wallyworld_> axw: i updated the pr to add pool validation in the constraints validation method, plus unified the pool creation
<wallyworld_> also unexposed the config translation
<axw> wallyworld_: cool, thanks
<jamespage> dimitern, sure
<jamespage> I can repro that easily
<jamespage> dimitern, in the specific case, the instance has multiple network interfaces onto the same openstack network
<jamespage> dimitern, juju appears to switch which ip address is the one associated with 'unit-get private-address'
<jamespage> dimitern, ok - so how do I get a 1.21 environment running now?
<jamespage> I can do 1.20 and 1.22-beta1
<wallyworld_> jamespage: 1.21 should be in trusty proposed
<wallyworld_> https://launchpad.net/~juju/+archive/proposed
<jamespage> wallyworld_, ta
<dimitern> jamespage, thanks for the update, however I'd appreciate to look at some logs if possible
<jamespage> dimitern, working on that for you now
<axw> wallyworld_: cool, thanks
<axw> oops
<axw> ignore :)
<wallyworld_> ok
<axw> wallyworld_: I've had to fix a bunch of stuff to do with the uniter, get the Location set on storage instances, and have made some more tweaks to storage-get
<axw> slow going, but nearly there I think
<wallyworld_> great, i can review when done
<wallyworld_> i was pondering the dirty hack to get loop devices provisioned
<jamespage> dimitern, now I can't reproduce...
<jamespage> gah
<wallyworld_> axw: the spec says each env provider has a default block device source, and cites EBS volumes for AWS. that still needs to be done. and even if there's a default block device source registered, if that fails to provision, i wonder if loop on rootfs should be used in an emergency
<axw> wallyworld_: I'd rather defer any plans around fallback, it gets messy very quickly
<axw> we should be retrying if it fails
<wallyworld_> ok, but i can reasonably implement default block device sources
<axw> wallyworld_: yes, that would be great
<wallyworld_> so EBS for AWS, loop for local
<wallyworld_> that will do for now for the demo
<axw> wallyworld_: yup. magnetic ebs specifically
<wallyworld_> yep
<wallyworld_> well, the "ebs" pool doesn't specify volume type
<wallyworld_> "ebs-ssd" does
<wallyworld_> perhaps i should add explicit volume type
<wallyworld_> to the "ebs" pool
<axw> wallyworld_: maybe, though the default is magnetic if unspecified.
<wallyworld_> that will do
<dimitern> jamespage, ok, please let me know if you manage to repro it
<jamespage> dimitern, marked the bug invalid - I've tried three different ways with no luck
<dimitern> jamespage, cheers!
<wallyworld_> axw: here's the default pool implementation https://github.com/wallyworld/juju/pull/9
<voidspace> dimitern: https://github.com/dimitern/juju/pull/6
<axw> wallyworld_: having dinner, will bbl to take a look
<wallyworld_> axw: no hurry at all
<wallyworld_> just letting you know
<axw> wallyworld_: LGTM
<wallyworld_> ty, will look
<axw> wallyworld_: I hit a snag with Ceph earlier :/  it wants me to reboot the server before the disks can be used
<axw> which I don't think will go down well with EC2
<wallyworld_> oh joy
<wallyworld_> sounds pretty dumb
<axw> wallyworld_: I haven't finished looking into it, it may be something I can work around
<axw> wallyworld_: otherwise I can try to hack the FS mounting code tomorrow
<wallyworld_> let's hope so
<axw> so we can use Cassandra or something else
<wallyworld_> FS would be great, as then we could look at the transcode charm
<wallyworld_> well, shared is needed
<axw> no, that needs shared...
<wallyworld_> yeah
<axw> shared needs a model change
<wallyworld_> might be able to hack something up though
<axw> maybe, I'll ponder
<wallyworld_> even if it's not properly modelled as shared and the charms are just given a different mount point
<wallyworld_> anyways, got to get the basics working first
<perrito666> morning
<anastasiamac> perrito666: o/
<hazmat> any mess folks around?
<hazmat> looks like its still pretty raw.. getting panics calling out to create
 * hazmat moves on
<hazmat> are people still working on the action specs?
 * hazmat scans prs and finds one
<rick_h_> hazmat: they've been landing/etc and have a demo charm setup https://github.com/binary132/actions-demo
<hazmat> rick_h_, cool, just concerned with the api oddities on it
<rick_h_> hazmat: yea, raise hell now as it's in feature frozen 1.22 I believe that had a beta go out today? /me had too much email today
<hazmat> http://pastebin.ubuntu.com/9900769/
<hazmat> type: object .. wierd nesting of parameters
<ahasenack> hi githubbers, is it possible to attach a file to a github issue? Or just images?
<rick_h_> ahasenack: drag/drop it and see? I think it does but can't recall
<ahasenack> rick_h_: " Unfortunately, we don't support that file type. Try again with a PNG, GIF, or JPG. "
<hazmat> ahasenack, not really. images.. or typically link to gist link for text
<rick_h_> ahasenack: :(
<ahasenack> hazmat: thanks
<rogpeppe> i'd appreciate it if someone could review this godeps change (and preferably QA it), because it runs some risk of killing all the CI bots... https://codereview.appspot.com/197110043
<rogpeppe> wallyworld_: perhaps you might want to take look?
<rogpeppe> sinzui, mgz: you also, i suspect, use godeps a fair amount. i'd be interested in your take on it
<sinzui> rogpeppe, interesting. I just wrote something like that for testing
<rogpeppe> sinzui: i've been looking for some solution in this space for a while, as we've got a bunch of projects with different dependencies files
<rogpeppe> sinzui: please have a play with it and see if it this feature can meet your needs in this area
<sinzui> your rules look similar to mine. including the warning on conflict
<rogpeppe> sinzui: that's a good sign :)
<rogpeppe> sinzui: i'm considering a rule (perhaps optionally enabled) to consider also the version of the project in which each dependency file lives as a dependency
<rogpeppe> sinzui: i *think* that might make intuitive sense
<sinzui> rogpeppe, yes, that is challenging. I was thinking the same to ensure we aren't errantly mixing versions
<sinzui> I am writing up my use case in my thanks for your changes.
<rogpeppe> sinzui: the way i was considering doing it was to implicitly insert a dependencies.tsv file after each one explicitly specified, holding the project info for the previous dependencies.tsv
<rogpeppe> sinzui: that sounds useful, thanks
<jw4> hazmat: about actions API - can you tell us more about your concerns?
<jw4> hazmat: I know it looks ugly, but what you've pasted is an example of an action specification, which is essentially arbitrary - at the charmers discretion
<hazmat> jw4, the presentation is completely inane and that's not charmers, thats the api
<jw4> hazmat: don't beat around the bush; how do you really feel?
<jw4> hazmat: ;)
<natefinch> perrito666: you have an OSX machine, right?
<hazmat> jw4, why are properties nested under params.. everything else next to properties is effectively api garbage.
<hazmat> jw4, at a sibling level.. type: object, description duplicated, title, etc.
<jw4> hazmat: what that result is depicting is a specification of the kinds of actions and associated parameters that the charm can perform
<hazmat> jw4, i know i'm in good company ;-)
<hazmat> jw4, i know what its describing
<hazmat> jw4, but the structure of that output is nonsensical
<hazmat> jw4, see how under ie. drop params, and bring properties up a level
<jw4> hazmat: I see - I would love some help on improving it
<hazmat> jw4, everything else there under params besides properties is basically a leaky implementation or duplicate info
<perrito666> natefinch: I sort of do, my wife has
<hazmat> jw4, also default around most apis is capital 'Results'
<hazmat> there's massive api consistency.. but fixing each bit helps
<jw4> hazmat: sure.. I know this is a bit of a big ask, but would you be willing to mock up an example of how you think it should look?
<jw4> hazmat: I know we've already simplified bits of this by getting rid of duplicate or redundant info
<jw4> hazmat: but an example of what you'd like to see would help if you have a bit of time
<natefinch> perrito666: ok, no big deal. There's a bug about OSX, but it honestly probably doesn't require OSX to fix....  https://bugs.launchpad.net/juju-core/1.21/+bug/1414800
<mup> Bug #1414800: OSX versionpanic: osVersion reported an error: unknown series <osx> <regression> <juju-core 1.21:Triaged> <https://launchpad.net/bugs/1414800>
<jw4> hazmat: fwiw, the actual spec is pretty light on these kinds of details
<hazmat> jw4, re example
<hazmat> jw4, drop the existing params, bring propeties up and rename as params or config.
<jw4> hazmat: +1 (I think we did that elsewhere already)
<hazmat> jw4, couldn't say i'm just running trunk from today.
<rogpeppe> sinzui: i'm done for the day now. thanks in advance for your comments on the review.
<jw4> hazmat: when you mentioned capitalized Results are you referring to: {u'results': [  becoming {u'Results': [
<hazmat> jw4, yes .. that's more consistent with the rest of the api if its capitalized 'Results'
<jw4> hazmat: +1
<jw4> hazmat: I'll add a card to our kanban to fix this and bring it up with our team
<hazmat> jw4, cool, thanks
<jw4> hazmat: I appreciate the feedback - please feel free to bring up anything else that looks inane :)
<hazmat> jw4, np, ... oh.. more thing.. client watch for actions completions.. polling is a bit silly, but that's additive not transformative.
<hazmat> ideally just injected into allwatcher which is what most clients use
<jw4> hazmat: I see - we do have a card outstanding for updating the allwatcher
<jw4> hazmat: thanks
<hazmat> jw4, thank you.. cheers
<jw4> hazmat: cheers
<bac> hi sinzui
<sinzui> hi back
<natefinch> ok, well, time to go rake the roof while it's still light out.
<natefinch> backport of a fix for yosemite to 1.21 is here if anyone has time: http://reviews.vapour.ws/r/809/
<natefinch> anastasiamac: ^^
<natefinch> (super simple, luckily)
<perrito666> did he say rake the roof?
<jw4> perrito666: lol
<natefinch> heh... when you get 3 feet of snow, it's best to get some of it off the roof (just about to go do it).  There's a thing called a roof rake for this job... probably not necessary in your neck of the woods, perrito666 ;)  Looks like this: http://www.homedepot.com/catalog/productImages/1000/66/6657ac9f-d400-4469-9f64-1d5957d8b873_1000.jpg
<natefinch> ok, going now :)
<perrito666> natefinch: nope, half of the roofs here are not even inclined
<perrito666> :p
<perrito666> I am a bit surprised there is no electrical gizmo to prevent that
<jw4> anastasiamac: I noticed your 'laconic' comment yesterday, but didn't realize it was on my PR :)
<jw4> anastasiamac: thanks for the review; I'll update
<sinzui> natefinch, I commented on your PR. I am not a reviewer so I don't mind if you ignore me
<jw4> OCR PTAL : http://reviews.vapour.ws/r/801/
<jw4> ^^ Add 'running' status and start time to Actions
<bodie_> hazmat, not sure if you got an explanation, but the top-level values under the 'actions' key (Params and Description) are keys for Juju's usage, belonging to Juju's action schema type; the values under "Params" are a one-for-one JSON-Schema mapping which is used by the schema lib verbatim to enforce param typing
<bodie_> basically, the JSON-Schema may be used by the frontend, and Go clients, the worker, etc, can use the Juju type's values without having to dig into the JSON-Schema or check whether it has nested keys present, etc
<hazmat> bodie_, ah.. that helps explain type: object syntax, though not the redundancy exposed to users. Params would logically correspond to the params defined in actions.yaml..
<hazmat> except they don't
<bodie_> right; that's a request from fwereade due to the fact that requiring it to be verbatim JSON-Schema causes the simplest (single depth) schemas to be overly complicated to write
<bodie_> the transform is specified on the docs site.  I'm not a huge fan, but it does make the writing of simple actions cleaner
<bodie_> https://juju.ubuntu.com/docs/authors-charm-actions.html#params-transformation
<hazmat> bodie_, while jsonschema has some value (i'd love vocabs etc in charms).. as a one off its not clear because its different then everything else that defines configuration schemas in juju atm.
<hazmat> bodie_, interesting so actions allow for complex value outputs?
<jw4> hazmat: fwiw, the requirement for using jsonschema was from sabdfl directly
<bodie_> yeah.... I think we were hoping to roll more things (config at least) over to use the same technique, but we definitely want refinement and clarification brought to bear on it
<hazmat> jw4, i recall
<hazmat> jw4, well now i do.. looking at the api output.. i was .. well unclear would be nice ;-)
<hazmat> bodie_, thanks
<bodie_> I think the ideal way to handle it in clients is actually to use the service JSON-Schema to define the UI
<hazmat> rick_h_, ^ this on your radar?
<rick_h_> hazmat: reading backlog, otp sec
<bodie_> the other keys (Description at the moment) could be fleshed out to include other things, like revision, or whatever might be needed by Juju specifically
<bodie_> not sure if that helps clarify intent
<bodie_> and yeah, not totally certain those need to be exposed to the user, the redundancy is confusing
<hazmat> bodie_, it does.. but it only really works.. if the uis  do adopt it on top of maintaining their existing interfaces ontop of config schemas, else its unclear
<jw4> hazmat: yeah - the UI team has been somewhat involved and should be acquainted with the plans to use the JSON-Schema to build the UI
<jw4> hazmat: we'll see if rick_h_ concurs ;-p
<bodie_> there's a JS lib that makes UIs out of JSON-Schemas
<bodie_> s/UIs/UI elements/
<rick_h_> jw4: bodie_ hazmat we had instruction to use https://github.com/jdorn/json-editor to build the actions UX from json-schema per sabdfl suggestion which we thought was good
<rick_h_> anything that could be expressed in jsonschema we could build a UI around
<hazmat> rick_h_, did the subject come of up of retrofitting charm and env config with the same?
<rick_h_> and it's a push for a standards based approach to data types across juju with the spearhead being actions as greenfield work
<rick_h_> yes
<hazmat> cool, yeah.. this was capetown last year..
<rick_h_> the goal was to test and push through with actions with juju charm config being updated as a follow up
<rick_h_> yep
<rick_h_> it's been a requirement from the start and we've been down this road a few times of trying to ditch it, but it fails to take the longer term idea into practice
<rick_h_> if it's not there, it needs super reasons and sabdfl sign-off imo
<anastasiamac> jw4: natefinch: u've pingd me at 5am. still 7am here -m taking kids to school. will look l8r :)
<jw4> anastasiamac: nothing important - cheers
<anastasiamac> jw4: thnx :D
<bodie_> hazmat, just to make sure you saw, the basic transform here might help make more sense of what happens to actions.yaml
<bodie_> hazmat, https://juju.ubuntu.com/docs/authors-charm-actions.html#simple-yaml-transform-example
<hazmat> bodie_, noted.. although the other confusing bit is renaming parameters to properties and reusing params for something else.. maybe rename the top level params to schema
<bodie_> hazmat, yeah, I think that would likely more sense.  TLDR: use the value of the "Params" key to interact with libs that use json-schema.
<hazmat> bodie_, ie the top level object under params is actually not a param its the action itself
<hazmat> http://pastebin.ubuntu.com/9900769/
<bodie_> brb a sec
<hazmat> just a repost of previous for context
<hazmat> yeah.. meetings over here.
<natefinch> perrito666, wwitzel3, ericsnow: any of you guys have nothing to do tonight and happen to be free at 8:30pm eastern?  There's a meeting with cloudbase, totally not mandatory, but if you happened to be available, it would be cool to have someone there, otherwise alexis will do it on her own (which she said is fine).
<ericsnow> natefinch: unfortunately not
<natefinch> ericsnow: no problem... I wouldn't expect you to be free at 6:30 your time :)
<natefinch> ericsnow: I'm skipping out too, just figured I'd ask in case someone had a boring night in planned... pretty much never applicable for homes with kids.
<natefinch> perrito666: not the best picture, but here's the results of me "raking the roof": https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-xpa1/v/t35.0-12/10951050_10152991996797348_1941075273_o.jpg?oh=820150bca86087f75043727217b74699&oe=54CA916F&__gda__=1422491522_a043b88ea0ceb52a051d1f3f2aa89eba
<menn0> thumper: quick review please: http://reviews.vapour.ws/r/808/
 * thumper looks
<menn0> this is the apiserver state cleanup change we discussed yesterday
<menn0> thumper: I just added some tests at anastasiamac's suggestion
<thumper> ok
<menn0> thumper: the thing that really slowed me down is that I didn't realise that sometimes the server is serving an apiRoot and other times it is serving an apiHandler. Kill and Clean had to be implemented on both.
<thumper> nit: Clean vs CleanUp ?
<thumper> menn0: thoughts?
<menn0> thumper: I was trying to stick with the Go convention of *er interfaces
<menn0> and Cleanuper sounds bad
<thumper> fine with Cleaner and CleanUp
<thumper> I don't like the 'er' always
<thumper> it doesn't always fit
<menn0> thumper: I thought you'd pick up on that :)
<thumper> :)
<menn0> thumper: I'll change it
<thumper> although I agree that Cleanuper is dumb
<menn0> thumper: CleanUp or Cleanup?
 * thumper thinks
<thumper> most our existing code uses Cleanup
<thumper> so probably go with consistency
<menn0> thumper: apparently "cleanup" is a word in American and Canadian English
<menn0> http://grammarist.com/usage/cleanup-clean-up/
<menn0> thumper: so I think it's fine
<menn0> I prefer Cleanup
<thumper> ok
<thumper> go with that
<thumper> love this: Australian and New Zealand publications are inconsistent on the matter.
<menn0> thumper: basically we can't decide who we're better friends with :)
<thumper> :)
<menn0> thumper: fixed. pushing and merging now.
<perrito666> well here I was mockig nate because he has to rake his roof and here I am now trying to stop wather from entering my kitchen
<perrito666> gotta love subtropical summer, the only thing stopping a heat wave is a flooding rain :p
<menn0> perrito666: ha :)
<wallyworld_> perrito666: has the rain washed away the present you were going to leave under my tree?
<perrito666> wallyworld_: nope, but still working on it, albeit interrupted by things like a new hole I just saw in my roof
<perrito666> brb
<wallyworld_> oh no
<perrito666> back, hole fixed by correct placement of kitchen hardware
<wallyworld_> lol
<perrito666> man i really want to fix that but i need to wait for the summer to end
<wallyworld_> buy a bigger bucket to put under it
<perrito666> wallyworld_: I think I want to replace the house instead
<perrito666> :p
<perrito666> I need to put epoxi paint on the roof, but that requires a given amount of days without rain, which cannot be granted in summer
<wallyworld_> always an issue
<perrito666> wallyworld_: well, I will eventually fix it summer gotta end eventually
<wallyworld_> unless global warming changes things
<perrito666> wallyworld_: oh, its changing things, but that does not include rain cycle so far
<perrito666> I wish there was a way to emulate rains on my roof to discover these issues in winter
<wallyworld_> it's called a hose
<perrito666> wallyworld_: trust me, you need a bit more than a hose to make these rains up
<perrito666> :p
<wallyworld_> i figured, was being facetious :-)
<thumper> wwitzel3: ping
<anastasiamac> menn0: thnx for the test :D
<anastasiamac> menn0: the more the merrier
<menn0> anastasiamac: usually :)
#juju-dev 2015-01-28
<hazmat> thumper, what's the state of mess atm?
<thumper> hazmat: it's a mess
<hazmat> thumper, when i try to use the create api, i get panic and restarts.
<thumper> hazmat: working hard to get a demo...
<hazmat> complaining about juju home
<hazmat> thumper, cool, carry on :-) just curious
<thumper> huh... interesting
<menn0> gotta love tests that pass for the wrong reason
<wallyworld_> menn0: quick upgrades question. i understood that fromVersion would be set to 1.16 if unknown. hence for new installs, all steps should run. I just added a new 1.23 step, and when I bootstrap from master (1.23-alpha1), no upgrade steps run. am I missing something?
<menn0> wallyworld_: hmmm... I think that behaviour changed recently
<menn0> wallyworld_: because it was causing problems
<wallyworld_> :-(
<menn0> wallyworld_: why would you want to run upgrade steps on bootstrap?
<wallyworld_> sec
<menn0> wallyworld_: let me check ... I might be getting confused with something else I changed recently
<wallyworld_> menn0: well, i am adding new things for the 1.23 release. the release will eventually be installed on top of 1.22. but of course, for development, the version is set to 1.23-alpha1
<wallyworld_> i need to be able to run a fresh 1.23 install and have things done
<wallyworld_> the things that are needed are not there in 1.22 nor a fresh install
<wallyworld_> the old behaviour would have been perfect
<wallyworld_> and the code comments still imply the old behavbiour woeks
 * menn0 is looking
<menn0> wallyworld_: I'm pretty sure that i've dealt with at least 2 bug reports from users because we were running upgrade steps straight after bootstrap
<thumper> wallyworld_: having from version set to old when bootstrapping was causing problems for landscape
<thumper> wallyworld_: and we shouldn't ever need to run upgrade steps on bootstrap
<menn0> wallyworld_: it's unnecessary and confusing and breaks scripting, meaning that you can't rely on the env being usable after "juju bootstrap" returns
<wallyworld_> hmmm
<wallyworld_> well that's a change in semantic that caught me by surprise
<menn0> wallyworld_: to test what you're testing couldn't you bootstrap and then "upgrade-juju --upload-tools"?
<wallyworld_> what about new installs that need to do things
<menn0> wallyworld_: that will use a new version number and trigger the upgrades
<menn0> wallyworld_: what kind of things?
<wallyworld_> sure, but that doesn't account for new installs
<wallyworld_> adding default storage pools
<wallyworld_> that needs to be done for upgrades and new installs
<menn0> can't that be done when state is opened?
<menn0> as per mongodb indexes?
<wallyworld_> it's not really the same - i wouldn't want to add bespoke application functionality to opening a db connection
<wallyworld_> mongo indexes are a db artefact so they belong in state opening
<wallyworld_> i don't get why we didn't just make bootstrap wait
<wallyworld_> let it complete what it needed to do, and then return
<wallyworld_> no scripts broken
<wallyworld_> now, we can't add functionality required to initialise a given version of juju
<menn0> but running all upgrade steps at bootstrap is completely unnecessary. a bootstrapped env is by definition up to date.
<wallyworld_> the steps are idempotent and will just skip
<menn0> they don't "skip"
<wallyworld_> skip = check if already done and return early
<menn0> the will still do things (query collections etc)
<menn0> they just have no net effect, apart from taking up time
<wallyworld_> regardless, how do we now add things that need to be done for the current version
<wallyworld_> a small time
<wallyworld_> compared with the overall bootstrap time
<menn0> sure
<wallyworld_> what we've now lost is the ability to in one place define steps to be run to upgrade or initialise a new version
<wallyworld_> all for the cost of a minimal extra 2% of bootstrap time
<menn0> and also avoiding problems for users
<wallyworld_> but if bootstrap waits, there are no problems
<wallyworld_> instead, the development complexity has been increaed
<menn0> well if you want to change bootstrap...
<wallyworld_> as i now have to add code in multiple places
<wallyworld_> i wish the change were mode widely socialised before being implemented
<wallyworld_> so tese issues could have been raised
 * thumper calls timeout
<wallyworld_> i'll have to add a work around for now
<wallyworld_> doesn't thumper like a good robust dicussion?
<thumper> wallyworld_: sure, in a week when we have time
<wallyworld_> would be voring if we always agreed
<thumper> right now we are super busy
<wallyworld_> fair point. as am i trying to get stuff done which is now much more difficult, hence the grumpyness
<menn0> sorry that this has made it harder for you
<wallyworld_> no need to apologise, i was just a little surprised at the change and hence frustrated
<menn0> i didn't realise that upgrades steps were intended for more than upgrades, and we had problem reports from users relating to the old behaviour
<thumper> wallyworld_: what are you actually wanting to do?
<wallyworld_> add default storage pools
<thumper> menn0: the thing is: they shouldn't be
<wallyworld_> these need to be done when coming from older versions, as well as bootstrapping 1.23
<thumper> wallyworld_: which is what?
<wallyworld_> just adding records to a collection
<wallyworld_> so we want some out of the box data
<thumper> surely this isn't a large amount of work
<wallyworld_> well, i now have to do it in two places
<thumper> just one func called from an upgrade step and the bootstrap process
<thumper> but then it isn't surprising
<wallyworld_> sure, but it's now messy, i'd rather fix the upgrades
<thumper> having bootstrap config happen in an upgrade step is surprising
<thumper> upgrades aren't broken
<thumper> your thinking is
<wallyworld_> have to agree to disagree. the semantic has been in place for many versions
<thumper> but the semantics were wrong
<wallyworld_> we can discuss next week
<thumper> and have been fixed
<thumper> relying on broken semantics is broken design
<thumper> well, you can
<wallyworld_> what was wrong ws that bootstrap didn't wait
 * thumper won't be there
<thumper> no, I continue to say that bootstrap config in an upgrade step is broken design
<menn0> wallyworld_ and I are sharing a room so we will have plenty of time for debate :)
<thumper> sorry, but that is just bollocks
<perrito666> menn0: learn to sleep with one eye open
<wallyworld_> this is a good argument for down at the pub with beer
<thumper> agreed
<menn0> (or wine in your case?)
<wallyworld_> yes :-)
<wallyworld_> red
<menn0> wallyworld_: so working towards an immediate solution for you...
<menn0> wallyworld_: adding some rows to a collection sounds like state functionality
<wallyworld_> yeah, will hack something in
<menn0> wallyworld_: so why could those rows be added if required when state is opened?
<menn0> coudn't
<wallyworld_> opening state is akin to opening a db connection
<wallyworld_> it's down a layer or two from the application layer
<menn0> i kinda see what you're saying but state is also full of business logic. it's not just "the database".
<wallyworld_> what was nice about upgrade was that it would be done once - the upgradedFrom would be empty, then the steps would run, and then it wouldn't happen again
<wallyworld_> now i have to encode extra checks in say state server start up tp ensure it's only done once
<wallyworld_> our state package is bollocks
<menn0> wallyworld_: ok well lets see if we can work something out next week
<wallyworld_> sure
<wallyworld_> i can reply on upgrades to handle coming from < 1.23, but have to add some bespoke stuff to handle new installs
<wallyworld_> but it's bad because if i add it to bootstrap, how do i know if upgrade did it or now
<wallyworld_> not
<wallyworld_> so potentially need to check fromVersion and all that, just like upgrade does, so it gets messy fast
<thumper> menn0: I've hacked together a branch that does what we talked about yesterday
<thumper> menn0: I'm going to try to use it locally before I submit for review
 * thumper has a sinking feeling
<thumper> oh yeah... everything is still erroring with notimplemented
 * thumper enfixorates the providers we care about right now
<thumper> (four of them)
<thumper> dummy, local, ec2, maas
<thumper> dummy for tests
<thumper> local for testing
<thumper> ec2 for demo
<thumper> maas for mark
<thumper> menn0: did your branch land?
<menn0> thumper: sorry was having lunch
<thumper> menn0: np
<thumper> it isn't holding me up
<menn0> thumper: branch not landed yet. my changes exposed brokenness in the machine agent tests which i'm fixing.
<thumper> ok
<menn0> this one has been a hard slog
<thumper> ok
<perrito666> wallyworld: I just packed half your gift, still need to fix the facade to do the proper calls but I pushed the general idea for you to look
 * perrito666 sleep time
<perrito666> cheers all
<wallyworld> ty, will do
<wallyworld> hope the roof holds out
 * thumper crosses fingers
<thumper> stabby stabby
<thumper> fark!!!!
<thumper> (â¯Â°â¡Â°ï¼â¯ï¸µ â»ââ»
 * thumper looks upwards emploringly
<thumper> WHY?
<thumper> why can't json serialization have integers?
<wwitzel3> thumper: pong
<menn0> thumper: b/c Javascript doesn't have integers...
 * menn0 ducks
<thumper> wwitzel3: got time for a quick 5 min hangout?
<wwitzel3> yep
<menn0> wallyworld: ping
<wallyworld> menn0: hi
<menn0> wallyworld: do you know anything about MachineSuite.TestManageEnvironDoesNotRunFirewallerWhenModeIsNone?
<menn0> wallyworld: the logs indicate you might have had something to do with it
<wallyworld> axw: remind me, were we considering supporting zones with deploy
<axw> wallyworld: last time I spoke to fwereadeabout it, he wanted that to be on add-machine only, and use machine placement in deploy/add-unit
<axw> <axw> wallyworld: last time I spoke to fwereadeabout it, he wanted that to be on add-machine only, and use machine placement in deploy/add-unit
<wallyworld> axw: np, i'll need to add a new check analogous to isZoneConstrained() to handle VolumeTypeNotAvailableInZone
<wallyworld> menn0: sorry, connection bouncing, not sure if you saw my pong
 * thumper headdesks
<menn0> wallyworld: I replied but it must have been while your connection was down
<wallyworld> i hate this kernel bug
<menn0> I was asking if you knew about  MachineSuite.TestManageEnvironDoesNotRunFirewallerWhenModeIsNone. you seem to have had something to do with it according to the logs.
<menn0> wallyworld: ^^^
<wallyworld> hmmm, mayb, i didn't write it, may have modified it
<wallyworld> let me look
<menn0> wallyworld: it appears to be broken and passing for the wrong reason
<menn0> wallyworld: it's testing that the firewaller doesn't start when firewall-mode is "none"
<menn0> wallyworld: but never actually sets the firewall mode
<menn0> wallyworld: the mode defaults to "instance"
<menn0> wallyworld: I think the test only passes because it waits for ShortWait time which is rather short
<wallyworld> menn0: dimiter wrote it, i just tweaked the worker running
<menn0> wallyworld: it's just that the firewaller isn't getting a chance to start
<wallyworld> i don't know offhand much about it without looking at the code in more detail
<menn0> wallyworld: that's cool
<menn0> wallyworld: any ideas on how I can set the firewall-mode for the agent in the test?
<menn0> this is what i'm doing:
<menn0> 	m, agentConfig, _ := s.primeAgent(c, version.Current, state.JobManageEnviron)
<menn0> 	agentConfig.SetValue("firewall-mode", config.FwNone)
<menn0> 	c.Assert(agentConfig.Write(), jc.ErrorIsNil)
<menn0> 	a := s.newAgent(c, m)
<menn0> but the agent still ends up with "instance" as the mode instead of "none"
<menn0> if you don't know off hand don't go digging. I know you're busy.
<menn0> thumper: do you know? ^^^
<thumper> menn0: pretty sure we hard code that in the dummy provider
<menn0> thumper: it looks like it default to "instance" in the dummy provider but there are dummy provider tests which change it
<thumper> hmm..
<thumper> menn0: isn't the firewall-mode in the environ config not the agent config
<thumper> ?
<menn0> thumper: maybe that's what i'm doing wrong :)
 * thumper takes a deep breath
<wallyworld> axw: when you have a moment https://github.com/wallyworld/juju/pull/10
<axw> wallyworld: LGTM
<wallyworld> axw: question - when the attached hook runs, the storage id is passed in via env var right? i couldn't see it
<wallyworld> ty
<axw> wallyworld: no, no env var set atm
<wallyworld> ah, ok, cause i was going to try using the storage-get hook tool
<axw> wallyworld: it's set in the context
<wallyworld> the bash script doesn't get to see that though right?
<axw> wallyworld: you may want to wait for my changes, I'm just doing a test and I can probably propose something shortly
<wallyworld> sure, np
<wallyworld> but works so far
<wallyworld> the storage show is also light on details
<axw> wallyworld: when my change goes in, storage-get will default to the hook storage instance
<wallyworld> so we'll need to wire that up
<wallyworld> ah, yes, that's right
<wallyworld> remember now
<axw> wallyworld: whee, I just deployed postgresql with storage :)
<wallyworld> \o/
<wallyworld> axw: i agree the hack to work around the change in upgrade semantics is yucky
<wallyworld> but works for now
<wallyworld> axw: is that using a raw block device or mounting a fs?
<axw> wallyworld: the latter
<wallyworld> even better
<axw> wallyworld: diskformatter will now mount the fs after formatting, if it's not already done, and then record the mount point in state
<wallyworld> oarsum
<axw> then the unit agent sees the change and can get the location with storage-get
<wallyworld> whoot
<axw> wallyworld: I have zero tests, do you want me to propose it anyway?
<wallyworld> yes :-)
<wallyworld> tested live is good enough :-)
<axw> wallyworld: I'll just confirm that postgres is actually working :)  it has created the db dir in the mount
<wallyworld> axw: after landing, might be an ides to sync up to plan the next bit of work
<wallyworld> ok :-)
<wallyworld> now that we have both landed stuff today
<axw> wallyworld: yep, sure
<wallyworld> or soon will
 * wallyworld needs a little food
<axw> wallyworld: https://github.com/wallyworld/juju/pull/11  -- I'm going to get something to eat too. bbs
<wallyworld> sure
<axw> wallyworld: I need to fix a bunch of tests in that branch
<wallyworld> ok, i wouldn't worry about new ones, just make current ones pass
<thumper> menn0: if you are bored with your changes, there is this: http://reviews.vapour.ws/r/811/
<thumper> waigani, mattyw: or one of you on call reviewers could have a look
<mattyw> thumper, will do
<mattyw> thumper, give me 10 minutes or so and I'll take a look
<thumper> mattyw: no pressure, I'm pretty much done for the day
<waigani> I'm about to break but plan to come back later tonight, if it's still up I'll take a look then
<axw> wallyworld: I'll just test once more, then my branch hsould be good to merge
<wallyworld> \o/
<axw> wallyworld: also, I've pushed my postgresql to lp:~axwalk, and it can be deployed from cs:~axwalk/trusty/postgresql
<axw> wallyworld: this is what I had to change: http://bazaar.launchpad.net/~axwalk/charms/trusty/postgresql/trunk/revision/112
<wallyworld> looking
<wallyworld> jeez, not many chnges
<wallyworld> which is good
<axw> wallyworld: I think in reality we could make it much better, but atm the charm is (i.e. already was) hard coded to look in /srv/data
<axw> but this is the minimal change needed to work
<wallyworld> yep
<wallyworld> axw: i have a meeting at 4, quick catch up now?
<axw> wallyworld: yep, sure
<axw> see you in 1:1
<axw> wallyworld: my branch is good to merge
<wallyworld> merge away
<axw> wallyworld: you need to, please
<wallyworld> oh doh
<wallyworld> sorry
<wallyworld> and done
<axw> cheers
<axw> anastasiamac: I'm going to fill out the details in "juju storage show" - you're not working on this, right?
<anastasiamac> axw: no ;)
<anastasiamac> axw: thnx :D
<axw> anastasiamac: nps
<jam> fwereade: are you around ?
<fwereade> jam, sorry, not really, we're all sick here :/
<jam> fwereade: sorry to hear that, I've been wondering why I haven't really seen you around.
<fwereade> jam, that's just today
<fwereade> jam, the last few days I've been trying to integrate the new uniter ops in a way that doesn't hurt more than it helps
<jam> fwereade: I wanted to check in how Leader election stuff is going.
<fwereade> jam, it's depressing at the moment, because I keep hitting roadblocks that I have to fix before I can progress
<perrito666> morning
<dimitern> perrito666, o/
<axw> wallyworld, anastasiamac: FYI, I've added "storage list" and modified "storage show" -- https://github.com/wallyworld/juju/pull/12
<jam>  mgz: poke
<jam> or maybe even dimitern
<dimitern> jam, yep, what's up?
<jam> dimitern: you saw the mailing list question about Rackspace, right?
<dimitern> jam, i did
<jam> I haven't personaly investigated their API, but I thought that is what we worked out
<jam> that Juju doesn't work there
<jam> I just wanted to check with other people to confirm it
<dimitern> jam, haven't read it thoroughly though
<dimitern> jam, well IIRC rackspace api does not support security groups the way juju uses them
<dimitern> it supports an older api, not the os-security-groups extension
<jam> dimitern: sure. I thought there were other limitations, as certainly security groups can be reasonably worked around
<dimitern> jam, well, looking in one of my earlier networking specs - part 1, last time I checked "Rackspace supports older/custom compute extensions (os-networksv2 and os-virtual-interfacesv2)"
<dimitern> jam, however now it seems it does support newer apis - but I need to investigate what and how
<jam> dimitern: k. I'd like to make sure I'm not talking garbage since there is interest there
<dimitern> jam, how critical is this to investigate now?
<natefinch> jam: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1
<jam> natefinch: it thinks its trying to connect
<voidspace> jam: ping
<voidspace> jam: are you able to add jamestunnicliffe (and dimitern ) to the canonical ec2 account?
<jam> natefinch: I'm back
<jam> voidspace: /wave
<natefinch> jam: I'm in moonstone
<jam> natefinch: I can
<jam> jamestunnicliffe: are you around now?
<jamestunnicliffe> yes
<jamestunnicliffe> jam: yes
<jam> jamestunnicliffe: do you have a preferred username ?
<jamestunnicliffe> jam: dooferlad
<voidspace> jam: thanks
<jam> voidspace: happy to. I should really coach someone else on how to do it. Reduce the bus factor.
<jam> dimitern: you up for it the next time it needs to be done?
<voidspace> jam: sounds good
<jam> voidspace: I'm pretty sure wallworld and dimitern also have the root credentials, but they may not realize it.
<voidspace> jam: dimitern doesn't even use the canonical aws account, he uses his own
<voidspace> jam: so I'm pretty sure he doesn't realize it...
<dimitern> jam, sorry - up for what?
<dimitern> jam, i probably have access to the canonical aws account, but I have to find where the info is and I haven't tried to add users to it
<dimitern> jam, that's not a priority for now though - we can sort it out later
<anastasiamac> axw: pool list  :D - https://github.com/wallyworld/juju/pull/13
<wallyworld> anastasiamac: ty, will look after i finish current review
<jam> dimitern: right. not a worry this week, but I want to make sure other people know how to do it. Probably we can just go over it quickly in some downtime next week.
<dimitern> jam, great, I'd appreciate that
<mgz> it's hailing... ugh
<jamespage> mgz, yes very hard
<jamespage> mgz, we must be in the same storm I think
<mgz> I wonder who's getting it first :)
<jamespage> mgz, not quite so hard now
<jamespage> so maybe I got it first
<jamespage> mgz, question - I think I see a behavioral change in 1.21
<mgz> from 1.20?
<mgz> or from a previous 1.21 dev release?
<jamespage> mgz, specifically I have a test process that bootstraps and adds a number of machines and then configures them prior to deploying services
<jamespage> 1.20 -> 1.21
<mgz> k
<jamespage> 1.20 would happily use said provisioned machines upon deployment
<jamespage> not so with 1.21 - it creates new machines
<mgz> how we use provisiond but not deployed-onto machines has been fiddled with, I don't recall at what point exactly...
<jamespage> mgz, just trying to understand whether I was using a undesigned feature ot nor
<mgz> there's almost certainly a way to do what you want still
<jam> hey mgz, did you do some of the rackspace investigation in the past? There's a thread on juju-dev about using Juju with Rackspace and I want to make sure I'm not regurgitating falsehoods
<mgz> jam: I'll have a look at the thread - the old issue was their old (pre-use-of-trunk-openstack) setup didn't have things like security groups
<mgz> I'd think everyone *should* have been switched to a more vanilla setup by now
<jam> jamespage: speaking of which, it'd be really good to get wider knowledge of why people want to preallocate, because there has been some team pushback as to "why would you want to do that". that I want to make sure we understand the use cases well.
<axw> wallyworld: can you please merge my PR?
<wallyworld> axw: sure
<mgz> jamespage: when you get to deploying the services, you're just to using --to right?
<mgz> *not using
<jamespage> mgz, not using --to
<wallyworld> axw: i think i have an acceptable hack for loop devices for local provider; if it tests ok that is
<mgz> which used to pick a created-but-un-united machine, but does no longer
<axw> wallyworld: cool
<axw> wallyworld: looking forward to not having to bootstrap ec2 constantly...
<jamespage> jam: right now I use pre-allocation to setup and environment with machines attached to six different networks so i can test out the multi-network support in the openstack charms
<jamespage> jam: I fully anticipate that juju will replace this requirement at some point
<wallyworld> axw: agreed, i'm not sure how long i'll be as i have some other stuff to review, plus a few chores to finish tonight. hopefully soon
<jam> jamespage: so why do that via "add-machine" rather than during "deploy" time? Easier for you to reason about ? Some flag that we support on add-machine that we don't in deploy ?
<axw> wallyworld: nps, I'll keep an eye out
<jamespage> jam: no - I want to have the base OS fully configured before deploying the charms
<jam> jamespage: as in you add-machine, then go fiddle with it, then deploy onto them?
<jamespage> jam: yes
<jamespage> jam: when we implemented the multi-network support we specifically don't try to configure anuy network devices in the charms
<jamespage> jam: that is not the resposibility of the charm
<jamespage> jam: so I'm using this approach to fill the gap until juju does this for us
<jamespage> jam: when "--networks" is actually fully implemented :-~)
<axw> wallyworld: ping?
<wallyworld> yo
<axw> wallyworld: can you please take a quick look at https://github.com/juju/names/pull/38?
<wallyworld> sure
<axw> I forgot to get that landed first...
<wallyworld> ah yeah i should have mentioned it
<wallyworld> LGTM
<axw> cheers
<anastasiamac> axw: wallyworld: and again... https://github.com/wallyworld/juju/pull/14
<axw> looking
<wallyworld> axw: i have something working except for one bit; ping me when you're free
<axw> wallyworld: sure, just reviewing anastasiamac's branch atm
<wallyworld> yep, np
<axw> wallyworld: done
<wallyworld> axw: so, the remaining part is to write the block devices back to machine in state after the loop devices are created
<wallyworld> https://github.com/wallyworld/juju/compare/wallyworld:storage-feature...provision-loop-devices
<wallyworld> the existing apis don't seem suitable
<axw> looking
<wallyworld> all works nicely though
<axw> wallyworld: I think provider type should just be part of VolumeParams
<wallyworld> yeah, didn't i do that?
<wallyworld> i did
<axw> wallyworld: you've got it next to it in ProvisioningInfo
<wallyworld> bah, old code,
<axw> possibly before you made the change to VolumeParams
<axw> :)
<alexisb> axw, wallyworld what time is it for you guys??
<wallyworld> 00:15 for me :-)
<axw> alexisb: early, only 22:15 here
<alexisb> crazy
<wallyworld> alexisb: storage is a beast
<alexisb> yes but I have to say it is very exciting to see the work land :)
 * perrito666 knows wallyworld had other adjectives in mind
<axw> wallyworld: I think you should just create a new "provisioner" API method that calls state.setProvisionedBlockDeviceInfo
<axw> wallyworld: it's all in the wrong place, but that's the least amount of work
<wallyworld> axw: yeah, that was what i was afraid would be needed, thanks for verifying
<wallyworld> works great apart from that last bit
<wallyworld> axw: i may look to merge what i have so katco can use it
<axw> cool
<natefinch> ericsnow: can you change the sharing settings on your plugin provider doc to "can comment"?  Possibly even "can edit" if you trust us not to muck it up
<natefinch> wwitzel3: mind if we move our 1:1?  I need to go snowblow 30" of snow off my driveway
<perrito666> natefinch: how many snow related activities do you have :p
<wallyworld> axw: should be good to go https://github.com/wallyworld/juju/pull/15
<natefinch> perrito666: I don't often rake the roof, but I always have to snowblow (if there's more than a couple inches of snow).
<axw> wallyworld: looking
<wwitzel3> natefinch: only 30"? ;)
<natefinch> wwitzel3: it's a guess based on the 28" we had yesterday noontime.... I'll definitely remeasure and get an accurate total.  It was a record breaking storm for much of Massachusetts.
<anastasiamac> axw: wallyworld: m eod.. would it b awesome to see PR#14 land :D
<wwitzel3> natefinch: yeah, had some friends posting pictures of some 60-70" drifts
<axw> anastasiamac: good night
<natefinch> wwitzel3: nice
<anastasiamac> axw: or morning :D thnx :D talk tomorrow
<wallyworld> anastasiamac: are the isues addressed?
<wwitzel3> natefinch: overall, looks like the area took the storm well (so far), no a lot of power issues that I've heard of
<anastasiamac> wallyworld: issues? no issues :D
<anastasiamac> wallyworld: suggetsions
<anastasiamac> wallyworld: suggestions even and yes addressed
<wallyworld> anastasiamac: merged
<anastasiamac> wallyworld: \o/
<anastasiamac> axw: wallyworld: will make a map formatted output tomorrow :D
<wallyworld> great
<axw> wallyworld: you're not going to do the change to update state?
<axw> I guess that can come later
<wallyworld> axw: followup branch
<wallyworld> axw: just fixing a test since the deletion of Pool Q= "" broke something
<axw> wallyworld: okey dokey
<wallyworld> i know it's a hack, just using what api was already easily available
<axw> evilnickveitch: that PR actually made me lol :)
<axw> wallyworld: yup, just making sure anyone else reading knows that too
<wallyworld> fair enough
<evilnickveitch> axw me too :)
<dimitern> voidspace, http://pad.lv/1403225
<wallyworld> axw: oh yeah, dependencies.tsv needs updating, we can do that tomorrow
<axw> wallyworld: for?
<wallyworld> names
<axw> wallyworld: anastasiamac already did it in her branch
<wallyworld> the new storage stuff
<wallyworld> ah, ok, didn't notice, ta
<wallyworld> katco: if you pull the latest rev of the feature branch, you can fire up a local provider and deploy a minimal charm that declares block storage and it will call the loop device source. it will currently panic since it's not done but if you plug your work in stuff should happen
<ericsnow> natefinch: done and done :)
<axw> wallyworld: I'm off, cya tomorrow.
<ericsnow> fwereade: mentoring?
<katco> wallyworld: thanks; any suggestions for what charm to deploy?
<perrito666> natefinch: standup?
<dimitern> a trivial review anyone? http://reviews.vapour.ws/r/815/
<natefinch> ,...well that took a lot longer than expected
<natefinch> I don't think I've ever snow blowed for 3 hours before.
<perrito666> natefinch: well wwitzel3 passed me a video of what snow blowing is and it really looks fun
<wwitzel3> :)
<wwitzel3> it is fun, but at the 3 hour mark, it isn't fun anymore
<natefinch> haha... yeah, it looks like fun, but really you're wrestling a 200 pound engine on wheels in the bitter cold
<perrito666> wrestling? these things dont have assisted direction?
<natefinch> nope... the wheels go straight... some models (like mine) you can disengage one of the wheels, which helps it turn left in a little circle... but when you're just going straight, one side or the other will slip or catch, and you have to man handle it to keep it going straight
<perrito666> anyway I am no parameter, I love playing in the snow
<perrito666> wow, that takes a lot of the fun out
<natefinch> haha
<perrito666> I mean looks like an extremely fun little racing car
<natefinch> yeah, it looks very majestic in videos with the snow flying... but it's actually a lot more work than it seems at first
<perrito666> natefinch: well the video wwitzel3 passed me uses pumpkins instead of snow
<natefinch> If you are lucky to have one you can attach to a tractor or something, then I'm sure it's a lot easier... mine's a walk-behind type (since they're a lot cheaper than buying a tractor ;)
<perrito666> wowowoow you walked that thing?
<perrito666> you lost a perfectly fun chance of real life mario kart
<natefinch> perrito666: mine basically looks like this: http://www.snowblowersdirect.com/product-images/921029_11240_600.jpg
<perrito666> ok, not all that fun
<natefinch> haha yeah
<natefinch> brb
<voidspace> dimitern: https://github.com/voidspace/juju/compare/dimitern:wip-ec2-addressable-containers...wip-preparecontainerinterfaceinfo
<perrito666> abentley: sorry for my unhelpful answer
<jw4> OCR PTAL : http://reviews.vapour.ws/r/817/  (Actions API :: Expose Enqueued, Started, and Completed times)
<sinzui> natefinch, do you have a minute to review https://github.com/juju/juju/pull/1496
<sinzui> or http://reviews.vapour.ws/r/818/
<natefinch> sinzui: done
<sinzui> thank you natefinch
<sinzui> natefinch, perrito666 I am setting up unit test runs for windows. Will all github.com/juju/juju be testable, or should I just test a sub package?
<perrito666> sinzui: by now all tests that dont run on windows should have been skipped
<sinzui> oh, well I have bad news then
<perrito666> bogdanteleaga: care to weight on that?
<sinzui> hi bogdanteleaga I am running "go test github.com/juju/juju/..." should I be using different args to test juju on windows?
<bogdanteleaga> sinzui, there are a couple of packages that are not ready yet
<bogdanteleaga> sinzui, I still have 2 PR's that need to be merged and then there's a lot of refactoring in worker
<bogdanteleaga> sinzui, still working on that
<sinzui> bogdanteleaga, okay. So we do expect "github.com/juju/juju/.." to one day pass
<bogdanteleaga> sinzui, definitely
<bogdanteleaga> sinzui, hopefully sometime next week
<sinzui> okay, the test will be just gathering results for a few weeks before we make it voting
<menn0> thumper: manual tests for my branch are looking good
<menn0> thumper: reviewing yours now
<thumper> menn0: cool, just looking at your code now
<menn0> thumper: dear god... that's a whole lot of changes
<thumper> menn0: oh c'mon, it isn't that big
<menn0> a lot of files anyway
<thumper> but yes... could have renamed Prepare to PrepareForBootstrap differently
<thumper> but it wouldn't have made as much sense out of context
 * menn0 agrees
<menn0> thumper: gotta love those floating point port numbers
<menn0> thumper: SMTP is on port 25.0 right?
<thumper> menn0: yeah, that sucks
<thumper> :)
<menn0> thumper: i haven't found anything beyond wwitzel3's comments
<menn0> thumper: thanks for the review. all good comments.
<thumper> wwitzel3: O
<thumper> wwitzel3: I've addressed your comments
<wwitzel3> thumper: thanks
 * thumper heads to the gym
#juju-dev 2015-01-29
<katco> does juju have a wrapper around exec.Command?
<perrito666> katco: iirc there is one in utils
<perrito666> sadly there is more than one around
<katco> perrito666: ty sir
<perrito666> it does little to wrap it :)
<perrito666> yw
 * perrito666 wonders why is he even reading this channel at thos hour
<jw4> perrito666: because you love it here - admit it
<katco> :)
<perrito666> jw4: well have no life :p
<jw4> :)
<katco> code is life!
<katco> (of course there are more important things, but coding is a lot of fun :))
 * jw4 whispers "don't tell katco any different"
<katco> lol
<jw4> :)
<katco> well right now my daughter is asleep by 6:30
<katco> 18:30
<jw4> wow - how old is she?
<katco> i'm sure when that changes i'll be a little more scarce
<katco> she is 8 months
<jw4> awww
<jw4> does she sleep through the night though?
<katco> most of the time
<jw4> nice!
<katco> she's a good little baby :)
<jw4> yeah; our first slept through the night at 8 weeks, but our son (second born) took much much longer
<katco> how many do you have john?
<jw4> two
<jw4> daughter 13 and son 9
<katco> wow, how nice :)
<katco> the teenage years haha
<jw4> don't envy the daipers and sleepless nights even though some monster occasionally replaces my daugter
<katco> well, i'm having fun so far :)
<jw4> good
<jw4> I get nostalgic until I think about some of the practicalities of infants
<katco> hehe
<jw4> perrito666: I forget - you don't have kids right?
<jw4> he took himself seriously
<katco> haha
<menn0> katco, jw4: we make baaad sleepers
<menn0> katco, jw4: my son is 19 months and he's only /just/ started sleeping through
<katco> oh man
<katco> menn0: i feel lucky for sure
<jw4> menn0: yeah? That's not unusual I think... but it's a long time to have disruptive sleep
<jw4> menn0: for the parents I mean
<perrito666> jw4: nope, it shows right?
<jw4> perrito666: you don't look weary? ;)
<perrito666> well if when I have kids they are anything like me the will begin to be good sleepers around 21
<perrito666> years that is
<jw4> perrito666: lol
<thumper> menn0: are you close to merging your env worker branch  ?
<perrito666> wallyworld: you are a hard client to please :p
<wallyworld> :-(
<wallyworld> did what i say make sense though?
<perrito666> wallyworld: not really but I havent read the last review just the email
<wallyworld> the review is just a note or two
<wallyworld> we have existing code to set machine/unit status
<wallyworld> so we need to use that instead of duplicating new code
<perrito666> lets go priv so logs dont suck later
<wallyworld> we add the agent status setter and the apiserver layer thunks accordingly
<wallyworld> ok
<katco> axw: wallyworld: boom! 2015-01-29 02:10:34 INFO juju.storage.provisioner provisioner.go:128 block devices created: [{1 1771eb70-b0ca-49e3-84d4-7b73b1401d18 /dev/loop1
<katco>     1  false}]
<axw> katco: sweet :)
<wallyworld> whoot
<katco> going to sanity check the image rq, but we might be good
<wallyworld> seems so
<axw> katco: are you generating a UUID for the volume ID now?
<katco> axw: currently, yes. with todos to use the algorithm you suggested
<katco> axw: i just didn't have a machine id to hang off of
<axw> UUID would probably be fine TBH
<axw> assuming you're creating the image file with that too
<katco> yes
<katco> axw: i think we'll still have to put some thought into how to name these things so persisting across reboots isn't a pain
<katco> axw: but that is trivial i think
<katco> /dev/loop1: [0801]:51515891 (/var/lib/juju/storage/block/loop/var/lib/juju/storage/block/loop/1771eb70-b0ca-49e3-84d4-7b73b1401d18)
<katco>  
<axw> cool
<katco> charm is hello worlding along just fine, i'm going to push these changes and have dinner
<katco> i'll check back in after to see if there are any issues
<katco> axw: https://github.com/wallyworld/juju/pull/4
<wallyworld> looking
<katco> axw: fair warning, the lxc tests are not passing (stupid trivial stuff to fix)
<katco> wallyworld: ^^
<katco> ok i'm eating some dinner. bbiab
<wallyworld> ty
<rick_h_> mattyw: let me know when the data is loaded and will check it out. It needs rating info because that's the api we hit
<rick_h_> mattyw: so it needs the sub, the plan, and a starting rating data so that the my accout page in the UI had some records to it
<mattyw> rick_h_, stop
<rick_h_> mattyw: party
<mattyw> thumper, ping?
<thumper> mattyw: hey
<thumper> mattyw: wwitzel3 reviewed that branch :)
<mattyw> thumper, hey there, ok cool - I didn't get around to much reviews yesterday so I'm going to try to get to some today
<mattyw> thumper, quick question for you
<thumper> shoot
<mattyw> thumper, recently I've been wanting to find the envuuid from juju via the command line - I don't think we have one at the moment, and it feels like it might be something that would be useful to have in juju status
<mattyw> thumper, wanted to know what thoughts/ plans you had for that kind of thing
<thumper> mattyw: there is one
 * thumper bootstraps to check
<thumper> $ juju api-info environ-uuid
<thumper> 607aa9ab-dd2e-4943-86c3-3fc65b800450
<thumper> mattyw: ^^
<thumper> although, in status?
<thumper> maybe
<mattyw> thumper, awesome!
<mattyw> thumper, that certainly makes me happy for now
<mattyw> thumper, I didn't know about the api-info command - feels like Christmas :)
<mattyw> thumper, thanks for the help :)
<thumper> np
<mattyw> thumper, as along as it's available somewhere I'm happy for now
<thumper> kk
<wallyworld> axw: another oarsum hack https://github.com/wallyworld/juju/pull/16
<axw> wallyworld: looking
<axw> wallyworld: reviewed
<wallyworld> ty
<katco> wallyworld: i've addressed your concerns. ok if i merge?
<wallyworld> katco: would be awesome
<katco> wallyworld: i think you might have to do it since i don't have write access
<wallyworld> katco: oh ffs i keep forgetting
<wallyworld> katco: merged \o/ tyvm
<wallyworld> i'll try it out now
<wallyworld> after meeting
<katco> wallyworld: i would feel a lot better if you or axw could poke at things and make sure this is working (multiple block devices etc.)
<wallyworld> will do
<katco> wallyworld: i'm not familiar enough yet to know what edges there are
<axw> katco: sure, I'll have a play with it
<katco> axw: tyvm
<katco> ok, i'm going to take care of a few things... please ping me if you find something!
<axw> wallyworld: FYI, the loop devices won't get cleaned up if you destroy the machine
<axw> wallyworld: we should be marking these as "persistent" later, and preventing env destruction till they're gone
<wallyworld> axw: i just noticed that
<wallyworld> i ate my disk with a large loop device
<thumper> axw,wallyworld: either of you remember where is the cloud-init.log file is on cloud machines?
<axw> thumper: I thought it was just /var/log/cloud-init-output.log ?
<thumper> axw: ta
<wallyworld> what he said
<wallyworld> axw: sorta works, but something is not quite wired up - attached hooks complaining storage foo/0 is not provisioned, just digging into that now
<wallyworld> this is for loop on local
<axw> wallyworld: yeah, I'm having some issues too but I've modified the provisioner
<wallyworld> so close
<anastasiamac> axw: wallyworld: there r separate discussion going on..
<anastasiamac> axw: wallyworld: is it pool manager responsibility to vaidate provider type supplied
<anastasiamac> axw: wallyworld:?
<anastasiamac> axw: wallyworld: on pool create that is...
<wallyworld> depends if we think pool manager should know about envieons
<axw> I guess not
<axw> it should validate the storage provider type, but probably not whether it is valid for the current environment
<wallyworld> that is my initial thought, but i'm happy to be wrong
<axw> it does make me a little nervous that something else will come along and use the pool manager without validating, that's all
<wallyworld> may need another component - EnvironPoolManager
<wallyworld> or maybe that's overkill
<axw> that might make it clearer, but you could still go through the PoolManager directly (unless that gets hidden)
<wallyworld> yes
<wallyworld> not too fussed either way
<wallyworld> was just trying to reduce external dependencies of pool manager
<wallyworld> would i think prefer an EnvironPoolManager to live in /environ/storage
<wallyworld> and the api calls that
<axw> I vote we defer this decision, and do validation in the apiserver code for now
<wallyworld> yep, sounds good
<wallyworld> was about to say that :-)
<wallyworld> am annoyed we have so much business logic in apiserver layer - need to fix that sometime
<thumper> heh
<thumper> wallyworld: I'm sure it is because we don't have a business layer
<thumper> we have apiserver and state
<thumper> and people didn't want it in state
<thumper> so guess where it goes
<wallyworld> thumper: i know too well the reason - been annoyed about it for ages as you know
<thumper> yeah.... I know
 * thumper has the munchies
<wallyworld> thumper: if i just install django, is there a default page like for apache or tomcat etc?
<thumper> kinda
<thumper> you mean locally or the charm?
<axw> wallyworld: sorry, I think your SetProvisionedBlockDevices method is broken. it's only calling state.DemoSetProvisionedBlockDeviceInfo if err != nil above
<wallyworld> thumper: i just want to expose the service and point a browser to it to see it worked
<thumper> wallyworld: it might work, but not entirely sure
<wallyworld> kk
<wallyworld> axw: ffs i'm stupid
<axw> wallyworld: I'm changing the storage provisioner to periodically check for unprovisioned storage instances
<axw> wallyworld: not using a watcher because the entities are in the wrong places (not associated with machine, but units)
<wallyworld> can you fix that stuff up while you're there?
<axw> yes
<wallyworld> ta
<axw> just testing now
<wallyworld> dontcha love hacks for demos
<menn0> wallyworld: you too huh? :)
<wallyworld> noooooo, we aren't using *any* dirty hacks
<wallyworld> we're far better than that
<menn0> wallyworld: of course not. we aren't either. ;-/
<wallyworld> glad to hear it
<axw> wallyworld: I think we should be running those LXC changes by someone, could be opening up security holes
<axw> wallyworld: in particular, looks like we're dropping all AA protection
<wallyworld> i think we can tighten up the rules
<wallyworld> after the demo :-)
<axw> of coursed
<wallyworld> axw: nice branch. we should also be able to register the new providers for ec2 as well as local i would expect?
<axw> wallyworld: yep
<wallyworld> wanna add that real quick?
<axw> wallyworld: sure, just testing a tmpfs provider.. I think it needs more lxc config hacks to work
<axw> likewise with formatting/mounting block devices in local, I think
<wallyworld> ah bollocks
<axw> wallyworld: actually I think I've just mucked up the mount syntax, but could still be broken... will come back to it
<wallyworld> righto
<axw> wallyworld: testing rootfs on ec2, will take a few mins
<wallyworld> sure, np
<wallyworld> my connection has been shit today, so ec2 testing is painful
<axw> wallyworld: updated
<wallyworld> ty looking
<wallyworld> thanks axw, i'll merge
<axw> cool
<axw> wallyworld: I was worried about nothing, tmpfs works on local
<wallyworld> farking oarsum
<axw> wallyworld: https://github.com/wallyworld/juju/pull/19
<wallyworld> looking
<wallyworld> sigh, ec2 is slooooow right now
<anastasiamac> axw:wallyworld: running tests after rebasing, m getting
<anastasiamac> cmd/juju/storage/listformatters.go:11:2: cannot find package "github.com/dustin/go-humanize"
<anastasiamac> axw: ?
<axw> anastasiamac: godeps -u dependencies.tsv
<wallyworld> axw: i did have a way of registering common providers but removed it due to import loops
<wallyworld> i plan on adding it back later
<axw> wallyworld: cool
 * anastasiamac rolles eyes
<anastasiamac> axw: of course :D
<axw> wallyworld: unless you think there's anything else we need for next week, I'm going to take a step back from code and go back to the model rework
<wallyworld> axw: should storage-dir in tmpfs default to <libdir>/storage or similar?
<axw> libdir?
<wallyworld> the juju lib dir setting
<wallyworld> can't remember the name
<wallyworld> comes from agentconfig
<axw> wallyworld: data-dir?
<axw> wallyworld: as in /var/lib/juju/storage?
<wallyworld> yeah
<axw> isn't that where I put them?
<axw> /var/lib/juju/storage/fs
<wallyworld> ah, haven't seen that in the diff yet
<wallyworld> ah there it is
<axw> wallyworld: I really think we need to change the way that works, since (as I noted in a TODO) the data-dir isn't necessarily the same on all machines
<wallyworld> at the bottom
<wallyworld> it should be decided late
<axw> so it can't be a property of the pool, but of the provider as instantiated in the agent
<wallyworld> yep
<wallyworld> will do for now :-)
<axw> yep
<wallyworld> axw: i'll fix that failing test anastasia mentioned. on my machine the cmd/juju tests *always* hang and time out :-(
<axw> okey dokey
<axw> thanks
<axw> wallyworld: actually before I go off to do model things, one more thing... we should be able to use the storage subordinate now
<wallyworld> axw: only web app i can find that works with postgres is discourse and it fails to install - apt issues, fixed, but now sig verification failure. just seems too out of date
<axw> :(
<wallyworld> oh storage subordinate
<wallyworld> hmmm
<wallyworld> if there were a bundle with that with a web app charm, might be good
<wallyworld> i'd still like to find a web apps just just uses postgres like wordpress uses mysql
<wallyworld> i might see about making mysql chamr use storage
<wallyworld> but first food
<axw> wallyworld: https://jujucharms.com/rails-example-single/6 ?
<axw> I'll test it out..
<wallyworld> nice, how'd you find that?
<wallyworld> my charm store search foo must be bad
<axw> wallyworld: jujucharms.com, bundles
<axw> looked for one with an elephant :)
<wallyworld> :-)
<wallyworld> right, i need to eat first then charms
<voidspace> TheMue: https://github.com/voidspace/juju/compare/dimitern:wip-ec2-addressable-containers...wip-preparecontainerinterfaceinfo
<voidspace> TheMue: there's a comment in there "// TODO: (mfoord) we should add the machine ID to the IP"
<voidspace> TheMue: that's out of date and can  be removed (the call to addr.AllocateTo does this)
<wallyworld_> axw: data=rootfs,10M   <- forces the use of the 10M or else it errors. we shouldn't need that for filesystemmounts
<axw> wallyworld_: that's not always true. Ceph will need it...
<wallyworld_> rightio, so it needs to be optional
<axw> wallyworld_: yep. it should default to 1G anyway, which will then be ignored
<wallyworld_> yep
<wallyworld_> axw: loop file systems, as accessed from inside container, are owned by root. we will need to make the ownership ubuntu.ubuntu
<axw> wallyworld_: do we? the charms run as root
<wallyworld_> unity also notified me of a new device and showed the disk icon in the launcher. and i can see what files i create from inside the container
<axw> nice :)
<wallyworld_> ah yeah, fair enough
<axw> wallyworld_: that fails app fails when I try to install it...
<wallyworld_> joy
<axw> wallyworld_: apparently you can use django, but I have nfi about it
<wallyworld_> me either
<wallyworld_> i was hoping django had a welcome page
<wallyworld_> like apache, tomcat etc
<wallyworld_> but i didn't see one
<wallyworld_> axw: i think juju storage list should be sorted by owner. thoughts? i had a few units and would prefer to see storages for the units listed together
<menn0> anyone able to look at this one line fix: http://reviews.vapour.ws/r/820/
<axw> wallyworld_: yeah, probably
<axw> wallyworld_: in the tabular format anyway
<wallyworld_> menn0: oh, you ask all the hard questions
<wallyworld_> axw: yep
<menn0> wallyworld_: sorry :)
<menn0> wallyworld_, axw: thanks guys
<wallyworld_> anastasiamac: can you please sort the tabular storage list by owner? when you are free
<perrito666> morning
<voidspace> perrito666: morning
<wallyworld__> axw: this fixes some tests (yours and mine :-) and also makes default size 1G and default count 1 always https://github.com/wallyworld/juju/pull/20
<menn0> wallyworld__, axw: pushing a branch now that fixes the panic that's currently being frequently seen during merge attempts
<wallyworld__> which test?
<menn0> wallyworld__: the leadership feature tests are failing frequently
<menn0> the problem isn't actually with those tests but with my recent change to how the machine agent workers are started
<menn0> subtle race
<wallyworld__> ok, i hadn't noticed as ee've been landing to a feature branch
<wallyworld__> great that you caught it
<menn0> wallyworld__: http://reviews.vapour.ws/r/821/
<menn0> wallyworld__: funnily enough, this related to a review comment you made :)
<wallyworld__> oh, i hate to ask, which one?
<menn0> wallyworld__: where we were discussing whether the envWorkerManager should clean up the returned runner even when an error was returned
<wallyworld__> ah right
<menn0> wallyworld__: I was using an unusual pattern, which you picked up on
<wallyworld__> ancient history, > 1 week ago :-)
<menn0> wallyworld__: this fix in the same area
<wallyworld__> menn0: done, ty for fix
<menn0> wallyworld__: thanks. good idea re the comment
<wallyworld__> menn0: i can't recall, maybe even a comment next to where singular runner is used if it's releavnt to do so
<menn0> wallyworld__: I actually have a better idea which is clearer and reduces the assumptions about the singular runner's behaviour
<wallyworld__> ok
<menn0> wallyworld__: pls take a look again. this is much clearer IMO. http://reviews.vapour.ws/r/821/diff/
<wallyworld__> ok
<wallyworld__> menn0: can we remove "&& singularRunner != nil" now?
<wallyworld__> looks much nicer
<menn0> wallyworld__: probably but I've been bitten by not have a guard like that before
<menn0> wallyworld__: let me try it
<wallyworld__> menn0: i can't see that it's needed as we exit above with an err before the defer
<menn0> wallyworld__: yeah I know, but I've thought that before and had problems
<menn0> wallyworld__: that was in a slightly different case though
<wallyworld__> menn0: runner and singular runner usage looks the same and runner has no guard
<wallyworld__> oh wait
<wallyworld__> sorry
<wallyworld__> can't read
<menn0> wallyworld__: it does for the defer that cleans
<wallyworld__> LGTM
<menn0> wallyworld__: i'm going to remove them and re-run the tests and my little tester that was triggering the panics
<menn0> wallyworld__: the guards irk me too
<wallyworld__> ok
<menn0> wallyworld__: yep, doesn't work. panics due to nil pointer dereference
<wallyworld__> doh
<menn0> wallyworld__: I don't get why... something special about defers perhaps and how they get executed?
<wallyworld__> nfi right now
<menn0> wallyworld__: I don't care right now. bed awaits and I still have some other (non-work) jobs to do
<wallyworld__> yep, land tha sucker
<menn0> wallyworld__: that's in. thanks for the late night review.
<wallyworld__> np, catch you later
<wallyworld__> really late for you
<menn0> yeah :)
<jam> wallyworld__: anything to go over before I see you in Capetown ?
<jam> fwereade: ^^
<wallyworld__> jam: nothing urgent - just need go over storage, health etc prior to seeing mark
<wallyworld__> jam: there's a few things re: storage we can disuss over there
<katco> wallyworld__: axw: hey, just waking up. how are things with storage?
<wallyworld__> katco: good :-)
<katco> wallyworld__: woohoo!
<wallyworld__> loop provider works
<wallyworld__> was tweaked a bit
<katco> wallyworld__: cool
<wallyworld__> axw added tmpfs and rootfs
<katco> wallyworld__: anything i should know going into today?
<katco> very cool
<wallyworld__> katco: we do need to 1. evaluate the permissions, especially the aa ones on the container (but that can wait); 2. loop at mounting an external block device
<wallyworld__> look not loop
<axw> and bind mounting into local provider
<katco> wallyworld__: understood on both accounts. 1 is part of a lot of cleanup we'll have to do
<wallyworld__> yep
<katco> wallyworld__: axw: i was planning on looking at 2 today
<wallyworld__> doesn't matter for demo of course
<wallyworld__> great :-)
<axw> cool
<katco> wallyworld__: axw: ok i'm off to get ready. you guys rock!
<katco> anastasiamac: and you as well if you're around
<axw> katco: later, hope you have a good day
<wallyworld__> axw: not sure if you saw before, i fixed some tests and added default size handling https://github.com/wallyworld/juju/pull/20
<axw> wallyworld__: I didn't. looking now
<wallyworld__> i made count default to 1 regardless of size
<wallyworld__> if count wasn't specified
<axw> wallyworld__: reviewed
<wallyworld__> ty
<wallyworld__> axw: changes pushed
<axw> wallyworld__: LGTM, thanks
<wallyworld__> merged
<wallyworld__> katco: also, if you're bored, there's also investigating how to provision block devices on openstack (as we do with EBS volumes on AWS)
<dimitern> jam, hey, as OCR can you have a look at this please? http://reviews.vapour.ws/r/822/
<dimitern> wallyworld__, axw, ^^
<dimitern> well not as OCRs but if you have some time :)
<alexisb> wallyworld__, axw you guys must be robots that don't need sleep
<jamestunnicliffe> dimitern: Could you review http://reviews.vapour.ws/r/823/ please?
<dimitern> jamestunnicliffe, sure, looking
<mgz> ....snow
<ericsnow> perrito666: are we having standup? (no natefinch, no wwitzel3)
<perrito666> ericsnow: we can wave it
<perrito666> ericsnow: yet Ill priv you a question
<ericsnow> perrito666: k
<TheMue> voidspace: why do you think so?
<voidspace> TheMue: no reason...
<jamestunnicliffe> dimitern: http://reviews.vapour.ws/r/824/ should work. Deleted the old merge and review request
<dimitern> jamestunnicliffe, reviewed
<dimitern> mgz, ping
<mgz> dimitern: hey
<dimitern> mgz, hey! so a quick question about the merge bot
<mgz> hit me
<dimitern> mgz, jamestunnicliffe - our new team mate wants to land a PR - https://github.com/juju/juju/pull/1502 - and he is in juju Hackers group on GH, has access to RB and all
<dimitern> mgz, but it seems the bot is not picking up his $$merge$$ comment
<mgz> jamestunnicliffe: have you set your membership of the juju group to public?
<dimitern> mgz, ah! that might be it :)
<dimitern> mgz, he went for some tea - will ask him in a moment :)
<mgz> cool, poke if that doesn't help
<dimitern> mgz, cheers! I can see his membership is private, that should be it
<dimitern> voidspace, http://reviews.vapour.ws/r/822/diff/
<perrito666> wallyworld__:  you are going to hate me when you are back
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs:  1416006
<wwitzel3> ericsnow: I'm back btw, just trying to finish up this final once over I promised for the cloudsigma provider
<ericsnow> cool
<ericsnow> wwitzel3: how'd it go?
<wwitzel3> ericsnow: how is the services stuff going? also we should probably start poking people to have another look at GCE now that the first set of review fixes are up.
<alexisb> wwitzel3, so are you officially moved in?
<ericsnow> wwitzel3: I'm wrapping things up (hope to have an initial patch up soon)
<ericsnow> wwitzel3: the systemd stuff on top of that will be almost trivial :)
<wwitzel3> ericsnow, alexisb: went great, keys in hand, we are going to take some stuff over tonight and tomorrow evening and stay the night tomorrow :)
<ericsnow> \o/
<wwitzel3> ericsnow: yeah, I was thinking I would do that, no better test of an abstraction than having someone who didn't write it, us it :)
<wwitzel3> s/us/use
<ericsnow> wwitzel3: perfect
<alexisb> wwitzel3, sweet, first nights are always fun
<wwitzel3> alexisb: yeah, pizza on the floor as been my tradition since my first house ~12 years ago, so probably do that again
<menn0> thumper, waigani: that CI blocker details are a little off but there is a problem
<menn0> thumper, waigani: I suspect this is unrelated to the issue I already fixed
<waigani> yay
<menn0> thumper, waigani: for whatever reason it seems much more likely on i386 than other archs
<menn0> thumper, waigani: in fact it hasn't happened yet on amd64
<menn0> waigani: you haven't committed any presence changes yet have you?
<waigani> menn0: nop
<menn0> waigani: cause it's now the presence pinger that's dying
<waigani> oh really?
<menn0> waigani: I have a theory that the presence pinger is still active when state is shut down
<waigani> menn0: could it be one of our multi-env tests - sets up two states, and the pinger for each will currently listen to everything from all envs?
<menn0> thumper, waigani : even though from a quick look at the code that doesn't seem possible
<menn0> thumper, waigani : I'll keep digging post-kids swimming. I have to go now.
<waigani> menn0: cool.
<sinzui> menn0: Right, you branch fixed some of the panics CI saw, but not all :(
 * thumper sad-face
<wallyworld__> perrito666: that maas issue is easy to fix - i can do it. the time to wait is more than sufficient on vmaas but clearly not on hardware so i'll increase it
<perrito666> wallyworld__: isnt there a better way?
<wallyworld__> sadly no
<wallyworld__> the deployment process involves a reboot
<wallyworld__> and all juju can do is poll
<wallyworld__> i discussed the approach with the maas guys
<thumper> waigani, menn0, cherylj, davecheney: http://paste.ubuntu.com/9943662/
<wallyworld__> trivial review for 1.22 maas fix http://reviews.vapour.ws/r/825/
<thumper> huh...
<thumper> why am I surprised that this is working?
<waigani> thumper: is that juju status of another env? Nice!
<thumper> waigani: yep
<thumper> and I just did 'juju ssh 0'
<thumper> and got in to it
<waigani> sweet
<waigani> thumper: next test would be to expose a service, i.e. wordpress/mysql and visit it
<thumper> there are problems with the rsyslog worker
<thumper> hmm...
<thumper> inside the machine, the logdir is /var/log/juju
<thumper> however on the host it is /var/log/juju-tim-another
<thumper> and the rsyslog worker is looking for the host log dir not the machine log dir
<waigani> thumper: extend RsyslogConfig to hold the dir path?
<thumper> I'll be looking into it next
<thumper> huh
<thumper> I now have ubuntu deployed in both environments
<thumper> time for some wordpress/mysql action
<waigani> =D
<thumper> :(
<thumper> now I have machines for the secondary environment, we need waigani's destroy branch
 * thumper taps fingers
 * thumper does things manually
<menn0> thumper: that's awesome that it's working
<menn0> jw4: ping
<menn0> thumper: can you push what you have to your repo soon so I can grab it?
<sinzui> wallyworld_: I am fighting google
<jw4> menn0: yep
<thumper> menn0: pushing it up now
<thumper> menn0: https://github.com/howbazaar/juju/tree/juju-environment-create
<menn0> thumper: cheers
<menn0> davecheney: do you know if "go test" run tests in parallel by default?
<waigani> menn0: I asked davecheney the same question a while ago: answer was no
<menn0> waigani: interesting. one of the stacktraces i'm looking had has teardowns running from different test suites in different packages (in separate goroutines)
<waigani> menn0: with pingInfo, do we need to add a docID and leave Slot as an int64? i.e. the same as you suggested for beingInfo?
<waigani> oh really?
<davecheney> menn0: the answer is yes
<davecheney> and no
<davecheney> tests inside a package are run in sequence
<menn0> waigani: re pingInfo, yeah that sounds right (so change the underlying field name for Slot)
<davecheney> multiple packages are tested in parallel, just like they are built in parallel
<menn0> davecheney: ok, that explains that then. thanks.
<waigani> ah right
<menn0> davecheney: I hadn't noticed that before so it wierded me out
<davecheney> menn0: you've not noticed that running the tests uses all your cores ?
<davecheney> sorry, that sounded passive agressive
<davecheney> what i meant to say is
<davecheney> go test .../juju/... would take over an hour if packages were not tested in parallel
<menn0> davecheney: I wasn't offended :)
<menn0> davecheney: I run tests for the package i'm working on most of the time which just uses one core
<menn0> davecheney: for full runs I often start it and walk away so don't really notice core usage
#juju-dev 2015-01-30
<davecheney> menn0: fair enough
<davecheney> there is an option for running tests within a package in parallel
<davecheney> but
<wallyworld__> axw: trivial review when you have a moment http://reviews.vapour.ws/r/825/
<davecheney> a. it's not the default (nobody writes tests taht can be run concurrently)
<davecheney> b. you have to specifically enable it on a per tets by test basis
<davecheney> c. it wouldn't matter because gocheck makes everything look like one test case anyway
<menn0> davecheney: thanks
<menn0> thumper, wallyworld__ : possible fix to the CI blocker. http://reviews.vapour.ws/r/826/
<thumper> menn0: how confident are you that it fixes it?
<thumper> it certainly looks like a candidate
 * thumper is munching then free for the call
<thumper> ah shoot
<thumper> menn0: I forgot about my chat with wallyworld__
<thumper> wallyworld__: what is your schedule like?
<wallyworld__> i can wait
<menn0> thumper: well I can't replicate the problem locally, it's timing dependent. it only seems to fail for i386 and ppc in CI.
<thumper> cool cool
<wallyworld__> ping when ready
<jw4> menn0: I can try it on my slow ec2 instance
<menn0> thumper: but there was definitely a problem which this PR fixes and I can see how the bug could lead to the problems we're seeing.
<jw4> menn0: although after reviewing the fix my error might be unrelated
<menn0> jw4: that would be great
<menn0> jw4: the error you added to the bug could definitely be related to this fix
<jw4> menn0: cool - I'll spin it up and try
<menn0> jw4: thank you
<jw4> menn0: hmm - first test run without your change did not fail - I'll try again
<thumper> wallyworld: chat?
<menn0> jw4: I would expect the problem to be somewhat intermittent
<jw4> yup
<axw> wallyworld: can you just change the timeout to be based on the bootstrap-timeout?
<axw> wallyworld: it's configurable
<thumper> wallyworld: I'm in the hangout, but I'm going to make a coffee
<jw4> menn0: still not repro'ing it - i'm gonna try with an i386 image
<menn0> jw4: ok thanks
<jw4> menn0: otherwise we'll just have to watch for it in the CI servers
<menn0> jw4: the CI unit test runs with the fix in are about to finish so that might tell us more as well
<jw4> menn0: +1
<menn0> jw4: this is the key one to watch: http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-i386/1327/console
<menn0> jw4: the jujud tests have already passed which is a good sign
<jw4> sweet
<thumper> wallyworld: missed you, I'm off to the vet, bbl
<wallyworld> thumper: np, i'm off to buy a new farking network adaptor
<wallyworld> can't stand these disconnects
<menn0> wallyworld: how common is it for the jujud test runs to time out on the (apparently slower) i386 CI hosts?
<menn0> wallyworld: because a lot of runs just did
<wallyworld> not sure, but often i suspect
<wallyworld> our tests timeout on my laptop
<wallyworld> axw: pr updated
<axw> wallyworld: just noticed. LGTM
<axw> wallyworld: your default size change is ineffective, sorry I didn't pick up on it earlier. your change just alters a temporary
<axw> I'll fix it in my branch
<wallyworld> ah ball ok, thanks
<wallyworld> that's what i get for rushing
<thumper> wallyworld: you back?
<wallyworld> yep
<wallyworld> haven't left yet
<thumper> chat?
<wallyworld> sure
<axw> wallyworld: https://github.com/wallyworld/juju/pull/21
<wallyworld> looking
<wallyworld> axw: gh not letting me comment. but i think we need allcons[name] = cons after cons.Pool = defaultPool
<axw> wallyworld: ah yeah, I'll fix it
<wallyworld> ta
<axw> wallyworld: updated
<wallyworld> thanks, merging
<thumper> night all
<wallyworld_> menn0: in case i don't catch you later, see you in capetown
<wallyworld_> axw: small problem to add to todo list - storage constraints reject pool names with "_" eg fast_ebs
<axw> wallyworld_: yeah, we need to decide what's a valid name. I just copied what was valid for service names
<wallyworld_> np, i'll add to spreadsheet
<wallyworld_> axw: i can't get ebs volumes with provisioned iops to work - the ec2 instance doesn't attach any ebs volumes and doesn't finish coming up in AWS
<axw> wallyworld_: just about to propose UML changes, will take a look in a sec
<wallyworld_> that's using volume-type = provisioned-iops
<wallyworld_> and iops = 2000 or whatever
<wallyworld_> i blame aws
<mattyw> folks - is it possible to change the juju admin secret in a deployed environment?
<mattyw> also - morning
<wallyworld_> not sure
<axw> wallyworld_: there is a ratio between size and IOPS that you need to adhere to, could be that
<wallyworld_> ah ok
<axw> wallyworld_: http://reviews.vapour.ws/r/828/
<axw> morning mattyw. I don't think so, but not 100% sure
<axw> wallyworld_: I'm going to $$__JFDI__$$ for hopefully obvious reasons :)
<wallyworld_> yeah, np
<wallyworld_> i ned to land the maas fix also
<wallyworld_> i'm off for a bit - going to computer shop to get network card
<axw> later
<axw> will let you know how I go with IOPS
<mattyw> axw, morning - I was in the process of bringing the env up - so I decided to just destroy and start again
<wallyworld_> axw: anastasiamac: i think we should enhance storage list to display the provider type
<anastasiamac> axw: wallyworld_: sounds good but let me merge my stuff 1st
<anastasiamac> axw: wallyworld_: i can add it next :D
<wallyworld_> sure :-)
<axw> wallyworld_: provider type, or pool?
<axw> wallyworld_: I was thinking pool...
<wallyworld_> axw: both maybe. as a user, i'd like to see provider type as well
<axw> wallyworld_: I'd hope the pools are named to make it obvious which provider is involved
<wallyworld_> that's up the the user :-)
<axw> well if you know what pool to select when you deploy... you should also know what it is when you list them
<wallyworld_> supporting "_" would help
<axw> that only holds if the person listing and deploying are the same tho
<axw> wallyworld_: why? you can use "-"
<wallyworld_> i agree in principal, but yes, as you just said
<wallyworld_> true
<wallyworld_> i guess i'm an old python guy at heart
<axw> :)
<axw> wallyworld_: https://github.com/wallyworld/juju/pull/23
<axw> wallyworld_: did you need any other docs or anything for next week, or is that UML update sufficient?
<axw> wallyworld_: btw I just confirmed that with a pool having iops=3000, I can deploy with a 100GiB volume
<axw> otherwise the block device mapping seems to get silently eaten up...
<axw> that branch just beefs up the validation
<mattyw> davecheney, ping?
<davecheney> mattyw: ack
<dimitern> voidspace, jamestunnicliffe, TheMue, fix for bug 1416134 - please take a look http://reviews.vapour.ws/r/830/
<mup> Bug #1416134: Unable to override network-bridge if container type is kvm (local provider) <cloud-installer> <config> <lxc> <network> <regression> <cloud-installer:Confirmed
<mup> for adam-stokes> <juju-core:In Progress by dimitern> <juju-core 1.21:Triaged> <juju-core 1.22:Triaged> <https://launchpad.net/bugs/1416134>
<dimitern> voidspace, http://reviews.vapour.ws/r/831/
<dimitern> TheMue, http://reviews.vapour.ws/r/832/
<dimitern> thanks for the reviews voidspace and TheMue!
<TheMue> dimitern: yw
<perrito666> morning
<perrito666> wallyworld_: still here
<perrito666> ?
<wallyworld_> hi, about to sleep, i catch a taxi soon
<perrito666> wallyworld_: the short answer is tounit <-- testing purposes
<wallyworld_> sure, but i think the function signature is wrong
<perrito666> you mean my findEntity?
<wallyworld_> yeah - why not have it return a StatusSetter
<perrito666> wallyworld_: because it needs to satisty an interface to be used by common statusetter?
<wallyworld_> right, so change that
<wallyworld_> the operation is to call SetStatus
<wallyworld_> the different is how the thing to set status on is found
<wallyworld_> for machines and units, shared logic can be used
<wallyworld_> for unit agents, the Agent() method is used
<wallyworld_> anyways, have a think about it, i gotta get a few hours sleep before i catch th plane
<perrito666> go, have a nice trip
<wallyworld_> i might be off base too, so see if it makes sense looking at the code
<perrito666> we can re-discuss this via mail whenever you are more awake, cheers
<wallyworld_> ty, see you later
<voidspace> anyone know anything about the proxyupdateworker?
<voidspace> wallyworld_: ping if you're still awake at this horrendous hour
<fwereade_> voidspace, might do a little, I touched it semi-recently
<fwereade_> voidspace, I am 80% convinced that it's fundamentally dangerous
<fwereade_> voidspace, but it was a reduce-duplication driveby, not a make-tings-right
<voidspace> fwereade_: so this worker updates the .juju-proxy file with the http(s) (etc) settings from the config
<fwereade_> voidspace, yeah
<voidspace> fwereade_: so these values are propagated from environments.yaml into .juju-proxy and /home/ubuntu/.profile is updated to source this file
<fwereade_> voidspace, it's the env vars that that http module uses that worry me
<voidspace> fwereade_: however the environment variables are never set in the jujud process
<voidspace> fwereade_: so http.DefaultClient (which uses these environment variables) does not see the proxy settings
<voidspace> fwereade_: so if they are needed (which is usually why they are set...) everything fails
<voidspace> fwereade_: so my intention is that the proxyupdater should also set the environment variables for the process
<voidspace> fwereade_: https://bugs.launchpad.net/juju-core/+bug/1403225
<mup> Bug #1403225: charm download behind the enterprise proxy fails <cloud-installer> <deploy> <proxy> <sync-tools> <cloud-installer:Confirmed for adam-stokes> <juju-core:Triaged by mfoord> <https://launchpad.net/bugs/1403225>
<fwereade_> voidspace, proxyupdater.go:151?
<perrito666> lol, just found in the code
<perrito666> 	// TODO(fwereade) GAAAAAAAAAAAAAAAAAH this is LUDICROUS.
<fwereade_> :)
<voidspace> fwereade_: ah right, but only the first time - right
<voidspace> fwereade_: interesting. I can attach to jujud and see that they are *not* set.
<fwereade_> voidspace, I think that says "always on first call or if they've changed since last time"?
<fwereade_> voidspace, best guess is that jujud changes somehow stopped it from being started properly?
<voidspace> fwereade_: hmmm... indeed. Well, it doesn't seem to be "working". I'll do some debugging. Thanks.
<fwereade_> voidspace, I thought I had tests for that, but possibly I screwed them up
<voidspace> no problem, I know where I'm looking now - so it should be easy from here ;-)
<sinzui> jam, dimitern: We have a licensing bug to address. bug 1416425 which has an easy fix I think
<mup> Bug #1416425: src/bitbucket.org/kardianos/osext/LICENSE is wrong <licensing> <packaging> <juju-core:Triaged> <juju-core 1.21:Triaged> <juju-core 1.22:Triaged> <https://launchpad.net/bugs/1416425>
<sinzui> jam dimitern: Can you ask someone to address the licensing bug
<fwereade_> voidspace, (also: it's started in both the unit agent and the machine agent, please check both)
<voidspace> fwereade_: yep
<sinzui> also dimitern , or anyone, why did the license to juju/cmd change?
<dimitern> sinzui, i'll have a look
<dimitern> sinzui, so is this about using rev 44140c5 in dependencies.tsv for bitbucket.org/kardianos/osext - in 1.23, 1.22, and 1.21 ?
<sinzui> dimitern, yes, tip has the fix
<perrito666> fwereade_: ftr, the ensuing line is indeed ludicrous
<perrito666> cmd := exec.Command("go", "build", "github.com/juju/juju/cmd/jujud")
 * fwereade_ knows :(
<dimitern> sinzui, ok, I'll prepare a fix for it
<dimitern> oh boy.. I found a bug with retry-provisioning
<dimitern> using it on a failed container in EC2 caused a new *instance* to be provisioned for "0/lxc/0"
<dimitern> how do you like this http://paste.ubuntu.com/9957564/
<sinzui> dimitern, Sorry, several more licensing bugs were found. Can you find people to look into the issues. they all affect 1.23, 1.22, and 1.21 https://bugs.launchpad.net/juju-core/+milestone/1.21.2
 * sinzui return to the broken publish-revision job
<dimitern> sinzui, I can fix the osext one, but unfortunately we're close to finishing the sprint here and somebody else can take over
<dimitern> alexisb, hey, are you around?
<coreycb> does anyone have any tips on getting around a "virbr0": net: no such interface error on bootstrap of local provider on 1.21.1?
<dimitern> coreycb, do you have virbr0 on your machine?
<coreycb> dimitern, nope
<coreycb> dimitern, I know I could create it, but wondering more about if my juju config is wrong
<dimitern> coreycb, and you're using container: kvm ?
<coreycb> dimitern, yes
<dimitern> coreycb, looks like you're affected by bug 1416134 which I just fixed this morning :)
<mup> Bug #1416134: Unable to override network-bridge if container type is kvm (local provider) <cloud-installer> <config> <lxc> <network> <regression> <cloud-installer:Confirmed for adam-stokes> <juju-core:In Progress by dimitern> <juju-core 1.21:Fix Committed by dimitern> <juju-core 1.22:Fix Committed
<mup> by dimitern> <https://launchpad.net/bugs/1416134>
<coreycb> dimitern, sweet, I think :)
<coreycb> dimitern, thanks
<dimitern> coreycb, in the mean time you can either create a virbr0 or use "network-bridge" setting but lxcbr0 there won't work due to that bug
<coreycb> dimitern, ok, thanks
<dimitern> np :)
<dimitern> voidspace, http://reviews.vapour.ws/r/833/, http://reviews.vapour.ws/r/834/, http://reviews.vapour.ws/r/835/ PTAL :)
<bodie_> natefinch around?
<perrito666> bodie_: no natefinch today
<menn0> sinzui: ping
<sinzui> hi menn0
<menn0> sinzui: so it's hard to know if I've fixed the CI blocker because we seem to be hitting test timeouts for the i386 and ppc unit test runs
<menn0> sinzui: I know my recent changes added a few seconds to the cmd/jujud/agent test run (on my machine at least)
<menn0> sinzui: I wonder if we were close to the 20min threshold on the i386 and ppc CI machines and I've just tipped it over?
<sinzui> menn0, I am not user. but looking at the FAIL: in the tests, these are very different from what we usually see
<sinzui> menn0, and the ppc test are run very quickly
<menn0> sinzui: run-unit-tests-trusty-ppc64el and run-unit-tests-precise-i386 seem to be hitting timeouts I don't see fails there
<sinzui> menn0: the fact that we are still seeing panic means something is wrong. We know that some tests will fail, but pass when retried, but we don't see panics
<menn0> sinzui: since the last fix was committed the only panics I see are about tests taking too long
<sinzui> menn0, the 386 tests  ridiculously slow. we cannot get  a fast cloud images
<menn0> sinzui: can you point me at another panic?
<sinzui> menn0, http://data.vapour.ws/juju-ci/products/version-2286/run-unit-tests-trusty-ppc64el/build-2281/consoleText
<menn0> sinzui: that's because "panic: test timed out after 20m0s"
<menn0> sinzui: go test triggers a panic because the timeout set for the test run was exceeded
<sinzui> menn0, but about that 386. We have a fast trusty 386 machine, but the tests don't pass because of two issues. I reported a bug about this. If the two tests are fixed or *skipped* you will see a passs http://data.vapour.ws/juju-ci/products/version-2286/run-unit-tests-trusty-i386/build-164/consoleText
<sinzui> bug 1408459
<mup> Bug #1408459: pingerSuite tests consistenly fail on trusty 386 <ci> <i386> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1408459>
<menn0> sinzui: ok, I see that and we need to fix that. but that's an unrelated issue to the current CI blocker.
<menn0> sinzui: what I'd like is if we could bump up the test timeout to 25 min or something to see if the panics related to the CI blocker have been resolved.
<sinzui> menn0,the machine is healthy
<sinzui> and I can see it doesn't need a updated
<sinzui> menn0, and we can see that 1.21 and 1.22 pass
<sinzui> menn0, so I think master + gccgo have a problem that 1.21 doesn't
<menn0> sinzui: I'm not saying there's anything wrong with the machine. I think that the addition of extra tests in trunk recently has pushed us over the timeout 20min timeout for the machine agent tests.
<sinzui> menn0, the whole suite when it was passing take 37 to 44 minutes to run
<sinzui> menn0, this is the last health mast run http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/run-unit-tests-trusty-ppc64el/2263/
<sinzui> disregard the tar timestamp issue. I fixed that this morning
<menn0> sinzui: ok, that's useful
 * menn0 checks how the timeout works
<sinzui> oh good, because I am running out of thinks to look for
<jw4> sinzui: are there any plans to get the tests working on windows and osx ?
<menn0> sinzui: ok... so i'm pretty sure the timeout is per Go package and it's looking like on i386 and ppc64 the cmd/jujud/agent package is now taking over 20mins
<sinzui> jw4 yes We test the client ever revision and we run windows units every revision http://reports.vapour.ws/releases/2286
<menn0> sinzui: that's a massive jump from the last successful runs
<menn0> sinzui: so it's not just tests taking a little bit longer... something is stuck
<menn0> sinzui: but these tests pass fine on amd64
<sinzui> menn0, yep
<menn0> sinzui: am I ok to use those hosts later on today
<menn0> ?
<menn0> sinzui: I think I still have the details to access the ppc64 host that you gave me some time ago
<sinzui> jw4, when there is a way to for juju to provision a windows machine and deploy an agent we will add that test to ever revision
<jw4> sinzui: cool.  That link doesn't work with my ubuntu credentials, probably because I'm not a cougar (<-- is that even a thing ? ;) )
<sinzui> menn0, oh, you my have visited a different machine even if you had. these tests are run on stilson-08. I can add your ssh keys if needed
<jw4> sinzui: I'm just curious how they're being run because on the surface they're not even close to running on my windows and osx machines
 * menn0 checks
<sinzui> jw4, only canonical staff can see the results, but the raw results are public
<jw4> sinzui: kk - tx
<menn0> sinzui: that's what I have access for and I can still get in
<sinzui> jw4, the last 2 tests are windows client and units http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/
<sinzui> fab
<menn0> sinzui: access to the i386 host might be useful too if that's possible
<sinzui> menn0, we spin up an instance for that
<jw4> sinzui: sweet - thanks again
<menn0> sinzui: ok, I can spin up my own then.
<sinzui> menn0, but I can get you acesss to the machine we want to run tests on. again, about that 386 bug. when those tests pass, we will remove the old 386 job
<menn0> sinzui: ok
<jw4> sinzui: hmm the last 'successful' run of win-client-deploy doesn't seem to be actually successful : http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/win-client-deploy/1381/console
<sinzui> jw4, looks like a successful bootstrap to me
<jw4> sinzui: it probably is - I just see a strange error about 5 lines up from the bottom
<sinzui> jw4, The SUCCESS was on the win machine the other lines are from ssh and the test running. we don't what them cursing the win success
<sinzui> and ssh did bugger up again
<jw4> sinzui: I see - also I noticed that the run-unit-tests-win2012-amd64 is getting the same '/usr/lib/juju/bin/mongod' file does not exist error that I'm getting on my windows tests
<menn0> sinzui: do you have any details or a script for spinning up the precise and trusty i386 test hosts?
<menn0> sinzui: I want to make sure I replicate as closely as possible
<sinzui> menn0, juju-ci-tool/run-unit-tests m1.medium ami-81dee0e8
<menn0> sinzui: perfect. thanks. i've got a few things to take care of right now but I'll jump back on this today.
<sinzui> menn0, aws is the only provider that permits 386 and they limit it to a slow machine, and aws is phasing out 386, which is why we need to switch to our special machine
<menn0> sinzui: got it
#juju-dev 2015-01-31
<menn0> long shot I know but are there any core developers around?
 * davecheney waves
<menn0> davecheney: any change you could review a fix for one of the CI blockers?
<menn0> davecheney: http://reviews.vapour.ws/r/839/
<menn0> davecheney: it's tiny.
<menn0> davecheney: i'm going to talk to thumper about making Factory.MakeEnvironment do this by default
<davecheney> menn0: done
<menn0> davecheney: the comment does match !
<menn0> davecheney: MakeEnvironment creates an env and then the following lines wait for workers to start
<menn0> davecheney: remove the blank line?
<davecheney> what does the Close() do
<davecheney> it looks to me, as a naieve reader of the code that it's expecting some side effect behavior
<eagles0513875_> hey guys im just wondering does the windows version of juju support manual provisioning to a particular server ip?
<eagles0513875_> or is that something i would need to go through the documentation and configure
<eagles0513875_> hi jog
<eagles0513875_> jonmasters:
#juju-dev 2015-02-01
<eagles0513875_> hey all
#juju-dev 2016-02-01
<thumper> wallyworld, axw: if we are bumping all the API versions and hacking at the wire protocol with a machette, can you please add explicit serialization directives to everything on the wire?
<wallyworld> we can try
<thumper> surely this is just a sed script right?
<wallyworld> :-(
<thumper> because sed is all you need
 * wallyworld growls at thumper 
 * thumper hums the "all you need is love" tune...
<thumper> all you need is sed... sed...
<thumper> sed is all you need
<wallyworld> groan
 * thumper is distracting himself from what he really needs to be doing
<mup> Bug #1540076 changed: Storage Hooks before Install <juju-core:Invalid> <https://launchpad.net/bugs/1540076>
 * thumper is wondering when master will get another CI run...
<thumper> last was several days ago...
<thumper> like four
<thumper> the dep engine merge hasn't been blessed or cursed yet...
<wallyworld> thumper: CI is paused over the weekend, should get another run later tonight our time. i'm keen to see it get a run because after i merged the dep endgine stuff into the api-command-rename feature branch CI reports intermittent leadership failures which i claim have nothing to do with renaming some methods
<axw> wallyworld: was out, Charlotte's first day of grade one. responded to your email
<wallyworld> np, hope she went ok
<axw> yep
<axw> wallyworld: I think one use-case for adding volumes after deploy is, e.g. if you have a MAAS and add a disk to the machine
<axw> wallyworld: MAAS has no agent, so you have no other way of discovering it
<wallyworld> that is true, but that then becomes an add storage thing maybe
<wallyworld> needs thought as to how it all fits the model
<axw> wallyworld: how do you mean? add-storage would always be required, to associate the volume with a unit
<wallyworld> exactly, that's after a unit is deployed, you add a volume, possibly one you added post deploy to the machine
<wallyworld> but at deploy time, i'm not sure it makes sense to have to have storage and placement directives macthing
<wallyworld> maybe i'm wrong
<axw> wallyworld: well if you have the machine already, then you wouldn't be able to deploy a unit that *requires* storage if you only have that OOB-added disk on the machine
<axw> wallyworld: anyway, this is just speculation, needs more requirements
<wallyworld> yep
<wallyworld> thumper: can you let me know if you have 5 minutes for a chat/question
<thumper> wallyworld: sure, now is as good as any
<wallyworld> ok, 1:1
<davecheney> thumper: it works!
<davecheney> root@ip-10-251-40-215:~# echo -e "GET /debug/pprof/goroutine?debug=1 HTTP/1.0\r\n" | socat unix-connect:/tmp/.5636.pprof STDIO | pastebinit
<davecheney> http://paste.ubuntu.com/14847387/
 * thumper took a minute disecting that command
<thumper> very cool
<davecheney> yeah, it's a mouthful
<thumper> davecheney: what are the lines like this: 2 @ 0x437b2a 0x446382 0x445372 0x713277 0x737040 0x464d41
<davecheney> most http tools are like "unix socket, wat?!"
<davecheney> those are the raw program counters
<thumper> ah
<thumper> is there memory dumps?
<davecheney> here is what the comp[uter see, http://paste.ubuntu.com/14847399/
<thumper> ah
<davecheney> the extra lines are for humans, see the ?debug=1 flag
<thumper> much nicer to see the full method
<thumper> what about memory?
<thumper> can we see that?
<davecheney> http://paste.ubuntu.com/14847418/
<davecheney> i don't immediately know how to interpret those numbers
<davecheney> but pprof can
<davecheney> i've never found the go heap profiler to tell me useful information
<davecheney> i wanted to implement this facility so I could find out what the goroutines were doing inside a program
<davecheney> http://paste.ubuntu.com/14847633/
<davecheney> friday the juju deployer worked fine for me
<davecheney> today, nothing
<thumper> see y'all tomorrow
<jog> davecheney, juju-deployer==0.3.6 is an old version of deployer. You probably want 0.6.1-1 from 'deb http://ppa.launchpad.net/juju/stable/ubuntu trusty main'
<mup> Bug #1540047 changed: Error in Jujuclient with Python3 <python-jujuclient:Fix Released by aisrael> <https://launchpad.net/bugs/1540047>
<mup> Bug #1540047 opened: Error in Jujuclient with Python3 <python-jujuclient:Fix Released by aisrael> <https://launchpad.net/bugs/1540047>
<mup> Bug #1540047 changed: Error in Jujuclient with Python3 <python-jujuclient:Fix Released by aisrael> <https://launchpad.net/bugs/1540047>
<frobware> jam: ho? (though I have no camera today.)
<jam> be there in just a sec
<dimitern> voidspace, frobware, fwereade__, standup?
<voidspace> dimitern: omw
<voidspace> dimitern: got booted out - rejoining
<dooferlad_> voidspace: are you rejoining?
<dooferlad> ok, 2fa for me.
<dooferlad> yay, in the middle of a hangout!
<voidspace> dimitern: jam: dooferlad: not sure if any of my last messages got through
<voidspace> dimitern: jam: dooferlad: internet went down briefly
<voidspace> dimitern: jam: dooferlad: dependency engine stuff may mean I get to simplify my tests a bit, I think that was all I wanted to say
<dooferlad> voidspace: we didn't get that bit. sounds goodf.
<wallyworld> fwereade__: hey, did you have 5 minutes, i just wanted to run something by you
<dimitern> voidspace, that sounds like a good next step, yeah
<mup> Bug #1540301 opened: adding storage to a unit should be able to take a specified block device <storage> <juju-core:Triaged> <https://launchpad.net/bugs/1540301>
<mup> Bug #1540316 opened: provider/maas: bridgescript should support vlans <maas-provider> <juju-core:New> <https://launchpad.net/bugs/1540316>
<axw> wallyworld: might be a minute or two late
<wallyworld> ok
<davecheney> jog: i'm running the version of hte juju-deplouyer that ships in trusty
<davecheney> and it worked right up til friday
<davecheney> but this week it just decided not to work
<wallyworld> perrito666: meeting?
<cherylj> fwereade__: ping?
<mattyw> wwitzel3, you around mate?
<mattyw> cherylj, don't suppose you know anyone who might be awake and knows about the lxd provider, I'm having problems
<wwitzel3> mattyw: ping
<mattyw> wwitzel3, hey hey, you worked on the lxd provider?
<wwitzel3> mattyw: yep, ericsnow and myself
<mattyw> wwitzel3, awesome, I'm having problems with it (it was working fine) I've almost finished typing it all out in an email I was going to put on juju-dev email list. If it's ok I'll finish that and ping when it's sent. I basically need help debugging and env that seems screwed
<mattyw> wwitzel3, I'll finish off the email to save time typing it all out again
<wwitzel3> mattyw: yeah, ok, I'm not on the list anymore, so if you want to also pastebin it to me, I'll give it a read and see if I have any insight
<mattyw> wwitzel3, I've emailed you as well
<wwitzel3> that works
<tasdomas> menno, ping?
<wwitzel3> mattyw: I'd also ping ericsnow and direct him to the list message, he might have some insight on it too .. but yeah, I'll poke it a bit more and see if anything shakes out
<mattyw> ericsnow, are you around?
<mattyw> I wish irc displayed timezones for everyone
<wwitzel3> mattyw: he'll be around soon (if he isn't out of office)
<wwitzel3> mattyw: moonstone standup is in 30 minutes
<frobware> mgz, did we ever get a dedicated 1.25 CI run for bug 1532167  - this was the backport of the maas-spaces bridge script
<mup> Bug #1532167: maas bridge script handles VLAN NICs incorrectly <addressability> <maas-provider> <network> <juju-core:Triaged> <juju-core 1.25:In Progress by frobware> <https://launchpad.net/bugs/1532167>
<mup> Bug #1540394 opened: Juju does not handle 429 "too many requests" from Azure <azure-provider> <bootstrap> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1540394>
<mup> Bug #1540394 changed: Juju does not handle 429 "too many requests" from Azure <azure-provider> <bootstrap> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1540394>
<mup> Bug #1540394 opened: Juju does not handle 429 "too many requests" from Azure <azure-provider> <bootstrap> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1540394>
<ericsnow> mattyw: present, but about to go to standup
<ericsnow> mattyw: I'll ping you when we're done
<mattyw> ericsnow, ok cool
<ericsnow> mattyw: nevermind, I'm the only one around apparently
<ericsnow> :)
<mattyw> ericsnow, take a look at my email to juju-dev for the strarting point
<ericsnow> mattyw: will do
<katco`> sinzui: ping
<katco`> or, mgz: ping
<sinzui> hi katco`
<katco`> sinzui: o/
<katco`> sinzui: at the charmer summit, have a person here trying to try out juju on joyent
<katco`> sinzui: and he's getting a strange error on bootstrap
<katco`> sinzui: (one sec getting err)
<sinzui> katco`: joyent has several ill regions over the last few weeks. us-east-3 is good. us-east-2 is locked down. us-east-1 has network issues, us-west-1 is not cleaning up networks
<katco`> sinzui: we're using eu regions
<katco`> sinzui: 2016-02-01 15:27:04 ERROR juju.cmd supercommand.go:429 there was an issue examining the environment: cannot create credentials: An error occurred while parsing the key: asn1: structure error: tags don't match (16 vs {class:0 tag:23 length:23 isCompound:true}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false} pkcs1PrivateKey @2
<katco`> sinzui: the environments.yaml looks like it could be ok, but wouldn't completely rule that out
<sinzui> katco`: I don't think someone setup their account keys properly. CI has never seen that kind of issue
<katco`> sinzui: so we checked the keys in his account, and they match what's in environments.yaml
<katco`> sinzui: i've not used joyent. what kind of setup could be wrong in this case?
<sinzui> katco`:  I've never seen an error like that to day
<sinzui> say
<katco`> sinzui: hm ok
<katco`> sinzui: manta information should be the same as sdc?
<sinzui> katco`: my config looks like this https://pastebin.canonical.com/148857/
<katco`> sinzui: ahhh possibly what's wrong. the key he posted looks possibly like a fingerprint
<sinzui> katco`: yes, sdc-key-id and anta-key-id are the same, as ar sdc-user and manta-user.
<katco`> sinzui: scratch that, he's specifying the path to the key
<sinzui> katco`: private-key-path works, though I don't use it often
<katco`> sinzui: he's on os x if that helps
<sinzui> katco`: I haven't seen a difference between OS X and Ubuntu for joyent. ssh and keys are the same.
<katco`> sinzui: hm ok
<sinzui> katco`: unless someone trued to compile their own juju on that host. OSX requires CGO=1 to link to the native keystore
<katco`> sinzui: nah he just brew installed
<katco`> sinzui: fyi when he tried to brew install alpha1, he got an error
<sinzui> katco`: we don't support it
<katco`> sinzui: looked like the brew recipe had some kind of utf error
<katco`> sinzui: ah ok
<sinzui> katco we onlt support jujus in *releaed* streams. other wise peoples configs break when they get updates
<katco`> sinzui: right, we're just showing alpha1 here to show everyone the new stuff
<katco`> sinzui: but seems like a bug in the brew recipe for 2.0? or are you saying we don't do the support for brew?
<sinzui> katco`: not a bug is the recipe, it is about streams not working at the time the binaries are meade
<katco`> sinzui: no, i'm saying the recipe seems broke. as in brew install doesn't work for alpha1
<sinzui> katco`: we did suport devel, users were not happy about incompatible configs
<sinzui> katco`: We solved this by creating cient for testers https://launchpad.net/juju-core/+milestone/2.0-alpha1 got clients *before* ubuntu users
<katco`> sinzui: right, that's what he ended up downloading
<mup> Bug #1540437 opened: lxd provider: machines stuck at "pending" <juju-core:New> <https://launchpad.net/bugs/1540437>
<mup> Bug #1540447 opened: error: r.runner.RunCommands: tomb: dying <run> <juju-core:Incomplete> <juju-core 1.25:Triaged> <juju-core api-command-rename:Invalid> <https://launchpad.net/bugs/1540447>
<mup> Bug #1540437 changed: lxd provider: machines stuck at "pending" <juju-core:New> <https://launchpad.net/bugs/1540437>
<mup> Bug #1540447 changed: error: r.runner.RunCommands: tomb: dying <run> <juju-core:Incomplete> <juju-core 1.25:Triaged> <juju-core api-command-rename:Invalid> <https://launchpad.net/bugs/1540447>
<mup> Bug #1540437 opened: lxd provider: machines stuck at "pending" <juju-core:New> <https://launchpad.net/bugs/1540437>
<mup> Bug #1540447 opened: error: r.runner.RunCommands: tomb: dying <run> <juju-core:Incomplete> <juju-core 1.25:Triaged> <juju-core api-command-rename:Invalid> <https://launchpad.net/bugs/1540447>
<voidspace> dimitern: omw
<natefinch> morning ericsnow
<ericsnow> natefinch: hey
<ericsnow> natefinch: interested in a late standup?
<natefinch> ericsnow: yeah, definitely
<frobware> cherylj, some success with VLAN nics on 1.25 - https://bugs.launchpad.net/juju-core/+bug/1532167/comments/28
<mup> Bug #1532167: maas bridge script handles VLAN NICs incorrectly <addressability> <maas-provider> <network> <juju-core:In Progress by frobware> <juju-core 1.25:In Progress by frobware> <https://launchpad.net/bugs/1532167>
<frobware> cherylj, this change was also going to get a CI branch as well. I haven't seen any reports. The PR is good to go, just waiting on sanity CI run.
<voidspace> dimitern: frobware: do we still have the networking thing this afternoon?
<frobware> voidspace, yes.
<voidspace> frobware: thanks
<cherylj> frobware: okay, I had held off since the initial feedback showed it had problems.  I'll get it on the queue
<frobware> cherylj, thanks.
<dimitern> voidspace, we do, but I might skip it
<mup> Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <juju-core:New> <https://launchpad.net/bugs/1540469>
<voidspace> dimitern: ok
<voidspace> dimitern: frobware: dooferlad: unless anyone has any questions/follow up for Jay, maybe we should skip it this week?
<dimitern> voidspace, +1 from me
<frobware> voidspace, dimitern, dooferlad: agreed. need to merge :)
<voidspace> I've emailed
<dimitern> cheers voidspace
<voidspace> dimitern: can I tell the difference, on a buffered channel as I *do not* want to block, between a closed channel and a channel with no messages available
<voidspace> dimitern: I'm starting to think that the bool is just easier
<voidspace> dimitern: I think I can
<voidspace> dimitern: unping
<natefinch> voidspace: a closed channel never blocks
<natefinch> voidspace: an empty channel always blocks
<voidspace> natefinch: and I need to tell the difference between those two cases
<voidspace> natefinch: but I think I can use the ok parameter for that
<voidspace> (thanks)
<voidspace> natefinch: ah, I misread
<voidspace> natefinch: I want a non-blocking way to tell if a channel is closed or not
<voidspace> natefinch: just reading from it isn't enough as it will block if not closed
<voidspace> natefinch: but I can provide a default path in the select for that case
<natefinch> voidspace: yeah, was goign to say, with a default you get the non-blocking, as long as you're ok with actually taking a value off the channel if there's something there
<natefinch> voidspace: also, for a := range ch {    works really nicely if you're looping anyway
<voidspace> natefinch: thanks
<voidspace> natefinch: in my case I'm not reading from the channel, just need to know if it's closed
<natefinch> voidspace: You can't tell if it's closed without trying to read from it.. the notification that it's closed is a value you read off the channel.... Why do you need to know if a channel is closed without reading off it?
<voidspace> natefinch: I thought it was a common pattern to use a channel to signal something - the signal being that you close the channel
<voidspace> natefinch: reading from the closed channel then doesn't block - which is how you know it's closed
<natefinch> voidspace: yes
<voidspace> natefinch: the problem was that attempting a read would block, and I need the check to be non-blocking
<voidspace> natefinch: but that's easy enough with a default path in the select
<voidspace> natefinch: so no problem :-)
<natefinch> voidspace: yep
<dimitern> voidspace, dooferlad, frobware, here it is: http://reviews.vapour.ws/r/3693/
<voidspace> dimitern: cool, mine is nearly ready for review too
<dimitern> in the mean time I'm still finishing a few tests
<voidspace> dimitern: dooferlad: frobware: http://reviews.vapour.ws/r/3694/
<voidspace> dimitern: looking at your review
<dimitern> voidspace, ta, looking at yours
<voidspace> dimitern: switching to a channel meant undoing a bunch of code, but it is nicer
<dimitern> voidspace, I suspected as much ;)
<frobware> dimitern, taking a look now; would some manual testing help at all?
<dimitern> frobware, sure, in fact what I'm seeing now is that it seems to work live (bindings included), but I still have hard time fixing the unit tests
<frobware> dimitern, how many unit tests are broken? what's the nature of the breakage? and depending on how many are to be addressed could these be done post the merge as collateral damage?
<dimitern> frobware, so the biggest issue is one of the apiaddressupdater tests hang
<dimitern> frobware, the rest are mostly trivial - e.g. endpoint_bindings_test.go, service_test.go - related to incomplete test setup or outdated asserts to fix
<frobware> dimitern, for the test that hangs is this something the CI test would run into?
<dimitern> frobware, well, yeah - the merge bot will hang
<dimitern> frobware, I'm close to actually skipping that test for now
<frobware> dimitern, right. :)
<frobware> dimitern, ok, another way. if we skip the test, is there likely to be a genuine functional failure that CI would run into.
<dimitern> frobware, I don't believe so
<frobware> dimitern, so we could throw caution to the wind here.
<frobware> dimitern, cherylj: another way is to get a CI run on your branch before we merge to maas-spaces
<dimitern> frobware, the test in question is related to the legacy address selection by scope followed by filtering of lxcbr0 addresses, both of which are skipped on 1.9 with controller space used for selection instead
<frobware> dimitern, as an aside just found an interesting problem with lxcbr0
<dimitern> frobware, that's perfect, if it can be arranged
<frobware> cherylj, ^^
<frobware> dimitern, is your branch now rebased against maas-spacs? I know on friday you were holding off
<dimitern> frobware, I rebased before proposing, that's why it took some time - a couple of conflicts - easy to resolve though
<voidspace> dimitern: frobware: manual test on my branch works fine by the way
<dimitern> voidspace, I'm still looking, but it will take some time as I have a few suggestions
 * cherylj reads scrollback
<voidspace> dimitern: ok
<natefinch> ericsnow: so, for deploy --resources, if we're doing multiple resources as a single string, space separated, how do we handle paths with spaces in them?  like --resources "foo=./bar foo2=./my docs/baz"?
<natefinch> rick_h__: ^^
<ericsnow> natefinch: --resources foo="./bar" foo2="./my docs/baz"
<ericsnow> natefinch: or  --resources "foo=./bar" "foo2=./my docs/baz"
<ericsnow> natefinch: or  --resources foo=./bar "foo2=./my docs/baz"
<voidspace> dimitern: are you passing controller space straight through to maas as a constraint at bootstrap time
<ericsnow> natefinch: any of those should work
<natefinch> ericsnow: that's different than how constraints work, though
<rick_h__> natefinch: understand, it'll have to be in order to be cross platform/etc. None of the others have actual directories in there like that.
<ericsnow> natefinch: if this is not a problem that constraints has then we'll probably have to do it differently than constraints, no?
<rick_h__> natefinch: constraints are built by us so we got to curate what was in there. I'd be curious if you can create a space or storage pool with a space in the name?
<rick_h__> guessing not
<natefinch> ericsnow, rick_h__ : it's also sorta hard to parse.... juju deploy mysql --resources foo=bar mydatabase  is mydatabase the name of the service or a resource?
<ericsnow> natefinch: you'd have to use quotation marks
<dimitern> voidspace, not exactly - it's a bootstrap argument which if unset will be inferred and set during bootstrap (see the finalizerWrapper)
<natefinch> ericsnow: no, that could be legal for deploying mysql with the servicename mydatabase
<natefinch> ericsnow: or it could be a typo for a resource called mydatabase without a path
<natefinch> my point is --flag item1 item2 item3   is really hard to parse unless you guarantee it has to be the last thing on the line
<natefinch> which is probably why constraints have to be a single string
<natefinch> ericsnow: looks like gnuflag doesn't even support that kind of thing
<ericsnow> natefinch: so we use a comma for a separator?
<natefinch> ericsnow: that or semicolon.... I think we use semicolon somewhere else....
<natefinch> (although now I can't find where... maybe I'm crazy)
<natefinch> ericsnow: oh, well, loggo uses semicolons *shrug*
<voidspace> dimitern: right, but is it used to select the space to bootstrap *to*?
<voidspace> dimitern: (as well)
<frobware> cherylj, ping
<dimitern> voidspace, no, that's what bootstrap constraints are for
<cherylj> frobware: in mtg, give me a few
<frobware> cherylj, ack
<voidspace> dimitern: so you could set controller-space and end up with an apiserver not in the controller space?
<dimitern> voidspace, and in fact if you specify --controller-space admin-api, you'll get a node there, as after validating the controller space we also update the bootstrap constraints to match
<voidspace> dimitern: ok, so the controller-space *is* turned into a bootstrap constraint?
<voidspace> dimitern: (that's what I was really asking - but I can't see the code that does that)
<voidspace> dimitern: because if so we have an issue with the space name
<dimitern> voidspace, but if you don't specify --controller-space, you'll get arbitrary node, from which we'll then infer contr.space and also set env. constraints
<voidspace> dimitern: I am specifically asking about the case where you *do* specify it
<dimitern> voidspace, yeah, the issue is commented in a todo
<voidspace> dimitern: we'll have an issue with space names
<dimitern> voidspace, that we need to apply the same transformation as in discoverspaces
<voidspace> dimitern: the bootstrap contraint should be a maas space name, but we require controller space to be a juju space name
<voidspace> dimitern: right
<voidspace> dimitern: that's the point I wanted to make
<dimitern> voidspace, well, both need to be a valid juju space name
<voidspace> dimitern: but it's still a todo then?
<voidspace> dimitern: except you can't "untranslate"
<voidspace> dimitern: our name transformation is one way only
<voidspace> dimitern: you can't go from juju name back to maas name
<dimitern> voidspace, so you can't use "space=Default space" in cons or --controller-space "Default space" as well
<voidspace> dimitern: we'll have to do discovery on the client I'm afraid and then match the name
<dimitern> voidspace, you can :) that's why we discover them and keep both id and name
<voidspace> dimitern: and for the bootstrap constraint the user will have to "guess" what the juju name will be
<voidspace> dimitern: not before discovery has been done you can't
<dimitern> voidspace, there are still things to fix, I'll give you that :)
<voidspace> dimitern: and discovery hasn't been done until after machine-0 has been selected and jujud is runnign
<voidspace> *running
<dimitern> voidspace, but it's a necessary first step
<dimitern> anyway, really should be going now, perhaps back later
<voidspace> dimitern: why don't we just use provider names in state instead of having restricted juju names requiring a transformation?
<voidspace> because of binding syntax restrictions I suppose :-(
<dimitern> and because we have provider ids on maas only - i.e. not the common ground
<voidspace> dimitern: maas is the only place where we need discovery
<voidspace> dimitern: so technically it is fully common ground...
<dimitern> voidspace, I'll think about it tomorrow and I'm happy to listen to ideas how to generalize this approach
<voidspace> but anyway, this is just a distraction
<voidspace> life would be easier if we dropped the name restrictions and didn't have to do the transformation though
<voidspace> (well - easier in places, maybe harder in others)
<dimitern> yeah - mongo special chars in keys, and bad cli ux for constraints and bindings..
<dimitern> there's a better way I'm sure
<katco`> sinzui: hey, got past the problem. getting a new error "cannot read product data, invalid URL "https://us-east.manta.jpyemt.com/cpcjoyentsupport/public/juju-dist/tools/streams/v1/com.ubuntu.juju:devel:tools.json""
<katco`> sinzui: what environments.yaml thing do i tweak to make that work?
<katco`> sinzui: again he's using 2.0-alpha1
<sinzui> katco`: per the annoucenment to the juju and juju-devel lists, users of 2.0-alpha1 in aws, joyent, and azure must set "agent-metadata-url: https://streams.canonical.com/juju/tools" which restore 1.x behaviour
<katco`> sinzui: ty trying now
<katco`> sinzui: sorry for the verbose questions, in a bar trying to get this to work lol
<sinzui> katco`: :) that situation is crazy. by manually setting the streams to the default location that juju is already checking, it will find the agents in the public cloud it said it could not find.
<katco`> sinzui: ty! that seems to be working
<perrito666> cherylj: will get coffee, might be a couple of mins late depending on the expreso machine mood
<cherylj> perrito666: np :)
<perrito666> back, its so hot outside that the machine didnt need to pre-heat
<frobware> cherylj, just wanted to say that dimiter has committed changes to his dev branch only. the tests that he mentioned in the meeting still need fixing but are simply commented out and skipped for now. Is it possible to get a CI run on his branch so we can look at the net result in the morning?
<frobware> cherylj, PR -> https://github.com/juju/juju/pull/4255
<mup> Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <juju-core:Invalid> <https://launchpad.net/bugs/1540469>
<mup> Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <juju-core:Invalid> <https://launchpad.net/bugs/1540469>
<natefinch> anyone online who knows how deploying bundles works with the juju client?
<perrito666> natefinch: dont you need deployer for bundles?
<natefinch> perrito666: it's been added to the client
<mup> Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <juju-core:Invalid> <https://launchpad.net/bugs/1540469>
<natefinch> it looks like we just blindly throw away all CLI flags if you target a bundle
 * perrito666 is suddenly attacked in linkedin by solo recruiters offering the dream job, I wonder what is the algo that triggers those
<natefinch> heh
<lazypower|summit> natefinch - are you referring to the new "juju deploy" or python juju client?
<thumper> cherylj: did you want to talk through those few bugs?
<thumper> o/ lazypower|summit
<lazypower|summit> \o heyo thumper
<cherylj> thumper: give me a minute, I'm chatting with frobware
<thumper> k
<thumper> mramm: are we chatting today?
<natefinch> lazypower|summit: juju deploy
<lazypower|summit> natefinch - interesting, what flags were you passing that weren't respected? i can try to reproduce over here to confirm if you'd like
<natefinch> lazypower|summit: I haven't actually tried it, was just looking at the code
<rick_h__> thumper: he's in DE still and might be be around. after 9pm atm
<lazypower|summit> rick_h__ - while i see that you're here - code challenge one. We still cool to riff on slides tomorrow am?
<lazypower|summit> s/one/done/
<rick_h__> lazypower|summit: definitely
<lazypower|summit> <3 awesome. 'preciate you
<natefinch> lazypower|summit: basically any deploy flag at all, other than storage, looks like we just drop on the floor... which, granted, most of them don't make sense when deploying a bundle... like series or numunits or spaces... it's just surprising that I don't see code to alert the user that they've typed a command that doesn't make any sense
 * perrito666 discovers that the dual channel setup on his computer was the exact oposite than what is on the mobo manual...
<lazypower|summit> natefinch - yeah, i can see that being obtuse... Messaging would be good there.
 * lazypower|summit hattips
<lazypower|summit> i'm outy 5k for now, see yinz tomorrow
<perrito666> natefinch: it would be nice to yell one time per flag that doesnt make sense
<perrito666> so we avoid the typicall loop that users have to endure on some apps to figure out all that is wrong
<natefinch> perrito666: yeah, definitely.  but right now it won't yell at all, it just silently ignores the flags and deploys the bundle, which is far worse
<perrito666> natefinch: sounds like you just bought yourself a bug report and a task? :p
<thumper> lazypower|summit: how goes the summit?
<mup> Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <juju-core:New> <https://launchpad.net/bugs/1540469>
<cherylj> thumper: you still free?
<thumper> cherylj: briefly
<cherylj> thumper: standup HO?
<thumper> kk
<mup> Bug #1540580 opened: Broken symlink to charm breaks `juju deploy` <juju-core:New> <https://launchpad.net/bugs/1540580>
<mup> Bug #1540580 changed: Broken symlink to charm breaks `juju deploy` <juju-core:New> <https://launchpad.net/bugs/1540580>
<mup> Bug #1540580 opened: Broken symlink to charm breaks `juju deploy` <juju-core:New> <https://launchpad.net/bugs/1540580>
<frobware> cherylj, before I go... there will be no blessed build if the CI tests still exercise against MAAS 1.7 and 1.8. Just a heads-up.
<cherylj> frobware: yes, thank you :)
<natefinch> ericsnow: meeting?
<cherylj> thumper: looks like master got a bless, so you can merge it
<thumper> sweet!!
<thumper> thanks
<mup> Bug #1540647 opened: local provider: Name com.ubuntu.Upstart does not exist <kanban-cross-team> <landscape> <juju-core:New> <https://launchpad.net/bugs/1540647>
<mup> Bug #1540647 changed: local provider: Name com.ubuntu.Upstart does not exist <kanban-cross-team> <landscape> <juju-core:Invalid> <https://launchpad.net/bugs/1540647>
<mup> Bug #1540650 opened: lxd provider: juju destroy-controller hangs <juju-core:New> <https://launchpad.net/bugs/1540650>
<wallyworld> axw: anastasiamac: a minute late
<anastasiamac> k. ping when ready
<alexisb> anastasiamac, we are on bantering if you would like ot join us
<alexisb> saying horrible things about wallyworld
<alexisb> cherylj, sinzui I see that the maas-spaces branch failed
<alexisb> any thoughts what is causes the deploys tests failing
#juju-dev 2016-02-02
<cherylj> alexisb: "cannot determine machine endpoint bindings: endpoint bindings
<cherylj>       for "s#dummy-source" not found'"
<cherylj> alexisb: not sure what that means in terms of their code change, though
<alexisb> ok
<davecheney> 2016-02-02 00:47:47 ERROR juju.worker runner.go:229 exited "toolsversionchecker": cannot update tools information: cannot get latest version: cannot find available tools: cannot read product data, invalid URL "https://juju-dist.s3.amazonaws.com/tools/streams/v1/com.ubuntu.juju:released:tools.sjson" not found
<davecheney> 2016-02-02 00:47:55 ERROR juju.worker runner.go:229 exited "toolsversionchecker": cannot update tools information: cannot get latest version: cannot find available tools: cannot read product data, invalid URL "https://juju-dist.s3.amazonaws.com/tools/streams/v1/com.ubuntu.juju:released:tools.sjson" not found
<davecheney> is it _really_ necessary to poll for new tools every 8 seconds
<cherylj> juju has OCD
<davecheney> honestly ,checking once and hour would be overkill
<davecheney> thumper: https://github.com/juju/juju/pull/4259
<davecheney> pprof PR for review
<davecheney> i wish the juju deployer hadn't stopped working
<davecheney> i know people tell me that i'm using an outdated versin
<davecheney> but frankly, if we (juju) have to live in a world where we have to keep making the versino in trusty work for the next 3 years
<davecheney> then the same should apply to the deployer
<mwhudson> axw: do you know things about azure?
<davecheney>   File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 96, in get_cli_status
<davecheney>     status = super(GoEnvironment, self).get_cli_status()
<davecheney> cherylj: thumper who changed the default juju status to be _not_ yaml
<davecheney> that has broken the juju-deployer shipped in trusty
<davecheney> yup, that's the problem
<davecheney> cherylj: where should I log this bug ? against juju
<davecheney> or against the deployer that it needs to call juju status with an explicit formatter ?
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1540697
<mup> Bug #1540697: Recent change to juju status has broken compatibility with trusty juju-deployer <juju-core:New> <https://launchpad.net/bugs/1540697>
<axw> mwhudson: I do, sorry, missed your message
<mwhudson> axw: heh you answered the thread anyway
<axw> davecheney: the (informal) plan for 2.0 is that anyone who needs structured output needs to explicitly specify the format
<axw> davecheney: otherwise you get tabular, and its format might change over time
<natefinch> wallyworld, axw: git blame says you guys might know about the implementation of deploying bundles with juju core?
<axw> so long as that holds, deployer needs to be changed
<axw> natefinch: I know some, I added storage support
<mwhudson> i guess deployer in trusty needs to be sru-ed then?
<wallyworld> natefinch: francesco did the work, we tweaked as needed
<natefinch> wallyworld: is it me, or do we just drop all the flags (except storage) on the floor if you target a bundle?
<mup> Bug #1540697 opened: Recent change to juju status has broken compatibility with trusty juju-deployer <juju-core:New> <https://launchpad.net/bugs/1540697>
<wallyworld> davecheney: deployer in trusty will work with 1.25
<wallyworld> tht is our mandate
<axw> natefinch: which flags?
<wallyworld> 2.0 will be co-installable with 1.25 andf 1.25 wil be supported for 2 years
<natefinch> axw: juju deploy flags
<wallyworld> 2.0 will not be backwards compatible
<wallyworld> we will support juju in trusty via 1.25
<natefinch> axw: like --constraints -n --networks etc ... looking at the code, it seems like we woudl totally let you 'juju deploy cs:/bundle/foo/bar --constraints=blah' and not complain, but completely ignore the constraints
<natefinch> axw: it's the "not complain" part that bothers me
<axw> natefinch: looks a bit like it. I'd say because it kind of changes the meaning of deploy entirely. not one service, but multiple
<axw> natefinch: yeah
<axw> we should error if you specify them
<natefinch> axw: exactly my thinking
<axw> natefinch: I added a bundle-specific syntax to the --storage command that allows you to specify for services in a bundle. we probably should do that for other things, but erroring would be a good start
<natefinch> axw: yep.  ok, just wanted to make sure I wasn't missing anything obvious before filing a bug
<davecheney> wallyworld: ahh, ok
<davecheney> thumper: http://paste.ubuntu.com/14854594/
<wallyworld> davecheney: we shoudl try and make deployer work though if possible i think. 2.0 has deployer built in but i don't think it is yet fully functionally complete
<davecheney> this is on a modest sized deployment
<davecheney> wallyworld: it's a trivial chang to the deployer
<davecheney> but that has to be backported to trusty
<davecheney> which is not trivial
<thumper> davecheney: nice
<wallyworld> davecheney: it is - my other concern is that in 2.0 old yaml fields like agent0state are gone
<davecheney> thumper: i'm going to commit this and backport it
<davecheney> so we can investigate that failure
<thumper> ack
<wallyworld> so deployer may need subsequent tweaking
<davecheney> precise environments still have two mongodbs installed ...
<davecheney> root@ip-10-250-24-28:/tmp# ps auxww | grep mongo
<davecheney> mongodb   5462  0.6  0.9 372940 38264 ?        Ssl  01:48   0:15 /usr/bin/mongod --config /etc/mongodb.conf
<davecheney> root      6410  3.2  1.2 3026932 46808 ?       Ssl  02:28   0:01 /usr/bin/mongod --auth --dbpath /var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile /var/lib/juju/server.pem --sslPEMKeyPassword xxxxxxx --port 37017 --noprealloc --syslog --smallfiles --journal --keyFile /var/lib/juju/shared-secret --replSet juju --ipv6 --oplogSize 512
<davecheney> root      6511  0.0  0.0   8100   932 pts/1    S+   02:29   0:00 grep --color=auto mongo
<davecheney> our mongob does not restart if the process dies for any reason :(
<thumper> really?
<thumper> isn't that what upstart or systemd does?
<davecheney> kill 6410
<davecheney> process does not restart
<davecheney> root@ip-10-250-24-28:/tmp# initctl list | grep juju
<davecheney> juju-db stop/waiting
<davecheney> juju-clean-shutdown stop/waiting
<davecheney> jujud-machine-0 start/running, process 5548
<davecheney> process does not restart
<mup> Bug #1512718 changed: Leader election fails with "leadership failure: leadership manager stopped" <arm64> <landscape> <sts> <juju-core:Triaged> <juju-core 1.24:Triaged> <juju-core 1.25:Incomplete> <https://launchpad.net/bugs/1512718>
<mup> Bug #1540697 changed: Recent change to juju status has broken compatibility with trusty juju-deployer <juju-core:Invalid> <juju-deployer:New> <https://launchpad.net/bugs/1540697>
<mup> Bug #1540701 opened: juju deploy <bundle> silently ignores most CLI flags <bundles> <deploy> <juju-core:New> <https://launchpad.net/bugs/1540701>
<thumper> wallyworld: under which situation will a service not have a status doc?
<wallyworld> thumper: never; there ws a bug in an early 1.24(?) release where that could have been the case
<thumper> wallyworld: so I can assume that there will always be a status doc for services?
<thumper> for migration?
<wallyworld> yep
<thumper> awesome
<thumper> ta
<wallyworld> the status doc is written first
<wallyworld> in the []txn.Ops slice
<thumper> kk
<davecheney> thumper: master is blocekd
<davecheney> can I have your permission to land this change ?
<davecheney> thumper: and environment with 11 machines, has 51 firewall watchers running
<thumper> 51?
<thumper> wow
<davecheney> no, sorry,
<davecheney> 20 @ 0x437c2a 0x446482 0x445472 0xa3f46f 0xa40234 0xd53b36 0x464e41
<davecheney> #	0xa3f46f	github.com/juju/juju/worker/firewaller.(*machineData).watchLoop+0x43f		/home/dfc/src/github.com/juju/juju/worker/firewaller/firewaller.go:767
<davecheney> #	0xa40234	github.com/juju/juju/worker/firewaller.(*Firewaller).startMachine.func1+0x44	/home/dfc/src/github.com/juju/juju/worker/firewaller/firewaller.go:243
<davecheney> #	0xd53b36	github.com/juju/juju/worker/catacomb.Invoke.func3+0x76				/home/dfc/src/github.com/juju/juju/worker/catacomb/catacomb.go:115
<davecheney> 20
<davecheney> we do have 58 moingodb conections in use
<davecheney> http://paste.ubuntu.com/14854643/
<davecheney> it's just a bloodbatch
<davecheney> it's just a bloodbath
<davecheney> and this environment is idle
<thumper> davecheney: I've added my mark on the github PR
<thumper> davecheney: try to land it now
<davecheney> ta
<davecheney> cmd/pprof/pprof.go:61:2: cannot find package "runtime/trace" in any of: /usr/lib/go/src/pkg/runtime/trace (from $GOROOT) /mnt/jenkins/workspace/github-merge-juju/tmp.f4uCbWbBZu/RELEASE/src/runtime/trace (from $GOPATH)
<davecheney> shake's fist at Go 1.2
 * davecheney deletes things til go 1.2 stops complaining
<mup> Bug #1540316 changed: provider/maas: bridgescript should support vlans <maas-provider> <juju-core:New> <https://launchpad.net/bugs/1540316>
<mup> Bug #1540316 opened: provider/maas: bridgescript should support vlans <maas-provider> <juju-core:New> <https://launchpad.net/bugs/1540316>
<mup> Bug #1540316 changed: provider/maas: bridgescript should support vlans <maas-provider> <juju-core:New> <https://launchpad.net/bugs/1540316>
<axw> wallyworld: not quite as big as your diffs, but sorry all the same: http://reviews.vapour.ws/r/3698/diff/#
<wallyworld> revenge is sweet
<axw> on a positive note, net reduction of ~1600 lines
<wallyworld> axw: reviewed, i love seeing code deleted
<axw> wallyworld: thanks
<axw> wallyworld: there's a card for creating the second model already, is a TODO necessary?
<wallyworld> nah
<davecheney> thumper, each client of a 1.25 api server consumes 50 goroutines
<axw> wallyworld: how do you feel about this? https://github.com/juju/juju/compare/cloud-credentials...axw:cloud-credentials-detect-regions
<anastasiamac> axw: love the name! detector :D
<wallyworld> axw: looking
<wallyworld> axw: it does give a way to avoid having a personal clouds file for openstack, if standard env vars are set via sourcing a credentials file or whatever
<wallyworld> i think it makes sense
<wallyworld> and we can use an lxd env var for the lxd host
<axw> wallyworld: might not even need to, you could list the remotes
<axw> wallyworld: e.g. "lxc remote list"
<axw> wallyworld: and report each of those as a region
<wallyworld> true
<axw> wallyworld: I'll tidy this up and propose then. it's a bit less hacky than what was there, and supports openstack. we can revisit if there's dissent
<wallyworld> yup
<voidspace> dimitern: thanks for the review, I replied to all your comments.
<voidspace> dimitern: is your WIP branch ready yet?
<dimitern> voidspace, it is ready - the ci run last night revealed the main issue, which I'm fixing now: don't assume bindings always exist
<dimitern> voidspace, so I'll be pushing a few more commits on top to fix that and re-enable the tests I disabled
<voidspace> dimitern: cool
<dimitern> voidspace, responded to your replies
<voidspace> dimitern: I think you're actually wrong about the test channel - the test really is the owner as it closes the channel.
<voidspace> dimitern: and the "code path" is still the same.
<voidspace> dimitern: the other two issues - ok.
<dimitern> voidspace, I don't mind that much if you leave the chan creation outside of the patched func
<frobware> dimitern, standup
<voidspace> frobware: dimitern: dooferlad: PR for merge of latest master changes back to maas-spaces. http://reviews.vapour.ws/r/3703/
<dimitern> voidspace, LGTM - you'll likely manage to land this before my branch which also includes it, so there shouldn't be a conflict
<mup> Bug #1540832 opened: Hard wired bridge prevents the use of the fan overlay network <juju-core:New> <https://launchpad.net/bugs/1540832>
<mup> Bug #1540832 changed: Hard wired bridge prevents the use of the fan overlay network <juju-core:New> <https://launchpad.net/bugs/1540832>
<mup> Bug #1540832 opened: Hard wired bridge prevents the use of the fan overlay network <juju-core:New> <https://launchpad.net/bugs/1540832>
<frobware> dimitern, dooferlad, voidspace, jam: the trusty/series issue we talked about in the standup is being tracked here atm: https://bugs.launchpad.net/juju-core/+bug/1540771
<mup> Bug #1540771: Bootstrap fails with trusty is not a valid distro_series <bootstrap> <juju-core:Invalid> <juju-core maas-spaces:Triaged> <https://launchpad.net/bugs/1540771>
<frobware> mgz, looking for help with bug 1540771. dimitern plans to update his branch again (RSN) and we'll want another CI run but little point if we keep running into said bug.
<mup> Bug #1540771: Bootstrap fails with trusty is not a valid distro_series <bootstrap> <juju-core:Invalid> <juju-core maas-spaces:Triaged> <https://launchpad.net/bugs/1540771>
<dooferlad> frobware/dimitern/voidspace: Updated bind syntax: http://reviews.vapour.ws/r/3569/diff
<dimitern> dooferlad, cheers, looking
<dimitern> dooferlad, reviewed
<dooferlad> dimitern: http://reviews.vapour.ws/r/3569 addressed those issues.
<dimitern> dooferlad, thanks! let's get it in then I guess
<mup> Bug #1540900 opened: juju deploy ignores model default-series <juju-core:New> <https://launchpad.net/bugs/1540900>
<jam> fwereade_: ping
<mup> Bug #1540900 changed: juju deploy ignores model default-series <juju-core:New> <https://launchpad.net/bugs/1540900>
<mfoord> frobware: what's the release notes link again please
<mfoord> frobware: never mind, found it
<frobware> mfoord, thought so as you show up as already viewing. :)
<mfoord> frobware: heh
<mup> Bug #1540900 opened: juju deploy ignores model default-series <juju-core:New> <https://launchpad.net/bugs/1540900>
<jam> fwereade_: I had a question about "connects", and another one about some lxd configuration stuff
<fwereade_> jam, heyhey
<voidspace> cherylj:  dimitern:  frobware: alexisb: given that 2.0 is dropping maas 1.8 support we should probably not mention improved support for maas 1.8 in the release notes (there's a maas 1.8 compatibility section)
<voidspace> bug 1534636
<mup> Bug #1534636: Destroying a hosted model in the local provider leaves the controller unusable <juju-release-support> <juju-core:Triaged> <https://launchpad.net/bugs/1534636>
<frobware> jam: saw your doc, but punting until I get the maas-spaces unit tests to work on ppc64
<voidspace> cherylj:  dimitern:  frobware: alexisb: we can probably also drop the mention of bug 1534636 as a known issue, as the local provider won't
<voidspace> *won't be there in 2.0
 * voidspace lunches
 * frobware follows suit
<dimitern> cherylj, frobware, alexisb, voidspace, I think that section was copied from 1.25 rel. notes, so we should indeed drop it for the 2.0 notes
<jam> fwereade_: hey, still around?
<perrito666> bbl
<fwereade_> jam, ha, yes, but apparently notification-blind
<mattyw> jam, any reason why we shouldn't do $$merge$$ on this https://github.com/juju/juju/pull/4131 ?
<cherylj> mattyw: yes, we are trying to get some branches landed and we'd rather not have master be a moving target
<mattyw> cherylj, ok cool thanks
<cherylj> frobware: I see now that maas-spaces has some new commits.  What's the expectation now for testing?  are we still testing dimitern's branch?
<mattyw> cherylj, ok - judging by the comment lxd provider is broken in xenail so it probably needs to land before the release - but I don't really know
<mattyw> cherylj, I'm not involved - I'm just an interested party
<cherylj> mattyw: we can look at getting it into alpha2, but really, if it misses, alpha3 is next week
<mattyw> cherylj, ack
<frobware> cherylj, yes, new commits for blocking API and bootstrap whilst discovery taking place (this is the one we talked about last night). bind syntax change for '@' -> '=' which came out of CT. and merge current master into maas-spaces.
<frobware> cherylj, dimitern is updating and fixing tests based on the CI run.
<cherylj> frobware: is dimitern's branch up to date with maas-spaces again?
<frobware> cherylj, dimitern: do you plan to rebase maas-spaces onto your branch?
<cherylj> otherwise we need to get a bless on dimitern's, then merge into maas-spaces, then get a bless on maas-spaces before it can all land in master
<frobware> cherylj, ack.
<frobware> cherylj, I would advocate that dimiter rebases so we only do test once
<cherylj> frobware: agreed
<cherylj> and don't forget to pull in master too.  There were some changes that came in
<frobware> cherylj, we did yesterday - need to check if we're talking about the same change.
<cherylj> frobware: probably same one.  the pprof thing
<frobware> cherylj, yep
<frobware> cherylj, 3 things to fix in dimiter's branch: CI test failures surrounding bindings, skipped unit tests (though for expediency we could skip these), and ppc64 unit test failures.
<cherylj> frobware: wallyworld has a PR up for a different bug that's only been seen on his branch (although the underlying problem exists on master too).  I think it should go into his branch rather than master so we can demonstrate in CI that it fixes the problem
<cherylj> frobware: what's the confidence level in getting a bless on dimitern's branch today
<frobware> cherylj, ppc64 might still fail. looking at that now. need to hear from dimiter about progress on the other problems. also bug 1540771 might cause a cursed build again.
<mup> Bug #1540771: Bootstrap fails with trusty is not a valid distro_series <bootstrap> <juju-core:Invalid> <juju-core maas-spaces:Triaged> <https://launchpad.net/bugs/1540771>
<cherylj> frobware: is that bug a problem with the MAAS setup?  or a problem on the branch?
<frobware> cherylj, I would say MAAS setup and/or CI setup.
<cherylj> frobware: is there someone who can work with jog_ to track it down so we don't hit it again?
<frobware> cherylj, put me down, though I have a hard commitment 7:30pm.
<cherylj> frobware: ok, jog_ should be in before then
<frobware> cherylj, I'm not entirely sure how quick a CI run can be but I would advocate we have a stripped down run that initially only runs the failures; we need to iterate faster
<cherylj> frobware: yes, but your changes could cause some of the previously passing tests to fail, so we need to get a complete run before a merge.  We could see about doing some initial tests for verification before a complete run
<cherylj> sinzui: is there a way we could just run selected tests for maas-spaces?
<cherylj> mgz: ^^
<frobware> cherylj, agreed.
<frobware> cherylj, I just don't want to wait $N hours to find out we haven't fixed the existing failures
<cherylj> frobware: yeah, I understand
<sinzui> cherylj: we can re-run selected tests for previously tested revisions. We cna do this beause we saved  the built packages and streams. We cannot run tests with a branch.
<frobware> cherylj, do you know if we have access to ppc64?
<frobware> I'm futzing trying to fix this unit test failure
<frobware> I have a qemu ppc64 running but it's not terribly efficient or productive.
<frobware> actually, I cannot build juju atm (on ppc64)
<cherylj> mgz, sinzui, is it possible to get frobware access to the CI ppc machines?
<sinzui> cherylj: yes, though I am informed that ports.ubuntu.com is down and that limits what we can do. No installs
<cherylj> sinzui: should be okay, it's just a unit test run
<frobware> sinzui, cherylj: ah, that explains some of my problems over the last 20 mins.
<cherylj> d'oh
<cherylj> frobware, dimitern, I've copied over the charms that CI uses for the basic deploy tests here:  https://private-fileshare.canonical.com/~cherylj/dummy-charms/
<cherylj> there's also a txt file with the steps used to deploy
<cherylj> you can start with testing on AWS, since that's failing on the maas-spaces-controller-space-config
<frobware> dooferlad, if you're between tasks please could you help out on the test failures ^^
<dooferlad> frobware: I am not quite, but will be soon.
<natefinch> ericsnow: it occurs to me that we need to have the resource download code on the server regardless, to support the GUI
<ericsnow> natefinch: I wasn't aware of any GUI-related requirements
<natefinch> ericsnow: I think rick_h__ would probably be upset if we couldn't deploy charms from the gui that used resources in the charmstore
<ericsnow> natefinch: regardless, you make a good point about what the GUI might need
<natefinch> ericsnow: but you're right that there was nothing in the spec about it
<cherylj> fwereade_: would you be able to review https://github.com/juju/juju/pull/4266 ?  It's related to that bug I talked to you about yesterday
<ericsnow> natefinch: I can follow up with the GUI team
<rick_h__> natefinch: can you email the context there?
<natefinch> rick_h__: sure
<ericsnow> rick_h__, natefinch: I just send an email
<rick_h__> natefinch: ty on my phobe aylt the conference and want to better follow what's up
<fwereade_> cherylj, alexisb: ack
<ericsnow> natefinch: let's get back together some time after lunch to put together the demo for last iteration
<natefinch> ericsnow: ok
<jog> hi frobware, I'm here if you have questions about bug 1540771
<mup> Bug #1540771: Bootstrap fails with trusty is not a valid distro_series <bootstrap> <juju-core:Invalid> <juju-core maas-spaces:Triaged> <https://launchpad.net/bugs/1540771>
<frobware> jog, immediate question is if we run again will this just repeat. is this at related to not running against MAAS 1.7 or 1.8?
<jog> frobware, it failed several time in a row in the same way. Master had run before maas-spaces and passed. The bug is open for failures against MAAS 1.9
<frobware> jog, but new against 1.9?
<jog> frobware, not sure what you mean?
<frobware> jog, trying to understand whether it is our branch that causes the failure or a separate 1.9 issue
<jog> well I just reran the last revision of master (1cb8f03) and don't see the failure there. I'm running the maas-spaces revision that failed again now, just to be sure there was no other issue.
<frobware> dimitern, ^^
<frobware> jog, thx - will keep an eye out for the result.
<frobware> jog, is your "reran" http://reports.vapour.ws/releases/3564
<frobware> jog, because that show as cursed
<frobware> jog, ah, but only for ppc64.
<natefinch> man, the function that actually does charm deployments from the client is a hairy mess
<natefinch> 100 lines of if statements
<perrito666> how sad
<natefinch> and weird implicit meanings to error values... good lord
<natefinch> perrito666: https://github.com/juju/juju/blob/master/cmd/juju/commands/deploy.go#L249
<perrito666> deploy charm Or bundle, just in case
<perrito666> is there a reason why that is not the at least 3 methods it should be?
<natefinch> perrito666: the problem is that our deploy command does way too much and has way too much encoded into the string we pass it
<perrito666> "we might have been given a bundle" <-- that is one.  " if not a bundle then maybe a local charm" <- there is another,
<natefinch> the fact that this code is a nightmare is just an indication that the UX is a nightmare...
<natefinch> I mean, the code could be better even given the bad UX
<perrito666> that is not true, code can be bad and ux be decent
<perrito666> I have  glanced at that function and already know it can be split
<perrito666> I do have the "fresh pair of eyes" advantage
<natefinch> yeah, it absolutely should be refactored
<perrito666> but cmon, a bit of good will might make things better
<natefinch> I think someone added bundles and just didn't want to refactor what was there
<perrito666> well mcdonalds lunch is not the best alternative to stay awake after lunch
<natefinch> heh
<perrito666> bbl
<natefinch> oh how nice, we're throwing away an error and assuming we know what was in it
<natefinch> oh hey, someone fixed it on master... ok, I gotta rebase.
<voidspace> dimitern: we have 19 code branches checking for the AddressAllocation feature flag
<dimitern> voidspace, so next step is to prepare a PR that drops them I guess
<voidspace> dimitern: did we ever get devices by default on master?
<voidspace> dimitern: it looks to me, from skimming the code, that if we lose AddressAllocation we lose containers as devices too
<voidspace> dimitern: and we go back to leaking DHCP leases
<perrito666> bq
<dimitern> voidspace, that's ok - we'll redo it properly anyway
<natefinch> gah, why do we have a client api function that no one calls? (ServiceDeployWithNetworks)
<natefinch> .....maybe we keep it so the tests can test the server's backwards compatibility
<natefinch> hmm, in which case it should be in test code only.
<mup> Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <run> <juju-core:New> <https://launchpad.net/bugs/1540469>
<mup> Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <run> <juju-core:New> <https://launchpad.net/bugs/1540469>
<perrito666> natefinch: are you sure no one calls it?
<natefinch> perrito666: grep says it's only called in tests
<perrito666> I do recall that there is a cascade of calls for service deploy
<perrito666> ServiceDeploy -> serviceDeployWithX -> ServiceDeployWithY and so on
<perrito666> natefinch: take a look a few versions back, perhaps its legacy and asking to go
<natefinch> perrito666: yeah, almost certainly... like I said, my guess is that it needs to stick around for testing that we keep backwards compatibility with old clients... but that means it should be moved to test code, so it's more obviously something you should let rot in a corner
<perrito666> actually, if you are working in 2, that is a death sentence
<mup> Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <run> <juju-core:New> <https://launchpad.net/bugs/1540469>
<natefinch> perrito666: *shrug* I'll file a bug
<mup> Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <run> <juju-core:New> <https://launchpad.net/bugs/1540469>
<mup> Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory <run> <juju-core:New> <https://launchpad.net/bugs/1540469>
<natefinch> dammit... something is throttleing the hell out of my laptop lately
<perrito666> natefinch: use htop?
<perrito666> aghh, sms spam, just what I needed
<natefinch> perrito666: htop is very pretty and I have no idea what it means
<perrito666> natefinch: use the man page
<perrito666> if the upper graphs for microprocessor are full (or the ram one) something is behaving wrongly, you can use f6 iirc to sort things by cpu or ram
<perrito666> and that would put the culprit at the top
<natefinch> perrito666: I'm running tests, and expect my processors to all be pegged, but they're all at like 50%, and my tests are taking a lot longer than usual to run
<perrito666> natefinch: try iotop then
<perrito666> perhaps something is hammering your hd
<natefinch> hmm disk read is always 0, I presume that's a bug, or my computer would not be working ;)
<natefinch> that weird feeling when you change the signature of like 3 very important functions.... and no tests fail
<natefinch> ...I'm sure it's fine
<perrito666> natefinch: http://www.quickmeme.com/img/a7/a7eb19ae524e1d066fae16e5e7c0438c86fa321468cd3e3d0764731cf86f869c.jpg
<natefinch> lol indeed
<davechen1y> no menn0 today ?
<davechen1y> paging the on call reviewer: http://reviews.vapour.ws/r/3700/
<davechen1y> to the OR, STAT!
<perrito666> that is me
 * perrito666 reviews
<perrito666> stat?
<natefinch> perrito666: means "right now"
<natefinch> perrito666: I believe it was backronymed to Sooner Than Already There
 * natefinch makes perrito666 google backronym instead of stat
<perrito666> natefinch: its ok, for me davechen1y 's now is 11 hours away so technically I can pull it
<natefinch> perrito666: https://www.youtube.com/watch?v=gNIwlRClHsQ
<natefinch> ericsnow: I gotta run a little early today (will be back later as usual).  Want to talk demo?
<ericsnow> natefinch: sure
<ericsnow> natefinch: moonstone
<perrito666> davechen1y: there you got a bunch of fixmes :p and all more than 10 hours before your asked time
<menn0> waigani: ship it on the auth worker PR
<waigani> menn0: thanks
 * perrito666 makes coffee before a long long review
<natefinch> bbl
<perrito666> I wish we had un-shipit as an option
<menn0> waigani: deployer review done
<menn0> wallyworld: http://reviews.vapour.ws/r/3711/ doesn't need a review right?
<wallyworld> menn0: nope, is a straight merge of a feature branch
<wallyworld> a huge one at that
<menn0> wallyworld: I noticed so I'm glad to hear I don't need to review it :)
<wallyworld> all the terminology changes etc
<wallyworld> as per your earlier email to dev list
<wallyworld> model model model
<wallyworld> cherylj: i wonder if the add-machine "regression" is just a timebomb in 1.25 say, waiting to show up like the leadership issue did, and we just haven't seen it yet
<wallyworld> you'd think it would show up in more places
<perrito666> menn0: tx for the rev
<rick_h__> just dropping as an FYI: https://twitter.com/mitechie/status/694506568000344064
<perrito666> neat
<rick_h__> yea, really good attendance here at the summit
 * perrito666 wanted to go
<rick_h__> we'll have to hit up the next one. We had a couple of core folks here
<davechen1y> cherylj: when will the next 1.25 build come out ?
<davechen1y> ie, if I want to get my profiling stuff into a 1.25 build
<davechen1y> what deadline should I focus on ?
<cherylj> davechen1y: I'm targeting late next week
<cherylj> for 1.25.4
<cherylj> hooray!  api-command-rename has LANDED!
 * rick_h__ does a giant happy dance
<cherylj> master will stay blocked until we get a bless
<cherylj> then we shall open up the flood gates
#juju-dev 2016-02-03
<anastasiamac> cherylj: we are all holding our breaths :D
<alexisb> anastasiamac, me too :)
<alexisb> wallyworld, I am free when you are ready to chat
<davechen1y> cherylj: thanks
<perrito666> wow save file still has that annoying create new folder bug
<alexisb> perrito666, sounds like you are volunteering
<perrito666> alexisb: for what?
<alexisb> fixing an annoying bug :P
<perrito666> alexisb: I am by no means fixing a bug in unity7
<wallyworld> alexisb: just finished with andrew, let's meet in our 1:1
<thumper> davechen1y, cherylj, wallyworld: while I remember, next Monday is a public holiday in NZ
<wallyworld> waitaingi day?
<thumper> aye
<alexisb> wallyworld, ack
<wallyworld> enjot
<thumper> will do
 * thumper goes to add to team calendar
<perrito666> wallyworld: and monday and tuesday are holidays in argentina (be cause we cant be less than nz)
<thumper> perrito666: what is your excuse?
<perrito666> thumper: carnival ;) south america rules
<thumper> :)
<perrito666> also, the fact that we stop the country 2 days for carnival might hint you on why the economy is going down the drain :p
<davechen1y> thumper: noted
<davechen1y> 3 day kiwi weekend
<thumper> yay
<anastasiamac> perrito666: well, in Oz, we have a horse race that apparently stops the country for the whole of 5 mins but "it's the race that stops Australia" :D
<thumper> perrito666: I think more countries should take two days off for a party
<thumper> alexisb: worth noting that even though I'm off monday, we still have our call on your monday :)
<davechen1y> anastasiamac: 5 minutes on the track; 24 hours on the terps
<anastasiamac> +100 for parties
<anastasiamac> davechen1y: \o/
<anastasiamac> :D
<menn0> waigani: there's something up with the diff for the deployer PR. Lots of errors that weren't there before.
<waigani> ?!
<waigani> menn0: I think I know why. Another worker landed into MADE-workers - and now there are conflicts
<menn0> waigani: that sounds likely
<menn0> phew! I have caught up on reviews.
<menn0> everyone stop working now :)
<anastasiamac> menn0: but then u'll have nothing to do \o/ what will become of u?... :D
<davechen1y> hello
<davechen1y> we have gc.Matches
<davechen1y> which is _sort_ of a regex
<davechen1y> but expects to match exactly
<davechen1y> do we have something like gc.Substring ?
<thumper> wow
 * thumper is a tad surprised
<thumper> tests pass
<thumper> it means I didn't screw up the refactoring
<thumper> \o/
<thumper> what I mean to say is
<thumper> of course it works!
<perrito666> I find your faith in our tests disturbing
<thumper> :-)
<davechen1y> prey that I do not change the scope further
<perrito666> ok, my brain just became pudding, see you all tomorrow
<davechen1y> booooooooooooooo
<davechen1y> gc.Match does not handle multi line input
<davechen1y> what a shit
<waigani> menn0: that was the problem. I've resolved conflicts. Other than conflicts, only change I made was to return the Uninstall err
<menn0> cool. Ship it then!
<thumper> holy crap
<thumper> just now looking at the number of comments that menn0 added to the migrate-machines branch
<menn0> thumper: sorry, most are small things IIRC
<thumper> that's fine
<thumper> just going through them now
<thumper> so I can get a re-review while you are oncall
<thumper> also, I want to land it
<thumper> services is almost ready
 * davechen1y considers writing a one off checker for what I want
<wallyworld> axw: huzah! just merged master into cloud-crdentials branch, 220 files with conflicts \o/
<axw> wallyworld: heh :)
<wallyworld> sigh
<wallyworld> guess what i'm doing today -( :-(
<anastasiamac> wow :(
<wallyworld> ffs, almost none of these are true conflicts. wtf is git doing
<blahdeblah> wallyworld: At least you're not using bzr...
<wallyworld> blahdeblah: i love bzr, *much* better thT GIT
<wallyworld> intiuitive cli
<blahdeblah> Much better at creating conflicts that aren'tr true conflicts, though...
<wallyworld> you don't need to understand the internal model
<wallyworld> that might be true
 * davechen1y decides not to write a checker
<davechen1y> i'll just write a function that takes a gc.C
<davechen1y> menn0: perrito666 http://reviews.vapour.ws/r/3713/
<davechen1y> ^ addresses your commentson the 1.25 backport
<davechen1y> will land on master and merge into my 1.25 backport branch
<menn0> davechen1y: looking
<thumper> menn0: i.makeMachineJobs(m.Jobs())
<thumper> nope
<thumper> wrong paste
<thumper> menn0: http://reviews.vapour.ws/r/3656/
<thumper> sometimes having two paste buffers is annoying
<menn0> thumper: I liked the first one better :)
<thumper> menn0: I agreed with almost all your review points
<thumper> perhaps you'd like a few minute call to talk about the others?
<menn0> davechen1y: looks good. ship it!
<menn0> thumper: lemme have a look first
<thumper> menn0: ack
<thumper> menn0: in the service branch, I had realised also that I wanted a method called Tag(), and a separate one for Name()
<thumper> menn0: luckily I didn't go back and do the machine ones
<thumper> I was thinking about it
<thumper> have done so for this review
<menn0> ok
<menn0> thumper: I'm fine with the issues you dropped. I'm happy with your reasoning and/or they were ones I either didn't feel too strongly about.
<menn0> still going through the diff now.
<thumper> cool
<thumper> I didn't sneak anything else in :)
<menn0> thumper: just seeing how you addressed the issues
<thumper> mostly by deleting things :)
<menn0> thumper: hmmmm.. maybe I shouldn't have asked for all those doc strings.
<menn0> a lot are pointless.
<thumper> heh
<thumper> yes
<menn0> e.g. SetStatus implements Machine.SetStatus.
<thumper> but 'required'
<thumper> jeez I'm hot
<thumper> physically hot
<thumper> it is mid 20s here
<thumper> I'd hate to have to live somewhere where it is regularly this hot
 * thumper looks at wallyworld, axw and anastasiamac
<thumper> no air con
<thumper> because this is NZ
<menn0> thumper: I just talked to my parents. It's 38 right now in Brisbane
<thumper> why would you want to cool things down?
<menn0> New Zealanders are often in denial about hot and cold
<axw> thumper: actually quite nice today, 26C. going to be 40 on the weekend though :o
<menn0> for example the freezing student flats in Dunedin
<thumper> heh
<menn0> thumper: ship it, with a few minor comments
<thumper> menn0: cheers
<thumper> menn0: I'm not sure on the 'canonical' comment for interface method implementations
<thumper> davechen1y: do you have an opinion on this?
<thumper> axw, wallyworld: or you folks?
<axw> wat?
<menn0> thumper: IMHO repeating the method name adds more repetition to an already low-value doc string
<menn0> axw: // Foo implements SomeInterface.Foo
<menn0> vs
<menn0> / Foo implements SomeInterface.
<menn0> / even
<thumper> or // Foo implements SomeInterface method
<axw> menn0: oh. I've never thought either was really that helpful, TBH.
 * thumper shrugs
<thumper> axw: but if we are wanting to have tools that complain if you are missing comments for exported methods / types
<menn0> mentioning the interface being implemented offers a little bit of value in terms of linking back to the intended interface
<thumper> we should have something
<thumper> yeah
<menn0> mentioning the method name twice doesn't add any value
<menn0> I don't care too much though
 * thumper nods
<axw> of the two, I'd go for the methodless one
<thumper> axw: so // Foo implements SomeInterface.
<axw> thumper menn0: I think in the past I've written "// Foo is part of SomeInterface"
<thumper> I like that
<menn0> that's fine too
<thumper> I'm thinking of what I have though
<menn0> although it's sort of vaguely incorrect
<thumper> will look like: // Id is part of Machine.
<menn0> "implements" is slightly more accurate
<axw> yeah, it might not have been that specifically
<thumper>  // Id implements Machine.
<thumper> is that better?
<axw> menn0: well... all the methods together implement that interface
<axw> I'm fine with that
 * axw doesn't really care
<thumper> shall we really bikeshed it and take it to the mailing list?
<axw> heh
<thumper> menn0: the fact that the method on partially implements the interface is why I added the name on the end
<thumper> but
 * thumper shrugs
 * thumper looks at our current code
<thumper> / SubnetId implements StateIPAddress.
<thumper>  // SubnetId implements StateIPAddress.
<thumper> ugh
<thumper> I'll go with that
<thumper> at least we'll be mostly internally consistent
<menn0> It really doesn't matter too much. All the approaches are fine. I was trying to avoid requiring a little extra typing and maintenance.
<davechen1y> thumper: can you bless this please https://github.com/juju/juju/pull/4275
<thumper> davechen1y: done
<menn0> waigani: I'm just pushing a PR now which adds the test helpers for PostUpgradeManifold based manifolds
<menn0> waigani: in includes new unit tests for proxyupdater's manifold
<waigani> menn0: awesome! I'll learn from the proxyupdater
<menn0> waigani: https://github.com/juju/juju/pull/4277
<menn0> waigani: yeah I figured a non-trival example would be useful
<waigani> menn0: shipit. Thanks for the unit test examples, all makes sense.
<davechen1y> thumper: tahnks
<davechen1y> thumper: relly $$__JDFI__$$ ?
<thumper> yeah
<davechen1y> TIL
<menn0> waigani: thanks
<thumper> laters folks
<thumper> I'll be checking in on my merge job
<mup> Bug #1541228 opened: deployed container in maas provider does not honor maas proxy settings <juju-core:New> <https://launchpad.net/bugs/1541228>
<mup> Bug #1541228 changed: deployed container in maas provider does not honor maas proxy settings <juju-core:New> <https://launchpad.net/bugs/1541228>
<mup> Bug #1541228 opened: deployed container in maas provider does not honor maas proxy settings <juju-core:New> <https://launchpad.net/bugs/1541228>
<dimitern> voidspace, morning
<dimitern> voidspace, it seems like apart from ci issues the most common cause of the last curse was due to "spaces still being discovered" - e.g. on a manual provider or joyent, etc.
<voidspace> dimitern: that should be a warning not an error
<voidspace> dimitern: and it shouldn't happen on providers that don't support discovery
<dimitern> voidspace, also the api rename branch has landed on master, I'm preparing a merge PR for maas-spaces
<voidspace> dimitern: oh dear in other words
<voidspace> dimitern: cool
<dimitern> voidspace, well, have a look at the ci report
<voidspace> dimitern: I'll look into the failures
<voidspace> dimitern: yep
<voidspace> dimitern: ah, just worked out what it could be
<voidspace> dimitern: if I'm right, will be a trivial fix (and extra test)
<voidspace> dimitern: coffee first
<voidspace> dimitern: (I suspect we don't close the channel if discovery isn't supported - I thought that was tested, but maybe not)
<dimitern> voidspace, cheers - I suspected something like that
<voidspace> dimitern: confirmed
<voidspace> dimitern: just pushing a fix, will write test after coffee but before standup
<dimitern> voidspace, sure, thanks
<voidspace> dimitern: that's the fix: https://github.com/juju/juju/compare/maas-spaces...voidspace:maas-spaces-discovery-fix?expand=1
<dimitern> voidspace, looks good, but can you please try a live test on something other than maas?
<voidspace> dimitern: hmmm... there was a test for that, obviously something wrong with the test
<voidspace> dimitern: yep
<dimitern> voidspace, ta
<voidspace> dimitern: ah no, it was just where the provider doesn't support networking - and you can't get a dummy provider that doesn't support networking
<voidspace> dimitern: so it wasn't testable
<voidspace> dimitern: so I'm pull requesting this as is
<voidspace> dimitern: http://reviews.vapour.ws/r/3716/
<dimitern> voidspace, networking maybe, but space discovery is testable with dummy
<voidspace> dimitern: yes, and it is tested
<voidspace> dimitern: the faulty code path is where the provider doesn't support networking
<voidspace> (was)
<voidspace> dimitern: where environs.SuppportsNetworkingEnviron(environ) returns false for ok
<dimitern> voidspace, ah, I see
<voidspace> dimitern: and you can't trigger that with the dummy provider
<dimitern> voidspace, there are ways to trigger it
<voidspace> dimitern: I couldn't see any
<dimitern> voidspace, in networkingcommon we use a stub that embeds environs.Enviorn
<voidspace> dimitern: it's a type cast and there isn't a dummy provider that doesn't have those methods
<voidspace> dimitern: not in networkingcommon we don't
<voidspace> dimitern: ah, apiservertesting StubProviderType
<voidspace> dimitern: ok, I'll look at those tests
<dimitern> voidspace, yeah - it's ok for this specific feature to be tested with a stub environ I think - since we test the rest of the logic already
<frobware> dimitern, voidspace, dooferlad: we also need to concentrate on just 1 branch to test (i.e., maas-spaces).
<voidspace> frobware: I don't really understand. We are concentrating on just maas-spaces aren't we?
<frobware> voidspace, nope. the last few CI runs have been based off dimiter's wip branch.
<frobware> voidspace, dimitern: we should get that merged into maas-spaces (assuming we're happy the CI results and changes per-se)
<voidspace> frobware: ah
<frobware> voidspace, dimitern, dooferlad: the CI tests look better in that I didn't see the empty series error
<dimitern> frobware, agreed - I'm almost done with the merge PR from master
<dimitern> frobware, yeah, that last minute refactoring around select|acquire|startNode seems to have worked
<frobware> dimitern, I took a look too. there were some conflicts but perhaps fewer than I feared.
<voidspace> dimitern: with the stub-non-networking-provider I can create an apiserver facade that uses it
<dimitern> frobware, most conflicts are around discoverspaces and the networkingcommon restructuring
<dimitern> voidspace, awesome - so it will work for the test?
<voidspace> dimitern: but I need an api facade (to create a discoverspaces.API to create the worker), that will return the right config from EnvironConfig to get the stub environ
<voidspace> dimitern: you were too quick :-) an apiserver facade is only part of the machinery
<frobware> mgz, ping
<voidspace> dimitern: setting up the dummy provider is done in JujuConnSuite in setUpConn
<voidspace> dimitern: and there's a *lot* of code there
<dimitern> voidspace, yeah, so you need another suite not based on jujuconnsuite
<voidspace> dimitern: but that's my point
<voidspace> dimitern: in order to get an api that can actually get back at the provider I think I probably need jujuconnsuite
<dimitern> voidspace, let's topic this for standup
<voidspace> dimitern: ok
<voidspace> dimitern: frobware: dooferlad: finally failed with "unable to connect to API" (not because of space discovery though)
<voidspace> dimitern: frobware: dooferlad: this is the error I expected, I just expected it a bit earlier in the process!
<dimitern> voidspace, right, that's perhaps due to the dep engine churning through dependencies
<dimitern> voidspace, also, if you're waiting for a fixed time that's flaky - it needs no to depend on the real clock if possible
<voidspace> dimitern: this is just normal bootstrap, there's no additional timeout code in any of the discoverspaces stuff
<voidspace> dimitern: maybe there's a timeout set in environments.yaml I can tweak?
<voidspace> not that I can see
<dimitern> voidspace, there's bootstrap timeout settings - see juju help bootstrap
<voidspace> dimitern: I'm not *sure* that's the problem though
<dimitern> voidspace, :) when in doubt - add more logging
<voidspace> heh
<voidspace> dimitern: what do you have in ~/.ssh/config for Canonistack?
<voidspace> dimitern: I have a ProxyCommand there, through chinstrap, which is probably why it "works"
<voidspace> specifically, I'm wondering what you use for User - I have "ubuntu"
<dimitern> voidspace, let me check
<dimitern> voidspace, I have these: https://pastebin.canonical.com/149020/, but I haven't tried if it still works
<voidspace> dimitern: thanks
<jamespage> jam, alexisb: hey - not sure if you are around but https://github.com/juju/juju/pull/4131 is required for local LXD provider on xenial right now
<dimitern> frobware, dooferlad, voidspace: https://github.com/juju/juju/pull/4279 please take a look - all tests pass, incl. a live test on both ec2 and maas 19
<frobware> dimitern, dooferlad, voidspace: please can I ask that we have two 'ShipIts' for this change. I was just trying this independently and what to see if I end up with the same tree. Thx.
<dimitern> frobware, it won't be the same tree
<dimitern> frobware, it's a cherry-picked merge from tip of master + a few more commits on top to fix the issues
<frobware> dimitern, I can pull your tree and compare, that's all I was thinking of. Unless I'm missing something....
<dimitern> frobware, but it also changes a few names - e.g. TestManageEnvironX -> TestManageModel; for consistency
<frobware> dimitern, I think we should have followed up with that kind of change.
<frobware> dimitern, i.e., do the simplest smallest change possible
<dimitern> frobware, it's not a lot more than the simplest thing - see the latter commits
<frobware> dimitern, ok
<mup> Bug #1541393 opened: apiserver/logsink.log gets created while running unit tests <juju-core:Triaged> <https://launchpad.net/bugs/1541393>
<dimitern> frobware, voidspace, dooferlad, any feedback or issues with the master merge while testing?
<dimitern> I've rebased my other branch on top of the merge PR with not a lot of churn
<frobware> dimitern, I didn't get the same-ish tree as you - was just looking to understand why
<dooferlad> dimitern: I am about to run the mediawiki demo. Just need to sort out a problem.
<dimitern> dooferlad, cheers
<dimitern> frobware, I've used the tip of master with: git cherry-pick -m1 9e7dbcf67cefe89dbaf291222ace59c74c28aa67
<frobware> dimitern, that was your initial merge, then followed up with name fixes?
<voidspace> I'm still trying to bootstrap with my branch - couldn't bootstrap canonistack with master and a 20minute timeout
<dimitern> frobware, yeah, however as I discovered later I had to change a few places (reverting a bit of the post-merge commit changes) and they are in the last commit
<voidspace> it *may* be that an even longer timeout is needed, not sure
<voidspace> but need this branch to land so testing with amazon and maas at least first
<dimitern> voidspace, did you use --debug ?
<voidspace> dimitern: not yet, good point - but need to do sanity test with amazon/maas *anyway*
<voidspace> so getting that done
<dimitern> voidspace, I'll give your PR a try
<voidspace> cool
<frankban> ocr I need a review for http://reviews.vapour.ws/r/3719/ (quick fix to juju deploy flags)
<frankban> cherylj: FYI ^^^
<voidspace> right, I'm nipping out
<voidspace> back soon
<dimitern> voidspace, (when you're back) I can confirm your fix works in maas 1.9 and aws, now trying local and manual to be certain
<dimitern> voidspace, manual bootstrap on wily works fine, a well as local - no unnecessary blocking
<dooferlad> dimitern: http://pastebin.ubuntu.com/14866490/
<dooferlad> dimitern: not sure what happened. Just digging through the logs now.
<dimitern> dooferlad, hmm interesting - that's reported by FinishBootstrap I think (or thereabouts)
<dooferlad> dimitern: SetBootstrapEndpointAddress
<dimitern> dooferlad, did you use -m maas by any chance?
<dimitern> it doesn't look like from the paste
<dooferlad> dimitern: no
<dooferlad> dimitern: I switched to maas, then did the bootstrap
<dimitern> dooferlad, so it completed, but the juju client messed up saving the endpoint
<dimitern> dooferlad, it you can repro this, try bootstrap with --debug
<dooferlad> dimitern: looks like it. I am trying again with --debug to see if I get more useful output
<dooferlad> :-)
<dimitern> :)
<dimitern> dooferlad, check if you have any surprising/stale stuff in ~/.juju/environments/ (or ~/.juju/ for that matter)
<dimitern> dooferlad, I found there a cache.yaml file with a section like: "": ... - perhaps that "" was supposed to contain "maas" in your case
<dooferlad> dimitern: yea, it does seem to contain some rubbish
<dooferlad> dimitern: yep, server-data: "": ...
<dimitern> dooferlad, that cache.yaml is news to me - perhaps a step towards dropping environments.yaml and using (another piece of news) --bootstrap-config argument
<dooferlad> dimitern: looks like it. I just deleted it and am trying again.
<dooferlad> dimitern: this time it has a UUID. Strange!
<dimitern> dooferlad, yeah, sounds like a bug
<dimitern> dooferlad, can you please file a bug if that's the cause - having a cache.yaml with empty uuid?
<dooferlad> dimitern: sure.
<dimitern> dooferlad, cheers!
<cherylj> dimitern, dooferlad  - the api-command-rename branch changed the default juju home, so you'll see weird things if you switch between a juju built before then, and one built after
<dimitern> cherylj, really, what changed there?
<cherylj> gah, I can't find what it changed it.
<cherylj> what it changed TO.  give me a minute
<dimitern> cherylj, np, thanks for the heads up :)
<frobware> dimitern, dooferlad: the only joy I have right now is to set JUJU_HOME=~/.juju2 if on master/maas-spaces. otherwise switching between that and 1.25 makes things confusing.
<cherylj> dimitern: the problem I run into is because cache.yaml is now in ~/.juju/models/  so if I try to use an older client, it thinks the environment isn't bootstrapped
<cherylj> I thought they changed the default juju home, maybe that hasn't happened yet.
<voidspace> dimitern: thanks for the review - I've tested with maas (spaces are discovered fine) and amazon (no spaces but bootstrap completes fine)
<voidspace> dimitern: will set to land and work on getting canonistack to work plus testing
<dimitern> cherylj, I see, thanks
<dimitern> voidspace, ta!
<mup> Bug #1541445 opened: empty uuid in cache.yaml after destroy-controller <juju-core:New> <https://launchpad.net/bugs/1541445>
<mup> Bug #1541445 changed: empty uuid in cache.yaml after destroy-controller <juju-core:New> <https://launchpad.net/bugs/1541445>
<mup> Bug #1541445 opened: empty uuid in cache.yaml after destroy-controller <juju-core:New> <https://launchpad.net/bugs/1541445>
<perrito666> aghh lxd stable ppa provides go 1.6 that is insane
<voidspace> perrito666: why is that insance?
<perrito666> because 1.6 rc1 is not someting I would call stable
<perrito666> voidspace: also it is a bit intrusive
<voidspace> perrito666: ah, RC1
<perrito666> voidspace: yep missed that, but anyway, replacing someones compiler for go is not that nice :p
<voidspace> perrito666: ah, it replaces it - yeah, agree
<perrito666> voidspace: if you apt-get install it in vivid you get golang-go from lxc stable ppa
<perrito666> which is by no means a good thing to do to someone's system
<voidspace> perrito666: do you ever deploy to canonistack
<perrito666> voidspace: I do not, why?
<voidspace> perrito666: I have the right ~/.ssh/config setup so it can bootstrap via ssh
<voidspace> perrito666: but I don't have the right sshuttle incantation so it can connect to the apiserver afterwards
<voidspace> the one I used to use doesn't any longer
<perrito666> ah my go to person for that kind of magic usually is sinzui
<voidspace> good idea
<voidspace> not around though
<voidspace> mgz: ping
<mgz> voidspace: hey
<voidspace> mgz: hey, hi
<voidspace> mgz: do you know the right sshuttle incantation to be able to bootstrap to canonistack
<voidspace> mgz:  I have the right ~/.ssh/config setup so it can bootstrap via ssh
<voidspace> mgz: but I don't have the right sshuttle incantation so it can connect to the apiserver afterwards
<voidspace> so bootstrap still fails
<mgz> voidspace: can ou just enable the vpn?
<voidspace> I used to use: sshuttle -D -r ubuntu@10.55.60.5 10.55.0.0/16
<voidspace> mgz: sshuttle is what I used to use instead of the vpn
<voidspace> mgz: I would need to setup the vpn to use that, I can do I suppose if it's required - sshuttle seems *better*
<mgz> yeah, it was the only option before
<voidspace> mgz: I'll try setting up the VPN, thanks
<mgz> what I do is actually have a machine in canonistack that I bootstrap from, I guess the other option is be fast on the sshuttle incantation when the machine comes up
<mgz> as you need the address of the bootstrap machine to route to, but if you can't get api to finish bootstrap you need to mid-process set that up
<mup> Bug #1541473 opened:  ensure-availability timesout in 1.25 <blocker> <ci> <ensure-availability> <regression> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1541473>
<dimitern> frobware, dooferlad, voidspace's fix landed, and now I'm rebasing my merge to include it - do you foresee any issues with the merge so far? I'd rather land it today if we can
<frobware> dimitern, not at the moment. I did have some problems bootstrapping but on reflection I believe that's because I'm jumping between old (1.25) and new.
<frobware> dimitern, my merge showed a diff between your api/service/client.go and mine - was just verifying.
<voidspace> dimitern: an easier way to test "no-networking" would be to patch out (and make it possible to patch out) environs.SupportsNetworking
<perrito666> so, the answer is "its a ppa" whch is completely fair but we should make sure we dont break people, I think we are recommending said ppa for lxd provider
<perrito666> katco: natefinch-afk ^^
<dimitern> frobware, anything surprising?
<dimitern> voidspace, agreed that seems the proper way to fix it and make it test-able
<frobware> dimitern, it was in ServiceDeploy() - I had the old len(placements) and len(bindings) ...
<frobware> dimitern, just a bit bemused why that it atm.
<dimitern> frobware, do you have a diff?
<frobware> dimitern, not to hand
<frobware> dimitern, why testing mostly on your branch
<frobware> s/on/from
<dimitern> frobware, ok, so ServiceDeploy used to take ToMachineSpec and []Placement, now the latter is used for both
<mup> Bug #1541479 opened: unable to completely destroy-model with LXD provider <juju-core:New> <https://launchpad.net/bugs/1541479>
<mup> Bug #1541482 opened: unable to download charm due to hash mismatch in multi-model deployment <juju-core:New> <https://launchpad.net/bugs/1541482>
<perrito666> cherylj: I thought master got unblocked last night
<cherylj> perrito666: not until we get a bless
<cherylj> :(
<perrito666> cherylj: ah I also thought we got a bless
<perrito666> well bot will kick me
<dooferlad> dimitern: at long last, I have a clean run of the mediawiki demo.
<frobware> dooferlad, \o/
<frobware> dooferlad, using which branch?
<dooferlad> frobware: maas-spaces-merge-master
<dimitern> dooferlad, great!
<frobware> dooferlad, what were the issues?
<mup> Bug #1541479 changed: unable to completely destroy-model with LXD provider <juju-core:New> <https://launchpad.net/bugs/1541479>
<mup> Bug #1541482 changed: unable to download charm due to hash mismatch in multi-model deployment <juju-core:New> <https://launchpad.net/bugs/1541482>
<dooferlad> just juju 2.0 not playing nice with old configurations and some changes to how it outputs status
<mup> Bug #1541482 opened: unable to download charm due to hash mismatch in multi-model deployment <juju-core:New> <https://launchpad.net/bugs/1541482>
<perrito666> go rename is wonderful
<mup> Bug #1541479 opened: unable to completely destroy-model with LXD provider <juju-core:New> <https://launchpad.net/bugs/1541479>
<mup> Bug #1541479 changed: unable to completely destroy-model with LXD provider <juju-core:New> <https://launchpad.net/bugs/1541479>
<dimitern> dooferlad, so it worked ok eventually?
<dooferlad> dimitern: yes
<dimitern> dooferlad, \o/
<dimitern> frobware, I think it's time to push the button :) I'm running make check on maas-spaces-merge-master after rebasing onto voidspace's fix - once it's done I'll push
<dimitern> frobware, voidspace, dooferlad, unless any of you feel strongly against merging today?
<dooferlad> dimitern: +1
<dimitern> frobware, voidspace, last chance? :)
<frobware> dimitern, go for it
<frobware> dimitern, there's no point waiting. :)
 * dimitern types $$merge$$ and hits [Comment]
<dimitern> frobware, cheers
<frobware> mgz, once maas-spaces is updated can we get a CI run?
<mgz> frobware: will do, we're just in some maas untangling now to get a clean 1.25 run
<frobware> mgz, ack
<voidspace> dimitern: frobware: awesome
<frobware> dimitern, voidspace, dooferlad: at some point RSN I want to delete upstream/maas-spaces-controller-space-config to avoid any inadvertent runs on that branch. Concerns?
<voidspace> frobware: not if it has merged, no
<frobware> voidspace, I will of course double-check. :)  But hopefully our "party" is all taking place in maas-spaces now.
<dimitern> frobware, yeah, i'll push another one which is based on maas-spaces after the merge
<dimitern> and drop the other
<frobware> dimitern, push another branch? and to where?
<frobware> dimitern, all that "stuff" is in maas-spaces, no?
<dimitern> frobware, I have maas-spaces-controller-space-after-master-merge almost ready to push (well, in only slightly better tested state)
<frobware> dimitern, but the essence of maas-spaces-controller-space-config is in maas-spaces now, no?
<dimitern> frobware, no, not at all
<frobware> dimitern, ohhhhhhhhhhhhhhhhhhh
<dimitern> frobware, it's not ready yet
<frobware> dimitern, well I have to confess that's a surprise for me.
<dimitern> frobware, but maas-spaces is up-to-date with master now, so we might get fewer issues
<frobware> dimitern, but we still get the failed '' series, no?
<dimitern> frobware, I didn't want to churn things too much by combining the relatively straightforward merge from master with the still highly untested controller-space stuff
<dimitern> frobware, hmm, well
<dimitern> frobware, no
<dimitern> frobware, as I believe that was only in my controller-space branch?
<frobware> dimitern, does maas-spaces-controller-space-config have all that's currently in maas-spaces? ie. is it rebased?
<dimitern> frobware, no, I branched of the merge branch and cherry picked the controller space commits onto it
<frobware> dimitern, ok, but testing maas-spaces means we will fail CI tests with space name issues
<dimitern> frobware, that's correct
<frobware> dimitern, so there's no point running that in CI
<dimitern> frobware, not maas-spaces, but maas-spaces-controller-space-after-master-merge is worth testing prior to merging I guess
<frobware> dimitern, we need to get that into CI; the easiest way is to push the changes into the existing upstream branch maas-spaces-controller-space-config as CI is setup to run that.
<dimitern> frobware, fair point, ok
<dimitern> frobware, then I'll rebase it and push
<frobware> mgz, ^^ we want to do another run of maas-spaces-controller-space-config, just waiting to land the bits.
<frobware> dimitern, which comes back to "any concerns for deleting that branch". Turns out the answer is emphatically yes!
<frobware> dimitern, is there a reason for tonight's CI tests we cannot also merge maas-spaces into upstream/maas-spaces-controller-space-config?
<dimitern> frobware, sorry, I meant concerns about the master merge branch
<dimitern> frobware, i'm doing the rebase of maas-spaces-controller-config onto current tip of maas-spaces now
<frobware> dimitern, if you want some sanity testing let me know and I'll try stuff out.
<dimitern> frobware, more testing is always welcome :)
<dimitern> frobware, will ping you when done and pushed
<perrito666> rick_h__: hey, are you around?
<mup> Bug #1541536 opened: Deployer and Quickstart failed setting annotations because of socket or json parsing <api> <blocker> <ci> <deployer> <quickstart> <regression> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1541536>
<perrito666> aghh why are there no good terminal emulators around :(
<dimitern> frobware, so I can't seem to convince git to push to the same branch
<frobware> dimitern, nope, I suspect I'll have to do it.
<dimitern> frobware, so the changes are pushed to https://github.com/juju/juju/compare/maas-spaces...dimitern:maas-spaces-controller-space-after-master-merge?expand=1
<frobware> dimitern, hmm.
<dimitern> frobware, make check passed, live test as well
<frobware> dimitern, when you say same branch, you mean your github/dimitern branch?
<dimitern> frobware, yep
<dimitern> frobware, so I think I should still drop https://github.com/juju/juju/pull/4255 along with the branch and pass dimitern:maas-spaces-controller-space-after-master-merge through CI instead
<frobware> dimitern, ok, need to coordinate with mgz
<dimitern> frobware, how did you arrange the first ci run? did you push the branch to juju/juju/ ?
<frobware> dimitern, the QA folk helped out
<dimitern> frobware, ok, so then can I leave you to it?
<frobware> dimitern, ok, I need to EOD soon. but I'm hoping mgz has seen "this" conversation
<dimitern> frobware, and just to reduce the confusion I'll drop #4255
<frobware> dimitern, yep
<dimitern> frobware, yeah, hopefully - I'll ping in #juju @c as well
<dimitern> frobware, cheers
<perrito666> bbl
<voidspace> mgz: ping
<rick_h__> perrito666: kind of, in a car atm
<rick_h__> cherylj: will be late to the meeting. in a car between the airport and apt.
<rick_h__> cherylj: don't see thumper to ping if you can share the info please
<cherylj> rick_h__: will do
<wallyworld> cherylj: i may be a touch late to 2.0 meeting
<cherylj> wallyworld: the release standup?
<rick_h__> thumper: cherylj all still up for a chat?
<thumper> rick_h__: happy to, not sure who else is around
<rick_h__> doh hit the end
<thumper> rick_h__: I edited the hangout
<rick_h__> thumper: k
<thumper> so it is the same as our standup
<thumper> rick_h__: I'm in the hangout
<cherylj> rick_h__: we can hear you
<thumper> cmars: ping
<cmars> thumper, pong
<thumper> cmars: can we have a quick call?
<cmars> thumper, certainly
<thumper> cmars: I have questions regarding migration
<thumper> cmars: https://plus.google.com/hangouts/_/canonical.com/casey-tim?authuser=1
<alexisb> changing locations will be back online in about 30 minutes
<wallyworld> cherylj: ah, i thought we had the 2.0 messaging meeting but i'm a week out
<cherylj> wallyworld: ah, ok
<cherylj> I was confused :)
<wallyworld> yeah, i can see why
<davecheney> menn0: perrito666 http://reviews.vapour.ws/r/3700/
<davecheney> PTAL
<menn0> davecheney: looking
<perrito666> looking
<menn0> davecheney: LGTM. You already had a Ship It from me. Land away.
<perrito666> same here
<davecheney> cherylj: 1.25 is blocked, do I have your permission to JFDI ?
<cherylj> davecheney: is this for the pprof stuff?
<cherylj> davecheney: yes, jfdi it.  We want that in 1.25.4
<davecheney> kk
<thumper> alexisb: I have no good internet right now
<thumper> going through my phone's data
<alexisb> fun
<thumper> hangouts suck too much data for my plan
<thumper> I've just rebooted the router to see if it was that
<alexisb> well we will just make all the important decisions without you then
<thumper> ok
<davecheney> wut ?
<davecheney> lucky(~/src/github.com/juju/juju) % juju ssh -
<davecheney> Warning: Permanently added '54.79.206.144' (ECDSA) to the list of known hosts.
<davecheney> nc: getaddrinfo: Name or service not known
<davecheney> ssh_exchange_identification: Connection closed by remote host
<thumper> davecheney: you need to specify a machine right?
<davecheney> https://github.com/juju/juju/wiki/pprof-facility
<davecheney> yes, if you give no machine, it will error out
<davecheney> but - is not a valid number, machine, or unit
<davecheney> that's clearly choosing machine 0
<davecheney> then bombing because it's passing - to the ssh child process
<perrito666> I just lost years of my life merging git...
<menn0> waigani: note that in master the Environment facade was renamed to Model, and then removed completely.
<menn0> waigani: see https://github.com/juju/juju/commit/95a766fd18728ac9331e4f282efcb7861a1e0277
<menn0> waigani: the Agent facade should be used to call EnvironConfig (now called ModelConfig)
<menn0> waigani: you probably want to remerge master into the MADE-workers branch and deal with with the fallout
<menn0> waigani: also note that the proxyupdater has been changed a bit in master too
<thumper> menn0: we need to chat at some stage this afternoon about migration of binaries
<thumper> particularly charms
<menn0> thumper: ok
<thumper> menn0: also, if you are bored - http://reviews.vapour.ws/r/3729/
<thumper> waigani: it looks like you on call reviewer day has dropped off the team calendar
<menn0> thumper: ok, that's on my queue. I want a few other things closed off first
 * thumper heads to lunch
<thumper> menn0: np
#juju-dev 2016-02-04
<waigani> thumper: yeah I don't know why. I'm on next Wed I should be on.
<wallyworld> axw: it was a nightmare yesterday merging master into cloud-crdentials. tl;dr; living the dream, if you notice any glitches, sorry, we can fix as drive by
<axw> wallyworld: ok
<axw> wallyworld: forgot about leads meeting this morning, was waiting for standup. yesterday I got add-user/register working. need to rework some of it to production quality tho
<wallyworld> axw: \o/ that is most awesome
<perrito666> wallyworld: fyi it was a nightmare tonight to :p
<wallyworld> :-(
 * perrito666 is still fixing artifacts
 * wallyworld stabs git in the heart
<perrito666> anyone remembers the key that lets you navigate github like a shell prompt?
<perrito666> wallyworld: does cmd/juju/model/jenv_test.go run for you?
<wallyworld> perrito666: yup
<perrito666> ok, I think I finally merged
 * perrito666 reruns the whole suite
<axw> thumper: kabaddi is .. interesting :)
<axw> looks like there's lots of variations, I have nfi what the one I just watched is
<wallyworld> thumper: yo
<wallyworld> axw: anastasiamac: can you guys spare 15 minutes for a chat?
<axw> wallyworld: yep, standup?
<wallyworld> ok
<thumper> wallyworld: ya?
<wallyworld> thumper: never mind, figured it out :-)
<axw> wallyworld: re base64-encoding username, controller host etc. I have a concern about that. you don't necessarily know which address another user should use. we could encode them all... I'm concerned that'll blow out the line length tho. I'll try it and see.
<wallyworld> ok, we can look to adjust how it may work if needed
<natefinch> huh, I thought our errors.IsNotFound also checked for os.IsNotExist .... it's too bad it doesn't.  so now I have to pick one :/
 * natefinch rages against the fact that errors.NotValidf appends "not valid" to my carefully crafted error string
<menn0> perrito666: better late than never, the Github key you're after is "t" (and "?" pops up some help)
<menn0> thumper: hmmm... the apiserver/controller.Controller interface doesn't appear to get used anywhere
<thumper> ?
<thumper> menn0: I think it was there purely as a contract of the API facade
<thumper> more for documentation purposes than real use
<menn0> thumper: do we want to keep it?
<thumper> I kinda like it, but not attached to it
<natefinch> A Tag tags things that are taggable.
<wallyworld> axw: damn, i forgot to replace state servers with controllers; user visible artifacts are affected, i've had to do a pr for alpha2. it also deletes format 1.16 agent conf support, but apart from that it's purely mechanical http://reviews.vapour.ws/r/3731
<axw> wallyworld: ok
<wallyworld> sorry
<axw> wallyworld: LGTM with one possible change
<wallyworld> awesome, ty
<wallyworld> axw: i'd rather change to "is-controller" and worry about upgrades later. ie get everuthing correct now for 2.0
<wallyworld> since we can't upgrade anyway
<davecheney> soooo
<davecheney> trusty cloud images are broken on ap-southeast-2
<davecheney> looks like launching the current 14.04 image on t2.medium results in an instance that does not know it's own ip address
<axw> wallyworld: do you want me to do the add-user/register bits on cloud-credentials, or just on master?
<axw> wallyworld: because there's nothing really specific to cloud-credentials to it
<wallyworld> axw: don't commit to master right now as it's frozen for alpha2. Bets to stick to out featuure beanch for now i think
<axw> wallyworld: ok
<axw> wallyworld: I have updated the doc just to have --config with two syntaxes. I think we should just implement that, and ask for stakeholder feedback
<axw> wallyworld: the original proposals are preserved in the alternatives section
<wallyworld> axw: sgtm, but i really hope they whinge about having to type --config each time
 * wallyworld off to soccer, bbiab
<axw> wallyworld: :)
<axw> later
<anastasiamac> wallyworld: i hope that "iab" is at least 90min of play time!!
<wallyworld> anastasiamac: it's training for the team i coach, so 90 minutes of me tell them what to do :-)
<anastasiamac> oh, like work then
<wallyworld> lol
<anastasiamac> :P
<voidspace> dimitern: frobware: we haven't had a CI run on maas-spaces since Monday
<frobware> voidspace, yep, the focus has been on the other branch
<voidspace> dimitern: frobware: today we got another on maas-spaces-controller-space-config though, is that expected?
<frobware> voidspace, yes, and it looks way better.
<voidspace> yep
<voidspace> "cannot update controller space in config: cannot set controller-space to "Default space": not a valid space name"
<voidspace> we need space name translation for the controller space
<dimitern> voidspace, yeah, it's expected - I wanted to make sure it wasn't badly broken somehow
<dimitern> voidspace, that's due to the CI job using pre 1.9.0 MAAS (beta3 or 4 I guess)
<voidspace> dimitern: heh, right
<voidspace> dimitern: shouldn't we handle it though?
<dimitern> voidspace, that depends on whether there is a way to handle space names with spaces
<dimitern> voidspace, if acquire node fails when you pass interfaces=0:space=Default space ..
<dimitern> voidspace, I'll do some experiments with that
<frobware> dimitern, I see a bootstrap error today. "failed to bootstrap model: cannot infer controller space name to use: PXE interface "eth0" (52:54:00:c6:17:82) has no links"
<dimitern> frobware, interesting - and the node managed to boot  ?
<frobware> dimitern, yep
<dimitern> frobware, what did the UI show for the node's interfaces?
<frobware> dimitern, but I was staring at my screen thinking... uh-oh and I noticed the console window finish the bridging.
<frobware> dimitern, I didn't think that stage was that async.
<dimitern> frobware, the async aspect is systemd related I believe
<frobware> dimitern, one wonders whether the nonce should get wrriten after we've finished fiddling.
<frobware> dimitern, actually, looking at the cloud-init output that seems to tbe the case already.
<dimitern> frobware, yeah - since the bridge script runs first
<dimitern> frobware, before the tools are downloaded or the nonce saved
<frobware> dimitern, UI shows two bonds, and deployed.
<frobware> dimitern, poor man's UI: http://pastebin.ubuntu.com/14876219/
<dimitern> frobware, well d'oh.. sorry, we should fix that
<frobware> dimitern, what's the issue?
<dimitern> frobware, since there's a bond, we should not just pick the first physical nic with the pxc mac, but also cater for bonds
<frobware> dimitern, I guess yesterday I was not using this node. :(
<dimitern> frobware, that's a good catch sir :)
<frobware> dimitern, I happened to release all my nodes; that was previously my 1.25 vlan/bond testing node.
<dimitern> frobware, so the logic around pxe nic detection should include: mac=pxe_mac, type=physical && len(children) == 0 || type=bond && len(parents) > 0, and having a link to subnet
<frobware> dimitern, I can take a look now
<voidspace> frobware: do you have the company VPN setup?
<dimitern> frobware, and there's the odd chance of picking a disabled nic (although we should see if this works in practice - disabled pxe interface)
<frobware> voidspace, yes
<voidspace> frobware: I followed the instructions here and it doesn't work, I get "VPN connection failed"
<voidspace> frobware: https://wiki.canonical.com/InformationInfrastructure/IS/HowTo/CompanyOpenVPN
<voidspace> frobware: did you set it up through Network Manager as described there?
<frobware> voidspace, yep
<frobware> voidspace, maybe a 15.10 issue
<dimitern> voidspace, one of the failures was functional-jes, which failed ultimately with "spaces discovery still in progress"
<voidspace> frobware: harrumph
<voidspace> dimitern: hmmm, odd
<dimitern> voidspace, so we should investigate how we're dealing with multiple models in play
<voidspace> dimitern: will look
<voidspace> dimitern: oh god
<voidspace> dimitern: I have no idea :-)
<voidspace> dimitern: I will look into it
<dimitern> voidspace, e.g. bootstrap went fine, then discovery was skipped as expected, followed by create-model, which exposed discovery blocking
<voidspace> dimitern: frobware: this tests the fix from yesterday - it passes with the new code and fails with the old code
<dimitern> voidspace, great!
<voidspace> dimitern: why was discovery skipped?
<dimitern> voidspace, the job runs on joyent
<voidspace> dimitern: ok
<voidspace> dimitern: so it should only be possible for login to be blocked if space discovery has started
<voidspace> dimitern: I'll check all code paths to see if that's really true
<voidspace> dimitern: is this with yesterdays fix in?
<frobware> voidspace, has to be yes -- we had so many "success" results today.
<voidspace> *grr*
<voidspace> frobware: thanks
<dimitern> voidspace, yes - have a look at http://reports.vapour.ws/releases/3569/job/functional-jes/attempt/620 - around 03:44:30
<voidspace> dimitern: yeah, looking
<voidspace> dimitern: I see two code paths that don't close the channel (and need fixing)
<voidspace> dimitern: an error calling api.ModelConfig and calling envrions.New
<voidspace> dimitern: but I doubt they're the cause of the problem
<voidspace> dimitern: I'll fix those in the current branch anyway
<dimitern> voidspace, I think that's exactly what it is, considering the error right after that is 'model "functional-jes-env2" not found'
<voidspace> dimitern: that'll be it then...
<dimitern> voidspace, but all code paths need to be tested, yeah
<dimitern> frobware, voidspace, dooferlad, are we doing the weekly call?
<frobware> dimitern, I'm skipping. want to concentrate on the build/test/merge.
<dimitern> frobware, ok, I think I'll do the same
<voidspace> ok
<dooferlad> dimitern, frobware, voidspace: I can go and report back.
<frobware> dimitern, voidspace, dooferlad: I would like to get a CI build kicked off during our day if we have additional fixes today.
<voidspace> I'll go as well
<voidspace> frobware: yep
<voidspace> dimitern: I've changed the channel close to be done in a defer (with a "maybeClose" function - so I don't have to play whack-a-mole with code paths
<frobware> dooferlad, ack & thanks.
<dimitern> voidspace, before calling anything that might fail, right?
<voidspace> dimitern: yep
<dimitern> voidspace, +1
<voidspace> dimitern: and as a "few" of the exits are tested, the defer is now tested
<dimitern> voidspace, awesome
<dimitern> voidspace, now, it might be worth jiggling the test around a bit
<voidspace> dimitern: in terms of?
<dimitern> voidspace, e.g. causing the dep engine to bounce the workers a few times before checking ultimately discoverspaces stops
<dooferlad> dimitern, frobware, voidspace: is there any value in a quick standup?
<dimitern> voidspace, it seems that when issues like that surface - e.g. the api caller dependency dies and pulls the other workers along
<voidspace> dimitern: ok, interesting - is there an example test I can look at that does this?
<voidspace> dimitern: otp
<dimitern> voidspace, there should be - looking
<dimitern> voidspace, have a look in worker/dependency/engine_test.go
<voidspace> dimitern: ok
<voidspace> dimitern: this should only be an issue for bootstrap - the bootstrap code is the only path that *checks* this channel
<voidspace> dimitern: so bouncing the workers shouldn't block anything
<voidspace> dimitern: maybe worth checking that discovery still completes if the worker is interrupted
<dimitern> voidspace, isn't there a singular runner per model?
<voidspace> dimitern: well, yes - but only one channel
<voidspace> dimitern: interesting, I'll have to look at this code
<dimitern> voidspace, then it can happen during/right after model creation, which is essentially the same as bootstrap
<voidspace> dimitern: it's only limitLogins that is affected though, and that is only used during bootstrap itself I *believe*
<voidspace> dimitern: that's where the channel is checked for closing
<voidspace> dimitern: I'll look at that code and try and understand the ramifications of jes here
<voidspace> (one channel per agent)
<voidspace> dimitern: so it should only be discovery on the JES controller itself that blocks anything
<voidspace> discovery for other models shouldn't block - that would be the intent anyway, and I think also the code
<frobware> dooferlad, I would say yes, but I see a lot going on here. dimitern, voidspace: want a quick standup?
<dimitern> frobware, dooferlad, voidspace, yeah, let's do it quickly
<voidspace> frobware: dooferlad: dimitern: if we can actually be quick :-)
<voidspace> omw
<voidspace> frobware: dimitern: dooferlad: http://reviews.vapour.ws/r/3735/
<voidspace> dooferlad: my network manager logs, showing the problem with openvpn (not very helpful) http://paste.ubuntu.com/14876407/
<voidspace> dooferlad: any ideas?
<frobware> mgz, ping
<voidspace> dooferlad: ah, now I get a missing file error for tls-auth
<voidspace> dooferlad: dimitern: frobware: fixed the problem (file in wrong place) and now it works :-)
<dooferlad> voidspace: sweet!
<voidspace> dooferlad: dimitern: frobware: so should be able to test bootstrapping to openstack
<voidspace> we'll see
<frobware> voidspace, dimitern, dooferlad: we seem to dump a lot of hex output to the logs - is this necessary / useful?
<dooferlad> frobware: example?
<voidspace> frobware: no...
<frobware> dooferlad, let me find an example in the CI reports
<frobware> dooferlad, http://data.vapour.ws/juju-ci/products/version-3569/maas-1_9-deploy-trusty-amd64/build-1900/consoleText
<dooferlad> frobware: looks useless
<frobware> personally I think we should drop this
<dimitern> frobware, that was the result of adding more logging
<dimitern> frobware, and indeed it was temporary to debug the trusty issue if it happens, will drop it
<frobware> dimitern, thanks
<dimitern> voidspace, you've got a review
<dimitern> voidspace, however, I've just discovered this:
<dimitern> 210669b2-3a14-411c-847a-b5ec79f4803e machine-0: 2016-02-04 11:02:27 ERROR juju.worker runner.go:229 exited "discoverspaces": error from CreateSpaces: adding space "admin": ProviderId "admin" not unique
<dimitern> voidspace, I bootstrapped, then created another model - it seems discover spaces runs there as well
<dimitern> voidspace, we have to deal with this to allow multiple models to work with spaces, either by dropping the requirement for provider-id uniqueness, or alternatively always storing the providerID with model-uuid like for doc-id
<dimitern> space names with spaces can be both created and used for selection of nodes, fortunately selection can use the ID just the same
<jamespage> anyone know if there is a plan to have the juju cli be able to export a bundle?
<jamespage> juju-deployerizer exists but that's a stop-gap IMHO
<jamespage> (thankyou for that niedbalski :))
<perrito666> morning all
<voidspace> dimitern: space discovery should run for the new model, shouldn't it?
<voidspace> dimitern: we knew we would have to face this (non-unique provider id
<perrito666> wallyworld: tx for the extra set of eyes on that XDG merge
<perrito666> I really needed that
<voidspace> dimitern: do you want me to move the doc comment to the variable (in my PR)?
<dimitern> voidspace, yes, I'm trying to fix this in my branch actually
<dimitern> voidspace, yes please
<voidspace> dimitern: provider id uniqueness?
<dimitern> voidspace, uniqueness by prefixing with model uuid
<dimitern> voidspace, experimenting with this now
<voidspace> dimitern: cool
<dooferlad> frobware/dimitern/frobware: our branch is missing https://github.com/juju/juju/commit/462a3d6e3c1a7a67085d6a092bafe0f0dd21b679 and that seems to be the problem the manual deploys are running into
<dimitern> dooferlad, great! thanks for tracking this down
<dimitern> dooferlad, can you propose a PR to merge this please?
<dooferlad> dimitern: yea, I am just resolving all the conflicts caused by renames
<dimitern> dooferlad, cheers
<frobware> dooferlad, heh. so I had done an independent merge yesterday and this was one of my diffs against dimiter's version.
<dooferlad> frobware/dimitern I am seeing lots of networker being deleted and environ/model renaming. Wasn't that already merged?
<dooferlad> maybe this is a git suckage problem
<dimitern> dooferlad, it's not merged in master
<dooferlad> yes, but shouldn't we already have fixed this stuff? We should have already got the renames, right?
<frobware> dooferlad, which branch are you looking at? maas-spaces or maas-spaces-controller-space-after-master-merge
<dooferlad> frobware: maas-spaces
<frobware> dimitern: the latest merge of master is all in maas-spaces-controller-space-after-master-merge, correct?
<frobware> dooferlad, ^
<natefinch> ericsnow: standup?
<dimitern> frobware, yeah, that's the one
<frobware> dimitern, dooferlad: this is what I'm testing and making changes against.
<dimitern> frobware, me too
<dooferlad> dimitern, frobware: why does that branch still exist? Why not go back to maas-spaces?
<frobware> dooferlad, dimitern: that's the plan, but it generally wasn't ready and we wanted to keep going with maas-spaces. we needed somewhere to work for both cases.
<frobware> dooferlad, the plan _is_ to go back to maas-spaces.
<dooferlad> frobware: so, should I propose a merge of master to maas-spaces-long-name-of-doom?
<dooferlad> frobware: I just don't want to replicate more work :-|
<frobware> dooferlad, I would say no...
<frobware> dooferlad, can we cherry-pick the change you identified?
<dooferlad> frobware: sure, but ISTR not much happened on master last night
<frobware> dooferlad, the reason I say "no" to a merge is that we have 4 identified failures. I didn't want to perturb the current state too much.
<dooferlad> frobware: maas-spaces-controller-space-config was tested last night. We were just talking about another branch. Which state are we trying not to perturb?!
<frobware> dooferlad, I'm in favour of a cherry-pick, plus michael's changes for discovery and dimiter's "stuff" means we should get down to either 1 or 0 failures.
<frobware> dooferlad, let's HO
<dooferlad> frobware: sure
<frobware> dooferlad, I'm done with typing. :)
<frobware> dimitern, ping - can you join the sapphire HO?
<dimitern> frobware, sure, omw
<voidspace> dooferlad: ping
<dooferlad> voidspace: pong
<voidspace> dooferlad: that was a test ping
<voidspace> dooferlad: thanks for the ack
<dooferlad> voidspace: :-)
<voidspace> dooferlad: I thought my connection had died and XChat was just being slow to catch up
<voidspace> it does that sometimes. Long timeout on connection.
<natefinch> Man I wish we had something other than params.Entity to use in the API.... especially for the 95% of the time when it has to be one specific type.
<perrito666> if you know what you are expecting just check it
<alexisb> perrito666, on a call w/ curtis and team; can you please rebase your mongo3 branch on the current master?
<alexisb> perrito666, cherylj and curtis are working to get a bless so we can merge
<perrito666> alexisb: I certainly can, you need that now?
<alexisb> today please
<perrito666> k, on it
<alexisb> thnaks
<alexisb> perrito666, please ping cherylj once it has been merged
<perrito666> alexisb: you want me to just put mongo3 branch up to date with master ?or to merge back that into master too
<alexisb> rebase off master
<alexisb> so we can get a fresh ci run
<perrito666> ah ok, ill do it
<perrito666> then roll into a corner of the room and cry
<natefinch> ericsnow: I keep thinking... we keep passing around the size of the resources... but why do we care?  The fingerprint ensures we have the right data... size really only matters possibly if we ever want to display it to the user as part of the info about a resource.
<ericsnow> natefinch: that and the blobstore sometimes only gives us the size (so we can't verify against the fingerprint)
<natefinch> ericsnow: ahh, hmm... weakness of the blobstore.  That's too bad
<ericsnow> natefinch: yep, though it doesn't hurt to store the size (accessible via YAML and JSON output)
<natefinch> ericsnow: yeah, totally is something we should store, just could be calculated on the server, instead of passing it up with requests.
<ericsnow> natefinch: true
<natefinch> ericsnow: what's the pendingID on UploadRequest for? I thought the point of the pending resources is that you *can't* generate an ID on the client side, because it won't be guaranteed to be unique on the server
<ericsnow> natefinch: the ID is the one you get from the server by calling "AddPendingResources()" before trying to upload
<cherylj> tych0: ping?
<tych0> cherylj: hi
<natefinch> ericsnow: oh, so it's two steps - add the pending metadata and then upload the bytes for that metadata?
<cherylj> hey tych0, I'm going to help you get your PRs into a feature branch, but need you to do a couple things for me first.
<ericsnow> natefinch: yep
<tych0> sure, sounds good
<cherylj> tych0: can you rebase off of master and create one PR that has both 4191 and 4131?
<tych0> cherylj: 4191 has both and has been rebased off of master yesterday, i can rebase again though
<cherylj> tych0: ah, then that should be fine.  Don't think that it's changed since then
<tych0> cherylj: just building the rebase now, i'll push when it builds successfully
<cherylj> tych0: ok, thanks.  I can make a feature branch off if 4191 and you can merge the change for 4131 into that branch
<cherylj> then we'll get a CI test run on that branch before we merge it into master
<tych0> cherylj: 4191 has both already
<tych0> i was planning to rebase if 4131 ever got merged
<cherylj> tych0: then maybe I didn't need you to do anything!  :)
<tych0> cherylj: ok, 4191 should be rebased on master as of a minute ago
<tych0> cherylj: np :)
<cherylj> kk, will get it in feature branch for testing.  Thanks!
<tych0> cherylj: sure, let me know if you need anything else
<tych0> i'll be in and out today, but happy to fix stuff as needed
<cherylj> thanks, tych0!
<natefinch> I kind wish the we had client-side logging
<natefinch> I mean, I guess we kinda do - but I wish we actually saved it somewhere.
<cherylj> tych0: the feature branch is up!  https://github.com/juju/juju/tree/lxd-container-type
<cherylj> tych0: you should also add go fmt to your pre-push script
<perrito666> anyone could hint me what is the new name of StateServerInfo?
<perrito666> ill guess its ControllerInfo
<natefinch> ericsnow: why do we need ListModelResources?  It looks like the only additional information it returns that ListResources doesn't, is the storage path... is that right?
<ericsnow> natefinch: and PendingID
<natefinch> ericsnow: but for ListModelResources PendingID is always empty
<ericsnow> natefinch: yep
<natefinch> ericsnow: so, it seems like a lot of mental overhead for just one additional value
<perrito666> bbl
<natefinch> ericsnow: seems like we don't even really need ModelResource if we just add ServiceID to resource.Resource
<natefinch> ericsnow: and storagepath I guess
<ericsnow> natefinch: and PendingID
<ericsnow> natefinch: perhaps I'm missing something, but is your concern that we have a type that contains multiple fields instead of returning each of those fields separately from the related methods?
<natefinch> ericsnow: we never use pendingID outside the database
<ericsnow> natefinch: and in state
<natefinch> ericsnow: maybe it just needs a better name?
<ericsnow> natefinch: fine with me :)
<natefinch> ericsnow: it just seems like most of the data in ModelResource isn't really ever used.  StoragePath seems to be there just to avoid having to calculate the storagepath each time.  ServiceID could easily be on resource.Resource.  PendingID is unique by definition, so shouldn't need any of the surrounding data most of the time.
<ericsnow> natefinch: StoragePath is necessary because when a pending resource is made active in the AddService() transaction, we do not change the storage path (and it cannot be re-calculated because the pending ID is gone)
<natefinch> ericsnow: that's fair.  Still seems like something that could be on resource.Resource
<ericsnow> natefinch: So you're suggesting that Resource and ModelResource be merged together?
<ericsnow> natefinch: the catch is that resource.Resource is the more user-facing type and ModelResource includes info that isn't meant for users
<natefinch> ericsnow: yeah.... it seems like there's very little benefit to having them in two types, and personally I find it confusing as to when I'd want one or the other, since they're so close
<ericsnow> natefinch: (namely PendingID and StoragePath)
<natefinch> ericsnow: right... though both those could, in theory, be useful if converted to something like a boolean for IsPending
<ericsnow> natefinch: It has to be an ID since there can be more than one pending for a given resource ID
<ericsnow> natefinch: trust me, I tried the boolean first :)
<natefinch> ericsnow: right, I just meant, since we were talking user-facing (I'm thinking conversion like for tabular output)
<ericsnow> natefinch: users should never see the pending ID except when it comes back from an AddPendingResource API call
<natefinch> ericsnow: right, and I don't think they will.  The API types don't need to expose it
<natefinch> ericsnow: the API being the best definition of what is userfacing
<davecheney> thumper: yup, I had an environment come up that was not listening on port 17070
<thumper> davecheney: well that port can be configured right?
<thumper> why is that an issue?
<davecheney>   manual3:
<davecheney>     type: manual
<davecheney>     bootstrap-host: ec2-54-79-138-39.ap-southeast-2.compute.amazonaws.com
<davecheney>     bootstrap-user: ubuntu
<davecheney> this is my config
<davecheney> if there is a default, it didn't default to what it should have defaulted to and/or the client tool did not respect the default
<thumper> ok, that is weird
<natefinch> ericsnow: can you explain modelID to me?  (I know you're changing the name, but the concept presumably stays the same)
<ericsnow> natefinch: its the identifier for the resource in the model, i.e. the resource ID
<thumper> can we please not call it modelID?
<natefinch> ericsnow: how is thast different than the doc ID (for non-pending resources) or the pendingID (for pending resources)?
<natefinch> thumper: already been brought up and I think fixed
<ericsnow> natefinch: FYI, I'm working up a patch that folds the ID and service ID into resource.Resource
<natefinch> ericsnow: ok
<ericsnow> natefinch: my aim is to eliminate resource.ModelResource
<natefinch> ericsnow: awesome. I really think that will simplify the code a lot.  It might be 5-10% imperfect, but I think it'll be a lot easier to work with and understand.
<ericsnow> natefinch: thanks for bringing it up
<davechen1y> thumper: this isn't a fw issue
<davechen1y> the port is not open
<davechen1y> i checked with `netstat -anp` and `ss
<tych0> cherylj: ah, cool, thanks
<tych0> cherylj: what steps to do we do from here to get it merged?
<cherylj> tych0: there should be a CI run happening now for your branch.  If it gets a bless (or realistically, shows the same failures as master), we'll merge it for you
<cherylj> tych0: otherwise, we'll need you to fix the failures
<tych0> cherylj: ok, cool. that sounds entirely reasonable :)
<alexisb> tych0, I am going to set up some time to chat next week as well, so we can check point where we are for removing lxc
<tych0> alexisb: ok, sounds good. there's definitely some stuff to be finished still
<alexisb> yep and I know jam has been doing some testing
<perrito666> its a good thing I decided to update my laptops ubuntu, I was tired of being able to boot without getting a video hang
<natefinch> perrito666: lol
<perrito666> well, and xorg seems to be under the impression that its doing a great job
<perrito666> off cource, intel video driver cause a nil pointer dereference
<natefinch> yay
<natefinch> ericsnow: thanks for the clarification about what the resource ID means.
<ericsnow> natefinch: np :)
<natefinch> ericsnow: and I agree, it makes sense to canonicalize the ID in a real field, instead of it being implicitly servicename + resourcename
<menn0> natefinch: I just replied to some of your replies for http://reviews.vapour.ws/r/3699/
<alexisb> cherylj, tych0 looks like the lxd broker branch had a fun first test run ;)
<cherylj> heh, yeah, but we know what the issue it :)
<cherylj> issue IS
<tych0> cherylj: so what happens when i push to my branch?
<tych0> does that get automatically pushed and re-run? are the results available somewhere?
<cherylj> tych0: you'll want to submit a PR to that feature branch
<cherylj> tych0: then once it gets merged, CI will see that it's changed and add it into the rotation
<tych0> cherylj: ok, so should i close my other PRs then?
<cherylj> tych0: did you just update dependencies.tsv?  or are you updating the lxc package?
<cherylj> tych0: yes
<tych0> cherylj: working on updating lxd
<tych0> cherylj: ok, will do
<cherylj> tych0: then once you have the lxd package updated, you'll want to update the juju dependencies.tsv to pull in that new lxd
<tych0> cherylj: yup
<cherylj> k :)
<mup> Bug #1538241 changed: 2.0-alpha2 stabilization <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
<mup> Bug #1538241 opened: 2.0-alpha2 stabilization <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
<mup> Bug #1538241 changed: 2.0-alpha2 stabilization <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
<wallyworld> perrito666: standup?
#juju-dev 2016-02-05
<alexisb> axw, ping
<thumper> fwereade_: you working?
<alexisb> thumper, I just sent him that same question over mail
<thumper> heh
<thumper> I saw the push
 * thumper stares evily in wallyworld's direction
 * thumper mutters something about renames
<thumper> wallyworld: does the client facade not exist any more?
<wallyworld> thumper: it does, but has gone on a diet
<wallyworld> it also is version 1
<thumper> where are the broken out facaces?
<wallyworld> all of the service methods are moved off to the service facade. before only half the methods had been moved across
<wallyworld> not all facades are broken out
<wallyworld> just the existing service facade had the remaining methods moved
<wallyworld> not enough time to do everything else
<thumper> hmm...
<wallyworld> i can ask superman to make the earh spin slower
<wallyworld> so we get 30 hours in a day
<wallyworld> i thought it best to complete the half done work for serivce
<wallyworld> remaining stuff will just have to wait
<thumper> wallyworld: can we have a call?
<wallyworld> sure
<thumper> I think talking through what I need will be heaps faster
<thumper> wallyworld: lets jump in our 1:1
<axw> alexisb: pong, sorry, was doing school stuff
<alexisb> axw, nws no rush, I am off for the day I will catch you next week
<axw> alexisb: okey dokey, have a nice long weekend. ttyl
<anastasiamac_> alexisb: have fun :D
<anastasiamac_> axw: did u have a chance to see my msgs?
<anastasiamac_> axw: wallyworld: can't bootstrap on FB at the moment - have to specify both controller name and cloud..
<axw> anastasiamac_: in 1:1, I'll help you later
<anastasiamac_> k
<thumper> wallyworld: \o/
<thumper> wallyworld: the test to check all the read only calls found two that were wrong
<mup> Bug #1542127 opened: CPC sjson triggers failed to parse public key: openpgp: invalid argument: no armored data found <bootstrap> <ci> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1542127>
<wallyworld> thumper: awesome, yay, i look forward to it landing so i can adjust those apis names
<natefinch> cherylj: re: min juju version... uh... in theory all it needs is to merge master into the feature branch and run through CI (and fix whatever falls out of that). However, we're also busting our butts to get resources in, so I don't know if I'll have the time.  I'll certainly try.
<mup> Bug #1542127 changed: CPC sjson triggers failed to parse public key: openpgp: invalid argument: no armored data found <bootstrap> <ci> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1542127>
<axw> anastasiamac_: "juju bootstrap <controller-name> lxd --upload-tools" should just work
<anastasiamac_> axw: tyvm... i'll look in a sec
<mup> Bug #1542127 opened: CPC sjson triggers failed to parse public key: openpgp: invalid argument: no armored data found <bootstrap> <ci> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1542127>
<thumper> axw: do we now create two models when bootstrapping?
<thumper> axw: is that bootstrap in master, or a feature branch?
<axw> thumper: not just yet
<axw> thumper: feature branch
<thumper> k
<thumper> looking forward to it though
<thumper> looking forward to it though
<thumper> ugh
<axw> thumper: I'll let you know when it's ready, if you want to play
<thumper> cheers
<mup> Bug #1542131 opened: Juju ignores index2.sjson in favour of index.json <bootstrap> <ci> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1542131>
<cherylj> natefinch: it's no problem to move min juju version to beta 1.  Just wanted to check on its status
<natefinch> cherylj: I don't know when beta 1 is, but I know feature freeze is the 16th and that's what I was talking about as being questionable.
<cherylj> natefinch: oic
<natefinch> cherylj: 12 days from now comes up pretty quick :)  But like I said... I really want it in, so I'll do my best.
<cherylj> thanks, natefinch
 * thumper afk for a bit
 * thumper back
<thumper> I'm  merging master into the model-migration
 * thumper sighs
<cherylj> that bad?
<thumper> only minor conflicts so far
<thumper> but I know some other changes are needed in my code
<thumper> so off to rename all my bits
 * menn0 imagine thumper renaming his body parts
<cherylj> I didn't want to go there...
<menn0> so is InitiateModelMigrationResults too unwieldy as a type name? :)
<menn0> thumper: ^^
<thumper> menn0: nah
<menn0> thumper: good. review for http://reviews.vapour.ws/r/3745/diff/ then please
<thumper> not a shitty review?
<natefinch> menn0: IMO, the model part there is probably redundant.  I presume there's no other migrations going on in that section of code.. I'd probably call it InitMigrationResults.
<natefinch> menn0: oh, I guess if it's in the API, maybe you do need the model part.
<menn0> natefinch: yeah, it's based on the API name that returns that result
<menn0> the API is InitiateModelMigration
<natefinch> menn0: yeah, figured that out after I typed it.  I wish our API types were partitioned into a package per facade, so we didn't need java style naming.
<menn0> natefinch: yep that would be much nicer. instead of having the one params package with all of them thrown together.
<menn0> thumper: and another one http://reviews.vapour.ws/r/3746/
<davecheney> https://github.com/juju/juju/pull/4305
<natefinch> cherylj: is there a task for someone to rename our various -unstable repos to no longer be called unstable for 2.0?
<davecheney> that's a very unwise question to be asking
<cherylj> natefinch: not that I'm aware of
<natefinch> cherylj: I figured
<natefinch> davecheney: heh, well, I'd be happy to fix it if I had time.  I never liked using -unstable anyway.
<wallyworld> thumper: so your branch landed, i can do that rename?
<wallyworld> i se eone landing, was that the one
<mup> Bug #1287718 changed: jujud on machine 0 stops listening to port 17070/tcp WSS api <cts-cloud-review> <mongodb> <state-server> <sts> <juju-core:Expired> <https://launchpad.net/bugs/1287718>
<mup> Bug #1469193 changed: juju selects wrong address for API <kvm> <local-provider> <lxc> <network> <sts> <juju-core:Expired> <https://launchpad.net/bugs/1469193>
<davecheney> cherylj: 1287718, that's what I am seeing trying to replicate an issue with the manual provider
<davecheney> why does juju use websockets when neither the server, or the client are a browser ?
<natefinch> davecheney: just in case?
<natefinch> davecheney: to be fair, one of the clients is a browser
<davecheney> so we have our own encoding scheme over net/rpc over websockets over https
<davecheney> not simple
<natefinch> Juju: It's Not Simpleâ¢
<davecheney> what a catchcry
<wallyworld> axw: a one pager! fixes the recently migrated Service facade method names http://reviews.vapour.ws/r/3749
<axw> wallyworld: looking
<wallyworld> am looking at yours too
<wallyworld> gawd, that sounds suss
<axw> mine's bigger?
<wallyworld> no mine is! more files
<axw> wallyworld: LGTM, just one changed method name in a comment needs to be capitalised
<wallyworld> ty
<wallyworld> axw: looks great
<axw> wallyworld: thanks
<wallyworld> np
<axw> wallyworld: what are you looking for in a feature test? I think the TestRegister checks everything we can at the moment
<wallyworld> axw: actualy, yeah, i was thinking add-user, then register, then login but we're not there yet
<wallyworld> ignore me
<axw> wallyworld: yep. I'll add one when I've finished the register branch
<wallyworld> ta, sorry premature test request :-)
<thumper> wallyworld: yes, that was the one
<wallyworld> thumper: ta, i'm about to land my update, got to update romulus dependency first
<voidspace> dimitern: hey, did you land my branch on controller-space-config?
<voidspace> dimitern: functional-jes was still failing last night
<dimitern> voidspace, yes, your last change is in maas-spaces-controller-config
<voidspace> hmmm
<dimitern> voidspace, and it seems the spaces discovery still blocked during create-model, but not during bootstrap
<voidspace> dimitern: ok, needs more looking at then
<voidspace> dimitern: where are you seeing that, it doesn't seem to be in the logs for machine-0/1/2
<dimitern> voidspace, I'm looking at http://reports.vapour.ws/releases/3574/job/functional-jes/attempt/626
<voidspace> dimitern: me too
<dimitern> voidspace, don't you see the artifacts section with all the logs in the beginning?
<voidspace> dimitern: yes and I looked through the logs of machine-0, machine-1 and machine-2
<voidspace> dimitern: couldn't see mention of space discovery
<voidspace> I may have just missed it
<voidspace> also not in the console output
<dimitern> voidspace, true.. let me see where I found that..
<voidspace> nor logsink
<dimitern> voidspace, there http://data.vapour.ws/juju-ci/products/version-3574/functional-jes/build-625/consoleText
<dimitern> voidspace, and in the other later attempt: http://data.vapour.ws/juju-ci/products/version-3574/functional-jes/build-625/consoleText
<dimitern> voidspace, but not in the last attempt 626
<voidspace> the first two links are the same
<voidspace> dimitern: so it *isn't* a problem in the most recent build
<voidspace> which is what I would hope
<dimitern> voidspace, sorry - 625 and 624 are the jobs I meant
<voidspace> hmmm... 625 was today
<dimitern> voidspace, all 3 runs on functional-jes though - 624, 625, and 626, were all with the most recent build with your fix
<voidspace> oh no it wasn't
<voidspace> (necessarily)
<voidspace> dimitern: really, odd
<voidspace> dimitern: I'm seeing run 3574 on Friday which has build 626 (not discovery)
<voidspace> dimitern: run 3569, on Thursday and before my fix, build 620
<voidspace> dimitern: ah, now I see them
<voidspace> dimitern: so if a test fails with a build it does multiple attempts?
<dimitern> voidspace, yeah, I guess it depends on how the job is set up
<voidspace> dimitern: ok, I'm looking into it - will see if I can reproduce it
<dooferlad> dimitern: hangout?
<dimitern> dooferlad, omw
<rick_h__> jam: ping
<voidspace> rick_h__: jam isn't usually around on Friday
<rick_h__> voidspace: ah, gotcha
<rick_h__> thanks
<voidspace> dimitern: frobware: dooferlad: my connection died
<frobware> dimitern, if we're changing how bootstrap space discovery we won't need this (http://pastebin.ubuntu.com/14886630/) anymore. Preserved should we need it.
<dimitern> frobware, yeah, but that's one heuristic less to deal with :)
<perrito666> morning
<anastasiamac_> perrito666: \o/
<voidspace> dimitern: frobware: so, on a naive test I can bootstrap to joyent and create a new model
<voidspace> dimitern: frobware: digging into what this test does that might be different
<voidspace> ah, it does a bunch of deployments
<voidspace> and it's the deploy that fails with the error
<voidspace> that obviously involves starting machine agents
<voidspace> it shouldn't involve starting a discovery worker though!
<dimitern> voidspace, the discovery worker is in the MA
<voidspace> dimitern: yes, but only with job ManageEnviron
<dimitern> voidspace, yeah, which is true for the bootstrap model, but also for the created model later
<voidspace> dimitern: but the failure is not during create model, but during deployment afterwards
<dimitern> voidspace, not sure if we run jujud per model or 1 instance that handles both
<voidspace> dimitern: the space discovery error is only a warning
<voidspace> dimitern: the error  is:
<voidspace> ERROR cmd supercommand.go:448 model "functional-jes-env2" not found
<dimitern> voidspace, yeah, that's troubling
<voidspace> ERROR Exception while environment "functional-jes-env2" active
<voidspace> investigating
<dimitern> voidspace, it might be that the model is not yet created when we try to see if it supports networking?
<dimitern> unlikely, but..
<voidspace> with the fix in place an error there should still terminate space discovery
<voidspace> but that *could* set a new channel on the agent, if it keeps getting bounced
<voidspace> I can add a check not to set a new channel if there's one there
<dimitern> voidspace, I was thinking along these lines as well
<dimitern> but anyway..
<voidspace> so, if we're bounced we don't keep waiting
 * dimitern steps out for ~1h btw
<voidspace> I think I can test that too
<voidspace> basically test for race conditions like that
<voidspace> although then we should protect the channel with a mutex
<perrito666> I wonder why psus dont come with sensors I can monitor from the pc, it would be very useful in a country such as this that has high temperatures
<cherylj> tych0: you around?
<mup> Bug #1542336 opened: cannot find package "github.com/gosexy/gettext" <blocker> <ci> <packaging> <regression> <test-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1542336>
<mup> Bug #1542336 changed: cannot find package "github.com/gosexy/gettext" <blocker> <ci> <packaging> <regression> <test-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged by tycho-s> <https://launchpad.net/bugs/1542336>
<mup> Bug #1542336 opened: cannot find package "github.com/gosexy/gettext" <blocker> <ci> <packaging> <regression> <test-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1542336>
 * dimitern is back
<mup> Bug #1542336 changed: cannot find package "github.com/gosexy/gettext" <blocker> <ci> <packaging> <regression> <test-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged by tycho-s> <https://launchpad.net/bugs/1542336>
<mup> Bug #1542336 opened: cannot find package "github.com/gosexy/gettext" <blocker> <ci> <packaging> <regression> <test-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1542336>
<cherylj> can someone review my patch to revert the change that broke master?  http://reviews.vapour.ws/r/3752/
<voidspace> cherylj: LGTM
<cherylj> thanks, voidspace !
<dimitern> cherylj, +1 from me as well
<cherylj> tyvm, dimitern!
<voidspace> dimitern: so, I have a fix and a test that proves it
<voidspace> dimitern: just factoring out code common to the two agent tests that test discovery
<dimitern> voidspace, great! what was the crux of the issue?
<voidspace> dimitern: restarting the worker no longer replaces the discovery channel
<voidspace> dimitern: well, that's a fix for what I *think* the error must be
<voidspace> I didn't reproduce as such
<voidspace> ...
<voidspace> but given the new code it *must* be the problem
<voidspace> if discovery is indeed the cause of the error
<voidspace> dimitern: when this is up for review I will put more time back into getting the deploy to work locally so i can attempt to repro and see if this fixes it
<voidspace> it's a good fix *anyway*
<voidspace> as the test proves
<dimitern> voidspace, right, so was it due to jingling and bouncing workers?
<voidspace> dimitern: if you bounce the worker and restart, the closed channel was replaced with a new one
<voidspace> dimitern: so repeatedly bouncing the worker could cause the api to be permanently blocked
<dimitern> voidspace, nasty :/
<voidspace> dimitern: I protected access to the channel with a mutex
<voidspace> probably not strictly necessary as it's only set in one place - but at least it's goroutine safe now
<dimitern> voidspace, I doubt that'll help though
<dimitern> voidspace, what does the mutex aim to prevent?
<voidspace> dimitern: *plus* I don't set it if it's already set
<voidspace> that's the key thing
<voidspace> the mutex makes *that* goroutine safe
<dimitern> voidspace, ah! that sounds like closer to it
<voidspace> let me show you
<voidspace> the test makes it clear what I'm protecting against
<dimitern> ok, I'll have a look when you're readt
<dimitern> ready even
<voidspace> dimitern: http://reviews.vapour.ws/r/3753/
<voidspace> I have to go pick up my daughter from school
<frobware> dimitern, voidspace, dooferlad: did you see the latest maas-spaces CI run?
<voidspace> dimitern: I have checked - the new test fails with the old code, the API remains blocked
<voidspace> frobware: good?
<frobware> dimitern, voidspace, dooferlad: foreget it -- it's not final.
<voidspace> hah
<voidspace> frobware: http://reviews.vapour.ws/r/3753/
<voidspace> back soon
<tych0> cherylj: hello
<cherylj> hey tych0, were you working on lxd to not pull in github.com/gosexy/gettext?
<cherylj> it seems jam merged your PR into master last night
<tych0> cherylj: yeah, it's done, just didn't get merged until this morning
<tych0> i'll send a juju patch shortly
<cherylj> tych0: cool, thanks. Remember to make it against your feature branch.
<tych0> even thought the other stuff landed in master?
<tych0> or did that get reverted?
<cherylj> I reverted the merge of 4131 from master
<tych0> ok
<cherylj> tych0: you may also want to merge master into your feature branch as a lot of stuff landed yesterday.
<tych0> ah, yes
<tych0> lots of conflicts.
<tych0> sweet
<cherylj> :(
<tych0> cherylj: https://github.com/juju/juju/pull/4313
<ericsnow> natefinch: oh, and were you going to help tych0?
<natefinch> ericsnow: yes, but it sounds like everything is pretty much under control
<ericsnow> tych0: ^^^ ?
<tych0> ericsnow: i'm not sure, cherylj is probably the person who knows best. i don't fully understand the juju process here :)
<ericsnow> tych0: no problem :)
<natefinch> cherylj, tych0: let me know if you need any help
<tych0> natefinch: cool, thanks
<ericsnow> tych0: for that PR (#4313), it looks like you included the merge from master...the actual change isn't that big or invasive, is it?
<tych0> ericsnow: yes, cherylj just told me to merge from master above
<tych0> the actual change is two lines
<perrito666> does anyone here knows a lot about multiwatcher?
<ericsnow> tych0: if so, it may make sense to merge master into your feature branch in a separate PR, land it, and then rebase your first PR
<ericsnow> tych0: otherwise your change gets lost in the noise on reviewboard
<tych0> lol, ok.
<ericsnow> tych0: cool, thanks :)
<tych0> well
<tych0> unfortunately there's not a good way to keep this conflict resolution :(
<ericsnow> tych0: you could use git rebase -i to move your specific changes out of the way to after the merge
<ericsnow> tych0: maybe?
<tych0> yeah, that doesn't work so hot
<ericsnow> tych0: :(
<tych0> i'll just redo the merge conflicts
<ericsnow> tych0: ugh, I've been in that situation before too :(
<tych0> ericsnow: https://github.com/juju/juju/pull/4314
<tych0> let me know when you have that one merged
<tych0> i have the one that goes on top of it as well. i think github is usually smart enough to resolve that in its UI, no idea about reviewboard, so i'll wait to make the PR until it's done so as not to confuse things further
<dimitern> voidspace, hey, sorry - I was in a call, but I've looked at your PR and it LGTM
<cherylj> dimitern, frobware_ have you guys taken a look at bug 1542206?
<mup> Bug #1542206: space discovery still in progress <ci> <juju-core:Invalid> <juju-core maas-spaces:New> <https://launchpad.net/bugs/1542206>
<cherylj> Seems to be weird behavior when creating hosted models
<cherylj> with space discovery
<dimitern> cherylj, yeah, and voidspace has a fix for it, which I've just reviewed - it should fix it
<cherylj> dimitern: awesome, thanks!
<mup> Bug #1287718 opened: jujud on machine 0 stops listening to port 17070/tcp WSS api <cts-cloud-review> <mongodb> <state-server> <sts> <juju-core:Confirmed> <https://launchpad.net/bugs/1287718>
<marcoceppi_> I need help, questions about storage
<marcoceppi_> is filesystem supported?
<marcoceppi_> perrito666 cherylj natefinch ^?
<cherylj> marcoceppi: you mean like adding storage as a new filesystem on a host?
<cherylj> I'm not sure what you're asking (I'm so not familiar with the storage piece)
<marcoceppi> cherylj: I'm going to mail the list
<frobware> dimitern, voidspace: ping
<frobware> dimitern, voidspace, dooferlad: IRC has been busted for since around 2pm. :(  anything to be aware of?
<dimitern> frobware, pong
<dimitern> frobware, I had a chat with fwereade_ about what we discussed and I'll compile a list of use cases updated with some nice suggestions from him to share and discuss the approach on the ML
<frobware> dimitern, ack
<voidspace> frobware: pong
<voidspace> dimitern: thanks
<frobware> voidspace, can we / should we cherry-pick your change into dimiter's branch so we get another test of -jes tonight?
<voidspace> frobware: yes
<dimitern> frobware, and I decided to do something useful as today is not going smoothly as I hoped :/, so I'm preparing a merge of master into maas-spaces (sans my other branch) to get at least closer to upstream, and see what is the simplest thing to do to fix the remaining issues
<frobware> dimitern, ahhh. we should co-ordinate. I was doing the same.
<frobware> dimitern, any objections if I cherry-pick voidspace's change into your branch and push to upstream?
<dimitern> frobware, ah :) ok, well - I'm about 4/5 done, we can compare later?
<frobware> dimitern, there's stuff landing in master. depends when you and I started.
<dimitern> frobware, is it worth pursuing that branch anymore?
<frobware> dimitern, nope. can we merge that back into maas-spaces before tonights CI run?
<dimitern> frobware, started when tip was at 1cd7ac8c and I'm about 4/5 done with it
<frobware> dimitern, though the timing of the CI run is based on any new changes in that branch
<dimitern> frobware, I doubt we need most of it - at least the contoller-space handling stuff
<frobware> dimitern, I really wanted to be back on maas-spaces -- reduces confusion.
<dimitern> frobware, well, that's also true for maas-spaces itself, so why not get up-to-date with master, and then see what to cherry pick from my other branch?
<frobware> dimitern, there's the manual deployer tests which are only fixed in your branch. correct?
<dimitern> frobware, well, in master as well
<frobware> dimitern, getting confused. what would you like to do? which branch should be tested tonight (over the w/end)?
<dimitern> frobware, I'm not saying all of that other branch of mine needs to go, some things are still useful, but the approach around controller spaces will most likely change
<dimitern> frobware, sorry, let me restate: I'd prefer to get maas-spaces up-to-date with latest master and get a CI run of maas-spaces with it
<frobware> dimitern, voidspace: I'm thinking the simplest thing for today is to cherry-pick the additional -jes fix
<frobware> dimitern, voidspace: without additional churn we can validate whether we're better/worse
<voidspace> yep
<voidspace> dimitern: is there any way your work can be parallelised?
<dimitern> voidspace, it is, I just need to write down a rough set of steps
<dimitern> voidspace, considering the discussions etc.
<voidspace> ok
<voidspace> thanks
<frobware> dimitern, voidspace: so let's take the -jes change into maas-spaces-controller-space-config and we can also move forward with master -> maas-spaces
<dimitern> frobware, voidspace, but I believe we can get maas-spaces (with master merged in it) to a bless with a handful of changes only
<voidspace> dimitern: great
<voidspace> dimitern: that should be priority #1
<dimitern> voidspace, frobware, agreed - those changes will bring back the compatible behavior (no controller-space to care about, no default space in bindings to cause issues)
<dimitern> so we can then do those (rather invasive changes I must say) in a simpler step-wise manner
<dimitern> frobware, ok, so as soon as I finish with merge conflicts will c-p the jes fix into the -config branch and push
<voidspace> right, EOW
<voidspace> have a good weekend everyone
<tych0> cherylj: ericsnow: i'm not sure i understand the failure here: http://juju-ci.vapour.ws:8080/job/github-merge-juju/6249/console
<tych0> it builds fine on my machine, but sine i added that directory in the previous feature branch, perhaps i need to add it to some other listing somewhere so that whatever is building the tarball picks it up?
<natefinch> tych0: the landing bot runs go 1.2, which means we have to mark all LXD code as // +build go1.3  so it only builds with 1.3 or higher
<ericsnow> tych0: perhaps "container/lxd/instance.go" imports "github.com/juju/juju/tools/lxdclient" and the lxdclient package isn't in the repo?
<natefinch> tych0: looks like you need to add that builds tag to container/lxd/instance.go
<ericsnow> tych0, natefinch: or that :)
<tych0> oh, hm
<tych0> cool, thanks guys
<ericsnow> natefinch: nice catch
<natefinch> ericsnow: went through all this rigamarole before, so I remember getting that dumb error
<tych0> :)
<natefinch> I wish the error were just a little bit smarter and would tell you "all go code in the directory is excluded by build tags"
<natefinch> but that probably breaks layers
<ericsnow> natefinch: I have my "drop ModelResource" patch up: http://reviews.vapour.ws/r/3759/
<natefinch> ericsnow: cool
<perrito666> omg that merge was painful
<tych0> cherylj: bah. it looks like i forgot one. one more time please? sorry about that :(
 * tych0 can't computer today
<cherylj> tych0: np :)  You can also use $$fixes-1542336$$
<cherylj> but I am also going to remove the blocker tag on that bug
<cherylj> so thank you for reminding me :)
<tych0> cherylj: oh, can i merge my own stuff?
<cherylj> tych0: yeah, that one is going into your feature branch, so merge away (once you get a ship it, generally)
<cherylj> it was blocking because of that bug, which was a critical blocker
<natefinch> ericsnow: added the upload code - http://reviews.vapour.ws/r/3699/
<natefinch> gotta go snowblow before it gets dark.
<ericsnow> natefinch: k
<perrito666> aghhh I spent half a day merging master into my branch and now is not compatible, how can that possibly be?????
<mup> Bug #1542488 opened: 'juju storage pool create' has no useful help and a generic error message  <docteam> <juju-core:New> <https://launchpad.net/bugs/1542488>
<mup> Bug #1542510 opened: aws-deploy-trusty-amd64-on-xenial-amd64 hooks not firing <ci> <ec2-provider> <hooks> <regression> <wily> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1542510>
<mup> Bug #1542518 opened: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1542518>
<mup> Bug #1542520 opened: TestDestroyRemovesContainers fails on centos <centos> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1542520>
#juju-dev 2016-02-07
<mup> Bug #1542518 changed: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1542518>
<mup> Bug #1542518 opened: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1542518>
<mup> Bug #1542518 changed: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1542518>
<mup> Bug #1542518 opened: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1542518>
<mup> Bug #1542518 changed: TestListAllEmpty <ci> <go1.5> <intermittent-failure> <unit-tests> <wily> <xenial> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1542518>
#juju-dev 2017-01-30
<axw> wallyworld babbageclunk: would you please take a look at https://github.com/juju/juju/pull/6884
<babbageclunk> axw: sure
<wallyworld> ok
<wallyworld> axw: was the change to introduce cloud name as an attribute of cloud pre 2.0 release? just thinking about 2.0.0 client bootstrapping with a 2.2 agent
<axw> wallyworld: bootstrap always bootstraps the same version
<axw> wallyworld: the addition of name was post 2.0 anyway
<wallyworld> unless you use --agent-version
<wallyworld> or --auto-upgrade
<axw> wallyworld: that still doesn't bootstrap a different version, it upgrades post-bootstrap
<wallyworld> hmmm yeah, that is true
<wallyworld> axw: lgtm, ty
<menn0> axw: are you looking at this by any chance? https://bugs.launchpad.net/autopilot-log-analyser/+bug/1655783
<mup> Bug #1655783: 2.1beta4: LXDs getting stuck pending in cloud-init during Landscape Autopilot deployment <cdo-qa-blocker> <Autopilot Log Analyser:Fix Committed by fginther> <juju:Triaged> <Landscape Server:Confirmed for fginther> <https://launchpad.net/bugs/1655783>
<axw> menn0: nope
<menn0> axw: ok thanks
<babbageclunk> axw: reviewed
<axw> babbageclunk: thanks (and wallyworld)
<axw> wallyworld: I've responded to comments on https://github.com/juju/juju/pull/6878, PTAL
<wallyworld> ok
<wallyworld> axw: yeah, ignore me, i misread the code
<axw> wallyworld: cool
<wallyworld> anastasiamac: axw: menn0: babbageclunk: are you guys ok to do standup an hour earlier tomorrow? uros wants to pop in
<babbageclunk> wallyworld: sure
<axw> wallyworld: ok, may not be very coherent tho
<menn0> wallyworld: fine with me. I imagine it's going to be hardest for axw
<babbageclunk> yeah, a bit early for axw
<axw> less coherent than usual anyway :>
<wallyworld> axw: yeah, it will be 11pm for him
<wallyworld> just a one off
<wallyworld> tim was supposed to email about it last week but is seems he forgot
<wallyworld> jam: you around? could we catch up and discuss CMR when you are free? maybe in say an hour?
<anastasiamac> wallyworld: fine by me. altho it'll conflict with release call
<wallyworld> shit happens
<anastasiamac> yeah, and apparently when u eat it, don't nibble
<wallyworld> ewww
<anastasiamac> not my survivor tip
 * anastasiamac shrugs
<mup> Bug #1658100 changed: Deployment of a large bundle fails or hoggs the system <juju:Incomplete> <https://launchpad.net/bugs/1658100>
<blahdeblah> Hi folks, just wondering if you saw https://bugs.launchpad.net/juju/+bug/1660087 yet?  Looks like a slightly different twist on some of the bugs we've been seeing lately.
<mup> Bug #1660087: Controller and all models unresponsive <canonical-is> <juju:New> <https://launchpad.net/bugs/1660087>
<mup> Bug #1658100 opened: Deployment of a large bundle fails or hoggs the system <juju:Incomplete> <https://launchpad.net/bugs/1658100>
<anastasiamac> blahdeblah: m getting to triage now :)
<blahdeblah> anastasiamac: ta - axw is gonna love me for this one. :-)
<anastasiamac> blahdeblah: how long did it take to get unresponsive?
<anastasiamac> blahdeblah: is it 2.0.x?
<blahdeblah> anastasiamac: I don't know, sorry; logs will hopefully show that
<blahdeblah> Yes, it's 2.0.2
<anastasiamac> blahdeblah: there was a memory leak that I believe have been fixed in later 2.0.x
<anastasiamac> blahdeblah: i wonder if it's related...
<blahdeblah> I'd have to double check graphs to find out if memory was leaking, but I don't think so
<blahdeblah> Unless it was a leak in mongodb, because it was the top user of memory
<anastasiamac> blahdeblah: if memory serves right, it was mongodb-connections-related..
<blahdeblah> Well, hopefully the profiling data gathered will give you something to go on; if not, please let us know and we can gather more.  This is happening several times per week across our public cloud services - I've seen it on AWS, Azure, and GCE.
 * anastasiamac afraid to ask...
<anastasiamac> blahdeblah: and what do u do when it happens?
<anastasiamac> blahdeblah: reboot?
<blahdeblah> Restart all the things! \o/
<blahdeblah> anastasiamac: We stop jujud-machine-0, restart juju-db & rsyslog, and start jujud-machine-0 again
<anastasiamac> blahdeblah: i'll update the bug with this info. I ma sure that axw will be thrilled indeed :D
<mup> Bug #1658100 changed: Deployment of a large bundle fails or hoggs the system <juju:Incomplete> <https://launchpad.net/bugs/1658100>
<mup> Bug #1658549 changed: Security issue: jujud is not owned by a user on the system <juju:Incomplete> <https://launchpad.net/bugs/1658549>
<mup> Bug #1659102 changed: juju status shows ip address from public-api space rather than internal-api space <juju:Incomplete> <https://launchpad.net/bugs/1659102>
<anastasiamac> blahdeblah: are u seeing similar behavours with 2.1.x?
<blahdeblah> anastasiamac: I'm sorry to disappoint, but we don't deploy non-stable versions into our production cloud mirrors. :-)
<anastasiamac> blahdeblah: i admire our definition of stable \o/
<mup> Bug #1658549 opened: Security issue: jujud is not owned by a user on the system <juju:Incomplete> <https://launchpad.net/bugs/1658549>
<mup> Bug #1659102 opened: juju status shows ip address from public-api space rather than internal-api space <juju:Incomplete> <https://launchpad.net/bugs/1659102>
<jam> wallyworld: I just got back from giving my wife a ride, I need a few minutes but then I'm happy to meeet
<wallyworld> ok, sgtm
<mup> Bug #1658549 changed: Security issue: jujud is not owned by a user on the system <juju:Incomplete> <https://launchpad.net/bugs/1658549>
<mup> Bug #1659102 changed: juju status shows ip address from public-api space rather than internal-api space <juju:Incomplete> <https://launchpad.net/bugs/1659102>
<axw> jam wallyworld: what's the status of merging 2.1 into develop? I'm working on 2.1 now, and want to forward-port to develop... merge brings in a whole lot of unrelated conflicts
<wallyworld> i tried this morning but it failed
<wallyworld> i didn't look into it after that
<jam> axw: I have the branch that should land, it had a problem in Bundle tests that seem spurious
<jam> testing here it passes, so I'll resubmit
<axw> jam: okey dokey, thanks
<axw> wallyworld: nps, i'll wait for jam's branch to land
<jam> axw: I *could* just hit that "merge now" button... but that would be an abuse of power. :)
<axw> jam: heh :) going through CI seems safest to me
<jam> http://juju-ci.vapour.ws:8080/job/github-merge-juju/10137/ says that 'grant' failed as well, but I don't see anything other than maybe an EOF in that file
<jam> and that doesn't give me any real clue as to what is wrong.
<axw> jam: it merged, thank you
<wallyworld> jam: i'm going to grab dinner, maybe I can ping you a bit later if you're free then?
<jam> sure
<menn0> jam: review done
<jam> thanks menn0, will go through it now
<menn0> jam: I found a lot of stuff but it's mostly minor
<wallyworld> jam: free when you are; finish whatever you're doing and give me a ping
<jam> k
<axw> jam: if at some stage you have a chance, I'd appreciate your review of https://github.com/juju/juju/pull/6887. I recommend reviewing the commits separately. they're in one PR because without the upgrade step the branch is unreleasable
<jam> sure axw
<jam> wallyworld: ping
<wallyworld> hey
<wallyworld> ho?
<wallyworld> https://hangouts.google.com/hangouts/_/canonical.com/a-team-standup
<jam> wallyworld: that sounds like a scary place to go
<jam> I'm not sure I'm welcome there
<wallyworld> huh? i'm the only on ehere, not that scary
<wallyworld> we can go elsewhere
<wallyworld> i just had a link jandy
<wallyworld> *handy
<jam> anyone else seeing failures in "grant" testing where it seems to be expecting a password prompt and isn't seeing it?
<jam> the code even mentions bug #1621532
<mup> Bug #1621532: grant-revoke could not enter password to login <ci> <login> <regression> <juju:Invalid> <juju-ci-tools:Fix Released by sinzui> <https://launchpad.net/bugs/1621532>
<jam> about macaroons
<perrito666> jam: nope, inquiere rogpeppe about that he might know what is going on
<perrito666> brb taxes pay day
<rogpeppe> jam: it might be because the command line behaviour has changed so the command-line mode is no longer the default (because it often fails)
<jam> rogpeppe: from what I can tell the issue is that the test is doing something like "juju login" and is expected to be prompted for a password, but it is getting no prompt
<jam> the related bug says that "when there are macaroons at play, you don't always get prompted"
<jam> axw: ping about 2.1 vs 2.2 on 'handleGetRegionError' stuff.
<rick_h> jam: congrats on getting the branch merged to 2.1
<jam> rick_h: thanks. still needs one more thing before it is releasable, but Nick asked that it land rather than not. as it feels better to be testing the real thing.
<rick_h> jam: cool, still big chunk of stuff there so <3
<jam> yeah, and I got the SSH-keyscan stuff landed, merging it to 2.2 now
<jam> got 2.1 synced into 2.2, so we can keep getting stuff in both
<jam> doing decent
<jam> still trying to sort out the real fix for bug #1657449
<mup> Bug #1657449: 2.1 needs to fallback to lxdbr0 for clouds that can't be bridged <aws> <lxd> <lxdbr0> <juju:In Progress by jameinel> <https://launchpad.net/bugs/1657449>
<jam> and things like vSphere and Manual both *wanting* to bridge-by-default
<rogpeppe> perrito666: i've just commented on https://github.com/juju/juju/pull/6865. I wonder what you think.
<perrito666> rogpeppe: looking
<perrito666> rogpeppe: mm, interesting comment
<perrito666> rogpeppe: what you say is true, I like your idea, thank you
<rogpeppe> perrito666: cool, thanks
<redir> morning
<perrito666> redir: sorry that was rude from my part, morning
<redir> nah
<balloons> axw, you introduced a new launchpad dependency? :-(
<balloons> d'oh, never mind, wrong branch :-)
<babbageclunk> so standup's still an hour early today, right?
<anastasiamac> babbageclunk: from calendar, it looks like it has been reverted to original time
<babbageclunk> anastasiamac: thanks, should've checked there first duh
<anastasiamac> babbageclunk: nps ;D
<babbageclunk> perrito666: since you're OCR today, mind taking a look at https://github.com/juju/juju/pull/6868?
<perrito666> babbageclunk: our definitions of today differ a bit but glad to
 * perrito666 is running tests so he can't do anything of consequence in the laptop
<babbageclunk> oh, duh - forgot
<babbageclunk> perrito666: thanks!
<perrito666> babbageclunk: lots of dereferencing, lovely provider API I see :)
<babbageclunk> perrito666: Yeah, the rest API client autogenerator that the azure SDK uses makes everything a pointer. Blech.
<axw> jam: pong about handleGetRegionError (sorry, was EOD)
<perrito666> babbageclunk: ship it, brb
<babbageclunk> perrito666: awse, thanks
#juju-dev 2017-01-31
<babbageclunk> axw: can you take a gander at https://github.com/juju/juju/pull/6868?
<axw> babbageclunk: looking
<axw> babbageclunk: all the other works are quiesced while this code runs, right?
<axw> workers*
<axw> babbageclunk: the provisioner won't be trying to update things, for example?
<anastasiamac> menn0: axw: for bug 1660087, we just need to know if it has been addressed in later revisions, on 2.1.x
<mup> Bug #1660087: Controller and all models unresponsive <canonical-is> <juju:Triaged by menno.smits> <https://launchpad.net/bugs/1660087>
<axw> anastasiamac: can't say until we know what the root cause is
<anastasiamac> axw: hopefully, logs attached to the bug will help :D
<axw> anastasiamac: yeah, hopefully. I'll look when I can
<anastasiamac> axw: nps \o/
<babbageclunk> axw: yeah, that's right - they're all gated by the migration flag
<axw> babbageclunk: ok, thanks. I've left some comments
<babbageclunk> axw: cool, thanks!
<menn0> anastasiamac: sorry I was out for a bit
<menn0> anastasiamac: what axw said :)
<anastasiamac> menn0: thnx :)
 * menn0 is going dark for a while to focus on CAAS 
 * redir goes eod
 * redir will look at #1654603 in the morning if still waiting for PPC64EL MAAS access.
<mup> Bug #1654603: kill-controller takes too long, is it waiting for hooks to finish? <landscape> <juju:Triaged> <https://launchpad.net/bugs/1654603>
<babbageclunk> Does anyone know how to get maas to see a change in max ram after changing it in vmm?
<babbageclunk> Do I need to remove and recommission it, or is there a more direct way?
<anastasiamac> babbageclunk: i wonder if maybe we should reach out to MAAS guys? dunno if any of them are in this channel..
<babbageclunk> anastasiamac: yeah, I'll go ask over there.
<mup> Bug #1660522 opened: Un-removeable "life: dead", "agent-state: pending"  agents  <canonical-bootstack> <juju-core:New> <https://launchpad.net/bugs/1660522>
<axw> jam: bridge testing again, didn't notice IRC failed to reconnect. wallyworld raised the same issue of remove-storage vs. detach-storage
<axw> jam: my question is what that means in the case of shared storage
<axw> jam: we can just have it mean "detach from the application", which might be reasonable?
<jam> axw: sorry, I've got a bunch of different things in my head right now. I just wanted to make sure I understood what made you uneasy, so that I understand the full problem. I probably won't be able to switch to that for a bit
<axw> jam: it's all good, I'm looking at the lxd/maas networking issue now
<axw> wallyworld: ping?
<wallyworld> hey
<axw> wallyworld: yo. about the firewaller thing. can you HO? might be easier
<wallyworld> sure, give me 5
<axw> wallyworld: np, ping when
<wallyworld> 1:1?
<axw> wallyworld: yes
<mup> Bug #1660542 opened: container mac addresses should use 'locally assigned' section <lxd> <mac-address> <network> <juju-core:Triaged> <https://launchpad.net/bugs/1660542>
<mup> Bug #1660543 opened: container mac addresses should use 'locally assigned' section <lxd> <mac-address> <network> <juju-core:Triaged> <https://launchpad.net/bugs/1660543>
<wallyworld> axw: changes pushed
<axw> wallyworld: looking
<axw> wallyworld: LGTM
<wallyworld> ty
<babbageclunk> axw: take another look? https://github.com/juju/juju/pull/6868
<axw> babbageclunk: looking
<babbageclunk> axw: thanks! you weren't kidding when you said callAPI was ugly.
<axw> babbageclunk: yeah :/
<axw> babbageclunk: probably should just bite the bullet and write our own wrapper API clients
<axw> (not in this PR though)
<babbageclunk> axw: no :)
<axw> babbageclunk: done
<babbageclunk> axw: cheers
<mup> Bug #1660543 changed: container mac addresses should use 'locally assigned' section <lxd> <mac-address> <network> <juju-core:Triaged> <https://launchpad.net/bugs/1660543>
<perrito666> morning all
<rick_h> redir: ping when/if you're about
<perrito666> rick_h: I doubt he will be here for a couple of hours
<rick_h> perrito666: yea, just realized. I ping'd him out of band
 * rick_h somehow thought the day was farther along
<jam> a review of https://github.com/juju/juju/pull/6893 is requested
<jam> it involves refactoring the code that checks what bridges to create for containers, but doesn't, itself, change any of the logic
<jam> just pulls it off of Machine where it didn't really belong.
<jam> frobware: ^^
<frobware> jam: reviewed
<jam> frobware: responded and pushed up an update
<jam> care to take another look
<frobware> jam: looking
<frobware> jam: LGTM
<frobware> jam: I was just taking another look on the files that were added.
<frobware> jam: those tests - they were just the tests that were moved?
<frobware> jam: if the error doesn't bubble up to the user then I think what you suggested is fine.
<jam> frobware: so how the code is *currently* used, the only caller of it ensures that its only dealing with bridges first
<jam> frobware: I added the check because I split the code from where it was created from where the check was done
<jam> frobware: and I believe in validating inputs in case someone uses it differently in the future
<jam> frobware: hypothetically, that sort of error could bubble to the user if we have a bug in our software, so I did tweak the wording
<frobware> jam: ack
<redir> rick_h: pong
<redir> I see a task in the Juju Show
<rick_h> redir: howdy, yep
<redir> todo made
<rick_h> redir: wanted to see if you had some details/etc you wanted/could flesh out
<rick_h> redir: ty
<redir> rick_h: not much, uvtool is gone and it works on ARM64 where kvm does on ARM64 now.
<rick_h> redir: and works everywhere else expected and this is 2.2 ?
<redir> otherwise there's no change to the API or feature set, but I can make an outline
<redir> rick_h: still trying to get access to ppc64el HW to test and debug there.
<redir> brb need to let the tree trimmers in the garden
<perrito666> brb, afternoon snack :)
<redir> perrito666: you made me hungry
<perrito666> redir: sad to be you mate :p
<perrito666> I actually had to drive 15 min to get the snack
<redir> develop is broken
<redir> same breakage in 2.1
<menn0> redir: what's the breakage? (I haven't looked)
<redir> # github.com/juju/juju/worker/machineactions
<redir> worker/machineactions/handleactions.go:70: unknown "github.com/juju/utils/exec".RunParams field 'User' in struct literal
<redir> worker/machineactions/handleactions.go:87: not enough arguments to return
<redir> menn0: ^ was going to look at the history after lunch
<menn0> how the hell did that get in?
<redir> the commit where that went in was the 26th
<redir> perhaps before verify.bash was enabled on CI
<redir> s/enabled/fully enabled
<redir> and someone prolly didn't have the pre-commit hook enabled
<menn0> redir: seems likely
<redir> OK nm, I think It is me
<redir> shockingly
<redir> I fatfingered godeps update and missed the error
<anastasiamac_> menn0: as an OCR, this PR addresses frequent intermittent bug: https://github.com/juju/juju/pull/6892
<perrito666> jam: YUNO squash?
<redir> perrito666: prolly because there were conflicts and or merges
<redir> juju deploy charm1 charm2 # is this valid syntax?
<redir> it appears not but it deploys charm2
<redir> but claims it deployed charm1
<redir> be a couple minutes late to s/u
<menn0> redir: that sounds like sensible behaviour :-/
#juju-dev 2017-02-01
<wallyworld> perrito666: https://bugs.launchpad.net/juju/+bug/1656205
<mup> Bug #1656205: Check for lxd ipv6 is failing for various cases <juju:Triaged by jameinel> <https://launchpad.net/bugs/1656205>
<perrito666> wallyworld: ack
<wallyworld> menn0: free when you are
<menn0> wallyworld: i'm back. pick a HO
<wallyworld> menn0: standup works
<axw> anastasiamac_: free to talk bugs now
<anastasiamac_> axw: m starving :( can i ping u in 30mins or so?
<axw> anastasiamac_: np, helping out IS now anyway
<anastasiamac_> \o/
 * redir goes eod
<redir> to answer my question above it deploys charm1 with the alias charm2. That is only confusing when you think it is deploying 2 charms. The second pos arg is an alias...
<redir> after a brief chat with wallyworld I created #1660872 so that the output might be more informative, less misleading.
<mup> Bug #1660872: juju deploy charm alias should provide more detailed output  <juju:New> <https://launchpad.net/bugs/1660872>
<mup> Bug #1592887 changed: juju destroy-service deletes openstack volumes <canonical-is> <landscape> <uosci> <juju:In Progress by axwalk> <juju-core:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1592887>
<jam> perrito666: as a developer of an alternative VCS that didn't encourage squashing as standard practice, I'm not a big fan. I have squashed here and there, but it depends what squashes you were asking for
<jam> perrito666: if you squash merging 2.1=>develop you've negated the whole benefit of the merge, as the *point* is to bring across the individual commits so you know they're in both
<anastasiamac_> wallyworld: axw: do u recall why ModelInfo only returns info if user has a write access?
<wallyworld> hmm, not sure, didn't know it did
<wallyworld> seems like read access should be sufficient unless i'm missing something
<anastasiamac_> wallyworld: looking at bug 1660745
<mup> Bug #1660745: write privileges should not be required to see model machine info <juju:New> <https://launchpad.net/bugs/1660745>
<anastasiamac_> wallyworld: i agree. was just wondering in case i've missed something
<wallyworld> well we both have if that's the case :-)
<anastasiamac_> that never happens \o/
<wallyworld> axw: only if you have time, a look at this PR would be great https://github.com/juju/juju/pull/6867
<wallyworld> i have other work to go on with as well
<axw> wallyworld: ok, in a little while
<wallyworld> sure, no rush at all
<wallyworld> do your other stuff first
<mup> Bug #1660542 changed: container mac addresses should use 'locally assigned' section <lxd> <mac-address> <network> <juju:Triaged> <lxd (Ubuntu):New> <https://launchpad.net/bugs/1660542>
<mup> Bug #1660907 opened: jujud and mongod consume 800% CPU, lots of RAM after restart in controller cluster <juju-core:New> <https://launchpad.net/bugs/1660907>
<mup> Bug #1660907 changed: jujud and mongod consume 800% CPU, lots of RAM after restart in controller cluster <juju-core:New> <https://launchpad.net/bugs/1660907>
<mup> Bug #1660907 opened: jujud and mongod consume 800% CPU, lots of RAM after restart in controller cluster <juju-core:New> <https://launchpad.net/bugs/1660907>
<perrito666> Morning
<mup> Bug #1660907 changed: jujud and mongod consume 800% CPU, lots of RAM after restart in controller cluster <juju:Incomplete> <https://launchpad.net/bugs/1660907>
<jam> small patch up for review: https://github.com/juju/juju/pull/6896
<perrito666> jam: Ill review it (not like you have other choices of reviewer at this time :p )
<jam> thanks perrito666
<perrito666> jam: lgtm with a small comment
<perrito666> I did not do the actual deploying im trusting in you for this one
<frobware> jam: ping
<frobware> jam: I have an hour before I EOD, anything I can help with?
<jam> frobware: actually there is
<jam> frobware: https://github.com/juju/juju/pull/6898
<jam> should be enough to have containers work on AWS
<jam> I still want to do more work there, to remove the code in "worker/provisioner" that treats any error as falling back to lxdbr0
<jam> but between that pull and one that just landed
<jam> I think I have lxd containers working again on AWS
<jam> frobware: perrito666: reviews of it are welcome, as that should unblock my parts for 2.1b5, and I can focus more on polish/other bug fixing
<jam> I'm going to try and trigger a CI run by landing it into 2.1-dynamic-bridges
<perrito666> jam: ill try to look at it
<frobware> jam: ping
<jam> frobware: pong
<frobware> jam: heh, nm... Might not have been running the right binary. testing again...
<jam> frobware: so *just* that branch needs to be merged with 2.1
<jam> I had another patch that was in merge review and pending landing
<jam> and I didn't want to merge it into the other branch
<jam> (always-observe is the extra bit)
<jam> frobware: the code also all landed in upstream/2.1-dynamic-bridging if you want to try from there
<frobware> jam: I was trying your branch: 2.1-containerizer-config-1657449
<frobware> jam: my wrong binary was... I hadn't source my `require juju-dev` bash scripts which ensure $HOME/go/bin is on my PATH first.
<jam> frobware: ah sure, that particular branch needs to merge 2.1, I'll do it now
<jam> done
<frobware> jam: that branch seemed to work OK
<frobware> jam: for AWS
 * frobware needs to find some power; back in a minute.
<frobware> jam: love this on azure - dns-search vlmw5tgdmhlejaynio3hmmlt4b.gx.internal.cloudapp.net
<frobware> jam: presumably the first part is hungarian notation :p
<frobware> jam: reviewed, QA'd and approved. I did not try MAAS, but I did try Azure, GCE and AWS.
<frobware> jam: I'm unlikely to be around much tomorrow.
<jam> frobware: thanks for the help and the heads up
<jam> have a good day
<redir> forgot to say... morning juju-dev
<rick_h> afternoon redir
<redir> yay relativity
<perrito666> EOD all see you
<menn0> perrito666: ciao
<menn0> veebers, anastasiamac: I'm fairly sure that the migration credentials failure is due to a change axw merged yesterday (fff2d6ff710ead222741eb61b7fbee3510eab769)
<menn0> confirming now
<anastasiamac> thnx, menn0
<babbageclunk> ugh, I'm having real trouble testing my change for the ec2 provider.
<redir> babbageclunk: anyway I can help?
<redir> what's the difficulty?
<babbageclunk> redir: no I think I just need to power through it - just hard to extract the information I want to check to see if it's done the right thing. I think I've broken the back of it now.
<babbageclunk> redir: thanks though!
<redir> phew
<redir> http://www.businessinsider.com/gitlab-outage-due-to-human-error-2017-2
<axw> menn0: oy, sorry. I'm here now
<menn0> axw: so fff2d6ff710ead222741eb61b7fbee3510eab769 has broken migrations on LXD
<axw> menn0: got a CI failure handy that I can look at?
<menn0> axw: I'll get one for you but I can also tell you what's failing
<menn0> axw: in state/migration_import.go, the Import method does a credentials check
<menn0> axw: and fails the migration if they don't match
<menn0> axw: it's the comparison of existingCreds.Attributes() to creds.Attributes() that's failing
<menn0> axw: the client-cert and client-key don't match
<menn0> axw: i'm not sure if the migration check is invalid and we've been getting away with it or if something not right with the LXD creds change
<axw> menn0: I'll need to see the steps taken to know why they might be different
<axw> menn0: each bootstrap creates new credentials, so that might explain it?
<menn0> axw: I can repro locally
<menn0> axw: that's probably it
<menn0> axw: i'll have to figure out why tim added that check
<axw> menn0: ok. I have an idea about caching the creds on disk for lxd, I can do that if it helps
<axw> menn0: i.e. so we get the same creds each time, so long as the on-disk creds remain the same
<menn0> axw: were you thinking of doing that anyway?
<axw> menn0: yeah, was planning to do it later but I can look at doing it now
<menn0> axw: another approach would be to not include cloud creds in the model description for LXD models
<menn0> axw: it looks like the model migration logic allows for that on the import side
<axw> menn0: would that make it more difficult to support remote-LXD?
<axw> because we're pretty close to being able to support that now I think
<menn0> axw: yeah I guess that could be an issue. quick chat after standup?
<axw> menn0: I will have to drop my daughter off at school, I'll ping you as soon as I get back
<menn0> axw: sounds good
#juju-dev 2017-02-02
<blahdeblah> menn0: I see you are the lucky winner of https://bugs.launchpad.net/juju/+bug/1660087; it may still be the same underlying cause as https://bugs.launchpad.net/bugs/1635311, so that might be useful background, depending on what you find in the profiling data.
<mup> Bug #1660087: Controller and all models unresponsive <canonical-is> <juju:Triaged by menno.smits> <juju 2.2:Triaged> <https://launchpad.net/bugs/1660087>
<mup> Bug #1635311: juju2 eating CPU, units in error <canonical-is> <landscape> <juju:In Progress by axwalk> <https://launchpad.net/bugs/1635311>
<menn0> blahdeblah: ok, thanks for joining the dots
<blahdeblah> menn0: No guarantees ;-)
<alexisb> wallyworld, ping
<wallyworld> hey
<alexisb> heya wallyworld did you want to meet in our 1x1 HO?
<wallyworld> alexisb: give me 5?
<wallyworld> or 3
 * alexisb taps her fingers impatiently on her desk
<alexisb> and stares at wallyworld
<wallyworld> alexisb: just looking for hangout, it's gone :-)
<alexisb> https://hangouts.google.com/hangouts/_/canonical.com/alexis-ian
<alexisb> you just have to look into the past
<mup> Bug #1660522 changed: Un-removeable "life: dead", "agent-state: pending"  agents  <canonical-bootstack> <juju-core:Fix Released> <https://launchpad.net/bugs/1660522>
<axw> menn0: back
<babbageclunk> axw (or anyone else) - is there any way to have the ec2test server return errors for operations?
<babbageclunk> axw: Or do I need to introduce indirection in the provider and patch things out with mocked ones? (that seems to be what the other tests do)
 * redir looks into the future and sees a beer
 * redir eods
<redir> maybe
<redir> but should find a bug to look at tomorrow
<axw> babbageclunk: I don't think so, but don't know for sure. what error do you want to return?
<babbageclunk> axw: I just want to test a few error paths in my code.
<babbageclunk> axw: Actually I'm not sure there's much need - the error handling is fairly trivial in this case.
<axw> for full coverage you probably need a layer of indirection. for more specific validation errors (for example), it would make sense to update ec2test
<babbageclunk> axw: I'm not looking for any specific validation errors or anything, so I think this is probably fine.
<menn0> axw: was having lunch. i'm back now too.
<menn0> axw: quick hangout?
<anastasiamac> wallyworld: for bug 1602192, you just want me to say that it'll b better communicated thru status msg and will b addressed in 2.2?
<mup> Bug #1602192: when starting many LXD containers, they start failing to boot with "Too many open files" <lxd> <juju:Triaged> <lxd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1602192>
<wallyworld> let me look, one sec
<axw> menn0: sorry missed your msg, yep
<axw> menn0: see you in standup HO?
<menn0> axw: yep, see you there
<wallyworld> anastasiamac: yeah. i think we also discussed that in clouds, there was no guarantee that the images could support those particular kernal parameters so best not to try and guess and better surface the error
<anastasiamac> wallyworld: ack
<wallyworld> that was the advice from nic and adam anyway
<wallyworld> juju often gets stuff wrong when it tries to guess, so we are trying to be more explicit
<redir> I often get stuff wrong when I try to guess
<babbageclunk> axw: can you take a look at this if you get a chance? https://github.com/juju/juju/pull/6900
<axw> babbageclunk: okey dokey
<babbageclunk> axw: thanks!
<axw> babbageclunk: done
<babbageclunk> axw yay! Yeah, much easier than the Azure one (although a lot of that was me getting to understand the SDK),
<veebers> babbageclunk: query is the model migration fix in for aws re: the bug where destroying the origin controller borks things?
<babbageclunk> veebers: no - the AWS specific bit is done, but I still need to do the part that calls that from the migration.
<veebers> babbageclunk: ack thanks, I'll leave that out from my test script for today then :-)
<babbageclunk> veebers: sorry - I'll let you know when it's all done.
<veebers> babbageclunk: nw, was just checking before I try it out in my manual testing. Also interested in GCE
<babbageclunk> veebers - gce's in the same state.
<babbageclunk> Hmm, there's no reason I can't do the migration bit now, if it would help?
<veebers> ack. it's not blocking me really at the moment, I'm just doing some manual testing for migrations in the public clouds
<menn0> axw: how far away are you with that lxd credentials fix?
<axw> menn0: not too far, just updating tests
<menn0> axw: cool. I ask b/c I have a fix for the charm URL migration issue but it would be easier to QA if your fix was in place.
<menn0> axw: I can work around it regardless
<axw> menn0: sorry was otp
<axw> menn0: I can push my branch if you want to use it incomplete
<menn0> axw: nah it's all good
<menn0> axw, babbageclunk or wallyworld: https://github.com/juju/juju/pull/6901
<wallyworld> looking
<wallyworld> menn0: i think i missing something - the sequence assigns rev numbers 1,2,3,4,5 etc for each base charm url right?
<menn0> wallyworld: yes
<wallyworld> so if the source model has had charm rev 4 deleted, and migrates rev-1 rev-2 rev-3 rev-5 rev-6, won't that stuff up?
<menn0> wallyworld: no b/c the revision is set to max(charm URL rev, next seq value)
<menn0> wallyworld: for each charm upload
<wallyworld> ah, right ok
<menn0> wallyworld: the rev in the uploaded charm is used if possible, unless the sequence is beyond that value already
<wallyworld> ok, thanks. lgtm
<menn0> wallyworld: thanks for looking
<wallyworld> np. was a pretty easy fix in the end
<menn0> wallyworld: a bit fiddly but not too hard
<menn0> thanks to babbageclunk too
<babbageclunk> wallyworld: it looks like there's no method in goose corresponding to the update volume metadata API call: https://developer.openstack.org/api-ref/block-storage/v3/#update-a-volume-s-metadata
<wallyworld> wouldn't surprise me
<wallyworld> i don't know for sure off hand though
<babbageclunk> wallyworld: Ok, cool - just checking I wasn't missing something. I'll add it then.
<wallyworld> babbageclunk: if you do change goose, there's a drive by that needs doing
<wallyworld> i'll find it
<babbageclunk> wallyworld: It's not "upgrade it to the new version of the openstack API" is it?
<wallyworld> it's a message that is logged as INFO that should be debug
<wallyworld> that could be the message, can't recall exactly
<babbageclunk> wallyworld: Oh yeah, that I can handle.
<babbageclunk> wallyworld: No I meant a big thing being jokingly described as a driveby. But changing a log level I'll do.
<wallyworld> just as well :-)
<wallyworld> actually, it's worse than info, it's warning
<wallyworld> line 220 of client/apiversion.go
 * menn0 is done for now. back again later.
<babbageclunk> wallyworld: ok, I'll change that too.
<wallyworld> ta
<babbageclunk> wallyworld: Do you mean 229? https://github.com/go-goose/goose/blob/v1/client/apiversion.go#L229
<wallyworld> babbageclunk: yis (that's kiwi for yes)
<babbageclunk> thenks
<Lalit>  Hello Experts 	I working on to develop charms . I have two questions  	1) Is there any way to fire command from one juju unit to another ? 	2) Is there a way to copy file from one unit to another ?
<axw> Lalit: juju itself does not provide any way to do those things. charms can coordinate with each other to do it using relations, though
<axw> wallyworld: can you please review https://github.com/juju/juju/pull/6903
<wallyworld> sure
<Lalit> Hello Andrew thank for replay . if possible could you please let me know any charm doing this So that i can refer that charm
<axw> Lalit: I'm sorry, I don't know of any. I suggest trying in #juju or on the mailing list
<axw> wallyworld: you're working on https://bugs.launchpad.net/juju/+bug/1643430?
<mup> Bug #1643430: Unassigned units in an error state cannot be removed <juju:Triaged> <juju 2.1:In Progress by wallyworld> <https://launchpad.net/bugs/1643430>
<wallyworld> axw: yeah, in between things, just fixing a state changing too quickly error. i also need to think about the fact that we can't assert A && B across collections
<jam> for those around that can review: https://github.com/juju/juju/pull/6905
<jam> frobware: perrito666: ^^
<perrito666> Jam it's 7am here once I finish waking up I'll take a look :)
<prankgm> Hi Folks,
<prankgm> Have a question. Is there a way to call config-changed hook in charm after relation hooks?
<pranav_> Hey folks. Have a query
<pranav_> need to fire config-changed hook after all relations are made. Any way to do this?
<pranav_> Hey. Can anyone here help me with a query on hooks?
<perrito666> pranav_: sorry almost no one is here you had a query regarding config hook right?
<perrito666> pranav_: most of these will most likely get a more useful answer in #juju
<pranav_> Oh ok thanks. Will try there
<redir> morning juju dev
<redir> new kernel brb
<redir> or bbiab
<babbageclunk> perrito666, redir: anyone familiar with the openstack API?
<perrito666> Babbageclunk slightly
<babbageclunk> perrito666: I'm adding a SetMetadata method to cinder volumes in goose.
<babbageclunk> perrito666: My reading of the docs says I want PUT - https://developer.openstack.org/api-ref/block-storage/v3/?expanded=create-metadata-for-volume-detail,update-a-volume-s-metadata-detail#update-a-volume-s-metadata
<babbageclunk> perrito666: Only confused because the corresponding method for servers is POST: https://developer.openstack.org/api-ref/compute/?expanded=create-or-update-metadata-item-detail,update-metadata-items-detail,create-or-replace-metadata-items-detail#update-metadata-items
<babbageclunk> perrito666: There's also a POST version of the volumes request, but I think PUT is what I want since it explicitly allows partial updates to the metadata (so I don't need to get all of the tags to update one).
<perrito666> Definitely not that into it :)
<babbageclunk> perrito666: :)
<babbageclunk> perrito666: I'll do PUT for now anyway.
<redir> babbageclunk: vry minimally
<redir> babbageclunk: PUT sounds correct to me
<babbageclunk> redir: sweet, thanks
<babbageclunk> perrito666: do you know how I can run the cinder live tests for goose?
<babbageclunk> perrito666: Can I use creds from cloud-city?
<babbageclunk> perrito666: hang on - following along in the nova tests it looks like maybe I don't need to.
 * redir lunches
<perrito666> babbageclunk: yes you can
<babbageclunk> perrito666: I couldn't work out where to get all of the cred bits: I can find URL, user, secret (I assume this can be password), region and tenant. What should domain be?
<perrito666> babbageclunk: let me peek at cloud city
<perrito666> babbageclunk: you need the credentials for something other than bootstraping?
<perrito666> otherwise just JUJU_DATA=/path/to/cloud-city
<babbageclunk> perrito666: no, trying to run the goose live tests - that seems to need 6 env vars set.
<babbageclunk> perrito666: Ah, looking at the credentials struct, domain is tagged as optional.
<babbageclunk> perrito666: Ok, I'll try this - thanks!
<perrito666> babbageclunk: source ec2rc
<perrito666> :p
<perrito666> ah sorry openstack
<perrito666> babbageclunk: I used to have hp cloud for tests in openstack
<perrito666> now I have no idea what we use, last time someone gave me one
<babbageclunk> perrito666: ok, if this doesn't work I'll ping some of the qa folk, thanks
<babbageclunk> wallyworld: ping?
<wallyworld> otp
<wallyworld> babbageclunk: i can ping you after release standup
<babbageclunk> wallyworld: ok, just trying to work out how to run goose live tests - if you don't know then no worries.
<wallyworld> babbageclunk: unit tests?
<wallyworld> ah live tests
<babbageclunk> wallyworld: yup
<wallyworld> i can help a bit, soon
<babbageclunk> wallyworld: ok, awse - thanks
<wallyworld> babbageclunk: i have to duckout real quick, but basically if you have your .novarc and source it, the env vars should be set up so you can go test --live
<wallyworld> from memory
<wallyworld> babbageclunk: but realistically, nowadays, a bootstrap / live functional test is probably best
<babbageclunk> wallyworld: what's a .novarc? (I know nothing about openstack)
<wallyworld> ah, oh. novarc contains your credentials
<wallyworld> and tenant etc
<babbageclunk> wallyworld: ok, I'll do some testing once I've done the migration master part
<wallyworld> you have not been givwn those for canonistack?
<wallyworld> ok
<babbageclunk> wallyworld: I've got cloud-city, so I can use those I think
<wallyworld> yes you can
<wallyworld> there should be a novarc file there
<wallyworld> save it as ~/.novarc
<wallyworld> then you juju autoload-credentials
<wallyworld> babbageclunk: i have to pop out for a bit, back soon
<alexisb> anastasiamac, ping
<anastasiamac> hi alexisb \o/
<alexisb> heya anastasiamac, sorry I missed standups today
<anastasiamac> u haven't yet. it's now
<alexisb> was curious where we are with the beta5 release
<alexisb> ah ok
<alexisb> I will join
<anastasiamac> it's ready to go. last PR just landed
#juju-dev 2017-02-03
<wallyworld> menn0: do you have a link to the release notes?
<anastasiamac> wallyworld: link is in canonical #juju channel
<wallyworld> ta
 * babbageclunk lunches
 * babbageclunk runs actually
 * redir eod
<axw> wallyworld: mind if we hold off on 1:1 till quarter past? I'd like to fix this LXD bug ASAP
<wallyworld> yep no problem, just ping when ready, no rush
<axw> wallyworld: would you please review https://github.com/juju/juju/pull/6912
<wallyworld> sure
<wallyworld> axw: in TestFinalizeCredentialLocalAddCertAlreadyThere, i can't see where "out" is used
<wallyworld> could we check its contents like in the similar test
<axw> wallyworld: yeah, I deleted it after running the test.. heh
<axw> wallyworld: ok
<wallyworld> i'd prefer we keep in and just do a check
<axw> wallyworld: yep, will do
<wallyworld> lgtm aprt from that
<axw> wallyworld: pushed
<axw> wallyworld: thanks
<wallyworld> axw: a message to balloons to hold the release would be useful. he may even see this ping
<balloons> I see jam is landing things too
<balloons> so yea :-)
<wallyworld> balloons: well, i merged for him :-)
<balloons> ahh
<balloons> go for it
<balloons> be busy, and make beta5 as shiny as it can be'
<wallyworld> great, agreed :-)
<wallyworld> thanks
<axw> wallyworld: 1:1 now?
<wallyworld> sure
<babbageclunk> wallyworld, axw, anastasiamac: I forgot to mention in the meeting, it's Waitangi Day here on Monday, so the kiwis won't be around.
<axw> babbageclunk: ok, thanks
<wallyworld> babbageclunk: bloody kiwi bludgers
<anastasiamac> wallyworld: axw: shall we skip standup on monday then?
<wallyworld> anastasiamac: don't you want to talkto me and andrew? :-(
<anastasiamac> wallyworld: m always happy to tlk to andrew \o/
<wallyworld> :-( :-( :-( :-( :-(
<anastasiamac> wallyworld: that's a lot of faces u r wearing.. masks? :D
<wallyworld> it's me crying
<anastasiamac> u and me both
<veebers> anastasiamac: lol, I think it's because you only want to talk to Andrew not Ian ^_^
 * wallyworld sobs quietly
 * anastasiamac hands tissues virtually
<babbageclunk> redir: ping?
<babbageclunk> redir: Entertainingly a bit of live testing shows that the PUT version doesn't do what I want (partial update), while the POST does.
<babbageclunk> redir: directly counter to the docs.
<babbageclunk> wallyworld, axw: my connection just dropped, so not sure whether anyone saw my last message: could someone review https://github.com/go-goose/goose/pull/39
<babbageclunk> Please?
<wallyworld> yeah, connection dropped
<axw> babbageclunk: will do
<babbageclunk> Thanks!
<babbageclunk> I need to duck out soon to take my sister to the airport but I'll be around later on.
<axw> babbageclunk: what openstacks have you QAd on? I'm curious about the discrepancy between docs and reality
<babbageclunk> axw: yeah, I thought that was weird too. I've only tried on canonistack.
<axw> babbageclunk: ok. would be good to test on rackspace also
<babbageclunk> axw: good call, I'll do that now
<babbageclunk> axw: rackspace behaves the same, although my test fails because they put some other stuff into the metadata. Fixing that now.
<axw> wallyworld: do you think it would be reasonable to add a new "introspection" access level? to enable prometheus, you would add a user for it with introspection access; and then other users could be granted that access also, and they can hit up the pprof endpoints without having full access
<axw> babbageclunk: okey dokey
<axw> wallyworld: otherwise we'll need to add separate auth for introspection, with a special user/pass
<wallyworld> axw: so introspection < read < write < admin?
<axw> wallyworld: there's no read on controller
<axw> wallyworld: login < introspection < superuser
<wallyworld> ah controller, right
<axw> wallyworld: hmm, but add-model ...
<axw> sigh
<axw> single dimension access does not work
<wallyworld> yeah
<wallyworld> we could chamge that
<wallyworld> any of login, add-model could have introspection added
<wallyworld> superuser implies it
<axw> wallyworld: or we could just have introspection imply add-model. you still have to have creds
<axw> wallyworld: so, login < add-model < introspection < superuser
<wallyworld> not a fan that
<wallyworld> because
<wallyworld> why would you *need* to allow someone to add-model just because you want them to be able to introspect
<axw> wallyworld: the permissions model just does not allow for multiple dimensions. what's the alternative?
<axw> wallyworld: (I agree that it *shouldn't imply add-model)
<axw> wallyworld: I guess another option is that you have *either* introspection *or* add-model
<axw> wallyworld: kinda crappy, means you can't grant introspection to someone who can add models
<axw> wallyworld: but maybe the least crappy option for now?
<axw> wallyworld: bleh, but of course everything relies on them being in a single dimension for comparison
<axw> wallyworld: new idea: support adding access for application tags, and introduce a conceptual "juju-controller" application. having "read" access on that would give you the ability to introspect it
<axw> wallyworld: that goes in the direction of modelling juju in juju
<blahdeblah> Is thumper not around today?
<anastasiamac_> blahdeblah: on holidays
<blahdeblah> OK - ta
<anastasiamac_> blahdeblah: back next week, but maybe not monday (NZ public holiday?)
<blahdeblah> I've got a reoccurrence of a bug he was helping me with, but the bug as described doesn't match what the symptoms were when we looked at it.
<anastasiamac_> :(
<blahdeblah> The bug is https://bugs.launchpad.net/juju-core/+bug/1484105, but the problem we saw at the time was that there was mongo corruption which prevented charms from reporting status correctly.
<mup> Bug #1484105: juju upgrade-charm returns ERROR state changing too quickly; try again soon <bug-squad> <canonical-is> <upgrade-charm> <upgrade-juju> <juju-core:Fix Released> <https://launchpad.net/bugs/1484105>
<blahdeblah> So what happens is the charm gets upgraded on disk, but the results don't show up in juju status as upgraded.
<blahdeblah> So I'm not sure whether I should log a new bug about this, or just update that one, even though the symptoms don't match. :-\
<anastasiamac_> blahdeblah: maybe a new bug? a link the old one to it to describe how u think they r related?...
<blahdeblah> Is there much point opening a new bug on 1.x?  It makes juju fairly broken, but we have a reasonably clear procedure for fixing it, so it's not likely to get much traction...
<anastasiamac_> blahdeblah: true. add new info to existing bug... m not even sure if we have capacity to dig into any 1.x at this stage
<anastasiamac_> blahdeblah: which 1.x is it?
<blahdeblah> 1.25.9
<wallyworld> axw: thanks for review, was good outcome
<axw> wallyworld: np
<axw> wallyworld: re my earlier spam. I think I've settled on supporting user being superuser on controller, or having read access to the controller model
<axw> wallyworld: sound reasonable?
<wallyworld> yeah, i think so. if you can read model you should be ok to introspect
<wallyworld> we can ask for "guidance"
<wallyworld> but keep it simple firs tup
<axw> wallyworld: I'll just email the list once as a heads up, guidance can be given later
<wallyworld> yep, that's what i meant sorry
<axw> cool
<axw> wallyworld: if you have time, please review https://github.com/juju/juju/pull/6913
<wallyworld> sure
<axw> wallyworld: do you know if it's intentional that when you use "controller-config" or "model-config" with a single key, that multi-line strings get formatted as YAML ?
<wallyworld> um, i *think* so, I *think* tim did it?
<wallyworld> recently?
<wallyworld> axw: just started looking - don't we already have a base auth http handler?
<axw> wallyworld: I don't think so? everything I've looked at uses httpContext
<axw> wallyworld: re controller-config, seems a bit crappy that "juju controller-config ca-cert" doesn't just give me the ca-cert unadorned. it comes out with YAML formatting
<wallyworld> hmmm, we used to have an authContext of some sort. but now we have httpContext with func (ctxt *httpContext) stateForRequestAuthenticated(r *http.Request)
<wallyworld> agree about the cert config, should ask tim next week
<axw> wallyworld: Server has an authContext, httpContext has a Server
<wallyworld> ok, np. seems there's been a bit of refactoring since i last looked at that code
<wallyworld> i have to do school pickup, bbiab
<jam> axw: wallyworld: single-key feels like we should be doing a raw-dump of the output, *not* formatted
<jam> its intended that you can do "FOO=`juju config foo`"
<jam> well probably FOO=$(juju config foo)
<axw> jam: yeah, that's what I thought. was surprised when it didn't work
<jam> axw: sounds like a bug to me
<axw> jam: yeah, I'll file one
<wallyworld> axw: that PR  LGTM. Will be a nice enahncement
<axw> wallyworld: thanks
<mup> Bug #1661576 opened: show-status-log formatting contains unwanted pad lines <juju-core:New> <https://launchpad.net/bugs/1661576>
<mup> Bug #1661624 opened: show-status-log bogus timeline <juju-core:New> <https://launchpad.net/bugs/1661624>
<mup> Bug #1661681 opened: Broken agent complaints about tomb: dying <juju-core:New> <https://launchpad.net/bugs/1661681>
<redir> morning juju-dev
<perrito666> redir: morning redir
<perrito666> I thought you where out today
<redir> fo a bit in a couple hours but not all day perrito666
<balloons> redir, you about?
<redir> yup
<redir> balloons:
<balloons> redir, I kind of want to land this for beta5 to fix the packaging issues: https://github.com/juju/juju/pull/6915
<balloons> look at whose being crazy now
<balloons> redir, the only real change is to allow 2.* and not 2.0. The rest is just renames for sanity
<redir> balloons: shipit
<balloons> redir, :-)
<balloons> redir, while you're at it, bump the version PR: https://github.com/juju/juju/pull/6914
<redir> balloons: looking
<redir> balloons: lgtm
<perrito666> k EOD, ill be online if annyone needs me
#juju-dev 2017-02-04
<hoenir> Was'up dudes? How ya doing?
#juju-dev 2017-02-05
<blahdeblah> Morning all; I have another occurrence of bug 1660087
<mup> Bug #1660087: Controller and all models unresponsive <canonical-is> <juju:Triaged by menno.smits> <juju 2.2:Triaged> <https://launchpad.net/bugs/1660087>
<blahdeblah> Unless I hear otherwise, I'll just gather profiling data & logs, and pop them on the bug for the next lucky contestant to review.
<axw> anastasiamac: your computer's back to life?
<anastasiamac> axw: \o/ after an upgrade from lts, yes :) took forever to figure out (and millions of driver/kernel combinations trials).. took the whole weekend :( 2 days out of my life to not get back :(
<axw> anastasiamac: :(
<anastasiamac> blahdeblah: on 2.0.2?
<blahdeblah> anastasiamac: That is the stable branch, so that is what we use. :-)
<anastasiamac> blahdeblah: yeah. since we r not planning another 2.0.x, would b awesome to figure out how to test ur scenario with 2.1.x/2.2.x to ensure we addressed the issue ;)
<blahdeblah> anastasiamac: Indeed it would, but this is a production archive mirror, so it would probably need to be discussed with managers to find out whether they're prepared to risk that.
#juju-dev 2018-01-29
<axw> anastasiamac: do you want to do standup today?
<anastasiamac> axw: i think i just lost u... reconnecting
<axw> yeah
<axw> anastasiamac: I've just restarted hangouts in case it was me...
<axw> jam: finally got raft things working again, after making the changes to store API server addresses directly in raft cluster config: https://github.com/juju/juju/pull/8308
<jam> axw: will look later today. is 8308 where I should start? It seems to say it was last updated 1 week ago
<axw> jam: it is. some commits are from 7 days ago, some are new or updated
<axw> jam: have you already started looking at my PR? I've got one last change to push... don't want to interrupt your reviewing if you've already started though
<jam> k. I just looked at the very beginning, I haven't gotten very far, currently in standup
<axw> jam: ok. I'll just add as extra commits, and rebase later
<jam> sgtm
<jam> wpk:  /wave
<wpk> jam: o/
<balloons> anyone working on updating lxdclient?
<wpk> balloons: in juju, for current master?
<wpk> balloons: if yes - yes, I am
<balloons> wpk, I thought someone was :-) I'm working on doing the socket updates so juju works with both lxd sockets. Do you have a branch I can base off of?
<wpk> balloons: I started today and I'm currently reading the code for both lxdclient and new client in lxd, so just branch off develop
<balloons> wpk.. hmm, I wonder if I should wait on the work then. I need to pull in the newer lxdclient as well, so I didn't want to duplicate.
<wpk> balloons: what do you want to have?
<balloons> wpk, i need a new enough lxd that understands snaps
<balloons> wpk, see https://bugs.launchpad.net/juju/+bug/1743719
<mup> Bug #1743719: Juju should prefer lxd snap socket <juju:Triaged by nskaggs> <https://launchpad.net/bugs/1743719>
<wpk> balloons: if you have something then you can push it to 2.3, I don't think we'll have the new LXD code in 2.3 anyways
#juju-dev 2018-01-30
<jam> axw: I sent a first set of review comments on your PR. I'm going to try and finish reviewing it today, but i realize that it would have been useful to submit the feedback I had earlier so you could respond
<axw> jam: thanks
<axw> jam: in one of your comments, it sounds like you're leaning back towards the original approach of mapping IDs to addresses not stored in the raft config, is that right?
<jam> axw: so I wanted to discuss it at least. I thought you were doing separate mapping on our side, not part of the raft config
<jam> you were saying that you needed to publish the alternative config and track it somewhere
<jam> which was my big concern
<axw> jam: originally I had (in the raft config) the server ID and address both equal to machine-tag
<axw> jam: then the transport code would map the "address" (machine tag) to the real API server hostport
<jam> axw: ah, that's probably the part I didn't want. I'd prefer not to have the "address" be a tag
<axw> jam: it eliminates the split-brain issue, since you have a stable address in the raft config (no need to remove/add). what's your concern exactly? about consistency?
<axw> jam: consistency/ordering isn't ideal for name resolution, you always want what's current?
<jam> axw: my main concern is that the raft config is no longer self contained. you need to look up somewhere else that you then need to keep consistent
<axw> jam: ok. I agree that it would be nice to have it self-contained. it's unfortunate that there's no atomic config change op
<jam> axw: so it separates the idea of what IP address a given node has, but it doesn't let you change the address of the node without removing it?
<jam> axw: that seems weird given the separation
<axw> jam: yup
<axw> jam: unless docs lie...
<axw> I see code that looks like it *might* allow an address change, which is not what the docs say
<jam> axw: looking at the code in configuration.go
<axw> yep, same
<jam> looks like you can just AddVoter(existingId, newAddress)
<jam> and it will overried
<jam> if server.Sufferage == Voter ; Servers[i].Address = serverAddress
<jam> axw: ^^
<axw> jam: yep, I just saw the same. I'll test it now. yay for misleading doc comments
<axw> "If the server is already in the cluster as a voter, this does nothing."
<jam> axw: yeah, there definitely is a discrepency between some of the code comments and the "design" vs the "implementation"
<jam> things like "AddVoter" actually adds it as staging, but promotion actually does nothing which means it actually adds it as voter
<axw> jam: ok, it works fine - I'll update the code to get rid of the intermediate remove op, and I'll add in a check to stop it from removing a server if it could lead to split-brain
<jam> wpk: I looked at PR 8323
<jam> axw: that changes your code also to handle the "localhost" to "public IP" case, doesn't it?
<axw> jam: sorry, don't follow - which case is that? you mean changing the bootstrap address (juju.localhost)?
<jam> axw: you have comments around "if len(X) < 3)" and some comments in the test suite about handling when we have 1 entry vs 3 entries
<jam> axw: can we simplify that code/
<jam> ?
<axw> jam: yes, as we speak :)   I'll just make sure that it doesn't remove any servers if that would lead to an even-sized configuration
<axw> add or remove...
<jam> axw: i'm less concerned about an even-sized config, because often it is necessary to go via that to get to what you want. I'd like to move our existing ensure-ha to not mandate even-always
<axw> jam: OK, I'll leave that for now then
<axw> I'll just drop the length check
<jam> axw: sgtm. (specifically, if there is a breakage on one of your machines, you'd really like to just kill that machine, and then bring it back into rotation, and our code around "protect you from yourself" usually gets in the way of people actually fixing stuff)
 * axw nods
<jam> axw: I updated my comments on 8308
<axw> jam: I've pushed updates
<axw> jam: I have no strong feeling regarding Internal vs. HA, so let me know if you'd like me to change it. also please see my reply regarding Debug vs. Info for cluster config updates
<balloons> how is the ami chosen when creating a unit?
<balloons> wallyworld, ^^?
#juju-dev 2018-01-31
<balloons> vern, were you able to get the spaces test job created on jenkins?
<thumper> morning people
<hml> morning thumper
<rick_h> howdy thumper, thuoght you were away this week?
<thumper> rick_h: hi there
<thumper> morning hml
<thumper> rick_h: nope, just until today
<thumper> back into it
<thumper> short week for me
<rick_h> gotcha, cool
<babbageclunk> hi thumper
<thumper> babbageclunk: morning
<thumper> babbageclunk: how goes the sprint?
<vern> balloons or veebers, quick review? https://github.com/juju/juju/pull/8333
<veebers> vern: LGTM with one comment (not a blocker)
<vern> veebers: I assume your comment meant to reference .set_region? I think that's clearer... I'll update that real quick
<veebers> vern: so sorry, copy and paste error and I didn't even check it :-(
<veebers> vern: yes I meant to add .set_region(...) to the end of that. Sorry for the additional overhead
<vern> all checks pass. merge requested :) https://github.com/juju/juju/pull/8333
<vern> grr... in ci the machines have ens3, not eth0... guess I need to improve the logic of that part of the network spaces test
#juju-dev 2018-02-01
<thumper> axw: can I grab you in about 30 min for a chat?
<axw> thumper: yes but I need to go out for an appointment in 50 mins
<thumper> ack
<thumper> axw: now?
<axw> thumper: sure, 1:1?
<thumper> ack
<thumper> axw: I had to go hunting for an old hangout link
<axw> heh me too
 * thumper EODs
<codenameba> ââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  onwbnzhmn: tinwood rogpeppe cmars ââââââââââ
<codenameba> âââââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  gcyboohq: alexlist coreycb jog_ ââââââââââââ
<codenameba> ââââââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  zebvh: rogpeppe tasdomas joedborg ââââââââââââ
<codenameba> âââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  onqha: blahdeblah Mmike meetingology âââââââââââ
<codenameba> Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  dvhznvpupg: meetingology jillr zeestrat Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢
<codenameba> ââââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  gkizhbv: rick_h elmo jose âââââââââââââ
<codenameba> âââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  fhhzaeg: ejat rick_h redir ââââââââââââââââ
<codenameba> âââââââââââââââââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  ksqzlqs: niemeyer chaology blahdeblah âââââââââââââââ
<codenameba> Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  zjwrxodat: mpontillo jw4 bogdanteleaga Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢â
<codenameba> Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââ SPECIAL ANNOUNCEMENT: IRC.SUPERNETS.ORG #SUPERBOWL IS HAVING A SUPERBOWL WATCH PARTY ON FEB. 4TH. MESSAGE CHRONO FOR DETAILS!!  lyndkhssz: niedbalski anastasiamac tvansteenburgh Ã¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢ââÃ¢
<jam> rogpeppe1: I might have targetted the wrong branch for the mgo change. I probably wanted to target the v1 branch, looks like I targeted master by mistake
<rogpeppe1> jam: no, looks like the one that landed was targeting v1
<rogpeppe1> jam: https://github.com/go-macaroon-bakery/macaroon-bakery/pull/196
<jam> rogpeppe1: thx, I guess I just missed seeing it from gopkg, but it seems to be there
<thumper> morning
<alexisb> morning thumper
<thumper> alexisb: hey, still miss you
<alexisb> :) I miss you guys as well
<zeestrat> thumper: morning. I've got about 30-45 min before EOD if you'd like me to poke at #1746265
<mup> Bug #1746265: juju-upgrade from 2.2.9 to 2.3.2 fails with state changing too quickly <upgrade-juju> <juju:Triaged> <juju 2.2:Won't Fix> <juju 2.3:Triaged> <https://launchpad.net/bugs/1746265>
<thumper> zeestrat: ack, otp just now
<zeestrat> Roger. If I'm out, just leave some steps in the comments and I can mess about and get some results for monday.
<thumper> zeestrat: can you do google hangouts?
<zeestrat> thumper: sorry, remoting from home atm without a good setup
<thumper> hmm...
<thumper> ok
<thumper> I suppose I can talk you through it :)
<thumper> zeestrat: so, am I right in thinking that you have a reproduction of the upgrade problem right now?
<thumper> where it is mid-upgrade
<thumper> and one agent hasn't restarted ?
<zeestrat> Yes. Though the agents in controller 0,1,2 all seem to be spewing different error messages so I'm not sure which one's are stuck
<thumper> :)
<thumper> kk
<thumper> the first thing to do is to work out which one is stuck
<thumper> so... open terminals into each of the three machines
 * thumper quickly bootstraps a lxd to make sure he gets the commands right
<zeestrat> :)
<thumper> you want to look for 'running jujud' in the machine log files
<thumper> grep machine*.log 'running jujud' | tail
<thumper> something like that
<thumper> look for the timestamp of when each of the agents most recently started
<zeestrat> in decending order, machine-0 11:12:51, machine-2 11:15:25, machine-1 11:15:27, so machine-1 was the last one to the party
<thumper> does the 11:15 time match the upgrade-juju?
<thumper> roughly?
<thumper> can you pastebin a bunch of the logs from machine-1 to pastebin.ubuntu.com ?
<zeestrat> They're all in the bug so you could take a look there :)
<zeestrat> Time sounds right, though I don't have the exact timestamps
<thumper> zeestrat: how is your mongo foo ?
<thumper> mongo fu?
<zeestrat> fubar
<zeestrat> But if you have some docs I'll probably get through them.
<thumper> let me pastebin you a cool command
<zeestrat> These are still usable? https://github.com/juju/juju/wiki/Login-into-MongoDB
<thumper> https://pastebin.ubuntu.com/26502452/
<thumper> zeestrat: this is better
<thumper> put that in your path somewhere as executable called 'juju-db'
<thumper> then you can just type 'juju db' to get a mongo shell on the controller'
<zeestrat> Very handy indeed. What's next?
<thumper> zeestrat: I need to see the content of the leases collection
<thumper> which you can get with db.leases.find().pretty()
<thumper> it appears that the leases are a bit fubared
<zeestrat> Would you like that for all controlelrs?
<thumper> no, just one
<thumper> since the db is replicated, it should be the same info
<thumper> no matter which we look at
<zeestrat> Roger
<thumper> although by default you can only query the master
<thumper> so if the command doesn't show "juju:PRIMARY"
<thumper> it is probably quicker to try another machine rather than me digging the command to query from replicas
<zeestrat> Yeah, sorry, was just finding a nice way to dump mongo into pastebinit
<zeestrat> https://paste.ubuntu.com/26502475/
<zeestrat> Ah, didn't spot the type "it" for more. Here's the complete dump: http://paste.ubuntu.com/26502478/
<thumper> cool
<thumper> now I want to get the transaction that is failing...
<thumper> this will be more interesting
<thumper> db.txns.find({"o.c": "leases", "s": 5}).sort({"$natural":-1}).limit(1).pretty()
<thumper> zeestrat: which says
<thumper> find the last transaction that was aborted that tried to modify the leases collection
<zeestrat> http://paste.ubuntu.com/26502504/
<thumper> ok, thanks
<thumper> now I have what I need to work out why it is failing
<thumper> if you want to wait a few minutes I might be able to get you past it...
<zeestrat> I've got about 10 minutes, but no rush to get it fixed right now. A comment in the bug will do fine.
<zeestrat> Do let me know if you want to run some other permutations or fixes over the weekend
<thumper> zeestrat: ok, you best go then
<zeestrat> thumper: Thanks for taking the time and effort. We owe you and the team a round of alcoholic beverages if any of you ever stop by Norway :)
<thumper> :)
<thumper> unlikely, but not impossible
<zeestrat> beeroverip.org just doesn't hit the spot
<zeestrat> Thanks again and have nice weekend.
<thumper> cheers, you too
<kwmonroe> hey thumper, is it true that only 1 hook will ever run on a unit?  so like if a million apps are smashed onto a single unit, the agent will coordinate such that only 1 hook is running at any given time?
<kwmonroe> the key part there being "at 1 time", which i didn't make clear in that first part
<thumper> kwmonroe: yes
<kwmonroe> thx
<thumper> kwmonroe: although I think you mean a million units on a single machine
<thumper> but the answer is the same
<kwmonroe> thumper: let's be real here, there are a lot of things i would have changed about that initial ping if i could.
<thumper> :)
#juju-dev 2018-02-02
 * thumper out, EOW
<anastasiamac_> jam: not point in conversing on github on a landed commit... it just makes sense that whoever worked on the code in a branch forward-ports it or ditributes it around - if ever there r any conflicts, this is the best person to deal with them...
<wallyworld> babbageclunk: meet  in standup ho
<babbageclunk> wallyworld: wilco
<mup> Bug #1727355 opened: Juju attempts to bootstrap bionic by default <cdo-qa> <cdo-qa-blocker> <cdo-release-blocker> <foundations-engine> <sts> <uosci> <verification-done> <verification-done-xenial> <verification-done-zesty> <juju:Fix Released by nskaggs> <juju-core:New> <distro-info-data
<mup> (Ubuntu):Won't Fix> <juju-core (Ubuntu):Invalid> <distro-info-data (Ubuntu Trusty):Won't Fix> <juju-core (Ubuntu Trusty):Invalid> <distro-info-data (Ubuntu Xenial):Fix Released by vorlon> <juju-core (Ubuntu Xenial):Fix Released> <distro-info-data (Ubuntu Zesty):Fix Released> <juju-core (Ubuntu
<mup> Zesty):Fix Released> <distro-info-data (Ubuntu Artful):Won't Fix> <juju-core (Ubuntu Artful):Invalid> <https://launchpad.net/bugs/1727355>
#juju-dev 2018-02-04
<mup> Bug #1654136 changed: Removing relationship didn't remove subordinate unit <canonical-bootstack> <juju-core:Expired> <https://launchpad.net/bugs/1654136>
