[00:04] wallyworld: r u getting my msgs?... [00:05] yes, but in the midle of something [00:05] be with you in a sec [00:06] wallyworld: k :D [00:53] anastasiamac_: 0/ how are you [00:53] katco: o/ m good :D [00:53] katco: u? [00:53] anastasiamac_: doing good! [00:54] thumper: did my performance review, with literally hours to spare [00:54] a new record [00:55] davecheney: thanks [00:55] davecheney: when are you off? [00:55] flight leaves at three [00:55] i need to get my ass in a cab [00:55] :) [00:55] have a good trip [00:56] will do === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 [01:51] sinzui & co. - fi scenario does indeed succeed with 1.20.x - bug updated https://bugs.launchpad.net/juju-core/+bug/1420049 [01:51] Bug #1420049: ppc64el - jujud: Syntax error: word unexpected (expecting ")")

[01:51] beisner, thank you very much. [01:51] s/fi/fyi/ [01:52] sinzui, happy to test it. holler if you need me to throw anything else against that hardware. [01:53] wallyworld, bug 1420049 is confirmed to be a regression from 1.20.x [01:53] Bug #1420049: ppc64el - jujud: Syntax error: word unexpected (expecting ")")

[01:58] wallyworld, bug 1407699 appears to be back. precise and aws and lxc are failing in CI for branch 1.22 [01:58] Bug #1407699: Precise services cannot be deployed

[01:59] wallyworld: http://reviews.vapour.ws/r/928/ - one issue unresolved, I've replied with a possible solution, see what you think === kadams54 is now known as kadams54-away === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1416928 1423036 [02:09] thumper: howdy? r u looking at bug 1421237?.... [02:09] Bug #1421237: DEBUG messages show when only INFO was asked for

[02:09] thumper: it currently blocks CI :D [02:10] anastasiamac_: I'm busy arguing about it - it isn't us [02:10] thumper: \o/ [02:13] I've just commented again [02:13] and moved it back to incomplete [02:13] I'm guessing the problem is environmental [02:21] thumper: :D [02:21] wallyworld: you around? [02:22] wallyworld: I'd like you to join waigani and me for a chat about destroy environment behaviour [02:22] * thumper looks at the time and guesses wallyworld is having lunch [02:23] waigani: probably best to wait for wallyworld [02:23] so I'm not repeating myself [02:23] we are going to need clear docs around this too :-) [02:24] thumper: ah yep okay [02:25] thumper: how goes? [02:25] rick_h_: good [02:25] rick_h_: I'm wanting more feedback on my spec BTW [02:25] thumper: k, will see what I can do [02:25] I'm going to chase up john too [02:25] thumper: good plan [02:25] rick_h_: I may take urulama off the approval list [02:25] thumper: k [02:26] as I've removed many references to other things out of the scope of the spec [02:26] rick_h_: and I'm wanting to get moving on this ASAP [02:26] thumper: understand [02:27] * thumper is considering a flag called --dont-look [02:27] or perhaps --jfdi [02:27] nah [02:27] what part needs jfdi'ing? [02:28] perhaps it should be --i-know-there-are-no-other-environments [02:28] juju destroy-environment --force [02:28] when there are possibly multiple environments [02:28] ugh [02:28] it'll leave other environments running [02:28] but in a deadish state [02:28] gets back to new commands for 'server' vs environment [02:29] this 'special' environment thing is going to kill us [02:29] do you think it should be 'juju server destroy' ? [02:29] juju destroy-environment --force ec2 does just that, juju destroy-server is what you need to kill a JES [02:29] yep, no way to do it on accident [02:29] but... [02:29] there is no --force [02:29] for a non-server environment [02:30] we do that anyway [02:30] provider, plz kill all these machines [02:30] ok, so what? [02:30] so there's not a flag --force on the new command for a JES [02:30] so 'juju server login' ? [02:30] big deal [02:30] +1 from me [02:30] juju server list? [02:30] make it less magical to users [02:30] what does that mean? [02:30] list servers that I have cached local info about [02:31] e.g. list the servers I've logged into [02:31] not list me the environments on the current server? [02:31] nope, that's juju login some-server [02:31] juju environment list [02:31] and I don't expect to see the JES in that list [02:31] hmm... [02:31] basically don't expose the implementation detail to user, make it a completely safe split [02:32] in my opinion and all that [02:32] otherwise there's too many 'oops' and 'doh' cases to think about [02:32] now we are breaking history [02:32] what history? JES is new :) [02:32] because destroy-environment now isn't how you stop a local environment [02:32] it's a new feature with new commands and ... [02:32] bootstrap still starts and environment [02:32] how does one stop a local environment then? [02:33] bootstrap starts a JES [02:33] don't call it an environment [02:33] this is going to fuck so many people off [02:33] again, implementation detail to the user [02:33] why? [02:34] I think if you list it out there's a whole lot less pain to deal with this way than trying to tell apart magic environments from non-magic environments across users/scripts/cli [02:34] hmm... [02:35] however... [02:35] renaming the function doesn't stop the problem [02:35] juju server destroy foo --force -y [02:35] could still leave environments running but dead [02:35] then it needs to clean them up before it dies or else block and stop and refuse to die [02:35] juju server destroy foo --force -y --trust-me [02:36] until those envs are gone [02:36] --force doesn't fail if the API is broken [02:36] ic [02:36] not sure --force even tries [02:36] * thumper looks [02:36] // If --force is supplied, then don't attempt to use the API. [02:36] // This is necessary to destroy broken environments, where the [02:36] // API server is inaccessible or faulty. [02:37] you know sabdfl will want it to be like his locking spec then. [02:37] thumper: you'll have to have a code or something to enter with giant warnings like this block stuff getting implemented [02:38] thumper: see https://docs.google.com/a/canonical.com/document/d/1oBZ2oIZOok64_QbWVPfY5Q2I053peESPZYtKSdF_L6E/edit for the original stuff on it [02:38] rick_h_: that didn't make sense [02:39] thumper: saying that if --force is going to potentially do bad things to running envs ^ document and https://docs.google.com/a/canonical.com/document/d/1XFYlNGmQH7-X68IXhdxsANwHN6DgAL_RMn7IDK7ZbJc/edit will come into play [02:39] thumper: well, they should come into play [02:39] * thumper is looking [02:41] rick_h_: the problem with all these solutions is they don't take into account the --force option [02:41] as it lives now [02:41] perhaps we have to remove the --force option altogether [02:41] thumper: right, but they're all trying to work in the space of your problem so whatever you do should take into account all of that [02:41] rick_h_: really this behaviour is outside the scope of what we are doing [02:41] and we have blocks now [02:41] thumper: possibly, or provide --force via a third party tool like the stuff to remove juju now [02:41] which anastasiamac_ has been doing [02:42] I think her current work addresses some of this [02:42] thumper: right, agree with that [02:42] rick_h_: I think perhaps a developer plugin, like the juju-nuke plugin [02:42] just saying it should fit whatever you end up doing as well. e.g. don't make it easy for users to accidentally make envs go boom that should not have [02:42] thumper: +1 [02:42] fark... [02:43] how to get all this done sanely and quickly... [02:43] but get it right vs fixing it later :) [02:43] yeah... [02:43] fuck fuck fuck fuck fuck [02:44] shit [02:44] rick_h_: I was feeling pretty happy with this document before [02:45] remeber, "I love my job I love my job" [02:45] and now it is all screwed [02:45] thanks [02:45] sorry man, you asked for feedback. :/ [02:45] don't get me wrong, I appreciate the feedback [02:46] it is just I'm getting real sick of speccing all this out [02:50] thumper: let me know if there's anything I can do to help. I really care/want this to come out awesome and happy to help. [02:50] thumper: but been on the road 12 hrs today so going to go to bed and crash. === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 [02:52] rick_h_: ack === _thumper_ is now known as thumper === kadams54 is now known as kadams54-away [03:07] * thumper heads away from the laptop to think this over and reread the spec with fresh eyes === kadams54-away is now known as kadams54 [03:40] sinzui: i saw the precise bug. i added some input. i suspect an out of date apt mirror, but could be wrong [03:41] wallyworld, aws, joyent, and hp? [03:41] sinzui: before i committed the previous fix, i tested on aws. the bug refers to an aws CI test. i just now tested with AWS again [03:41] so the CI run fails on AWS [03:42] wallyworld, yes [03:42] but i have no trouble running up an aws env [03:43] but theory about an apt mirror out of date is just a theory [03:43] i could be totally off the mark [03:43] aws got [03:43] Unpacking cloud-image-utils (from .../cloud-image-utils_0.27-0ubuntu9~ctools0_all.deb) .. [03:44] it's cloud-utils we are worried about [03:44] https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/tools [03:45] cloud-archive-utils 0.1-0~35~ubuntu12.04.1 Ubuntu Cloud Archive Team (2015-02-13) [03:45] waigani, Removing cloud-utils ... [03:45] the metadata for cloud-utils was wrong, and the fix is in the above [03:45] if an old cloud-utils is used, the issue as per the failed CI test occurs [03:46] wallyworld: got a few minutes? [03:46] the issue is that installed cloud-image-utils causes cloud-utils to be removed [03:46] sinzui: RE: the debug logging bug, I'm pretty sure it is environmental [03:46] sinzui: and "hai" [03:46] wallyworld: I want to run some ideas past you [03:46] wallyworld, I just attached the cloud-init-output log to https://bugs.launchpad.net/juju-core/1.22/+bug/1423036 [03:46] Bug #1423036: precise services cannot be deployed (again)

[03:47] i'm confused as to why i can run up an AWS env ok, but CI fails [03:47] wallyworld: same region? [03:48] sinzui, wallyworld: I have experienced differences in AWS over different regions before [03:48] not sure if they still operate weirdly [03:49] thumper, good example. wallyworld I think this case is broader since the precise machine in Hp and precise tests in joyent also failed [03:49] wallyworld, I know mirrors should update every day, but some can be a week behind [03:50] but for all 3 to be behind? that is surprising [03:51] wallyworld, I also blew away the lxc cache on the precise lxc machine [04:04] wallyworld, https://launchpad.net/ubuntu/trusty/amd64/cloud-image-utils says the new cloud-image-utils is *proposed* not updates as cloud images require [04:06] thumper: https://bugs.launchpad.net/juju-core/+bug/1413052 feedback on my reply when you have a second [04:06] Bug #1413052: juju status hangs

[04:09] sinzui: your aws deployment is indeed pulling out of date cloud-utils. I added some info to bug. But I don't know enough packaging and the repos involved [04:10] there's different data here http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/precise-updates/cloud-tools/main/binary-amd64/Packages and here https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/tools [04:10] the latter is up to date, the former is out of data [04:10] date [04:10] why my AWS deployment works I have no idea; I would have though it would be consistent [04:11] wallyworld, I can see the joyent pulled the old package from the real archive, but Lp does agree the new package arrived a few days ago [04:12] unless joyent, aws, and hp have proxies between ctools [04:28] wallyworld, I need to sleep. When I look at http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/precise-updates/cloud-tools/ and everything below, all the data is from last year. no files mention the new packages [04:30] wallyworld, but I can see that proposed does know about the new packaged http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/precise-proposed/cloud-tools/ [04:32] Maybe the archive is not really at the state Lp thinks it is. This is true for releasing juju. I need to count the packages in the archive because Lp doesn't read the file systems [04:36] sinzui: do you have any idea about the archive issues? [04:36] wallyworld, waigani: we should hold off on the destroy-environment landing until we get some agreement on the spec [04:37] wallyworld: please hold off any more reviewing until you, me and waigani have had a chat [04:39] wallyworld: did you get either of those? [04:40] those what? [04:40] stupid freenode conection :-( [04:40] wallyworld, waigani: we should hold off on the destroy-environment landing until we get some agreement on the spec [04:40] wallyworld: please hold off any more reviewing until you, me and waigani have had a chat [04:40] sure [04:40] ta [04:40] thumper: is that due to the conversation with rick? [04:41] yup [04:41] i was wondering about that === kadams54 is now known as kadams54-away [05:57] wallyworld: dunno if u saw it (with ur connection misbehaving and all)... [05:58] wallyworld: re-proposed PTAL http://reviews.vapour.ws/r/941/ [06:02] ok, might have to be after soccer === urulama_ is now known as urulama [08:53] morning [09:17] fwereade_: when you're around, give me a ping [09:20] jam, heyhey, just finished with domas [09:23] fwereade_: hey, brb. I'll be in leader-election hangout when you're available just using the restroom quickly [09:23] dimitern: ping [09:24] TheMue, hey [09:25] dimitern: seen your comments on GH, some are already done in the PR [09:26] dimitern: regarding the MachinePortsResults, this type relies on MachinePorts which is older and looks different than our networking stuff [09:26] dimitern: but still move anything related to networks? [09:27] dimitern: at least it uses a NetworkTag [09:27] dimitern: in MachinePorts [09:27] TheMue, yeah, it's older but still networking related and used by the firewaller IIRC [09:27] dooferlad, ping [09:27] dimitern: it is, yes [09:27] dimitern: ok, so will move it too [09:27] dimitern: Hi, was just looking at that bug [09:27] TheMue, cheers [09:28] dooferlad, hey! so the foreport to 1.22 should be easy, but let's talk about the fix during/after standup [09:28] dimitern: will do [09:29] dooferlad, ta [10:03] TheMue, dooferlad, sorry, I'll be ~5m late for standup [10:28] fwereade_: http://reviews.vapour.ws/r/949/ for some of the cleanups that I've done so far. [10:49] fwereade_: ping [10:50] jam, sorry, missed the review link [10:50] fwereade_: np. Not urgent. I wanted to check something else with you [10:50] specifically what we found with livesource.go [10:50] jam, the for loop? [10:50] where it was using wg.Add() wg.Done() which means that the goroutine is blocked if you never call the func [10:50] jam, ah yes [10:51] I have a test that fails, by observing that source.Changes() channel is never closed if you call source.Stop() but don't call the func. [10:51] I'm trying to figure out what we want instead of the for loop. Is it that we need to go to a full func loop() ? [10:51] morning all [10:51] hi perrito666 [11:00] fwereade_: thoughts on what the correct fix is? I think we talked about NewLiveSource needing its own Tomb [11:01] or alternatively use a channel implementation [11:01] (create a channel, close it in the func(), rather than wg.Add, select over that channel and the tomb so we know when we need to clean up.) [11:02] I think with a channel we don't have to have a full "loop" but we probably need a Tomb, and if we have a tomb, going to a loop() is probably clearer code as it follows the model of similar code. [11:05] jam, yes, I think we do in general just want our own tomb [11:05] jam, hence for loop [11:06] jam, and I think it's generally nice practice to pull the loop out into its own method, returning an error, for consistency's sake if nothing else [11:06] k [11:16] jam, I think I remember how to do it "right" [11:17] jam juju/state/watcher.Stop(Stopper, *tomb.Tomb) [11:17] jam, so if it were in a less unhelpfully-samed package [11:18] jam, it would be clear that goroutine-wrapping-some-resource is a common pattern [11:18] jam, and that Stop and EnsureErr are both generally useful for any components that wrap one resource [11:19] jam, so [11:20] defer self.tomb.Done() [11:20] defer resource.Stop(self.resource, &self.tomb) [11:20] self.tomb.Kill(self.loop()) [11:21] jam, ...along with [11:21] case _, ok := <-self.resource.Changes(): [11:21] [11:21] if !ok { [11:22] return resource.EnsureErr(self.resource) [11:22] } [11:22] fwereade_: "resource" isn't in livesource.go, you just mean in general? [11:23] jam, yeah, the current package is state/watcher [11:23] jam, but Stop and EnsureErr aren't specific to watchers [11:24] fwereade_: RelationUnitsWatcher only has Stop Err and Changes [11:24] jam, oh wait maybe watchers have Stop() methods where workers have Kill() and Wait()? [11:24] jam, gaah [11:24] jam, I think the worker interface is saner [11:25] jam, but, see? these fixes become fractally tentacular if we'renot careful :( [11:25] :) [11:25] yak shaving [11:25] jam, indeed [12:46] jam, (btw: LGTM as is, despite quibbling, progress not perfection) [13:02] * TheMue is out for lunch [13:03] dimitern: updated PR is in, see review board [13:05] TheMue, cheers, will look soon [14:00] * dimitern steps out for ~1h [14:05] why are we getting so many curses? [14:08] perrito666: curses? [14:08] from CI [14:08] i.e. blockers? lol [14:27] any chance I can get a senior reviewer to took at this? http://reviews.vapour.ws/r/926/ [14:32] marcoceppi: looks good to me too, is eric's stamp not okay? [14:32] mgz: I was told to wait for a more sneior reviewer [14:34] marcoceppi: done [14:35] ta [14:36] TheMue: i am not being an incredible fan of http://reviews.vapour.ws/r/947/diff/2 why is it that you are not adding a method to apihostportresults that returns Servers as NetworkHostPorts ? [14:39] perrito666: could be a helper too, yes. so far concentrated on structs or nested slices of hostport [14:41] * TheMue sees our already large params with lots of conversion helpers in the future ... [14:42] TheMue: well borrowing from python :p better explicit [14:42] TheMue: what botters me the most is that I see a lot of params.NetworkHostPorts(someparam.Servers) [14:42] which is a bit noisy [14:43] perrito666: yeah, you get network, want params, get params, want network [14:44] so basically what I mean is, you are using params.NetworkHostPorts to convert a member of params.APIHostPorts :) [14:44] sounds like that should be happening on params :p [14:45] * perrito666 laughs because param is the sound used in spanish for tada [14:47] perrito666: could you point me to file and line [14:48] TheMue: gimme a sec, trying to beat some sense into reviewboard :p [14:48] hehe [14:49] ohh ffs http://reviews.vapour.ws/r/947/diff/# file: api/state.go line 80 [14:49] sometimes I really hate that ui [14:52] perrito666: thx, yes, here it creates the network.HostPorts out of the params.HostPorts [14:53] perrito666: and those are [][].HostPort which are struct, gna [14:54] gna? [14:56] inner sound of "excitement" about this construct [14:56] :D [15:02] wwitzel3: ? [15:06] TheMue: I added the comment in the code [15:07] perrito666: great, thx [15:42] TheMue, hey, did you see my review about the live tests? [15:43] dooferlad, ping [15:43] dimitern: yep, just doing it with local provider. looks good so far. [15:43] TheMue, cool! [15:44] TheMue, I'll double check it here as well [15:44] * dimitern doesn't want more regressions :) [15:44] dimitern: great, thx [15:44] hehe [15:44] dimitern: yeah, and this change already went too large. not in the sense of structure, but changed places. [15:45] TheMue, yeah, but I'd rather get it landed :) [15:47] dooferlad, how goes the backport of bug 1416928 to 1.22? [15:47] Bug #1416928: juju agent using lxcbr0 address as apiaddress instead of juju-br0 breaks agents

[15:55] sinzui, he mailed me a couple of branches with the backports for 1.22 and trunk; I'm checking them now [15:56] oops - s/backports/foreports/ that is [16:01] dimitern, bug 1420049 is now confirmed to be a regression with 1.20. We need someone to address it. [16:01] Bug #1420049: ppc64el - jujud: Syntax error: word unexpected (expecting ")")

[16:01] * sinzui would also ask natefinch to make arrangements, but he might be lost in the snow [16:01] sinzui, I'll have a look [16:05] dimitern: sinzui: Happy to apply the fix to 1.20 as well. [16:06] dooferlad, which fix? [16:07] dimitern: Oh, hang on, I got my bug numbers muddled! Thought both comments were about 1416928 [16:07] dooferlad, yeah :) [16:07] dooferlad, I've pulled your branches with the foreports and once I'm done with live testing TheMue's changes I'll look at yours [16:08] dimitern: turns out tests run better when the heatsink isn't dangling off your northbridge. Top computer stability tip. [16:08] dooferlad, 1.20 is not broken and we aren't release it any more. just 1.22 and master need the love [16:08] dooferlad, LOL :) [16:08] dimitern: are there instructions for live testing, or is it in your head? [16:09] dooferlad, in my head, but I can try to summarize them, good point [16:09] dooferlad, i'll send them by mail [16:09] dimitern: thanks. [16:26] dimitern: just had 1:1 and during this time I've seen an error on EC2. have to look for the reason now. [16:27] dimitern: hmm, maybe already found it [16:28] TheMue, I'm testing on maas first [16:28] dimitern: ah, fine, so different providers [17:26] dimitern: thought I found it but cannot get the reason for my "environment uuid was not supplied" [17:27] TheMue, you can easily test if your changes caused this (but I doubt it) - switch to master, rebuild, bootstrap :) [17:28] dimitern: yes, will check it this way [17:29] dooferlad, please go ahead and propose a PR for the 1.22 foreport [17:29] dimitern: will do [17:29] dooferlad, cheers [17:29] TheMue, live tests on maas pass ok [17:29] TheMue, trying manual now and then ec2 and local [17:29]