=== kadams54 is now known as kadams54-away [00:37] thumper: can i pick your brain? let me know if you have time for a hangout [00:47] how about in 15 min? [00:47] * thumper is just eating [00:53] sure [00:53] * wallyworld reboots [01:00] wallyworld_: https://plus.google.com/hangouts/_/gyff6xf73v7pnxrzkv4lm3t7fqa?hl=en [01:01] Party is over ;( [01:29] do we have a good tool for navigating the history of the git branch? === kadams54 is now known as kadams54-away [01:31] wallyworld_: here is a suspect comment: // Write non-empty contents to the file, otherwise delete it [01:31] so the question becomes, why is the info value empty [01:31] thumper: almost finished standup, will look soon === kadams54-away is now known as kadams54 [01:32] hmm... it seems like it should be in the agent config === kadams54 is now known as kadams54-away [01:49] sinzui: you around? [01:50] I am [01:50] sinzui: thumper has questions about bug 1392745 [01:50] Bug #1392745: juju run doesn't after upgrade to 1.20.11 [01:50] we know 1.19 was broken [01:50] sinzui: oh hai [01:50] so comment #1 we can put don to that [01:51] also... [01:51] the bug targets 1.22 and 1.21-beta3 but the bug talks about 1.20.11 [01:51] yep, CI was 1.18 and 1.20 === kadams54-away is now known as kadams54 [01:51] sinzui: first question, what is machine 6? [01:51] sinzui: is it a state server or not? [01:52] sinzui: are you running in HA or not? [01:52] thumper, CI itself... http://juju-ci.vapour.ws:8080 [01:52] thumper, no HA [01:52] sinzui: is machine 6 a state server? [01:52] or is it just a normal machine? [01:53] how many machines are in the environment? [01:53] sinzui: can I get access to the state server machine? [01:53] thumper, no, not a state-server, that is 0, 6 really is juju-ci's jenkins master. [01:53] sure [01:54] sinzui: isn't there some magic script to import someones launchpad ssh keys to a machine? [01:54] thumper, https://drive.google.com/a/canonical.com/?ddrp=1#search/juju-ci3.jenv will show the jenv [01:55] thumper, sure I can do that if you just want access to the machine, the staging key in cloud-city also gives you that power [01:55] * sinzui places keys [01:57] sinzui: I could just use the juju command to add my ssh key [01:58] sinzui: has there been any change to the github-merge-juju job in the last day? it just failed three times in a row with intermittent test errors... at least two mongo related [01:59] axw, I don't thinks so. mgz, didn't mention he made changes [01:59] okey dokey [01:59] wallyworld_: when adding an ssh key, do I specify the path or the actual key? [01:59] wallyworld_: help isn't clear [02:00] um [02:00] let me look to refresh memory [02:01] the key itself [02:01] or you can import from lp [02:01] juju authorised-keys import lp:thumper [02:02] nm, there already [02:03] haha [02:03] ha [02:03] oops [02:03] * thumper does some digging [02:03] this could be 'orrible [02:04] thumper, I think I put your keys on manually because the above command exited with error 1 [02:04] sinzui: I also put them in through the environment [02:04] :) [02:05] * thumper facepalms [02:07] sinzui: well, I can fix the problem but it doesn't make it clear why it happened in the first place... [02:07] although I do have some ideas [02:07] * thumper continues digging [02:08] sinzui: do you want me to fix the environment? [02:08] thumper, just helping us fix is will let us get on with our lives [02:08] kk === kadams54 is now known as kadams54-away [02:28] sinzui: my attempted fix didn't work :-( [02:29] :( [02:29] * thumper tries something else [02:32] davecheney: thanks for taking a look at http://reviews.vapour.ws/r/485/ [02:33] davecheney: if you like I can roll the rest of removing-backups-dependency-in-state into the patch [02:42] wallyworld__: managed to get into the inside of the mongo database, it says "system-identity" is "" [02:42] \o/ [02:42] that that explains why the file is being removed [02:42] but not why system identity in db is "" [02:43] right [02:43] wallyworld__: did you want to talk this through? [02:43] * thumper needs a teddybear [02:44] thumper: i'm just about to get a pie out of the oven, give me 5 mins to eat it [02:44] kk [02:44] then i'll spoon you [02:45] wallyworld__, thumper I'm guessing that means something different in that hemisphere? [02:45] * jw4 is from South Africa though and spooning meant the same there [02:45] jw4: no... I think he just used the wrong word [02:45] * thumper hopes [02:45] lol [02:52] thumper: am in our 1:1 waiting for a cuddle [02:52] jw4: he meens "huddle" [02:52] * jw4 blushes and tiptoes away [03:06] LOL [03:06] i just looked at the channel after having not looked for a while and oh dear... === Spads_ is now known as Spads [03:13] menn0: I feel safer now that theres someone else here [03:13] :) [03:33] menn0: well, he is very cuddly, you should try it sometime [03:33] wallyworld__,thumper: "huddly"?.. [03:38] anastasiamac: :-D === kadams54-away is now known as kadams54 [03:44] jw4: m relieved this channel is PG-rated ;-) [04:00] anastasiamac: hehe - I'm sure thumper is squirming right now [04:00] nah [04:01] calm but annoyed by minions running around the house [04:01] hehe [04:01] for some reason my minions have been CRAZY the last few days [04:03] thumper: u'll need to teach me someday how u manage to "be calm but annoyed" :-) [04:04] anastasiamac: outer calm [04:05] inner rage [04:05] deep breathing [04:07] wallyworld__: so... about this system identity problem... [04:07] wallyworld__: are you looking after it? [04:07] no, i was hoping you would :-) [04:08] i was delegating :-D [04:08] as i'm stuck on the other one [04:08] sharing the love, john and nate's teams have already fixed stuff for beta3 [04:09] thumper: yep, I know the feeling. JUst got off the phone with the school re: discipline ;-( [04:09] :-( [04:09] thumper: it's less than 2 weeks left! [04:09] :-( [04:10] wallyworld__: ok, I'll take it, and look in the morning [04:10] \o/ ty [04:11] wallyworld__: I expect it will be an upgrade step specifically for 1.20.12 [04:11] wallyworld__: and put back in for 1.21 too [04:11] yeah, cause 1.21 and on run all the other steps [04:12] also, was only going to put the upgrade step in the 1.20 branch for 1.20.12 [04:12] and in the normal 1.21 steps for the 1.21 branch and trunk [04:12] wallyworld__: sound reasonable? [04:12] yep, do it as a 120 step, right [04:12] um... no [04:12] well [04:13] if done as a 120 step, 1.21 and later will run it [04:13] no it won't [04:13] not if on 1.20 already [04:13] you may have upgraded, and just be broken [04:13] so to unbreak, the step needs to reinsert the file into the db [04:14] shit, i guess it becomes an agent startup step [04:14] like ensureMong() [04:14] it needs to create a new system identity file and put it in the authorized keys and update the db [04:15] wallyworld__: no, just an upgrade step [04:16] maybe i'm missing something - if the install is 1.20.11 and broken, having new 1.20.12 won't rerun the 1.20 upgrade stes wil it? [04:16] no, but I can make an upgrade step for 1.20.12 [04:16] oh, i thought we only did 1.20 [04:16] not 1.20.x [04:16] so far we haveā€¦ [04:16] cool [04:16] but that doesn't mean we are limited to it [04:17] rightio [04:59] waigani: here's the NewEnvironment change [04:59] http://reviews.vapour.ws/r/489/diff/ [05:00] waigani: this is what's required to easily allow multiple test environments to be set up === urulama___ is now known as urulama === kadams54 is now known as kadams54-away [06:55] morning all [07:01] morning dimitern :) [07:02] hey jw4 :) === Ursinha_ is now known as Guest58037 [08:35] morning [08:35] jw4: still awake? [08:36] morning TheMue [08:36] Yeah, but off to bed in minutes [08:36] TheMue: :) [08:36] jw4: good, already wondered [08:36] dimitern: o/ [08:50] morning all [10:04] wallyworld__: any more bugs to tackle today ? [10:06] jam: just eating, be with you soon [10:07] wallyworld__: np, I looked at the milestones and everything at least looks assigned [10:11] jam: so bug 1392745, the system identity was moved into the state database from the file system, presumably for HA, but there was no upgrade step written. tim is onto that one. [10:11] Bug #1392745: juju run doesn't after upgrade to 1.20.11 [10:11] and then there's bug 1392602 [10:11] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [10:11] i can't see what the root cause is [10:11] i can reproduce it, but not find the cause to fix it [10:12] i have manually stopped the juju agents, and rsyslog, removed the update conf files, everything is good [10:12] ie the log dir is still there [10:12] then a reboot results in the whole log dir being deleted [10:12] with juju seemingly not involved because it had been stopped and the conf files moved away to avoid upstart stating the agents [10:13] a manual stop and start of the agents (but no reboot) does not reproduce the issue [10:13] it's only a reboot [10:13] i have nfi right now [10:16] is there some diagnostic process that can be run to see what process is removing adir? === fuzzy_ is now known as Fuzai === Guest58037 is now known as Ursinha [11:22] wallyworld: fuser [11:23] i'm trying auditd but can't get the rules to stic [11:23] k [11:26] fuser shows nothing is referencing the file after bootstrap [11:50] dimitern: should ReleaseIPAddress take instance id and net id - or just net id (plus address of course)? [11:50] dimitern: I think only net id is sufficient [11:51] voidspace, you mean Environ.ReleaseAddress? [11:53] voidspace, I think, for consistency, it can take the same args as AllocateAddress (in maas we won't use instId and netId, but for ec2 we will right?) [11:55] dimitern: for maas we'll use netid I think [11:56] dimitern: for ec2 we might *need* instance id [11:56] dimitern: so yeah, probably taking the same is best [11:56] dimitern: ah no, for maas we just need ip address [11:56] voidspace, +1 [11:57] cool [11:57] morning folks :) [12:30] hey, anyone knows what the Total attribute of utils.AttemptStrategy is for? [12:40] Total Warfare [12:40] sorry.... hehe [12:43] oh I get how it works.. mm [13:26] perrito666: it bounds the total length of time of all attempts [13:27] anyone know how local charm revisions are meant to work? [13:27] dimitern: ^ [13:27] rogpeppe: yes, I read the code because it was not all that clear, It is odd that there is no clear way to ask for a given number of attempts instead, the only way seems to be to set any short time and min retries [13:27] perrito666: just specify min retries, but don't specify Total [13:28] rogpeppe, yes, so the revision in the revision file is respected only if there's no duplicate charm with the same revision in state (it's bumped then until unique) [13:28] am I the only one getting breakage on master again? [13:29] I rolled back to go1.2.2 thinking it would make things work better, cleaned my $GOPATH/pkg, ran godeps, and now everything is broken. lol [13:29] throw in a few go get -u -v ./... for good measure [13:29] ... with godeps at the end, of course [13:29] dimitern: so if there's a local charm with no revision file and there's an existing charm in the environment, but no existing charm with revision 0, what should happen? [13:30] bodie_: what breakage are you seeing? [13:30] dimitern: i'm seeing what i think is broken behaviour, but perhaps it's deliberate [13:31] rogpeppe, looks like the same newline breakage we were getting last night when we updated the yaml.v1 version [13:31] rogpeppe, if there's no rev file and no local charm with the same url, revision is set to 1 [13:31] jw4 and I went over that, but it looks like the update changed some of the behaviors expected by many tests in master [13:32] rogpeppe, so, I wasn't able to get the new dep in, perhaps somebody poked it into the deps without checking the tests, or possibly I'm just doing something silly :P [13:32] dimitern: ok, so that will upload to revision 1 in the environment? so won't actually upgrade the charm? [13:32] bodie_: remind me what the issue was again? [13:32] rogpeppe, hmm.. wait a sec - how can you upgrade a charm not yet deployed (i.e. missing from the env and state)? [13:32] rogpeppe, we needed to update yaml.v1 version in order for the new charm content to pass tests [13:33] dimitern: it's already deployed [13:33] rogpeppe, but when updating yaml.v1, many tests in master broke with slight changes to newline behavior [13:33] dimitern: but the local charm doesn't have a revision file [13:33] rogpeppe, so we didn't push the change since it was already past EOD for us [13:33] bodie_: example test output? [13:34] rogpeppe, iirc, the revision file is only used if present, and only if the revision is unique in state; if it's missing it's assumed rev 1 (and if rev 1 exists, rev 2 and so on) [13:34] rogpeppe, can't promise that's what I'm seeing right now, jw4 was the one who was seeing the test breakage, as I wasn't able to roll back to 1.2.2 due to some network issues and was seeing different breakage [13:34] but, it looks like this, as far as I can tell -- [13:35] dimitern: the behaviour i'm seeing is that i run "juju upgrade-charm myservice" and it says "Added charm "local:trusty/myservice-41" to the environment." (that's later than any other revision) but the service isn't actually seeing an upgrade [13:35] rogpeppe, http://paste.ubuntu.com/9072227/ [13:35] bodie_: ok. so those are actually broken tests, i'd say [13:35] rogpeppe, you mean looking at the uniter logs you don't see the upgrade happening? [13:36] dimitern: yup, and the charm source didn't change [13:36] dimitern: (when i looked in /var/lib/juju/agents/unit-myservice-0/charm) [13:36] rogpeppe, weird [13:36] rogpeppe, this is my master with latest godeps, so I'm hoping it's an issue with my rolling back to 1.2.2, but yes [13:37] rogpeppe, sounds like a bug to me [13:37] bodie_: so if we don't change those tests, we won't be able to use the latest version of goyaml [13:37] dimitern: me too :) [13:37] bodie_: but the plus side is that it should be relatively easy to fix them [13:37] yes, rogpeppe, this is on my master with current godeps, however [13:38] unless we ugpraded the yaml version in master already [13:38] bodie_: i'll see if i can replicate [13:39] hoping it's just a failure with my packages [13:50] bodie_: this is such awesome test output http://paste.ubuntu.com/9072392/ [13:50] bodie_: now where's the difference again? :) [13:51] rogpeppe: what, isn't it obvious to you? [13:51] hmmm... [13:51] bodie_: if i update yaml deps to current juju-core version, all tests in that package pass for me [13:52] hmm... okay, good to know [13:55] bodie_: ok, here's the diff: http://paste.ubuntu.com/9072508/ [13:57] rogpeppe, for? deps.tsv? [13:57] I don't know if I'm supposed to know what I'm looking at here, heh [13:57] bodie_: the odd thing is why the obtained value doesn't seem to be formatted with the same goyaml [13:57] bodie_: that's the diff between expected and obtained output from that test output i pasted above [13:58] bodie_: ah! i have a suspicion [13:59] bodie_: hmm, unfounded [14:24] bodie_: ah, i've found the problem, and i'm not sure of the best way of fixing it [14:25] bodie_: in windows_userdata_test.go, it has an expected literal agent configuration [14:25] bodie_: that seems bogus to me [14:27] bodie_: i think that test should be using assertScriptMatch [14:28] bodie_: then it could just leave out the problematic places, i think [14:37] rogpeppe, I apologize, family woke [14:37] bodie_: np [14:42] bodie_: ok, i've got that package passing its tests [14:43] rogpeppe, excellent [14:52] bodie_: i've pushed the branch to github.com/rogpeppe/juju under the 019-fix-environs-cloudinit-for-new-yaml branch [14:52] bodie_: if you could move forward with that, that would be great, as i've already spent too much time on it [14:53] bodie_: ideally that enormous string in windows_userdata_test.go would be honed down to stuff we actually care about. [14:56] rogpeppe, sure thing. I need the charm working to push forward on my work anyway, so this will have to come first either way :) [14:57] but, we are pretty slammed with trying to push actions forward [14:57] I don't imagine this will be a big timesink though [14:59] natefinch: wwitzel3 ericsnow brt, restarting a few things to see if hangouts works better [15:01] bodie_: sorry about that. it is paying off tech debt, at least, though === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 [15:19] natefinch: ericsnow wwitzel3 could you hear any of that? [15:19] perrito666: when are you going to upgrade to that 5800 baud modem? :) [15:19] perrito666: nope [15:19] natefinch: I am saving to buy the serial cable [15:19] lol [15:20] well, apparently chrome decided to take my 4 cores [15:20] in the good side, almost no ram [15:20] except for the 84G virt memory [15:20] that is [15:21] I have no clue what is going on there but I fear I will have to switch to firefox if this keeps going on [15:21] natefinch: btw, I did not see your rant [15:21] perrito666: I'm still writing it. Almost done. [15:21] natefinch: rant early, rant often === kadams54 is now known as kadams54-away [15:25] well that explains it... its is a freaking virtual machine https://en.wikipedia.org/wiki/Google_Native_Client [15:28] dimitern: ping [15:28] dimitern: why do some providers use errors.NotSupportedf for not implemented methods [15:28] dimitern: whereas others use NotImplementedf [15:30] voidspace, well supposedly because some *can* support something (once implemented) while others don't even support that feature [15:30] dimitern: ok [15:30] dimitern: makes sense [15:30] if true in actual usage patterns... [15:31] voidspace, it's not a strictly followed policy :) [15:32] perrito666: rant sent [15:34] dimitern: looks like they're used in local and manual [15:34] anyone remember how I turn off the 10m timelimit for tests? [15:34] dimitern: is it true that local and manual will not support AllocateAddress and ReleaseAddress? [15:34] mgz: -timeout [15:35] mgz: also go help testflag [15:35] voidspace, possibly not, at least not in the near future I think [15:35] natefinch: ta [15:35] dimitern: cool [15:37] dimitern: for the dummy environ, do I just write an "OpReleaseAddress" to the estate.ops ? [15:38] dimitern: or do we need to mimic the behaviour of the real release and undo the previous allocate? [15:39] natefinch: I had to do that very thing (report a bug) and had exactly the same problem as you... [15:39] natefinch: I just picked "unity" as the package name in the end [15:40] voidspace, let's have a OpReleaseAddress - that way we can test it gets called [15:40] voidspace: heh... on my second try, I picked "X.org", just because it was a UI related thing, but then it went to a screen asking what particular thing crashed or whatever, and none of that was right either, so I finally said screw it and wrote that rant instead. [15:40] dimitern: sure [15:41] natefinch: hah [15:41] voidspace, and the implementation should allow us to test it, without expecting too smart behavior of the dummy provider :) [15:41] natefinch: I think its lightdm [15:41] although I have no clue what the package for it its called [15:41] natefinch: mine was a bluetooth/touchpad/multi-touch gesture issue [15:41] fuck-knows where the actual bug is with that [15:41] I shouldn't need to know what the f'ing package name is to report a bug!! [15:42] a three-fingered (or four fingered) gesture causes an event storm and the magic trackpad stops responding [15:42] natefinch: really all it took is a few years of fighting with dms and dpkg -l | grep -i light [15:42] :p [15:51] hi natefinch, could I trouble you to look at a small PR I've got, http://reviews.vapour.ws/r/488/. fixes a go 1.3 map ordering issue bodie_ and I ran into yesterday [15:52] cmars: interestingly, I couldn't make it fail of gccgo [15:53] mgz, that is interesting... does gccgo implement the same nondeterministic map ordering? [15:53] it's different non-determinism [15:54] cmars: why do we care what order the port ranges are returned in? [15:54] "non-deterministic" [15:54] natefinch, so that we can more easily test them [15:54] natefinch, go 1.3 breaks map ordering on purpose I think, so that people won't rely on map ordering in tests for example (like us) [15:55] there is no doubt also another good reason [15:55] natefinch: in my opinion, api returns should be deterministic [15:55] natefinch, I would like your thoughts on this bug. I hope to defer it to another milestone or show that is it a local setup issue. https://bugs.launchpad.net/juju-core/+bug/1392602 [15:55] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [15:55] if we're providing a public method to retrieve some stuff, it's better if you get the same stuff back when nothing has changed, rather than a random shuffle [15:56] rogpeppe, I love your branch naming! lol [15:56] natefinch, that apiserver/firewaller test fails intermittently on go 1.3. easy to reproduce if you repeat it in a loop. the test deep-equals a slice, whose order was coming from map iteration. [15:56] rogpeppe, I am constantly trying to convince people to use expressive branch names :/ [15:57] bodie_: :) [15:58] bodie_: in the old juju-core, my branch numbers had reached 562 [15:59] anyone have time for a go question? [15:59] (^ wording Kapil suggested :) [16:01] ask away, #go-nuts is also a great place if people are too busy here :) [16:01] I've got a https client in go, talking to an https server in go. On first connect, I'd like the client to grab the server's certificate from the connectio nand present the fingerprint to the user for verification. In the server code I can grab the certificate from the http.Request.TLS. but that .TLS doesn't exist on the client end. [16:01] I guess I should probably join there === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 [16:02] But so I'm wondering how I can grab the cert. I.e maybe override the Dial command to intercept it, but I still don't have access to my own structs then to store the result [16:03] ooh, can i get it fro mtr.clientsessionstate? [16:04] from config.ClientSessionCache that is [16:07] mgz, cmars: I guess it makes sense to return the same data each time, but I kinda hate guaranteeing an order, because then people will depend on it, and if we change the sorting mechanism, it'll break people. [16:09] natefinch: I think when we're returning a slice of stuff, having it in a random platform/compiler dependent order is perverse [16:09] dimitern: ping [16:09] dimitern: http://reviews.vapour.ws/r/491/ [16:10] dimitern: I'm pinging you specifically because I'm not happy that the test for the dummy provider implementation of ReleaseAddress is basically a duplicate of the one for AllocateAddress [16:10] dimitern: all that changes is the op type, and in the absence of generics (or reusing the OpAllocateAddress which would be bad) I can't see an obvious way round it [16:10] dimitern: suggestions welcomed! [16:11] mgz: I don't ever assume lists are sorted unless it is very specifically called out as such. That being said, if we decide we want this to be a sorted list, that's fine.... but we need tests around it so we don't break it later. [16:15] xwwt: https://launchpad.net/juju-core/+milestone/1.21-beta3 [16:18] natefinch, I really do need an opinion on bug https://bugs.launchpad.net/juju-core/+bug/1392602 it block the release [16:18] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [16:18] mgz, cmars: I'd like to bring this up on the mailing list, because we should probably be consistent with any API that returns a list of things - either explicitly sorted or not-sorted., [16:19] natefinch: if you fancy a review [16:19] natefinch: http://reviews.vapour.ws/r/491/ [16:19] natefinch: note what I said to dimitern above [16:19] natefinch: I'm not happy that the test for the dummy provider implementation of ReleaseAddress is basically a duplicate of the one for AllocateAddress [16:19] natefinch: all that changes is the op type, and in the absence of generics (or reusing the OpAllocateAddress which would be bad) I can't see an obvious way round it [16:19] natefinch: so suggestions welcomed... [16:21] In other news, it seems Nokia is still a thing... [16:21] voidspace, sorry, was afk for a while, will have a look now [16:21] dimitern: np, thanks [16:21] voidspace: nearly duplicate tests do not bother me === Spads_ is now known as Spads [16:21] natefinch: heh, fair enough [16:21] voidspace: merging two tests that just happen to be very similar *does* bother me [16:21] natefinch: it just seems like there ought to be an elegant solution [16:22] natefinch: they're *identical* other than type [16:23] voidspace: it's really just a coincidence. There's nothing inherently "the same" about allocation or deallocation. They happen to be the same here, but that's not a contractual thing we want to ensure. [16:23] natefinch: in terms of implementation in the dummy provider it's more than coincidence [16:23] natefinch: for both of them it's "write a typed op into the log" [16:24] natefinch: it's not even a coincidence that they have exactly the same parameters... [16:24] natefinch: it's very deliberate [16:24] natefinch: I'm not fighting you on saying it's alright for the tests to be duplicates though [16:24] natefinch: and agreed we may not want to bake in the fact that the parameters are the same [16:24] natefinch: although that would be trivially easy to change of course [16:25] the fact that we have to test the dummy provider at all is appalling to me. [16:25] we're testing our mock... awesome [16:25] natefinch: there's code there [16:25] it needs to be tested [16:25] that's the problem [16:25] you should certainly test your test infrastructure [16:25] anyway [16:25] not your tests themselves of course [16:26] natefinch: I would hope the mock servers in gomaasapi and goamz are tested === Spads_ is now known as Spads [16:26] natefinch: but they're "merely mocks"... [16:26] I would hope you could test the code without a mock server :) [16:26] but, yes, for exporting a mock server for others to test again, that's valid, I suppose [16:27] and dummy provider is essentially that, no? [16:27] I wouldn't wish testing against the dummy provider on my worst enemy, however [16:27] although "others" is us... [16:27] hah [16:27] ok [16:28] sinzui: thanks for pinging me, I'll get someone on it [16:39] voidspace, LGTM [16:39] dimitern: cool, thanks [16:39] dimitern: maas implementation done - just testing it [16:39] not really an achievement - it's about ten lines of code or so... [16:40] voidspace, sweet! :) [16:40] can someone on utopic either confirm bug 1392602 exists or doesn't? [16:40] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [16:40] we need that to see if we have to block beta3 on it [16:41] dimitern: what version of ubuntu are you on atm? [16:42] mgz, trusty [16:43] 14.04.1 [16:43] mgz: I thought wallyworld_ confirmed that this morning [16:43] mgz: he said he could reproduce and was working on it IIRC [16:43] mgz: he just had no idea why the log dir would disappear last I heard [16:44] mgz: it happened even with jujud stopped and the upstart config blown away to prevent it restarting [16:44] mgz: so it didn't look like it was *juju* that was deleting the log dir... [16:44] cmars, anastasiamac, hey, thanks for taking care of that port ranges bug with http://reviews.vapour.ws/r/488/ - sorting LGTM [16:44] mgz: ah, you're asking about utopic specifically [16:44] mgz: I missed that, sorry :-) [16:46] voidspace: he can, but sinzui cannot - we want a third opinion :) [16:47] mgz: hah :-) [16:47] basically, to see if the bug should be bumped or not [16:55] is gc.ErrorMatches *really* a regex match? [16:57] hmmm... it really is [16:57] voidspace: yes.... [16:57] but .* doesn't seem to be doing its job [16:57] voidspace, but with some added nastiness [16:57] http://pastebin.ubuntu.com/9074743/ [16:57] it's a substring match [16:58] is it [16:58] yep, you can just do "failed to release ip address 192.168.2.1" [16:58] it puts ^ and $ around the regext [16:58] natefinch: nope, that failed [16:58] which is why I added .* [16:58] really? dang, sorry, I think it did [16:59] voidspace, if it's a multiline string you're trying to match, add (.|\n)* at the beginning and at the end of the regexp [16:59] dimitern: yuck [16:59] dimitern: probably - it's an annotated error I'm trying to check [16:59] voidspace, yeah, that's one of the ways to get around the auto-appending/prepending of ^ and $ [17:00] dimitern: that worked... [17:00] dimitern: thanks [17:02] dimitern: see what you think of the test here [17:02] http://reviews.vapour.ws/r/492/ [17:02] dimitern: hmm... I could get rid of the mocking indirection, as the test server is sufficient for testing this [17:02] voidspace, there are a few places in the codebase this is used [17:02] voidspace, ok, looking [17:05] voidspace, reviewed [17:06] dimitern: thanks [17:06] voidspace, np [17:06] * dimitern needs to stop now [17:07] dimitern: see you tomorrow [17:07] natefinch: would you mind taking a look at http://reviews.vapour.ws/r/484 [17:08] natefinch: it should be a quick review [17:13] ericsnow: done [17:13] natefinch: thanks === Spads_ is now known as Spads === kadams54 is now known as kadams54-away [17:50] fwereade: does it make sense to worry about bulk calls for the backups API? === kadams54-away is now known as kadams54 [18:00] wwitzel3: did you look at this bug at all? Can't remember if you were looking at it or not: https://bugs.launchpad.net/juju-core/+bug/1392602 [18:00] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [18:10] ericsnow, I would say that in general yes it does, I can absolutely see myself wanting to create backups of all my envs in one shot [18:10] fwereade: ah, got it [18:15] natefinch, wwitzel3 voidspace : I really want to know if this issue affects utopic users. I don't see this affecting trusty users and I don't think this is a juju-core bug https://bugs.launchpad.net/juju-core/+bug/1392602 [18:15] Bug #1392602: local provider agent fails to restart on reboot of host - log dir missing [18:16] natefinch, wwitzel3 voidspace : If we can prove the bug is not in juju-core, I want to release beta3 today [18:19] I don't have utopic I'm afraid [18:20] :-( [18:20] gah, guess I can upgrade [18:20] 8//// [18:20] g'night all [18:20] natefinch: good luck! [18:20] thanks [18:23] natefinch: when you have a minute, could you take a look at http://reviews.vapour.ws/r/493/? [18:23] natefinch: effectively I take state/backups.go and move it to state/backups/storage.go (plus deal with various consequences) [18:29] ericsnow: sorry, gotta try to work on the release blocker... [18:30] ericsnow: unless you want to upgrade to utopic while I look at your review... ;) [18:31] natefinch: no worries [18:35] natefinch, I am not asking anyone to update to utopic. I don't want to ruin someone's desktop. I am hoping someone on the engineering team is on utopic [18:36] sinzui: too late, I've pulled the trigger... [18:36] natefinch, ouch [18:36] if I don't come back, tell my family I love them [18:36] natefinch, I am staying on trusty since it is where juju really needs to run [18:55] perrito666: sorry, I'm going to call it a day [18:55] ericsnow: no worries [18:55] are you ok? [18:56] perrito666: I've got a sudden head cold that's kicking my but [18:56] well go rest then [18:59] [18:59] natefinch: wise words [18:59] well, couple notifications about chrome crash- [18:59] sorry 1 year old on my la [18:59] lap [19:00] anyway, ubuntu thinks chrome crashed a couple times, but my browser windows look ok [19:00] otherwise, no problems so far in the first 2 minutes [19:00] 5 [19:00] natefinch: same happens to me [19:00] I would guess auxiliary chrome apps [19:00] if you ps you will find chrome runs a load of processes [19:00] 3andthis i why I like having an off switch on my keyboard [19:01] one of them a virtual machine [19:01] yeah [19:01] I had to go to firefox even tough I dont like it at all [19:02] I feel kind of dirty runing xul apps [19:04] reminds me of this: https://www.youtube.com/watch?v=lg7MAacSPNM&t=15 [19:05] lool [19:05] havent seen that one in a while [19:06] me either [19:08] (a while <1 year) [19:25] gah, dependencies.tsv is wrong on the 1.21 branch [19:26] where's our logging bot gone for this channel? [19:26] actually, wait, now [19:26] no [19:26] mgz: dunno, where is chanserv? [19:27] nvm, maybe it is here after all... [19:32] sinzui: couldn't repro that bug on my machine using 1.21 on utopic [19:32] natefinch, thank you very much. I am going to defer that bug and release beta3 === kadams54 is now known as kadams54-away [19:38] morning [19:38] party thumper [19:39] sinzui: hey, looked more into the juju-run issue yesterday [19:39] sinzui: we *think* we know what the problem is [19:39] sinzui: but not a quick fix, needs a patch, backport and new version [19:39] hi rick_h_ [19:40] thumper, thank you! I removed the bug from the beta3 milestone since we are certain the regression is older [19:40] sinzui: yeah, I think it came about from 1.18 -> 1.20 upgrade [19:40] I'll write an update in the bug and I agreed with wallyworld_ to take it [19:41] * thumper checks sprint agenda for more surprise hangouts [19:41] thumper: but why ruin the surprise? [19:41] rick_h_: I was trying to rest lying in bed when the hangout request came through from mramm2 [19:42] rick_h_: when I answered and heard sabdfl in the background, I was like "oh bugger, better get up" [19:42] thumper: yea, I got home from bringing the boy home from school and saw an email "we're in this hangout" [19:43] "hmm, wonder what hangout that is...click link...guess I'll see if anyone's still there. Oh crap, that's the boss in front of a whiteboard!" [19:43] heh... yeah === kadams54-away is now known as kadams54 [19:55] natefinch, mgz, do you have a minute to review https://github.com/juju/juju/pull/1183 [19:57] on it [20:22] fwereade: are you around by any chance? [20:23] wallyworld_: what about you? === kadams54 is now known as kadams54-away [20:36] thumper: when you're free can you have a look at http://reviews.vapour.ws/r/489/ pls? [20:36] sure [20:37] menn0: I'm looking now too [20:37] waigani: thanks. that'll make 4 reviewers :) === Spads_ is now known as Spads [20:39] menn0: so this is moving business logic out of initialize into NewEnvironment and then calling NewEnvironment from Initialize [20:43] waigani: that was the original plan but it didn't quite work out that way. [20:44] waigani: Initialize and NewEnvironment do share the code to create and initialize an environment though [20:44] so you've got envSetupOps now [20:45] menn0: ^ why couldn't that go in NewEnvironment ? [20:46] katco: hiya [20:48] because Initialize needs to take those ops and add more [20:48] wallyworld_: did you see my pms? [20:48] they should happen in one trxn [20:48] txn [20:48] waigani: ^ [20:49] waigani: when the state server env is created, the initial user needs to be created and various docs about the state servers need to be created [20:49] waigani: these ops aren't required when setting up further envs [20:51] right [21:02] can I bother someone about an lxc container start issue/local provider file storage? see bug 1393932 [21:02] Bug #1393932: 'container failed to start' with local provider === kadams54-away is now known as kadams54 === kadams54 is now known as kadams54-away [21:18] anyone know why I might have to source my .profile every time I start a terminal to get my prompt stuff set up? [21:21] natefinch: you broke your conf most likely :p [21:21] can't tell you how much I hate the linux profile configuration stuff === Spads_ is now known as Spads [21:28] mgz: it could be that an old container template is being used as tools are no longer in provider storage and the template has the bind mount entry for that still [21:28] i'd delete the lxc template container [21:33] wallyworld_: as in, the template in /var/lib/juju/containers ? we tried deleting the contents of that dir [21:33] or should we try wiping anything else? [21:33] mgz: no, /var/lib/lx c/juju-trusty-template or something like that [21:34] is this something we expect to break for users who do an apt-get upgrade and get a new version of juju? [21:34] it could do [21:35] yeay. [21:35] this is the problem of template containers being stale [21:35] we should do something about this issue though [21:35] we were talking yesterday about whether a big-bang client upgrade from 1.18 to 1.21 would work the other day, if we want to sru back a modern juju [21:36] if i'm correct, if there's containers already running, that will work, but if you destroy env and recreate, it will break [21:36] but [21:37] it needs more investigation [21:37] what i say above is an educated guess about the root cause [21:37] ericsnow: http://reviews.vapour.ws/r/493/diff/ "There was an error displaying this diff" [21:41] is there a consensus on leaving liberal loggo Debugf statements in the code? [21:44] sinzui: not sure what you want to do with bug 1393932 - a lxc template created in 1.20 has invalid config when using 1.21 [21:44] Bug #1393932: 'container failed to start' with local provider [21:45] jw4: if you're going to be really liberal, trace is better.... debug still gets turns on with --debug, but if you want super verbose, one can manually turn on trace [21:45] natefinch: yeah; I went back and converted my most chatty ones to Tracef [21:45] natefinch: thanks [21:47] wallyworld_, I have seen that report [21:47] sinzui: i just sorta confirmed the root cause [21:48] wallyworld_, beta3 is in the cpcs and will be on streams.canonical.com in 30 minutes. [21:48] there's a now obsolete bind mount entry in the template config [21:48] deleting the template container fixes [21:48] cause juju will recrete it === Spads_ is now known as Spads [21:49] maybe that's a workaround we add to the release notes [21:49] maybe juju should put a version in the template config file [21:50] and if a newer juju is incompatible, advise the user to delete the template [21:50] or something [21:51] wallyworld_, to rephrase users should lxc- destroy their juju-*-lxc-templates after upgrading? [21:51] a comment in release notes sounds good to me [21:51] wallyworld_, is this just local, or are lxc containers on other substrates affected? [21:51] sinzui: that's a big hammer approach, but yeah. the image tarball will be cached still in /var/cache/lxc so it's not too bad [21:51] just local i think [21:53] wallyworld_, I disagree. the templates are stale within days of creating them. I purge /var/cache/lxc and juju templates when I start each development effort to be sure my charms run with new images [21:54] wallyworld_, weekly purges with apt update and upgrade off are fastest [21:55] fair enough [21:57] wallyworld_, we already have a section about new local defaults in the 1.20.0 release notes. I will update it with some advice about best practice. [21:57] wallyworld_, so you you like to declare that bug as documentation, not a defect? [21:57] sinzui: it's only in 1.21 that this particular issue occurs [21:58] wallyworld_, I might release 1.21.0 on Friday :) [21:58] it's because we no longer use provider storage for tools [21:58] sinzui: i personally am +1 with doco, bu that's IMHO [21:58] wallyworld_, could you review https://github.com/juju/juju/pull/1183 [21:58] there's an argument we could add an upgrade step [21:59] done [21:59] i hope we can release on friday too [22:01] wallyworld_, since the 1.21.x user is not getting upgrades, the template needs to be recreated regularly, or we explain how to upgrade a template. [22:01] wallyworld_, deletion is fastest. [22:01] wallyworld_, in fact, my own preference for an upgrade step is to remove stale templates [22:02] how do we define stale? [22:02] menn0: I think I see a problem with the new env branch [22:02] menn0: plz don't land it yet [22:02] menn0: just about to talk to fwereade [22:02] menn0: I'll chat with you after [22:03] thumper: no problems. i wasn't going to land it without you having a look. [22:05] wallyworld_, the packages are more than a fortnight stale update settings set 2 weeks as the longest interval. [22:05] menn0: ah...no actually, I think it may be ok, still reading... [22:05] wallyworld_, sorry, that makes no sense [22:05] thumper: ok good :) [22:07] wallyworld_, We have dealt with ppc64el bugs caused by users who went out of their way to disable updates. juju requires current images, and now that updates and upgrades are off by default for local-provider, users need to learn to manage their images [22:07] sinzui: updates should no longer be off by default [22:07] only upgrades [22:08] wallyworld_, maybe you want to review and revise https://docs.google.com/a/canonical.com/document/d/1SDhb5UKpsPL4jwnOhgHMJyNZb0Vj01LHY0mDgWMNU1w/edit [22:08] the docs based on previous release notes says they are off by default [22:09] sinzui: they were off by default at one point, but kapil asked for that be be changed so updates were on, i'll check the state of it all [22:11] sinzui: i also have a plan brewing to hopefully better manage the template staleness issue moving forward [22:11] those notes are 16 pages... [22:12] wallyworld_, a cunning plan? [22:12] sinzui: yes, my lord [22:12] mgz, the release was due 7 weeks ago... [22:13] features kept landing [22:13] sinzui: is is as cunning as a fox what used to be Professor of Cunning at Oxford University but has moved on and is now working for the U.N. at the High Commission of International Cunning Planning [22:13] :)) [22:15] wallyworld_, I am sure your plan cannot possibly fail. [22:15] sinzui: of course not, it is so cunning [22:15] thumper: re your comment about using StateServerEnvironment intead of StateServerInfo [22:15] thumper: I originally did it that way, but all I wanted was the UUID [22:16] thumper: so using StateServerInfo seemed like a more direct way to get it [22:16] thumper: StateServerEnvironment does a bunch of things I don't need at that point [22:16] thumper: what do you think? I don't mind changing it if you think it's better. [22:17] sinzui: it involves a combination of changing the current EnsureCloneTemplate() method, storing lxc images in state blobstore, and redirecting lxc to download images via state server so we get to determine if we should continue to use a cached image or reach out and download a new one [22:17] so we continue to cache for performance, bt can make a decision to update images when needed [22:19] wallyworld_, my that is cunning [22:19] i hope it works, it should do [22:19] i have a prototype [22:19] where the redirection happens, but minus logic to look at staleness [22:20] menn0: I thought about that, and I think that it just makes it a little more easy to understand [22:20] menn0: comments could also help [22:21] menn0: alternatively make a function that returns just that [22:21] and use the new function [22:22] thumper: ok... i'll make this clearer one way or another === kadams54 is now known as kadams54-away [22:44] menn0: if you can kill stateserverinfo, that would be great [22:50] davecheney: i'm not going to kill it in this PR, but I will probably avoid adding another use of it [22:50] menn0: can we have a chat about upgrade order [22:50] thumper: sure [22:50] menn0: just jump in the standup hangout [23:11] sinzui: are we merging the 1.21 branch into master periodically? [23:11] sinzui: and also want to confirm that we aren't merging the 1.20 branch into anything any more [23:12] this fix is horrible [23:12] but I know what to do now [23:12] thumper: not as a matter of course, we've been backporting changes from 1.22 to 1.21 [23:12] we're not upmerging 1.20 [23:12] mgz: ok, I gan go with the back port option [23:12] will land on trunk first, them push into 1.21 [23:15] I have proposed for an edge case failure, it's possible a reviewboard thing will show up for it [23:16] thumper, I don't think we are. jam has done it on occasion. [23:16] thumper, I am sure we are not merging 1.20 branch into master. we are hoping to never commit to it again [23:17] sinzui: to fix juju-run we'll need 1.20.12 [23:19] thumper, really? we cannot count on everyone to upgrade to the latest. in each minor. If 1.21.1 gets the fix, can't I upgrade my 1.20.11 to 1.21.1? [23:19] thumper, many envs went from 1.18.1 to 1.20.5 [23:20] sinzui: ok, what I mean is that if we want the 1.20 release to have a fixed juju-run, we'll need 1.20.12 [23:20] sinzui: if we are fine with it just being fixed in 1.21, then that makes my job easier [23:21] sinzui: AFAICT, juju-run was broken for everyone with the 1.20 branch [23:21] sinzui: for everyone that upgraded to it [23:21] rather than bootstrapped it [23:21] thumper, okay, yes, I agree. juju-ci will support 1.20.x for a while. we want to remove the dual support for old streams soon [23:21] sinzui: so... fix in 1.20 series or not? [23:22] thumper, if it isn't patch hell, yes please. [23:22] ok [23:22] so what I need now is a 1.18 binary so I can bootstrap and test upgrades [23:22] * thumper pokes around [23:23] bugger [23:23] /usr/bin/juju is 1.20.11 [23:23] * thumper goes to get files out of the package by hand [23:23] * thumper goes to make lunch first [23:24] thumper, ci has a cache of what was published in trusty [23:24] * sinzui looks [23:24] sinzui: can we find the amd64 version for 1.18.x where x is the latest? [23:24] yep, Lp will have it too [23:24] * sinzui looks [23:26] thumper, expand 1.18.4 and download the juju-core and juju-local package https://launchpad.net/ubuntu/+source/juju-core