[00:32] wallyworld, anastasiamac: if I wake up tomorrow with even a cursory pre-review of http://reviews.vapour.ws/r/263/diff/ I will be most grateful [00:32] * fwereade bed right now [00:33] ok, will do [00:33] <3 [00:33] good night :-) [00:34] fwereade: k :-) [00:52] hey folks, what's the quickest you've seen an LXC come up, given its image is already cached? [00:52] bigjools: 32 seconds [00:52] stokachu: precisely? lol [00:52] bigjools: yea ive got a counter [00:53] that's fairly swift, thanks [00:53] thats using ubuntu-cloud template btw [00:53] regular ubuntu is slightly longer [00:53] we're trying to work out if it's a problem when juju uses maas to make LXCs if the DNS entry takes up to a minute to appear [00:53] and with lxc-clone enabled [00:54] bigjools: im actually testing juju+maas 1.7 ill let you know if i see anything like that [00:54] stokachu: well this feature is not present yetr [00:54] ah ok [00:54] we're working on it now, but trying to get a feel for what will be a problem [00:54] we're having to make some compromises [00:54] cool, understood [00:54] https://code.launchpad.net/~gmb/maas/add-ptr-records-bug-1382190/+merge/239493 if you're interested [00:55] nice will take a look [00:56] stokachu: wait, are you working on your Sunday?! [00:56] bigjools: yea :( trying to get our unmanaged installer ready for tuesday's lds release [00:56] so we're testing daily maas 1.7 right now [00:57] stokachu: ah ok. LXCs don't get DNS until that branch lands :) [00:57] good to know, fortunately we're using KVM for our bootstrap node in maas [00:57] so it works well [00:59] stokachu: ok so what would be good to know is if a delay getting the DNS entry up after the LXC appears will be a problem in practice [01:00] bigjools: ok cool, i subscribed the team to that bug and added a note on our tasklist to keep an eye on that [01:00] great thanks [01:01] np [01:46] axw, wallyworld: isLoopback is great idea but it does not work for explicit "localhost"... [01:46] anastasiamac: no, you'd still need a string comparison for that [01:46] you're just cutting out the comparison to 127.0.0.1 and ::1 [01:47] axw: k... m with u :-) [01:54] axw: it looks like it does not cater for ip where hostport is only ::1 either... [01:54] axw: to be fair, I suspect that it's url.parse that does not cater for this case... [01:55] axw: just how it does not cater for malformed urls... [01:56] anastasiamac: ah, no, you're right, SplitHostPort expects there to be a port... in that case if you get an error you could tack on a ":0" and try again. starting to get a little convoluted... [01:58] axw: hmm... do u think that mayb regular experession could work neater? it will abbrv8 the if- and is easier to modify to cater for "funny" cases... [02:01] anastasiamac: actually, is the port even optional? I'm not sure there is a default port for apt proxies? [02:03] axw: from apt.conf man page... " http [02:03] HTTP URIs; http::Proxy is the default http proxy to use. It is in [02:03] the standard form of http://[[user][:pass]@]host[:port]/. Per host [02:03] proxies can also be specified by using the form http::Proxy:: [02:03] with the special keyword DIRECT meaning to use no proxies. If no [02:03] one of the above settings is specified, http_proxy environment [02:03] variable will be used." [02:03] ok [02:03] axw: it looks like port could be optional... unless square brakcets have other meaninng... [02:03] nope, that would mean optiona [02:03] l [02:04] axw: ;-( [02:04] i'll do a regexp for now... we can alsways neat it up once url.parse and net.IP.isLoopback cater for every possibility... :-) [02:05] okey dokey [02:05] axw: i do prefer to call on framework rathern than rely on regexp [02:05] axw: thnx for pointing me in the right (i.e. proper) direction! :-) [02:05] nps [02:15] folks, when juju destroys an environment, does it keep issuing commands to destroy individual machines if it later notices that one is not in the state it expected (ie getting destroyed)? This might be provider-specific, in which case I'm talking about maas. [02:16] axw: you worked on the maas provider recently :) --^ [02:17] bigjools: I'll check what it does [02:17] cheers [02:17] I think it issues a single release call [02:18] bigjools: http://pad.lv/1319016 [02:18] it does a single release API call [02:18] bigjools: why do you ask? [02:18] axw: because https://bugs.launchpad.net/maas/+bug/1381619 [02:18] Bug #1381619: Failed to destroy-environment when node is in commissioning or new state

[02:19] axw: ignore the bug title, look at what greg posted [02:19] I'll need to explain some background perhaps: [02:19] we added some more states. It used to go from ALLOCATED straight to READY as soon as an API call was issued to release the node, however now it goes via a RELEASING state [02:20] but because this broke some API users, we folded RELEASING into ALLOCATED so that API users never see it [02:20] I wondered if juju was issuing a release call, and still seeing ALLOCATED when in reality it's RELEASING, so juju issues another release? [02:21] bigjools: ah ok. nope, we don't interpret the error or retry at the moment [02:21] oh ,weird then [02:21] good to know for the bug I mentioned tho... [02:21] yeah [02:21] I'll paste that in there [02:22] axw: how does it know when the env is destroyed? Or does it just fire and forget? [02:23] bigjools: it'll do do a release of all nodes, and check if it passed/failed based on HTTP status code only [02:23] ok [02:23] hmmm [02:23] that bug makes no sense then, unless people were mannually titting about with things first [02:24] bigjools: hmm. actually, Bootstrap will do a release of the bootstrap node if it manages to acquire it and then something goes wrong [02:24] then when the env is destroyed, it'll try to release it again [02:25] ah [02:25] I *think* anyway [02:25] axw: so releasing twice? [02:25] why did the instance stop showing up tho? i.e. why "ERROR bootstrap failed: refreshing addresses: no instances found [02:25] Stopping instance..." [02:25] NFI what was going on there [02:26] it's a shame people don't report what actions they were taking, and just post log snippets [02:27] bigjools: actually no, that error message greg pasted is from the first attempt to release [02:28] there may or may not be a second attempt, depending on what MAAS returns to Juju when it tries to list nodes with agent-name later, but I don't think it's relevant [02:28] axw: looking at the event log, it did fail while coming up [02:28] well [02:29] something caused bootstrap to release the node while it was deploying [02:29] (the second attempt would occur after that error is listed) [02:31] I asked greg to file another bug with more info [03:28] wallyworld: can you please take a look at http://reviews.vapour.ws/r/137/diff/2-3/ [03:28] sure, justing finishing a review, will look real soon [03:28] there's a drive by fix to ssh/run_test.go, which failed when I tried to land my change once [03:28] yep no rush [03:29] the main change is that I've made the metadata commands use environs.New, rather than environs.Prepare; i.e. they require an existing env now [03:47] axw: i think the changes are ok - any env should already be existing when using those metadata commands [03:48] wallyworld: thanks. utils/ssh is used by testing, so can't use the existing ShortWait/LongWait [03:48] ah ok [03:50] * wallyworld -> get more coffee, it's an emergency [04:11] wallyworld: I've removed myself from that credential bug and moved it back to the planning lane. azure and joyent still need doing. [04:31] axw: ok, np [08:08] morning everyone [08:42] axw: if you get a chance before your EOD, cloud you look at http://reviews.vapour.ws/r/264/ ? it's a large number of file but 99% of the changes are mechanical - the tools stream needs to be added as a parameter to many test methods [08:43] wallyworld: ok [08:43] ty, if you don't get to it, that;s fine [08:44] nah I need a break from storage, will look now [09:04] morning all [09:09] o/ morning core devs [09:27] axw: thanks for review :-) i prefer to use string literals in tests to cause breakage when things change, so that tests are thought about [09:28] morning voidspace [09:29] voidspace: did you see: https://github.com/juju/juju/pull/962 wallyworld wants to cache your lookup, which I think we were talking about doing a different way [09:30] ah, read it wrong, you proposed it :) [09:30] the email comes as "no-reply" and then is responded by "ian" so the mail preview always gives me the wrong impression [09:30] * wallyworld doesn't want to cache the lookup - the pr implements a cache and i had a query about it [09:31] wallyworld: we can't change vpc at all for the lifetime of an environment [09:31] wallyworld: ok [09:31] wallyworld: instances in 1 vpc can't communicate via private network to instances in another vpc [09:31] wallyworld: so the "lifetime of juju process" is a misnomer [09:31] it is "lifetime of a juju environment" [09:31] jam: ok, thanks. the pr said the vpc *could* change [09:31] it was unclear if that would be in the lifetime of an env or not [09:31] wallyworld: first thoughts were that it could, further thinking realized "no we can't actually support that" [09:32] ok, np. i think then my only concern (from memory) is that a muctex should be used [09:32] mutex [09:34] yeah, my mistake in the description [09:34] the vpc changing would be "bad" for an environment - but it isn't likely to happen during the lifetime of a process anyway [09:34] * TheMue wonders about the Google Calendar. It shows the correct time now after switching back from DST, but the alerts are for a false time?!?!?! [09:35] wallyworld: I'll add the mutex [09:35] voidspace: ok, thanks, that then conforms to existing practice [09:35] wallyworld: that's in code already committed - it was just missed in the review of that code [09:35] easy to add it now [09:36] voidspace: np. i also had a question about one of the tests [09:36] wallyworld: yes, I just answered that [09:36] wallyworld: the test is the same because I want to *prove* that cached value is used [09:36] wallyworld: so even after changing what the api would return I expect to get the previous value back [09:36] ah, let me take another look [09:36] wallyworld: proving that a second api call wasn't made [09:36] wallyworld: that's the intent [09:40] voidspace: i may be dumb, but from what i can tell, the code could still make an api call and return those result values and make the test pass. the more correct approach would be to mock out the api call and put in an assert that it is not called [09:42] TheMue: you didn't get your reminder for our 1:1? I didn't see you around, I should have pinged, I guess [09:42] wallyworld: if it made an api call it would get the opposite result [09:42] wallyworld: if you remove the caching and run the test it fails [09:42] wallyworld: I did check that [09:42] wallyworld: also, with the mutex [09:43] voidspace: ok, then i was being dumb :-) [09:43] jam: I got it, for in 18 minutes. And later I thought that it would conflict with our standup. [09:43] wallyworld: there is a mutex in the call to e.ec2() [09:43] wallyworld: the common pattern in ec2.go where e.ec2() is used seems to be *not* to use an extra mutex [09:43] jam: but no I got the alert for the standup, also one hour too late. [09:43] voidspace: ok, np. i missed that nuance [09:44] TheMue: are these email alarms or popup ones? [09:44] jam: email [09:44] wallyworld: the mutex in use seems to be a config mutex - preventing the ec2 configuration changing whilst we're building the ec2 instance [09:44] although it doesn't prevent it changing whilst we're using it :-/ [09:45] voidspace: yeah, that was my worry [09:45] voidspace: anyway, that's a separate issue it seems, so i marked it lgtm [09:45] wallyworld: locking every use would serialise all our api calls though [09:45] wallyworld: awesome, thanks [09:45] np, thanks for putting up with my questions [09:45] heh :-) [09:45] jam: but simply won't care for them in the future now as I now it :D and will be at standup at the correct time [09:46] wallyworld: I'll change the PR description before merging, so the comment about default-vpc is correct [09:46] awesome, ty [09:55] wallyworld, thanks for the review [09:55] wallyworld, lots of that stuff is preexisting, but now someone's complained about them I can muster up the emotinal strength to work on some of them a bit ;p [09:55] fwereade: np, it looks great but as i said, i am unfamiliar with the finer detail [09:56] wallyworld, it needs more steps before it's really approaching properly sane tbh [09:56] fwereade: yeah, i figured as much, hard to tell what was new vs moved [09:56] wallyworld, that's the trouble [09:56] eat an elephant one bite at a time [09:57] wallyworld, exactly [09:57] wallyworld, and I've only really been going on this path for a week and it's already feeling noticeably better [09:57] yep :-) [09:57] wallyworld, actions needs a bunch of work too [09:57] if it all works, i reckon commit what you've done so far [09:57] so it doesn't bit rot [09:57] wallyworld, well, it's worth another pass to fix your suggestions [09:57] ok [09:57] they were not mandatory [09:58] if the code was existing [09:58] but would be nice to see them fixed sometime [09:58] wallyworld, yeah, I reserve the right to ignore the ones that turn out to be rabbit holes [09:58] wallyworld, at least for this branch [09:58] yep, np there [09:59] TheMue: standup ? [09:59] wallyworld, but I am very firmly trying to steer a happy course between fix-only-what's-needed and FIX-ALL-THE-THINGS, cos the former leads where we are today and the latter stops me doing anything directly [09:59] fwereade: +100 [10:00] jam: coming, my clock say :59 ;) === ashipika is now known as ashipika_afk [10:28] wallyworld, just pushed a couple of updates, would you give it a quick once-over? I didn't fix the yucky docstring because whenever I start trying to edit that file I start pulling on loose threads everywhere and get sucked in [10:28] sure [10:37] fwereade: changes look sound [10:37] wallyworld, awesome [10:38] wallyworld, is that a ship-it, then? :) [10:41] fwereade: if you are happy, yeah :-) as i said, take my +1 with a little skepticism :-) [10:41] but i like the direction it is heading very much [10:42] wallyworld, cool [10:42] wallyworld, I'm making a big effort to avoid changing functionality [10:43] wallyworld, not that it always works, I just realised that I broke relation settings caching in one of the previous branches, so that's th enext one before I can fix actions [10:43] yep, tests will tell us :-) [11:06] something in 1.21alpha2 seems to have broken the allwatcher [11:06] its no longer reporting all the services in an environment [11:15] morning [11:15] morning [11:16] tasdomas, as OCR could you take a look at this http://reviews.vapour.ws/r/265/? [11:16] * perrito666 tries to convince his ubuntu that it can upgrade to a new version [11:18] jam: so I haven't yet found the neutron api for assigning a private p address [11:19] jam: one example I saw created a port to reserve the address [11:20] http://blog.felipe-alfaro.com/2014/05/09/fixed-ip-addresses-with-openstack-neutron-for-tenant-networks/ [11:21] I'm not sure a port is quite what we want [11:21] https://wiki.openstack.org/wiki/Neutron/APIv2-specification#Create_Port [11:21] The APIv2 spec doesn't seem to have an alternative api though [11:21] I'll look and see if dimiter has written about this in the networking spec [11:26] voidspace: I'm back in our 1:1 hangout [11:26] jam: ok [11:27] I found "UpdatePort" which I think is what we want [11:27] jam: except the addresses you provide overwrite the existing ones [11:27] voidspace: yeah [11:27] jam: so effectively that would be manual management [11:27] I don't see a way to "request a new one" [11:27] voidspace: I'm seriously considering manual management [11:28] voidspace: as that gives us a token [11:28] "this IP address that I'm assigning is going to go to this LXC" [11:29] I would really appreciate a review on this http://reviews.vapour.ws/r/238/ looks big but its not [11:29] mattyw, reviewed your PR - looks good [11:40] tasdomas, thanks very much [12:08] Hmm, did something changed on runTransaction()? My so far working migration now aborts since I merged master. [12:08] Sadly it doesn't says why it is aborting. [12:16] perrito666, LGTM with trivials [12:16] perrito666, good change, thanks [12:20] fwereade: tx for the review [12:20] jam, can I send you into #juju-gui to have a talk with frankban about something networky-looking please? [12:20] * perrito666 visists a friend company for coworking for the day and he gets his space bar fixed [12:21] jam, no public networks from the AllWatcher: http://pastebin.ubuntu.com/8703311/ (although I see a localhost with public scope..?) [12:23] fwereade: yeah, but no ipv4 address with public scope [12:24] frankban, indeed, but localhost doesn't helpmost people very much ;p [12:24] fwereade: yeah, and this breaks local quickstart usage [12:25] frankban, I'm sorry I can't give you my full attention right now -- dimitern may be on a swap day? otherwise jam or TheMue are most likely to have something relevant to say about networking [12:25] frankban, it sounds like a critical bug regardless, please go ahead and report it [12:27] fwereade: this seems to be part of a more general problem with the mega-watchers: hazmat filed bug 1386143 [12:27] Bug #1386143: 1.21 alpha 2 broke watch api, no longer reports all services

[12:28] frankban, ok, cool, would you add it to that bug then (assuming status gives you what you want, but the watcher is broken?) [12:29] fwereade: sure, and thanks [12:29] waigani, heads up: ^^ I think you/menn0/thumper have been in state lately? [12:32] fwereade: morning; were you saying you were going to do a bunch of refactoring in actions? [12:33] jw4, have you been seeing the changes going by in uniter? [12:33] jw4, it's an extension of that really -- trying to move context/runner related stuff out of uniter [12:34] fwereade, ah, you mean the hook context stuff? I haven't caught up the last 24 hours yet :) [12:34] fwereade: sure I remember now [12:34] jw4, in particular, I think the validation stuff needs to move into the context package [12:34] fwereade: kk [12:34] jw4, and the RunActions stuff needs at least *some* explicit testing beyond "RunAction fails on a non-action context" [12:34] jw4, but, I'm getting there [12:35] jw4, I'm 90% focused on structural stuff and testability [12:35] fwereade: you're on a tear... did you rest this weekend at all? [12:35] jw4, I just managed to ignore everything else and focus on code for the week :) [12:35] jw4, the weekend was mostly relaxed [12:35] fwereade: lol... good [12:36] jw4, once I've got a load of the uniter stuff out of the way I should be able to rework the modes into an explicit hook queue [12:36] jw4, it's what I "should" be doing right now but I need to clear the underbrush first iyswim [12:36] fwereade: yep, yep [13:14] frankban: is this on EC2? And do you have default-vpc available there? [13:14] 172.* looks suspiciously like the cloud-local addresses EC2 likes to give out [13:15] ah, I guess this was a local environ [13:15] so not ec2 [13:16] voidspace: since you're on at this time, do you have a chance to look at the networking part of bug #1386143 ? [13:16] Bug #1386143: 1.21 alpha 2 broke watch api, no longer reports all services

[13:25] jam: the addresses problem is on a local env, yes [13:25] frankban: so I think the issue is that the 10.* address *is* the public address on local [13:25] jam: exactly, and it was previously [13:41] jam: sure === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1386204 [13:49] natefinch, jam, can you find someone to fix a test that fails when we increment to alpha3? bug 1386204 [13:49] Bug #1386204: metadataSuite.TestAsJSONBuffer fails when version is alpha3

[13:51] eric wrote the test, but should be pretty simple for anyone to fix up [13:51] voidspace: so I'm guessing how we determine public vs private cahnged, [13:52] jam: ok [13:52] *looks* like it should be easy... [13:57] a backup test [13:58] jam: ah, I thought you were referring to 1386204 [13:58] sinzui: jam: 1386204 we can fix by formatting the current version into the test data [13:59] currently it's hardcoded and will fail every time the version changes [13:59] sinzui: jam: I'll propose a branch for that [14:00] thank you voidspace [14:01] voidspace, you might want to merge https://github.com/juju/juju/pull/970 to get the version change too. [14:04] natefinch: ericsnow wwitzel3 getting there [14:08] sinzui: I have a branch that works with trunk (trivial) and also passes with the version changed manually [14:08] sinzui: I'll just propose it separately [14:09] thank you voidspace [14:13] https://github.com/juju/juju/pull/971/ [14:13] There's no reviewboard url yet [14:14] natefinch: care for a (hopefully) easy review? [14:14] voidspace: seems fine to me [14:14] mgz: cool, will this merge or do I need to force? [14:16] * voidspace is grabbing coffee [14:17] it is the blocking bug, so you just need to mark as fixes-NNN [14:28] ericsnow: natefinch wwitzel3 the call got unusable === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1386204 1385289 [14:37] natefinch, jam I have escalated Bug #1385289 because the last 8 devel revisions tested failed because upgrades consistently timeout on all providers...including stable and devel maas [14:37] Bug #1385289: local storage migration is very slow

[14:38] perrito666: np [14:44] ericsnow: I didn't see a reviewboard review created for this PR [14:44] ericsnow: https://github.com/juju/juju/pull/971 [14:44] voidspace: checking [14:46] voidspace: looks like github is rate-limiting the requests [14:47] voidspace: I'll see if I can work around that better (may not happen immediately) [14:47] ericsnow: does it just drop them you think? [14:47] ericsnow: ok, np [14:48] voidspace: I'm thinking it drops requests ("API rate limit exceeded ...") [14:48] voidspace: I should be able to work around that though [14:49] voidspace: did it not create a review request at all? [14:49] ericsnow: as far as I can tell not [14:49] sinzui: how can tell which revision was used for the 1.21alpha2 release? [14:49] sinzui: looking for the version number change in the commit history? [14:49] voidspace: bummer (I would have expected it to make one) [14:50] voidspace: in the meantime you can use rbt post [14:50] ericsnow: no need on this occasion, but thanks [14:50] voidspace: k [14:50] voidspace, https://github.com/juju/juju/releases [14:51] voidspace, reports.vapour.ws will one day republish that ^ info [14:51] sinzui: ah, cool - thanks [14:52] I checked my tags in my local clone, but I don't have them all [14:52] maybe this is a git thing [14:52] I have the major release tags, but not the alphas [14:52] well, minor release tags [14:53] ah [14:53] git pull upstream master --tags [14:53] now I have them [14:55] sinzui: my fix for issue 1386204 landed [14:55] sinzui: I've marked it as fix committed [14:55] voidspace, rock [15:34] Gna, first needed to find why PR failed (overlapping change regarding DocID) and now the fixes block it. :) [15:47] Is anyone else looking at bug 1386143 [15:47] https://bugs.launchpad.net/juju-core/+bug/1386143 [15:47] Bug #1386143: 1.21 alpha 2 broke watch api, no longer reports all services

[15:50] frankban: ping [15:58] now do I get the megawatcher output? [15:59] Bug report above states "Probably a side effect of this regression: on local environments, the mega-watcher for machines no longer reports the ipv4 address in the public scope" [16:02] voidspace: pong [16:02] frankban: hey [16:02] frankban: see above [16:02] frankban: I'd like repro instructions for getting the mega-watcher log [16:03] frankban: as you reported for bug 1386143 [16:03] Bug #1386143: 1.21 alpha 2 broke watch api, no longer reports all services

[16:03] voidspace: ok [16:05] voidspace: do you have quickstart installed? otherwise I can quickly write a script to see the mega-watcher output [16:06] frankban: I don't have it installed no [16:06] frankban: I'm happy to install it if it helps [16:06] is it just from juju-plugins? [16:07] voidspace: no, ok I am adding dupe instraction re quickstart to the bug [16:07] frankban: cool, thanks [16:16] Ok, I now have juju-quickstart installed [16:19] frankban: so, running juju-quickstart dies with: [16:19] http://pastebin.ubuntu.com/8706138/ [16:19] frankban: is this the error you're getting? [16:20] I'd still like to be able to see the megawatcher output directly [16:20] voidspace: no, this is a very old version of quickstart [16:20] frankban: hah, ok [16:20] I need to add the right ppa then [16:20] I assumed I had it already [16:20]