[00:08] mwhudson, davechen1y: do we already know about the panic inside Go 1.7 that the worker/uniter tests induce? [00:08] menn0: no0 [00:09] mwhudson, davechen1y : http://paste.ubuntu.com/17312432/ [00:09] mwhudson, davechen1y : doesn't happen under 1.6 [00:09] menn0: exciting [00:09] menn0: that's with golang-1.7 from my ppa? [00:10] mwhudson: it is indeed [00:10] menn0: what command did you run? [00:10] go test -c github.com/juju/juju/worker/uniter ? [00:11] go test ./worker/uniter/... (while in the github/juju/juju directory [00:11] heh i don't have my own ppa installed [00:11] enabled [00:11] whatever [00:11] axw: you where right its --nojournal [00:12] perrito666: cool, hopefully easy to fix then [00:12] well its a commit to testing then another to master [00:12] mwhudson: no -c when I did it... just trying that now [00:12] menn0: well it's clearly the compiler crashing [00:13] just wondering which package it's crashing on [00:13] mwhudson: go test -c for worker/uniter works. [00:13] menn0: hmm [00:13] i'll track down the subpackage [00:13] thank [00:13] s [00:14] i'm installing and updating things and building go tip [00:15] mwhudson: I've got to go for lunch but I have a loop doing go test over each subpackage [00:15] menn0: ok, i'll see if it reproduces for me [00:16] menn0: news to me [00:20] mwhudson, davechen1y: it's worker/uniter/runner/context [00:20] huh [00:20] works for me [00:21] /usr/lib/go-1.7/bin/go test -c ./worker/uniter/runner/context [00:21] thumper: you're not the only one who gets to delete stuff: http://reviews.vapour.ws/r/5049/ [00:21] menn0: do you have golang-tip installed? maybe you could try with that? (after lunch when a new version should have built) [00:21] mwhudson: i'll try with current master [00:21] or that, yeah [00:21] there have been some compiler fixes [00:23] mwhudson: it still happens with master for me [00:24] mwhudson: i'll wait for the new go version and try again [00:24] menn0: hmm [00:24] menn0: what exact command line? [00:24] menn0: also, are you still on wily? [00:24] go test -c ./worker/uniter/runner/context [00:24] (just trying to think up differences...) [00:24] nope on xenial [00:24] ok [00:24] and "which go"? [00:25] $ which go [00:25] $ readlink -f /home/menno/bin/go [00:25] stupid irc [00:25] $ which go [00:25] /home/menno/bin/go [00:25] $ readlink -f /home/menno/bin/go [00:25] /usr/lib/go-1.7/bin/go [00:25] haha [00:25] menn0: reviewed [00:25] ok thanks [00:26] mwhudson: [00:26] $ dpkg -S /usr/lib/go-1.7/bin/go [00:26] golang-1.7-go: /usr/lib/go-1.7/bin/go [00:26] menn0: and it happens every time for you? [00:26] mwhudson: seems to [00:27] mwhudson: it's not stopping me doing what I need to do. just noticed it when running tests. [00:27] mwhudson: this is the package version: 1.7~~devel.201605030049.499cd33+ppa1~ubuntu16.04.1 [00:27] back in a bit [00:27] menn0: ooh [00:28] menn0: that's the old ppa [00:28] menn0: use ppa:gophers/archive instead [01:04] mwhudson: that could be it then. installing the updating package from the newer ppa now [01:07] mwhudson, davechen1y : well that fixed it. nothing to see here. sorry for the noise [01:07] 2np [01:11] Bug #1592210 opened: juju2beta9 agent stuck in allocating state

[01:11] hey anyone seen http://paste.ubuntu.com/17313966/ line 184 before? [01:19] wallyworld: I'm back, just replying to an email and will ping you to talk about openstack/controller-uuid [01:19] axw: ok, i'm just finishing a review, haven't looked too deeply again yet, will ping you [01:20] wallyworld: ok [01:20] stokachu: ah, that was an upstream bug in godbus [01:20] i thought it had been fixed [01:21] maybe it was not fixed properly [01:21] wallyworld: yea i can pretty easily reproduce [01:21] can you file a bug and i'll get someone to look? [01:21] wallyworld: that was the bug 1592210 [01:21] Bug #1592210: juju2beta9 agent stuck in allocating state

[01:21] wallyworld: ^ [01:21] ok, ta [01:21] thanks! we were going crazy trying to figure out why all but 1 service was failing [01:21] natefinch-afk: ^^^^^ can you see if your previous fix maybe needs tweaking? [01:25] wallyworld: i can reproduce it when doing a bunch of single application deploys one right after the other [01:25] wallyworld: not so much when doing just a bundle deploy [01:25] damn ok [01:25] i am not entirely familiar with the upstream issue off hand' [01:26] wallyworld: cool np, just have nate ping me and i can test or answer any other questions [01:26] but i did think it ws fixed, maybe it was done in 1.25 not master, will need to check [01:26] ty [01:26] ok cool [01:38] axw: i can just use the goose client to get all groups and do filtering in the openstack code, so i'll look at that and see how it pans out [02:05] Bug #1592221 opened: restore-backup empty CloudName not valid

[02:06] oh cmoooon [02:17] Bug #1591940 changed: juju failed unmarshal the /server/{server_id} api response body [02:28] thumper: i've had 3 attempts at trying to land a PR now and I keep getting random "unexpected message" errors from mongo [02:28] seems like "unexpected message" is the new "bad record MAC" [02:28] :( [02:28] menn0: linkage [02:28] maybe it's something I broke [02:28] davecheney: http://juju-ci.vapour.ws:8080/job/github-merge-juju/8086/consoleFull [02:29] davecheney: the failures are in areas not touched by the PR [02:29] davecheney: these are into the model-migration branch which a bit behind master [02:29] nope, that's not my fault [02:29] that's just the usual mongo couldn't actaully talk TLS when it said it could [02:29] nonsense [02:30] followed by it segfaulting or just crapping the bed [02:30] so i'm just unlucky then [02:30] * menn0 tries again [02:30] menn0: don't forget the bug that was introduced that means if mongo is not running with /tmp on a ramdisk [02:30] then it will take 20+ seconds for the test to kick off [02:31] that's a mongo 3.2 thing right? [02:31] nope mongo 2.4 [02:31] which is what CI runs [02:31] davecheney: hmm ok. I didn't know about that one [02:31] * menn0 does have tmp on a ramdisk though [02:31] menn0: have you been following hte "ci times increasing at alarming rates" thread ? [02:32] davecheney: yes [02:32] menn0: if you didn't have /tmp on a ramdisk [02:32] then you might be upset about the reduction in lifespan of your ssd :) [02:35] davecheney: yep, I remember seeing that [02:36] that'll be hurting CI as it runs mongo 2.4.9 === natefinch-afk is now known as natefinch [02:41] how come we have enable-ha but no disable-ha? [02:41] just curious [02:41] thumper: I don't think we have code for disable HA [02:41] if I want to go from 3 back to 1, can I? [02:41] does going from 5 -> 3 work? [02:41] thumper: not to my knowledge [02:42] thumper: you can go up, but you can't go down [02:42] why? [02:43] thumper: there's nothing inherently stopping us from doing it, it was just going to be some significant added work, which we were going to get to later.... [02:43] thumper: is it later yet? :) [02:43] heh [02:43] I think I can get what I want by going : [02:43] ensure-ha [02:43] kill machine 2 [02:43] and then ... [02:43] what? [02:44] ensure-ha again? [02:44] davecheney: menn0 we should not have journal enabled in the test for 2.4, that was a bit of a careless change [02:44] I don't know what the commands do in 2.0. in juju1, ensure-availability would start new machines when you did ensure-ha again [02:45] so basically, ensure-ha is "please try to get me into the state I original asked for when I did ensure-availability the first time" [02:45] menn0: https://pastebin.canonical.com/158638/ in github.com/juju/testing will make your test be less sad [02:46] perrito666: ok, but i'm seeing the problems in CI... why was --nojournal removed in the first place? [02:47] menn0: someone did changes to testing to enable mongo3 but droped support for mongo2.4 in the process [02:48] and by support I just mean optimization [02:53] natefinch: looks like you forgot to take out the ubuntu-cloudimg-query stuff [02:53] davecheney, perrito666: the model-migration branch is sufficiently behind master that it doesn't have that change in place [02:53] wallyworld: last I'd heard it was still up in the air.... was going to ask you since I saw it failing. I guess I know what the answer is. [02:54] you're getting mixed up [02:54] so that's not relevant to what I'm seeing [02:54] menn0: thumper https://github.com/juju/juju/pull/5607 [02:54] there's a different between the mechanism to download lxc images and image caching in general [02:54] menn0: you probably have it by accident as it is on the juju/testing repository [02:54] wallyworld, perrito666: but still good to know (and we should fix it) [02:55] davecheney: nope. I made sure. [02:55] natefinch: we need to remove the bits to download and cache lxc images, but not yet the image caching store [02:56] menn0: no idea then [02:56] just mongo be awful then [02:58] wallyworld: I didn't realize there were different parts of it... kinda thought of it all as one feature. So I can drop the image downloading stuff, I take it? [02:59] yep [02:59] davecheney: reviewed [03:00] ta [03:06] davecheney: it just failed - known flaky resumer test [03:06] not my day [03:12] thumper: here's WatchMinionReports: http://reviews.vapour.ws/r/5051/ [03:13] * thumper looks [03:13] menn0: I'm about to run out and take Maia to BJJ [03:13] back soonish [03:13] thumper: no rush [03:13] * menn0 stacks the PRs [03:30] menn0: having the same problem http://juju-ci.vapour.ws:8080/job/github-merge-juju/8088/console [03:30] on master branch [03:31] yeah, it's pretty awful [04:00] wallyworld: I'm probably going to have to tack and do the add-model changes before finishing validation [04:00] wallyworld: otherwise everything except lxd will be broken [04:00] eek, ok [04:00] wallyworld: I'll push what I've got if you want to review what's there [04:00] I don't *think* it will change any more [04:01] i'm stuck fixing tests with this controller uuid change, won't get to look till a little later [04:01] wallyworld: no worries. there's a WIP PR up now, whenever you can [04:01] will do, ty [04:13] natefinch: ping? [04:13] cherylj: yo [04:14] did you land the update for godbus in master? (I think so) [04:14] I see the 1.25 update here: https://github.com/juju/juju/pull/5363 [04:14] cherylj: I didn't [04:14] natefinch: was it because master was blocked? or some other reason? [04:15] cherylj: yes, because master was blocked, and then it fell off my radar [04:15] heh, perfectly understandable [04:16] natefinch: could you merge it as $$fixes-1592210$$ ? [04:16] stokachu: is running into that issue for bug 1592210 [04:16] Bug #1592210: juju2beta9 agent stuck in allocating state

[04:16] yeah, it's an ugly bug, definitely should go in [04:17] thanks!! [04:42] Bug #1583893 changed: 1.25.5: goroutine panic launching container on xenial

[05:05] gah... helps if I actually commit my changes before rerunning CI :/ [05:39] wallyworld: does this sound sensible to you? "juju add-model --credential foo" will automatically upload a credential "foo" from the client to controller if it doesn't exist in the controller already [05:40] i think so - it should print what it does [05:41] wallyworld: yep will add some output [05:41] ok [08:01] frobware: sorry, I'll be right there [08:01] dimitern: 1:1 in 5? Ok? [08:01] frobware: ok, sure [09:03] jam, dooferlad?: standup? [09:15] dimitern: https://github.com/frobware/juju/tree/master-lp1566801-strike2 [09:15] dimitern: requires your change === Spads__ is now known as Spads [09:22] babbageclunk: http://pastebin.ubuntu.com/17320468/ [09:23] dimitern: for i in /usr/bin/apt* ; do sudo ln -s /usr/bin/eatmydata /usr/local/bin/$(basename $i); done === Spads_ is now known as Spads [09:26] dimitern: http://pastebin.ubuntu.com/17320499/ [09:49] wallyworld: I've just put up a new WIP branched off the other one that updates add-model, tested live with lxd and aws [09:56] dimitern: frobware: babbageclunk: ok, I think it's ready for review http://reviews.vapour.ws/r/5019/ [09:56] voidspace: looking [09:56] voidspace: adding to my review TODO... [09:56] thanks [09:57] dimitern: frobware: migration import (and test) is where the fiddly stuff with global keys and references is done [09:58] validation of the imported model still to be added as a separate step - in fact that's next as the network entities have all been done now [09:58] ok, cheers [10:18] dimitern: AA-FF reviewed [10:24] babbageclunk: you're right about that comment being incorrect - it's from a copy/paste + search & replace [10:24] babbageclunk: the filename plurality is consistent with ipaddresses and ipaddress_test - but that's inconsistent with the others [10:24] babbageclunk: so I'll change both [10:24] voidspace: :) [10:44] frobware: thanks! [10:46] dimitern: do you run maas20 in lxd at all? [10:48] frobware: I tried some weeks ago, but couldn't get it to work (mounting loop devs was the issue) [10:48] any juju devs here? [10:49] dimitern: I tried many months ago. IIRC, you have to mknod the devices first [10:50] frobware: i didn't get that far tbo [10:50] tbh even [10:51] admcleod: you're at the right place :) what do you need? [10:52] dimitern: theres a guy in #juju (brunogirin) with a juju 2.0 / go stack trace, and ive hit the extent of my troubleshooting :} [10:53] admcleod: ok, let's have a look then [11:00] dimitern: thanks :) [11:14] Bug #1592366 opened: Juju machines failed to die, with added storage [11:44] Bug #1592365 opened: Juju 2.0 crashes on bootstrap with datacentred.co.uk OpenStack cloud [12:19] dimitern, voidspace, babbageclunk: backport from master to 1.25 -- http://reviews.vapour.ws/r/5054/ [12:28] frobware: looking === Spads_ is now known as Spads === Spads_ is now known as Spads === Spads_ is now known as Spads === Spads_ is now known as Spads [13:19] axw: Hey, are you still the best person to talk to about storage? [13:28] frobware: LGTM on that backport btw [13:28] aisrael: +1 to the right person but he's APAC so zzzz atm [13:28] rick_h_: ack, thanks. I'll send him an email === Spads_ is now known as Spads [13:33] dimitern: ping [13:33] dimitern: can you join the networking tests call? [13:33] frobware: omw [13:57] dimitern, voidspace, babbageclunk, dooferlad: FYI (last comment in): https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1513529 [13:57] Bug #1513529: cloud images should be built with the same /etc/apt/sources.list as server images

[14:01] dimitern: ping - have 10 minutes to talk about the removal of "ifup -a || true"? [14:02] frobware: yeah [14:02] frobware: it wasn't necessary anymore (tests on aws and maas confirmed it) [14:02] dimitern: this was my conclusion too [14:04] frobware: hey, no eth0.cfg in the maas trusty cloud image from dailies [14:05] dimitern: yep, saw that too. I added -proposed and installed curtin-389 this morning [14:05] frobware: and trusty-backports is there [14:05] sweet! [14:05] dimitern: didn't look for that, but \o/ [14:05] dimitern: things are beginning to come together [14:06] frobware: due to that curtin issue, every other trusty deployment fails, but still, once it's done lxd also works [14:06] dimitern: we still have the 50-cloud-init.cfg conflation with LXD containers on MAAS [14:08] frobware: because of that missing cloud-init fix? [14:09] dimitern: we need a variation of: https://github.com/frobware/juju/tree/master-lp1566801-strike2 [14:10] dimitern: one that isn't so intrusive and that will work on KVM too [14:11] dimitern: but with that branch and latest curtin/daily cloud image it should be just peachy! [14:15] frobware: yeah! re kvm - we should see what can be done with virt-copy-in [14:15] dimitern: I really want to look at seeding cloud-into to DWIM & DTRT [14:16] dimitern: because that's a common interface for both LXD and KVM [14:23] frobware: that would be better, indeed === Spads_ is now known as Spads [15:03] gsamfira, bogdanteleaga: ping? [15:04] or anybody from cloudbase? there's a guy in #juju (gennadiy) who asks how to get the password of a windows instance deployed with juju on openstack === Spads_ is now known as Spads [15:21] frobware: still there? [15:22] dimitern, he will be back later [15:22] alexisb: ok [15:29] natefinch: hey, did you see my comment and wallyworld's comments on the LXC-to-LXD? [15:33] Bug #1592456 opened: juju publish hangs

[16:15] cherylj: ping - any concerns for landing stuff into 1.25 (e.g., http://reviews.vapour.ws/r/5054/)? [16:25] frobware: that's fine, thanks :) [18:14] * katco goes to run errand [18:23] jam: just to amke sure... there's also an imagemanager facade.... that's ok to stay around? [18:23] jam: or is that part of what should be deleted? [18:23] natefinch: looking [18:26] natefinch: so I think that should stay. I think that could be adapted to how we cache LXD images as well, so that you can save space, etc. [18:26] jam: ok [18:26] I wouldn't cry if we removed it, because it won't do anything in 2.0 [18:27] but when we do bring something like it back, it would be nice for a 2.0 client to be able to ask a 2.1 server to list/delete images. [18:27] jam: mit means we have a command that doesn't do anything, juju cached-images [18:28] natefinch: so as stated above, in 2.0 it won't do anything, but likely it will in 2.1 and I don't see a reason why it couldn't look like that [18:28] which would mean a 2.0 client could ask a 2.1 server to list/delete cached images. [18:29] *but* if there is something that makes it hard to keep around [18:29] I wouldn't feel terrible removing something that doesn't do anything right now. === alexisb is now known as alexisb-afk [18:55] cmars: https://www.youtube.com/watch?list=PLKfWL8IXgKBte4TfD53pLaHONfSYCX0RH&v=ZZTp8NeGbmk === Spads_ is now known as Spads === alexisb-afk is now known as alexisb [19:46] sinzui: my PR had its job cancelled, but now the bot isn't picking it up - https://github.com/juju/juju/pull/5583 [19:46] mgz: ^ [19:47] natefinch: I will find it it and attempt a rebuild. [19:47] natefinch: the lander didn't call back to the PR so it doesn't know it needs to look again [19:49] natefinch: You job is rebuiling. http://juju-ci.vapour.ws:8080/job/github-merge-juju/8111/console [19:49] sinzui: thanks [19:49] natefinch: once it calls back to the PR you can try again if you need...our you can just rebuild [19:49] sinzui: I don't know how to make it rebuild [19:52] natefinch: from http://juju-ci.vapour.ws:8080/job/github-merge-juju/ I can see "natefinch lxc-to-lxd" among the jobs. Its build 8104. when I follow that link to see the actual build I also see a link to rebuild on the left. [19:52] natefinch: ah, but you also need to be logged in [19:53] sinzui: ahh, ok, I see, thanks. [20:07] Bug #1592537 opened: No test for upgrade-juju upload-tools for non-controller model [20:34] natefinch, looks like there are some issues with building your branch is it clear from the build log? [20:35] alexisb: gah... I had to rebase on top of some changes, and there were conflicts. I'm sure I can figure it out. [20:35] alexisb: I think I lost some changes dimiter made, because I renamed a file [20:36] alexisb: should be a trivial fix [20:36] natefinch, thanks [20:49] alexisb: fixed and resubmitting [20:56] natefinch, thank you [20:57] natefinch, were are we on the deps update for the dbus package? [20:57] that is the last thing for a beta9 release [20:57] do you need me to hand that off to someone else? [21:00] alexisb: it's super trivial, one sec [21:01] alexisb: if someone can just cherry pick this commit and fix the conflcits... 9bdc72a76405b5e247f4561aafbdb4775136afc6 [21:01] alexisb: sorry, I gotta run, the kids are crazy and need dinner. I'll be back on in a few hours, but that change is super trivial [21:01] natefinch, yep figured it was that time :) [21:02] alexisb: doesn't even need to be cherry pick... it's just 2 lines in dependencies .tsv [21:02] thanks for the work today natefinc, I will follow-up with someoen on the deps issue [21:02] alexisb: thanks === natefinch is now known as natefinch-afk [21:08] ericsnow, katco Is this something that one of you can pick up from natefinch-afk quickly? ^^^ [21:08] alexisb: the cherry-picking is easy, but the resolving conflicts always requires context [21:08] relates to bug https://bugs.launchpad.net/juju-core/+bug/1592210 [21:08] Bug #1592210: goroutine panic launching container on xenial

[21:08] alexisb: we are both under tight deadlines per wallyworld; up to him? [21:09] katco is should be a couple like dependancy change [21:09] alexisb: ok, let me get to a stopping point and i'll TAL rq [21:09] i can do it also [21:09] katco, if you are actively work something continue [21:10] wallyworld, I would like to get this in in the next hour if possible as it will allow for a green light for beta9 [21:10] yep, sure, figured as much :-) [21:10] wallyworld, thanks [21:13] alexisb: as per my email, you know we need to land a PR unless we want to ship with add-model broken right? [21:14] ug [21:15] wallyworld, I didnt realize it was broken [21:15] that changes things [21:15] sorry, i should have been more explicit [21:15] where are we with that pr? [21:15] it will land asap this morning [21:15] wallyworld, no I could of looked harder not your fault [21:15] wallyworld, ok that is fine [21:16] we just need to get a CI run started your morning [21:16] so that we can release asap tomorrow [21:16] ok [21:59] wallyworld, thumper I will some of your time post the sts call [21:59] ack [22:07] Bug #1589680 changed: Upgrading to cloud-archive:mitaka breaks lxc creation

[22:07] Bug #1592582 opened: Failing unit tests: TestAddStorageMixDistinctAndNonDistinctErrors [22:07] Bug #1592583 opened: Unit test failures in github.com/juju/juju/cmd/juju/commands [22:27] https://hangouts.google.com/hangouts/_/canonical.com/core-leads-call [22:27] wallyworld, thumper ^^ [22:30]