[00:46] babbageclunk: got 5-10 minutes to catch up soonish? [00:47] thumper: sure - give me 2 mins? [00:47] ack [00:50] wallyworld: ok, take another look at https://github.com/juju/juju/pull/8138 plz? [00:50] thumper: go for babbageclunk [00:50] babbageclunk: 1:1? [00:51] ja [02:02] * babbageclunk goes for a run [02:18] wallyworld: how do I push a new image into the running k8s without updating the registry? I see Make targets, but I guess I need to configure docker to talk to the remote host somehow? [02:19] ah, local-operator-import, not the other one [02:19] * axw tries [02:35] axw: I don't know if anyone told you, but we're co-opting the tech board today to discuss the interview candidates. [02:36] (I just got my reminder for t-b and I meant to cancel the meeting yesterday) [02:36] jam: yeah wallyworld mentioned this morning, thanks tho [02:42] wallyworld: FYI, with minikube: eval $(minikube docker-env); make operator-image; kubectl set image po/juju-operator-ubuntu juju-operator=juju/caas-jujud-operator:latest [04:20] hey wallyworld: are you happy with https://github.com/juju/juju/pull/8138? [04:20] looking [04:22] babbageclunk: just one small comment fix [04:22] wallyworld: cool thanks! [04:54] axw: here's a refactpromg branch to extract common hook command logic https://github.com/juju/juju/pull/8150 [05:00] wallyworld: ok, will take a look after lunch [05:00] no rush [05:38] wallyworld: not sure why it's showing up as outdated immediately, but I left a comment [05:38] wallyworld: also I got the operator downloading/unpacking teh charm, will send a PR soon [05:38] axw: ah, sorry. i pushed a change just now - thought you were still at lunch [05:39] axw: there's a drive by fix for a windows error, plus i renamed the common/hooks package [05:39] yay for thr charm work! [05:39] wallyworld: yeah I saw the fix, thanks [05:39] rename... [05:40] ah I didn't see that [05:40] that's fine [05:40] axw: ok, i'll rename the hooks/testing package to hookstesting and land. [05:40] wallyworld: thanks [05:41] axw: so next i'll be extracting common runner stuff so that the operator *could* run a hook if it needed to [05:41] wallyworld: sounds good [05:42] axw: but that really needs to land first before we mess with the operator itself to get it to run the first hook i think [05:42] not sure if you wan tto look a tthat bit [05:42] wallyworld: I've still got tests to write for this stuff [05:42] k, sounds good [05:43] oh and SHA256 validation, nearly forgot [05:43] wallyworld: it would be nice if we could just mount the charm in as a volume, but I suppose that's tying too closely to k8s [05:44] also would require a custom volume driver I think [05:44] yeah, i sort of went there but had the same reservation === frankban|afk is now known as frankban [09:06] jam: hiya. do you know what happens to panic output from jujud these days? i just did a kill -QUIT of a jujud instance and i don't see any traceback in /var/log/juju/machine-0.log [09:08] rogpeppe: I've used SIGQUIT not too long ago, I thought it would have ended up in the machine log. So if you're not seeing that, then we probably have introduced an issue. [09:08] jam: yeah, that's what i'm concerned about [09:08] jam: i'm using 2.2.6 [09:11] jam: well /var/lib/juju/init/jujud-machine-0/exec-start.sh *looks* like it should send stderr to /var/log/juju/machine-0.log [09:18] jam: hmm, i think i might see the issue. [09:18] jam: jujud no longer prints its logs to stdout [09:19] jam: so there will be a race between juju's file descriptor and stderr, both going to the same file. [09:19] jam: i'll raise an issue [09:25] jam: https://bugs.launchpad.net/juju/+bug/1735120 [09:25] Bug #1735120: jujud panic output does not appear in log [09:53] wallyworld: https://github.com/juju/juju/pull/8151 [09:53] righto [10:17] axw: i wonder - the unit currently just gets the charm url and not the url plus sha56. that seems like a bit of an ommission [10:48] wallyworld: the uniter? it does get it, it's just buried deep in the bowels [10:48] ok [10:48] worker/uniter/charm/BundlesDir.download [10:49] i haven't looked at that code in a bit - i recalled thr CharmURL() API [10:57] wpk: care to review https://github.com/juju/juju/pull/8152 [11:03] jam: do you have any opinion on https://bugs.launchpad.net/juju/+bug/1734725 ? [11:03] Bug #1734725: add-model sometimes ignores specified region [11:04] jam: and do you know of any workaround for it? [11:06] axw: are you still around? It seems the new leadership test failed under '--race': bug #1735153 [11:06] Bug #1735153: state_leader_test LeadershipSuite.TestCheck has a race condition

[11:06] rogpeppe: I haven't paged it in to have a specific opinion yet, I can try to look [11:07] jam: looking [11:07] axw: there were other failures during --race, one of which is genuine and reproducible and I have a fix for [11:08] I haven't reproduced the failure yet [11:09] wpk: lgtm on your Clock change [11:09] wpk: that's for 'develop', right? [11:09] yes [11:10] well, it's just a test change so it could be backported, but for now freeze is freeze, and it's not critical [11:17] wpk: is it blocking us getting a clean run? [11:17] in which case, its worth a 2.3 target (as my --race fix is) [11:18] jam: it's -really- rare [11:27] wpk: k, then we can not worry about it. [11:28] rogpeppe: I'm unable to reproduce bug #1735120, I get a traceback every time with 2.2.6 [11:28] Bug #1735120: jujud panic output does not appear in log [11:28] jam: you see the traceback in machine-0.log ? [11:29] rogpeppe: yes [11:29] I did a -QUIT about 4 times, and it showed up every time [11:29] jam: i'll try again [11:30] if I -QUIT the exec-start.sh it just gets ignored [11:30] rogpeppe: so make sure you see the "running jujud" line, which means we really did die and start a new process [11:30] also, its certainly possibel that there is a race, but I'm unable to reproduce on LXD or whatever. [11:33] jam: oops, i left jujud stopped, and now i have to work out how to ssh to the localhost machine 0 again... [11:34] rogpeppe: lxc list ? [11:34] jam: ssh -i /home/rog/.local/share/juju/ssh/juju_id_rsa ubuntu@10.0.8.149 [11:39] axw: so the leadership thing doesn't look like a 'drop everything, must fix', just something I came across while digging into different issue with getting a CI bless [11:39] jam: pretty sure I know the issue (bug in test), just verifying now [11:41] jam: ok, so i got a stack trace this time [11:41] jam: but i definitely didn't last time [11:41] I'm out for a bit [12:07] jam: thanks for your reply to https://bugs.launchpad.net/juju/+bug/1735120 [12:07] Bug #1735120: jujud panic output does not appear in log [12:08] jam: FWIW the problem is made worse because there's no way to find out what credentials are stored on the controller [12:45] jam: i have a theory as to what's happening with the stack trace. not sure of a good way to fix it though. https://bugs.launchpad.net/juju/+bug/1735120 [12:45] Bug #1735120: jujud panic output does not appear in log === frankban is now known as frankban|afk [19:07] hml: morning [19:08] thumper: good morning [19:08] hml: I'd like to chat with you about the cloud-init stuff if you have some tome [19:08] time [19:08] thumper: sure - give me a few minutes first [19:10] thumper: ready [19:11] hml: https://hangouts.google.com/hangouts/_/canonical.com/hml-thumper === salmankhan1 is now known as salmankhan [21:09] wallyworld: when are you on? [21:26] thumper: after looking at our cloud-init for a controller machine - i’m rethinking the list we agreed to, new list is postruncmd, preruncmd and users? [22:00] thumper: yep [22:07] hml: I'll go through the doc and comment there, so we have history [22:07] wallyworld: hangout? [22:07] sure [22:08] 1:1 [22:09] thumper: sounds good [22:37] wallyworld: https://github.com/juju/juju/pull/8153 change is +25/-11, so really not big