[00:46] <thumper> babbageclunk: got 5-10 minutes to catch up soonish?
[00:47] <babbageclunk> thumper: sure - give me 2 mins?
[00:47] <thumper> ack
[00:50] <babbageclunk> wallyworld: ok, take another look at https://github.com/juju/juju/pull/8138 plz?
[00:50] <babbageclunk> thumper: go for babbageclunk
[00:50] <thumper> babbageclunk: 1:1?
[00:51] <babbageclunk> ja
[02:02]  * babbageclunk goes for a run
[02:18] <axw> wallyworld: how do I push a new image into the running k8s without updating the registry? I see Make targets, but I guess I need to configure docker to talk to the remote host somehow?
[02:19] <axw> ah, local-operator-import, not the other one
[02:19]  * axw tries
[02:35] <jam> axw: I don't know if anyone told you, but we're co-opting the tech board today to discuss the interview candidates.
[02:36] <jam> (I just got my reminder for t-b and I meant to cancel the meeting yesterday)
[02:36] <axw> jam: yeah wallyworld mentioned this morning, thanks tho
[02:42] <axw> wallyworld: FYI, with minikube: eval $(minikube docker-env); make operator-image; kubectl set image po/juju-operator-ubuntu juju-operator=juju/caas-jujud-operator:latest
[04:20] <babbageclunk> hey wallyworld: are you happy with https://github.com/juju/juju/pull/8138?
[04:20] <wallyworld> looking
[04:22] <wallyworld> babbageclunk: just one small comment fix
[04:22] <babbageclunk> wallyworld: cool thanks!
[04:54] <wallyworld> axw: here's a refactpromg branch to extract common hook command logic https://github.com/juju/juju/pull/8150
[05:00] <axw> wallyworld: ok, will take a look after lunch
[05:00] <wallyworld> no rush
[05:38] <axw> wallyworld: not sure why it's showing up as outdated immediately, but I left a comment
[05:38] <axw> wallyworld: also I got the operator downloading/unpacking teh charm, will send a PR soon
[05:38] <wallyworld> axw: ah, sorry. i pushed a change just now - thought you were still at lunch
[05:39] <wallyworld> axw: there's a drive by fix for a windows error, plus i renamed the common/hooks package
[05:39] <wallyworld> yay for thr charm work!
[05:39] <axw> wallyworld: yeah I saw the fix, thanks
[05:39] <axw> rename...
[05:40] <axw> ah I didn't see that
[05:40] <axw> that's fine
[05:40] <wallyworld> axw: ok, i'll rename the hooks/testing package to hookstesting and land.
[05:40] <axw> wallyworld: thanks
[05:41] <wallyworld> axw: so next i'll be extracting common runner stuff so that the operator *could* run a hook if it needed to
[05:41] <axw> wallyworld: sounds good
[05:42] <wallyworld> axw: but that really needs to land first before we mess with the operator itself to get it to run the first hook i think
[05:42] <wallyworld> not sure if you wan tto look a tthat bit
[05:42] <axw> wallyworld: I've still got tests to write for this stuff
[05:42] <wallyworld> k, sounds good
[05:43] <axw> oh and SHA256 validation, nearly forgot
[05:43] <axw> wallyworld: it would be nice if we could just mount the charm in as a volume, but I suppose that's tying too closely to k8s
[05:44] <axw> also would require a custom volume driver I think
[05:44] <wallyworld> yeah, i sort of went there but had the same reservation
[09:06] <rogpeppe> jam: hiya. do you know what happens to panic output from jujud these days? i just did a kill -QUIT of a jujud instance and i don't see any traceback in /var/log/juju/machine-0.log
[09:08] <jam> rogpeppe: I've used SIGQUIT not too long ago, I thought it would have ended up in the machine log. So if you're not seeing that, then we probably have introduced an issue.
[09:08] <rogpeppe> jam: yeah, that's what i'm concerned about
[09:08] <rogpeppe> jam: i'm using 2.2.6
[09:11] <rogpeppe> jam: well /var/lib/juju/init/jujud-machine-0/exec-start.sh *looks* like it should send stderr to /var/log/juju/machine-0.log
[09:18] <rogpeppe> jam: hmm, i think i might see the issue.
[09:18] <rogpeppe> jam: jujud no longer prints its logs to stdout
[09:19] <rogpeppe> jam: so there will be a race between juju's file descriptor and stderr, both going to the same file.
[09:19] <rogpeppe> jam: i'll raise an issue
[09:25] <rogpeppe> jam: https://bugs.launchpad.net/juju/+bug/1735120
[09:25] <mup> Bug #1735120: jujud panic output does not appear in log <juju:New> <https://launchpad.net/bugs/1735120>
[09:53] <axw> wallyworld: https://github.com/juju/juju/pull/8151
[09:53] <wallyworld> righto
[10:17] <wallyworld> axw: i wonder - the unit currently just gets the charm url and not the url plus sha56. that seems like a bit of an ommission
[10:48] <axw> wallyworld: the uniter? it does get it, it's just buried deep in the bowels
[10:48] <wallyworld> ok
[10:48] <axw> worker/uniter/charm/BundlesDir.download
[10:49] <wallyworld> i haven't looked at that code in a bit - i recalled thr CharmURL() API
[10:57] <jam> wpk: care to review https://github.com/juju/juju/pull/8152
[11:03] <rogpeppe> jam: do you have any opinion on https://bugs.launchpad.net/juju/+bug/1734725 ?
[11:03] <mup> Bug #1734725: add-model sometimes ignores specified region <juju:Triaged> <https://launchpad.net/bugs/1734725>
[11:04] <rogpeppe> jam: and do you know of any workaround for it?
[11:06] <jam> axw: are you still around? It seems the new leadership test failed under '--race': bug #1735153
[11:06] <mup> Bug #1735153: state_leader_test LeadershipSuite.TestCheck has a race condition <intermittent-failure> <leadership> <test-failure> <juju:In Progress by jameinel> <https://launchpad.net/bugs/1735153>
[11:06] <jam> rogpeppe: I haven't paged it in to have a specific opinion yet, I can try to look
[11:07] <axw> jam: looking
[11:07] <jam> axw: there were other failures during --race, one of which is genuine and reproducible and I have a fix for
[11:08] <jam> I haven't reproduced the failure yet
[11:09] <jam> wpk: lgtm on your Clock change
[11:09] <jam> wpk: that's for 'develop', right?
[11:09] <wpk> yes
[11:10] <wpk> well, it's just a test change so it could be backported, but for now freeze is freeze, and it's not critical
[11:17] <jam> wpk: is it blocking us getting a clean run?
[11:17] <jam> in which case, its worth a 2.3 target (as my --race fix is)
[11:18] <wpk> jam: it's -really- rare
[11:27] <jam> wpk: k, then we can not worry about it.
[11:28] <jam> rogpeppe: I'm unable to reproduce bug #1735120, I get a traceback every time with 2.2.6
[11:28] <mup> Bug #1735120: jujud panic output does not appear in log <juju:Incomplete> <https://launchpad.net/bugs/1735120>
[11:28] <rogpeppe> jam: you see the traceback in machine-0.log ?
[11:29] <jam> rogpeppe: yes
[11:29] <jam> I did a -QUIT about 4 times, and it showed up every time
[11:29] <rogpeppe> jam: i'll try again
[11:30] <jam> if I -QUIT the exec-start.sh it just gets ignored
[11:30] <jam> rogpeppe: so make sure you see the "running jujud" line, which means we really did die and start a new process
[11:30] <jam> also, its certainly possibel that there is a race, but I'm unable to reproduce on LXD or whatever.
[11:33] <rogpeppe> jam: oops, i left jujud stopped, and now i have to work out how to ssh to the localhost machine 0 again...
[11:34] <jam> rogpeppe: lxc list ?
[11:34] <rogpeppe> jam: ssh -i /home/rog/.local/share/juju/ssh/juju_id_rsa ubuntu@10.0.8.149
[11:39] <jam> axw: so the leadership thing doesn't look like a 'drop everything, must fix', just something I came across while digging into  different issue with getting a CI bless
[11:39] <axw> jam: pretty sure I know the issue (bug in test), just verifying now
[11:41] <rogpeppe> jam: ok, so i got a stack trace this time
[11:41] <rogpeppe> jam: but i definitely didn't last time
[11:41] <jam> I'm out for a bit
[12:07] <rogpeppe> jam: thanks for your reply to https://bugs.launchpad.net/juju/+bug/1735120
[12:07] <mup> Bug #1735120: jujud panic output does not appear in log <juju:Incomplete> <https://launchpad.net/bugs/1735120>
[12:08] <rogpeppe> jam: FWIW the problem is made worse because there's no way to find out what credentials are stored on the controller
[12:45] <rogpeppe> jam: i have a theory as to what's happening with the stack trace. not sure of a good way to fix it though. https://bugs.launchpad.net/juju/+bug/1735120
[12:45] <mup> Bug #1735120: jujud panic output does not appear in log <juju:Incomplete> <https://launchpad.net/bugs/1735120>
[19:07] <thumper> hml: morning
[19:08] <hml> thumper: good morning
[19:08] <thumper> hml: I'd like to chat with you about the cloud-init stuff if you have some tome
[19:08] <thumper> time
[19:08] <hml> thumper: sure - give me a few minutes first
[19:10] <hml> thumper: ready
[19:11] <thumper> hml: https://hangouts.google.com/hangouts/_/canonical.com/hml-thumper
[21:09] <thumper> wallyworld: when are you on?
[21:26] <hml> thumper: after looking at our cloud-init for a controller machine - i’m rethinking the list we agreed to, new list is postruncmd, preruncmd and users?
[22:00] <wallyworld> thumper: yep
[22:07] <thumper> hml: I'll go through the doc and comment there, so we have history
[22:07] <thumper> wallyworld: hangout?
[22:07] <wallyworld> sure
[22:08] <thumper> 1:1
[22:09] <hml> thumper: sounds good
[22:37] <thumper> wallyworld: https://github.com/juju/juju/pull/8153 change is +25/-11, so really not big