[01:18] <wallyworld> thumper: https://github.com/juju/juju/pull/11609
[01:31] <tlm> wallyworld: doing the rc1 upgrade I am seeing this in the controller after upgrade
[01:31] <tlm> ERROR juju.worker.dependency engine.go:671 "api-caller" manifold worker returned unexpected error: [bf8dc1] "controller-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused
[01:31] <tlm> any thoughts ?
[01:33] <kelvinliu> wallyworld: tlm ho?
[01:34] <wallyworld> sure
[01:34] <tlm> k
[01:58] <thumper> wallyworld: lint failure
[02:08] <wallyworld> thumper: should be fixed
[02:17] <thumper> wallyworld: going trhough it now
[02:28] <thumper> wallyworld: bug 1878329
[02:28] <mup> Bug #1878329: stuck k8s workload unit following upgrade-charm with new image <k8s> <juju:New for kelvin.liu> <https://launchpad.net/bugs/1878329>
[02:29] <thumper> tlm: did you want to chat?
[02:30] <tlm> thumper: in standup with wallyworld and kelvinliu discussing bugs atm
[02:30] <thumper> tlm: ok
[02:30] <thumper> tlm: I may leave you to work with them on the rc2 issues
[02:30] <tlm> no worries. Enjoy your arvo
[02:30] <thumper> I may give a brief try to see if I can reproduces the caas operater intermittent test failures locally
[02:30] <thumper> I have about 30 minutes
[02:31] <tlm> k, I am banging my head against the wall at the moment as I can do an rc1-rc2 upgrade. Juju doesn't bump the k8's image locally
[02:31] <tlm> seems to work for everyone else
[02:39] <thumper> tlm: running caas operator tests in a loop could reproduce in 400 iterations
[02:39] <thumper> but ran with -race and hit it in 10
[02:40] <thumper> tlm: this line appears in failing test, but not passing test: [LOG] 0:00.087 WARNING test unit running chan["gitlab/0"] is blocked
[02:43] <tlm> hmmmm have you found where that log message comes from thumper ?
[02:44] <thumper> line above says:
[02:44] <thumper> 						// This should never happen unless there is a bug in the uniter.
[02:44] <thumper> caasoperator.go line 527
[02:44] <thumper> can't get it to fail again
[02:48] <thumper> got it to fail eventually, but running stress-race and make check in another window
[02:48] <thumper> stress-ng doesn't seem to tickle it
[02:48] <thumper> I think I'd need to understand all the channels to debug further
[02:48]  * thumper reads code for 10 minutes
[02:49] <tlm> two ticks thumper, just writing a response in launchpad
[02:52] <tlm> free now thumper
[02:54] <thumper> tlm: which hangout?
[02:54] <tlm> standup ?
[02:55] <thumper> ack
[03:42] <wallyworld> kelvinliu: did you see paul has pasted links to the charm on the bug?
[03:43] <kelvinliu> wallyworld: yep
[03:43] <wallyworld> i'm just landing some other stuff, will test soon
[04:52] <tlm> wallyworld, kelvinliu
[04:52] <tlm> whoops
[04:53] <tlm> PR https://github.com/juju/juju/pull/11610 no rush it's more of a look and see if you agree type situation
[05:06] <wallyworld> tlm: i'm always a little wary about creating a buffered channel to solve a race; in this case, I can't see from the code ho we can get events from the containerStartChan  before we start selecting from the channel, so as yet i can't see how this solves the race... is there a causal link between the unbuffered chnnel and the race?
[05:06] <kelvinliu> wallyworld: https://bugs.launchpad.net/juju/+bug/1879598  HO to discuss this issue?
[05:06] <mup> Bug #1879598: terminating pod confuses k8s charm operator <k8s> <juju:Triaged by kelvin.liu> <https://launchpad.net/bugs/1879598>
[05:07] <wallyworld> sure one sec
[05:09] <wallyworld> kelvinliu: free now
[05:10] <kelvinliu> stdup?
[05:12] <wallyworld> yup
[05:36] <tlm> wallyworld: I can try and explain it better over HO if you want
[08:52] <stickupkid_> manadart, CR https://github.com/juju/juju/pull/11608
[08:55] <manadart> stickupkid_: Swap you. https://github.com/juju/juju/pull/11607
[09:31] <manadart> stickupkid_: Got a sec to HO?
[09:31] <stickupkid_> sure
[12:35] <stub> cory_fu: Know what is happening here? I think layer:status is broken with focal (py3.8), but I haven't traced why: https://pastebin.ubuntu.com/p/6bXFzkxB6b/
[12:36] <stickupkid_> manadart, when you get a second, want to talk through the next stages of work on bindings thing
[12:37] <stub> cory_fu: Weirdly, hooks work fine, but actions failing (which use the charm-env shebang)
[12:43] <manadart> stickupkid_: Want to HO now before standup?
[12:44] <stickupkid_> sure
[12:48] <stub> cory_fu: nm, found charms.layer.basic.activate_venv where the docstring explains it
[12:53] <cory_fu> stub: I'm not sure about activate_venv, but I think actions need to explicitly call import_layer_libs: https://github.com/juju-solutions/layer-basic/blob/master/lib/charms/layer/__init__.py#L6
[12:54] <stub> cory_fu: activate_venv does that, and some other twiddling.  'This is handled automatically for normal hooks, but actions might need to invoke this manually,'
[12:55] <cory_fu> Ah, right, I see
[12:56] <cory_fu> charm-env does part of it by ensuring the venv is active, but not the extra twiddling
[12:57] <stub> activate_venv works in any case \o/ . I have no idea when this broke; I haven't rebuilt the charm for ages.
[12:57] <stub> Was thinking it was py3.8 problems
[12:59] <cory_fu> I think it's mostly due to the `from charms import layer` pattern vs `from charms.layer import status`
[15:18] <stickupkid_> manadart, if you're still around https://github.com/juju/juju/pull/11611
[15:19] <stickupkid_> manadart, this lifts up the endpoints binding calls along with AllSpaces call...
[15:19] <stickupkid_> manadart, I'm going to tackle the merging in another PR
[16:25] <manadart> stickupkid_: I will have a proper look in the AM. Have to head off now.
[16:25] <stickupkid_> manadart, wicked, nearly finished myself
[22:25] <hatch> What api facade/method can I use to get a controllers cloud/region ?
[22:36] <thumper> hatch: not sure about that, but show-controller has the details you need
[22:36] <thumper> check what facade it hits
[22:37] <hatch> thanks, I'll check there
[22:44] <hatch> all these side effects.....
[22:57] <hatch> thumper I'm not really sure where `details` is being populated https://github.com/juju/juju/blob/d59b8637e150943cb987a8194addbdc116746b63/cmd/juju/controller/showcontroller.go#L139