[00:05] <wallyworld> kelvinliu_: forgot to ask, this PR can be closed right? https://github.com/juju/juju/pull/8936
[00:08] <kelvinliu_> wallyworld, yeah, i just closed this for now. we can solve it later.
[00:13] <wallyworld> kelvinliu_: can you make sure there's a card on the caas trllo board for it so we don't forget
[00:16] <kelvinliu_> wallyworld, sure, done.
[00:16] <wallyworld> ty
[00:19] <kelvinliu_> np
[01:54] <thumper> bollocks
[01:54] <veebers> wallyworld: I'm I going crazy, using a fresh develop build I deploy a caas charm, juju status never changes the workload or message from active/Started Container
[01:54] <thumper> intermittent allwatcher test failures
[01:54] <veebers> I thought my changes introduced that hence why trying develop. I built a fresh operator image and set that too incase there was some caching or something
[01:55] <thumper> I think we have had them for a while,
[01:55]  * thumper disd
[01:55] <thumper> digs
[01:55] <wallyworld> veebers: i don't quite follow. status should not start out as active
[01:57] <veebers> wallyworld: sorry, this is status after ages: https://pastebin.canonical.com/p/3p5yYBFnKf/ I see with kubectl -n failing-message log -f juju-operator-mariadb that everything is happy etc. and the unit is fine
[01:57] <veebers> I assumed that status should have been updated right?
[01:58] <wallyworld> veebers: sorry, what's wrong with status? it looks correct? what are you expecting to see?
[01:59] <veebers> wallyworld: ah shit,  you're right; sorry I was thinking that 'started container' was the first message and something else should replace that.
[01:59] <veebers> wallyworld: I blame Friday + lack of sleep
[01:59] <veebers> I'll continue on my way ^_^
[01:59] <wallyworld> no worries. not that the charm is broken a bit
[02:00] <wallyworld> it will sometimes send the wrong status after a restart
[02:00] <wallyworld> *note
[02:01] <veebers> ack
[02:27] <anastasiamac> babbageclunk: \o/
[02:28] <babbageclunk> anastasiamac: hey hey
[02:28] <anastasiamac> just celebrating ur PR's merge...
[02:29] <babbageclunk> anastasiamac: oh yay!
[02:29]  * babbageclunk dances
[02:49] <thumper> i need a teddybear... babbageclunk your soft...
[02:51]  * babbageclunk sighs
[02:51] <babbageclunk> ok!
[02:51] <babbageclunk> in 1:1
[03:49] <veebers> ah man, wallyworld I'm all of sudden seeing this error, any thoughts on debugging it? (I use make operator-image and docker push to publish it: Failed to pull image "veebers/caas-operator@sha256:fc83ad5cbba1247daa1623d9b102201e56a655abd3d3b680d1ca3d456645ec5d": rpc error: code = Unknown desc = Error response from daemon: repository veebers/caas-operator not found: does not exist or no pull access
[03:49] <veebers> as far as I'm aware it's public and should have access
[03:50] <wallyworld> um
[03:50] <wallyworld> sure you you are using --config to set docker username when bootstrapping
[03:51] <wallyworld> caas-operator-image-path
[03:51] <wallyworld> not user name
[03:52] <wallyworld> other than that, not sure. i've bootstrapped today using the official image with no problems
[03:52] <wallyworld> do you need a non-released operator image?
[03:53] <veebers> wallyworld: ah shoot, maybe using wrong username will check
[03:54] <veebers> wallyworld: hmm, I am setting caas-operator-image-path as I was previously.
[03:54] <veebers> yeah bootstrapping without setting caas-operator-image-path works, I'll keep diggin
[03:55] <wallyworld> ok
[03:58] <thumper> babbageclunk: I think the race is still there just a lot smaller than it was...
[04:56] <babbageclunk> thumper: stink
[05:09] <thumper> babbageclunk: https://paste.ubuntu.com/p/DwCgvSNYfC/
[05:09] <thumper> babbageclunk: getting there...
[05:10] <thumper> babbageclunk: I can't remember if the squishing of events is done in the txn watcher or the hub watcher
[05:10] <thumper> if it is the former, then there is no race
[05:10] <babbageclunk> I'm pretty sure it's the former
[05:10] <thumper> then, yay?
[05:11]  * babbageclunk yays?
[05:11] <thumper> I sorta hadta do a refactoring to move NewModel on to the controller object
[05:11] <thumper> from state
[05:11] <thumper> it made a bunch of things easier
[05:11] <thumper> and more correct
[05:11] <babbageclunk> yeah, it makes more sense being there now that you say it.
[05:12] <thumper> I was going to say all the tests are passing...
[05:12] <thumper> but just noticed a few watcher failures in migration tests
[05:12] <thumper> so almost there...
[05:13] <thumper> all failures so far just in modelmigration_test.go
[05:13] <thumper> I am getting hastled about finishing today though from Rachel
[05:14] <thumper> I had said I'd finish early because of the extra hours this week
[05:14] <thumper> but I'm trying to get these tests passing
[05:14]  * thumper sigh
[05:14] <thumper> never ending bollocks
[05:14] <thumper> and likely merge conflicts with devel now...
[05:15] <thumper> ... and StateSuite.TestWatchAllModels
[05:15] <thumper> and another...
[05:16]  * thumper sighs again
[05:16] <thumper> where is my wine
[05:16] <thumper> OOPS: 2263 passed, 3 skipped, 9 FAILED
[05:16] <thumper> better than before...
[05:16] <babbageclunk> yeah, I'm pretty sure it's beer oclock
[05:18] <thumper> huh
[05:18] <thumper> found that six of those failures are due to a replacement of the state clock
[05:18] <thumper> which would screw up the watchers
[05:22] <thumper> 6 fixed
[05:29] <babbageclunk> thumper: juju-engine-report isn't working for me on a freshly bootstrapped controller machine?
[05:30] <thumper> babbageclunk: underscores now
[05:30] <thumper> juju_engine_report
[05:30] <babbageclunk> thumper: ah, of course - thanks!
[06:20] <wallyworld> babbageclunk: you having luck with the bounce fix?
[06:20] <wallyworld> kelvinliu_: here's a small k8s storage fix https://github.com/juju/juju/pull/9108
[06:20] <kelvinliu_> wallyworld, looking now
[06:21] <wallyworld> ty
[06:21] <babbageclunk> wallyworld: yeah, I think so - just fixing some tests that needed a lot of thinking about, but the basic change is pretty simple. Will try it with the cmd/jujud/agent tests next.
[06:21] <wallyworld> ok
[06:22] <wallyworld> good that the big mother farker landed :-)
[06:32] <kelvinliu_> wallyworld, LGTM, thanks
[06:32] <wallyworld> thanks!
[09:01] <stickupkid> manadart: you got a second?
[09:04] <manadart> HO? Let me put a shirt on.
[09:04] <stickupkid> haha
[10:20] <stickupkid> manadart: I got it working
[10:20] <stickupkid> YESSSSS!
[10:21] <stickupkid> manadart: we can't do this here for older lxd -- https://github.com/juju/juju/blob/develop/provider/lxd/server.go#L245
[10:23] <manadart> Ahh.
[10:24] <stickupkid> I could check the version I guess, because it does work for new LXD, or do I follow the todo and move it?
[10:24] <stickupkid> manadart: also I don't think we can move it prior to instance creation?
[10:32] <manadart> I did some work on that on account of the frequency of the logging messages, but backed it out because the bridge must be there before enabling the HTTP listener.
[10:38] <stickupkid> what do you think the best course of action is, it looks like we don't need it for 2.0.x, but it's fine with 2.3.x >
[11:02] <manadart> stickupkid: I think we still need it called once on the host at the outset, we just don't want it called by model provisioning...
[11:03] <stickupkid> manadart: so we just need it hoisting further up the code flow?
[11:05] <manadart> stickupkid: Somehow, but where to put the conditional is the tricky part. This is the factory created with the environ...
[11:08] <stickupkid> manadart: so it doesn't know what's launching it, only the site of execution does (juju vs jujud)
[11:09] <manadart> stickupkid: I found the PR for the TODO that was closed: https://github.com/juju/juju/pull/8964/files
[11:12] <stickupkid> manadart: why did we close it?
[11:13] <manadart> stickupkid: I mentioned it ^^. If you don't call it, local bridge name is not set, then there is no IP for enableHTTPSListener.
[11:15] <manadart> It was one of the last LXD things I worked on. At the time I had to get an Openstack bug fixed, so it was parked. Didn't realise it was an actual bug at the time - I was addressing logging frequency.
[11:16] <stickupkid> ha
[11:16] <stickupkid> so I'll have a look at the PR and try and see what we can do with it
[12:42] <stickupkid> manadart: if we move that call up the code, we then don't end up with a local bridge name :sigh:
[12:42] <manadart> stickupkid: Exactly.
[12:42] <stickupkid> haha
[12:42] <stickupkid> right, i'm getting your comment before now, i'm just re-living it
[12:49] <stickupkid> manadart: right it seems we still need to ensure the bridge for local setups see: https://github.com/juju/juju/blob/2.2/tools/lxdclient/client.go#L158-L167
[13:50] <manadart> externalreality: It seems to be working now. Refreshed from develop, installed deps and rebuilt.
[13:53] <manadart> manadart: Panic still. Different error.
[13:53] <manadart> externalreality: ^
[14:09] <externalreality> manadart, ack
[14:10] <externalreality> manadart, looking
[14:10] <manadart> externalreality: I think I've sorted it. One minute.
[14:10] <externalreality> manadart, ok
[14:24] <manadart> externalreality: Yep. Works end-to-end. Will put a PR up now.
[14:26] <externalreality> manadart, cool
[14:34] <externalreality> manadart, I see you comment on the PR
[14:34] <manadart> externalreality: The lock cleanup? Yes.
[14:34] <externalreality> manadart, I feel dumb, not sure how I left that bit out I was sure I put it in there... thus the code review process works.
[14:35] <externalreality> manadart, In any case what I see is actually somewhat different that what you suggest on the pr
[14:35] <externalreality> manadart, as in what the main loop of the worker is doing... it seems to be switching on the retreived state. Its quite nice.
[14:36] <externalreality> manadart, anyway I updated the PR if you could have a look for quickly, I would appreaciate it. Its only a 3+ line change.
[14:36] <manadart> externalreality: Sure.
[14:37] <manadart> Also the PR that gets end-to-end working again is: https://github.com/juju/juju/pull/9109
[14:44] <stickupkid> manadart: https://github.com/juju/juju/pull/9110
[14:48] <manadart> stickupkid: Will look.
[14:49] <stickupkid> manadart: i need to do some more manual testing, around snap 2.0.x and apt-get 3.0.x
[14:49] <stickupkid> so many combinations :(
[15:50] <stickupkid> hml: updated the description https://github.com/juju/juju/pull/9110
[15:51] <stickupkid> note this needs backporting to 2.4.x - I need to check that to be sure.
[15:57] <hml> stickupkid: ack, ty
[16:49] <externalreality> manadart, if you are still around do you know why I get the error "CRIU is not installed" when trying to make a stateful snapshot with lxc - this when CRIU is infact installed.
[17:17] <hml> rick_h_: you prefer this: https://paste.ubuntu.com/p/mhJQKCFPcZ/ with the newline?  at first glance the Cloud line blends into the Enter questions to me.  either way
[17:32] <rick_h_> hml: sure, let's try it thanks.
[17:33] <hml> rick_h_: k
[17:33] <hml> rick_h_: on the PEM-encoded… i thinking a few more words would help too.
[17:34] <hml> rick_h_: Enter the LXD client certificate, name of PEM-encoded file (optional):
[17:34] <hml> rick_h_: or
[17:34] <hml> rick_h_: certificate filename, PEM-encoded (optional):
[17:47] <rick_h_> hml: plus one to the second one. I like filename
[17:50] <hml> rick_h_: ack
[18:18] <hml> rick_h_: i have 2 quick PRs up for your viewing pleasure: https://github.com/juju/juju/pull/9112 and https://github.com/juju/juju/pull/9111
[18:19] <rick_h_> hml: cool will look. Thanks.
[18:55] <rick_h_> hml: quest back to you on one of them
[19:07] <hml> rick_h_: option 2 matches more closely to the o7k cert file requestion
[19:14] <rick_h_> hml: which one is option 2?
[19:15] <hml> rick_h_: the path to the PEM-encoded LXD server certificate file
[19:15] <rick_h_> hml: cool, let's do that then. Thank you for adjusting it!
[19:38] <hml> rick_h_: 9112 is updated
[19:39] <rick_h_> hml: cool, will look once done with this call ty