[00:05] kelvinliu_: forgot to ask, this PR can be closed right? https://github.com/juju/juju/pull/8936 [00:08] wallyworld, yeah, i just closed this for now. we can solve it later. [00:13] kelvinliu_: can you make sure there's a card on the caas trllo board for it so we don't forget [00:16] wallyworld, sure, done. [00:16] ty [00:19] np [01:54] bollocks [01:54] wallyworld: I'm I going crazy, using a fresh develop build I deploy a caas charm, juju status never changes the workload or message from active/Started Container [01:54] intermittent allwatcher test failures [01:54] I thought my changes introduced that hence why trying develop. I built a fresh operator image and set that too incase there was some caching or something [01:55] I think we have had them for a while, [01:55] * thumper disd [01:55] digs [01:55] veebers: i don't quite follow. status should not start out as active [01:57] wallyworld: sorry, this is status after ages: https://pastebin.canonical.com/p/3p5yYBFnKf/ I see with kubectl -n failing-message log -f juju-operator-mariadb that everything is happy etc. and the unit is fine [01:57] I assumed that status should have been updated right? [01:58] veebers: sorry, what's wrong with status? it looks correct? what are you expecting to see? [01:59] wallyworld: ah shit, you're right; sorry I was thinking that 'started container' was the first message and something else should replace that. [01:59] wallyworld: I blame Friday + lack of sleep [01:59] I'll continue on my way ^_^ [01:59] no worries. not that the charm is broken a bit [02:00] it will sometimes send the wrong status after a restart [02:00] *note [02:01] ack [02:27] babbageclunk: \o/ [02:28] anastasiamac: hey hey [02:28] just celebrating ur PR's merge... [02:29] anastasiamac: oh yay! [02:29] * babbageclunk dances [02:49] i need a teddybear... babbageclunk your soft... [02:51] * babbageclunk sighs [02:51] ok! [02:51] in 1:1 [03:49] ah man, wallyworld I'm all of sudden seeing this error, any thoughts on debugging it? (I use make operator-image and docker push to publish it: Failed to pull image "veebers/caas-operator@sha256:fc83ad5cbba1247daa1623d9b102201e56a655abd3d3b680d1ca3d456645ec5d": rpc error: code = Unknown desc = Error response from daemon: repository veebers/caas-operator not found: does not exist or no pull access [03:49] as far as I'm aware it's public and should have access [03:50] um [03:50] sure you you are using --config to set docker username when bootstrapping [03:51] caas-operator-image-path [03:51] not user name [03:52] other than that, not sure. i've bootstrapped today using the official image with no problems [03:52] do you need a non-released operator image? [03:53] wallyworld: ah shoot, maybe using wrong username will check [03:54] wallyworld: hmm, I am setting caas-operator-image-path as I was previously. [03:54] yeah bootstrapping without setting caas-operator-image-path works, I'll keep diggin [03:55] ok [03:58] babbageclunk: I think the race is still there just a lot smaller than it was... [04:56] thumper: stink [05:09] babbageclunk: https://paste.ubuntu.com/p/DwCgvSNYfC/ [05:09] babbageclunk: getting there... [05:10] babbageclunk: I can't remember if the squishing of events is done in the txn watcher or the hub watcher [05:10] if it is the former, then there is no race [05:10] I'm pretty sure it's the former [05:10] then, yay? [05:11] * babbageclunk yays? [05:11] I sorta hadta do a refactoring to move NewModel on to the controller object [05:11] from state [05:11] it made a bunch of things easier [05:11] and more correct [05:11] yeah, it makes more sense being there now that you say it. [05:12] I was going to say all the tests are passing... [05:12] but just noticed a few watcher failures in migration tests [05:12] so almost there... [05:13] all failures so far just in modelmigration_test.go [05:13] I am getting hastled about finishing today though from Rachel [05:14] I had said I'd finish early because of the extra hours this week [05:14] but I'm trying to get these tests passing [05:14] * thumper sigh [05:14] never ending bollocks [05:14] and likely merge conflicts with devel now... [05:15] ... and StateSuite.TestWatchAllModels [05:15] and another... [05:16] * thumper sighs again [05:16] where is my wine [05:16] OOPS: 2263 passed, 3 skipped, 9 FAILED [05:16] better than before... [05:16] yeah, I'm pretty sure it's beer oclock [05:18] huh [05:18] found that six of those failures are due to a replacement of the state clock [05:18] which would screw up the watchers [05:22] 6 fixed [05:29] thumper: juju-engine-report isn't working for me on a freshly bootstrapped controller machine? [05:30] babbageclunk: underscores now [05:30] juju_engine_report [05:30] thumper: ah, of course - thanks! [06:20] babbageclunk: you having luck with the bounce fix? [06:20] kelvinliu_: here's a small k8s storage fix https://github.com/juju/juju/pull/9108 [06:20] wallyworld, looking now [06:21] ty [06:21] wallyworld: yeah, I think so - just fixing some tests that needed a lot of thinking about, but the basic change is pretty simple. Will try it with the cmd/jujud/agent tests next. [06:21] ok [06:22] good that the big mother farker landed :-) [06:32] wallyworld, LGTM, thanks [06:32] thanks! === alephnull_ is now known as alephnull [09:01] manadart: you got a second? [09:04] HO? Let me put a shirt on. [09:04] haha [10:20] manadart: I got it working [10:20] YESSSSS! [10:21] manadart: we can't do this here for older lxd -- https://github.com/juju/juju/blob/develop/provider/lxd/server.go#L245 [10:23] Ahh. [10:24] I could check the version I guess, because it does work for new LXD, or do I follow the todo and move it? [10:24] manadart: also I don't think we can move it prior to instance creation? [10:32] I did some work on that on account of the frequency of the logging messages, but backed it out because the bridge must be there before enabling the HTTP listener. [10:38] what do you think the best course of action is, it looks like we don't need it for 2.0.x, but it's fine with 2.3.x > [11:02] stickupkid: I think we still need it called once on the host at the outset, we just don't want it called by model provisioning... [11:03] manadart: so we just need it hoisting further up the code flow? [11:05] stickupkid: Somehow, but where to put the conditional is the tricky part. This is the factory created with the environ... [11:08] manadart: so it doesn't know what's launching it, only the site of execution does (juju vs jujud) [11:09] stickupkid: I found the PR for the TODO that was closed: https://github.com/juju/juju/pull/8964/files [11:12] manadart: why did we close it? [11:13] stickupkid: I mentioned it ^^. If you don't call it, local bridge name is not set, then there is no IP for enableHTTPSListener. [11:15] It was one of the last LXD things I worked on. At the time I had to get an Openstack bug fixed, so it was parked. Didn't realise it was an actual bug at the time - I was addressing logging frequency. [11:16] ha [11:16] so I'll have a look at the PR and try and see what we can do with it [12:42] manadart: if we move that call up the code, we then don't end up with a local bridge name :sigh: [12:42] stickupkid: Exactly. [12:42] haha [12:42] right, i'm getting your comment before now, i'm just re-living it [12:49] manadart: right it seems we still need to ensure the bridge for local setups see: https://github.com/juju/juju/blob/2.2/tools/lxdclient/client.go#L158-L167 [13:50] externalreality: It seems to be working now. Refreshed from develop, installed deps and rebuilt. [13:53] manadart: Panic still. Different error. [13:53] externalreality: ^ [14:09] manadart, ack [14:10] manadart, looking [14:10] externalreality: I think I've sorted it. One minute. [14:10] manadart, ok [14:24] externalreality: Yep. Works end-to-end. Will put a PR up now. [14:26] manadart, cool [14:34] manadart, I see you comment on the PR [14:34] externalreality: The lock cleanup? Yes. [14:34] manadart, I feel dumb, not sure how I left that bit out I was sure I put it in there... thus the code review process works. [14:35] manadart, In any case what I see is actually somewhat different that what you suggest on the pr [14:35] manadart, as in what the main loop of the worker is doing... it seems to be switching on the retreived state. Its quite nice. [14:36] manadart, anyway I updated the PR if you could have a look for quickly, I would appreaciate it. Its only a 3+ line change. [14:36] externalreality: Sure. [14:37] Also the PR that gets end-to-end working again is: https://github.com/juju/juju/pull/9109 [14:44] manadart: https://github.com/juju/juju/pull/9110 [14:48] stickupkid: Will look. [14:49] manadart: i need to do some more manual testing, around snap 2.0.x and apt-get 3.0.x [14:49] so many combinations :( [15:50] hml: updated the description https://github.com/juju/juju/pull/9110 [15:51] note this needs backporting to 2.4.x - I need to check that to be sure. [15:57] stickupkid: ack, ty [16:49] manadart, if you are still around do you know why I get the error "CRIU is not installed" when trying to make a stateful snapshot with lxc - this when CRIU is infact installed. [17:17] rick_h_: you prefer this: https://paste.ubuntu.com/p/mhJQKCFPcZ/ with the newline? at first glance the Cloud line blends into the Enter questions to me. either way [17:32] hml: sure, let's try it thanks. [17:33] rick_h_: k [17:33] rick_h_: on the PEM-encoded… i thinking a few more words would help too. [17:34] rick_h_: Enter the LXD client certificate, name of PEM-encoded file (optional): [17:34] rick_h_: or [17:34] rick_h_: certificate filename, PEM-encoded (optional): [17:47] hml: plus one to the second one. I like filename [17:50] rick_h_: ack [18:18] rick_h_: i have 2 quick PRs up for your viewing pleasure: https://github.com/juju/juju/pull/9112 and https://github.com/juju/juju/pull/9111 [18:19] hml: cool will look. Thanks. [18:55] hml: quest back to you on one of them [19:07] rick_h_: option 2 matches more closely to the o7k cert file requestion [19:14] hml: which one is option 2? [19:15] rick_h_: the path to the PEM-encoded LXD server certificate file [19:15] hml: cool, let's do that then. Thank you for adjusting it! [19:38] rick_h_: 9112 is updated [19:39] hml: cool, will look once done with this call ty