| wallyworld | vino: did you have a minute, i just looked at juju run pr, quick hangout? | 00:52 |
|---|---|---|
| vino_ | yes | 00:57 |
| vino_ | wallyworld: yes | 00:58 |
| wallyworld | vino: see you in standup | 00:59 |
| wallyworld | anastasiamac: small PR if you have a chance - needed for 2.4 beta 1 https://github.com/juju/juju/pull/8613 | 01:00 |
| anastasiamac | wallyworld: nws, m about to pr up the comment u've asked and will need a review too :D | 01:03 |
| anastasiamac | babbageclunk: veebers: could u please TAL https://github.com/juju/juju/pull/8614 does it make sense to u coming in cold into the context? | 01:18 |
| babbageclunk | anastasiamac: will take a look soon | 01:21 |
| wallyworld | anastasiamac: will do just let me know the PR | 01:22 |
| anastasiamac | wallyworld: https://github.com/juju/juju/pull/8614 :) | 01:27 |
| wallyworld | looking | 01:27 |
| wallyworld | anastasiamac: lgtm, ty | 01:28 |
| * anastasiamac does a fast-landing dance \o/ | 01:37 | |
| babbageclunk | oops, sorry anastasiamac | 01:45 |
| babbageclunk | Oh, should have looked first before holding off! | 01:45 |
| anastasiamac | babbageclunk: u r sorry i dance? :D | 01:46 |
| babbageclunk | No, I just should have paused what I was doing! | 01:47 |
| anastasiamac | babbageclunk: nws, i just needed someone to tell me that that i made sense :D wallyworld is good at keeping my english straight :D PR is heading for landing now | 01:48 |
| anastasiamac | babbageclunk: no no, it was not worth interuppting anyone... was just a review ;D | 01:49 |
| babbageclunk | Well I'll do better next time. | 01:53 |
| veebers | sorry anastasiamac was at lunch :-\ | 02:06 |
| anastasiamac | veebers: all good! | 02:06 |
| * thumper goes to make a coffee | 02:13 | |
| veebers | wallyworld: when you have a spare moment if you could ack the test addition I made since your approval: https://github.com/juju/juju/pull/8599 | 02:23 |
| wallyworld | sure, sorry i totally forgot | 02:23 |
| veebers | nw | 02:23 |
| wallyworld | veebers: just a minor tweak | 02:25 |
| veebers | gah, my illusion of perfection is broken ;-) I'll fix it up and merge | 02:27 |
| wallyworld | anastasiamac: thanks for review, i missed those other errors, i've updated PR with a new commit | 02:35 |
| veebers | wallyworld: remind me the command that shows charm hooks run? | 02:46 |
| veebers | hah, that's almost not english :-) | 02:46 |
| wallyworld | juju show-status-log | 02:47 |
| veebers | awesome thanks | 02:48 |
| veebers | wallyworld: is install hook retried by default perchance? | 02:49 |
| wallyworld | veebers: yah, all hooks are retried i think 3 times | 02:49 |
| veebers | wallyworld: ah right cheers, I'll update the test charm to take that into account :-) | 02:49 |
| wallyworld | yeah sorry should have mentioned that | 02:50 |
| veebers | nw, no biggie | 02:50 |
| wallyworld | anastasiamac: i just noticed you already approved but given the changes i made for errors are non-trivial, would appreciate another look; they are all in the 2nd commit | 02:53 |
| anastasiamac | wallyworld: k, looking now :D | 02:53 |
| wallyworld | awesome, ty | 02:53 |
| wallyworld | yay, thanks for review | 02:59 |
| === mup_ is now known as mup | ||
| thumper | babbageclunk: got a minute, I'm a little confused | 03:00 |
| babbageclunk | thumper: sure | 03:00 |
| thumper | babbageclunk: jumping in the HO | 03:00 |
| anastasiamac | wallyworld: that was really good !!! loved how u consolidated all these errors. once i saw it, it totally made sense | 03:01 |
| wallyworld | yeah, we do tend to scatter shite everywhere | 03:02 |
| wallyworld | was good to clean it up | 03:02 |
| wallyworld | thanks for noticing in the PR | 03:02 |
| wallyworld | was late when i did it :-) | 03:02 |
| anastasiamac | \o/ | 03:02 |
| veebers | It appears that juju is attempting to re-try the install hook at least 8 times. Am I doing something odd in my charm to trigger this? | 03:31 |
| veebers | all my charm is doing is in the install hook increment the attempt count (file stored in tmp) and setting: "status-set blocked "Install hook failed on purpose." then exit 1 | 03:31 |
| veebers | wallyworld: oh you;re back :-) I was just complaining: It appears that juju is attempting to re-try the install hook at least 8 times. Am I doing something odd in my charm to trigger this? | 03:35 |
| veebers | all my charm is doing is in the install hook increment the attempt count (file stored in tmp) and setting: "status-set blocked "Install hook failed on purpose." then exit 1 | 03:35 |
| wallyworld | yeah, using vpn drops irc :-( | 03:35 |
| wallyworld | i don't think you're doing anything amiss | 03:36 |
| wallyworld | i think juju reties 3 times | 03:36 |
| wallyworld | maybe just exit 1 and see what happens though that shouldn't make a difference | 03:36 |
| thumper | wallyworld: it retries more than 3 times | 03:37 |
| wallyworld | really? :-( | 03:37 |
| wallyworld | i wonder why | 03:37 |
| wallyworld | even 3 seems too many | 03:37 |
| wallyworld | i had recalled that we settled on 3 but obviously i'm wrong | 03:37 |
| veebers | it's tried 10 times now ^_^ | 03:38 |
| wallyworld | i'm sure in my testing it didn't do that | 03:38 |
| wallyworld | it retried what i could have sworn was 3 times and then gave up | 03:38 |
| wallyworld | and left the unit in error state | 03:38 |
| veebers | wallyworld: if your testing was just exit 1 always you might not have seen it, but because I'm waiting to make sure it stops trying to ensure my resolve is the thing that fixed it I see it happen | 03:39 |
| wallyworld | but could be wrong there too | 03:39 |
| wallyworld | if you always exit 1 then resolve will be the thing that moves it on | 03:39 |
| * thumper thrashes his computer somewhat | 03:40 | |
| veebers | wallyworld: right, I can resolve --no-retry to move it on, but I can't ensure that resolve will re-run the hook when requested (although it seems perhaps that's not needed as it's *always* retrying) | 03:42 |
| thumper | babbageclunk: got 5 minutes? | 03:43 |
| thumper | babbageclunk: I'm invoking the live demo demons | 03:44 |
| thumper | babbageclunk: demonstrating as yet untried code | 03:44 |
| babbageclunk | sure | 03:44 |
| wallyworld | veebers: yeah, the code will need to be read to see what's going on. i can't recall off hand | 03:44 |
| wallyworld | --no-retry would be a good start | 03:44 |
| veebers | wallyworld: ack. I'll proceed with a functional test for no-retry. If I get nosey / have time I might try figure out what the code is doing there too | 03:45 |
| wallyworld | sgtm | 03:45 |
| * thumper jumps back in HO | 03:46 | |
| === mup_ is now known as mup | ||
| veebers | wallyworld: I'm confused, my juju 2.3.5 resolve works as expected, passing --no-retry skips the install hook and goes onto the start hook | 03:56 |
| === kelvinliu__ is now known as kelvinliu | ||
| === meetingology` is now known as meetingology | ||
| === mup_ is now known as mup | ||
| thumper | babbageclunk: I realise there was one more space where I haven't hooked it up :) | 04:15 |
| babbageclunk | thumper: I mean, I guess that had to have been the problem | 04:15 |
| thumper | :) | 04:16 |
| babbageclunk | thumper: so it's working now with quick status changes? | 04:18 |
| thumper | shtill threading | 04:18 |
| thumper | babbageclunk: got it using the new code, but all coming back as unknown not alive | 04:34 |
| thumper | so all machine showing down | 04:34 |
| * thumper sighs | 04:34 | |
| babbageclunk | doh | 04:34 |
| thumper | added more logging | 04:35 |
| thumper | ah FFS | 04:35 |
| * thumper knows | 04:36 | |
| === mup_ is now known as mup | ||
| thumper | babbageclunk: I think I have it this time... | 04:46 |
| babbageclunk | by George, I think he's got it! | 04:46 |
| thumper | nope | 04:49 |
| thumper | got logic backwards | 04:49 |
| thumper | trying again | 04:49 |
| thumper | this is the culmination of around two years of effort | 04:49 |
| thumper | I'll be celebrating when this is in | 04:49 |
| wallyworld | thumper: sent email about upgrade test for 2.3.6 - short answer NFI, can't reproduce | 04:50 |
| thumper | BOOM!!! | 04:51 |
| thumper | working | 04:51 |
| thumper | wallyworld: ok | 04:51 |
| thumper | babbageclunk: wanna see it before I run to get Maia | 04:51 |
| thumper | ? | 04:51 |
| thumper | too late | 04:53 |
| thumper | gotta go | 04:53 |
| thumper | I'll be back on later tonight to polish and put up for review | 04:54 |
| thumper | wallyworld: this is new presence hooked up in status | 04:54 |
| wallyworld | oh joy | 04:54 |
| thumper | with user updatable controller feature flags | 04:54 |
| wallyworld | faaark, nice | 04:54 |
| thumper | wallyworld: jump on a HO | 04:54 |
| thumper | I want to show someone | 04:54 |
| wallyworld | righto | 04:54 |
| wallyworld | lol | 04:54 |
| wallyworld | vino_: go fmt is still sad. you'll need to do a quick fix for run_test.go | 04:54 |
| vino_ | that same line 95 ? | 04:54 |
| thumper | wallyworld: in our 1:1 | 04:55 |
| vino_ | ok let me look | 04:55 |
| wallyworld | vino_: there's about 4 errors | 05:01 |
| babbageclunk | thumper: oops, sorry | 05:02 |
| === mup_ is now known as mup | ||
| vino_ | i cudnt get this githook fix in my setup. | 05:06 |
| vino_ | huh.. | 05:06 |
| anastasiamac | thumper: wallyworld: PTAL licencing for 2.3 - https://github.com/juju/juju/pull/8615 | 05:08 |
| wallyworld | ok | 05:09 |
| wallyworld | jeez, 31 files | 05:09 |
| anastasiamac | wallyworld: but mostly tests :D | 05:09 |
| anastasiamac | 31? m seeing 25... | 05:10 |
| wallyworld | maybe i can't count | 05:14 |
| veebers | wallyworld: you're 'resolve --retry' fix, did that go in for 2.3.5 or 2.3.6? | 05:21 |
| wallyworld | 2.3.6 | 05:21 |
| veebers | wallyworld: so with 2.3.5 we should see that 'resolve --no-retry' does in fact try the errored hook? I'm not seeing that, but I"ll double down now and confirm that | 05:23 |
| wallyworld | yeah, i think 2.3.5 has the issue but hmmm, not sure now | 05:24 |
| vino_ | wallyworld: run_test.go - 4 errors correct ? | 05:30 |
| wallyworld | sounds right yeah, that's whay my go fmt said i think | 05:31 |
| vino_ | line: 105, 110, 119 | 05:31 |
| wallyworld | simething like that, don't have the putput anymore | 05:31 |
| vino_ | no worries. | 05:31 |
| vino_ | wallyworld: done. plz verify | 05:38 |
| wallyworld | looking | 05:40 |
| wallyworld | vino_: i thought there were four errors, https://pastebin.ubuntu.com/p/b3DM8qm4m3/ | 05:42 |
| kelvinliu_ | wallyworld: i still got the same problem. so after I run `juju deploy ./mysql`, I got this message immediately, after unit status changed to `allocating`. | 05:47 |
| vino_ | can u see that ? | 05:48 |
| wallyworld | which message? | 05:48 |
| vino_ | next to @line104 | 05:48 |
| wallyworld | lookin again at pr | 05:49 |
| wallyworld | vino_: ah, look ok. the pr showed 2 changes together | 05:50 |
| wallyworld | sorry, looks good, will approve | 05:50 |
| vino_ | yea. | 05:50 |
| vino_ | but i shd get this githook and few other things fixed sooner.. | 05:50 |
| vino_ | no sorry required :) | 05:50 |
| wallyworld | yeah, see if you can get it working | 05:50 |
| wallyworld | also, our landing bot shoulld not accept prs with these errors | 05:51 |
| wallyworld | we need to fix that | 05:51 |
| wallyworld | kelvinliu_: which message are you seeing? | 05:51 |
| kelvinliu_ | wallyworld: some logs are here https://pastebin.ubuntu.com/p/D9sjnXvY4K/ | 05:51 |
| wallyworld | kelvinliu_: i can't see anything obvious. does juju status show an incremented version number to indicate the upgrade went ok? ie 2.4-beta1.2 or something like that | 05:55 |
| wallyworld | you could try creating a new model and trying again | 05:55 |
| kelvinliu_ | wallyworld: the controller upgrade went ok, and tried it a few times, now I got 4 testcaas models | 05:57 |
| wallyworld | you should see in the operator logs that the charm reactive code is calling set-pod-spec | 05:58 |
| wallyworld | this triggers juju to create the unit | 05:58 |
| wallyworld | which then creates the mysql pod | 05:58 |
| kelvinliu_ | wallyworld: no, it's not there | 05:59 |
| wallyworld | ok, let me run up a k8s cluster and see what my logging says | 05:59 |
| kelvinliu_ | wallyworld: so after `juju-unit executing running start hook` then `workload unknown`, then `juju-unit idle` | 06:01 |
| wallyworld | i can't recall the exact order off hand, i'll see when my k8s cluster starts | 06:03 |
| kelvinliu_ | wallyworld: thx | 06:03 |
| kelvinliu_ | wallyworld: or do u have a couple of minutes for a screen share session? | 06:04 |
| wallyworld | kelvinliu_: i won't be much help until i run up my k8s bundle to compare | 06:05 |
| kelvinliu_ | wallyworld: ok, sure, we will wait for the cluster up and running first | 06:05 |
| wallyworld | kelvinliu_: i just started a k8s cluster and deployed mysql without any problem | 06:26 |
| wallyworld | did you want to do a hangout? | 06:26 |
| kelvinliu_ | wallyworld: yes, plz | 06:27 |
| wallyworld | see you in standup | 06:27 |
| kelvinliu_ | yup | 06:27 |
| vino_ | have a quick question.. only snap install, will install the service files in /var/lib/juju/init ? | 06:48 |
| vino_ | i dont see in /var/lib/juju/tool | 07:01 |
| wallyworld | vino_: snap install of juju is just the client | 07:23 |
| wallyworld | when you bootstrap, it pulls down the agent binary and creates the systemd files on the vm | 07:23 |
| wallyworld | so you need to bootstrap and ssh into the controller machine to see what was done | 07:24 |
| kelvinliu_ | wallyworld: just raised a bug for the remove-relation failure on apps running k8s -> https://bugs.launchpad.net/juju/+bug/1764649 and also created a card on caas trello board plz let me know if any questions. thx | 07:25 |
| mup | Bug #1764649: juju caas remove-relation does not work <juju:New for wallyworld> <https://launchpad.net/bugs/1764649> | 07:25 |
| wallyworld | great ty | 07:32 |
| vino_ | wallyworld: sorry i was a bit away for my tea. | 07:40 |
| vino_ | yes i did that. | 07:41 |
| vino_ | and i am there | 07:41 |
| wallyworld | vino_: great ok, so the service files - you should see a symlink from files /etc/systemd/system to their actual location under /var/lib/juju/init | 07:43 |
| vino_ | :) yes. | 07:43 |
| vino_ | i did ssh to instance but was looking in another terminal didnt realize :p | 07:45 |
| wallyworld | ah no worries | 07:45 |
| thumper | wallyworld: still around? | 08:12 |
| wallyworld | thumper: sorta | 08:13 |
| wallyworld | soccer soon | 08:13 |
| thumper | wallyworld: how soon? | 08:13 |
| thumper | just getting PR ready | 08:13 |
| wallyworld | 15 | 08:13 |
| wallyworld | maybe 0 | 08:14 |
| wallyworld | 20 | 08:14 |
| thumper | long enough | 08:15 |
| wallyworld | ok | 08:15 |
| thumper | poo, need to remove some critical logging... | 08:19 |
| thumper | wallyworld: https://github.com/juju/juju/pull/8617 | 08:21 |
| thumper | wallyworld: did you want to get on a hangout to talk through? | 08:21 |
| thumper | smaller than the penultimate branch | 08:21 |
| wallyworld | thumper: can do | 08:22 |
| * thumper jumps in HO | 08:22 | |
| thumper | phew... | 08:53 |
| * thumper waits for the merge | 08:53 | |
| thumper | manadart: ping | 08:58 |
| manadart | thumper: Pong. On a call with jam. | 09:15 |
| thumper | manadart: hey, was just wondering if you had an ETA on removing the horrible lxd patches we have | 09:15 |
| manadart | thumper: Not yet, but I can peel off this intermittent failure I am working on and round up such an estimate. | 09:16 |
| thumper | heh... no, we're good | 09:17 |
| thumper | what is the intermittent failures? | 09:17 |
| manadart | https://bugs.launchpad.net/juju/+bug/1753418 | 09:20 |
| mup | Bug #1753418: intermittent failure in kvmProvisionerSuite.TestKVMProvisionerObservesConfigChanges <intermittent-failure> <test-failure> <juju:In Progress by manadart> <https://launchpad.net/bugs/1753418> | 09:20 |
| thumper | night all | 09:41 |
| manadart | G'night. | 09:42 |
| hml | g’morning o/ | 13:48 |
| rick_h_ | morning hml | 13:49 |
| === frankban is now known as frankban|afk | ||
| thumper | morning | 20:27 |
| hml | morning thumper | 20:30 |
| veebers | Morning o/ | 20:38 |
| thumper | wallyworld: so... no power nor internet at home now | 21:31 |
| thumper | my laptop battery has several hours I guess | 21:32 |
| thumper | and tethered to my phone | 21:32 |
| thumper | they are replacing a damanged pole outside my house, so have turned off power, but we never got notified | 21:32 |
| thumper | from 9 till 3 | 21:32 |
| wallyworld | thumper: oh that's what happended | 21:33 |
| thumper | yeah, I was a little surprised, but you should see the kids | 21:33 |
| thumper | they are like, I can't charge my phone? | 21:34 |
| thumper | can't watch netflix? | 21:34 |
| thumper | what am I going to do? | 21:34 |
| hml | ha! | 21:35 |
| wallyworld | lol | 21:36 |
| wallyworld | thumper: so no release call :-) i thought i might be good to chat about that upgrade issue even thoigh nix is away | 21:36 |
| thumper | wallyworld: let's do a voice only hangout | 21:38 |
| thumper | to save my data | 21:39 |
| wallyworld | ok | 21:39 |
| veebers | wallyworld: I'm scratching my head on this one, re: resolve command: every build I've tried it works as expected, --no-retry skips the failing install hook and continues. | 22:07 |
| veebers | I tried 2.3.5 (snap installed), I tried the commit before your one that fixed the issue | 22:08 |
| babbageclunk | ha, if you're benchmarking something where you're appending to logs, make sure part of the thing isn't copying all of the logs so your bechmark gets progressively slower. | 22:08 |
| wallyworld | veebers: i don't have an explanation for that off hand, will need to investigate | 22:09 |
| wallyworld | thumper: we can fill out much of the 2.4 release notes but the enable-ha/remove-machine and ha space sections are todo (john/joe) as are model owner changes (you). i think we may need a day to get these sorted | 22:19 |
| thumper | yeah | 22:19 |
| wallyworld | so we maybe should take a view tomorrow after forcing folks to fill out notes today :-) | 22:20 |
| wallyworld | we can do 2.4 and 2.3.6 a day apart | 22:20 |
| wallyworld | different people do each | 22:20 |
| vino | hey Wallyworld: morning! fixed the format issues u have mentioned. | 22:35 |
| wallyworld | great ty | 22:35 |
| wallyworld | looking at pr | 22:35 |
| wallyworld | lgtm | 22:36 |
| wallyworld | babbageclunk: ready! | 23:47 |
| wallyworld | still in same hangout | 23:48 |
| babbageclunk | ok | 23:49 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!