| veebers | kelvinliu, babbageclunk: re: add-k8s command: it requires a kubernetes config, at the moment that's done by deploying a cluster and grabbing it off that. That would point that to add-k8s you need a cluster deployed, which would mean you need a model thus a bootstrapped controller. Does that sound right kelvinliu? | 00:28 |
|---|---|---|
| veebers | kelvinliu: or is the k8s config a requirement that can be satisfied outside of juju? | 00:31 |
| veebers | and really the req is just that config, regardless of controller/model exisiting Or is the content of the config dependant on the cluster being deployed using juju | 00:32 |
| kelvinliu | veebers, I also mentioned this in the email. In my understanding, we should be able to add cloud by selecting the cluster context/credentials from the `.kube/config` file, so it doesnot matter if the cluster was created by juju or not | 00:36 |
| veebers | kelvinliu: ok, so seems you shouldn't need an active model nor a bootstrapped controller, just a valid kubernetes config to add-k8s | 00:37 |
| kelvinliu | veebers, so we probably consider to replace the model name with the cluster name | 00:37 |
| veebers | kelvinliu: where is that change needed? | 00:38 |
| kelvinliu | veebers, but we might need the name at least coz the endpoint will be uniq per cluster | 00:38 |
| kelvinliu | veebers, one cluster one cloud | 00:39 |
| kelvinliu | veebers, I think this need to be discussed | 00:40 |
| kelvinliu | veebers, ah, if we decide the use the cluster name as the cloud name, then `juju add-k8s <name>`, so just one arg | 00:41 |
| veebers | kelvinliu: ok, agreed some discussion/clarification needed. I'll hold off actioning that bug then :-) | 00:44 |
| kelvinliu | veebers, sounds good, it actually rely on #1768837. | 01:09 |
| mup | Bug #1768837: add-k8s should be more flexible in loading config <caas> <k8s> <juju:Triaged by kelvin.liu> <https://launchpad.net/bugs/1768837> | 01:09 |
| veebers | kelvinliu: are you looking at https://bugs.launchpad.net/juju/+bug/1768845 or should I take a look? (seeing as that other bug needs further discussion :-)) | 01:45 |
| mup | Bug #1768845: add-k8s incorrectly reports "cloud already exists" <add-cloud> <add-k8s> <caas> <k8s> <remove-cloud> <juju:Triaged by kelvin.liu> <https://launchpad.net/bugs/1768845> | 01:45 |
| kelvinliu | veebers, I am not looking on this yet. | 01:59 |
| veebers | kelvinliu: Cool, you mind if I have a look? I suspect it needs less k8s context than the 3rd bug there | 02:00 |
| kelvinliu | veebers, u can take a look on this if you'd like to | 02:01 |
| veebers | kelvinliu: sweet, if it's too hard I'll hand it back ;-) | 02:01 |
| kelvinliu | veebers, ah, haha | 02:02 |
| veebers | kelvinliu: re: your email, would you mind updating the bug (https://bugs.launchpad.net/juju/+bug/1768845) with it's contents? That way there is pubic record of it | 02:02 |
| mup | Bug #1768845: add-k8s incorrectly reports "cloud already exists" <add-cloud> <add-k8s> <caas> <k8s> <remove-cloud> <juju:Triaged by veebers> <https://launchpad.net/bugs/1768845> | 02:02 |
| kelvinliu | veebers, this card is more related with remove-cloud(clean up clearly) for one cloud but not directly related with multi cloud, i think | 02:07 |
| veebers | kelvinliu: indeed, seems perhaps add-k8s is adding stuff that remove-cloud isn't removing. I'll be diving in shortly | 02:08 |
| kelvinliu | yeah, we can add the context to the other relatd cards later after we are all clear about the final solution for this. | 02:09 |
| veebers | kelvinliu: I take it I need an actual cluster deploy to add-k8s or can I fake it with fake contents in the config? A quick look at the code, it doesn't seem to validate the data or try reach out to anything | 03:30 |
| veebers | kelvinliu: would you have an example config I could use in my testing (just need to add-k8s) | 03:30 |
| vino | babbageclunk: the issue is fixed by just making use of my WriteSystemdAgent() which i refactored for updateseries. | 03:37 |
| vino | its simple. | 03:37 |
| vino | no need to go thru complete Installation process | 03:37 |
| babbageclunk | oh, nice! | 03:37 |
| vino | yea. | 03:37 |
| vino | however, i do see some issues if the I Fix the other issue in the way i swap the PerfomrUpgrade steps.. | 03:38 |
| vino | it can be a problem. | 03:38 |
| vino | u r correct. | 03:38 |
| kelvinliu | veebers, yes, take a look this https://pastebin.ubuntu.com/p/yS4mv6fwZ5/ | 03:38 |
| veebers | kelvinliu: mean cheers | 03:39 |
| vino | babbageclunk: so i shd find another way to fix that problem. | 03:40 |
| kelvinliu | veebers, cheers | 03:47 |
| babbageclunk | vino: yeah, maybe - it might be that we just need another category of upgrade steps that we run before running state upgrade steps, and the service install step would go in that category? | 03:48 |
| vino | babbageclunk: yes. something similar to newUpgradeOpsIterator | 03:49 |
| vino | which only calls installServiceFiles and it is for 2.4 upgrade | 03:50 |
| babbageclunk | vino: that sounds reasonable - although you may as well make it so that we could add other steps to it (for a different version maybe) if we needed to. | 03:53 |
| vino | babbageclunk: yup. | 03:53 |
| veebers | babbageclunk: you have a quick minute? I won't take up much of your time | 05:00 |
| babbageclunk | veebers: for you, a long minute! | 05:00 |
| babbageclunk | in standup? | 05:00 |
| veebers | babbageclunk: \o/ quick text should be ok: | 05:01 |
| babbageclunk | oh, ok | 05:01 |
| * babbageclunk removes headset sadly | 05:01 | |
| veebers | babbageclunk: this is logs of interest re: the CMR test and tearing down the model failing: https://pastebin.ubuntu.com/p/km7Ndcmxn6/ | 05:01 |
| veebers | hah sorry :-| | 05:01 |
| babbageclunk | ok | 05:01 |
| veebers | This test started failing when I used percona-cluster instead of just mysql, but it passes (when using percona-cluster) on a slightly older 2.4-beta2 build (I haven't narrowed down when the failure was introed) | 05:02 |
| babbageclunk | ok | 05:02 |
| veebers | babbageclunk: This is errounous behaviour right? Different to the suggested by Ian? | 05:02 |
| babbageclunk | hang on, reading more carefully | 05:03 |
| babbageclunk | in the email about CMR? | 05:03 |
| veebers | babbageclunk: I hijacked the email to talk about the bug I'm looking at, it's talking about CMR and offers | 05:04 |
| babbageclunk | yeah, I see the one - just comparing what he's saying with the log now | 05:04 |
| babbageclunk | So the crit logs are your additions? | 05:05 |
| babbageclunk | veebers: ^ | 05:05 |
| veebers | babbageclunk: huh, maybe it is what Ian outlined in the email "And because the application can't be deleted, model destruction never completes" | 05:05 |
| veebers | babbageclunk: hmm, if they are they shouldn't be there. Let me confirm (I just re-built an older juju, the merge of the macaroon changes) | 05:06 |
| veebers | babbageclunk: ugh no, this is a build of develop sorry, but yeah I'll check the crit log addition | 05:06 |
| babbageclunk | oh, I thought they might have been something you'd added chasing this. | 05:07 |
| veebers | babbageclunk: ok, so yes they are my crit logs, but the should not be there, they are left over cruft | 05:07 |
| babbageclunk | it kind of does look like what he was suggesting - the offer keeping the application alive because of the remote applications that are still connected to it. | 05:08 |
| babbageclunk | veebers: | 05:09 |
| veebers | babbageclunk: ok, so presumably behaviour added sometime in the 2.4-beta2 lifespan. | 05:11 |
| veebers | babbageclunk: although odd that it doesn't happen when using mysql :-\ | 05:11 |
| veebers | babbageclunk: I'll email with the log and my questions to try get clarity on it | 05:12 |
| babbageclunk | veebers: maybe? yeah, that is odd. Seems unlikely when you put it like that. | 05:12 |
| veebers | I'll also push up a PR shortly removing those extra logging that snuck past me | 05:12 |
| babbageclunk | So it doesn't happen with mysql either on develop or beta1, and only happens with percona-cluster on develop? | 05:13 |
| veebers | babbageclunk: correct. I'm just going to re-run using mysql now see if I can't capture something useful | 05:14 |
| veebers | babbageclunk: oh, also seems odd that the machine the application was on is removed even though the app wasn't removed. Or is that not so surprising? | 05:17 |
| babbageclunk | veebers: yeah, that's surprising, usually the application is removed when the last unit goes. | 05:17 |
| veebers | babbageclunk: ack thanks, I'll fire off an email | 05:18 |
| babbageclunk | veebers: oh, also - looking at the log it seems like it's actually the model log rather than the controller log? You wouldn't normally see messages about applications and relations starting in the controller log. Maybe there are some details in the controller log that would be useful? | 05:21 |
| veebers | babbageclunk: that is the controller machine-0.log. I didn't catch the failure before the models machine was removed :0\ | 05:23 |
| veebers | babbageclunk: SInce I grabbed that log I see further entries, namely: ERROR juju.worker.dependency engine.go:551 "environ-tracker" manifold worker returned unexpected error: cannot read environ config: settings not found (not found) | 05:23 |
| veebers | the model is still there (controller and relation-sink) | 05:24 |
| veebers | and I see this error in status; attempt 2 to destroy model failed (will retry): model not empty, found 1 application (model not empty) | 05:25 |
| babbageclunk | veebers: ah, ok | 05:25 |
| babbageclunk | veebers: any errors in the log? | 05:35 |
| babbageclunk | veebers: I mean, other errors? | 05:35 |
| veebers | babbageclunk: heh, I just killed those machines. I don't think so, but I'll set it up to run again and collect the useful bits after dinner (the test annoyingly takes a while) | 05:36 |
| babbageclunk | veebers: Could it be that the handling that was added to deal with dots in config need to be added somewhere in the CMR config propagation as well? Maybe it was succeeding in beta1 because the dot handling bug was there? | 05:48 |
| veebers | babbageclunk: oh, interesting idea! | 06:18 |
| === Spads_ is now known as Spads | ||
| === Cynerva_ is now known as Cynerva | ||
| === cnewcomer_ is now known as cnewcomer | ||
| bdx | elastic-peeps: ELK 6.x on bionic https://paste.ubuntu.com/p/5GyQ2pQFNY/ | 19:05 |
| bdx | https://jujucharms.com/u/omnivector/elk/ | 19:05 |
| === magicalt1out is now known as magicaltrout | ||
| === petevg_ is now known as petevg | ||
| === eriklonroth_ is now known as eriklonroth | ||
| === kina___ is now known as kina__ | ||
| === vern_ is now known as vern | ||
| veebers | ah man, I keep deleting my test machines out from under juju, I need to stop trying to do 2 things at once | 21:19 |
| magicaltrout | thats not bad, I wiped my entire work laptop yesterday ;) | 21:20 |
| veebers | magicaltrout: 0_0 d'oh that's a wee bit more annoying! | 21:22 |
| babbageclunk | yeah, I keep removing machines when I meant to ssh to them. (Still not as bad as wiping laptop though. Pushes to github just in case.) | 21:30 |
| * veebers wonders if he should *finally* get a proper backup solution sorted | 21:30 | |
| veebers | I think I may have jinxed myself now | 21:30 |
| veebers | when I 'juju add-k8s' it hits the controller and inserts data in it's DB. Is that because that's needed for add-k8s, or needed for add-cloud in general or is that happening now because I have a controller bootstrapped, if I didn't have a bootstrapped controller and added a cloud (at the moment you need that for add-k8s) it wouldn't happen until I did bootstrap and it would happen at that point? | 21:42 |
| veebers | Ok, seems that add-k8s is special, add-cloud does not hit the controller to add data | 22:14 |
| === blahdeblah_ is now known as blahdeblah | ||
| wpk | veebers: I recently put an odroid hc2 + 4TB drive on another continent, in case of a gamma ray burst. | 23:10 |
| wpk | veebers: should something happen I'll get it shipped with fedex, will be faster than transferring it back | 23:11 |
| veebers | wpk: now that's a backup strategy! | 23:58 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!