| hpidcock | thumper: looking | 00:09 |
|---|---|---|
| thumper | hpidcock: ta | 01:32 |
| kelvinliu | just pushed and released the new mariadb and mediawiki testing charms for CI, got this tiny PR to upgrade those charms for test, anyone got 1min to take a look plz? thank you https://github.com/juju/juju/pull/11738 | 02:42 |
| kelvinliu | and https://github.com/juju/juju/pull/11739 ports the change to develop | 02:42 |
| thumper | https://github.com/juju/juju/pull/11740 | 03:44 |
| * thumper looks at kelvinliu's branch | 03:45 | |
| thumper | kelvinliu: good to go | 03:45 |
| kelvinliu | ty | 03:45 |
| hpidcock | looking thumper | 03:50 |
| Chipaca | thumper: đź‘‹ | 05:06 |
| thumper | ah poo | 05:25 |
| thumper | sorry Chipaca | 05:25 |
| thumper | Chipaca: got busy watching some testing | 05:25 |
| thumper | and for whatever reason, didn't actualy create a calendar item | 05:26 |
| thumper | Chipaca: I could chat now for a bit if you are still around | 05:27 |
| thumper | this was on me sorry, I should have made the calendar entry, then I would have remembered | 05:27 |
| jam | thumper, I think he went back to sleep, but we'll plan for it next week | 05:30 |
| thumper | jam: ok, sorry | 05:30 |
| Chipaca | thumper: you are forgiven :-) i got a nice early run out of it (the "go back to bed" plan didn't pan out) | 08:26 |
| Chipaca | the sleep-deprived me of 10 years ago scoffs at current-day me | 08:26 |
| thumper | :) | 08:30 |
| === Guest84896 is now known as psydroid | ||
| === narindergupta is now known as narinderguptamac | ||
| Eryn_1983_FL | hi guys | 14:24 |
| Eryn_1983_FL | so i am trying to figure out what logs i need to pull for ssl cert issues on openstack/juju | 14:24 |
| Eryn_1983_FL | barbican/1* waiting idle 0/lxd/7 10.3.251.11 9311/tcp,9312/tcp 'shared-db' incomplete | 14:24 |
| Eryn_1983_FL | barbican-vault/11* blocked idle 10.3.251.11 'secrets-storage' missing | 14:24 |
| Eryn_1983_FL | i think something is wrong with the vault | 14:24 |
| hml | Eryn_1983_FL: try “juju debug-log —include unit-barbican-1 —include unit-barbican-vault-1 —replay” these are the logs for those 2 units | 14:26 |
| Eryn_1983_FL | ty | 14:26 |
| Eryn_1983_FL | juju debug-log —include unit-barbican-1 —include unit-barbican-vault-1 —replay | 14:27 |
| Eryn_1983_FL | ERROR unrecognized args: ["—include" "unit-barbican-1" "—include" "unit-barbican-vault-1" "—replay"] | 14:27 |
| hml | Eryn_1983_FL: sorry… things got auto corrected on me. the include and replay args use 2 dashes. | 14:29 |
| Eryn_1983_FL | ok | 14:29 |
| Eryn_1983_FL | https://paste.debian.net/1153329/ | 14:31 |
| Eryn_1983_FL | https://paste.debian.net/1153329/ | 14:32 |
| hml | Eryn_1983_FL: hrm.. just those 53 lines? what are your logging levels set to? “juju model-config logging-config” | 14:34 |
| hml | was there anything for barbican-vault? | 14:34 |
| Eryn_1983_FL | thats all that came out | 14:34 |
| Eryn_1983_FL | juju model-config logging-config | 14:34 |
| Eryn_1983_FL | <root>=WARNING | 14:35 |
| Eryn_1983_FL | no | 14:35 |
| Eryn_1983_FL | i dont see it | 14:35 |
| hml | Eryn_1983_FL: can we change it to “juju model-config logging-config=`<root>=WARNING;<unit>=DEBUG`” ? | 14:36 |
| Eryn_1983_FL | i can try.. | 14:36 |
| Eryn_1983_FL | back ticks? | 14:36 |
| hml | Eryn_1983_FL: yes, or quotes work too | 14:37 |
| Eryn_1983_FL | got it | 14:37 |
| Eryn_1983_FL | maybe its been too long and the errors were last week when we done it | 14:38 |
| hml | Eryn_1983_FL: unfortunately that will only change for logs moving forward… has the charm with the relation for “shared-db” deployed? | 14:39 |
| hml | what’s the db’s status? | 14:39 |
| Eryn_1983_FL | barbican/1* waiting idle 0/lxd/7 10.3.251.11 9311/tcp,9312/tcp 'shared-db' incomplete | 14:39 |
| Eryn_1983_FL | i dont think so | 14:39 |
| Eryn_1983_FL | https://paste.debian.net/1153330/ | 14:40 |
| Eryn_1983_FL | can we rip out and start over? | 14:41 |
| hml | Eryn_1983_FL: looks like the db is in an error state. | 14:41 |
| Eryn_1983_FL | the vault and barbican | 14:41 |
| Eryn_1983_FL | yeah | 14:41 |
| hml | Eryn_1983_FL: i’d get the db happy - then see what happens with barbican | 14:42 |
| Eryn_1983_FL | ok | 14:43 |
| rick_h | petevg: did you bootstrap to the cluster? I'm failing to bootstrap at "Contacting controller to verify accessibility" and wonder what I've missed | 14:50 |
| jamespage | Eryn_1983_FL: is barbican directly related to mysql-innodb-cluster? it really should be related via a mysql-router subordinate | 14:57 |
| jamespage | that might be the cause of the shared-db-relation-departed errors | 14:58 |
| Eryn_1983_FL | not sure | 15:01 |
| Eryn_1983_FL | he just restarted | 15:01 |
| Eryn_1983_FL | barbican-mysql-router?? | 15:02 |
| Eryn_1983_FL | ill see what happens after the reinstalls | 15:05 |
| Eryn_1983_FL | https://paste.debian.net/1153341/ | 15:39 |
| Eryn_1983_FL | ok guys how do i fix the certs | 15:40 |
| Eryn_1983_FL | vault is initalized and i got the keys | 15:40 |
| Eryn_1983_FL | https://paste.debian.net/1153347/ | 15:56 |
| Eryn_1983_FL | https://paste.debian.net/1153349/ | 15:56 |
| rick_h | petevg: what's your az --version output looking like? | 15:56 |
| petevg | rick_h: 2.5.1 * | 15:57 |
| petevg | Full paste: | 15:57 |
| petevg | https://paste.ubuntu.com/p/cCjyqBCV3J/ | 15:57 |
| rick_h | petevg: ok, far ahead of me even on the edge snap then. Let me try all this with your version. You confirmed you got it from the deb link in your email? | 15:58 |
| rick_h | geeze, nothing like pipe to bash... | 16:00 |
| petevg | rick_h: I don't think that I got it w/ the deb link. I'm actually not sure what I did to install :-/ | 16:01 |
| rick_h | petevg: ok, well getting 2.7 from the deb link now so will try that | 16:02 |
| Eryn_1983_FL | are there juju ovn docs i can use that are for this kind of deployment? | 16:02 |
| Eryn_1983_FL | forum? | 16:02 |
| Eryn_1983_FL | discourse? | 16:02 |
| hml | jamespage: ^^ | 16:07 |
| Eryn_1983_FL | anyideas what i should do rick_h ? | 16:13 |
| rick_h | Eryn_1983_FL: what to do about using vault? I'm not familiar with driving that tbh. | 16:16 |
| Eryn_1983_FL | driving? | 16:16 |
| Eryn_1983_FL | allright who do i gotta murder to get this working right, | 16:17 |
| rick_h | Eryn_1983_FL: configuring, managing, etc. Looking at the traceback to see what the ask is. You mention having keys? | 16:17 |
| Eryn_1983_FL | yeah | 16:17 |
| Eryn_1983_FL | vault is open we got keys | 16:17 |
| Eryn_1983_FL | how do i resolve the certificate missing error | 16:17 |
| rick_h | Eryn_1983_FL: this is maintained by the openstack team. They've written the charms and know how to hold them right to dance for you. | 16:17 |
| Eryn_1983_FL | network wont build yet | 16:17 |
| rick_h | Eryn_1983_FL: that looks like a missing relation | 16:17 |
| rick_h | Eryn_1983_FL: juju status --relations | 16:18 |
| rick_h | Eryn_1983_FL: do you see a relation on barbican-vault and vault? | 16:18 |
| Eryn_1983_FL | Relation provider Requirer Interface Type Message | 16:18 |
| Eryn_1983_FL | barbican-mysql-router:shared-db barbican:shared-db mysql-shared subordinate | 16:18 |
| Eryn_1983_FL | barbican-vault:secrets barbican:secrets barbican-secrets subordinate | 16:18 |
| rick_h | Eryn_1983_FL: oh, but your vault is in an error state "hook failed: update-status" | 16:18 |
| rick_h | Eryn_1983_FL: so you have to mark it resolved and get it to kick again | 16:18 |
| rick_h | Eryn_1983_FL: juju resolve vault/0 | 16:19 |
| Eryn_1983_FL | ok | 16:19 |
| rick_h | Eryn_1983_FL: might have to look at the log file on vault to see why update-status is erroring | 16:19 |
| Eryn_1983_FL | juju resolve vault/0 | 16:19 |
| Eryn_1983_FL | ERROR unit "vault/0" is not in an error state | 16:19 |
| rick_h | Eryn_1983_FL: ok, looking at your paste, is it ready now then? | 16:19 |
| Eryn_1983_FL | here let me make anew one | 16:20 |
| Eryn_1983_FL | https://paste.debian.net/1153354/ | 16:20 |
| Eryn_1983_FL | vault says read and active. | 16:21 |
| Eryn_1983_FL | ready | 16:21 |
| rick_h | Eryn_1983_FL: ok, good there. I do see this though | 16:21 |
| rick_h | barbican-mysql-router/0* error idle 10.3.251.28 hook failed: "install" | 16:21 |
| rick_h | Eryn_1983_FL: so have to check out why that's unhappy | 16:21 |
| Eryn_1983_FL | pulling logs | 16:22 |
| Eryn_1983_FL | would that prevent the ovn certs from being there | 16:23 |
| rick_h | Eryn_1983_FL: no idea on that one. That's the question for the openstack folks. They have a community setup for the charms that would be good to check with. Let me get the link | 16:23 |
| Eryn_1983_FL | ok | 16:23 |
| Eryn_1983_FL | thanks | 16:23 |
| Eryn_1983_FL | juju debug-log --include barbican-mysql-router-0 --replay is this suppose to take forever? | 16:24 |
| rick_h | https://docs.openstack.org/charm-guide/latest/find-us.html | 16:24 |
| Eryn_1983_FL | thank rick we been at this for months | 16:24 |
| Eryn_1983_FL | ty | 16:24 |
| rick_h | Eryn_1983_FL: hmm, I'd suggest just juju ssh barbican-mysql-router/0 | 16:24 |
| rick_h | Eryn_1983_FL: and from there less /var/log/juju/unit<tab> | 16:24 |
| rick_h | and it'll complete the unit name and get you into the unit log | 16:24 |
| * rick_h is old school and wants to see the log file for himself I guess | 16:24 | |
| Eryn_1983_FL | hehe | 16:25 |
| Eryn_1983_FL | rick_h is teaching me the ways of the kungu fu of juju | 16:26 |
| Eryn_1983_FL | https://paste.debian.net/1153356/ | 16:26 |
| rick_h | Eryn_1983_FL: lol you're getting the same thing I am on a k8s install right now | 16:26 |
| rick_h | mystery error | 16:27 |
| rick_h | Eryn_1983_FL: what version of Juju? | 16:27 |
| Eryn_1983_FL | :( | 16:27 |
| * rick_h hopes you saying 2.8.0 | 16:27 | |
| rick_h | petevg: ^ same error, on a MAAS, 2.8.0 | 16:27 |
| Eryn_1983_FL | juju version | 16:27 |
| Eryn_1983_FL | 2.8.0-bionic-amd64 | 16:27 |
| Eryn_1983_FL | fml | 16:27 |
| Eryn_1983_FL | bugs? | 16:27 |
| rick_h | manadart_: achilleasa anyone around? this is kinda :( | 16:27 |
| rick_h | Eryn_1983_FL: I've been working to pull notes together to file one today. | 16:27 |
| Eryn_1983_FL | ok | 16:28 |
| Eryn_1983_FL | well im going to go fight with openstack peeps, | 16:28 |
| rick_h | Eryn_1983_FL: ok, well good news you're not alone and I don't think this is you. Bad news is I don't know why we're getting this error and have to chase down smarter folks that work on Juju to HALP! | 16:28 |
| Eryn_1983_FL | see who wins | 16:28 |
| Eryn_1983_FL | ok | 16:28 |
| rick_h | Eryn_1983_FL: lol well you can check on the certs working order, but they're not going to be able to help much with this issue | 16:29 |
| Eryn_1983_FL | ovn vs openvswitch? | 16:29 |
| Eryn_1983_FL | boss is wanting to put in openvswitch instead | 16:29 |
| rick_h | Eryn_1983_FL: yea, they can help more with that. fnordahl is a bit of an expert there and might be able to help but we'll see | 16:31 |
| Eryn_1983_FL | ok | 16:32 |
| pmatulis | Eryn_1983_FL, you just need to pass a CA cert to vault | 16:42 |
| petevg | rick_h: catching up on the backscroll (I'm technically at lunch right now): I don't see the Juju bug? I do know that vault can be tetchy about certificates, and the OpenStack folks can help out w/ that. | 16:42 |
| petevg | Speaking of the OpenStack folks, I see that pmatulis has jumped in. Thx! | 16:43 |
| pmatulis | Eryn_1983_FL, juju run-action --wait vault/leader generate-root-ca | 16:43 |
| pmatulis | Eryn_1983_FL, please tell me what documentation you were following so i can improve | 16:44 |
| rick_h | petevg: it's the exit 1 error on install that got hit on there as well. | 16:44 |
| manadart_ | rick_h: I have failures connecting to both achive.ubuntu.com and the charm store. Failed charm deployment earlier, now I can't bootstrap. | 16:46 |
| petevg | rick_h: right. Which I think has to do w/ certs not being generated. Things fail in really bad ways if your vault isn't set up properly. (Security software is always a headache.) I'll keep an eye on the convo -- if there is a bug in Juju, I'd definitely like to get it filed and fixed. | 16:46 |
| Eryn_1983_FL | im not sure what he followed | 16:47 |
| pmatulis | Eryn_1983_FL, i see, you are helping someone else? (too much backscroll, sorry) | 16:47 |
| Eryn_1983_FL | its cool | 16:48 |
| Eryn_1983_FL | no worries | 16:48 |
| Eryn_1983_FL | he install openvswitch now | 16:48 |
| Eryn_1983_FL | great | 16:51 |
| Eryn_1983_FL | now i we got sqllite errors on octavia | 16:52 |
| Eryn_1983_FL | how do i get missing relations i need neutron-plugin and messaging | 16:54 |
| rick_h | manadart_: yea, I'm seeing the archive one now I think | 16:57 |
| rick_h | manadart_: the issue seems to be that things hang/don't get marked as an error so there's no retry/etc path really | 16:57 |
| Eryn_1983_FL | anybody know where octavia stores it sqlite db | 17:07 |
| jamespage | Eryn_1983_FL: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-ovn.html is the general deployment guide for the charms (that section covers ovn) | 17:23 |
| jamespage | octavia - https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-octavia.html | 17:24 |
| jamespage | apologies - you caught me at me end of day | 17:24 |
| jamespage | I see pmatulis is helping as well | 17:24 |
| Eryn_1983_FL | i think we are going to try this doc | 17:28 |
| Eryn_1983_FL | https://jaas.ai/openstack-lxd/bundle/1 | 17:28 |
| Eryn_1983_FL | and see if that helps us solve some issues, | 17:28 |
| Eryn_1983_FL | sigh i swear every time we try to install something it breaks something else | 17:35 |
| Eryn_1983_FL | now the networks are down on two machines | 17:36 |
| Eryn_1983_FL | ffs | 17:36 |
| rick_h | kwmonroe: ping, when you get a few can I bug you please for k8s info for the demo prep? | 17:48 |
| kwmonroe | ping away rick_h | 17:49 |
| rick_h | kwmonroe: https://meet.google.com/zhg-vdsi-rgi?authuser=1 I'll bring a list :) | 17:50 |
| Eryn_1983_FL | i got a wacked error | 19:25 |
| Eryn_1983_FL | raise SourceConfigError( | 19:25 |
| Eryn_1983_FL | charmhelpers.fetch.SourceConfigError: Invalid Cloud Archive release specified: bionic-train on this Ubuntuversion (focal) | 19:25 |
| Eryn_1983_FL | for ovn | 19:25 |
| josephillips | hey | 19:25 |
| josephillips | someone know what is the cicle of the charms i mean what is the charm i cna download normaly from juju | 19:26 |
| josephillips | in github from stable? | 19:26 |
| josephillips | stable/year.month right? | 19:26 |
| Eryn_1983_FL | hi guys we are trying to do https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/rocky/app-certificate-management.html | 19:36 |
| Eryn_1983_FL | https://paste.debian.net/1153390/ | 19:37 |
| petevg | josephillips: I'm not sure what you're asking? | 19:43 |
| petevg | You can visit the charmstore at https://jaas.ai/store | 19:43 |
| petevg | If you want to deploy a charm on a machine from a specific ubuntu release, you can do that by adding the --series flag to the juju deploy command. | 19:44 |
| petevg | If you want to deploy a specific charm revision, you do so by specifying it like so: juju deploy cs:<charm>-<revision> | 19:45 |
| tvansteenburgh | hello juju friends | 19:47 |
| tvansteenburgh | i've got a bundle where i deploy some stuff "to: [lxd:0, lxd:1]" etc... | 19:48 |
| tvansteenburgh | when the dust settles i've got one lxd with both and ipv4 and ipv6 address, and the rest have ipv6 addresses but no ipv4 | 19:49 |
| tvansteenburgh | is there a way to tell juju, "hey, give all my lxds an ipv4 address" | 19:49 |
| rick_h | tvansteenburgh: :/ since Juju normally hates that ipv6 on lxd that's surprising. Is the host machine setup with some custom lxd config? | 20:07 |
| tvansteenburgh | rick_h: no, nothing custom. it's a machine newly provisioned by maas | 20:07 |
| tvansteenburgh | rick_h: the machines *are* getting both ip4 and 6 addresses | 20:08 |
| tvansteenburgh | which is fine, i'm just not sure what's happening with lxd | 20:08 |
| rick_h | kwmonroe: do we have any default alerts/dashboards for graylog that are interesting? | 20:29 |
| Eryn_1983_FL | hey guys how do i get the ca private key from openstack | 20:42 |
| kwmonroe | rick_h: unfortunately we don't have any default dashboards for graylog. we leave it completely up to the user as to what log they choose to care about. that said, graylog's UI is pretty good for drilling down. the System->Inputs->Show Recvd Msgs gives you a host of fields that you can drill on. It's not a dashboard, but it's still pretty powerful ootb. | 21:00 |
| rick_h | kwmonroe: yea, all good just checking | 21:02 |
| rick_h | petevg: lol well found the memory limits on kvm pods on guimaas | 21:06 |
| rick_h | 21 deployed machines and it's cranky now | 21:06 |
| hml | rick_h: that’s it? :-D | 21:11 |
| rick_h | hml: well a few are hardware, but 15 vms nuc4 cried uncle and said no more | 21:12 |
| rick_h | but my juju status is now scrolling off the terminal so I just got it going well enough lol | 21:13 |
| kwmonroe | rick_h: tvansteenburgh, can you come back to me in https://meet.google.com/zhg-vdsi-rgi?authuser=1 re: dashboards | 21:23 |
| rick_h | kwmonroe: yea omw | 21:31 |
| tlm | very tiny PR if anyone has two seconds https://github.com/juju/juju/pull/11742 | 22:09 |
| kwmonroe | rick_h: remove-unit --force doesn't mess around. | 22:39 |
| kwmonroe | s/unit/machine | 22:39 |
| rick_h | kwmonroe: yea, it's pretty brutal | 23:50 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!