[12:21] <stickupkid> this is interesting, never spotted this in github - https://github.com/juju/python-libjuju/network/dependents?package_id=UGFja2FnZS01MjI0MzQ2NA%3D%3D
[12:21] <stickupkid> see whom is a dependant on pylib in github
[12:27] <rick_h> hah that's cool
[12:27] <rick_h> though seems odd layers are depending on it. Maybe testing/etc?
[14:51] <achilleasa> did we make any watcher-related changes to develop? I am getting lots of random watcher-related test failures on CI
[15:01] <rick_h> achilleasa:  lots of thumper stuff around watchers?
[15:02] <rick_h> stickupkid:  what's up with the root job build queue listing the nw-deploy-eoan-amd64-lxd a ton of times? Any ideas?
[15:03] <stickupkid> wow
[15:03] <rick_h> stickupkid:  yea, cancelling them up right now
[15:04] <stickupkid> looks like this is 2.7 issue
[15:04] <stickupkid> no wait, I'm wrong
[15:04] <rick_h> ok, cleaned those out...
[15:05]  * rick_h moves to internal chat
[16:58] <skay> I added a postgresql unit after ripping out the old one and this one's status message is 'Installing wal-e snap' and it's been like that for quiet a while now
[16:58] <skay> how do I get it unstuck?
[17:12] <achilleasa> rick_h: for the remote-lxd use-case can we assume that the client can _always_ talk to the remote-lxd or do we assume that only the local controller can do so (e.g. very strict fw rules)?
[17:13] <stickupkid> we want both right?
[17:13] <achilleasa> for juju login the fingerprint fetch/check happens client-side whereas for the above I could imagine that this might not always be possible (for the remote)
[17:13] <rick_h> achilleasa:  yes, in this sample case the client probably can, but not always
[17:13] <rick_h> achilleasa:  right, I think that's jam's point is that our lxd uses are very client driven
[17:13] <rick_h> achilleasa:  and so the logic isn't avilable on the server side when running add-cloud to a different controller
[17:14] <rick_h> achilleasa:  however we can't trade one use for for the other. In the end we need both
[17:14] <rick_h> achilleasa:  but it's a bit of work. I'd prefer if we identify issues, file bugs, and see what we can do to unblock folks vs biting the whole thing off
[17:14] <stickupkid> achilleasa, what rick_h said
[17:14] <stickupkid> :)
[17:14] <rick_h> achilleasa:  as I think there's a lot in there. You can see the same story with things like maas checking endpoints when you add-cloud a maas, or the same sanity checks we do with openstack when you add an openstack cloud
[17:15] <rick_h> achilleasa:  there's a fundamental design we've missed in that juju add-cloud local vs to a controller should be more in line
[17:15] <rick_h> it's just one more thing in "multi-cloud controller makes life interesting"
[17:15] <stickupkid> rick_h, achilleasa maas is client only, yeah - that was a big miss
[17:16] <rick_h> stickupkid:  achilleasa so from my PoV I'd like to identify how it should be "done right" and spec/file bugs on that vs trying to just make the lxd case work better
[17:16] <stickupkid> rick_h, achilleasa agreed, it's not just LXD as you pointed out
[17:16] <achilleasa> still investigating... for the lxd case in particular, the fix must live in the controller (in the lxd provider to be precise). however, my question is whether we could have our client try to connect to the remote (its endpoint is known to the client)
[17:17] <rick_h> achilleasa:  no, because it's no promise the controller will work from where it sits
[17:17] <rick_h> achilleasa:  this case of adding a lxd cloud to a k8s controller is the perfect example. Nothing says those two work together
[17:17] <rick_h> achilleasa:  and the client is the perfect middle man providing the "best case" vs the worst one
[17:18] <rick_h> stickupkid:  you free for a sec then if you're still around?
[17:19] <achilleasa> still, the problem (as far as this bug is concerned) lies in that lxd behaves in a certain way (jam has linked an issue where they explain why it is implemented like this) which means that we need to do a custom TLS verification step
[17:19] <rick_h> before I break CI to bits?
[17:19] <stickupkid> rick_h, yeah in daily
[17:19] <achilleasa> (just as lxc does)
[17:19] <stickupkid> rick_h, i'll trade you for writing tests in goose
[17:20] <achilleasa> happy to jump in on a HO later to explain my proposal for a solution (that may be applicable to other providers)
[17:25] <rick_h> achilleasa:  it'd be good to write up (tomorrow maybe, getting late) and that way more folks can comment/discuss
[17:25] <rick_h> I think jam, wallyworld, etc would like to think about it as well
[17:26] <achilleasa> rick_h: as a discourse post or an email?
[17:28] <rick_h> achilleasa:  either/or
[17:57] <achilleasa> argh! still cannot replicate the TLS validation issue with remote lxd. I will ping John tomorrow for more details and try with k8s
[20:03] <thumper> morning
[20:04]  * thumper waves at alexisb
[20:04] <thumper> alexisb: I hope you and family are staying healthy
[20:08] <rick_h> morning thumper