[19:34] <holmanb> caribou / meena / minimal: Thanks all for discussion last week about this cloud networking requirement we have.
[19:34] <holmanb>  A couple of thoughts:
[19:34] <holmanb> 1. "bring your own IP" is a fairly common cloud requirement
[19:35] <holmanb> 2. some clouds have decided that the implementation of this requirement involves assigning the public IP directly to an instance's interface (I'm curious if any clouds diverge from this approach)
[19:37] <holmanb> Considering 1 and 2, this requirement is probably a higher priority than my first glance afforded.
[19:40] <Teridon> I'd appreciate some advice re: workarounds for this bug :  https://bugs.launchpad.net/cloud-init/+bug/1999164   .  Should I just use late-commands to write the extra lines manually? 
[19:40] -ubottu:#cloud-init- Launchpad bug 1999164 in cloud-init "when multiple SSH host key certificates are defined, only one HostCertificate is referenced in sshd_config" [High, Confirmed]
[19:44] <holmanb> Teridon: The workaround for that cloud-init bug falls into subiquity territory, so this probably not the best place to ask.
[19:45] <holmanb> Teridon:  However, based on a quick google of what late-commands are I'll go so far as to say: "that's what I'd try first".
[19:45] <Teridon> holmanb:  oh that make sense, my mistake.  I'll ask there.  Thanks
[19:49] <minimal> holmanb: in terms of "bring your own IP" I've only seen that mentioned in the context of someone having their own PI address space and using BGP to advertise it
[19:49] <Teridon> do you have the channel name?  Apparently it's not #subiquity? 
[19:50] <holmanb> Teridon: recent addition to the FAQ page ;) https://cloudinit.readthedocs.io/en/latest/topics/faq.html
[19:50] <holmanb> #ubuntu-server
[19:51] <Teridon> ty
[19:51] <holmanb> np
[19:55] <falcojr> Teridon: heh, sorry for bouncing you back and forth, but yes, I'd think that using a runcmd for now should be able to achieve that
[19:57] <falcojr> /etc/ssh/sshd_config.d/50-cloud-init.conf should contain "HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub", and you could add the other ones manually in the mean time (assuming nothing else is going on with that file)
[19:59] <Teridon> so if I use a late-command to edit 50-cloud-init.conf to add those lines, when the system is rebooted (after the install), it will re-write sshd_config ? 
[20:00] <blackboxsw> alternative #3:
[20:00] <blackboxsw> write_files:
[20:00] <blackboxsw> - path: /etc/ssh/sshd_config/50-cloud-init.conf
[20:00] <blackboxsw>   defer: true
[20:00] <blackboxsw>   append: true
[20:00] <blackboxsw>   content: |
[20:00] <blackboxsw>     HostCertificate /etc/ssh/ssh_host_ecdsa_key-cert.pub
[20:00] <blackboxsw> alternative #3:
[20:00] <blackboxsw> write_files:
[20:00] <blackboxsw> - path: /etc/ssh/sshd_config/50-cloud-init.conf
[20:00] <blackboxsw>   defer: true
[20:00] <blackboxsw>   append: true
[20:00] <blackboxsw>   content: |
[20:00] <blackboxsw>     HostCertificate /etc/ssh/ssh_host_ecdsa_key-cert.pub
[20:00] <blackboxsw>     HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub
[20:00] <blackboxsw> sorry late and bad paste twixce
[20:01] <holmanb> minimal: True. I guess any implementation that supports public addresses directly on the instance interface might do something like this.
[20:01] <falcojr> if it works like the other .d dirs, it'll merge in the .d contents when the service starts. That's not like a cloud-init specific dir or anything
[20:01] <Teridon> yeah sorry my brain was stuck on cloud-init and I failed to see you were just right in the sshd_config.d dir.  
[20:02] <blackboxsw> better paste here if you are using autoinstall: https://paste.opendev.org/show/bpJ4w4SbZtrHIaYOHmsn/
[20:03] <blackboxsw> so the runcmd falcojr referenced would be under an  autoinstall:user-data:runcmd  key path in the original YAML referenced in that bug.
[20:03] <blackboxsw> same w/ a write_files:  key if you chose to use that
[20:04] <Teridon> awesome!  ty!
[20:05] <blackboxsw> and/or just write a separaete /etc/sshd_config.d/60-your-extra-host-overrides.conf
[20:05] <blackboxsw> the `defer: true` for write_files ensures it is late enough in cloud-init to write things after ssh config is already generated
[20:06] <blackboxsw> the `defer: true` for write_files ensures it is late enough in cloud-init to write things after ssh config is already generated. or `runcmd` is late enough by default 
[20:07] <Teridon> hmm. does that "content" line have a dash? "- content: | "  ?  that's what I see from the example at https://cloudinit.readthedocs.io/en/latest/topics/examples.html 
[20:08] <Teridon> * does==should
[20:11] <Teridon> nvm; I see I'm reading that wrong
[20:11] <blackboxsw> Teridon: no worries the content: |     indicates to YAML that it's a multi-line value below
[20:11] <minimal> holmanb: what Hetzner seemed to be doing is using their (rather than the customer's) public address space but, in order to reduce on IP address overhead (as a typical subnet approach means 1 IP is "wasted" for a gateway), they seem to be using a PtP-like approach combined with private addresses (from memory) for the gateways
[20:13] <blackboxsw> Teridon: BTW the paste I put up wasn't tested, but I **think** it should work.. you might have to specify `permissions: "0600"` in the write_files if that's what you are using
[20:14] <Teridon> blockboxsw:  I think content should be indented.  And I don't see the "append:" attribute documented
[20:15] <holmanb> minimal: +1 -> and it looks like scalaway has a similar approach, but uses a public default gateway (same public IP as the one caribou dropped in the example above)
[20:16] <Teridon> blackboxsw: ok I found docs for append https://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files 
[20:21] <minimal> holmanb: from a quick search regarding Scaleway I note "You can configure a unique gateway for your virtual machines. In doing so, your VMs can move between your hypervisors without changing network configuration of the virtual machine" so perhaps in their case its a way to handle "funky" routing somehow for VMs that may move
[20:23] <minimal> they also talk about /32 failover IPs
[20:24]  * holmanb nods
[20:29] <blackboxsw> Teridon: a quick way to test your user-data locally if you have lxc installed: https://cloudinit.readthedocs.io/en/latest/topics/tutorials/lxd.html.     If you just popped off top-level `autoinstall:user-data` keys and wrote a `#cloud-config` file your-data.yaml with the top-level keys timezone,disable_root, ssh_keys, ssh_getkeytypes etc an lxc launch ubuntu-daily:focal -c user.user-data="$(cat your-data.yaml)"   you can check the lxc
[20:49] <Teridon> blackboxsw:  tyvm!  I didn't know that (lxd) existed!  That will greatly reduce my test time!
[20:57] <blackboxsw> yeah does for us too  :)
[21:10] <blackboxsw> minor PR to bump pycloudlib integration test dependencies to fix support for systems without azure-cli installed https://github.com/canonical/cloud-init/pull/1908
[21:10] -ubottu:#cloud-init- Pull 1908 in canonical/cloud-init "Pycloudlib bump" [Open]
[21:42] <Teridon> the sage continues:  I used the "write_files" stanza suggested, and it does indeed seem to create the sshd_config.d/foo.conf file on my (autoinstall) test system.  However, for some reason, I had to restart sshd manually before it actually *used* that config.   If I didn't restart sshd, only the RSA cert was served.  
[21:43] <Teridon> sage==saga if only I could type correctly
[21:49] <Teridon> would a "runcmd:" to restart sshd be run after the write_files part ? 
[21:54] <falcojr> hmmm, runcmd actually comes before write_files_deferred
[21:55] <falcojr> you can see the order in /etc/cloud/cloud.cfg
[21:59] <Teridon> well that's funny because I just tested adding runcmd to do a service restart and that *seemed* to work.  
[22:02] <Teridon> side-note: This is what I use to test the server host certificates:  https://gist.github.com/teridon/b25c9041dc3486b3fa33a5dbbb3e1b03