=== chris14_ is now known as chris14
[06:50] <lunatiq> whats a method to protect against DDoS on ubuntu 22.04 LTS
[07:13] <lunatiq> https://devanswers.co/how-to-get-php-mail-working-on-ubuntu-16-04-digitalocean-droplet/ It was suggested to install this to enable mail() in PHP
[07:13] <lunatiq> do I setup a domain for "internet site"
[07:54] <rfm> lunatiq, honestly, what are you trying to accomplish?  You surely do not appear to be clueful enough to run a wide-open mail server, shouldn't you just let google or somebody do it for you?  I do.
=== JanC is now known as Guest8007
=== JanC_ is now known as JanC
=== JanC is now known as Guest7254
=== JanC_ is now known as JanC
=== JanC is now known as Guest2780
=== JanC_ is now known as JanC
=== JanC is now known as Guest191
=== JanC_ is now known as JanC
=== JanC is now known as Guest803
=== JanC_ is now known as JanC
=== cpaelzer_ is now known as cpaelzer
=== JanC_ is now known as JanC
=== JanC is now known as Guest1127
=== JanC_ is now known as JanC
[16:31] <rbasak> blackboxsw: o/ on https://github.com/canonical/cloud-init/commit/8f569d45111dce368f92883e35bfbb40bbb86091, shouldn't this also be the default behaviour any time the instance-id changes, rather than being limited to a CLI command? IOW, shouldn't there be a module on this, just like cc_ssh.py?
[16:31] -ubottu:#ubuntu-server- Commit 8f569d4 in canonical/cloud-init "clean: add param to remove /etc/machine-id for golden image creation"
[16:31] <rbasak> IOW, AIUI "golden image creation" should continue to be the default and automatic cloud-init behaviour - golden images shouldn't need to be special or declared as such, cloud-init allows any image to be a golden image without special configuration.
=== JanC is now known as Guest7146
=== JanC is now known as Guest5659
=== JanC_ is now known as JanC
=== JanC is now known as Guest6929
=== JanC_ is now known as JanC
[17:32] <blackboxsw> rbasak: instance-id changes could be used to indicate some smaller config change such as network config has changed due to new routes being added or a device hotplug event due to LXD system config changes or any number of cloud-native config changes. At some point instance-id changes will indicate whether  storage is added/removed. Some of these cases don't warrant rewriting /etc/machine-id (it's a pretty big hammer).
=== JanC is now known as Guest3813
=== JanC_ is now known as JanC
[17:38] <blackboxsw> This specific corner case which warranted this parameter was someone manually preparing an image installed via the Ubuntu Desktop Live installer and Server Live installer (subiquity-based) for re-use as a golden image in other cloud platforms (VMWare). Specific to the reused of the live installer, the 'golden image' may be running on bare metal or an env with no instance metadata, or inability to change instance-id in config.
[17:39] <blackboxsw> In those cases, we suggest people redact /etc/machine-id to avoid surprises on cloned image boot so allow systemd to take care of that setup.
[17:42] <rbasak> blackboxsw: isn't the hammer already used with ssh host key regeneration?
[17:42] <rbasak> My impression was that it's a fundamental design principle of cloud-init that every image is a golden image. Is that no longer the case?
[17:43] <rbasak> I get that running outside a cloud environment that can identify to cloud-init that it's a clone will require manual intervention.
[17:44] <rbasak> I'd just like to see the behaviour when it *is* in such an environment not disappear.
[17:44] <blackboxsw> that said, I believe most cloud images already handle /etc/machine-id configuration across the "clone my vm" operation to ensure that machine-id is regenerated across clone image reboot.
[17:44] <rbasak> What's the mechanism used for that? That's what I couldn't find.
[17:44] <rbasak> So I assumed that it's not happening, but perhaps I'm wrong?
[17:45] <blackboxsw> blackboxsw: isn't the hammer already used with ssh host key regeneration? Yes true, when instance-id delta is triggered by IMDS, hostkeys are regen'd and the instance is considered new and configured as such.
[17:48] <blackboxsw> in cases where IMDS is not longer available after reboot (sandboxing a VM, bare-metal with no longer a NoCloud or DataSourceNone configuration available, ConfigDrive no longer attached) cloud-init won't be activated at systemd-generator timeframe so this cloud-init clean --machine-id helps in that environment to allow reuse of those images across that tranansition.
[17:48] <blackboxsw> My impression was that it's a fundamental design principle of cloud-init that every image is a golden image. Is that no longer the case?   (Correct in that every first boot with new instance-id detected is a golden image that needs full configuration)
[17:52] <blackboxsw> And currently that's how LXD datasource behaves on Jammy++ for many `lxc config set` configuration changes. the instance re-runs and regenerates SSH host-keys across that instance-id delta. Though, I guess I'm trying to present cases where I don't think cloud-init should re-configure everything and reset SSH host keys, create users, set passwords. I think we'll need to define granularity there to avoid cfg re-run where unneccessary
=== JanC_ is now known as JanC
[17:55] <rbasak> blackboxsw: what you just said matches my expectation I think, but then shouldn't machine-id be reset any time ssh host keys are?
[17:56] <blackboxsw> rbasak: sorry was talking out of both sides of my mouth. The future desired case w/ scoped reaction to only subsets of changed IMDS data, vs current state of the world.
[17:59] <blackboxsw> rbasak: so, long conversation short. Yes, I think that is the case. The an instance-id delta from IMDS is currently equivalent to the IMDS indicating that this is a full new instance/machine/configuration and that all cloud-init setup needs to be performed as if it were a brand new image.
[18:00] <blackboxsw> rbasak: so yes, I think you are right, we want to build in that logic as well for EventType.NEW_INSTANCE_ID  (IMDS instance-id delta) to also clean/redact the /etc/machine-id
[18:02] <rbasak> blackboxsw: OK! Would you like a bug? :)
[18:06] <blackboxsw> rbasak: also note the above behavior of removing /etc/machine-id is actually buggy because on some non-systemd distros will regenerate /etc/machine-id when the file is absent, but most systemd environments rely on   systemd-machine-id-commit.service which depends on the presence of `ConditionPathIsMountPoint=/etc/machine-id
[18:06] <blackboxsw> `
[18:06] <blackboxsw> here's a PR I hope to land today tomorrow related to this bug https://github.com/canonical/cloud-init/pull/1953
[18:06] -ubottu:#ubuntu-server- Pull 1953 in canonical/cloud-init "machine-id: set to uninitialized to trigger regeneration on clones" [Open]
[18:07] <blackboxsw> rbasak: I think a separate bug is good thanks Robie. I'll followup with a separate fix to generalize that functionality and use it across instance-id deltas
[19:08] <lemoi> is there a way to statically IP ubuntu server 20.04.5? I'm trying to do an automated install via packer and I'm lost
[19:08] <lemoi> I need to statically IP the installation medium and then point it to meta-data + user-data
[19:11] <sarnold> lemoi: maybe this? https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html
[19:19] <lemoi> I hope it works in 20.04.x
[19:20] <lemoi> ideally I'd want to use vanilla install media and IP it through command line parameters. Is this doable?
[19:21] <sarnold> you can certainly run the ip command in your own scripts if you wish, but netplan is right there..
[21:19] <rbasak> blackboxsw: I filed bug 2003121, and also found a case where the bug is quite severe - cloud-init "golden images" don't at all on Ubuntu + cloud-init + libvirt at the moment.
[21:19] -ubottu:#ubuntu-server- Bug 2003121 in cloud-init "machine-id is not reset when instance-id changes" [Undecided, New] https://launchpad.net/bugs/2003121
[21:19] <rbasak> don't *work* at all
[21:49] <falcojr> blackboxsw: I thought we decided the other day that we DON'T want cloud-init doing golden image creation
[21:50] <falcojr> there's a lot more than just resetting the instance id / machine id, and we don't do any of it currently
[21:55] <rbasak> falcojr: my understanding was that part of the principle of cloud-init's design was to do all the things necessary. Did that change?
[21:56] <falcojr> rbasak: I'm not aware of that expectation
[21:57] <Odd_Bloke> cloud-init's instance ID behaviour enables the use of ~any image as a "golden" image, without a "creation" step.  It's not incompatible, IMO, to say cloud-init shouldn't create golden images but should reset machine-id when it detects a new instance.
[22:02] <Odd_Bloke> falcojr: To clarify: you aren't aware of the expectation that cloud-init should enable launching fresh instances from images(/instance snapshots) which have not been specifically prepared for that purpose (i.e. non-golden images), or you aren't aware of an expectation that cloud-init should support preparing golden images?
[22:03] <falcojr> Odd_Bloke: that last one...support preparing golden images
[22:03] <Odd_Bloke> (I believe the former to be a crucial part of cloud-init's utility, and I don't think any commitment to the latter has been made except that `clean` might be helpful if someone is doing that.)
[22:04] <falcojr> I agree about the former
[22:04] <rbasak> What's the difference between the two cases?
[22:04] <rbasak> In the former case, every VM that uses cloud-init is a golden image.
[22:05] <rbasak> The only benefit of a specific "make this a golden image" command is to clean up for size purposes.
[22:05] <falcojr> blackboxsw had a list of things we don't want to do the other day that I don't remember at the momemt. One example I can think of is we don't clean the journal logs on cloud-init clean
[22:05] <falcojr> (but we do clean cloud-init's logs)
[22:05] <rbasak> Or for an environment where instance-id isn't available I suppose, sorry.
[22:05] <falcojr> ((if you pass --logs))
[22:11] <blackboxsw> sry catching up. falcor, Correct per cloud-init doing golden image creation, we don't want cloud-init to boil the ocean and be "the tool" to perform golden image creation because there are a lot of config or first boot artifact to cope with outside the scope of cloud-init.
[22:12] <blackboxsw> We want cloud-init to be able to provide some semblance of tooling to help support folks who may be trying to perform some semblance of golden image creation themselves to prepare such an image for reuse as a golden image.
[22:14] <blackboxsw> cloud-init also can't easily clean up all artifacts on the system due to first boot (nor should it).   Things like user creation, ssh keys imports, host keys laid down, any runcmd or bootcmd artifacts executed during initial vm/instance  boot.
[22:15] <Odd_Bloke> rbasak: To my mind, an image is only golden if it's been specifically prepared for use as an image.  So using cloud-init enables you to launch _non-golden_ images in a similar fashion to golden images.  But if an instance has, say, Postgres installed with some databases, it's ~impossible for cloud-init to know if "golden" for that means keep the databases or trash them etc.
[22:16] <rbasak> Odd_Bloke: sure. That sounds like a difference in the definition of "golden" only.
[22:16] <rbasak> I think we agree, but to be clear, my view is that a golden image is a golden image if the person who created it says it is. cloud-init helps with that by dealing with all the pitfalls on the OS side, such as ssh host keys and (when the bug is fixed) /etc/machine-id.
[22:17] <blackboxsw> I recall end of last week we chatted about potentially starting a discussion with CPC team as they have a lot more goodness, tooling and knowledge when it comes to golden image creation required by official cloud images. The intent is to see if we might work with them to help provide a utility that may have more broad usefulness that could help assist community members interested in creating their own images .
[22:17] <rbasak> OTOH, a golden image is not a golden image if, when booted, it's broken, such as with a duplicate ssh host key or /etc/machine-id.
[22:18] <rbasak> So maybe I should correct myself and say that a golden image is a golden image if the person who created it says it is, _and_ it isn't otherwise broken like that. cloud-init's job IMHO is to ensure the latter for all cases where the OS does something that would make a clone non-golden.
[22:19] <Odd_Bloke> Right, I agree that we agree: I guess my point is that this isn't a problem with _golden_ images, it's a problem with _all_ images.
[22:20] <rbasak> Only if you clone though, right? I can't imagine a reason to clone in a non-golden case.
[22:20] <falcojr> I think there's still a distinction I'm not understanding, but I don't see how having a stale ssh host key is any more broken than a stale username and user key and cloud-init doesn't clean those up on new instance creation
[22:21] <falcojr> and I'm also not sure we intend "new instance id" == "golden image"
[22:21] <Odd_Bloke> An SSH host key is necessarily stale on a new instance, a username/key are not necessarily stale on a new instance.
[22:22] <rbasak> falcojr: a stable ssh host key is a security vulnerability. For example, a well known hosting company provided OS images that all had the same ssh host key when they launched, meaning that their customers could MITM each other.
[22:22] <falcojr> not necessarily on a new instance...in fact it's one of the biggest source of frustration for our users
[22:22] <blackboxsw> So, for cloud-init's role we can provide simple entry points that help support one-off cases for manual image creation and reuse. If someone is using cloud-init and already knows they want to trigger cloud-init to manually re-run 'from scratch' on next boot, they are already told to run `sudo cloud-init clean --logs`.
[22:22] <falcojr> they get assigned a new instance id when the machine didn't change and now they have new host keys
[22:22] <rbasak> OTOH, a known user with a given authorized_keys might be _intentionlly_ part of a golden image.
[22:23] <sarnold> just in case [redacted] needs to debug an ubuntu-core system in the field? :)
[22:23] <rbasak> If the instance id changed but the user didn't intend the machine to change, then perhaps instance id from that particular cloud provider is the wrong identifier to use to detect when the machine has been cloned, but the principle remains.
[22:24] <rbasak> sarnold: :)
[22:24] <rbasak> sarnold: but seriouly, within a particular org it might be a perfectly reasonable thing to do.
[22:24] <blackboxsw> If they want to ensure that image will come up with an unique machine-id, it's easy to extend cloud-init for that element of "preparing your image" but cloud-init will probably not grow a full suite of features to tackle all  systemd recommendations for https://systemd.io/BUILDING_IMAGES/
[22:24] <sarnold> *nod*
[22:25] <rbasak> blackboxsw: I think maybe it should
[22:26] <rbasak> I haven't reviewed each of those in detail, but the principle behind cloud-init's design is that it is supposed to take care of those things, AIUI.
[22:26] <rbasak> The principle being that if its identity has changed, it's a clone and it should adapt correctly.
[22:28] <rbasak> If instance-id changes when network config changes, that's either a) wrong; or b) an environment where instance-id is no longer an indicator of identity, so cloud-init cannot work as designed any more unless that's provided some other way.
[22:28] <rbasak> I mean: "instance identity" should be a clue :-P
[22:29] <rbasak> "This instance has a different identity" -> "OK, so the ssh host keys should be different then" seems entirely reasonable.
[22:29] <rbasak> And same for /etc/machine-id, etc.
[22:31] <falcojr> rbasak: that's not how it works today, and I think that would need to be a larger discussion
[22:31] <falcojr> I don't see why cloud-init couldn't grow that functionality, but I disagree that that is the principle behind cloud-init's design
[22:31] <falcojr> (unless smoser wants to pop in and correct me XD )
[22:33] <Odd_Bloke> `/var/lib/cloud/instances` exists for this reason: so that the same filesystem can be used as multiple different instances, as identified by the underlying cloud platform.
[22:36] <blackboxsw> rbasak: I think the broader conversation of whether  cloud-init needs to prune things well beyond the scope of cloud-init needs to be had w/ CPC and Foundations before cloud-init signs up for that level of feature/scope for "golden image" preparation.  Maybe this is something that needs broader adoption as a separate tool/script delivered in ubuntu-dev-tools?  Also, are there images which don't have cloud-init installed that would 
[22:36] <blackboxsw> need a more broad tool/utility to support golden image setup?
[22:38] <rbasak> blackboxsw: for the "every image with cloud-init should be able to safely and correctly boot with a new identity" use case, your statement sounds odd to me because I've taken that to *be* the scope of cloud-init all along.
[22:39] <rbasak> That doesn't include cleaning everything, to be clear. Just the things that would be broken to carry forward, such as security keys, machine-id, etc.
[22:40] <rbasak> For images that don't have cloud-init installed, my answer would be "use cloud-init; that's what it's for".
[22:52] <blackboxsw> rbasak falcojr, Odd_Bloke I'm not sure if we are talking past each other or maybe falcojr responded to an earlier aspect of the discussion when he said "I think that would need to be a  larger discussion").   Do we think that instance-id delta from a cloud datasource is sufficient to indicate that this is a new system "enough" that we don't want cloud-init to also uninitialize machine-id when instance-id changes?
[22:54] <blackboxsw> the case I'm wondering about is services like Canocial livepatch  or Ubuntu Pro which may take into account the current machine-id on the filesystem, if there are clones launched from that filesystem (which would receive a new instance-id) I'd potentially like cloud-init to perform the work to purge/regenerate that machine-id during boot to ensure other services which key on /etc/machine-id are aware of such an identity change
[22:55] <rbasak> blackboxsw: I'm not sure I follow your question. I think that if instance-id changes, machine-id should be reset, although with anything else needed.
[22:56] <rbasak> If there's a case where instance-id changes where you don't think that should happen, then I'd like to hear more about that case.
[22:56] <rbasak> *along* with anything else needed
[23:02] <blackboxsw> I think when an instance-id changes in metadata cloud-init loks at, host keys should be regenerated (already done), and I do think we should think about regenerating machine-id across that boundary.  Your "anything else needed" comment I'd like to limit to only things attributed strictly to identity of that host. 
[23:07] <rbasak> blackboxsw: from your link, would you include /var/lib/systemd/random-seed and /var/lib/systemd/credential.secret?
[23:18] <blackboxsw> rbasak: I think this is asking a lot that cloud-init would have to grow into if we decide it's the right place to do this. Ideally I'd want cloud-init to invoke an  external tool or script to perform this "image clean" there was a product cpaelzer/holmanb referenced virt-sysprep as a tool for performing 'full clone image creation' and this really can only be done safely on an inert image with the host stopped.
[23:24] <blackboxsw>  https://www.libguestfs.org/virt-sysprep.1.html#copying-and-cloning as a point of reference.
[23:28] <blackboxsw>  But, cloud-init shouldn't be that broad a utility at runtime to ensure we fix  everything related to a hosts identity, cleanse various service logs, etc by default. I think we *can* do a best effort in scope of cloud-init where there is friction because a cloud is telling us via instance-id that identity needs to change. It would be nice though, if cloud-init could leverage some external tool in the OS itself.
[23:29] <blackboxsw> I want to note as well that pulling in libguestfs-tools  as a dependency of cloud-init is a non-starter due to the sheer number of dependencies it also pulls in
[23:29] <rbasak> blackboxsw: cleaning service logs is scope creep. I'm not asking for that.
[23:30] <blackboxsw> +1 thx
[23:30] <blackboxsw> yet so is /var/lib/systemd/random-seed and /var/lib/systemd/credential.secret :)
[23:30] <rbasak> I agree that's work, but I think they fit in the same category as ssh host keys.
[23:31] <blackboxsw> yes, I think it's worth a spec for broader discussion w/ CPC/Foundations too. We'll tag you on it and can reflect the conversation on that here too for others.
[23:33] <blackboxsw> I think we have to be certain what we think cloud-init should automatically be in charge of as far across instance-id delta because this is processed after reboot, not before reboot so certain services may be in flight trying to consume those artifacts that cloud-init is changing out from under them
[23:34] <blackboxsw> the machine-id uninitializing is something safely performed prior to the eventual reboot of that machine image, so we aren't pulling the rug out from under any service which may have just started
[23:34] <rbasak> Yeah the DHCP case is an interesting one
[23:34] <rbasak> With networkd using machine-id to derive the DHCP client identifier.
[23:35] <blackboxsw> sorry to clarify: the cloud-init clean --machine-id cmd is performed pre-reboot. So it's safer than at runtime
[23:35] <blackboxsw> right on DHCP
[23:35] <rbasak> That's not the use case I'm interested in though.
[23:35] <rbasak> I want the "just boot it and cloud-init will do the right thing" use case to work, without a pre-boot step.
[23:40] <falcojr> That category feels very squishy to me. What makes a new instance broken? Ask 10 different people and you'll probably get 10 different answers. 
[23:45] <rbasak> Like all bugs :)