| bswinnerton | Hello, first time playing around with cloud init. Things are going great in Ubuntu, but having some problems configuring networking in Debian. I'm using a nocloud database with a v2 network configuration file to set a static ip address. The user-data and meta-data is applied successfully, but after the machine comes up, it's falling back to DHCP. | 12:21 |
|---|---|---|
| bswinnerton | Upon further inspection, this is because /etc/network/interfaces does a source-directory for both /etc/network/interfaces.d/ (which has the correctly configured file), and /run/network/interfaces.d/ which has a file that conflicts and falls back to DHCP. Is there a way to turn off the latter? | 12:21 |
| bswinnerton | s/database/datastore | 12:25 |
| bswinnerton | hmm, there's also something weird going on similar to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867921 | 12:36 |
| ubot5 | Debian bug 867921 in openstack-debian-images "cloud-init: Network configuration doesn't work" [Normal,Fixed] | 12:36 |
| bswinnerton | So to recap, when using nocloud with a v2 network config that sets a static ip address, when the Debian host comes up, a few things need to happen in order for the machine to get networking: 1) `rm /run/network/interfaces.d/50-cloud-init.cfg` (which includes the DHCP setting), and 2) `mv /etc/network/interfaces.d/50-cloud-init.cfg | 12:38 |
| bswinnerton | /etc/network/interfaces.d/50-cloud-init`, once those have been completed, a `systemctl restart networking` can be run and everything works fine. | 12:38 |
| bswinnerton | This is cloud-init 18.3 | 12:39 |
| meena | bswinnerton: have you tried a more recent version to see if this is fixed? | 13:46 |
| bswinnerton | I haven't, I'm using what's built into the debian cloud images | 13:47 |
| bswinnerton | In what scenarios would cloud-init write to `/run/network/interfaces.d/`? Is that the fallback described in https://cloudinit.readthedocs.io/en/latest/topics/network-config.html#fallback-network-configuration? | 13:56 |
| meena | I'm trying to remember if we have deb packages pre-built anyhere | 13:57 |
| meena | my brain isn't working very well today, so you might be quicker looking yourself | 13:58 |
| bswinnerton | https://packages.debian.org/buster/cloud-init | 13:59 |
| bswinnerton | They're quite old, which is why the debian images include them, I assume | 13:59 |
| meena | no backports? | 14:00 |
| bswinnerton | ah yes: https://packages.debian.org/source/buster-backports/cloud-init | 14:00 |
| meena | there's https://launchpad.net/ubuntu/+source/cloud-init but i don't know how well those will work on Debian | 14:00 |
| meena | bswinnerton: so let's try at version, and if that still fails, we'll file a bug report | 14:03 |
| meena | and by we, i mean, can you please | 14:03 |
| meena | s/at /that / | 14:04 |
| bswinnerton | Sounds good, installing 20.2 now | 14:05 |
| bswinnerton | Is there an effective way to test this with cloud-init on the guest without rebuilding the image? | 14:07 |
| bswinnerton | Or does the image need to be rebuilt? | 14:07 |
| meena | install, run cloud-init clean --logs --reboot | 14:07 |
| bswinnerton | Neat, wasn't aware of `clean`. Doing that now | 14:08 |
| meena | that will regenerate everything on the server, including ssh server keys | 14:08 |
| bswinnerton | Sigh, that fixed it | 14:10 |
| bswinnerton | Okay, off to pester the Debian folks. Thanks meena | 14:11 |
| meena | 😽 | 14:11 |
| meena | if i really were useful, I'd help you pinpoint where that was fixed, exactly | 14:12 |
| meena | but screw that, let them upgrade to the latest version :P | 14:12 |
| bswinnerton | A quick clarifying question, even after bumping to 20.2 I notice a very long delay when trying to bring up the interface, it appears that it's trying to get a DHCP lease with "Started ifup for ens3". It then eventually times out and continues along, and sets the static IP. Any idea how I could tell it not to try and DHCP? Is that the difference | 14:13 |
| bswinnerton | between a dsstore of local vs net? | 14:13 |
| bswinnerton | I have a feeling it was this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867921 | 14:13 |
| ubot5 | Debian bug 867921 in openstack-debian-images "cloud-init: Network configuration doesn't work" [Normal,Fixed] | 14:13 |
| bswinnerton | But the dates don't quite align | 14:13 |
| meena | I'd have to see your full config to figure that out | 14:14 |
| meena | maybe there's a v1 config sent with dhcp? | 14:14 |
| bswinnerton | It's using v2 and no shouldn't be DHCP, but let me throw it in a gist for a second set of eyes | 14:14 |
| meena | cloud-init query | 14:15 |
| meena | something or the other | 14:15 |
| bswinnerton | Here's the user-data, meta-data, network-config that's being thrown into the nocloud datastore, as well as `cloud-init query -a`: https://gist.github.com/bswinnerton/86e2e44d4dee4b5271be92580afd056c | 14:18 |
| bswinnerton | Started a conversation with the debian cloud folks here: https://lists.debian.org/debian-cloud/2020/06/msg00073.html | 14:28 |
| bswinnerton | Answered my own question: it's not related to the local vs net dsstore option of cloud-localds. It still tries to get a lease on boot even with the value set to local | 14:34 |
| bswinnerton | I bet it's because of the dhcp config that lives in /run/network/interfaces.d/ | 14:34 |
| bswinnerton | meena: any idea where files in that directory come from? Is that part of cloud-init? | 14:34 |
| bswinnerton | Oh yeah, that's definitely it. Removing the file leads to super quick boot times | 14:35 |
| bswinnerton | God what a mess | 14:36 |
| meena | that a patch from Debian? | 14:50 |
| bswinnerton | https://github.com/canonical/cloud-init/commit/a6faf3acef02bd8cd4d46ac9efeebf24b3f21d81.patch | 14:51 |
| bswinnerton | That's the first problem, the second is the fact that a file gets added to `/run/network/interfaces.d/` that defaults to DHCP | 14:53 |
| bswinnerton | And I'm not sure where that comes from, but am working on terrible hacks with user-data to get rid of it | 14:53 |
| meena | logs should be helpful | 14:55 |
| meena | maybe | 14:55 |
| bswinnerton | Not seeing any references to that directory in the log, so it must be coming from the Debian cloud image | 15:00 |
| bswinnerton | Is runcmd the best hook for modifying network configurations early in the boot process? It only needs to run once | 15:01 |
| meena | network is usually the first thing to be run, so there's almost no chance of doing anything before that dhcp thing | 17:15 |
| meena | unless | 17:15 |
| meena | https://cloudinit.readthedocs.io/en/latest/topics/modules.html#bootcmd | 17:17 |
| meena | bswinnerton: this could be worth a shot, if you Anna rm that file, so you don't have to modify the image :D | 17:17 |
| meena | wanna | 17:21 |
| meena | i am now second stage tired / exhausted. | 17:22 |
| meena | Good evening. | 17:22 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!