=== esv__ is now known as esv [17:34] cloudinit identifies datasource as OpenStack on 23.3 instead of Ec2 [17:34] this is FreeBSD so there may be other differences [17:35] https://www.irccloud.com/pastebin/MWxQ1bGc/cloud.cfg [17:36] https://www.irccloud.com/pastebin/8OBlUpcr/ds-identify [17:36] at least from command line can I force the datasource? [17:37] this is what I'd expected `datasource_list: ['Ec2']` to do, which is what was previously relied upon [17:39] cc meena [17:47] dch: yes, but I think also need, 'None' to terminate the list for good [17:48] meena: just tried that, same result [17:48] dch: you can also disable the ds-identify service to disable the feature [17:48] I just say this "no datasource_list found, using default: MAAS ConfigDrive... etc [17:49] and by disable, i mean, delete [17:49] lemme get my laptop, might make writing easier [17:50] meena: only if you have the time, its not urgent [17:51] 22.4-28-gaecdcbf8 was the previous version [17:53] actually, it's bedtime, and i have to put kiddo ti bed now, dch, so anywhere between half an hour, and two hours when I'm actually getting back to my laptop [17:55] meena: similar story here, how about I keep debugging and you can drop in later to throw some peanuts at me? [17:55] or popcorn, I'm easy [17:56] we got monkey nuts for the crows === dbungert1 is now known as dbungert [18:20] dch: this is a physical machine (Supermicro) ? [18:27] dch: a physical machine won't "look" like AWS (i.e. the DMI values) and so it makes sense that Ec2 is not chosen. There was a change made to the OpenStack DS a while to support physical machines (as well as VMs/Cloud) and I expect that is why ds-identify decides "maybe" [18:29] look at the likes of the ec2_identify_platform in ds-identify to see its' logic [18:50] minimal: yep, all real tin [18:50] this btw works under 22.x so I'm trying to find how to address this [18:51] exploring /etc/cloud/ds-identify.cfg to disable it [19:02] meena: where do you want me to log some bugs for this? /etc/cloud/ds-identify.cfg should look in %PREFIX%/etc/cloud/ds-identify.cfg IM(H)O but it doesn't [19:04] and even if that is set, I see `DSNAME=Ec2 DSLIST=Ec2 in ds-identify log, but this doesn't appear to change the DS as seen by cloudinit itself [19:04] so I shall dig into that next [19:05] https://www.irccloud.com/pastebin/elrefaG2/ds-identify [19:15] dch: to disable it, you literally just `rm %PREFIX%/etc/rc.d/ds-identify` [19:15] meena: thanks, let me try that. [19:16] but every time I get pkg update, this will re-appear I guess [19:16] . is there any way to disable it by config, rather than chainsaw? [19:16] i should probably introduce a variable to disable only that thing, eh? [19:17] dch: do you wanna open up a bug? [19:17] meena: sure, on cloudinit repo? [19:17] mmm [19:18] I think the nicest way / easiest way is to add a cloudinit_dsidentify_enable=NO var [19:18] yeah, i don't have access to mail rn, so bugzilla is kinda meh [19:19] In the bigger picture I would like to understand why there appears to be no obvious way to hard-wire this, using the parameters that worked happily under 22.x. [19:19] dch: lots of code changes since then [19:20] indeed I skimmed the changelog [19:21] 1 datasources returned maybe: OpenStack [19:22] yeah, i wonder if you can say no maybes [19:23] so. [19:23] that alone isn't sufficient, it insists on picking openstack [19:52] I think I need to remove /run/cloud-init/* as well as /var/lib/cloud/* /var/log/cloud* to clean up [19:59] ok that looks better [20:28] |`->no network data found from DataSourceEc2 @01.04000s +125.16800s` [20:28] if I read this right, despite locking Datasource=Ec2 this still takes over 2 minutes to fetch the metadata? [20:28] that seems longer than expected [20:36] I wonder if this is 2x 60s metadata server timeouts [20:38] some services have 5 minutes timeouts [20:44] I think the instance just takes ~ 2 minutes before its allowed to connect to the network [21:25] meena: did the rc.d/dsidentify only get added in 23.3 ? I think this the case [21:25] dch: yes [21:26] dch: so you have something on your local network emulating a Ec2 metadata server? [21:26] minimal: yes, although its not actually mine, so I don't get to change it [21:27] so is it providing network config info? [21:27] minimal: not network info, because thats pre-injected (as its complicated), but the rest - ssh keys, userdata scripts, the usual [21:28] dch: pre-injected? but you're trying to use the Ec2 DataSource in which case the Ec2 metadata server is expected to provide network config [21:28] `network: {config: disabled}` in cloud.cfg prevents that [21:28] i.e. the images are deployed with the requisite network config injected during ipxe stage [21:29] before OS is started [21:29] dch: so you don't really want a "proper" Ec2 DS then? [21:29] the rest is (largely) user configurable [21:30] minimal: TBH all I really want is for cloud-config to have a schema for its config files, that is largely consistent and stable across versions [21:30] with a "real" Ec2 datasource cloud-init brings up a temporary network config in order to contact the metadata server to then obtain the "real" network config so that it can configure the OS accordingly [21:30] minimal: right, in this case we skip the first bit [21:30] but if you skip that then you don't really want "real" Ec2 then... [21:31] its a bit hard for me to introspect it from the outside, but I think ~ 2 minutes is waiting for something (networking, routing idk) to "come up" and let traffic flow [21:31] minimal: true, we also have `strict_id: false` for the ec2 config [21:31] again, that bit is not my network so it is what it is [21:31] so if you don't want "real" EC2 then why don't use use another DS, like NoCloud-Net? [21:33] minimal: did it exist 4-5 years ago? that would be the main reason its done this particular way [21:34] I skimmed https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html and its not clear to me whether the network-fetched metadata like ssh-keys and stuff would still work the same way (from same url etc) [21:34] I didn't think cloud-init supported FreeBSD 405 years ago... [21:34] s/405/4-5/ [21:35] minimal: you'd be surprised what FreeBSD people hack on when we think nobody's looking [21:36] it was first added in 2014 to FreeBSD ports, and works tolerably well apart from the slow drifts of time and config changes [21:36] dch: "is for cloud-config to have a schema for its config files, that is largely consistent and stable across versions" - cloud-init *does* have a schema for cloud-config, indeed the provided user-data is validated during boot. However regarding stable across versions, when things are changed the old version is marked depricated in the schema and still supported for a period of time [21:37] I wonder if there's a way for me to validate our configs externally perhaps? [21:37] like, in CI when I build the images? [21:37] actually you referred to cloud-init as "files" - cloud-init is one particular type of user-data [21:37] true [21:38] network config and meta data are NOT cloud-config [21:38] the cloud-init cli can validate cloud-config files [21:39] so the "data" that are fetched from www servers which may or may not be, or end up, as files, ... [21:39] do these follow a schema from cloud-config? or is this actually the DataSource provider, that owns this? [21:40] no, the specific cloud-config user data i.e. a YAML file with first line "#cloud-config", not metadata or network config files/documents [21:40] recent releases of cloud-init have added a schema for network v2 config but I think that is a work-in-progress [21:42] aciba90 also recently wrote a server for validating cloud-config docs: https://github.com/aciba90/cloud-config-validator [21:43] dch: cloud-config schema is independent of DataSource [21:45] dch: "network-fetched metadata like ssh-keys and stuff would still work the same way (from same url etc)" - ssh-keys etc are NOT metadata and NoCloud-Net fetches from multiple URLs relative to the same base-url (i.e. meta-data, network-config, vendor-config, and user-data appended to the base url) [21:48] oops, forget "network-config", it's not relevant for NoCloud-Net [22:11] ok I think I got to the bottom of all of this [22:12] - on FreeBSD, stuff live in /usr/local/etc/cloud/... not /etc/cloud [22:12] - ds-identify script doesn't accommodate that yet, so: [22:12] thus: [22:12] - it looks for the default cloud.cfg in /etc/cloud/cloud.cfg [22:12] - fails to find the `datasource_list: ...` parameters [22:13] - falls back on OpenStack [22:13] if the default cloudinit path is fixed, this seems to work again [22:19] meena: catch you tomorrow I'll add an issue for this too with [23:52] I'm having an issue I just can't seam to figure out. I've got cloud-init installed on a disk image that is booted by Xen (virtualization). The OS is almalinux8 and it boots as it should however it for some reason cloud-init doesn't run on boot. [23:52] I've used systemctl to enable cloud-init, cloud-init-local, cloud-config, and cloud-final. [23:54] Still wont run. cloudinit clean doesn't seam to change anything either. [23:55] cloud-init init will actually run as it should. [23:58] nothing in /var/log/cloud-init.log?