=== IchabodCrane is now known as Kuraokami [14:50] holmanb: can this getting merged be my b-day present? 😄 ( tomorrow is b-day ) - https://github.com/canonical/cloud-init/pull/4654 [14:50] -ubottu:#cloud-init- Pull 4654 in canonical/cloud-init "dmi: support OpenBSD native reading" [Open] [15:06] Code_Bleu: heh, maybe [15:06] Code_Bleu: happy birthday! [15:07] Code_Bleu: no promises on merging, but I'm planning on getting in a review for it today [15:09] * holmanb has a long todo list of PR's to review and wrap up (too many WIP PR's too) [15:24] holmanb, Code_Bleu: looking at that code I'm thinking it would be nice to build a fallback to dmidecode, but in a new PR. [15:45] holmanb: thanks 😄 [15:47] holmanb: meena: I was also wondering if there could be some logic in cloud-init somewhere that if ds-identify doesn't work, that it would fallback to using the cloud.cfg. I tested the new stuff ( without the new ds-identify ) and I assumed it would use my cloud.cfg, but it didnt. [15:47] Code_Bleu: no, you have it backwards [15:47] holmanb: it is Friday...so I understand the no merge on Fridays 😉 [15:47] if you know that ds-identify won't work, you disable the service and hard-code it into cloud.cfg.d/ [15:48] Code_Bleu: I'll break the world on friday with a bad merge and sleep just fine [15:48] Code_Bleu: (but then fix it on monday) [15:48] meena: ok, with the ds-identify being in my /etc/rc.local ( broken version ), I was just hoping it would see the failure and use the cloud.cfg...but ok. [15:49] Code_Bleu: fyi, with only one datasource in datasource_list, ds-identify will use that datasource [15:49] Code_Bleu: I use cloud-init on Alpine without ds-identify [15:49] Code_Bleu: is that what you want? [15:49] holmanb: i don't see what the point is of running ds-identify when you know the datasource_list [15:50] holmanb: I had datasource_list: [ ConfigDrive, CloudStack, None ] in my cloud.cfg [15:50] meena: sure, if you know what you're running on, why would you try to identify it? [15:50] meena: good point i guess [15:50] holmanb: that's very succintly put [15:50] on the other hand, however [15:51] ..... [15:51] what happens if your datasource "changes" in a flaky way? [15:51] I'm going to steal that sentence for my docs [15:51] holmanb: if you're in a hybrid environment and move from OpenStack to AWS [15:51] exactly, when I build a disk image for a specific cloud or hypervisor I just set the specific datasource [15:51] way to keep things minimal 😉 [15:52] for generic use the default cloud.cfg has a list of all the datasources and it checks each in turn until it finds the appropriate one [15:52] which can take a looooong time [15:52] very long time [15:53] minimal: does ds-identify work on alpine? [15:53] hadn't noticed, then again I don't use generic images often [15:53] meena: never tried it [15:53] disabling cloud-init is really a better choice than generating bad ssh keys via DataSourceNone [15:53] minimal: why not try it now?! [15:56] it would be reasonable to add ds-identify as a service before init-local [15:56] I'd have to check if it has any deps I'd need to add [15:57] alsof ro some reason I'd thought it was Bash-specific but seems it isn't ~(now) - shellcheck is happy its POSIX [15:57] minimal: it uses dash on buntu [15:58] dash isn't 100% POSIX though, neither is ash on Alpine ;-) [15:58] minimal: there are a couple of non-posix things I think (local keyword isn't posix iirc) [15:58] hmm, must be warnings-ignore for that then as shellcheck normally points that out [16:00] but I don't think there are any serious bash-isms besides a couple of things like that [16:00] and happy to modify for ash compatibility [16:00] lol [16:00] $ apt search almquist [16:00] Sorting... Done [16:00] Full Text Search... Done [16:00] dash/lunar,now 0.5.12-2ubuntu1 amd64 [installed] [16:00] yeah I use "local" myself all the time [16:01] so it simply needs to be run before cloud-init-local? ~Ok, I'll give ti a try later today [16:02] to be fair, if ds-identify works on FreeBSD, it should work on alpine, too [16:04] minimal: yep, before init-local is the right time [16:04] meena: I'm not saying it won't work, just that I've never tried it & never needed it [16:04] minimal: and really, it would be best to be able to disable cloud-init if no datasource is detected [16:04] and the pre-existing init.d scripts didn't provide one for ds-identify [16:05] rather than running into a 2-15 minute timeout… [16:05] what timeout? [16:07] I assume meena is talking about the "timeout" of waiting for attempting to detect each datasource in series for all of the default datasources [16:08] hmm, don't remember any noticable delay in the past [16:09] I'll have another look and maybe time it [16:09] I think dch complained about 5~ minutes a few weeks back, because his machine was detected to be on OpenStack [16:10] wasn't that due to an issue around changes made for BareMetal Openstack use (by OVH) that were later reverted due to unexpected side effects? [16:11] meena: ahhh, openstack detection, running on arm? [16:12] yes, most likely on arm64 [16:13] gotta dash, back on in a while [16:14] minimal: that might have been the case too [16:14] but openstack doesn't have a good detection mechanism on arm [16:19] s/a good/any/g [17:17] meena: should I be able to change hostname and password in CloudStack and cloud-init should pick that up without having to do a cloud-init clean, right? [17:18] I don't know, i have to check [17:20] I'm seeing in the logs that the set_password and update_hostname are running "successfully", but it's not updating until I do a clean and reboot. [17:21] hostname i can Kind of understand, but password not so much [17:23] I'm ok with the reboot, but I didn't think it needed to basically do a clean run of cloud-init to get that. The set_passwords is in the cloud_config_module section, which i thought it would run without a clean. [17:23] could it be cached in /run or somewhere and it's a weird thing with OpenBSD again? [17:29] I do see those in the /run directory...so maybe it isn't clearing that out like and then the running of cloud-init would then query and get the new values [17:29] /run/cloud-init/combined-cloud-config.json:90: "password": "xxxx" [17:31] Code_Bleu: I need to see if I can get this reproduced and debugged [17:31] meena: ok, thanks! [17:32] i just have this stupid habit of overengineering things…like, I want to create a port before-hand :P [17:32] https://dpaste.com/2BT3U6DRK - hostname is in /run file too. Weird thing is ...it has both local- and local_ [17:32] also, CloudStack is not something I can setup easily lol [17:32] so I'll be testing other codepaths, and maybe having to fix those too [17:33] meena: actually there is a quick way to setup cloudstack locally..let me find what it's called again. [17:33] to be fair, fixing those things would be good [17:37] let's see how far i'll get [17:40] * meena 's not been feeling great today [17:40] :crossedfingers: [17:40] 😉 [17:40] sorry to hear that. Hope you feel better soon [17:40] maybe the plague that was … plagueing my family last week has finally caught up with me [17:40] Its not Covid!™ at least. [17:41] So that's 1 out of a billion other things that's floating around trying to kill us / make a home inside us. [17:42] meena: I believe this is what I was talking about with being able to spin up local cloudstack easily. It's been a min. since I've messed with it...but from what I recall it was pretty simple. - https://github.com/apache/cloudstack/blob/4e46f5ad17fa16684b689c061c7cc52ad2e5b7cc/tools/docker/README.md#L9 [17:43] oh no not docker, anything but docker please 🙈 [17:43] come on...docker is fine 😄 [17:44] but then again...I don't know what your main OS you run is...and if it's BSD, then well..maybe docker sucks on that ? [17:52] meena: the more I'm looking into things...it's looking more and more like the /run should be cleaned out on reboot and OpenBSD is not doing that. Now the question is...where does cloud-init look to know not to run the "on first boot" stuff? Is that also in /run? [17:54] Code_Bleu: https://github.com/canonical/cloud-init/issues/4180 [17:54] -ubottu:#cloud-init- Issue 4180 in canonical/cloud-init "/run is not ephemeral on *BSD" [Open] [17:54] it's not highly critical, but it's annoying [17:56] oh, Code_Bleu have you seen https://github.com/canonical/cloud-init/issues/4231 [17:56] -ubottu:#cloud-init- Issue 4231 in canonical/cloud-init "On FreeBSD 13 password is not set after initial reboot with 'set-passwords, always' on CloudStack Datasource" [Open] [17:57] actually, let's do that hack for now: rm -rf /run ; ln -s /var/run /run [17:58] 👍 [18:01] meena: that's weird. who is this igalic person that opened up a bug report on this back in June? 😉 [18:02] June? Good grief [18:03] You mentioned "not highly critical, but it's annoying"....well I pretty much need it to work so my customers can clicky clicky buttons in our UI to change hostname and reset password 😄 [18:03] yupp, making /run a symlink works. so we can do that before ds-identify and before any other thing runs [18:04] 👍 [18:04] as in, it's ephemeral now [18:04] I still need to check the password change still [18:04] so, you adding this to existin PR or creating new one? [18:05] I'm pretty sure this will fix the password too. I see the password in /run and /run isn't being removed...so [18:07] lemme prototype something [18:08] meena: I just tested manually clearing /run and running the "reset password" from my UI that talks to CloudStack...and it worked and changed the pw this time [18:08] O_o [18:08] cool [18:09] we can commit to the hack for now, but I'd still like to be a good Unix citizen and make /run relocatable in our code/installer [18:09] so however you plan on getting /run to be /var/run and ephemeral...should fix those issues [18:11] meena: is FreeBSD or some BSD your main driver OS you use...or you use something else? [18:12] Code_Bleu: I'm running some Ubuntu derivate on this Laptop (elementary OS), and FreeBSD on my servers. [18:13] In my first job, I was working on Solaris, and even had a Solaris workstation! (tho… amd64, not sparc) [18:13] and even tho it was amd64, that's still a long time ago lol [18:15] at various times in past I had NeXT Cube, Solaris/SPARC "pizza box", HP-UX workstation, and Alpha workstation on my desk ;-) [18:16] actually I think it was SunOS rather than Solaris [18:17] meena: you almost sound as old as me 😄 [18:18] * Code_Bleu is a Gentoo user, but work on all kinds of other Linux distros [18:19] Sorry meena: maybe I was thinking minimal 😄 HP-UX...yuck! [18:20] I have not had a computer in my house before 2000. Tho that doesn't necessarily say anything about my age, just about my economic background. [18:21] Code_Bleu: a lot of the telco stuff it did in late 90s/early 2000s was either HP-UX or SunOS/Solaris based [18:21] s/it did/I did/ [18:24] meena: also just noticed that in cloud.cfg i have the default user shell: /bin/bash, but for whatever reason the user was created with /bin/ksh. Hrmm [18:25] well 💩 it's /usr/local/bin/bash [18:25] Code_Bleu: isn't that controlled by the OpenBSD shell defined in cloud.cfg.tmpl? [18:30] the default is /bin/ksh, yes...but I just modified my default user and wanted to put bash...so i put /bin/bash and it should be /usr/local/bin/bash [18:30] cloud.cfg is created from cloud.cfg.tmpl so unless you override the default_user block via a file in /etc/cloud.cfg.d/* then the default will be used [18:31] Code_Bleu: yeah that seems bogus [18:32] I'll have to modify ds-identifty as its detect_virt (on Linux) only checks via systemd [18:34] here's the patch for now https://gist.github.com/igalic/9b090874cf1cc82009fff6de00cd3470 [18:34] won't be able to commit this until my goblin child is asleep [18:35] cool, thanks [18:37] meena: openbsd? [18:44] Code_Bleu: i haven't gotten to OpenBSD yet, but also, just put the contents of that precmd function into rc.local [19:14] meena: simple enough...just wanted to make sure you didn't have some other way in mind. [19:21] Code_Bleu: update, https://gist.github.com/igalic/ea855d280b4f778a9692a23824c8d5d2 [19:21] now to write some sensible commit message(s) [19:35] meena: https://whatthecommit.com/ [19:35] just keep refreshing page 😉 [19:41] Code_Bleu: https://github.com/canonical/cloud-init/pull/4677 [19:41] -ubottu:#cloud-init- Pull 4677 in canonical/cloud-init "hack(sysvinit): On *BSD, create /run as symlink" [Open] [20:26] that's what I generally try to do: https://whatthecommit.com/c8a751333acd8052477112b0dd9a24d4 [21:01] meena: nice work on the openbsd PR [21:01] meena: just submitted a review [21:01] Code_Bleu: ^ [21:08] rm -rf /run [21:08] * holmanb blinks [21:15] holmanb: we live dangerously ;) [21:16] I could issue a warning instead to have the admin do it themselves [21:18] heh [21:19] if /run is not expected to be ephemeral, what does it mean for any other project that expects files there to exist across boots? [21:20] * holmanb thinks about how we might integrate Path's assumptions into the build system [21:21] holmanb: /run doesn't exist on BSD [21:22] then why do we need to rm -rf it? [21:22] would it not suffice to leave the symlink where it is? [21:23] that's in case this is an upgrade, and it already is created, but not a symlink [21:23] why would it be anything other than a symlink? [21:23] something across upgrade do that? [21:23] ds-identify creates it on / right now [21:25] as normal directory structure. if we preempt that by making /run a symlink, the directory structure it creates is ephemeral [21:30] right, but that's only because ds-identify doesn't know that PATH_RUN should point at /var/run [21:31] I assume you're not wrong, but this hack and doing this in the build system should be about the same level of work, I think [21:33] if we do it in the build system, we'd have to make ds-identify a template and that would complicate things [21:33] speaking of, should openbsd be setting the same file variables as freebsd/dragonfly? [21:33] hmm? [21:33] meena: I don't think templetizing ds-identify is necessary [21:33] or even modifying it [21:34] just set the PATH_RUN environment variable in the init script that runs ds-identify [21:36] the build system integration would be to get helpers.Path to do the right thing [21:37] actually, build system integration wouldn't be necessary either [21:38] ugh, this reminds me why I hate Paths [21:40] env PATH_RUN=/var/run ds-identify [21:42] the problem with that is that we then need to teach admins to do this too during debugging, and that we need to teach the rest of python cloud-init where to find /run [21:42] python cloud-init can be set in cloud.cfg [21:43] via system_info.paths.run_dir [21:46] and wouldn't admins use the dsidentify init script anyways when debugging? [21:47] I… dunno? [21:47] depends on what we tell them to do, i guess [21:49] I'm hesitant to add a hack when we have what we need already (in a more elegant way), just need to update the default config for bsd and set the env variable [21:51] i agree but still think the environment variable is too fragile [21:54] if we could pull uname parsing before evaluation of the RUN_PATH we could just say if $DI_UNAME_KERNEL = Linux, do current behaviour, else set the default to /var/run [21:59] True, that could work [22:10] FYI simple "generic" test on Alpine with NO ds-identify involved and cloud.cfg containing DS list of: ConfigDrive,LXD,OpenNebula,Azure,AltCloud,OVF,MAAS,GCE,OpenStack,AliYun,Vultr,Ec2,CloudStack,Bigstep,Scaleway,Hetzner,IBMCloud,Oracle,Exoscale,RbxCloud,UpCloud,VMware,NWCS,Akamai,NoCloud,None [22:10] in init-local stage it took 2.5 *seconds* to check through that list (including searching for various local ISOs/fses) before finding the NoCloud ISO [22:11] a far way from 2-15 minutes... [22:13] obviously would be a bit different for a test using Metadata Servers rather than local ISO [22:33] minimal: I would've thought it goes searching in that order [22:40] never mind, when i properly think about it, this makes sense [23:32] meena: it DOES go searching in the defined DS order yet only takes 2.5 seconds [23:34] it checks filesystems to mount, it checks /sys/class/dmi/, does ephemeral DHCP (for GCE Local) etc