[15:07] falcojr: https://github.com/canonical/cloud-init/pull/1779#issuecomment-1292195028 I'm not seing travis tests running the unit tests [15:07] -ubottu:#cloud-init- Pull 1779 in canonical/cloud-init "Net: add BSD ifconfig(8) parser" [Open] [15:16] folcojr, blackboxsw, rharper: wondering if a PR to change cloudinit/util.py's udevadm_settle function to add a "if not subp.which("udevadm"):\nreturn" would be accepted? [15:17] this would be to handle the situation where udev is not installed (i.e. a Linux distro not using udev, specifically Alpine in my case) [15:18] I don't believe udevadm_settle is called at all for BSD systems (though not sure about cc_disk_setup.py) [15:18] meena: You don't get something like this? https://i.imgur.com/E9avfYC.png [15:19] falcojr: ^^^ [15:19] minimal: I think that'd be reasonable, though you might want an alternative [15:20] is there anything similar that alpine uses? [15:20] falcojr: Alpine has mdev and mdevd as alternatives to udev but they don' have the equivalent of a settle command [15:21] actually looking at cc_disk_setup.py I don't see anything code to prevent udevadm_settle being called for BSDs - meena, have you ever tested cc_disk_setup with BSDs? [15:27] falcojr: many months ago growpart (in cloud-utils) was modified to also only run "udevadm settle" if udevadm was present [16:11] minimal: one thought is, if we're on linux where udevadm should be, then we might silently skip calling it, rather than failing. Not sure the best way to have the distro object tell the code path whether to expect udevadm or not. [16:20] rharper: I'd guess if udevadm is missing when it shouldn't be then you've got (other) bigger problems ;-) [16:20] possibly [16:21] I can wrap it with a condition for Alpine, just wasn't sure if udevadm_settle was ever called for BSDs [16:22] minimal: not yet. I have a plan, and going thru all modules is part of that [16:22] I'm not saying you have to do it that way, was just a thought. [16:22] as I mentione earlier from a quick look at cc_disk_setup I see nothing to prevent it being called for BSDs [16:22] falcojr: I couldn't see any tests that… aren't lint tests there [16:23] yet cc_disk_setup is enabled for all distros (i.e. including [Free/Open/Net]BSD [16:25] that seems to be the obly module using udevadm_settle that doesn't seem to check for non-Linux OSes [17:05] meena: in the image, there's a Travis line that's used to click through to the unit tests. You don't see the Travis line? [17:11] falcojr: oh. that's another 9 checks that i haven't noticed so far [17:11] but any way, this build is is failing for everyone, not just me [17:12] right? [17:12] yes, https://app.travis-ci.com/github/canonical/cloud-init/jobs/586661364 [17:14] minimal: I'm not sure disk_setup can work on BSD, I'll need to properly read thru [17:14] it's not enabled in the config anyway [17:18] meena: I was referring to the "distros": [ALL_DISTROS] in its code [17:18] yeah I wasn't sure if the various utils it uses (sfdisk, sgdisk, etc) were present in BSDs [17:22] not without additional installation. [17:23] it would be better to use base system utilities to… you know… manipulate disks and partitions [17:24] meena: so I guess that line should be changed (as it is used to populate the "Supported distros:" entry in the online docs for modules [17:24] can you open a bug? [17:25] I would actually like to support disk_setup for at least FreeBSD [17:27] falcojr: any idea how to get these tests fixed? [17:28] meena: better still I can do a PR for it ;-) There's no to do "all distros EXCEPT x" so guess will just need to define the full list of all non-BSDs distros [17:29] meena: looks like they're all subp mocking related [17:29] minimal: there's also no distros=Linux, which is true… sometimes… [17:29] except maybe [17:29] > ifs_txt, _ = subp.subp(["ifconfig", "-a"]) [17:29] E ValueError: not enough values to unpack (expected 2, got 0) [17:29] falcojr: that's the whole point of the mock i committed [17:36] meena: I'm seeing a common traceback in a lot of tests [17:36] https://paste.ubuntu.com/p/4DMwHcj6vx/plain/ [17:36] they're not BSD-related tests, yet it's trying to return a BSD distro [17:37] that shouldn't be happening, so might be worth checking if you accidentally changed something to make that happen [17:37] cls = distros.fetch(distro) if distro else MockDistro [17:37] yet somehow BSD is coming back there for a number of tests? [17:38] falcojr: I've fixed at least two functions that now can run on BSDs, and a third one that can run on FreeBSD [17:39] 18:37 <@falcojr> cls = distros.fetch(distro) if distro else MockDistro ⬅️ which commit is that? and which function is it? [17:39] look at the traceback...though...now that I look a little closer those tests do all seems to specifically requesting a BSD distro [17:42] https://github.com/canonical/cloud-init/pull/1779/commits/bd5c1de7f1829e1e7407e606b1ef6c2fbbfd739e here's the commit that tries to fix this, and seems to have no effect [17:42] -ubottu:#cloud-init- Pull 1779 in canonical/cloud-init "Net: add BSD ifconfig(8) parser" [Open] [17:42] the mock will be popped as soon as the context manager exits [17:42] so you're right...it has no effect [17:43] actually, not 100% sure on that...but it's worth double checking [17:44] oh [17:46] i figured from… this https://github.com/canonical/cloud-init/blob/main/tests/unittests/distros/test_networking.py#L20 i figured it should just work [17:47] but I reckon i should be using yield then, and return [17:47] yeah, this is a fixture, so the entire test is going to run within that context manager [17:48] well, if i knew how use fixtures [17:49] pytest docs are pretty good [17:49] I'd start with the example and then move up/down from there https://docs.pytest.org/en/6.2.x/fixture.html#quick-example [17:50] it's the standard way to do setup/teardown in pytest, so worth understanding [19:07] Hello [19:09] I need a hand with getting Cloudinit working on freebsd. I am getting logs of cloud-init.log and cloud-init-output.log and its taking the script but its not making any changes. It runs the modules but no changes [19:19] Rick14: hi! I'm the main developer for the FreeBSD bits [19:20] what cloud are you running on, and what does your user-data look like, exactly? [19:26] Hi Mina! Im running on a private openstack [19:26] #cloud-config [19:26] users: [19:26] - name: user [19:26] ssh-authorized-keys: [19:26] - KEY [19:26] gecos: user [19:26] groups: [adm, wheel] [19:26] sudo: ["ALL=(ALL) NOPASSWD:ALL"] [19:26] shell: /bin/csh [19:26] passwd: "$6$ hash" [19:26] lock_passwd: false [19:26] ssh_pwauth: yes [19:26] timezone: US/Eastern [19:26] I can see in the logs that cloud init is doing something. Just not making any chnages [19:26] Rick14: you… could've used a pastebin for that… [19:27] Sorry [19:27] anyway, do you have /usr/local/etc/cloud/cloud.cfg.d/99_freebsd.cfg enabled? or even customized? [19:28] IRC doesn't have any kind of auto-text formatting options. It's customary to share code/snippets using a pastebin service. paste.ubuntu.com is a common one here [19:30] Sorry about that and noted. Never used IRC before today. [19:31] And There is a 99_freebsd.cfg in that path. But its just the default [19:33] okay, so, let's start fresh: cloud-init clean -slr (cleans logs, seed, and reboot) [19:35] Ok Done [19:38] Cloud init just ran and injected the user [19:38] Rick14: so… it thought before that it had already ran? [19:39] The user that I was injecting didnt exist till after I ran the command you sent [19:41] Rick14: no, i meant: cloud-init thought it had already ran, so when it was reading its configuration and user-data, it determined it had nothing to do (most modules are per-once) [19:43] https://cloudinit.readthedocs.io/en/latest/topics/modules.html#users-and-groups Module frequency: once-per-instance [19:44] Rick14: so, maybe the way the image was created, cloud-init clean wasn't called. [19:45] I see ok [19:46] So when creating the golden image, I should run cloud-init clean before turning it into a template? Sorry for the dumb question, im just trying to figure this out as this is all new [19:47] do everything you need before booting the guest [19:48] that is to say, if you're using packer you should have a "provision" stage [19:48] when the guest boots it'll run cloud-init as you'd expect [19:48] ok Noted [19:49] https://developer.hashicorp.com/packer/tutorials/docker-get-started/docker-get-started-provision what are you using to build the images? [19:53] So we had a freebsd image on our stack and Im just updating it and something happened from the previous image where could-init was just gone. So I was tasked with getting it running again [19:59] Rick14: was it uninstalled in a conflict? pkg logs to /var/log/messages [20:02] Based on what logs are from the previous image, it was uninstalled but nothing much more I could see. So im just trying to reinstall it [20:03] if I hit a conflict I will mention it here [20:05] Rick14: which FreeBSD version is this? and which cloud-init version is it? Are you using quarterly or latest? [20:07] This is freebsd 13.1 and Release 22.3.4 [20:18] Rick14: I'm not seeing 22.3.4 anywhere here: https://www.freshports.org/net/cloud-init/ [20:19] I installed from https://github.com/canonical/cloud-init/releases/tag/22.3.4 [20:19] Rick14: whyyyyyyyyy [20:21] * meena should submit a port upgrade… [20:21] Rick14: this is my port: https://www.freshports.org/net/cloud-init-devel/ [20:21] So I should install from freshports? [20:22] Rick14: freshports is just an interface. you should install from `pkg`, and if you're not getting the latest version of a port you're expecting, you should file a bug / submit a patch for a port-upgrade. [20:23] Ok noted. [20:24] Rick14: you can join #freebsd for help on FreeBSD. You can join #bsdports on EFNet to talk with porters. [20:24] Ok Cool Ill save those for later if I come into port issues [20:24] * meena should update net/cloud-init… not just net/cloud-init-devel… [20:25] jrm: ping [20:25] So after I install from net/cloud-init, Do I have to run any commands to stage cloud init before turning it into an image? [20:27] Rick14: you should check the config files, and you need to enable it via service cloudinit enable / sysrc cloudinit_enable=YES [20:28] the net/cloud-init port doesn't enable logging, some people like it that way… [20:33] Ok. Im installing it fresh now [20:34] Rick14: there's a bunch of fixes in net/cloud-init-devel, like we can configure ntp now, but, they're not released yet. [20:37] * meena still can't get over how FreeBSD's cloud-init runs on more platforms than Linux' [20:37] (this is because we eliminated dmidecode) [20:55] falcojr: I've now reverted that silly mock patch, and the failures, (now) unsurprisingly still look the same. [20:56] I'm… starting… to consider… a terrible approach [20:59] https://gist.github.com/e05440564d52fa70f78fa48230085d44 [20:59] "fixes" two failing tests [21:13] I think the idea behind the mock patch was a good one. If you have a common class you're going to be using in a bunch of tests but it has some code you don't want being executed in unit tests, then replace the problematic piece with something not problematic [21:15] meena: pong [21:16] jrm: i think we should give net/cloud-init also an upgrade [21:16] meena Thank you for your help! [21:17] meena: K. Still reading the backlog. Should I pull from the HEAD of upstream's main for the port upgrade? [21:18] jrm: no, just use the latest tag, and we think about what fixes from net/cloud-init-devel we can push down to net/cloud-init [21:18] but, basically, Rick14 installing cloud-init from source, rather than pkg prompted me to check what version we're on, and it's a bit behind [21:21] My last message was stupid me thinking you wrote net/cloud-init-devel, not cloud-init. [21:22] falcojr: btw, is there a way to only @mock.patch that only triggers on a specific subp.subp call? (this is well possible in Ruby…) [21:33] like only from a particular call point? Not really [21:35] falcojr: person in #python is recommending: https://bpa.st/YPLA [21:36] but… i'm starting to wonder if we're not overriding @mock.patch [21:37] that's one way to make it happen [21:37] falcojr: btw, re https://docs.pytest.org/en/6.2.x/fixture.html#quick-example — in my code, this doesn't work: https://gist.github.com/fcefd4a0afee6e156fd7e3f691b0784c [21:38] even tho, I already have this exact pattern in test_networking.py [21:38] fixtures aren't called, they're passed into the test function [21:39] oh wait, nm, that's what you're doing [21:40] have you tried mocking cloudinit.distros.networking.subp.subp ? [21:41] I read this a while ago and found it helpful btw, https://nedbatchelder.com/blog/201908/why_your_mock_doesnt_work.html [21:42] falcojr: in networking.py, we're just importing from cloudinit import net, subp, util; so i assumed that's the one i have to mock. [21:43] i wish there was a way to … initialize the BSD networking class only on (first) use [21:46] but your test code is importing networking, so it should be patched there [21:50] falcojr: either way, that's not waht's failing with this particular attempt… [21:53] (also, in test_networking.py, we don't import networking) [22:00] okay slowly starting to get a better grasp on this [22:26] https://gist.github.com/ee97505d7b297237efbcfbe40563fd38 this works, but I really hope I'll stumble into a more generic solution [22:26] tomorrow.