=== hackedhead1 is now known as hackedhead
lanefuOkay i've tried to RTFM a few dozen times but I have an order of operations issue with user-data trying to include before networking is up15:11
lanefui've got a iso payload with user-data,meta-data,network-config,   (nocloud)  userdata strictly includes a remote resource.. btu its failing becuase network isn't setting up first15:12
lanefuconfigs, notes here: https://armbian.lane-fu.com/linx/m3zebigz.md15:12
lanefuQ: how do i get network-config to execute before #include in users data?15:13
blackboxswlanefu: not sure if this answers your question. But it looks like the launch is trying to detect nocloud "local" datasource which happens before your network config is "up".  Offhand if you are using qemu to launch that ISO I think you'd want to specify `nocloud-net` datasource with smbios directives https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=nocloud-net. That'll keep cloud-init from 15:29
blackboxswtrying to run with your user-data in pre-networking timeframe15:30
lanefublackboxsw: need to avoid doing smbios because eventually will be using some UEFI only images with qemu15:32
lanefufor the record : In my previous config i was doing bios payloasd for the user-data url  + dhcp and it was aweosme15:33
blackboxswlanefu: can you explain for me how you are launching this ISO currently with nocloud metadata? Just so I make sure I'm understanding the problem fully15:35
lanefueyah. did you happen to take a look at the link I shared? it has all my configs documented 15:38
lanefuwill give you raw qemu command in a sec15:38
lanefuto establish a baseline real quick:  the ISO _is_ working.. the network is getting configured, hostname is getting updated.. it's just the #include remoteuserdata is happening too early15:39
lanefublackboxsw: okay this will make your eyes bleed a bit, but the raw command and templated https://armbian.lane-fu.com/linx/kjfuylbl.json15:40
lanefuthe extra stuff you see in there, which you're probably WTFing about is I have to overload some stuff to make it do what i want.15:41
lanefuthe iso generation, and deployment is all done via hashicorp nomda15:41
lanefuit's qemu stuff works, but you have to sometimes twist its arm a bit15:42
blackboxswthanks lanefu, it'll take me a bit to digest this and try a wrap my head around the use-case here as I specifically haven't tried pushing `#includes` into nocloud user-data. If unable to set ds=nocloud-net in smbios directives to the launch.15:47
lanefuI'm the king of unearthing edge-cases that are reasonable, but edge cases nontheless. :P15:48
lanefuthank you for taking a look15:48
lanefusomeone suggesed that LEGACY network config might solve the order of operations issue.. but was trying to stick with the more modern option for purity purposes15:50
blackboxswlanefu: I'm just throwing out an untested idea here. I think in your provided meta-data you may be able to replace `dsmode: local` with `dsmode: net`16:01
lanefuokay let me try that!16:02
blackboxswthat should tell the NoCloud datasource to mandate that it run in init-network timeframe instead of init-local (before networking is setup)16:02
lanefunaturally having to unwind another mess i created first :P16:11
lanefuwill report shortly16:11
blackboxswfalcojr: also if we are adding test dependencies we might be able to quickly validate something like `./tools/run-container --unittest rockylinux/8`16:11
blackboxswlanefu: story of my life16:12
falcojrblackboxsw: I checked and it's in rocky 816:12
blackboxswyeah whatever we do as long as our copr builds don't break we have a prettty good smoke test16:12
lanefublackboxsw: worked!16:25
lanefuthank you!16:25
blackboxswbooom, I'll take that to the bank. Thanks. I might be able to put some docs together on this for future16:26
lanefunice.. any snippets I can offer?16:27
lanefui think that pastebin thing i shared pretty much is what i have tho16:27
blackboxswlanefu: have a github useraccount?16:28
blackboxswI'll send you a PR review request. I think we want to add some docs to https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html16:28
lanefublackboxsw: @lanefu16:28
blackboxswGHuser makes complete sense :). I think we also want to provide a dsmode: net/local section at the top and a `::note ` about any remote #include resources require net mode instead of local16:29
lanefusweet yeah as I move further along on the project i'll try to circle back and see what other lessons i can share16:30
lanefufun triva fact.. my user-data and meta-data are in a consul KV store and i just have a small nginx write sitting in front of it16:30
blackboxsw+1 lanefu if you'd like to adapt the current text a bit and provide a PR, that'd be awesome. the content is here https://github.com/canonical/cloud-init/blob/main/doc/rtd/topics/datasources/nocloud.rst16:31
blackboxswwe can walk you through getting into the project and shepherding that PR in16:31
blackboxswthe more the merrier16:32
lanefutis the way16:32
lanefuokay lemme drop that in ym notes so i dont forget16:32
blackboxswlanefu: great. to test doc generation it's basically just edit the rst file and `tox -re doc; xdg-open doc/rtd_html/topics/datasources/nocloud.html`16:33
lanefuthanks so much... uncorked blocker in my long-term science project that i only rarely get to do deep work on.. aka only on holiday lol16:35
minimallanefu: interesting to see someone using nomad+consul with cloud-init, I'm heading in that direction myself16:38
minimallanefu: BTW as a side note you're using virtio-blk, I assume that Ubuntu ARM should support virtio-scsi which is recommended and likely more performant16:39
lanefuminimal: let me sanitize a job file and i'll share you some insanity16:39
lanefuminimal: yeah..baby steps.. step one was get it to use virtio... i had to twist the qemu driver's arm with overload tos to od it16:39
minimallanufu: my end goal is completely throwaway VMs using Alpine Linux run-from-ram combined with cloud-init16:40
lanefulove it.. yeah lots of reallyt cool lean options these days especially with some microvm tech16:44
lanefuwhich even qemu has a flavor now.. so  you dont need firecracker etc16:44
lanefuanyway yeah i'll drop some stuff in a repo shortly16:44
minimallanefu: I hadn't thought as far as keeping the user-data inside consul (was thinking of some sort of script serving it up)16:45
minimallanefu: my own script for cloud-init enabled Alpine images: https://github.com/dermotbradley/create-alpine-disk-image16:46
minimalnot currently run-from-ram, is using "classic" disk install mode16:46
minimalblackboxsw: re lanefu's problem, is the issue not that networking config hasn't yeat been read from ISO but rather that the interface hasn't yet been brought up before the user-data #include is referenced?16:48
blackboxswminimal: yes that's ultimately the base problem. cloud-init has already rendered the network-config v1 that the seed ISO presented. But the networking-online.target in systemd hasn't been reached (and setup) yet.16:50
blackboxswbecause NoCloud detected during the boot local stage, basically happens before a `pre-networking.target` systemd unit which is before systemd-networkd actually brings up the configured network16:51
minimalblackboxsw: whereas for other DS the ephimeral DHCP would be up to give some network access...16:52
blackboxsw`Before=network-pre.target` rather16:52
blackboxswminimal: I think  yes. The ephemeral DHCP context manager for other datasources wouldn't get affected by this I believe because the cloud/backplane/environment DHCP servers would provide the necessary connectivity to access and pull down the remote #include resources.16:56
blackboxswI think I need to refresh on context there too, as I though in most cases user-data #includes generally shouldn't be processed until the init-network boot stage https://cloudinit.readthedocs.io/en/latest/topics/boot.html#network16:57
blackboxsw*as I thought*16:57
lanefuminimal: https://github.com/lanefu/lanecloud-snippets17:03
minimallanefu: rather than using ISO have you though of using seed-urls?17:08
lanefuminimal: yes. :P17:08
lanefui started from seed urls and came to ISO17:09
minimallanefu: the only problem I had with seed-urls is that you cannot provide a network-config, its using "fallback" DHCP, and so any things like VLANs etc can't be configured17:09
lanefuso my first iteration was great.. seed urls on DHCP nodes and rock n roll17:10
lanefubut now I'm actually doing stuff like static public IPs17:10
lanefuso need static config17:10
lanefuand then with the custom armbian images we're doing... the x86 ones are UEFI only17:11
lanefuso sadly its just cleaner to do the ISO17:11
lanefuand then send remote payload later 17:11
minimallanefu: you might find my script interesting - lightweight OS images :-)17:11
minimallanefu: have you looked at firecracker support in nomad? It seems promising17:14
lanefuminimal: starred your script looks cool.... I typically distance myself from Alpine strictly due to relgious issues, BUT I RESPECT IT :P17:17
blackboxswsame on those repos. TIL a bit about consul && nomad thanks for the chat here 17:19
minimallanefu: religious issues? you mean OS-specific opinions? (glibc vs musl, systemd vs non-systemd etc)17:22
lanefuyeah musl and just the package manager17:22
lanefualpine just annoys  me cuz i'm in too much of a hurry to think differently17:22
lanefudebian brain is just easier for me code for17:23
lanefuespecially if i wanna use ansible etc17:23
lanefualso i'm an Armbian dev17:23
lanefuso just sometimes you gotta stick with your team even if they lose a lot of games17:23
lanefure firecracker: yeah def been on my radar, just gonna be a while before i get there17:25
lanefuorder of operations for project is: gracefully deploy any cloud-init generic images, with appropriate payloads sourced from netbox, consul, vault17:27
lanefuthen containers17:27
lanefuthen microvims17:27
minimallanefu: I know the feeling - I've got a long list of stuff like that to try and integrate17:27
lanefuso there's some additional context for all of this of course17:28
lanefubut TLDR; a joke got out of hands after i discovered fiverr and a LOT of random cheap VPS nodes 17:29
lanefuand now i'm in the joking, but not joking phase  17:29
lanefuso really going "make my own cloud provider"  but an opinionated modern take as opposed to the mom and pop shops using proxmox and a billing/control panel platform on top of it for provisioning17:30
lanefuand eventually mimic a lot of fly.io17:31
lanefubut running on cheaper infra17:31
lanefucascading prerequisites lol17:31
minimallanefu: interesting you mention fly.io - was looking at their docker->firecracker stuff and their use of Nomad recently17:31
lanefuyeah... i was really mad when I found out they existed and already were doing "my idea"17:32
minimalI think though we're getting a bit off-topic for this channel ;-)17:32
blackboxsw'tis friday17:36

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!