| === vrubiolo1 is now known as vrubiolo | ||
| sdhd-sascha | Did anybody know an easy tutorial for creating an bond, with pxe and maybe install "kubeadm", "rke" or some other way to create an Kubernetes cluster? (on bare-metal....) | 12:12 |
|---|---|---|
| Holiday | I know this isn't directly cloud-init related but I wasn't able to find an answer in #Ubuntu. Any reason why after an apt-get install cloud-init there would be 3 systemctl units on ubuntu 18.04 and cloud-init runs on boot as expected but in 20.04 after the apt install there's no units listed in systemctl and nothing runs on boot even after a systemctl enable cloud-init? | 12:21 |
| Holiday | It looks like everything is there and good but nothing kicks off. a 'systemctl start cloud-init' fires it off and displays what I'd expect to the terminal, just wasn't sure what by apparent default would keep it from doing so on boot in 20.04 | 12:21 |
| sdhd-sascha | @Holiday I don't know. I'm only guest here... But i figured out, since years. That LTS ubuntu is only LTS after some years... It maybe better to use some older distro... (Except you are a expert, and can handle this kind ;-) ) | 12:27 |
| sdhd-sascha | Maybe i try landscape today. Some weeks ago it didn't support 20.04 | 12:31 |
| Holiday | @sdhd-sascha ah sadly hands are tied as this is for a provision portal for internal devs/users/etc to spin things up. Our home grown solution used vmware-tools and ansible so it was just "does ssh work", where as the new solution I'm assisting in rolling out we went cloud-init a couple years ago for the linux side which worked well.. until 20.04 made me a liar haah | 12:40 |
| Odd_Bloke | Holiday: Hmm, I'm not sure off-hand. What type of Ubuntu system are you installing cloud-init on that doesn't already have cloud-init? | 12:40 |
| Holiday | it's a 20.04 LTS, built from kickstart to be mostly bare bones, so I did an apt install cloud-init like I did with our 18.04 build (built the same way with kickstart and the same overall packages) | 12:41 |
| sdhd-sascha | @Holiday @Odd_Bloke : As i know MAAS use cloud-init ... | 12:41 |
| Holiday | just confused on why the systemd files all seem there, systemctl enable doesn't yell at me about it not existing, but then the darn thing doesn't run on boot but will by hand haha. I know, ubuntu prob | 12:42 |
| Holiday | or I just have some fub in my script that's preventing run on boot but everything is the same as the 18.04.. hence my 'what. the. heck.' | 12:43 |
| Odd_Bloke | Holiday: Can you run `cloud-init collect-logs` on an affected 20.04 machine and upload the tarball it produces somewhere? (A Launchpad bug filed using the link in /topic would work.) | 12:45 |
| Holiday | rgr I'll work on that one | 12:55 |
| Odd_Bloke | Thanks! | 12:56 |
| Odd_Bloke | (My guess is that the cloud-init generator is tripping you up, but I'm not entirely sure; cloud-init is generally pre-installed in Ubuntu images where it would be used, so post-first-boot installation isn't a path that is regularly walked.) | 12:57 |
| Holiday | @odd_bloke https://bugs.launchpad.net/cloud-init/+bug/1887668 I hope I put enough detail without being too like.. who knows. Did this while on a zoom haah | 13:13 |
| ubot5 | Ubuntu bug 1887668 in cloud-init "cloud-init doesn't appear to be running on boot" [Undecided,New] | 13:13 |
| Odd_Bloke | Holiday: OK, so it looks like my guess was correct: if you look at /run/cloud-init/ds-identify.log, then you can see "No ds found ... Disabled cloud-init", which means that cloud-init is entirely disabled. | 13:22 |
| Odd_Bloke | Looking further up in the log, you can see that "rpctool query returned 1". | 13:22 |
| Odd_Bloke | Holiday: Which is emitted by https://github.com/canonical/cloud-init/blob/master/tools/ds-identify#L791-L808 | 13:23 |
| Odd_Bloke | So we're expecting `vmware-rpctool "info-get guestinfo.ovfEnv"` to return 0 and it instead fails. | 13:24 |
| Holiday | so that is related to the vmware tools from my quick google.. which open-vm-tools is in place | 13:26 |
| Holiday | okay I see running the vmware-rpctool "info-get guestinfo.ovfEnv" returns "No value found". I also ran on my 18.04 but it returns the same value | 13:30 |
| Odd_Bloke | Holiday: Has the exit code changed between the two Ubuntu releases? | 13:35 |
| Odd_Bloke | (`echo $?` after running it.) | 13:35 |
| Holiday | they're the same. I'm not 100% sure that's the main cause as the software as far as I remember is forming an iso with the user rundata to inject (mounts to the VM pre-boot), and is using the root account via vmware tools rpc for the other bits (morpheus is the portal software). | 13:37 |
| Holiday | but again I'm not super well versed in this area. The exit code for both is 1 | 13:37 |
| Odd_Bloke | Hmm, OK, could you pastebin ds-identify.log from the 18.04 system? (https://paste.ubuntu.com/) | 13:39 |
| === vrubiolo1 is now known as vrubiolo | ||
| Holiday | https://paste.ubuntu.com/p/hd95RyBbZY/ | 13:54 |
| Holiday | quick skim between the two systems seems everything there matches up | 13:56 |
| Odd_Bloke | *polishes monacle and peers through it* "Disabled cloud-init", you say? | 13:57 |
| Odd_Bloke | Oh, actually, a major difference there is uptime. | 13:57 |
| Odd_Bloke | Holiday: If you reboot the bionic instance, does cloud-init run? | 13:59 |
| Holiday | hrm well it did, something I changed in this clone apparently made it stop haha | 14:17 |
| Holiday | basically it was running every time it booted/rebooted. now for some reason it stopped, I did just run an 'apt update; apt upgrade' | 14:18 |
| Holiday | going to clone from my base again and check that one pre-updates | 14:19 |
| Holiday | okay yeah so I cloned my base bionic to a new VM as well and on power up cloud-init runs and I see all that "calling http://169.254.169.254/<etc>' failed stuff as expected for the 120 seconds | 14:21 |
| Holiday | interesting.. on the one I ran 'apt upgrade' on, a systemctl list-units | grep cloud once again returns no results like the 20.04 system that's not running on boot either | 14:22 |
| Odd_Bloke | Oh, that is interesting; what version of cloud-init is that upgrading from? | 14:24 |
| Holiday | so pre-upgrade, an apt list --installed | grep cloud returns cloud-guest-utils @ 0.30-0ubuntu5 and cloud-init @ 19.4.33-gbb4131a2-0ubuntu1 | 14:26 |
| Holiday | hrm same version after the apt upgrade | 14:26 |
| Holiday | I did see systemd get and what not get updated (although since the base auto-updates on provision via cloud-init there were a ton of packages that got updated), so apparently not cloud-init related at all since the versions are the same but another package that's being updated | 14:27 |
| Holiday | just needed that second set of ideas and extra thoughts/pokes to realize that. Still doesn't make sense why on 20.04 where I installed cloud-init and haven't updated after that doesn't work on boot like expected | 14:27 |
| Holiday | funny thing is I remember getting hit by the "updating systemd killed my network" bug. I'm going to slide the cause over into that area | 14:31 |
| Odd_Bloke | Holiday: So one thought I have: is the OVF ISO available to the instances for the entire time they're up? | 14:37 |
| Odd_Bloke | Because if it is removed, then cloud-init will potentially no longer identify that there is anything for it to do, and disable itself. | 14:37 |
| Odd_Bloke | (Which could be a difference between the two instances which isn't tied to Ubuntu version.) | 14:38 |
| Odd_Bloke | And my other thought: a systemd upgrade will reload the daemon, so it will pick up new services on-disk at that point. | 14:38 |
| Odd_Bloke | So if you install cloud-init and later (or in the same transaction, probably) upgrade systemd on one system, but don't on the other; then that could explain the difference in reports. | 14:39 |
| Odd_Bloke | (A `systemctl daemon-reload` after you install cloud-init should have them appear available.) | 14:39 |
| Holiday | I'll check that daemon-reload | 14:40 |
| Holiday | the only ISO in use is with the rundata that morpheus passes over. Other than that there's no OVF or ISO as these are existing VMs within vCenter. Historically every reboot cloud-init would run (which was a bit of an 'ugh' because I'd have to wait for the 2 minute timeout before I could do something in the base VM we clone from but boy do I miss that delay now haha) | 14:41 |
| Holiday | trying to purge cloud-init now via apt and re-installing too to see if there's any change that way | 14:42 |
| Odd_Bloke | Yeah, so I think the problem you're having is that your instance doesn't appear to be running on a cloud, so cloud-init disables itself. | 14:42 |
| Odd_Bloke | From what I can tell, it sounds like focal is behaving as we would expect; you've been getting away with it on bionic for some reason. :p | 14:43 |
| Holiday | haha could be | 14:43 |
| Holiday | guess worst case I'll make an rc.d entry to kick it off and then use the rundata to wipe that entry or something. | 14:43 |
| Odd_Bloke | Holiday: As you build these images, you could configure cloud-init to look only for the datasource you're expecting to use. | 14:43 |
| Odd_Bloke | That would have two effects: the 120s timeout would go away, and ds-identify won't disable cloud-init if only a single datasource is configured. | 14:44 |
| Odd_Bloke | (Because it assumes that whoever wrote the configuration is smarter than it is, precisely for this sort of case.) | 14:45 |
| Holiday | haha obviously I proved that wrong | 14:45 |
| Holiday | alright I'll look into that area and see if I can figure something out there. This is the only area so far where we've used cloud-init and it initially was setup in short time as we were on an initial time crunch. I'll do some reading up in that area and see if that helps at all | 14:46 |
| Holiday | I still just find it weird it worked before the update but not after, yet the cloud-init package version didn't change | 14:47 |
| Holiday | arlight, time to google and read. Thank you for all the assistance, though, input and direction pointing!~ | 14:48 |
| Odd_Bloke | Holiday: If there's anything more we can do, let us know! | 14:48 |
| smoser | https://github.com/canonical/cloud-init/pull/492 makes me think of https://youtu.be/3m6Blqs0IgY | 18:41 |
| Odd_Bloke | falcojr: https://github.com/canonical/cloud-init/pull/493 <-- the Oracle PR | 19:27 |
| falcojr | will take a look | 19:27 |
| Odd_Bloke | smoser: You may also be interested in ^ as the original author of DataSourceOracle; it's refactoring DataSourceOracle to use Oracle's metadata endpoint. | 19:28 |
| smoser | Odd_Bloke: are instance ids the same between opc/v1 and openstack ? | 19:31 |
| smoser | +1 to oracle for not having putting HTML in their metadata responses. | 19:33 |
| blackboxsw | community-notice: cloud-init 20.2 finally released! to Ubuntu Xenial, Bionic, Eoan and Focal. Thanks all for the patience and verification efforts! email/blog post forthcoming | 19:40 |
| === blackboxsw changed the topic of #cloud-init to: pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting July 28 16:15 UTC | 20.2 (Jul 15) | https://bugs.launchpad.net/cloud-init/+filebug | ||
| blackboxsw | expect to see 20.2 in daily published cloud images within the next day or two | 19:41 |
| Odd_Bloke | smoser: Good question! I'll check. | 19:50 |
| Odd_Bloke | smoser: N=1, yes. I'll ask Oracle for confirmation. | 19:53 |
| === vrubiolo1 is now known as vrubiolo | ||
| powersj | blackboxsw, I think the date for 20.2 should stay at Apr 28. That is when upstream cut the release. | 20:27 |
| blackboxsw | ahh right +1 | 20:27 |
| powersj | 20.3 should then be planned for sometimes from now till sept. so we stay on track for 4 releases/year | 20:28 |
| === blackboxsw changed the topic of #cloud-init to: pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting July 28 16:15 UTC | 20.2 (Apr 26) | https://bugs.launchpad.net/cloud-init/+filebug | ||
| blackboxsw | powersj: yep, plan for 20.3 is in short order when some Oracle work is complete. (and to avoid this costly an SRU) | 20:29 |
| powersj | sweet | 20:29 |
| blackboxsw | will send an email to the list early next week | 20:29 |
| blackboxsw | -> camping for a real woodsy outdoor wedding Th/Fri | 20:30 |
| blackboxsw | :w | 20:30 |
| blackboxsw | falcojr: followup on your https://github.com/canonical/pycloudlib/pull/3/files | 22:42 |
| blackboxsw | or first review round rather | 22:42 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!