[01:35] <prologic> Still trying to debug this VM
[01:35] <prologic> Trying to run my cloud-init user data manually (after rm -rf /var/lib/cloud and cloud-init clean) results in the same behavior, no userdata is run
[01:35] <prologic> I can see the userdata with the vmware-rpctool
[01:36] <prologic> it is syntactically correct YAML
[01:36] <prologic> this is just weird
[01:55] <prologic> Anyone able to look at these logs?
[02:40] <prologic> Now I have tow VMs one with a failed cloud-init run and another with a successful one
[02:40] <prologic> No changes whatsoever, just different VM name
[05:43] <prologic> anyone around?
[05:54] <prologic> systemctl status cloud-init reports dead on the failed VM, "active" on the succcessful one
[06:23] <prologic> this is totally nuts
[06:23] <prologic> besides the failed cloud-init metdata crawler service
[06:23] <prologic> I cannot see any other obvious differences
[06:23] <prologic> or any obvious errors or failues in any cloud-init logs
[06:23] <prologic> this is total nonsense
[06:41] <johnsonshi> I've created a small PR to fix a DataSourceAzure bug where the Azure http handler is not sleeping in between http retries. The PR is here: https://github.com/canonical/cloud-init/pull/842
[06:43] <prologic> man that's nice johnsonshi :)
[06:43] <prologic> I wish I could figure out this bug/issue I'm having and submit a PR to fix it !
[08:07] <dam_> Hi, I am new to cloud-init and I was able to create an image with a / and /home partitions with nosource using subiquity. I would like to do that with cloud-init directly if possible. There is "fs_setup" but for now I do not know how I can use it create an image with / and /home.
[08:13] <meena> dam_: how are you creating images?
[08:15] <dam_> From my understanding, cloud-init boots on base image and apply modification.
[08:16] <dam_> meena: With subiquity, I used ubuntu server iso.
[08:17] <dam_> meena: To use cloud-init directly I thought I could use ubuntu cloud image.
[08:18] <meena> dam_: you could ask in #ubuntu-server
[08:18] <meena> there might be more folks awake there
[08:19] <dam_> Do I have to create an image myself?
[08:19] <meena> I'm only a contributor, not one of the devs, and my main contributions are for FreeBSD
[08:21] <meena> dam_: i don't know, we've only recently gained the ability to resize LVM partitions, which to me would be the precondition to working with different partitions
[08:22] <meena> https://github.com/canonical/cloud-init/pull/721
[08:24] <meena> the alternative is that you put your different partitions on different devices
[08:55] <dam_> Do you have an example of configuration file to put different partitions on different devices?
[08:56] <meena> dam_: 09:19 <meena> I'm only a contributor, not one of the devs, and my main contributions are for FreeBSD
[08:56] <meena> i haven't worked with Linux since the start of the first Lockdown
[08:58] <dam_> If I ask you to create a FreeBSD image with 2 partitions how will proceed?
[08:59] <dam_> will *you* proceed
[09:01] <dam_> Will you install it by hand?
[09:02] <meena> right now, the automation of the installation of FreeBSD is a mess. you can script bsdinstall and bsdconfig, but usually what people do is to use zfs in these scenarios
[09:02] <meena> in that case /home is on its own dataset
[09:03] <meena> you can increase the entire pool, for which we already have code in cloud-init
[09:03] <meena> but that's like LVM but very different
[09:05] <meena> from what i gather, all Unices have very different storage management, and even between Linux distros the installers are very different
[09:07] <meena> also, in the past, I've only ever created images with packer
[09:08] <meena> I've never worked with subiquity
[09:13] <dam_> My goal is to define a cloud-config that use "fs_setup" and other stuff to automatically create a VM that contains all I need.
[09:16] <dam_> The idea is to use the same cloud-config to create a VM based on Ubuntu, FreeBSD, Arch, whatever.
[09:17] <dam_> But, maybe this is not what cloud-init permits.
[09:28] <meena> FreeBSD does not support fs_setup, so this https://github.com/canonical/cloud-init/blob/master/cloudinit/config/cc_disk_setup.py#L78 is a lie
[10:12] <dam_> What is the difference between focal-server-cloudimg-amd64-disk-kvm.img and focal-server-cloudimg-amd64.img?
[10:57] <dam_> With NoCloud datasource it is possible to specificy user-data and the example contains "password: xxx". Can this file contains all stanzas like the ones in https://cloudinit.readthedocs.io/en/latest/topics/examples.html or is it limited to specific ones?
[11:11] <meena> NoCloud can have all the modules your config contains
[11:20] <dam_> I am confused then. When I use "focal-server-cloudimg-amd64.img" I have to specify "password: xxx" as in https://blog.dustinkirkland.com/2016/09/howto-launch-ubuntu-cloud-image-with.html not "users:…".
[11:25] <meena> probably because that's for the default user, and we still have a bunch of options that crowd the global namespace
[13:05] <stevenm> Odd_Bloke, hey only just seen your reply :)
[13:05] <stevenm> http://hackstack.org/x/blog/2014/08/17/openwrt-images-for-openstack/
[13:05] <stevenm> https://github.com/dtroyer/openwrt-packages/tree/master/rc.cloud
[13:05] <stevenm> "rc.cloud" seems to be another implementation of "cloud-init" but for OpenWRT
[13:05] <stevenm> as well as (as you also said) cloudbase-init
[13:06] <stevenm> so I guess if I'm working with a hypervisor (Proxmox VE) that supports configuring "cloud-init" strings on a per-VM basis... these would work regardless of "cloud-init"/"rc.cloud"/"cloudbase-init"/"etc..."
[13:07] <stevenm> perhaps it would be good for it to become a standard... rather than just a "defacto standard" (as I've often seen it referred to)... to stop any potential fracturing
[13:27] <jadsh02> I am using find and change permission of a file command under runcmd section but its not executing
[13:32] <jadsh02> "/usr/bin/find /var/log -type f -exec chmod g-w,o-wx {} +"
[14:23] <Odd_Bloke> prologic: Feel free to drop me a link to a log/tarball. :)
[14:26] <Odd_Bloke> dam_: Did you figure out what you were trying to do, or are you still having issues?
[14:31] <Odd_Bloke> stevenm: So we do have an ongoing strand of work to define a JSON schema for cloud-init's config format; perhaps that would meet some of what you're looking for?
[14:33] <stevenm> yeah that kind of thing
[14:33] <stevenm> but I couldn't find any mention of such a thing - got a link?
[14:34] <Odd_Bloke> https://bugs.launchpad.net/cloud-init/+bug/1917626 is the bug to actually publish it.
[14:34] <ubot5> Ubuntu bug 1917626 in cloud-init "Publish jsonschema with documention" [Undecided,Triaged]
[14:34] <Odd_Bloke> You can use it to validate a config today with `cloud-init devel schema`.
[14:37] <Odd_Bloke> And you can get the full schema we generate internally for that with: python3 -c 'from cloudinit.config import schema; import json; print(json.dumps(schema.get_schema()))'
[14:38] <Odd_Bloke> (Which works with Python's jsonschema module at least: I don't think I can promise it's spec-compliant beyond that.)
[14:56] <stevenm> Odd_Bloke, part of what made me start this train of thought was this... https://cdimage.debian.org/cdimage/openstack/current/
[14:56] <stevenm> afaics those images have got absolutely nothing to do with openstack really
[14:57] <stevenm> but OpenStack and EC2 and Amazon (as in the words) is all over them for a description
[14:57] <stevenm> when they're just (again... afaics) generic raw/qcow2 images meant for qemu/kvm usage - and happen to be 'cloud-init' compliant
[14:57] <stevenm> which apparently started due to Ubuntu's initial work with EC2?  apparently!?
[14:58] <stevenm> it's similar to when you're trying to set up 2 Factor Auth on something and it starts mentioning things like "Google Authenticator" at you - you translate that to RF6238 and know it'll work with many things actually :P
[14:58] <stevenm> the language has got all confused
[14:58] <stevenm> this especially is a confusing sentence "If your platform supports the EC2 style metadata server (which is contacted by cloud-init)"
[14:59] <stevenm> so is that saying "cloud-init" (the package) is an implementation of something which communicated in "EC2 style metadata" ?
[14:59] <stevenm> or is the "EC2 style metadata" a product of the cloud-init effort itself?!: d
[15:08] <Odd_Bloke> stevenm: cloud-init started life as ec2-init, implemented in shell scripts; back when EC2 was the only cloud.
[15:13] <Odd_Bloke> cloud-init fetches data from a variety of metadata sources, including the EC2 Instance MetaData Service (IMDS); it's common for clouds to provide an IMDS that behaves the same way, so that any software written to work on EC2 will also work on their platform.
[15:17] <Odd_Bloke> (This is part of where specifying cloud-init would fall down: we have to be bug-compatible with all the metadata sources with which we interact, else cloud-init won't work on those platforms.  If a cloud changes their platform in a way that breaks cloud-init, we can try and convince them to do something different (and given the number of cloud-init users, this does sometimes work), but ultimately we have
[15:17] <Odd_Bloke> to follow what they're doing to continue functioning on their platform; even if the spec says something else.)
[15:31] <beantaxi> Odd_Bloke: Have you ever read what Eric Raymond has written about creating ntpd?
[15:32] <beantaxi> Iirc, early on they were getting killed by incoming data, because there would be a header specifying the version of the standard, and then what followed was, how-you-say, ah yes, not so compliant
[15:33] <beantaxi> And eventually they made a crucial decision: to completely ignore the header, completely ignore any claims the data made about itself, and simply infer them by inspecting the data
[15:34] <beantaxi> And now they're always right.
[15:55] <beantaxi> (and ntpd runs basically everywhere)
[15:55] <beantaxi> My takeaway was that standards are great, but if you want Epic Adoption you better focus on the data that's actually out there ... and that data might not care about standards.
[16:16] <beantaxi> falcojr: I just added some quick questions on #475. If you cant look at them today no woories, just wanted to give a heads up. Thanks!
[16:18] <stevenm> Odd_Bloke, ok so when I see Proxmox VE (for example) has "cloud-init" support... and a config tab for "cloud-init" per each kvm/qemu VM that you can spin up... what we're actually talking about is how Proxmox have made a cloud-init-compatible IMDS?
[16:18] <stevenm> does the "cloud-init" project make an IMDS or just the thing inside the guest listening for an IMDS?
[16:20] <stevenm> because it kinda sounds like the closest things to a "reference" IMDS for cloud-init ready images... is actually still EC2?! which sounds horrible and not very vendor agnostic
[17:05] <falcojr> less so a cloud-init compatible IMDS, and more-so somebody adding the support for that particular IMDS into cloud-init
[17:05] <falcojr> https://github.com/canonical/cloud-init/tree/master/cloudinit/sources shows the list of current datasources
[17:06] <falcojr> each one has separate code used to fetch the correct data from the corresponding IMDS (or other data source)
[17:14] <Odd_Bloke> stevenm: Right, what Proxmox are saying is that they will present your data to the system in a way that means cloud-init in the system will find the data that you provide.
[17:15] <Odd_Bloke> cloud-init is solely concerned with running within the instance: the implementation of an IMDS is very cloud-specific, so I'm not sure what a reference implementation would look like.
[17:19] <Odd_Bloke> The closest to a reference "IMDS" is likely what the NoCloud datasource does: it will use a variety of discovery methods but ultimately is looking to read 3-4 files (off the top of my head: meta-data, user-data, vendor-data, network-config) from whatever "IMDS" it discovers.
[17:20] <Odd_Bloke> Worth noting that an IMDS usually refers to a network Service: that's not the only way to get data into systems in a way cloud-init will find.
[17:20] <Odd_Bloke> The other commonly used route is to mount an ISO containing those files (or a more complex set of files, for some clouds).
=== vrubiolo1 is now known as vrubiolo
[22:47] <dam_> Is it possible to use "Disk setup" module to change a base image with one partition ("/") to an image with 2 partitions ("/" and "/home")?