[14:54] <otubo> Hey all! Quick question: When setting up the ephemeral network for EC2, I can't see any code handling resolv.conf. Does it relies on the first dhclient call?
[14:59] <akutz> otubo: I don't think it would need to handle DNS/resolve.conf since the metadata is accessed via IP address, or maybe I'm missing something?
[15:00] <akutz> For example, there's a static IP address in the DS at https://github.com/canonical/cloud-init/blob/0bf8d575dc91b68084ef4d88869ac719b23924d0/cloudinit/sources/DataSourceEc2.py#L60
[15:03] <rharper> otubo: what akutz said, and in particular, for ephemeral we don't run any dhclient hook scripts (which typically would configure resolv.conf) since the *real* instance networking will come in the metadata we fetch 
[15:03] <akutz> (oh good, I wasn't totally off base!)
[15:05] <otubo> akutz: rharper the problem here is that there's some calls to getaddrinfo() to detect dns redirection and there's also http://instance-data.:8773
[15:05] <otubo> those are getting timeout on some tests
[15:05] <otubo> and are slowing down the boot time dignificantly 
[15:05] <rharper> what dhcp client?   
[15:05] <rharper> what would be hitting/requesting a DNS entry in ec2 ? 
[15:06] <otubo> rharper: dhclient 4.3.6
[15:07] <otubo> rharper: sorry, I've been looking at the code all day and forgot a little context :) I'm talking about cloudinit/util.py, function is_resolvable()
[15:08] <otubo> rharper: this function calls getaddrinfo() 3 times for bad names to detect redirection. And at cloudinit/sources/DataSourceEc2.py:60 there's the extra host lookup for EC2.
[15:11] <rharper> sure, but Ephemeral DHCP doesn't hit URLs;  it can do a connectivlty check, if configured, for Ec2, none is configured ...   
[15:16] <otubo> rharper: I don't follow. Ephemeral DHCP for EC2 does hit URL twice for http://169.254.169.254 and http://instance-data.:8773 
[15:17] <otubo> rharper: on cloudinit/sources/DataSourceEc2.py:258
[15:20] <rharper> sorry, I meant DNS based urls, if you have logs that'd help;  once Ephemeral is up, then we do a get() on the metadata.   Something doesn't feel right, this codes been here quite some time though; I'd think we'd have seen timeouts and slow boots on ec2 
[15:29] <otubo> rharper: https://pastebin.com/Wbpi6U9W more specifically on lines 114 and 115
[15:29] <otubo> rharper: we're running RHEL-8 on AWS EC2 and having a consistent 50s delay on every deploy. 
[15:33] <rharper> is that in a custom vpc or stock EC2 networking?  the socket request goes through nsswitch path , is there something else configured in the OS that would attempt to resolve things to some end point?   do you know if resolv.conf is empty (or typically a symlink to systemd-resolve stub?) 
[15:33] <rharper> otubo: and does cloud-init master rpm fail the same way?    As I mentioned, there's no changes to this path for multiple years;  
[15:35] <otubo> rharper: let me check on that.
[15:39] <otubo> rharper: default vpc aws provides when you sign up, resolv.conf contains "192.168.122.1" at the time it delays, I don't believe there's anything else trying to resolve at that time, by cloud-init master rpm you mean upstream vanilla rpm?
[15:47] <rharper> yeah, why does resolv.conf have anything in it ? 
[15:48] <rharper> that doesn't seem to match the DHCP response in your logs (10.X network)  
[15:52] <rharper> https://bugs.centos.org/view.php?id=16948 
[15:52] <ubottu> bugs.centos.org bug 16948 in CentOS-8 "CentOS 8 cloud images have static 192.168.122.1 nameserver in /etc/resolv.conf" [Minor, Open]
[15:53] <rharper> typically resolv.conf is symlink to /run or where ever the dns resolver is ... 
[15:53] <rharper> $ stat /etc/resolv.conf 
[15:53] <rharper>   File: /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf 
[15:58] <otubo> rharper: I think that's it!
[15:58] <rharper> \o/ 
[15:59] <otubo> rharper: 192.168.122.1 comes from image builder and messes up the lookups. If I erase that and redeploy, the calls bypass resolv.conf to nsswitch and the delays are over
[15:59] <rharper> yep
[15:59] <otubo> rharper: that's a bug on the image builder, not cloud-init.
[15:59] <rharper> naught image builder =P
[15:59] <rharper> right
[15:59] <otubo> rharper: thanks a lot! :-)
[16:00] <rharper> yw
[20:33] <bahamat> Is users.<user>.groups an array or a comma separted string?
[20:48] <rharper> bahamat: you're taking cloudinit config or lxd ?    for cloud-init users:  accepts an array of dicts, each dict has a number of fields,  the groups key inside a user dict is a list of groups, if it's a string, comma separated is converted to list,  
[20:48] <bahamat> cloud-init config
[20:49] <bahamat> so I can do `groups: [g1, g2]` as well as `groups: g1, g2`?
[20:50] <rharper> yes, or groups:\n - g1\n -g2 ;  yaml list or comma separated string 
[20:50] <bahamat> Ok, cool.
[20:50] <bahamat> And, is there a defined execution order for cloud-init? Or is it the order it's written in the file?
[20:51] <bahamat> Like if package foo creates group foo, and I want user bar to be in group foo, can I ensure that packages are done before users?
=== blackboxsw_afk is now known as blackboxsw
[21:22] <blackboxsw> falcojr: I'm not going to get through the ssh PR today. https://github.com/canonical/cloud-init/pull/984. I want to walk through more potential use-cases in my head to see if it holds.
[21:22] <ubottu> Pull 984 in canonical/cloud-init "Fix home permissions modified by ssh module (SC-338)" [Open]
[21:23] <falcojr> blackboxsw: all good. I wasn't expected we'd get through it all today
[21:23] <blackboxsw> https://github.com/canonical/cloud-init/pull/985 landed and your other PR
[21:23] <ubottu> Pull 985 in canonical/cloud-init "Ignore hotplug socket when collecting logs (SC-345)" [Merged]
[22:10] <rharper> bahamat: it's defined , in /etc/cloud/cloud.cfg  you can see the order of init, modules and final   , the user-groups runs first, any package installs happen *last* (during final stage);  so, for users that are created before the package installs, you'll need to include a runcmd to do a usermod to add them to the group;  
[22:10] <rharper> the runcmd writes a script which is called almost last during 'scripts-user' in the final stage 
[23:13] <FE80CC1E> Hey Folks, having an issue getting cloud-init to work with terraform created image. This is on ESXi using the following https://github.com/josenk/terraform-provider-esxi/tree/master/examples-0.13/05%20CloudInit%20and%20Templates
[23:15] <FE80CC1E> it appears the guestinfor.userdata shows up in gzip+base64 but never acutally deploys. 
[23:16] <FE80CC1E> image is trusty-server-cloudimg-amd64.ova
[23:26] <akutz> Trusty is 14.04 and the VMware Datasource that uses guestinfo was not introduced until last week, scheduled for cloud-init 21.3 and Ubuntu Impish (21.10).
[23:27] <akutz> FE80CC1E: If you are using the old cloud-init-vmware-guestinfo version of the DS on 14.04, then I'm not sure I even tested the DS with a version of Ubuntu that old. I started the DS in 2018, and I believe I tested it with Ubuntu 18.04+.
[23:28] <FE80CC1E> @akutz the trust image states cloud-init is included and I see it initialized......the version I pulled is 14.04.6. 
[23:28] <akutz> cloud-init may be included, but the datasource that introduces support for guestinfo is brand new
[23:29] <FE80CC1E> akutz gotcha, is there an ova for this or do I have to deploy ubuntu and install cloud-init manually and use ovf?
[23:30] <FE80CC1E> just want a simple linux version to deploy a web server using terraform and cloud-init 
[23:32] <rharper> https://cloud-images.ubuntu.com/daily/server/trusty/current/ 
[23:32] <rharper> FE80CC1E: has up-to-date ova and ovf ;  cloud-init pre-installed of course 
[23:35] <rharper> for the older images and OVF, there have been "challenges" w.r.t expected behavior due to the design of OVF datasource;  the cloud-init in trusty  is rather old; so it might not be new enough for behaviors you're looking for. 
[23:36] <FE80CC1E> any other thoughts here without having to deploy, add cloud-init, then export as ovf? Any Linux distro that comes with the right version of cloud-init?