=== rezroo1 is now known as rezroo
[11:21] <otubo> Odd_Bloke, blackboxsw I was actually going to send a PR today for this fixes :-) But thanks anyway! Sorry for not being online at the time you requested the review. I was multitasking on meetings and other fixes yesterday.
[12:16] <otubo> Anyone interested in discussing this particular bug? https://bugzilla.redhat.com/show_bug.cgi?id=1803928
[12:16] <ubot5> bugzilla.redhat.com bug 1803928 in cloud-init "[RHEL8.2] Race condition of starting cloud-init and NetworkManager" [High,New]
[12:16] <otubo> Not sure if that hits upstream as well, perhaps it does.
[12:24] <otubo> I'll do the rubber duck here until someone reads the log :-D
[12:27] <otubo> So if I understood correctly, cloud-init-local unit will call cloudinit/cmd/main.py which should start *before* NetworkManager
[12:29] <otubo> cloud-init will perform the network configuration and *only then* NetworkManager starts and brings the interfaces up
[12:30] <otubo> So question 1) Why is there the pretty log on cloudinit/cmd/main.py to print all network info, if that's responsible only for applying the configuration (networmanager should bring that up later)?
[12:32] <otubo> And question 2) on the bugzilla, there's 2 scenarios the one that cloud-init "starts" (prints net info) "in the middle" of NetworkManager and that when it fails. The second is when cloud-init prints net info after NetworkManager does its stuff and that's when it works.
[12:38] <otubo> In the meanwhile I'll just try to get access to the VM and check some logs.
[12:39] <StyXman> user expiredate vs chpasswd expire. comments?
[13:40] <Odd_Bloke> StyXman: Without more context, none. :)
[13:41] <StyXman> Odd_Bloke: part of the context is the bug I opened yesterday
[13:41] <StyXman> but basically, I can't make the password expire with either method
[13:42] <StyXman> in both cases I'm also setting the password; in the bug's case, the password is set *after* the expiration is set, so ir resets it
[13:42] <StyXman> and I'm gessing somehitng similar happens with chpasswd
[13:43] <StyXman> hmm, is the yaml processed sequentially?
[13:44]  * StyXman moves chpasswd to the end of the file
[13:45] <Odd_Bloke> otubo: No worries!  As I said, we had a real time pressure else we'd have waited a whole day for it.  That said, I think there are more fixes required to that code: the create_swap function swallows ProcessExecutionError but the body of create_swapfile uses ProcessExecutionError to indicate that it should attempt dd after fallocate.  As a result, the fallback will never happen (I believe).
[13:45] <Odd_Bloke> StyXman: The processing is not sequential; the order of processing is defined in /etc/cloud/cloud.cfg.
[13:50] <StyXman> Odd_Bloke: ok
[14:28] <otubo> Odd_Bloke, Understood, I'll take a look at that right after I fix that BZ I posted above. Looks like that's a little more urgent. Boy I need some PTO
[14:42] <Odd_Bloke> blackboxsw: rharper: OK, so the fix we cherry-picked yesterday didn't mock util.get_mount_info which returns None on the Launchpad builders (rather than any valid content at all), causing test failures.  https://github.com/canonical/cloud-init/pull/319 does mock that and I just did a test PPA build with it included that built successfully:
[14:42] <Odd_Bloke> https://launchpad.net/~daniel-thewatkins/+archive/ubuntu/temp2/+build/19168108  So a review of that would be appreciated, and then we can have another go at a focal upload.
[15:47] <Odd_Bloke> blackboxsw: https://github.com/canonical/cloud-init/pull/320 has successfully built locally, and is building in a PPA now.
[15:47] <Odd_Bloke> blackboxsw: If you review it (and approve, obvs), I'll wait for the PPA build to succeed and then upload.
[15:49] <kqkq95> Hi al ! someone is using cloud-init with vmware under centos8 ?
[15:55] <blackboxsw> Odd_Bloke: approved for you to build and push #320
[15:55] <blackboxsw> not often kqkq95  but if you ask a specific question some folks may have an answer
[15:56] <blackboxsw> people drop in frequently with vmware/centos questions
[15:57] <Odd_Bloke> blackboxsw: Thanks!
[16:04] <kqkq95> blackboxsw : I am trying to configure my vmware with cloud-init template, I use foreman to deploy my servers (they are compatible with cloud-init), I followed this doc: https://docs.theforeman.org/guides/build/doc-Provisioning_Guide/index-foreman.html#Provisioning_Virtual_Machines_in_VMware_vSphere-Provisioning_with_cloudinit_and_userdata_templates
[16:04] <kqkq95> Except that when my vm starts, I have errors in the logs: https://hastebin.com/retipezonu.sql any ideas ?
[17:56] <kqkq95> ?
[18:45] <rharper> kqkq95: sorry, blackboxsw stepped out for a bit;   looking at your trace;  it's hard to say what the problem is;  OVF has had some challenges with the different customization modes and versions of cloud-init;
[18:48] <rharper> kqkq95: there are also a number of vmware related fixes that are in newer release of cloud-init;
[19:09] <kqkq95> rharper: ok no problem, i use cloud-init v18.5.7  and vsphere 6.5, how can i debug ? maybe documentation ?
[19:11] <blackboxsw> sorry back
[19:13] <rharper> kqkq95: I would suggest trying to run a newer cloud-init; let's see what we have in our daily yum repo ... https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
[19:14] <rharper> that has 20.1;   we also have centos7 19.4, https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/
[19:15] <rharper> those will have a number of OVF related fixes; which may resolve your issue.    We don't have access to VMware infrastructure making testing/verification not possible;   it's been a rough spot for quite some time unfortunately; we reliant on downstream and VMware to test and fix their datasource.
[19:17] <blackboxsw> rharper: kqkq95 from your share hastebin, it looks like the only datasource searched is only (DataSourceNoCloudNet) on line 111 ..... instead of DataSourceOVF detection. Typically whe we SRU test we are deploying in a manner that detects OVF
[19:18] <blackboxsw> kqkq95: here's one of our output logs and scripts that show DataSourceOVF being detected https://github.com/cloud-init/ubuntu-sru/blob/master/manual/vmware-sru-18.5-45-g3554ffe8-0ubuntu1.txt
[19:18]  * blackboxsw reads back on your guide you linked 
[19:19] <kqkq95> ok
[19:23] <blackboxsw> kqkq95: you have a bogus seedfrom configure
[19:23] <blackboxsw> kqkq95: you have a bogus seedfrom configured
[19:23] <blackboxsw> https//deploy.phys.pack/userdata/
[19:23] <kqkq95> why ?
[19:23] <blackboxsw> is not a valid url that's why you get the message 2020-04-15 19:41:59,845 - DataSourceNoCloud.py[DEBUG]: Seed from https//deploy.phys.pack/userdata/ not supported by DataSourceNoCloud [seed=None][dsmode=net]
[19:23] <kqkq95> this my url
[19:24] <blackboxsw> not https//deploy....  should be https://deploy...
[19:24] <blackboxsw> missing the ':'
[19:24] <blackboxsw> silly one character always gets ya ;)
[19:24] <kqkq95> eu
[19:24] <kqkq95> '=D
[19:24] <kqkq95> i test again
[19:25] <kqkq95> I did a lot of testing
[19:26] <blackboxsw> so I found this by looking for the debug message "Seed from " in cloudinit code base. and found this https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/DataSourceNoCloud.py#L166-L171
[19:27] <blackboxsw> which pointed me at potentially invalid protocol being the issue.
[19:27] <blackboxsw> it would have been a better/real message if cloud-init emitted a warning saying ignoring your seedfrom because it isn't one of the supported protocols: [ ("http://", "https://", "ftp://") ]
[19:28] <kqkq95> i try a new deployment
[19:53] <kqkq95> blackboxsw: ok so I corrected the url, but I have a new error: https://hastebin.com/apuvuyafuc.sql
[20:00] <blackboxsw> hrm kqkq95 do you have access to the system's interactive serial console? it looks like cloud-init isn't finding the expected */metadata URL and it's getting a 404 on line 261 after 11 failed retries.
[20:03] <kqkq95> blackboxsw: yep of couse
[20:04] <kqkq95> blackboxsw: why he returns me a 404
[20:09] <blackboxsw> dunno, something on your foreman system (deploy.phys.pack) isn/userdata/meta-data is inaccessible. you'd probably have to test a system console that
[20:09] <blackboxsw> isn't surfacing userdata/meta-data.
[20:10] <blackboxsw> sorry about that. I hit enter mid sentence.   something on the system deploy.phys.pack isn't responding to route userdata/meta-data. You might have to check the deploy.phys.pack system to see what's truly configured there
[20:12] <blackboxsw> but, that's not really a cloud-init problem. cloud-init nocloudnet expects to see both a route <your_url>/user-data and  <your_url>/meta-data from whatever seedfrom you provide it https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
[20:12] <blackboxsw> so if those routes don't exist on the host/ip you provid in seedfrom cloud-init rejects it
[20:13] <kqkq95> blackboxsw: ok indeed, by making a curl from my server deployed to the url https://myforemanurl/userdata/user-data, it responds well 404 instead of showing me my template, I will dig ...
[20:28] <kunalbansal> Hey I needed some hep regarding the release versioning for cloud-init. Is this the right channel?
[20:29] <powersj> kunalbansal, possibly
[20:30] <blackboxsw> kunalbansal: ask away
[20:31] <kqkq95> blackboxsw: ok i found that was coming from my haproxy front servers by making a curl on my foreman server directly it works! thank you :)
[20:32] <blackboxsw> good to hear kqkq95
[20:32] <blackboxsw> another satisfied customer ;)
[20:32] <kunalbansal> How is minor version for cloud-init is decided. Like right now we get cloud-init-18.5-3-centos when we do "yum install cloud-init-18.5". Is it possible that in future, cloud-init-18.5-4 would be downloaded or 18.5 has been closed.
[20:32] <kqkq95> blackboxsw: ;)
[20:33] <blackboxsw> kunalbansal: release versioning is just time/year-based and we update the upstream version of cloud-init around 4 times per year
[20:34] <blackboxsw> kunalbansal: our current release is 20.1  (the first release of 2020 released on Feb 18th). and 20.2  is scheduled for Apr 28 (in the topic of this IRC channel)
[20:37] <kunalbansal> Does that mean we would not have a release for 18.5 anymore?
[20:38] <kunalbansal> and releases always go in a forward session and no bug fixes in previous ones.
[20:39] <kunalbansal> And the releases that would be coming out would be 20.x versions
[20:39] <kunalbansal> or forward
[20:39] <Odd_Bloke> kunalbansal: Upstream, we generally do not backport bug fixes to existing releases.  However, the "-3" part of that version comes from CentOS, and they may choose to do their own backporting.  You would really need to talk to them to understand the update policy you should expect.
[20:40] <Odd_Bloke> (And presumably they _have_ done some sort of backporting, as -3 would be a strange number to choose for your first version. ;)
[20:41] <kunalbansal> Yeah that was the confusion i was having. Thanks for clearing that up.
[20:43] <blackboxsw> sorry I got pulled away before I could properly answer the actual question. my bad .
[20:43] <blackboxsw> thx Odd_Bloke
[20:44] <kunalbansal> Thank you that really helps. Do you know a channel where I can get this information.
[20:53] <blackboxsw> kunalbansal: otubo may have a reference or pointer for some CentOS  related release info. I haven't seen a definition/policy for ongoing intended ports for CentOS. As a note, we (upstream cloud-init) just dropped python 2* support December 2019. So it's a relatively recent change that will adversely affect the ability to backport changes from tip of cloud-init upstream into CentOS7 if that is the environment in
[20:53] <blackboxsw> which you are running. So, maybe there is a CentOS distro plan for how to handle backports for bug fixes etc.
[21:05] <blackboxsw> kunalbansal: one path to go is if you have a specific feature or bug for centos you could try filing a bug there and see where the response goes from CentOS folks https://bugs.centos.org/main_page.php