/srv/irclogs.ubuntu.com/2020/08/25/#cloud-init.txt

blackboxsw_same falcojr I'll close out on that review now and put up a 20.3 branch for review02:47
paridesmoser, TIL https://github.com/chrlutz/apt-repos, not packaged yet09:48
paride`apt-repos dsc` + dget could finally be a good way to download a src package with its .orig from a repo without setting it up in sources.list09:48
meenaoh, nice09:49
Carbrich_R2HI all, I am having a weird issue with cloud-init, for some kind of strange reason it gets terminated. And I cannot figure out what is causing the issue. I only have this issue with CentOS8,  You can find information on09:51
Carbrich_R2https://community.theforeman.org/t/cloud-init-does-not-install-puppet-but-runs-the-final-module-and-succeeds-according-cloud-init/20155/23 of my issue. The only part that is missing that if I remove /var/lib/cloud/ and reboot the client all scripts and providers work. Sometimes 1 of 100 cloud-init works without issues.09:51
Carbrich_R2on should be at but ok. As long as you guys understand it :)09:52
meenaCarbrich_R2: what's makecache, and why is that part of the puppet installation?09:54
Carbrich_R2HI, Initially I tried a cloud config with installing puppet, later on the road I also removed that part to investigate what is going wrong. At this point I am only trying to install package "SL" with the native package provider. And that also fails.09:55
meenahttps://community.theforeman.org/t/cloud-init-does-not-install-puppet-but-runs-the-final-module-and-succeeds-according-cloud-init/20155/26 ⬅️ this is interesting09:56
meenathe easiest way to figure out if cloud-init got updated is to check the version before and after the run. but dnf's logs should have some clues, as well09:56
Carbrich_R2Even more strange sometimes let`s say 1 of 100 it works. and after removing '/var/lib/cloud/ and restart the machine again wil result in OK.09:57
Carbrich_R2to anwser your qeustion meena there is nothing logged about updating cloud-init09:58
Carbrich_R2in the dnf lgos09:58
Carbrich_R2logs. If I do a casual ping with count 20 it stops after 8-10 tries. It executes runcmd part for example and it get a sigterm. Nothing in dnf logs etc.09:59
Carbrich_R2 All end up with the conclusion that something is killing cloud-init. And I cannot figure out what is happening under the hood. Any idea`s?10:00
meenaI'm wondering if there's some kind of time limit on execution, but that would be new to me10:02
Carbrich_R2It almost looks like some race condition with system services,  that causes this issue. I have a similar setup on our CentOS7 deployment street and that works fine.10:04
meenaoh, hrm, i thought we fixed those10:04
meenain newer versions10:05
meenaCarbrich_R2: can you upgrade and run your test?10:05
Carbrich_R2which versions? I updated to 19.4 yesterday10:05
Carbrich_R2and tried with that version also10:05
Carbrich_R2:)10:05
Carbrich_R2 meena I have no problems with upgrading, but which version do you advice?10:07
meenahttps://github.com/canonical/cloud-init/pull/227 hrm, i thought we merged this "fix"10:07
meenaCarbrich_R2: the current release is 20.210:08
meenahttps://copr.fedorainfracloud.org/coprs/g/cloud-init/el-stable/ ⬅️ im confused about the last release being 9 months ago10:11
meenamaybe these are just the SRU releases?10:11
meenaaaaaand I'm very confused about the versions listed there10:13
Carbrich_R2meena No problem, That confused me also. But in what version is that issue fixed you are referring to?10:53
Carbrich_R2meena https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/ I could try a Daily build, but the repo file is missing..10:56
Carbrich_R2They are referring to " # Save the above yum config to  /etc/yum.repos.d/CentOS-cloudinit.repo" But there is nothing above that text.. Confusing10:57
meenahttps://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/repo/epel-8/group_cloud-init-cloud-init-dev-epel-8.repo10:58
Carbrich_R2meena THX found it already indeed :)  clicked on one of the icons down on the page.10:59
meenasame11:00
Carbrich_R2Cool, I1`ll have to rebuild my base image with this repo and import in vcenter before i can test. I`ll come back to you if I know the outcome. It will take some time :)11:00
beantaxi8 days ago (https://github.com/canonical/cloud-init/pull/518), make-mime.py was moved from tools to devel, and going forward will be part of the CLI. Seems like a nice idea, and the docs have been able to reflect it.11:47
beantaxiHowever, while the code is in master, it's not in a distro, and it looks like it's not tagged by 2.2-94 either. And so obviously it's not on my ubuntu EC2. Any idea when that will all happen? And, is it considered a docs bug, when the docs on the public web refer to changes that have not happened yet?11:49
smoserparide: thanks for the apt-repos reference.12:59
smoserbeantaxi: what is 2.2-94 ?13:01
smoserreadthedocs has docs for older versions also. and source code for your distro should contain docs that match exactly with your packaged version.13:02
smoserwith regard to "when will make-mime" become part of ubuntu, probably in the next SRU (stable release update)13:05
AnhVoMSFT@rharper @smoser if the customer has security requirements on /var/tmp being noexec, is there any workaround to allow cloud-init to still do DHCP to talk to metadata service?13:42
beantaxismoser: Thanks. And sorry - I meant 20.2-94, or more precisely 20.2-94-g3d06abc2-0ubuntu1, the github tag. My fault for trying to be terse instead of copy/pasting. (Really sorry -- I13:52
beantaxi'I'd be mad if someone did that and I wasted time figuring out what they were talking about)13:53
beantaxiAlso thanks for pointing out the docs version option, which I now found, and it does indeed refer to tools/ rather than devel. All good.13:55
beantaxiAs for in which release will the move to devel happen ... I suppose that question was meta, and I'm curious about if there is an easy way to tell, or a reliably way to make assumptions, about what release a PR will be a part of, other than asking here.13:56
smoserbeantaxi: well, looking at the debian/changelog it appears every ~ 2m14:18
smoseri think thie goal was 1m.14:18
smoserAnhVoMSFT: not that i'm aware of.14:23
beantaxismoser: Thanks ... I'm now looking here for Ubuntu 18.04 -> https://launchpad.net/ubuntu/bionic/+source/cloud-init/+changelog, where the release schedule has more variance, and release dates are included at the end of the changelog, when they are included. But I'm new at this; maybe this is the wrong place to look.14:26
beantaxiEither way, a heursitic of 'on the order of months' is valuable.14:26
smoserbeantaxi: maybe Odd_Bloke or blackboxsw_ could point on what the expected cadence is.14:46
smosersome other info at https://github.com/cloud-init/ubuntu-sru and https://wiki.ubuntu.com/CloudinitUpdates14:47
beantaxiThat looks great. I don't think I'll need anything precise but if I do I know who to ask. Thanks!14:58
blackboxsw_@ahosmanMSFT  I thought there was a recent bug/branch to try to address the noexec on /tmp/noexec  for dhcp https://github.com/canonical/cloud-init/commit/db86753f81af73826158c9522f2521f210300e2b15:04
Carbrich_Rmeena It only worked two times from the 10 and now i am only getting to see the sigterm again.15:06
Carbrich_R2020-08-25 15:03:53,142 - subp.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/runcmd'] with allowed return codes [0] (shell=False, capture=False)2020-08-25 15:03:54,145 - util.py[DEBUG]: Cloud-init 20.2+137.gc0450c02-1.el8 received SIGTERM, exiting...  Filename: /usr/lib64/python3.6/subprocess.py  Function: _try_wait  Line number:15:06
Carbrich_R1424    Filename: /usr/lib64/python3.6/subprocess.py    Function: wait    Line number: 1477      Filename: /usr/lib64/python3.6/subprocess.py      Function: communicate      Line number: 8552020-08-25 15:03:54,145 - handlers.py[DEBUG]: finish: modules-final/config-scripts-user: FAIL: running config-scripts-user with frequency15:06
Carbrich_Ronce-per-instance2020-08-25 15:03:54,146 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)15:06
blackboxsw_so I think that pr was to use the known dhclient path if var/tmp is noexec. If that doesn't work, then cloud-init falls back to use the dhclient in /usr/sbin/dhclient15:10
AnhVoMSFTThanks @blackboxsw, I'll take a look at the PR15:50
blackboxsw_@AnhVoMSFT that said, I think ubuntu will still run into an issue with AppArmor profiles on ubuntu in such scenarios15:53
blackboxsw_but it'll 'work' for other distros which don't use apparmor15:53
blackboxsw_beantaxi: the plan for cloud-init releases is to try to create an upstream release (20.2, 20.3, 20.4 etc)any time we have a new significant feature to make SRU generally this will likely mean at least 4 times per year. We document the estimated upstream release date (20.3) in the topic of this channel, and Canonical cloud-generally.16:10
blackboxsw_specifically for your commit, we are trying to cut that 20.3 release today (we were waitong on an azure-related fix for cloud-init related boot timeouts which just landed last night)16:10
blackboxsw_I expect we'll have a release of 20.3 (containing the commitish you are interested in) into Ubuntu 20.10 (groovy) and will start an SRU process to release into Ubuntu Xenial (16.04), bionic and focal16:11
blackboxsw_agreed though. we can/should add some semblance of docs with that intent16:12
blackboxsw_well looks like it's that time again16:19
blackboxsw_#startmeeting cloud-init status meeting16:24
meetingologyMeeting started Tue Aug 25 16:24:27 2020 UTC.  The chair is blackboxsw_. Information about MeetBot at http://wiki.ubuntu.com/meetingology.16:24
meetingologyAvailable commands: action commands idea info link nick16:24
blackboxsw_#chair smoser rharper falcojr lucasmoura16:24
meetingologyCurrent chairs: blackboxsw_ falcojr lucasmoura rharper smoser16:24
blackboxsw_hey folks, welcome to another cloud-init bi-weekly (or bi-monthly) community status meeting16:25
blackboxsw_or semi-monthly16:25
blackboxsw_... anyhow. We use this platform/channel to discuss latest and greatest cloud-init, as well as ensuring that there are a couple of upstream developers present to field questions or discussion as needed.16:26
blackboxsw_We gather here in this IRC channel every 2 weeks to discuss current development tasks and progress on cloud-init. All questions and side-conversations welcome16:26
blackboxsw_we keep our meeting minutes from previous meetings here:16:26
blackboxsw_#link https://cloud-init.github.io16:27
blackboxsw_The topics we'll cover today: Previous Actions, Recent Changes, In-progress Development, Community Charter, Office Hours (~30 mins).16:27
blackboxsw_#topic Previous Actions16:27
blackboxsw_Looks like no carryover actions from last meeting16:27
blackboxsw_#topic Recent Changes16:27
blackboxsw_The following changes have landed in tip of master since last meeting (08/14/20).16:30
blackboxsw_found via git log --since 2020-08-01416:30
blackboxsw_#link https://paste.ubuntu.com/p/h2qxwKwvFw/16:30
blackboxsw_things to note. we *JUST* cut 20.3 upstream release as of 20 seconds ago. Thanks a bunch James(falcojr) for getting the Azure timeout pr up to close out this upstream release.16:31
blackboxsw_and thanks smoser, rharper and meena for all the reviews and work here.16:31
blackboxsw_in the last couple weeks we got fixes to reduce boot timeouts for certain azure accelerated network instances, fix oracle datasource retries, handle compressed user-data on juju deployed machines in 'cloud-init query', and early boot dhclient will not attempt to run outside of the /var/tmp sandbox directory if that directory is marked no-exec16:34
blackboxsw_thanks otubo for that noexec branch16:34
blackboxsw_#topic In-progress  Development16:35
blackboxsw_So thanks all for the 20.3 upstream release. We will be tagging that release and pushing that tag to master just after this meeting16:36
blackboxsw_falcojr: is our release lead for this upstream release and SRU so he'll be working through the release process work items. Thanks falcojr.16:36
blackboxsw_-next step on the release process is to publish to Ubuntu Groovy (20.10) and then queue up a -proposed cloud-init SRU upload into xenial, bionic and focal.16:37
blackboxsw_we will then begin the SRU testing (which we hope to keep at around ~7days)16:37
blackboxsw_as smoser mentioned earlier. cloud-init has to follow this process to update cloud-init in stable Ubuntu releases16:38
blackboxsw_#link https://wiki.ubuntu.com/CloudinitUpdates16:38
blackboxsw_An email will be sent to the cloud-init mailinglist notifying the community about the SRU under test in the event that folks have spare cycles to pitch in on some of the verification effort16:39
blackboxsw_also "in-progress" paride is cleaning up a bunch of automated CI-related failures and lint issues in cloud-init, so expect some branches against cloud-init on that front as well16:40
blackboxsw_#topic Community Charter16:40
blackboxsw_The following topics are still topics for ongoing community development anyone new to cloud-init, or with a bit of time could easily grab one of these bitesized tasks:16:41
blackboxsw_ JSON schema extensions to validate user-data before instance launch: https://bugs.launchpad.net/cloud-init/?field.tag=bitesize16:41
blackboxsw_- Datasource documentation and updates16:41
blackboxsw_-  cloudinit.net refactor into distro-specific networking subclasses cloudinit.distros.networking: https://bugs.launchpad.net/cloud-init/+bugs?field.tag=net-refactor16:41
blackboxsw_As always: thank you all for bug contributions, PR submissions, triage and discussion participation.16:41
blackboxsw_If anyone would like to be involved more than they currently are, please feel free to contact us here in IRC #cloud-init on Freenode or on the mailing list cloud-init@lists.launchpad.net and we can see how best we can get you "set up"16:41
blackboxsw_ #topic Office Hours (next ~30 mins)16:42
blackboxsw_his time of the meeting is really just an open door for any discussions, concerns, bugs, questions or general prodding of upstream devs to make sure existing development work is unblocked where possible.16:42
blackboxsw_*This time16:42
blackboxsw_while we're at it with meeting time.... I'll set the next cloud-init status meeting date in the topic of this IRC channel16:42
AnhVoMSFTWhen do you anticipate the next SRU would be?16:42
AnhVoMSFT(after 20.3)16:43
blackboxsw_AnhVoMSFT: thanks for the question. I hope today for 20.3, I think 20.4 will likely align with the next Ubuntu release cycle (20.10)  so, October, 202016:44
blackboxsw_as mentioned in passing earlier, we are looking at trying to create an upstream release of cloud-init when any significant feature set has landed in tip to make release verification easier (and hopefull include less sprawling sets of broad commit streams).16:46
blackboxsw_this ultimately may translated to 4-5 releases per year16:46
blackboxsw_I expect a frequency of < 3 months16:46
blackboxsw_#topic #cloud-init pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting Sep 8 16:15 UTC | 20.2 (Apr 26) | 20.3 (estimated Aug 19th) https://bugs.launchpad.net/cloud-init/+filebug16:48
blackboxsw_next status meeting sept 8th, same time ^16:49
AnhVoMSFTThanks. There's some planned work for Azure's pre-provisioning v2 and will need some change in cloud-init. We're trying to get a sense of when cloud-init SRUs land so that we can communicate the timeline16:49
blackboxsw_@AnhVoMSFT generally cadence has been ~3 months between SRUs, I think we'd like to see a slightly higher frequency than that because our last SRU was so costly. but I think expectation for this pass is likely 10/15/202016:52
blackboxsw_also, if we are looking to test features in Ubuntu Groovy 20.10 images, they don't require an SRU, so upload to that development release are possible any time (we could do that as frequently as weekly if need be)16:53
AnhVoMSFTthanks for the clarification @blackboxsw_16:56
blackboxsw_so as of today, I expect we'll have an SRU in October and likely something beginning of Jan16:57
blackboxsw_I'll take an action for us to communicate via mailinglist the next estimated SRU once this SRU for 20.3 closes out16:57
blackboxsw_#action bbsw add workitem to SRU release process to announce to mailinglist estimated next SRU timeframe16:59
meetingologyACTION: bbsw add workitem to SRU release process to announce to mailinglist estimated next SRU timeframe16:59
blackboxsw_and again we hope to have 20.3 SRU complete by next week16:59
blackboxsw_complete, as in publish the 20.3 SRU for next week into Ubuntu xenial and later16:59
blackboxsw_I expect today we publish tip of master (20.3) into Ubuntu Groovy (20.10) so expect to see it in your friendly neighborhood ubuntu cloud-images in the next day or two17:00
blackboxsw_falcojr: I just annotated tag 20.3 and pushed to upstream17:04
blackboxsw_so we have a signed tag17:04
blackboxsw_falcojr: so next task is https://trello.com/c/KxShylli/14-upload-source-tarball-to-launchpad17:04
falcojrCool, I can jump on that17:05
blackboxsw_ok I think we'll wrap up the status meeting. Thanks all for tuning in!17:12
blackboxsw_#endmeeting17:13
meetingologyMeeting ended Tue Aug 25 17:13:02 2020 UTC.17:13
meetingologyMinutes:        http://ubottu.com/meetingology/logs/cloud-init/2020/cloud-init.2020-08-25-16.24.moin.txt17:13
Carbrich_rmeena I Was logged out, so I have no idea if you wrote back.17:21
blackboxsw_meeting minutes for the cloud-init status meeting just posted to https://cloud-init.github.io/status-2020-08-25.html#status-2020-08-2517:48
rharperblackboxsw_: thanks!17:49
blackboxsw_np; and ditto17:49
blackboxsw_it'll be good to get this release out and verified17:49
meenaCarbrich_r: nope, sorry. i had left to pick up my daughter17:56
meenawho's still not asleep17:56
Carbrich_rmeena No problem I also went a away to pick-up my two boys :)17:56
Carbrich_rI have create a bug report on: https://bugs.launchpad.net/cloud-init/+bug/1892902 With some more log details.17:57
ubot5Ubuntu bug 1892902 in cloud-init "Cloud-init received SIGTERM and is thereby unable to install packages or running runcmd completely" [Undecided,New]17:57
Carbrich_rmeena Now seeing ubot5 I am in a doubt if that was the right place :S17:57
Carbrich_rI`ll have to arrange a few things I`ll check later on.18:00
meenaI've been bringing my daughter to daycare by bike, and picking her up, by bike, since Friday. and today, i can't feel my body where it doesn't hurt.18:00
meenaCarbrich_r: now that North America is awake, and the meeting is over, it would be good to repeat your question18:01
Carbrich_rI`ll try to do it later on this evening :_18:02
Carbrich_rdoorbell is going18:02
meenaall the people with an @ in front of their name know vastly more than i… aaaaand gone18:14
Carbrich_rHI All I have the strange issue that cloud-init sigterms during a runcmd or package install with the native package provider. I have create a bug on https://bugs.launchpad.net/cloud-init/+bug/1892902. That explains my issue  I hope.18:15
ubot5Ubuntu bug 1892902 in cloud-init "Cloud-init received SIGTERM and is thereby unable to install packages or running runcmd completely" [Undecided,New]18:15
Carbrich_rmeena Thx for the tip! about U.S. part ot the world :)  That is getting awake.18:16
=== Carbrich_R_ is now known as Carbrich_R
Carbrich_R.19:58
anankeI'm having a hell of a time trying to figure out why cloud-init throws an error with the following apt config: https://dpaste.com/GZMDAVVEM20:52
anankethe error is TypeError: 'str' object does not support item assignment20:52
anankeand here's a bigger section of the log: https://dpaste.com/F526QRPAW20:53
anankeI'd appreciate any hints on what I may be doing wrong20:53
anankeseems the issue persists if I remove the key part, so it's just the source21:02
beantaxiUntil someone more knowledgable comes along ... which wouldn't be hard ...21:03
beantaxiIt does not appear that your config matches the structure in the example on https://cloudinit.readthedocs.io/en/20.2/topics/modules.html#apt-configure21:04
=== meena7 is now known as meena
=== Gaffel_ is now known as Gaffel
beantaxiThey have sources.source1.source and .key. You have sources.source1 and .key21:05
beantaxiI have zero idea if that is relevant or not. Just something I noticed.21:05
anankeI was following the example from https://cloudinit.readthedocs.io/en/latest/topics/examples.html#install-and-run-chef-recipes, not sure if that differs in that aspect21:07
beantaxiYep. looks like you do match that example. Sorry - that's all I got.21:08
anankethough I have nothing to lose to try the other way, thank you :)21:08
beantaxiGood luck!21:10
anankeha! it worked! thank you so much21:11
beantaxiNo way21:12
beantaxiHaha -- good on you for trying it!21:12
powersjananke, if you could, can you file a doc bug @ https://bugs.launchpad.net/cloud-init/+filebug21:12
anankepowersj: will do! later tonight if you don't mind, parent-teacher conference time21:13
powersjno rush, but want to remember to fix this21:13
blackboxsw_falcojr: I'm reviewing your upload pr https://github.com/canonical/cloud-init/pull/548   by performing the same basic new-upstream-snapshot steps now to confirm we see nearly identical diffs21:39
blackboxsw_confirmed no reall diff (other than pkging author) between your branch and mine for upload21:42
blackboxsw_falcojr: https://paste.ubuntu.com/p/Jhvp3YQwZF/21:42
blackboxsw_I'm confirm sbuild-it works and then I'll hit build-and-push to perform the upload21:44
blackboxsw_assuming that succeeds, we are good to go on groovy release of 20.321:44
blackboxsw_bah james I missed the following diff on the ubuntu/devel branch22:05
blackboxsw_https://paste.ubuntu.com/p/RhN4fpbkX5/22:06
blackboxsw_putting up a minor pr now22:06
blackboxsw_n/m I just changed UNRELEASED to groovy in debian/changelog.  which we should have done via a sed command on your branch, but that was my fail to catch it (why I'd like to do more frequent releases, so my muscle memory can take care of things so my brain can forget them)22:12
blackboxsw_falcojr: upload to groovy complete. we can send out the release email now22:12
blackboxsw_thanks @!22:12
blackboxsw_community notice: [ubuntu/groovy-proposed] cloud-init 20.3-0ubuntu1 (Accepted)22:15
blackboxsw_ok 20.3 release is "in" for Ubuntu Groovy (20.04) should show up in cloud images in the next day or two22:15
blackboxsw_falcojr: so we can now start the SRU for 20.3 into Xenial, Bionic and Focal. (once your 20.3 release email is out to the mailing list)22:16
blackboxsw_thanks again22:16
falcojrSounds good. Thank you22:26

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!