/srv/irclogs.ubuntu.com/2019/12/17/#cloud-init.txt

=== tds6 is now known as tds
Gonerimeena, hum, let's me check01:45
Gonerimeena, my bad, I forgot it during the last rebase. sorry. I just pushed a fix.01:48
rharpersmoser: http://paste.ubuntu.com/p/CQRgW9HQRw/03:05
smoserrharper: yeah, i verified that works too.  was ripping the ctn_cfg out from other places.03:13
=== MrGeneral_ is now known as MrGeneral
=== paride4 is now known as paride
=== blackboxsw_ is now known as blackboxsw
=== benj_- is now known as benj_
=== cyberpear_ is now known as cyberpear
meenapowersj: vendor-data is a bit underdocumented.10:33
meenaI fucked up.11:02
Odd_Blokemeena: Should we revert https://github.com/canonical/cloud-init/commit/e2840f1771158748780a768f6bfbb117cd7610c6 ?15:10
meenaOdd_Bloke: i guess we an leave the test15:14
meenaI'll blame it on mental overload15:14
Odd_BlokeThese things happen!15:17
Odd_BlokeParticularly in complex code bases. :p15:17
Odd_Blokemeena: Do you want to propose the revert, or shall I?15:17
powersjmeena, is that a kind hint that you want me to go write more docs :)15:24
smoserrharper: https://github.com/cloud-init/qa-scripts/pull/1515:26
meenaOdd_Bloke: im on the go, rn, so if you're in the code, go ahead15:26
meenapowersj: if i knew how it works, i'd do it myself 😅15:27
Odd_Blokemeena: Cool, will do.15:27
Odd_Blokemeena: https://github.com/canonical/cloud-init/pull/11615:34
Odd_Blokepowersj: I don't appear to be able to add non-Canonical folks as reviewers on PRs. :/15:34
Odd_Blokemeena: (I would have added you as a reviewer if not for ^. :p)15:34
powersjOdd_Bloke, nor am I15:35
rharpersmoser: thanks!15:37
Odd_Blokepowersj: I've tried adding meena as a Read permitted collaborator.15:51
Odd_BlokeTo see if that adds them to the list.15:51
Odd_Blokesmoser: rharper: So is that profile PR a fix for the pstart issue we're seeing, or a simplification so we can reproduce it with less moving parts?15:54
smoserthat shoudl fix it, Odd_Bloke15:56
smoserthe issue was the "cached" (memory) config that we restored on exit was stale.15:56
rharperOdd_Bloke: I used smoser simplified reproducier to file the issue with lxd yesterday15:59
smoserand then we got the response expected15:59
rharperand stgraber pointed out the error and the solution as well15:59
smoserand used --dump-commands15:59
rharperah, yes15:59
smoserwhich was helpful.15:59
rharperand I followed up with a, and what _should_ we do15:59
rharperso, team effort to extract a solution16:00
smoserthe one thing i'im not sure about is how old 'lxc profile add'16:00
smoseris16:00
rharperright, I've not verified it on Xenial16:00
rharperbut we're running bionic + snap-based lxd in CI16:00
smoserseems old enough.16:00
rharperooo, it's 19.4 day16:03
smoserhttps://github.com/cloud-init/qa-scripts/pull/16 gets rid of:16:04
smoser  rcontainer = remote + ":" + name if remote else name16:05
meenaOdd_Bloke: did i approve too early (before accepting your invitation?) or do i still have no write access, like my github message says?17:04
Odd_Blokemeena: I didn't grant you write access, I just granted you explicit read access, in the hopes that would mean you showed up in the drop-down listing potential reviewers.17:06
Odd_BlokeAnd I _think_ that worked.17:06
blackboxswOdd_Bloke: that looks good on my side and I see someone was able to set meena as reviewer17:10
blackboxswon the PR17:10
Odd_BlokeI believe meena just reviewed it of their own accord.17:16
Odd_BlokeRather than it being requested.17:16
Odd_Bloke(Obviously it _was_ requested, just not through the GH UI. :p)17:17
blackboxswfollowup suggestion on powersj's set hostname branch https://github.com/canonical/cloud-init/pull/109#discussion_r35854398817:17
Odd_Blokeblackboxsw: Which platforms base dynamic DNS on the initial DHCP requests?  (I thought those were the platforms that the current behaviour doesn't work well for?)17:20
blackboxswand per our conversation yesterday Odd_Bloke I'm in agreement with you I think. If metadata or user-data set a hostname for a platform, it seems correct for cloud-init to assert that the hostname remains set per the opinionated deployment directives.17:20
blackboxswOdd_Bloke: juju driving some version of vSphere (so OVF + juju). I believe the interactions of both substrates rely on DDNS that falls over during our init-local sandbox DHCP attempts caused the DDNS in that scenario to keep transfering DNS entries for the 'ubuntu' hostname to different vms as they are brought up17:21
blackboxswOdd_Bloke: more context on that is here https://bugs.launchpad.net/cloud-init/+bug/1746455/comments/5  and here https://bugs.launchpad.net/cloud-init/+bug/1746455/comments/2317:24
ubot5Ubuntu bug 1746455 in cloud-init "cloud-init vSphere cloud provider DHCP unique hostname issue" [High,Fix released]17:24
blackboxswOdd_Bloke: so juju deploys (or deployed) some magic that adds DHCP hostname request logic to the deployed target (and a lot of cloud-init user-data). It is possible that there is an issue in juju machine deployment logic that needs a bit more investment to make sure to avoid triggering this case17:27
Odd_Blokeblackboxsw: I'm a little confused.  That's a case where this behaviour causes problems, but your proposed paragraph suggests that that's the motivation for having this behaviour.17:29
blackboxswOdd_Bloke: sorry I wasn't clear here. That is the case where cloud-init's behavior of setting hostname from metadata before running sandboxed dhcp client fixes that issue.17:30
blackboxswif we didn't set hostname before our sandbox dhclient run, we would break juju deploying on vsphere17:30
Odd_BlokeOh, you know what, I'm thinking of Azure-specific behaviour.17:31
Odd_BlokeWhich is handled by the datasource, not through this route.17:31
blackboxswI don't really know about platforms on which us setting hostname early and repeatedly resetting is causing problems17:31
blackboxswahh right Or17:31
blackboxswahh right Odd_Bloke17:31
Odd_BlokeWhich explains why I couldn't understand what was going on. :p17:32
blackboxswyet, doesn't explain why I didn't. But, some mysteries are better left unsolved17:32
* powersj looks for 19.418:25
blackboxswOdd_Bloke: powersj rharper I know we are in discussion about best process for reviewing PRs quickly19:24
blackboxswcan we in the nearterm label a PR with incomplete if it is waiting on Proposer feedback/resolution?19:24
Odd_Blokeblackboxsw: I have reservations about introducing a stopgap when we plan on having a full process in the next few days.19:32
blackboxswthanks Odd_Bloke +119:32
blackboxswwill leave PRs undecorated at the moment19:32
* blackboxsw was going through initial upstream 19.4 board tasks 19:33
Odd_Blokeblackboxsw: Did you cut 19.4 yet?19:33
blackboxswOdd_Bloke: nope, doing that in a matter of a few minutes19:33
Odd_Blokeblackboxsw: Because I think https://github.com/canonical/cloud-init/pull/116 is worth including.19:33
blackboxswjust reviewing CI right now19:33
blackboxswok Odd_Bloke will review that19:33
blackboxswOdd_Bloke: was there a bug associated with that issue?19:34
blackboxswthat PR I mean19:34
blackboxswas in, how did we/you discover the issue?19:34
blackboxswjust an errant extra chaneset that snuck  in in https://github.com/canonical/cloud-init/commit/e2840f1771158748780a768f6bfbb117cd7610c6 or something else triggered discovery of the issue?19:36
blackboxsw*changeset*19:36
blackboxswmaybe here is the context I was missing https://bugs.launchpad.net/cloud-init/+bug/1854594/comments/819:39
ubot5Ubuntu bug 1854594 in cloud-init "lock passwd implemented wrong on FreeBSD" [Medium,Incomplete]19:39
blackboxswthe last comment being, oops shouldn't have done that19:39
meenablackboxsw: basically, the thing that hid the issue was system_info/distro being overwritten ➕ me misinterpreting pw(8) wrong.19:41
blackboxswmy question around that PR is that https://bugs.launchpad.net/cloud-init/+bug/1854594 isn't really fixed then right?19:41
ubot5Ubuntu bug 1854594 in cloud-init "lock passwd implemented wrong on FreeBSD" [Medium,Incomplete]19:41
blackboxswand wasn't really broken either right19:41
meenablackboxsw: you said something about reading comprehension too this morning, i believe.19:41
meenablackboxsw: yes, wasn't broken ☹ shouldn't have fixed it.19:41
meenai'm very sorry about this mishap.19:42
blackboxswroger doger. ok I'm marking that bug as invalid then. no worries19:42
blackboxswok merged19:43
meenamy perfect score is gone19:43
Goneriyou're still perfect <3 :-)19:44
Odd_Blokemeena: Honestly, don't worry about it, these things happen.19:44
Odd_BlokeAnd we caught it!19:44
blackboxswheh.19:45
blackboxsw+119:45
meenaOdd_Bloke: i just wish we caught it before SRU release xD19:45
Odd_Blokemeena: The SRU doesn't affect FreeBSD, so no harm there.19:45
Odd_Blokeblackboxsw: So I've got someone to open up the CLA migration MP and PR.  Do I merge the PR and close the MP out?19:46
blackboxswOdd_Bloke: since we can't use review-mps from git@github.com:CanonicalLtd/uss-tableflip.git  anymore due to our security config, we'll have to do that manually. Right, merge PR locally once you confirm the matching LP Branch proposal on associated LP account. And Manually close the LP-side branch proposal with a comment pointing to the upstream github commitish19:48
blackboxswand you've probably seen ad-m's next PR that can followup after this19:48
Odd_BlokeYep, I'm working through his things ATM.19:48
blackboxswtkx19:48
Odd_Blokeblackboxsw: Why do I need to merge the PR locally?19:49
Odd_Bloke(As opposed to using the GH UI.)19:49
blackboxswOdd_Bloke: sorry I meant using the GH UI. (I meant 'local' in the sense of in github, not over in launchpad)19:50
Odd_BlokeCool19:50
blackboxswOdd_Bloke: rharper what do we think? https://github.com/canonical/cloud-init/pull/7719:50
ahosmanMSFTHi all, I'm working on this cloud-init (Azure) bug that might be connected to cloud config. Here is the behavior: start a vm, ssh via password, cloud-init clean, then restart. Now the user can't ssh via password. Looking into DataSourceAzure, the functionality is there, but the behavior doesn't match. Am I missing something? https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceAzure.py#n120619:51
blackboxswstatic6 eni support for 19.4?19:51
blackboxswI'm thinking hold as it needs a bit more review and we will have a 20.1 early next year.19:52
blackboxswhi ahosmanMSFT.19:52
blackboxswahosmanMSFT: could it be related to the byteswapping branch that just landed?19:53
blackboxswI didn't see this in SRU testing19:53
blackboxswwhich didn't have your fix19:53
ahosmanMSFTblackboxsw: This bug has apparently existed for a while19:54
blackboxswahosmanMSFT: a good attempt would be to try reproducing on current xenial vms19:54
blackboxswto see if it has the same broken behavior19:54
ahosmanMSFTI've reproduced the behavior on both xenial and bionic19:55
blackboxswgenerally after a cloud-init clean all cloud-init semaphores should be removed and all modules should get re-run19:55
blackboxswok is there a bug filed ?19:55
ahosmanMSFTblackboxsw: Not sure, saw this in our internal backlog and picked it up19:56
ahosmanMSFTIt's because the password get's redacted in ovf-env.xml file19:57
blackboxswbest bet is to check to see if a bug is filed @ https://bugs.launchpad.net/cloud-init/+bugs. If not, then try filing a bug at https://bugs.launchpad.net/cloud-init/+filebug file a bug  with the cloud-config required to show the problem and attach logs tarfile from cloud-init collect-logs .  Generally we'd expect to see the ssh, password and user_groups cofig modules running after a clean.  But, due to sourcing20:00
blackboxswcontent from ovf-env.xml this information might not be present anymore after a clean.20:00
blackboxswas you said, if it was redacted and only run once20:00
blackboxswthen new cloud-init boots after clean may not have that metadata available anymore20:00
blackboxswTBH, cloud-init clean is a developer tool, and a blunt hammer.  It's not intended for typical use-cases for ongoing system maintenance20:01
blackboxswso it is not a typical consumer use pattern that I'd expect you to jump through hoops to support if the initial provisioning of a vm does properly setup that configuration20:02
ahosmanMSFTblackboxsw: Is there a way to persist metadata, or mitigating the issue cleaning+restarting causing password lock. You are right, the typical consumer wouldn't run in to this. But I'd like to find some resolution20:22
rharperahosmanMSFT: sounds similar to this, https://bugs.launchpad.net/cloud-init/+bug/184967720:31
ubot5Ubuntu bug 1849677 in cloud-init "azure locks existing user if instance id changes" [Medium,Fix committed]20:31
rharperahosmanMSFT: w.r.t persistent metadata, it's not clear to me what you mean;   for debugging, I suggest creating an additional user on the system after the initial launch with known password and sudo privs;  then after a clean + reboot; login as the secondary user you created, so you can collect cloud-init logs20:33
ahosmanMSFTrharper: That is the bug I am dealing with. Creating a new user/pass was how I was getting on the vm to collect logs.20:38
rharperok, I didn't see any logs from the secondary boot to contrast with the first one;20:39
rharperahosmanMSFT: interesting, so it seems to me that it' may be a side-effect of redacting the password value in ovf-xml file;  that's not kept in original state in your boot, clean, reboot scenario, right ?20:41
rharperin which case, on the secondary boot (after clean) the xml contents are still redacted instead of the original password supplied ?20:41
rharperdoes that sound plausible ?20:41
rharperahosmanMSFT: on comment #2, https://bugs.launchpad.net/cloud-init/+bug/1849677/comments/2  ; there is mention that the iso only exists on first boot20:42
ubot5Ubuntu bug 1849677 in cloud-init "azure locks existing user if instance id changes" [Medium,Fix committed]20:42
ahosmanMSFTrharper: exactly, the ovf is cached via waagent20:57
ahosmanMSFTTrying the committed fix now...20:57
rharperah, ok20:57
rharpersounds like you've reproduced the issue that we now have a fix for20:58
meenaanyone considered adding CircleCI to Travis?21:14
meena… instead of just having travis.21:16
ahosmanMSFTblackboxsw rharper: Has this fix worked for you? https://bugs.launchpad.net/cloud-init/+bug/1849677/comments/521:32
ubot5Ubuntu bug 1849677 in cloud-init "azure locks existing user if instance id changes" [Medium,Fix committed]21:32
ahosmanMSFTI just tried it and it didn't work, still get locked out21:32
rharperahosmanMSFT: we did not test it directly21:34
rharperdo you have logs from first and second boot ?21:34
Odd_Blokemeena: Why would we use both?21:42
meenaOdd_Bloke: for freebsd.21:42
Odd_Blokemeena: Oh, I didn't know they supported it.21:45
blackboxswok upstream release bug complete for 19.421:46
blackboxswhttps://bugs.launchpad.net/cloud-init/+bug/185676121:46
ubot5Ubuntu bug 1856761 in cloud-init "Release 19.4" [Undecided,New]21:46
blackboxswadding a simple script update PR to make this a bit more automated21:46
meenaOdd_Bloke: at least last time i looked at it21:54
powersjblackboxsw, thank you!21:55
meenaOdd_Bloke: cirrus ci21:59
meenahttps://cirrus-ci.org/guide/FreeBSD/22:00
Odd_BlokeAha, OK.22:01
Odd_BlokeThat explains why I couldn't find it in the Circle docs. :p22:01
meenathey're all ci and c and stuff22:05
blackboxswhttps://github.com/CanonicalLtd/uss-tableflip/pull/2722:10
blackboxswupstream release process update and simple script22:10
blackboxswupstream release commit https://github.com/canonical/cloud-init/pull/12122:11
meenaso, anyone got anything against merging Goneri's pr, so we can go on with life and refactor cloudinit.net? https://md.hecke.rs/z17JGX4HT4emH5jTEhMuTA22:34
blackboxswthanks Odd_Bloke . scubbed changelog (we probably need that change for log2dch too now to clean out merge markers)22:39
blackboxswsrubbed even22:39
Odd_Blokeblackboxsw: Well, we shouldn't need to any longer, as we've disabled merge commits.22:48
meenawho allowed hetzner to be merged without documentation???22:50
blackboxswhrm I think I may have run out of daylight and reviewers on the upstream 19.4 cut for today, rharper or powersj or smoser if around I've queued  https://github.com/canonical/cloud-init/pull/121 from tip for cloud-init 19.4 release23:33
blackboxswif that lands today, I'll publish 19.4 to focal, and send a release email & discourse post out23:34

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!