=== frickler_ is now known as frickler
meenaabsolutely love how the most innocent do this on *BSD functions turns into a can of worms https://github.com/canonical/cloud-init/pull/29810:50
=== beezly is now known as beezly[away]
=== beezly[away] is now known as beezly
beezlyHello all - I'm trying to configure an EFS mount using the mounts module in cloud-init, but I'm hitting a problem. It appears to completely ignore the entry in the mounts yaml.13:24
beezlyHello all - I'm trying to configure an EFS mount using the mounts module in cloud-init, but I'm hitting a problem. It appears to completely ignore the entry in the mounts yaml.13:26
beezlyoops - sorry about that.13:26
beezlyI can see it reading the YAML ok and it says... mounts configuration is [['fs-10e8d4db:/', '/opt/jenkins_workspace', 'efs', 'defaults,_netdev', 0, 0]]13:26
beezlybut then it says "Attempting to determine the real name of fs-10e8d4db:/" followed by "Ignorming nonexistant named mount fs-10e8d4db:/"13:27
Odd_Blokebeezly: It looks to me like the code doesn't handle "devices" that don't look like an actual /dev/... device, unfortunately: https://github.com/canonical/cloud-init/blob/master/cloudinit/config/cc_mounts.py#L11513:42
beezlyyeah - i'm just looking through that. Might have a patch soon-ish.13:42
beezlyI'm making an assumption that anything that matches ^.+:/.* should be considered a network device.13:43
beezlywhat version of python do people develop cloud-init against?13:44
Odd_BlokeWe support 3.4+.13:45
Odd_Blokebeezly: https://cloudinit.readthedocs.io/en/latest/topics/hacking.html lists the steps to get setup for development (presumably you know what you're doing with git &c., but you will need to sign the CLA).  Thanks for looking at this!13:46
beezlyok, thanks.13:46
beezly@Odd_Bloke I'm finding the Launchpad/Github process really confusing. I'm trying to create my launchpad fork at lp:~beezly/cloud-init but each time I push I get "fatal: remote error: Path translation timed out."14:47
Odd_Blokebeezly: Apologies, we appear to have a doc building error, but there are new docs on CLA signature here: https://github.com/canonical/cloud-init/blob/master/HACKING.rst14:49
Odd_Blokebeezly: But I can see your GH username in our CLA records, so you're good to go.14:49
beezlyah great!14:49
Odd_BlokeThe docs I pointed you at (which I fixed in master yesterday) predate the GitHub username field being in the CLA form.14:50
Odd_Blokerharper: blackboxsw: If I could get a quick review of https://github.com/canonical/cloud-init/pull/301 to add beezly as a CLA signer, I'd appreciate it.15:13
blackboxswOdd_Bloke: approved15:14
=== powersj changed the topic of #cloud-init to: pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting April 14 16:15 UTC | 20.1 (Feb 18) | 20.2 (TBD) | https://bugs.launchpad.net/cloud-init/+filebug
=== hjensas is now known as hjensas|afk
beezly@Odd_Bloke thanks for the super-quick turnaround on that stuff. I just tried out that PR on an EC2 instance of mine and it works as expected.15:56
Odd_Blokebeezly: Nice, I'm bouncing between a lot of meetings today, but will hopefully get to reviewing it before too long. :)16:02
beezlyno problem - I'll try and respond quickly, but I have a pretty full day tomorrow so it might be the weekend.16:03
Odd_Blokerharper: https://github.com/canonical/cloud-init/pull/300/ <-- is r"^.+:/.*" a reasonable regex to match NFS shares (remote:/path/on/server) and no other devices that you're aware of?16:27
rharperOdd_Bloke: hrm16:30
Odd_Bloke(Disk devices, that is.)16:30
Goneri.+ will catch the spaces. This is not what you want.16:32
Odd_BlokeYou can't have spaces in NFS mount targets?16:33
Gonerinot in the hostname, maybe in the mount point.16:34
Odd_BlokeHmm, not as we're going to put it in fstab, I think.16:34
Odd_BlokeThey'd have to be escaped to \040.16:35
Goneriand here come the non-breaking space :-)16:35
rharperOdd_Bloke: AFACIT, ":" in the spec will be considered NFS,  left of the colon is a hostname or FQDN, and afterward is any "path" spec16:39
rharperFQDN/hostnames cannot have whitespace IIUC, and paths cannot either without encoding/escaping16:40
Odd_BlokeI think you _might_ be able to have unencoded/escaped whitespace if you're just running mount yourself, but it definitely won't work in fstab (which is whitespace-separated) regardless.16:41
Odd_BlokeSo I think what we would want in an ideal world is two checks.  First, we check if something looks like it's intended to be a network share (which the proposed regex is sufficient for, I think).  If it does then, second, we check if it's valid for use in an fstab, and emit a warning if it isn't.16:46
Odd_BlokeOr, in fact, that second step could perform the encoding and only emit a warning if that also fails.16:47
Odd_BlokeSo I think I've convinced myself that this check is fit for purpose: it allows people to configure NFS shares through cloud-init, and shouldn't match anything other than NFS shares.16:48
Odd_BlokeWe could further improve the experience to not configure broken mounts, but users could also do that themselves by modifying their cloud-config.16:49
Odd_BlokeWhereas today they can't even do that.16:49
rharperWe don't issue the mount directly; right, we only encode it into fstab entry and we mount -a16:52
rharperso, it has to be correct enough to have fstab parsable16:52
Odd_BlokeIf `mount -a` will skip unparseable lines, then we don't necessarily have to have it parseable.16:55
rharperI think we get a unit failed error16:56
rharperit's not fatal to cloud-init16:56
Odd_BlokeArguably it's easier for a user to figure out what's going on if we write out the unparseable line than if we just emit a warning.16:56
rharperthinking, we don't normally sanitize/fixup cloud-config provided values16:56
rharperI was heading just there16:56
Odd_BlokeYep, I wasn't disagreeing with you. :)16:56
rharperand I was just violently agreeing with you as well , haha16:57
Odd_BlokeOK, so I think that regex is fine, except I'm not sure if there _must_ be a leading slash on the path.16:58
rharpercould we not just check for ":" in the line ?16:59
rharpersince we're not parsing it ?16:59
Odd_BlokeYeah, that's what I'm wondering about.17:00
rharperand we could possibly help with the _netdev option , if that's not present17:01
rharperhttps://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html  is useful;17:02
Odd_BlokeYeah, I think we could improve the experience further, and I might ask beezly if they're interested in doing more work after this.17:02
rharperI wonder if the efs-utils is in the ubuntu image? or the aws snap ?17:03
Odd_BlokeThere isn't a likely-looking package in the archive, so I don't think it's deb-packaged at least.17:03
GoneriOdd_Bloke, could you please remove the "wip" flag here https://github.com/canonical/cloud-init/pull/29817:04
blackboxswdone Goneri17:04
blackboxswI'm assigned to that PR as of today. will get you a review there17:05
Gonerialso, is there anything required here: https://github.com/canonical/cloud-init/pull/28917:05
Goneriblackboxsw, I'm mostly just some over due cleanup/refactoring.17:05
blackboxswsounds good17:05
Goneriblackboxsw, ultimately I would like to move these functions in distros/17:05
Odd_Blokerharper: OK, so I'm thinking I'll ask beezly to drop the "/" from the regex, and call that good.  OK with you?17:06
rharperOdd_Bloke: any reason for the regex over just the ':'  ?17:20
blackboxswOdd_Bloke: what git commits should I be cherry picking into ubuntu/devel for the dropping invalid underscores from hostnames18:51
blackboxswI was thinking the following in this order: 4f825b3e6d8fde5c239d29639b04d2bea6d95d0e  c478d0bff412c67280dfe8f08568de733f9425a1.... but I think I might need another before c47818:54
vasartoriHi all, I'm getting a strange behaviour in cloud-init with https://github.com/vmware/cloud-init-vmware-guestinfo. When the vm is customized by open-vm-tools, it works, after reboot, cloud-init (from user-data) fails, because an directory exists (/var/lib/cloud/instance). If I remove manually this directory, works. Any tip?18:55
rharperOdd_Bloke: so, I just rebased Goneri 's set-passwd change, and now the squash-and-merge wants to put in a Co-Authored by me (which it is not) ... how do I merge the PR and ensure that Goneri is in the author field ?18:58
rharpervasartori: there are known issues with vmware workflows and cloud-init, specifically around changing the instance id.  There's upstream work on trying to sort out how to prevent changing the instance-id but still running for additional "customizations" ;  it's somewhat a misunderstanding of how cloud-init is designed (it performs most customizations just once) where the vmware workflows invole multiple rounds of customization aainst the same19:00
Gonerirharper, I can rebase it myself if you prefer19:01
blackboxswGoneri: that'd work , but rharper that issue was a github snaffu. they broke something on the backend when I had merged. I think make sure that your squashed commit message does not contain Co-Authored-By: ryan.harper as a footer19:03
rharperGoneri: I think if you do and force push, that should reset it to you as last author19:03
vasartorirharper So,  should i give up using it?19:03
rharpervasartori: I don't know;19:04
GoneriI've a script for that: https://gist.github.com/goneri/c0fab52388465f5da2b2979ee5166f0819:04
blackboxswvasartori: it'd be nice if we could ultimately work with vmware somehow to get that datasource upstream in cloud-init to ease support also it'd be good to file a support ticket against VMWare somehow reporting that issue. But, there may be a bug in cloud-init upstream that still needs resolving as rharper states19:04
rharpervasartori: as a community, we don't have access to all the different versions of vmware, customizations, platforms; so it's difficult for us to develop a way to 1) provide uniqie instance-ids to vmware VMs  2) design a workflow that translates the "recustomize" and alreadu "customized" VM;   we continue to work with the VMware engineers when they submit PRs to cloud-init;19:08
blackboxswyeah it's a strange relationship that we hope can be improved with engagement from VMware and the cloud-init community in general through fix requests against vmware (and our discussions with VMWare reps/developers)19:12
rharperblackboxsw: they left19:14
blackboxswwell shucks19:15
vasartorirharper and blackboxsw Thanks a lot for the answers... I'll try to talk with vmware...19:20
Odd_Blokerharper: I believe that they reverted things so that the submitter of the PR is the author of the squashed commit.19:20
blackboxswthanks vasartori I had worried you got frustrated and took off :)19:21
Odd_BlokeSo you shouldn't need to do anything special, just remove the Co-Authored line.19:21
blackboxswvasartori: the more voices, the more likely we can improve engagement thanks19:21
vasartoriblackboxsw my internet has gonna away :p . You are 100% correct, I'll try to make some noise there :p19:22
Odd_Blokeblackboxsw: I believe you'll want (in reverse order): 1bbc4908f c478d0bff 2566fdbee c5e949c0219:25
Odd_Bloke(`git cherry-pick c5e949c02 2566fdbee c478d0bff 1bbc4908f` onto ubuntu/devel looks right to me, and pytest runs.)19:25
Odd_Blokes/runs/passes/ of course. :p19:26
rharperOdd_Bloke: I see (just remove the co-author bit);19:29
rharperI hope so, even after a rebase / force-push by Goneri the UI still wants to inject that Co-Authored line...  dropping that and merging19:29
rharperOdd_Bloke:  yeah, that worked.  merged and Goneri has credit.19:35
blackboxswrharper: Odd_Bloke I've put up a cherry-pick branch for review into ubuntu/devel with neew tooling20:03
meenaoh no github down20:53
blackboxswwell at least github has a near 500 error splashscreen21:01
meenawell, time to sleep then!21:05
meenaGood night.21:05
blackboxswOdd_Bloke: rharper, the big question with https://github.com/canonical/cloud-init/pull/303 is do we want to support --force push of dropped commits in ubuntu/devel per cherry-pick -t in https://github.com/CanonicalLtd/uss-tableflip/pull/4521:14
blackboxswor do we want to re-add cherry picks that we dropped to support fixing our daily recipes?21:15
* rharper has to shift metal gears .. 21:15
rharperIIUC, we shouldn't have to force push, if we cherry pick, when we merge, the merge will see we've already landed the cherry picks ... no ?  /me 's git fu is weak-sauce21:16
rharperwe have a list of cherry picks;  we need to cherry pick each one, refresh patches, update changelog.   then post release, we'll do a new-upstream-snapshot, the merge should see the cherry picks, I though it ill replay the commits that are interleaved automatically ?  or does it not do it that way ?21:17
blackboxswrharper: we'll have to --force if we drop the 'dropped cherry-picks to make daily recipe build'21:20
rharperblackboxsw: I thought daily was master with merged release branch ... and IIRC, the steps post cherry-pick and release is to remove the cherry picks on the release branch, right ?21:20
rharperwe only need the cherry picks *in* for a release21:20
rharperonce we've uploaded, we can un do them21:20
rharperright ?21:20
rharperwhich is what the refresh --patches only was to be ?21:20
blackboxswrharper: correct. we only need cpicks 'in' for a release21:20
blackboxswin this circumstance, we've already released a cherry pick once and applied a followup couple commits to remove that cpick so dialies work21:21
blackboxswrharper: right but refresh --patches is fairly shallow in that it actually imports all commits via new-snapshot, but only uses that to drop cpicks, so the code in ubuntu/devel will now officially match tip21:21
blackboxswthe intent of --refresh-patches was that our next release would perform the full new-upstream-snapshot and end up listing all the commits is had already pulled into ubuntu/devel in the debian/changelog21:22
blackboxswduring feature-freeze, we can't import all commits from tip to properly refresh/drop patches, because we also can't release those other non-cherry commits21:23
rharperthen I'm not sure what problem we're fixing ... we manually cherry pick, release/upload,  and commit changes to drop cpick patches right ?21:23
rharperat that point, daily still works;21:23
blackboxswrharper: right I'll draw up a hackmd doc with the scenarios we are handling.21:23
blackboxswthen we can comment on best plan for a second cherry-pick release21:24
rharperthe new case is, we'd like to release a second cpick only release ?21:24
rharperand what does't work there ?21:24
blackboxswrharper: here are the use-cases https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?both21:43
blackboxswand yes this new cases is second cpick release instead of new-upstream-snapshot21:43
blackboxswso we already have dropped the earlier released cpick from ubuntu/devel so we either need to unwind that dropped cpick to add new cpicks, or re-add all cpicks and re-release21:44
blackboxswI'm not sure about the best approach. maybe the best approach is not to unwind/pop the dropped cpick commits. and just re-add the original cpicks that we dropped from ubuntu/devel which allowed us to fix daily builds21:45
blackboxswalso, maybe I'm misunderstanding what we did with new-upstream-snapshot --update-patches-only. I thought it actually pulled in all commits. rerunning now to confirm.21:50
rharperblackboxsw: so, I was thinking for cpick releases, we'd have a file with the cpick hashes we wanted ... then we could easily revert those to get daily working, and then if we did a second cpick release, the list is just longer this time (we update it with new cpicks) and then pull them out afterwards21:52
Odd_BlokeFound a gap in our pytest testing vs non-pytest testing, which I've addressed in https://github.com/canonical/cloud-init/pull/30421:53
Odd_Bloke(As it happens, no pytest tests are using util.subp, but this will catch any future additions.)21:54
blackboxswrharper: the problem is if we are reverting the original cpick hashes from ubuntu/<series> then we've botched the commits that we ended up releasing already21:57
blackboxswso we don't have an exact point anymore in ubuntu/<series> that was used to build/dput21:57
rharperdo you want to do hangout?  I might be faster21:57
blackboxswso maybe the best bet is to just re-cpick21:57
blackboxswyeah I'd like to21:57
rharperstandup ?21:58
Odd_Blokeblackboxsw: I'm not on the call so this may not be helpful, but reverts in git create a new commit (whereas in bzr they reset the history to before the commit in question).  Just in case there's a confusion in nomenclature, a git revert would not mean we'd lose the commit that was previously uploaded.22:00
Odd_Bloke(Apologies to you and rharper if I've just stepped all over your conversation unhelpfully. :p)22:00
blackboxswOdd_Bloke: always a help. will read now22:10
blackboxswrharper: Odd_Bloke ok PR 303 in in good shape I think for cloud-init22:45
blackboxswupdated description with the manual steps I performed22:45
blackboxswand shouldn't involve a --force push due to mangled commitishes22:45
* blackboxsw starts reworking the uss-tableflip script changes for handling --pop-all22:48
rharperblackboxsw: +1, Iemme look22:57
rharperblackboxsw: "I manually deleted the 'drop ec2 secondary nics cpick changelog" ; you just opened the file and deleted content ? or is there some dch command that will drop it ?23:00
blackboxswrharper: hrm my dch command dropped me into that (maybe because I actually wasn't running that as a bash script but individual commands on the commandlin23:01
blackboxswif you try that separate cherry-pick 6600c64 on the commandline directly (instead of inside a bash script) it might interactively prompt you for the changelog changes that were added. I always have to :wq after each step23:02
blackboxswmaybe it's because I have EDITOR=vi in my env?23:04
rharperblackboxsw: I see, you manually removed it after running cherry pick, during the dch -i interactive session23:08
rharperI've reproduced your branch;  have you build-package and test-built it yet ?23:08
blackboxswrharper: will do that now. was testing tooling (which I think is almost there for --pop-all)23:09
rharperblackboxsw: I'll hold off approving; but I've commented that I can reproduce your branch23:11
rharperI'll check back later23:11
blackboxsw+1 thanks rharper23:12
blackboxswrharper: ok tooling isn't going to change this manual process I forgot23:27
blackboxswso 303 is good23:27
blackboxswif +1'd then I can upload that.23:28
blackboxswtomorrow we can sort the uss-tableflip and my followup PR to --pop-all for daily recipe builds23:28
* blackboxsw needs to make dinner23:28
blackboxswbuild-package works on my branch23:47
blackboxswcan build-and-push if agreed23:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!