/srv/irclogs.ubuntu.com/2020/04/08/#cloud-init.txt

=== tds0 is now known as tds
apollo13hello, anyone around to help me with network configuration?09:05
apollo13my cloud-init on debian properly writes the config to /etc/network/interfaces.d/50-cloud-init.cfg09:05
apollo13but also generates a dhcp template in /run/network/interfaces.d which obviously takes precedence but won't work because of no dhcp server in that network09:05
apollo13any idea what is causing that?09:06
apollo13also I need to rename 50-cloud-init.cfg to 50-cloud-init that it works09:07
=== cpaelzer__ is now known as cpaelzer
apollo13okay, I fixed that by upgrading cloud-init to > 19.211:06
apollo13sadly that is not yet in the buster images, but apparently on it's way11:06
apollo13so, cloud-init network configuration configures dns-nameserver in /etc/interfaces yet the debian cloud images do not have resolvconf, how is that supposed to work?11:09
tribaalapollo13: just a heads up - most of the active cloud-initers are in US timezones (I don't know how to answer your question personally)11:19
apollo13I have opened https://salsa.debian.org/cloud-team/debian-cloud-images/-/issues/2411:20
apollo13tribaal: thanks, my bouncer will stay online :)11:20
tribaalapollo13: hehe ack :) I guess opening the bug report was the right reflex - it is a good asynchronous contact point as well.11:24
apollo13in all fairness I am surprised how badly cloud-init seems to work every time I try it :(11:24
apollo13I mean I guess it is probably my fault for using a static network configuration instead of DHCP and I fully understand that getting those things right is hard, but I fail massively every time I try cloud-init :(11:25
Odd_Blokeapollo13: What is creating the file in /run/network/interfaces.d?13:40
apollo13Odd_Bloke: the ifupdown-cloud-init-helper13:40
apollo13but it only does this because cloud-init wries 50-cloud-init.cfg int /etc/network/interfaces.d which is a invalid name13:41
apollo13so that part got fixed by upgrading to cloud-init > 19.213:41
Odd_BlokeI'm not familiar with ifupdown-cloud-init-helper, what is that?13:41
Odd_BlokeWell, regardless, sounds like you're past that part of the issue.13:42
apollo13yes13:42
apollo13all that remains (and I also fixed that by remastering the debian cloud images) was DNS which got fixed by adding resolvconf to the image13:42
Odd_BlokeSo it sounds like this is an issue with the mastering of the Debian cloud images more than it is with cloud-init?13:43
apollo13yes, seems like it13:43
apollo13hence I opened an issue at https://salsa.debian.org/cloud-team/debian-cloud-images/-/issues/2413:43
apollo13apparently they only test the dhcp boot cases, dunno…13:43
Odd_BlokeOK, cool, was just making sure I'd understood fully.  Thanks for bringing the issue here (even if you fixed it before the core team woke up ;).13:45
apollo13hehe, no issue after all…13:46
apollo13although one could argue that the cloud-init debian package should probably depend on resolvconf (or at least something that is able to write dns into resolv.conf or somewhere else)13:46
apollo13or can I use a different network renderer on debian that would work13:46
apollo13Open for suggestions if anyone has any13:49
Odd_BlokeI believe Debian package cloud-init themselves, so a Depends change would be a bugs.debian.org conversation.13:52
apollo13Odd_Bloke: yeah, or at least for the cloud-images, anyways thanks for trying to help! much appreciated. It works kinda lovely from proxmox now14:16
apollo13now if I were to figure out how to easily remove snap from the ubuntu server images, I could also use that :D14:16
Odd_BlokeYou could potentially automate that with cloud-init user-data.14:18
apollo13sure, sadly proxmox cloud-init support is rather limited14:20
kqkq95hi all ! i hope you are well :) , is cloud-init ready for CentOS 8 ?16:31
=== tds5 is now known as tds
blackboxswkqkq95: cloud-init supports CentOS. tip of cloud-init has python3 support and looks like centos8 looks to be python3.6. We should be good there, though I see stock centos8 has a  dated cloud-init version 18.5 available18:38
blackboxswwe are at 20.1 now18:39
blackboxsw~ 1.5 years of missing bugfixes/features18:39
blackboxswrharper: Odd_Bloke here are the  use-cases our release tooling is trying to automate https://hackmd.io/VbmtcZLyR4650aqqmfMMYg18:40
meenablackboxsw: and how many bugs? have we introduced since then? :P18:40
rick_h_meena:  3 definitely 318:42
blackboxswhaha!18:43
blackboxswkqkq95: also you can get latest cloud-init development bits from https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/18:44
Odd_Blokeblackboxsw: Scanning through that now, thanks for putting it together!18:45
Odd_BlokeMy first note: Feature Freeze only applies to the devel release, but the wording suggests that we should behave differently for non-devel series during FF.18:46
blackboxswgood pt Odd_Bloke changing18:46
Odd_BlokeThanks!18:47
blackboxswfixed18:47
kqkq95blackboxsw: ok i have the latest version ;)  i have a problem with my template vmware (OS : Centos 8)  with open-vm-tools and cloud-init) the network configuration is not set :(18:50
blackboxswkqkq95: not quite sure, might be related to https://bugs.launchpad.net/cloud-init/+bug/1835205 maybe?18:54
ubot5Ubuntu bug 1835205 in cloud-init "OVF datasource should check if instant id is still on VMware Platform" [Medium,Triaged]18:54
Odd_BlokeSo I am beginning to wonder if Robie's idea for a third git branch would be a cleaner way of handling this.  Currently, we merge `master -> release` and build from that.  If instead we merged `new_branch -> release` and then `master -> release`, we could (a) revert cherry-picks on `new_branch` as soon as we apply them to `release`, and (b) resolve any other merge conflicts in `new_branch` rather than19:02
Odd_Bloke`release`.  (The key here is that `new_branch` only need be updated when it's necessary to fix the builds, most of the time it won't need to be touched.)19:02
kqkq95blackboxsw: not really, at boot my network interfaces are disconnected, only with cloud-init, i test without cloud-init in my template that works well :(19:04
Odd_BlokeBasically `new_branch` is where we handle the munging required for `master` to cleanly merge into `release`.  This would be instead of us doing it in `release` (because doing it there is what leads us to have to think about popping cherry-picks etc.).19:07
Odd_Bloke(Oh and when I say "if instead we merged", I'm talking in the daily recipe build.)19:10
rharperOdd_Bloke: I guess I don't understand what's so objectionable to popping of cherry picks in the release branch;19:11
rharpers/of/off19:11
rharpermy question with a third branch (or daily sub branch to the release branch is):  where do you land release changes first  (I want to make a change to debian/control on ubuntu/devel ) ; we will ultimately release this *eventually* but certainly we want daily to get it first ... so if we put in ubuntu/devel first, we have to remember to merge that into ubuntu/devel/daily (for example)  or reverse this, and we still need to remember to land the change19:13
rharperon two branches19:13
rharperif we are changing recipes, I would prefer the idea of having daily recipe "fix itself" by popping off cherry-picks, if any19:14
Odd_BlokeWell, we don't have to remember per se, daily builds will fail and then we can fix it (or they won't and we don't).19:14
rharperheh19:14
rharperwe already have that today;19:14
rharperI was hoping to not have daily builds fail if we have a well-defined process for updating release branches and automatically cleaning up after releasing19:15
rharperat least now we have exactly one release branch to maintain  vs. two per release going forward19:15
Odd_BlokeFrom a practical perspective, popping off cherry-picks fails in (5) in Chad's doc.19:16
Odd_BlokeHaving to reapply them to add another one, and then pop them all back off feels fragile.19:17
Odd_BlokeFrom a less directly practical perspective, having the release branches not represent what's actually in those releases is confusing (and annoying in a practical sense, `git diff ubuntu/devel` won't show me how my current tree differs from focal, for example).19:18
rharperhrm, I would suggest that if only the recipe build popped cherry-picks off; that seems the least fragile;    the release branch stays in it's "released" state;  daily reverts any cherry picks if present and builds it;19:18
Odd_BlokeWell, I don't believe that's an option in the recipe builder.19:19
Odd_BlokeAnd wouldn't help in the d/control scenario you laid out.19:20
rharperhttps://help.launchpad.net/Packaging/SourceBuilds/BzrBuilder   suggest that we can use run <shell code here>19:20
Odd_Bloke"Note: Launchpad does not support the run command."19:21
rharperah I see, the bzr-builder did but lp doesn't19:21
Odd_BlokeYep, for security reasons I believe/assume.19:21
rharperso forced to put what we'd run into a branch;  it does help in the d/control scenario as we merge in the release branch where the new d/control change was made, and then just need to pop off the cherry picks before building19:21
rharperdue to the recipe build restrictions it sounds like we have to maintain a different branch if we don't want to push/pop for daily builds if the release has cherry picks19:22
Odd_Bloke"merge in the release branch" <-- merge from where into where?19:22
rharpermaster merges release into itself for recipe builds19:23
rharperit's git checkout master git merge <release>19:23
rharperthat's daily ppa recipe19:23
Odd_BlokeSure, just wasn't sure if that's specifically what you meant. :)19:24
rharperI wonder if we could test to see if a branch was present ...19:24
blackboxswyes per recipe builder, I was trying to utilize the 'run ' command in the build recipe to do that popping for us. but it seems unsupported in git build recipes19:24
rharperit would be nice if the recipe would prefer ubuntu/devel/daily  if present, and if not fallback to ubuntu/devel19:24
rharperblackboxsw: yeah, lp does not support run for security reasons19:25
Odd_BlokeI guess the d/control case I'm thinking of is the pytest one: the daily build and the release build needed _different_ d/control values.19:25
rharperbut we cherry picked in a d/control fix though IIUC19:25
Odd_BlokeSo I would expect that if we merge in the correct order, `ubuntu/devel/daily` would not need to be kept up-to-date.19:25
rharperso it;s just one more cpick to pop off19:25
Odd_BlokeWe accepted having both test frameworks in our Build-Depends for a while, I think.19:26
rharperOdd_Bloke: intersting  ...19:26
rharpertell me more  as there is multi branch merging  in the recipe19:26
blackboxswhrm ahhh19:26
blackboxswright so if all we care about for dailies is to drop debian/patches/cpicks- we could merge the ubuntu/devel/daily/debian/patches directory that only contains the base patches that matter for the release branch19:27
Odd_BlokeThat isn't where I was going (which isn't to say that it isn't worth going there :p).19:28
blackboxswso any cpick patches that we happen to apply to ubuntu/devel would be absent once ubuntu/devel/daily/debian/patches was 'merged' ontop19:28
rharperblackboxsw: I don't think you can merge away changes ...19:29
rharperunless the fix includes a commit that removes the files ..19:29
Odd_BlokeSo essentially what we're trying to avoid when we pop cpicks back off is merge conflicts, right?19:31
blackboxswright19:31
rharperOdd_Bloke: yes19:31
rharperso, if we create ubuntu/devel and ubuntu/devel/daily  at the same time;  then update our recipe to be:  lp:cloud-init master; merge ubuntu-pkg lp:cloud-init ubuntu/devel; merge ubuntu-pkg-fix  lp:cloud-init ubuntu/devel/daily ;  the last merge is no-op as all of the commits in daily are already merged ;; if that doesn't break19:34
Odd_BlokeI think we need to merge ubuntu-pkg-fix before ubuntu-pkg, so that it can resolve merge conflicts.19:34
rharperthen if we added cpicks to ubuntu/devel and  update ubuntu/devel/daily with the same cpicks;  then we pop off on ubuntu/devel/daily and commit that19:34
rharperOdd_Bloke: maybe;  I'm weak around the ordering ...19:35
Odd_BlokeSo my proposal is that we merge a third branch into `master` before `release`, which can resolve those merge conflicts.  When we have a conflict, we manually merge `master -> new_branch` and then `release -> new_branch` (which should cause the conflicts, which we can manually resolve).19:35
Odd_BlokeThat would mean, in the immediate term, that the second merge in the recipe would be a noop.19:35
Odd_Bloke(The second merge being `release ->` the result of `new_branch -> master`.)19:36
rharperI think two steps here 1) sorting out the right recipe with the third branch  2) updating the tooling to keep daily recipe sub branch in sync when we modify the release branch ;  for (2)  I know if we cpick we need to copy and then revert in daily    for new changes to release branch; (like d/control)  it's not clear to me if we need to keep daily in sync or not19:37
* blackboxsw was starting to document the possibility here https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?both#proposed-build-recipe19:38
blackboxswahh but the 'merge' attempt of u/d/daily would result in merge conflicts19:38
Odd_BlokeSo, to be clear, I'm proposing all three are full trees that are merged together fully.19:39
Odd_BlokeBut there might be a better solution that doesn't do that.19:39
rharperwhy would there be merge conflicts; sorry I'm a little git merge slow19:40
rharperonly if ubuntu/devel changes touched a file that is cpicked, right ?19:41
blackboxswrharper: folowing Odd_Bloke's suggestion of 3 separate branches. I guess we wouldn't if we did the following during cherry-pick19:41
blackboxsw(from ubuntu/devel): cherry-pick mycommit; git push ubuntu/devel:ubuntu/devel/daily;  git checkout ubuntu/devel/daily; cherry-pick --pop-all;19:42
blackboxswso we cherry pick into u/d; push to u/d and u/d/daily; checkout u/d/daily and pop-all;19:43
blackboxswthen we won't have merge conflicts; and we won't have to revert ubuntu/devel19:43
rharperOdd_Bloke:  we have master, we have ubuntu/devel which is a fork of master on which we keep the debian dir (and sync it up with master from time to time);  and ubuntu/devel/daily which is a fork of ubuntu/devel which is updated only when we modify ubuntu/devel and daily recipe would fail ; the cpick scenario  ;19:43
rharperblackboxsw: yes19:43
rharperand if ubuntu/devel has a new non-cpick commit to d/control, if we merge ubuntu/devel/daily over it;  there shouldn't be any conflicts AFAICT ;  daily is just behind ubuntu/devel but they don't overlap in changes19:44
blackboxswrharper: right any non-cpick is present already in ubuntu/devel/debian/patches and untouched by the --pop-all19:45
blackboxswso u/devel/daily would still contain the non-cpick debian/patches file19:46
blackboxswok I *think* this clarifies what we  can do in cherry-pick for me if we are ok with a separate ubuntu/series/daily branch that automatically is updated during new-upstream-snapshot and cherry-pick calls19:46
blackboxswif cherry-pick/new-upstream-snapshot pushes to both ubuntu/<series> and ubuntu/<series>/daily then we don't have to any reverts in ubuntu/<series>19:47
blackboxswand we don't have to manually maintain those two branches as it'll be hidden under the tooling19:48
Odd_BlokeLet's figure out what the behaviour should be (e.g. when applying a cherry-pick to ubuntu/$series, a corresponding revert should be pushed to ubuntu/$series/daily) before thinking about automating it.19:48
* blackboxsw tries writing it out on the hackmd doc19:48
blackboxswOdd_Bloke: rharper if we were ok supporting a seprate full ubuntu/<series>/daily branch anyway, there may be no reason to merge ubuntu/<series>/daily into ubuntu/series as ubuntu/<series>/daily should have all the commits it needs plus only the drop of any cpicks.19:51
blackboxswso build recipe could remain simple. but I'm writing that out now19:51
Odd_BlokeOK, hang on, I'm a little confused: we haven't popped the latest round of cherry-picks, but daily builds are succeeding.19:55
rharperhttps://code.launchpad.net/~cloud-init-dev/+recipe/cloud-init-daily-devel19:57
rharpersays no19:57
Odd_BlokeOh haha, I was looking at bionic.19:57
Odd_BlokeVery smart.19:57
rharperblackboxsw: AFAIK we always want to merge ubuntu/devel/daily  ; it just may not have any content to add19:57
rharperunless we have cpicks19:57
Odd_BlokeOK, so the issue isn't really merge conflicts, actually, it's that quilt patches don't necessarily cleanly apply.19:57
rharperblackboxsw: we want a stable recipe that always works; and once we diverge a release branch then daily contains the "fixups"19:58
rharperOdd_Bloke: we can also put our refresh patches work on to daily19:58
Odd_BlokeRight, though we would probably just revert them there?20:01
blackboxswrharper: if ubuntu/devel/daily has no cpicks to drop ubuntu/devel == ubuntu/devel/daily20:02
blackboxswif we just always push ubuntu/devel -> ubuntu/devel daily with new-upstream-snapshot and cherry-pick, we never really have to merge20:03
blackboxswI think I tried capturing this in https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?view#proposed-branching-strategy20:03
=== tds0 is now known as tds
Odd_BlokeThat would require a force push to ubuntu/$series/daily, I think20:04
Odd_Bloke?20:04
rharperI don't want to force push;  and I don't see why we need to push anything into daily except cpicks20:06
Odd_Bloke"tip of ubuntu/<series> plus commits to revert of any debian/patches/ named cpick-* or fix-cpick-*" implies force-pushing, I think.20:06
rharperdaily only needs to hold refresh debian/patches  and cpick pop-all20:06
blackboxswOdd_Bloke: I believe a force push would be required to daily right20:07
rharperwhy ?  we merge in cpicks from release branch to apply a revert ;20:07
Odd_BlokeYeah, to have it be "tip + ..." it would have to be a force push.20:07
rharperand we hold changes to debian/patches needed for refreshing release branch patchs20:07
Odd_Bloke(Maybe I'm just getting hung up on the wording here.)20:07
blackboxswrharper: because we reverted cpicks in u/$series/daily  but didn't revert in u/$series base branch20:08
rharperif I branch ubuntu/devel and then ubuntu/devel/dail from ubuntu/devel;   and Ihave master, merge in ubuntu/dlevel, and then merge in ubuntu/devel/daily20:08
blackboxswso how do we update u/$series/daily from u/$series without a force push?20:08
rharperbut the reverts play on top of the commits already merged in from ubuntu/delvel ?20:08
Odd_BlokeMerge?20:08
blackboxswyeah I guess we could keep merging u/$series20:08
rharperdaily only needs to apply on top of release20:08
Odd_BlokeWe shouldn't really need to touch /daily all that often.20:09
rharperand it's "undo commits only"20:09
rharperyes20:09
rharperthat's the idea20:09
Odd_Bloke(Not even for most releases, I would hope.)20:09
rharpernormaly daily will be the same state as ubuntu/devel was at the start of the new series20:09
rharperOdd_Bloke: yes;20:09
Odd_BlokeI'm putting together a test recipe now.20:11
rharpernice20:12
blackboxswyeah I think that recipe would be just this right? https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?view20:14
Odd_BlokeOh ubuntu/devel/daily is an invalid name because we have ubuntu/devel, I've gone for ubuntu/daily/devel for now.20:14
* blackboxsw didn't understand that Odd_Bloke . why is that invalid?20:14
blackboxswbecause the branch name we reference looks like a path under ubuntu/devel?20:15
Odd_BlokeBecause `.git/refs/heads/ubuntu/devel` is already a file, so it can't also be a directory, basically.20:15
blackboxswahh righto20:17
rharperbummer20:18
Odd_BlokeOK, so I have a working prototype recipe here: https://code.launchpad.net/~daniel-thewatkins/+recipe/oddbloke-cloud-init-daily-devel20:33
Odd_BlokeMy ubuntu/daily/devel has reverts of every cherry pick added since the previous release.20:34
blackboxswnice. /me wonders why the order seems backwards20:34
blackboxswI have the same understanding. if daily/devel20:35
Odd_BlokeYou have to revert in reverse order.20:35
blackboxswbut I'd expect merge conflicts when you are merging ubuntu/devel into lp:cloud-init  on line 220:36
Odd_BlokeThe issue is not merge conflicts, the issue is patches failing to apply.20:36
Odd_BlokeAnd they aren't applied until the recipe is complete and both merges have happened.20:36
blackboxswahh right. right.20:36
blackboxsw+120:36
blackboxswyes that makes sense in my head now20:36
Odd_Bloke(I was earlier mistaken about this, so my earlier proposals presupposed that we needed to order the merges to be able to resolve conflicts.)20:37
Odd_Bloke(Which is why they expressed them in a different order.)20:37
Odd_Blokeblackboxsw: So I would suggest that we document this process and see how it works for us manually a couple of times before enshrining it in tools.20:37
blackboxswbut again, why couldn't we just drop the merge ubuntu-pkg lp:~daniel-thewatkins/cloud-init/+git/cloud-init ubuntu/devel20:38
blackboxswas ubuntu/daily/devel will be devel + drops20:38
Odd_BlokeIt is currently.20:38
blackboxswI mean 'omit the line that says " merge ubuntu-pkg lp:~daniel-thewatkins/cloud-init/+git/cloud-init ubuntu/devel" in your recipe20:39
Odd_BlokeI understood.20:39
Odd_Blokeubuntu/daily/devel is currently devel + drops.20:39
Odd_BlokeBut we don't want to have to update ubuntu/daily/devel every time we update ubuntu/devel.20:40
Odd_BlokeOK, so I just did a `new-upstream-release master` and pushed to my repo and the build is now failing.20:43
blackboxswok Odd_Bloke I see. right we only need to update it when we cherry-pick or drop cherry-picks20:43
blackboxswOdd_Bloke: I'd expect that if we aren't pushing to ubuntu/daily/devel every change to ubuntu/devel20:44
Odd_BlokeIt looks like because the fix-cpick-* patch wasn't dropped automatically by new-upstream-release.20:44
blackboxswOdd_Bloke: right that fix is also in my PR #45 on uss-tableflip20:44
blackboxswfix-cpick* and cpick-* get handled20:44
blackboxsws/handled/popped20:44
blackboxswby both new-upstream-snapshot and cherry-pick --pop-all20:44
Odd_BlokeOK, with all patches dropped, the build succeeded.20:47
Odd_BlokeWithout updating ubuntu/daily/devel.20:47
blackboxswso Odd_Bloke if we only update ubuntu/daily/devel when cherry-picks are applied. then procedure looks like this maybe?:   git checkout ubuntu/devel; cherry-pick <upstream_cpick_commit>; git checkout ubuntu/daily/devel; git merge ubuntu/devel; git revert <local_cpick_commit>; git push ubuntu/daily/devel;20:50
blackboxswso individually each time we cherry-pick, we git revert that local cherry-pick commit and push to ubuntu/daily/devel20:51
Odd_BlokeSomething along those lines, yes.21:01
rharperOdd_Bloke: catching up;   "with all patches drop the build succeeded/Without updating ubuntu/daily/devel"   -- that exactly what I was hoping.  Excellent!21:36
johnsonshiAnybody have issues with running `tox -e pylint` recently? I constantly hit the "unable to import pytest" error, but when I try to do so through the py2/py3 console everything seems fine. This is on cloud-init master btw so unsure if this is a bug.22:03
rharperjohnsonshi: I've not, let me see if I can reproduce22:06
rharperjohnsonshi: tox -r -e pylint works for me22:09
rharperI used -r to recreate the env fresh which means pulling down the pip packages into the virtenv22:09
johnsonshirharper: Yeah -r passed. Thanks!22:12
rharpercool!22:12

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!