[09:05] <apollo13> hello, anyone around to help me with network configuration?
[09:05] <apollo13> my cloud-init on debian properly writes the config to /etc/network/interfaces.d/50-cloud-init.cfg
[09:05] <apollo13> but also generates a dhcp template in /run/network/interfaces.d which obviously takes precedence but won't work because of no dhcp server in that network
[09:06] <apollo13> any idea what is causing that?
[09:07] <apollo13> also I need to rename 50-cloud-init.cfg to 50-cloud-init that it works
[11:06] <apollo13> okay, I fixed that by upgrading cloud-init to > 19.2
[11:06] <apollo13> sadly that is not yet in the buster images, but apparently on it's way
[11:09] <apollo13> so, cloud-init network configuration configures dns-nameserver in /etc/interfaces yet the debian cloud images do not have resolvconf, how is that supposed to work?
[11:19] <tribaal> apollo13: just a heads up - most of the active cloud-initers are in US timezones (I don't know how to answer your question personally)
[11:20] <apollo13> I have opened https://salsa.debian.org/cloud-team/debian-cloud-images/-/issues/24
[11:20] <apollo13> tribaal: thanks, my bouncer will stay online :)
[11:24] <tribaal> apollo13: hehe ack :) I guess opening the bug report was the right reflex - it is a good asynchronous contact point as well.
[11:24] <apollo13> in all fairness I am surprised how badly cloud-init seems to work every time I try it :(
[11:25] <apollo13> I mean I guess it is probably my fault for using a static network configuration instead of DHCP and I fully understand that getting those things right is hard, but I fail massively every time I try cloud-init :(
[13:40] <Odd_Bloke> apollo13: What is creating the file in /run/network/interfaces.d?
[13:40] <apollo13> Odd_Bloke: the ifupdown-cloud-init-helper
[13:41] <apollo13> but it only does this because cloud-init wries 50-cloud-init.cfg int /etc/network/interfaces.d which is a invalid name
[13:41] <apollo13> so that part got fixed by upgrading to cloud-init > 19.2
[13:41] <Odd_Bloke> I'm not familiar with ifupdown-cloud-init-helper, what is that?
[13:42] <Odd_Bloke> Well, regardless, sounds like you're past that part of the issue.
[13:42] <apollo13> yes
[13:42] <apollo13> all that remains (and I also fixed that by remastering the debian cloud images) was DNS which got fixed by adding resolvconf to the image
[13:43] <Odd_Bloke> So it sounds like this is an issue with the mastering of the Debian cloud images more than it is with cloud-init?
[13:43] <apollo13> yes, seems like it
[13:43] <apollo13> hence I opened an issue at https://salsa.debian.org/cloud-team/debian-cloud-images/-/issues/24
[13:43] <apollo13> apparently they only test the dhcp boot cases, dunno…
[13:45] <Odd_Bloke> OK, cool, was just making sure I'd understood fully.  Thanks for bringing the issue here (even if you fixed it before the core team woke up ;).
[13:46] <apollo13> hehe, no issue after all…
[13:46] <apollo13> although one could argue that the cloud-init debian package should probably depend on resolvconf (or at least something that is able to write dns into resolv.conf or somewhere else)
[13:46] <apollo13> or can I use a different network renderer on debian that would work
[13:49] <apollo13> Open for suggestions if anyone has any
[13:52] <Odd_Bloke> I believe Debian package cloud-init themselves, so a Depends change would be a bugs.debian.org conversation.
[14:16] <apollo13> Odd_Bloke: yeah, or at least for the cloud-images, anyways thanks for trying to help! much appreciated. It works kinda lovely from proxmox now
[14:16] <apollo13> now if I were to figure out how to easily remove snap from the ubuntu server images, I could also use that :D
[14:18] <Odd_Bloke> You could potentially automate that with cloud-init user-data.
[14:20] <apollo13> sure, sadly proxmox cloud-init support is rather limited
[16:31] <kqkq95> hi all ! i hope you are well :) , is cloud-init ready for CentOS 8 ?
[18:38] <blackboxsw> kqkq95: cloud-init supports CentOS. tip of cloud-init has python3 support and looks like centos8 looks to be python3.6. We should be good there, though I see stock centos8 has a  dated cloud-init version 18.5 available
[18:39] <blackboxsw> we are at 20.1 now
[18:39] <blackboxsw> ~ 1.5 years of missing bugfixes/features
[18:40] <blackboxsw> rharper: Odd_Bloke here are the  use-cases our release tooling is trying to automate https://hackmd.io/VbmtcZLyR4650aqqmfMMYg
[18:40] <meena> blackboxsw: and how many bugs? have we introduced since then? :P
[18:42] <rick_h_> meena:  3 definitely 3
[18:43] <blackboxsw> haha!
[18:44] <blackboxsw> kqkq95: also you can get latest cloud-init development bits from https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
[18:45] <Odd_Bloke> blackboxsw: Scanning through that now, thanks for putting it together!
[18:46] <Odd_Bloke> My first note: Feature Freeze only applies to the devel release, but the wording suggests that we should behave differently for non-devel series during FF.
[18:46] <blackboxsw> good pt Odd_Bloke changing
[18:47] <Odd_Bloke> Thanks!
[18:47] <blackboxsw> fixed
[18:50] <kqkq95> blackboxsw: ok i have the latest version ;)  i have a problem with my template vmware (OS : Centos 8)  with open-vm-tools and cloud-init) the network configuration is not set :(
[18:54] <blackboxsw> kqkq95: not quite sure, might be related to https://bugs.launchpad.net/cloud-init/+bug/1835205 maybe?
[19:02] <Odd_Bloke> So I am beginning to wonder if Robie's idea for a third git branch would be a cleaner way of handling this.  Currently, we merge `master -> release` and build from that.  If instead we merged `new_branch -> release` and then `master -> release`, we could (a) revert cherry-picks on `new_branch` as soon as we apply them to `release`, and (b) resolve any other merge conflicts in `new_branch` rather than
[19:02] <Odd_Bloke> `release`.  (The key here is that `new_branch` only need be updated when it's necessary to fix the builds, most of the time it won't need to be touched.)
[19:04] <kqkq95> blackboxsw: not really, at boot my network interfaces are disconnected, only with cloud-init, i test without cloud-init in my template that works well :(
[19:07] <Odd_Bloke> Basically `new_branch` is where we handle the munging required for `master` to cleanly merge into `release`.  This would be instead of us doing it in `release` (because doing it there is what leads us to have to think about popping cherry-picks etc.).
[19:10] <Odd_Bloke> (Oh and when I say "if instead we merged", I'm talking in the daily recipe build.)
[19:11] <rharper> Odd_Bloke: I guess I don't understand what's so objectionable to popping of cherry picks in the release branch;
[19:11] <rharper> s/of/off
[19:13] <rharper> my question with a third branch (or daily sub branch to the release branch is):  where do you land release changes first  (I want to make a change to debian/control on ubuntu/devel ) ; we will ultimately release this *eventually* but certainly we want daily to get it first ... so if we put in ubuntu/devel first, we have to remember to merge that into ubuntu/devel/daily (for example)  or reverse this, and we still need to remember to land the change
[19:13] <rharper> on two branches
[19:14] <rharper> if we are changing recipes, I would prefer the idea of having daily recipe "fix itself" by popping off cherry-picks, if any
[19:14] <Odd_Bloke> Well, we don't have to remember per se, daily builds will fail and then we can fix it (or they won't and we don't).
[19:14] <rharper> heh
[19:14] <rharper> we already have that today;
[19:15] <rharper> I was hoping to not have daily builds fail if we have a well-defined process for updating release branches and automatically cleaning up after releasing
[19:15] <rharper> at least now we have exactly one release branch to maintain  vs. two per release going forward
[19:16] <Odd_Bloke> From a practical perspective, popping off cherry-picks fails in (5) in Chad's doc.
[19:17] <Odd_Bloke> Having to reapply them to add another one, and then pop them all back off feels fragile.
[19:18] <Odd_Bloke> From a less directly practical perspective, having the release branches not represent what's actually in those releases is confusing (and annoying in a practical sense, `git diff ubuntu/devel` won't show me how my current tree differs from focal, for example).
[19:18] <rharper> hrm, I would suggest that if only the recipe build popped cherry-picks off; that seems the least fragile;    the release branch stays in it's "released" state;  daily reverts any cherry picks if present and builds it;
[19:19] <Odd_Bloke> Well, I don't believe that's an option in the recipe builder.
[19:20] <Odd_Bloke> And wouldn't help in the d/control scenario you laid out.
[19:20] <rharper> https://help.launchpad.net/Packaging/SourceBuilds/BzrBuilder   suggest that we can use run <shell code here>
[19:21] <Odd_Bloke> "Note: Launchpad does not support the run command."
[19:21] <rharper> ah I see, the bzr-builder did but lp doesn't
[19:21] <Odd_Bloke> Yep, for security reasons I believe/assume.
[19:21] <rharper> so forced to put what we'd run into a branch;  it does help in the d/control scenario as we merge in the release branch where the new d/control change was made, and then just need to pop off the cherry picks before building
[19:22] <rharper> due to the recipe build restrictions it sounds like we have to maintain a different branch if we don't want to push/pop for daily builds if the release has cherry picks
[19:22] <Odd_Bloke> "merge in the release branch" <-- merge from where into where?
[19:23] <rharper> master merges release into itself for recipe builds
[19:23] <rharper> it's git checkout master git merge <release>
[19:23] <rharper> that's daily ppa recipe
[19:24] <Odd_Bloke> Sure, just wasn't sure if that's specifically what you meant. :)
[19:24] <rharper> I wonder if we could test to see if a branch was present ...
[19:24] <blackboxsw> yes per recipe builder, I was trying to utilize the 'run ' command in the build recipe to do that popping for us. but it seems unsupported in git build recipes
[19:24] <rharper> it would be nice if the recipe would prefer ubuntu/devel/daily  if present, and if not fallback to ubuntu/devel
[19:25] <rharper> blackboxsw: yeah, lp does not support run for security reasons
[19:25] <Odd_Bloke> I guess the d/control case I'm thinking of is the pytest one: the daily build and the release build needed _different_ d/control values.
[19:25] <rharper> but we cherry picked in a d/control fix though IIUC
[19:25] <Odd_Bloke> So I would expect that if we merge in the correct order, `ubuntu/devel/daily` would not need to be kept up-to-date.
[19:25] <rharper> so it;s just one more cpick to pop off
[19:26] <Odd_Bloke> We accepted having both test frameworks in our Build-Depends for a while, I think.
[19:26] <rharper> Odd_Bloke: intersting  ...
[19:26] <rharper> tell me more  as there is multi branch merging  in the recipe
[19:26] <blackboxsw> hrm ahhh
[19:27] <blackboxsw> right so if all we care about for dailies is to drop debian/patches/cpicks- we could merge the ubuntu/devel/daily/debian/patches directory that only contains the base patches that matter for the release branch
[19:28] <Odd_Bloke> That isn't where I was going (which isn't to say that it isn't worth going there :p).
[19:28] <blackboxsw> so any cpick patches that we happen to apply to ubuntu/devel would be absent once ubuntu/devel/daily/debian/patches was 'merged' ontop
[19:29] <rharper> blackboxsw: I don't think you can merge away changes ...
[19:29] <rharper> unless the fix includes a commit that removes the files ..
[19:31] <Odd_Bloke> So essentially what we're trying to avoid when we pop cpicks back off is merge conflicts, right?
[19:31] <blackboxsw> right
[19:31] <rharper> Odd_Bloke: yes
[19:34] <rharper> so, if we create ubuntu/devel and ubuntu/devel/daily  at the same time;  then update our recipe to be:  lp:cloud-init master; merge ubuntu-pkg lp:cloud-init ubuntu/devel; merge ubuntu-pkg-fix  lp:cloud-init ubuntu/devel/daily ;  the last merge is no-op as all of the commits in daily are already merged ;; if that doesn't break
[19:34] <Odd_Bloke> I think we need to merge ubuntu-pkg-fix before ubuntu-pkg, so that it can resolve merge conflicts.
[19:34] <rharper> then if we added cpicks to ubuntu/devel and  update ubuntu/devel/daily with the same cpicks;  then we pop off on ubuntu/devel/daily and commit that
[19:35] <rharper> Odd_Bloke: maybe;  I'm weak around the ordering ...
[19:35] <Odd_Bloke> So my proposal is that we merge a third branch into `master` before `release`, which can resolve those merge conflicts.  When we have a conflict, we manually merge `master -> new_branch` and then `release -> new_branch` (which should cause the conflicts, which we can manually resolve).
[19:35] <Odd_Bloke> That would mean, in the immediate term, that the second merge in the recipe would be a noop.
[19:36] <Odd_Bloke> (The second merge being `release ->` the result of `new_branch -> master`.)
[19:37] <rharper> I think two steps here 1) sorting out the right recipe with the third branch  2) updating the tooling to keep daily recipe sub branch in sync when we modify the release branch ;  for (2)  I know if we cpick we need to copy and then revert in daily    for new changes to release branch; (like d/control)  it's not clear to me if we need to keep daily in sync or not
[19:38]  * blackboxsw was starting to document the possibility here https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?both#proposed-build-recipe
[19:38] <blackboxsw> ahh but the 'merge' attempt of u/d/daily would result in merge conflicts
[19:39] <Odd_Bloke> So, to be clear, I'm proposing all three are full trees that are merged together fully.
[19:39] <Odd_Bloke> But there might be a better solution that doesn't do that.
[19:40] <rharper> why would there be merge conflicts; sorry I'm a little git merge slow
[19:41] <rharper> only if ubuntu/devel changes touched a file that is cpicked, right ?
[19:41] <blackboxsw> rharper: folowing Odd_Bloke's suggestion of 3 separate branches. I guess we wouldn't if we did the following during cherry-pick
[19:42] <blackboxsw> (from ubuntu/devel): cherry-pick mycommit; git push ubuntu/devel:ubuntu/devel/daily;  git checkout ubuntu/devel/daily; cherry-pick --pop-all;
[19:43] <blackboxsw> so we cherry pick into u/d; push to u/d and u/d/daily; checkout u/d/daily and pop-all;
[19:43] <blackboxsw> then we won't have merge conflicts; and we won't have to revert ubuntu/devel
[19:43] <rharper> Odd_Bloke:  we have master, we have ubuntu/devel which is a fork of master on which we keep the debian dir (and sync it up with master from time to time);  and ubuntu/devel/daily which is a fork of ubuntu/devel which is updated only when we modify ubuntu/devel and daily recipe would fail ; the cpick scenario  ;
[19:43] <rharper> blackboxsw: yes
[19:44] <rharper> and if ubuntu/devel has a new non-cpick commit to d/control, if we merge ubuntu/devel/daily over it;  there shouldn't be any conflicts AFAICT ;  daily is just behind ubuntu/devel but they don't overlap in changes
[19:45] <blackboxsw> rharper: right any non-cpick is present already in ubuntu/devel/debian/patches and untouched by the --pop-all
[19:46] <blackboxsw> so u/devel/daily would still contain the non-cpick debian/patches file
[19:46] <blackboxsw> ok I *think* this clarifies what we  can do in cherry-pick for me if we are ok with a separate ubuntu/series/daily branch that automatically is updated during new-upstream-snapshot and cherry-pick calls
[19:47] <blackboxsw> if cherry-pick/new-upstream-snapshot pushes to both ubuntu/<series> and ubuntu/<series>/daily then we don't have to any reverts in ubuntu/<series>
[19:48] <blackboxsw> and we don't have to manually maintain those two branches as it'll be hidden under the tooling
[19:48] <Odd_Bloke> Let's figure out what the behaviour should be (e.g. when applying a cherry-pick to ubuntu/$series, a corresponding revert should be pushed to ubuntu/$series/daily) before thinking about automating it.
[19:48]  * blackboxsw tries writing it out on the hackmd doc
[19:51] <blackboxsw> Odd_Bloke: rharper if we were ok supporting a seprate full ubuntu/<series>/daily branch anyway, there may be no reason to merge ubuntu/<series>/daily into ubuntu/series as ubuntu/<series>/daily should have all the commits it needs plus only the drop of any cpicks.
[19:51] <blackboxsw> so build recipe could remain simple. but I'm writing that out now
[19:55] <Odd_Bloke> OK, hang on, I'm a little confused: we haven't popped the latest round of cherry-picks, but daily builds are succeeding.
[19:57] <rharper> https://code.launchpad.net/~cloud-init-dev/+recipe/cloud-init-daily-devel
[19:57] <rharper> says no
[19:57] <Odd_Bloke> Oh haha, I was looking at bionic.
[19:57] <Odd_Bloke> Very smart.
[19:57] <rharper> blackboxsw: AFAIK we always want to merge ubuntu/devel/daily  ; it just may not have any content to add
[19:57] <rharper> unless we have cpicks
[19:57] <Odd_Bloke> OK, so the issue isn't really merge conflicts, actually, it's that quilt patches don't necessarily cleanly apply.
[19:58] <rharper> blackboxsw: we want a stable recipe that always works; and once we diverge a release branch then daily contains the "fixups"
[19:58] <rharper> Odd_Bloke: we can also put our refresh patches work on to daily
[20:01] <Odd_Bloke> Right, though we would probably just revert them there?
[20:02] <blackboxsw> rharper: if ubuntu/devel/daily has no cpicks to drop ubuntu/devel == ubuntu/devel/daily
[20:03] <blackboxsw> if we just always push ubuntu/devel -> ubuntu/devel daily with new-upstream-snapshot and cherry-pick, we never really have to merge
[20:03] <blackboxsw> I think I tried capturing this in https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?view#proposed-branching-strategy
[20:04] <Odd_Bloke> That would require a force push to ubuntu/$series/daily, I think
[20:04] <Odd_Bloke> ?
[20:06] <rharper> I don't want to force push;  and I don't see why we need to push anything into daily except cpicks
[20:06] <Odd_Bloke> "tip of ubuntu/<series> plus commits to revert of any debian/patches/ named cpick-* or fix-cpick-*" implies force-pushing, I think.
[20:06] <rharper> daily only needs to hold refresh debian/patches  and cpick pop-all
[20:07] <blackboxsw> Odd_Bloke: I believe a force push would be required to daily right
[20:07] <rharper> why ?  we merge in cpicks from release branch to apply a revert ;
[20:07] <Odd_Bloke> Yeah, to have it be "tip + ..." it would have to be a force push.
[20:07] <rharper> and we hold changes to debian/patches needed for refreshing release branch patchs
[20:07] <Odd_Bloke> (Maybe I'm just getting hung up on the wording here.)
[20:08] <blackboxsw> rharper: because we reverted cpicks in u/$series/daily  but didn't revert in u/$series base branch
[20:08] <rharper> if I branch ubuntu/devel and then ubuntu/devel/dail from ubuntu/devel;   and Ihave master, merge in ubuntu/dlevel, and then merge in ubuntu/devel/daily
[20:08] <blackboxsw> so how do we update u/$series/daily from u/$series without a force push?
[20:08] <rharper> but the reverts play on top of the commits already merged in from ubuntu/delvel ?
[20:08] <Odd_Bloke> Merge?
[20:08] <blackboxsw> yeah I guess we could keep merging u/$series
[20:08] <rharper> daily only needs to apply on top of release
[20:09] <Odd_Bloke> We shouldn't really need to touch /daily all that often.
[20:09] <rharper> and it's "undo commits only"
[20:09] <rharper> yes
[20:09] <rharper> that's the idea
[20:09] <Odd_Bloke> (Not even for most releases, I would hope.)
[20:09] <rharper> normaly daily will be the same state as ubuntu/devel was at the start of the new series
[20:09] <rharper> Odd_Bloke: yes;
[20:11] <Odd_Bloke> I'm putting together a test recipe now.
[20:12] <rharper> nice
[20:14] <blackboxsw> yeah I think that recipe would be just this right? https://hackmd.io/VbmtcZLyR4650aqqmfMMYg?view
[20:14] <Odd_Bloke> Oh ubuntu/devel/daily is an invalid name because we have ubuntu/devel, I've gone for ubuntu/daily/devel for now.
[20:14]  * blackboxsw didn't understand that Odd_Bloke . why is that invalid?
[20:15] <blackboxsw> because the branch name we reference looks like a path under ubuntu/devel?
[20:15] <Odd_Bloke> Because `.git/refs/heads/ubuntu/devel` is already a file, so it can't also be a directory, basically.
[20:17] <blackboxsw> ahh righto
[20:18] <rharper> bummer
[20:33] <Odd_Bloke> OK, so I have a working prototype recipe here: https://code.launchpad.net/~daniel-thewatkins/+recipe/oddbloke-cloud-init-daily-devel
[20:34] <Odd_Bloke> My ubuntu/daily/devel has reverts of every cherry pick added since the previous release.
[20:34] <blackboxsw> nice. /me wonders why the order seems backwards
[20:35] <blackboxsw> I have the same understanding. if daily/devel
[20:35] <Odd_Bloke> You have to revert in reverse order.
[20:36] <blackboxsw> but I'd expect merge conflicts when you are merging ubuntu/devel into lp:cloud-init  on line 2
[20:36] <Odd_Bloke> The issue is not merge conflicts, the issue is patches failing to apply.
[20:36] <Odd_Bloke> And they aren't applied until the recipe is complete and both merges have happened.
[20:36] <blackboxsw> ahh right. right.
[20:36] <blackboxsw> +1
[20:36] <blackboxsw> yes that makes sense in my head now
[20:37] <Odd_Bloke> (I was earlier mistaken about this, so my earlier proposals presupposed that we needed to order the merges to be able to resolve conflicts.)
[20:37] <Odd_Bloke> (Which is why they expressed them in a different order.)
[20:37] <Odd_Bloke> blackboxsw: So I would suggest that we document this process and see how it works for us manually a couple of times before enshrining it in tools.
[20:38] <blackboxsw> but again, why couldn't we just drop the merge ubuntu-pkg lp:~daniel-thewatkins/cloud-init/+git/cloud-init ubuntu/devel
[20:38] <blackboxsw> as ubuntu/daily/devel will be devel + drops
[20:38] <Odd_Bloke> It is currently.
[20:39] <blackboxsw> I mean 'omit the line that says " merge ubuntu-pkg lp:~daniel-thewatkins/cloud-init/+git/cloud-init ubuntu/devel" in your recipe
[20:39] <Odd_Bloke> I understood.
[20:39] <Odd_Bloke> ubuntu/daily/devel is currently devel + drops.
[20:40] <Odd_Bloke> But we don't want to have to update ubuntu/daily/devel every time we update ubuntu/devel.
[20:43] <Odd_Bloke> OK, so I just did a `new-upstream-release master` and pushed to my repo and the build is now failing.
[20:43] <blackboxsw> ok Odd_Bloke I see. right we only need to update it when we cherry-pick or drop cherry-picks
[20:44] <blackboxsw> Odd_Bloke: I'd expect that if we aren't pushing to ubuntu/daily/devel every change to ubuntu/devel
[20:44] <Odd_Bloke> It looks like because the fix-cpick-* patch wasn't dropped automatically by new-upstream-release.
[20:44] <blackboxsw> Odd_Bloke: right that fix is also in my PR #45 on uss-tableflip
[20:44] <blackboxsw> fix-cpick* and cpick-* get handled
[20:44] <blackboxsw> s/handled/popped
[20:44] <blackboxsw> by both new-upstream-snapshot and cherry-pick --pop-all
[20:47] <Odd_Bloke> OK, with all patches dropped, the build succeeded.
[20:47] <Odd_Bloke> Without updating ubuntu/daily/devel.
[20:50] <blackboxsw> so Odd_Bloke if we only update ubuntu/daily/devel when cherry-picks are applied. then procedure looks like this maybe?:   git checkout ubuntu/devel; cherry-pick <upstream_cpick_commit>; git checkout ubuntu/daily/devel; git merge ubuntu/devel; git revert <local_cpick_commit>; git push ubuntu/daily/devel;
[20:51] <blackboxsw> so individually each time we cherry-pick, we git revert that local cherry-pick commit and push to ubuntu/daily/devel
[21:01] <Odd_Bloke> Something along those lines, yes.
[21:36] <rharper> Odd_Bloke: catching up;   "with all patches drop the build succeeded/Without updating ubuntu/daily/devel"   -- that exactly what I was hoping.  Excellent!
[22:03] <johnsonshi> Anybody have issues with running `tox -e pylint` recently? I constantly hit the "unable to import pytest" error, but when I try to do so through the py2/py3 console everything seems fine. This is on cloud-init master btw so unsure if this is a bug.
[22:06] <rharper> johnsonshi: I've not, let me see if I can reproduce
[22:09] <rharper> johnsonshi: tox -r -e pylint works for me
[22:09] <rharper> I used -r to recreate the env fresh which means pulling down the pip packages into the virtenv
[22:12] <johnsonshi> rharper: Yeah -r passed. Thanks!
[22:12] <rharper> cool!