[13:16] <mutantturkey> hello
[13:16] <mutantturkey> is there a flag for cloud-init to only use specific modules?
[13:17] <mutantturkey> right now i am playing whack-a-mole with cloud-init to prevent it from overwriting my various configs. However, I basically just need cloud-init's networking module.
[13:19] <mutantturkey> or maybe the answer is 'provide my own cloud-init config'
[13:19] <mutantturkey> that just imports one module
[13:19] <mutantturkey> because mine right now is the stock one from digital ocean
[13:36] <Odd_Bloke> mutantturkey: You can configure which modules cloud-init will execute by modifying /etc/cloud/cloud.cfg, but you can't do that after first boot (because that's too late, of course) so that would involve image modifications.  What is cloud-init doing that you would like to disable?  (There may be user-data you can pass to disable the particular undesirable behaviours you're seeing.)
[14:07] <rick_h> 372795
[14:08] <Odd_Bloke> Time to get a new phone.
[14:08] <rick_h> pretty much
[14:08] <rick_h> or yubikey
[14:08] <rick_h> oops
[14:08] <Odd_Bloke> :p
[15:34] <blackboxsw> Odd_Bloke: rharper. so branching strategy for cloud-init ubuntu/daily/<series>  vs ubuntu/<series>  do we want to talk about concerns with the approach mentioned here https://hackmd.io/VbmtcZLyR4650aqqmfMMYg   or https://github.com/CanonicalLtd/uss-tableflip/pull/45 ?
[15:37] <blackboxsw> It may be faster to chat about that in a virtual hangout to make sure we aren't exposed to gaps. but, generally it feels like PR #45  simplifies our cherry-pick process a bit
[15:37] <Odd_Bloke> blackboxsw: Which is currently the canonical source of the proposed process?
[15:38] <blackboxsw> Odd_Bloke: https://github.com/CanonicalLtd/uss-tableflip/pull/45/files#diff-d6b5a9d4866846cffb4df4461090dfbb
[15:39] <Odd_Bloke> OK, so I think we can discard the HackMD entirely then?
[15:40] <blackboxsw> Odd_Bloke: yes, we can. As I think the use-cases I mentioned there are handled by the proposal in PR #45
[15:41] <Odd_Bloke> OK, cool.
[15:44] <blackboxsw> I think the biggest set of open conversation on it was here https://github.com/CanonicalLtd/uss-tableflip/pull/45#discussion_r406505316
[15:44] <blackboxsw> s/conversation/questions/
[15:59] <rharper> Odd_Bloke:  blackboxsw: I have time now, meet in standup ?
[15:59] <blackboxsw> sure I can swing on by
[16:06] <Odd_Bloke> Oops, was AFK, OMW.
[16:44] <mutantturkey> Odd_Bloke: well, I do use a custom image
[16:45] <mutantturkey> that's so i can put things on like, users keys, and custom packages, its an entire dev environment in a single VM for my work stack, so it has lots of stuff on it
[16:45] <mutantturkey> Odd_Bloke: what module seems to generate the files under /etc/network/interfaces?
[16:57] <blackboxsw> rharper: I think you are right, we can git cherry-pick && git revert single cpick commitish properly I misinterpreted the output from the CLI. the revert is valid and applicable. it was a PEBCAK issue on my side thursday.   https://paste.ubuntu.com/p/QnFnvJHMGx/
[16:57] <blackboxsw> Odd_Bloke: rharper per the above paste. I think we can use individial git cherry-pick <cpick_commit_from_devel>; (daily-branch) git revert <cpick_commit_from_devel>;
[16:58] <blackboxsw> so no full merge necessary each time we cherry pic.
[16:58] <blackboxsw> I can update the doc on PR #45 now for the minimal approach
[17:04] <rharper> blackboxsw: even with changelog entries ?
[17:05] <blackboxsw> rharper: that's without the changelog entries
[17:05] <rharper> changelog is one file that could get in conflict
[17:05] <rharper> with new commits to release that are not present in daily
[17:06] <rharper> so maybe replay your test but with changelog commits to release before the recipe build  step such that you're merging daily into a branch that has release + changelog commit not in daily;
[17:06] <rharper> and see if it blows up
[17:07] <rharper> mutantturkey: cloudinit/net/eni.py writes to /etc/network/interfaces.d/50-cloud-init.cfg     we don't touch /etc/network/interfaces file directly
[17:10] <rharper> mutantturkey: networking isn't a config module;  we don't yet have a "render-network-config" command cli that's does the right thing at this time (there are several places cloud-init looks for input for network-config);   but it has been requested on a few fronts;  I'll see about getting that on our next cycle features
[17:11] <mutantturkey> rharper: so eni.py writes the config, cool. that config file is exactly what i want
[17:12] <mutantturkey> rharper: if i remove everything elsefrom my /etc/cloud/cloud.cfg... will it still run eni.py and not the other modules?
[17:13] <rharper> mutantturkey: eni will run independent of the config modules
[17:14] <mutantturkey> okay
[17:14] <mutantturkey> does anything else run independent of the config modules hten?
[17:14] <mutantturkey> then?
[17:14] <rharper> it's part of cloud-init stages (cloud-init init --local stage runs network rendering code)
[17:14] <mutantturkey> my goal: get DO to provide the correct network information, but that's it
[17:14] <rharper> DO as in digital ocean ?
[17:14] <mutantturkey> yes exactly
[17:14] <rharper> how is the information incorrect ?
[17:14] <rharper> they provide network metadata IIRC
[17:15] <mutantturkey> they provide exactly what i need yes. the /etc/network/interfaces.d/50-cloud-init.cfg works perfectly
[17:15] <rharper> oh, I see;  you only need network-config
[17:15] <mutantturkey> however, other things are done by cloud init that overwrite my config
[17:15] <mutantturkey> i think so :-)
[17:15] <rharper> yes; so if you make the init_modules and config modules and final modules lists empty; that should be rather sparse
[17:16] <mutantturkey> is an empty /etc/cloud/cloud.cfg valid?
[17:16] <rharper> yes but likley won't do what you want
[17:16] <mutantturkey> okay.
[17:17] <mutantturkey> i know my goal, i am not quite sure how to make cloud-init do what i want
[17:17] <rharper> you want:  cloud_init_modules: [] , cloud_config_modules: []  and cloud_final_modules: []
[17:17] <rharper> this will tell cloud-init to not run any of the config modules
[17:17] <mutantturkey> okay
[17:17] <mutantturkey> so something lik
[17:17] <mutantturkey> echo "cloud_init_modules: []" > /etc/cloud/cloud.cfg
[17:17] <mutantturkey> echo "cloud_config_modules: []" >> /etc/cloud/cloud.cfg
[17:17] <mutantturkey> echo "cloud_final_modules: []" >> /etc/cloud/cloud.cfg
[17:18] <rharper> I'm not sure what removing the system_info section will do
[17:18] <rharper> likely break
[17:18] <rharper> I'd put your config you have in a different file that will get merged
[17:18] <mutantturkey> okay, that would be in
[17:18] <rharper> so write that to /etc/cloud/cloud.cfg.d/99-disable-config-modules.cfg
[17:18] <mutantturkey> /etc/cloud/cloud.d/
[17:18] <rharper> yes
[17:18] <mutantturkey> got it
[17:18] <mutantturkey> thanks let me try that
[17:19] <rharper> sure
[17:19] <mutantturkey> (not that i dislike cloud init, its very cool)
[17:19] <mutantturkey> i just am only needing it for one thing
[17:19] <rharper> so what about resizing root and ssh host key clean/gen etc ?
[17:20] <mutantturkey> hm
[17:20] <mutantturkey> good point
[17:20] <rharper> so, keep growpart, resizefs
[17:20] <rharper> in cloud_init_modules
[17:20] <mutantturkey> er how do i do multiline bash things
[17:20] <rharper> for those;  and ssh if you want keygen
[17:20] <mutantturkey> <<<EOF
[17:20] <rharper> cat  > /myfile <<EOF
[17:21] <rharper> ...
[17:21] <rharper> ..
[17:21] <rharper> EOF
[17:21] <mutantturkey> right
[17:21] <mutantturkey> do the numbers mean anything? 99, 50 etc?
[17:21] <mutantturkey> is it order of operation?
[17:21] <rharper> yes, order of merging
[17:21] <rharper> so 99 will *clobber* previous keys in the dictionary by default
[17:22] <mutantturkey> understood
[17:22] <rharper> which is what you want in this case
[17:22] <mutantturkey> cloud_init_modules: [growpart, resizefs]
[17:23] <mutantturkey> just want to make sure that syntax is correct
[17:24] <rharper> also, DO sends vendor-data which contains a lot of stuff, you may want to disable their vendor-data,  you can include "vendor_data: {enabled' False}"
[17:24] <rharper> mutantturkey: that looks right
[17:24] <mutantturkey> vendor_data: {enabled: False}"
[17:24] <mutantturkey> ?
[17:25] <rharper> mutantturkey: one thing you can do to test this out on existing instances is to write your config file out, and then: sudo cloud-init clean --logs;  then sudo cloud-init init --local;  sudo cloud-init init;  that runs the first two stages
[17:25] <rharper> the platform can send additional cloud-config to your instance
[17:25] <rharper> you don't have to accept it, but that turns it off
[17:25] <rharper> you can check /var/lib/cloud/instance/vendor-data.txt to see if they sent you some;  ISTR DO had quite the bundle
[17:27] <mutantturkey> rharper: could you review this quickly?
[17:27] <mutantturkey> http://ix.io/2hTi
[17:27] <rharper> y
[17:28] <rharper> add #cloud-config to the first line of your config file you generate
[17:28] <mutantturkey> awesome, done.
[17:32] <mutantturkey> alright, i shall report back in a bit with my results!
[17:51] <blackboxsw> rharper: one thing I don't understand is why we would want to cherry-pick the debian/changelog commitish too into daily for a revert from u/daily/devel? Since the changelog entry also increments debian/changelog version as well, a revert will mean that when we merge back into ubuntu/devel, we'll be down-reving cloud-init as built during daily images.
[17:52] <blackboxsw> so this would mean that every daily deb build would be 1 subrevision lower than the version present (and released) in ubuntu/devel.
[17:52] <blackboxsw> making it a bit harder to install and test a daily build vs the current released version of cloud-init
[18:27] <mutantturkey> rharper: cool, i borked it up (whitspace after EOF, but let me build and run it again).
[18:28] <mutantturkey> the other problem i currently have is that it doesn't seem to apply the new network information after cloud-init puts the information on the first boot. after the first boot it works just finewith thecorrecet info
[18:28] <mutantturkey> though i wonder if thats debian doing something different, need to look ta logs
[18:35] <punkgeek> I use this sample to config static ip on ubuntu 16.04 vm, But it doesn't work, https://paste.ubuntu.com/p/Hh8SkhkS9P/
[19:00] <mutantturkey> rharper: yeet
[19:00] <mutantturkey> No 'config' modules to run under section 'cloud_config_modules'
[19:00] <mutantturkey> No 'final' modules to run under section 'cloud_final_modules'
[19:12] <adrian27> hello, I have the following in /etc/cloud/cloud.cfg.d/01_runcmd.cfg
[19:12] <adrian27>   runcmd:
[19:12] <adrian27> but it is not executed. Any idea what am I doing wrong?
[19:12] <adrian27> tanks
[19:12] <sarnold> it looks like your paste may have been destroyed, better to use a pastebin site
[19:12] <adrian27> thanks
[19:13] <adrian27> its just those two lines
[19:13] <sarnold> that's the thing, we only got one: https://paste.ubuntu.com/p/n93Dk5pBFB/
[19:14] <rharper> mutantturkey: w.r.t network-config, we only write network-config once (per-instance)  ; if you're repeat testing on the same instance you want to use the cloud-init clean --logs (which wipes instance data and the log files)
[19:15] <rharper> blackboxsw: I don't *want* daily to be down level; I was just worried that we'd see conflicts due to release and daily both writing to debian/changelog
[19:15] <adrian27> he he, okay, here it is: https://paste.ubuntu.com/p/kRQMxmRBVf/
[19:16] <sarnold> adrian27: try /usr/bin/echo or /bin/echo, depending upon where your echo executable is located
[19:16] <sarnold> adrian27: (I have no idea if this'll help, but giving explicit paths to things often helps)
[19:16] <blackboxsw> rharper: the way PR #45 is setup, we are only reverting the single commitish that introduced the debian/patches/cpick-* file. the changelog commit is separate, so if we don't cherry-pick it into ubuntu/daily/<series> then we essentially don't touch it, or risk merge conflicts. But we have the caveat of daily build recipe debian/changelog representing that a cherry-pick is applied, when it really isn't
[19:17] <blackboxsw> BTW rharper Odd_Bloke I just force pushed all doc changes and script changes to use git revert instead of git merge ubuntu/devel -> ubuntu/daily/devel
[19:17] <adrian27> sarnold thanks, I will try that
[19:17] <rharper> blackboxsw: I see, so we don't conflict at all since we only revert the patch;   I'm not super worried about daily changelog
[19:18] <blackboxsw> rharper: right, and I just double checked if we wanted to also revert the debian/changelog entries and yes it causes conflicts making it a far more manual process than we want :/
[19:19] <adrian27> sarnold: it still does not work :(
[19:20] <sarnold> adrian27: dang :( sorry. how about a touch command to create a file, instead? maybe stdout isn't going where you expect?
[19:21] <adrian27> sarnold: hmm, good idea. Will be back
[19:25] <adrian27> sarnold: nope, still does not work :(
[19:25] <sarnold> adrian27: dang, sorry, I'm out of ideas
[19:26] <adrian27> sarnold: thanks anyway :)
[19:33] <adrian27> do you need to have the #cloud-config at the beginning in config files from  /etc/cloud/cloud.cfg.d/ ?
[19:33] <rharper> adrian27: I don't think so but it can't hurt
[19:36] <adrian27> can I see somewhere if cloud-init tried to execute my runcmd commands?
[19:36] <rharper> adrian27: yes, cat /var/lib/cloud/instance/scripts/runcmd
[19:38] <adrian27> rharper: well, that folder its empty. What does that mean? That it did not even tried to execute commands in https://paste.ubuntu.com/p/kRQMxmRBVf/
[19:38] <adrian27> actually /var/lib/cloud/instance/scripts/ is empty
[19:40] <rharper> it's hard to say, cloud-init collect-logs will produce a tarball with debugging info;  the log file will show whether it ran into issues with your userdata or something else
[19:42] <adrian27> isn't that /var/log/cloud-init.log ?
[19:43] <rharper> https://paste.ubuntu.com/p/8zb2qMmYbF/
[19:43] <rharper> adrian27: it's more that just that, but take a look at that
[19:45] <rharper> I'm not sure how you're re-running your test on your config but 1)  user-data needs a #cloud-config at the top  2) config modules typically only run once per instance so if you are calling cloud-config.service etc; it's skipping over the runcmd module as it's already run  3)  you can test with cloud-init single --   see the paste for an example;   when you're have it working, you can cloud-init clean --logs --reboot and see a "clean boot" run
[19:48] <sarnold> rharper: oh handy, thanks
[19:50] <rharper> sarnold: =)   -- yeah, we are in need of some cli refactor to make testing/validating config eaiser ;  we do have the schema subcommand but we've not completed writing schemas for all of the config modules
[20:06] <rharper> blackboxsw: ok; yeah so touching changelog on both branches means merge conflicts ... that was expected;  so are we OK then with the "downlevel" changelog entry?  you also mentioned that the daily PPA version would not be newer than what's in the archive?
[20:11] <blackboxsw> rharper: as PR 45 currently the pkg version generated  will be identical for packages generate from (ubuntu/devel) and (ubuntu/devel with ubuntu/daily/devel merged into it)
[20:11] <blackboxsw> but the second merged case will have the cpick reverted (just no changelog dicc)
[20:11] <blackboxsw> diff*\
[20:12] <Odd_Bloke> I don't think the changelog is used in recipe builds.
[20:12] <Odd_Bloke> Because the recipes generate their own version string
[20:12] <blackboxsw> hahah, right
[20:12] <blackboxsw> good pt
[20:12] <Odd_Bloke> I thought of this at some point over the long weekend, forgot about it until just now. :p
[20:12] <blackboxsw> right so it'll always be the  deb-version {latest-tag}-{revno}-g{git-commit}-0ubuntu1+{revno:ubuntu-pkg}~trunk
[20:13] <blackboxsw> so, what would our preference be (that the unused debian/changelog in ubuntu/daily/devel contain an item that says "reverted" cpick? or is it a don't care (because we have a revert commit in the git log anyway
[20:14] <rharper> don't care on daily build
[20:14] <rharper> though maybe as Odd_Bloke pointed out, we'll suddenly find so many users really do look at daily ppa changelog ...
[20:14] <rharper> I'd defer tackling it
[20:15] <blackboxsw> ok, if we don't care there on daily build, I think #45 is in shape then, and only performs a single git  revert of the cpick file in the ubuntu/daily/devel branch.
[20:15] <rharper> from my POV daily ppa is a way to verify that some bug works;  that's almost always an integration type of test
[20:15] <rharper> blackboxsw: ok, let me re-review shell code
[20:16] <rharper> I assume you added all of the git command || clean-up  bits I requested ?
[20:16] <adrian27> rharper: thank you
[20:17] <blackboxsw> rharper: I tried to mark resolved as I addressed each comment. I'll make sure I've pushed all those cleanup review comment resolutions
[20:20] <rharper> adrian27: all good?
[20:20] <rharper> blackboxsw: k
[20:20] <blackboxsw> rharper: +1 on handling your review comments. I noticed a typo in one of doc my fixes and just pushed it now. 'a older series' instead of 'an older series'
[20:21] <blackboxsw> the others look resolved
[20:21] <adrian27> rharper: yes. you were right, rumcmd is run only once. I booted the instance, wrote the /etc/cloud/cloud.cfg.d/ file, and expected to see it running when rebooted
[20:23] <adrian27> why cloud-init does not have a man page? :)
[20:30] <rharper> adrian27: ok;  fair point on man page
[20:32] <adrian27> well, you guys are doing a great job with cloud-init
[20:34] <powersj> https://github.com/canonical/cloud-init/tree/master/doc/man
[20:34] <rharper> adrian27: thanks for the kind words ;  powersj tells me we should have a man page ... so double checking on that
[20:34] <blackboxsw> rharper: powersj hah!
[20:35] <blackboxsw> time to bundle it in the deb package
[20:35] <blackboxsw> or see where we reverted that
[20:35] <blackboxsw> hah it *is* there on latest, and bundled
[20:36] <Odd_Bloke> `man cloud-init` doesn't work in my focal EC2 instance (launched last week).
[20:36] <blackboxsw> cloud-init 19.3-68-gc78fddea introduced it
[20:36] <powersj> https://github.com/canonical/cloud-init/pull/101
[20:37] <powersj> so apparently that needs to happen somewhere else?
[20:37] <blackboxsw> I have it on focal lxc images
[20:38] <Odd_Bloke> From the archive?
[20:39] <Odd_Bloke> Because `man cloud-init` is failing in this container I just launched.
[20:42] <rharper> Odd_Bloke: me too
[20:42] <rharper> daily focal lxd does not have man page
[20:45] <blackboxsw> Odd_Bloke: /me relaunches. interesting. it was on 20.1
[20:45] <blackboxsw> and failed on latest current build
[20:46] <blackboxsw> 20.1-10-g71af48df-0ubuntu3 for failed case... and my success case was locally built
[20:46] <Odd_Bloke> Yeah, locally built will be fine.
[20:46] <Odd_Bloke> Assuming you did it from master.
[20:46] <blackboxsw> yeah right ;/
[20:46] <Odd_Bloke> Because that PR only updated the packaging used in master.
[20:47] <blackboxsw> +1. thanks for the bug adrian27 :)
[20:50] <blackboxsw> :q
[20:50] <blackboxsw> wrong term :/
[21:41] <blackboxsw> ok addressed https://github.com/canonical/cloud-init/pull/152 comments rharper as well thanks for the schema review
[21:42] <blackboxsw> ahh and thanks for the review on uss-tableflip
=== tds1 is now known as tds
[23:14] <Goneri> blackboxsw, https://github.com/canonical/cloud-init/pull/298 :-)