[05:05] <enleth> Hi there. I'm trying to write a simple network-config for a VM running on an OVH dedicated server, with a public IP for the VM. The problem is, OVH's IP assignment is kinda wonky and requires using a gateway from a completely different subnet than the machine's IP.
[05:06] <enleth> This is a working network config I set manually: https://paste.q3k.org/paste/6VsfMrhe#DWJwJjp+6nlZCcd5aGvqWolB7VXzqr8FztVi2cJgHPQ
[05:07] <enleth> yes, it works, yes, it uses a gateway address in a subnet different than the interface address, yes, the interface has no address in the gateway's subnet - and it must not have one
[05:07] <enleth> so, how do I define a link-scope route in network-config?
[10:28] <LaborejoGuest9> hi there, it there any module doesn't listed in /etc/cloud/cloud.cfg?
[10:29] <LaborejoGuest9> since I didn't found document says every module belongs to which stage
[13:49] <mgerdts> I submitted a merge proposal the other day and needed to make some updates.  After I made the updates, I deleted the remote branch and pushed my squashed changes to a branch of the same name.  1) was this the right way?  2) Do I need to do anything else to alert reviewers that it was updated?
[13:49] <mgerdts> https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+ref/bug-1746605
[14:22] <smoser> mgerdts: i'll review. i'm sorry, just beenbehind. always am.
[14:23] <smoser> mgerdts: you can basically do 1 of 2 ways
[14:23] <mgerdts> no worries.  I've made other changes aside from the commit message as well.  Those changes are not in the same files as were in the original commit.
[14:23] <smoser> a.) push up original merge proposal, get feedback, add commits that say 'address feedback', rince and repeat
[14:23] <smoser> rinse
[14:24] <smoser> b.) push up original, get feedback, git commit --amend, git push --force
[14:24] <smoser> i do both
[14:24] <smoser> when we pull, we squash and use the commit message from the merge proposal
[14:24] <smoser> so the "address feedback" comments get tossed.
[14:25] <mgerdts> ok, thanks for clarifiying.
[14:26] <smoser> mgerdts: so, in the 'Set commit message' there
[14:27] <smoser> put a good git commit message
[14:27] <smoser>  Summary:
[14:27] <smoser>  blank
[14:27] <smoser>  mer stuff
[14:27] <smoser>  blank line
[14:28] <smoser>  LP: #1746605
[14:28] <ubot5> Launchpad bug 1746605 in cloud-init "DataSourceSmartOS needs locking and retries" [Undecided,New] https://launchpad.net/bugs/1746605
[14:28] <smoser> make sense ?
[14:28] <smoser> your 'Description of change' is reasonable, just format in summary and body, and < 74 chars wide.
[14:29] <mgerdts> I think when I ran tox the first time around I must have done it as a non-root user because I wasn't seeing serial failures.  If it doesn't have write access to the serial port, it skips that test.  My updates this time around (aside from commit message) were to add pyserial and fix a bug with deb package creation.
[14:30] <smoser> you sould not have to run tox as root
[14:30] <mgerdts> I updated the commit message in the changeset.  Are you saying I was supposed to do that in LP?
[14:30] <smoser> it should mock anything that needs.
[14:30] <smoser> https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/337106 right ?
[14:30] <mgerdts> As the comments in the new test explain, that's not really possible.
[14:30] <smoser> i just see one commit, with a one line message.
[14:31] <mgerdts> that's stale
[14:31] <mgerdts> I don't know how to tell it that I have replaced that changeset.
[14:31] <mgerdts> that is, I deleted the branch then pushed again.
[14:32] <mgerdts> perhaps this request needs to be closed and refiled?
[14:32] <smoser> ok. hm.. lets just do that.
[14:32] <smoser> http://paste.ubuntu.com/26536015/ <-- just to not lose that
[14:33] <smoser> so now go https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+ref/bug-1746605 and propose for merging
[14:33] <smoser> is 37ea241 your tip ?
[14:34] <mgerdts> https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/337271
[14:34] <mgerdts> 67cd602d
[14:34] <mgerdts> oh wait...
[14:35] <mgerdts> yeah, you got the right one
[14:35] <mgerdts> I was in a different branch when I gave you the bogus response
[14:36] <smoser> :)
[15:03] <smoser> mgerdts: do you think you have fixed 1667735 also ?
[15:04] <mgerdts> No, I have a separate fix for that just about ready.
[15:04] <smoser> ok
[15:33] <mgerdts> @smoser if the test isn't a good fit for unit tests, I can remove it.  Since this (and the fix for 1667735) depend on behavior that is specific to serial ports, I'm unsure of if/how any automated tests can be written.
[15:35] <smoser> mgerdts: yeah... i know. you end up having to mock a lot of infrastructure
[15:35] <smoser> write a "pretend mock server" sort of thing.
[15:36] <mgerdts> If Linux had something like FreeBSD's nmdm (null modem) driver that could be used.  Unless I'm missing something, any attempt to mock a serial port with a pipe or socket will result in invalid tests.
[15:36] <mgerdts> pipes and serial ports will create one stream (thinking of solaris, don't know the linux implementation details) that ensure there is no cross-talk.
[15:36] <smoser> yeah.
[15:37] <mgerdts> Also, there's no guarantee that the VFS ops related to locking with non-serial ports are representative of those used on serial ports.
[15:37] <smoser> you'd just have to fabricate the crosstalk
[15:37] <smoser> but yeah. i know . its a lot of pain.
[15:38] <smoser> i'll ask blackboxsw to have a think also.
[15:41] <mgerdts> ok.  I guess I need to come up to speed on mock so that I can maybe be a bit more imaginative in coming up with test strategies.
[15:59] <enleth> I'm having a problem getting network-config to work with nocloud local ds
[15:59] <enleth> https://paste.q3k.org/paste/tbFmVALJ#N1adKr6domaWXz-ayWaK7phn3aRzxoUNFkbc5I1Y0if - here's my configuration and the error I'm getting
[16:00] <enleth> meta-data, user-data and network-config are packed into an iso file attached to a VM
[16:00] <enleth> hostname and user are being set up correctly
[16:01] <enleth> googling for the error message suggests that cloud-init detected the network-config file but for some reason parsed it as an empty (?) config
[16:01] <smoser> enleth: i need a key to read that.
[16:01] <enleth> smoser: the key is in the URL, it's the anchor
[16:01] <enleth> needs JS to work, decrypts it in the browser
[16:02] <enleth> I can re-post it to a regular pastebin
[16:02] <enleth> https://pastebin.com/1jKntbhx
[16:03] <enleth> smoser: ^
[16:03] <enleth> if I run this without the network-config file, it just runs DHCP on the intherface as expected and everything else works too
[16:04] <smoser> enleth: well, it says "could not decrypt data (Wrong key?)
[16:04] <smoser> so... i dont know. i'm just using firefox and no js disabled or anything
[16:04] <enleth> no matter, use the second link
[16:07] <enleth> if that's of any importance, the iso file is generated like this: genisoimage  -output seed.iso -volid cidata -joliet -rock user-data meta-data network-config
[16:29] <smoser> enleth: i'msorry.. pulled away. can you file a bug ?
[16:42] <enleth> is there anything I can check now to actually get it over with and continue my work?
[16:46] <smoser> enleth: well, i suggest trying the network config v1
[16:46] <smoser> which is a bit more explicit
[16:49] <enleth> smoser: how does matching by MAC address work there? what I hoped to achieve was a shared config ISO for several VMs, with interface entries for each of them matching by MAC - so that the correct one would be used to set a static IP on first boot and the rest would be ignored
[16:49] <enleth> the docs for v2 explicitly say this is how it works
[16:49] <enleth> but it's not so clear for v1, where the interface name is described as the primary piece of data
[16:53] <enleth> maybe let me rephrase my question - if I define several interfaces in v1 config format, with names that don't match any default network adapter names assigned by kernel/udev (like, say, "ethpub1", "ethpub2", etc. instead of ethX or enoX or whatever) and define their MAC addresses, will it simply match one of those to an existing interface by MAC, rename it and ignore the rest?
[17:04] <smoser> enleth: i'm sorry, not really all the way here righ tnow.  i can look some later.
[17:16] <blackboxsw> per the cloud-init status bug rharper, you mentioned checking results.json presence, but that file could also exist already if we are running some non-local stage of cloud-init via cli too. so I think I'll have to check both, presence of result.json & no stage in status.json with a non-zero start and no finished time.
[17:19] <rharper> blackboxsw: hrm, generally I don't think cloud-init status would react to anything after cloud-final as completed and result.json is written
[17:19] <rharper> cloud-init cli invocations are not asynchronous , they're blocking
[17:22] <rharper> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1747965
[17:22] <ubot5> Ubuntu bug 1747965 in cloud-init (Ubuntu) "cloud-init status reports done before boot is finished" [Undecided,New]
[17:33] <blackboxsw> http://paste.ubuntu.com/26536818
[17:34] <blackboxsw> rharper: ^ that's the status example on ec2 where a stage that's never going to offically get run is leaked/written into status.json
[17:35] <rharper> modules-init should be ignored AFAIK, smoser ?
[17:35] <blackboxsw> maybe it's worth us ensuring modules-init isn't written
[17:35] <blackboxsw> yeah
[17:35] <rharper> that said, how is that stage related to reporting 'done' early ?
[17:36] <blackboxsw> rharper: status logic used to say if any stage didn't have a start time report 'running'
[17:36] <blackboxsw> now logic says if start != null and finished == null RUNNING
[17:37] <blackboxsw> but we really could just simply rely on result.json existing and no incomplete (started but not finished) stages in status.json
[17:37] <blackboxsw> or we could look at ps :) but this feels like overkill
[17:38] <rharper> I think the result.json parsing would be best,
[17:38] <rharper> err, status
[17:38] <blackboxsw> not much to parse there, but ..
[17:38] <blackboxsw> ahh
[17:38] <blackboxsw> right status.json
[17:39] <blackboxsw> also one things status didn't take into account is stage: != null means we are in flight on a given stage
[17:40] <blackboxsw> but as a stage completes I believe that always gets set back to null before starting the next stage. so there are gaps where cloud-init status would mistakenly represent 'done'
[17:40] <blackboxsw> so anyway, I'll work it and ping you with a branch on this
[17:44] <rharper>  cool
[17:52] <smoser> "ignored" ?
[17:52] <smoser> yeah, we shouldnt write that to status
[17:52] <smoser> only the stages that we run.
[17:52] <enleth> smoser: FYI v1 config loads properly
[17:52] <enleth> shame it can't do wildcard matches on the MAC
[17:52] <blackboxsw> so how it looks as far as status.json. cloud-init modules --mode=init will write a start and finished time for that stage when running those 'modules-init'  modules if we run on the commandline
[17:53] <enleth> or can it? the docs don't say
[17:53] <blackboxsw> during normal system startup/config of cloud-init we don't explicitly run modules --mode=init
[17:53] <blackboxsw> so that will always have null/null for start/finished times
[17:56] <smoser> enleth: ok. let me loop at your issue again.
[17:56] <blackboxsw> those modules-init modules are run during cloud-init's init(network) stage.  the cloud_config_modules are run during modules:config stage
[17:57] <smoser> enleth: so https://pastebin.com/raw/1jKntbhx is actually waht you put in ?
[17:57] <smoser> i dont understand "wildcard"
[17:57] <blackboxsw> so in terms of status.json, the "init" section represents modules-init during typical system startup and cloud-init configuration
[17:58] <enleth> smoser: not related to the issue, I just wanted to use "match: macaddress: 52:*" for another interface which is why I tried v2
[17:59] <enleth> smoser: yes, the pastebin shows my entire cloud-init config
[17:59] <enleth> didn't paste the user-data file but it only contains the users entry with my ssh keys and stuff
[18:00] <smoser> enleth: this is fixed in trunk. i'm pretty sure.
[18:00] <enleth> when I converted the config to v1, it loads and works
[18:00] <enleth> smoser: you mean v2 not being loaded?
[18:00] <smoser> so your v2 would work with 17.1 or 17.2 release.
[18:00] <smoser> yeah.
[18:02] <enleth> wonder when debian releasing publishing openstack images with cloud-init upgraded to that version
[18:02] <enleth> *s/publishing//
[18:03] <smoser> i'd be itnerested in knowing for sure
[18:03] <smoser> you can build a deb gfrom trunk easily enough, or probably just ggrab it from
[18:03] <smoser> https://code.launchpad.net/~cloud-init-dev/+archive/ubuntu/daily/+packages
[18:03] <smoser> and patch your image
[18:06] <enleth> are those going to work on debian?
[18:06] <smoser> i do not know that they dont.
[18:08] <enleth> I was under impresion that some ubuntu packages do work with debian but it's more "by accident" than anything deliberate that can be depended upon
[18:08] <smoser> enleth: i'm not suggesting you use a daily build for anything production.
[18:09] <smoser> just to test if it is fixed.
[18:09] <enleth> sure
[18:29] <enleth> the network interface name changes need cloud-init to run on every boot, right?
=== r-daneel_ is now known as r-daneel
[18:31] <enleth> I tried creating /etc/cloud/cloud-init.disabled after the first boot so that existing VMs don't break when I do something stupid to the config in the datasource, but the interfaces are no longer getting renamed so whatever got created in /etc/network/interfaces.d/50-cloud-init.cfg no longer works at all
[18:38] <smoser> enleth: hm..
[18:38] <smoser> enleth: is that ifupdown ?
[18:38] <smoser> cloud-init writes udev rules to make that work
[18:39] <smoser> but *it* might not support the 'match' if you're using that.
[18:39] <enleth> debian9, ifupdown
[18:39] <enleth> nothing in /etc/udev/rules.d/
[18:40] <enleth> using v1 config right now
[18:40] <smoser> really.
[18:40] <smoser> it really should write /etc/udev/rules.d/70-persistent-net.rules
[18:41] <enleth> is there a "verbose" mode I can enable to see what it does and when?
[18:44] <enleth> ah, there's quite a bit of logs in /var/log/cloud-init.log
[18:46] <enleth> ok, I did a clean boot - it set the runtime interface names correctly
[18:47] <enleth> /etc/udev/rules.d/ empty
[18:48] <enleth> grep rules /var/log/cloud* comes up empty
[18:48] <enleth> same for udev (save for a mountpoint list)
[18:50] <enleth> smoser: https://pastebin.com/Y1m5PxRx this is the most interesting part I found
[18:51] <enleth> other refernces to "network" in the log are accidental or just refer to the network-config file being found at the beginning
[18:55] <smoser> enleth: so in newer cloud-init you'll get https://hastebin.com/kaxelapipo
[18:56] <enleth> is that just a matter of an old version in debian?
[18:57] <smoser> i think so yes.
[18:59] <enleth> something's not right, I grepped /usr/lib/python3/dist-packages/cloudinit/ for "persistent-rules" and two hits come up
[18:59] <enleth> the code is there
[19:08] <enleth> yes, the code to use netrules_path and write to it is definitely there
[19:09] <enleth> I'm trying to figure out why wouldn't it run
=== r-daneel_ is now known as r-daneel
[19:26] <enleth> ah, so
[19:27] <enleth> distro/debian.py initializes the Rendered with 'netrules_path': None
[19:33] <enleth> why would anyone do that?
[19:43] <mgerdts> I thought that https://mail.python.org/pipermail/python-dev/2013-September/128287.html would mean that the development branch of cloud-init wouldn't need to support Python 2.6.  Sigh.
[19:57] <enleth> eh, setting netrules_path correctly for debian works
[19:58] <enleth> I'm at loss as to why it was set to None
[19:58] <smoser> mgerdts: we're still supporting centos 6. which is python 2.6
[20:00] <Odd_Bloke> Until November 2020, apparently.
[20:01] <Odd_Bloke> Though RHEL 6 is also 2.6, and supported until 2024. :p
[20:10] <enleth> smoser: OK, thanks for all the help, for now I've jury-rigged a patch to support udev rules on debian with the old version I have and switched to v1 network config, I'll upgrade whenever debian starts shipping the new version
[21:57] <blackboxsw> rharper: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/337306 for cloud-init status cmdline fix
[22:06] <rharper> blackboxsw: thx, will look at it
[23:55] <jchevalier> paulmey: Hi, I'm sorry I'm that long to answer, I have out of sync work hours!
[23:56] <jchevalier> So I tried to add the .cfg file to my vm before to make it a image
[23:56] <jchevalier> but then the image never spawns
[23:56] <jchevalier> the boot fails in Azure dashboard