[13:36] <larsks> smoser: harlowja: are either of you familiar with the new systemd configs introduced between 0.7.6 and 0.7.7?  There are a couple of new targets, some udev configs, a systemd generator...are there docs on how all that is supposed to work?
[13:38] <smoser> larsks, i do want to write some docs on taht :)
[13:39] <smoser> but yeah, i'm familiar.
[13:39] <smoser> the generator really makes it so you can completely disable cloud-init easily
[13:39] <larsks> That seems useful.
[13:39] <smoser> just by: touch /etc/cloud/cloud-init.disabled
[13:39] <smoser> then it never is even considered by systemd in boot
[13:39] <smoser> also by cloud-init=disabled on the kernel command line
[13:40] <smoser> so it wont do any of the annoying bottlenecks in boot that it woudl otherwise impose.
[13:40] <smoser> the udev stuff stops networking from coming up until cloud-init-local is done
[13:40] <smoser> it blocks the hotplug events
[13:40] <smoser> and then when cloud-init has rendered the desired networking, it allows it to go on through
[13:41] <larsks> Okay, that helps.
[13:41] <smoser> that way we're not dealing with a system that started ifup on a device that cloud-init re-wrote the configuration for
[13:41] <larsks> Quick question about dependencies in the unit files:  there is a dependency on "networking.service", which I'm assuming is an ubuntu-ism.  Would it make sense to simply replace that with a dependency on network.target (which is a standard systemd target)?
[13:42] <larsks> Otherwise I need to muck about with the unit files for packaging, and I would like to minimize mucking as much as possible.
[13:42] <larsks> Or make the dependency part of a drop-in rather than part of the main unit file, I guess (and the ubuntu packages could include the drop-in).
[13:48] <smoser> larsks, yeah, networking.service is an ubuntu-ism (well, an ifupdown-ism)
[13:49] <smoser> network.target isnt really good enough
[13:49] <larsks> What exactly does networking.service provide?  Because there are *also* dependencies already on network-online.target...
[13:49]  * larsks is obviously ubuntu-ignorant w/r/t network setup.
[13:50] <smoser> well, it is really quite specific.
[13:50] <smoser> networking.service is guaranteed to run before network.target is reached
[13:51] <smoser> if we were just After network.target, then 'cloud-init' would run in parallel with all other things that are really only blocked on network.target
[13:51] <smoser> which is a lot of things.
[13:52] <smoser> since cloud-init basically consumes user-data at this stage, we want it to block as much of boot as possible
[13:52] <smoser> by consumes, i mean it runs the boot-hooks
[13:52] <larsks> Okay.  What about moving that dependency to a drop-in?  That would get you exactly the same behavior, but if we could keep the main unit files generic that makes packaging for non-ubuntu enviornments easier.
[13:52] <smoser> o
[13:53] <smoser> i'm not opposed to it if it works.
[13:53] <smoser> or some other way that we could do it.
[13:53] <larsks> I think drop-ins are the systemd approved way of doing this sort of thing.
[13:53] <smoser> the generator could actually probably determine what the system will use and dtrt
[13:53] <larsks> Oh, huh.
[13:53] <larsks> That would be interesting.
[13:53] <smoser> what do you mean by drop-in?
[13:54] <larsks> You create a directory /etc/systemd/system/cloud-init.service.d, and you can drop files in their to extend the unit file without having to modify it.
[13:54] <smoser> thats what i thought you meant.
[13:54] <larsks> So from systemd's perspective, you end up with same final configuration, but you get to keep distribution-specific dependendecies out of the main unit file.
[13:56] <smoser> yeah. i'm not opposed to that.
[13:56] <smoser> your'e looking to get this functional on centos/rhel ?
[13:56]  * smoser would like that too
[13:56] <larsks> Yeah.
[13:56] <larsks> I guess we will need the network setting stuff, too.
[13:56] <smoser> so, for that...
[13:57] <smoser> right now in cloud-init on ubuntu it basically works like:
[13:57] <smoser> a.) read openstack networking configuration format
[13:57] <smoser> b.) convert it to "internal" format
[13:57] <smoser> c.) render it
[13:57] <smoser> c.) render it to /etc/network/interfaces and systemd.link files
[13:58] <smoser> there is code i think in clod-init that will take a /etc/network/interfaces file as a "networking configuration format" and render it to redhat style networking format
[13:58] <smoser> so that might be the easiest way to do this.
[13:58] <larsks> Yeah, that should already exist.
[13:58] <smoser> although ENI is kind of ...  sucky as a network config format
[13:58] <smoser> ie, i'm afraod the conversion might be lossy
[13:59] <smoser> afraid
[13:59] <larsks> I will try to take a look at that soon unless mgagne or dmsimard gets to it first.
[14:01] <dmsimard> yeah but "/etc/network/interfaces" format != network_data.json
[14:01] <dmsimard> and network_data.json is the agreed way upstream to do it
[14:01] <larsks> dmsimard: right, but smoser  was saying there exists network_data.json format -> ENI, and we also have ENI->redhat already.
[14:02] <larsks> So you could use ENI as an intermediate format.
[14:02] <dmsimard> ah, yes.
[14:02] <larsks> Modulo smoser 's comments re: lossy conversion.
[14:02] <smoser> because there exists ENI -> RH for the old crappy path where openstack declared networking in ENI format
[14:02] <dmsimard> so that patch added the "translator" and ubuntu support
[14:02] <dmsimard> we can definitely re-use the "translator", just need to add RHEL support
[14:02] <smoser> but doing it well probably isnt that much work.
[14:03] <larsks> Yeah.
[14:03] <smoser> you have a fairly clean networking config format declared to you, and we can fix the "internal" stuff in cloud-init to be better if we need to
[14:03] <smoser> the harder part is the next step..
[14:03] <dmsimard> I can help testing an implementation for RHEL/CentOS but adding the support is a bit out of my league
[14:04] <smoser> where we want to take a hotplug event and then hit the datasource and say "hey, i just got a network device, what should i do with it"
[14:04] <smoser> and apply that.
[14:08] <larsks> I need to fix https://bugs.launchpad.net/cloud-init/+bug/1424710 first before I can play with 0.7.7 much :)
[14:09] <dmsimard> TIL about hostnamectl
[14:25] <smoser> rharper, ping when youre in.
[14:32] <rharper> smoser: here
[14:33] <smoser> hey.
[14:33] <smoser> https://bugs.launchpad.net/maas-images/+bug/1570142
[14:34] <smoser> the one difference between cloud-init fallback rendered networking config (which doesnt work on iscsi root) and cloud-initramfs-dyn-netconf (which *does*) is 'auto eth0'
[14:35] <smoser> rather than manual
[15:08] <rharper> smoser: really here: was at the car shop, back home now
[15:09] <smoser> k.
[15:09] <smoser> so theres 2 things there really
[15:10] <smoser> a.) we need a way for config to say 'manual'
[15:10] <smoser> b.) cloud-init fallback needs to determine "this is manual" for the ip= fallback path.
[15:12] <rharper> can you expand on (a); as it sounds like you mean more than just iface foo inet manual
[15:12] <rharper> which can already be expressed in network config
[15:15] <smoser> allow-auto
[15:15] <smoser> or allow-hotplug
[15:17] <smoser> rharper, http://paste.ubuntu.com/15831923/
[15:17] <smoser> that'd be one suggestion, arbitrarily picked name 'control' to mean how should this interface be broought up or down
[15:19] <rharper> smoser: anything that's not 'auto' (which is alias for allow-auto) or 'hotplug' (which we don't support in network config yet) get's ignored by ifquery as "manual"
[15:19] <rharper> so the default is manual control
[15:19] <rharper> certainl fallback code can use ifquery to determine which ifaces are present but manual (i.e. not in ifquery)
[15:20] <smoser> not really.
[15:20] <smoser> fallback is dictating which are present but manual
[15:20] <smoser> it can't ask ifquery that
[15:23] <rharper> fallback runs when networking didn't come up (at all or just one expected iface);  in the case that there's an eni, we can use ifquery vs. the set of nics we find from the kernel; and apply heuristics w.r.t what we select first and what we ignore;  in the case that there is no eni; then  we just have the heuristics and ignore list.
[15:24] <rharper> I don't quite follow how a new field in eni would help
[15:28] <smoser>  brb
[15:28] <smoser> http://paste.ubuntu.com/15832136/ is what i think is sane.
[15:31] <rharper> AFAICT, you're just excluding them from allow-auto (or auto); which means ifquery won't pick them up by default;  allow-manual removes it from the list of ifaces that ifquery looks for by default for bring all ifaces up;
[16:29] <larsks> Slightly off topic, but...does anyone here have Azure experience?
[16:32] <waldi> some
[16:41] <larsks> waldi: in particular, do you have experience convert disk images to vhd format and uploading them successfully?
[16:42] <larsks> convert*ing*
[16:43] <waldi> yep
[16:44] <waldi> code: https://gitlab.credativ.com/de/azure-manage/blob/develop/azure_manage/vhd.py
[16:45] <larsks> waldi: thanks!  Looking.
[16:46] <waldi> the code in qemu-img is broken unfortunately. i need to write a bug-report about it
[16:47] <larsks> Interesting. At least that means I'm not crazy.  Or at least, not because of the conversion failures...
[16:48] <waldi> qemu-img alligns the images to cylinder/head boundary. this is futile on Azure, which expects images alligned to whole MiB
[16:52] <larsks> Do you know if earlier versions of qemu-img worked?  The microsoft docs seem to assume that there is a working version out there somewhere.
[16:54] <waldi> not sure
[21:12] <larsks> waldi: already fixed upstream; staged for next qemu-release: http://serverfault.com/questions/770378/problems-preparing-a-disk-image-for-upload-to-azure/770425#770425