[13:21] catphish: most of the time "dhcp on eth0" [13:22] at least for the initial connection to the metadata service. [13:22] smoser: thanks, i thought that might be the case, unfortuinately in my environment i don't want to rely on being able to run a dhcp and web server on all networks a vm might be connected so, so i'm looking at using a config disk instead [14:01] catphish: hardware ? [14:07] i don't understand [14:08] x86-64 qemu/kvm VMs [14:08] in a "public cloud" environment, but where some VMs might be on a private network [14:17] catphish: so no... not hardware ;) [14:17] "physical" is what i meant. [14:17] oh right nope :) [14:18] you're looking to create a public cloud ? or your wanting to use/ammend an existing one? [14:26] i'm making one [14:27] or rather, just adding cloud-init to one with the hope of replacing a proprietary postinstall mechanism [14:58] if you just wanted to "ammend" an existing one. [15:00] i think config disk (config drive) is a reasonable solution. You can put it on any partition (it doesn't have to be a separate disk... with gpt, you've got 128 partitions and spending 10MiB on a config location sin't really costing you anything) [15:00] but if you wanted to ammend a network solution, then one way you could do it would be to provide the MAC of the nic where the metadata servcie lived in dmi info [15:01] somehow just tell cloud-init "this is the mac to dhcp on" and then it can go on as it does [15:01] what options can i provide in smbios? [15:01] because that would be idea if i can force it to use a link-local IP [15:02] i'm puzzled because the default address seems to be APIPA, but it doesn't seem to assign an APIPA IP [15:03] ie by default it appears to try to download a config from 169.254.169.254 but never assigns an IP [15:20] i can't see any way to use network config without ipv4 dhcp [15:43] catphish: yeah.. i think you're prbably looking at amazon [15:43] or at least, yes. that is how ec2 works. [15:43] but one thing crazy about ec2... they run their server on a ipv4 link local address, as you've found [15:43] but if you configure ipv4 link local, you wont get a response from them. [15:43] smoser: same with digitalocean [15:43] digital ocean i think *does* do ipv4ll [15:44] yeah, it does, it assigns a random address [15:44] but on ec2, if you don't request from the ipv4 address that the dhcp request would give you, then you can't talk to the metadata service [15:44] :) [15:44] digital ocean's is better. [15:44] yeah, that was my interpretation too [15:44] as it is right now, what you're wanting to do is not implemented fully. [15:45] i'm tempted to write my own to do one of 2 things: 1) fetch the data from a virtual serial port or block device 2) use ipv6 link local properly [15:45] config-disk (I'd suggest NoCloud) is the only real option. [15:45] oh... [15:45] yeah nocloud (config-disk) basically works [15:45] serial... one of the datasources does that [15:45] joyent's [15:45] yeah i was just looking that that joyent thing [15:45] it does a metadata service over serial [15:45] yeah [15:45] annoyingly they use a proprietary format [15:45] but it could work [15:46] and then you avhe to serialize requests over serial [15:46] yeah, that's not *too* painful [15:47] if i were going to design a new datasource now, i htink i would [15:47] a.) provide instance-id and mac address of "the right" nic in dmi data (or possibly some other "local" manner) [15:48] b.) use ipv6 link local to hit a well known address (or maybe provide the ipv6 address in 'a' also) [15:48] c.) get the rest of the network configuration from the metadata service there. [15:49] i think joyent actually did a reasonable job. [15:49] oh... other "local" transports are available. hypervisor <-> guest sockets . [15:50] vsock i think is the thing [15:51] a local (unixy) socket would be absolutely amazing [15:52] basically i'm down to 2 options: 1) write a datasource that pulls data from the hypervisor by some kind of virtual socket 2) use an ISO datasource (basic nocloud) [15:53] i'm currently going with option (2) because it doesn't require somehow getting my data source into every OS image i want to use, i'm running into trouble with caching though, qemu doesn't seem to refetch the iso for subsequent requests, and i'd like to to able to change the contents [15:53] since cloud-init unmounts the iso, it should be safe to change it, then reprovision later, but something is caching it, hopefully i can address this [16:17] one specific question - can i use an ISO to configure the network, but ALSO configure a network data source to run at the next stage? [16:32] smoser: after much internal discussion, we're going to write a datasource :) it's going to 1) get an instance-id and security key from DMI 2) get an interface MAC from DMI 3) bring up the appropriate interface and fetch the remaining config from a well-known ipv6 link-local IP [16:33] and of cource we're going to be dickheads and make sure our config is slightly nonstandard, and name the module after our platform, so that nobody else can use it ;) [16:34] thank you for your assistance === lachesis_ is now known as lachesis [17:19] catphish: unattaching the iso will be a pain. [17:19] cloud-init will look for the instance-id on each boot [17:19] so you have to disable cloud-init on subsequent boots or set 'manual_cache_clean' [17:19] and, fwiw, it doesn't have to be an ISO [17:20] as in iso9660. it has to be a filesystem with a label. [17:22] catphish: what would be neat... and faster development cycle [17:22] is if you also made it able to take instance-id, security-key and interface from environment . [17:22] then you can iterate on lxd containers quite quickly [17:23] actually, now remembering ... there was a pan to do a more advanced datasource for lxd [17:24] using /dev/lxd which is a socket [17:38] smoser: yeah, i'm not a fan of the iso (or ext2 or whatever) filesytstem idea, seems kinda messy having an extra filesystem dangling around, so yeah, going to have a proper stab at the IPv6 LL approach, the only downside is that i'll have to modify every OS image to include it