[13:21] <smoser> catphish: most of the time "dhcp on eth0"
[13:22] <smoser> at least for the initial connection to the metadata service.
[13:22] <catphish> smoser: thanks, i thought that might be the case, unfortuinately in my environment i don't want to rely on being able to run a dhcp and web server on all networks a vm might be connected so, so i'm looking at using a config disk instead
[14:01] <smoser> catphish: hardware ?
[14:07] <catphish> i don't understand
[14:08] <catphish> x86-64 qemu/kvm VMs
[14:08] <catphish> in a "public cloud" environment, but where some VMs might be on a private network
[14:17] <smoser> catphish: so no... not hardware ;)
[14:17] <smoser> "physical" is what i meant.
[14:17] <catphish> oh right nope :)
[14:18] <smoser> you're looking to create a public cloud ? or your wanting to use/ammend an existing one?
[14:26] <catphish> i'm making one
[14:27] <catphish> or rather, just adding cloud-init to one with the hope of replacing a proprietary postinstall mechanism
[14:58] <smoser> if you just wanted to "ammend" an existing one.
[15:00] <smoser> i think config disk (config drive) is a reasonable solution. You can put it on any partition (it doesn't have to be a separate disk... with gpt, you've got 128 partitions and spending 10MiB on a config location sin't really costing you anything)
[15:00] <smoser> but if you wanted to ammend a network solution, then one way you could do it would be to provide the MAC of the nic where the metadata servcie lived in dmi info
[15:01] <smoser> somehow just tell cloud-init "this is the mac to dhcp on" and then it can go on as it does
[15:01] <catphish> what options can i provide in smbios?
[15:01] <catphish> because that would be idea if i can force it to use a link-local IP
[15:02] <catphish> i'm puzzled because the default address seems to be APIPA, but it doesn't seem to assign an APIPA IP
[15:03] <catphish> ie by default it appears to try to download a config from 169.254.169.254 but never assigns an IP
[15:20] <catphish> i can't see any way to use network config without ipv4 dhcp
[15:43] <smoser> catphish: yeah.. i think you're prbably looking at amazon
[15:43] <smoser> or at least, yes. that is how ec2 works.
[15:43] <smoser> but one thing crazy about ec2... they run their server on a ipv4 link local address, as you've found
[15:43] <smoser> but if you configure ipv4 link local, you wont get a response from them.
[15:43] <catphish> smoser: same with digitalocean
[15:43] <smoser> digital ocean i think *does* do ipv4ll
[15:44] <catphish> yeah, it does, it assigns a random address
[15:44] <smoser> but on ec2, if you don't request from the ipv4 address that the dhcp request would give you, then you can't talk to the metadata service
[15:44] <smoser> :)
[15:44] <smoser> digital ocean's is better.
[15:44] <catphish> yeah, that was my interpretation too
[15:44] <smoser> as it is right now, what you're wanting to do is not implemented fully.
[15:45] <catphish> i'm tempted to write my own to do one of 2 things: 1) fetch the data from a virtual serial port or block device 2) use ipv6 link local properly
[15:45] <smoser> config-disk (I'd suggest NoCloud) is the only real option.
[15:45] <smoser> oh...
[15:45] <catphish> yeah nocloud (config-disk) basically works
[15:45] <smoser> serial... one of the datasources does that
[15:45] <smoser> joyent's
[15:45] <catphish> yeah i was just looking that that joyent thing
[15:45] <smoser> it does a metadata service over serial
[15:45] <smoser> yeah
[15:45] <catphish> annoyingly they use a proprietary format
[15:45] <catphish> but it could work
[15:46] <smoser> and then you avhe to serialize requests over serial
[15:46] <catphish> yeah, that's not *too* painful
[15:47] <smoser> if i were going to design a new datasource now, i htink i would
[15:47] <smoser> a.) provide instance-id and mac address of "the right" nic in dmi data (or possibly some other "local" manner)
[15:48] <smoser> b.) use ipv6 link local to hit a well known address (or maybe provide the ipv6 address in 'a' also)
[15:48] <smoser> c.) get the rest of the network configuration from the metadata service there.
[15:49] <smoser> i think joyent actually did a reasonable job.
[15:49] <smoser> oh... other "local" transports are available.  hypervisor <-> guest sockets .
[15:50] <smoser> vsock i think is the thing
[15:51] <catphish> a local (unixy) socket would be absolutely amazing
[15:52] <catphish> basically i'm down to 2 options: 1) write a datasource that pulls data from the hypervisor by some kind of virtual socket 2) use an ISO datasource (basic nocloud)
[15:53] <catphish> i'm currently going with option (2) because it doesn't require somehow getting my data source into every OS image i want to use, i'm running into trouble with caching though, qemu doesn't seem to refetch the iso for subsequent requests, and i'd like to to able to change the contents
[15:53] <catphish> since cloud-init unmounts the iso, it should be safe to change it, then reprovision later, but something is caching it, hopefully i can address this
[16:17] <catphish> one specific question - can i use an ISO to configure the network, but ALSO configure a network data source to run at the next stage?
[16:32] <catphish> smoser: after much internal discussion, we're going to write a datasource :) it's going to 1) get an instance-id and security key from DMI 2) get an interface MAC from DMI 3) bring up the appropriate interface and fetch the remaining config from a well-known ipv6 link-local IP
[16:33] <catphish> and of cource we're going to be dickheads and make sure our config is slightly nonstandard, and name the module after our platform, so that nobody else can use it ;)
[16:34] <catphish> thank you for your assistance
[17:19] <smoser> catphish: unattaching the iso will be a pain.
[17:19] <smoser> cloud-init will look for the instance-id on each boot
[17:19] <smoser> so you have to disable cloud-init on subsequent boots or set 'manual_cache_clean'
[17:19] <smoser> and, fwiw, it doesn't have to be an ISO
[17:20] <smoser> as in iso9660. it has to be a filesystem with a label.
[17:22] <smoser> catphish: what would be neat... and faster development cycle
[17:22] <smoser> is if  you also made it able to take instance-id, security-key and interface from environment .
[17:22] <smoser> then you can iterate on lxd containers quite quickly
[17:23] <smoser> actually, now remembering ... there was a pan to do a more advanced datasource for lxd
[17:24] <smoser> using /dev/lxd which is a socket
[17:38] <catphish> smoser: yeah, i'm not a fan of the iso (or ext2 or whatever) filesytstem idea, seems kinda messy having an extra filesystem dangling around, so yeah, going to have a proper stab at the IPv6 LL approach, the only downside is that i'll have to modify every OS image to include it