| smoser | catphish: most of the time "dhcp on eth0" | 13:21 |
|---|---|---|
| smoser | at least for the initial connection to the metadata service. | 13:22 |
| catphish | smoser: thanks, i thought that might be the case, unfortuinately in my environment i don't want to rely on being able to run a dhcp and web server on all networks a vm might be connected so, so i'm looking at using a config disk instead | 13:22 |
| smoser | catphish: hardware ? | 14:01 |
| catphish | i don't understand | 14:07 |
| catphish | x86-64 qemu/kvm VMs | 14:08 |
| catphish | in a "public cloud" environment, but where some VMs might be on a private network | 14:08 |
| smoser | catphish: so no... not hardware ;) | 14:17 |
| smoser | "physical" is what i meant. | 14:17 |
| catphish | oh right nope :) | 14:17 |
| smoser | you're looking to create a public cloud ? or your wanting to use/ammend an existing one? | 14:18 |
| catphish | i'm making one | 14:26 |
| catphish | or rather, just adding cloud-init to one with the hope of replacing a proprietary postinstall mechanism | 14:27 |
| smoser | if you just wanted to "ammend" an existing one. | 14:58 |
| smoser | i think config disk (config drive) is a reasonable solution. You can put it on any partition (it doesn't have to be a separate disk... with gpt, you've got 128 partitions and spending 10MiB on a config location sin't really costing you anything) | 15:00 |
| smoser | but if you wanted to ammend a network solution, then one way you could do it would be to provide the MAC of the nic where the metadata servcie lived in dmi info | 15:00 |
| smoser | somehow just tell cloud-init "this is the mac to dhcp on" and then it can go on as it does | 15:01 |
| catphish | what options can i provide in smbios? | 15:01 |
| catphish | because that would be idea if i can force it to use a link-local IP | 15:01 |
| catphish | i'm puzzled because the default address seems to be APIPA, but it doesn't seem to assign an APIPA IP | 15:02 |
| catphish | ie by default it appears to try to download a config from 169.254.169.254 but never assigns an IP | 15:03 |
| catphish | i can't see any way to use network config without ipv4 dhcp | 15:20 |
| smoser | catphish: yeah.. i think you're prbably looking at amazon | 15:43 |
| smoser | or at least, yes. that is how ec2 works. | 15:43 |
| smoser | but one thing crazy about ec2... they run their server on a ipv4 link local address, as you've found | 15:43 |
| smoser | but if you configure ipv4 link local, you wont get a response from them. | 15:43 |
| catphish | smoser: same with digitalocean | 15:43 |
| smoser | digital ocean i think *does* do ipv4ll | 15:43 |
| catphish | yeah, it does, it assigns a random address | 15:44 |
| smoser | but on ec2, if you don't request from the ipv4 address that the dhcp request would give you, then you can't talk to the metadata service | 15:44 |
| smoser | :) | 15:44 |
| smoser | digital ocean's is better. | 15:44 |
| catphish | yeah, that was my interpretation too | 15:44 |
| smoser | as it is right now, what you're wanting to do is not implemented fully. | 15:44 |
| catphish | i'm tempted to write my own to do one of 2 things: 1) fetch the data from a virtual serial port or block device 2) use ipv6 link local properly | 15:45 |
| smoser | config-disk (I'd suggest NoCloud) is the only real option. | 15:45 |
| smoser | oh... | 15:45 |
| catphish | yeah nocloud (config-disk) basically works | 15:45 |
| smoser | serial... one of the datasources does that | 15:45 |
| smoser | joyent's | 15:45 |
| catphish | yeah i was just looking that that joyent thing | 15:45 |
| smoser | it does a metadata service over serial | 15:45 |
| smoser | yeah | 15:45 |
| catphish | annoyingly they use a proprietary format | 15:45 |
| catphish | but it could work | 15:45 |
| smoser | and then you avhe to serialize requests over serial | 15:46 |
| catphish | yeah, that's not *too* painful | 15:46 |
| smoser | if i were going to design a new datasource now, i htink i would | 15:47 |
| smoser | a.) provide instance-id and mac address of "the right" nic in dmi data (or possibly some other "local" manner) | 15:47 |
| smoser | b.) use ipv6 link local to hit a well known address (or maybe provide the ipv6 address in 'a' also) | 15:48 |
| smoser | c.) get the rest of the network configuration from the metadata service there. | 15:48 |
| smoser | i think joyent actually did a reasonable job. | 15:49 |
| smoser | oh... other "local" transports are available. hypervisor <-> guest sockets . | 15:49 |
| smoser | vsock i think is the thing | 15:50 |
| catphish | a local (unixy) socket would be absolutely amazing | 15:51 |
| catphish | basically i'm down to 2 options: 1) write a datasource that pulls data from the hypervisor by some kind of virtual socket 2) use an ISO datasource (basic nocloud) | 15:52 |
| catphish | i'm currently going with option (2) because it doesn't require somehow getting my data source into every OS image i want to use, i'm running into trouble with caching though, qemu doesn't seem to refetch the iso for subsequent requests, and i'd like to to able to change the contents | 15:53 |
| catphish | since cloud-init unmounts the iso, it should be safe to change it, then reprovision later, but something is caching it, hopefully i can address this | 15:53 |
| catphish | one specific question - can i use an ISO to configure the network, but ALSO configure a network data source to run at the next stage? | 16:17 |
| catphish | smoser: after much internal discussion, we're going to write a datasource :) it's going to 1) get an instance-id and security key from DMI 2) get an interface MAC from DMI 3) bring up the appropriate interface and fetch the remaining config from a well-known ipv6 link-local IP | 16:32 |
| catphish | and of cource we're going to be dickheads and make sure our config is slightly nonstandard, and name the module after our platform, so that nobody else can use it ;) | 16:33 |
| catphish | thank you for your assistance | 16:34 |
| === lachesis_ is now known as lachesis | ||
| smoser | catphish: unattaching the iso will be a pain. | 17:19 |
| smoser | cloud-init will look for the instance-id on each boot | 17:19 |
| smoser | so you have to disable cloud-init on subsequent boots or set 'manual_cache_clean' | 17:19 |
| smoser | and, fwiw, it doesn't have to be an ISO | 17:19 |
| smoser | as in iso9660. it has to be a filesystem with a label. | 17:20 |
| smoser | catphish: what would be neat... and faster development cycle | 17:22 |
| smoser | is if you also made it able to take instance-id, security-key and interface from environment . | 17:22 |
| smoser | then you can iterate on lxd containers quite quickly | 17:22 |
| smoser | actually, now remembering ... there was a pan to do a more advanced datasource for lxd | 17:23 |
| smoser | using /dev/lxd which is a socket | 17:24 |
| catphish | smoser: yeah, i'm not a fan of the iso (or ext2 or whatever) filesytstem idea, seems kinda messy having an extra filesystem dangling around, so yeah, going to have a proper stab at the IPv6 LL approach, the only downside is that i'll have to modify every OS image to include it | 17:38 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!