| magicalChicken | smoser, rharper: Hey, I have fallback configuration working in (what I believe) is a mostly sane way | 06:29 |
|---|---|---|
| magicalChicken | lp:~wesley-wiedenmeier/cloud-init/net-fallback | 06:29 |
| smoser | reading | 06:29 |
| magicalChicken | Currently I am not attempting to rename the device to eth0, as that fails usually, but the code to do that is in place, a parameter just has to be changed | 06:29 |
| smoser | merge lines | 06:31 |
| magicalChicken | sure, just a second | 06:31 |
| magicalChicken | just pushed | 06:34 |
| smoser | ok | 06:34 |
| smoser | sitll ther ei think | 06:34 |
| magicalChicken | there's one or two that kidna make sense for separating sections, I can pop them out too | 06:35 |
| smoser | no. i'm complaining about the merge conflict lines | 06:35 |
| magicalChicken | oh, whoops | 06:35 |
| smoser | :) | 06:35 |
| magicalChicken | haha sorry about that i'm pretty tired :) | 06:35 |
| smoser | ok, so comments on what is see there. | 06:36 |
| smoser | find_fallback_network_device() | 06:37 |
| smoser | what id' like is somethign more like: | 06:37 |
| smoser | generate_fallback_config() | 06:37 |
| magicalChicken | sure, that makes sense | 06:37 |
| smoser | and that would simply return {'config': {....}, 'version': 1} | 06:38 |
| smoser | the dictionary | 06:38 |
| magicalChicken | okay, and we could generate the .link in distros.debian? | 06:38 |
| magicalChicken | it might be okay to just omit the .link file for now until we can figure out the veth naming thing | 06:39 |
| smoser | right! | 06:39 |
| magicalChicken | haha awesome, okay | 06:39 |
| smoser | here... let me show you the diff i have. | 06:39 |
| smoser | http://paste.ubuntu.com/15470570/ | 06:40 |
| smoser | so basically we call init.apply_networking() | 06:40 |
| smoser | and it decides which networking source to look at (via find_networking_config) | 06:40 |
| smoser | selects the right one, and then calls the distro's "apply" for that. | 06:41 |
| magicalChicken | yeah, that makes sense | 06:41 |
| smoser | the transfer stuff is sort of unrelated | 06:42 |
| magicalChicken | is that from trunk? | 06:42 |
| smoser | that diff is versus my net1 | 06:42 |
| magicalChicken | ah, okay | 06:42 |
| magicalChicken | i can merge in from there and apply the rest of the diff | 06:42 |
| smoser | dont worry about grabbing my paste that isnt in my branch yet | 06:43 |
| magicalChicken | okay sure | 06:44 |
| smoser | you get 'fallback_config()' to just return the config | 06:44 |
| magicalChicken | okay | 06:44 |
| smoser | make sense ? | 06:44 |
| magicalChicken | yeah, I'll get that done real quick | 06:45 |
| smoser | oh. and magicalChicken yrou return value should be a dict with 'config' and 'version' | 06:53 |
| magicalChicken | Oh, right yeah | 06:53 |
| magicalChicken | I've got the change made but something went wrong and the file didn't get written, so debugging rn | 06:53 |
| smoser | k | 06:54 |
| smoser | you had started taht from my net1 branch, right? | 06:57 |
| magicalChicken | yeah, but from around 1191, I think I merged at 1199 though | 06:58 |
| smoser | well, the one thing we wanat from your branch right now is the fallback_config() | 07:04 |
| smoser | so if you want to just quick re-do that, that'd be fine. | 07:05 |
| magicalChicken | just pushed, I think it has everything | 07:05 |
| magicalChicken | I renamed the function, and it generates the config fully formatted with the 'config' and 'version' keys | 07:05 |
| smoser | ok. i'll get it pulled into my branch and then push up. i will drop some of the other changes though for now. | 07:08 |
| magicalChicken | Awesome okay | 07:08 |
| magicalChicken | yeah, some of the other stuff was mainly for testing | 07:08 |
| magicalChicken | it was cool to see it boot and get a dev configured without the timeout, that'll be pretty nice in the future | 07:09 |
| smoser | i think you need to push | 07:10 |
| smoser | oh. i needed rto reload | 07:10 |
| magicalChicken | haha yeah, sometimes launchpad doesn't show new commits for a couple minutes | 07:10 |
| magicalChicken | oh, wait, I'm gonna remove the merge lines, I forgot to push that | 07:11 |
| magicalChicken | actually nvm, I think that was just launchpad failing to generate a diff | 07:12 |
| smoser | http://paste.ubuntu.com/15470624/ | 07:12 |
| smoser | that is because i have enp0s25 | 07:12 |
| magicalChicken | Aah, whoops, I thought str.strip would get the ones in the middle too, but I guess it's only at the beginning and end | 07:13 |
| magicalChicken | I can fix that real quick | 07:13 |
| smoser | k | 07:17 |
| magicalChicken | just pushed, it's fixed now, although the fix is kinda ugly, but I couldn't think of another way to do it on one line | 07:25 |
| smoser | k | 07:27 |
| magicalChicken | I think I'm gonna sign off for the night, that's pretty much everything I was working on done. I'll look into vmtests for this tomorrow | 07:29 |
| smoser | magicalChicken, thanks. | 07:44 |
| smoser | magicalChicken, it still needs some work, but good job | 07:54 |
| smoser | $ python3 -c 'from cloudinit import net; print(net.generate_fallback_config())' | 07:54 |
| smoser | {'config': {'routes': [], 'interfaces': {'vethVJLV0G': {'subnets': [{'type': 'dhcp4'}, {'type': 'dhcp6'}], 'type': 'physical', 'mode': 'manual', 'name': 'vethVJLV0G', 'mac_address': 'fe:15:28:38:df:27', 'inet': 'inet'}}, 'dns': {'search': [], 'nameservers': []}}, 'version': 1} | 07:54 |
| smoser | pulled yours into my branch | 07:54 |
| smoser | man... | 09:38 |
| rharper | smoser: if I'm reading trunk.net1 right , now with fallback from magicalChicken , we can use nocloud-net instead of nocloud for ds mode ? | 13:54 |
| rharper | smoser: which openstack did you use to get network_data via config drive ? | 14:22 |
| smoser | rharper, sorry. in now. | 16:13 |
| smoser | openstack that reads config drive is ConfigDrive | 16:13 |
| smoser | i had forgotten that yesterday too. | 16:13 |
| rharper | smoser: I meant which cloud | 16:13 |
| smoser | serverstack or canonistack will provide a config drive. | 16:13 |
| rharper | as you said, serverstack doesn't populate the metadata service with 'network_data' link | 16:13 |
| smoser | right. | 16:13 |
| rharper | yeah, I saw the config drive | 16:13 |
| smoser | so that sucks. | 16:14 |
| rharper | so I'm using the example json from the link | 16:14 |
| smoser | i dont know where i can get one.... | 16:14 |
| rharper | but Ideally we'd be able to test this parser out | 16:14 |
| smoser | yeah | 16:14 |
| rharper | we'd have to standup one and enable it | 16:14 |
| smoser | let me search a couple public clouds and see if we can get network md anywhere for openstack. | 16:14 |
| rharper | or poke around any of the stacks you have access to | 16:14 |
| rharper | yeah | 16:14 |
| smoser | i'm not sure why we dont get it | 16:14 |
| rharper | so, one thing, I think the routes handling in the network stuff needs an adjustment | 16:14 |
| smoser | can you bother someone on opentsack team to see if you can get them to make it go on serverstack ? | 16:15 |
| rharper | yeah | 16:15 |
| rharper | I need to add support to the subnets section to allow one to specify a list of routes (which we already emit the 'up route ' stanzas for) | 16:17 |
| rharper | the 'networks' section in the config drive network bits is made just for that (network netmask gateway) | 16:17 |
| smoser | rharper, wel... | 16:44 |
| smoser | i got half | 16:44 |
| smoser | ssh root@162.253.53.94 | 16:45 |
| smoser | ok. so half way good news. | 16:45 |
| smoser | serverstack has the network config in the MD , but not on the config drive | 16:46 |
| smoser | which is actually the same as i'm seeing on vexxhost where iw as able to launch an instance after adding some networks. | 16:46 |
| smoser | suro-patz, you had some questions about networking and such, right ? | 16:51 |
| suro-patz | smoser: yes regarding https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1549403 | 17:02 |
| suro-patz | basically I am working on MaaS equivalent in OpenStack, i.e. Ironic - and looking for configuring multiple IP (v4 & v6) on the target node through cloud-init | 17:02 |
| smoser | ah. ok. | 17:04 |
| smoser | is it config drive datasource ? | 17:04 |
| smoser | where will the network info come from | 17:05 |
| suro-patz | smoser: yes, config drive data source | 17:06 |
| suro-patz | Neutron | 17:06 |
| smoser | rharper, ^ | 17:07 |
| smoser | that is exactly what we were wanting to work on next. | 17:08 |
| smoser | suro-patz, so in https://code.launchpad.net/~smoser/cloud-init/trunk.net1/ | 17:09 |
| smoser | that branch, the code is mostly there to do this. | 17:09 |
| suro-patz | smoser: I will look into the diff | 17:09 |
| smoser | well at least the infrastructure. | 17:09 |
| suro-patz | that would be helpful | 17:09 |
| suro-patz | I made some changes to get it working for RHEL for us | 17:10 |
| smoser | suro-patz, the thing that is missing is the ConfigDrive datasource needs a 'network_config' method (see the one in DataSourceNoCloud) | 17:10 |
| suro-patz | but I would love to adhere to the infrastructure | 17:10 |
| suro-patz | smoser: I will check | 17:10 |
| smoser | and taht network_config basically needs to return a "network config" dictionary | 17:10 |
| smoser | and then, in theory, it'd all "just work" | 17:10 |
| smoser | ah. | 17:11 |
| smoser | there are 3 bits of work | 17:11 |
| smoser | a.) getting DataSourceCondfigDrive.network_config to return the right stuff. That will require a parser/converter from openstack entworking format into 'network_config' | 17:12 |
| smoser | b.) some improvements to cloudinit/net/ for rendering that network config on ubuntu | 17:12 |
| smoser | c.) support for rendering that into Centos style networking | 17:12 |
| smoser | because i think you want centos, suro-patz | 17:12 |
| smoser | so now i have a question for you... | 17:13 |
| smoser | do you know how to convince nvoa to put a 'network_cofnig.json' on a libvirt config drive ? | 17:13 |
| suro-patz | smoser: I will work on that as I get things working end-to-end for me, I expect to start on that next week | 17:14 |
| smoser | --config-drive=1 gets me networking information in the metadata service but no network_data.json on the config drive | 17:15 |
| suro-patz | nova-compute uses a template to parse using jinja2 | 17:15 |
| suro-patz | we can pass new template/conditionally | 17:15 |
| suro-patz | or we can modify the template too | 17:15 |
| suro-patz | I had to modify some of the nova-compute code too | 17:15 |
| suro-patz | once I get things working for me, which I expect by end of this week, I will work with the community to upstream | 17:16 |
| smoser | suro-patz, ? | 17:17 |
| smoser | template to parse? | 17:17 |
| suro-patz | nova-compute uses a template for the content file | 17:18 |
| smoser | i dont want interfaces | 17:18 |
| smoser | here. let me show. | 17:19 |
| suro-patz | smoser:https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L76 | 17:20 |
| smoser | http://paste.ubuntu.com/15473498/ | 17:20 |
| smoser | suro-patz, so i dont want the "/etc/network/intefaces" style file. | 17:21 |
| suro-patz | smoser: got your point | 17:21 |
| smoser | i want the more structured network_config.json | 17:21 |
| smoser | so the above came from an instance. | 17:22 |
| smoser | but the config drive does not have that file | 17:22 |
| smoser | :-( | 17:22 |
| suro-patz | I am not aware of nova-community's argument on why we should not put network_data.json | 17:22 |
| smoser | there is an argument ? | 17:22 |
| smoser | it really looksl ike its just busted | 17:22 |
| suro-patz | I am not aware of | 17:23 |
| smoser | i poked through the code, and it seems to be *trying* to do it. | 17:23 |
| suro-patz | we can fix it | 17:23 |
| suro-patz | even I felt the same | 17:23 |
| suro-patz | we do process meta_data.json | 17:23 |
| suro-patz | but the network config is in old format | 17:23 |
| smoser | right. i want to dump the old code. | 17:23 |
| smoser | not support the network_interfaces reading... well, leave it for dead i guess. | 17:24 |
| smoser | try to keep it working. | 17:24 |
| smoser | but in cloud-dinit want to focus on reading the network_data.json | 17:24 |
| suro-patz | smoser: will work in that direction | 17:24 |
| smoser | suro-patz, ggreat. one last thing... | 17:24 |
| smoser | for now the target is to read and apply network information from a local data source only | 17:25 |
| smoser | ie, config drive tells me what networking should look like, and we apply it before networking ever comes up | 17:25 |
| smoser | the more complex scenario of where you need *some* networking to get to network provided network configuration is for later. | 17:26 |
| smoser | that make sense ? | 17:26 |
| suro-patz | are you referring to metadata service sort of scenario | 17:26 |
| smoser | that branch does a good job (on ubuntu at least) of enforcing the NICs dont just come up before cloud-init has had a chance to write the system config. | 17:27 |
| smoser | suro-patz, yes, | 17:27 |
| smoser | metadata service. | 17:27 |
| suro-patz | its like bootstrap and download further config to apply | 17:27 |
| smoser | ie, for openstack metadata service, wed' have to first dhcp on some device (or bring up local networking) and then read the network config. | 17:27 |
| smoser | right. | 17:27 |
| suro-patz | only catch is they should not be conflicting | 17:27 |
| smoser | right. | 17:28 |
| smoser | the first step (config drive) is pretty easy | 17:28 |
| smoser | and almost done there on ubuntu. we write all the config before the OS gets a chance to bring it all up. so we dont have to worry about changing it. | 17:28 |
| smoser | rharper, http://paste.ubuntu.com/15473498/ is a real world example. | 17:29 |
| rharper | smoser: nice! | 17:47 |
| rharper | smoser: nice and scary with 3 interfaces with dhcp =) | 17:49 |
| rharper | smoser: is there a definitive markup for what will get populated there? services is new; would like to see bridge examples as well | 17:50 |
| smoser | rharper, i dont know. | 17:52 |
| rharper | looks like the code | 17:52 |
| rharper | I looked at the above nova link, I can dig through there | 17:53 |
| smoser | i kind of doubt anything in openstack will be declaring bridges and such at this point | 17:53 |
| rharper | almost done parsing the json into network state so we can render eni | 17:53 |
| smoser | oh. /me remembers | 17:53 |
| smoser | how was it... jayf . works at rackspace on bare metal. they might have somethign like that. | 18:06 |
| smoser | rharper, JayF is going to get us some baremetal data | 18:11 |
| rharper | smoser: cool! | 18:11 |
| smoser | and suro-patz https://bugs.launchpad.net/nova/+bug/1513267 <-- that seems to be maybe what is missing | 18:11 |
| JayF | Just don't mine any dogecoin while you're on the box :P | 18:12 |
| * smoser googles dogecoin and assumes he should probably stop the bitcoin mining that he was doing :) | 18:13 | |
| JayF | lol | 18:14 |
| smoser | rharper, jump on that box and look around. | 18:26 |
| smoser | JayF, mentions that the vlan stuff is patched in in baremetal | 18:27 |
| smoser | not in upstream. | 18:27 |
| suro-patz | smoser: with "sudo rm -rf /var/lib/instances/*; sudo rm -rf /var/log/cloud-init.log; sudo cloud-init init —local' did not reconfigure network from content/0000 - I had changed the content of the file, but it did not apply | 18:27 |
| smoser | hm.. | 18:28 |
| suro-patz | any trick, how can I re-run cloud-init to reconfigure network, without reboot | 18:28 |
| suro-patz | even reboot also din't help | 18:28 |
| smoser | i dont knwo why that wouldnt have done it. | 18:28 |
| smatzek | rm -rf stuff under /var/lib/cloud/data as well? | 18:29 |
| suro-patz | smatzek: no, I had not done that | 18:30 |
| suro-patz | will give a shot | 18:30 |
| JayF | smoser: I can probably even supply the nova patch we use if oyu want | 18:30 |
| smatzek | at a minimum you need to take /var/lib/cloud/data/instance-id or change the UUID in it, or cloud-init sees the UUID hasn't changed and doesn't run | 18:31 |
| smatzek | take > take out | 18:31 |
| smoser | JayF, your metdata is odd | 18:32 |
| smoser | the config drive layout | 18:32 |
| smoser | has 'latest/' that has network_data.json in it | 18:32 |
| smoser | but the LIBERTY version (2015-10-15) does not | 18:32 |
| smoser | is that by design ? | 18:32 |
| JayF | probably means we haven't deployed that upstream bugfix you linked me in pm | 18:32 |
| smoser | well, JayF the bug was that it did not appear *anywhere* | 18:33 |
| JayF | oh. hm | 18:33 |
| JayF | I don't know why :) | 18:33 |
| JayF | Asking about stuff we did, at this point, 2y ago :) | 18:33 |
| smoser | rharper, for reference http://paste.ubuntu.com/15474058/ | 18:36 |
| rharper | smoser: thanks! | 18:37 |
| smoser | rharper, and you might as well grab config-drive.img.gz and cnonfig-drive.tar.gz of that system. just in case. | 18:38 |
| rharper | k | 18:39 |
| rharper | got it | 18:39 |
| suro-patz | smatzek: works, after removing the instance-id! Cool! | 18:42 |
| smoser | JayF, we're done if you want to kill that thing. | 19:05 |
| randeffects__ | nocloud question: if I use nocloud ds on a vmware vm does the disk image have to stay attached after the initial boot? if I reboot the vm and the disk image is not attached, cloud init runs again with ds none. | 19:20 |
| smoser | randeffects__, yes | 19:25 |
| smoser | you're right :) | 19:25 |
| smoser | randeffects__, https://launchpad.net/bugs/1553815 | 19:25 |
| smoser | that is fixed in trunk, and should soon be in xenial | 19:25 |
| smoser | but actually the fix i added for nocloud was to only check the seed direcdtories or the kernel command line (not mounting the disk) | 19:27 |
| randeffects__ | smoser thanks | 19:27 |
| smoser | so... but you can get the behavior you want. you just have to be explicit | 19:27 |
| smoser | and set 'manual_cache_clean' in the image or in user data | 19:27 |
| smoser | actually, that wont work from user-data | 19:28 |
| smoser | i just put a comment in that bug | 19:29 |
| JayF | smoser: cool, hopefully it helped? | 19:38 |
| smoser | yeah. thank you very much JayF . | 19:40 |
| JayF | np anytime I can help with something like that feel free to ask | 19:42 |
| vox_clamantis | hi all ... anyone know where to find info on configuring cloud-init to use OpenStack metadata service? | 20:21 |
| smoser | vox_clamantis, it should "just work". | 20:27 |
| vox_clamantis | :) i was afraid of that | 20:28 |
| vox_clamantis | it always picks another like configdrive or e2c. if i specify OpenStack as the only option in datasource_list it never finds it. although i have no problem accessing with curl | 20:29 |
| smoser | hm. | 20:30 |
| smoser | definitely if config drive is enabled it will select it. | 20:30 |
| smoser | but openstack shoudl run before ec2 | 20:30 |
| smoser | check the order | 20:30 |
| vox_clamantis | let me try again. thanks for the suggestion | 20:31 |
| smoser | magicalChicken, around ? | 20:38 |
| magicalChicken | smoser: Yeah | 20:38 |
| smoser | hey.... | 20:38 |
| smoser | shoot. sorry to ping you . | 20:39 |
| magicalChicken | it's okay | 20:39 |
| magicalChicken | is there anything I should be working on today aside from tests? | 20:39 |
| smoser | heres one | 20:39 |
| smoser | kernel command line | 20:39 |
| smoser | with your chagnes for kenrel command line in theory curtin should not need any patching of the image | 20:40 |
| smoser | ie, the ENI should be able to be left in tact | 20:40 |
| magicalChicken | Right yeah | 20:40 |
| smoser | so you could use your kernel command line branch | 20:40 |
| smoser | and integrate with net1 | 20:40 |
| smoser | and then try vmtest for curtin | 20:40 |
| smoser | the other thing there... | 20:40 |
| magicalChicken | cool, okay | 20:40 |
| smoser | we should no longer need persistent names | 20:40 |
| smoser | in the curtin xenial command line | 20:40 |
| smoser | that make sense ? kernel cmdline ... | 20:40 |
| smoser | net.ifnames=0 | 20:41 |
| smoser | i never rememberr that string | 20:41 |
| magicalChicken | Okay yeah, that makes sense | 20:41 |
| smoser | i have to run *right now* | 20:41 |
| smoser | i'll check in later. | 20:41 |
| smoser | bye | 20:41 |
| magicalChicken | bye | 20:41 |
| rharper | smoser: ok, I've got something that converts the 3 different network-data json into network_state, when then we can render_eni etc; I'm looking at cloudinit/source/DataSourceConfigDrive ; where do we determine which dir of the config drive to use? for example, the network_data.json is only in latest ; | 21:17 |
| rharper | openstack helpers | 21:20 |
| rharper | OS_HAVANA is the latest, prolly should update that | 21:21 |
| * rharper relocates | 21:30 | |
| smoser | rharper, did you actually see it in latest ? | 21:31 |
| smoser | because i've not seen it anywhere. althought hat file sys it will be. | 21:31 |
| smoser | i think we should assume correct working order of the cloud though and use the LIBERTY date | 21:31 |
| smoser | 'latest' is not really something you can use. they're explicilty stating there is a ABI. using latest/ is guaranteed to fail at some point. | 21:32 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!