| bob_cheesey | morning all - I've been having some issues setting the hostname on debian jessie using 0.7.6 (and 0.7.7) - is there somewhere I can raise issues like this? I'm not sure whether it's a bug or my implementation | 11:25 |
|---|---|---|
| bob_cheesey | well clearly this isn't the place to ask! | 13:31 |
| rharper | bob_cheesey: it's mostly quiet here except during US Central hours; ask away and we usually get back to you if your still in-channel ; you can report bugs here: https://bugs.launchpad.net/cloud-init | 13:41 |
| * rharper has to take kiddos to school, bbiab | 13:41 | |
| bob_cheesey | rharper: ah thanks, that's useful to know! | 13:42 |
| bob_cheesey | i'll try again later | 13:42 |
| smoser | rharper, http://paste.ubuntu.com/23911739/ | 16:07 |
| magicalChicken | bob_cheesey: are you getting a timeout from hostnamectl in cc_set_hostname? | 18:43 |
| magicalChicken | I've had similar issues on centos but haven't had time to debug | 18:44 |
| burgerk | smoser, sorry I was out of the office yesterday... to followup on the issue I was having with rebooting without the ConfigDrive attached. I captured the logs you requested. I believe this is likely the section of logs of interest. http://paste.ubuntu.com/23912850/ | 19:27 |
| burgerk | I can attach a tar of all the logs, just tell me where | 19:29 |
| smoser | burgerk, yeah... where is this running ? | 19:30 |
| smoser | what os are you on ? | 19:30 |
| smoser | and what openstack ? | 19:30 |
| smoser | xen, isnt it. | 19:30 |
| burgerk | Ubuntu 16.04.1 | 19:32 |
| burgerk | or host hardware ? | 19:32 |
| smoser | your guest is 16.04 | 19:32 |
| burgerk | yes | 19:32 |
| smoser | do you know what openstack version is under you ? | 19:32 |
| burgerk | Newton ... or do you want a specific driver # | 19:33 |
| smoser | burgerk, is it xen ? | 19:39 |
| burgerk | ppc64le | 19:39 |
| smoser | rharper, ^ | 19:39 |
| smoser | wow. | 19:39 |
| rharper | \o/ | 19:40 |
| smoser | this very thing rharper and i were wondering about | 19:40 |
| smoser | :) | 19:40 |
| rharper | virsh dumpxml need | 19:40 |
| smoser | burgerk, can you do that ^ ? can you get at the host at all ? | 19:40 |
| rharper | smoser: IIRC ppc64le has a uuid out in the device tree sys path | 19:41 |
| smoser | so in your guest, can you do this, burgerk | 19:42 |
| smoser | sudo grep . -r /sys/firmware/devicetree/ | grep -v Binary | 19:42 |
| smoser | sudo grep . -r /sys/firmware/devicetree/ | grep -v Binary | pastebinit | 19:42 |
| rharper | /sys/firmware/devicetree/base/vm,uuid | 19:43 |
| rharper | but yeah, a full collection will be super useful | 19:44 |
| smoser | that is crazy helpful, burgerk | 19:44 |
| burgerk | http://paste.ubuntu.com/23912946/ | 19:45 |
| smoser | booo | 19:45 |
| burgerk | not much there | 19:45 |
| smoser | can just | 19:46 |
| smoser | cat /sys/firmware/devicetree/base/vm,uuid | 19:46 |
| smoser | shows no file ? | 19:46 |
| smoser | (ie,m wondered if grep -r didn't follow a symlink maybe) | 19:46 |
| burgerk | yes | 19:47 |
| burgerk | no such file | 19:47 |
| burgerk | contents of /sys/firmware/devicetree/base http://paste.ubuntu.com/23912964/ | 19:50 |
| smoser | ibm,partition-uuid ? | 19:51 |
| smoser | and artition-name | 19:51 |
| smoser | anything in those ? | 19:51 |
| smoser | is this kvm ? | 19:51 |
| smoser | or is it powervm | 19:51 |
| smoser | oh. wait. i dont knwo if there is ppc64el powervm | 19:51 |
| smoser | and system-id | 19:51 |
| smoser | theres just binary stuff in there ? | 19:51 |
| burgerk | powervm | 19:52 |
| burgerk | ibm,partition-uuid 7ab415a2-30e7-482f-8806-e9ec4197b7ba | 19:53 |
| smoser | that by chance match the uuid ? | 19:54 |
| smoser | from the openstack c onfig drive? | 19:54 |
| burgerk | instance -> /var/lib/cloud/instances/fab415a2-30e7-482f-8806-e9ec4197b7ba | 19:55 |
| burgerk | same except 1st character | 19:56 |
| rharper | eww | 19:58 |
| burgerk | I don't have config drive anymore, removed to repeat test | 19:58 |
| rharper | that looks bugy | 19:58 |
| smoser | wow | 19:58 |
| smoser | that would be a massive coincidence | 19:59 |
| smoser | :) | 19:59 |
| rharper | https://review.openstack.org/#/c/200879/ | 20:00 |
| rharper | but doesn't show where it would appear | 20:00 |
| smoser | burgerk, so why didnt our 'grep ." above show us that file ? | 20:00 |
| burgerk | powervm | 20:00 |
| burgerk | sorry bad paste | 20:00 |
| rharper | https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/vm.py#L310 | 20:01 |
| burgerk | so that covert method is changing the system UUID byte 0, bit 0 to 0 | 20:17 |
| rharper | eww | 20:22 |
| rharper | certainly then the OpenStack instance UUID ought to match the UUID inside the vm | 20:23 |
| rharper | even if they need to drop a bit to avoid collisions (or for whatever reason) | 20:23 |
| rharper | nova list $UUID should match inside the instance ; I would think | 20:23 |
| burgerk | I believe the converted UUID is being sent as system UUID when the VM is created and the config drive contains the original openstack UUID | 20:26 |
| rharper | but your instance symlink says otherwise | 20:27 |
| rharper | vs. the device-tree value which is created by the hypervisor | 20:27 |
| burgerk | instance symlink matches config drive value | 20:27 |
| rharper | but not the vm id | 20:27 |
| rharper | which is what the hypervisor got | 20:27 |
| rharper | ibm,partition-uuid ; maybe that's by design | 20:28 |
| burgerk | yes I believe it is some sort of requirement that the system UUID bit 0 be 0 | 20:28 |
| rharper | what we were hoping for was something that indicated this was an openstack instance; on KVM and Xen, for example the SystemVendor is set to Openstack | 20:29 |
| rharper | that's mostly for the network-facing metadata service; config-drives indicate their presence via filesystem attributes; however; IIRC, your issue was that you removed the configdrive, which also set the UUID, then when cloud-init runs a second time somewhere else, its looking for a datasource, it's going to search the system for a uuid to see if that matches existing instance directories | 20:31 |
| burgerk | yes that appears to be the issue | 20:31 |
| rharper | that seems to have gone awry in that we're poking for DMI data (which we won't see on power); also, even if we did look at the partition,uuid, we'd need to apply a 1 to bit-0 of byte-0 to match if we're on powervm | 20:32 |
| * rharper will bbiab, kiddo pick up time | 20:33 | |
| rharper | burgerk: in your situation, did you set manual_cache_clean: True ? ISTR something (or someone) not wanting to set that; though for configdrive it's required if you expect it to boot with the same instance data; if you do have that set, and it's failing (due to the search for dmi data on power) then definitely file a bug, https://bugs.launchpad.net/cloud-init | 20:58 |
| burgerk | The drawback of that was that I can't just snapshot the VM and deploy that image then, right ? | 21:00 |
| burgerk | there would be manual steps before snapshoting? | 21:01 |
| rharper | I think smoser pointed out that if you snapshot and boot somewhere else it _is_ a new instance; so you shouldn't deploy the snapshot without re-running cloud init | 21:04 |
| rharper | you don't want nodes with the same ip, the same sshd host keys, etc | 21:04 |
| burgerk | right, I want cloud-init to run again if it is a new instance, i.e. new config drive and new UUID on it | 21:06 |
| burgerk | what I don't want is to have to manually delete something before I snapshot | 21:07 |
| rharper | how are you deploying your snapshot? with openstack still, so nova boot points to an existing snapshot image? | 21:08 |
| burgerk | yes, snapshot the current VM with openstack, then nova boot pointing to that snapshot | 21:09 |
| rharper | IIUC, when you boot the snapshot again, cloud-init will run again, as it needs to since you're on a new instance; but I'm guessing there is something that you don't want to run again as you're on this snapshot? | 21:09 |
| burgerk | I will give that a value a try and see what happens | 21:11 |
| smoser | rharper is right, it should all "just work" | 21:12 |
| burgerk | smoser, after snapshot and deploy of the snapshot, it is not writing network info from the config drive on first boot using manual_cache_cleanup: True | 21:59 |
| smoser | burgerk, i'm confused. | 22:01 |
| smoser | why are you setting manual_cache_cleanup=True ? | 22:01 |
| smoser | (and its manual_cache_clean (no "up"). | 22:02 |
| smoser | i thoguht your problem before was that cloud-init *was* re-writing network configuration | 22:02 |
| rharper | smoser: my fault | 22:03 |
| burgerk | rharper suggested using that property to fix the situation for removing the config drive | 22:03 |
| rharper | smoser: when using a config drive, and then removing it, we have to use that , right? | 22:03 |
| rharper | it sounds like in manual_cache_clean mode, we don't re-render network-config? but generate fall-back config (dhcp on first interface) | 22:05 |
| rharper | burgerk: in general the static network config is going to break if someone else also boots that static network config in the same tenant network | 22:05 |
| smoser | manual cache clean should basically mean that cloud-init always re-uses the stale data in /var/lib/cloud/instance/ | 22:06 |
| smoser | and thus, the second time or any time after (even after snapshotting and re-deploying) it boots will think it is not first boot | 22:07 |
| smoser | basically its *supposed* to just never go looking for a datasource again | 22:07 |
| smoser | as long as there is one in /var/lib/cloud/instance/ | 22:07 |
| rharper | right; how does that intersect network config | 22:08 |
| burgerk | ok, that was my understanding of the property and why I didn't want to use it ... I need something like use datasource if it exists, otherwise stale data in /var/lib/cloud/instance/ :) | 22:08 |
| rharper | I've not seen the config-drive, but apparently it's including some static eni configuration which gets written to /etc/network/interfaces.d/50-cloud-init.cfg | 22:08 |
| burgerk | correct | 22:09 |
| smoser | burgerk, well, that i think is the behavior you'd get on intel | 22:09 |
| smoser | in a vm on openstack. | 22:09 |
| rharper | if we set manual_cache, it appears that we're still doing fallback networking (re-writing the 50-cloud-init.cfg file) but not with the network info which should be embedded in the instance.obj right? | 22:10 |
| smoser | because on intel, cloud-init comes up and checks the product uuid and says "oh, its the same as the one in /ar/lib/cloud/instance. no need to look again". | 22:10 |
| smoser | i think. | 22:10 |
| rharper | burgerk: is it possible to share your user-data/meta-data files in your config-drive ?? (or a variant of it) ? | 22:10 |
| smoser | it shouldnt do that | 22:10 |
| rharper | but burgerk is not using the same instance | 22:10 |
| rharper | snapshot to cinder, nova boot -from-snapshot | 22:11 |
| smoser | for sure its possible it is,b ut it shoudl only render networkign on first instance boot | 22:11 |
| rharper | it *is* a new instance | 22:11 |
| rharper | but we're saying, use stale data (on purpose) | 22:11 |
| burgerk | with the property set to true it is using stale data, not fallback data | 22:11 |
| burgerk | rharper, here is network config drive files https://paste.ubuntu.com/23913828/ | 22:19 |
| rharper | burgerk: thanks, I'll recreate; I suspect you're still going to have issues with using the snapshot with stale-data; but if we say to use stale-data, then it shouldn't wipe the network config | 22:20 |
| burgerk | correct it appears to be using stale-data on all boots ( including first boot ) | 22:21 |
| burgerk | rharper, smoser I will also see if something can be done on the powervm side to get matching UUIDs for system UUID and ConfigDrive UUID value | 22:42 |
| rharper | ok | 22:42 |
| rharper | if you asking, we're very interested in getting the Openstack Nova string into the devicetree output so we can detect that we're Openstack on ppc64le platforms | 22:43 |
| burgerk | ok | 22:44 |
| rharper | or, somewhere else if devicetree isn't possible | 22:50 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!