[11:25] morning all - I've been having some issues setting the hostname on debian jessie using 0.7.6 (and 0.7.7) - is there somewhere I can raise issues like this? I'm not sure whether it's a bug or my implementation [13:31] well clearly this isn't the place to ask! [13:41] bob_cheesey: it's mostly quiet here except during US Central hours; ask away and we usually get back to you if your still in-channel ; you can report bugs here: https://bugs.launchpad.net/cloud-init [13:41] * rharper has to take kiddos to school, bbiab [13:42] rharper: ah thanks, that's useful to know! [13:42] i'll try again later [16:07] rharper, http://paste.ubuntu.com/23911739/ [18:43] bob_cheesey: are you getting a timeout from hostnamectl in cc_set_hostname? [18:44] I've had similar issues on centos but haven't had time to debug [19:27] smoser, sorry I was out of the office yesterday... to followup on the issue I was having with rebooting without the ConfigDrive attached. I captured the logs you requested. I believe this is likely the section of logs of interest. http://paste.ubuntu.com/23912850/ [19:29] I can attach a tar of all the logs, just tell me where [19:30] burgerk, yeah... where is this running ? [19:30] what os are you on ? [19:30] and what openstack ? [19:30] xen, isnt it. [19:32] Ubuntu 16.04.1 [19:32] or host hardware ? [19:32] your guest is 16.04 [19:32] yes [19:32] do you know what openstack version is under you ? [19:33] Newton ... or do you want a specific driver # [19:39] burgerk, is it xen ? [19:39] ppc64le [19:39] rharper, ^ [19:39] wow. [19:40] \o/ [19:40] this very thing rharper and i were wondering about [19:40] :) [19:40] virsh dumpxml need [19:40] burgerk, can you do that ^ ? can you get at the host at all ? [19:41] smoser: IIRC ppc64le has a uuid out in the device tree sys path [19:42] so in your guest, can you do this, burgerk [19:42] sudo grep . -r /sys/firmware/devicetree/ | grep -v Binary [19:42] sudo grep . -r /sys/firmware/devicetree/ | grep -v Binary | pastebinit [19:43] /sys/firmware/devicetree/base/vm,uuid [19:44] but yeah, a full collection will be super useful [19:44] that is crazy helpful, burgerk [19:45] http://paste.ubuntu.com/23912946/ [19:45] booo [19:45] not much there [19:46] can just [19:46] cat /sys/firmware/devicetree/base/vm,uuid [19:46] shows no file ? [19:46] (ie,m wondered if grep -r didn't follow a symlink maybe) [19:47] yes [19:47] no such file [19:50] contents of /sys/firmware/devicetree/base http://paste.ubuntu.com/23912964/ [19:51] ibm,partition-uuid ? [19:51] and artition-name [19:51] anything in those ? [19:51] is this kvm ? [19:51] or is it powervm [19:51] oh. wait. i dont knwo if there is ppc64el powervm [19:51] and system-id [19:51] theres just binary stuff in there ? [19:52] powervm [19:53] ibm,partition-uuid 7ab415a2-30e7-482f-8806-e9ec4197b7ba [19:54] that by chance match the uuid ? [19:54] from the openstack c onfig drive? [19:55] instance -> /var/lib/cloud/instances/fab415a2-30e7-482f-8806-e9ec4197b7ba [19:56] same except 1st character [19:58] eww [19:58] I don't have config drive anymore, removed to repeat test [19:58] that looks bugy [19:58] wow [19:59] that would be a massive coincidence [19:59] :) [20:00] https://review.openstack.org/#/c/200879/ [20:00] but doesn't show where it would appear [20:00] burgerk, so why didnt our 'grep ." above show us that file ? [20:00] powervm [20:00] sorry bad paste [20:01] https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/vm.py#L310 [20:17] so that covert method is changing the system UUID byte 0, bit 0 to 0 [20:22] eww [20:23] certainly then the OpenStack instance UUID ought to match the UUID inside the vm [20:23] even if they need to drop a bit to avoid collisions (or for whatever reason) [20:23] nova list $UUID should match inside the instance ; I would think [20:26] I believe the converted UUID is being sent as system UUID when the VM is created and the config drive contains the original openstack UUID [20:27] but your instance symlink says otherwise [20:27] vs. the device-tree value which is created by the hypervisor [20:27] instance symlink matches config drive value [20:27] but not the vm id [20:27] which is what the hypervisor got [20:28] ibm,partition-uuid ; maybe that's by design [20:28] yes I believe it is some sort of requirement that the system UUID bit 0 be 0 [20:29] what we were hoping for was something that indicated this was an openstack instance; on KVM and Xen, for example the SystemVendor is set to Openstack [20:31] that's mostly for the network-facing metadata service; config-drives indicate their presence via filesystem attributes; however; IIRC, your issue was that you removed the configdrive, which also set the UUID, then when cloud-init runs a second time somewhere else, its looking for a datasource, it's going to search the system for a uuid to see if that matches existing instance directories [20:31] yes that appears to be the issue [20:32] that seems to have gone awry in that we're poking for DMI data (which we won't see on power); also, even if we did look at the partition,uuid, we'd need to apply a 1 to bit-0 of byte-0 to match if we're on powervm [20:33] * rharper will bbiab, kiddo pick up time [20:58] burgerk: in your situation, did you set manual_cache_clean: True ? ISTR something (or someone) not wanting to set that; though for configdrive it's required if you expect it to boot with the same instance data; if you do have that set, and it's failing (due to the search for dmi data on power) then definitely file a bug, https://bugs.launchpad.net/cloud-init [21:00] The drawback of that was that I can't just snapshot the VM and deploy that image then, right ? [21:01] there would be manual steps before snapshoting? [21:04] I think smoser pointed out that if you snapshot and boot somewhere else it _is_ a new instance; so you shouldn't deploy the snapshot without re-running cloud init [21:04] you don't want nodes with the same ip, the same sshd host keys, etc [21:06] right, I want cloud-init to run again if it is a new instance, i.e. new config drive and new UUID on it [21:07] what I don't want is to have to manually delete something before I snapshot [21:08] how are you deploying your snapshot? with openstack still, so nova boot points to an existing snapshot image? [21:09] yes, snapshot the current VM with openstack, then nova boot pointing to that snapshot [21:09] IIUC, when you boot the snapshot again, cloud-init will run again, as it needs to since you're on a new instance; but I'm guessing there is something that you don't want to run again as you're on this snapshot? [21:11] I will give that a value a try and see what happens [21:12] rharper is right, it should all "just work" [21:59] smoser, after snapshot and deploy of the snapshot, it is not writing network info from the config drive on first boot using manual_cache_cleanup: True [22:01] burgerk, i'm confused. [22:01] why are you setting manual_cache_cleanup=True ? [22:02] (and its manual_cache_clean (no "up"). [22:02] i thoguht your problem before was that cloud-init *was* re-writing network configuration [22:03] smoser: my fault [22:03] rharper suggested using that property to fix the situation for removing the config drive [22:03] smoser: when using a config drive, and then removing it, we have to use that , right? [22:05] it sounds like in manual_cache_clean mode, we don't re-render network-config? but generate fall-back config (dhcp on first interface) [22:05] burgerk: in general the static network config is going to break if someone else also boots that static network config in the same tenant network [22:06] manual cache clean should basically mean that cloud-init always re-uses the stale data in /var/lib/cloud/instance/ [22:07] and thus, the second time or any time after (even after snapshotting and re-deploying) it boots will think it is not first boot [22:07] basically its *supposed* to just never go looking for a datasource again [22:07] as long as there is one in /var/lib/cloud/instance/ [22:08] right; how does that intersect network config [22:08] ok, that was my understanding of the property and why I didn't want to use it ... I need something like use datasource if it exists, otherwise stale data in /var/lib/cloud/instance/ :) [22:08] I've not seen the config-drive, but apparently it's including some static eni configuration which gets written to /etc/network/interfaces.d/50-cloud-init.cfg [22:09] correct [22:09] burgerk, well, that i think is the behavior you'd get on intel [22:09] in a vm on openstack. [22:10] if we set manual_cache, it appears that we're still doing fallback networking (re-writing the 50-cloud-init.cfg file) but not with the network info which should be embedded in the instance.obj right? [22:10] because on intel, cloud-init comes up and checks the product uuid and says "oh, its the same as the one in /ar/lib/cloud/instance. no need to look again". [22:10] i think. [22:10] burgerk: is it possible to share your user-data/meta-data files in your config-drive ?? (or a variant of it) ? [22:10] it shouldnt do that [22:10] but burgerk is not using the same instance [22:11] snapshot to cinder, nova boot -from-snapshot [22:11] for sure its possible it is,b ut it shoudl only render networkign on first instance boot [22:11] it *is* a new instance [22:11] but we're saying, use stale data (on purpose) [22:11] with the property set to true it is using stale data, not fallback data [22:19] rharper, here is network config drive files https://paste.ubuntu.com/23913828/ [22:20] burgerk: thanks, I'll recreate; I suspect you're still going to have issues with using the snapshot with stale-data; but if we say to use stale-data, then it shouldn't wipe the network config [22:21] correct it appears to be using stale-data on all boots ( including first boot ) [22:42] rharper, smoser I will also see if something can be done on the powervm side to get matching UUIDs for system UUID and ConfigDrive UUID value [22:42] ok [22:43] if you asking, we're very interested in getting the Openstack Nova string into the devicetree output so we can detect that we're Openstack on ppc64le platforms [22:44] ok [22:50] or, somewhere else if devicetree isn't possible