[14:51] hi [14:51] what usually comes after the ssh key generation? [14:51] have a node that is stuck after the ssh key generation [15:22] Raboo: physical system? cloud? [15:22] could be a RNG issue. [16:00] dpb1 physical [16:01] it gets stuck when running trusty, but when i tested xenial it worked.. [16:01] but i would like it to work on trusty again [16:02] Raboo: through maas I assume? [16:02] dpb1 nope, I use The Foreman [16:03] https://theforeman.org/ [16:03] Raboo: your images are up to date for Ubuntu? [16:03] yes, i build it daily [16:03] Raboo: most likely, trusty is just not going to work easily [16:03] it has no dynanmic network configuration [16:03] and i even triggered a manual build today when it failed [16:04] meaning that the images have 'dhcp eth0' [16:04] and if you dont have a 'eth0' or its not dhcp.. [16:04] but maybe thats not your issue. [16:04] the network came up [16:04] yeah, then i assume it is that you dont have a eth0 ? [16:04] it printed ip before it generated ssh keys [16:04] or eth0 is not plugged in and some other. [16:04] oh. [16:04] it's a bond0 actually [16:04] hm... ... i'm not really sure where youd' be hung then. [16:05] and i could ping the node [16:05] the most annoying thing is that it got stuck. It didn't fail and drop to a console so I could read logs [16:05] so very hard to debug [16:07] i think it got stuck at the end of "init" phase [16:12] Raboo: what cloud init version are you using in your trusty images? [16:13] Raboo: is it the one in the trusty-updates pocket? [16:16] dpb1 hmm let me try to find out, i think it was 17 something [16:20] is there a binary or a config file that contains a version string? [16:21] could be 0.7.5. [16:24] it's the same as in https://cloud-images.ubuntu.com/trusty/current/ [16:25] it contains this file ./usr/lib/python2.7/dist-packages/cloud_init-0.7.5.egg-info [16:25] so i guess it's 0.7.5 [16:26] ./usr/share/doc/cloud-init/changelog.Debian.gz says at the top of the file "cloud-init (0.7.5-0ubuntu1.22) trusty; urgency=medium" [16:29] Raboo: ok [16:31] this used to work before, don't know if ubuntu has bricked something since I update my images daily [16:47] :-( [17:20] Raboo: you can open a bug. just please provide as much information as you can. [17:20] you can just "backdoor" the image too [17:21] then you can go in with the backdoor to get other info [17:21] http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image [17:32] smoser what does the backdoor do in short? [17:33] cause i have a local user already [17:36] problem was it doesn't even drop to the console where i can login [17:56] you said it set up network [17:56] no ? [17:56] just ssh in [18:17] smoser i don't think sshd is started [18:19] i'm going to reinstall another node tomorrow, see if i can replicate the issue on another node. [18:44] Raboo: from the sounds of it, that could be what is stalling, actually (sshd startup) [19:36] rharper: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334147 [19:36] can you ack that ? [19:38] acked [21:17] ok I'm out of the way on the two branches (no more changes from me on these): https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115 [21:17] just pushed latest changes/rebase and fixes [21:17] hitting review queue [22:21] smoser: seems you pulled your KVM fix, so I am going to enable the KVM tests in jenkins [22:30] smoser: rharper http://pastebin.ubuntu.com/26075320/ shouldn't our v1 instance-data.json actually surface public-hostname instead of local-hostname from metadata? Right now the instance-data.json sources cloud.get_hostname() which grabs local-hostname from metadata if present [22:30] checkout the ec2 section of that pastebin [22:31] blackboxsw: what does hostname on the instance say now ? [22:31] I suspect it retains the .internal hostname [22:31] "public-hostname": "ip-10-0-20-27", [22:31] an external/public hostname is for facing outward IIUC [22:31] no, run 'hostname' on your instance [22:31] which is actually grabs from hostname on the cmdline [22:31] ie, what actually ended up in the Linux hostname value [22:31] ubuntu@ip-10-0-20-27:~$ hostname [22:31] ip-10-0-20-27 [22:32] ok [22:32] but for ec2 instances, you can't really get ahold of that local 'hostname' [22:32] then I would think that should match in the data (which is what you're pointing out, yes?) [22:32] can you not resolve that now ? [22:32] the .internal ? [22:32] what resolver is configured on the instance ? [22:32] I can from within amazon, but not from home [22:33] right [22:33] which is fine [22:33] the instnace.json is from within the instance [22:33] but, I think what's happening is that the hostname public value is not yet when dump the instance data [22:34] but it's a valid hostname on the instance [22:34] so, there's a little fuzziness w.r.t what that value should reflect [22:34] should it be the hostname at the time we process the metadata (which is what I think it is doing now) [22:34] right it's a valid hostname on the instance, what would you call the public-hostname which you can reference external to a given machine? [22:34] or should it reflect what the hostname value will be eventually (after cc_hostname runs) [22:34] public-hostname [22:34] I think [22:35] we might need other cloud examples to see what makes the most sense [22:35] @ the time we process metadata, we could query md['network']['public-hostname'] and it'd give us ec2-18-216-195-116.us-east-2.compute.amazonaws.com [22:35] I think I'd be fine with tagging hostnames with public or private [22:36] and let instance users pick [22:36] and skip having a instance_json['hostname'] value [22:36] yeah +1 [22:36] or have it include a dictionary of public/private values [22:36] I have ['v1'][ [22:36] right, I see that [22:36] I have ['v1']['public-hostname'] which I think is a misnomer (as it's actually the local/private hostname) [22:37] so I'll surface public/private hostname keys which will allow us to differentiate [22:37] "public-hostname": "ip-10-0-20-27", [22:37] yes, that seems wrong [22:37] that's the .internal one [22:37] yeah definitely feeling like that [22:37] but you're saying it has a hostname that matches [22:37] even though that's what our cloud.get_hostname() logic defaults to [22:38] I even believe cloud.get_hostname() should probably default to the public hostname where available instead of private [22:38] I wouldn't change what that does [22:38] but, it currently doesn't (and that behavior is surfaced in multiple cc modules already. [22:38] yeah [22:38] but in our rendering of the data into json, we could pick something else (or call it differently) [22:38] ok will tweak that. as public/private is more explicit [22:39] depending on the scripts folks may want the local/private name, for configuring services, etc [22:39] but if they're injecting values to configure a service to point to another host, those would be the public ones [22:39] right [22:39] so I think we need both values [22:39] +1 [22:39] vote 'private-hostname' vs 'local-hostname'? [22:52] hrm; I dunno [22:53] I kinda like public/private symmetry [22:53] but I honestly have no strong feelings