[14:51] <Raboo> hi
[14:51] <Raboo> what usually comes after the ssh key generation?
[14:51] <Raboo> have a node that is stuck after the ssh key generation
[15:22] <dpb1> Raboo: physical system?  cloud?
[15:22] <dpb1> could be a RNG issue.
[16:00] <Raboo> dpb1 physical
[16:01] <Raboo> it gets stuck when running trusty, but when i tested xenial it worked..
[16:01] <Raboo> but i would like it to work on trusty again
[16:02] <dpb1> Raboo: through maas I assume?
[16:02] <Raboo> dpb1 nope, I use The Foreman
[16:03] <Raboo> https://theforeman.org/
[16:03] <dpb1> Raboo: your images are up to date for Ubuntu?
[16:03] <Raboo> yes, i build it daily
[16:03] <smoser> Raboo: most likely, trusty is just not going to work easily
[16:03] <smoser> it has no dynanmic network configuration
[16:03] <Raboo> and i even triggered a manual build today when it failed
[16:04] <smoser> meaning that the images have 'dhcp eth0'
[16:04] <smoser> and if you dont have a 'eth0' or its not dhcp..
[16:04] <smoser> but maybe thats not your issue.
[16:04] <Raboo> the network came up
[16:04] <smoser> yeah, then i assume it is that you dont have a eth0 ?
[16:04] <Raboo> it printed ip before it generated ssh keys
[16:04] <smoser> or eth0 is not plugged in and some other.
[16:04] <smoser> oh.
[16:04] <Raboo> it's a bond0 actually
[16:04] <smoser> hm...  ... i'm not really sure where youd' be hung then.
[16:05] <Raboo> and i could ping the node
[16:05] <Raboo> the most annoying thing is that it got stuck. It didn't fail and drop to a console so I could read logs
[16:05] <Raboo> so very hard to debug
[16:07] <Raboo> i think it got stuck at the end of "init" phase
[16:12] <dpb1> Raboo: what cloud init version are you using in your trusty images?
[16:13] <dpb1> Raboo: is it the one in the trusty-updates pocket?
[16:16] <Raboo> dpb1 hmm let me try to find out, i think it was 17 something
[16:20] <Raboo> is there a binary or a config file that contains a version string?
[16:21] <Raboo> could be 0.7.5.
[16:24] <Raboo> it's the same as in https://cloud-images.ubuntu.com/trusty/current/
[16:25] <Raboo> it contains this file ./usr/lib/python2.7/dist-packages/cloud_init-0.7.5.egg-info
[16:25] <Raboo> so i guess it's 0.7.5
[16:26] <Raboo> ./usr/share/doc/cloud-init/changelog.Debian.gz says at the top of the file "cloud-init (0.7.5-0ubuntu1.22) trusty; urgency=medium"
[16:29] <dpb1> Raboo: ok
[16:31] <Raboo> this used to work before, don't know if ubuntu has bricked something since I update my images daily
[16:47] <smoser> :-(
[17:20] <smoser> Raboo: you can open a bug. just please provide as much information as you can.
[17:20] <smoser> you can just "backdoor" the image too
[17:21] <smoser> then you can go in with the backdoor to get other info
[17:21] <smoser> http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image
[17:32] <Raboo> smoser what does the backdoor do in short?
[17:33] <Raboo> cause i have a local user already
[17:36] <Raboo> problem was it doesn't even drop to the console where i can login
[17:56] <smoser> you said it set up network
[17:56] <smoser> no ?
[17:56] <smoser> just ssh in
[18:17] <Raboo> smoser i don't think sshd is started
[18:19] <Raboo> i'm going to reinstall another node tomorrow, see if i can replicate the issue on another node.
[18:44] <dpb1> Raboo: from the sounds of it, that could be what is stalling, actually (sshd startup)
[19:36] <smoser> rharper: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334147
[19:36] <smoser> can you ack that ?
[19:38] <rharper> acked
[21:17] <blackboxsw> ok I'm out of the way on the two branches (no more changes from me on these): https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115
[21:17] <blackboxsw> just pushed latest changes/rebase and fixes
[21:17] <blackboxsw> hitting review queue
[22:21] <powersj> smoser: seems you pulled your KVM fix, so I am going to enable the KVM tests in jenkins
[22:30] <blackboxsw> smoser: rharper http://pastebin.ubuntu.com/26075320/ shouldn't our v1 instance-data.json actually surface public-hostname instead of local-hostname  from metadata? Right now the instance-data.json sources cloud.get_hostname() which grabs local-hostname from metadata if present
[22:30] <blackboxsw> checkout the ec2 section of that pastebin
[22:31] <rharper> blackboxsw: what does hostname on the instance say now ?
[22:31] <rharper> I suspect it retains the .internal hostname
[22:31] <blackboxsw>   "public-hostname": "ip-10-0-20-27",
[22:31] <rharper> an external/public hostname is for facing outward IIUC
[22:31] <rharper> no, run 'hostname' on your instance
[22:31] <blackboxsw> which is actually grabs from hostname on the cmdline
[22:31] <rharper> ie, what actually ended up in the Linux hostname value
[22:31] <blackboxsw> ubuntu@ip-10-0-20-27:~$ hostname
[22:31] <blackboxsw> ip-10-0-20-27
[22:32] <rharper> ok
[22:32] <blackboxsw> but for ec2 instances, you can't really get ahold of that local 'hostname'
[22:32] <rharper> then I would think that should match in the data (which  is what you're pointing out, yes?)
[22:32] <rharper> can you not resolve that now ?
[22:32] <rharper> the .internal ?
[22:32] <rharper> what resolver is configured on the instance ?
[22:32] <blackboxsw> I can from within amazon, but not from home
[22:33] <rharper> right
[22:33] <rharper> which is fine
[22:33] <rharper> the instnace.json is from within the instance
[22:33] <rharper> but, I think what's happening is that the hostname public value is not yet when dump the instance data
[22:34] <rharper> but it's a valid hostname on the instance
[22:34] <rharper> so, there's a little fuzziness w.r.t what that value should reflect
[22:34] <rharper> should it be the hostname at the time we process the metadata (which is what I think it is doing now)
[22:34] <blackboxsw> right it's a valid hostname on the instance, what would you call the public-hostname which you can reference external to a given machine?
[22:34] <rharper> or should it reflect what the hostname value will be eventually (after cc_hostname runs)
[22:34] <rharper> public-hostname
[22:34] <rharper> I think
[22:35] <rharper> we might need other cloud examples to see what makes the most sense
[22:35] <blackboxsw> @ the time we process metadata, we could query md['network']['public-hostname'] and it'd give us ec2-18-216-195-116.us-east-2.compute.amazonaws.com
[22:35] <rharper> I think I'd be fine with tagging hostnames with public or private
[22:36] <rharper> and let instance users pick
[22:36] <rharper> and skip having a instance_json['hostname']  value
[22:36] <blackboxsw> yeah +1
[22:36] <rharper> or have it include a dictionary of public/private values
[22:36] <blackboxsw> I have ['v1'][
[22:36] <rharper> right, I see that
[22:36] <blackboxsw> I have ['v1']['public-hostname'] which I think is a misnomer (as it's actually the local/private hostname)
[22:37] <blackboxsw> so I'll surface public/private hostname keys which will allow us to differentiate
[22:37] <rharper> "public-hostname": "ip-10-0-20-27",
[22:37] <rharper> yes, that seems wrong
[22:37] <rharper> that's the .internal one
[22:37] <blackboxsw> yeah definitely feeling like that
[22:37] <rharper> but you're saying it has a hostname that matches
[22:37] <blackboxsw> even though that's what our cloud.get_hostname() logic defaults to
[22:38] <blackboxsw> I even believe cloud.get_hostname() should probably default to the public hostname where available instead of private
[22:38] <rharper> I wouldn't change what that does
[22:38] <blackboxsw> but, it currently doesn't (and that behavior is surfaced in multiple cc modules already.
[22:38] <blackboxsw> yeah
[22:38] <rharper> but in our rendering of the data into json, we could pick something else (or call it differently)
[22:38] <blackboxsw> ok will tweak that. as public/private is more explicit
[22:39] <rharper> depending on the scripts folks may want the local/private name, for configuring services, etc
[22:39] <rharper> but if they're injecting values to configure a service to point to another host, those would be the public ones
[22:39] <blackboxsw> right
[22:39] <rharper> so I think we need both values
[22:39] <blackboxsw> +1
[22:39] <blackboxsw> vote 'private-hostname' vs 'local-hostname'?
[22:52] <rharper> hrm; I dunno
[22:53] <rharper> I kinda like public/private symmetry
[22:53] <rharper> but I honestly have no strong feelings