| Raboo | hi | 14:51 |
|---|---|---|
| Raboo | what usually comes after the ssh key generation? | 14:51 |
| Raboo | have a node that is stuck after the ssh key generation | 14:51 |
| dpb1 | Raboo: physical system? cloud? | 15:22 |
| dpb1 | could be a RNG issue. | 15:22 |
| Raboo | dpb1 physical | 16:00 |
| Raboo | it gets stuck when running trusty, but when i tested xenial it worked.. | 16:01 |
| Raboo | but i would like it to work on trusty again | 16:01 |
| dpb1 | Raboo: through maas I assume? | 16:02 |
| Raboo | dpb1 nope, I use The Foreman | 16:02 |
| Raboo | https://theforeman.org/ | 16:03 |
| dpb1 | Raboo: your images are up to date for Ubuntu? | 16:03 |
| Raboo | yes, i build it daily | 16:03 |
| smoser | Raboo: most likely, trusty is just not going to work easily | 16:03 |
| smoser | it has no dynanmic network configuration | 16:03 |
| Raboo | and i even triggered a manual build today when it failed | 16:03 |
| smoser | meaning that the images have 'dhcp eth0' | 16:04 |
| smoser | and if you dont have a 'eth0' or its not dhcp.. | 16:04 |
| smoser | but maybe thats not your issue. | 16:04 |
| Raboo | the network came up | 16:04 |
| smoser | yeah, then i assume it is that you dont have a eth0 ? | 16:04 |
| Raboo | it printed ip before it generated ssh keys | 16:04 |
| smoser | or eth0 is not plugged in and some other. | 16:04 |
| smoser | oh. | 16:04 |
| Raboo | it's a bond0 actually | 16:04 |
| smoser | hm... ... i'm not really sure where youd' be hung then. | 16:04 |
| Raboo | and i could ping the node | 16:05 |
| Raboo | the most annoying thing is that it got stuck. It didn't fail and drop to a console so I could read logs | 16:05 |
| Raboo | so very hard to debug | 16:05 |
| Raboo | i think it got stuck at the end of "init" phase | 16:07 |
| dpb1 | Raboo: what cloud init version are you using in your trusty images? | 16:12 |
| dpb1 | Raboo: is it the one in the trusty-updates pocket? | 16:13 |
| Raboo | dpb1 hmm let me try to find out, i think it was 17 something | 16:16 |
| Raboo | is there a binary or a config file that contains a version string? | 16:20 |
| Raboo | could be 0.7.5. | 16:21 |
| Raboo | it's the same as in https://cloud-images.ubuntu.com/trusty/current/ | 16:24 |
| Raboo | it contains this file ./usr/lib/python2.7/dist-packages/cloud_init-0.7.5.egg-info | 16:25 |
| Raboo | so i guess it's 0.7.5 | 16:25 |
| Raboo | ./usr/share/doc/cloud-init/changelog.Debian.gz says at the top of the file "cloud-init (0.7.5-0ubuntu1.22) trusty; urgency=medium" | 16:26 |
| dpb1 | Raboo: ok | 16:29 |
| Raboo | this used to work before, don't know if ubuntu has bricked something since I update my images daily | 16:31 |
| smoser | :-( | 16:47 |
| smoser | Raboo: you can open a bug. just please provide as much information as you can. | 17:20 |
| smoser | you can just "backdoor" the image too | 17:20 |
| smoser | then you can go in with the backdoor to get other info | 17:21 |
| smoser | http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image | 17:21 |
| Raboo | smoser what does the backdoor do in short? | 17:32 |
| Raboo | cause i have a local user already | 17:33 |
| Raboo | problem was it doesn't even drop to the console where i can login | 17:36 |
| smoser | you said it set up network | 17:56 |
| smoser | no ? | 17:56 |
| smoser | just ssh in | 17:56 |
| Raboo | smoser i don't think sshd is started | 18:17 |
| Raboo | i'm going to reinstall another node tomorrow, see if i can replicate the issue on another node. | 18:19 |
| dpb1 | Raboo: from the sounds of it, that could be what is stalling, actually (sshd startup) | 18:44 |
| smoser | rharper: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334147 | 19:36 |
| smoser | can you ack that ? | 19:36 |
| rharper | acked | 19:38 |
| blackboxsw | ok I'm out of the way on the two branches (no more changes from me on these): https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115 | 21:17 |
| blackboxsw | just pushed latest changes/rebase and fixes | 21:17 |
| blackboxsw | hitting review queue | 21:17 |
| powersj | smoser: seems you pulled your KVM fix, so I am going to enable the KVM tests in jenkins | 22:21 |
| blackboxsw | smoser: rharper http://pastebin.ubuntu.com/26075320/ shouldn't our v1 instance-data.json actually surface public-hostname instead of local-hostname from metadata? Right now the instance-data.json sources cloud.get_hostname() which grabs local-hostname from metadata if present | 22:30 |
| blackboxsw | checkout the ec2 section of that pastebin | 22:30 |
| rharper | blackboxsw: what does hostname on the instance say now ? | 22:31 |
| rharper | I suspect it retains the .internal hostname | 22:31 |
| blackboxsw | "public-hostname": "ip-10-0-20-27", | 22:31 |
| rharper | an external/public hostname is for facing outward IIUC | 22:31 |
| rharper | no, run 'hostname' on your instance | 22:31 |
| blackboxsw | which is actually grabs from hostname on the cmdline | 22:31 |
| rharper | ie, what actually ended up in the Linux hostname value | 22:31 |
| blackboxsw | ubuntu@ip-10-0-20-27:~$ hostname | 22:31 |
| blackboxsw | ip-10-0-20-27 | 22:31 |
| rharper | ok | 22:32 |
| blackboxsw | but for ec2 instances, you can't really get ahold of that local 'hostname' | 22:32 |
| rharper | then I would think that should match in the data (which is what you're pointing out, yes?) | 22:32 |
| rharper | can you not resolve that now ? | 22:32 |
| rharper | the .internal ? | 22:32 |
| rharper | what resolver is configured on the instance ? | 22:32 |
| blackboxsw | I can from within amazon, but not from home | 22:32 |
| rharper | right | 22:33 |
| rharper | which is fine | 22:33 |
| rharper | the instnace.json is from within the instance | 22:33 |
| rharper | but, I think what's happening is that the hostname public value is not yet when dump the instance data | 22:33 |
| rharper | but it's a valid hostname on the instance | 22:34 |
| rharper | so, there's a little fuzziness w.r.t what that value should reflect | 22:34 |
| rharper | should it be the hostname at the time we process the metadata (which is what I think it is doing now) | 22:34 |
| blackboxsw | right it's a valid hostname on the instance, what would you call the public-hostname which you can reference external to a given machine? | 22:34 |
| rharper | or should it reflect what the hostname value will be eventually (after cc_hostname runs) | 22:34 |
| rharper | public-hostname | 22:34 |
| rharper | I think | 22:34 |
| rharper | we might need other cloud examples to see what makes the most sense | 22:35 |
| blackboxsw | @ the time we process metadata, we could query md['network']['public-hostname'] and it'd give us ec2-18-216-195-116.us-east-2.compute.amazonaws.com | 22:35 |
| rharper | I think I'd be fine with tagging hostnames with public or private | 22:35 |
| rharper | and let instance users pick | 22:36 |
| rharper | and skip having a instance_json['hostname'] value | 22:36 |
| blackboxsw | yeah +1 | 22:36 |
| rharper | or have it include a dictionary of public/private values | 22:36 |
| blackboxsw | I have ['v1'][ | 22:36 |
| rharper | right, I see that | 22:36 |
| blackboxsw | I have ['v1']['public-hostname'] which I think is a misnomer (as it's actually the local/private hostname) | 22:36 |
| blackboxsw | so I'll surface public/private hostname keys which will allow us to differentiate | 22:37 |
| rharper | "public-hostname": "ip-10-0-20-27", | 22:37 |
| rharper | yes, that seems wrong | 22:37 |
| rharper | that's the .internal one | 22:37 |
| blackboxsw | yeah definitely feeling like that | 22:37 |
| rharper | but you're saying it has a hostname that matches | 22:37 |
| blackboxsw | even though that's what our cloud.get_hostname() logic defaults to | 22:37 |
| blackboxsw | I even believe cloud.get_hostname() should probably default to the public hostname where available instead of private | 22:38 |
| rharper | I wouldn't change what that does | 22:38 |
| blackboxsw | but, it currently doesn't (and that behavior is surfaced in multiple cc modules already. | 22:38 |
| blackboxsw | yeah | 22:38 |
| rharper | but in our rendering of the data into json, we could pick something else (or call it differently) | 22:38 |
| blackboxsw | ok will tweak that. as public/private is more explicit | 22:38 |
| rharper | depending on the scripts folks may want the local/private name, for configuring services, etc | 22:39 |
| rharper | but if they're injecting values to configure a service to point to another host, those would be the public ones | 22:39 |
| blackboxsw | right | 22:39 |
| rharper | so I think we need both values | 22:39 |
| blackboxsw | +1 | 22:39 |
| blackboxsw | vote 'private-hostname' vs 'local-hostname'? | 22:39 |
| rharper | hrm; I dunno | 22:52 |
| rharper | I kinda like public/private symmetry | 22:53 |
| rharper | but I honestly have no strong feelings | 22:53 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!