[00:13] bdx: if you can pastebin /var/log/cloud-init.log we can see what's going on; [03:19] rharper: https://bugs.launchpad.net/cloud-init/+bug/1729715 [03:19] Ubuntu bug 1729715 in cloud-init "nocloud datasource network-config not working" [Undecided,New] [14:00] rharper: in netwokr config v1, i think that we can't rename an interface without knowing the mac. is that right ? [14:09] smoser: correct [16:49] https://git.launchpad.net/~smoser/cloud-init/commit/?h=feature/json-socket-server [16:49] thats my json server [16:49] see that commit message for how to use. [16:59] Can you write templates in write_files? [17:13] not sure what you mean Sargun, you can write any file content you want. if you wrote templates in write_files I'd guess you'd be processing templates in your runcmd config. [17:18] the files written by the write_files is a list with a single file declared per list item. [17:53] @rharper, @smoser does static interface network-config for nocloud work for you guys? [18:07] bdx: working; I replied, you need network-config in separate file, and drop the top 'network' key [18:10] rharper: that is awesome! [18:10] I just got it to work [18:10] ]o/ [18:10] many thanks [18:10] what happened to my arm!? [18:10] \o/ [18:10] there we go [18:10] :) [18:10] np, sorry it wasn't clear from the docs [18:11] its cool, its pretty clear, there are just a few gotchas [18:11] like you *must* use the network-config file [18:11] which I did [18:11] geh [18:11] I've filed https://bugs.launchpad.net/cloud-init/+bug/1708255 [18:12] Ubuntu bug 1708255 in cloud-init "empty or invalid network config dictionaries are not handled well" [Undecided,New] [18:12] * bdx beats his own head [18:12] which should have made it obvious that it got a config but was unhappy with it [18:12] now just to allocate time toward that one [18:12] ha - right [18:12] thanks all the same [18:13] np [18:13] I may repurpose your bug into updating the docs around NoCloud's use of network-config since that part wasn't clear [18:14] that would be great [18:15] blackboxsw: I have a file I want to write in write_files, but I want to substitute instance_id into the config files in some places. Is there an easy way to do this? [18:40] Sargun: no. :-( [18:40] but https://trello.com/c/AYaCdQyT [18:40] that is the trello card that i descried what i'd want. [18:41] powersj: what am i doing wrong [18:43] Sargun: i'd be interested in your thoughts on that. [18:43] smoser: not sure yet, I was just finishing lunch let me dig in [18:49] smoser: hmm now it is working, but super slow... 20 seconds for each pull. [18:50] https://paste.ubuntu.com/25881348/ [18:51] are you seeing similar results? [18:52] for each pull ? [18:52] what is pull [18:52] sorry each cmd run [18:54] no.. [18:54] 2017-11-03 18:53:28,938 - tests.cloud_tests - DEBUG - running collect script: result.json [18:54] 2017-11-03 18:53:28,938 - tests.cloud_tests - DEBUG - Executing "collect: result.json" [18:54] 2017-11-03 18:53:29,321 - tests.cloud_tests - DEBUG - running collect script: status.json [18:54] 2017-11-03 18:53:29,322 - tests.cloud_tests - DEBUG - Executing "collect: status.json" [18:56] hmm [18:56] powersj: i have a thoguht that it could be sudo and dns related possibly. [18:57] or it could be paramiko related. [18:58] maybe the wway we are sending stdin now is not rigiht [18:58] (we were not sending before) [19:07] powersj: can you try [19:08] http://paste.ubuntu.com/25881453/ [19:08] that re-uses the connected ssh client [19:35] smoser: powersj I'm seeing successes on nocloud-kvm (artful) [19:35] will pull in your paste too [19:37] blackboxsw: Are you seeing timings like smoser, which are quick? [19:39] smoser: still getting 20-second delays between each collect... that definitely feels like something is timing out somewhere along the path. [19:39] I'll pastebin shortly [19:40] lemme, check timestamps on that powersj [19:41] https://paste.ubuntu.com/25881623/ [19:41] meh powersj yeah 20 second delays between collect calls [19:41] http://pastebin.ubuntu.com/25881628/ [19:43] * powersj goes and find's smoser's venv cmd [19:47] powersj: ./tools/tox-venv [19:47] blackboxsw: where are you running that ? [19:48] * blackboxsw hasn't tried smoser I'm running that on artful desktop at home. [19:48] oh. i thought you said you had. [19:48] tox -e citest -- run --verbose --os-name xenial --deb cloud-init_17.1-46-g8a49e051-1~bddeb_all.deb --data-dir ~/collection --preserve-data --platform=nocloud-kvm --test=tests/cloud_tests/testcases/modules/final_message.py [19:51] smoser: still having issues: https://paste.ubuntu.com/25881674/ [19:53] smoser: sorry about the last couple comments, I had tried running artful with the following commandline I see various successes there, but I'm seeing the 20 second gap between each collect [19:54] and that 20 second gap persists even across your re-use ssh patch [19:55] could one of you letl me in somewhere ? [19:55] so i can run. i'm kind of out of ideas. [19:56] smoser: yeah adding you [19:57] gotta open up my router one min [20:00] blackboxsw: hm.. ok . i ran here and i see it. (locally on artful desktoip) rather than on diglett. [20:00] smoser: ok. I've got creds for you if needed [20:06] Running on torkoal seems fine [20:06] it hangs on [20:06] rc = channel.recv_exit_status() [20:14] should we be checking exit_status_ready ? [20:17] not going to help [20:17] I looped in a pdb session [20:17] http://pastebin.ubuntu.com/25881811/ [20:17] it also is false for about 20 seconds [20:17] then true [20:17] which presumably is when the exit status becomes available to paramiko [20:22] blackboxsw: yeah, iwas looking at that too. [20:22] the examples of paramiko and sending data are not a lot. [20:22] just tried here doing a channel.shutdown(2) [20:22] http://docs.paramiko.org/en/2.3/api/channel.html [20:22] rather than shutdown_write() [20:22] and taht didnt help [20:42] blackboxsw: dropping remvoes the hang. [20:43] self.ssh(["sudo", "cat", "/etc/cloud/build.info"]) [20:43] ^^ slow ---- vv faster [20:43] self.ssh(["cat", "/etc/cloud/build.info"]) [20:45] ok. [20:45] well... not paramiko fault [20:45] this hangs similarly [20:45] ssh ubuntu@localhost -p 37425 -i results.nocloud-kvm.d/id_rsa "cat /etc/passwd" [20:46] i really, really hate that feature in sudo [20:51] geez right, it's sudo trying to hit resolve or something for the hostname? [20:51] yeah [20:51] you need the hostname in /etc/hosts or sudo barfs; [20:52] so we could set the hostname in /etc/hosts [20:52] yeah [20:52] or, set NOPASSWD in sudoers [20:52] ? [20:52] we do have nopassword for that user [20:52] ah, right, I guess it still does the lookup [20:53] looks like ALL would work [20:53] ok in ip tools world (no netstat) how do I check what open ports I'm listening too [20:55] I'm so used to netstat -ln [20:55] its such a non-feature [20:55] https://superuser.com/questions/429790/sudo-command-trying-to-search-for-hostname [20:55] that describes what it is and what is doing it. [20:55] hey, you want to share your sudoers file across your entire company [20:55] it needs to know which host it's on [20:56] ss -l [20:57] im' not sure why, but [20:57] 'time host ubuntu' [20:57] that returns in 1 second on my system (where this is slow) [20:57] and 0.014 seconds on diglett [20:57] dns search order? [20:57] which i think is basically the source of why it works fast on diglett [20:58] what does host -v show ? [20:58] that walks the dns search list here [20:59] with vpn, it rns through all of those domains in the list [20:59] all those domains have to say no before your caching nameserver can give up and let /etc/hosts respond IIUC [20:59] wow, localhost even has to do that with the "host" command [21:01] so on artful I'm talking to systemd/resolved, what's diglett running again? [21:01] artful [21:01] hmm ok [21:01] diglet is bionic [21:01] oh, when did we upgrade ? [21:02] https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1728616 [21:02] Ubuntu bug 1728616 in apt (Ubuntu) "using 'devel' in sources.list causes apt-get update to fail" [Undecided,Opinion] [21:02] looks like i did on 2017-10-30 [21:02] :) [21:02] heh [21:03] we really should fix sudo [21:03] to not do a dns lookup if it doesnt need to [21:13] the claim is that it wants to, ifyou set the section to (ALL) for the hostname it won't do that [21:13] it doesnt claim that. [21:13] https://bugs.launchpad.net/ubuntu/+source/sudo/+bug/32906 [21:13] Ubuntu bug 32906 in sudo (Ubuntu Hardy) "sudo fails if it cannot resolve the local hostname and no MTA is installed" [High,Fix released] [21:13] well, it doesnt fail [21:14] …yeah, sudo, it’s all very clever until someone loses an eye! [21:14] i like that :) [21:14] now, it appears are are using (ALL); so now I'm confused [21:16] https://www.sudo.ws/repos/sudo/rev/1e10105ade5b [21:16] fun stuff [21:16] sudo + PAM [21:18] well, acouple of options here. [21:18] a.) cloud-init can be set to write the hostname to /etc/hosts [21:18] b.) we can just eat this once when we first run a command [21:18] but even then, 20 seconds is suck [21:19] c.) maybe this is sufficient, for nocloud we can just modify it pre-boot [21:20] also in these scenario.. [21:20] we can be trickly. [21:20] for nocloud-kvm, if i add an 'ubuntu' to the *hosts* /etc/hosts [21:20] then qemu dns -> host resolv -> that entry [21:20] and magic fast [21:21] 127.0.0.1 localhost ubuntu [21:22] yeah I think I don't want to eat that 20 second cost even on setup [21:22] that makes every test that much more costly [21:23] with that in place, on diglett [21:23] I like being trickly [21:23] http://paste.ubuntu.com/25882159/ [21:23] looks much better [21:23] ~ 0.06 [21:23] instead of 0.014 right? [21:24] oh. no. instead of something like a half a second it hink [21:24] Ohh I was thinking you were talking time host ubuntu ... and 0.014 seconds on diglett [21:25] I'm +1 for writing ubuntu into hosts [21:26] oh, heh, I forgot we had something like this for m-i-c [21:26] yeah, I think that's warranted given we know sudo's shortfall w.r.t. hostname lookup