=== jdm_ is now known as jdm === sambetts|afk is now known as sambetts === shardy is now known as shardy_lunch === shardy_lunch is now known as shardy [12:30] hello, I'm trying to write a connector for Scaleway (https://www.scaleway.com/) [12:30] By running `cloud-init --file ~/cloudinit.yaml init` with "datasource_list: [Scaleway]" in ~/cloudinit.yaml, I have an exception I don't understand: [12:30] https://pastebin.com/98PwbVBR [12:31] any chance someone can help me understand what is going on? [12:52] rharper, you do not need python-argparse on centos7 [12:52] python-argparse is really just a 2.6 thing. as its builtin to 2.7+ [12:52] $ python3 -c 'import argparse; print(argparse)' [12:52] [12:53] $ python2 -c 'import argparse; print(argparse)' [12:53] [12:53] i'm not saying that the rpm isn't broken, but the answer is to probably not depend on it. not sure how really to do that in rpm an dsuch. === rangerpbzzzz is now known as rangerpb [14:16] niluje, hm.. not sure what woudl have ahppened there. [14:17] i suspect you must have a network_config thing that is returning invalid data [14:17] smoser: I know we don't need it; but somehow it's in the rquires.txt file which python checks when loading [14:18] rharper, that gets used for the rpm i guess ? [14:18] smoser: harlowja: running with this which works but not sure that's the right thing to do: http://paste.ubuntu.com/24466885/ [14:19] smoser: it's used by python setuptools or whatever, entry point thingy that reads egginfo [14:19] about my issue, here is the code of the connector: https://github.com/brmzkw/cloud-init-scaleway/commit/110bb3e02608cb29ec67fe180cac4a48e3bc548b and my configuration + the errors I have: https://pastebin.com/qjBW8TGg [14:19] https://github.com/certbot/certbot/commit/8f101034960ffc1e47879314585898efda234e60 [14:20] this is sorta the issue as well; [14:20] niluje, can you get a cloud-init.log ? [14:20] IIUC, special case python2.6 to add that dep (in packaging) and from > 2.6 it's standard library, so shouldn't be in requirements (or at least not show up in egginfo) ? [14:20] smoser: where is it stored? [14:20] /var/lib/cloud? [14:20] i suspect that you've gone down the path of reading command line parameters that cloud-init has... and that it went wrong. [14:20] /var/log/cloud-init.log [14:21] if it sees ip= on the command line, it takes *that* as the fallback networking, and then tries to read files that ubuntu's initramfs tools writes [14:21] so, on ubuntu, we package the requires.txt file, but it's empty [14:22] smoser: cloud-init.log is empty [14:22] ubuntu ? [14:24] yes, xenial [14:24] I git-clone'd the cloud-init repository and pip install -e . [14:26] hm.. [14:27] i suspect that there is no /etc/cloud/cloud.cfg.d/loggin* ? [14:27] (i'd install from package and just make your updates that way) [14:27] or build a package from trunk and install it. [14:27] ./tools/bddeb [14:27] installing with pip is not something i've really ever tested. [14:27] ewww [14:27] ok [14:27] let me try installing the package with apt then [14:28] you'll still fail for sure [14:28] but hopefully get a log :) [14:32] how to you work on cloud-init then? [14:32] apt-get install cloud-init, then git clone, pip install -e ., $> cloud-init init? [14:35] I just apt-get install'ed the package, updated /etc/cloud/cloud.cfg to add "datasource_list: [Scaleway]" and call "cloud-init init" but cloud-init is trying to contact the EC2 APIs :/ [14:40] niluje, generally i install the package. taht is what a distro is going to do anyway. [14:40] hm.. [14:40] and then? [14:41] how do you edit the code? [14:41] when hacking i just change files in /usr/share/// [14:41] and re-try and such [14:41] then i take those chagnes and re-build / dpkg -i [14:42] yeah, i know its not the best. but dpkg -i isn't actually *all* that different from pip install. [14:42] hm [14:42] how do you build the package? [14:42] ./tools/bddeb [14:43] soryr [14:43] ./packages/bddeb [15:37] https://bugzilla.redhat.com/show_bug.cgi?id=1194451 [15:37] bugzilla.redhat.com bug 1194451 in cloud-init "[PATCH] Add dnf support to cloud-init" [Low,Closed: rawhide] [16:28] niluje, so... what was happenging for you was that [16:28] a.) you boot with 'ip=' on the kernel cmdline (which makes good sense) [16:28] b.) cloud-init recognizes that as saying taht the initramfs configured networking and that it should read its networkign from that. [16:29] rather than generating "fallback" config, which is "dhcp on the first nic you think looks good" [16:29] because obviously bouncing the nic would be bad on a network moutned root device [16:30] the problem is that your initramfs does not generate /run/net-.cfg files that cloud-init expected to read some information from. [16:30] and thus it ends up generating a completely empty config that fails [16:30] ok [16:30] but should our initramfs generate the /run/net-.cfg file? [16:31] well, mkinitramfs-tools does (ubuntu/debian) [16:31] but there is no official format of that, its just kidn of happensatnce [16:31] the other option would be to have cloud-init realize there are no net-* and do some other sort of searching to figure out what the network config is. [16:32] ok so what you did (I haven't checked yet) is remove the read from /run/net-xxx.cfg and it works, right? [16:32] no. those files do *not* exist for you [16:32] and cloud-init is expecting them [16:32] I know [16:32] I understand that [16:32] oh. right. yeah. [16:32] http://paste.ubuntu.com/24467477/ [16:32] i just made it not pay any attention to the kernel cmdline [16:33] but that probably fails eslwehere [16:33] ok [16:33] as cloud-int will write a /etc/network/interfaces that says it should dhcp on eth0 [16:33] ok [16:33] which best case fails on 'ifup' (because that is already up) [16:33] and worst case brings down the link that you have to the root device [16:34] so what I need to do is 1/ first, create manually the file un /run, 2/ make cloud-init work 3/ in //, update our initrd to create the files in /run/net-xxx.cfg [16:34] and everything should work [16:34] correct? [16:34] if your initramfs writes those files, it should work yeah. [16:34] ok [16:34] theres no standard way to do this. [16:35] I don't even know what is the format of those files :p will check [16:36] thanks a lot for your help smoser [16:43] niluje, see cloudinit/net/__init__.py _klibc_to_config_entry [16:45] blackboxsw, http://paste.ubuntu.com/24467573/ [16:45] thats what i'll just upload with [16:46] smoser, ok so that's our bug list for the SRU? [16:47] yeah [16:47] hm.. and maybe i'd rip out the yum one [16:47] (not reference it) [16:48] yeah, i think i iwill [16:53] there's no reference to _klibc_to_config_entry in cloudinit/net/__init__.py [16:53] k in cloudinit/net/cmdline.py [16:55] sorry [16:55] yeah [16:55] niluje, and then in the tests therea re examples [16:58] smoser: this is the function reading /run/net-xx.cfg and convert it to a cloud-init config object, right? [16:59] sorry for the stupid questions I'm not familiar with cloud-init so I struggle a bit to understand what needs to be done [17:00] ok yes, that's it [17:00] I will look into it tomorrow [17:00] thanks a lot again for your help smoser, I'll post a message here when the connector will be working [17:02] _klibc_to_config_entry is what converts [17:02] and the test functions show examples of the net-* [17:08] smoser, so SRU info, last time you created https://public.etherpad-mozilla.org/p/cloud-init-sru-info will we do the same type of thing for this sru? [17:09] blackboxsw, i think its useful, yeah. [17:09] https://docs.fedoraproject.org/en-US/Fedora/8/html/SELinux_FAQ/index.html#id503239 ; that's troublesome; ifconfig output is thrown away due to selinux, you can get it by reading it from the pipe (instead of the terminal) and writing to file (ifconfig -a | cat >out) [17:09] blackboxsw, https://public.etherpad-mozilla.org/p/cloud-init-sru-info [17:11] ... clunky rharper [17:11] yeah, I'm more worried about cloudinit.util.subp [17:11] not sure that's working as expected [17:11] * rharper is testing that now [17:13] rharper, see, i told you enforcing=off [17:13] :) [17:13] lol [17:13] not the *right* answer [17:13] but yes [17:13] you were right [17:14] if everyone in the world says that 2+2 = 5, then that is 'right' for some definition of right ;) [17:14] i wonder [17:15] rharper, if we close the standard input [17:15] playing with subp now [17:15] hm.. [17:15] I think it's already closed though [17:15] well, standard output is probably indirectly attached to a temrinal [17:16] i don tknow [17:16] serial console [17:16] well, /dev/console [17:16] but yeah; this gets into the magic of tty [17:16] and other things [17:17] we should be ablet to just close enough file handles and such though so that , before that subp (or all with 'capture') no open file handles are a termianl [17:17] id' think that would do it [17:18] so, subp defauts to data=None (stdin is fd of /dev/null) and capture=True, sets up a pipe [17:18] I just need to confirm we get output; there's a separate netstat -rn returning 1; I don't yet know why [17:19] https://bugs.launchpad.net/ubuntu/+source/net-tools/+bug/1251563 [17:19] Ubuntu bug 1251563 in net-tools (Ubuntu) "netstat command returns nozero even if successively executing" [High,Fix released] [17:19] well, tha;s just strange [17:19] bbiab [19:19] https://paste.ubuntu.com/24468378/ [19:19] ^ unit test failures on centos 6 + 7 [19:19] I'll go play with https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/323265 shortly [19:24] powersj, great. === sambetts is now known as sambetts|afk [20:22] https://bugzilla.redhat.com/show_bug.cgi?id=1406520 [20:22] bugzilla.redhat.com bug 1406520 in libselinux "calling libselinux python restorecon fails on /var/lib/nfs/rpc_pipefs" [High,Verified] [20:22] it appears that's hitting in centos7 image [20:22] * rharper is confirming [20:26] bummer, I don't see 2.5-7 released yet === rangerpb is now known as rangerpbzzzz [20:37] https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/323351 [20:37] All I got rid of was the use of the array for the patches and got rid of one that was useless [20:38] oh and cleaned up pylint errors [20:38] rharper: if you could help me better understand what else you would want changed to smoser original proposal that would be good. You mentioned moving the asserts [20:38] powersj: yeah, lemme get a pointer [20:42] powersj: so, each of the patch_XXX should be @mock.patch.object(util, 'write_file') on the helper function; then pass that in as a mock_xxx to the _apt_source_list method; the decorator does the start/stop automatically for you; then in them ethod, the mocks that have return values or side-effects, you just need to apply those based on the input, [20:43] oh you wanted decorators [20:43] powersj: I think all of them are constant values, except the mock_shouldcfg, in which case you just mockshouldcfg.return_value = (cfg_on_empty, "this is for test") in the method [20:43] first [20:44] and then, I really want the caller who knows if they sent "debian", "ubuntu" to run the asserts; [20:45] so end that function after running cc_apt_configure.handle [20:45] so, test_apt_v3_source_list_debian knows what config it sent and which distro is being used [20:45] knowing those things, it can test that it expects for should_config to return true [20:46] the other variable is "is_system_snappy" which I think I'd like to assume its no , and then have a single test_apt_v3_source_list_snappy; where we return True on that and then can confirm that apt_configure wasn't run [20:47] that'll break up much of the if block of asserts based on the config_on_empty list [20:48] the test_ method calling the helper should construct the path and test it explicitly; if there is something that we know *always* gets called then I think the helper could assert it; but I'd rather the caller do the asserts since it's setting up the test scenario [20:48] if that makes sense [20:48] rharper: ok I think I follow all that. I'll give it a shot and come back. [20:48] it does [20:49] cool [20:49] would you like me to paste this log into the MR comment ? [20:49] sure :) [20:49] k [21:02] smoser: http://paste.ubuntu.com/24468995/ ; looks like we get to have "fun" with centos7 network/cloud-init.local ordering in systemd [21:04] or I need to figure out how in brpm to tell it we require systemd; noticed that difference in the 0.7.5 vs. a local build [21:10] * rharper learns of -b systemd in brpm [21:10] now for more fun [21:21] ha, systemd units worked!