[06:16] smoser: CentOS 7.4 & cloud-init 0.7.9 didn't set the gateway when used earlier user-data and meta-data files with NoCloud. Managed to figure out the network-config syntax and now all the network configurations are set correctly. === shardy is now known as shardy_afk [10:04] Dear experts, I've a problem with Cloud-init or at least I think so [10:04] can you please help me? === shardy_afk is now known as shardy [14:17] robjo, around ? [14:20] responded to both your mps [14:32] on the phone, will check after, thanks [14:54] k [16:09] smoser: https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331142 should hopefully meet the needs [16:30] robjo, are you ok if i change the indent on 'Copyright' ? [16:31] sure [16:35] robjo, ok. and is it ok if i just drop the copyright ? [16:35] i think they're 99% largely harlowja work anyway. [16:36] Yeah, I guess, this is going away in 2019 anyway when SLE 11 dies, I can't wait..... [16:37] http://paste.ubuntu.com/25587157/ [16:39] thats the diff from redhat/* to suse/* [16:41] yup, just off enough to be annoying, I know [16:42] i think then we shoudl just do a straight copy and then make the changes that matter [16:42] is that ok ? [16:42] we could do some integration or templating, but sysvinit in suse and redhat are both really EOL [16:42] so that doesn't make much sense. [16:43] hey guys, I have a Libvirt VMs that are using cloud-init provided by the 'config drive'. Is this possible using cloud init to setup the eth0 interface when booting the instance? I use in my network only static addressing and I dont want to add DHCP server [16:43] Yes copy and then changing the bits where it matters also works for me [16:44] So instance starts with no configured interfaces -> cloud init configures statically the eth0 [16:48] robjo, http://paste.ubuntu.com/25587206/ [16:48] thats what i would do on top of your branch [16:49] which leaves me with [16:49] http://paste.ubuntu.com/25587211/ [16:49] as (redhat -> suse differences) [16:49] robjo, i'm sorry that i didnt realize at the start that you had copied the redhat/ scripts [16:49] i assumed they'd come from your packaging or something [16:49] sorry [16:49] kszarlej, not statically [16:50] oh. sorry. wait. [16:50] kszarlej, "config drive" [16:50] ye [16:50] is that meaning "OpenStack config drive" ? [16:50] or meaning "configuration disk" more generically [16:50] I am creating .iso with 'user-data' and 'meta-data' [16:50] right. NoCloud then. [16:50] and I mount it to VM as 'cdrom' device [16:51] and the VM recognizes it just fine [16:51] 'config drive' is a term that means something specifically to openstack. [16:51] sorry. i was just trying to clarify what you were asking [16:51] smoser: http://paste.ubuntu.com/25587206/ looks fine, do not recall the origins os the sysvinit stuff it's been too long, been carrying those patches in cloud-init package forever [16:51] I have problem setting up the dnsmasq in my network infrastructure that would be leasing DHCPs to VMs.. and I am thinking if it would be possible to [16:51] ok. then robjo i'll pull those in with my changes. [16:51] configure the interfaces using cloud-init instead of DHCP [16:52] thanks [16:52] kszarlej, so on your disk with 'user-data' and 'meta-data'. [16:52] you need to [16:52] also replied on the other proposal [16:52] a.) set the label to be 'cidata' [16:52] b.) provide 'network-config' file in addition to user-data and meta-data [16:53] the network-config is described http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html#network-config-v1 [16:53] so like that https://gist.github.com/Informatic/0b6b24374b54d09c77b9d25595cdbd47 [16:53] ?? [16:53] or http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html#network-config-v2 [16:53] ok smoser [16:53] thanks!! [16:53] I was looking for "cloud init config drive network configuration" [16:54] you do not need dsmode: local now [16:54] i'm 99% sure [16:54] and tried the network-interfaces: in meta-data [16:54] but I am failed :) [16:54] will try the network-config :) [16:54] network-interfaces is legacy [16:54] (as described in that do up there) [16:54] thanks! [16:54] I have a case of vanishing tempdir... Any idea what might cause this? http://paste.ubuntu.com/25587235/ [16:55] kszarlej, if you're able, its best to provide the mac address of the nic [16:55] that way cluod-init can [16:55] a.) rename the nic to whatever you want [16:55] Is systemd mounting tmpfs on /tmp while cloud-init is in the middle of something? [16:55] b.) actually identify it, rather than you guessing the name that the kernel will give it. [16:55] paulmey, yes [16:56] sort of [16:56] paulmey, https://bugs.launchpad.net/cloud-init/+bug/1707222 [16:56] Ubuntu bug 1707222 in cloud-init "usage of /tmp during boot is not safe due to systemd-tmpfiles-clean" [High,Confirmed] [16:56] not "mounting", but deleting [16:57] robjo, i'm going to pull the sysvinit changes for you [16:58] hmmm... nasty [17:02] smoser: thanks. That will be in the upcoming release, I assume? [17:02] yeah, i just marked it fix-committed. [17:02] awesome thanks [17:02] note, it had fallout though [17:02] we changed to using /run/cloud-init/tmp [17:02] so that systemd wouldnt think it should just delete things for us [17:03] and that broke because /run is (typcially) mounted with 'noexec' [17:03] and at times we need exec [17:03] fun [17:03] sigh... sorry for your pain... [17:03] i find the idea of deleting files under almost any criteria by a central daemon questionable. [17:04] but getting into earlier boot races is a good condition to be in... ;-) [17:04] ie, there is stuff that will delete your files in /tmp if they're 30 days old [17:04] smoser: thanks [17:04] well... sometimes i put stuff there :) [17:04] i dont plan ahead of time how long i'll want to keep something for [17:04] yeah... systemd is getting power hungry... [17:09] smoser wut [17:09] lol [17:09] harlowja, cloud-init misses you [17:09] ya [17:09] i miss her to [17:09] lol [17:11] * dpb1 hands smoser /home [17:12] dpb1, i have ~/t/ for tmp files in /home that i think i might need later. [17:12] but sometimes i just dont know. [17:13] smoser: I have a ~/tmp/ [17:13] for same [17:13] * smoser is 66% more effective than dpb1, but has to think about typing 'affective' or 'effective' [17:14] hahaha @ affective [17:14] (googled that for kids last week. RAVEN: Remember Affect Verb Effect Noun) [17:17] rharper, what do you think... [17:17] https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331140 [17:17] robjo's merge above [17:17] we can drop the duplicated 'Default user' section (line 31-44) with this change instead [17:18] http://paste.ubuntu.com/25587352/ [17:18] ie, the only change that robjo put in there was different default groups [17:30] blackboxsw, what do you think ? [17:30] was looking it over [17:33] basically i'm asking about whether to "nested loop" or "copy block" [17:33] er... [17:33] "nested if/else" [17:34] smoser: it looks like it's the same content as centos/rhel/fedora too [17:34] ? [17:34] my diff should make it clear what is different [17:34] http://paste.ubuntu.com/25587416/ [17:34] ahh didn't know your were talking about a pastebin. I was on robjo's branch [17:34] yeah looking now [17:35] my pastebin there is on top of robjo's branch. (and *this* pastebin doesn't use undefined variables) [17:35] which is a nice feature i think (the last one spelled variant wrong) [17:35] +1 on that pastebin as I was thinking kinda the same, we didn't want to copy a separate block for suse as it's nearly the same as rhel etc [17:36] yeah I like that fix [17:36] s/fix/patch [17:36] gotta go for kid pickup [17:36] ok. then i am going to pull that also. [17:36] with my change there. [17:40] robjo, are you there ? [17:40] i do have some questions though on that mp. [17:40] smoser: on the phone again [17:42] welll, please when you get done take a look at my comments. [17:47] Does multiple interface configuration work for anyone using the OpenStack datasource on Xenial using 0.7.9? [17:47] i'd expect it to work. [17:47] ah. [17:47] with config drive [17:47] it works with config drive but not the OpenStack datasource. [17:48] yeah [17:48] er.. yes. that is what i would expect. [17:48] why is that? A bug? I've dug through the code and have some ideas. [17:48] cloud-init successfully reads the network json: 2017-09-21 13:07:58,001 - url_helper.py[DEBUG]: Read from http://169.254.169.254/openstack/latest/network_data.json (200, 626b) after 1 attempts [17:48] smoser: I did respond on the inline comments, hmm not sure what I screwed up that they didn't arrive on your end [17:49] basic.target vs. sysinit.target [17:49] but then says this: stages.py[INFO]: Applying network configuration from fallback bringup=True [17:49] likely because the OpenStack datasource doesn't have the network_config() property in 0.7.9 and doesn't override it from the base class at master in github either [17:49] https://git.launchpad.net/cloud-init/tree/cloudinit/stages.py#n623 [17:49] robjo, maybe you didn't hit 'submit [17:50] r... 'Save comment' [17:50] i dont see them. i'm sorry [17:50] probably :( [17:50] I bet I cannot get back there now, darn..... [17:51] smatzek, right now, only "local" datasources can provide network data. [17:51] this is because the point at which cloud-init.service (non-local) runs is too late in boot to declaritively set up networking. [17:52] ah, I suppose because you don't want to depend on the networking to come up to allow metadata service gets to then configure networking.... [17:52] and we don't currently have any configuration of hotplugged networking [17:52] thanks! [17:52] smoser: do you see the comments now? [17:52] They were not lost that's nice [17:53] the real problem woudl be that at cloud-init.service point in boot osmehting could already be using the network (and rightfully so, as network.target is reached) [17:53] so taking it down to fix it would be difficult and possibly break things [17:53] so it is as it is right now, smatzek but we do have intent on fixing feature to address it [17:53] Well, that's not intuitive, I comment in-line but hen have to click "Save-Comment" for an empty text field.... [17:53] something learned [17:53] robjo, yeah, launchpad has saved me there before :) [17:54] smoser: +1 on the smaller delta to the config template for suse [18:06] robjo, ok. if you're ok with my suggested changes, then i'll make some small comments in the template based on your responses and pull it [18:06] my only concern is that you're never going to figure out why basic.target v sysinit.target so i'm accepting this technical debt when what i should tell you to do is properly justify it [18:06] smoser: sounds good thanks [18:08] I will put a card on my trello board to re-visit this topic before the end of the year [18:10] back [18:12] robjo, http://paste.ubuntu.com/25587623/ [18:12] that is my total diff against your current branch (73acbf4a00741f8) [18:12] seem sane ? [18:14] smoser: yes, thanks [18:14] my heart goes pitter-patter when I hear others use trello. /me ♥'s agile w/ trello. [18:14] ... yeah I don't get out much [18:16] lol [18:18] but look at the mastery of unicode usage. 💕 [18:19] heh [18:19] the world was much simpler without .decode() ;) [18:23] f = '💕'' [18:23] File "", line 1 [18:23] f = '💕'' [18:23] SyntaxError: EOL while scanning string literal [18:23] so masterful [18:34] blackboxsw, rharper robjo i'm looking at [18:34] https://cloud.google.com/compute/docs/storing-retrieving-metadata#querying [18:34] and then planning on making sure that goes thorugh an integration run and then change versions and tag [18:35] smoser: did you mean a code link not gce metadata ? [18:35] yes, puzzled... [18:36] smoser is having cut-n-paste fun after switching main browsers [18:36] but did I tell you about gcemetadata ;) https://github.com/SUSE/Enceladus/tree/master/gcemetadata [18:37] I recall from the summit, thanks for the link! [18:38] i meant [18:38] https://code.launchpad.net/~arnd-arndnet/cloud-init/+git/cloud-init/+merge/331148 [18:38] btw /run/cloud-init/instance-data.json is queued for review an will land post this release [18:39] * blackboxsw wants to get a branch up for DataSource.crawl_metadata ++ cloud-init devel crawl-metadata [18:43] powersj, [18:43] yo [18:43] smoser: sounds like a plan [18:43] once i push above, i'd like to push a branch candidate/17.1 [18:44] then i'd like to run through integration tests on that [18:44] and also as much other stuff as possible [18:44] then just pull that. [18:44] ok [18:44] what all do you recommend we run and how can we do that ? [18:44] honestly we should be doing all the tests we say we will run for an SRU exception: https://wiki.ubuntu.com/CloudinitUpdates [18:45] so touching each of the data sources [18:45] for integration tests, run through all tests on LTS + devel via lxd [18:45] I could spin up a couple instances on azure, gce and aws to test too [18:45] hm.. [18:45] `python3 -m tests.cloud_tests run --verbose --platform --os-name xenial --os-name artful` should do the lxd side [18:46] woops forogt the lxd part [18:46] python3 -m tests.cloud_tests run --verbose --platform lxd --os-name xenial --os-name artful` [18:46] powersj, well, i'd like for jenkins to run that for me [18:46] then there are some thing sthat ideally we'd test that dont (currently) take a branch [18:47] you are looking for a release testing job that runs that based on a given branch? [18:47] ie, testing xenial is actually much more valid to test the daily ppa [18:47] which builds from trunk [18:47] powersj, so yes, sort of :) [18:47] what you said [18:47] this doesnt have to be perfect now [18:48] You know our current daily integration tests build for each distro and then run using the built deb right? [18:48] well, do they ? [18:48] yes [18:48] https://github.com/canonical-server/jenkins-jobs/blob/master/cloud-init/integration.yaml [18:48] do they use the packaging branch ? [18:48] no they use trunk [18:49] I sbuild each [18:49] I do not have direct testing on your release branches, only master [18:49] sounds like a test gap [18:49] right. [18:49] * powersj adds a card for release branch testing [18:50] it'd be best for us to test those daily ppas or some other way build trunk + ubuntu packaging [18:50] and test the result. [18:50] I think I am testing the latter option, right? [18:50] Our image build system has been terribly slow the last couple of days. I had hoped to have some rudimentary test results on SLES in EC2 by now but well, no images, no testing..... [18:51] however rather than pulling master and pushing more on the long queue I'll just wait until this round goes through that should be a reasonably close approximation [18:56] powersj, i suspect you're not testing ubuntu packaging [18:56] i think its just using packages/bddeb [18:56] which builds trunk's packaging [18:56] can you define what you mean by that... to mean ubuntu packaging == debian directory [18:56] which is missing some patches [18:57] yeah, the debian/ directory in the ubuntu/xenial branch [18:57] so I'm not testing the specific release patch sets [18:57] right. [18:57] ok table that for next week? [18:57] from a "we are upstream" perspective, its arguable that that is fine. [18:57] yeah [18:58] more immediately you want a jenkins job to run lxd backed integration tests that i proposed above? [18:58] powersj, yes [18:58] ok [18:58] but it will just do that [18:58] right ? [18:58] if i push a branch for review [18:59] The review ill run 2-3 integration test cases on xenial only [19:02] powersj, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331161 [19:02] * smoser wonders if that will pass check_version in c-i [19:02] smoser is that using log2dch? [19:03] heh we will find out shortly :) [19:03] blackboxsw, it did, yeah [19:03] I just pushed that into our cloud-init/qa-scripts repo [19:03] so we have a copy of it in non-gist format :) [19:03] i un-indented [19:04] figured we'd collect any tools we are using for our varios dev practice in powers [19:04] figured we'd collect any tools we are using for our varios dev practice in powers' shared repo [19:04] various even [19:04] bak 2 typoe skool [19:05] :D [19:06] ok created two jenkins jobs for release based testing, given a repo + branch it will run all LXD tests, one for xenial, one for artful. [19:15] smoser: build passed [19:15] install deb: "cloud-init_17.1-0-gbf6456f-1~bddeb_all.deb" into target [19:15] * blackboxsw runs that branch on aws [19:15] smoser: are we really still including the git tag? [19:16] powersj, ? [19:16] when you release will the version string have "-gbf6456f" or some revision string in it and not just "17.1"? [19:18] the version string in the debian package will have that. [19:20] hm.. [19:22] CI passed [19:22] running integration tests [19:26] smoser: python-jsonschema needs to not be required by the package on that branch [19:26] dpkg: dependency problems prevent configuration of cloud-init: [19:26] cloud-init depends on python3-jsonschema; however: [19:26] Package python3-jsonschema is not installed. [19:27] Hey. I am having an exception when trying to use `network-config` with NoCloud cloudinit that is suppose to provision my libvirt instances. I tried both version1 and version2 of the network-config. Details can be found in http://paste.ubuntu.com/25588011/ (both configurations and an error) [19:27] blackboxsw, so the debian packages built from trunk will get -0-gXXXX the ones that go into ubuntu will have [19:27] 17.1-0ubuntu1 [19:27] but then the first time we grab a snapshot again we'll get [19:28] 17.1-0-gXXXXX-0ubuntu1 [19:28] blackboxsw, "package on that branch" ? [19:28] blackboxsw, trunk requires python3-jsonschema [19:29] xenial patches that dependency out [19:29] sorry, just a reminder that we carry a patch for no python3-jsonschema [19:29] yeah [19:29] ok good good [19:31] this is one of the things that testing the daily builds from the ubuntu/xenial + trunk build would cover [19:31] https://git.launchpad.net/cloud-init/tree/debian/patches/stable-release-no-jsonschema-dep.patch?h=ubuntu/xenial [19:31] ist hte patch [19:32] xenial lxd tests passed https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-release-lxd-xenial/2/console [19:33] 2017-09-21 19:31:43,001 - util.py[DEBUG]: Cloud-init v. 17.1 running 'init-local' at Thu, 21 Sep 2017 19:31:42 +0000. Up 5.54 seconds. [19:33] 2017-09-21 19:31:43,523 - handlers.py[DEBUG]: finish: init-local/search-Ec2Local: SUCCESS: found local data from DataSourceEc2Local [19:33] ok looking good aws [19:37] kszarlej: I've seen that error before; I can't find the bug but it should be fixed in trunk; if possible you can test/try the daily cloud-init rpm https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/ [19:41] kszarlej, please do try trunk. i can't reproduce the error there. [19:41] i suspect you're on centos/rhel based on python2.7 [19:43] yep [19:43] 2 bad [19:43] I have to modify the [19:43] original centos cloud image [19:45] http://paste.ubuntu.com/25588132/ [19:45] kszarlej, ^ shows how you can use a tool in trunk to more quickly iterate [19:45] which probably would have been useful to you earlier [19:45] :) [19:48] 2017-09-21 19:45:36,396 - util.py[DEBUG]: Cloud-init v. 17.1 running 'init-local' at Thu, 21 Sep 2017 19:45:36 +000 [19:48] 0. Up 9.36 seconds.2017-09-21 19:45:39,401 - handlers.py[DEBUG]: finish: init-network/search-GCE: SUCCESS: found network data from Dat [19:48] aSourceGCE [19:48] https://www.irccloud.com/pastebin/7XrX4aVw/ [19:48] gce looking good [19:48] and now to remember my azure creds [19:49] 7a3nf07sf [19:49] something like that =) [19:49] blackboxsw: created card and added to you [19:49] blackboxsw: can you add links to your pastebin results there? [19:50] I'm pretty good w/ ctrl-v. let's see :) [19:50] thx powersj [19:50] https://trello.com/c/sd5qvgJx/428-171-release [19:50] blackboxsw: maybe you can help smoser; there was some ctrl-v/ctrl-p trouble earlier [19:50] hahaa [19:54] in firefox with pentadactyl i could just hit 'y' rather than ctrl-v. [19:54] but cvim doesn't seem to work as well. thats supposed to work i think, but doenst sometimes. in chrome. [19:54] vimium! [19:54] vimperator [19:54] did that get ported to chrome ? [19:55] it used to be ff only (*years* ago ) [19:55] idk, actually [19:55] ya [19:55] my cube neighbor at hp was hardcore vimperator user [19:55] sadly, it's not as complete as vimperator was (input boxes) so I've added also wasavi (which is inputbox only vim support) [19:56] but the link/tab highlighting is super cool in vimium [19:56] pentadactyl was well better follow-on to vimperator [19:56] but is kind of dead. [19:56] https://github.com/5digits/dactyl/issues/99 [19:57] google seemed to think that cvim was better than other options for chrome [19:58] and wasavi is pretty nice. but can't read or write to files. [19:59] yeah [20:00] artful lxd tests passed https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-release-lxd-artful/2/console [20:04] running nocloud kvm [20:06] azure clean boot test completed [20:07] ok, looks like that'll cover it w/ powersj' test [20:07] * blackboxsw remebers to teardown my azure resource group [20:08] smoser: ok got coverage on ec2, azure, aws, nocloud, lxd [20:08] +build +ci [20:09] +MAAS Compatability Testing [20:09] I use wasavi too. ... though I keep accidentally triggering it [20:16] same [20:16] it's also not the same [20:16] https://public.etherpad-mozilla.org/p/cloud-init-17.1 [20:24] smoser: I upgraded to newest from daily rpm builds [20:25] (v0.7.9) [20:25] and still same error [20:27] the exact version is cloud-init-0.7.9+286.gd3a8777-1.el7.centos.noarch [20:29] kszarlej, hm.. [20:29] ah [20:29] wait [20:30] kszarlej, so... [20:30] first thing [20:31] is that 'network:' should not be present in the network-config file [20:31] as its already namespaced to 'network' [20:31] clearly better error would be ncie. [20:31] but try out-denting that and dropping network: [20:31] the 'version: 2.... i'm not sure [20:31] https://bugs.launchpad.net/cloud-init/+bug/1708255 [20:31] Ubuntu bug 1708255 in cloud-init "empty or invalid network config dictionaries are not handled well" [Undecided,New] [20:32] ok trying to out-dent [20:32] trying with version 1 synta [20:33] x [20:33] oh. and kszarlej you didn't have a mac_address in your v1 [20:34] smoser: I have [20:34] network: [20:34] version: 1 [20:35] config: [20:35] - type: physical [20:35] name: eth0 [20:35] mac_address: 0e:a5:95:fd:a3:b2 [20:35] subnets: [20:35] - control: auto [20:35] type: static [20:35] address: 10.10.1.120/24 [20:35] gateway: 10.10.1.3 [20:35] dns_nameservers: [20:35] - 8.8.8.8 [20:35] yeah, in your paste you didn't have {{ mac address }} [20:35] - 8.8.4.4y [20:35] here is the exact config. (the y on the end is by accident) [20:35] sry I should have used paste [20:35] thats the exaact config I used now and it failed [20:36] well you should not have 'network' [20:36]