[02:55] blackboxsw, good evening. I was wondering if you guys had managed to look into that bug I submitted last month. [09:41] smoser: Not sure if i said thanks for everything earlier: Thanks :) === shardy is now known as shardy_lunch [15:59] blackboxsw: urg, the c-i meeting is on top of our standup [16:01] urg [16:01] o/ [16:01] \o [16:01] it [16:01] o/ [16:01] it's that time again [16:03] #startmeeting Cloud-init bi-weekly status [16:03] Meeting started Mon Nov 13 16:03:13 2017 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. [16:03] Available commands: action commands idea info link nick [16:03] o/ [16:03] time change got to us [16:04] #meetingtopic Recent Changes [16:05] hey folks. thanks for joining just pulling together the content for the last couple weeks of work for the cloud-init project [16:06] http://paste.ubuntu.com/25954862/ [16:06] $ git log a90a8b1cb3104ee3250ac79d6e25a9ff4f527baa.. | log2dch | sed 's,^ ,,' | pastebinit [16:06] most of the ubuntu-side of the house was involved in handling the SRU of 17.1 into ubuntu and handling any discovered regressions [16:06] Published cloud-init packages to Bionic Beaver release [16:06] Update Gentoo Linux support to "rc-service" scripts as "service" is deprecated, thanks to ckonstanski! [16:06] Detected and fixed a pre-release regression of resizefs when root path is specified by UUID on the kernel cmdline (LP: #1725067) [16:06] Launchpad bug 1725067 in cloud-init (Ubuntu Zesty) "cloud-init resizefs fails when booting with root=PARTUUID=" [Medium,Fix committed] https://launchpad.net/bugs/1725067 [16:06] #link http://paste.ubuntu.com/25954862/ [16:07] #info SRU queued for release today [16:07] Here's the cloud-init content we published for the last two weeks: [16:07] #link https://github.com/canonical-server/dev-summary/blob/master/doc/2017-10-31.md [16:07] #link https://github.com/canonical-server/dev-summary/blob/master/doc/2017-11-07.md [16:09] last week we handled an EC2 behavior regression for xenial, whereby we didn't want to change cloud-init to configure all nics based on ec2 metadata, we will only configure the primary nice [16:09] last week we handled an EC2 behavior regression for xenial, whereby we didn't want to change cloud-init to configure all nics based on ec2 metadata, we will only configure the primary NIC [16:09] with those SRU regresssions fixed and published to master, we expect cloud-init 17.1 updated in Xenial,Zesty and Artful today [16:10] #meetingtopic In-progress development [16:10] smoser: rharper anything here? [16:10] #link http://bit.ly/ci-reviews [16:11] robjo has done a couple fixes for SuSE and i've pulled a few of them. [16:11] he has one up i saw yestderday for ntpSuSE [16:11] others ther.e we've been delinquent due to some distractions recently. [16:11] blackboxsw: nothing new for me at the moment [16:12] and chad had one up for clean and status [16:12] btw thx robjo ckonstanski and Dave Mulford for the fixes over the last iteration. We also expect that a couple VMware branches for the OVF datasource will last this week or next [16:12] which is nice. [16:13] moving the meeting an hour foward while we are on Standard time or is this a one time occurance, did I miss an announcement? [16:13] #meetingtopic Office Hours (next 30 minutes) [16:14] lp#1731619, chrony support, should that also be driven through ntp config or should there be a new config option? [16:14] so we'll hang out with eyes on this channel for any burning questions/bugs/questions [16:14] robjo: well, the meeting is listed in UTC time [16:14] that pays no attention to US legislation to change clocks at random points in the year :) [16:15] oK, my fault when I added it to my calendar, eay enough to fix ;) [16:15] but the humans here were also affected :) [16:15] heh, anyone opposed to shifting this meeting time +30 from now during the next few months? [16:15] as the meeting now collides w/ another meeting for us [16:15] :/ [16:16] officially 16:30 UTC? [16:17] Well, I'd prefer to either follow the "randomness" clock manipulation or not follow it [16:19] meaning don't change the meeting time because there exists a conflict when standard time switches to daylight savings or vice versa, becaus if you do that you might as well follow the silliness of the government to begin with [16:19] fair point. ok let's keep the new time as is. [16:20] we've discussed side-channel, we can shift our meetings out of the way of this [16:20] so robjo +1 [16:20] 16:00 UTC [16:24] also related to CI side, powersj and rharper spent quite a bit of time w/ our continuous integration infrastructure fixing/addressing memory & storage pressure issues to make sure we avoid intermittent false test failures due to timeouts or system resource contention [16:24] #link https://jenkins.ubuntu.com/server/view/cloud-init/ [16:28] is there a way to use metadata in the cloud-init file? specifically, if i want to use the aws-provided instance id in an attribute [16:28] OK, back to my question about chrony: lp#1731619, chrony support, should that also be driven through ntp config or should there be a new config option? [16:28] like configuring the chef node name to have my instance id in it [16:32] #link https://bugs.launchpad.net/cloud-init/+bug/1731619 [16:32] Launchpad bug 1731619 in cloud-init "Support chrony as a client for ntp" [Undecided,New] [16:33] it's a good bug, we've had a couple of discussions about ntpd versus timesyncd for different system environments [16:34] current implementation of cc_ntp module is to return False ('ntp' not installable) on certain known environments where we know we want systemd timesyncd to run instead by default [16:34] via: i think what your asking is (i htink) covered in https://trello.com/c/AYaCdQyT [16:35] well, i'm trying to do it in a yaml cloud-config file [16:35] right. as it is right now, via you cann't reference anything from the metadata. [16:35] does that mean i need to use #jinja and if so how does that play with #cloud-config ? [16:35] oh [16:35] bummer [16:36] should i just switch to a shell script? [16:36] but we'd hope to implement that. [16:36] via: thats really the only way right now. and then in the shell scripty you'd have to query the metadata service yourself. [16:36] okay, damn [16:36] thanks [16:36] robjo: we think that's a good approach/feature suggestion. We could add chrony template files etc like the ntp templates, and we might be able to have the distro report what time sync daemon it wants to run [16:36] basically... we realize what you're asking is quite helpful and reasonable but dont have a way to do it right now [16:36] but we do plan on implementing it. [16:37] no worries, i'm stuck on an ancient version anyway [16:38] blackboxsw: That was my thinking, move the "service_name" setting to the distro as "time_service_name" and then drive cc_ntp based on that [16:39] since with a third option the black/white decision being made today will no longer work [16:39] +1 robjo yeah. rharper was chatting about this potential approach as well [16:39] look there is also grey ;) [16:39] heh yeah [16:40] Next question.... network config. [16:40] yeah might have to 'grow' an override option in cc_ntp module eventually [16:41] as those grey use-cases come up (per bugs/requests ;) ) [16:41] A long timi ago the RHEL implementation was re-written to use sysconfig renderer, but RHEL sysconfig and SLE sysconfig are different, why wouldn't they be [16:42] that also implies that the openSUSE/SLES implementation for network config rendering still uses the "old" implementation and thus produces a warning in the log file [16:43] * blackboxsw is looking for the warning generated [16:43] this would imply some refactoring is in order if we want to move openSUSE/SLES to using the newer API to render the network config [16:44] blackboxsw: "apply_network_config is not currently implemented " [16:44] "for distribution '%s'. Attempting to use apply_network" [16:45] ahh. right-o [16:45] the question from my point would be is, when I want to implement the SUSE bits am I also on the hook for the refactoring part or can I get some help with that? which of course will make my life easier ;) [16:47] And yes, I realize a bug will need to be filed, but I haven't figured out how to formulate this nicely [16:47] robjo: I think we should be able to help out a bit with that refactor to make sure it's cleaner and easier to maintain. [16:47] OK :) [16:50] there are a couplengeneric distro fixes which need to get designed (just like in the datasources) to make the common distro classes a bit easier to maintain as well as making classes a bit more modular and more easily tested. [16:51] we still haven't landed some of the common datasource changes we had talked about during the Summit because we've been avoiding risk during the 17.1 release. But, similar/minor architecture changes should start taking shape here for datasources and distros now that we see a light at the end of the tunnel on the release. [16:52] we'll keep our eyes open for discussions/suggestions from folks [16:54] Speaking of data sources, for the SUSE Container As A Service Platform, we implemented a data source to read from local disk, is that something that would be of interest upstream? Yes, this might seem silly but in our use case it makes perfect sense ;) [16:56] robjo: I'm curious how different that datasource would be from nocloud datasource [16:56] http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=nocloud [16:56] #link http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=nocloud [16:56] which allows for providing local data instead of dealing with metadata [16:56] well network metadata [16:57] I wasn't really involved, just accepted the patch to the package and have not done a comparison to nocloud, but I'll take a look [17:00] good deal.... think we are at the top of the hour... so I'll probably end meeting now [17:01] thanks via robjo rharper powersj & smoser. next meeting 2 weeks same early time [17:01] #endmeeting [17:01] Meeting ended Mon Nov 13 17:01:43 2017 UTC. [17:01] Minutes: http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-11-13-16.03.moin.txt [17:01] thanks blackboxsw for running === blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 11/13 16:00 UTC | cloud-init 17.1 released [17:02] np [17:04] smoser: will $(cloud-init query instance-id) work in said shell? it doesn't seem to evaluate [17:05] via: no. cloud-init query doesn't really exist. [17:05] :-( [17:05] the goal is to have a good solution for you [17:05] but there is not one at the time :-( [17:05] so wait, there is actually no way to use cloud-init at all to get my instance id? [17:06] i have to query the aws 169 address with curl or whatever? [17:08] via: if you don't mind hackiness you could resolve the /var/lib/cloud/instance symlink [17:08] ok [17:08] instance_id=$(basename $(readlink -f /var/lib/cloud/instance)) [17:09] (assuming 0.7+) [17:09] you mentioned you might be using something much older [17:09] looks like ec2metadata is installed by default [17:10] via theres something in progress that will help https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115 [17:10] I'm hoping we can land this soon. it'll put a /run/cloud-init/instance-data.json blob thatt will report that for you [17:11] well, tbf, if ec2metadata is installed, i dunno why i'd use anything but it [17:11] i didn't realizeit was [17:11] and we'd support it as a public interface that folks can consume [17:13] that branch referenced is the precursor to cloud-init query subcommand [17:13] precursor/prerequisite [17:21] * ajorg changes his calendar entry for the meeting to UTC... [17:23] thanks ajorg. then it can surprise us again in spring [17:23] haha [17:25] smoser: https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/331660 < this is okay to merge? or did you have other concerns? [17:27] ajorg: i'll look now. [17:27] thanks [17:27] I blame Ben Franklin for the meetings I've missed. [17:28] you should stop hanging out with that guy ajorg, he's a bad influence [17:28] totally. the guy likes to fly kites in the rain [17:28] ... ok I'm back, wrapping up review comments on the cloud-init clean/status branch [17:29] thx smoser for the review, yeah I had pulled out cloud-init status disabled as I had wanted to chat w/ you about all the ways it could be 'disabled' [17:30] rharper: smoser if ds-identify 'disables' cloud-init, we just expect /etc/cloud/cloud.cfg.d/90_dpkg.cfg to have an empty datasource list? [17:33] rharper: per your inotify comment, I tried not depending on python inotify libs as they aren't currently pulled in by cloud-init as a dependency. Shall we add that as a pkg dependency? [17:34] no, /run/cloud-init/cloud.cfg will be empty [17:34] ahh ok rharper thx [17:34] cloud-init reads the cloud.cfg that ds-identify writes [17:35] blackboxsw: no. ds-identify exits wihtout putting the target in systemd's goals [17:37] blackboxsw: bummer on not having inotify tools; [17:41] yeah even the base-pkg inotify-tools !present yet. [17:48] I saw a python syscall binding "inotify-basic" package [17:48] but it would be nice to just have that built-in [18:12] rharper: blackboxsw fyi there are cloud-images for bionic now [18:26] rharper: do you know why we get an ipv6 address in lxc on artful ? [18:26] that seems wrong [18:40] your lxd config enabled ipv6 dhcp [18:41] --dhcp-range fd42:3cd1:a543:bb7::2,fd42:3cd1:a543:bb7:ffff:ffff:ffff:ffff,1h [18:41] smoser: ^^ [18:47] https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1732002 [18:47] Launchpad bug 1732002 in systemd (Ubuntu) "cloud images in lxc get ipv6 address" [Undecided,New] [18:47] rharper: its a bug. they should not. they did not request it. [18:47] doesnt ipv6 ra get you that ? [18:48] xenial doesnt get it [18:48] doesn't run networkd [18:48] networkd does ipv6 ra by default [18:48] that is a bug [18:48] https://bugs.launchpad.net/ubuntu/+source/nplan/+bug/1717404 [18:48] Launchpad bug 1717404 in nplan (Ubuntu Zesty) "IPv6 support regresses with nplan transition" [Undecided,Fix committed] [18:49] https://bugs.launchpad.net/maas/+bug/1655440 [18:49] all seem related [18:49] Error: Could not gather data from Launchpad for bug #1655440 (https://launchpad.net/bugs/1655440). The error has been logged [18:50] rharper: so 1655440 may imply that cloud-init needs to write 'accept-ra: no' [18:50] ? [18:51] I think 1) they do request it, the kernel has ipv6 enabled, which means it does RA 2) dnsmasq for lxd, can be configured to disable ipv6; your host does not have it disabled ; if there is a bug it's related to dhclient6 not running by default in xenial [18:51] no. [18:51] it's hard to say where it should be written [18:51] lxd and cloud-init config specifically say (via omission) not to have ipv6 [18:52] that's unclear [18:52] so behavior that just enablels an ipv6 is wrong. [18:52] look at the lxd bug that stgraber filed [18:52] if you don't want ipv6, you have to pass kernel config [18:52] it's certainly a change in behavior from xenial; I think that needs some discussion w.r.t what that means for artful/bionic [18:54] stgraber's comment does not hmake sense. [18:54] "ever since the switch to nplan by default in Ubuntu, we're not getting an IPv6 address anymore when a RA reaches the container. [18:54] " [18:54] as if we *were* getting ipv6 address by default previously [18:54] but we are not [18:55] IPv6AcceptRA=no [18:55] nplan was disabling IPV6RA by default [18:55] it's no longer doing that [18:55] and you have ipv6 enabled on dnsmasq [18:55] so you're now getting ipv6 addrs in your containers [18:56] https://bugs.launchpad.net/ubuntu/+source/nplan/+bug/1717404/comments/3 ; m-h-d suggests a 3 way switch in the config; but someone is going to be unhappy; this needs discussion AFAICT [18:56] Launchpad bug 1717404 in nplan (Ubuntu Zesty) "IPv6 support regresses with nplan transition" [Undecided,Fix committed] [19:02] rharper: im' missing something [19:02] but i just made cloud-init render [19:03] I suspect that we likely didn't have a dnsmasq that was ipv6 enabled, which means there were not IPV6 RAs going to our containers; [19:03] http://paste.ubuntu.com/25955872/ [19:03] and rebooted [19:03] and i still have the ipv6 [19:03] look at /etc/default/lxd-bridge.upgraded which shows we've got ipv6 enabled [19:03] rharper: no. i had in the past explicilty enabled dnsmasq for ipv6 [19:03] (and i think lxc did that by default) [19:03] it does by default [19:04] i dnot follow. [19:04] what does your /run/systemd/network-* files look like ? [19:04] netplan *used* to disable ipv6 [19:04] so even if you had ipv6 for lxd enabled, it never worked [19:04] ifyou used netplan [19:04] they filed a bug to get netplan to not disable RA by default [19:04] that is bad. [19:05] its worse to open people to attack that were not expecting it [19:05] it's complicated depending on who wants what "default" behavior [19:05] you mean having your kernel enable ipv6 by default ? [19:07] you have an accept-ra option in netplan if you really want avoid getting RAs. [19:07] ok. so maybe that is "complicated" and i somehat agree with m-h-d =. [19:07] but in my newly updated cloud-init i just rendered [19:08] please check the run dir to see what got generated [19:08] http://paste.ubuntu.com/25955901/ [19:09] no [19:09] accept-ra is per-interface. [19:09] http://paste.ubuntu.com/25955908/ [19:09] in any case, cat /run/systemd/network/10-netplan* [19:09] ok. let me try [19:09] cyphermox: should it fail to parse on that ? [19:10] configs present under tree that doesnt' parse them ? [19:10] rharper: I fail to parse your question [19:10] you said he put it in the wrong spot [19:10] should be under eth0: [19:10] and I'm saying should that fail to parse with the value in the wrong spot [19:10] yep [19:11] if it didn't fail, that's a bug [19:11] yes [19:11] agreed [19:11] it did not complain. [19:11] so you can consider that a bug. [19:11] but I'm a little surprised, netplan usually is failing to parse pretty aggressively [19:11] indeed [19:11] and putting it in the right place does get the behavior i expected. [19:12] so my feeling is that cloud-init should render 'accept-ra': no unless there is a dhcp6 stanza [19:12] but this still imo seems broken [19:13] s/cloud-init/netplan; and that's one of the discussion points; I'm not sure we've reached consensus w.r.t the "right default"; [19:13] so in your lxd case were it has a dhcp6 enabled subnet; then should lxd emit DHCP6 config in addition to the v4 dhcp ? [19:13] lxd should, yes. [19:14] I suspect stragber has an opinion to that [19:14] which is counter to what you're suggesting; hence the bug [19:14] rharper: smoser: if I put your config on my system and run 'netplan generate', it does fail to parse [19:15] network config should be deterministic [19:15] what version of netplan is in your image ? [19:15] smoser: ^ [19:15] smoser: except you may really want to no do DHCP for an IPv6 network and do SLAAC, which needs the RAs. [19:15] not "well, you'll get config if someon (nefarious or not) was sending packets on your network) [19:15] smoser: you're arguing with the wrong person; I suggest a comment in the lxd/nplan bug if you want to strike up a discussion [19:15] netplan is bionic image today [19:16] smoser: ergo, disabling RAs in MAAS might be what you want, since MAAS does DHCPv6 and is supposed to manage all, but it's very not the right default elsewhere. [19:17] an oracle (lxd or maas) should declare. [19:18] I think 0.30 doesn't have the accept-ra parsing [19:19] rharper: it does, it's added in 0.28 [19:19] huh [19:20] I see an error when I run generate as well; smoser do you have the cloud-init.log ? [19:20] rharper: bionic does work if we make cloud-init interpret no 'dhcp6' as 'accept-ra: no' [19:20] which i think we should [19:20] oh. yes. we do get a 'generate' error [19:21] wonder how network came up ? [19:21] or did it ? [19:21] the /run/systemd/network/ file shouldn't have been written [19:21] no it didnt [19:21] which means no networking [19:21] yeah. [19:21] oh, ok [19:21] fail. [19:21] just looked at the ipv6 [19:21] and saw nothing [19:21] so that seemeed right [19:21] hehe [19:21] ok [19:26] http://paste.ubuntu.com/25955976/ [19:26] cyphermox: ^ that is what you were saying should work right ? [19:27] no [19:27] hows this for confusing [19:27] $ python3 -c 'import yaml; print(yaml.load("false") is yaml.load("no"))' [19:27] accept-ra is per-interface, it should be under eth0 [19:27] True [19:27] how is that confusing? [19:28] you're asking python if a boolean is X for a language with a pretty loose grammar, not unlike javascript [19:29] javascript has a much more strict gramar. [19:30] its confusing that 'no' is interperted as a boolean. [19:30] the lack of quoting requirements in yaml makes it wierd [19:49] the netplan parser does some additional string -> boolean conversion IIRC [19:51] true == on == yes == y; false == off == no == n ; for fields which are designated boolean : accept-ra uses the handle_netdef_bool which does the conversion === natorious_ is now known as natorious === boxrick_ is now known as boxrick [22:46] rharper: smoser addressed most comments on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513. I [22:48] blackboxsw: cool [22:48] am not quite sure what we should do about standardizing config paths etc. Right now, it feels a bit painful the way content is organized in /etc/cloud/cloud.cfg.d/05_logging.cfg and cloudinit.helpers.Paths and various cmd files. I think we may need to look over a way to centralize more of these config defitinitions in an official config file that can be sourced by other modules [22:49] s/defitinitions/definitions [22:50] blackboxsw: is that to help with schema? [22:50] dpb1: nope just to help with cloudinit modules knowing where the proper log_file_path or run_dir is etc. [22:50] k [22:51] there's a bit of hardcoding of strings and some duplication that I've introduced that'd be nice to bring together in one place [22:51] hardcoding paths [22:51] I was just looking over your branch, makes sense now [22:52] well, there is the confg paths; which consumes the system config into an object [22:52] so I suspect we'll need to use the stage wrappers to ask for path to various keys [22:52] which allows the system config to override [22:53] yeah, it feels like the Init() -> Init.read_cfg() -> cfg object is almost the right place. it just doesn't feel 'accessible' or parameterized quite enough [22:54] it feels like it's still a beast of an object to tear apart into the component you want to look at (as it still contains unparsed yaml strings as the attributes) [22:54] or maybe I've just dug into the wrong hole on it [22:57] oh, that's completely fair [22:58] blackboxsw: I think you want the init object paths properties [22:59] which gives you the cloudinit/helpers.py:Paths() object [22:59] that has alot of the methods that combine the system config with "standard" paths (like instance_link, run_dir, etc) [23:25] yeah, though it doesn't seem to surface log paths. [23:25] maybe we need to extend Paths to be our source of truth... no quite sure [23:26] that sounds reasonable (extending paths)