/srv/irclogs.ubuntu.com/2018/05/29/#cloud-init.txt

=== rangerpb is now known as baude
rharperhatifnatt: hrm, that looks like a bug, could you file one   https://bugs.launchpad.net/cloud-init/+filebug  with your config that you posted and log ?14:29
smoserrharper: i think hatifnatt posted on bug 1746455 and i responded there.14:50
ubot5bug 1746455 in cloud-init "cloud-init vSphere cloud provider DHCP unique hostname issue" [High,Fix released] https://launchpad.net/bugs/174645514:51
dmbaturinHi everyone!15:03
smoserhi dmbaturin15:11
dmbaturinWhat determines which of these: https://github.com/cloud-init/cloud-init/tree/master/cloudinit/distros will be run?15:13
dmbaturinAlso, what's the policy on including new distros?15:14
blackboxswdmbaturin: /etc/cloud/cloud.cfg  system_info: distro15:14
blackboxswor any override of the same system_info:distro key present in yaml files in /etc/clouc/cloud.cfg.d/*15:15
blackboxswIt'15:16
dmbaturinI see, thanks.15:16
blackboxswdmbaturin: it'd be nice to have a merge proposal with a new distro module proposed against cloud init following the devel instructions @ http://cloudinit.readthedocs.io/en/latest/topics/hacking.html15:17
blackboxswthen we can pull that into tip of cloud-init so that the new distro aligns with existing work.15:18
blackboxswwe can help you get any branches in fairly quickly15:18
dmbaturinI want to add support for cloud-init to VyOS (https://vyos.io), but since it's only loosely debian-based and the configuration system is completely different, we need to have cloud-init interact with its config system specifically, so I'm looking how I can make that integration.15:20
dmbaturinLet me read the hacking guidelines.15:20
dmbaturinWhat I'm supposed to put in the "Please add the Canonical Project Manager or contact" field?15:35
dmbaturinIn the contributor agreement.15:35
powersjsmoser: ^15:37
dpb1dmbaturin: are you signing for just you? or for a company?15:38
dpb1dmbaturin: if just for an individual, writing David Britton is fine.15:39
dpb1(that's me)15:39
dpb1In the end, it's just for Canonical legal to have a contact point in case they aren't sure why you are signing.15:40
dmbaturinAs an individual. Can it later be converted to a company contact at some point?15:41
dpb1dmbaturin: at that future point, just have your company rep do a new one15:41
dmbaturinWell, if anyone is going to be a company rep, most likely that will be me again.15:42
dpb1dmbaturin: ya, you can follow up later with that once you get confirmation from the right entity at your company15:43
dpb1it shouldn't lock you out or anything15:43
dmbaturinAh, ok. Signing for myself for now then.15:43
dmbaturinAll contributions to cloud-init must be a allowed to be dual licensed under GPLv3 or Apache 2.0?15:49
dpb1dmbaturin: yes, the current project is licensed gpl/apache2 https://github.com/cloud-init/cloud-init/blob/master/LICENSE15:52
smoserdmbaturin: from a legal perspective, any thing you do that is licensed apache 2.0 is effectively allowed to be licensed GPLv315:54
smoser(per the FSF, thats not my legal opinion)15:55
dmbaturinYes, I know. The original internal libraries of VyOS are GPL, need to be careful not to link to them then.15:55
smosererr... more clear, that is the FSF's legal opinion, and I do not disagree... I just wanted to be clear that I was not stating *my* legal opinion (which would have no value as IANAL)15:56
dmbaturinWe didn't choose it, our own new libs are either LGPL or MIT, but the ones that were there before the fork are GPL.15:56
dmbaturinOh, don't worry, even if you were a lawyer, from worldwide perspective it would be just as worthless as when you are not. ;)15:57
dpb1hah15:57
dmbaturinI mean, in quite a few jurisdictions GPL (or Apache) would technically be legally void just because it's not in their national language; and in an even greater number of jurisdictions no one really tried it in court.15:59
dmbaturinA more practical question: would it be fine to introduce dependencies on something that will be in the target distro into those distro-specific modules?16:01
blackboxswok looks like we're at the cloud-init status meeting time again (delayed 1 day for us holiday)16:05
blackboxsw#startmeeting Cloud-init bi-weekly status meeting16:05
meetingologyMeeting started Tue May 29 16:05:51 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.16:05
meetingologyAvailable commands: action commands idea info link nick16:05
dpb1dmbaturin: there will be a time for open questions in this meeting in just a few minutes. :)16:06
dpb1so you will have the right people around16:06
blackboxswwelcome folks to another cloud-init community status meeting,  today's meeting delayed by one day due to US holiday. Next meeting will be June 11th. same time16:06
blackboxswI've added an actions  topic  to this meeting so we can wrap up  or carry over any actions discussed last time16:07
blackboxswthe topics will be Previous Actions, Recent Changes, In-progress Development,  and Office Hours16:08
dmbaturinOh, cool.16:08
blackboxswAs always notes will be posted to the following site16:09
blackboxsw#link https://cloud-init.github.io/16:09
blackboxswwelcome dmbaturin good timing. :)16:09
blackboxsw#topic Previous Actions16:09
dmbaturinYeah, I'm just in time it seems. ;)16:09
blackboxsw2 weeks ago we had a couple of followup items that needed some extra review:16:09
blackboxsw* ACTION: blackboxsw review distro dection and empty modules list16:10
blackboxsw* ACTION: robjo review existing chrony support in master per rharper's work16:10
blackboxsw* ACTION: blackboxsw carryover network hotplug vs network maintenance on reboot-only16:10
blackboxswwe did get through robjo's branches on distro *detection* and landed them\16:10
blackboxswand I know our team also discussed a potential approach to network hotplug vs network maintenance to better enable SmartOs folks who want to handle network config across reboots only16:11
blackboxswI think we decided we needed to draw up a quick shared document on a proposal which would allow for maintenance on reboots only vs true hotplug.16:12
blackboxswI'll carry over that action to write up a doc on this and send it to list by the next meeting16:12
blackboxswACTION blackboxsw write up short doc/branch on  hotplug versus network maintenance on reboot for comment16:13
blackboxswand I believe robjo from SuSE was able to get through rharper's chrony support branch with a couple comments too16:13
blackboxswso no other actions from last meeting16:13
blackboxsw#topic Recent Changes16:13
blackboxswthis following content landed in cloud init tip over the last two weeks16:14
blackboxsw- Do not use the systemd_prefix macro, not available in this environment16:14
blackboxsw      [Robert Schweikert]16:14
blackboxsw    - doc: Add config info to ec2, openstack and cloudstack datasource docs16:14
blackboxsw      [Chad Smith]16:14
blackboxsw    - Enable SmartOS network metadata to work with netplan via per-subnet16:14
blackboxsw      routes [Dan McDonald] (LP: #1763512)16:14
blackboxsw    - openstack: Allow discovery in init-local using dhclient in a sandbox.16:14
ubot5Launchpad bug 1763512 in cloud-init "DataSourceSmartOS ignores sdc:routes" [Medium,Fix committed] https://launchpad.net/bugs/176351216:14
powersjlol!16:14
powersjwelcome back16:14
blackboxswheh looks like I got kicked for the paste :)16:14
powersjblackboxsw: your last message was - openstack: Allow discovery in init-local using dhclient in a sandbox.16:14
blackboxsw    - tests: Avoid using https in httpretty, improve HttPretty test case.16:15
blackboxsw10:14       (LP: #1771659)16:15
blackboxsw10:14     - yaml_load/schema: Add invalid line and column nums to error message16:15
blackboxsw10:14       [Chad Smith]16:15
blackboxsw10:14     - Azure: Ignore NTFS mount errors when checking ephemeral drive16:15
blackboxsw10:14       [Paul Meyer]16:15
ubot5Launchpad bug 1771659 in cloud-init "unittests fail in OpenSuSE 42.3 with httpretty issues" [Medium,Fix committed] https://launchpad.net/bugs/177165916:15
blackboxsw- packages/brpm: Get proper dependencies for cmdline distro.16:15
blackboxsw10:14     - packages: Make rpm spec files patch in package version like in debs.16:15
blackboxsw10:14     - tools/run-container: replace tools/run-centos with more generic.16:15
blackboxsw10:14     - Update version.version_string to contain packaged version. (LP: #1770712)16:15
blackboxsw10:14     - cc_mounts: Do not add devices to fstab that are already present.16:15
blackboxsw10:14       [Lars Kellogg-Stedman]16:15
blackboxsw10:14     - ds-identify: ensure that we have certain tokens in PATH. (LP: #1771382)16:15
blackboxsw10:14     - tests: enable Ubuntu Cosmic in integration tests [Joshua Powers]16:15
blackboxsw10:14     - read_file_or_url: move to url_helper, fix bug in its FileResponse.16:15
blackboxsw10:14     - cloud_tests: help pylint [Ryan Harper]16:15
blackboxsw10:14     - flake8: fix flake8 errors in previous commit.16:15
ubot5Launchpad bug 1770712 in cloud-init "It would be nice if cloud-init provides full version in logs" [Medium,Fix committed] https://launchpad.net/bugs/177071216:15
blackboxsw10:14     - typos: Fix spelling mistakes in cc_mounts.py log messages [Stephen Ford]16:15
blackboxsw10:14     - tests: restructure SSH and initial connections [Joshua Powers]16:15
ubot5Launchpad bug 1771382 in cloud-init "ds-identify: fails to recognize NoCloud datasource on boot cause it does not have /sbin in $PATH and thus does not find blkid" [Low,Fix committed] https://launchpad.net/bugs/177138216:15
blackboxsw10:14     - ds-identify: recognize container-other as a container, test SmartOS.16:15
blackboxswok hopefully we ended on ds-identify16:15
dmbaturinYes, we did.16:16
blackboxswexcellent. sorry for the paste, I'll send this out to cloud-init@lists.canonical.com a day before the next meeting so we don't have to IRC flood here16:16
blackboxswmake that cloud-init@lists.launchpad.net16:17
blackboxswalso we finished our SRU (stable release update) of cloud-init 18.2.27 to Bionic.16:17
blackboxswUbuntu Cosmic currently reflects near tip of master 18.2.5916:18
blackboxswok that's all for Recent Changes16:18
blackboxswanything I'm missing powersj ?16:18
powersjI think you are good16:19
blackboxsw#topic In-progress Development16:19
blackboxswWe track upstreams progress publicly in trello16:19
blackboxsw#link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin16:19
blackboxswany blue labeled cards are cloud-init core work16:20
blackboxswwe have been fixing a couple of bugs raised by our CI infrastructure on newer series of Ubuntu . currently a minor issue with salt minion on Bionic or later, and a couple of unit and integration test race conditions16:21
blackboxswbig ticket items for cloud-init in the nearterm are metadata standardization across clouds, so cloud-init scripts/cloud-config template can source these cloud-provided values16:23
dmbaturinMetadata standardization is something I really would like to see, if you need more hands for that, let me know.16:24
blackboxswthe standardization of this instance-data will allow folks to script against any standard values provided to cloud-init in the same way on any cloud. Think hostname, fqdn, ip addrs, region name etc.16:24
dmbaturinSSH keys too!16:24
blackboxswdefinitely dmbaturin I'll point you at a couple branches and what we're thinking16:24
blackboxswthis conent will show up in /run/cloud-init/instance-data.json16:25
blackboxsw#link https://cloudinit.readthedocs.io/en/latest/topics/datasources.html?highlight=instance-data#instance-data16:25
blackboxswand will also be referenced via jinja template variables16:26
blackboxswand a cloud-init query CLI16:26
dmbaturinThe data will be updated whenever a change in the environment is made?16:28
blackboxswAlso powersj will be working toward a common library for cloud-testing in the weeks to come which cloud-init integration tests will leverage to drive lxd, ec2, openstack azure etc for a cloud testing16:28
dmbaturinAlso, will it be possible to stop cloud-init from doing anything but writing that data and starting an external script to process it?16:29
blackboxswdmbaturin: some of that functionality will be handled in the hotplug work we are starting on. There will be operations that can be triggered by either a hotplug monitor on metadata or by cloud-init's CLI to say query from cache (the instance-data.json file) versus query fresh/update16:30
dmbaturinI see.16:30
blackboxswdmbaturin: cloud-inits init-local or init-network stage is what calls "get_data" on the give datasources to collect and write that data to file. Spawning a script is generally done through runcmd which happens in cloud-init's 'final' stage. Trying to decouple them (and skipping the modules:config stage) is possible by altering /etc/cloud/cloud.cfg in a custom image to specify no modules in a given stage. Though it's not16:33
blackboxswreally recommeded as most of the modules only do a quick sanity check to see if they are specifically enabled before trying to do any realy work16:33
blackboxswwe try to keep boot time as fast as possible and cut out the fat where we can16:33
blackboxswif that's the concern you had16:33
blackboxswok I think that's it for In-progress development we can move to office hours for all addtional discussion16:34
blackboxsw#topic Office Hours (next ~30 mins)16:34
dmbaturinNo, boot time is not the primary concern here, my concern is how to ensure no module is trying to treat our system as if it was a normal Debian (which either doesn't work or can potentially get the system into an inconsistent state).16:35
dmbaturinI guess if we are having a real meeting, it may be a good idea to formally introduce myself and the project. :)16:36
blackboxswAll topics of interest to cloud-init development can be brought up and discussed here. If there are merge proposal that need attention, bugs that need work just bring them up here we should have a few sets of eyes on this channel to discuss and comment16:36
blackboxswsounds good dmbaturin introduce away :)16:36
blackboxswChad smith, Canonical, one of the maintainers of cloud-init. We have a few others here (some on vacation). powersj rharper smoser dpb1  all canonical as well.16:37
dpb1idk who blackboxsw is16:38
dpb1he might be crazy16:38
blackboxswfrequently we have other distribution developers and cloud devs here too (SuSE, RedHat, Microsoft Azure, SmartOS, VMWare )16:38
blackboxswheh, I'm just a bot16:38
dmbaturinSo, I'm one of the maintainers of the VyOS project (http://vyos.io). It's a distro for routers and firewalls whose primary goal is to be just like hardware routers, but not tied to any hardware, which includes a single config file and unified CLI with a commit/rollback model, versioning, and cross-checks (e.g. if you try to reference a non-existent NIC in DHCP configuration, commit fails).16:38
dpb1nice to meet you dmbaturin16:39
blackboxswahh makes sense. So debian-based os kindof, which is why you'd want to lock down what modules run.16:40
dmbaturinWe support all major virtualization platforms now in the sense of including all required drivers and utilities, but autoconfiguration on cloud platforms is only supported for EC2 via a custom script, so we are looking to ways to support more clouds, ideally without doing the work that is already done, or at least contributing those general things into something where more people can benegit from it, not just us.16:40
blackboxswalos dmbaturin each config module claims what distro is supported in a distro property, so you could vet what modules you want to run, and only add VyOS to the list of compatible distros. Config modules all live in source at cloudinit/config/cc_*py.16:41
blackboxswbut we can discuss that confuig module support (or not) once you dig in to look at supporting VyOS16:41
dmbaturinYes, I'm thinking how exactly it should be done.16:42
blackboxswdmbaturin: cloud-init's a pretty good choice for getting that cloud-support breadth for free16:42
* robjo sorry I'm late16:43
blackboxswrobjo: sorry for the late change from yesterday's normal meet time16:43
dmbaturinThe least intrusive option would be to indeed improve the instance data format, so that we can simply pass it to our own script, which is why I'm all for contributing to it.16:43
robjoblackboxsw: noLnxDistro branch has not yet been merged16:43
blackboxswbah robjo ahh you're right16:44
blackboxswok looks like you handled all review comments. I'll get it landed today16:44
smoserpowersj: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/34706016:44
blackboxswACTION blackboxsw land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/336794 today16:44
smoseryou moved that to 'approved' i guess ?16:45
smoserwhich meant the bot didnt comment (sorry blackboxsw ... interupted)16:45
dmbaturinblackboxsw: Could you point me to the branches were the work on instance data is going on?16:45
blackboxswdmbaturin: here's one stale one I need to get back to this week. for enabling the template reference of instance-data.json content16:45
powersjsmoser: ah sorry you are right16:46
smoserpowersj: i'm going to land it anyway16:47
robjoalso I think emptyStageOK branch should be ready to go16:47
powersjok16:47
blackboxswhrm digging on the metadata branch.16:47
blackboxswdmbaturin: the trello card I'll be tying branches to is this one16:48
blackboxsw#link https://trello.com/c/5n5B8x23/802-cloud-init-query-standardized-json-information16:48
robjoand https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333904 should be on smoser plate16:48
robjoor anyone else who wants to pick it up and get it merged, please16:49
blackboxsw#link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/33529016:49
blackboxsw^ dmbaturin initial template handling thoughts.... I have to create at least 2 branches to standardize datasource class apis to make the metadata content easier to generalize and I can add your launchpad username to the review as I put them up16:50
rharpersmoser: re: hostname, yes, that's right;  we probably could update the set_hostname docs to mention that detail w.r.t early hostname setting16:50
rharperblackboxsw: sorry to interrupt16:51
blackboxswdmbaturin: what's your launchpad user name?  (mine's chad.smith)16:51
dmbaturinblackboxsw: dmbaturin16:51
blackboxswheh.16:51
blackboxswthx16:51
dmbaturinI'm too predictable. ;)16:51
blackboxswyeah, I lost a bet on blackboxsw :)16:51
blackboxswrobjo: okay adding that branch too for review/landing16:51
blackboxswACTION land  https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/33390416:51
blackboxswACTION land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/33679416:52
blackboxswalright, any other topics or discussions?16:53
dmbaturinTrello is integrated with launchpad?16:53
blackboxswdmbaturin: nope, just easy to use for our agile workflow. And simple to cut-paste links, assign people, drag to different lanes as the work progresses16:55
blackboxswhas a lot of github integraiton if you get the right plugins16:55
blackboxswwe have some minimal tooling that can talk to lauchpad and inject cards, but that's hand-written, not part of trello product.16:56
dmbaturinI mean, if you add my username there, will I get any notifications about card changes.16:59
blackboxswdmbaturin: I get emails from all trello card moves,changes. let's see16:59
blackboxswI can subscribe you to the card (you want the standardized json stuff?)16:59
dmbaturinYes.17:00
blackboxswhrm can't find your user17:02
blackboxswahh17:02
blackboxswI think I invited you17:03
blackboxswand added your user to the card so you can watch it progress17:03
blackboxswok I think that about wraps up our meeting for today17:05
blackboxswany parting shots?17:05
blackboxswI'll post these notes to our github project page17:05
blackboxsw#link https://cloud-init.github.io/17:05
blackboxswthanks again all17:05
blackboxsw#endmeeting17:05
meetingologyMeeting ended Tue May 29 17:05:59 2018 UTC.17:05
meetingologyMinutes:        http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-05-29-16.05.moin.txt17:05
dmbaturinblackboxsw: Since I'm not very familiar with the codebase as you can see, some tasks in form of "do this and that to this module" would be perfect.17:07
dpb1blackboxsw: what was the bet?17:07
blackboxswdpb1:  During world cup I thought the US was going to lose and Dean bet otherwise, at stake was that I had to change my nick from csmith -> blackboxsw (my former side-job company name) because dean(aka, Beret) thought that nick was cooler than name-based nick17:09
dpb1blackboxsw: us men?17:10
blackboxswUS men. a couple years ago17:10
* blackboxsw secretly wanted to change my nick anyway17:10
dpb1blackboxsw: seems like a safe bet on your part then, I guess it didn't work out17:11
dpb1:)17:11
Beretit's amazing how I remember that so differently17:11
blackboxsw... and yes I invoked Beret in #cloud-init +1 bbsw17:12
dpb1I question beret's sanity if he bet on the US men to win something in the world cup17:12
blackboxswheh, I think it was a semi's game, but Beret can correct me, now that the source of truth is here. :)17:12
Beretsame here17:13
Beretthat's why I remember it differently17:13
blackboxswheh, maybe I bet they would win17:13
Beretnow US women would have been different17:13
blackboxswabsolutely17:13
dpb1hehe17:16
dpb1the picture is now becoming clear17:16
blackboxswheh, good think meetingbot didn't record this conversation, otherwise I'd have too many sources of truth refuting my lies ;)17:18
dmbaturinI use a name-based nick after 70's hackers.17:26
jocharharper: re: "the change (agent_command using __builtin__) upstream to the AzureDS would break existing behavior in ubuntu xenial", is this by design or is there more work to be done to address this? :)17:31
=== blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 6/11 16:00 UTC | cloud-init 18.2 released (03/28/2018)
rharperjocha: by design, we don't want to change behavior on Xenial instances18:21
jochaalrighty, thanks!18:35
=== Beret- is now known as Beret

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!