| redguy | so for posterity - in Debian jessie (systemd 215) when you restart journald as a part of cloud init it makes the cloud-init (and possibly other systemd units) fail when they want to write something into stdout/stderr. Debian stretch (systemd 232) is not affected. | 09:47 |
|---|---|---|
| ajorg | meeting today? | 16:01 |
| blackboxsw | ajorg: morning. just in time | 16:03 |
| blackboxsw | was just determining who would kick it off | 16:03 |
| blackboxsw | <--- winner | 16:03 |
| ajorg | morning | 16:03 |
| blackboxsw | #startmeeting Cloud-inin bi-weekly status meeting | 16:03 |
| meetingology | Meeting started Mon Jan 8 16:03:53 2018 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. | 16:03 |
| meetingology | Available commands: action commands idea info link nick | 16:03 |
| blackboxsw | Happy 2018 cloud-initers! Thanks ajorg for helping kick us off. | 16:04 |
| blackboxsw | Welcome back from break hope the holidays were good for folks. | 16:04 |
| blackboxsw | It | 16:04 |
| blackboxsw | It's been a while since we've held the meeting due to holidays and vacation time. So, not a ton of content to report for the last bit. Digging up those details now | 16:05 |
| blackboxsw | Testing of 17.2 on EC2, Azure, and GCE and release to Ubuntu Bionic | 16:06 |
| blackboxsw | Complete 17.1.46 SRU to Ubuntu Xenial, Zesty, and Artful | 16:06 |
| blackboxsw | Fix documentation around 'init' mode for modules subcommand (LP: #1736600) | 16:06 |
| blackboxsw | Tooling to merge community authored branches into master | 16:06 |
| ubot5 | Launchpad bug 1736600 in cloud-init "CLI: cloud-init modules -h documents unsupported --mode init" [Low,Fix committed] https://launchpad.net/bugs/1736600 | 16:06 |
| blackboxsw | So the canonical side of the team worked a bit on getting the latest SRU updates 17.1.46 into Xenial, Zesty and artful. The testing and verification of that release took a bit of time, but we are getting better(faster) | 16:07 |
| blackboxsw | I think this last SRU only took us 2 weeks instead of 4 weeks. so that frees up more time on upstream reviews and increasing cloud-init's velocity | 16:07 |
| ajorg | great | 16:07 |
| blackboxsw | we also added team tools for streamlining community authored branches. so that we stop slowing folks down :/ | 16:08 |
| blackboxsw | then the only problem is the reviewer :) | 16:08 |
| blackboxsw | #topic Recent changes | 16:09 |
| blackboxsw | ^ should have been that topic. | 16:09 |
| blackboxsw | Also 17.2 release was 'cut' prior to Christmas break, this opened master up for more changes to land. so we've pulled in good fixes for VMWare NoCloud and SLES | 16:10 |
| blackboxsw | digging up the changests now. | 16:11 |
| blackboxsw | Also, keep in touch with our active development and the "done" lane on trello. It's out bookkeeper for anything we are working and Done represents anything landed | 16:11 |
| blackboxsw | #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin | 16:11 |
| blackboxsw | so high-level content that landed between 17.1.46 and 17.2: | 16:13 |
| blackboxsw | * CLI added the clean and status subcommands | 16:14 |
| blackboxsw | * Support for identifying OVF datasource provided by VMware | 16:14 |
| blackboxsw | * NoCloudKVM tests now run in continuous integration | 16:14 |
| blackboxsw | * Formalize DataSource get_data and related properties | 16:14 |
| blackboxsw | * Remove prettytable dependency and introduce simpletable | 16:14 |
| blackboxsw | * VMWare pre and post-customization script support | 16:14 |
| blackboxsw | Thanks ajorg I think you were the author of note on simpletable stuff, it's nice to drop dependencies where we can to increase speed of cloud-init | 16:15 |
| ajorg | it was done selfishly | 16:15 |
| ajorg | we dislike taking on new dependencies :-) | 16:15 |
| blackboxsw | and thanks to robjo(suse) maitree(vmware) too and dojordan and Ryan McCabe(redhat) for recent branches too | 16:17 |
| blackboxsw | :) | 16:17 |
| blackboxsw | Post our 17.2 release we've started work on improved integration..... I think we just got powersj's ec2 integration tests landed right johs? | 16:18 |
| blackboxsw | josh even | 16:18 |
| powersj | \o/ yep! | 16:18 |
| ajorg | nice | 16:18 |
| blackboxsw | sweet, so an extra security blanked for us when we have significant changesets landed in master to ensure ec2 is happy. | 16:19 |
| blackboxsw | powersj: what are out plans for continuous integration frequency | 16:19 |
| blackboxsw | with ec2 specifically | 16:19 |
| ajorg | Can those integration tests be run by others with EC2 accounts? | 16:19 |
| blackboxsw | ajorg: yes they can | 16:19 |
| powersj | I am working on the jenkins jobs this week and hope to have a weekly run as well as a manual run for backport testing | 16:19 |
| blackboxsw | I'll get the cmdline | 16:19 |
| ajorg | thanks! | 16:19 |
| blackboxsw | tox -e citests -m tests.cloud_tests run --os-name=artful --platform=ec2 --preserve-data --data-dir=../results --verbose | 16:20 |
| blackboxsw | or something like that | 16:20 |
| ajorg | got it | 16:20 |
| ajorg | thanks! | 16:20 |
| blackboxsw | powersj: documented it too I think | 16:20 |
| blackboxsw | getting link | 16:20 |
| powersj | https://cloudinit.readthedocs.io/en/latest/topics/tests.html#ec2 | 16:20 |
| blackboxsw | #link https://cloudinit.readthedocs.io/en/latest/topics/tests.html#ec2 | 16:20 |
| blackboxsw | :) | 16:20 |
| blackboxsw | excellent work Josh | 16:21 |
| powersj | thanks for all the reviews :) | 16:21 |
| blackboxsw | anything else I'm missing about landed work? rharper powersj smoser1 ? | 16:21 |
| blackboxsw | otherwise next topic | 16:22 |
| rharper | blackboxsw: nothing from me | 16:22 |
| blackboxsw | #topic In-progress Development | 16:22 |
| blackboxsw | So we've got a fairly healthy review queue that we need to get through as we get the year started.... | 16:23 |
| blackboxsw | we also have a few things we are in flight currently: | 16:24 |
| blackboxsw | - continuous integration improvements per powersj | 16:24 |
| blackboxsw | - dropping dependence on ifup ifdown utils where possible as that's not supported (or installed in some cases) in systemd world | 16:24 |
| smoser1 | blackboxsw: wow. sorry, missing. | 16:24 |
| blackboxsw | who is that smoser1 guy anyway | 16:25 |
| smoser1 | yeah, i didnt see anything missing sorry. | 16:25 |
| === smoser1 is now known as smoser | ||
| smoser | wonder how that happened. | 16:25 |
| blackboxsw | welcome ;) | 16:25 |
| blackboxsw | - netplan improvements per rharper and jinja template support for all cloud-config modules | 16:25 |
| blackboxsw | - and softlayer support per smoser | 16:26 |
| blackboxsw | know the Azure guys are also posting a couple branches on getting a pre-provisioning setup going for thier datasource which looks pretty exciting | 16:27 |
| blackboxsw | I can't think of anything else off the top of my head. | 16:27 |
| robjo | chrony support | 16:28 |
| ajorg | we're only talking feature work in this topic? | 16:28 |
| blackboxsw | any in progress development to highlight is fair game. bug work. refactoring, feature etc | 16:29 |
| blackboxsw | +10 robjo and again thanks for working with us getting all those branches up and (hopefully soon) landed | 16:29 |
| ajorg | what does "jinja template support for all cloud-config modules" mean? | 16:29 |
| ajorg | I'd guess most modules don't need templating? | 16:31 |
| blackboxsw | ajorg: two things. 1. since we have now landed /run/cloud-instance/instance-data.json to store metadata/userdata it'd be that #cloud-config can new be specified with ## template:jinja header and could leverage anything jinjia has to offer plus sourcing any of the instance-data.json metadata fields | 16:31 |
| ajorg | Ah, right. Is that not being done above the module level? | 16:33 |
| blackboxsw | so if people have repetitive or template-driven content in the runcmd or write_files portion or their #cloud-config they'd be able to leverage jinja templates etc | 16:33 |
| smoser | ajorg: yes, above the module level. | 16:33 |
| blackboxsw | ajorg: not anywhere in cloud-config currently | 16:33 |
| blackboxsw | one sec I misunderstood the question | 16:33 |
| blackboxsw | smoser: can you clarify what you mean? | 16:33 |
| ajorg | I mean, shouldn't #cloud-config template expansion happen before the module sees the config? | 16:33 |
| smoser | blackboxsw: we could/should also allow other part types to be rendered | 16:34 |
| smoser | ttps://trello.com/c/xyqxyOxg | 16:34 |
| smoser | er... bad url. in 2 ways | 16:35 |
| ajorg | The the part handler would be the one to do that expansion. | 16:35 |
| smoser | https://trello.com/c/AYaCdQyT | 16:35 |
| blackboxsw | ahh ok, right that makes sense. I think the cut I made was limited in focus to cloud-config modules and custom scripts supporting the ## template:jinja header.. but nothing would preclude handling other parts | 16:35 |
| blackboxsw | so the link to my WIP branch was | 16:36 |
| blackboxsw | #link https://trello.com/c/xyqxyOxg | 16:36 |
| blackboxsw | and the general feature per smoser | 16:36 |
| blackboxsw | #link https://trello.com/c/AYaCdQyT/21-cloud-init-query-standardized-json-information | 16:36 |
| ajorg | Is there a design doc of some kind of this? | 16:36 |
| blackboxsw | not yet.. but we probably should have a spec as it'd be a good template for the docs we'll need to write | 16:37 |
| blackboxsw | scott captured most of the use cases we'd be going for in that last trello link above | 16:38 |
| ajorg | Small example of where some clarity is needed: if Jinja is interpreting {foo} in a user-script, what will it do when it sees a shell variable ${foo} | 16:38 |
| ajorg | ? | 16:38 |
| smoser | you declare that the content is a jinja template | 16:39 |
| smoser | if you provide it something that is not renderable as a jinja template | 16:39 |
| smoser | then it will fail | 16:39 |
| smoser | it requires input to explicitly say "this is jinja". it does not just attempt to render anything. | 16:39 |
| smoser | (unless explicitly told to) | 16:39 |
| blackboxsw | some brief working examples are in the description of the branch @ https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334030 | 16:39 |
| ajorg | Sure. But as a content author, I need to know if Jinja is going to try to render ${foo} or not. | 16:40 |
| smoser | then as a content author you can read jinja docs :) | 16:40 |
| blackboxsw | jinja would try to render {{ foo }} | 16:40 |
| ajorg | :- | 16:40 |
| smoser | ajorg: we'll document a simple case, and we can even document "for shell, you'll have to be aware that ...." | 16:41 |
| smoser | but we're not going to document all of jinja | 16:41 |
| ajorg | I see. | 16:41 |
| ajorg | My understanding was that Jinja was highly customizable in what it interpreted and how, so that it's important to document how you've configured it to work. | 16:42 |
| blackboxsw | and since to burden is on the #cloud-config or script writer to provide the header ## template: jinja\n#cloud-config\n they *should* understand what they are doing | 16:42 |
| blackboxsw | we won't implicitly run the #cloud-config through jinja | 16:42 |
| ajorg | I get that, no problem, what I'm saying is that Jinja is an engine that you configure to do something, not a markup that always does the same thing for everyone. | 16:43 |
| ajorg | Am I making any sense? | 16:43 |
| blackboxsw | understood (though I thought it was fairly constrained it it's application and functionality). We'll make sure that the mechanism by which jinja operates is well documented and confined as best we can... for our own sanity we don't want that template engine to be too flexible... too many tough support use cases | 16:44 |
| blackboxsw | ok anything else for "In progress development" otherwise we can move to Office hours for 30 mins | 16:45 |
| blackboxsw | #topic Office Hours (next 30 minutes) | 16:46 |
| blackboxsw | robjo: you've got quite a few branches of goodness up for us to review. Any prioritization on those branches or just take them as we can? | 16:47 |
| rharper | I don't think there are issues w.r.t jinja and shell; they use different variable escape methods, jinja uses {{ variable/expression }}; and it doesn't consume $ AFAIK, ajorg do you know differently ? | 16:47 |
| blackboxsw | #link https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/334992 | 16:48 |
| blackboxsw | I'm guessing is top of the list | 16:48 |
| ajorg | I saw {instance_id} at https://trello.com/c/AYaCdQyT/21-cloud-init-query-standardized-json-information so I assumed it was being customized to look for { instead of {{ | 16:48 |
| robjo | blackboxsw: The chrony support should probably be the last as it will take longer over all and more back and forth | 16:48 |
| ajorg | (for one thing) | 16:48 |
| ajorg | rharper: also, there's the whole question of the "extends" feature | 16:49 |
| ajorg | We integrated Jinja into an internal tool a few years back and we spent a very long time making sure the loaders did the right thing. | 16:50 |
| blackboxsw | ajorg: I thought I read somewhere that you couldn't exend jinja for custom functions. maybe I was mistaken | 16:50 |
| robjo | I am also not certain that the "re-write everything" on the first go around for chrony is really what we want to do initially | 16:50 |
| ajorg | blackboxsw: I don't think I'm referring to custom functions | 16:50 |
| robjo | That's probably where we ant to end up, but I am not certain that a "step function" approach is in order | 16:51 |
| rharper | ajorg: hrm, I've always seen {{ variable }} or {% expression %}; so maybe blackboxsw can just update the templates; | 16:51 |
| rharper | the examples in the cards | 16:51 |
| ajorg | rharper: sure, that would have helped in this case. | 16:52 |
| robjo | If we do go down the route of the step function I'll need more gudance then in rharper's comments | 16:52 |
| ajorg | blackboxsw: I was referring to the ability of one template to extend another. | 16:52 |
| ajorg | blackboxsw: and the question of where does the engine look when it's asked to extend another template. It can be tricky. | 16:53 |
| blackboxsw | yeah I honestly hadn't gotten past step one of handling the template markup within an existing single template. so this may need a bit of thought/work | 16:54 |
| ajorg | Personally, I'd be a lot happier with limiting things to Python format() templates, even though it means you can't have loops, but I won't get in the way as long as we're cognizant of the problems we can run into by accepting the full power of an advanced engine like Jinja. | 16:55 |
| smoser | i'm not opposed to allowing ## template: python-format | 16:56 |
| ajorg | heh | 16:56 |
| smoser | honestly. | 16:56 |
| smoser | you can pick a differnt name if you dont like that one. | 16:56 |
| smoser | but we already use jinja, so it makes sense to support jinja | 16:57 |
| * smoser has to run. sorry. | 16:57 | |
| rharper | I do feel that supplying the template means the user is opting in; and specifically if we've got a good way to provide dry-run based on a instance.json and a script; that certainly can help folks work out the kinks in the template of their choosing | 16:57 |
| ajorg | I'm really not opposed so much as wary of the extensive power of the thing | 16:57 |
| rharper | ajorg: that's a fair warning; given you've experience here; help drawing the line is most welcome | 16:58 |
| ajorg | I'm trying to think of a way to read in /etc/shadow using Jinja, you know? | 16:58 |
| rharper | well, cloud-init is root anyhow; so, what's the deal with that ? | 16:58 |
| blackboxsw | ajorg: heh, right though you can read that with your runcmd section in #cloud-config :) | 16:59 |
| ajorg | If I can come up with a way to do it that doesn't make it look obvious that I'm doing it, and then post that as something others can copy, or use with #include <url> then I win. | 16:59 |
| rharper | I don't think jinja makes that any more troublesome | 16:59 |
| rharper | folks already wget | bash with shell they don't understand either | 17:00 |
| ajorg | I suspect Jijna makes it more opaque. | 17:00 |
| ajorg | The answer to "what file does Jinja read when I use {% extends foo %}" is a very lengthy "it depends" | 17:01 |
| ajorg | anyway, I've said my piece | 17:02 |
| * ajorg is a bit of a template naysayer. | 17:03 | |
| blackboxsw | +1, there's one in every group. We'll try to keep that in mind as this feature evolves | 17:05 |
| blackboxsw | :) | 17:05 |
| ajorg | nice | 17:06 |
| ajorg | :-) | 17:06 |
| blackboxsw | any pet bugs, new features or burning reviews that need mention? | 17:06 |
| blackboxsw | ajorg: we could do something simple like disable the extends option via policies | 17:07 |
| blackboxsw | it looks like | 17:07 |
| blackboxsw | #link http://jinja.pocoo.org/docs/2.10/api/#policies | 17:08 |
| blackboxsw | or maybe I'm misunderstanding the issue I'll read up more on it | 17:08 |
| ajorg | thanks | 17:08 |
| ajorg | It looked like https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334074 was blocking https://code.launchpad.net/~yeazelm/cloud-init/+git/cloud-init/+merge/331897 but shouldn't be anymore. | 17:09 |
| ajorg | I'll remind Matt to try it again now. | 17:09 |
| blackboxsw | thanks good dela | 17:13 |
| blackboxsw | dela | 17:13 |
| blackboxsw | deal | 17:13 |
| blackboxsw | geez | 17:13 |
| blackboxsw | on that note. I think it's time for coffee | 17:14 |
| blackboxsw | and time to end the meeting | 17:14 |
| blackboxsw | Happy New Year again folks. Good to be back in the office. | 17:14 |
| blackboxsw | thanks again for the chat, until next time.. | 17:15 |
| blackboxsw | #endmeeting | 17:15 |
| meetingology | Meeting ended Mon Jan 8 17:15:30 2018 UTC. | 17:15 |
| meetingology | Minutes: http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-01-08-16.03.moin.txt | 17:15 |
| ajorg | bye now | 17:15 |
| blackboxsw | see you ajorg and robjo have a good one | 17:15 |
| ajorg | oh, actually before I leave, but not necessarily before you leave, what conferences are folks attending this year? | 17:16 |
| blackboxsw | powersj: mind promoting me to opp again? | 17:16 |
| blackboxsw | or rharper | 17:16 |
| blackboxsw | thx rharper | 17:16 |
| rharper | np | 17:16 |
| blackboxsw | I hadn't chosen yet. I was think about LISA or debconf. | 17:17 |
| blackboxsw | cloud foundry looks interesting too. | 17:18 |
| blackboxsw | actually, no it doesn't, to me. | 17:22 |
| ajorg | I went to LISA when it was in Seattle. I'd probably go again if it was in Seattle again. | 18:00 |
| powersj | blackboxsw: debconf is in taiwan this year | 18:15 |
| dpb1 | chad and I went in San Diego, it was a good conference, IMO | 18:16 |
| powersj | blackboxsw: smoser: some fun for you https://paste.ubuntu.com/26348213/ | 18:16 |
| blackboxsw | powersj: guess that narrows down that choice | 18:16 |
| blackboxsw | anyone heard much about http://devopssummit.sys-con.com/general/papers2018east.htm ? | 18:19 |
| blackboxsw | cloud/devops expo | 18:19 |
| blackboxsw | ? | 18:19 |
| blackboxsw | I think marco ceppi went | 18:19 |
| blackboxsw | not sure if it's the 'right' venue | 18:20 |
| === shardy is now known as shardy_afk | ||
| smoser | powersj: i really should upload simplestreams to pypi | 18:34 |
| smoser | and do a release of that. | 18:34 |
| powersj | smoser: yes you should | 18:34 |
| smoser | powersj: responded to all of them. | 18:55 |
| powersj | smoser: thanks! | 18:56 |
| powersj | after lunch I'll go through 'em | 18:56 |
| smoser | rharper: or blackboxsw https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/335108 review woudl be appreciated | 19:01 |
| blackboxsw | will do. just posted archive of meeting notes https://cloud-init.github.io/status-2018-01-08.html#status-2018-01-08 | 19:11 |
| === blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 1/22 16:00 UTC | cloud-init 17.2 released (Dec 14, 2017) | ||
| dojordan | @blackboxsw @smoser can you guys take a look at the new changes I made? https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341 | 20:55 |
| blackboxsw | smoser: quick discussion point on your maas oauth branch | 21:04 |
| blackboxsw | looking dojordan | 21:04 |
| smoser | blackboxsw: k | 21:38 |
| smoser | blackboxsw: ? | 21:48 |
| blackboxsw | sorry, I missed something? | 21:48 |
| blackboxsw | was working on review comments ATM | 21:48 |
| smoser | "smoser: quick discussion point on your maas oauth branch" | 21:48 |
| smoser | i expected follow up on that | 21:49 |
| blackboxsw | ohh posted on your branch mso | 21:49 |
| blackboxsw | smoser: | 21:49 |
| blackboxsw | just wondered about pulling the required oauth token logic down into OauthUrlHelper | 21:49 |
| blackboxsw | so that other callers (as they grow) could benefit from same behavior... warning and proceed w/out oauth | 21:49 |
| smoser | blackboxsw: yeah, i kind of saw that duplicated also... i think that sounds reeasonable. | 22:11 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!