[13:34] smoser: blackboxsw So where do we stand with https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149 [13:34] From my point of view things become ever more confusing [13:35] if we agree on the format then I will carry the patch that we have in the SUSE package, I need to move forward on that front [13:37] robjo, sorry... we stand where i was in a sprint all last week and delinquent [13:37] if the patch cannot move forward for whatever reasons then so be it, given the current state I do not see how fiddling anymore with this patch set before committing makes any sense as we have comments upon comments upon changes and more comments, I've lost track of it all [13:37] i'll look and/or talk to chad today. oh. looks like chad just commented 11 minutes ago. [13:37] i'll read some [13:38] thanks, [13:42] smoser: and I still cannot reproduce the bot failure locally :( worse yet I do not understand what the log is telling me [13:52] robjo, ok. i have one other thing i need to be looking at at the moment, but someone will get a look at it today [13:52] smoser: thanks [15:30] robjo: I'm backing down on the patch suggestion, but I'll add some inline comments about some of the code in add zypper to reorder a bit of the required/defaulted checks to avoid what feels like a bit of extra work. [15:34] meh robjo you took the patch. sorry 'bout that. nits etc. [15:35] blackboxsw: took some of the changes yes, will look at the next set of comments [15:36] I'm looking over the latest and it feels good. just CI complaints on testing. I'm giving it a thorough review now (not a mid-trevel review) [15:36] mid-travel rather [15:42] question: I thought if my boothook had output I'd see it on the console and in the cloud-init-output.log, but I don't. Is this expected behavior? (on Ubuntu 16.04 LTS) [15:58] ajorg, you should yes. [15:58] hmm, will have to troubleshoot after the meeting then [15:58] thanks [16:02] yikes. [16:02] meeting [16:02] blackboxsw, rharper [16:03] oops I set wrong UTC time [16:03] too early? :-) [16:03] right should be now [16:03] htis is right. [16:03] yeah === blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 10/2 16:00 UTC | cloud-init 17.1 released [16:03] oops [16:04] is the but here, blackboxsw ? [16:04] should that work ? [16:05] bah [16:05] here [16:05] bot [16:05] yep it should [16:05] smoser: I see meetingology [16:05] #startmeeting cloud-init [16:05] Meeting started Mon Oct 2 16:05:47 2017 UTC. The chair is smoser. Information about MeetBot at http://wiki.ubuntu.com/meetingology. [16:05] Available commands: action commands idea info link nick [16:06] #topic Recent Changes / Highlights [16:06] hm.. [16:06] 17.1 released https://lists.launchpad.net/cloud-init/msg00106.html [16:06] thats the biggest thing and only thing i have for this topic [16:06] #link https://lists.launchpad.net/cloud-init/msg00106.html [16:06] horay for a release, thanks to those who contributed. [16:07] \o/ [16:07] 17.2 is set for 2017-12-14 [16:08] https://launchpad.net/cloud-init/+milestone/17.2 [16:08] yay! [16:08] feel free to target bugs to that release. [16:08] #link https://launchpad.net/cloud-init/+milestone/17.2 [16:08] #info please target bugs to the next release [16:09] does that do anyting ? [16:09] in the meeting summary, it does [16:09] smoser: the links showed up in meeting minutes last time [16:09] k. i always expected the bot to tell me that in a pm [16:09] meetingology didn't echo though [16:09] well, smoser may have to do those [16:09] #topic In Progress Development / Highlights [16:09] same. but something is up that needs attention. [16:10] I'll properly handle it when publishing [16:10] cool [16:11] Merge Proposals [16:11] #link http://bit.ly/ci-reviews [16:11] there are some there for sure. i know that robjo has some he's interested in, and i think ajorg's instance-identity deserves a look [16:12] as well as simpletable for 17.2 [16:12] I'd be grateful for both, yes. [16:12] i am looking at the 'networkd' one which is a blocker as currently Ubuntu does not work properly on azure [16:12] we should have more bandwidth this week [16:12] and all sysstemd-networkd systems will not work properly on CloudStack. [16:12] anything else there? [16:13] The task limit increase has been tested and solves our problem, thus merging would be great [16:13] robjo, yeah. athat does seem unlikely to cause issue [16:13] Trello Board [16:13] # https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin [16:13] #link ? [16:13] #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin [16:14] https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+ref/taskLimit that's the branch for the task limit [16:15] Sorry, I can never find the link for the open merge proposals :( problem exists between the keyboard and the chair [16:16] will get eyes on that today robjo as your addZypper is about wrapped [16:16] Bugs [16:16] #link https://bugs.launchpad.net/cloud-init [16:16] just mostly pointers... above. [16:16] for Ubuntu in 16.04, there is one SRU in progress that blackboxsw and i need to verify today [16:16] and then we will be soon looking at doing an SRU of 17.1-ish [16:17] Thanks, still in europe, my day is about done, will pick it up tomorrow [16:17] robjo, link is in topic [16:17] http://bit.ly/ci-reviews [16:17] ok. so open discussion i guess. no w? [16:17] #topic Open Discussion / Office Hours [16:17] we'll hang around and pay attention to pings and discussion fro the next 30 minutes or so. [16:18] and after that, feel free to ping. [16:19] I'm writing a unit test for a bugfix I'm preparing to submit. It works, but it's slow because readurl retries and waits a second each time [16:20] I'm a bit of a mock / patch newb. Anyone can help me see how to patch that argument? [16:20] you can feel free to mock readurl [16:20] and set its return_value and look at calls [16:21] then you dont have to httppretty. [16:21] The only thing I want to change about it is the sleep though. Is there a straightforward way to just change one arg? [16:21] tests/unittests/test_handler/test_handler_chef.py is an example and tests/unittests/test_datasource/test_maas.py [16:21] I think [16:21] Hmm, I could do that, yeah. Would be sufficient for this. [16:22] I'd still like to know if there's an easy way to just change one argument to something that gets called somewhere else. [16:22] you could also mock the time.sleep from url_helper [16:22] hahaha, yes. that's what I need to do. [16:22] ajorg: if you want to mock the sleep you can decorate the test_ method with @mock.patch(time.sleep) [16:22] we're suggesting other ways generally because there isn't :) at least that i know of. [16:23] that's cool, patching time.sleep will do nicely [16:24] is there an integration test that looks at what lands on the console? [16:24] (going back to my question from before the meeting) [16:25] no. there coudl be on the nocloud-kvm backend [16:25] but i think there is no console access currently on lxd [16:25] and i think that we do not collect console access on nocloud-kvm [16:25] but as you suggest we should for sure [16:25] k [16:26] I wondered if systemd might be swallowing my output, or maybe python (boothook in this case is a python script) is making a strange choice when it sets up logging. [16:27] certainly possible; I know there were issues with cloud-init starting before say rsyslog on non-systemd boots [16:28] and the python logging has changed w.r.t the default configuration; cloud-init main sort of expects this transition as it starts up in init-local and it has not yet read the cloud-config for logging configuration yet, so it reads that and then does some replay [16:28] interesting [16:29] ajorg, you're writing to stdout/err with logging from a boothook ? [16:29] at least I would have thought cloud-init-output.log would contain my logs though, since it's more a redirect of stderr and stdout, right? [16:29] yeah [16:29] correct [16:29] this is from a bootcmd ? [16:29] no, #cloud-boothook [16:29] * rharper hasn't used boothook [16:29] hm.. [16:30] boothooks are super useful [16:32] smoser: when do boot_hooks run ? local ? net ? [16:33] I'm not sure yet where stdout/err for boot hooks occur, but you can dump each from the units via: journalctl -u cloud-init-local.service (or cloud-init.service) [16:33] "This is the earliest hook available. Note, that there is no mechanism provided for running only once" [16:34] I'm checking the code now (local/pre-local maybe)? [16:34] yeah, saw that; it runs in-image scripts programs [16:34] I did that too, and don't see anything. [16:34] (journalctl) [16:34] you should see some output [16:34] but not seeing your hook in there ? [16:34] I can tell by other means that the hook ran [16:34] just don't see that it printed anything [16:35] my prime suspect if systemd can't do something bad here is that python makes some decision about not actually logging. [16:35] and I need to be more explicit that it should log [16:36] so, the boot_hook will sub out each part, I see no capture on the stderr/stdout ; so I would expect those to go to whatever is currently capturing those; [16:37] s/sub/subprocess [16:38] yup, that's what I expect too, and what I recall seeing on Amazon Linux [16:38] since it's using cloud-init's util.subp, there *should* be a debug level message saying 'Running command %s with allowed return codes %s' which maches a path to the boothook [16:38] I would expect to see that in the cloud-init.log [16:39] http://paste.ubuntu.com/25661692/ [16:39] yup, I do see that [16:39] the output of script, I *think* should go to cloud-init-output.log [16:39] i'm testing ^ on serverstack now. [16:39] i verified the commands run on lxd but not look at console there. [16:39] thanks [16:40] actually.. yeah, BOOTHOOK does run, but doesnt seem to have stdout tied to same place as bootcmd [16:41] oh? [16:41] I'm slightly surprised it's not just me [16:42] http://paste.ubuntu.com/25661714/ [16:42] it really should be in the journal/console if stdout is not redirected to a file, cloud-init-local service [16:43] that reproduces in lxc [16:44] smoser: in your instance on serverstack, do you see your BOOTHOOK in journalctl -b -u cloud-init-local.service ? [16:44] ajorg, please go ahead and open a bug. [16:44] cool (not cool :-P) [16:44] will do [16:44] # journalctl -b -u cloud-init.service | grep BOOT [16:44] Oct 02 16:37:41 a5 cloud-init[121]: BOOTCMD: Mon, 02 Oct 2017 16:37:41 +0000: a5 [16:44] ^ that is from lxd [16:45] and what about the welcome message from cloud-init ? [16:45] i think not. [16:45] bootcmd does: util.subp(cmd, env=env, capture=False) ; where boot_hook does: util.subp([filepath], env=env) [16:46] yeah, so boot_hook definitely swallowed. and should not be. [16:47] yeah [16:49] https://bugs.launchpad.net/cloud-init/+bug/1720841 [16:49] Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Undecided,New] [16:50] Meanwhile I need to setup more explicit logging from that boothook anyway, so I'm okay. [16:51] #link https://bugs.launchpad.net/cloud-init/+bug/1720841 [16:51] Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Medium,Confirmed] [16:55] Here's another one, which I've confirmed with a unit test and proposed a patch: https://bugs.launchpad.net/cloud-init/+bug/1720844 [16:55] Ubuntu bug 1720844 in cloud-init "UrlError from #include aborts stage" [Undecided,New] [16:55] (what is this #link thing you're doing there?) [16:55] well, the bot is supposed to care and do somethign with it. [16:56] but /me thinks it doesnt work right :) [16:56] ajorg: since meetingology is still active, the theory is it is keeping track of links during this meeting [16:56] (during the meeting) [16:56] ah, okay [16:57] I think simpletable is completely ready to merge, btw. Any objections? https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525 [16:57] The theory is it writes the meeting notes out for us so we can publish to https://cloud-init.github.io [16:57] #link https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525 ;-) [16:57] geg [16:57] heh [16:58] ajorg: I think we were good on that changeset, and we only wanted to wait post 17.1 cut to avoid potential regression [16:59] great [16:59] I'm +1 on that will give it a spin today and then I think we can land it [17:01] blackboxsw, ajorg my thoughts on the simpletable... [17:01] here it comes... [17:01] :-P [17:01] :) [17:02] i would like to have a machine friendly output available [17:02] human friendly is good, but machine friendly solves the actual goal of writing the stuff. [17:02] So I modified it to display *exactly* as prettytable did [17:02] oh really. [17:02] wow [17:02] which is admittedly not machine friendly [17:02] not especially [17:02] then i guess i can't object at all. [17:02] but it's at least as good as prettytable [17:03] in that its backwards compat [17:03] right. [17:03] ^ this was my goal, to get you to not object [17:03] i still think we should probaly additionally write some machine friendly json [17:03] agreed, that would be better [17:03] i tihnk having something human friendly is good though [17:03] as i know *I* look at that output [17:04] it meets my goal of not depending on prettytable [17:04] and parsing json would be less nice [17:04] yeah [17:05] so based on your assertion that it outputs the same as pretty table, i have no objections. only future hopes. [17:05] and i do agree dropping pretty table is nice [17:05] winning [17:06] smoser: yeah I was wondering how we generally expect people/machines to parse cloud-init-output.log. Right now it's kindof hard to do machine parsing of cloud-init-output.log. [17:07] blackboxsw, yeah. cloud-init-output.log is not intended to be machine friendly. [17:07] do we know already of non-human consumers of the formatting in cloud-init.output.log? [17:07] but the console actually has value in having machien friendly things on it. [17:08] my feeling is if you have access to /var/log/cloud-init.log, then you could very easily have written whatever you wanted to another file that was purely machine friendly. [17:08] was wondering where we would intend to dump machine-friendly json [17:08] yeah [17:08] but the console (/dev/ttyS0) boot log is different [17:08] in that it can give you a couple things you'd not find easy access to [17:09] a.) ssh public keys for the system (providing out of band communication of this data) [17:09] b.) network configuration info: providn useful bits of data on how you might get to this system [17:10] systemd makes it "fun" to get that data to the console in a safe way [17:12] ugh yes. systemd like to stomp all over your consoles. [17:19] ok so did we want to iterate on simpletable to dump json, or just look for cloud-init to write supplementary json files under /run/cloud-init [17:20] .. after we land ajorg's branch [17:21] I think it's a good idea to dump json fragments to console for some things, but I suspect smoser will still not want to break anyone who might be using the tables, so probably best to land my branch as-is [17:23] yeah, i agree with ajorg. for now, we can just take a replacement that drops prettytable [17:23] there are improvments to be done there, but this is a simple win [17:29] we'll go ahead and end meeting here. [17:29] i'll still be around. thanks all! [17:29] #endmeeting [17:29] Meeting ended Mon Oct 2 17:29:26 2017 UTC. [17:29] Minutes: http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-10-02-16.05.moin.txt [17:29] blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664 [17:29] would appreciate input there. [17:30] looks like links got handled by meetingology, not infos. ok will fix that for next meeting [18:27] blackboxsw, http://paste.ubuntu.com/25662210/ [18:27] that was my "mark-released" that i used for cloud-init [18:28] powersj, ^. mentioned we'd want to put that somewhere and then can improve on it for other things too [18:28] mainly i'm using pastebin and irc logs as a persistent storage system right now [18:29] smoser: thx [18:37] smoser: powersj pushing it to qa-scripts [18:37] on the review too [18:58] ajorg: landed simpletable [19:01] \o/ [19:01] thank you [19:09] robjo, sorry I forgot to hit approve button w/ final comments https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149 === blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 10/16 16:00 UTC | cloud-init 17.1 released [19:10] smoser: powersj rharper time/date check on next cloud-init meeting in topic [19:10] k [19:10] so I don't mess that up again [19:11] % date --utc -d "2 weeks" [19:11] Mon Oct 16 19:11:22 UTC 2017 [19:12] :) just checking that we weren't all up to something else (trips/early Halloween etc.) [19:12] I think we are good [19:12] lol [19:12] should be fine [19:14] blackboxsw, [19:14] $ python3 -c 'from cloudinit.simpletable import SimpleTable' [19:14] Traceback (most recent call last): [19:14] File "", line 1, in [19:15] ModuleNotFoundError: No module named 'cloudinit.simpletable' [19:15] i think somehow you didn't get the simpletable.py file added [19:15] bah, merge fail. grabbing it [19:17] pushed [19:18] going fresh checkout again to be sure [19:21] tox passes clean master [19:22] starting off the week right [19:22] powersj, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331670 [19:23] if you review that i'll get the other branches [19:23] by 'get', i mean i'll just commit to the ubuntu packaging branches. [19:37] smoser: approved w/ nits https://code.launchpad.net/~smoser/curtin/trunk.doc-interface/+merge/331608 [19:38] wrong channel [19:40] CI isn't happy at the moment... ERROR: Failure: ImportError (No module named simpletable) [19:40] smoser: looking [19:40] +1 [19:46] powersj: just pushed a fix [19:46] b3acdff390329c8091b880b490e1f27441a7a486 [19:47] I had missed it when merging ajorg's branch [19:47] I forgot to 'git add' the new files [19:47] oh? [19:48] blackboxsw, did you say you did https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664 [19:48] i dont see a comment [19:49] sorry ajorg, I shouldn't have hilighted you. it was PEBKAC on my merge of your branch [19:49] all good now [19:49] blackboxsw: did I prepare that MP in some wrong way that lead to the mistake? [19:49] nope, all my fault [19:49] won't happen next merge [19:49] k [19:50] no judgement [19:50] <- the judgement is all in this guy's head [19:50] smoser: I hadn't yet commenting now, wrapped up your curtin branch and robjo's, now the dhcp [20:07] only comment I have so far smoser is that the sort order should be reversed I think for systemd's/leases directory (isn't higher # more recent? ) [21:13] testing meetingology for a sec [21:14] #startmeeting Test captured commands [21:14] Meeting started Mon Oct 2 21:14:28 2017 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. [21:14] Available commands: action commands idea info link nick [21:15] #commands [21:15] Available commands: action commands idea info link nick [21:15] #info test info [21:15] #link test link [21:15] #nick test nick [21:15] #idea test idea [21:15] #meetingtopic topic change doesn't work [21:16] #topic do topics work? [21:16] lol [21:16] #action sedition joins in on the fun (but actions don't work) [21:16] ACTION: sedition joins in on the fun (but actions don't work) [21:16] ahh thank you meetingology [21:17] #idea I think only circle of friends commands work here [21:17] guess not [21:17] #subtopic sub topic doesn't work [21:17] #endmeeting [21:17] Meeting ended Mon Oct 2 21:17:34 2017 UTC. [21:17] Minutes: http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-10-02-21.14.moin.txt [21:17] this concludes our broken broadcast. I'm hitting up #meetingology to see what gives [21:18] what an interesting concept [21:18] looks like action, idea, subtopic work [21:18] :) [21:18] i dont think ive ever seen a meeting minutes bot [21:19] heh, it's supposed to make meeting minutes publishing easier... per commands listed here: https://wiki.ubuntu.com/meetingology [21:22] ok looks like most of those commands were properly handled [21:23] just that meetingology doesn't repeat (it's silent) [21:23] ok I think we are good then. [21:34] blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664 updated [21:35] i can hp back in later tonight and upload that or address further feedback [21:49] smoser: looks good on the changeset. [21:49] I'll approve and give it a whirl on on upgrade azure [21:49] upgraded [21:52] && lxd [22:01] blackboxsw: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331678 (if you have time) [22:03] should be able to get that today [22:03] straight forward fix