[02:08] <smoser> ckonstanski: reviewed your two MP, i'll grab those tomorrow.
[02:09] <smoser> note my updates to commit messages we'll use those.  I realize they're long windeded for the changes but i like to read git logs (yeah, i'm that nerdy).
[02:09] <smoser> also, LP: #XXXXX
[02:09] <smoser> is the way you reference a bug.
[02:09] <smoser> Thanks!
[02:20] <ckonstanski> Will do. At Charter se used Issue: #xxxxx. In Openstack it's Closes-Bug: xxxxx. One more to learn.
[02:27] <ckonstanski> The perfect couple of bugs to learn the ropes.
[15:11] <powersj> smoser: here is the merge https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331736
[15:11] <powersj> blackboxsw: ^
[15:19] <blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1725067
[15:19] <ubot5> Ubuntu bug 1725067 in cloud-init (Ubuntu Artful) "cloud-init resizefs fails when booting with root=PARTUUID=" [Medium,Fix committed]
[15:19] <blackboxsw> so this is the bug I think we expect foundations to approve
[16:26] <blackboxsw> hrm smoser rharper https://launchpad.net/cloud-init/+packages shows cloud-init for artful at rev 17.1-25-g17a15f9e-0ubuntu1~17.10.1 yes apt update/apt policy on artful still shows only 17.1-18-gd4f70470-0ubuntu1
[16:27] <blackboxsw> what 'else' do we wait for before we can see a published package version via apt repos once an SRU has landed?
[16:27] <nacc> blackboxsw: it's in artful-proposed
[16:27] <blackboxsw> pending-sru.html still shows the cloud-init package in pending status for artful
[16:27] <nacc> (per rmadison)
[16:27] <blackboxsw> nacc right, but once we pass SRU, doesn't it move to artful-updates?
[16:27] <blackboxsw>  
[16:28] <nacc> blackboxsw: after 7 days
[16:28] <blackboxsw> ahhh
[16:28] <nacc> blackboxsw: depending on the package, there are exceptions, but usally there is a back period to flush out regressiosn
[16:28] <blackboxsw> was missing "what else needs doing" there. Answer:patience child, good things come to those who wait
[16:29] <blackboxsw> thx nacc makes sense
[16:29] <nacc> blackboxsw: http://people.canonical.com/~ubuntu-archive/pending-sru.html
[16:29] <nacc> blackboxsw: it's pending still, but green on bugs meaning all are verified
[16:29] <blackboxsw> yeah that is still sitting in artful (all green)
[16:29] <blackboxsw> yeah
[16:30] <blackboxsw> didn't know if there was another button/person I needed to push (as the artful bugs themselves have been commented by landscape janitor as uploaded
[16:30] <nacc> yep
[18:16] <smoser> blackboxsw and powersj http://paste.ubuntu.com/25825019/
[18:16] <smoser> that is what i have run/am-running fro nocloud tests. and will post to the bugs.
[18:16] <smoser> bug
[18:18] <powersj> smoser: that looks good
[18:18]  * blackboxsw likes it. I'm reviewing powersj kvm branch 
[18:19] <blackboxsw> I find myself wanting a make target for running out integration tests in general so I don't have to lookup what we are running from jenkins
[18:19] <smoser> yeah. something needs easier-ing
[18:21] <powersj> blackboxsw: so like a --proposed and it just does the right thing?
[18:27] <blackboxsw> powersj: yeah like that, I just want to document it somewhere in cloud-init src proper.
[18:27] <powersj> blackboxsw: https://cloudinit.readthedocs.io/en/latest/topics/tests.html
[18:27] <powersj> that page would be a good place for it
[18:27] <powersj> Have an example runs
[18:27] <powersj> can put what we do for nightly testing + proposed testing + ci
[19:39] <smoser> rharper: so.. looking at cyphermox's image
[19:39] <rharper> y
[19:40] <smoser> crazy. if i enable debug logging to the console (change 'WARNING' to 'DEBUG' in the stderr logger in /etc/cloud/cloud.cfg.d/05_logging.cfg)
[19:40] <smoser> then ENOREPRODUCE
[19:40] <rharper> hrm
[19:40] <smoser> but by default, it reproduces ;)
[19:40] <rharper> doesn't change the logging
[19:40] <rharper> and if you just remove the import random from util.py in your image
[19:40] <rharper> what happens ?
[19:41] <smoser> apparently writing stuff to the console constitues entropy
[19:41] <smoser> i'll try that.
[19:41] <cyphermox> smoser: oh, hold on
[19:41] <rharper> oh, yeah
[19:41] <rharper> for sure
[19:41] <rharper> console data
[19:41] <cyphermox> smoser: if you boot once and the reboot you won't necessarily get the same result
[19:41] <smoser> i'm not booting.
[19:41] <smoser> mounting, changin file, unmounting booting.
[19:41] <cyphermox> fair enough
[19:42] <rharper> do we have the KCONFIG delta ?
[19:43] <cyphermox> you'll have to ask kamal for that
[19:43] <rharper> doesn't the image have the config-X file in /boot ?
[19:43] <rharper> that's enough then we can diff from some other image
[19:43] <smoser> removing 'import random' didnt help
[19:44] <cyphermox> rharper: what are you looking for?
[19:44] <rharper> delta
[19:44] <cyphermox> presumably for something in particular
[19:44] <rharper> not yet
[19:44] <smoser> # pastebinit /boot/config-4.4.0-1009-kvm
[19:44] <smoser> http://paste.ubuntu.com/25825393/
[19:47] <smoser> $ diff -u config-4.4.0-59-generic  config-4.4.0-1009-kvm  | pastebinit -f diff
[19:47] <smoser> http://paste.ubuntu.com/25825403/
[19:52] <rharper> no CONFIG_USER_NS ?  aren't the minumals used for containers ?
[20:12] <blackboxsw> powersj: how'd you setup /srv/citest/nocloud-kvm for integration test runs?
[20:12] <blackboxsw> I'm trying to run your nocloud-kvm branch
[20:12] <powersj> mkdir -p /srv/citest/; chown -R jenkins:jenkins /srv/citest
[20:12] <powersj> something like that...
[20:13] <blackboxsw> kk thx. just trying to document shortcomings on a fresh artful install
[20:13] <powersj> basically make sure you own that dir
[20:13] <powersj> :)
[20:13] <powersj> thx
[20:13] <blackboxsw> np
[20:13] <powersj> artful!
[20:13] <powersj> did you upgrade?
[20:13] <blackboxsw> my desktop is artful, laptop1 xenial laptop2 zesty
[20:13] <blackboxsw> :)
[20:14] <blackboxsw> want to play w/ all the pretty ponies
[20:15] <sedition> hi team.
[20:18] <blackboxsw> hi sedition
[20:19] <sedition> y'all are always working hard. one of the most active channels i'm in.
[20:22] <blackboxsw> :) maybe we type too much in IRC :)
[20:22] <sedition> no such thing. i've been an irc junkie for too long
[20:22] <blackboxsw> heh
[20:23] <sedition> i've really been digging cloud-init so i figured i would idle and watch
[20:24] <blackboxsw> that's nice to hear. File bugs/comments/code etc if you find features you think are needed etc. we have that status meeting Monday where we brain dump what we've been working toward.  You are welcome to raise questions concerns there too if there are things you are itching to discuss with a broader audience
[20:25] <blackboxsw> reminds me I need to update the topic
=== blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/30 16:00 UTC | cloud-init 17.1 released
[20:25] <sedition> ok great! i can do that.
[20:25] <blackboxsw> so Monday 16:00 UTC there are a few more eyes on this channel
[20:25] <blackboxsw> :)
[20:25] <blackboxsw> should be every 2 weeks
[20:29] <blackboxsw> grr and I need to publish last meeting logs to the github.io page.
[20:57] <blkadder> Hi all, I am trying to figure out the appropriate incantation to get cloud-init to run multiple commands via pipes while sudo'd to a user. The following works just fine when done as root: " gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export" When I try to invoke it via sudo -u -i user however it simply ignores the entire list of runcmd directives.
[20:57] <blkadder> sudo -u user -i that is...
[20:58] <blkadder> I've also tried sudo -u user -i sh -c which works on the command line but not via cloud-init.
[21:01] <blkadder> Am I relegated to making this a script or something?
[21:06] <rharper> hrm
[21:07] <rharper> blkadder: do you have a paste of your runcmd yaml ?
[21:07] <blkadder> Yes, but I have been fiddling with it endlessly to try to get it to work.
[21:08] <blkadder> How should I post it?
[21:08] <blkadder> It's just the one line that is causing problems...
[21:08] <blkadder> If I comment it out everything else works.
[21:08] <blkadder> And if I do it all as root (no sudo) it all works...
[21:10] <rharper> yeah, just runcmd: and the lines that don't work
[21:10] <rharper> we can generally replace the gpg and stuff with piped echos and greps just to simulate the pipes
[21:10] <rharper> also your cloud-init.log may be helpful in case there's something else going on
[21:11] <blkadder>    - sudo -u jenkins "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"
[21:11] <blkadder> That was one invocation.
[21:11] <rharper> k
[21:11] <blkadder> I have also tried with - sudo -i -u jenkins
[21:12] <blkadder> And even sudo -i -u jenkins sh -c "...foo..."
[21:19] <rharper> try this
[21:19] <rharper> -  [sudo, -u, jenkins, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
[21:19] <blkadder> Will do, thanks.
[21:19] <rharper> that gets put into the shell'fied config in /var/lib/cloud/instance/scripts/*  so you can see the script format there; possible run it by hand in the instance to see if there are other errors
[21:20] <rharper> root@a1:/var/lib/cloud/instance/scripts# cat runcmd
[21:20] <rharper> #!/bin/sh
[21:20] <rharper> 'sudo' '-u' 'jenkins' 'gpg --list-keys --with-colons | grep sub | awk -F: '\''{print $5}'\'' | gpg --armor --output /repo/qbis-repo.gpg.pub --export'
[21:21] <blkadder> That's interesting...
[21:21] <blkadder> That invocation creates a shell script?
[21:22] <rharper> runcmd elements get shellified
[21:22] <rharper> that was the latest cloud-init; possible the older versions just exec them;
[21:22]  * rharper reads the docs and code
[21:23] <blkadder> That worked, thanks.
[21:23] <blkadder> I am doing my best to read docs and googling before bugging the channel. :-)
[21:23] <blkadder> My google fu failed on this particular nuance.
[21:23] <rharper> no
[21:23] <rharper> I know it changed
[21:24] <blkadder> Ahh. ok.
[21:24] <rharper> so the string is execve'd
[21:24] <rharper> the list is shellified
[21:24] <blkadder> Oh no it didn't quite work.
[21:24] <blkadder> The parser works now but...
[21:24] <blkadder> sudo: gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export: command not found
[21:25] <blkadder> Before it wasn't even parsing the yaml blob (errored in logs)
[21:25] <rharper> yea, I can see that, lemme play with the escaping
[21:26] <blackboxsw> asking in channel is fine. it helps us know the probs people are hitting. (and it helps me learn from rharper/smoser)
[21:26] <blkadder> Ok good deal, just didn't want to bother with noob questions...
[21:28] <rharper> ok, try this
[21:29] <rharper> -  [sudo, -u, jenkins, --, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
[21:29] <blkadder> Ok, one sec.
[21:29] <rharper> the -- tells sudo that it doesn't have any more argument parsing to do
[21:30] <blkadder> Thanks for the info I was wondering...
[21:33] <blackboxsw> rharper: I also just validated pipes in general in runcmd with this
[21:33] <blackboxsw>  cat runcmd_pipes.yml
[21:33] <blackboxsw> #cloud-config
[21:33] <blackboxsw> runcmd:
[21:33] <blackboxsw>  - cat /etc/os-release | grep CODE > /tmp/cloud-init-output
[21:34] <blackboxsw> no quotes etc worked for me (I know blkadder's using single quotes etc on a more complex command).
[21:34] <rharper> well, the real trick is knowing that the  - [run, my, cmd] gets shellified, which makes | easy
[21:34] <rharper> otherwise you need the exec'ed command to invoke a shell for you sudo -u jenkins -- sh -c 'do this | eat that | write here'
[21:34] <blkadder> The pipes work fine, it's the sudo that seems to cause issues...
[21:35] <blkadder> If I run that same command that I posted previously as root, no problem.
[21:35] <rharper> right, you're in a shell
[21:35] <rharper> vs. exec, which expects a binary
[21:35] <blkadder> Same result rharper...
[21:35] <rharper> no dice, huh; I don't get that --export command not found
[21:35] <blkadder> Let me double check my yaml file
[21:36] <blkadder>   - [sudo, -u, jenkins, --, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
[21:37] <rharper> I think I answered my own question;  you're sudo'ing to jekins to dump the keys, and then running the rest of that as non-jenkins
[21:37] <blkadder> Ahh, I want to run it all as jenkins. :-)
[21:37] <rharper> ok, that makes it easier
[21:37] <rharper> we'll have sudo run a shell as jenkins
[21:38] <blkadder> I tried that.
[21:38] <blkadder> But you have a smarter way I am sure.
[21:38] <blkadder> I tried sudo -i -u jenkins sh -c ...
[21:38] <rharper> the trouble you'll have is that you can't write to /root as jenkins
[21:39] <blkadder> I don't want to.
[21:39] <blkadder> Am I missing something?
[21:39] <blkadder> I am writing to /repo
[21:39] <blkadder> Which is owned by jenkins.
[21:39] <rharper> oh, right
[21:39] <rharper> that was a local error
[21:39] <rharper> I don't have a jenkins, so I'm subbing something else
[21:39] <blkadder> Ahh
[21:40] <rharper> -  [sudo, -u, jenkins, sh, -c, "gpg --list-keys --with-colons | grep sub | awk -F: '{print $5}' | gpg --armor --output /repo/qbis-repo.gpg.pub --export"]
[21:40] <rharper> I replaced jenkins with ubuntu; but running that gets me gpg running and no complaint about export commnd
[21:41] <blkadder> Hmm...
[21:41] <blkadder> Let me give 'er a shot..
[21:46] <blkadder> And we have key!
[21:46] <blkadder> Thanks!
[21:47] <rharper> \o/
[21:48] <sedition> :)
[21:48] <rharper> blkadder: thanks for stopping by =)
[21:48] <blkadder> So on a side note would you be willing to explain the difference in invocation method?
[21:48] <rharper> yeah
[21:48] <blkadder> I tried sudo -u -i jenkins sh -c which did not work.
[21:48] <rharper> you had -i
[21:48] <rharper> which does the login shell
[21:48] <blkadder> Right.
[21:49] <rharper> which I didn't try; but resets the env and other things
[21:49] <rharper> but, you also didnt use the [, list  ] method
[21:49] <blkadder> I did not.
[21:49] <rharper> IIURC
[21:49] <rharper> so, cloud-init attempted to exec("string")
[21:49] <blkadder> I have used it elsewhere.
[21:50] <rharper> I suppose that should work if quoting and all were correct
[21:50] <blkadder> That's what I am getting hung up on... Works fine on command line but not via cloud-init
[21:52] <rharper> if you have logs of the failure (/var/log/cloud-init*.log ) I might be able to see what went wrong
[21:52] <blkadder> Well, I can certainly reproduce if it is helpful.
[21:53] <blkadder> Not a huge deal since I have a working solution, but just more curious why what works on the command line doesn't work the same...
[21:54] <rharper> well, I think it was the quoting since your were getting yaml parsing errors
[21:54] <blkadder> Ack.
[21:55] <rharper> http://paste.ubuntu.com/25825970/
[21:55] <rharper> you can try that
[21:56] <rharper> using the | to let you inline the command as a string
[21:56] <blkadder> Ahh, cool.
[22:00] <blkadder> Now to figure out why I am seeing read errors in the log...
[22:01] <blkadder> Thanks again rharper.
[22:01] <rharper> np
[22:20] <blackboxsw> powersj: minor comment on https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331736
[22:20] <powersj> blackboxsw: re: tox, correct we pulled that in on a different commit, but that is in master
[22:21] <powersj> https://github.com/cloud-init/cloud-init/commit/aa024e331f8196855fa8d707a2dd7e26e1deab40
[22:21] <blackboxsw> ok yeah I didn't recal until I saw it working on my other master checkout
[22:22] <powersj> blackboxsw: what do you think about the use of /tmp
[22:22] <blackboxsw> powersj: as mentioned inline. use /var/tmp instead
[22:22] <blackboxsw> it doesn't get clobbered as frequently
[22:22] <blackboxsw> by systemd
[22:24] <blackboxsw> it's what cloud-init does too for anything we write out that needs to be somewhat persistent
[22:24] <powersj> *sigh*
[22:24] <powersj> ok :)
[22:25] <blackboxsw> though I don't think it should really affect these tests, best to get rid of any known intermittent failure conditions as that's been the bane of our existing with CI in the last couple weeks
[22:25] <blackboxsw> s/in the last/over the last/
[22:25] <dpb1> why do we need persistence of files in tmp?
[22:25] <powersj> I feel that this should be equally documented then
[22:25] <blackboxsw> by *our* I mean you and rharper
[22:25] <dpb1> past a run that is 1 hour long
[22:26] <blackboxsw> powersj: yeah want me to add it to readthedocs? like runcmd? or write_files ?
[22:26] <dpb1> I don't think so
[22:26] <dpb1> it's not a cloud-init specific thing
[22:26] <blackboxsw> like a Note: don't write to /tmp because of https://bugs.launchpad.net/cloud-init/+bug/1707222?
[22:26] <ubot5> Ubuntu bug 1707222 in cloud-init (Ubuntu) "usage of /tmp during boot is not safe due to systemd-tmpfiles-clean" [High,Fix released]
[22:27] <powersj> dpb1: how else would an end user know about this? I get that maybe it isn't a cloud-init only thing
[22:27] <dpb1> hm
[22:27] <dpb1> because of the after= before=
[22:28] <powersj> as an end user writting user-data I'm not sure I'd ever see that...
[22:28] <dpb1> ok
[22:28] <dpb1> guess so
[22:28] <dpb1> strike my comments
[22:28] <dpb1> yes, we should put it on readthedocs
[22:28] <dpb1> and yes, powersj should fix his branch
[22:28] <blackboxsw> it's likely because cloud-init is so early in the boot process we are hit by this more than anywhere else
[22:28] <powersj> :) I will
[22:28] <blackboxsw> also, I think we probably shouldn't have examples that use this
[22:29] <dpb1> yup
[22:29] <blackboxsw> like https://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd
[22:29] <dpb1> blackboxsw: new bug would be fair
[22:29] <blackboxsw>  wget, "http://example.org", -O, /tmp/index.html
[22:29] <blackboxsw> yeah will add it and can clean up docs
[22:29] <blackboxsw> add it and own it
[22:32] <blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1727876
[22:32] <ubot5> Ubuntu bug 1727876 in cloud-init "Document in RTD that cloud-init can't write to /tmp due races with systemd LP:1707222" [Medium,Triaged]
[22:40] <powersj> blackboxsw: thx for review I'll make changes and let you know when they are up