[09:13] <asdawqweq> Hello everyone, could any one give an advice how to reset cloud-init so it runs normally after next restart. I tried to delte ../instances unfortunatelly does not work, using openstack. Any tips?
[10:44] <asdawqweq> a]\
[13:26] <smoser> asdsad, rm -Rf /var/lib/cloud/
[13:46] <dpb1> hi smoser
[13:59] <smoser> hello dpb1
[14:19] <blackboxsw> good morning. hopefully all made it swiftly back home
[14:52] <smoser> this is wierd
[14:52] <smoser> https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/325424
[14:52] <smoser> causes
[14:52] <smoser> https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/509/console
[14:52] <smoser> there are issues in the MP, but the failures there show that util.append_file() is just broken
[14:52] <smoser> i'm not sure how we've not seen those failures before
[14:54] <blackboxsw> hmm so octal is more strict than %s  in that %s convert None to "None".  So we've been passing in an empty self.args in the past and just not validating the output in TestGenericDistro maybe
[14:55] <smoser> blackboxsw, you're correct there.
[14:55] <smoser> but if you ever call append_file("/tmp", "foo\n") it will just fail
[14:55] <smoser> without that change.
[14:55] <smoser> as that calls write_file with mode=None
[14:56] <smoser> which calls os.chmod("/tmp/myfile", None)
[14:56] <smoser> should have read append_file("/tmp/myfile, "foo\n")
[14:56] <smoser> above
[15:09] <ajorg> It's funny. I submitted https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/325424 as a no-brainer test of the current merge process (last time we were still using bzr).
[15:09] <ajorg> I thought, "surely this can't go wrong."
[15:09] <ajorg> "... it's a two character change"
[15:09] <ajorg> :-)
[15:09] <smoser> :)
[15:11] <smoser> (and you did so 0o3 years ago... thought i'd be clever and write that as octal, but its hard to read)
[15:11] <smoser> looks like some funny ascii art
[15:15] <ajorg> smoser: so if I submit a 2 char fix and you counter with an 8 line fix and a test case, I still get my name on it?
[15:15] <ajorg> ;-)
[15:17] <ajorg> I need to setup an Ubuntu instance to run tox on. Hopefully I can keep carving off time for it.
[15:18] <powersj> ajorg: out of curiosity what OS are you developing on?
[15:20] <ajorg> powersj: Amazon Linux. I'm Amazon's cloud-init maintainer, when I can find time for it.
[15:20] <ajorg> yes, getting unit tests running on Amazon Linux is also somewhere on my TODO list.
[15:21] <powersj> ajorg: cool :) does tox bomb out on Amazon Linux today?
[15:22] <powersj> we recently got them going on centos 6/7, so I was curious how we were doing on other distros
[15:22] <ajorg> powersj: I last ran the unit tests before it was moved to tox (I think?) and a bunch of the tests were failing. I'll give it another try.
[15:22] <powersj> ok thanks!
[16:02] <ajorg> powersj: the unit tests do work on Amazon Linux. Neat-o!
[16:02] <smoser> https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
[16:03] <powersj> ajorg: woohoo! thanks for trying and that is good to know
[16:06] <smoser> ajorg, yeah, i'd keep your name on it.
[16:06] <smoser> :)
[16:07] <ajorg> smoser: heh, okay, verifying fix now.
[16:17] <ajorg> bleh, how did append_file ever work?
[16:18] <ajorg> it passes mode=None... chmod() doesn't accept None
[16:22] <smoser> ajorg, ah. i see.
[16:22] <smoser> in util, 'chmod' is not os.chmod
[16:23] <smoser> and it does:
[16:23] <smoser> if path and real_mode:
[16:23] <smoser>    ... os.chmod()
[16:23] <smoser> so calling None just does nothing
[16:23] <ajorg> oh, okay
[16:25] <smoser> rharper, https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/325402
[16:25] <smoser> that is good ?
[16:25] <rharper> I think I landed that, no ?
[16:26] <smoser> yes. fixed mp to mark that
[16:26] <rharper> ack
[16:31] <smoser> rharper, you want me to grab https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/325404  ?
[16:31] <smoser> i can drop the 'pass' line in pulling it
[16:31] <rharper> yes
[16:31] <rharper> thx
[16:33] <smoser> rharper, do you have a link to the selinux guard python bug ?
[16:33] <rharper> I think that's in the MP or commit message
[16:33] <rharper> https://bugzilla.redhat.com/show_bug.cgi?id=1406520
[16:33] <smoser> yep
[16:33] <smoser> tghanks
[17:03] <ajorg> smoser: not sure if I twiddled the state of https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/325424 correctly
[17:09] <smoser> ajorg, please write a nicer commit message. i'll make the c-i bot re-run on it and then i think it looks pretty good
[17:10] <ajorg> smoser: mind showing me an example of a nicer commit message to emulate?
[17:10] <ajorg> (or should I pick just any from the past?)
[17:11] <smoser> just 'git log'
[17:11] <smoser> basically
[17:11] <smoser> Summary
[17:11] <smoser> <blank line>
[17:11] <ajorg> and that's for the merge commit? Or you want nicer messages on all the commits (or squashed)
[17:11] <smoser> More info
[17:11] <smoser> put it on the 'Commit Message' in the merge proposal
[17:11] <smoser> and then i will squash with that commit message
[17:11] <smoser> make sense ?
[17:11] <ajorg> k, I can do that
[17:13] <smoser> blackboxsw, i'd appreciate a review from you on https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/325424
[17:14] <smoser> powersj, https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-centos-6/32/consoleFull
[17:14] <powersj> smoser: sup
[17:15] <smoser> ^ that job, can you give me an explanation of what it does ? i think that is supposed to be unit test on trunk ? right ? and eventually i think move to use run-centos ?
[17:15] <powersj> smoser: yes, exactly. Unit test + build
[17:15] <powersj> the run-centos is broken atm because it is missing a dependency. I was going to wait on blackboxsw merge for dependencies to fix that up, to remove the hard-coded ones
[17:16] <smoser> ok.
[17:20] <ajorg> smoser: okay, I think I made the commit message better. Please be critical if I failed.
[17:22] <smoser> ajorg, thanks.
[17:22] <smoser> i responded there. will wait for blackboxsw but other than that, great.
[17:26] <ajorg> cool, I'll try to prep a couple more of our smaller fixes today
[17:46] <blackboxsw> smoser: reviewing ajorgen's branch now
[17:47] <blackboxsw> thx BTW ajorg
[17:47] <ajorg> I'm super happy to be pushing our fixes upstream again finally.
[17:48] <ajorg> We have larger ones too, but I want to learn how you like to do things on smaller ones.
[17:54] <blackboxsw> the more the merrier. I'm learning too.
[17:57] <dpb1> ajorg: what are some of the larger ones?
[17:57] <ajorg> dpb1: lemme look so I can give you an example...
[17:58]  * dpb1 nods
[18:00] <ajorg> Ah, best example: I have a large patch for the migrator module that allowed us to seamlessly upgrade from 0.5 to 0.7.x. I had to restructure some things.
[18:01] <ajorg> We also have some patches that add compatibility for how our fork of cloud-init worked before RPM / Yum support was added upstream
[18:01] <ajorg> Some of those may be less useful generally.
[18:02] <dpb1> ajorg: ah, so re-basing those and see what was unique
[18:02] <ajorg> We have a new feature too.
[18:02] <ajorg> Right. Upstream took a different path, but we didn't want our customers to break, so we had to add some compat.
[18:03] <ajorg> We have a new feature (write-metadata) that we use to write out /etc/yum/vars/* to configure repositories.
[18:04] <ajorg> And we have a feature that lets the user choose what level of security updates to install by default.
[18:04] <ajorg> Other than that it's mostly small bugfixes.
[18:04] <ajorg> Some more important than others.
[18:05] <rharper> larsks: do you have a pointer to your downstream systemd unit files for cloud-init?  I'd like to take a look at those instead of fumbling to come up with what you already have working
[18:16] <dpb1> ajorg: that level of security updates to install sounds interesting
[18:23] <smoser> rharper, http://pkgs.fedoraproject.org/cgit/rpms/cloud-init.git/
[18:25] <rharper> smoser: thx
[18:53] <rharper> hrm, we don't have something equivalent  to 'networking-wait-online.target'  ;  the rhel network service starts bringing up interfaces, but the network-online.target only requires that network.target has been reached, so, cloud-init.net runs before networking is completely up
[18:56] <dpb1> rharper: sounds bad
[18:57] <rharper> well, it's a known problem; it's a matter of how to wait effectively
[18:57]  * rharper looks into ifup on rhel 
[19:10] <blackboxsw> smoser: ajorg just finish review on https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/325424   see what you think
[19:11] <blackboxsw> I wanted to use with_logs more if we could instead of mocking logging
[19:11] <blackboxsw> http://paste.ubuntu.com/24843185 specifically for that branch
[19:12]  * ajorg looks
[19:21] <smoser> blackboxsw, did you test your s?
[19:22] <smoser> decode_perms has a somewhat silly signature (which we could change) where you pass in a 'log'
[19:22] <blackboxsw> smoser: ajorg I ran tox on my patch ________________________________________________________________________________ summary ________________________________________________________________________________
[19:22] <blackboxsw>   py27: commands succeeded
[19:22] <blackboxsw>   py3: commands succeeded
[19:22] <blackboxsw>   flake8: commands succeeded
[19:22] <blackboxsw>   xenial: commands succeeded
[19:22] <blackboxsw>   pylint: commands succeeded
[19:22] <blackboxsw>   congratulations :)
[19:22] <blackboxsw> ooops
[19:22] <ajorg> blackboxsw: yeah, that looks good to me, pushing that soon
[19:22] <blackboxsw> smoser: +1 on that thought. I felt the same
[19:22] <ajorg> (to my branch)
[19:22] <smoser>  oh. i see. you did the self.logger
[19:22] <blackboxsw> just didn't want to hold up ajorg's branch for that sentiment (about logging)
[19:23] <smoser> er... i dont see.
[19:23] <smoser> oh.
[19:23] <smoser> i do
[19:23] <smoser> get with the program, smoser
[19:23] <blackboxsw> yeah self.logger comes now out of tests.unittests.helper
[19:23] <blackboxsw> heh
[19:24] <smoser> i say we change the signature
[19:24] <smoser> but i do think you're being overly pedantic/brittle on
[19:25] <smoser> self.assertEqual("WARNING: Undecodable permissions '999', returning default None\n", ...)
[19:25] <smoser> (linke 113 of your paste
[19:25] <smoser> i would (and did) assertIn
[19:26] <blackboxsw> smoser: works for me I was only trying to keep with the fact that len(warnings) was being used in your tests (to assert that we didn't have more than 1 warning in the log). But yes that was overly pedantic. assertIn would work instead of assertEqual when checking logs. we don't really have to count # of warnings etc in unit tests.
[19:27] <ajorg> So http://paste.ubuntu.com/24843292/
[19:27] <smoser> blackboxsw, true.
[19:27] <smoser> i guess i was *as* pedantic
[19:28] <smoser> ajorg, yeah, that looks fine to me.
[19:28] <blackboxsw> +1 from me ajorg
[19:28] <blackboxsw> heh, I just like passing the scapegoat smoser
[19:28]  * ajorg reruns tox because Jenkins is a cruel master
[19:34] <ajorg> k, pushed that latest, anything I need to do on the merge proposal to push it along?
[19:35] <powersj> ajorg: as long as it is in "Needs review" and not "work in progress" you should be good to go. I usually do like to add a comment to the merge so folks see I addresses the changes.
[19:36] <ajorg> powersj: thanks
[21:50] <ajorg> so I'm confused by the distros / os families thing. I've got 'amazon' added to the 'redhat' os family, and there's a module (cc_resolv_conf.py) that has distros = ['fedora', 'rhel', 'sles'] ...
[21:50] <ajorg> I thought that changing that to distros = ['redhat', 'sles'] would enable it for 'amazon', but it doesn't.
[21:53] <rharper> ajorg: the distro tag is mostly for documentation, IIRC, if you want a module to run, then you need to update /etc/cloud/cloud.cfg ; in there the modules list for each stage (local, init, config, final) is what controls which modules run
[21:54] <ajorg> rharper: I get stages.py[INFO]: Skipping modules ['resolv-conf'] because they are not verified on distro 'amazon'.  To run anyway, add them to 'unverified_modules' in config.
[21:54] <rharper> interesting, if amazon is like rhel, then you shouldn't needed it per the documentation which suggests that sysconfig to manage resolv.conf
[21:56] <rharper> but it sounds like you can add a 'unverified_modules: ['resolv_conf'] to the user-data to make it work
[21:56]  * rharper looks up unverified-modules in the code
[21:56] <ajorg> rharper: we don't use that specific module by default. we had a customer specifically ask that we enable it for them
[21:56] <rharper> ack
[21:56] <ajorg> There's another module (cc_spacewalk) that has 'redhat' in the distros list. My hunch is that it doesn't work on 'rhel'
[21:56] <rharper> yeah, looks like the stages code will check
[21:57] <ajorg> rharper: I'm less worried about how to let a customer us an unverified module than I am about understanding how that distros list in a module is used to decide if it's verified.
[21:58] <ajorg> If I'm wrong that 'redhat' is supposed to enable both 'rhel' and 'fedora' then I'll be on my merry way.
[21:58] <rharper> I'd say it get's set in merge of contributions
[21:58] <ajorg> I'm trying to figure out if there's a bug or a misunderstanding
[21:59] <rharper> ajorg: it's likely a bug here, the distro names and variants can mismatch, a distro variant of 'redhat' should match 'rhel'
[21:59] <rharper> ajorg: it's likely a bug here, the distro names and variants can mismatch, a distro variant of 'redhat' should match 'rhel'
[22:00] <rharper> sorry for the dupe
[22:00] <ajorg> I'll try to find out if it's something that broke recently.
[22:01] <rharper> ok
[22:01] <ajorg> ah, will have to revisit it tomorrow though, got a meeting
[22:01] <rharper> python -c 'import platform; print(platform.linux_distribution()[0].lower())'  that's what gets the distro value, and then in cloudinit/util.py there's a mapping of variants
[22:02] <ajorg> be back tomorrow, thanks for help today!