[00:13] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Add importlib for python 2.6 *only*  https://review.openstack.org/197691
[00:39] <harlowja> ok seems good to go ^ smoser ^
[00:39] <harlowja> all fixed up
[12:17] <smoser> Odd_Bloke, do you have some time today that we could look at the reporting stuff ? i'd like to make some progress on that.
[12:17] <Odd_Bloke> smoser: Potentially; looking at relatively important partner stuff at the moment.
[12:21] <smoser> Odd_Bloke, how do i make jenkins re-test something ?
[12:25] <Odd_Bloke> smoser: Pushing up a new patchset should do it automatically.
[12:25] <smoser> yeah, but i think theres a way to say "re-run".
[12:25] <smoser> ohwell.
[12:25] <Odd_Bloke> harlowja will probably know, when he wakes up in 3 days or so. :p
[12:26] <Odd_Bloke> smoser: Why do you need to rerun without any changes?
[12:26] <smoser> i wasnt sure how the py26 stuff was fixed. thought it was actually not related.
[12:35] <openstackgerrit> Scott Moser proposed stackforge/cloud-init: pip2distro: work in progress: build tox env for specific targets  https://review.openstack.org/197671
[12:56] <Odd_Bloke> smoser: Ah, good point about using setup in timeit.
[12:56] <Odd_Bloke> OTOH, we _aren't_ using the regex more than a handful of times, so I think my examples were more valid than they appeared.
[12:56] <Odd_Bloke> But it was a minor point; I don't particularly mind which we do. :)
[13:59] <smoser> harlowja, please ping when you wake up.
[13:59] <smoser> really want to resolve the py26 stuff today.
[14:00] <smoser> https://review.openstack.org/#/c/197691/
[14:01] <smoser> Odd_Bloke, i have no objection to using unittest2
[14:01] <smoser> i dont fully understand:
[14:01] <smoser> This would mean we couldn't use testtools' TestCase class, but we would still be able to use their matchers in our assertions, if we so wished (http://testtools.readthedocs.org/en/latest/for-test-authors.html#assert-that-function).
[14:02] <smoser> claudiupopa, do you have any objection to unittest2 ?
[14:02] <claudiupopa> does it have assertRaises as a context manager? ;-)
[14:02] <claudiupopa> No, no objection.
[14:05] <Odd_Bloke> smoser: So we're using testtools.TestCase, which inherits directly from unittest.TestCase.
[14:05] <Odd_Bloke> smoser: It is therefore incompatible with using unittest2.TestCase as our root test case.
[14:05] <smoser> ah. ok.
[14:05] <claudiupopa> Why are we using testtools? What benefits does it bring us?
[14:06] <Odd_Bloke> ^ I'm curious about this as well.
[14:06] <Odd_Bloke> (I think testtools has useful stuff, I just don't think we're using any of it ATM :p)
[14:07] <smoser> i have no attachment to it
[14:07] <smoser> it blames to
[14:07] <smoser>  http://paste.ubuntu.com/11810295/
[14:08] <smoser> is setUp and tearDown stuff it provides ?
[14:09] <smoser> guess not
[14:10] <claudiupopa> Nope, that's in unittest as well.
[14:33] <smoser> ok, we really need to resolve this today, as all tests fail on on the py26 gate right now.
[14:54] <smoser> Odd_Bloke, did you see my comment at https://review.openstack.org/#/c/170257/4/cloudinit/templater.py  ?
[14:54] <Odd_Bloke> smoser: Re: timeit?
[14:54] <smoser> yeah.
[14:54] <smoser> am i wrong ?
[14:55] <smoser> there definitely *is* a pentalty for compiling, but it does make up for itself.
[14:55] <smoser> (assuming it gets used)
[14:55] <Odd_Bloke> smoser: If you use the regex a lot, yeah.
[14:55] <Odd_Bloke> So the truth is somewhere between our two benchmarks.
[14:55] <Odd_Bloke> :p
[14:56] <Odd_Bloke> Mine tests the worst case (we use it zero or one times).
[14:56] <Odd_Bloke> Yours tests the best case (we use it 1000s of times).
[14:56] <smoser> well, but yours has so much noise
[14:56] <smoser> re.compile() seems to take about as much time as 'import re'
[14:57] <Odd_Bloke> The actual truth: we've spent more time discussing/thinking about this than is likely to be saved by selecting the correct option across all instances forever. ;)
[14:57] <smoser> correct
[14:58] <smoser> now lets get thsi stupid py26 stuff settled.
[14:58]  * smoser waits for harlowja 
[15:03] <Odd_Bloke> That guy.
[15:03] <Odd_Bloke> Does he ever wake up.
[15:24] <Odd_Bloke> harlowja: What does testtools actually get us over unittest2?
[15:24] <Odd_Bloke> (Except for ugly exception testing :p)
[15:26] <harlowja_at_home> fixtures integration, having to figure out what to do about py3 (unittest2 only works on py2.x), so we'd end up making a mini testtools anyway, testtools also heavily used throughout openstack so its common ground.... the author is helpful and involved in openstack to
[15:26] <harlowja_at_home> same author owns unittest2 :-P
[15:27] <harlowja_at_home> a bunch of other matchers, https://github.com/testing-cabal/testtools/tree/master/testtools/matchers
[15:27] <harlowja_at_home> integration with https://github.com/testing-cabal/fixtures
[15:27] <harlowja_at_home> ...
[15:28] <harlowja_at_home> so thats my 2 cents
[15:29] <harlowja_at_home> testtols is using unittest2 anyway and its providing the py2/py3 compat
[15:30] <harlowja_at_home> need me to ask the author to jump in here (he's in new zealand) so he's probably not awake yet
[15:30] <Odd_Bloke> harlowja_at_home: lifeless?
[15:30] <harlowja_at_home> yes
[15:30] <Odd_Bloke> harlowja_at_home: So you can use the matchers outside of the TestCase.
[15:31] <Odd_Bloke> (I know this because I wrote the function that supports doing it ;)
[15:31] <harlowja_at_home> ok
[15:31] <Odd_Bloke> I thought unittest2 worked on 2 and 3.
[15:31] <harlowja_at_home> https://hg.python.org/unittest2/file/d091f0086b03/setup.py#l40
[15:32] <harlowja_at_home> so seems not
[15:32] <Odd_Bloke> harlowja_at_home: The description at https://pypi.python.org/pypi/unittest2 says "It is tested to run on Python 2.6, 2.7, 3.2, 3.3, 3.4 and pypy."
[15:32] <Odd_Bloke> *shrugs*
[15:33] <Odd_Bloke> SHURG
[15:33] <harlowja_at_home> lol
[15:34] <harlowja_at_home> guess the classifiers haven't been updated
[15:35] <Odd_Bloke> harlowja_at_home: Maybe because it doesn't make sense to use for later versions?
[15:35] <Odd_Bloke> (Or maybe just olde)
[15:36] <harlowja_at_home> probably
[15:38]  * harlowja_at_home shrugs
[15:38] <harlowja_at_home> lol
[16:17] <harlowja_at_home> anyways, will be off most of today, for some reason they gave us 2nd & 3rd off
[16:25] <smoser> dagummit.
[16:25] <smoser> harlowja_at_home, want to get through this  py2.6 thing
[16:25] <smoser> i dont have strong feelings on fixtures/unittest2/testtools
[16:25] <smoser> but do have strong feelings on jenkins gate job failing :)
[16:25] <smoser> and also my feeling is that less dependencies is better.
[16:26] <harlowja_at_home> smoser, ya, only feeling i have is that this is what the rest of openstack uses, it has some nice helpers, losing a context manager, meh
[16:26] <smoser> also, harlowja_at_home right now onyour branch, if i 'tox -e py27' it doesn't work.
[16:26] <harlowja_at_home> its a test-time-dependency
[16:26] <smoser> complains of the importlib;python_ver==2.6
[16:26] <smoser> or whatever that is.
[16:26] <harlowja_at_home> yup, upgrade pip
[16:26] <smoser> :-(
[16:26] <smoser> upgrade pip is not too easy though.
[16:27] <harlowja_at_home> ya, not much i can do about that one, this change for all these in pbr is forcing this throughout all of openstack
[16:27] <smoser> but i just remembered system i'm running it on is 14.04 where usually i do wily
[16:27] <harlowja_at_home> for better or worse
[16:27] <harlowja_at_home> ya, i've also got my 14.04 vm and now have to uprade pip, sorta blows
[16:27] <smoser> what version do you need ?
[16:28] <smoser> https://launchpad.net/ubuntu/+source/python-pip
[16:28] <harlowja_at_home> 6+ i think
[16:28] <smoser> wily is 1.5.6
[16:28] <harlowja_at_home> afaik only pip 6+ understands version markers
[16:29] <harlowja_at_home> http://lists.openstack.org/pipermail/openstack-dev/2015-June/067823.html
[16:29] <harlowja_at_home> and such from robert
[16:30] <harlowja_at_home> i'm pretty sure that these changes are gonna force many distros to have a massive pip upgrade
[16:30] <smoser> are these version numbers the same ? 1.5.6 is in wily, but i need 6?
[16:30] <harlowja_at_home> 6.0+
[16:30] <harlowja_at_home> they did a huge version number bump, from 1.5 -> 6.0
[16:31] <harlowja_at_home> now its 7.0 i think, lol
[16:31] <harlowja_at_home> * https://pypi.python.org/pypi/pip (7.1)
[16:31] <smoser> ok.
[16:31] <smoser> so i'd kind of like to avoid this.
[16:31] <smoser> the only thing we're needing it for is python2.6 needing importlib
[16:31] <smoser> (at least at this point)
[16:31] <smoser> right ?
[16:32] <harlowja_at_home> ya
[16:32] <smoser> in our tox environment we can easily enough just use a different py26 requirements environment then everything else.
[16:32] <harlowja_at_home> that is possible to
[16:32] <smoser> i think i like that better at the moment.
[16:33] <smoser> than making people (including me) install pip from outside of distro
[16:33] <smoser> it'd be more acceptable if at least wily had a suitable pip
[16:33] <harlowja_at_home> ya, i'm pretty sure now that openstack is using those markers, alot of people are gonna have to upgrade
[16:34] <harlowja_at_home> *for better or worse*
[16:35] <smoser> ok. so my take out of the above is
[16:35] <smoser> a.) use testtools unless Odd_Bloke cries
[16:35] <smoser> b.) remove requirement of newer pip as smoser is a luddite
[16:36] <harlowja_at_home> :-P
[16:36] <smoser> if you're on holiday i can try to do that.
[16:36] <harlowja_at_home> i get it done, but probably heading out soonish
[16:36] <smoser> i just need some sane path to running this on ubuntu, and "download pip from the intertubes" isn't really sane
[16:36] <harlowja_at_home> time to get an upgrade :-P
[16:36] <smoser> harlowja_at_home, you are allowed to not do this today :) its ok for you to celebrate your countries independence from the queen
[16:37] <smoser> country's even.
[16:37] <harlowja_at_home> lol
[16:37] <smoser> (if you only have the one)
[16:37] <harlowja_at_home> obama the queen
[16:37] <Odd_Bloke> smoser: FWIW, using a virtualenv on Ubuntu solves the pip problem.
[16:37] <harlowja_at_home> ya, its still a pita though :-P
[16:38] <harlowja_at_home> create virtualenv, enter virtualenv, install newer pip
[16:38] <smoser> using virutalenv to run tox which runs creates a virtualenv
[16:38] <smoser> i really like the simplicity of 'tox'
[16:41] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Add importlib for python 2.6 *only*  https://review.openstack.org/197691
[16:41] <harlowja_at_home> ok let's see what happens
[16:46] <smoser> what'd you do ?
[16:46] <smoser> you took importlib out ?
[16:47] <smoser> oh. isee.
[16:47] <smoser> but the py26-gate thing wont figure that out, will it ?
[16:51] <smoser> seems to have figured it out;
[16:51] <smoser>  https://jenkins02.openstack.org/job/gate-cloud-init-python26/1/
[17:04] <smoser> i have a feeling that we should have a template engine that uses python formatting.
[17:05] <smoser> as probably immediately more powerful and better documented than 'builtin'
[17:05] <smoser> and no dependencies
[17:06] <smoser> but it might not be sufficent for how we want to expose objects to templating.
[17:10] <Odd_Bloke> The new formatting syntax is pretty comprehensive.
[17:10] <harlowja_at_home> cloud-init-formatter-v1000
[17:10] <harlowja_at_home> lol
[17:10] <Odd_Bloke> Though we should investigate if you can do anything untoward using it. :p
[17:11] <smoser> Odd_Bloke, i dont think i can access object.property
[17:11] <smoser> can i ?
[17:11] <Odd_Bloke> smoser:
[17:11] <Odd_Bloke> smoser: Sure.
[17:11] <Odd_Bloke> smoser: http://paste.ubuntu.com/11811141/
[17:12] <smoser> awesome.
[17:12]  * harlowja_at_home pretty sure i made that possible :-P
[17:12] <smoser> ok. so yeah, i like this.
[17:12] <smoser> harlowja_at_home, you did. you rock.
[17:12] <harlowja_at_home> oh, i think thats the built-in python formatting, nothing i did :-P
[17:12] <smoser> right.
[17:13] <smoser> thats what i'm saying we should offer as a templating option
[17:13] <harlowja_at_home> ah
[17:13] <harlowja_at_home> except afaik it only got good on 3.0 +
[17:13] <harlowja_at_home> https://www.python.org/dev/peps/pep-3101/
[17:14] <Odd_Bloke> Oh, Python 2.6 might be screwed.
[17:14] <harlowja_at_home> and 2.7?
[17:14] <Odd_Bloke> That example is on 2.7.
[17:14] <harlowja_at_home> k
[17:14] <harlowja_at_home> guess they must of backported some of it
[17:15] <harlowja_at_home> or something
[17:15] <Odd_Bloke> Just tested it, and the above works on 2.6.
[17:16] <harlowja_at_home> cool, up to u guys
[17:22] <smoser> i think makes sense to add it
[17:22] <smoser> and will just fail on py2.6 .
[17:22] <smoser> https://review.openstack.org/#/c/197691/5/cloudinit/sources/openstack/httpopenstack.py
[17:22] <smoser> as seen there, you had to make that change as the .format didnt' work on py26 right?
[17:23] <smoser> Odd_Bloke, could you review https://review.openstack.org/#/c/197691/
[17:23] <smoser> it is sane to me.
[17:26] <smoser> the one thing ihave left is that we added the py26 environment, which means 'tox' will fail now on any recent ubuntu. where as previously it would pass ( as we didn't need py26)
[17:29] <harlowja_at_home> u running all the tox envs?
[17:29] <harlowja_at_home> for fun
[17:32] <smoser> well, i want to run 27 and 34 for sure by default
[17:51] <openstackgerrit> Merged stackforge/cloud-init: Add importlib for python 2.6 *only*  https://review.openstack.org/197691
[17:52] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Bring over the 'templater' from bzr  https://review.openstack.org/170257
[17:53] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: pip2distro: work in progress: build tox env for specific targets  https://review.openstack.org/197671
[17:53] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: tools/tox-venv: support running other than ./tools/tox-venv  https://review.openstack.org/195631
[17:54] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Expose api response properties and cache buffer decoding  https://review.openstack.org/195800
[17:54] <smoser> why does tox docs not work
[17:54] <smoser> :-(
[17:55] <smoser> http://paste.ubuntu.com/11811416/
[17:57] <smoser> https://bugs.launchpad.net/oslotest/+bug/1379998
[17:59] <harlowja_at_home> no idea, jump on #openstack-oslo ask pbr folks?
[18:03] <smoser> harlowja_at_home, well. this fixes it.
[18:03] <smoser> but i dont know how that bug reports fixed in oslotest
[18:04] <harlowja_at_home> ya, not sure
[18:04] <smoser> http://paste.ubuntu.com/11811486/
[18:04] <smoser> that fixes it.
[18:04] <smoser> but will break again when 1.3b4 is released unless its fixed there.
[18:04] <harlowja_at_home> ya
[18:05] <harlowja_at_home> ok, i'm off
[18:05] <harlowja_at_home> will be on maybe at night or something, ha
[18:06] <smoser> ok.
[18:09] <openstackgerrit> Scott Moser proposed stackforge/cloud-init: fix 'tox -e docs' by limiting sphinx versions  https://review.openstack.org/198080
[18:13] <openstackgerrit> Merged stackforge/cloud-init: Bring over the 'templater' from bzr  https://review.openstack.org/170257
[18:14] <smoser> if someone else could check my work on https://review.openstack.org/#/c/198080/
[18:14] <smoser> i'd appreciate it. Odd_Bloke claudiupopa ^
[18:16] <openstackgerrit> Merged stackforge/cloud-init: Bring over the 'safeyaml' from bzr  https://review.openstack.org/170252
[18:16] <openstackgerrit> Merged stackforge/cloud-init: tools/tox-venv: support running other than ./tools/tox-venv  https://review.openstack.org/195631
[21:18] <openstackgerrit> Scott Moser proposed stackforge/cloud-init: fix 'tox -e docs' by limiting sphinx versions  https://review.openstack.org/198080
[21:37] <openstackgerrit> Scott Moser proposed stackforge/cloud-init: Expose api response properties and cache buffer decoding  https://review.openstack.org/195800
[21:38] <openstackgerrit> Merged stackforge/cloud-init: fix 'tox -e docs' by limiting sphinx versions  https://review.openstack.org/198080
[21:42] <openstackgerrit> Merged stackforge/cloud-init: Expose api response properties and cache buffer decoding  https://review.openstack.org/195800
[21:44] <smoser> why can't anything be easy.
[21:44] <smoser> attempt at running http://paste.ubuntu.com/11812524/ on vivid
[21:44] <smoser> results in http://paste.ubuntu.com/11812519/
[21:45] <smoser> and, oh joy. py34-coverage and py27-coverage now fail.
[21:45] <smoser> fail as in we dont have 90% coverage.
[21:45] <smoser> Odd_Bloke, ^ claudiupopa ^ feel free to fix that. :)