[00:00] <wgrant> Night.
[00:01] <wgrant> lifeless: Could you please send both the branches on https://code.edge.launchpad.net/~wgrant/launchpad/+activereviews off to ec2?
[00:03] <lifeless> night jml
[00:03] <lifeless> wgrant: url me up
[00:04] <wgrant> lifeless: https://code.edge.launchpad.net/~wgrant/launchpad/bug-661109-buildable-architectures/+merge/38529 and https://code.edge.launchpad.net/~wgrant/launchpad/publisher-release-cleanup/+merge/38530
[00:07] <lifeless> any special flags?
[00:08] <wgrant> lifeless: The latter is incremental but has a bug of its own, so probably not.
[00:08] <wgrant> Oh, hm, the bug might not be linked.
[00:09] <wgrant> It is.
[00:10] <lifeless> first one is spinning up now
[00:11] <lifeless> ‽
[00:11] <lifeless> Traceback (most recent call last):
[00:11] <lifeless>   File "bin/test", line 73, in <module>
[00:11] <lifeless>     assert os.environ['STORM_CEXTENSIONS'] == '1'
[00:11] <lifeless>   File "/usr/lib/python2.6/UserDict.py", line 22, in __getitem__
[00:11] <lifeless>     raise KeyError(key)
[00:11] <lifeless> KeyError: 'STORM_CEXTENSIONS'
[00:12] <lifeless> funky
[00:12] <lifeless> 9fixed)
[00:14] <lifeless> -ugh-
[00:14] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/ftests/pgsql.py", line 263, in tearDown
[00:14] <lifeless>     self.dropDb()
[00:14] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/ftests/pgsql.py", line 321, in dropDb
[00:14] <lifeless>     cur.execute('VACUUM pg_catalog.pg_shdepend')
[00:14] <lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/ftests/pgsql.py", line 108, in execute
[00:14] <lifeless>     return self.real_cursor.execute(*args, **kwargs)
[00:15] <lifeless> psycopg2.InternalError: right sibling's left-link doesn't match: block 260 links to 259 instead of expected 314 in index "pg_shdepend_depender_index"
[00:15] <wgrant> ... pardon?
[00:15] <wgrant> That's a new one.
[00:15] <lifeless> data corruption in pg
[00:16] <wgrant> Looks like it.
[00:18] <wgrant> lifeless: Both branches are running reasonably happily now?
[00:27] <lifeless> just started second one
[00:28] <wgrant> Thanks.
[00:31] <lifeless> and my pg is repaired.
[00:32] <wgrant> By recreating everything?
[00:32] <lifeless> no
[00:32] <lifeless> you reindex pg_catalog.pg_shdepend in single user mode
[00:33] <wgrant> Ah.
[00:33] <lifeless> I had to lookup how to do single user mode in 8.4
[00:33] <lifeless> command line changed :)
[00:40]  * wgrant cleans up the next 6 branches.
[01:10] <LPCIBot> Project devel build (120): STILL FAILING in 1 hr 33 min: https://hudson.wedontsleep.org/job/devel/120/
[02:02] <lifeless> wgrant: hey
[02:03] <lifeless> http://dev.launchpad.net/Contributions?action=diff&rev1=306&rev2=307
[02:03] <lifeless> lists jcsackett as outside the lp team
[02:03] <lifeless> this is bogus ;)
[02:03] <LPCIBot> Project db-devel build (74): STILL FAILING in 4 hr 10 min: https://hudson.wedontsleep.org/job/db-devel/74/
[02:03] <lifeless> likewise danilo, ian booth, and benji
[02:04] <wgrant> lifeless: Yeah, and a few others.
[02:04]  * wgrant fixes.
[02:06] <lifeless> thanks
[02:10] <lifeless> wgrant: is the script in the lp tree, that does that ?
[02:10] <wgrant> lifeless: utilities/community-contributions.py
[02:11] <wgrant> (kfogel used to run it on a cron job, but that obviously doesn't happen any more.)
[02:16] <lifeless> oh, we should do that
[02:17] <lifeless> should be easy to put it in a service account on dosium
[02:24] <wgrant> lifeless: All looks reasonably sane now.
[02:27] <wgrant> Missed one.
[02:27] <wgrant> Curses.
[02:33] <lifeless> :>
[02:46] <lifeless> the sweet sound of deleted code vanishing
[02:47] <wgrant> What are you eliminating?
[02:47] <lifeless> ftests.harness
[02:47] <wgrant> Aha.
[02:47] <lifeless> I didn't mean to
[02:48] <lifeless> but I needed to make LaunchpadTestSetup be used as an instance not a magical stateless beasite
[02:50] <lifeless> and in the course of that I found only one user each of its subclasses
[02:50] <lifeless> with those gone it was trivial to do stub's XXX and move LTS to layers.py
[02:52] <wgrant> Heh.
[02:54] <wgrant> lifeless: Thanks. Will you lp-land it?
[02:59] <lifeless> already spinning up
[03:00] <lifeless> my databasefixture branch (pushing now) has the cleanup I mentioned
[03:01] <lifeless> pushed
[03:02] <lifeless>  10 files changed, 106 insertions(+), 296 deletions(-)
[04:28] <wgrant> Yay, both succeeded.
[04:28] <wgrant> lifeless: Thanks.
[04:30] <StevenK> You mean something actually passed ec2?
[04:31] <StevenK> Dear ohloh, import faster.
[04:31] <StevenK> It's doing 10k every 12 hours or so.
[04:32] <wgrant> StevenK: It was sitting in a queue for most of yesterday.
[04:32] <wgrant> It took more than a month the first time, but I think their bzr importer is better now.
[04:32] <StevenK> Step 2 of 3: Importing source code into database (Running 31872/90554)
[04:32] <StevenK> It has another 4 days, by my hand-wavy estimate.
[04:32] <wgrant> deryck is still beating me by a few revs :(
[04:33] <wgrant> And then I have to defeat henning to make it onto page 3.
[04:33] <StevenK> I've lost count of mine
[04:33] <wgrant> It's going to be hard to rise above jml, I fear.
[04:34] <StevenK> Yes, but jml has been inactive for a while.
[04:35] <lifeless> are both of these ignorable ?
[04:35] <wgrant> Ah, I will pull ahead of deryck in about 10 minutes :P
[04:35] <lifeless> lp.soyuz.windmill.tests.test_archive_packages.TestArchivePackagesSourcesExtra.test_sources_extra_available
[04:35] <lifeless> and
[04:35] <lifeless> lp.translations.windmill.tests.test_documentation_links.DocumentationLinksTest.test_documentation_links
[04:35] <wgrant> lifeless: The latter I haven't seen before.
[04:35] <lifeless> [<Thread(Thread-74, started daemon 47049195071248)>]
[04:35] <lifeless> [<Thread(Thread-183, started daemon 47678012303120)>]
[04:35] <wgrant> But the former has been happening for ages.
[04:36] <lifeless> is it 'retoss at ec2'
[04:36] <StevenK> That's the same symptom as the leak I fixed.
[04:36] <lifeless> or direct to pqm time ?
[04:36] <wgrant> lifeless: Which branch?
[04:36] <lifeless> bzr+ssh://bazaar.launchpad.net/~lifeless/launchpad/paralleltest
[04:36] <lifeless> bzr+ssh://bazaar.launchpad.net/~lifeless/launchpad/paralleltests
[04:36] <wgrant> The first one is ignorable. I'd also be ignoring the second one, depending on what you changed.
[04:36] <StevenK> But I'm bloody stumped if I want to print thread counts during every windmill test.
[04:36] <wgrant> Mmm. Not sure.
[04:36] <lifeless> I added a file
[04:37] <lifeless> which nothing calls
[04:37] <lifeless> and has a few unit tests of its own
[04:37] <lifeless> and I bumped the fixtures dep
[04:39] <wgrant> I'd run it again for safety, personally.
[04:39] <StevenK> lifeless: Keep in mind that Hudson {db-,}devel tests fail with similar errors.
[04:39]  * StevenK needs to pester mars again.
[04:39] <wgrant> but I haven't seen the test_documentation_links one before.
[04:40] <StevenK> wgrant: Go back through Hudson's history -- they keep changing.
[04:40] <StevenK> test_worker was the only constant for a while.
[04:41] <wgrant> Come on PQM... work faster.
[04:42] <wgrant> How does it take half an hour to merge?
[04:42] <StevenK> I've been wondering that myself.
[04:42] <lifeless> bzr branch
[04:42] <StevenK> Why doesn't it just merge directly from the remote branch?
[04:42] <lifeless> its how long its taking to do a full clone
[04:43] <lifeless> StevenK: because it runs the precommit checks in a chroot, assuming maximum hostility from the submitter
[04:43] <StevenK> Yes, but we're only hostile towards PQM, not trunk.
[04:44] <lifeless> I can think of a few ways to address it, including 'allow merges to fuck the repository'
[04:44] <lifeless> but you asked why. thats why.
[04:45] <lifeless> if the precommit checks didn't call bzr at all (which they certainly used to) we could use a lightweight checkout
[04:46] <StevenK> lifeless: Out of curosity, what are the pre-commit checks doing? Just 'make'?
[04:46] <wgrant> I presume it's no longer test_on_merge.py.
[04:47] <lifeless> I don't have access to the pqm both these days (haven't for years)
[04:48] <lifeless> so I can't answer that
[04:48] <StevenK> lifeless: Okay, so what did it used to run?
[04:48] <StevenK> Idle curosity is all, so I'm not after a definite answer.
[04:49] <wgrant> Yay, PQM loves me.
[04:49] <lifeless> well it used to do check-on-merge, uhm
[04:49] <lifeless> it ran the tests obviously
[04:49] <lifeless> but also checked lint for the commit, which needed a diff, which needs bzr history
[04:50] <lifeless> another way we might be able to structure it is a http heavyweight checkout of the target branch with/stacked local branch stacked on the target.
[06:09] <wgrant> lifeless: :( you added code to canonical.*
[06:15] <lifeless> wgrant: and?
[06:15] <lifeless> wgrant: I know that there's this big goal to nuke it
[06:15] <lifeless> but I have very limited coding time.
[06:15] <lifeless> Should I a):
[06:15] <lifeless>  make the test suite 16 times faster
[06:15] <lifeless> or b)
[06:15] <lifeless>  move some code around to no practice difference
[06:16] <lifeless> wgrant: or are you referring to testing.parallel? I guess that could be in lp.services.testing or wherever
[06:17] <wgrant> lifeless: testing.parallel, yeah.
[06:17] <wgrant> Altering existing code in there is fine IMO. But adding a new file there is probably bad.
[06:20] <lifeless> shrug
[06:20] <lifeless> can move it later
[06:20] <lifeless> ideally that wouldn't exist at all, but I haven't had time to work on testrepository recently, nor am I sure that everyone uses testr
[06:29] <lifeless> \o/ unique db names
[06:32] <wgrant> Nice!
[06:32] <wgrant> I will be very interested to see how this performs.
[06:34] <lifeless> now, I need to glue the unique db names up to the config system
[06:34] <lifeless> in process and out of process
[06:37] <wgrant> Oh urrgh.
[06:37] <wgrant> Out of process... hm...
[06:38] <lifeless> in process should be easy once I figure out what layer the idiotic configs are read in
[06:46] <lifeless> hmm
[06:47] <lifeless> whups
[06:47] <lifeless>  launchpad_ftest            | robertc  | UTF8     | C         | C     |
[06:47] <lifeless>  launchpad_ftest_1099       | robertc  | UTF8     | C         | C     |
[06:47] <lifeless>  launchpad_ftest_1470       | robertc  | UTF8     | C         | C     |
[06:47] <lifeless>  launchpad_ftest_1579       | robertc  | UTF8     | C         | C     |
[06:47] <lifeless>  launchpad_ftest_929        | robertc  | UTF8     | C         | C     |
[06:47] <lifeless>  launchpad_ftest_978        | robertc  | UTF8     | C         | C     |
[06:48] <lifeless> wgrant: do you happen to know the dead chicken for increasing a vm disk image size?
[06:51] <wgrant> lifeless: libvirt?
[06:57] <lifeless> http://wiki.libvirt.org/page/Tips#Increasing_the_disk_size_of_a_virtual_machine
[06:58] <wgrant> If it's LVM it's easy. qcow2 I don't really know... I normally just create a new image and copy the data across.
[06:59] <lifeless> which is odd, I've have expected to be able to just loopback it
[06:59] <wgrant> Hm?
[07:00] <wgrant> I loopback the new and old devices then mount them both.
[07:02] <lifeless> loopback as a disk, then ext2resize
[07:02] <lifeless> well, fdisk, ext2resize
[07:03] <wgrant> Oh, I just resize the virtual block device then do the rest from within the VM.
[07:03] <lifeless> well, thats what they say on that wiki page
[07:03] <lifeless> it just seems a bit redundant to me
[07:11] <wgrant> lifeless: Can you manually submit that c-c.py branch?
[07:11] <wgrant> It failed with that LayerIsolationFailure.
[07:11] <wgrant> Despite only changing an untested script.
[07:21] <lifeless> what should the mangled commit message be
[07:21] <wgrant> [r=lifeless][ui=none][no-qa] Update community-contributions.py with new name mappings and statuses.
[07:22] <lifeless> and the branch url
[07:23] <lifeless> wgrant: ^
[07:24] <wgrant> lifeless: lp:~wgrant/launchpad/community-contributions-updates
[07:24] <lifeless> sent
[07:24] <wgrant> Thanks.
[07:24] <lifeless> wow, we have some leaky shit here
[07:26] <lifeless> oh, -ha-
[07:26] <lifeless> layers blows up very badly when testSetup goes boom
[07:27] <lifeless> wgrant: if you're interested, rev 11738 pushing now
[07:29] <lifeless> or 739 which has devel for the parallel tests.
[08:00] <lifeless> jml: https://code.edge.launchpad.net/~lifeless/launchpad/databasefixture/+merge/38643 I'm hoping you'll review that today, so I can get some momentum up.
[08:08] <StevenK> lifeless: But it's a Sunday?
[08:09] <lifeless> StevenK: yes
[08:09] <lifeless> and? You can review it if you like; its very mechanical
[08:09] <lifeless> I would love it if you did review it :)
[08:11] <StevenK> I would, but I'm about to run out the door
[09:01] <LPCIBot> Project devel build (121): STILL FAILING in 4 hr 5 min: https://hudson.wedontsleep.org/job/devel/121/
[10:07] <wgrant> lifeless: What did you do about that STORM_CEXTENSIONS thing?
[10:10] <lifeless> make
[10:11] <wgrant> Ah.
[10:11] <wgrant> Odd.
[10:11] <wgrant> Thanks.
[10:28] <jml> hi ho
[10:32] <lifeless> jml: hi
[10:32] <jml> lifeless: I'm just getting to your branch :)
[10:34] <lifeless> awesome
[10:34] <LPCIBot> Project db-devel build (75): STILL FAILING in 4 hr 8 min: https://hudson.wedontsleep.org/job/db-devel/75/
[10:34] <lifeless> I'm polishing the next one - setting a uniqe env variable in BaseLayer
[10:35] <lifeless> after that it will be using a custom ftesting.zcml + config dir, though I need to really understand WTF the overrides we write are
[10:35] <jml> oh
[10:35] <jml> I think I have an answer to your layer question
[10:35]  * jml checks
[10:35] <lifeless> then, in theory, we're down to fine tuning, fatsam etc.
[10:37] <jml> actually, I don't
[10:37] <jml> my guess was going to be that it had something to do with the way zope goes up the inheritance tree for you
[10:38] <jml> but since there aren't setUp or tearDown methods on LayerProcessController, I have no idea
[10:38] <lifeless> I think its because DatabaseLayer has multiple tasks
[10:38] <lifeless> but for external tests environments like windmill, we don't need them all
[10:38] <lifeless> or something
[10:47] <jml> yeah, maybe
[10:47] <jml> lifeless: working through your branch now
[10:47] <lifeless> jml: thank you
[10:52] <lifeless> jml: lp:~lifeless/launchpad/uniqueconfig is where my next thrust is going, if you're interested. It is selfcontained but  not useful until I do config mangling which is why I am not planning on putting it up for review just yet.
[10:52] <jml> lifeless: thanks
[10:52] <jml> this is a big branch
[10:53] <lifeless> I mention it because it alters the branch you are looking at, a little.
[10:53] <lifeless> jml: its nearly all mechanical.
[10:53] <lifeless> uhm
[10:53] <jml> yeah, so I'm seeing :)
[10:53] <lifeless> in fact I think its reasonable to say that its all mechanical
[10:53] <lifeless> static -> instance
[10:53] <lifeless> unused stuff deleted
[10:54] <lifeless> and the one tiny patch, adding dbname dynamic allocation support
[10:54] <jml> ooh, I think I've got to the meat :)
[10:55] <jml> > -        DatabaseLayer._reset_sequences_sql = None
[10:55] <jml> lifeless: why'd you delete that line?
[10:55] <lifeless> because its now held by the _db_fixture
[10:57] <lifeless> the fixture has two uses, calculating the reset sequences logic, and doing test db initialisation and restoration (which uses the sequences *from the template db* - but the choice to do that is held by the layer
[10:57] <lifeless> not by the fixture
[10:57] <lifeless> thats something we can clean up further in future I think.
[10:58] <jml> lifeless: > +            self.dbname = self.__class__.dbname + "_" + str(os.getpid())
[10:58] <jml> lifeless: why self.__class__.dbname and not self.dbname?
[10:59] <lifeless> because self.dbname would be wrong
[10:59] <lifeless> mmm
[10:59] <lifeless> rephrase
[10:59] <lifeless> self.dbname would be more magical
[10:59] <lifeless> I want to be explicit here.
[11:00] <jml> cool.
[11:00] <jml> just wanted to make sure it was deliberate
[11:01] <lifeless> sure
[11:02] <lifeless> DoesNotStartWith sure looks like it could be a simply curry of Not(StartsWith)
[11:03] <jml> I haven't seen it, but probably yes
[11:05] <jml> lifeless: also, I've found a case for which Matchers are not a natural fit.
[11:05] <jml> lifeless: "assert that this Deferred will fail with this exception type"
[11:05] <lifeless> well, matching would match the exception
[11:06] <lifeless> I have one of those I need to port to testtools, its in testrepository atm
[11:06] <lifeless> 'assert that this deferred will raise something matching <matcher>'
[11:06] <jml> return d.addCallback(self.assertThat, Matches(FooError))
[11:06] <jml> fsvo Matches
[11:06] <lifeless> yeah, I guess
[11:06] <lifeless> I spelt it, in regular code as
[11:07] <lifeless> err = self.assertRaises(Exception, thing)
[11:07] <lifeless> self.assertThat(err, MatchesException(...))
[11:07] <jml> right
[11:07] <lifeless> this doesn't seem unnatural to me
[11:07] <jml> well, for deferreds it's a bit awkard
[11:08] <jml> I guess I could add a convenience function like lambda d, *exc_types: return d.addCallback(self.assertThat, FailureMatches(*exc_types))
[11:09] <jml> actually, ugh
[11:09] <jml> it would need to be more complex
[11:09] <jml> because you actually want to addErrback instead, but also add a callback that will fail if it gets called
[11:10] <lifeless> yes
[11:10] <jml> but it can be done
[11:10] <lifeless> but this glue, isn't it directly equivalent to assertThat + assertRaises ?
[11:10] <lifeless> e.g., its the adaption to let the concerns be separated
[11:10] <jml> well...
[11:10] <jml> the issue is that assertRaises doesn't fit Deferreds
[11:11] <jml> and the tricky thing is where to put the deferred glue if not on the base class
[11:11] <lifeless> thats getting interesting ;)
[11:11] <lifeless> assertThat lives on the base class
[11:11] <lifeless> hmm
[11:12] <lifeless> this needs some consideration
[11:12] <lifeless> hey, do you want to review me moving StartsWith and DoesNotStartWith to testtools?
[11:12] <jml> so what I meant is that matchers as a solution for adding extra assertion types without doing inheritance magic aren't a good fit for this particular problem, even if they might be a part of the solution
[11:12] <jml> sure
[11:13] <lifeless> jml: ok, I get that now, and completely agree; matchers are always going to be just a component
[11:13] <lifeless> jml: https://code.edge.launchpad.net/~lifeless/testtools/matchers/+merge/38645
[11:15] <lifeless> jml: I'd like to roll that into an updated testtools snapshot for lp
[11:15] <jml> lifeless: sure.
[11:15] <lifeless> jml: or perhaps even a release, though its late and I don't want to do a release at this time of night (but if you did I could update LP in the morning)
[11:15] <jml> lifeless: yesterday, I was looking into the snapshot bug you filed on testtools
[11:16] <jml> lifeless: I don't know how to fix it
[11:17] <jml> lifeless: https://bugs.edge.launchpad.net/testtools/+bug/613734
[11:17] <lifeless> oh, I remember
[11:17] <_mup_> Bug #613734: Hard to make snapshots for use with buildout <testtools:Triaged> <https://launchpad.net/bugs/613734>
[11:17] <jml> I mean, I can think of ways to fix it
[11:17] <jml> but surely there's already a bog standard way of dealing with this
[11:18] <lifeless> presumably
[11:18] <lifeless> or we could go to commits are releases.
[11:18] <lifeless> if we mad it a little more automated
[11:22] <lifeless> jml: so, a testtools release, or a snapshot? I'm easy.
[11:23] <jml> lifeless: release sounds good
[11:23] <lifeless> jml: do you perhaps feel like doing one?
[11:23] <jml> lifeless: not right at this minute, but I could do one today
[11:23] <lifeless> that would be awesome
[11:24] <lifeless> I'll update testtools in lp tomorrow, rip out the now duplicated code.
[11:24] <jml> cool
[11:24] <jml> lifeless: I'm also thinking of making a web page summarizing a bunch of testing stuff that we've done
[11:25] <jml> lifeless: fixtures, testscenarios, testr, subunit, testtools, tribunal – anything else?
[11:25] <lifeless> testresources
[11:25] <jml> ahh, of course
[11:25] <lifeless> bzrlib.tests
[11:25] <lifeless> python's unittest addCleanup, load_tests - we provided the protocol and discussion, call it 'moral creation'
[11:26] <lifeless> or something catchy ;)
[11:26] <jml> lifeless: good point
[11:27] <jml> wgrant: I almost wasn't going to review the branch you just posted, and then I read the description
[11:27] <wgrant> jml: Hm?
[11:27] <jml> more-a-f-cleanup, re pockets
[11:28] <wgrant> The code is old and revolting :(
[11:28] <wgrant> And I have another 7 or so branches in this series :(
[11:31] <jml> wgrant: that's a good thing :)
[11:31] <jml> wgrant: I added getSuite when I was doing the package branches work. I'm glad it's now actually helping more broadly.
[11:32] <wgrant> jml: Ahhh, I wondered why nobody was using it.
[11:32] <wgrant> I didn't realise it was brand new.
[11:32] <jml> "brand new" meaning early 2009
[11:33] <wgrant> jml: This code is largely unmodified from 2005...
[11:33] <jml> if I did it again now, I might have even added a Suite class.
[11:33] <jml> point.
[11:35] <wgrant> What is the policy on sampledata changes in devel these days?
[11:36] <jml> sodomy non sapiens
[11:37] <wgrant> Every few times I make a change somebody tries to kill me. I guess it's worth a try.
[11:37] <jml> wgrant: I say edit boldly
[11:39] <jml> wgrant: ec2 landing that branch
[11:39] <wgrant> jml: Thanks.
[11:47] <lifeless> wgrant: depends on what you mean by sampledata change
[11:47] <lifeless> wgrant: if you mean 'deleteing a tonne' DOIT
[11:47] <lifeless> if you mean 'adding a bunch'. Uhm, lets talk.
[11:50] <wgrant> lifeless: I'm removing DistroSeries.lucilleconfig in favour of ComponentSelection. But one of the sampledata series is broken, having no ComponentSelections.
[11:50] <wgrant> So I am adding four rows to ComponentSelection rather than rewriting hundreds of tests.
[11:51] <lifeless> doit
[11:51] <lifeless> no question that that is the right thing to do.
[11:51] <wgrant> Thanks.
[11:58] <wgrant> The sampledata regeneration bloats the diff by 700 lines :(
[12:00] <lifeless> wgrant: why, its only four rows you said?
[12:01] <lifeless> wgrant: put them in by hand :)
[12:01] <wgrant> lifeless: The sampledata hasn't been regenerated for a while.
[12:01] <lifeless> wgrant: thats not entirely true
[12:01] <lifeless> what pg version have you used?
[12:01] <wgrant> I can see columns being added.
[12:01] <wgrant> The order is fine.
[12:01] <lifeless> hmm
[12:01] <wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/destroy-distroseries-lucilleconfig/+merge/38648
[12:02] <lifeless> wgrant: its midnight, I'm pumpkining
[12:02] <lifeless> but I smell fishy
[12:02] <wgrant> Some of it is order, you're right.
[12:02] <wgrant> But there is also new stuff.
[12:02] <wgrant> I'm running whatever pg maverick has.
[12:03] <wgrant> 8.4.5
[12:03] <wgrant> The order in my branch is right.
[12:03] <wgrant> It's alphabetical.
[12:03] <wgrant> The old one wasn't.
[12:03] <wgrant> I wonder how that happened.
[12:03] <lifeless> so someone else had a sampledata patch
[12:03] <lifeless> last week
[12:04] <lifeless> and it didn't have columns
[12:04] <wgrant> db-devel has been merged since.
[12:04] <lifeless> hmm
[12:04] <lifeless> gnight ;)
[12:04] <wgrant> Night!
[12:04] <lifeless> wgrant: I have one thought
[12:05] <lifeless> if you care to do it.
[12:05] <wgrant> Sure.
[12:05] <lifeless> wgrant: write a test that the sampledata is in some reference order; pin that to the pg version on ec2 at the moment (so it skips if it can't run)
[12:05]  * lifeless laughs maniacally
[12:05]  * lifeless goes to bed
[12:07] <jml> lifeless: g'night.
[12:25] <jkakar> lifeless: When you wake up it'd be nice if you could review https://code.launchpad.net/~niemeyer/storm/bug-620615/+merge/38459
[12:25] <jkakar> lifeless: It's a blocker for releasing 0.18 which will fix at least two problems you're experiencing with Storm in Launchpad.
[12:43] <LPCIBot> Project devel build (122): STILL FAILING in 3 hr 42 min: https://hudson.wedontsleep.org/job/devel/122/
[12:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Update community-contributions.py with new name mappings and statuses.
[12:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=jml][ui=none][no-qa] parallel testing facility that works with subunit.
[12:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][bug=655690] Replace the messy
[12:43] <LPCIBot> FTPArchiveHandler.release_files_needed dict with a simple
[12:43] <LPCIBot> Publisher.release_files_needed set of suites.
[14:23] <LPCIBot> Project db-devel build (76): STILL FAILING in 3 hr 49 min: https://hudson.wedontsleep.org/job/db-devel/76/
[14:23] <LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11729
[14:23] <LPCIBot> included.
[18:47] <lifeless> jkakar: hi
[18:47] <jkakar> lifeless: Hi!
[18:47] <lifeless> jkakar: sure, for future ref I'm be happy with storm folk reviewing such things
[18:48] <jkakar> lifeless: Yeah, I figured.  At the time, I was the only reviewer, was mentioning it so you could take second if no one stepped up, but therve has now with some interesting questions about it.
[18:51] <lifeless> jkakar: that layer of code is pretty opaque, I have to reread it three times each time I approach it :(
[18:51] <jkakar> lifeless: Me too. :)
[18:59] <lifeless> that makes me a little sad
[19:01] <lifeless> jml: I'm not sure why you're using a branch to do the release. The setup.py changes look trivially fine.
[19:02] <lifeless> jkakar: also thanks for reading the guide/presentation
[19:02] <lifeless> jkakar: its nice to know you felt it made some useful points
[19:03] <jkakar> lifeless: Yeah, I really enjoyed it.  Everything you said aligns with my perspective on the issues (and clarified some things I've thought but somewhat vaguely).  Thanks for putting effort into documenting those ideas. :)
[19:03] <lifeless> jkakar: cool; my pleasure. What things did it clarify for you?
[19:05] <lifeless> jkakar: also, you might like http://pypi.python.org/pypi/fixtures - its starting to mature/shape up
[19:05] <jkakar> lifeless: Specifically the comments about transparency.
[19:06] <jkakar> lifeless: Often, when I review code, I notice problems related to transparency and bring them up.
[19:06] <jkakar> lifeless: I hadn't attributed the term "transparency" to these observations, but seeing your list of things that contribute to transparency turned the light on.
[19:06] <jkakar> lifeless: I will definitely check out fixtures!
[19:07] <lifeless> this is my core-concept-of-testresources extracted to a dedicated library; testresources will be layering on it soonish
[19:08] <jkakar> lifeless: Cool.
[19:20] <lifeless> james_w: if you're around, was DoesNotStartWith unused in LP?
[19:26] <lifeless> jml: https://code.edge.launchpad.net/~lifeless/launchpad/testtools/+merge/38665
[19:49] <LPCIBot> Project db-devel build (77): STILL FAILING in 4 hr 9 min: https://hudson.wedontsleep.org/job/db-devel/77/
[20:33] <LPCIBot> Project devel build (123): STILL FAILING in 4 hr 7 min: https://hudson.wedontsleep.org/job/devel/123/
[20:56] <james_w> lifeless, I don't know
[20:58] <lifeless> james_w: you could review https://code.edge.launchpad.net/~lifeless/launchpad/testtools/+merge/38665 if you like
[20:59] <james_w> lifeless, looks fine to me
[21:11] <lifeless> mwhudson: good morning. https://code.edge.launchpad.net/~lifeless/launchpad/testtools/+merge/38665
[21:13] <mwhudson> lifeless: done
[21:13] <lifeless> thanks!
[21:14] <thumper> morning
[21:18] <lifeless> hi thumper
[21:45] <wgrant> lifeless: So you propose to stick the config in a request annotation?
[21:52] <lifeless> wgrant: I propose to use a single way of finding global state in each environment and have a single abstraction for accessing and changing that
[21:52] <lifeless> wgrant: the IAnnotations participation seems like the obvious place at the moment
[21:55] <thumper> lifeless: 2010-10-16 23:12:26 ERROR   Unhandled exception
[21:55] <thumper>  -> http://launchpadlibrarian.net/57750288/dQsXM3omQy3kDeJMenvfogrK9JQ.txt (FATAL:  Ident authentication failed for user "session"
[21:55] <thumper> FATAL:  Ident authentication failed for user "session"
[21:55] <wgrant> lifeless: Right, that sounds like a good idea.
[21:55] <thumper> lifeless: that is what garbo-daily is finishing with
[21:55] <thumper> lifeless: it is after the script has finished
[21:56] <thumper> lifeless: I'm guessing it is the script reporting bit failing
[21:56] <thumper> garbo-daily is reporting as not having run since 8th of September
[21:59] <thumper> is change assignee never completing for anyone else?
[22:00] <thumper> it seems tragically slow
[22:00] <lifeless> thumper: I haven't tried recently; I had it on my 'this is a little slow' list though, so it wouldn't surprise me. Do you get an oops ?
[22:00] <thumper> lifeless: is succeeded this time
[22:01] <wgrant> thumper: The person picker was timing out a lot last week.
[22:01] <thumper> lifeless: but I don't recall an oops
[22:01] <wgrant> I don't think we have an OOPS ID.
[22:01] <lifeless> wgrant: yes, that oopsed
[22:01] <wgrant> Since you need to get it with Firebug.
[22:01] <lifeless> wgrant: there is a bug
[22:01] <wgrant> Ah, good.
[22:02] <thumper> lifeless: I've had a thought about that weird xmlrpc behaviour
[22:05] <lifeless> cool
[22:28] <jml> lifeless: hello
[22:28] <lifeless> mwhudson: hi, another small branch you might like
[22:28] <lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/fixtures/+merge/38668
[22:28] <lifeless>  4 files changed, 11 insertions(+), 274 deletions(-)
[22:28] <lifeless> jml: hi
[22:29] <jml> lifeless: thanks for the review
[22:30] <jml> lifeless: regarding the deferred support, I'd like to know what you think of assert_fails_with
[22:30] <lifeless> ok, I'll look at that
[22:31] <jml> lifeless: other than that, I'm going to hold off on making unhandled errors in deferreds look different to regular errors for now
[22:32] <jml> lifeless: if you're ok with the assertion helper, then I'll probably land it
[22:32] <jml> lifeless: no use endlessly polishing the darn thing
[22:33] <mwhudson> lifeless: just one question: should the cleanUp methods upcall?
[22:34] <lifeless> mwhudson: which ones?
[22:35] <mwhudson> lifeless: the ones in lib/lp/poppy/tests/test_poppy.py
[22:35] <lifeless> jml: I'll look now
[22:35] <lifeless> mwhudson: oh, it doesn't inherit from Fixture yet
[22:35] <lifeless> mwhudson: that part of it is just tearDown->cleanUp rename
[22:35] <mwhudson> lifeless: ah, ok
[22:35] <lifeless> mwhudson: to harmonise on one protocol
[22:36] <mwhudson> we need to implement meld in the browser and use that on the mp page :-)
[22:36] <mwhudson> lifeless: approved
[22:36] <lifeless> schweet
[22:36] <jml> lifeless: ta
[22:38] <mwhudson> active and inactive tabs look very similar on maverick :(
[22:40] <lifeless> jml: so thats assertRaises spelt differently ?
[22:40] <jml> lifeless: spelt with deferreds
[22:40] <jml> lifeless: and using failureException shenanigans
[22:41] <lifeless> I'd like to tweak the name, the implementation seems ok (but we should consolidate the core of it and assertRaises to a single helper, in principle - that can wait)
[22:42] <jml> subclasses of testtools.TestCase that set different failureExceptions will get errors instead of failures unless they also inform assert_fails_with of their change
[22:42] <jml> lifeless: I'm happy to put a warning in the docstring saying that it's liable to change API.  All of the twisted stuff is experimental as far as I'm concerned
[22:42] <jml> caveat invoker
[22:42] <lifeless> jml: perhaps it could be self.assertFails(exc_types, d)
[22:43] <jml> lifeless: put Deferred know-how into base TestCase?
[22:43] <lifeless> to be more like assertRaises and let it use failureException directly
[22:43] <lifeless> jml: well, a mix in, like TestWithFixtures, perhaps
[22:43] <lifeless> jml: not thrilled, just speculating
[22:44] <jml> yeah, it could be a mix-in, but I'm frowning a little as I read that :)
[22:44] <lifeless> jml: anyhow, I'm fine with adding it as unstable subject to name-and-location-change-etc-etc
[22:44] <jml> lifeless: fwiw, the Twisted version is TestCase.assertFailure
[22:45] <jml> (I don't like that name either.)
[22:45] <lifeless> heh
[22:45] <lifeless> so future stuff: more like assertRaises, support case.failureException
[22:45] <jml> lifeless: noted, thanks.
[22:46] <jml> lifeless: on https://code.edge.launchpad.net/~jml/testtools/run-test-improvements/+merge/38659, you say "Setting the attribute on the object not the method"...
[22:46] <lifeless> yeah
[22:46] <jml> lifeless: the decorator doesn't have access to the object. got any objections to a dict on the class?
[22:46] <lifeless> not at all, I think I even proposed that
[22:46] <jml> good good.
[22:47] <lifeless> (in #subunit the other day)
[22:47] <jml> lifeless: you did propose it, I just wanted to make sure you hadn't changed your mind.
[22:47] <jml> :)
[22:47]  * jml frowns
[22:47] <lifeless> no, its still seems ok given the contract python offers
[22:47] <jml> I bet f.__class__ only works on some of the supported Python
[22:47] <jml> s
[22:47] <lifeless> jml: whats the lowest python we support ?
[22:48] <mwhudson> lifeless: could i get you to look at http://bazaar.launchpad.net/~jameinel/launchpad/lp-service/revision/11235#lib/lp/codehosting/tests/test_acceptance.py ?
[22:48] <jml> lifeless: 2.4
[22:48] <lifeless> jml: you can say 'decorator for functions only works on 2.4 up' if needed
[22:48] <lifeless> mwhudson: looks like a url
[22:48] <jml> lifeless: I think f.im_class works, but not sure about Python 3 support for it
[22:48] <mwhudson> lifeless: it's related to external processes in tests and i just want to check that it doesn't conflict overly with your vision for how they are going
[22:49] <mwhudson> (and also that there's nothing nicer in tree he can use yet)
[22:49] <jml> mwhudson: at some later point, I'm going to find out why jam's thing isn't a twisted daemon. not tonight though, please.
[22:49] <mwhudson> jml: because it forks
[22:49] <mwhudson> and forking with a running reactor sounds terrifying
[22:49] <jml> quite
[22:49] <jml> does it have to fork?
[22:49] <mwhudson> yes, it's basically the whole point of the excercise
[22:50] <mwhudson> jml: i don't understand how f.im_class and f.__class__ can be considered in the same category of thing
[22:50] <lifeless> mwhudson: derive from fixtures.Fixture
[22:50] <jml> mwhudson: they aren't, f.__class__ was a mistake
[22:50] <mwhudson> jml: ok
[22:51] <lifeless> mwhudson: then you can use useFixture as a convenience for use it, or 'with forker:'
[22:52] <mwhudson> lifeless: it's a helper for a layer at the moment, but i guess that we'll be wanting to move away from that in due course
[22:52] <jml> >>> Foo.f.im_class
[22:52] <jml> Traceback (most recent call last):
[22:52] <jml>   File "<stdin>", line 1, in <module>
[22:52] <jml> AttributeError: 'function' object has no attribute 'im_class'
[22:52] <jml> screw you Python 3
[22:52] <lifeless> mwhudson: also, the atexit is bong
[22:52] <jml> why do you hate on players?
[22:52] <lifeless> which reminds me to ask poolie about SIGTERM
[22:53] <poolie> hello jml
[22:53] <jml> poolie: hi
[22:53] <mwhudson> jml: im_class is moderately useless, iirc
[22:53] <poolie> jml i was thinking this morning my dkim branch may not have shown you the changes because it was in the wip state when i pushed
[22:53] <jml> mwhudson: how do you find the class of an unbound method then?
[22:53] <poolie> just a guess
[22:53] <jml> poolie: that was my guess, but I have a different branch that has been wip & shown changes
[22:53] <mwhudson> jml: i didn't think python 3 had unbound methods
[22:54] <lifeless> poolie: hey, so what did you learn about SIGTERM and python
[22:54] <lifeless> poolie: does SIGTERM cause a stack unwind by default ?
[22:54] <poolie> what about it?
[22:54] <jml> mwhudson: how do you find out which scope a function is in then?
[22:54] <poolie> i don't think so, no
[22:54] <lifeless> ahh
[22:54] <poolie> i think by default python has no OS-level handler for it, therefore *zap*
[22:54] <lifeless> poolie: did you add glue for that to bzr?
[22:54] <poolie> i think we did
[22:55] <lifeless> so we'll need that for:
[22:55] <lifeless>  - appserver
[22:55]  * poolie looks
[22:55] <lifeless>  - twisted daemons
[22:55] <lifeless>  - test runner
[22:55] <mwhudson> jml: i guess you don't?
[22:55] <poolie> no, we don't catch it
[22:55] <poolie> i think there's a bug saying we should
[22:56] <lifeless> ok
[22:56] <poolie> it may be
[22:56] <jml> lifeless: twisted installs a sigterm handler
[22:56] <poolie> - we should do it and we just haven't
[22:56] <lifeless> jml: ah, it does? cool.
[22:56] <jml> mwhudson: that sucks
[22:56] <poolie> - catching it is messy; other things like sigwinch have been hugely messy in python
[22:56] <jml> lifeless: it logs & stops the reactor
[22:56] <lifeless> jml: does it unwind properly? pids get deleted etc?
[22:56] <lifeless> jml: there are buildbot failures that smell like SIGTERM.
[22:57] <lifeless> pifiles
[22:57] <jml> lifeless: pidfiles are at a higher level than the reactor. the rephrased question would be something like "does twistd delete pidfiles at reactor shutdown?"
[22:58] <jml> lifeless: unless you are talking about piffle :P
[22:58] <jml> lifeless: and I don't know the answer to that question.
[22:59] <lifeless> jml: I'm talking about machine reboots of the bb slaves leaving a librarian pid behind
[22:59] <jml> lifeless: hmm.
[22:59] <jml> lifeless: I just don't know. it could be a bug in twistd
[22:59] <lifeless> jml: ok
[22:59] <jml> lifeless: but the solution would be fixing twistd not installing a different sigterm handler.
[23:00] <lifeless> jml: sure
[23:00] <lifeless> jml: details ;)
[23:00] <jml> lifeless: you architects have it so easy :P
[23:01] <poolie> what's the context? making sure that external fixtures are torn down?
[23:01] <lifeless> poolie: yes
[23:08] <jml> good grief, Python 3 has also crippled Foo.f.__dict__
[23:11] <jml> nm
[23:14] <lifeless> ok, raising in a sigterm handler works
[23:14] <lifeless> trivially
[23:16] <lifeless> and we have a SIGTERM handling in bin/test already
[23:17] <thumper> ah.. whut? LayerIsolationError: Librarian has been killed or has hung.Tests should use LibrarianLayer.hide() and LibrarianLayer.reveal() where possible, and ensure the Librarian is restarted if it absolutely must be shutdown: [Errno socket error] [Errno 111] Connection refused
[23:17] <thumper> I'm just trying to run a damn test :(
[23:17] <wgrant> Try bin/kill-test-services
[23:17] <wgrant> If it failed the first time.
[23:18] <wgrant> Except that might be broken now.
[23:18] <wgrant> So maybe remove the librarian pid manually.
[23:18] <lifeless> thumper: check you don't have a /var/tmp/fatsam.test/librarian.pid
[23:18] <lifeless> wgrant: it shouldn't be broken
[23:18] <thumper> $ bin/kill-test-services
[23:18] <thumper> Traceback (most recent call last):
[23:18] <thumper>   File "bin/kill-test-services", line 39, in <module>
[23:18] <thumper>     from canonical.librarian.ftests.harness import (
[23:18] <thumper> ImportError: No module named harness
[23:18] <thumper> heh
[23:19] <wgrant> Yeah, that.
[23:19] <lifeless> oh, thats broken.
[23:19] <lifeless> damn thing should be tested shouldn't it.
[23:19] <wgrant> I pointed that out to you last week, IIRC.
[23:19] <lifeless> wgrant: I don't remember
[23:20] <thumper> boo... hiss... FeatureError: Can't remove a sliced result set
[23:29] <jml> g'night
[23:31] <lifeless> night
[23:31] <lifeless> wgrant: that import is just bong
[23:32] <thumper> why does store = IMasterStore(IPerson) fail in the harness?
[23:33] <mwhudson> don't you do it to objects or database classes?
[23:33]  * mwhudson might be wrong
[23:34] <wgrant> thumper: IMasterStore(Person) or IMasterObject(some_person)
[23:34] <thumper> ah...
[23:34] <thumper> that's right
[23:39] <wallyworld> morning
[23:39] <thumper> wallyworld: morning