[00:03] <poolie> spiv, go ahead and land to 2.2
[00:03] <poolie> i might add something to the coding guide about making exceptions error objects?
[00:04] <poolie> i wonder if we should just have call a '.hint' method if it exists?
[00:05] <poolie> actually maybe we do have a rule about repr methods being passive?
[00:06] <spiv> Hmm, a .hint method might work well.
[00:07] <spiv> I'm not sure a written rule about repr methods being passive would have caught this problem much sooner.
[00:08] <poolie> it would have to either trip something in the brain of the person writing it or a person doing a review
[00:08] <poolie> i don't know if writing it in a doc really makes that more likely
[00:08] <poolie> _saying it_ now might
[00:08] <spiv> I'm not sure I realised that repr(BzrError) would invoke that code path when I wrote it.  If someone had specifically asked about it I probably would have guessed correctly, but it's thinking about it in the first place that's the tricky bit...
[00:09] <poolie> putting it in a doc gives people something to point to as "we agreed we wouldn't do this"
[00:10] <spiv> Actually, that reminds me of something I was wondering: perhaps BzrError.__repr__ shouldn't invoke str(self)
[00:11] <spiv> Precisely because people writing subclasses might not realise that __str__ and thus _format need to be as careful as a __repr__ should be.
[00:12] <spiv> I guess in practice it would have only prevented this one problem in however many years BzrError.__repr__ has had that implemenation.
[00:15] <poolie> ah, i didn't realize that calling str was something we added, rather than the default
[00:15] <poolie> i have seen several other bugs to do with complex repr methods
[00:17] <spiv> Yeah, the default repr doesn't call str.
[00:18] <spiv> I wonder if the use cases for repr-of-BzrError are better served by showing repr(self.__dict__) rather than str(self) there?
[00:19] <poolie> could very well be
[00:19] <poolie> i wonder what an uncaught exception does?
[00:22] <spiv> Shows the traceback and str(e)
[00:22] <spiv> http://paste.ubuntu.com/542865/
[00:23] <poolie> i'm just trying to work out why we called repr here
[00:23] <spiv> It's a mystery to me :)
[00:23] <spiv> I guess because it looked nice?
[00:24] <poolie> it's not obviously called by anything on the stack
[00:24] <poolie> perhaps twisted was trying to warn about a (what would you call it) leaked exception?
[00:25] <poolie> how about
[00:25] <poolie> http://pastebin.ubuntu.com/542866/
[00:25] <spiv> Possibly.  But also Twisted turns raised exceptions during callbacks/errbacks into "Failure" objects, as a sort of async error+traceback
[00:26] <spiv> But Failure has a fairly obnoxious behaviour of eagerly taking the traceback and turning it into a string (so it can discard the original traceback and all the probably circular references)
[00:26] <spiv> And that stringification reprs lots of stuff.
[00:29] <spiv> poolie: I'm ok with that.  It's a shame we don't have a good answer for "what about when we want to show hints for that error that may require IO to generate".  But that's just a wishlist thing rather than something important.
[00:30] <poolie> i'd be ok with having .get_suggestions() optionally present
[00:30] <poolie> then the ui code can call it, and perhaps suppress knock on errors while calling it
[00:30] <poolie> ok, i'll put that patch up
[07:26] <vila> hi all !
[07:31] <poolie> hi there vila
[07:32] <vila> poolie: hey !
[07:33] <vila> hmm, where are core dumps dumped ? In the process working dir ? In the user home ? In some fixed place ?
[07:34] <spm> bonjour vila!
[07:34] <vila> :)
[07:34]  * spm exhausts his high school french in one word.
[07:39] <vila> spm: ...on the other hand, I'm sure you can teach me a lot about core dumps :D
[07:39] <spm> apple cores are nicest
[07:44] <poolie> vila, normally they go into the process cwd
[07:44] <poolie> but on ubuntu they're configured into /var/crash by default i believe
[07:45] <poolie> there is a /proc file that controls it
[07:45] <vila> well, in this case it's a karmic VM that says: 'invalid environment block' and the Virtual Box is supposed to have crashed, but I can't find any core...
[07:45] <vila> poolie: ha, one more place to rey
[07:45] <vila> no luck, no /var/crash dir on the host either... puzzling
[07:45] <poolie> they may just be off by default
[07:46] <poolie>  'invalid environment block' is an interesting error
[07:46] <vila> I turned them on in /etc/limits.conf
[07:46] <vila> I turned them on in /etc/security/limits.conf or something
[07:46] <poolie> that probably only takes effect on new logins or something
[07:46] <poolie> try 'ulimit -c'
[07:46] <gorakhargosh> Can bzr determine when a file is renamed/moved to a different directory on Windows?
[07:47] <gorakhargosh> s/determine when/determine whether/
[07:47] <poolie> i think 'bzr mv --auto' will automatically detect it
[07:47] <poolie> but you need to run that command
[07:47] <gorakhargosh> ah, so if i did a mv foobar blah/foobar it wouldn't?
[07:47] <vila> poolie: checked, I did that last Friday, I'm lazily hunting them, but that's the third time the VM can't boot at all anymore which make the problem more... interesting
[07:49] <vila> ouchy, > 100 errors fixed by fsck...
[07:51] <vila> make that >1000.... sounds like I may restore from a backup instead...
[07:51] <poolie> gorakhargosh: if you do that it will also work
[07:52] <poolie> there's a lot of it around :)
[07:52] <gorakhargosh> poolie: so my next question would be: how does it detect that? windows has no concept of inodes. how does it know it's the same file that is moved?
[07:52] <poolie> vila, i was going to ask what you were going today but i think now i know :)
[07:52] <vila> hehe, no, it's only a VM
[07:55] <vila> wt.... stupid hudson ! Don't even try to start this VM, I just told you it was offline !
[07:59] <gorakhargosh> poolie: I'm asking this because I'm writing a directory monitoring library which needs to do this on Windows (detect file movement, that is) and there is no reliable way of determining movement to my knowledge.
[08:06] <gorakhargosh> Without this bit of info, I would think the 'bzr mv' command is not irrelevant. It is very relevant on filesystems like NTFS. It's hard (or I don't know yet how) to detect file movement on Windows.
[08:08] <vila> gorakhargosh: mv --auto looks at the content of the files to guess renames, it didn't track it at the file system level
[08:08] <gorakhargosh> vila: What do you mean by "guess renames"?
[08:09] <gorakhargosh> <— currently downloading the source for bzr
[08:09] <vila> gorakhargosh: we know the last versioned content of a file and we look at the actual content of files for close enough content
[08:10] <gorakhargosh> vila: bzr mv should need to guess renames. it's a command known to bzr that renames. how would bzr determine movement if i used plain old "mv" instead?
[08:10] <gorakhargosh> s/should need/shouldn't need/
[08:10] <vila> mv doesn't guess, mv --auto does
[08:10] <gorakhargosh> vila: ah
[08:11] <gorakhargosh> ah
[08:11] <gorakhargosh> also, by the latter "mv" i don't mean the bzr subcommand. i mean the UNIX mv tool.
[08:13] <vila> gorakhargosh: I meant 'bzr mv'
[08:13] <gorakhargosh> vila: yep
[12:12] <DonDiego> hi everybody
[12:12] <DonDiego> i patched the bzr text_checker plugin
[12:12] <DonDiego> what is the preferred method for submitting a patch?
[12:14] <beuno> DonDiego, merge proposals on launchpad
[12:17] <DonDiego> so i push my local branch to my private space on lp and then connect it to the text_checker lp home with a merge proposal?
[12:19] <jelmer> DonDiego: yep
[12:20] <DonDiego> ok, thx, will do
[18:11] <dev001> hi.  i've added to my .bzrignore, "./sites/mysite/files/".  but when i 'bzr add', that subdir & contents is still added.  wrong sytax on my part?
[19:23] <soren> Can someone smarter than me take a peek at http://hudson.openstack.org/job/nova-tarmac/43312/console ?
[19:23] <soren> Search on that page for "Lockable".
[19:23] <soren> What is that?
[19:27] <lifeless> soren: oh
[19:28] <lifeless> either a bug, or a bad bzrlib script that isn't calling 'thing.unlock()'
[19:28] <lifeless> make sure you have the latest release, then file a bug
[19:29]  * soren glances at tarmac
[19:29] <soren> ...and by extension rockstar`.
[19:34] <soren> mtaylor: btw ^
[19:36]  * mtaylor pokes rockstar
[19:53] <lifeless> mgz: can you join #subunit ?
[20:31] <lifeless> jam: hey, around?
[20:31] <jam> hi lifeless, yep
[20:31] <lifeless> wondering if you could do a small favour
[20:31] <lifeless> we had a test failing on windows
[20:31] <lifeless> in lp:testtools
[20:32] <lifeless> rev 158 was broken, 159 should be fixed
[20:32] <lifeless> wondering if you could run the tests on native windows python
[20:32] <lifeless> for those two revs
[20:32] <jam> lifeless: certainly
[20:32] <lifeless> thanks!
[20:33] <jam> lifeless: how does one run the tests
[20:34] <jam> I think I found it
[20:34] <lifeless> make check
[20:34] <lifeless> or look at the makefile
[20:36] <jam> lifeless: "make check PYTHON=.../python26" has 2 failures at r158, but 1 failure at r159
[20:36] <lifeless> interesting!
[20:36] <lifeless> what are the test ids
[20:37] <jam> Failure for 159: http://paste.ubuntu.com/543271/
[20:39] <lifeless> ahha
[20:39] <lifeless> ok we didn't know about that one
[20:39] <lifeless> jam: what was the additional failure in 158 ?
[20:42] <jam> lifeless: testtools.tests.test_testresult.TestNonAsciiResults.test_unprintable_exception
[20:42] <jam> and testtools.tests.test_testresult.TestNonAsciiResultsWithUnittest.test_unprintable_exception
[20:42] <lifeless> ah, you didn't see a spinner test failure in either case?
[20:42] <jam> nope
[20:43] <jam> r157 has the same failures
[20:43] <lifeless> and this is with native python ?
[20:43] <jam> yep
[20:43] <jam> python 2.6
[20:43] <lifeless> its bizarre that 159 would fix testtools.tests.test_testresult.TestNonAsciiResults.test_unprintable_exception
[20:43] <jam> well, now I'm getting the failure again
[20:43] <jam> maybe there is a race?
[20:43] <lifeless> perhaps
[20:44] <lifeless> mgz: ^ wrote those tests
[20:44] <jam> is there a randomize?
[20:44] <lifeless> I was hoping to see a fix for a threaded twisted support thing.
[20:44] <lifeless> no, I don't think so
[20:44] <jam> lifeless: I might also need to have appropriate deps installed. how many tests should be running?
[20:45] <lifeless> oh, good point!
[20:45] <lifeless> 585 with no skips
[20:45] <jam> lifeless: test suite at r159 runs 100% clean on python2.5
[20:45] <jam> yeah, I'm getting 585 both ways
[20:45] <jam> of course, r158 also passes on py2.5 fails on py2.6
[20:45] <lifeless> mgz: https://bugs.launchpad.net/testtools/+bug/689858
[20:46] <lifeless> jam: can you import twisted successfully in those pythons ?
[20:46] <jam> lifeless: no twisted for py2.5, there *is* twisted for py2.6
[20:46] <jam> but both claim 585 tests ran and no skipping
[20:47] <lifeless> grr :(
[20:47] <lifeless> we need a skip summary like bzr does
[20:48] <jam> lifeless: Ran 585 tests in 1.091s
[20:48] <jam> seems awfully fast
[20:48] <jam> and -v doesn't show any actual "running test XX"
[20:49] <lifeless> Ran 585 tests in 0.950s
[20:49] <lifeless> testtools test suite is reasonably healthy
[20:49] <lifeless> we could bear to make it a bit faster I think
[20:49] <jam> lifeless: well, I miss stuff like "-v" printing what tests are running and individual status
[20:49] <lifeless> yeah
[20:49] <lifeless> agreed
[20:49] <jam> Verbose doesn't seem very, verbose
[20:50] <lifeless> real    0m1.221s
[20:50] <lifeless> real    0m1.512s
[20:50] <lifeless> time() with parallel/serial
[20:52] <jam> lifeless: anyway, I can't confirm or deny what tests are being run, it says everything is fine
[20:52] <jam> at least for py2.5, but I don't have twisted there
[20:53] <jam> so I *know* it is lying :)
[20:53] <jam> installing 10.2 now
[20:55] <lifeless> thank you!
[20:56] <lifeless> if that still shows nothing unusual changing between the two revs, I think we should let mgz look at it
[20:57] <jam> lifeless: 2.5 passes cleanly on both 158 and 159. This time it prompted me that it was trying to open up a port, so installing twisted did change something.
[20:57] <lifeless> interesting, thanks!
[20:58] <lifeless> I think mgz has something different again
[20:58] <jam> lifeless: he used to have 2.4, and then I've seen him run bzr against 2.7
[21:25] <yhager> Hi, we are working on a central SVN repo, and I want a subteam to work on a DVCS repo. I am seeing that git-svn has quite a bit of limitations, and would like to check bzr-svn - as it looks better. Any good references to this? I know git, but don't know bzr, so any pointers to "bzr for gitters" would be appreciated too
[21:26] <spiv> Good morning folks.
[21:27] <spiv> yhager: http://doc.bazaar.canonical.com/migration/en/foreign/bzr-on-svn-projects.html http://doc.bazaar.canonical.com/migration/en/survival/index.html
[21:28] <yhager> spiv: thanks!
[21:28]  * yhager sits back and starts reading...
[21:40] <mgz> lifeless: that's certainly odd. it's possible that the twisted somehow screws up that test, as it involves traceback formatting, but really twisted shouldn't see it.
[21:40] <mgz> jam: if you could get that rev where you were having the random failures again, and try moving twisted off your python path, might answer that question.
[21:41] <lifeless> mgz: can you check if 159 fixes the _spinner issue you reported ?
[21:41] <mgz> lifeless: on the r159 change, that doesn't fix it for my default install
[21:42] <lifeless> mgz: grah, srs?
[21:42] <lifeless> mgz: can you pastebin the code in your threadpool.py then ?
[21:46] <mgz> gra, I hate twisted, the package structure makes no sense to me and the names are all too cute to be meaningful
[21:46] <mgz> (under 'python' for future self-reference)
[21:46] <yhager> Can anybody testify that working with bzr under an SVN repo is almost seamless, as opposed to trying to do the same with git?
[21:47] <mgz> http://float.endofinternet.org/temp/threadpool.py
[21:47] <lifeless> yhager: many folk have praised it
[21:47] <mgz> lifeless: if the result is "upgrade your damn twisted install", I can live with that.
[21:48] <yhager> lifeless: You see, I usually use git, and I want to introduce DVCS to the team. git-svn has put me down in that it is fairly easy to get wrong, especially for DVCS newbies. bzr looks better in that sense..
[21:51] <lifeless> cool
[21:52] <yhager> I am now looking for reassurance :)
[21:52] <lifeless> mgz: that source appears to have the same bug - check _worker and stop()
[21:52] <lifeless> mgz: what I don't understand is why it wouldn't be working
[21:52] <mgz> reading your notes on the bug now.
[21:52] <lifeless> mgz: care to put a print in before the self.threads.remove(ct) in _worker, before/after the threads copy
[21:52] <lifeless> mgz: also try a gc.collect :P
[21:57] <mgz> http://paste.ubuntu.com/543309/ <- like that?
[21:57] <mgz> or do I also need to weakref the ct itself?
[21:58] <lifeless> yeah
[21:58] <lifeless> try that
[21:58] <lifeless> In principle the thread exits to the Threading wrapper function
[21:58] <lifeless> yhager: Lots of folk use bzr-svn and enjoy it
[21:58] <lifeless> yhager: the mailing list has active helpful users of it
[21:59] <lifeless> yhager: and launchpad.net uses it for mirroring many many svn repositories
[21:59] <lifeless> yhager: bzr is somewhat different to git, and some folk love that, others don't :)
[22:00] <mgz> hm, I think I'd need a context switch for that to work, I'm guessing threading.currentThread will mean something in the interpreter is holding onto it.
[22:00] <yhager> lifeless: well, I like git, but I am not religious. If bzr works better for me, I'll use it just as happily (or maybe even more..)
[22:02] <lifeless> mgz: its interesting reading in the core
[22:03] <jam> mgz: I don't think it is twisted. it is python 2.5
[22:03] <jam> vs 2.6
[22:03] <jam> py2.6 always fails
[22:03] <jam> py2.5 never fails with or without twisted
[22:03] <jam> but *with* twisted it prompted my Firewall alert
[22:03] <jam> meaning the tests that were running did something different
[22:04] <jam> (even though the test suite always says "ran 585 tests in 1.0s"
[22:04] <jam> )
[22:06] <mgz> mphf, 2.7.latest works for me though, and that isn't really a complex test
[22:06] <mgz> (it does involve what happens when exceptions are raised from magic methods, but python doesn't change what happens there if it can help it)
[22:07] <jam> mgz: Py2.7 without twisted passes cleanly
[22:07] <jam> looks like 2.6 formats exceptions differently from the rest
[22:07] <mgz> what's your minor version?
[22:07] <lifeless> __repr__ and __str__ got fiddled with
[22:07] <lifeless> :P
[22:08] <lifeless> yhager: so, give it a spin ?
[22:08] <yhager> lifeless: yes, I guess I will. I want to read a bit more first...
[22:10] <mgz> lifeless: failing to make that twisted method do anything different. what exactly is the intention of the workaround? because it seems to me as soon as the `self.threads.remove(ct)` line gets hit we're (maybe) screwed
[22:11] <lifeless> mgz: see stop()
[22:11] <lifeless> mgz: its possible that the thread isn't getting any timeslice at all though
[22:11] <lifeless> mgz: throw some more prints in, and paste the instrumented file/diff so that I can see what you've changed
[22:13] <mgz> ha, joy. extra prints = delay = bug vanishes!
[22:14] <mgz> presumably that's io sync
[22:18] <mgz> okay, os.write saves me.
[22:19] <mgz> lifeless: http://paste.ubuntu.com/543321/
[22:19] <poolie> hi mgz, jam, lifeless
[22:19] <mgz> hey poolie.
[22:21] <mgz> whoops, c/p error, not printing the local threads variable in stop.
[22:21] <lifeless> hi poolie
[22:21] <mgz> fixing that gives me the last line instead as:
[22:21] <mgz> GZ: stop after joining [<Thread(PoolThread-twisted.internet.reactor-1, stopped)>]
[22:23] <lifeless> mgz: three tweaks
[22:23] <mgz> so, the thread does get stopped, but threading.enumerate in the test case still pulls it up.
[22:23] <lifeless> stop before and stop after joining should prit threads
[22:23] <mgz> right, did that.
[22:23] <lifeless> and can you put a print in the for thread in threads loop
[22:24] <mgz> sure.
[22:24] <lifeless> and a print of threading.enumerate() as well Iguess
[22:24] <lifeless> mmm, no that shouldn't be needed
[22:25]  * poolie is curious about the context
[22:26] <lifeless> poolie: https://bugs.launchpad.net/testtools/+bug/666345
 and a print of threading.enumerate() as well Iguess <- already done as well :)
[22:27] <lifeless> mgz: so, how does it look ?
[22:28] <mgz> http://paste.ubuntu.com/543326/
[22:28] <mgz> looks like I need to check out what threading.enumerate does.
[22:29] <lifeless> ok, so we're definitely joining
[22:29] <lifeless> phew
[22:29] <lifeless> and you can see the context switch to the thread triggered by join
[22:29] <mgz> yup.
[22:30] <lifeless> wtf is enumerate returning a stopped, joined thread
[22:31] <lifeless> 'Return a list of all Thread objects currently alive. The list includes daemonic threads, dummy thread objects created by current_thread(), and the main thread. It excludes terminated threads and threads that have not yet been started.'
[22:31] <lifeless> per 2.6 docs
[22:32] <lifeless> 2.4.4 claims the same
[22:32] <lifeless> mgz: try a sleep(0) after the join
[22:32] <lifeless> I think you proposed that in the test case
[22:34] <mgz> yup. that's sufficient.
[22:34] <lifeless> so
[22:34] <lifeless> theory:
[22:34] <lifeless> join() isn't calling the os level pthread_join [the equivalent of, on windows]
[22:35] <lifeless> mgz: do you get this failure on 2.6/2.7?
[22:35] <lifeless> jam: how many cores does your machine have?
[22:35] <lifeless> mgz: ^
[22:35] <jam> lifeless: 2
[22:35] <mgz> I don't have twisted installed for my svn python, 1.
[22:36] <mgz> but this may be some long-fixed threading bug.
[22:36] <spiv> lifeless: I've seen enumerate return stopped, joined threads.
[22:36] <spiv> It's caused me a bit of grief in the past.
[22:36] <spiv> I forget if it's a bug fixed in later upstream, but the practical result is that you have to filter the result of enumerate.
[22:37] <spiv> (if you care about the threads all being alive)
[22:38] <lifeless> spiv: thanks
[22:39] <lifeless> mgz: ok, so I'm very glad we investigated this, as I do believe it would have caused memory pressure issues and potential bugs in large test suites
[22:39] <lifeless> mgz: the fix I've landed fixes that correctness issue (though twisted upstream is still borked if you downsize a threadpool)
[22:39] <lifeless> mgz: so I think:
[22:40] <lifeless>  - check for a fix in upstream python
[22:40] <lifeless>  - if not, file a bug
[22:40] <lifeless>  - we should filter enumerate(), with a comment and a link to the python bugtrackers bug about this.
[22:41] <mgz> just installed 10.20 on 2.7 and don't get failure, which may or may not mean it's fixed.
[22:41] <lifeless> mgz: I suspect its not fixenated
[22:41] <lifeless> otoh perhaps it is :P
[22:41] <mgz> timing stuff is sensitive.
[22:42] <lifeless> http://www.mail-archive.com/python-bugs-list@python.org/msg11648.html
[22:42] <mgz> http://bugs.python.org/issue1703448
[22:43] <mgz> heh, same thing.
[22:43] <mgz> either lifeless or me could pass for a psychic this evening
[22:45] <spiv> Oh look, I filed that.
[22:47] <lifeless> mgz: would love https://bugs.launchpad.net/testtools/+bug/689858 to  be figured out ;)
[22:47] <mgz> metoo.
[22:47] <spiv> Gosh, I filed that 3½ years ago, and fixed within a year.  It's such a shame we still have to cope with it :(
[22:48] <lifeless> mgz: I've pushed a fix, can has confirmation ?
[22:48] <mgz> spiv: you deserve some kind of reward
[22:48] <mgz> lifeless: passes.
[22:49] <lifeless> ok great
[22:49] <mgz> nit: 2.5.2
[22:49] <lifeless> mgz: are you going to look into jams' test failure?
[22:49] <spiv> mgz: I'm just glad I've forgotten whatever horrible thing it was that made me diagnose that.
[22:49] <lifeless> mgz: E-pic shrug :P
[22:49] <lifeless> spiv: bzr server threading tests
[22:49] <mgz> lifeless: I'll branch release26-maint and see if I get anywhere, but I have doubts.
[22:50]  * spiv puts his fingers in his ears and sings "la la la la"
[22:54] <lifeless> grrr
[22:54] <lifeless> python.threading doesn't say whether one must call join() or not. Just 'can call it'
[22:58] <lifeless> spiv: http://twistedmatrix.com/trac/ticket/4770
[23:05] <mgz> jam: http://paste.ubuntu.com/543343/ <- is there anything else you can suggest?
[23:12] <mgz> hm, something's up with the importing.
[23:18] <mgz> okay, twisted tests working, but I can't repo jam's failure.
[23:23] <lifeless> jam: still playing wow at all?
[23:41] <lifeless> mgz: oh
[23:41] <lifeless> mgz: i finished my parallel test stuff for testrepository