[00:06] <__monty__> Mmm, socket.error is a child class of IOError, so should the test for socket.error happen before the test for IOError? [00:10] i wonder why it isn't being handled by the existing code that tries to treat IOError as a user error? [00:10] in python 2.6 and later. [00:11] hi mgz [00:11] ah, perfect [00:11] <__monty__> Ah, yes the user was using 2.5 [00:11] in that case you could just list it in the same exception clause, and perhaps add a comment why it's being caught separately [00:12] <__monty__> What do you mean by listing it in the same exception clause? [00:14] having it before with a special note about connectivity would also be reasonable [00:17] i meant having (OSError, IOError, socket.error) [00:17] you should also add a test [00:17] i think this is pretty well covered in test_trace so you can go from those examples [00:19] <__monty__> What would be the best solution, testing it first with a specific message about connectivity or making the message for IOError more general and including OSError and socket.error? [00:22] can you explain the second alternative more? [00:22] <__monty__> The second alternative is what I understood you were proposing. [00:23] add a test that raises socket.errer [00:23] *error [00:23] check it does not print a traceback [00:24] the easiest way to make this test pass is probably to just also catch that exception in the same way as for IOError [00:24] hm [00:24] actually, on trunk, since we need py2.6 now, this bug may be obsolete [00:24] you could just interactively test that it works [00:25] <__monty__> How exactly do I test it? How do I try to get a socket.error? [00:25] look at eg test_format_io_error [00:26] probably a good way to get a realistic one is to try to connect to some unlikely port on localhost [00:26] like tcp port 2 [00:29] it's safe to just construct your own socket.error [00:29] it is [00:30] <__monty__> Yes, but how? [00:30] i feel if you do it yourself, you may miss quirks in how python constructs them [00:30] for instance there have been some bugs along the lines of actual python not setting named attributes on errors the way we expected, iirc [00:30] so if it's easy i'd lean towards generating a real owne [00:30] *one [00:32] <__monty__> I should write a new method in test_trace.py to test if this is no longer a problem in python 2.6, right? [00:32] yes [00:33] so, for instance, in bzrlib.tests.test_trace use test_format_pywintypes_error as a template but raise a likely looking socket.error instead [00:33] ...and I note there I did do what poolie said rather than what I suggested [00:33] in that I called a function that will raise a real error rather than constructing my own [00:35] this will probably cover bug 598158 [00:35] Launchpad bug 598158 in Bazaar "Socket is closed errors are not caught/wrapped." [Medium,Confirmed] https://launchpad.net/bugs/598158 [00:35] well, it's probably a dupe [00:36] in fact both are probably dupes of bug 393173 [00:36] Launchpad bug 393173 in amarok (Ubuntu) "Amarok does not transfer album art to iPod (dup-of: 260091)" [Undecided,New] https://launchpad.net/bugs/393173 [00:36] Launchpad bug 260091 in amarok (Ubuntu) "Amarok does not correctly re-format cover art when transferring to Ipod" [Undecided,Won't fix] https://launchpad.net/bugs/260091 [00:38] sorry bug 393713 [00:38] Launchpad bug 393713 in Bazaar "[master] socket.error causes a traceback" [Medium,Confirmed] https://launchpad.net/bugs/393713 [00:39] some socket errors are deeply unhelpful so some extra text in the advice param is probably a good idea [00:41] <__monty__> As extra text just point to bzr.log or something more helpful? [00:41] socket.socket(0) gives me a nice fast error, but that's probably a bad idea [00:41] depends if the constants are standardised at all I guess [00:42] i think there probably the thing is to catch them close to the point they're raised and rethrow something more meaningful [00:42] like try: socketblah: except socket.error: [00:42] raise something("failed to connect to smtp server %s:%d: %s" % (host, port, e)) [00:44] ehehe, funny mail from jam about the 2.4b3 release [00:44] :) [00:44] that was such a fun sprint [00:46] <__monty__> So, the error should be caught before we have to handle it in trace.py? Or did you mean something else poolie? [00:46] poolie, I've got half a thing written trying to convey that sense of fun, if I mail it to you will you proof read? [00:46] sure [00:46] i'd kind of like to consciously understand what we (or i) did right [00:46] to reproduce it [00:47] it seemed like there was just enough focus on having a hit list, but it wasn't overstructured [00:54] <__monty__> How does the test method work, I make sure an error is raised in the try block, then 'pass' the except block, then 'msg = _format_exception()', then self.assertContainsRe(msg, r"The string for the error here")? [00:57] that sounds right [00:58] <__monty__> Still how do I raise a socket.error that's realistic? [00:59] like mgz said just a socket.socket(0) could be a good start [00:59] __monty__: hmm, perhaps try to call sock.accept without first calling sock.listen? [00:59] o/ spiv [00:59] or socket.socket() then socket.send("something") yells at you for not being connected [01:00] really realistic ones like timeouts are probably not something we want in the test suite :) [01:00] mgz's idea sounds good too. [01:00] Hi poolie [01:01] <__monty__> Which one of these is least likely not to fail? [01:02] they should all fail! :) [01:02] either of those last two should be reliable [01:02] if it turns out they're not we can fix it [01:05] <__monty__> What would the error string look like for the socket.socket() socket.send("string") scenario? [01:06] just try it :) [01:06] python -c "import socket; s = socket.socket(); s.send('string')" :) [01:10] <__monty__> Is this correct for the string? Are the backslashes necessary since it's a raw string? [01:12] <__monty__> r"^bzr: ERROR: \[Errno .*\] Socket is not connected" [01:15] the backslashes are needed if you're trying to match it as a regexp [01:16] <__monty__> so it's not really necessary in this case since the error will always be: socket.error: [Errno 57] Socket is not connected [01:17] i don't think it's guaranteed to be error 57 on all platforms [01:17] spiv could i mumble at you about bug 220464? [01:17] Launchpad bug 220464 in Bazaar "Bazaar doesn't detect its own stale locks" [High,In progress] https://launchpad.net/bugs/220464 [01:19] <__monty__> Then this is good? r"^bzr: ERROR: \[Errno .*\] Socket is not connected" [01:19] yup [01:19] does it pass? [01:19] try running ./bzr selftest -s bt.test_trace [01:20] <__monty__> I get this: bzr: ERROR: No module named testtools [01:20] <__monty__> You may need to install this Python library separately. [01:20] <__monty__> bzr: warning: some compiled extensions could not be loaded; see [01:20] yes, you'll need testtools to run the tests [01:21] on ubuntu or debian, apt-get install python-testtools [01:26] <__monty__> This is the output: [01:26] <__monty__> bzr selftest: /Users/toon/src/bzr/connection-timed-out-364462/bzr [01:26] <__monty__> /Users/toon/src/bzr/connection-timed-out-364462/bzrlib [01:26] <__monty__> bzr-2.4.0dev4 python-2.6.5 Darwin-9.8.0-i386-32bit [01:26] poolie: sure; via which medium? [01:26] <__monty__> ---------------------------------------------------------------------- [01:26] <__monty__> Ran 25 tests in 0.577s [01:26] <__monty__> OK [01:26] <__monty__> Missing feature 'meliae' skipped 1 tests. [01:26] <__monty__> Missing feature 'pywintypes' skipped 1 tests. [01:26] <__monty__> bzr: warning: some compiled extensions could not be loaded; see [01:27] <__monty__> Sorry for the excessive pasting. [01:28] rockin [01:28] spiv, here, or mumble, or the phone [01:28] mostly it's been a while since i touched it so i can't recall if i meant to do anything else before proposing it [01:29] actually if you could just review it now that would be nice [01:29] <__monty__> Should I submit the change now? [01:29] yes please [01:29] poolie: sure, already started reading [01:30] poolie: So far it looks very nice! [01:31] there's another thing connected but just out of scope [01:31] which is to do with trying to release locks as we terminate [01:31] and there was some discussion at the sprint [01:32] which leaned me a bit closer towards a crash-only design [01:32] iow better to just pretty much terminate when something unexpected happens and then pick up next time [01:38] __monty__: all ok? [01:39] <__monty__> Yes, I'm composing the cover letter(?) I should mention all three of the bugs right? [01:39] since they're all dupes only the master is enough [01:39] thanks for fixing thins! [01:39] *this [01:40] poolie: I've emailed you a thingy. [01:40] ta [01:41] that's really nice, thanks! [01:41] i didn't know about bialix's shirt [01:41] i don't have any particular corrections or suggestions [01:41] i'd like to put it on the blog too [01:42] which will make it easy to add links [01:42] do you have access? [01:43] not currently. [01:44] ok sent [01:47] <__monty__> Merge proposed. Thank you for all the help poolie, mgz. :) [01:48] thanks poolie, and thanks monty :) [01:48] np, thanks for your contribution [01:49] <__monty__> See you later! [01:51] okay, gzlist on wordpress is now me. [01:51] darn minimum four character username restrictions :) [01:56] mgz: your nick is too compressed :) [01:57] poolie: branch reviewed, very nice [02:06] thanks spiv [02:07] okay, I think the post should be in the review queue on the blog [02:07] feel free to edit or change if anyone spots something that could do with improving [02:14] mgz, it looks like it is published [02:14] "Someone even" in the second sentence a mistake? [02:15] *how [02:15] dammit, I thought I'd changed that :) [02:15] i published it [02:15] well spotted [02:15] you should still be able to edit it? [02:15] spiv interesting question about reraising incidental errors [02:16] hm [02:16] hm I don't see the edit button [02:20] i'll fix it [02:21] I've got some fixes for your lock breaking branch in return :) [02:22] :) [02:39] mgz are you going to comment on the mp - should i wait for you? [02:39] yes please. [02:39] will try and be fast. [02:40] np [02:40] just didn't want to stand around looking silly [02:48] okay, nearly there [02:49] I don't really need to implement this twice... [02:49] requiring 2.6 means has_ctypes should always be true [03:14] okay, tests pass [03:15] i wonder if i should change this to cast to unicode to get the human form, rather than to str [03:15] to make it safer for py3 and indeed also for unicode values [03:22] it's generally the right thing to do [03:23] but it's a little annoying in Python 2 because it can be dangerous to pass somethings unicode rather than str [03:23] it can break exceptions and such like [03:29] okay, posted branch poolie [03:29] thanks mgz [03:30] I'm a bit past a coherant review, hopefully the changes make sense, otherwise it'll have to wait till tomorrow :) [03:31] sure [03:31] i have some other things to fix in it anyhow [03:31] have a good night [03:31] I got a few of them in the branch I think. Night poolie! [07:02] woohoo, nice blog post mgz ! [07:52] it is isn't it [07:52] hi vila [07:53] i might call it a day though [07:53] poolie: heya [07:53] thanks for the release [07:53] vila i don't know if you saw the mail from https://bugs.launchpad.net/bzr-cvsps-import/+bug/788914 [07:53] Ubuntu bug 788914 in CVS to Bazaar importer "unexpected keyword argument 'pb'" [Critical,Confirmed] [07:53] but it would be nice to review and merge his patch [07:53] *yawn* [07:54] hmm, never touch this code so far :-/ [07:54] they should be shallow fixes [07:54] or maybe we can get lifeless to review them [07:55] anyhow i should really go [07:55] have a good day [07:55] will try to be online later next week [07:55] have a nice weekend [07:55] you too [08:02] vila: you might enjoy https://code.launchpad.net/~spiv/bzr/subprocess-log-in-test-details/+merge/62612 [08:03] It might help demystify some babune failures. [08:04] spiv: amazingly, I just found a trick to avoid the hangs... [08:04] Heh. [08:04] What's the trick? [08:04] redirecting *selftest* stderr at the invocation seems to be enough (how that's inherited is yet unclear, but probably works for all cases where we don't explicitly handle stderr) [08:05] so, what did I collect there ? [08:05] I'm glad you asked: http://paste.ubuntu.com/613637/ [08:06] *2* deprecation warnings... sound very small to block a pipe no ? [08:06] Well, depends on what the pipe's buffer size is [08:06] hmm, 277 chars... [08:06] awfully close to 256 :) [08:06] And how close we were to hitting the buffer limit without the extra noise :) [08:07] Yeah. [08:08] That worryingly suggests that jenkins or whatever is driving selftest in the slave isn't draining both stdout and stderr concurrently. [08:09] my head spins a bit trying to understand the ramifications... [08:09] dosen't clearly explain why your patch for osx/windows didn't work, I should retry that now [08:10] It also appears to disprove my optimism in saying β€œIt's hopefully safe to assume that the terminal or whatever is draining both our stdout and stderr reliably!” :( [08:11] yup, that remark triggered my trick test [08:11] on the other hand, selftest is not supposed to output anything on stderr other than potential issues that needs to be addressed [08:11] does that sound correct ? [08:12] vila: well, deprecation warnings sound like fair game for stderr to me [08:12] yes, as things that should be addressed [08:12] And it's always possible to have lots of them [08:12] hmm [08:12] let me clarify [08:13] we seem to encounter various issues with anything that appears on stderr [08:13] And there's some value in testing that we work in environments that do emit deprecation warnings. [08:13] i.e. with older libraries [08:13] jenkins and subunit being brittle there [08:14] so diverting stderr at the top level and investigating issues found there separately may help us reach a more robust babune [08:15] And in extreme situations (i.e. sufficiently bad bugs) we might even get subprocesses exploding with tracebacks on stderr [08:15] I agree there is valuable information there, I don't mean to redirect to dev/null ! [08:15] Sure I didn't take it to mean that. [08:15] right, my guess is that this should trigger test failures nevertheless [08:15] I'm just saying we can't rely on driving stderr output in babune down to zero either [08:16] We have to be able to cope with it (even if we do manage to get it down to zero normally, bugs happen) [08:16] oh, right, may be not in the short term, but that's still a good target ;) [08:16] I don't feel especially at ease with this deprecation warnings [08:16] It's probably a good target, but we can't depend on it. :) [08:16] indeed [08:17] so 2>selftest.stderr and cat selftest.stderr may be a useful trick [08:17] In the extreme case imagine someone introduces a syntax error in lazy loaded code that is only hit by a test subprocess [08:17] vila: +1 [08:17] buildbot gets this right :/ [08:17] ? [08:18] oh, you mean by handling it by default ? [08:18] It reads stdout and stderr from the subprocess just fine (and even shows stderr in red in the default console output display) [08:18] Right [08:19] I can't believe (yet) the solution to so many babune problems could be that simple... [08:19] time for more coffee :) [08:19] hmm, by the way... [08:20] OMG ! [08:20] fullermd is not there ! [08:20] Hell must be frozen ! [08:21] ok, so plan: 1) more coffee, 2) look at spiv's stderr handling patch..... 4) profit ! [08:28] vila: given the state of http://webnumbr.com/.join%28bzr-active-reviews,bzr-pqm-queue-length,bzr-merged-reviews.derivative%29 you have quite a challenge as patch pilot today :) [08:29] Hangover from the sprint I guess! [08:29] well, I backuped jelmer so I have a bit of lag, but I'm also waiting for reviews for my own proposals... [08:30] may be I should just land them pretexting a user error while landing and approved one without realizing some pre-requisites weren't approved ;-D [08:47] spiv: still hanging on OSX :-/ [08:49] vila: :/ [08:49] vila: does subprocess-log-in-details help at all? [08:49] not tried that one yet [08:49] (Obviously it won't help the hang, but maybe it'll give better info if you interrupt the hang) [08:50] vila: I just did a couple of very quick reviews for you [08:50] And now I'm off! [08:50] great ! [08:50] have a nice week-end ! [08:50] Happy hacking, piloting, and babune-wrangling. [08:51] And have a nice weekend when you finally get there :) [09:21] Riddell: hello there ! [09:22] Riddell: I realized my comment on your https://code.launchpad.net/~jr/bzr/330063-merge-non-existing/+merge/61728 may need explanations about bb:tweak [09:22] Riddell: or do you understand the meaning already ? [09:22] it means tweaking the blackbox tests? [09:22] hehe no [09:23] BB == bundle buggy, that's what we used before reviews were available on lp [09:23] different votes were available there and we still miss the 'tweak' one on lp [09:23] ah [09:24] we use it to convey: your patch looks good, you can land it without an additional review as long as you agree with the modifications asked by the reviewer and implement them [09:24] but shorter ;) [09:25] since this doesn't exist on lp, we use an idiom which is: review: needsfixing, status: approved [09:26] Riddell: do you need further help with this proposal ? [09:26] hopefully not, I'll get to it shortly once I've looked at my e-mail (I was taking a swap day yesterday) [09:27] but I'll let you know if i do :) [09:27] Riddell: ok, cool, welcome back :) [10:43] vila: heh, TestImportTariffs.start_bzr_subprocess_with_import_check explicitly uses the real $HOME! [10:44] vila: so you could argue that including ~/.bzr.log in the details is therefore appropriate ;) [10:44] oh my, I routinely have Megs there ! [10:45] I just didn't look at the result of my first run even if it was limited to ~10 tests, imagine what it will contain for a whole test suite ! [10:46] vila: it's just TestImportTariffs [10:46] Megs * ~20.000, we will end up with GOs ! [10:46] ha right, still we will end up with gOs :) [10:47] It shouldn't be much: [10:47] * details are discarded for passing tests [10:47] * only TestImportTariffs uses start_bzr_subprocess with a real $HOME [10:47] Worth checking my assumptions are right, of course. [10:48] why not make them use a temp home then ? [10:48] But they should be ;) [10:48] β€œ # Normally we want test isolation from the real $HOME but here we [10:48] # explicitly do want to test against things installed there, therefore [10:48] # we pass it through. [10:48] ” [10:48] argh [10:48] i.e. test_import_tariffs wants to make use ~/.bazaar/plugins [10:48] There's probably a better way to do that [10:49] we rotate at 4M I think [10:49] Maybe set BZR_PLUGIN_PATH for the subprocess to explicitly be exactly what the parent uses [10:49] that's 2M of noise on average ;-/ [10:52] vila: should I be on the patch pilot schedule? [10:52] vila: I don't follow [10:52] vila: it only keeps the log contents if the test fails [10:53] vila: so normal cost: 0 [10:53] vila: no matter how large ~/.bzr.log is for that one test [10:53] Riddell: up to you, feeel free to insert yourself, probably at the end of the cycle so you get some time to accomodate [10:54] spiv: meh, maybe I misread your patch but I thought your were installing some automatic trigger for all subprocess calls ? [10:55] vila: for all start_bzr_subprocess calls [10:55] vila: but for all tests *except* TestImportTariff $HOME is some temporary directory [10:55] vila: and all in-process logging goes to an in-memory log during selftest [10:56] vila: so the only contents it will find are from test subprocesses, with the exception of TestImportTariff [10:56] (Ok, the doctests use real bzr $HOME too, but I don't think they use start_bzr_subprocess) [10:59] spiv: right, worth adding as comments then, I'll look into it again later (I thought I said that in the review no ?) [10:59] anyway, thanks for the explanations, will avoid useless roundtrips [11:00] vila: sure, just some drive by info as I was peeking at my mail :) [12:57] * Riddell blogs http://www.kdedevelopers.org/node/4435 [13:37] here's an odd question: is there anyways, from bzr command line, to "know" if someone has proposed a merge request? [13:37] Riddell: 1) Nice. 2) I'm confused by your user icon. 3) s/zx/xz/ [13:37] i have a private lp account, and I need to integrate it with some CI tools, and it requires me knowing merge requests. [13:37] Not via the bzr command line, no [13:38] damn. ok. [13:38] so i instead need to figure out some way of automating a browser request to the LP MR page. [13:38] which would normally be easy if it were public, but it's private. [13:38] maxb: the user icon comes from Umbrello which was my university project back in the day [13:39] Morbus: can't you use launchpad APIs? [13:39] Riddell: didn't even know LP had APIs :) [13:39] take a look at https://help.launchpad.net/API/launchpadlib [13:39] I'm not sure if merges have APIs, but they ought to if there's any justice in the world [13:40] heh, heh. [13:40] ask in #launchpad to find out [13:40] Morbus: You probably want to look at the Launchpad Web Services API, and in particular, if you use python and launchpadlib, lp.branches.getByUrl(url="lp:.......").getMergeProposals(...) [13:40] of course, i don't know any python, but i could figure it outl, I suspect. [13:40] maxb: thanks! [13:40] Riddell: you too :D [13:41] hrm. if it's just a webservice that seems to export JSON, I shouldn't have to use python lib... [13:42] Python is not the only integration possibility, but it is the one with the most advanced client libraries [13:42] sure. [13:42] i don't necessarily need a whole shebang - i /just/ need the list of branches that have a merge request in. [13:42] everything else i can handle from the command line locally. [13:43] Morbus: Try hitting this URL, then: https://api.launchpad.net/1.0/~bzr-pqm/bzr/bzr.dev?ws.op=getMergeProposals&status=Approved [13:44] maxb: right. i presume that's protected by the same cookie auth as a private project is? [13:44] cookie/session auth? [13:45] If it works, that's accidental. [13:45] how do you mean? [13:45] private thingies aren't supposed to be in the API? [13:45] The api.launchpad.net vhost is supposed to be accessed via OAuth tokens for authorization [13:46] ah. [13:46] does the pythonlib handle that too? [13:47] yes [13:47] Launchpad.login_with is the main entry point for authenticated access [13:48] hrm. i think i tweaked this wrong: https://api.launchpad.net/1.0/~morbusiff/examiner/trunk?ws.op=getMergeProposals&status=Approved [13:49] * maxb disappears for lunch, back ~15mins [13:49] * Morbus heads off to try the pythonlib. [13:56] or 5mins as the case may be [13:56] Morbus: Try hitting https://launchpad.net/api/1.0/~morbusiff/examiner/trunk?ws.op=getMergeProposals&status=Approved with your cookie [13:56] same error. [13:56] Object: , name: u'trunk' [13:57] Hm [13:57] No idea then, sorry [13:57] i woudln't worry too much about it now. got launchpadlib installed and i'm gonna fiddle with that for a bit. [13:57] Probably time to hop over to #launchpad for any further questions [13:58] <__monty__> If I want to rename a branch can I just rename the directory? [13:59] <__monty__> Or can I use bzr mv oldbranch/* newbranch/ ? [14:00] just rename the directory [14:00] <__monty__> ty === Ursinha is now known as Ursinha-afk [14:07] hrm. where does this launchpadlib store the OAuth details I'm abotu to put in? [14:08] the script will be running as a different (non-Unix) user than my current Unix user, so I don't want it to overwrite my own data. === Ursinha-afk is now known as Ursinha [14:09] wrong chan. [14:14] ug, I keep using the broken "s" for submit option on feed-pqm [14:14] lifeless: is it true you have a good reason for keeping that around? [14:17] Riddell: hehe, was just reading such emails :) [14:21] poolie: if you add me to ~hydrazine-core I'll set up daily builds so the packaging is no longer a year out of date [14:21] I might also get rid of the broken "s" option on feed-pqm while nobody is looking :) [14:27] Riddell: Propose a branch for merge, I run from sources ;) [15:27] Riddell: nope [15:27] * lifeless disappears [15:28] oh good [15:57] spiv: wake up ! wake up ! I have good news ! [16:03] spiv: what ? Not even 1AM and you're not there... Sheesh [16:04] For the rest of the crowd: one source of selftest hanging on babune shut got a head-shot, thanks to spiv ;) [16:04] For the rest of the crowd: one source of selftest hanging on babune just got a head-shot, thanks to spiv ;) [16:04] just not shut... tyops are back [16:05] I blame fullermd [16:05] it's generally his fault. [16:07] vila: can you run a branch of mine on one of your forking babune slaves? [16:07] I think I fixed the remaining pain, just want to double check before proposing [16:07] mgz: sure [16:08] which branch and which subset ? [16:08] lp:~gz/bzr/cleanup_testcases_by_collection_613247 as always hang fix before r [16:08] ...interesting edit [16:08] that branch, but merge in whatever you need to make a full run work too. [16:08] say that again ? [16:09] so, the import tariff hang fix if that was an issue on the slae [16:09] *slave [16:09] I'd prefer a significant subset, not *all* hangs are fixed, I have a couple of pretty good silver bullets now tough [16:09] * mgz blames vila's infectious typos [16:09] hehe [16:10] justasec, a couple of zombies to kill first [16:10] `-s bb. -s bt.test_` isn't bad, but I want a full run to compare resource usage really [16:11] I got it through a complete selftest on the laptop (after stopping X, had som OOMs trying to fork otherwise), so it should run inside 240MB or so [16:11] hmm, a full run will be more complex *right now*, let's try with the subset and ping me again if I forget the full one [16:11] okay [16:12] but how will you check resources usage on babune ? [16:12] but parallel=fork please, and optionally -Euncollected_cases [16:12] ^...it doesn't give metrics? er... speed I guess then. [16:12] hmm, you know what ? File a bug on lp against babune so I can better track that [16:13] yeah, speed could be a good indirect indicator [16:14] hmm, bt.test_ will include tariff, no good, can *you* merge lp:~vila/bzr/import-tariff-test-subprocess-deadlock instead ? [16:15] can we start with -s bb instead ? [16:15] okay, -s bb. then [16:15] or is it not good enough ? [16:15] that should be clean though. [16:16] mgz: http://babune.ladeuil.net:24842/view/debug/job/selftest-subset-all/109/aggregatedTestReport/ [16:19] crap typo in the branch name ;) [16:21] ehhee [16:21] add -s bt.test_selftest to the subset if you've not restarted yet [16:21] that should give some output [16:22] I did, but it' soon enough to cancel [16:24] hmm, 3 slaves running concurrently, if bad things happen, don't take the blame... [16:24] ...or not too fast ;) [16:25] http://babune.ladeuil.net:24842/view/debug/job/selftest-subset-all/111/aggregatedTestReport/ [16:26] mgz: gentoo failures already available, shallow ? [16:26] hm, that's the hard test case. [16:27] mgz: same number on freebsd, karmic, osx (macadam is osx) [16:27] Oh, I didn't specify -Exxx [16:27] if that helps... [16:27] should work anyway, but those failures are from trying to exercise the parallel=fork code [16:28] which is... tricky [16:28] unexplored territory.. unfortunately [16:29] mgz: which makes me believe you're not splitting your proposal enough :-D [16:29] the code needs to work... [16:30] I can continue landing things that don't actually fix the issue for ever [16:30] but maybe I'll split fixing parallel=fork from the rest. [16:31] mgz: well, -Euncollected_cases as a first step would be good to start with no ? [16:31] mgz: if it requires *not* using --parallel, I'll still buy it [16:31] not if it prints every single test case when you run selftest with --parallel=fork :) === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [16:32] mgz: I can live with such a limitation as long as it doesn't break --parallel=fork when I *don't* use it ! [16:33] mgz: if we reach the point where the test suite can fully run without OOMing that would make me extremely happy [16:34] what's the subunit version on those slaves? [16:34] hmm, close to trunk I think, except for the osx one maybe [16:35] oh no, even the osx one mounts my source dir [16:36] and testtools? [16:36] so, trunk for subunit [16:36] hey, let me type :) [16:36] trunk too [16:37] hum hum. [16:37] but that's not true for the chroots (but you don't care at that point (OMG this is becoming complex again :-/))) [16:37] I'll have to boot the laptop again, darn 2.6 requirement [16:38] ha [16:38] I updated this week after months of using very old versions... [16:41] hm, passes here. wonder if it's something about being a real multicore machine. [16:44] not passing here [16:44] just a note: [16:45] have a look at my last --exclude ORed proposal for a trick to use short names for tests, you will then be able to use assertEquals instead of assertContains [16:45] no big deal but I feel better not using regexps when I can avoid them [16:46] yeah, saw that. [16:46] still need contains, because it looks at the whole output [16:46] which is good here, because it tells me no tests are being run under fork [16:46] ...not sure why though [16:47] anyway, problem with the test not the case cleanup code [16:47] http://paste.ubuntu.com/613816/ [16:47] after running ./bzr selftest -s bt.test_selftest [16:48] I'll ignore the two errors blaming some weird merge [16:48] ...and there it's the reverse problem [16:49] all the cases are leaking [16:49] but the test works. [16:49] ha crap, forgot =-site [16:50] ok single error, s//&/ :) [16:50] ...does that help? if not, I'd check bzrlib.tests.fork_for_tests and make sure the two list emptying statements survived the merge [16:50] what ? You pushed ? [16:51] nope. [16:51] I'm just trying to understand your results [16:52] ...I'm not sure I have ssh to any machines with python 2.6 [16:52] oooh, I love "notify = "uncollected_cases" in selftest_debug_flags" :D [16:56] mgz: lol, reading your patch, tu quoque mi fili ! using test_suite_factory :) [16:57] mgz: but you don't support the arguments, a bit more risky but good enough here [16:58] mgz: come to think of it, the conditional code in sefltest for test_suite_factory may be better *outside* of tests.selftest [16:58] there are quite a lot of oddities with how code is arranged for selftest [16:58] or rather, always called in the same way [16:58] yeah, history [16:58] even having taken an axe to decorators, they're still a pain in the bum [16:59] I can't get a simple param from run_suite down to the code that does the forking [16:59] so have to monkey patch subunit [17:01] I'm still none the wiser about why the slaves didn't manage to run the forking test correctly [17:01] only 3 failures on windows [17:02] can't you just override the forking code ? [17:02] no fork! ... and they look very familiar failures I thought we'd fixed in the past. [17:03] basically about trying to read a locked pack file. [17:04] huh ? Those are new to me... may be I didn't notice them earlier ? Worth investigating [17:04] but right, not now, indeed, different failures [17:04] ^I want to test the actual code that selftest uses to do the test splitting and forking, because it has issues now [17:04] I don't like fixing problems if I'm not testing that they won't regress [17:05] I was suggesting wrapping them to address your params passing issue [17:06] the problem is there are already too many wrappers :) [17:07] nah... the functions you're after are not *yet* wrapped ;D [17:09] can you re-run the subset with the flag on? [17:09] ...I don't think it'll help me understand why the fork tests aren't happy on babune, but I'm out of ideas [17:11] fired [17:13] mgz: I wonder if your overrides survives the fork... [17:14] ooh, interesting [17:14] ...or something [17:14] may be silly but since you ran out of ideas, I'm trying to help :D [17:14] of course I've not been running the whole suite under --parallel=fork here [17:15] I'll see if I can get the failure by forcing that [17:15] yeay! [17:15] I can repo. [17:15] okay, will fix. [17:16] ...also, annoying, you don't get a full test count with --parallel=fork just a running total of completed tests [17:16] thanks vila! [17:18] me ? Thank my daemon, he send me stupid ideas all day ! [17:24] mgz: seriously, I have no idea why you're thanking me... any hint ? [17:25] your suggestion was enough to tell me how to reproduce the failures babune got locally [17:26] if I can reproduce, I can fix. [17:26] haa, great ! [17:26] so, the why might be wrong, but the wondering was useful [17:52] vila, or maxb, could someone handle bug 788736 for me ? if its not a terribly painful thing. [17:52] Launchpad bug 788736 in Ubuntu Distributed Development "python-boto parallel import repair" [Undecided,New] https://launchpad.net/bugs/788736 [17:53] <__monty__> u [17:55] smoser: justasec, just found it the mail in my backlog (and this has been a busy day...) [17:55] <__monty__> vila are you online [17:56] __monty__: yes, but expect lag, I should have stopped working already ;) [17:58] <__monty__> vila: About the error for which I'm trying to add a test, do you have any idea why the result differs for you and me? [17:59] __monty__: I don't know *what* result you get, but search for socket.error in the code base, there are plenty of cases where the same pattern occurs (EPIPE among others) [17:59] smoser: on it [18:00] smoser: branches overwritten, deleting tags [18:00] smoser: expectedly getting No such tag errors :) [18:01] maxb rocks... [18:04] vila: Whilst looking at that one, fancy actioning maxb's suggestion for sysvinit :) ? [18:04] <__monty__> vila: Could you glance at the results I get for selftest? https://code.launchpad.net/~toonn/bzr/socket.error-traceback-393713/+merge/62585 [18:04] smoser: requeued [18:05] smoser: running [18:05] Daviey: meh, which one ? I think I requeue that recently [18:06] vila: https://lists.ubuntu.com/archives/ubuntu-distributed-devel/2011-May/000807.html :) [18:09] Daviey: :-/ I don't know enough there to act without maxb [18:09] * maxb waves [18:09] rejoice ! [18:10] maxb: sudo what ? :D [18:10] I guess the first thing we need to do is decide which of my proposed options w.r.t. sysvinit we are going to go with [18:10] Daviey: How anxious are you to get this sorted? [18:11] maxb: today doesn't matter [18:12] I would quite like to get james_w's opinion on my email [18:12] smoser: comment on the bug when you get some results. Thanks in advance [18:12] maxb: Throwing away the trivial merge doesn't seem to matter, but it's not a really a good generic fix IMO. As it's not ideal for other situations [18:13] __monty__: just a sec [18:13] vila: thanks for looking into it [18:13] Daviey: Agreed. Though, the situation that sysvinit is in on Launchpad, with some Ubuntu branches existing but no Debian branches existing, should never occur naturally anyway [18:13] Daviey: np, thank maxb ;) [18:14] maxb: I blame smoser. [18:14] So I'm guessing that someone has been doing some manual fiddling there anyway [18:14] eh? [18:14] Have we crossed sysvinit and python-boto? [18:14] no, i just enjoy blaming smoser for generic things. [18:14] i probably did it. i am constantly fighting with bzr. it sees things differently than i do. [18:14] __monty__: ok, so what OS are you on ? [18:14] Oh, oSX [18:14] <__monty__> vila: os X.6 [18:15] and since i don't have a 'smoser' to blame like Daviey does, i usually just blame bzr. [18:15] <__monty__> vila: no os X.5.8 (sry) [18:16] __monty__: ha ha, an OSX 10.5 user using python2.6... [18:16] __monty__: you know what ? We may have more interesting bugs for you :) But let's focus on that one first [18:17] <__monty__> vila: Sure. [18:18] sockets can raise different kind of execptions depending on the platform so you need to be ready for that [18:19] <__monty__> vila: Anywhere I can find what exception they raise on what platform? Or how do I handle this? [18:20] __monty__: we care more about catching all the relevant ones that tracking which platform track which kind, so try grepping the sources for socket.error [18:23] __monty__: more concretely, what you should check is not the *message* but rather the class of the exception and then the associated args [18:23] you should find plenty of examples in the code [18:23] <__monty__> Indeed, they are plentiful. [18:32] __monty__: so, if you start working again, it's better to put your mp into the 'Work In Progress' state [18:33] __monty__: because it will pop up in the list again once you put it back to 'NeedsReview' [18:33] <__monty__> Ah yes, hadn't thought of that. [18:34] Argh, what?! The python-boto reimport has somehow retained the revisions that were not supposed to be there?! [19:03] <__monty__> vila: Care to give me another clue? I don't really understand what to learn from the examples in the code. === medberry is now known as med_out === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [19:40] __monty__: sorry, was afk [19:40] so, what are you trying to test exactly ? [19:54] <__monty__> vila: I'm working on the test for the socket.error which apparently behaves different on linux(?) and on os X. [19:55] right, I got that, but why are you doing that ? What is your final goal ? [19:57] <__monty__> Proving that the socket.error is allready caught? Because it's a subclass of IOError. [19:57] maxb, so any ideas? [19:58] it looks like the score is still : smoser's stupidity: 1, bzr: 0 [19:59] hi all [19:59] __monty__: then you want a test that succeeds at catching IOError but fails at catching socket.error ? [20:00] poolie: wow, hey ! [20:00] poolie: is it late for me, early for you or both ? [20:00] :) early for me [20:01] rats, I thought I could stop and enjoy my week-end ;-D [20:04] :) well, you can [20:04] isn't it something like 9pm there anyhow? [20:04] i just thought i'd poke at some discretionary lp patches since i'm up so early [20:04] one last bug ! [20:04] :) [20:04] <__monty__> vila: No I don't think so. Because IOError is allready handled by test_trace.py and socket.error is a subclass thereof (since python 2.6) the cause of bug 393713 no longer produces a traceback. I am now trying to add a test that proves this. So the bug can be closed. [20:04] Launchpad bug 393713 in Bazaar "[master] socket.error causes a traceback" [Medium,Confirmed] https://launchpad.net/bugs/393713 [20:06] __monty__: if you raise a socket.error and have two exception handlers, one for IOError and one socket.error, if socket.error is a subclass, the IOError handler will trigger, if not, the socket.error handler will trigger, which one triggers tells you if your proof is correct [20:07] <__monty__> Ah, like so. So I try catching IOError first? [20:08] __monty__: just try ! That's what tests are for ! Experimenting with your understanding of the code ! [20:09] <__monty__> Still, I don't really understand how the tests work. Is there someplace I can read up on them? My main difficulty with them is that the except blocks are just 'pass'ed and then the error string is checked. [20:11] __monty__: try adding a self.debug() at the first line of your test and execute it step-by-step under pdb, that should help you understand how it works [20:20] <__monty__> vila: How do I run it in pdb? import pdb; import bzrlib.tests.test_trace; pdb.run('test_format_sockets_error()') ? [20:20] __monty__: try adding a self.debug() at the first line of your test [20:20] __monty__: and just re-run ./bzr selfest [20:23] <__monty__> vila: Can I go on step by step? Because now it just stops, then I type continue and the results for selftest are shown. [20:24] vila, tests using threading seem highly likely to be flakey [20:24] except KeyboardInterrupt, e: [20:24] out, err = e.args [20:24] wow that's quite creative [20:25] __monty__: you don't know how to use pdb ? [20:25] well, good luck to you [20:25] __monty__: to step, use 's' or 'n' [20:25] the latter steps over function calls [20:25] <__monty__> vila: No, but 'step' seems to do the trick. [20:26] doh, I'm tired, that was a silly question :( [20:26] __monty__: http://docs.python.org/library/pdb.html?highlight=pdb#module-pdb [20:27] poolie: I disagree [20:27] <__monty__> vila: I allready have that open, it's just I don't really see what's the point in using pdb. I can go over the code just reading it no? It doesn't seem to give me extra information. [20:28] poolie: they are flaky only when they are buggy, at least that's my experience based on debugging the thread leaks [20:28] poolie: most of the bugs where bad synchronization [20:29] __monty__: you said you didn't understand how the tests worked, the best way to understand them is to see them run, just reading them reading doesn't have the same taste ;) [20:29] poolie: s/where/were/ [20:30] poolie: and unsurprisingly here, that's still a race, involving processes instead of threads but still a race [20:30] <__monty__> vila: What I meant was, why is(are) there except blocks if they don't do anything? [20:31] __monty__: try removing them and you'll soon realize they *do* something [20:31] monty, where? [20:31] __monty__: probably swallowing an execption [20:32] <__monty__> Ah, didn't realise that a trp [20:32] <__monty__> *Ah, didn't realise that a try block still raises the error if it's not handled. [20:37] it's not so much it still raises it, it just doesn't stop it propagating [20:39] <__monty__> vila: I'm still not sure about what you said earlier, 'the handler that's triggered will proof socket.error is a subclass'. Why should I proof that, it is true(since python 2.6). I think what I need to proof is that the socket.error is allready handled in test_trace.py, no? [20:39] <__monty__> poolie: Thank you for correcting. [20:40] __monty__: I don't know, does test_trace catch IOError ? [20:40] <__monty__> Yes. [20:41] that's not what your test is showing [20:41] <__monty__> vila: That's true but how do I show that? [20:42] we're running into cycles, where did you start ? What was failing ? [20:44] __monty__: the bug report says: "I think that on python2.6 (which is required for bzr2.4) this bug will be implicitly fixed because socket.error has become a subclass of IOError, which does not generate a traceback. We should test that though." [20:44] <__monty__> nvm what I was typing. In short that's the reason for the test ^. [20:47] __monty__: that's what I described above: proof that IOError is a subclass of socket.error, this can be as simple as assertIsInstance(socket.error(), IOError) or assertTrue(issubclass(socket.error, IOError)) or a bit more detailed as I explained above [20:49] if you want to go further then you need to find a case which indeed raise a socket.error, explore the call stack, remove the now useless socket.error handlers and check (with tests) that the behavior is still the same [20:49] <__monty__> vila: What socket.error handlers are you talking about? [20:50] hey guys, how can I publish my branches on my own server? [20:50] any try block with a except socket.error clause [20:51] pl0sh: by allowing people to connect to your server with any protocol supported by bzr: ftp/http/ssh/bzr [20:52] pl0sh: the setup ranges from none to quite sophisticated depending on the protocol === med_out is now known as medberry [20:55] let's say http, because my dev's uses winsucks, but my server is a freebsd box [20:56] pl0sh: do they need read write or readonly access? [20:56] if they're developers i guess rw? [20:56] the easiest thing then is to let them ssh in and run bzr over ssh [20:56] the bzr windows client includes ssh support [20:57] <__monty__> vila: I think I don't really understand the problem then. I thought what I was trying to solve was the fact that people got a traceback when a socket.error occured. Bug 364462 is what I started on but poolie(I think) pointed out it's a dupe of 393713. [20:57] Launchpad bug 364462 in bzr email commit hook "smtp "connection timed out" causes full stack trace to be dumped (dup-of: 393713)" [Medium,Confirmed] https://launchpad.net/bugs/364462 [20:57] Launchpad bug 393713 in Bazaar "[master] socket.error causes a traceback" [Medium,Confirmed] https://launchpad.net/bugs/393713 [20:57] poolie well, they have to be able to commit and merge the changes [20:57] monty, where are you with it now? [20:57] ok, so bzr+ssh is the way to go [20:58] Ok, where can I find some info? [20:58] <__monty__> poolie: Still nowhere pretty much. [20:58] I have installed bzr on my freebsd and on my windows boxes [20:59] pl0sh: there's an admin guide [20:59] but, basically, start an ssh server on the bsd box [20:59] then just from a windows box run eg [20:59] 'bzr init bzr+ssh://bsd/~/test' [21:00] great [21:01] and bzr push bzr+ssh://bsd/~/test right? [21:01] to push changes to the repo [21:01] is there an easy API in bzrlib available to get the LP API branch URL from a working tree? [21:01] i have a working tree of a branch on lp and want to make API call on its LP representation [21:08] hi flacoste [21:08] hi poolie [21:08] i was about to try [21:08] pl0sh: correct [21:08] i think there is one in plugins/launchpad [21:08] from bzrlib.plugins.launchpad import lp_api [21:08] lp_api.from_bzr(lp, branch) [21:21] <__monty__> poolie: I'm trying to write a test for 393713 but I'm not sure what to write in it. (Doing this because you say in comment #4: 'We should test that though.') [21:22] flacoste: so is that ok [21:22] poolie: still writing the script :-) [21:23] from the look of it, it seems like it's exactly what i need [21:23] flacoste: separately i was wondering if you could nudge lp reviewers to land more approved 3rd party patches [21:23] not me, other peolpe [21:23] if you want to add a tip to the api guide docs or wiki page that would be welcome [21:23] what are you writing, just for curiouisyt? [21:24] g'night all, have fun ! [21:24] <__monty__> vila: gn, thanks for helping. [21:24] poolie: a script to make a principia package branch official [21:25] monty, have you pushed your current code? [21:25] poolie: inspired by set_official.py in udd but less coupled to the package-importer implementation [21:25] i'll have a look at the review [21:25] cool [21:25] * __monty__ nudges poolie to stop ignoring his question? :) [21:26] poolie: people approving a third party branch are supposed to take care of landing it, isn't that happening? [21:26] apparently not [21:26] poolie: remember that it's not visiale if they have submitted it to ec2test and it will take some time to update [21:26] just sliping through the cracks i think [21:26] well, i'm seeing 3rd party branches merged [21:26] nigelb and chris' one [21:27] which one haven't landed? [21:27] like https://code.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057 [21:27] it's lifeless here that needs nudging [21:27] 3 from chrisjohnson [21:27] he might not be aware of it [21:27] oh to rereview it? [21:27] he approved it [21:27] i've just put one of chris's up for ec2 [21:27] poolie thanks [21:28] i'm not saying it never happens, just it apparently doesn't always happen [21:28] i am very happy chris & others got prompt reviews [21:29] i can send a reminder [21:29] vila, you just need a couple of tweaks to the utextwrap thing [21:29] __monty__: sorry, not trying to ignore you [21:30] show me the test you have currently? [21:30] if any [21:30] <__monty__> poolie: It's worthless currently. [21:30] i thought yesterday we got up to having a test based on the other error-reporting tests [21:31] that raised a socket error and then checked it was printed in a reasonable way [21:31] did that break? [21:31] <__monty__> Well, turns out the error message is different on different platforms. [21:32] oh, that's too be expected, yeah [21:32] has anyone got bzr bisect to work http://doc.bazaar.canonical.com/plugins/en/bisect-plugin.html ? [21:32] the main thing we want to assert in the test is just that it does not give a stack trace [21:32] ironcamel: what specifically is going wrong? [21:33] poolie: i installed it, but bzr bisect doen't do anything [21:33] bzr: ERROR: unknown command "bisect" [21:33] <__monty__> poolie: Agreed, so just test the string for not containing 'traceback' ? [21:33] how about 'bzr plugins -v' [21:33] __monty__: i think it's "Traceback" but yes [21:34] poolie: it's not listed [21:34] how did you install it? [21:34] poolie: python setup.py install [21:34] got the source from https://launchpad.net/bzr-bisect [21:36] hm, i wonder where that installed it to? [21:36] perhaps something that's not on your bzr plugin path [21:36] not sure [21:36] typically you'd install from source with just eg [21:36] bzr branch lp:bzr-bisect ~/.bazaar/plugins/bisect [21:37] oh really [21:37] it comes with a setup.py, so that's why i did that [21:38] poolie: i think that did it :) [21:39] a little documentation would have made this easier [21:39] please file a bug against it about the setup.py not doing the right thing [21:40] sure [21:42] done [21:51] <__monty__> poolie: This is what the test should look like right? https://code.launchpad.net/~toonn/bzr/socket.error-traceback-393713/+merge/62585 [22:00] looks pretty good [22:02] <__monty__> Ok, then I'm off, thanks for the help. Hope I can work on a different bug tomorrow. :) [22:12] hm... lp:ubuntu/nagios-plugins seems to be screwed up the same way python-boto is. [22:14] opened bug 789342 to address that. [22:14] Launchpad bug 789342 in Ubuntu Distributed Development " lp:ubuntu/nagios-plugins gives strange diffs" [Undecided,New] https://launchpad.net/bugs/789342 === Ursinha is now known as Ursinha-afk [22:15] thanks smoser === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk === JayFo is now known as JFo [23:26] poolie, did you get a chance to look at my additions to your stale locks branch? [23:26] I... was slightly losing track last night but I think the code makes sense [23:28] mgz i did merge it in [23:28] thanks for that [23:28] i have some more items from spiv still to fix [23:28] at the moment i'm just finishing off some pending lp work [23:32] in unrelated bookkeeping, do I need to bug someone for an expenses form for last week? [23:33] yes, marianna [23:33] will email.