[01:16] Good morning. [01:17] oh hai === CcxCZ_ is now known as CcxCZ [03:05] Hm. Can you suggest a reason why installing the bzr-builddeb package wants to bring in the entire libqt stack? [03:14] its in love! [03:15] is qbzr being dragged in somehow? [03:16] mwhudson: as far as I could tell, no [03:16] This was a nearly naked server system; [03:16] (ie, no Desktop) [03:16] I subsequently removed the *qt* and apt didn't try to remove bzr-builddeb [03:17] so I was really at a loss as to wtf [03:17] recommends vs depends [03:17] I install with -o Install-Recommends=false [03:17] (and it still came in; I look closely for that sort of thing) [03:18] what does apt-cache bzr-builddeb show for you [03:18] under depends, recommends & suggests [03:18] (ie, there was something in the Recommends: column that was nothing to do with libqt* [03:18] lifeless: I looked through that too. Its innocent, as far as I can tell === CcxCZ_ is now known as CcxCZ [03:19] [which is why I raise this casually here; it was _weird_] [03:19] so, to diagnose it [03:19] apt-cache rdepends [03:19] the package that was being brought in [03:19] yeah, I didn't think to try that at the time [03:19] and/or remove bzr-builddeb and install it again [03:25] lifeless: I tried that too [03:25] oh [03:25] and /var/log/apt/ has some debug stuff [03:25] (I'll keep an eye out for this; at the moment that machine is churning away building things, so...) [03:25] true, [03:25] looking [03:27] ah ha [03:27] it was arora, depending on libqt4-webkit [03:27] * AfC wonders what arora is [03:27] some kind of web browser, obviously... but what pulled it? Weird. [03:28] Well, embedding mail reading in everything was SO 1980's. So now all apps grow until they can render HTML. [03:28] lifeless: nice call on /var/log/apt/history.log [03:29] if don't have a browser [03:29] then the first explicit one listed (if one is) is pulled in [03:29] someone has a clause like [03:29] some kind of virtual package requirement, presumbably [03:29] arora | web-browser [03:30] yeah [03:30] eek [03:53] hi all, i'm back [03:55] wb [03:59] poolie: wb [04:00] thanks === timchen1` is now known as nasloc__ [05:09] Hi. Is there any gotchas in makine /etc a bzr working tree. [05:09] andrewblack: bzr doesn't track permissions very closely [05:09] andrewblack: permissions on funky files within the /etc tree within the .bzr directory. [05:09] so i think you might want to use a wrapper like etckeeper [05:10] or lock down /etc/.bzr [05:10] http://joey.kitenet.net/code/etckeeper/ [05:10] ok - will look at etckeeper [05:10] mwhudson: nice tag teaming there. high-five :-) [05:11] spm: aren't we supposed to do that in a privmsg? [05:11] :) [05:11] um.... [05:13] It says it tracks changes during package updates. What about manual changes to /etc/xxxx files [05:19] you can manually check those in [05:19] it'll also auto check those in next package update [05:19] if you want; you can always setup some sort of cron script that alerts/or auto-commits changes [05:20] at $job-1, we had a process rule; any /etc or other config changes were always done thru a wrapper script that would check the edited file in/out/diff of an RCS store. [05:24] ho hum. said article has gone the way of the dodo. was on the samag.com website; now defunct. [05:25] Thanks spm [05:26] so basically it sets up the repos in a way that is permission friendly [05:32] ok - I wanted to edit a cron job (and get it under bzr first) [05:32] turns out the that thing I needed to change was in a file it calls that was already in a bzr tree. [05:36] hrm. not so much permission friendly; more permission *unfriendly*. /etc/.bzr ==> 0700 type of thing. [05:57] http://bazaar.canonical.com/en/ seems to be a little old on the released bzr status [06:06] thanks thumper, i'll check why later [06:06] poolie: we should have a chat some time [06:06] soonish [06:06] we should [06:06] now is good with me, if you like [06:06] (or in about 2m) [06:10] poolie: I'm sprinting this week, but would still like to organise something... [06:10] induction for wallyworld_ [06:17] yay wallyworld_ ! [06:17] thumper: how about now [06:18] hi poolie [06:18] * thumper fetches headset === r0bby is now known as robbyoconnor [07:45] hi all ! [07:45] wv poolie ! [07:45] meh [07:45] wb [07:45] hello vila, how are you? [07:45] first typo in second msg, looks like a good way to start the week :) [07:46] poolie: great, about to half the active review queue by landing those long awaited fixes :) [07:49] only 660 messages to go .. [07:50] Hi all. [07:51] hey GaryvdM ! [07:52] spiv, vila, did either of you do anything about unlocking on interrupts, or coping when we were interrupted? [07:52] poolie: nope, I thought you were working on it and then... went to other things [07:53] poolie: ping [07:55] * bialix waves at ddaa [07:55] Hi bialix [07:56] Heya Gary! I happy to see you [07:59] I'm trying to write some tests for qlog. I need to create branches with interesting graphs. I looked at BranchBuilder to see if it would make things easier, it helps some what, but one thing that is crufty is having to branch my branch on disk so that I can create a diverging graph. Is there a helper to make this easier? [08:01] np, i might look at it this week [08:02] GaryvdM: invoke `bzr branch` command via self.run_bzr? [08:05] poolie: no, I didn't either. [08:14] what did you two get up to? [08:17] poolie: resuming working on transform targeting allowing deleting dirs containing .pyc files [08:17] s/resuming/resumed/ [08:18] nice [08:24] GaryvdM: can you look at https://code.launchpad.net/~s-dumbie/qbzr/ench_qdiff/+merge/33986 [08:25] one russian guy contacted me with patch couple of months ago. then yesterday friend of him decided to finish that patch [08:25] GaryvdM: as I understand some of their changes should be reworked with new toolbar in qdiff. but we don't have one yet. [08:26] bialix: Ok [08:26] we (I) may ask him to split the patch into smaller pieces [08:26] if it helps [08:27] * vila banhs head on desktop, from pqm: "Exception processing merge: Unknown branch format: 'Bazaar-NG Loom branch format 7\\n'" [08:27] bialix: Hopefully the search stuff can be made common with annotate too. [08:27] yeah, banhs, brohe a teeth doing that [08:28] vila: we're with you! [08:28] * fullermd broke a tooth on the bottom of a pool once. [08:29] tooth yeah, the only remaining one in fact [08:29] fullermd: forgot to check for water ? [08:30] Oh, no. There was water. It was pretty bizarre; I probably couldn't do it again if I tried. [08:30] Not that I plan on trying, though. Unpleasant experience. [08:30] fullermd: and *that* was the last time you left your basement right ? [08:31] Well, it shoulda been :p [08:31] :-D [08:31] But I was young and foolish, and therefor sociable, back then. [08:31] fullermd: such a loss of time... [08:32] I know! All those afternoons I could have spent plotting the downfall of humanit... er, studying and improving myself. [08:45] spiv, what's new with you? [08:50] errr, what did I miss ? pqm got a faster machine ? I wasn't able to see my requests on http://pqm.bazaar-vcs.org/, yet they have landed... [08:51] today speed of light is much faster I suppose [08:51] hi bialix [08:51] vila, oops [08:51] i suggest you send one you know will fail [08:52] hi poolie [08:52] and make sure that it does [08:52] spiv does my guess on bug 626649 sound plausible? [08:52] Launchpad bug 626649 in Bazaar "pull from one ssh branch into another fails with TooManyConcurrentRequests (affected: 1, heat: 6)" [Undecided,New] https://launchpad.net/bugs/626649 [08:54] poolie: you're kidding ? Time to send all those controversial patches instead :) [08:55] poolie: yes, very plausible, I waited far too long to report it wanting to write a test for it :-} [08:57] poolie: I work around it with: 'bzr unbind ; bzr pull -v ; bzr push ; bzr bind' [09:00] yup, it's a dupe of bug 483661 [09:00] Launchpad bug 483661 in Bazaar "bzr pull bzr+ssh into a bound branch fails with TooManyConcurrentRequests (affected: 3, heat: 4)" [High,Confirmed] https://launchpad.net/bugs/483661 [09:00] poolie: yep, that sounds plausible. Not 100% sure, but definitely plausible. [09:03] poolie: I did some measurement of where we're at with gcc-linaro, and found a 15% win for one of the cases. [09:03] oh, nice, what was that [09:04] Although for some reason I'm having trouble with getting useful results with --lsprof-file and kcachegrind, I'm not sure why yet. It might be something about trunk, or something about maverick... [09:04] It turns out that InterTree.iter_changes was doing "if file_id in self.target.all_file_ids()" in a loop. [09:05] so recalculating all the ids every time? [09:05] Right, which means set(tree.inventory) [09:07] maybe that should be deprecated [09:07] hm [09:07] Anyway, Tree already has a __contains__ method that does that check without iterating over an entire inventory, so "if file_id in self.target" is the trivial and faster replacement. [09:07] ok [09:07] no, i guess profiling is probably a better way to find expensive things called repeatedly [09:07] i'd like to delete __contains__ etc [09:08] vila does this mean all those recently merged things of yours perhaps should not have merged? [09:08] or did you integrate them and then send the whole thing? [09:08] poolie: investigating [09:09] poolie: I went with pqm-submit instead of feed-pqm because looms were involved (and not supported by pqm) but that still use the "regular" pqm way [09:10] hmm, looks like I used syntax accepted by python-2.6 but not 2.5, so probably not 2.4 either [09:11] that in turn would mean that pqm accepted the merge when selftest failed with a syntax error... bad bad [09:13] vila: ! [09:14] rhaaaa, on no more py25 on lucid... shudder [09:22] spiv: Isn't it a nice way to land a patch ? [09:25] poolie, spiv: here is the fix: http://paste.ubuntu.com/485753/ [09:27] poolie, spiv: I will send this to pqm *now* and check for fallouts (unless you yell loud) [09:28] * poolie looks [09:28] vila, is this because py25 won't let you use *args and also regular kwargs? [09:29] poolie: seems so, 2.4 and 2.5 fail without and succeed with this patch [09:29] but doesn't pqm use 2.4? [09:30] I don't remember if it stills uses 2.4 or 2.5, but I've filled bug #626667 [09:30] Launchpad bug 626667 in PQM "syntax error doesn't prevent merge (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626667 [09:31] losa: can you help diagnose bug #626667 by checking the recent logs on the bzr pqm instance ? [09:31] poolie: ok to land the above fix ? [09:31] vila, i guess so [09:31] i really want to work out why it wasn't rejected though [09:32] sent [09:32] as i'm sure you do too :) [09:32] poolie: yup, hence the bug [09:32] losa: first data point: which python version is used there ? 2.4 or 2.5 ? [09:33] hey [09:33] http://paste.ubuntu.com/485756/ hmpf [09:33] I accidentally uploaded some stuff to lp without fixing bzr whoami first [09:34] is there a way to fix my commits? [09:34] vila: not loud yells from me [09:34] s/not/no/ [09:35] bug 626667 updated... just lovely [09:35] Launchpad bug 626667 in PQM "syntax error doesn't prevent merge (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626667 [09:36] lvh: uncommit, or push --overwrite [09:37] bialix: so if I fix whoami, and then push --overwrite, it'll be fixed? Cool, thanks [09:38] lvh: no [09:38] bialix: huh? what do I do then [09:38] lvh: you have to redo your commits locally and then push --overwrite [09:38] how many of them do you have? [09:38] so 10:36 lvh: uncommit, or push --overwrite => uncommit *and* push locally [09:38] twenty or so [09:39] I didn't have a working internet connection to push to lp [09:39] so it took me about a day to notice [09:39] what's wrong with your old whoami? [09:40] lp doesn't link it to my user profile correctly [09:40] it doesn't have my email address [09:40] lvh: and you don't get karma ? [09:40] it's not too bad [09:40] vila: maybe, I don't know [09:40] yes, it's not a major issue [09:41] I was just hoping there's a way of letting lp know "oh hey that guy, that's me" [09:41] lvh: that's the only consequence I can think of [09:41] lvh it has no address at all? [09:41] poolie: worse, it has a useless one [09:41] 20 commits... you can redo them one by one, or maybe use bzr-rewrite plugin [09:41] It says Laurens Van Houtven [09:42] lvh: try adding this email to your lp profile [09:42] you can add it to your profile as alternate mail [09:42] vila: aha, okay [09:42] bialix: Is that possible without mail confirmation? [09:42] lvh: not sure lp willl accept an invalid email but worth a try [09:43] A confirmation message has been sent to 'lvh@bruichladdich'. Follow the instructions in that message to confirm that the address is yours. (If the message doesn't arrive in a few minutes, your mail provider might use 'greylisting', which could delay the message for up to an hour or two.) [09:43] That'll backfire :) [09:43] lp is too smart [09:43] lvh: all you have to do now is register this TLD :-D [09:44] lvh: You could as in #launchpad if there is a way around that, or fix up the commits and push again.. [09:44] maybe you can file a bug report against lp itself and ask for such deature [09:44] Okay, thanks :) [09:44] s/feature [09:47] https://launchpad.net/bzr-rewrite is that the rewrite plugin I want? [09:47] lvh: yes [09:49] vila is it just me or is --parallel still a bit flaky? [09:49] i get "failed to connect to ssh server" [09:49] oh, heh, and the strace tests are broken by maverick's policy change there [09:50] poolie: ENOTENOUGHCONTEXT [09:50] poolie: is that with the leaking tests fix landed ? [09:50] ERROR: bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh: Unable to connect to SSH host localhost:51613; [Errno 111] Connection refused [09:50] that's just in trunk as of a few minutes ago [09:51] Okay, so lets say I want to fix all my commits and then push --overwrite, how does rewrite make my life easier [09:51] poolie: on maverick? That's a paramiko bug relating to IPv6 [09:51] poolie: sounds like either a transient one or something related to a spiv fix for paramiko [09:51] oh ok, thanks [09:51] how about bug 626679 [09:52] Launchpad bug 626679 in Bazaar "test_strace_result_has_raw_log fails on maverick (affected: 1, heat: 6)" [Medium,Confirmed] https://launchpad.net/bugs/626679 [09:52] I should nag upstream and/or the Ubuntu packager to include my patch, I guess. [09:53] poolie: I'd like to draw your attention to bazaar.canonical.com/en/index.html page [09:54] * poolie looks [09:54] i need to go soon [09:54] hi bialix [09:54] that page has section "Developer's Blog" [09:54] what about it ? [09:54] hi poolie [09:54] but it links to "None", i see [09:54] the section is empty for me. [09:55] but when I'm regenerating the index on my machine it's not empty [09:55] does this index regenerated on the production server? [09:55] it seems production server has no access to wordpress feed, or something like that [09:56] I have no idea how that page deployed, but there is definitely some problem [09:57] i'll check it out on the production server [09:57] ping LOSA [09:57] and btw local index has link to http://bazaarvcs.wordpress.com/ and not to None [09:57] something weird [09:58] vila: hi [09:58] mthaddon: hey ! [09:59] mthaddon: really bad bug on pqm bzr instance: bug #626667 [09:59] Launchpad bug 626667 in PQM "syntax error doesn't prevent merge (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626667 [09:59] losa: can you help diagnose bug #626667 by checking the recent logs on the bzr pqm instance ? [09:59] losa: first data point: which python version is used there ? 2.4 or 2.5 ? [10:00] Okay, I installed bzr-rewrite. I don't think rebase is what I want if it's anything like git's [10:00] mthaddon: a fix for *bzr* is currently landing, but pqm needs to be fixed, we prefer to rely on pqm to *detect* such problems :) [10:00] vila: it's a hardy chroot, so python 2.5 is the default [10:01] mthaddon: ok, that's one thing [10:01] vila: in what way does PQM need to be fixed? [10:02] see the bug report, despite an *error* it happily merged the submission [10:05] hi [10:05] bialix: it's a problem with the proxy setting being not seen [10:05] when doing bzr export i get the following error: bzr: ERROR: zlib.error: Error -3 while decompressing data: incorrect data check [10:05] fixed now, thanks for letting me know! [10:05] poolie: :-) [10:05] after googling i found that this means an data error [10:06] is there some way to fix this [10:06] bzr status / bzr commit and so on still work [10:07] nomatter001: try to branch it to new location [10:08] nomatter001: try 'md5sum .bzr/repository/packs/*' and check that the md5sums match the filenames [10:10] vila: pqm just works by submitting if the command you give it returns a "0" exit code - if for some reason the conditions you're describing still return a 0 exit code, then PQM is still doing the right thing [10:11] mthaddon: right, so what is this command for the bzr instance ? [10:11] spiv: not for all files [10:11] Hi. I don't understand how to use the rewrite plugin to change the author of a bunch of commits. [10:12] vila: for which branch? [10:12] bzr rebase appears to be for rebasing branches like git's rebase, which isn't quite what I want (I think). [10:12] mthaddon: bzr.dev [10:12] mthaddon: but I hope it's the same for all bzr branches :-} [10:12] spiv: for 1 file it is different [10:12] spiv: what can i do? [10:12] vila: "make check PYTHON=python2.4" :( [10:13] mthaddon: LOL, good so it's 2.4 finally, I'll look into 'make check' now [10:13] spiv: rename the file? [10:15] i'm off, see you later [10:15] mthaddon: wow, reproduced, indeed, there are two commands for the make target and the second one is eating the first error :( [10:16] good to have reproduced... [10:16] mthaddon: thanks a lot [10:16] mthaddon: I've enough info to propose a fix [10:16] ok, cool [10:17] nomatter001: if they don't match, the data in the repository doesn't match what bzr thought it was writing to disk, which probably indicates filesystem corruption has occurred :( [10:17] * vila re-re-re-re-reading the make doc to find the right prefix [10:17] spiv: so what can i do? [10:18] spiv: can i get a diff between real data and what bzr thinks it should be? [10:18] nomatter001: ideally, rebuild the repository from backups or other (good) copies that don't exhibit that error [10:18] First, you reverse MD5... [10:19] spiv: i have an online repository which i branched right now where no error happend when i exported [10:19] but i made a push into it (i think after error happend) [10:20] can this still be valid? [10:20] nomatter001: is the file that doesn't match an empty one ? [10:20] villa: no [10:20] nomatter001: ok, so I'd make a new repository from that one, and then if you have new data in the damaged one branch/pull the new branches/revisions into it and hope they aren't affected. [10:21] spiv: ok thx [10:21] 'bzr check' when you're done should tell you if it the result is ok [10:22] (Although just checking the md5sums would be a good and quick sanity check too) [10:22] nomatter001: out of interest, which filesystem and OS, and has your system crashed at all? [10:23] ha, yeah, 'cmd1 | cmd2' in a Makefile... nice way to ignore errors in cmd1... [10:24] spiv: archlinux, ext3 and no [10:24] no crash at all [10:25] nomatter001: then it could be your HD slowly dying :-/ [10:25] vila: hd is 2 months old [10:25] nomatter001: the pack files are named with their md5sum and are *never* modified after that [10:26] nomatter001: wow, that's really surprising then! [10:26] vila: would be quickly dying [10:26] :-/ [10:26] nomatter001: what the english expression for that: infant mortality ? [10:26] vila: exactly [10:26] nomatter001: some electronic devices die young or never, something like that [10:27] I've also heard "bathtub curve of reliability" [10:27] vila: yes i think its called mtbf [10:27] nomatter001: *I* would closely monitor this HD [10:27] mean time between failure [10:27] These are hard drives. It's not a bathtub, so much as an endless abyss. [10:28] It's a question of the shape of the curve, not just where the mean is :) [10:28] nomatter001: all reports I've seen about such corruptions were massively empty files after crashes, the few others were hardware problems [10:28] vila: ok great ... [10:29] nomatter001: I wish you the best but... if you really had no crashes nor rude reboots... [10:29] nomatter@nomatter-laptop ~ % uptime [10:29] 11:30:52 up 11 days, 14:47, 1 user, load average: 0.00, 0.02, 0.05 [10:29] scary, and you're sure the pack file has been created since then ? [10:30] vila: hmm thats another question [10:30] i have no idea when this files are created? [10:30] nomatter001: oh, the modification time should be good for that :) [10:30] nomatter001: they are *never* modified ;) [10:30] 'stat that_file' should show you the mtime (and the ctime) [10:31] vila: -rw-r--r-- 1 nomatter nomatter 55804179 Aug 10 07:40 925afb86da175268da810b3337d50eea.pack [10:31] Except by the hard drive ;) [10:31] ok no before [10:31] nomatter001: right, so that's before your uptime [10:31] fullermd: ;) [10:31] vila: i have no idea what happend in that timeframe [10:32] but what i know is that i could make exports since then [10:32] nomatter001: so hope for a crash, cross your fingers, kill a chicken, and monitor this HD for any suspicious activities ;) [10:33] nomatter001: follow spiv directions, if 'bzr check' is ok, then you're fine [10:36] ok seems that the online repo is finde [10:36] *fine [10:36] i'm lucky [10:36] so i have only to hope that another of my local branches is ok then i'm rescued [10:37] thx for the help so far [10:38] nomatter001: if you use a shared repo, you only need to fix the repo [10:38] nomatter001: you're welcome, and good luck with the mysterious corruption :/ [10:38] vila: how can i fix the repo? [10:40] nomatter001: ECONTEXT, I thought you just did that ? [10:41] vila: i created a new repository [10:41] and branched the trunk branch from the online repo [10:42] nomatter001: a shared repo ? [10:43] vila: no normal repo [10:43] or no shared-repo [10:43] damn [10:43] nomatter001: let step back [10:44] how many branches did you have on your laptop, are they sharing a repo or are they standalone branches ? [10:44] i have no idea i just followed the tutorial [10:44] i made a repo with bzr init-repo [10:44] wo i think a shared one [10:44] yup [10:44] and in that i had several branches [10:44] err, init-repo creates a repo, if your create branches below that, they'll use this shared repo [10:45] ok [10:45] run 'bzr info' in such a branch and you can confirm that [10:45] yes [10:45] and this shared repo doesn't work [10:45] ok [10:45] cause one pack doesn't match the md5 [10:45] i also have a online-repo [10:46] so a shared repo contains all revisions ever created by any branch there [10:46] and now i just created a new shared repo [10:46] and branched the one branch in the online-repo into the new shared repo [10:46] and that seems to work [10:46] good [10:47] how many branches do you still need to get back ? [10:47] vila: only one is important [10:47] nomatter001: is it in the online repo ? [10:47] vila: the other ones have no changes i still need (all important ones are in trunk and online) [10:47] no that one is not online [10:48] but if it doesn't work i have the code (outside of the repo online) [10:48] ok, but all the changes are still in the working tree, right ? [10:48] vila: yes [10:48] nomatter001: try running 'bzr revno' in this branch [10:49] ok [10:49] its 50 [10:49] nomatter001: try branching this branch under the new repo, this will transfer all needed revisions from the broken repo to the new one OR fail [10:50] vila: ok i'll try right after fixing the testserver (last scriptrun destoyed code there after incorrect export) [10:51] nomatter001: meh, this is unrelated right ? [10:51] nomatter001: then ping me when you're back [10:52] vila: 1 minute [10:53] vila: so now i'll try branching [10:54] ok seems to work [10:55] thx a lot [10:56] nomatter001: good [10:57] perfect timing: its mealtime ^^ [11:34] hi. how can i preview what bzr up would do? [11:38] chx: bzr missing ? [11:39] ooooooo [11:39] nice. [11:39] thanks [11:39] chx, well, not exactly, but it should help [11:40] vila: yes, it helped [11:43] mthaddon: can you check the last submission on bzr pqm instance, it has not been merged but I got no email either [11:44] mthaddon: I've seen it run there but not until the end [11:46] vila: all I see is this, but that doesn't seem like it would cause a failure - https://pastebin.canonical.com/36445/ [11:48] * vila shudders [11:48] mthaddon: indeed, I can't see why it would change anything for this submission [11:49] mthaddon: but there should exist some more precise log of what happened for it no ? [11:49] mthaddon: or is it yet another email-too-big problem ? [11:50] vila: https://pastebin.canonical.com/36446/ [11:51] not sure why you wouldn't have received that... [11:51] mthaddon: great, that's a better explanation [11:52] mthaddon: this is a random failure I've seen for a long time, first time it happens on pqm though... I'll try to re-submit :-/ [11:52] k [11:53] mthaddon: double shudder, yeah, AFAIK, I'm the only one not receiving such failures, I *receive* some failures, but not all... go debug that... [11:53] spam filter? [11:54] mthaddon: me ? no. My ISP ? May be, but on which criteria ? Size ? spam filtered on size ? Hard to believe [11:55] delayed because of size maybe... === oubiwann is now known as oubiwann-away === oubiwann-away is now known as oubiwann [13:21] salve [13:22] !list [13:22] This is not a file sharing channel (or network); be sure to read the channel topic. If you're looking for information about me, type « /msg ubottu !bot » [13:59] mthaddon: sorry to bother you again, but re-submitting led to the same result: no merge, no mail :( [13:59] mthaddon: if you could confirm that the failure is the same for the same test, it would help [14:02] vila: sure, gimme a sec [14:03] mthaddon: a whole minute even :) [14:03] * vila tries to setup an hardy vm to reproduce [14:03] vila: yep, same error [14:03] same test then ? [14:04] hmm, I wonder what the return address is on pqm emails... may be some mailbox contain same feedback on mails not passing through... [14:05] ha, pqm@bazaar-vcs.org, mthaddon, is that a mailbox you can reach ? [14:06] nope [14:06] mthaddon: pff, silly me, that's the one that receive the submissions... I wonder what happened when it receives a "can't deliver" mail... [14:22] mthaddon: ok, I can reproduce the msg you got here, but that's not a failure and selftest succeds, there should be something else [14:22] i need help with diverged branches [14:23] i've got rev 43 on my local branch commited and another has rev 43 on their branch, both are connected with trunk [14:24] when i want to pull or push to trunk, the branches have diverged [14:24] amogorkon: both ? [14:24] bzr missing tells i'm missing one rev [14:24] dunno, at least on my side [14:24] when i try to merge, it says nothing to do [14:24] when i try to commit, there's nothing to do [14:25] amogorkon: do you have uncommitted changes ? ok, that answers my question [14:25] no conflicts [14:25] amogorkon: try 'bzr update' [14:25] Tree is up to date at revision 43 of branch /home/amogorkon/Dropbox/PiLife bzr [14:25] is that good? [14:25] meh [14:26] bzr missing [14:26] bzr info -v [14:26] check the later, it seems you should find that something is not as you think it is [14:27] amogorkon: and paste the ouput of 'bzr info -v', it will help to understand your setup [14:28] http://pastebin.com/5RXJz5Df [14:30] what does submit branch mean? [14:36] amogorkon: that's where you want your changes to land and is also the default merge source. Generally, that's trunk [14:36] that.. is the cause of the issue then, i suppose [14:36] how can i change that? [14:37] could be. [14:37] edit .bzr/branch/branch.conf and either remove the submit_branch line or put your trunk url there [14:39] bzr send --remember also changes the submit branch location. [14:39] many thanks [14:39] i'll try that [14:40] now if 'bzr missing' says you're *missing* one rev, 'bzr pull -v' should pull it [14:41] amogorkon: are your sure 'bzr missing --mine-only' is empty ? [14:42] vila, now it displays me "1 extra revision" - the one i commited previously [14:43] amogorkon: then you should push it [14:44] vila: would you be interested in a bug report with the tests that hit hangs with your leak fixes? [14:44] so we don't lose track of them even if we can't fix them now. [14:44] mgz: indeed [14:44] yeah, i think it works now [14:44] mgz: if only to see we get the same ones... [14:45] another 40 min of my life spent fixing diverged branches :p [14:45] thanks anyway :) [14:45] I'll set up a full before and after run now. [14:46] amogorkon: may be worth changing your workflow :) [14:48] vila, i'll take it into consideration if this submit branch thing doesn't fix it [14:51] okay, thanks again [14:51] * amogorkon bows out [15:14] urk, syntax error in test_config I take it you know about vila and that change to the makefile was involving? [15:15] mgz: yes and yes [15:15] mgz: the problem *right now* is that I can't land the fix because pqm is failing somewhere else [15:15] what's the patch? [15:15] I'll apply it locally so I can do this comparision [15:15] mgz: and since I don't get email failures I don't know where [15:16] mgz: lp:~vila/bzr/integration [15:16] ta. [15:16] hmm, now I know which test is failing....tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh [15:19] oh my, what a mess. I succeds on lucid/py26, fails on hardy/py24, succeeds on hardy/py25 with a traceback, succeeds on karmic/py24, failed on karmic/py25 (like hardy/py24), succeds on karmic/py26 (with the same tb than hardy/py25) [15:20] s/I succeds/It succeeds/ [15:21] quite interesting for a test but obviously it's trying to test too much at once.... [15:23] ...and poolie mentioned it this morning on maverick... [15:26] jam: ping, pp advice needed [15:26] hi vila [15:26] what's up? [15:27] I sent a submission this morning with a syntax error that pqm happily accepted [15:27] that was the lockable config files one, 3 things happened then [15:27] vila: yeah, I've run into that in the past, too [15:28] complained at the time, not much happened, I guess [15:28] 1) I sent the leaking test submission (accepted too on top of the previous syntax error merged cleanly) [15:28] 2) I noticed that pqm was damn quick and filed bug #626667 [15:28] Launchpad bug 626667 in Bazaar "syntax error doesn't prevent merge (affected: 1, heat: 6)" [Critical,In progress] https://launchpad.net/bugs/626667 [15:29] 3) I fixed the syntax error and submit the fix [15:29] The fix is at lp:~vila/bzr/integration right now [15:29] This last submission wasn't merged and I didn't get the failure email [15:30] I've been able to reproduce locally (make check PYTHON=python2.4) and this generates a 47M selftest.log which I suspect leads to a too-big email that is dropped somewhere between pqm and me [15:31] Anyway, the offending test is bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh [15:31] And as I mentioned above: [15:31] oh my, what a mess. It succeds on lucid/py26, fails on hardy/py24, succeeds on hardy/py25 with a traceback, succeeds on karmic/py24, failed on karmic/py25 (like hardy/py24), succeds on karmic/py26 (with the same tb than hardy/py25) [15:31] okay, I'm going out but have full run of r5395 and r5396 going with the syntax error fix [15:32] I feel a urgent need to disable this test to unblock pqm :) Thoughts ? [15:32] mgz: and ? [15:33] and nothing till it finishes, I guess :) [15:33] jam: I feel a urgent need to disable this test to unblock pqm :) Thoughts ? [15:33] mgz: ha, I thought you had the results not that you launched the runs :) Cu ! [15:34] vila: I'd feel best if you reverted to old code and worked back up [15:34] jam: my gut feeling is that this is a latent bug revealed not *caused* by the leaking fixes [15:34] so instead of just disabling it, submit a revert patch [15:35] that goes before the original borked submission [15:35] otherwise I can look more closely [15:35] if you want [15:36] I'd prefer that yes, that would be more productive, I'm filing a bug right now with details [15:36] this test have been failing intermittently for... months across all the platforms on babune [15:36] vila: given that you have 3 + patches that you don't actually know if it passed ... [15:37] jam: no, I just re-run the tests on my pqm-like setup and that's the only one [15:37] I'm not worried about that one test, but if you have 45MB of test failure [15:37] that is indicative of many tests failing [15:38] jam: no, only one test failure, the 47M are the *normal* subunit output [15:38] vila: here it is only 7MB, are you sure? [15:38] I've gotten several :) [15:38] 7MB compressed ? [15:38] possibly [15:39] well, compressed and then base64 encoded, I think [15:39] 47M uncompressed unfiltered here [15:39] jam: subunit stream or | subunit2pyunit stream ? [15:40] pqm-stdout.gz IIRC [15:40] note the discussion about whether we should remove 'logs' from skipped tests [15:40] jam: I have never been able to receive one so I really can't tell [15:40] jam: that's why I mentioned the 47M :) [15:41] jam: but getting rid of only the skipped ones will not help enough I'm afraid [15:41] My mailhost is configured to 10MB attachments [15:41] not sure about yours [15:41] default is often smaller, like 2-5MB, IIRC [15:43] I'd like you to chime in there [15:43] since being unable to get PQM output is pretty serious [15:43] jam: bug number ? [15:43] vila: merge proposal [15:44] https://code.launchpad.net/~jameinel/bzr/2.3-filter-tests/+merge/33938 [15:44] hmm oh yes, I thought it was on testtools [15:45] well, lifeless saying "you really should have the log on skips" ... [15:46] given that we used to have a single "reason" line, that is now 50 lines of reason + formatting + log [15:46] not to mention subunit overhead [15:48] Well, I'd be happy to get this stream. I don't really like getting *less* details to debug. Skipped tests are only a minor part of the whole... [15:48] jam: can you check which stream you get on failures ? [15:48] I often can't get *failure* emails because they go above 10MB [15:49] pqm-stdout.gz is the interesting stream [15:49] for the subunit stream, everything is in the same output file [15:49] jam: yes, I don't have such mails, can you open or forward me one of yours [15:49] well, I can't forward it, since you can't get them :) [15:49] but I can post it somewhere :)( [15:49] * vila slaps himself [15:59] jam: further evidence involving this test: bug #579530 [15:59] Launchpad bug 579530 in Bazaar "paramiko does not try all available address families (affected: 1, heat: 3)" [Medium,Confirmed] https://launchpad.net/bugs/579530 [16:00] think it is a ipv6 vs v4 thing? [16:00] jam: not sure in the case at hand, but it confirms that the tests is trying to test too many things at once [16:12] How can I get my bazaar hook to access branch specific config values? Specifically, those stored in branch.conf. [16:12] http://dpaste.com/236288/ [16:12] http://dpaste.com/236289/ [16:18] vila: on chinstrap:~jameinel/pqm-stdout.bz2 [16:19] Nevermind. The push_result.local_branch or push_result.master_branch is the target of the push. [16:30] jam: 'time: 2010-08-24 16:55:43.668882Z' is enough identify it as a subunit stream right ? And it's 45M uncompressed anyway [16:30] vila: the log is in the subunit stream *and* 'bzr selftest' normal [16:30] if that is what you are asking about [16:30] so also in "subunit2pyunit' output [16:31] and 7MB of stuff we won't look at obscures the stuff we do care about [16:31] well, 47MB from what you said :) [16:31] jam: but how do you the stuff is not relevant until you understand the failure ? [16:32] jam: but how do you know the stuff is not relevant until you understand the failure ? [16:32] vila: not failures, *skips* [16:32] and test-not-applicable [16:32] and xfail [16:33] is what *I'm* trying to change [16:33] not actual failures [16:33] basically everything that wasn't *success* was getting the log attached [16:33] then why not keeping *only* failures ? [16:34] well, that and errors [16:34] but I think that is ~ what I've done [16:37] though doing it as a blacklist rather than a whitelist [16:37] anyone know of a wiki that uses bazaar as a backend? [16:38] jam: no, you said that you only get rid of the skipped tests, this account for ~10%, this doesn't even guarantee that *I* will get email for failures [16:38] itamar: wikkid and ikiwiki both can [16:38] itamar: http://launchpad.net/wikkid, http://www.ikiwiki.info/ [16:38] ikiwiki had some caveats I didn't quite understand [16:38] vila: well, skips and xfail, I think account for most of it [16:39] at the least, it will account far a fair amount of shrinkage [16:39] though yeah, probably the subunit overhead might be part of it, too [16:39] I'll look at wikkid [16:39] thanks! [16:39] jam: I've been trough enough subunit/testools bugs on xfail and skip to know that they represent only a fraction of the tests... [16:40] err, xfail, not applicable not skipped [16:40] err, xfail and not applicable not skipped [16:40] vila: 2k out of 21k is a significant fraction, I'm trying to determine the verbosity now [16:42] jam: will that be enough to goes below the threshold that blocks the emails pqm is trying to send me ? [16:42] ah, I see what you mean, I didn't realize success also had a log, since it isn't output by selftest -v, which is where I really cared about it [16:42] how about an option for the logs ? [16:44] AAAAAAAAAARGHHHHHHH [16:44] vila: ?? [16:44] jam: who was able to land parthm submission ? [16:44] other than parth? [16:45] anybody ! Currently pqm will accept *any* submission since they all inherit the syntax error [16:45] well, anybody than can submit that is [16:45] and of course it *has* to refuse the one that fix the error.... [16:46] bah, yes of course, this the one that fixes the error is the only one that runs the test suite... [16:48] 5397 Canonical.com Patch Queue Manager 2010-08-30 [merge] [16:48] (parthm) bzr homepage url fixed in docs. (Parth Malwankar) [16:48] looks like parth did it [16:49] anyway, yeah, we should do something quick to get pqm back to correct. either your patch or the idea of set -e. [16:49] set -e ? [16:49] I think for right now, just revert back pre-fix, add the Makefile fix [16:49] 'set -e' in bash says if any command fails with non-zero, exit immediately [16:50] I don't think we can do that from/for make [16:51] ha come on, the leaking test fix have been waiting for far too long and I'm 98% sure it doesn't cause this failure. Don't let the better be the ennemy of the good :-( [16:51] vila: we have failures in what was submitted, we don't know how much [16:51] jam: have you *looked* at this test ? [16:52] of what was submitted since then [16:52] I'm not worried about your 1 test [16:52] I also don't know why it would fail now when it has been passing for *years* [16:52] no, I said it already, only one failure, this test, reproduced locally [16:52] vila: how do you know if you aren't getting pqm failure reports :) [16:52] ? [16:52] I can propose various explanations, and no, it has not been passing for years, it has been *failing* randomly for years [16:53] other than knowing that you have an occasional failure locally which doesn't seem to have been happening on pqm [16:53] because I don't get my submissions landed either [16:53] vila: I don't think I've had a submission kicked back because of that test that I can think of [16:53] and I've submitted a lot [16:53] vila: right, but you don't know why *PQM* think you can't land it [16:54] read the logs, mthaddon gave enough info for me to reproduce [16:54] vila: set -e seems to work for me in Makefile [16:54] vila: so if it *is* random, then you should also be able to resubmit and have it work [16:54] ok, I resign, you're the patch pilot, go, send a revert-fix and don't forget to also unmark the bugs fixed === beuno is now known as beuno-lunch === Meths_ is now known as Meths [17:02] vila: I'm not trying to be extra difficult, but what you're saying doesn't match up to what I understand. You ran with python2.4 on your machine, and that was the only test that failed? [17:03] vila: also, I think we still have a bug with your fix that if the selftest.log has content, but has *no tests* then it still succeeds by accident [17:03] jam: yes, starting with bzr.dev as current before parthm submission and adding the fix in lp:~vila/bzr/integration, aka skipping this test until it gets rewritten as proposed in bug #626876 [17:03] Launchpad bug 626876 in Bazaar "tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh considered harmful (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626876 [17:04] jam: right, but that's a step in the right direction and plug a hole that have been present for *months* [17:05] Is there anyway to get a launchpad type interface for locally hosted branches? I ask, because I really like the way lp's UI deals with branches and their relationships, but my boss will never go for a third party hosting our code, ever. [17:05] vila: the test really isn't that big, all it is doing is implementing an ssh server, which you could pull out anywhere you want if the number of lines concerns you [17:05] the test *is important* to make sure that we actually hook up the whole via ssh code [17:05] rubbs: well, worst case you could install launchpad :) [17:05] since *all the other tests* skip it and talk over a loopback [17:06] jam: I fully realize how important it is, but it fails far too often for far too many different reasons [17:06] dash: is there documentation on how to do that? I knew it was Open Source, but I couldn't find any way to do that itself. [17:06] vila: we need an (single) integration test or we have just as big of a whole [17:06] We've certainly in the past broken the ssh connection code [17:07] rubbs: there is some stuff around here: https://dev.launchpad.net/ [17:07] but the big thing is, you have to create your own icons because the lp ones are trademarked [17:08] (similar to CentOS vs Redhat, and whatever Debian did with Firefox, IIRC) [17:08] jam: icons are no big thing. We got a designer here with little to do atm so I can give him that task [17:08] jam: thank you very much. [17:08] jam: I realize that, but the actual test is brittle, I've said it for months as I had to diagnose several failures where it was involved and delete/re-run countless jobs on babune just because it's capricious [17:09] jam: and I'm not surprised to see it fail on pqm as fixing the leaks shakes the socket handling code BIG times [17:09] vila: but then you remove one test that covers the actual case that people care about (nobody cares if you can connect via a loopback :) [17:09] jam: I don't want to suppress it, I want to replace it with several tests to get a better isolation of the failures [17:10] dash: jam: Oh, this is perfect thank you. I'll look into this right away [17:10] vila: the only thing it does after creating an ssh server is 'mkdir' [17:10] jam: at least one slave on babune is running a version of paramiko provided by spiv *explicitely* to *not* skip this test already [17:10] (well, creating one, connecting to it, and then issuing one command) [17:10] jam: and it currently fails for a ConnectionReset ! === Ursinha is now known as Ursinha-lunch [17:12] vila: which honestly sounds like we have a bug in our connection handling... [17:12] vila: anyway, my specific point... how can you get more granular on that test [17:12] jam: yeah, one that is not reproducible on all platform/python combos ? [17:13] I really don't see a way to rewrite it that isn't *connect to an ssh server* [17:13] vila: for example, on Windows and different versions of Python, they've had OSError.errno change from being the *windows* error, to a regular errno [17:13] (ENOENT versus WINDOWS_ERROR_NO_ENTRY as a not-quite accurate example) [17:14] vila: and I know we've had spew to the terminal for a while on Windows because when the client disconnected we would raise the exception rather than suppress it as we did on other platforms [17:14] (but it wouldn't die because it was an exception raised in a thread, etc) [17:18] well, separating a simple connect/disconnect first, then the simplest hpss handshake and so on up to mkdir [17:18] vila: I honestly doubt it is failing because of the mkdir + stuff [17:19] jam: exactly why a better isolation will be achieved by *not* having it [17:19] vila: if you really thought that mkdir was the problem, you could have trivially fixed that by now. I have the feeling you feel it runs deeper [17:19] also ,we only really want 1 test that exercises the handshake code [17:20] since it is fairly slwo [17:20] slow [17:20] if you want to pull out mkdir, I don't care *at all* [17:20] I care that we connect to a real ssh server [17:20] and this is test is lying anyway, there is no process launched, it's all in the same process and fixing leaks revealed that python + sockets + client + server in the same process... can lead to exceptions being thrown at different points from one excution to the other [17:22] my point is: I think we'll be better served by disabling this test until it's fixed (which can only occur soon since it's Critical) than reverting all the work already done and discussing endlessly while pqm is accepting any patch :) [17:24] (I think your optimism about 'only occur soon' is a bit high :) however, I am looking over your integration patch, etc now while we're discussing this [17:24] and I've queued up some stuff in pqm, as well [17:24] meh, it spwans a process [17:25] jam: you won't be able to land anything once you fix the syntax error [17:25] jam: unless you revert the leking tests submission that is, which doesn't mean that test won't fail either one day or the other [17:26] that's the whole point of fixing the leaks, selftest can blow up at any time as mgz realized lately [17:26] it hasn't occurred on pqm so far, we're still lucky [17:28] not fixing the leaks also means windows test can't be fixed using babune, osx slave can't be added on babune, etc === jam changed the topic of #bzr to: Bazaar version control | try https://answers.launchpad.net/bzr for more help | http://irclogs.ubuntu.com/ | Patch pilot: jam | bzr 2.2-final has gone gold, build those installers [17:31] vila: so the typo... it is only a python2.4 typo? As 'test_config' loads just fine on python2.6 [17:32] jam: 2.4/2.5, pqm runs in an hardy chroot with 2.5 as the default python but we override that by running 'make check PYTHON=python2.4' as checked with mthaddon earlier [17:33] jam: I discussed the fix with poolie and spiv before sending it, short-circuiting the mp to go faster.... [17:33] vila: the typo fix? [17:33] yes [17:33] sure, I'm certainly fine with the change, just trying to understand, as it looks like that fix is already in pqm, but you said it had failed [17:34] huh [17:35] vila: well, I was trying to submit the 'make check' fix you had, but it is behind your 'integration' fix [17:35] which claims to be at 50% complete [17:36] yeah, I send that when I saw parthm submission before realizing why it was useless (unless the failure is random...) [17:37] jam: check your setup, the test_config.py fix is not in bzr.dev AFAICS [17:38] vila: Oh I agree, as I said I'm running 2.6 and wasn't sure why it was a problem === deryck is now known as deryck[lunch] [17:45] GaryvdM: ping, don't upgrade your bzr.dev branch yet, pong me first [17:50] GaryvdM, bialix: can you test lp:~vila/qbzr/config-get-filename-deleted before upgrading bzr.dev ? [17:52] jam: if you submitted straight from my branch it should fail, I'd be interested to know if you get a pqm failure mail though :) [17:53] vila: well, it has passed 18k tests so far, and is on: bzrlib.tests.test_bundle.V08BundleTester.test_binary_bundle [17:53] not sure if that is before or after the test in question [17:56] it's the 24199th test out of 24758 [18:01] vila: well doesn't that suck :) === beuno-lunch is now known as beuno [18:01] jam: I had a brief, very brief, instant of pure joy this morning when I saw my submissions land in no time [18:02] that was quickly followed by the depressing understanding... === abentley_ is now known as abentley [18:02] ..and the problem is still not fixed :( === deryck[lunch] is now known as deryck [19:07] gra, r5396 didn't run, thought it'd merge over the syntax fix but it lost it [19:07] * mgz runs it now === Ursinha-lunch is now known as Ursinha [19:39] * jelmer waves [20:02] vila: I did get the failure with attachment this time [20:10] vila: and it is giving a "ConnectionReset" error after spending ~5s (the test runs in 5.6s, I assume that is a timeout of some sort) [20:11] and as near as I can tell, that is the server failing to actually respond to the 'mkdir()' request... [20:12] log file is in the same plcae [20:12] plcae [20:12] place [20:12] (if my fingers could type) [20:30] jam: glad you got this email :) [20:30] jam: but it's failing here in 0.533s so not the kind of timeout you're suspecting [20:30] vila: so 'set -e' works as I wanted it to [20:31] I get the failure early rather than late [20:31] jam: good, as long as it works everywhere, I'm happy [20:32] vila: well at this point 'test_bzr_connect_to_bzr_ssh' seems to be failing reliably (though it also happens in 665ms here [20:32] jam: here ? [20:32] I just got it by only running test_transport [20:32] on my machine [20:32] and on pqm [20:32] python and paramiko versions ? [20:33] vila: http://www.davidpashley.com/articles/writing-robust-shell-scripts.html#id2384206 [20:33] python 2.6, paramiko ... ? [20:34] regarding the interaction with leaks, the previous sftp server was just leaving threads around never joining them, so this may have hidden the problem [20:34] 1.7.5 [20:34] jam: can you add that to the bug report please [20:34] jam: and that's with my fixes for leaks right ? [20:34] vila: that is current bzr.dev [20:34] ok [20:35] jam: only the traceback or more ? [20:39] vila: I don't understand "only the traceback" [20:39] * vila on phone [20:39] jam: but look at bug #628876, there are a lot of variations [20:39] Error: Could not parse data returned by Launchpad: 628876 (https://launchpad.net/bugs/628876) [20:39] fullermd: you use csh? (well, something other than bash), right? Do you know if 'set -e' works there? [20:40] vila: wrong bug, I think [20:40] 628876 isn't a bug [20:40] bug #628876 [20:40] Error: Could not parse data returned by Launchpad: 628876 (https://launchpad.net/bugs/628876) [20:40] meh [20:40] vila: bug #626876 [20:40] Launchpad bug 626876 in Bazaar "tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh considered harmful (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626876 [20:41] bug #636876 gettubg there [20:41] Error: Could not parse data returned by Launchpad: 636876 (https://launchpad.net/bugs/636876) [20:41] you had a 6=>8 [20:41] twice :) [20:41] can type and phone :) [20:41] cant :) [20:41] vila: everything I saw looks like a variant on connection reset [20:42] jam: there are other tbs sometimes [20:42] in your python2.5 it is raising EOFError (aka, no more data, aka connection closed) [20:42] that can be ignored [20:42] on which platform ? [20:42] vila: https://bugs.edge.launchpad.net/bzr/+bug/626876/comments/1 [20:42] Launchpad bug 626876 in Bazaar "tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh considered harmful (affected: 1, heat: 6)" [Critical,Confirmed] [20:43] jam: that's freebsd [20:43] vila: everything looks like connection reset, just variants on how well the python + paramiko + bzr stack handle translating it as such [20:44] jam: we shouldn't lose the connection, that's the root problem [20:45] jam: and I would happily suspect my fixes if there wasn't *successful* tests [20:47] so I'm more inclined to suspect the bug to reveal itself more blatantly now, since, after all, this is what happened numerous times while debugging the leaks, [20:47] vila: well it fails if I run the test by itself, so I'm not sure that it has anything to do with leaking threads... [20:48] jam: have you tried to establish how many threads/sockets/pipes are involved in this test ? [20:48] have a guess [20:49] Approx 5 [20:49] but paramiko does threads also, so I'm not positive [20:51] I'd say 2 processes, 7 threads, 3 sockets and 3 pipes [20:52] we should be able to get down to 2 processes 3 threads 3 sockets and 0 pipes [20:53] vila: perhaps, but it certainly passes @ -r5395 and fails @ -r5396 [20:53] that's a fact, but it says little about the cause [20:54] vila: I think I found it [20:54] vila: http://paste.ubuntu.com/485997/ [20:54] note that you don't actually save a reference to *test_case_server* [20:55] nope, not it [20:55] at least, not trivially so [20:56] that would not explain a succesful test [20:56] vila: what successful test, btw [20:56] also, I don't seem to be able to get trace information in my log file from the sftp thread, any ideas why? [20:57] https://bugs.edge.launchpad.net/bzr/+bug/626876/comments/2 [20:57] Launchpad bug 626876 in Bazaar "tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh considered harmful (affected: 1, heat: 6)" [Critical,Confirmed] [20:57] also succeds on hardy/py2.5/paramiko-1.6.4 with a traceback from a thread that can be ignored [20:57] - succeeds on karmic/py26/paramiko-1.7.4 with the same traceback than hardy/py25 [20:58] all of them mentioned in bug #626876 [20:58] Launchpad bug 626876 in Bazaar "tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh considered harmful (affected: 1, heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/626876 [20:58] Ha ! Got it right ! [20:59] all of the run involve 0 to 2 failures in ferry_bytes (including the ConnectionReset except this one comes a bit later in the code) [21:00] vila: I'm also getting: No handlers could be found for logger "paramiko.transport" [21:00] which would hint that paramiko is trying to warn us about something, but the logging module isn't initialized [21:02] which may be "I couldn't actually start up as requested" [21:02] that's not in your comment or is it ? [21:02] givien the other hints [21:02] vila: not sure [21:02] I probably didn't paste that part [21:03] ok, I got it too on some combinations [21:03] vila: I get it much before the final traceback [21:03] yeah, at the start of the run, probably when loading [21:04] where I get it it's about Random.Pool being BROKEN or something [21:04] vila: nope [21:04] if I add a "trace.enable_default_logging()" in the middle [21:04] nope what ? 8-) [21:04] I get: http://paste.ubuntu.com/486002/ [21:04] vila: not during import [21:04] maybe during import of a subprocess or something [21:05] vila: that is with this patch: http://paste.ubuntu.com/486003/ [21:05] Basically, on my machine, I'm not *getting* to ferry_bytes [21:05] it fails first [21:08] oooh, right, yet another different failure ! Fireworks ! [21:09] vila: well still a connection failure [21:09] but still a problem while client and server are trying to communicate [21:09] with this patch: http://paste.ubuntu.com/486003/ [21:10] on my machine, I'm not even getting into check_channel_exec_request I think [21:10] at least, I am unable to log what is happening there [21:10] not the way to go then ? :) [21:13] vila: I fixed logging, a bit more info now: [21:13] http://paste.ubuntu.com/486008/ [21:13] paramiko is trying to say: Socket exception: Bad file descriptor (9) [21:14] right, that's still match the: "someone is trying to read or write at the wrong time" pattern and paramiko use timeouts [21:15] jam: no traceback ? [21:15] vila: doesn't seem to be one on the server side [21:15] trying to sort it out [21:15] Connected (version 2.0, client paramiko_1.7.5) [21:15] Connected (version 2.0, client paramiko_1.7.5) [21:15] Twice ??? [21:16] vila: I'm playing around with trace stuff, so it is possible I'm just getting 2 outputs to stderr [21:17] but yes, it does say that [21:19] vila: if I change paramiko to 'raise' where it was just logging I get: http://paste.ubuntu.com/486010/ [21:21] here is my traceback on babune: http://paste.ubuntu.com/486012/ ferry_bytes is involved [21:22] forget about the .bzr.log error [21:25] meh, no .bzr.log error there, forget the 'forget' remark ;) [21:26] vila: 'failed to load compiled extension' is a bit surprising [21:26] the weird thing is that I ran this on windows [21:27] jam: fresh branch [21:27] we don't care about extensions for this test [21:28] hmm, indeed, and it was failing already: http://babune.ladeuil.net:24842/job/selftest-windows/139/#showFailuresLink [21:29] I just didn't notice it because it was down to 7 failures on windows (from 306 but for runs interrupted by can't start new thread) [21:31] anyway, way too late to continue, EODing [21:32] http://babune.ladeuil.net:24842/job/selftest-windows/138/testReport/junit/bzrlib.tests.test_transport/TestSSHConnections/test_bzr_connect_to_bzr_ssh/ [21:33] is evidence that we *had* a failure with ConnectionReset *before* my patch [21:34] jam: so make sure it was passing for you on *windows* *before* my patch [21:34] vila: it passes [21:34] if I revert [21:34] fails after [21:35] -r5395 passes, -r5396 fails [21:35] jam: Not as set -e, no. There's an -e command line arg just like sh. Why? [21:35] rha, right, same branch, I'm too tired [21:35] fullermd: trying to get "make check" to fail if there are any failures. I don't think it is required to run a bash, though [21:36] fullermd: we're having the bug that the right kind of error will make PQM think everything is ok [21:36] Well, it better run a sh. set -e is standard on Bourne shells AFAIK. [21:36] (currently a syntax error) [21:42] jam: do you need any help with this [21:43] fullermd: I think its pipefail thats not set [21:43] fullermd: -e affects the results of a pipe, not the steps [21:45] fullermd: yup, as lifeless said, the root cause is an action in a makefile: cmd1 | cmd2 [21:45] fullermd: cmd1 fails, cmd2 succeeds, make is happy [21:46] fullermd: how will you convince make to abort when cmd1 fails ? [21:46] * fullermd isn't sure why people are telling me ;) [21:46] vila: pipefail [21:46] fullermd: oops, it's GNU make :) [21:47] vila: man bash - look for pipefail [21:48] lifeless: just found it, but it's bash, if we require bash in addition to GNU make.... well, that may work [21:48] you're using bash [21:48] make doesn't have a shell interpreter [21:48] _I'm_ not using bash. [21:48] fullermd: you aren't you're [21:48] make use ${SH} [21:49] true [21:49] fullermd: but is it always available on freebsd > [21:49] anyhow [21:49] Yes, but make won't use it. [21:49] easiest way - put the test command in a bash script [21:49] call that script [21:49] SHELL=/bin/bash in the Makefile and it will [21:49] There is no /bin/bash on all sorts of systems. [21:50] fullermd: this is just for pqm, so that point, while entirely true, is irrelevant [21:50] easier : https://code.edge.launchpad.net/~vila/bzr/626667-check-no-docs-swallow-errors/+merge/34059 [21:50] lifeless: ! could be [21:51] lifeless: but could fail if the log file is not empty.. somehow [21:51] * fullermd just sees this as a good reason to keep working on his make(1) replacement :p [21:53] fullermd: You and everybody else? ;-D [21:53] There actually are like two or three good autotools/make replacements. [21:53] Well, sure, but mine is obviously the important one. Everyone else is just marking time 'till I'm done :p [21:54] fullermd: lol [21:54] I don't really need to build a replacement for autotools. I can slit my wrists with a rusty spoon already. [21:54] LOL [21:56] I wasn't overly thrilled with any of the make replacements I saw. Hence, firing up the ol' Arrogance Engine. [21:56] Heh. :-) [21:56] Well, nothing wrong with that. :-) [21:57] Part of the problem is that I don't want a "build system", so scons and the like are out. I really do just want a make(1), except without the suck. [21:57] jam: so, it seems bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh has never been run on babune except for my tests for leaks where it has always failed :) [21:58] jam: so good luck with it, see you tomorrow, keep me informed of your progress ! [21:59] g'night all [22:16] lifeless: I think we have a handle on how to make 'make check-nodocs' fail (set -e works, as would Vincent's suggestion, or figuring out how to set pipefail). [22:16] the problem right now is do we disable a test that used to pass, but was a bit brittle in some cases, but now *always* fails [22:16] I don't think I've ever had it fail on PQM, but it now fails on PQM and on my machine (where it also used to pass) [22:17] self.skip ? [22:18] I mean, mark it as a skip? would that do? then once pqm is locked down safely again, start landing attempts that un-skippify it [22:24] lifeless: yeah, to get it unblocked I'll probably do that. I was hoping to actually fix it [22:24] but since I'm going off the clock in about 5min, I'll have to take the easy route [22:28] there was a file in my directory that somehow got renamed to "\", and when I did a "bzr add ." it would crash bzr trying to add that file [23:22] okay, film was fun. seems like the entertainment has continued here as well.