[00:00] I've added (and its landing now) an earlier cache of revision trees, but that doesn't help here because its not a revision tree [00:01] and as the tree is modified it seems to me it will be doing the wrong thing - this could be exacerbating up-thread/down-thread/update conflicts as well, if I understand it correctly. [00:05] lifeless: I don't understand jam's change, but it looks wrong in two ways. [00:06] ok; I'll file a bug about this, as its not 'just a locking issue' then. [00:06] lifeless: You shouldn't call set_base_revision if you don't want to set the base tree, and you shouldn't set base_rev_id for the revision-id of a WorkingTree. [00:07] lifeless: For working trees, Merger does know about a basis revision, e.g. Merger.this_basis and Merger.other_basis. There's no corresponding variable for base trees, but that's what he would want to set. [00:09] lifeless: The smallest change that would get this working would be to set Merger.base_rev_id directly instead of calling set_base_revision. [00:10] lifeless: Confirmed, that passes. [00:10] abentley: thank you. It could have fall out elsewhere though? [00:11] lifeless: It's conceivable. Most likely in a test case where we had coded it to expect the wrong behaviour. [00:11] ok. I was doing this locking stuff as a warmup, so I'll put this bug in for now, and come back to it later. [00:11] do you have a diff of that change you just made? === beuno-afk is now known as beuno [00:14] lifeless: http://pastebin.ubuntu.com/259556/ [00:22] thanks [00:38] win 83 [00:59] sorry about bug noise on 305006 [00:59] trying to get lp to do what I need >< [01:22] igc: when you get here [01:22] igc: my faster commit was broken; it should have been faster; fixed and landing now [01:23] igc: so you may want to retest it speed wise :) [02:04] morning [02:04] lifeless: shall do [02:05] anyone want to point me in the right direction for using bzr to merge two heretofore unrelated files? [02:07] hm [02:07] now that i say that out loud, I don't see how it would reasonably be possible [02:07] hand-merge, here we come... [02:08] igc: its landing at the moment [02:08] lifeless: excellent. well done! [02:08] http://pqm.bazaar-vcs.org/ [02:09] lifeless: as soon as I wake up this morning, I'll give it a spin :-) [02:09] kk [02:09] it wasn't using specified_files in iter_changes [02:09] which is why 'partial commit' was only as fast as full commit === kiko is now known as kiko-zzz [02:11] poolie: my focus today will be on reviewing critical/high patches and ... [02:12] poolie: announcing explorer 0.7 + fastimport 0.9 [02:12] hopefully james_w or someone can get those uploaded to karmic in time [02:17] igc, that sounds good [02:17] mine is to merge the 2.0b1 branches and get that out [02:30] igc: its landed [02:34] lifeless: sweet [02:35] lifeless: so IIUIC, shelve/unshelve now work correctly on Windows? [02:35] igc: yes [02:35] lifeless: landed? [02:35] or rather [02:35] unit tests say they do [02:35] yes, landed. [02:35] lifeless: wow, you rock! [02:36] also pushing locally [02:36] there are now no tests that should fail due to locking interactions on windows that represent things users can do [02:36] there are a few that we use to test race conditions [02:36] lifeless: you may want to email the bzr-windows list calling for testing on this [02:36] I've a branch up for review that guards those tests on windows more aggressively [02:36] igc: I'm not on the windows list - could you send such a mail? [02:37] lifeless: sure [02:37] lifeless: that's mega cool because I'll now get some interest in someone writing qsheve/qunshelve [02:37] we can of course still collide between separate processes [02:37] but same process stuff should be rock solid [02:38] lifeless: as long as they weren't working on Windows, there was limited interest [02:40] lifeless: pulling the 2.0 branch doesn't show your commit changes, nor the shelve-related changes [02:40] lifeless: are they only in trunk or is lp mirroring to blame? [02:40] only trunk [02:40] landing a big batch to 2.0 later [02:41] ok [02:42] ok time for brunch, or whatever it is :) [02:44] poolie: it's not a weekend, so I think yes, you need to have brunch before noon. Weekends you're ok until 13:30 :) [03:00] igc: my testing shows - 1.3 seconds for commit w/out iter_changes, 0.25s for commit w/iter_changes 0.36s to commit the full tree [03:00] igc: (alll with a single file changed) [03:00] lifeless: which project? [03:00] samba in 2a [03:00] the first two times are 'commit README', the last one is 'commit' [03:13] spiv: ping [03:13] IncompatibleRepositories: CHKInventoryRepository('chroot-97810960:///stack-on/.bzr/repository/') [03:13] is not compatible with [03:13] KnitPackRepository('chroot-97810960:///to/.bzr/repository/') [03:13] different rich-root support [03:13] thats what I'm generating at the moment [03:13] spiv: this isn't ideal, but AFAICT its roughly what we do elsewhere [03:14] and its at least in the category of 'class of thing we already have to fix'. [03:14] spiv: seem ok to run with this? [03:20] lifeless: ugh [03:21] What's the benefit we'd trade this against? [03:21] well at the moment its an UnknownSmartServerError [03:21] the client hasn't ever opened 'stack-on', so it can't translate it without going to the network. [03:22] and 'to' doesn't exist because the server is aborting a 'create foo for me' operation. [03:22] so in terms of benefits; it will error about 600ms faster [03:23] possibly my unit tests are not representative of what launchpad itself will do [03:23] I'm not really fussed about making that error faster :) [03:23] I think we need to fix the server to return relpaths fairly soon. [03:23] spiv: note that I'm going to be catching this in cmd_push *anyway* [03:24] this is 'bzr push a 1.9 branch somewhere with a stacking policy that points at a 2a branch' [03:24] a precondition on that is that I can catch the error; so my motivation here isn't 'make this error nice', its 'make it typed rather than 'unknown'' [03:25] IncompatibleRepositories can occur on other operations, although I agree that's the most common. [03:25] they can, and at the moment - well see above - unknown smart server error. [03:25] What does the error serialisation look like? I'm wondering if it will be reasonably easy to replace the useless in-proc URLs with relpaths later without disrupting old clients. [03:26] I argue that what I've done so far is an unqualified improvement, even if its not an ideal end-state. [03:26] ('Incompa..', str(err.source), str(err.target), str(err.detail)) [03:26] Yeah, I agree it sounds like an improvement. [03:26] but as the client doesn't process at all it would be easy to change later [03:27] The crummy URLs are essentially the same issue as in the lock contention message, I guess. [03:28] yes [03:28] I'm satisfied that this is an improvement. I'm a little worried it'll be an impediment to doing something better later. [03:28] there are two cases [03:29] either we need more fields, or we need to change the serialised value of fields [03:29] the client doesn't care about how many fields [03:29] so we can add some (including adding different representations of existing fields) [03:29] Ok, if you leave that wiggle room in the client that sounds ok. [03:29] and the client doesn't parse the fields, so we can change fields if we want- knowing that the client will just show the strings it receives [03:30] i.e. effectively reserving some fields for future use. [03:32] its no worse than [03:32] Using default stacking branch /~bzr/bzr/trunk at lp-45211856:///~lifeless/bzr [03:32] I feel you're very concerned about a special case of a pervasive problem. [03:32] which is a bit odd. [03:33] Oh, it is pervasive. [03:33] That doesn't mean we should spread it even further without thought :) [03:34] Actually, that particular message is extra crummy. [03:34] It's actually emitted on stderr by the server process! [03:34] the incompatible one? [03:34] Oh, actually, not that one. [03:34] merge review request sent. [03:35] I'm thinking of the 'Source format doesn't support stacking' one. [03:35] as we've discussed, I figure you've just volunteered to review ;) [03:45] spiv: [03:45] https://code.launchpad.net/~lifeless/bzr/bug-393677/+merge/10710 [03:48] spm: whats the url configured in pqm's bzr config for 2.0 ? [04:08] spm: ping [04:08] poolie: whats the url for submitting to 2.0? [04:09] lifeless: I'll kick off that benchmark now while I grab some lunch [04:09] * igc food [04:09] igc: its going to be awesome [04:11] lifeless: similar to the 1.17 & 18: bzr+ssh://bazaar.launchpad.net/~bzr-pqm/bzr/2.0 [04:11] spm: thanks [04:11] we need to doc this better. /me adds a yak to the list [04:20] spiv, hi, shall we catch up? [04:20] lifeless: do you think https://code.edge.launchpad.net/~ian-clatworthy/bzr/faster-dirstate-saving/+merge/7537 is good to merge now? igc says you read a previous version [04:22] poolie: the last one I read looked like it could cause confusion about the internal state; jam's comments were along those lines and I agreed. [04:22] I don't know if its been improved since then; and while its a functional improvement it is a risky area of code. [04:22] so it needs to be reworked? [04:23] jml: regarding the test suite [04:23] i don't think running it twice from pqm is, um [04:23] likely to be actioned very fast [04:23] so, i'd rather rely on buildbot, which is apparently now working ok [04:24] poolie: do you mean 'lifeless: regarding the test suite' ? [04:24] well, it is being run twice from PQM right now [04:24] no :-P [04:24] jml: now that we have a buildbot and are getting results from it i propose to just remove the second run from make check [04:25] and get vila to do that on a slave somewhere [04:25] poolie, that sounds fine [04:25] poolie: I have been watching carefully since we last discussed this [04:25] poolie: I haven't landed any utf8-related patches to see if it helps or not :P [04:26] i loked through all my recent pqm mails and didn't see any failures in ascii tests [04:26] but that may be unrepresentative because people tend to work in different areas [04:26] poolie: right, but what fraction were ... exactly [04:26] anyhow i'm not going to do it now [04:26] anyhow, I've voiced my opinion - to keep it - before. I don't think the gain is worth the pain if something slips through. [04:26] or not today [04:27] but I won't try to stop the change, if I can't convince you, the arguments aren't convincing [04:27] so, i respect your opinion, and i'm open to changing it back, but i think we should try changing it [04:27] i'm sure we get more breakage across platforms or distro versions [04:27] so we need buildbot to work well for that coverage [04:28] so i'd be ok relying on it for that too [04:28] s//for lang=c too [04:28] I get that. OTOH we've known about windows breakage for ages. It got fixed when it became possible for us to test it locally and quickly. [04:28] [locking specifically] [04:29] My experience with fix-later environments is that they are consistently lower quality than fix-before ones. [04:29] But as you say, open to trying the change and measuring. [04:31] so eventually it might be nice to test them all synchronously but in parallel [04:32] poolie: what url did you give pqm to land bialix's patch? [04:32] on 2.0 [04:32] i didn't [04:32] spm: whats the public address configured in the pqm conf? [04:32] i think vila sent it [04:32] * lifeless hunts yaks [04:32] lifeless: http://bazaar.launchpad.net/~bzr-pqm/bzr/2.0 [04:33] spm: thanks [04:33] * lifeless submitifies [04:37] how about https://code.edge.launchpad.net/~jameinel/bzr/2.0b1-402645-fragmentation/+merge/10621 [04:39] spm: can you just paste me the stanza please.. [04:39] econfused [04:39] poolie: jam and I don't agree about the short term fix [04:39] lifeless: https://pastebin.canonical.com/21499/ [04:40] I'm worried about unordered being pathological because of whats out there already; he's worried about preventing existing branches getting worse [04:40] its not a strong difference [04:41] but as we rather have to fix this properly (IMO) for 2.0 to be released, I don't feel a strong urge to land a bandaid: users of 1.16/1.17/1.18 are already plentiful [04:42] spm: can you check if there is a pqm job in process on there? (not via the web UI... I suspect I may have tickled a bug) [04:43] poolie: bug 418998 is probably the symlink dereferencing bug [04:43] Launchpad bug 418998 in bzr-svn "bzr crashes when adding .vim/* from my home directory" [Undecided,New] https://launchpad.net/bugs/418998 [04:43] poolie: rather than bzr-svn [04:44] the 'didn't add files' aspect will be the 'silently ignores subtrees' bug in bzr core [04:44] lifeless: technicall yes a job is running; but isn't a bzr one - if you ken what I'm not saying :-) [04:44] I do [04:44] can you check the log for requests from me [04:44] aye [04:44] oh ok [04:44] spm: actually. [04:45] spm: cron spam! [04:45] ha [04:45] bzrlib.errors.ConnectionReset: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist. [04:45] Permission denied (publickey). [04:45] (*^%)@*(#&$^)@#*(^%()*!&@%$_@#$&*())@*(#&$^@(*&^!!!!!!!!!!!!!!!!!! [04:45] spm: please be fixing that file, that file you love so well. [04:45] poolie: I replied asking if it was a symlink, that will tell us [04:46] spiv, tell me things? [04:46] lifeless: pls to try try again [04:46] bombs aaway [04:48] poolie: heading out to lunch in a sec. I've been responding to jam's review of my groupcompress batching branch. [04:48] arjenAU: hi [04:48] arjenAU: when are you in sydney? [04:49] spiv, so how is it overall? [04:49] spm: no go?! [04:49] * lifeless tries with http <-> http [04:50] lifeless: 7-9 Sep (7pm on the 6th) [04:50] arjenAU: I'm going to try to swing past the mechanics school [04:51] lifeless: hmm? [04:51] aren't you giving a talk? [04:51] at SMUG? [04:52] poolie: he saw some possible holes in the batching/caching interaction. I think I've fixed that up, although it's perhaps a little bit more complex now. It's looking good; I'll submit it for re-review by jam just in case, but I expect it'll get a thumbs up. [04:53] poolie: http://pqm.bazaar-vcs.org/ will hopefully land all the cumulative fixes from bzr.dev. If it fails, they are pushed to lp:~lifeless/bzr/2.0 so that people can tweak. [04:53] I don't expect failures - but theres > 10 patches in the set so it may find something to bail on [04:55] arjenAU: http://www.pythian.com/news/3728/sydney-mysql-user-group-meetup-9-in-depth-mysql-with-arjen-lentz specifically ;) [04:55] lifeless: oh right! sure [04:56] but if you want to get together and chat bzr/lp stuff before/after/some-other-timeslot that would be cool too [04:59] lifeless: so the code as it stands now only makes things worse and never better. Why wouldn't switching to 'unordered' be a reasonable short term fix? [05:00] jam: I'm concerned that it will also make things worse, just differently, as it will apply to all fetches too. And its a chance for more bug reports while we're doing the longer term fix. [05:00] lifeless: well I guess one positive - that file is back. So I suspect we know where it's coming from... [05:01] spm: wheres that then ... [05:01] lifeless: so.. if the source has been packed (ever) then at least we preserve that [05:01] my using lp: as the source for the merge, I'd suspect [05:01] jam: I'm mainly coming from paranois :) [05:01] it doesn't effect Packer [05:02] I need food [05:02] back shortly- jam, land if if you're quite sure we won't have fallout while we develop the longer fix. [05:02] lifeless: that was my guess, yes. [05:02] spm: so, what happened when we tried an empty file? [05:02] lifeless: given that we've been fetching pack => pack 'unordered' for a while.... [05:03] lifeless: kaboom [05:03] actually, I don't want to know. I want to debug it with you tomorrow and file copious bugs [05:03] heh [05:03] we can't reliably switch to lp until thats fixed. [05:03] right [05:05] arjenAU: but if you want to get together and chat bzr/lp stuff before/after/some-other-timeslot that would be cool too [05:05] poolie: EOD(); suspend(); [05:06] ok [05:07] i'm going to get as many of these things merged as we can then do a beta [05:07] unless something comes up [05:07] lifeless: we'd be done at 5pm, barack st in city. [05:07] sounds good [05:07] I'll tee it up with you next week [05:08] poolie: http://pqm.bazaar-vcs.org/ is good so far [05:10] jml: after work today can you do me a small favour. subunit progress review-or-rubber-stamp [05:10] lifeless, sure thing [05:45] abentley: pipeline ping [05:50] spiv, back? [05:54] poolie: yeah [05:54] i guess rather than nagging i want to know [05:54] when i should do 2.0b [05:54] so can you tell me either when you have nothing more to do with it, or alternatively [05:55] if there's anything i should be doing, or if it's pear-shaped [05:55] poolie: if 2.0b means 'we think we're done', in a few days [05:55] poolie: if it means 'get a snapshot of where we are at' - well anytime really. [05:56] my backport of the last weeks progress looks like it will land [05:56] in 30-40 minutes [05:56] it means we think there's no more critical bugs [05:57] ok, well I don't think we're going to be there today [05:57] So I'm currently working on the fix for bug 402657, by doing the last tweaks from jam's review. I'm going to ask him to re-review it, though, because he's touched this more than I have and I'd like to be cautious with this. [05:57] Launchpad bug 402657 in bzr "2a fetch over dumb transport reads one group at a time" [Critical,Fix committed] https://launchpad.net/bugs/402657 [05:57] Which means I expect it to land tomorrow morning. [05:57] well, if you submit it right now [05:57] I might get to it :) [05:58] that would be much better [05:59] Hmm, I'll see how quickly I can take care of the last little bits :) [06:02] thanks === thumper is now known as thumper-cooking === thumper-cooking is now known as thumper === thumper is now known as thumper-cooking [06:20] poolie: '(mbp) fix IndexError printing CannotBindAddress' [06:20] thats in my branch thats landing [06:21] might want to ask spm to cancel your merge [06:21] hm [06:21] why is it in your branch? [06:21] because I did what I said I would do earlier [06:21] which is to cherrypick all the 2.0 candidates from trunk [06:22] ah, ok [06:22] i understood you meant you'd do one merge with all of your changes [06:22] so spm would you please cancel my pending pqm job? [06:22] have a look at http://pqm.bazaar-vcs.org/ at my commit message :P [06:28] poolie: cancelled [06:28] oh thanks [06:31] spiv: I'm going to bed now. So better to do a good job then a fast one. [06:31] I'll try to review first thing when I wake up [06:31] sleep well jam [06:31] night all [06:31] thx lifeless [06:33] spm: please tell me you didn't cancel the running job [06:33] spm: (and mean it!) [06:33] jam: g'night [06:33] d'oh, missed him. [06:34] lifeless: I didn't cancel the running job. ?? [06:34] Just pushing it up now, anyway. [06:34] whew [06:34] lifeless: rm patch.1251263587 ==> star-merge http://bazaar.launchpad.net/~mbp/bzr/prepare-2.0 http://bazaar.launchpad.net/~bzr-pqm/bzr/2.0 [06:34] ah found it [06:34] poolie: its all landed [06:37] spiv, so what now? [06:43] poolie: Well, the review reply with a fairly small incremental diff (+68, -45) has been sent and I think addresses jam's concerns. [06:44] so.. [06:45] Gar, wireless dropout. [06:45] (on wired for the moment) [06:53] poolie: btw, record_entry_contents has exhaustive permutation tests. [06:54] do they pick up the case where it shouldn't record a new text? [06:54] yes [06:54] they pin it down very very precisely. [06:54] they assumed that the interface they were building on returned accurate data. [06:55] grep for unchanged in bzrlib/tests/per_repository/test_commit_builder.py === vila-dinner is now known as vila [06:57] ow, almost crossed jam... [06:57] hi all :) [07:01] hi vila! [07:02] poolie: Its my contention that only explicit tests for the problem would have exposed it - be they integration tests or unit tests of the interface. [07:03] poolie: I don't think it is a lack of tests on the record_entry_contents interface though; given accurate data it will behave well and is tested to do that. [07:03] given bogus data I think its extremely likely to misbehave - but thats why we have tests for the behaviour of tree :) [07:03] https://bugs.edge.launchpad.net/bzr/+bug/418931 spiv [07:03] Launchpad bug 418931 in bzr "AssertionError: _remember_remote_is_before((1, 18)) called, but _remember_remote_is_before((1, 15)) was called previously" [High,Confirmed] [07:11] poolie: I guess I'm trying to say that you're very unlikely to be able to create a test that when given valid data the api does the wrong thing [07:11] poolie: because I spent a bunch of time trying to make the interface tests as defensive as possible [07:31] poolie: reviewing that cf patch now and have some questions ... [07:32] ok [07:32] poolie: in repository.py, why the new exception wrt nested tree [07:32] ? [07:32] more context? [07:32] did that fall out of tests? [07:32] lines 98-100 [07:33] see https://code.edge.launchpad.net/~mbp/bzr/415508-content-filtering/+merge/10640 [07:33] it came out of testing it [07:33] it should never be hit, but trapping it here gives a more meaningful message [07:33] ok [07:34] poolie: also the docstring in workingtree4.py ... [07:34] code doesn't seem to match? [07:34] well match precisely [07:34] that it holds the hash and size? [07:35] i thought it did :/ [07:36] igc: how did the perf test go? [07:36] did it rock? [07:36] igc: also, all the stuff for windows users is in the 2.0 branch now. [07:36] lifeless: the one over lunch sucked because the lp mirror wasn't up to date :-( [07:36] lifeless: I kicked off another test 30 minutes ago [07:36] igc: ah, shame. eta? [07:37] lifeless: *this* time with your code :-) [07:37] lifeless: so, possibly with the test_commit_builder tests, i just want to add a new one asserting that it copes ok if the size and hash are null [07:37] none* [07:37] and if the size is none but the hash is known [07:38] and maybe for both those cases, with the case that the file actually either has or hasn't changed [07:38] poolie: So unknown I'd expect to be covered (I'd have to check though). But unknown size was not conceived of in the original interface definition. [07:38] poolie: so I definitely adding tests to cover the change to the interface is appropriate. [07:39] lifeless: full test is still running but the number you want just popped on my screen 10 seconds ago ... [07:39] s/adding/think & [07:39] 1.2 seconds [07:39] down from 73+ seconds [07:39] igc: and full commit is? [07:39] 1.96 seconds [07:39] booyah! [07:39] :-) [07:39] i guess it returns an indication of whether something was recorded [07:39] igc: that 0.7 seconds is 'status on the rest of the tree' [07:39] right [07:40] poolie: yes, we also check that the inventory gets the right last-modified value for that entry. [07:40] poolie: remembering that this is an awkward interface we are deleting, so don't go overboard :) [07:40] poolie: 'rm the interface' is not-all-that-far-away. [07:41] igc: If you recally, I was expressing concern that that overhead would be substantial vis-a-vis layering on iter_changes rather than changing iter_changes. [07:41] so i've sent that thing for merge [07:41] igc: you can see why now, I hope. [07:41] there's a bug open about adding more tests but i'm not very motivated to do it now [07:42] I think thats fine; I'd be inclined to close the bug wontfix [07:42] diminishing returns [07:42] if i'm going to do more here it would be primarily [07:42] adding integration tests [07:43] or trying to refactor it into a clearer layer that does filtering [07:44] igc, tell me about bug 374735? [07:44] Launchpad bug 374735 in bzr "Plan and UI for upgrading multiple stacked branches " [High,In progress] https://launchpad.net/bugs/374735 [07:45] poolie: one sec [07:47] poolie: so the issue is ... [07:48] could the Upgrade Guide recommend a less complex upgrade recipe for branches stacked on LP? [07:48] poolie: iiuic, thumper thinks no [07:49] so, um [07:49] poolie: I'm happy to document whatever we agree on [07:49] i suppose what i want to know now is, what's the implications for the 2.0 release? should i be doing something to unblock it, or should we considering that it might block 2.0? [07:49] poolie: maybe we need a conference call with thumper, lifeless, you and I to resolve it [07:50] poolie: I think the current recipe and doc is acceptable [07:50] poolie: I agree it *could* be better one day but I don't think it's a blocker [07:51] poolie: my personal opinion is that making shared repository upgrades easier is more important [07:52] by upgrading branches in that repo implicitly [07:52] my patch for patch got some love last Friday but hasn't moved since [07:52] right [07:52] s/for patch/for that/ [07:52] so that's only weakly connected to the issue of upgrading stacked branches? [07:52] like technically unconnected, just in the same area? [07:53] my patch is currently unconnected from stacked btranches [07:53] an earlier cut tried doing them in a certain order but ... [07:53] I backed that out because it didn't apply ... [07:54] stacked branches aren't supported in a shared repo at all [07:54] i'm thinking about narrowing down the 2.0-targeted bugs to just things that we must or nearly must fix [07:54] just for easier thinking [07:54] right [07:54] the best kind of thinking! [07:54] I think the Upgrade Guide is fine for 2.0 [07:55] after all, it actually works [07:56] I wish the Data MIgration Guide (fastimport) had reached that lofty state [07:56] poolie: jml: spiv: I wonder if making scenario application do a shallow copy would be safe. It looks like it might be substantially faster. [07:56] we'd probably want to change some static parameters to factories. [07:56]