=== garyvdm_ is now known as garyvdm [00:19] I really want a 'bzr edit-diff' which gives me the diff, lets me edit it, then saves the result back [00:23] Obmar: yes, that's an ugly bug, need to get some time and fix it [00:26] verterok: That would be great. I would love to be able to do bzr from in the IDE. [00:28] spiv: ping [00:29] spiv: the _commit_write_group check; I think that that may be better on the RepositoryPackCollection level ? [00:33] lifeless: rather than on the KnitPackRepository? [00:33] lifeless: sure, I essentially flipped a mental coin when deciding where to place that. [00:36] I've moved it down [00:36] def _commit_write_group(self): [00:36] all_missing = set() [00:36] for prefix, versioned_file in ( [00:36] ('revisions', self.repo.revisions), [00:36] ('inventories', self.repo.inventories), [00:36] ('texts', self.repo.texts), [00:36] ('signatures', self.repo.signatures), [00:36] ): [00:36] # We use KnitVersionedFiles exclusively so can rely on _index. [00:36] missing = versioned_file._index.get_missing_compression_parents() [00:36] all_missing.update([(prefix,) + key for key in missing]) [00:36] if all_missing: [00:36] raise errors.BzrCheckError( [00:36] "Repository %s has missing compression parent(s) %r " [00:36] % (self, sorted(all_missing))) [00:37] ^ ok ? [00:37] self.repo at the end obviosuly [00:40] Looks ok. [00:44] spiv: ok, that branch is on its way to pqm [00:44] where is the next one? [00:55] spiv: ping [00:58] (some people at) nasa \heart bzr [00:58] yay [00:58] nice [01:04] lifeless: tests all passing now I think; just removing a cruft [01:12] beuno: http://blog.crisp.se/davidbarnholdt/2009/02/18/1234986060000.html really is nice [01:21] spiv: well the scan api is gold [01:21] spiv: I'd like to take the next lowest layer on, or ?...? [01:23] lifeless: I think I've removed the worst of the cruft now [01:23] lifeless: I still haven't added a flag to add_records, but otherwise I'm happy with this [01:24] lifeless: I'm pushing it up to lp:~spiv/bzr/missing-parents-integration [01:25] spiv: k, give me a shout when its done [01:27] spiv: also, have you had a chance to review network-name? [01:27] spiv: if not thats fine, I might interrupt poolie [01:28] who has the context for this [01:28] lifeless: I haven't, but now is a good time for me to do so [01:40] lifeless: reviewed, one typo found :) [01:45] spiv: bombs away [01:46] decided against a comment, because of too much duplication [01:47] * spiv nods [01:54] lifeless: nice drawings :) [01:55] spiv: so, I'll review this branch; make any changes i need, and you ack the changes? [01:55] lifeless: +1 [02:00] spiv: looking at it, it seems easier to put back the older buffer code, and just make a single call at the end with the new flag, to add_records [02:00] spiv: sound ok? [02:02] lifeless: yep [02:03] lifeless: having a single call to add_records would be more clearly correct, I think. Less room for accidental double-insertion. [02:03] (Which is a bug I noticed I had during our phone chat this morning...) [02:09] * spiv -> lunch [02:15] lifeless: ping for a quick question [02:15] i'm just looking at mark's vcsrace on status [02:15] the results are generally really good except for status which spends a considerable amount of user time compared to the competition [02:16] i think this is because of overhead of building tuples from the dirstate etc [02:17] poolie: [waiting for the question] [02:18] would you agree? [02:19] from memory i mean [02:19] well [02:20] One planned experiment is to write a pure C implementation [02:20] yes [02:20] actually i see something interesting in the profile [02:20] we may be able to squeeze some more out of chdir [02:20] the system time is actually pretty good [02:20] running up oprofile over it would be interesting too [02:20] the interesting thing is that we're using the generic InterTree.compare [02:20] poolie: pending merge? [02:20] because mark's measuring the special case of changes from the empty tree [02:21] oh [02:21] i suspect dirstate just gives you an EmptyTree [02:21] and this is unoptimized [02:21] grah [02:21] * lifeless hates on special cases [02:21] i may be wrong [02:21] quick, add another special case :) [02:21] measurement is the only way to be sure :) [02:22] well, it is definitely using the generic one [02:23] hm no it's not literally EmptyTree, which is deprecated [02:24] spiv: get_missing_compression_parents: perhaps on _index only ? [02:24] spiv: I don't think it needs to be top level [02:25] spiv: actually scratch that, the tests would be ugly [02:25] spiv: its fine [02:34] crapzor. [02:34] it's doing it again [02:34] same branch. one more revision. [02:38] poolie: IIRC we've gone back and forth on the patch that removes the special case when there is no entries [02:38] at least, I seem to remember fixing that in the past [02:38] # the source revid must be in the target dirstate [02:38] if not (source._revision_id == NULL_REVISION or [02:38] source._revision_id in target.get_parent_ids()): [02:38] spiv: done; bzr sending it [02:39] Looks like we should be handling the NULL tree [02:40] If we aren't, something else is wrong [02:41] spiv: mail sent; review @ will. next is suspend/resume yeah? [03:00] hi when I use ftp push, Does it actually upload the files. So that if I were to have a HDD failure, i could retrieve those files from the FTP server? [03:02] It does it just upload some kind of data about changes to the files [I Dont understand versioning systems] [03:02] PHPlover: it creates a database [03:03] PHPlover: you can inspect what it has uploaded by using bzr - e.g. 'bzr log URL' [03:03] ok, and does that database store the complete data of every file in my project, Or just info about the files [03:03] it stores the complete copy of each commit, which includes the files that *are versioned*, but not those that are not [03:04] PHPlover: the easiest way to see what its done, is to 'bzr branch URL /tmp/spare-directory' and have a look at what bzr puts in spare-directory [03:05] poolie: ping [03:05] poolie: does bencode avoid \n ?, or can/should I avoid a \n some other way . (network_name currently has \n, but that breaks SmartServerRequest.do() [03:08] lifeless: \n in HPSS args are fine with HPSS v3 [03:08] spiv: interesting. [03:08] bencode length prefixes strings. [03:08] spiv: how do a I register a verb to be only used with one HPSS version then ? :P [03:08] File "/home/robertc/source/baz/push.roundtrips/bzrlib/remote.py", line 2002, in _translate_error [03:09] raise errors.UnknownErrorFromSmartServer(err) [03:09] UnknownErrorFromSmartServer: Server sent an unexpected error: ('error', 'do() takes exactly 4 arguments (3 given)') [03:09] the blackbox tests pass [03:09] but the per-repository RemoteRepositoryFormat-v2 tests are failing [03:09] for the reason-under-discussion [03:09] Currently that's taken care of in the client; the client never tries sending various requests to older servers [03:09] ok, so I should guard based on a check? [03:10] Right. The client asks the medium if .is_remote_before((1,2)) or whatever. [03:11] And also does _remember_remote_is_before((x,y)) when it notices a smart method is not implemented to save it trying that or other new methods again. [03:11] is that _protocol_ or _bzrlib version_ [03:12] [how do I get these tests to work :P] [03:12] It's the bzrlib version. [03:12] When the protocol is v2, the medium immediately "remembers" that remote is before (1,6) (or whenever v3 was added). [03:13] ok, so its a lie, but don't look under the hook ? [03:13] *hood* [03:13] Yeah. [03:13] I'm not 100% happy with it, but it does the job. Well, jobs. [03:14] (Avoid trying calls that require \n in args and whatever that are new in v3, and also avoid repeatedly trying an RPC once we know the server doesn't implement it.) [03:15] where is the code so I can figure out what .is_remote_before to use [03:15] like is 1,6 right [03:15] or 1,2 [03:15] or ? [03:16] * spiv is looking [03:17] protocol_version in medium still returns '2' :P [03:18] or should I say is_before(1,13) ? [03:20] Probalby 1,13 would be best. [03:21] * spiv can't find where the remembered version is set low for v2 protocols. [03:23] ok I'm unblocked on that, though 1,13 fails the current version test :P [03:23] spiv: *and* [03:23] it still runs the verb on the Remotev2 tests [03:23] hmm, checking [03:24] Ah, found it. [03:24] ok fixed [03:24] cool [03:24] so the branch for partial insertion is polished and up for review [03:24] bzrlib/smart/client.py calls self._medium._remember_remote_is_before((1, 6)) [03:25] once it finds out that v3 doesn't work. [03:25] Cool, I'll take a look. [03:25] I'm waiting on the next unit of work [03:26] the network data serialisation for record streams is also outstanding for review [03:26] http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C1234854080.11037.18.camel%40lifeless-64%3E [03:36] spiv: ok, and I've just flushed the new verbs to qm [03:36] *pqm* [03:36] spiv: how should we split up the remaining work? Want a quick call? [03:41] lifeless: ok [03:41] lifeless: (I'm reviewing the network serialisation atm) [03:42] ok, I'll ring you now if thats ok [03:42] Yep [03:48] lifeless: http://people.ubuntu.com/~andrew/bzr/streaming-push [03:48] lifeless: ...nearly pushed [03:49] lifeless: done. [03:50] spiv: cool [03:54] * igc lunch [04:05] spiv: so RemoteVersionedFile has some of our fetch refactoring [04:09] spiv: it looks like RemoteVersionedFile is a little confused, the loom basically resets @ Very fetching [04:10] spiv: or something. econfused:P cherrypicking and reviewing [04:12] lifeless: weird. [04:12] jam: dropping the group size will shrink things because of the full texts at new groups === jfroy is now known as cad_monster === cad_monster is now known as jfroy [04:42] mwhudson: ping? [04:43] igc, can you remind me why content-filtering needs to change dirstate? [04:43] would it be at all possible to avoid it? [04:43] poolie: sha on disk != sha in storage ? [04:46] well, of course that's true for filtering [04:46] it's been a while but what lifeless says sounds right [04:47] poolie1:^ [04:47] it was more the next logical step after that that I'm thinking about [04:47] spiv: testing a massaged Very fetching with the lower layers removed [04:47] spiv: I'm keeping length_prefix, though really its in the wrong home [04:49] aaargh [04:50] Raawfle: some context might help [04:50] spiv: as we knew, progress breaks [04:52] spiv: so I'm going to commit it, and then start working on the progress bars [04:53] lifeless: sounds good [05:01] poolie: can you decode what this means for me ? [05:01] /home/robertc/source/baz/fetch.sinks/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/1, msg='fetch texts') is not the active task ProgressTask(None/None, msg='') [05:01] % (task, self._task_stack[-1])) [05:04] lifeless: probably that you're missing a finally block [05:05] prior to the code that constructs ProgressTask(0/1, msg='fetch texts') [05:05] hth [05:05] poolie1: ok, will look for that [05:05] poolie1: if it helps we have generators around generators [05:06] no, that doesn't help :-) [05:06] the previous pb seems to contain nothing [05:07] so you can probably just remove it [05:07] is previous a higher one? [05:07] or one that wasn't cleaned up from an earlier test? [05:07] so, there's supposed to be a stack [05:08] this message indicates that you tried to finish the one that's not on top of the stack [05:08] so actually "prior" is not necessarily true [05:08] so there are (at least) two here [05:08] A and B [05:09] it may be that B is conceptually inside A, and now all of A is finished but you forgot to finish B first [05:09] or, it may be that B is a sibling of A, and you forgot to finish A before starting B [05:09] if you see what i mean [05:10] yeh [05:10] I'm worried that generators are interacting badly [05:10] spiv and I overhauled fetch() [05:11] one answer is to remove the progress entirely for now [05:12] for many generators i'd say that giving an activity indicator may be more appropriate [05:13] because, firstly you may not know how many things are generating [05:13] indeed [05:13] and, as a technical reflection, finishing up is hard [05:15] ah found it [05:15] we have a function that passses a pb into a method [05:15] that method finishes it and rebinds it [05:15] I'll change it to an instance variable, bet it fixes it [05:15] hm as i may have said in a review, i think passing pbs around is whack [05:15] it's one thing i'd like to fix here [05:16] except in pretty special cases, a progress bar should correspond to a for loop in one function [05:16] poolie1: I'm not justifing the current code ;) [05:16] and the nesting of them i think can be done by the ui global :) [05:16] i know, i'm just repeating this thought in case you want to agree :) [05:17] oh, I agree [05:18] no warnings now, that was it [05:18] ok [05:19] well, i'm glad it wasn't spurious and didn't take too long to track down [05:19] :) [05:19] igc, so, i'm not quite done on filtering but it's not all that hadr [05:20] i am inclined to see if doing it on top of dirstate is within reach [05:20] at the least i'd like to document why [05:20] poolie1: did you see my comment about sha on disk and sha in repo? [05:20] yes but no but yes but [05:21] obviously they are different, [05:21] but this doesn't change dirstate to store both of them [05:21] what does it do [05:22] it seems to change it to record the filtered sha1 [05:23] sorry, i mean the sha1 of running the input filters on the file [05:24] Peng_: hola [05:28] mwhudson: Hi. [05:28] Um. [05:31] mwhudson: I have a dumb question about your changes to LH's served_url stuff. I used to monkeypatch served_url to return URLs relative to /bzr instead of /loggerhead, but now that doesn't work. So...what do you think I should do? [05:32] Peng_: um [05:32] :D [05:32] spiv: sending it for review, I'm happy and only trivial changes; if you're happy with my arbitrary cleanups we can just land it [05:32] Peng_: think of a way you'd like this to work and tell me about it :) [05:33] mwhudson: Heh. I was happy with how I was doing it before. :P [05:33] Peng_: you can probably monkey patch BranchFileSystem.app_for_branch [05:35] Peng_: alternatively, one could extract the calculation of the default value out into a method that you could then monkey patch [05:40] I'm barely awake enough to understand what that means, but I think it sounds good. [05:41] lifeless: fancy that, I just sent you a review too. [05:41] lifeless: If you only made arbitrary trivial cleanups to my code, then yes I'm definitely +1 on principle :) [05:41] lifeless: I can eyeball it anyway if you want. [05:41] spiv: just sent it [05:41] spiv: its basically - delete the RVF stuff [05:42] spiv: fix progress bars [05:44] spiv: we can't do RemoteStreamSink until the vf network bytes lands [05:45] I know. I tried not to let immiment achievements cloud my review judgement, though... [05:49] lifeless: there's some odd stuff in that diff you just sent; additions to the makefile etc. [05:49] lifeless: perhaps the branch you sent has more recent bzr.dev than bzr send diffed against? [05:51] unlikely [05:51] however wrong parent is likely [05:54] Ah, yeah. [05:54] resent [05:55] out to the shops before they close, will finish igc's review today though [05:56] morning [05:58] Heh, I see _length_prefix did sneak into that diff. [05:59] spiv: I kept it deliberately [06:00] * spiv nods [06:00] lifeless: reviewed (tweak) [06:01] * spiv keeps reviewing [06:03] spiv: ok network serialisation on its way to pqm [06:04] spiv: so we can merge that into the fetch refactoring branch [06:08] lifeless: just bb:approved your "insert record streams with missing compression parents" patch [06:08] excellent [06:08] I'll kick that off now [06:09] spiv: so what do we have left [06:09] suspend resume? is that up for review yet? [06:11] Hmm, I don't think so. [06:11] so it depended on missing parents and partial streams [06:11] both are fully acked [06:11] why don't you polish that, and I'll get the network stream integrated with RemoteSink [06:11] which will fail on stacked branches until suspend resume is done [06:12] Ok. Let's see if I can figure out which branches to merge into suspend-resume to bring it up to date... [06:12] :P [06:12] so administrivia [06:12] I'd *love* to finish this today [06:12] but its after [06:12] 5 [06:13] if you're up for a sprint-to-finish, so am I. And I'm sure we can convince poolie for some slack on Monday :) [06:13] Oh that's right, we were sensible and made that a branch off bzr.dev. That helps ;) [06:14] I'm still going strong atm ;) [06:14] http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C1235097663.6233.119.camel%40lifeless-64%3E is what you need for suspendresume [06:14] I'll probably need to stop about 6:30 [06:14] the clock is on :) [06:15] :) [06:18] you probably want bzr.dev too or that matter [06:21] Yeah, already grabbed that :) [06:31] spiv: ok, failure in my streams bytes thing in reconcile [06:31] while I investigate that the other approved patch is on the way [06:31] igc, still around at all? [06:38] poolie1: yep [06:40] i'm going to send a review in a sec [06:40] do you want to talk about it/ [06:41] lifeless: http://rafb.net/p/wgjSMm15.html is failing on the assertRaises; because the add_lines after the resume_write_group doesn't actually make a record with a compression parent. [06:42] lifeless: what's a good way to add a record with a compression parent there, should I crib from test_versionedfiles perhaps? [06:42] With a *missing* compression parent, specifically... [06:42] spiv: you need two repos [06:43] in repo one you do add_lines(base, (), [... ... ...]) [06:43] lifeless: that's what I feared :) [06:43] * spiv does that [06:43] and add_line(child, (base,), [.....]) [06:44] then repo2.texts.insert_record_stream(repo1.texts.get_record_stream(base,...) [06:44] sorry not base there [06:44] just delta [06:44] * spiv nods [06:45] or [06:45] use knit.make_pack_factory [06:45] to get a vf object [06:45] in lieu of a second repo [06:53] Existing tests passing, I think... [06:53] cool [07:02] Ok, wrote a test for filling in a missing parent after a suspend/resume, currently failing :) [07:02] Fixing... [07:02] spiv: I'm modifying RemoteSink to fallback to _real when stacked, pending the stuff you're doing [07:04] spiv: minor tweak needed, our substreams need to encode a prefix record [07:05] spiv: dealing with the keywidth mismatch [07:08] Oh, right. [07:09] so in do_chunk on the reader [07:09] does it know its a pack stream and give us only the bytes_record ? [07:09] Ok, filling in a missing parent after suspend/resume works. === chx is now known as chx_sleeping [07:10] woooooooooooooo [07:10] ooooo [07:10] ooo [07:10] and the current branch is into ascii tests [07:10] which is fetch refactor [07:11] Hmm, I think I need more context to understand your question. [07:11] do_chunk is called as bytes of the request *body* arrive. [07:11] yes [07:11] SmartServerRepositoryInsertStream [07:12] I think it will be fine [07:12] So long as the client is generating a request body stream of a pack, which IIRC it is... [07:13] i.e. it's basically just sending what the ContainerWriter is emitting [07:13] So that should be fine, yeah. [07:20] hi all (completely forget my ping this morning :) [07:20] Ok, added what I think is the last test, that suspend/resume doesn't accidentally make an uncommittable write group committable. [07:20] Not passing yet :) [07:21] hello vila [07:22] hi poolie1 :) === poolie1 is now known as poolie [07:30] lifeless: passing, just fixing up duplication added to make it pass [07:32] spiv: cool [07:32] spiv: I think I'm done too, bit of staring involved [07:32] spiv: the glue is a little awkward, pack -> bytes -> pack reader -> bytes -> NetworkRecordStream [07:33] lifeless: ok, pushing [07:33] lifeless: it'll be at http://people.ubuntu.com/~andrew/bzr/suspend-write-group shortly [07:33] spiv: was there a test to run to check this works [07:34] spiv: one branch landed [07:34] pulling .dev and sending the next [07:35] lifeless: there were acceptance tests in that loom [07:36] lifeless: although now that I look at it again, the test_push_smart_stacked_streaming_acceptance test is slightly wrong :) [07:37] lifeless: we expect 2 insert_stream RPCs now, not 1. [07:37] If things are working, that is :) [07:37] spiv: cool cool [07:37] lifeless: so, my branch is pushed, and all tests seem to be passing here. [07:38] (running a bigger selftest to be sure, but so far so good) [07:38] spiv: whats a good test to test RemoteSink ? [07:39] lifeless: hmm [07:40] fetch tests pass [07:41] lifeless: https://code.edge.launchpad.net/~lifeless/subunit/polish/+merge/2735 [07:42] dammit, its creating manually >< [07:42] lifeless: I suppose a simple test_remote test that shows that RemoteRepo._get_sink().insert_stream(...) invokes the right RPC would be good. [07:42] lifeless: btw, "bzr send --no-bundle" to merge@code.launchpad.net Just Works. [07:42] Possibly even with a stream of [] ;) [07:46] I'm sending up the suspend-write-group branch for review [07:46] cool [07:46] resending partial inserts (typo :() [07:47] :( [07:49] jml: thank; commonprefix? [07:50] lifeless: os.commonprefix. [07:50] lifeless: it's a blight on the standard library. only use it if you want your file operations to be broken [07:50] lifeless: oh, heh, I already had that exact one-liner in my suspend-write-group branch. [07:50] lifeless: os.commonprefix('/foo/bar/baz', '/foo/baz/bop') => '/foo/ba' [07:50] spiv: the typo ? [07:52] lifeless: yeah [07:53] spiv: :) [07:54] spiv: I don't see the patch [07:59] spiv: ok, its writing a pack directly [08:00] night all [08:02] lifeless: ok, patch sent. [08:03] lifeless: I'm done for the evening, but I'm very happy with the shape of the resume_write_group code. Maybe it's my Friday evening judgement, but I think that patch only has a few minors warts, rather than any glaring problems ;) [08:04] lifeless: thanks for helping me get it into shape [08:04] * spiv -> food, weekend, world of goo, etc! [08:11] spiv: approved [08:12] spiv: I'm going to keep plugging at joining these dots [08:12] * lifeless will be begging for reviews I think [08:12] lifeless: I'll pqm-submit it, thanks. [08:13] the dependent patch is onto ascii [08:13] should land fine I think [08:19] spiv: woo, 74 -> 60 [08:20] lifeless: the ratchet? [08:20] for non-stacked [08:20] stacked goes 99 -> 101 [08:20] Nice. [08:20] the latter is uninvestigated as yet [08:20] but I think its going to be due to my if block pending suspend integration [08:20] Is stacked still hitting the thing in InterPackRepo that explicitly uses something dumb? [08:20] so commiting and sending for review [08:21] (the "if len(fallback_repos) > 0" check) [08:21] I have a merge request from a friend who didn't set his 'whoami' info on the shared computer he was using. How can I modify the commit email address before I merge into my main branch? [08:24]