[00:00] I understand the first two about the protocol version, but the third sounds like server and client can't communicate [00:01] If the server doesn't support protocol v2, it is far older than 1.5. [00:02] oh wait [00:03] gah, the network I am connecting to is not homogenous, this is actually an older Debian box with 0.11.0 [00:04] Wow, that's *ancient*. [00:05] Raim: You'd have to use sftp; modern clients dropped support for smart servers that old. [00:05] 0.11? What disk format would that use? I didn't even use bzr back then... [00:06] Peng_: nah, I just have to use another server which also has access to the NFS share where the repo lives [00:06] Peng_: works fine now. I just made a bad assumption and did not check the versions properly [00:07] OK, that works too. [00:16] morning [00:19] Good morning. [00:23] hi Peng_ [00:23] hi igc [00:23] hi lifeless [00:26] lifeless: Thanks for the review. [00:28] abentley: no probs [00:28] abentley: you registered it with the new hooks registry; all was good in the world [00:36] * igc out for a few hours - chemo day - bbl [00:36] hello all [00:36] good luck igc [00:36] thanks poolie [00:42] poolie: hi there. [00:42] igc: good luck. [00:42] hi abentley [00:43] abentley: i heard you made me the reviewer for the nested trees doc [00:43] i'll do something on it today [00:43] poolie: Maybe we could have a quick chat? [00:44] abentley: sure, can you wait 5m? [00:44] poolie: sure. [01:26] jelmer: so did you upload subunit? or should I [01:26] lifeless: I did some work on it (fixing lintian warnings and the like) [01:27] oh noes [01:27] s/warnings/issues pointed out by lintian/ [01:27] I also sent in a merge request for a shell bug [01:27] did you see that one? [01:27] no [01:28] lp:~jelmer/subunit/merge [01:28] nice, https://code.edge.launchpad.net/~subunit/ shows package branches now [01:28] lifeless: it's shown package branches forever :) [01:28] jml: oh cool [01:29] lifeless: drop the tilde and they'll disappear [01:29] jelmer: I can see the reviews, but I didn't et mail [01:29] lifeless: they are also very badly sorted. [01:29] jml: how can I debug this [01:29] jml: I don't get notification on merge proposals to subunit trunk [01:29] I'm subscribed [01:30] lifeless: how did ~subunit get 1 registered branch? [01:30] I checked my subscription and it says all code review stuff [01:30] nm [01:30] lifeless: https://code.edge.launchpad.net/~subunit/subunit/trunk says you aren't subscribed. [01:30] it is old [01:30] oh subunit developers [01:30] wait no [01:31] I can't tell whats going on [01:33] jelmer: land it [01:34] lifeless: ack [01:36] thumper: what do you mean by 'it is old'? [01:36] jml: so even though subunit-developers is the default review team, they are not notified? [01:36] lifeless: before we had a registrant for branches [01:37] thumper: I would like a clear thing saying whether I will get notified about a given branch [01:37] thumper: at the moment I can't tell [01:38] * lifeless files a few bugs [01:38] if you are not subscribed, you don't get anything [01:38] lifeless: assign to beuno :) [01:40] thumper: I'll let you decide [01:41] lifeless: tracking down the 1.13.2 == 1.14.xx, I introduced the issue into the 1.13 branch when I merged ~spiv/bzr/stacking-inventory into 1.1.13 branch as part of the fix to Bug 354036 [01:41] Launchpad bug 354036 in bzr "ErrorFromSmartServer - AbsentContentFactory object has no attribute 'get_bytes_as' exception while pulling from Launchpad" [Undecided,Confirmed] https://launchpad.net/bugs/354036 [01:42] There was a bug task for 1.13, assigned to me. I need help understanding what I should have done, since a flatout merge of spiv's branch was not the right thing to do [01:43] revno: 4112.1.1 [merge] is the offending revno [01:43] I can post to the ML if that is better [01:43] BasicOSX: you should done a cherrypick into your local branch [01:44] BasicOSX: the easiest way to make this happen is to: [01:44] - get the branch you want merged to trunk if it isn't already [01:44] - bzr merge -c bzr.dev in your release branch [01:44] - bzr commit [01:44] - submit to pqm for that release branch [01:45] making notes [01:46] 'bzr merge' should never be used from a feature branch to a release branch, unless the feature branch is intended specifically for that release [01:47] I see that now. In the past someone has done the cherrypicking for me (not an excuse) [01:48] To fix the mess, do I issue bzr uncommits? [01:49] mail the list re the mess [01:49] ok [01:49] we need to consider who may have 1.13.14 [01:52] abentley: i reviewed the docs and posted a comment [01:52] looking at the other threads now [01:57] poolie: thanks. [02:27] Running 'bzr' on my /home/me directory. Also, /home/me/fw02/root. When in /home/me/fw02/root it is referencing the .bazaar/bazaar.conf file in /home/me and not the one in /home/me/fw02/root. Is that the way it should be? [02:28] solo mode if it makes any difference. [02:29] kvanderw: the .bazaar/bazaar.conf file has user-wide configuration, so bzr always looks it up in your home directory. [02:29] kvanderw: so yes, that's expected behaviour. [02:30] kvanderw: perhaps you are getting confused with the .bzr directory that is part of branches, repositories and working trees? [02:31] spiv: Thanks... just making me wonder... The subdir is an rsync copy of a firewall. I had forgotten that I had installed bzr over my home directory. [02:31] Just bad management from a noob :) [02:32] spiv: though, it sounds like, even if I put it outside that dir structure, it still always looks to my 'home' dir for the conf. No prob... Just trying to learn the ropes. [02:32] spiv: Thanks! [02:33] kvanderw: right [02:33] spiv: is this a TODO? [02:33] + # Also, perf tests: [02:33] + # - if all invs present, then no texts are checked [02:34] lifeless: oops, yes. [02:35] lifeless: actually, I was intending to not bother as it seemed too much hassle for minimal benefit... [02:35] have you done a user level perf test [02:35] say to people? [02:37] Not explicitly, I have done a bit of branching locally to confirm that the final form of the fix really did fix the original issue and didn't notice any slowdown in branching all of bzr.dev locally. [02:37] I'll let you make the call [02:38] I'll run another quick experiment. [02:38] what do you think of my -Dhpss backtrace patch? [02:42] I don't think I've seen it... where is it? [02:43] Oh, something to do with getting backtraces from _ensure_real? [02:47] lifeless: I like the idea of making the remaining VFS causes easier to identify, but I'm fairly sure I haven't seen an actual patch for it. [02:52] spiv: yes [02:54] spiv: no idea where it went [02:54] resending [03:04] spiv: its hit the list [03:04] Cool, I'll take a look. [03:04] lifeless: thanks for the review, btw [03:13] lifeless or spiv: PendingAncestryResult.get_keys() returns ghosts, is it supposed to? [03:14] The StreamSource code then calls Graph.iter_topological_order(keys) [03:14] which strips ghosts [03:14] so StreamSource works [03:14] but it is causing stacking w/ bbc to fail [03:17] spiv: also, why in your new code did you create 1 set per key, rather than having a single set? [03:17] I'm a bit concerned about having 100k sets in memory [03:17] (unlikely to be intentional for something which is stacked, though) [03:17] jam: heh [03:17] I just don't see what it gets you [03:17] versus 1 big set [03:17] jam: I just sent mail on this :) [03:18] since you only call stuff like 'get_referrers()' [03:18] jam: before I saw that you were on IRC [03:18] spiv: :) I just got the email [03:19] spiv: are there direct tests of PendingAncestryResult? [03:19] jam: there are some in test_graph [03:20] yeah, I just found it [03:20] no tests for ghosts [03:20] jam: PAR returning ghosts is sarguably a bug [03:20] I guess that leaves it open for me to interpret :) [03:20] jam: I see no reason to keep that [03:20] lifeless: agreed, just wanted to make sure everyone else thought so, too [03:21] jam: IDS breaks streaming push cross-format :( [03:21] which I knew [03:22] but jelmer filed a bug saying 'it go sloweth' [03:22] lifeless: does it go 'slowether' than doing it any other way? [03:22] jam: much [03:23] interesting, considering it goes *much* faster locally [03:23] spiv: inventory deltas top of the pile now? [03:23] it doesn't really query the target [03:23] jam: latency [03:23] I suppose this is a 'pull' issue? [03:23] I would assume it is faster for 'push' [03:23] jam: push [03:23] lifeless: theoretically yes, but I'm on leave from tomorrow till All Hands [03:23] jam: no, every write is a round trip [03:23] spiv: oh, I didn't know. [03:24] lifeless: well, isn't that just NewPack.set_buffer_size() ? [03:24] spiv: I will see if I can squeeze it in; please ensure you have it pushed and a mail to me with the url of the current state [03:24] Considering we hold a write group for 100 revisions at a time [03:24] Crap, I haven't mailed the about that yet... I'll do that now. [03:24] And we transfer texts 'in bulk' as well [03:24] jam: even so, its still many round trips rather than streaming [03:24] and each commit is about 10 round trips [03:24] and then a re-read to refresh data [03:25] jam: I can pretty much guarantee that disabling IDS over the network will be faster in all cases [03:25] jam: particularly if we do what I propose and set the fetch order to gc-optimal [03:26] well, other than the 4-5 things I listed, sure... [03:26] IDS wasn't specifically tuned for remote ops [03:28] I realise IDS is fast locally [03:28] AFAICT there are two specific issues; long conversion times, and writing many small chk pages meaning we can't really do gc-optimal cross-format [03:28] because we'll get optimal texts but not optimal chk pages [03:29] I don't want to delete IDS until there is a better option; if thats even possible. However I also want network ops to be fast [03:29] I'd like to see IDS split into a sink and stream pair and just hooked into the generic paths [03:30] lifeless: so other than wanting to trigger commit_write_group on the target [03:30] I'm pretty sure all the work is done in the source [03:30] and there isn't specific chatter between both sides [03:32] I suggest we add a stream flag saying 'the data from start to here is topologically consistent' [03:32] we should design this next week/sprint week [03:32] I guess it switches from texts => inventories => revisions [03:32] I have a few things I want to include [03:32] oh, and it wants 'add_inventory_by_delta' [03:33] which requires 'streaming inventory deltas' [03:33] jam: network deltas will cover that [03:33] which you keep claiming exists somewhere :) [03:33] yes, spivs laptop [03:35] The delta serialisation is already in bzr.dev [03:35] It just isn't hooked up to anything yet. [03:35] spiv: oh you landed that? [03:35] lifeless: we did :P [03:35] spiv: sweet, I hadn't noticed. [03:35] spiv: there's no other outstading branch then is there [03:36] Yeah, I forgot too until I went over my old branches looking for half-done work :) [03:36] Right, no other outstanding branch. [03:36] ok [03:36] jam: ^ [03:36] spiv: where is it? [03:36] jam: small amount of glue left; get_delta(); serialise(); yield; [03:38] jam: bzrlib/inventory_delta.py [03:39] spiv: .... PendingAncestryResult.get_recipe() does not return ghosts as 'exclude_keys' [03:40] Which makes 'bzr branch from-stacked' search all the chk pages, and want to copy all texts that are referenced from the stacked branch... [03:40] jam: PendingAncestryResult doesn't have any exclude keys [03:41] jam: par is a full-history grab [03:41] it's just "get me the ancestry for " [03:41] lifeless: but on a Stacked location, *without* fallbacks set [03:41] ... [03:41] which is how the stacking code works [03:41] If you want "get me the keys from to ", that's SearchResult [03:41] jam: so I think you still have a bug :) [03:41] jam: here is how it is meant to work [03:42] the server gets a par for the branch tip [03:42] it then finds all the revisions it actually has [03:42] it then calculates a delta for the new content in all those revisions and sends that [03:42] it should have enough data in the inventories to do that, *because* we include the adjacent inventories [03:42] spiv: so the idea was that if we have 'exclude_keys' *already* computed from the search result, why should I compute them *again* [03:43] now, PAR may never have excluded_keys [03:43] in this situation note that there are no exclude keys [03:43] and cannot be, because we're always grabbing all the history if we have a par [03:43] so... can I just never trust the recipe? [03:43] its never used if we have a shared repo [03:43] or should I trust it, unless it is empty? [03:43] jam: the recipe is completely accurate [03:43] note that PAR without stacking [03:43] will have no exclude keys [03:43] and require searching the *entier* graph to find that it is correct [03:44] jam: I don't understand why you think you can't trust it [03:44] what do you mean "find that it is correct"? [03:44] lifeless: I want to transmit all revisions from the stacked branch [03:44] jam: yes, which the par will have found [03:44] And all inventories for only those revisions, and no chk pages for previous revisions [03:44] I was using the 'exclude_keys' to figure out what inventories *not* to transmit [03:44] rather than doing another get_parent_map call [03:44] exclude_keys is wrong wrong tool to use [03:44] because it is redundant with SearchKey [03:45] SearchResult [03:45] sorry [03:45] lifeless: except SearchResult has already done that sort of get_parent_map for me [03:45] and avoids having to do double duty [03:45] especially on a 'branch everything from a non stacked branch' case [03:45] jam: you want to send search.get_keys() [03:46] lifeless: and *not* send get_parent_map(search.get_keys()).itervalues().discard(search.get_keys()) [03:46] yse [03:46] yes [03:46] I just didn't want to have to re-compute 'exclude_keys' for the case when SearchResult has already *done* that. [03:46] so, feel free to improve PAR is perhaps the right answer [03:47] well, I don't like doing: _, _, exclude_keys, _ = search.get_recipe() [03:47] its sketchy, it was created to avoid the 'billion round trips to find graph' case [03:47] so I would rather do it a different way [03:48] like "search.get_tails()" (get_exclude_keys(), etc) [03:48] and PAR could compute that when it computes get_keys() [03:48] I don't think andrew or I are tightly attached to any of these internals [03:48] though I notice that the Result objects often don't actually cache anything [03:48] so it seems a bit off to assume that get_keys() is called before get_tails() [03:49] lifeless: I'm trying to understand what you *do* like, so I can not make a mess [03:49] In the short term, I can just do another get_parent_map() [03:49] it just seemed redundant [03:49] (not that we don't do that *all the time* already...) [03:49] jam: Whats wrong with _, _, exclude_keys, _ = search.get_recipe() [03:49] is it doing more work [03:49] or is it ignore the variables you don't want that you don't like? [03:49] lifeless: get_recipe() seems like it should really be opaque [03:50] jam: I think our graph search stuff is pretty primitive at the moment, so I'm definitely in favour of improving it. [03:50] jam: unfortunately, and I sighed when I realised, we messed up PAR and its not as opaque as it could be [03:50] jam: that said, if you want to add methods that you'll like that would be fine. [03:50] jam: caching should be done with care, obviously. [03:51] lifeless: well, you cache 'heads'... I suppose caching 'self.tails' if we have encountered them? [03:51] jam: right [03:52] jam: you actually want to cache ghosts in this case [03:52] right [03:52] and really only ghosts, since PAR is 'everything that you can reach' [03:52] Yeah, the step from generalising SearchResult to SearchResult+PendingAncestryResult wasn't perfect. [03:54] * igc back - chemo delayed for a week [03:56] hey igc, glad to hear you feel well today, sorry to hear it is because of delayed chemo [03:56] hi jam! yeah - I feel fine fwiw [03:59] Hi, when branching from https://code.launchpad.net/~breathe-dev/breathe-icon-set/trunk I got http://pastebin.com/m6ad04457 [03:59] err http://pastebin.com/m64c5c70a sorry [04:04] ziroday: I suspect a broken http proxy at your ISP [04:04] ziroday: can you try using bzr+ssh please [04:04] lifeless: doing so now [04:05] ok [04:10] maybe something should be added to the smart protocol to help diagnose buggy http proxies ? [04:11] SamB: patches considered helpful :) [04:11] (in general its hard to tell 'bad proxy' from 'data attack' [04:11] some kind of handshake to be used after that kind of error to check for typical bugs, maybe? [04:11] lifeless: data attack? [04:11] SamB: that error isnt from the smart protocol though [04:12] SamB: a bad proxy gives bad bytes that it claims where what you asked for, but aren't. [04:12] oh, you mean it doesn't just get confused ? [04:12] or munge data ... or something like that? [04:14] there are all sorts of things which may be causing it [04:14] but we haven't detected an http protocol issue if we get the errror that ziroday saw [04:14] but if it is (and ssh working says it is) the http proxy, then the proxy is effectively hostile [04:15] well it appears to be working via bzr+ssh, I'll just advise the loco to use that [04:15] lifeless: yeah, what I'm saying is that if you get something like that, it might be a good time to check for such bugs in any proxy ;-) [04:16] SamB: yes, but exactly which bugs, and how? There are an infinite number of ways a proxy can screw up your data... [04:16] ziroday: cool [04:17] lifeless: spiv SamB: thanks for all the help, you've made my day :) [04:17] spiv: frequently seen bugs, I guess [04:17] SamB: I agree with the sentiment, I'm not smart enough to see how to do it though [04:18] SamB: because the symptoms are 'sometimes, when you ask for data you get different data' [04:18] yeah, me neither! [04:18] well, mostly because I don't know what any of those bugs are [04:18] SamB: many of these intercepting proxies are proprietary [04:19] so very few people will know exactly what the bug i [04:19] s [04:19] well, I mean, what kind of corruptions they create [04:19] or at least what kind of data they are likely to corrupt [04:19] who knows ;) [04:22] * mwhudson is pretty sure his isp uses good ol' squid [04:22] if I knew that, I might be able to cook up an http smart-server handshake protocol that would allow the client to construct a desired HTTP response and see if it comes through unscathed ... [04:22] and some probe data to check for common bugs [04:23] SamB: this isn' smart-server related [04:23] SamB: this is simple plain HTTP GET's failing. [04:23] well, yes, I know ... but I figure you'd need cooperation from the remote that plain-old HTTP isn't going to give you [04:24] I think it would be undesirable for bzr to issue arbitrary gets when something goes wrong on HTTP. In some organisations that would get a sysop coming around to your desk to 'talk' to you. [04:24] hmm. [04:25] a much simpler idea would be to simply point out that that type of exception is often caused by , where is derived from the access method in use ... [04:26] thats possible. Which comes back to - corrupt data has a pretty good chance of causing a zlib decompression error [04:26] you might like to file a bug about that approach [04:27] * SamB should file a bug against himself for being too lazy to file a bug ... [04:33] You guys know if there are any books on Bazaar? [04:54] Logomachist: there's a book in progress, for now the best docs are the user guide on doc.bazaar-vcs.org [05:18] hello vila [05:25] Cool. Thanks Poolie [05:44] what is the difference between: 2645 ghost revisions [05:44] 2485 revisions missing parents in ancestry [05:44] in bzr check? [05:44] I get this after reconcile [05:45] the former is ghosts, the latter is revisions that are present and should have been fixed up [05:45] I think [05:49] lifeless: what, if anything, should or could I do about them? [05:50] if reconcile left it like that, file a bug [05:50] I'm currently overhauling check [05:51] and will make sure its actually a relevant warning [05:51] ok [05:51] lifeless: what information would you like in the bug report? [05:51] it is the launchpad branch [05:51] I reconciled the 1.6 branch [05:51] then ran check [05:51] and that is what it said [05:52] sure that would be fine. [05:52] however it did fix the 3547 inconsistent parents [06:05] thumper: thats good and will help annotate give better answers [06:16] sadness IllegalUseOfScopeReplacer: [06:17] can't figure out why I'd get that in bzrlib.smart.medium [06:23] What *are* inconsistent parents? [06:25] Peng_: when the cached index <> the revisions list of parents [06:39] 65->61 seconds for check tests [06:40] poolie: ping [06:40] poolie: I need a pointer; whats the current idiom for doing 'I have three tasks, they may take varying time each' [06:40] one minute [06:41] sure [06:41] I'll just delete progress for now ;) [06:42] lifeless: and they hppen sequentially? [06:42] i guess you should still use phases [06:42] i don't think that's quite right [06:42] or maybe the api is right and the way they're used is not always optimal [06:43] at any rate i don't have a clear better answer [06:45] well [06:45] gnerators but yes there are some fairly discrete phases [06:48] did that answer your queston? you can call [06:48] sufficiently [06:48] I've spend the whole day on check so far [06:48] it is faster [06:48] but I'm not done [06:51] igc, i sent a response to your doc [06:51] about nested trees [06:52] it was good, but it opened up some questions for me [06:52] poolie: thanks [06:52] poolie: that's good - that was the point of writing it :-) [06:53] indeed :) [06:53] lifeless: OK, thanks. :) How's that fixed? Change the index? [06:55] * Peng_ goes to bed. I shouldn't ask questions right before leaving. :\ [07:07] hi all [07:15] poolie: there appears to be significant overlap between the _render stuff in progress view, and progressbar [07:15] poolie: I'd like to encourage you to delete a bunch of stuff [07:21] i owe poolie a patch in this area too [07:36] hmmm [07:36] bzr: ERROR: Server sent an unexpected error: ('error', "'pqm@pqm.ubuntu.com-20090505195559-0qmeyyua7e407sym'") [07:36] not encouraging [07:36] spiv: you seen that pushing to lp? [07:37] lifeless: no, I haven't. [07:37] That is worrisome. [07:37] lp:~lifeless/bzr/check [07:37] lifeless: in reply to which verb? [07:37] which is a tad ironic [07:38] Heh. [07:38] oh, I see [07:38] File "/home/robertc/.bazaar/plugins/branchfeed/branch_feed.py", line 41, in post_change_hook [07:38] BranchFeed(change.branch).update() [07:38] it doesn't handle ghosts yet. [07:38] mea culpa. [07:39] its odd that its getting a nonstacked repo or whatever is going on [07:39] branchfeed? Does that generate RSS or something? [07:39] yes [07:39] https://edge.launchpad.net/bzr-branchfeed [08:01] * jml wants 1.15dev on Launchpad [08:03] HPSS calls: 72 (23 vfs) [08:03] hmm. [08:04] 49 is still a lot. [08:04] jml: doing what [08:04] and is that against 1.15? [08:04] lifeless: pushing up a new stacked branch. no, it's not against 1.15: from 1.15dev to 1.14 [08:05]