[00:00] SamB: so yes, it would kill us. [00:00] lifeless: how does it move the data? [00:00] we'd have to figure out a lot more structure than we do, which would push memory usage up, processing time up, and increase network traffic - in pathological cases by thousands of times. [00:00] tarball? [00:01] we send the delta compressed data as-is [00:01] with the minimum edits to its storage to meet user expectations about when data propogates [00:01] right. [00:02] is that how it works when pulling between formats? [00:02] depends on the formats [00:02] examples? [00:03] I don't mean to be rude, but I've a tonne of code to write [00:04] unless this is more than academic, can I encourage you to look at the various formats in repoformats/ [00:04] what code should I read, then? [00:04] oh, repoformats [00:04] which define constraints like needing topological insertion, atomic transactions etc [00:04] lifeless: bug 390563 is marked in-progress, should it be marked assigned to you [00:04] Launchpad bug 390563 in bzr "absent factory exception from smart server when streaming 2a stacked branches" [Critical,In progress] https://launchpad.net/bugs/390563 [00:05] cross format fetching is parameterised by the RepositoryFormat objects + the stream behaviour / InterDifferingSerializer (streaming is for network, IDS for local) [00:05] SamB: we try to do the least work at any time when copying data [00:06] its the whole background theme to all this code [00:06] poolie: no [00:06] mmmhmm. [00:06] poolie: jam has a branch to land [00:06] thanks [00:13] * SamB settles for running bzr branch with -rsvn:1000, then bzr pull with -rsvn:2000 etc. [00:15] (for the moment) [00:19] * SamB keeps expecting "bzr revert" to ask him hunk-by-hunk for some reason ... [00:25] I installed bzr from the source... how do I uninstall it? [00:33] delete it ;) [00:35] it installed like a million files in different directories [00:38] it should have written the bzr script, and a single directory 'bzrlib' in your python site-packages directory === kiko is now known as kiko-afk [00:45] jam: around? [00:52] I'm running bzr (1.16 or 1.17rc1) froma Fedora 11 installed inside VirtualBox, and it always hangs around 1054K or 1060K when checking out a branch... [00:57] any ideas why? [00:57] GungaDin: how long have you left it to conclude that it's hung? [00:59] hours? [00:59] It succeeds from other machines [01:01] SamB: i fully support getting better api docs, but filing bugs on every class without them, if that's where you're heading, is probably not the most efficient way to do it [01:02] how about if for instance you put up a branch containing your guesses at better descriptions or the answers people give to your questions about the api? [01:03] GungaDin: oh ok [01:03] GungaDin: no idea then really :( [01:03] hello mwhudson [01:03] hi poolie [01:04] yeah, weird. [01:04] Can somebody explain the correct way to merge two branches with the same ancestory? [01:06] are there any logs I can check? [01:22] MT-: bzr merge BRANCH-B [01:23] GungaDin: you can look in ~/.bzr.log [01:23] GungaDin: and/or use strace or lsof to see whats going on [01:23] garyvdm: I was confused because I would do that and then get a message when I tried to commit the merge about no changes having been made [01:24] garyvdm: I added --unchanged to the commit and it seems to have worked - thanks [01:24] Er. If commit said there were no changes, there probably were no changes... [01:25] (and if there were, and commit wasn't seeing them, --unchanged probably didn't make it start) [01:27] fullermd: that's why I was so confused [01:28] I suspect merge's output will be more enlightening. [01:30] http://bzr.pastebin.com/m1d9af2f9 [01:30] exactly like I'd expect [01:37] poolie: I wasn't going to do it for every class, no [01:37] for starters, I'm skipping all the ones whose names begin with _ ;-P [01:37] there probably should be some better way to track this [01:38] it just doesn't map well to bugs [01:38] poolie: yeah [01:38] but I couldn't think of any other thing to do that wouldn't get completely lost [01:39] and honestly I don't really have any but the vaguest guesses what it's for ... [01:39] http://starship.python.net/crew/mwh/bzrlibapi/undoccedSummary.html [01:39] (not linked to or very useful though) [01:40] lots of things might be clearly-enough named/subclassed that a docstring would, in fact, be redundant [01:40] I'm not planning to file bugs against those [01:40] I think its reasonable, when something confuses you, to ask for help on it [01:41] if you write a docstring and submit a merge as a result of the help, a bug is kindof wasteful [01:41] OTOH if you don't plan to submit a patch yourself, a bug is accurate, but perhaps its unlikely anyone will work on it [01:42] agree [01:42] i think one can be kind of usefully provocative by asking the question and then turning it into the patch [01:45] lifeless: I agree, but to do that ... wouldn't I have to have understood what you said about InterDifferingSerializer ? [01:46] * SamB does a lastlog to see if he mentioned it more than once? [01:47] jml: did my inventory checks stuff get into 1.17rc1? [01:48] lifeless, I don't know. [01:55] jelmer: what the heck is a "native foreign revision"? [02:01] lifeless - how would lsof or strace help me to understand what's going on? [02:04] GungaDin: well, it might end up telling you that something was actually happening [02:04] or at least give you an idea where they go wrong [02:08] How can I uninstall a version installed from the source? [02:09] SamB: where is that mentioned? [02:10] jelmer: bzrlib.foreign [02:10] GungaDin: as I said before you need to delete it [02:11] GungaDin: one way would be to install it from the source again, and take note of all the files installed [02:11] and then delete them [02:12] why don't you supply an uninstall option?? [02:15] GungaDin: python's distutils don't (currently) provide any mechanism to do that. [02:15] There's a proposal on python-dev atm that would address that, I think. [02:15] SamB: revisions that were not roundtripped from bzr [02:15] alright. [02:15] thx [02:16] jelmer: yeah ... that was my eventual guess [02:16] GungaDin: if I'm installing something system-wide from source I usually use stow, mostly because it means I can always uninstall it later. [02:17] jelmer: now to figure out how to say that without using an oxymoron [02:17] hmmm... never used stow. [02:17] never even heard of it, to be honest. [02:19] GungaDin: http://www.gnu.org/software/stow/ [02:19] Although it's probably packaged for Fedora, which I think you said you are using? [02:20] yeah, it is. [02:20] justed yummed it :P [02:20] GungaDin: tastey, is it? [02:20] missing a bit of salt. [02:21] hmm, what column should comments be wrapped at? [02:21] more or less [02:22] 79 [02:25] would it make sense to wrap a bit sooner if it puts a >40 char parenthetical on it's own line? [02:26] well, let me just paste ... [02:40] okay, here it is: http://paste.ubuntu.com/217434/ [02:40] which of those comments is better wrapped? [02:41] the first [02:42] okay, next question: does jelmer like the comment? [02:58] jml: why 79 and not 80? Does that apply for all code? [02:59] garyvdm, 79 is what pep8 says for general code wrapping. [02:59] garyvdm, I don't know whether it's binding on bzr (I suspect it isn't) [03:00] garyvdm, but if someone asks a wrapping question about a Python project, you need a good reason to say anything other than 79 :) [03:00] (pep 8 actually recommends 72 for comments & docstrings, but that's pretty silly, since none of the editors on my system will do that automatically) [03:01] jml: I was just under the impression that it was 80 not 79. [03:01] garyvdm: Emacs displays an \ and wraps if you go over that [03:01] (under SamB's configuration) [03:01] (with an 80-character-wide terminal, or the default window size) [03:01] jml: not just mine [03:02] Ah - the -1 is for the \ [03:02] that's, of course, the *default* default [03:02] garyvdm: yeah [03:03] Historically, some terminals had Issues(tm) with drawing characters in the last column of the last line, which is one reason to go with 79 over 80. [03:03] (probably none of them have been USED in a long count of years, but I probably have one in my closet :P) [03:04] well, depends how you define issues [03:04] * garyvdm changes editors edge line from default of 80 to 79 [03:05] there are some terminfo/termcap flags to specify standard "issues" terminal type has [03:06] jml: also, my editor seems to have somehow been given the impression that like 72 is good for comments in Python ... [03:06] +something [03:06] I sometimes wonder how many of those flags still work with modern apps. [03:07] I was looking through termcap the other week and reflecting that, "Gee, 'xterm' is used all the time, and 'vt100', and probably a few others like 'cons25', but I'll bet 98% of these entries haven't been used in 20 years." [03:08] fullermd: could be, but some of these flags *apply* to xterm and/or some of it's ilk [03:09] (oh, and better not forget "linux" and "screen") [03:10] Yeah. But then there are all these comments about "I don't know what this entry is for, there's no model number or attribution", with timestamps in the early 80's. [03:11] yeah, probably those ones aren't really used much ;-) [03:11] jml: I routinely use all 80 [03:11] because the bottom right character works these days [03:12] lifeless: vi-using jerk! [03:12] :P [03:14] Yeah. And all these entries for homebrew terminals... [03:14] Including (at a spotcheck) some that call for tabset files the system doesn't even include :p [03:14] what files ? [03:15] anyway, a lot of these flags are basically used by curses + screen [03:15] Beats me. I'm not insane enough to have learned termcap that deeply. [03:15] as far as I can tell [03:16] I learned enough to trick screen into doing some insane things and mess with VT10x and such. That was quite enough for me. [03:17] I only try to learn it when something isn't working right [03:18] test_transport_implementations has been renamed to per_transport? [03:21] jml: sounds plausible [03:23] jml: i have a branch fixing that [03:24] mwhudson, oh cool. anything I can do to get it landed? [03:25] jml: i'll ask you to review it [03:34] my bzr always hangs on "Fetching revisions:Get stream source" [03:34] .... [03:34] any ideas why? [03:35] look at it with strace or lsof [03:35] how do I look at it with lsof? [03:35] lsof just tells me what files are open. [03:36] thats right [03:36] its data about whats going on [03:36] just as strace would b e [03:45] back [04:00] * igc hits the review queue [04:03] when I do strace on the process, the output is: read(6, [04:03] what does that mean?? [04:05] it means its waiting for IO [04:05] on fd 6 [04:06] so why isn't any IO coming? :) [04:07] well, I don't know [04:07] lsof can tell you what fd 6 is [04:11] are there any open source projects that use bzr? [04:12] yes, lots [04:12] such as? [04:12] mysql [04:12] I appreciate you're having some trouble, and I'd like to help [04:12] you need to gather data though [04:13] I don't know how really... [04:13] I think fd 6 was just a pipe. [04:13] apart from that there's nothing much I can say... [04:13] checkout works on oher machines. [04:13] what url scheme are you pulling from? [04:13] (bzr+ssh? http? ...?) [04:13] bzr+ssh [04:13] that should just work really [04:13] and you can SSH out from that VM fine? [04:13] is it a very big project or some such? [04:14] yup [04:14] it's quite big [04:14] but it shouldn't hang like this [04:14] so the client is waiting for data from the server [04:14] do other clients work on the same project ok? [04:15] yup [04:15] that's what I said: the checkout works on other machines [04:16] to answer your other question more fully - http://bazaar-vcs.org/WhoUsesBzr - lists /some/ of the open source projects using bzr [04:16] cool [04:16] there are more (based on people asking for support) [04:16] I'll try checking out from one of those project [04:16] we'll see how it goes [04:16] theres something like 20K projects on launchpad with branches [04:16] all of ubuntu is imported, for instance [04:17] imported into bzr? [04:17] yes [04:17] from cvs? svn? [04:17] depends on the thing [04:17] most of the ubuntu packaging branches are imported from the dpkg source packages [04:17] * wgrant points over to https://code.launchpad.net/ for lots of branches. [04:17] there are also thousands of svn/cvs/git imports [04:18] GungaDin: I also suggest you file a bug to provide a place we can gather systematic data about your problem [04:19] I don't know if it's a bug. [04:20] something is wrong somewhere [04:20] it may be system config [04:20] or a bug [04:21] either way, having a place to record notes about the problem is a good idea [04:21] and we usually do that in our bug tracker [04:22] hmmm... seemed to work with https on alltray. [04:55] how can I checkout using bzr+ssh from an open source project on launchpad? [04:55] or somewhere else. [04:55] I'd like to see that the exact scheme works with a different project. [04:58] setup your ssh keys in launchpad [04:59] GungaDin: you need a lp user name with a ssh key set to use bzr+ssh on lp [04:59] then bzr branch lp:PROJECTNAME [04:59] then turn http:// into bzr+ssh://USERNAME@ in any of the http urls [04:59] *sigh* [05:18] doesn't seem like it's working [05:18] where do I check the branches of each project on lp? [05:22] The "code" link for a project will show the branches. e.g. https://code.launchpad.net/bzr [05:27] i'd like to say just ssh://host... [05:27] well, if I bzr+ssh://myaccount@code.launchpad.net/bzr/trunk - it hangs forever... [05:27] i know there's a bug for it [05:27] poolie: that still really squicks me [05:27] sorry but i think you'll just have to cope [05:27] you don't have to use it :) [05:28] as long as we don't drop support for bzr+ssh I'll deal [05:29] but I really think its liable to lead to trouble down the track [05:29] lifeless - I can't bzr+ssh the way you mentioned. [05:29] GungaDin: what happens [05:29] nothing [05:30] no error? [05:30] like the onion or someone said [05:30] I tried: bzr co bzr+ssh://myaccount@code.launchpad.net/bzr/trunk [05:30] "why don't we have _optional_ gay marriage?" [05:30] eventually I get a timeout error from ssh [05:30] GungaDin: branches are on bazaar.launchpad.net [05:30] GungaDin: bzr+ssh://myaccount@bazaar.launchpad.net/bzr/trunk [05:30] does that really still time out? [05:30] that's lame [05:30] GungaDin: e.g. bzr+ssh://myaccount@bazaar.launchpad.net/~bzr.dev/bzr/trunk [05:31] i thought the firewall was changed a long time ago [05:31] GungaDin: actually spiv has it right [05:31] maybe it regressed [05:31] poolie: I don't follow the onion ref [05:31] GungaDin: if you use "bzr launchpad-login" you can just do "bzr co lp:bzr" and it will look up the right bzr+ssh URL for 'bzr' for you. [05:31] nm, it's a joke [05:31] poolie: I'm concerned that url handlers in firefox are going to end up fighting with ssh: urls [05:31] poolie: in a way that won't happen with what we do today [05:31] as if all the people objecting to gay marriage were afraid they personally would have to gay marry [05:32] i'm not saying you're being that silly [05:32] poolie: and further that we're adding ambiguity that doesn't need to exist [05:32] or indeed that they are [05:32] poolie: I could care less what individuals use as local scheme aliases; things that are liable to make the ecosystem more fragile are something I do care about. [05:33] spiv - I get an error saying it's not a trunk [05:33] we could possibly apply a kind of 'be strict in what you emit' here [05:33] and just treat them as a shortcut on input [05:33] sorry.. [05:33] I mean "not a branch" [05:33] i think bzr rejecting somebody typing the url is too pedantic [05:34] GungaDin: oh sorry [05:34] GungaDin: .../~bzr/bzr/trunk [05:34] GungaDin: that's what I get for typing from memory! [05:34] GungaDin: like I say, I usually just use "lp:bzr" [05:34] spiv: however that can use http [05:34] better [05:34] spiv: and I don't think it tells the user [05:34] thx [05:35] spiv: which is why I suggested explicitness here for debugging [05:35] lifeless: I'm not disagreeing :) [05:35] ;) [05:36] poolie: bzr does tell the user what url they should be using. [05:37] poolie: if bzr were to nag that its altering the url to be bzr+ssh and translated, well that might be ok. But I think its pragmatically bad to start emitting ssh:// urls ourselves, or encouraging people to put them in app handles [05:37] ah [05:38] i'd forgotten it gave a suggestion in the message [05:38] let's discuss it if/when somebody actually starts on it [05:39] I'd actually like to put this to bed permanently someday [05:39] its a bit of a recurring nightmare [05:39] unless we embed a terminal in bzr [05:40] it's a URL [05:40] it locates a resource [05:40] it doesn't need to say everything about what to do with the resource [05:40] if you click on a ssh url in a browser, what should happen? [05:41] presumably it'll go through the 'what app do you want to use for this' function? [05:41] of course at present: nothing happens afaict [05:42] to me this seems reasonable [05:43] what do you think should happen? [05:43] As a user I'd expect the sshshell (there is a gui ssh shell colin walters has been working on - its really nice) to fire up by default [05:43] with no questions [05:44] k, that sounds good then [05:44] what's not to like? [05:44] if the web page wants to say "open this in bzr" there should definitely be a syntax for that [05:44] hm [05:45] i mean really it should be something like handwavey ssh://example.com::bzr/some/dir [05:45] ssh there *and then what*? [05:45] just pointing to the directory on the host doesn't really mean a lot [05:45] maybe this is in the ssh url spec [05:45] if any [05:45] this is a bit of a weakness in std66 [05:46] and AFAIK the only workaround at the moment is in the scheme [05:46] I'll note that bzr+ is less letters than ::bzr :P [05:46] lifeless: you just need a keyboard with a :: key ;P [05:47] You could maybe adapt a keyboard with a -- key for tla ;) [05:47] * lifeless shudders [05:47] i shall treat that trolling with the respect it deserves [05:48] pragmatically, web browsers dispatch off three things: [05:49] embedded controls where a plugin is invoked [05:49] the scheme [05:49] and the mime type [05:49] How big is the the trunk of bzr? [05:49] around 68147K? [05:49] GungaDin: about 70MB [05:49] seems like my bzr hangs again [05:50] strace shows the same thing [05:50] read(6, [05:50] GungaDin: and I guess if you strace the ssh subprocess it's stuck in a recv or a select on a socket? [05:51] yeah, select. [05:51] GungaDin: but it's managed to download 68147kB to .bzr ? [05:52] seems like it. [05:52] apparently it always hangs at the end. [05:52] like with my repo. [05:52] I wonder if you have a broken plugin installed? [05:52] Can you try sending SIGQUIT to your bzr process? [05:52] That will drop it into the Python debugger. [05:52] or -Dhpss perhaps [05:53] Then type "bt" and pastebin the result. [05:53] kill -? [05:53] kill -QUIT [05:53] well.. I already stopped it with CTRL-C [05:53] Or hit Ctrl-\ in the terminal. [05:53] Ah. [05:53] any other smaller repo? [05:53] Did it give a backtrace (or did it leave one in ~/.bzr.log)? [05:54] GungaDin: you can use the one you were having trouble with earlier, if thats smaller [05:54] ~jml/testtools/trunk on the same server is quite small. [05:56] ok [05:56] brb [05:57] GungaDin: another test to try would be to add --no-plugins to a command that hangs. [05:57] tried that already [05:57] it didn't help [05:57] Ok. [06:01] igc: I have more inventory delta stuff [06:02] igc: I'll be updating the merge request this avo [06:02] igc: I'd love if if you could tell me if it sucks perf wise [06:02] lifeless: shall do. I want to get through some more reviews then I'll make some measurements ... [06:03] I sent a SIGQUIT to bzr... what do you want to do now, spiv? [06:03] lifeless: I'm all for safety first but ... [06:03] lifeless: we need to be sure we don't throw away all the recent gains accidentally [06:04] for sure [06:04] the reason this has taken a week is I've been looking and thinking carefully about every change [06:05] I'm sure there will be fallout, but I'm also sure it will be much less than the gains we've made by better disk structure [06:09] GungaDin: type "bt" and pastebin the traceback that makes [06:10] GungaDin: that will tell us where in bzr it is stuck, which might help diagnose what's going wrong. (Even though it's almost certainly not bzr's fault) [06:18] jml: so, can code auto update diffs please?:) at the moment, clicking superseded and aking a new review seems to me to be *more* work for lp... [06:18] lifeless, you're asking the wrong person. [06:18] thumper isoffline [06:19] lifeless, well, I think Launchpad should auto-update diffs too [06:19] ok cool [06:21] speaking of which [06:21] how does one do the superceded dance? [06:22] I get an error about an existing proposal [06:24] EOD (started at 6:30 :P) [06:25] igc: updated thingy submitted [06:25] poolie: I'll get to your patch early tomorrow [06:25] lifeless: thanks [06:27] * spiv makes a late lunch [06:27] igc: I think (I still have to do a careful audit) that I now have complete coverage of things that make deltas go wrong [06:28] igc: which means that once I've done that audit I can get onto iter_changes again and be protected [06:28] lifeless: and in theory, my patch will fail, right? [06:28] i.e. be trapped by your new tests [06:28] igc: your tests won't, but I can construct cases using the regular command line that will make it generate bad deltas [06:29] and those would be caught [06:29] anyway, as I promised I'm going to fix iter_changes itself [06:29] theres no need to layer on it when iter_changes is, itself, broken [06:30] for instance, a simple example is [06:30] lifeless: thanks. I still never got why my stuff was wrong so I'm looking forward to understanding that soon [06:30] init; mkdir a, touch a/b [06:30] commit [06:30] mv a c [06:30] mkdir a touch a/b [06:30] commit a/b [06:31] this sort of situation would turn up after merging parallel imports, for instance. [06:32] igc: the layering approach isn't /wrong/ in a correctness sense; its wrong in a performance sense because of the full-iteration being done under the hood. Thats tolerable. [06:32] the correctness wrongness is that you have to spider out upwards just like we do downwards [06:32] in what we include [06:33] and that may have further implications (e.g. a spider down impact) that I haven't yet thought all the way through [06:33] it was while I was thinking it through that I realised our delta sanity checks were totally insufficient to protect us against mistakes [06:33] anyhow, I'm really done for today. [06:33] see you all tomorrow [06:33] lifeless: sure, seeya tomorrow [06:51] spiv: http://pastebin.com/d5fb72ec7 [06:53] GungaDin: hmm, so it's stuck while waiting to receive a response to a get_stream request (i.e. the big request that fetches all the data) [06:54] but why???? [06:54] Yeah, that's the question. [06:55] lifeless: let's talk more when it's a bit closer to the top of my stack [06:55] I mean, I can fetch branches from Launchpad via bzr+ssh just fine. [06:55] I tend to think it could be a problem with one of the VM drivers. [06:55] I can fetch it fine from other machines as well. [06:55]