[00:03] I saw that lifeless - thanks [00:03] I'm good with your reply [00:03] you've seen the new bundle ? [00:04] it's come thru but I'm yet to look at it [00:04] in email at the moment [00:15] New bug: #156003 in bazaar ""Tree transform is malformed" error with "bzr import-dsc"" [Undecided,New] https://launchpad.net/bugs/156003 === mwh_ is now known as mwh === kiko is now known as kiko-zzz [00:51] New bug: #156015 in bzr "Typo in doc/en/user-guide/conflicts.txt" [Undecided,New] https://launchpad.net/bugs/156015 [01:26] Turns out gutsy's kernel doesn't like my firewall box. [01:27] ouch [01:27] seen that reconcile isn't ? [01:27] spiv: ^ [01:28] lifeless: no, I haven't. I assume there's mail about it? [01:28] there's a bug [01:28] * lifeless -> chemist [01:28] ring my mobile if you want to chat [01:29] * spiv finds the bug [01:34] poolie: ok, I understand the problem; the bug describes the situation pretty clearly. I'll add a test scenario for it. [01:35] spiv, ok [01:35] do you need to narrow it down [01:35] what i mean is, we want a narrower test than reconciling the whole branch [01:35] do you know what narrower test is likely to fail? [01:36] Well, I can write a narrower test than reconciling bzr.dev, I think. [01:37] it seems likely that creating a simple repository with about three revisions should be enough to reproduce it. [01:38] Is that what you meant? [01:41] it is [01:41] it's just that i would have thought that's what the existing tests do [01:42] in other words, do you know what's untested and causing this problem? [01:42] It appears the "file version X is actually removed from the repo (versions with X as a parent rewritten appropriately)" case is untested. [01:44] I'm re-reading the existing scenarios to double-check that. [01:45] but wasn't that the main point of the previous patch? [01:46] igc, ping? [01:46] hi poolie [01:46] Hmm, FileParentsNotReferencedByAnyInventoryScenario would seem to cover this case. [01:46] * spiv digs further [01:47] spiv, could it be to do with this being a no-op file merge (random guess) [01:47] igc, quick call? [01:47] sure - I'll call now [01:47] j [02:31] ` bzr diff`'s help says "Shows the difference in the working tree versus the last commit". when i run the command i dont get any output. i'm running the version of bzr shipped in Debian 4.0 [02:32] am i doing something wrong? [02:32] kgoetz, are there in fact any changes in the working tree? [02:33] what does bzr status show? [02:34] bzr status returns nothing, and yes i can confirm theres been changes made. [02:34] what does bzr --version show? [02:34] Bazaar (bzr) 0.11.0 [02:34] and bzr diff FILE on one modified file? [02:36] returns nothing [02:36] ok, this is pretty strange [02:37] has this happened before? [02:37] 0.11 is pretty old, is upgrading an option at all? [02:37] no, but this is the first time i've tried using bzr diff [02:38] yes it is, if theres decent debs [02:39] would these work? http://backports.org/debian/pool/main/b/bzr/ [02:39] i don't recall any bug like this, but that release is over a year old... [02:40] i'll check those out. [02:40] 0.18 doesnt seem much newer then what i'm on though [02:44] kgoetz: if status shows nothing [02:44] kgoetz: are the changes actually in this tree, to files that have been added and are not ignored ? [02:44] lifeless: i dont understand the 2nd bit [02:47] kgoetz: if you have ignored a file [02:47] then it can change, and bzr diff will not show it [02:47] whats the name of a file you have changed ? [02:48] lifeless: bzr info says i have 0 ignored [02:51] how many versioned files? [02:52] http://pastebin.ca/746222 heres some of the info fromb zr [02:52] 25+2 subdirs [02:52] what does bzr ls show? [02:53] did you just create this with bzr init. bzr add, or did you get the branch from somewhere else? [02:54] i branched rev 39 from home a few hours ago, bzr merged rev 40 from my mate , tried to bzr diff and it didnt [02:55] just instaleld the new bzr from backports [02:56] and because it asked, i upgraded the 'working tree format' [02:56] bzr info now has no info in it [02:57] just 'location' and 'related branches' [02:59] and 'bzr diff' and 'bzr diff file' still return nothing [02:59] bzr --version [02:59] Bazaar (bzr) 0.18.0 [03:00] if bzr st shows nothing [03:00] then I bet that your friend has not pushed their changes [03:00] so the merge had nothing to merge [03:00] if you try merge now it will probably say that, our feedback has improved since 0.11. [03:01] i can see a change i didnt make: [03:01] ### THIS IS BAD COUPLING - SHOULD BE RUN FROM builderrvcore (andrew) ### [03:01] perhaps you committed the merge already? what does 'bzr log -r -1' show ? [03:02] what part of it? revo: 40; commiter: andrew [03:02] I think perhaps you ran 'bzr pull' not 'bzr merge' [03:03] not sure. what would that mean? [03:03] pull gives you a mirror of another branch [03:03] it would mean you have what andrew has done in front of you [03:03] assuming you are not andrew ;) [03:04] then why dont i see uncommited files in bzr status? [03:04] because there are no changes [03:04] you have a copy of andrew's branch, not your branch with his changes merged and pending review [03:05] * kgoetz is confused. perhaps looking at the documentation can be this arvos timespender [03:05] a good way to check this would be [03:05] bzr missing ANDREWS_BRANCH_URL|less [03:08] says 'branches are up to date' [03:08] then it looks like you did pull from him rather than merge [03:08] in other words - he's already done the merge and there's nothing for you to do [03:09] so, be happy :) [03:09] so i have to bzr ci before i can see his stuff? [03:09] you can modify a file yourself to check that diff works [03:09] no [03:09] his stuff is in your tree now [03:09] lifeless, could it be that in 0.11 merge did --pull by default? [03:09] you're already looking at his stuff. If you want to see a diff of it, you can probably do 'bzr diff -r -2' [03:09] poolie: no [03:09] is there some way i can see what changes he made then? [03:09] poolie: I'm pretty sure it never did that [03:09] i didn't think so [03:09] oh, with switches [03:10] kgoetz: you can use bzr log to see the commit messages; bzr log -v will show the files changed in each commit, and if you find your prior commit in log (which I bet is one commit back), bzr diff -r REVNO will show you the difference from that to whats on disk. [03:11] lifeless: ok, thanks. [03:11] so before i start hacking again: is it ok for bzr ifno to not contain any info past thelocation+parent branch (having just updated the format after installing 0.18) [03:12] yes, we removed some bits that were expensive to calculate [03:12] you can get them with info -v [03:13] ah sweet. not sure how committers is calculated, but i'm guessing by system name [03:14] yes, or you can set it with bzr whoami [03:14] thanks. [03:15] I'm going to go for a walk; [03:16] good, enjoy [03:16] I need to think through this apply delta logic [03:16] can call if you want [03:17] thanks, but its not one of those problems, its one of these [03:17] I think I have a failing test for the reconcile bug. [03:17] you/spiv can ring me if you want to talk about anything [03:17] yay [03:17] spiv, tell me more? [03:19] Reconcile doesn't seem to actually remove unwanted file versions, although in all the tests so far it correctly updates other file versions to not use them as parents. [03:20] Perhaps it's more accurate to say that I have a failing test that appears to be related to the reconcile bug, but maybe not the same issue. [03:22] so I got as far as a shave before I had my answer [03:24] igc: so why do you object to bugfixed ? [03:24] it's not a word :-) [03:25] 85000 examples of it on google [03:25] try answers.com [03:26] what about answers.com ? [03:26] heh [03:26] spiv, may i call? [03:26] it's neat - dictionary definitions, synonyms, etc. [03:27] poolie: sure [03:27] still not getting the point :). [03:28] bugfixed appears to be more than uncommon [03:28] you asked why I objected - I told you [03:28] if not actually common [03:28] bugfixed doesn't add anything over just saying "fixed" but whatever :-) [03:28] it's not important right now [03:29] ok [03:33] lifeless: commit tweaks is now approved as well. Just the spelling errors raised by Rob Weir. [03:35] thanks [03:36] * igc food [04:56] jam-laptop: ping [04:56] lifeless: hi, typing one handed, though [04:57] TMI [04:57] lol [04:57] so, in dirstates memory representation [04:57] are fileids utf8 ? [04:57] yes [04:57] and have we made fileids utf8 outside that yet ? [04:57] IIRC, in memory everything is [04:57] file ids == utf8 in the core [04:58] greate [04:58] same with rev ids [04:59] kgoetz: this is why http://john.arbash-meinel.com/images/baby_face_01_small.jpg [05:00] lifeless: I'm probably heading off to bed [05:00] anything else you need? [05:01] sleep well [05:01] jam-laptop: later mate. vcute [05:04] jam-laptop: actually there is something [05:04] jam-laptop: do you recall a function to iterate the entries within a given path:fileid pair in dirstate ? [05:19] abentley, ping [05:20] lifeless, did packs land? [05:22] keir: its in final review at the moment [05:23] poolie is sheparding it through, I'm hacking on a native dirstate tweak for commit [05:23] nice [05:23] i want to write that faster index but i have been super busy :( [05:42] ok, my plan now is to make a patch to the pack code that makes it complain when copying a delta whose base is not copied [05:43] keir: no worries; at worst your work on taking my design goals and coming up with something concrete to meet them has been great. [05:43] keir: thats very nearly as valuable as code [05:44] poolie: FWIW, the smart server fetch does a similar sanity check. [05:52] spiv: why isn't the test for insert_data_stream doing that check a per-implementation test ? [05:53] spiv: or in other words, why are some implementations allowed to not check that ? [05:54] lifeless, is create_pack_from_packs active even when fetching from knits? [05:55] lifeless: hmm, good question. Probably an oversight :/ [05:56] poolie: no, but copying from knits copies all the data. [05:56] spiv: rule of thumb: all tests should be interface tests unless they cannot apply to the interface in general. [05:57] poolie: its pack to pack thats a problem [06:07] Hello all. [06:08] I would like to create a new branch (from an existing branch), but only of a subtree. Is this possible with BZR? What is the recommended procedure? [06:09] To be clear, I have a tree under source control: ~/a [06:09] And I'd like to create a branch from a subtree of it... bzr branch ~/a/path/to/subtree [06:09] Possible? [06:10] lifeless, are you sure? i think i'm seeing problems when branching from an existing knits repository into a new pack repository [06:10] i'll check [06:11] mrtuple, can you tell us more about what you want to do with the new branch after you've made it? [06:11] will it eventually merge back into the whole tree? [06:12] ooh. that would be cool... [06:12] poolie: yes; pulling from knits copies everything in the knit reachable by that knits index. Remember that in your source pack repo all the data was present. [06:14] Yes, i'd like to place the branch in a http:// space so that I can commit to it and a partner can branch from and merge from it. [06:14] i thought i had tested that... [06:14] ring me if you need a reminder; though I thought you were doing the review changes first ? [06:14] (And I would do the same with my partners branch -- I believe this workflow has a name in bzr docs, but I forget what it is). [06:15] Then, finally, I'd like to remerge with the original ~/a [06:15] (Once the project is complete). [06:15] New bug: #156091 in bzr "create_pack_from_packs should check that knit compression parents are present" [Undecided,New] https://launchpad.net/bugs/156091 [06:38] spivvo, how's it going? [06:39] mrtuple, there's some partial support for this at the moment, but for production use you're probably better off using the whole tree fro now [06:39] mrtuple, if you want to try it there is an experimental 'split' command [06:48] hi guys [06:49] hi [06:51] poolie: [06:51] All the changes in the delta should be changes synchronising the basis [06:51] tree with some or all of the working tree, with a change to a directory [06:51] requiring that its contents have been recursively included. That is, [06:51] this is not a general purpose tree modification routine, but a helper [06:51] for commit which is not required to handle situations that do not arise [06:51] outside of commit. [06:51] thats what I'm adding to the docstring, do you think it's clear enough? [06:51] i have the task of maintaining a patch and i need to forward port this to newer versions of the same code. my problem is that i want to continue developing the code on the old base, yet somehow update my forward ports when i do this. making any sense? [06:52] effbiae: just keep a branch for each one. Then you can merge from old to new whenever you want. [06:52] effbiae: if you have the patch in a branch [06:52] as AfC says :) [06:52] lifeless, that sounds clear [06:52] poolie: I've confirmed that reconcile isn't finding anything wrong with the join-branches.txt-... knit. I think I have a change that may fix it, I'm testing now. [06:53] is there any point keeping the more general version around, or yagni? [06:53] "not finding anything wrong" even though it should [06:53] AfC: so i won't have to go through resolving conflicts again? [06:54] poolie: right. [06:54] effbiae: in theory [06:55] effbiae: so long as you get the original state into Bazaar, then when you apply your patch on 'new' branch, it'll be a series of revisions that will diverge from the common base [06:55] poolie: if a file version is never used by any inventory, then clearly no other file version should be using it as a parent. But join-branches.txt-... has that problem. [06:55] poolie: yagni I think [06:55] effbiae: meanwhile, changes to 'old' will likewise be change to that common base. You _should_ be able to then just merge from 'old' to 'new' periodically [06:55] Hmm, while this runs I'm going to pop out for a quick lunch. [06:56] effbiae: it's not foolproof, but you'll find that Bazaar is incredibly robust in its merge performance. You should be alright. [06:57] AfC, thanks for all your help. i'll try that. [07:04] AfC, do i have to bind one branch to another? [07:04] Huh? No [07:04] poolie: (see what I mean about the bind command?) [07:04] effbiae: just have each of 'old' and 'new' be a full power branch in its own directory. [07:24] AfC: i tried that, but i continue to have to resolve conflicts each time a do a merge... [07:24] effbiae: if thats the case, are you changing the same region of code a lot ? It may be that basically your patch needs to be updated lots. [07:27] effbiae: as lifeless says :) [07:27] poolie: so here's a funny thing. [07:28] lifeless: How much faster than knits are packs? [07:28] poolie: the 'bad_revid' from your bug is one of unreferenced versions in that versioned file. [07:29] jelmer: depends on the use case. The most dramatic difference I'veheard of is 10 minutes or so down to 30 seconds for local operations [07:29] lifeless: And compared to rsync/ [07:29] poolie: so in some sense it shouldn't matter what the parent in the knit for it is, because it shouldn't be used. [07:29] jelmer: well, we already beat rsync, again depending on the scenario [07:30] lifeless: For initial push? [07:30] jelmer: in fact though, because you could teach rsync not to touch existing index and pack files, and only sync new ones and deletes, and pack-names, rsync should still be faster (if you don't want selected data copying) [07:30] jelmer: A month or so ago I benched it over my adsl to london as 80% of rsync speed [07:31] spiv: you sure? 20050919... ? Thats referenced [07:31] I thought. Dagnammit. [07:31] lifeless: hmm, reconcile thinks not anyway. [07:31] lifeless: (Pdb) pp revision_versions.get_text_version('join-branches.txt-20050309044946-f635037a889c0388', 'robertc@robertcollins.net-20050919060519-f582f62146b0b458') [07:32] 'mbp@sourcefrog.net-20050309045105-d02cd410a115da2c' [07:32] lifeless: and independently looking at the inventory agrees. [07:33] Well, if that file version is referenced, it's not referenced by the revision with the same name ;) [07:33] spiv: indeed, it is. [07:33] erm [07:33] I mean, indeed, that revision does not itself point at it. [07:34] but clearly something does or it wouldn't be getting copied [07:34] >>> i = r.get_inventory('robertc@robertcollins.net-20050919060519-f582f62146b0b458') [07:34] >>> i['join-branches.txt-20050309044946-f635037a889c0388'] [07:34] InventoryFile('join-branches.txt-20050309044946-f635037a889c0388', 'join-branches.txt', parent_id='doc-20050309044934-a811c79dd26eef58', sha1='b8148e34e1546e83acf1ed32f4ec4bb4cc3502ae', len=3260) [07:34] >>> i['join-branches.txt-20050309044946-f635037a889c0388'].revision [07:34] 'mbp@sourcefrog.net-20050309045105-d02cd410a115da2c' [07:35] file_ids_altered.... [07:35] is a good way to trap this [07:35] add a if fileid == thing and version == thing: pdb() [07:36] lifeless: is this all for a smart server using packs or just a dumb server? [07:36] dumb server [07:36] probably worse on the smart server today, at least until spiv gets push optimised too [07:37] because the file_stream method uses append() to write data incrementally [07:37] and that will bite. painfully. [07:37] down to 4 errors on the native update routine [07:40] lifeless, ah, ok - thanks [07:41] pull will be ok I think [07:41] just push will splatter chunky bits everywhere [07:47] lifeless: ok, I see, there are 1290 revisions in bzr.dev with inventories referencing that file version. [07:47] [earlier talking about merging with AfC] but if i do a merge, how can i continue developing my patch on an old branch and retain the resolution i had to provide for the new branch (i think i'm confused) [07:48] lifeless: and none referencing the "parent" that the knit claims it has. [07:48] spiv: but clearly, the inventory that *should* doesn't. Grah. [07:48] lifeless: right. [07:49] effbiae: perhaps we're confused. :( [07:49] lifeless: oh well, happily I think I can make a test case out of this. [07:49] spiv: so we need to rewrite that inventory too [07:50] spiv: or the inventories that point to it incorrectly. How can we tell which is which ? [07:50] lifeless: Will that break signed testaments? [07:50] spiv: I don't recall if they refer to content purely or last-changed. content only I thought [07:51] woot [07:52] all tests passing, but the code is incomplete, so clearly so are the tests. [07:53] however, for testing, I'm uncommenting poolies comment [07:53] and we'll see how long it takes on moz [07:54] lifeless: so, can we tell which inventories are wrong? [07:54] lifeless: hmm, in this case it's easy [07:54] fuck me, 4 seconds faster on initial commit [07:55] lifeless: oh, sorry, nm [07:55] 5 seconds faster on regular incremental commit [07:55] that's pretty nice [07:56] real 0m19.527s [07:56] user 0m17.473s [07:56] sys 0m1.388s [07:56] real 0m12.243s [07:56] user 0m10.905s [07:56] sys 0m0.568s [07:56] hmm thats hard to read [07:56] lifeless: neat [07:56] the first group of three is for 'bzr commit' with 5 new files [07:56] That's for a commit on Mozilla? [07:56] the second is 'bzr commit FILENAME' [07:56] with filename changed [07:56] Lo-lan-do: yes [07:56] Nice [07:56] lifeless: so, here's a fun fact [07:56] jelmer: no luck - "out of memory: killed process" [07:56] without this patch: [07:56] real 0m25.148s [07:56] user 0m22.729s [07:56] sys 0m1.368s [07:56] real 0m16.834s [07:57] user 0m15.537s [07:57] lifeless: all 12 file versions of that file have the same sha1 [07:57] sys 0m0.776s [07:57] spiv: ok [07:57] mdke: run it again, as-is [07:57] mdke: there is a memory leak :(, it will resume [07:58] poolie: I might finish this tomorrow, I need to write a new iterator for dirstate [07:59] to handle renames with children that are not altered [07:59] lifeless: I will, but it looks like it didn't get very far [07:59] mdke: well, start it, and what does the progress bar look like [08:00] lifeless: so presumably then the inventories that reference that version are mistaken, rather than the inventory in the revision corresponding to that version. [08:05] excuse me... [08:05] given this: [08:05] A---B---C topic [08:05] / [08:05] D---E---F---G master [08:05] and between E and F, files were moved around [08:05] i have to produce patches E->C, F->C, G->C [08:05] but i will be continuing development on branch 'topic' and have to continue prov [08:05] iding combinations of patches [08:05] lifeless, just one question if you're still here [08:06] when you've replied to a review, have you either done or responded to everything in it [08:06] poolie: rofl. [08:06] common sense says yes [08:06] I don't believe I have dropped items from the reviews silently. [08:07] if I have elided anything, I think it was either (a) in a review from someone else, or (b) actioned and clearly trivial. [08:08] sure [08:08] I'm still staggered by the performance difference [08:09] I expected 25% as an upper bound based on lsprof [08:09] to get 33% is very pleasing [08:09] That's great work [08:15] lifeless: looks like it has started again. Anyway, when it was killed from the scrollback it looks like it only got to about five "=" signs, still 0/9 on the first branch [08:16] mdke: what is the status message ? [08:16] lifeless: no "=" signs, 0/9 on the first branch [08:16] lifeless: i guess the best way forward is for me to ask the vcs-import guys in Launchpad to do an import for me [08:17] mdke: it should have english at the right hand side [08:17] mdke: I'd really like to know what words are there. [08:18] lemme see [08:18] "branches/dapper:3442 0/9" [08:19] that's all it has, aside from the progres bar [== ] [08:19] jelmer: ^ does that mean its finished the mapping ? [08:19] mdke: there is a big mapping it does right at the start, which I think is completed for you [08:20] lifeless: I think you are right, it has whizzed through "Finding branches" [08:20] and it has obviously downloaded quite a lot of material, the directory is 191MB already [08:21] mmm, a 64Gb flash chip [08:33] hi guys [08:34] What is the official delay between deprecation declaration and deletion from the code base ? [08:35] lifeless: you know, it says "branches/feisty" now, perhaps it is going a bit faster than I thought :) [08:37] spiv: the inventories [08:37] spiv: is the one 20050919 a merge/different exec bit/path/name/parent id ? [08:37] spiv: e.g. should it have recorded a new version in the per file graph ? [08:42] * spiv looks [08:43] * lifeless -> counterstrike source [08:44] if anyone needs my input tonight, ring my house [08:44] otherwise, chat tomorrow. [08:44] lifeless: thanks for your help. [08:49] lifeless: the answer to your question is "no", btw; there's no changes to the inventory entry for that fileid in that revision compared to the two parent revisions. [08:50] lifeless: so AFAICT it should not have recorded a new version in the per file graph. [09:00] right, thats teh calculation you need to perform to decide whether its good or not. [09:00] lifeless: I see. [09:00] lifeless: there's an alternative to rewriting inventories... [09:02] lifeless: I could (somewhat arbitrarily) make the parents in the versioned file be [version-referenced-by-corresponding-inventory]. It's not "correct", but I think it would solve poolie's problem. [09:03] lifeless: I only mention it because it might be more expedient to do that than fix it properly. I'll have a go at fixing it properly now though... [09:07] lifeless: I'm a little surprised that a change to the executable bit and nothing else apparently means there should be a new version in the knit? [09:07] lifeless: why shouldn't the inventory entry still point to the same old file version in that case? [09:10] lifeless: oh, it's because the per-file graph is used for merge calculations, and is punned with the knit index? [09:23] * igc food [10:13] hi all [10:13] i have downloaded loggerhead source for front end of bazaar [10:14] but there are no steps for installation or configurations in it [10:14] could some one please help me out in configuring the loggerhead for bazaar [11:20] spiv: the per file graph record all changes to that fileid, not just content changes. [11:21] spiv: content reconstruction is separated in pack repositories, in principle. [11:29] indraveni: hi, still looking for help with loggerhead? [11:29] mwhudson, hi yes === mrevell is now known as mrevell-hospital [11:34] mwhudson, i have installed bzr [11:34] and then downloaded loggerhead [11:34] but there are no instauctions of how to install or configure loggerhead [11:34] indraveni: there should be a loggerhead.conf.in file in the download [11:35] copy that to loggerhead.conf and edit it as appropriate [11:35] mwhudson, yes its tehre [11:35] then run ./start-loggerhead.py -f [11:35] where should i copy it to ? [11:35] loggerhead.conf in the same directory [11:36] i'm not sure what you mean by *installing* loggerhead [11:36] tbh there's no real point [11:36] i have saved the loggerhead source in my dekstop [11:37] and there is a loggerhead.conf under the Desktop/loggerhead/loggerhead.conf [11:37] i should copy and paste this where? [11:37] oh, if it's called loggerhead.conf already [11:37] then just edit it in place [11:37] it has lots of comments in [11:39] mwhudson, what should i edit in this line [11:39] folder='/Users/robey/code/loggerhead' [11:40] have you read the comment in the line above this line? [11:40] mwhudson, there is no comment above this line [11:40] you want to put in the file path of the directory that contains the branch you want to browse [11:40] hm [11:40] mwhudson, only a line like [11:40] [[loggerhead]] [11:40] indraveni: where did you get the loggerhead source from? [11:41] from http://www.lag.net/loggerhead/ [11:41] ah [11:41] https://code.edge.launchpad.net/~mwhudson/loggerhead/production <- this version of the code is better [11:44] ok [11:50] mwhudson, i downloaded, but it placesd all the source in a folder named production [11:51] can i change the folder name? [11:51] yes [11:54] mwhudson, what should i spcify for cachepath [11:54] ? [11:55] indraveni: for now i would just comment it out [11:55] indraveni: how big are the branches that you want to browse (in terms of number of revisions, number of files)? [11:56] it may reach atleast for 10 revisions for each branch [11:57] i'm not sure i understand [11:57] 10 revisions is a tiny number [11:57] mwhudson, the project is in starting stage [11:57] ok [11:57] so don't worry about caching for now [11:58] it's only needed for pretty huge projects these days (10000+ revisions or 5000+ files) [11:58] mwhudson, but if needed, how do i need to do ? [11:58] indraveni: you would put a path to some directory there [11:59] mwhudson, ok, what does it do then ? [11:59] indraveni: it caches some information about the branch [11:59] mwhudson, ok [11:59] most importantly the list of files changed between revisions [11:59] mwhudson, next, the url value, is this needed now ? [11:59] (which is expensive to compute in bazaar today, at least for large branches) [12:00] url = 'http://bazaar-ng.org/bzr/bzr.dev' [12:00] indraveni: the file you're reading now looks like this http://codebrowse.launchpad.net/~mwhudson/loggerhead/production/annotate/pqm%40pqm.ubuntu.com-20070910193149-jrabz72cwth1abup?file_id=loggerhead.conf-20061215091925-m5t3bxgioe500tx9-1 ? [12:01] mwhudson, exacly [12:01] mwhudson, exactly [12:01] indraveni: then it has comments in [12:01] indraveni: i suggest you read them [12:16] lifeless, still awake? [12:19] passing by from time to time [12:19] whats up? [12:19] jelmer: ^ [12:20] lifeless: what's the timeframe on packs landing? [12:20] jelmer: couple of days [12:20] mwhudson: btw, the list copy reduction optimisation I landed for Peng, will help inventory extraction [12:21] lifeless: for packs or generically? [12:21] lifeless: cool, though [12:21] mwhudson: but only when the two inventories are asked for separately; so doing the further optimisation I put in as FUTURE there would help a lot [12:21] The discussion about VCS for Samba has come up again (git's UI is confusing people) and I'd like to show some impressive graphs (-: [12:21] generically. [12:21] lifeless: will still help the revision view, i expect [12:21] jelmer: did you see my commit speed stuff? 10 seconds for a specific file commit in a mozilla tree in my current uncommitted code === kiko-zzz is now known as kiko [12:22] the changelog view gets a bunch of inventories all at once with getRevisionTrees, iirc [12:22] mwhudson: that will not benefit as yet then [12:22] well, repository.revision_trees rather [12:22] lifeless: okidoke [12:23] lifeless: no, I haven't [12:23] mwhudson: turns out applying 3 list copies to thousands of items long lists is expensive. [12:23] when you do that hundreds of times [12:23] 16:56 < lifeless> real 0m25.148s [12:23] 16:56 < lifeless> user 0m22.729s [12:23] 16:56 < lifeless> sys 0m1.368s [12:23] 16:56 < lifeless> real 0m16.834s [12:23] 16:56 < lifeless> user 0m15.537s [12:23] 16:56 < lifeless> sys 0m0.776s [12:23] thats packs in my branch today [12:23] lifeless: who would have thought it [12:24] first is commit of 5 new files [12:24] second is commit of one changed file when we provide the file name [12:24] 16:55 < lifeless> real 0m19.527s [12:24] 16:55 < lifeless> user 0m17.473s [12:24] 16:55 < lifeless> sys 0m1.388s [12:24] 16:55 < lifeless> real 0m12.243s [12:24] 16:55 < lifeless> user 0m10.905s [12:24] 16:55 < lifeless> sys 0m0.568s [12:24] lifeless: in the firefox tree? [12:24] thats with a patch that is not quite complete to update the dirstate from the delta that commit calculates [12:24] for the same twocases [12:25] you can see it saves 5 seconds consistently [12:25] mwhudson: moz tree, yes. [12:25] 55K files. [12:25] well, 55K paths [12:26] lifeless: cool [12:26] lifeless: Should the weave_store for packs still work as it used to earlier or should I use some other interface to add revisions to it? [12:27] jelmer: as long as you take out a write group, it will still work, its just slower than the commit builder interface [12:27] lifeless: well, it's still a bit slow really, but at least it's competitive with the competition :-) [12:27] mwhudson: pffft [12:27] probably the right answer is not to have branches with 55k paths in [12:27] mwhudson: my laptop has slow disk, and its cryted [12:27] mwhudson: *crypted* [12:28] lifeless: i know, i'm not being very realistic [12:30] lifeless: it's a bit like "how do escape from this pit full of lions" -- the preferred solution involving not getting thrown into the pit full of lions in the first place [12:31] mwhudson: well. I think we can get down to 4-5 seconds for sure. [12:32] going lower will require...some more time [12:32] on thing I'm starting to seriously consider is a serialised euler tour variant for the revision graph [12:33] stored persistently and synthesised together on access [12:33] but I need a really clear head [12:33] and about three industrial whiteboards [12:34] i guess there are probably higher priorities [12:35] lifeless: is tree building any faster with packs? [12:35] its different [12:35] ;) [12:35] its not optimised yet [12:36] so over sftp it will be worse [12:36] but on local disk it appears a little worse, but branch is overall faster, so -for now- shrug. [12:36] it's pretty slow with launchpad currently [12:38] well 5000 files [12:39] convert a branch to packs. [12:39] then you can see [12:40] hm, maybe it's not as bad as i thought [12:40] 20s the second time [12:41] thats knit disk latency [12:41] echo 1 to /proc/vm/drop_caches [12:41] or whatever it is [12:41] then it will be slow again. [12:42] $ cat ~/bin/drop-caches [12:42] #!/bin/sh [12:42] # get written data to disk (not that its guaranteed) [12:42] sync [12:42] # (drop the unmodified pages) [12:42] echo 1 | sudo tee /proc/sys/vm/drop_caches [12:42] --- [12:44] right [12:44] packs suffer from that less [12:44] cool === Mez|Away is now known as Mez === Mez is now known as Mez|Away === Mez|Away is now known as Mez [13:07] lifeless: is it correct that packs appear to be using a significant amount of memory less than knits? [13:07] likely [13:08] but much of the tweaks should apply to knits too [13:08] so if you are comparing between different versions of bzr as well as disk formats, your measurements will be suspect [13:09] there are really three interesting data points [13:09] knits in last release your team tested [13:09] knits in the packs branch [13:09] packs in the packs branch [13:10] ok, I'll keep that in mind [13:11] (if you want before and after graphs I mean) [13:12] I may actually do some unscientific benchmarking later, comparing to git and mercurial [13:13] so far, it does at least feel as fast as things like mercurial or git [13:14] If it turns out to be so, I guess a benchmark made on the kernel tree would be an interesting thing to brag about :-) [13:20] By the way: is 0.92 right around the corner, or should I try and install it from bzr? [13:23] Lo-lan-do: for packs? You'll need bzr.dev + the packs patch posted to the list or roberts packs branch [13:24] Not for packs, but I'd like to check a recent bzr-svn [13:25] I'm not sure what the plans for 0.92 are [13:40] mwhudson, hi again [13:41] mwhudson, i have configured all accordingly [13:41] mwhudson, now i ran the command ./start-loggerhead.py -f [13:41] mwhudson, but the output is as follows, [13:42] http://pastebin.ca/746697 [13:46] mwhudson, its saying distribution not found [13:46] mwhudson, there? [13:55] any one there who can solve my error in loggerhead [13:59] indraveni: I've never tried loggerhead, but have you got TurboGears installed? [14:01] jelmer: I see you haven't closed #145148... is it just because the fix hasn't been released, or is the bug still present in current code? [14:02] Lo-lan-do: it's not fixed for certain corner cases yet [14:03] Ah, but it might be fixed in my case? [14:04] I'm trying to decide whether I want to install a recent bzr and bzr-svn, based on whether it'll unblock the changes I need to push to the Gforge SVN [14:07] Lo-lan-do: yes, that should work now [14:08] Lo-lan-do: there may be some issues replaying still, but recommitting your patches should fix that [14:08] That's great :-) [14:16] Aehm, so how exactly do I recommit my patches? Should I clean the SVN cache and the properties? [14:16] (I get the same old error message when I just do ~/bzr.dev/bzr push) [14:18] Lo-lan-do: create a separate copy of the svn branch using bzr-svn (in a different repository, or a standalone branch) [14:19] and then manually apply + commit your changes from the original branch [14:20] merging them from the original branch may also work, but I'm not sure [14:21] * Lo-lan-do tries that [14:22] Merging would be best, since I have derived branches too. [14:27] Urgh, segfault [14:27] During the "get" [14:28] ~/bzr.dev/bzr get svn+https://svn.gforge.org/svn/gforge/trunk gforge-trunk-new [14:28] / [ ] copying revision 1/4883Erreur de segmentation [14:28] * Lo-lan-do cries [14:29] :-/ [14:29] what versions of bzr/bzr-svn? [14:30] the bzr.dev and 0.4 branches? [14:30] Up-to-date from bzr.dev andbzr-svn [14:30] Right [14:30] does "bzr selftest svn" pass or does that segfault too? [14:31] I'll check [14:31] * jelmer meanwhile also tries to clone gforge trunk [14:31] Erm, re-trying a get seems to progress this time. [14:32] It's copying revisions (40/4883 so far) [14:32] And the selftests seem to work, too. Currently at 46/691 [14:33] I'll let both run for a while, as I need to go out for a few minutes. [14:45] k [14:52] [691/691 in 989s, 3 known failures, 9 skipped] bzrlib.plugins.svn.tests.test_blackbox.TestBranch.test_log_empty [14:53] sleep well all [14:54] Can I get a little help on how to use bzr to run a webpage? [14:56] The same as other SCM tools? [14:58] So I did a checkout, but the site needs to basically do a bzr update [14:58] but I don't have shell access on it. what do I do? [14:59] do I do a push? [15:00] Rotund: what access do you have? [15:00] sftp [15:01] no ssh though [15:01] rsync, perhaps? [15:01] push over sftp doesn't update the working tree [15:01] crap [15:03] what DOES update the working tree? [15:03] update, pull [15:03] running 'bzr up' on the server :/ [15:04] is there a secure rsync? [15:05] i think you can rsync over sftp [15:05] never tried it myself, mind [15:05] or ssh [15:05] RSYNC_RSH=ssh rsync ... [15:05] okay, well I don't have a shell [15:05] crap. this stinks [15:07] may need to write an extension === cprov is now known as cprov-lunch === mrevell-hospital is now known as mrevell [15:51] effbiae: I think I saw that your questions got a little lost. If you are still around, I can try to help. [16:24] effbiae: A pretty darn good discussion of it is here: http://www.venge.net/mtn-wiki/DaggyFixes === cprov-lunch is now known as cprov [17:08] Lo-lan-do, any luck? [17:08] The bzr get hasn't completed yet. [17:09] It grabs about one revision every 2s, and there are ~4900 revs to grab. [17:09] ouch [17:10] sorry you have to go through this :-/ [17:10] Yeah. 800 revs to go... [17:10] I'll definitely save a local copy of the branch before attempting to merge and push :-) [17:14] (-: [17:14] (700 left) [17:15] I'll play a game or two while waiting [18:13] jelmer: Trying to push the first revision now (which I merged from my old branch) [18:13] It seems to take quite some time, and strace tells me bzr is opening and closing lots of sockets to the SVN server. [18:14] Well, just the one, but it opens and closes it quite a lot. [18:14] has it committed anything yet? [18:14] Not as far as I can tell. [18:14] (in SVN) [18:15] Nope. svn update in a checkout brings no new revision. [18:16] Other strange thing: my bzr branch is now 225 MB large, while the previous one was only 55 MB, is that known? [18:17] Even the SVN checkout is "only" 139 MB [18:18] Oh, I know. My fault actually. [18:18] The previous branch was actually a lightweight checkout. [18:21] Still running. I'll go for dinner, and leave bzr running for now. [18:51] Pushed up to revision 4884. [18:51] real 16m18.448s [18:51] Yay! [18:52] Although, 16 minutes to commit three properties... [18:53] * Lo-lan-do tries to push another merged change [18:56] Looks like it's going to take another quarter of an hour [19:05] Ah, no, 8 minutes this time. [19:12] Less than 2 minutes for the third change [19:12] jelmer: It looks like I'll stop bothering you for a while :-) [19:13] Lo-lan-do: It worked!? Nice :-) [19:13] It's not over, note, and I'll need your help to sync the new branch into the old one (so I can continue using the old one, whose URL has been made public) [19:14] But it seems to work, step by step. [19:14] l=$(bzr missing --line ../gforge-trunk+bzr | tail -1) ; r=$(echo $l | cut -d: -f1) ; m=$(echo $l | cut -c29-) ; echo $r -- $m ; bzr merge -r$r ../gforge-trunk+bzr ; bzr commit -m"$m" ; time bzr push [19:14] One commit at a time :-) [19:17] is there a way to temporarily disable a plugin temporarily without having to remove the package from /bzrlib/plugins/ ? [19:19] gdoubleu: bzr --no-plugins might be too much for you. [19:20] gdoubleu: otherwise no, you have to move the directory. [19:20] gdoubleu: you can put it in a subdirectory [19:20] I use ~/.bazaar/plugins/DEACTIVATED myself [19:21] ah, --no-plugins will work for me as I don't need any other plugins at the moment either [19:22] I didn't see that option as it doesn't appear in the output of bzr help commands [19:25] thanks, now back to coding [19:31] bzr help global-options lists it. [19:31] lets hope that bends the space time continuum to reach him before he actually logs off. [19:32] You have an STC bender? [19:32] well, it's a work in progress, I wouldn't say it bends it yet, more strokes it gently. [19:48] jelmer: Okay, apparently it all went smoothly. I did a pull from SVN in the old branch, and both branches seem identical now. [19:48] bzr missing doesn't list any differences, nor does diff -ruN [19:49] cool [19:49] If weigon_ can also no longer reproduce the bug, I'll close it [19:49] jelmer: at least I don't have the problem case anymore [19:50] weigon_: how do you mean? [19:50] I can merge and commit again after I uncomitted the change [19:55] weigon_: As a workaround you mean? [19:55] yes [19:56] I havn't really turned it into a real testcase [20:02] weigon_: but can you still reproduce the bug? [20:02] nope [20:02] I'm clean now [20:05] Here to ask some more really silly questions, if I may.. I'm used to the model of something like perforce, where the entire structure of the remote repository is mirrored on the local machine, e.g. if something remotely is in DEPOT/ProjA/TestingBranch then that is where it is going to be relatively on your working system when you check it out... [20:06] With bazaar would it be possible to have a single "working" directory on your workstation and check out whichever branch you wanted to work on at the time to that location? [20:07] In otherwords, does it matter to bazaar where you make the branches on your local system? [20:07] VSpike: Bazaar doesn't care where you put them [20:07] I might recommend putting a shared repository (bzr init-repo --trees DEPOT) [20:07] at the root [20:08] jam-laptop: I'm doing that on what might be considered the main machine... shoudl I do the same on the second (the laptop) as well? [20:09] VSpike: having a shared repository means if you have 2 branches of the same code, they can share storage [20:09] so it sort of depends how you will work [20:09] If I do that, the locations under that repository will become important, will they not? [20:12] jelmer: I'm afraid you'll have to bear with me for some time still. [20:13] Lo-lan-do, what's going wrong? [20:13] SubversionException: ("File '/svn/gforge/trunk/gforge/debian/po/de.po' already exists", 175005) [20:13] (again) [20:13] I had the new and the old branch synced with SVN just fine. [20:14] I tried pulling from another branch called sid, which was originally based from trunk [20:14] Then pushing to SVN. [20:15] Tried that via the old gforge-trunk+bzr branch and the new gforge-trunk-new branch (both being bzr-svn branches), got twice the same result. [20:15] you can probably never directly pull from those branches that have the broken revision in their mainline [20:15] only merge them [20:15] Darn. [20:15] If I merge them and then pull into them, will I still have the problem? [20:16] I don't think so [20:17] It's worth a try. [20:17] the problem is that Subversion (the data in the properties) and Bazaar disagree about what the broken revision contains [20:18] I'm also trying to understand how to manage the relationship between a project and a library with bzr. Both are under development by me. Again, with something like perforce, because it takes a view of the state of the entire repository, the relationship between the project and the library is always controlled, even if you branch both of them, because the branches are just new copies under the depot's tree. How can I control this with bzr? [20:18] Lo-lan-do: this is the sort of bug that is a nightmare for bzr-svn, luckily you've been the only one so far with an issue like this, and this bug is not in any releases [20:18] Do I need to make my repo structure contain the lib and the application side by side in each branch, and always keep them together? [20:20] Because I started out going with /libA/main, libA/Branch1, /AppA/main, /AppA/BranchX and so on... [20:20] Lo-Lan-Do: Once you've merged your branches, I'd recommend thrashing your existing branches and continuing from a fresh clone of the Subversion branch [20:21] but will I have to do /main/libA, /main/AppA, /somebranch/libA, /somebranch/AppA instead... [20:21] and always make sure I take both the lib and the app at the same time when branching? [20:21] I'd love to, but will I be able to do so without changing them if they're stored in a shared repo? [20:22] IOW: can I replace (or delete) a branch in a shared repository? [20:23] No, but you could start over with a clean repository that only contains the main branch imported from Subversion [20:24] Blargh, even merging sid into trunk and trying to push trunk to svn fails :-( [20:25] if there are things you can't merge, you could "replay" them using the replay command [20:25] bzr: ERROR: unknown command "replay" [20:25] Lo-lan-do, it's part of the rebase plugin [20:26] More recent than 0.1 then? [20:26] yes, 0.2 [20:26] * Lo-lan-do fetches [20:29] $ bzr replay -r4877 ../gforge-sid/ [20:29] bzr: ERROR: exceptions.NameError: global name 'replay_delta_workingtree' is not defined [20:30] * jelmer sighs [20:30] * Lo-lan-do pulls his hair off [20:30] Another plugin I need to fetch? :-) === cprov is now known as cprov-out [20:32] one sec [20:36] VSpike: hi. [20:37] VSpike: can I explain some things that will hopefully help you understand better the model of Bazaar? You're questions got a little lost so it is hard to address them directly. [20:38] james_w: I'd appreciate it, thanks [20:38] Firstly, the branch is the most important concept in Bazaar. Everybody has a branch that they work on. You can work in lockstep so that each branch is a mirror of the others. [20:39] or you can have branch as the term branch is used in SVN etc., to mean a separate line of development. [20:39] I'm just having trouble making a mental switch from the model I'm used to... doesn't help that I just flew +7 time zones [20:40] Fundamentally Bazaar doesn't make the distinction here, so when 'branch' is used it doesn't always mean 'a separate line of development' it can just mean 'a copy of the code that allows you to make changes and commit'. [20:40] VSpike: well, give it 7 hours until your brain catches up and all should be fine. [20:41] :) [20:41] The next concept to consider is the 'repository'. There is an unfortunate naming clash here, this isn't the same as an SVN repostiory. I've never used perforce, so I can't say how it differs there. [20:42] There have been discussion about using a different term in Bazaar, such as 'vault', 'locker' or 'archive', but other terms also have a disadvantage. [20:43] perforce is very similar to svn in overall concepts [20:43] Think of a repository is just a store of revisions. There is no 'tip' revision or anything, it is just a big box where the revisions are kept. A standalone branch has a repository hidden away to keep this separation (you can see this if you do 'bzr init; ls .bzr'). [20:44] This means that a branch is just a pointer in to the 'box' that is the repository, which points at the commit that is the tip of the branch. [20:45] This tip then points to its parents, and they point to there parents, so if you pull the string that starts at this tip revision you get the full history of the branch. [20:46] Now, as the repository is just a 'box' you can have many branches that point to tip revisions within. The repository doesn't care if they are related in any way, or if two point to the same tip revision. [20:47] Therefore you can create a 'shared repository', which is just a box where multiple branches can store their revisions. [20:47] This is what you do with the 'init-repo' command. [20:47] jelmer: I added replay_delta_workingtree to the "from rebase import ..." clause for class cmd_replay, seems to work better [20:49] Now when you have two branches of the same project inside a shared repository they can share some of the same revision data, and so save space compared to each having their own repository. This means you can just consider the shared repository as a storage optimisation. [20:50] and as the repository doesn't care about the branches that use it, you can name a branch inside a repository anything you like. [20:51] so, to hopefully address what I think was your first question, if you create a shared repository on your central server, then it doesn't matter whether you create one on your development machine or not. [20:51] If you do also create one on your development machine then you can name the branches within differently to the server if you like. [20:52] OK, with you so far [20:52] This is because the repository knows knothing about the other machine. If you want to associate a branch on your development machine with a branch on your central server then you use the URI to do so. [20:53] This means you would use something like bzr+ssh://central/wherever/branch1 [20:53] so branches are associated using URIs, rather than by using their position relative to a repository like in SVN. [20:54] OK [20:55] so if you have repo/branch1 and repo/branch2 on your server, you can have repo/branch3 and repo/branch4 on your development box, where 'bzr pull' in branch3 would pull from branch1, but 'bzr pull' in branch4 could pull from a completely different project on a completely different server. [20:55] This can lose you some convenience, but can be more flexible. [20:55] VSpike: happy with that? Does it answer your first set of questions? [20:56] Yes thanks, I get that. It doesn't answer one of them, but I don't think that other question was possibly a red herring anyway [20:57] So the next big one was about relationship between two sub projects [20:57] btw, excellent explanations - thanks [20:57] < VSpike> With bazaar would it be possible to have a single "working" directory on your workstation and check out whichever branch you wanted to work on at the time to that location? [20:57] that one? [20:59] yeah, that one I think is a red herring, but I'm curious [20:59] ok, short answer, yes it is possible. [21:00] I can see how it would work without using a shared repository on the workstation, but I would have thought using a shared repository would make it impossible? [21:00] (but probably not as easy, as you don't have a known layout on the server). [21:00] to answer this we need to consider the third bzr concept: working trees. [21:01] A working tree is the files that you have on your machine to edit and commit from. [21:01] Many people wouldn't distinguish this from the branch, but they are different. [21:01] Each working tree has a pointer to a branch that it will commit to when you commit. [21:02] Usually this is not obvious, as it is just the branch that is in the same directory as the working tree. [21:02] However you can separate the two physically, and even put them on two different machines. [21:03] Doing this separation gives you a 'lightweight checkout'. [21:03] ahhh okay [21:03] I see how that would work I think [21:03] (Yes, there is a heavyweight checkout, I can explain them later if you like). [21:03] So, you create a lightweight checkout on your local machine of the branch you want to change on your server. [21:04] when you want to change the branch you install bzrtools and run 'bzr switch URL', where the URL is another branch on the server. [21:05] so as I said, it's not as easy, as you have to give a full URL. We would be glad to hear a good scheme to improve this. (You can use env vars to do it now if you do it load). [21:05] bzrtools is a set of plugins that add sometimes useful features to bzr. [21:05] does that answer the question? [21:06] Yes thanks [21:06] good. [21:07] now for the libraries question. [21:07] :) [21:07] This is a very common task, and one that we want to support. [21:07] The idea is to do something like svn:externals if you have used those. [21:07] We call it 'nested trees'. [21:08] The format that will hopefully support doing this is in recent versions of bzr. [21:09] The commands to manage this are not. [21:09] I would interject one small thing, though [21:09] (I might have to go with 'these are not the commands you are looking for' if you look too hard). [21:10] In that "nested-by-value" does work [21:10] "nested-by-reference" (svn:externals) does not [21:10] I've done it with nested-by-value, and it actually works pretty well [21:10] but it means you have to be more careful when you want to pull any changes out of your subproject [21:11] I've never properly grasped by-value. [21:11] james_w: It is just merging the files into the other tree [21:11] such that they maintain their history and file ids [21:11] But Aaron is being quite insistent on this issue currently, so I didn't want to encourage it. [21:11] (bzr merge -r0..? or bzr merge-into from the plugin) [21:11] james_w: He is talking about by-reference [21:11] but yeah, that was a pretty strong discussion [21:12] jam-laptop: I helped someone get merge-into working, did you get the patch for it? [21:12] I don't remember seeing a patch for it [21:12] jam-laptop: ah, that distinction wasn't clear, thanks for clarifying. [21:13] So maybe the simplest solution for me for now is to keep the two together and treat them as a single entity in bzr? [21:13] that would work yes. The other alternative is to keep them separate and just branch them both whenever you need them. [21:13] VSpike: well, *I* would have 2 separate branches, which I then merged together [21:14] And I would try to make changes to the sub-project in the split-out branch [21:14] and then merge them into the combined one [21:14] I think that will help my other other problem, which is how to make visual studio deal with it... that way, the relative path between one and the other will be unchanging [21:14] but it *is* extra work [21:14] jam-laptop: bug #144108 [21:14] Launchpad bug 144108 in bzr-merge-into "Doesn't work with 0.90 -- no attribute 'base_is_other_ancestor'" [Undecided,New] https://launchpad.net/bugs/144108 [21:14] VSpike: yes, simplest solution is to make one branch, but it gives you less flexibility later. [21:15] you can always "bzr split" but your sub-project will keep tracking all the extra files from the combined project [21:15] jam-laptop: the problem that he mentions is what I wanted to ask you about. [21:15] unfortunately the salient details elude me. [21:15] (well, grabbing just a branch of the subproject will also copy the history files for files which *used* to be there) [21:15] james_w: you mean about 144108? [21:16] true, but I think - without having used it yet - that the flexible branching in bzr will still allow me to do what I want [21:16] jam-laptop: yes, his final comment in the patch mail. [21:16] I'm seeing lots of other failures trying to run with bzr.dev [21:16] So I'm guessing some api's changed in ways I didn't expect [21:17] Like instead of getting a path at one point, I'm getting a number [21:17] this was a while ago that we worked on it, so it might well be broken again. [21:17] jam-laptop: is it intened to support continually merging from the original branch? [21:17] 'merge-into' ? [21:18] Yes, you should be able to continue to merge from the original [21:18] The only issue is that you need unique tree roots [21:18] or adding a file in the root of the merged project [21:18] will add it in the root of the combined project [21:18] actually I don't know whether he was doing 'merge' or 'merge-into' on subsequent tries. [21:19] I think that was one issue. [21:19] The other was that you got some sort of conflict when a file was deleted from the merged project. [21:19] I said this was a limitation at the time, but I can't remember why. [21:20] james_w: I think the problem is that you have a rename of the file, versus a deletion [21:20] so there is a conflict about paths [21:20] (you have a change, they have a change that cannot apply exactly) [21:20] that's the badger. [21:20] but I could be wrong [21:20] I haven't tried it a lot [21:21] I need to see if I have the latest version, something is weird [21:21] yeah, so every deletion is a conflict, because every file is always a rename compared to the other branch. [21:23] VSpike: all sorted now? [21:24] moin [21:24] jam-laptop: can you explain what you meant when you said you'd have two separate branches which you'd then merge together, and make changes to the subproject in the split out branch? [21:25] bzr init project [21:25] bzr init library [21:25] cd project [21:25] jam-laptop: I read it a few times, but I can't quite get it :) brains still lagging [21:25] hi lifeless [21:25] bzr merge-into../library library [21:25] bzr commit -m "Add library to project" [21:25] VSpike: At this point, you have a project which has merged library into it [21:26] Then when you make changes in the standalone "library" project [21:26] james_w: on the verge of it, i think, thanks to you both [21:26] you can then go to "project" [21:26] and [21:26] bzr merge ../library [21:26] bzr commit -m "Merge the latest changes to library" [21:26] VSpike: do you follow me so far? [21:26] (of course, this is provided that I get merge-into working again) [21:27] VSpike: no problem, come back if you have any more issues. [21:27] jam-laptop: i do .. yep.. amazingly [21:27] Now, if you make changes to the library [21:27] (the copy that is in "project") [21:28] you have to "cherry-pick" them back into the separate 'library' project [21:28] cd library, bzr merge -r 10..12 ../project; bzr commit -m "Cherry pick the changes out of project for library" [21:29] The 10 should be the last revision *before* the changes, and the 12 is the last revision *of* the changes [21:29] it may be easier if I give more specific examples [21:29] which I can do [21:29] I think I follow you [21:29] VSpike: but if you follow me so far, it means I don't have to :) [21:29] VSpike: Anyway, Bazaar should be smart enough to merge changes to the correct files after you do this [21:30] though as james_w points out, it may tree deleted files as conflicts [21:30] What is the advantage of doing it that way as opposed to keeping it monolithic? [21:30] VSpike: if you want to have 4 projects which all share "library" [21:30] Or you have philosophical reasons [21:30] (easy to rectify, but perhaps annoying if you do it often) [21:30] like wanting to version "library" independently [21:30] right, makes sense [21:31] One company I worked for had about 90 separate branches that built together for the project [21:31] where each plugin/module was a separate branch [21:31] but the build stage built everything togethehr [21:31] together [21:32] because I was wondering how easy it would be if I have two branches, each with a monolithic proj+lib combination, and I made some fixed in the in library in branch A and wanted to merge them into branch B without getting any of the changes to the main project [21:32] *I* think our eventual nested-by-reference will be the "Right Way" to do it. [21:32] VSpike: that is where having a separate library branch helps [21:32] in that you can cherry pick those changes out [21:32] and then just merge them into the other project [21:33] You actually could do that between just the monolithic projects [21:33] But if I had three or four branches, i'd have to do it three or four time, while with your method i basically do the cherry picking once, and then merge into the other branches, right? [21:34] but that wouldn't necessarily satisfy my definiton of fun for some of the cases you could hit. [21:34] VSpike: yes, that's correct. [21:34] or just fix it in the library branch and merge in to all projects, depending on where you want to fix it first. [21:34] true [21:35] I think the fog is clearing :) [21:37] VSpike: obviously any help to make by-reference nested trees work better would be appreciated :) [21:37] :) [21:37] I'm not convinced that the time spent doing that would be less that the time you spent cherry picking etc. with jam's system. [21:38] Time spent getting by-reference working? [21:38] yeah. [21:39] Lo-lan-do: thanks, now fixed upstream too [21:39] jam-laptop: I have native update_basis_via_delta working. [21:39] jam-laptop: 5 second win [21:39] Lo-lan-do, any luck using replay? [21:40] lifeless: nice [21:40] certainly avoiding rebuilding the full tree is nice [21:40] jelmer: Nope. I ended up doing a shell loop around bzr diff and patch -p1 [21:40] lifeless: I had a question about _make_absent [21:40] for bug #114615 [21:40] Launchpad bug 114615 in bzr "Commit can fail and corrupt tree state after encountering some merge/conflicts" [High,Confirmed] https://launchpad.net/bugs/114615 [21:40] The idea is that it tracks down all references to the row that is being removed [21:41] I'm almost done, then I'll try pushing. If it works, then I'll trash the old branches and replace with clean new ones. [21:41] and marks the current value as absent [21:41] It has an assertion that they are not already marked as absent [21:41] But that breaks when you rename inside a directory [21:41] and then remove that directory [21:41] (so that commit unversions the whole subtree) [21:41] Because it finds one half of the rename, marks both sides absent, and then finds the other side [21:41] and tries to do it again [21:42] uhm [21:42] so this is in set_parent_trees? [21:42] unversion() [21:42] oh [21:42] I take it back [21:42] it finds one half of the rename [21:42] and marks just that entry as absent [21:42] then finds the other half [21:42] and tries to mark *both* sides absent [21:43] Because we don't pay attention to the working tree [21:43] but we do to all parents [21:43] so this is a merge [21:43] we have 3 trees [21:43] what's the kind and pointer values at the unversioned path ? [21:43] path being unversioned that is [21:44] (it also has a chance to accidentally remove a file which is actually moved out of that directory, but I haven't simplified that down to a simple case yet) [21:44] lifeless: doesn't have to be a merge [21:44] see my last test case [21:44] bzr init; mkdir dir; touch dir/file; bzr add; bzr commit [21:44] bzr mv dir/file dir/z; rm -rf dir; bzr commit -m "boom" [21:45] That first finds that all entries in "dir/" need to be removed [21:45] Then it finds "file" [21:45] which is shown to be renamed to "dir/z" [21:45] oh, sorry [21:45] it finds "dir/file" [21:45] and marks it as absent [21:45] then it finds "dir/z" [21:46] and sees that it is renamed from "dir/file" (because that was the path in the basis" [21:46] ) [21:46] so it tries to mark dir/file absent again, and dir/z as absent [21:46] but it fails because dir/file was already marked absent [21:47] My "fix" is to just remove the assertion [21:47] There is another bug present [21:47] but that fixes the assertion error [21:47] And it passes all the other tests [21:49] and were considering dir/file at all because ? [21:49] dir/file is not present in tree 0, unversion shouldn't be touching it [21:49] lifeless: it is in the dirblock of 'dir' [21:50] with 'r' [21:50] while entry_index < len(block[1]): [21:50] # Mark this file id as having been removed [21:50] which is 'not here but renamed' [21:50] entry = block[1][entry_index] [21:50] ids_to_unversion.discard(entry[0][2]) [21:50] if (entry[1][0][0] == 'a' [21:50] or not state._make_absent(entry)): [21:50] entry_index += 1 [21:50] That is 'unversion' [21:50] You could argue for "entry[1][0][0] in ('a', 'r')" [21:50] Which I could also accept [21:51] I think that would be more correct [21:51] I'll see if the tests pass with that... [21:51] because turning an 'r' into an 'a' is a massive problem [21:51] what if the 'r' is outside this directory [21:51] we'll lose our pointer, status will start misbehaving [21:53] real 1m30.277s [21:53] user 1m22.405s [21:53] sys 0m4.648s [21:53] initial [21:53] real 0m25.044s [21:53] user 0m17.357s [21:53] sys 0m1.284s [21:53] lifeless: do we have a specific "target" time? [21:53] adding 5 files [21:53] real 0m11.646s [21:53] user 0m10.825s [21:53] sys 0m0.632s [21:53] changing one and specifying it on the command line [21:53] lifeless: also, it would be easier to read if you did "/usr/bin/time XXX" rather than "time XXX" [21:53] 0.14 real 0.00 user 0.00 sys [21:54] jam-laptop: I'd like to get our cpu for initial commit down to only twice tar czf [21:54] lifeless: and what time is that? [21:54] for incremental operations, I'd like it to be roughly the time for 'st' [21:54] sure, I'm just wondering what the actual number is [21:55] on my machine, tar czf is [21:55] hmm, not sure, but gzip of the tar is 35 seconds [21:56] I find the vertical layout easiest to read, it may just be me [21:56] lifeless: it is much easier to compare 2 copies [21:56] when they line up [21:56] right, I do that left to right [21:56] then you should paste them here that way :) [21:57] these three sets are not meant to be compared [21:57] they are different use cases [21:57] it still would be easier to read :) [21:57] heh [21:59] well, the fact that my IRC program uses a proportional font doesn't help [22:00] so fix it ? :) [22:05] except it is actually easier to read most text [22:05] (softer on the eyes) [22:06] lifeless: well if gzip of the tar is 35s, then you still have all the fs overhead, etc. 1m22s seems pretty close to your mark [22:06] Unless you are just considering user/sys time? [22:07] 1m22 is close [22:07] but not there :) [22:08] 1m30 wall clock time is considerably more clearly 'not there' [22:08] 15 failures with this patch. [22:09] jelmer: It's pushed! [22:09] LarstiQ: ping [22:09] Lo-lan-do: cool! [22:09] lifeless: pong [22:09] how are you? [22:10] is your life stabilised ? [22:10] lifeless: sorta kinda [22:10] Lo-lan-do: hope this bug is finally fixed then [22:10] I'd love it if you got the subtree patch out and made hot sweet love to it [22:10] jelmer: I hope so too [22:11] LarstiQ: good to see you around [22:11] lifeless: anything in specific (ie, I recall a dirstate problem), or the entirety? [22:11] jam-laptop: thanks [22:11] * LarstiQ wouldn't call himself 'around' just yet, but thanks nonetheless [22:11] LarstiQ: there's a heated discussion on malone at the moment [22:12] lifeless: got a link to that? [22:12] that revolves around knit3 repositories existing, but not being something that folk hacking on bzr.dev as the majority of their hacking time want to consider 'fully supported' yet. [22:12] bug 131667 [22:12] Launchpad bug 131667 in bzr "--dirstate-with-subtree is documented in the man page" [Medium,Fix committed] https://launchpad.net/bugs/131667 [22:12] LarstiQ: ^ [22:12] LarstiQ: the 'most recently touched' bug links are quite nice [22:13] lifeless: ah, heated would imply that, yes [22:14] lifeless: quick question on "unversion" does it's list include child entries? [22:14] or does it just get the top-level dir ? [22:17] lifeless: I see. [22:21] jam-laptop: IIRC its handed in the user supplied path [22:21] jam-laptop: so its expected to recurse [22:21] lifeless: unversion() takes a list/set of file_ids [22:21] it is called by the commit code [22:21] after commit determines things to auto-remove [22:22] But I can see that the code *does* recurse [22:22] so I'm starting with that [22:22] right, commit stops recursing at the top level path it finds missing [22:23] jelmer: poke [22:24] LarstiQ: porrrr [22:24] jelmer: are you attending the bapc? [22:24] yup, I'll be there [22:24] jelmer: ok, have time for a chat then? [22:25] Sure :-) [22:25] bapc ? [22:25] lifeless: I haven't really kept up, but I thought I saw some changes go into bzr.dev that might make subtrees slightly easier. [22:26] hi LarstiQ [22:26] lifeless: I need to take a look at what the current combined status is before an assesement of the time involved can hope to be accurate, but I hope the major hurdles are solved. [22:26] lifeless: Benelux Algorithm Programming Contest [22:26] for some reason I got drafted into a spectator team *boggle* [22:28] lifeless: I can reproduce having a massive time difference between initial commit and recommit, but the circumstances where it's completely reproducible, the extra time on the recommit appears to be mostly I/O -- it shows up under "real", but not "user" or "sys" under time, as opposed to the first time where it was just about all accounted to user. I'm generating callgrind files. [22:28] james_w: heya [22:28] s/but the circumstances/but under the circumstances/ [22:29] nDuff: thanks [22:34] nDuff: are you saying that the total real time doesn't change? Just that it moves out of user time? [22:35] nDuff: oh, hangong. Are you saying that recommit got faster ? [22:35] or that recommit was *slower* ? [22:36] jam-laptop: no; the real time changes dramatically (4min initial vs 45min recommit), but the user time changes comparatively little (~2m40s on both). [22:36] so recommit is 10x slower than initial commit [22:36] hmm... [22:36] that is after just doing [22:36] bzr uncommit [22:36] bzr commit [22:36] right? [22:36] yup. [22:36] Is this the first commit? [22:37] or an incremental commit [22:37] right now, I'm testing with the first commit. previously, it was an incremental. [22:37] so 4 versus 45 is for the first commit [22:37] yes. [22:38] nDuff: the only thing I can think of is autopack [22:38] if this is a shared repository, or even just a branch you are reusing extensively [22:38] lifeless: a second commit shouldn't autopack, right? [22:38] nDuff: have a look in ~/.bzr.log [22:39] it would be the 10th [22:39] jam-laptop: right, but a reused branch or a shared repo will, because its the global rev count not the branch reachable revcount [22:39] lifeless: sure [22:39] but it would be a multiple of 10 [22:40] nDuff: look for autopack [22:40] jam-laptop: well not strictly no. Its whenever the desired pack count is greater than the real pack count [22:41] jam-laptop: the desired pack count changes for every 10 revisions; and the real increments by one every write group [22:41] ah, sure [22:42] lifeless: I was thinking that maybe it was looking up the indexes for the files [22:42] which are now non-empty [22:42] even though the existing revision is not in the current history [22:43] but autopack could also be an issue [22:44] it's not the hashcache being dropped again is it? [22:44] james_w: I don't think that would account for 45 min [22:44] and doing [22:44] jam-laptop: it will look to see that the inserted revisions are really new I think still [22:44] bzr uncommit; bzr status [22:44] would restore the hash cache [22:44] oh, %#$^ [22:44] and while the index layer in packs has different tradeoffs, 10x slower is totally unexpected [22:45] nDuff: ? [22:45] I dropped the --experimental from my "bzr init"s [22:45] okay, going back and starting from scratch. [22:45] nDuff: rofl [22:45] well, thank you for letting us know how long commit on knits is :) [22:46] nDuff: is that for all your tests? [22:46] (the 4 versus 45?) [22:46] Or is that just for your --lsprof ones? [22:47] jam-laptop: yes. consider everything I've said on the topic invalidated at this point; I'll get back after redoing them. [22:48] (actually, this might explain why I couldn't reproduce the 15min-of-user-time case) [22:49] oh. we're logging to ~/.bzr.log, eh? [22:49] I just thought of something. [22:50] my home directory is on very nonperformant network storage. [22:50] large amounts of logging could explain cases where I see lots of wall-clock time but not user or system time. [22:51] * nDuff looks into redirecting that to a path on local disk. [22:51] does BZR_HOME work for that? [22:53] yes [22:53] I think there is also a specific pathf or that [22:53] we shouldn't be logging all that much [22:54] lifeless: looking through ~/.bzr.log, I at least see a case where we're logging a line per file added. [22:54] lifeless: on my tree, that's significant. [22:54] nDuff: yes, that would be. damn, I thought we'd corrected that. [22:54] its basically spam. [22:55] lifeless: I didn't look at when it was logged; it may be from an older version of bzr. [22:55] nDuff: oh, btw, bzr commit -q is faster too [22:55] * nDuff has been using -q [22:55] cool :) [22:55] as is 'bzr add -q' :) [22:55] ok, time to see if I've fixed all bugs [22:56] fingers crossed [22:56] okay, using -packs, I don't get that 4-vs-45min behavior on the initial commit. [22:56] whew [22:56] wow, you mean we get to set Fixed Released to all 581 bugs? [22:56] blah [22:56] thank you very much lifeless [22:57] thou knowest what I mean [22:57] sure, but a man can dream [22:57] I had 15 failing tests [22:57] I mean, I know you wake up early and all [22:59] mwhudson: The production loggerhead branch seems to have a pretty serious memory leak. [23:00] mwhudson: My process was 1.1GB after a few weeks of running. [23:04] hi. i've got an Interesting Situation [23:05] i have two branches and I want to merge one into the working copy of the other [23:05] but I don't want to merge the two branches together. [23:05] is this even reasonable to think about? [23:06] dash: That's what merge does. [23:09] wooooo [23:09] dash: merge followed by revert --forget-merges is likley what you want [23:13] lifeless: --forget-merges was indeed the thing I was looking for. [23:13] basically, i've got an Eclipse workspace for a java project... and I want to merge in some changes for deployment on another server, in order to make an archive in eclipse [23:18] jam-laptop: if you have time, a review of my just-mailed patch would be cool. [23:20] huh. is --forget-merges in 0.90? [23:22] lifeless, ~3m30s for the initial commit (and recommit) using -packs, ~20m (15m user time) for the first incremental commit. [23:23] dash: no its not [23:23] nDuff: thats rather bad isn't it. I've never seen incremental go *up* [23:23] (over initial) [23:24] assuming the incremental is a small relative change, of course. [23:24] dang. and I thought I was doing so good, running gutsy [23:24] certainly I would expect it to go up if the basic diffs are drastically different [23:24] (every file doubles in size) [23:25] lifeless: it's a pretty significant set of changes. [23:25] nDuff: what revno of the packs branch are you using ? [23:25] * nDuff goes to gather some stats on that. [23:25] nDuff: have you done a 'make' tobuild the C extensions? [23:25] there was a bad list-copy on text construction [23:26] lifeless: r2851 [23:26] nDuff: hmm, that has that fix [23:26] 2852 has some bzr.dev fixes, and the new incremental update to the working tree [23:26] nDuff: you have 100K files right ? [23:26] evening all [23:26] hi sabdfl [23:27] how's tricks in this part of the world? [23:27] good [23:27] just sent in a review request for a 30% saving on 'bzr commit FOO' [23:27] in addition to prior numbers, or a new win? [23:27] sabdfl: incremental on top of the existing [23:27] measured with packs naturally [23:28] lifeless: in the initial tree, about 100,000 files. after the first update, 127000 files in 25000 directories. [23:28] so 15seconds to 10 seconds [23:28] nDuff: ok, for this I expect 2852 will help some [23:28] nDuff: but not at the scale you're seeing, something else is wrong there - jam-laptop's question about doing 'make' is a good one [23:29] 2852 is just reannotating back to knit format for you to pull [23:34] time for a new lsprof run with this update_basis code in place [23:34] jam-laptop: argh. did that on the tarball snapshot, but didn't have the C extension built for the checkout; that makes a dramatic difference on the initial incremental commit, at least if recommit times are representative. [23:34] nDuff: recommit is representative [23:34] that is using python to do "diff" versus "C" [23:34] which on my testing can be [23:34] 100ms [23:34] versus 5ms [23:34] jam-laptop: good call there :) [23:35] lifeless: is there a big queue of bits to land on bzr.dev ? [23:35] jam was just commenting yesterday that there is [23:36] theres about 15 branches the authors think are finished in flight at the moment [23:36] yikes. hope it can all land before 0.92, before all-hands [23:36] well, our test suite finishes a bit faster than LP's :) [23:37] haha [23:37] harsh [23:38] yeah, 15 branches would be 15 hours of PQM happiness in LP [23:38] just 15? [23:38] I thought the LP suite was closer to 2 hrs [23:38] we got the test suite down to 59 minutes at present [23:38] At least it felt that long when we used to be waiting behind it for bazaar merges [23:38] I'm glad to hear that you've improved it [23:39] there's still plenty of room for improvement [23:39] nDuff: pull now [23:39] r2852, in the pack-repository.knits branch [23:40] nDuff: I'll expect this to cut about 10 seconds off your incremental commits [23:40] lifeless: bzr: ERROR: Cannot lock LockDir(http://people.ubuntu.com/%7Erobertc/pack-repository.knits/.bzr/repository/lock): Transport operation not possible: http does not support mkdir() [23:40] ...doing an update [23:40] nDuff: oh, if you did 'checkout' just do 'bzr update' ;) [23:40] ...yes, that is doing 'bzr-packs update' [23:40] oh! [23:41] * nDuff unbinds and does a pull [23:41] I swear this has regressed. [23:41] but poolie spent time looking at it and thought it hadn't. [23:42] don't suppose you could grab the backtrace from your ~/.bzr.log and file a bug with it ? [23:42] can do. [23:43] you don't need to rebuild the C extensions after pulling [23:43] they haven't altered [23:43] lifeless: new bug, or look for a previous instance to this to reopen? [23:43] uhm, new please [23:43] I'll worry about joining if it is in fact a dupe [23:45] ok, the callgraph is starting to look tolerable for commit [23:45] populate_from_inventory is now up to 70% time [23:46] jam-laptop: serialisation of the new inventory is now up to 12% of commit, claimeth lsprof [23:46] and get_inventory is 8% + 5% [23:49] lifeless: https://bugs.launchpad.net/bzr/+bug/156462 [23:49] Launchpad bug 156462 in bzr ""cannot lock LockDir" on update over http" [Undecided,New] [23:51] thanks [23:51] so whats your incremental commit time now ? [23:53] what I'm most impressed with right now is the time to do a status with a dirty cache. don't know if it was the C extension or the update, but that's dropped *dramatically*. haven't gotten to incremental commit yet... [23:55] (I'm doing the initial add and commit in one tree then moving the .bzr directory to a different tree with the updates already applied; consequently, the cache is always dirty at that point) [23:55] reading the .bzr/checkout/dirstate file has a C extension [23:55] but I wouldn't think it would effect the dirty or not... [23:56] oh, depending on the update, it might include Martin's "sha_file_by_name" fi [23:56] fix [23:56] New bug: #156462 in bzr ""cannot lock LockDir" on update over http" [Undecided,New] https://launchpad.net/bugs/156462 [23:56] Which for a 55k tree changed it from 8s to 6s [23:58] lsprof I hate thee [23:58] brb [23:58] jam-laptop: so - can you review that patch ? [23:58] lifeless: currently in review [23:59] cool thanks