[00:01] igc: I'm popping up to the doctor [00:01] lifeless: ok [00:01] igc: I'm back on bzr now; if you need me in the next few hours you can SMS me [00:02] thanks === jfroy|work_ is now known as jfroy|work [01:49] hi [01:49] i'm considering writing a hook for my Bazaar server [01:50] i need to know if this will work [01:51] i know a guy who is willing to help me out on one of my projects, but i don't know if i can trust him because he specifically said he would replace all tab characters will 4 spaces and commit with that...... is there a way i could make a server hook that swaps all 4-space tabs with tab characters when he pushes? [01:51] right now,i really can't trust him [01:51] or at least deny his client from pushing if he uses 4-space tabs (that way at least he can't cause damage) [01:55] Well. That would require rewriting all the revs, so he'd have to pull --overwrite afterward... [01:56] is there a pre-push hook on the server? [01:56] But if you can't trust him, why the putz would you let him push in the first place? [01:56] that would allow me to end the push [01:56] fullermd: i have no idea... i don't think i will [01:56] i just want to know if i can just incase i ever do give him write access [02:46] * igc back in a few hours [02:48] is there an easy way to check what branch another branch was branch off? [02:51] meoblast001: there is a pre change hook yes [02:51] see bzr help hooks [02:51] lifeless: can i use that to completely kill the push? [02:52] that is why I mentioned it [02:53] ah, ok :) [03:38] lifeless: you're both a Bazaar and Launchpad dev right? [03:40] roughly [03:40] I've written code for both [03:40] not really active on lp codebase at the moment though [03:41] the code viewer in LP, that browses code in Bazaar [03:41] i'm thinking about scripting up something small like that in PHP for my server, so people can browse my sources [03:41] does Bazaar have utilities for getting file sources at certain revisions? [03:42] yes [03:42] bzr cat [03:42] ah, ok thanks :) [03:42] bzr xmloutput might have something [03:42] but personally, I'd suggest you just install loggerhead [03:42] what does loggerhead do? [03:44] meoblast001: loggerhead == "the code viewer in LP, that browses code in Bazaar" [03:44] ah, is it a Python/Django script? [03:46] django? not that I'm aware of. python yes. https://edge.launchpad.net/loggerhead/ [03:47] ah, ok [03:47] spm: does that use bzr cat? [03:47] or is it a Bazaar plugin? [03:48] meoblast001: perhas if we back up a bit and you ask what you're trying to get to? else we're just playing 1001 questions here :-) [03:49] well, does Loggerhead use bzr cat, or does it actually consist of a Bazaar plugin to make it more efficient? [04:09] meoblast001: its a bzr plugin [04:09] ah, ok [04:09] meoblast001: and standalone WSGI app [04:09] using python-paste for serving [04:09] I think django can speak wsgi too [04:09] what advantage does that provide over using a pipe in PHP? [04:14] its already written and feature complete? [04:14] it uses bzr's library, so can keep state in memory, making it ~ 1000 times faster [04:27] hey there, im a little confused about what bazaar is capable of doing [04:27] btu for our project, we're basically looking for SVN with forks [04:28] people can create their own fork of the main repo, and we (administrators) can cherry pick and merge specific changes from remote forks [04:34] Talidan: bzr can do that (among other workflows) [04:35] Talidan: http://wiki.bazaar-vcs.org/Workflows might help you. [04:41] thanks RAOF, i'll check it out [05:46] I need merge revno 519, 521, and 522. Is there some way I can do it in a single merge? [06:26] Howdy, all [06:26] hi [06:26] Any news on the nested-trees blueprint on launchpad? Look like it blocks support for implementing svn:externals in bzr-svn [06:26] https://blueprints.launchpad.net/bzr/+spec/nested-tree-support [06:27] Branch hasn't been updated in quite a while [06:27] no news [06:27] its essentially stalled [06:27] 2 years worth :( [06:27] there are other projects like scmproj and bzr-externals that may interest you [06:28] I was looking at bzr-externals [06:28] you've been able to do svn-externals like stuff for about 5 years now, using config-manager [06:28] would be cool if bzr-svn would use one of them to handle the mapping [06:28] true [06:29] the main thing is that noone is scratching the itch in the core [06:30] Its a blocker for using bzr against "big" SVN projects, which use externals a bunch [06:30] sure [06:30] you know how open source goes though :) [06:30] So far, the only one I have stumbled over, using bzr 2.1 and bzr-svn 1.0.2 [06:30] yup [06:45] Hi [06:48] woo: Running [ 1% 3 test(s) ] Current test: bzrlib.tests.blackbox.test_add.TestAdd.test_add_control_dir(pre-views) [06:48] I'm about to select bazaar , however there is a feature i pretty much need which it seems that bazaar does not support. [06:49] Do you the status of partical checkout/branch , i know there is a roadmap but don't know when it will be released [06:49] partial in what sense ? [06:49] like subversion update --depth [06:50] I don't know what that deos. [06:50] s/deos/does/ [06:50] When i want to checkout a large project in order to work only on an specific folder. i don't want to download the whole thing [06:50] Only that folder. [06:51] does it make sense know? [06:51] there is a feature called 'views' that avoids putting the content for other folders on disk. [06:51] However, bzr still downloads the same backend data for the repository content. [06:53] i'm aware of that, but i couldn't figure out how to use it with branch sub command [06:54] there nothing mentioned in documentation or man pages [06:55] probably needs a patch to let you set the view when you branch [06:56] Yeah, that's the problem. what do suggest me to do? [06:57] see if there is a bug open [06:57] if there isn't, file one [06:57] if you'd like, I can then mentor you on making the code change [07:00] Than lifeless i appreciate it. So you're saying that it's implemented ? [07:01] no [07:01] I'm saying that it needs to be implemented [07:01] and we need a bug open to track it [07:02] and its likely small, so if you want to do it, we can help you do it. [07:03] spm: http://pqm.bazaar-vcs.org/ - check it out :) - thanks [07:04] wooo! [07:04] the line 'Running [ 19% 4408 test(s) ] Current test: bzrlib ... [07:04] is the UI benefit here [07:05] failures will include a subunit trace if we've done it right [07:05] I'm going to send a failing merge after this [07:06] lifeless: I see [07:06] lifeless: i'll search the issue and come back. tx [07:30] jelmer: ping [07:32] lifeless: i submitted a new issue.https://bugs.launchpad.net/bzr/+bug/556236 [07:32] Ubuntu bug 556236 in bzr "branch --view feature is missing" [Undecided,New] [09:59] vila: see the shiny shiny pqm progress bar ? [10:02] lifeless: yup :) === radoe_ is now known as radoe [10:35] * igc dinner [11:11] Hi, just wondering if someone could help me please? I'm getting "Unable to find old fileid for..." errors when using bzr with an SVN repository in work. Any ideas why I would be getting this problem or what the problem means? [11:14] can you pastebin full traceback? [11:14] have you changed anything on your svn server? [11:14] maybe root id f svn server? [11:14] maybe root id of svn server? [11:15] well the repository recently went from 1.5 to 1.6 format but I was having the same problem before the migration [11:16] root id? sorry bit of a newbie to the finer workings of vcs :) [11:23] I've put a stack trace on pastebin, here http://pastebin.com/4g9Xz0ez [11:23] hopefully I've done that right :) [11:24] I changed some of the path names, filenames, etc as like I said it's an svn repo in work [11:28] you'll see in the traceback I'm doing an update, but I get the same error doing a checkout (lightweight) of a different path as well (albeit due to a different file). [11:36] mobby: it looks like the bug in bzr-svn [11:37] our bzr-svn wizard is not here right now [11:37] can you file a bug against bzr-svn on LP, please? [11:38] mobby: also, can you say how you get the working copy? with `svn co` or some bzr command? [11:41] Ok I'll raise a bug. I use "bzr checkout --lightweight http://....." I can't quite remember how I got hold of the one I'm doing the "update" in. I think it was using "bzr checkout ...." (I'll put this info in the bug) [11:41] thanks for your help [11:42] mobby: you can try to re-create working copy from the scratch in some empty folder [11:43] mobby: also there is cache of svn metadata in your bazaar config directory. maybe you need just delete it [11:43] oh ok, I'll try deleting that... [11:43] though this is just stab in the dark [11:53] I'm trying a new checkout and if that fails I'll try deleting the metadata and try again. Fingers crossed as I've been wanting to use bzr for a while now with our repo but keep hitting brick walls, usually due to our svn server which bzr isn't liking. [11:53] small issue with bazaar & merges: a colleague merged a branch in our trunk, but it turned out that he'd incorrectly resolved conflicts/missed some and hadn't tested the merge correctly. Except he still pushed it. So we decided not to uncommit (because it's ugly) and we reverted the merge commit instead (with merge -r{commit-revision}..{revision-before-commit}). [11:53] Broken code gone, but now when I try to rebase the old branch into the reverted trunk, whatever options I provide, it tells me the trunk is a parent of my branch (technically true) and just performs a pull [11:53] is there a way out of that? [11:54] s/into the/onto the/ [11:55] Other than hacking the bzr-rewrite source to remove that check, I can't think of one [11:56] dammit [11:56] mmm or maybe I can try a rebase on the revision right before the merge, it should change all revision ids, and then I rebase again to the current tip [12:00] bialix: Is that metadata the "subversion.conf" file or are there other cache files? [12:00] masklinn: that revision is already present in the parent branch, that's why it's trying a pull [12:01] jelmer: yeah I understand why it behaves as it does and why you'd usually want that, but that doesn't help me much as I'm specifically in a case where that's not what I want [12:09] masklinn: I have commented out this check in the past and things have worked fine [12:09] maxb, ok, I'll try to do it in to steps first, and if that doesn't work I'll do the commenting thing [12:10] s/to/two/ [12:13] masklinn: please make sure there is a bug erport for what you need [12:13] lifeless: are you sure? it's a fairly specific (and I'd hope rare) need [12:14] and I'm not sure whether it'd be a bug in bazaar (bug: impossible to revert a "merge" revision, merged commits are still considered being in the history) or rebase (wishlist: be able to force rebase even when rebased revisions are already in target branch) [12:15] mobby: ask jelmer [12:15] bialix: ok thanks :) [12:16] jelmer: Hi, before you were here I asked the following question. I wonder if you could help me please? - I'm getting "Unable to find old fileid for..." errors when using bzr with an SVN repository in work. Any ideas why I would be getting this problem or what the problem means? [13:04] mobby: btw, are you using bzr.exe? from standalone windows installer? [13:12] bialix: yeah it's the standalone windows install [13:13] mobby: I'm not sure which libsvn version there is bundled. in the past it was 1.5 [13:13] mobby: hi [13:13] jelmer: hi [13:13] so it's not really possible to work with 1.6 server [13:13] maybe it's not true anymore [13:13] mobby: Can you pastebin the error? [13:13] http://pastebin.com/4g9Xz0ez [13:15] bialix: from the .bzr.log the svn version is 1.6.6 in bzr-svn. (I think) [13:15] mobby: please file a bug, this is almost certainly a bzr-svn issue [13:15] mobby: ok, thx for the info [13:16] jelmer, bialix: ok will do. I've cleared the cache and I'm doing a new checkout, so far it is going on longer than before so the fix for the bug might be to clear the cache. I'll let you know how it goes. [13:16] s/fix/workaround/ [13:18] I don't think clearing the cache will help, unless you've been changing existing revisions in your repository [13:24] jelmer: Not sure, I know our repo was upgraded from 1.5 format to 1.6 recently but I *think* I was having the same problem before that change. [13:25] masklinn: a rebase bug [13:25] there are bugs already for cherrypick merge [13:26] jelmer: hmm, after clearing the cache and doing a clean checkout using bzr I got a different error - bzr: ERROR: subvertpy.SubversionException: ("REPORT of '/PATHXXX/!svn/bc/601278': 200 OK (http://XXX)", 175002 ) [13:26] jelmer: I'll raise the other bug now anyway [13:31] jelmer: Bug raised here - https://bugs.launchpad.net/bzr-svn/+bug/556451. Hopefully it has the information you require. [13:31] Ubuntu bug 556451 in bzr-svn "Unable to find old fileid error on checkout/update" [Undecided,New] [13:33] mobby: thanks [13:35] jelmer: no problem :) [13:43] Hi all [13:44] hi Lo-lan-do [13:45] jelmer: Question about bzr-svn… apparently branches stored in /branches/dir/subdir in the SVN repo don't get recognised by "bzr branches", is that known? [13:45] I'd like to tell FusionForge hackers to feel free to host their branches in /branches/people/$login/$feature [13:46] (Or even without the /branches component, ie straight in /people/$login/$feature) [13:46] Am I likely to run into problems? [13:47] Lo-lan-do: where /branches is inside of a Subversion repository? [13:47] Yes [13:48] Lo-lan-do: You won't run into problems trying to clone those branches later, but they e.g. won't be imported as separate branches by "bzr svn-import" [13:49] Lo-lan-do: This is all because Subversion doesn't really have a notion of what a branch is [13:49] I see. Yes, I know of that limitation, but I guess I can live with that. [14:02] morning all [14:03] So vila, about history-db, you mentioned there were bits you didn't understand, care to elaborate? [14:03] jam: So, about Caching 'dotted_revnos' and merge_sort, first, what are you caching exactly for a given revision ? [14:04] the same revision (in the worst case) can have a different revno in each branch [14:05] vila: so yes, "worst case" the cache would be N revs * M branches, where M could get close to N [14:05] however, I posted some results for both mysql and bzr [14:05] yes, [14:05] and importing the whole history, with each rev as a different "branch" [14:05] that's the part I don't get [14:06] take the history of bzr.dev [14:06] walk all revisions [14:06] you mean a branch the branch is the tip ? Something else ? [14:06] consider this revision as the tip [14:06] import and cache [14:06] ok [14:06] I optimize it slightly [14:06] by walking backwards through history [14:06] so I'll often get shared mainlines [14:06] that doesn't give me the revno when this branch is merged though [14:06] and only import 1 of a series [14:07] vila: so #1, import the merge_sorted graph [14:07] splitting it by mainline revno [14:07] hi all, a tailor user asked me about this problem: http://codepad.org/ru75we4F that happens on a big CVS repo, both with bzr 2.0.1 and with 2.1.1. Anybody knows if there is some kind of "finalization" I'm missing, when dealing with open_workingtree() objects? [14:07] which *does* tell you what revisions were merged into a given mainline rev [14:08] it doesn't tell you the info for that merged revision as the tip, though [14:08] that comes later. [14:08] vila: does that help? [14:09] Right, so that's an intermediate result, [14:09] and you use that to speed merge_sort by importing the "bottom" of the graph, right ? [14:10] lelit: looking at "stat_and_sha1" the file_obj = ... is shortly followed by a "try/finally:file_obj.close()" [14:10] So I don't think it would be leaking *those* file handles [14:10] but it may be leaking elsewhere? [14:11] vila: so... I *think* I haven't finished implementing "speed merge_sort" yet [14:11] That's part of what I wanted to discuss, to see if my thoughts there made sense. [14:11] *Right now* the current code is: [14:12] for a given branch (bzr.dev): KG.merge_sort(), import everything [14:12] jam: the tailor side is doing this: http://progetti.arstecnica.it/tailor/browser/vcpx/repository/bzr.py#L365 [14:12] jam: also, in your mail for mysql-6.0 --expand-all, you say: 32m import time [14:12] with --expand-all [14:12] 32 minutes ? [14:12] I then walk backwards from tip [14:12] hi jam, vila [14:12] morning bialix [14:12] hey bialix [14:12] morning jam, there is afternoon :-P [14:13] and if a given revision isn't already imported as a 'tip' (as identiified by tip_revision = merge_revision), then I do another KG.merge_sort(new_tip) and cache the result [14:13] Without --expand-all, mysql imports in about 11s [14:13] With --expand-all, it finds ~17k unique branches, and takes 32 minutes, yes [14:13] jam, after reading your cache db mail, I've suddenly realized we need for qlog only dotted revno of the tip of merged branch [14:13] I'm not sure what is KnownGraph stuff you and Gary merged recently [14:14] but 17k * 11s >> 32 minutes [14:14] bialix: not really related to what I'm doing now, I suppose tangentially [14:14] haaa, imports is reading all the graph *today* [14:14] vila: so you have to read the whole graph 1 time, but after that we can do better [14:15] jam: do we have any way to say "this revision is mainline", or this is also should be cached? [14:15] sorry to cross you and vila [14:15] bialix: its ok. for an arbitrary revision? [14:15] jam: yes [14:16] jam: when you traverse merged branch you need to know where to stop, when you come back to mainline. is it correct? [14:16] You can call Branch.revision_id_to_revno [14:16] and if it raises an exception, the revision isn't on the mainline [14:16] jam: I have trouble with the numbers you mention as I can't easily say which relates to our existing code and which depend on your wip [14:17] bialix: there is quite a bit more too it, as some of the intermediate 'merged' revisions may have been merged earlier [14:17] http://paste.ubuntu.com/410035/ [14:17] bialix: with time going up [14:17] you need to know that C is merged into E when you go to expand F [14:18] because you don't display it [14:18] jam: I see [14:18] http://paste.ubuntu.com/410036/ [14:18] merge_sort is how we determine what revisions go where in the graph [14:19] and then we put dotted_revnos from that fairly easily [14:19] actually qlog expand entire merged branch, which sometimes is not the best thing. but I see what you mean [14:19] bialix: sure, but even then you need to know that C was merged [14:19] somewhere [14:19] and to determine that [14:19] you currently have to walk backwards from all mainline revs [14:19] in *case* it was merged there [14:19] vila: so, the *counts* should be correct [14:19] jam: I think loading the whole graph for a mysql branch < 2s, not 11s [14:20] jam: ok [14:20] vila: importing the whole graph into the database is 11s [14:21] "bzr walk-ancestry --method=bzr" was 2s [14:21] --method=bzr-kg was 1.4s [14:22] --method=db-dotted was 0.087s [14:22] jelmer: Switching to /branches/$login_$branchname, I seem to run into "Operation denied because it would change the mainline history." when committing a merge, which I don't understand since I'm doing no such thing. [14:22] vila: so the time to get data *out* of the db should be accurate, and the counts of objects *in* the db should be accurate for future reference [14:22] (subject to tuning, etc :) [14:22] the time to put data *in* the database is still being worked on [14:23] ok, clearer, loading the db is not the most relevant, it should occur only one and then incrementally [14:23] vila: so I think loading the db taking 11s the first time is probably going to stay [14:23] Lo-lan-do: you're not pushing merges? [14:23] hmm, may be not, well it depends, your focus is loggerhead only/first/dunno ? [14:24] Loading an arbitrary real-world branch (today) requires grabbing get_known_graph().merge_sort() [14:24] which has a similar ~1s overhead [14:24] jelmer: I'm trying to commit a merge in a branch bound to SVN, so I probably am. [14:24] but then loading the unmerged data into the db is quite fast [14:24] --expand-all is sort of the ultimate "bring in as many branches as I can possibly find" [14:24] meant to stress-test the system [14:24] though you may want to run that in the future [14:25] In the import code *today* I believe the bottleneck is calling KG.merge_sort() for each one [14:25] jelmer: svn cp $svnrepo/trunk $svnrepo/branches/lolando_test, then a commit in a bzr branch bound to svnrepo/branches/lolando_test, then a merge of that into a checkout of $svnrepo/trunk, then commit. [14:25] Once I finish what I'm looking at now [14:25] both should be possible as an 'incremental update', and then you should be sub-second for all imports after the fisrt one [14:26] (note that right now you are at around 44ms for --expand-all per branch, and I think 40ms of that is merge_sort) [14:26] mysql is around 100ms per expanded-branch [14:26] vila: do those numbers make sense to you? [14:27] jam: that sounds sensible, my first interpretation is that you're demonstrating that the IOs are killing us, so batching them is good :) [14:28] jam: the db size is still a problem though, may be not for loggerhead... well, at least lp should make it possible to have one by project [14:29] vila: well, I was pretty surprised to see that "walk-ancestry" using the database was still 300ms or so, until you start walking 100 mainlines at a time, and it suddenly drops by an order of magnintude [14:29] jam: your dn can be shared across all branches of project right ? [14:29] vila: in theory it can be shared across all branches of all projects [14:29] though like shared repos, you don't benefit as much [14:31] jam: the numbers for walk_ancestry are for iterating the whole ancestry ? [14:31] vila: yes [14:31] 80ms to get the *entire* ancestry [14:31] 30k revs [14:32] or 68k revs for mysql [14:32] though for the db numbers, that isn't always in 'revision_ids' [14:32] you would have to probe one more table to get back to raw revision_ids [14:32] jelmer: http://whiteboard.debian.net/d6d21f.wb has the log [14:40] vila: so I'd like to discuss some of the algorithms for generating merge_sort, and see if they make sense to you === kfogel is now known as emacs === emacs is now known as kfogel [14:43] jam: sure [14:45] so I have some updated docs, and a start of an implementation in lp:~jameinel/+junk/bzr-history-db [14:45] (currently on rev 45) [14:45] there are a few ideas in there, I didn't really get it polished [14:46] but the general idea is to figure out what revisions need to be adedd [14:46] added [14:46] aeddd [14:46] (to the dotted_revno cache) [14:46] and I thought of some interesting uses of gdfo and parent => child mappings [14:48] the idea is that if all known children of a revision can be accounted for, then you can know that a rev was/wasn't merged [14:48] rather than having to walk in pure gdfo order [14:49] "accounted for" ? [14:49] oh, and how do you handle ghosts ? [14:49] http://paste.ubuntu.com/410050/ [14:49] time going upwards for you [14:50] sorry, missed a step, just a sec [14:51] well, maybe I can work with that [14:51] the idea is that X : B could be very long [14:51] which would make E have a high gdfo [14:51] however, thinking about it, it would still only be proportional to the diffs in the graph [14:51] so that doesn't quite fit my case yet [14:53] vila: at the moment ghosts are just treated as roots [14:53] I plan on accounting for them by putting them in a 'ghosts' table [14:53] since they are rare, I don't want to add a column on the 'revision' table for all the ones that aren't ghosts [14:54] so one possiibility: http://paste.ubuntu.com/410052/ [14:54] In this case, B is from a very old revision, so X has a few revs to it, but can have a much lower gdfo than C [14:55] the idea is that the only known child of X is D [14:55] so you know it wasn't merged before C [14:55] and the only known child of parent_of(X) is X [14:55] so again, you know it wasn't merged before [14:55] which continues until you get to B [14:55] at which point, you have to see if B was merged before, [14:55] so you start walking the mainline [14:56] however, I do wonder how the design works with: http://paste.ubuntu.com/410056/ [14:57] (no merge) [14:57] It will need to walk all the way back to A, and will load the dotted-revno info inbetween [14:58] may be unavoidable, but I think with known_children you could instead just walk the mainline revs, and not their merged revs [14:59] your db provide the children ? Or you build that as you walk the graph ? [15:00] vila: the parent table intrinsically provides children [15:00] SELECT child FROM parent WHERE parent =? [15:00] vs [15:01] SELECT parent FROM parent WHERE child = ? [15:01] I believe I have 2 indexes on the table [15:01] Lo-lan-do: it wants to copy from one of the merged branches I suspect [15:01] Lo-lan-do: and keep that as mainline [15:01] Lo-lan-do: please file a bug [15:02] jam: well, then you have even more shortcuts than with gdfo :) [15:04] jelmer: I've tried to do a svn commit in trunk just now, and now I can't bzr update the bound branch :-/ [15:04] AssertionError: Tried registering as parent while None already was parent for [15:04] vila: right [15:13] jam: you're still loading the whole graph anyway right ? (by whole I mean the revisions reachable from tip) [15:16] vila: the goal is to not do so [15:16] I believe I can compute merge_sort by only loading "enough" revisions [15:16] which is, the last merge sorted parent [15:16] of all newly merged revisions [15:16] and possibly a bit more context to get the numbers right [15:16] I still haven't sorted all that out, but I think I have a feel for it [15:24] jam: the thing I like with your approach is that it makes it easier to try various new tricks, some of them could be backported with a bit of luck [15:24] jam: can we query the bzr indices from a starting key for batch of keys ? [15:27] vila: not trivially [15:27] though that is what _known_graph_ancestry tries to do [15:32] sorry for the interruption, but did cherrypicking ever cause a pending merge? I was surprised when it didn't just now [15:32] cherrypicking never does [15:33] It would be illogical for it to do so [15:33] IslandUsurper: if you give a "cherrypick" range that actually includes a full ancestry, then it will record a pending merge [15:33] if it is 'accidentally' not actually a cherrypick [15:33] yeah, that makes sense [15:39] vila: I think I've found a couple holes in my new design [15:39] I pushed up a test case in rev 46 that might show it [15:39] specifically, when computing the second value in the dotted revno scheme, you pretty much *have* to load all children of the first revno [15:40] because there can be shortcuts that bump the 'branch' number, that are hidden by more recent merges [15:40] :( [15:40] vila: what were you saying about changing how we number again ? :) [15:42] jam: hehe, don't resign too fast :) [15:43] jam: but yeah, since there is not much added value in our actual scheme, changing for one that would give the merged-in mainline revno, can't be a loss [15:43] so one option is still "load all the ancestry between now and the parent you want to number from" [15:46] jam: but if merge_sort is only 40ms for a complete graph, the problem to address is really to load the graph (or the needed part) faster (and preferably with an acceptable space cost ;) no ? [15:47] vila: 40ms * 4k is minutes to expand all of bzr.dev. With incremental merge numbering we should be able to cut that by at least an order of magnitude [15:47] the bigger issue [15:47] is that I'd like to have a cache that gets updated with *commit* [15:47] 4k ? [15:47] Tiny bugreport: bzr should ignore SIGWINCH... I got a failure of "bzr pull" with "Interrupted system call" from resizing the window [15:47] (4k branches in the history of bzr) [15:47] so the 1-2s to load the ancestry of bzr.dev plays a much bigger deal there [15:48] which is why I wanted an incremental algo [15:48] I'm talking about a *single* merge_sort, isn't it 40ms *without* the loading part ? [15:48] right [15:48] right, gdfo is the easiest trick to manage at commit time :-) [15:48] gdfo = mac(gdfo) + 1 :) [15:48] s/mac/max/ === deryck is now known as deryck[lunch] [15:49] but the actual numbering scheme is the problem since it requires loading the whole graph [15:50] even if you manage to "load all the ancestry between now and the parent you want to number from", you still have the case where some daggy-fixed from a merged branch and merge again [15:50] s/some/someone/ [15:50] vila: true, though even daggy fixes don't go back to revno 1 [15:50] jam: indeed ! [15:50] the painful one in bzr's history is aaron's *long* lived integration branch [15:51] however, I do think those cases are the exception rather than the rule [15:51] and having an incremental algo will be good [15:51] just that i have to be aware of edge cases so I don't get them *wrong* [15:53] and the algo can be fast as long as it detects when it needs to fall back [15:53] so far, the 'detecting' part still looks expensive, which I'm unhappy about [15:53] jam: I think that even branches that are merged multiple times could be addressed too with a different revno scheme, they'll get dotted revnos with different first parts [15:55] jam: and again, using gdfo will help ensuring we number each revision based on its first landing (but gdfo may not be enough here, we may need a bit more help) [15:56] jam: so, concerning roots, you haven't addressed the filling part yet right ? [15:57] vila: ghosts/roots, no. But wipe and restart is a valid option. [15:57] The parent graph is still correct in the db [15:57] so you don't have to go back and touch branches again [15:58] vila: note that gary told me earlier that qlog makes use of the fact that a "branch" maintains its second number [15:58] well, first 2 numbers [15:58] jam: yeah, I think it's not such a good idea, but a good shortcut because we don't have cheaper ancestry checks [16:00] jam: and I would be delighted if he could magically find back the true branches in mysql mazes :) [16:04] jam: I just noticed: "If a revision X is in the ancestry of tip T, then the dotted revno for X will be the same for all descendents of T." [16:04] this is true *only* if T was a tip and is *still* on the mainline [16:05] jam: have you tried reusing your db on different mysql branches ? [16:21] vila: well I did --expand-all, which should be approx the same as "different branches" [16:22] vila: and for the "same for all descendents", sure, but that is how I'm caching [16:23] jam: ok, just wanted to make sure, there is really no mainline shared between mysql branches (apart from the initial import maybe) [16:26] vila: see my results [16:26] there is an average of only 68 rev divergence [16:26] and the rest is 'shared mainline' [16:26] vs bzr which has an average of 170 rev divergence [16:26] put another way, even though the ancestry of mysql-6.0 is 2x that of bzr, the final size of the db was == bzr after filling out the dotted revno table [16:27] jam: that's where I'm really surprised :) [16:27] vila: yep, I was too [16:27] I haven't probed enough to figure out why [16:28] jam: as in: a bit too much to trust these numbers wit... yeah :) [16:31] jam: what commands should I run in a given branch to get the same stats ? [16:33] vila: generally 'bzr create-history-db --db=filename -d branch --expand-all' [16:34] It *might* be broken right now, let me check [16:34] ah, it should be fine if you don't use --incremental [16:35] with --expand-all it should spit out some stats about how many revs were expanded, etc. And how many total entries you have [16:35] which is how I figured the numbers [16:35] you can also use "sqlite3 filename" and then do custom queries in there === mobby is now known as mobby_ [16:41] jam: random thought: at *merge* time it's cheap to record how "deep" the merged branch is (for external branches) or how many new revisions are merged from the other mainline === mobby_ is now known as mobby [16:42] jam: some hints may help devising a new incremental revno scheme [16:42] vila: I'm not really sure what you are proposing. [16:42] when merging, all branches are 1 deep [16:42] all *merged* branches are 1 deep [16:43] 'how many new revisions' is probably not all that easy [16:43] graph-difference vs just finding a common ancestor [16:43] jam: think merge-into and the consequences on the revnos [16:43] jam: sure, but you have the relevant graph already loaded anyway [16:43] at least most, probably [16:44] jam: so we are in the "40ms for merge_sort" area, hence cheap === deryck[lunch] is now known as deryck [16:55] Coworker came to bzr from git. He wanted to get rid of the last 3 revisions, so he did a 'bzr update -r -4'. Then did some work, and was surprised when he couldn't commit. [16:55] we fixed it with shelve and 3x 'bzr uncommit', but... [16:55] is there a better way to do that? [16:56] bzr pull? [16:56] yeah, that works [16:56] or bzr uncommit -r [16:57] Ok. What's the purpose of being able to do an update to a past revision, then? [16:57] seems weird if you can't commit to it. [16:59] Folks: there is a student applying for Google Summer of Code to work on the Tahoe-LAFS project and to integrate Tahoe-LAFS with a DVCS. [16:59] bzr is a candidate. [16:59] Would any bzr hackers be interested in mentoring or at least consulting? === IslandUsurper is now known as IslandUsurperAFK [17:05] NfNitLoop: you can if the branch is not bound I think [17:05] zooko: Yeah, np [17:06] it was not a bound branch. [17:07] 'bzr up -r -4' seemed to update the working copy and even the current revision, but when he tried to commit it said that he couldn't because the working copy was out of date. [17:07] even though it's a standalone branch. (Well, within a shared repo.) === salgado is now known as salgado-lunch === beuno is now known as beuno-lunch [17:30] I need merge revno 519, 521, and 522. Is there some way I can do it in a single merge? === salgado-lunch is now known as salgado [18:24] What is the english expression about a round thing into a squared other thing ? [18:25] Round peg, square hole? [18:25] (or vice versa, really.. I expect either way works) [18:25] yes ! But what is the sentence used ? [18:26] trying to fit a round peg in a square hole ? [18:26] Oh. something to the effect of "trying to fit.." yes. that === beuno-lunch is now known as beuno === IslandUsurperAFK is now known as IslandUsurper [18:45] vila: are you still around? [18:47] just to mention "square peg in a round hole" is the more common phrasing [18:47] http://www.google.com/search?q=round+peg+square+hole [18:47] (notice that I searched for round peg square hole and got the reverse) [18:47] anyway, I had another idea about the partial graph, in case vila comes back around. [18:48] jam: still there for a few minutes [18:48] http://paste.ubuntu.com/410151/ [18:48] vila: basic observation [18:48] jam: and another random idea: you can add gdfo into your db right ? [18:48] vila: gdfo is already in the db [18:48] and I'm using it now [18:49] basic observation: you don't need to load back to revno 0 [18:49] you need to load back to revno x.y.? that you are based on [18:49] That can potentially be a big difference [18:49] when finding a new branch number based on 1.5000.1, you don't need the data for anything 1.<5000 (maybe) [18:50] jam: you got it [18:51] jam: and then, if you had a bit more info in the ancestry graph you may even get there without a cache [18:51] jam: like: next merge point in my ancestry is at distance x [18:52] jam: which should help addressing the problem of the last part of the revno (otherwise you have to walk it until its origin to start at 1) (for various definitions of origin) [18:57] jam: but I'm still not sold about numbering in one way or the other (M: 5.1.1, L: 5.2.1, K: 5.1.2, etc) or (M: 5.1.2, K: 5.1.1) [19:03] Gah, I can never remember what should be merged! Do I merge a THIS and OTHER on to BASE? [19:05] vila: sure, I can see changing the numbering, but finding that while I need more-than-minimal but I need less-than-back to 1 makes it worth moving forward [19:05] I just wanted to make sure someone else agreed with my logic [19:10] balor, THIS is your working tree, OTHER came from the merged branch, and BASE is what they have in common. So for each chunk, ask which change is more important, the one you had or the one you're getting. [19:11] the tricky ones have some of each [19:12] So merge THIS and OTHER into each other [19:12] right [19:12] they're displayed together in the original filename [19:13] So what's the point of BASE? What information does it give me? [19:13] As I get common chunks in the same colour in Meld [19:14] if you diff BASE and either THIS or OTHER, you can see exactly what they changed [19:14] ah [19:14] thanks [20:16] jam: sure, sure, I'm very happy you work on *that* (reducing the cases requiring loading all the history) [20:24] jam: your logic ? [20:29] full discussion is in the text file in my lp:~jameinel/+junk/history-db branch. [20:30] basically, [20:30] for simple children, you just need to know there isn't another child [20:30] and then you can do X.Y.Z+1 [20:31] for a new sub-branch [20:31] you need to know if there is an X.Y+1 [20:31] to know that, you need to check all descendants merged into trunk from X.Y.1 [20:31] sorry [20:31] all merges into trunk since X.Y.1 was merged [20:32] however, you don't care about when X.Y-1.* was merged [20:32] lifeless: ^^ [20:33] maybe put another way [20:33] if you are walking backwards, you need to load X.Y.Z if it is a parent of yours, so that you can do any of X.Y+1.Z, or X.Y.Z+1 [20:33] to determine which [20:33] you need to load the children of X.Y.Z [20:33] if there are no other children, you can do X.Y.Z+1 and do no further loading [20:33] if there is one [20:34] then you need to load from X.Y.1 to determine any X.Y+1.1 that exist [20:34] (+N if you prefer) [20:34] The key is that you don't need to load from X [20:35] since anything landed before X.Y.1 was landed, will have have a second number < Y. [20:48] vila: can I purchase a review ? [20:48] jam: bzr+ssh://bazaar.launchpad.net/~jameinel/%2Bjunk/history-db/ - not a branch [20:49] lifeless: sorry bzr-history-db [20:49] lp:~jameinel/+junk/bzr-history-db [20:50] so, 'only the right hand side *children* of X can influence revision numbers X.*.*' ? [20:54] lifeless: I'm pretty sure only lh children [20:54] sorry [20:54] you were saying X [20:55] children of X with X in their LH history will be numbered based on it, yes [20:57] the way the data is stored in the dotted_revno table today does affect things [20:57] you need to know that a child of X is in the ancestry of the current branch [20:57] just being a child doesn't affect X.*.* for *this* branch [20:57] sorry, I should have said 'rh parents of X' [20:58] lifeless: rh parents of X have no affect [20:58] X is the ancestor of X.*.* [20:59] it is, specifically, a LH ancestor === ubott2 is now known as ubottu [21:21] jam: https://code.edge.launchpad.net/~lifeless/bzr/commit/+merge/22848 [21:22] jam: vila: https://bugs.edge.launchpad.net/bzr/+bug/530265 [21:22] do you guys have a preference between [21:22] *) a line to delete [21:22] Launchpad bug 530265 in bzr "Not modifying a suggested commit message commits anyway" [Medium,Confirmed] [21:23] *) a y/n prompt if the message is the unaltered template [21:23] lifeless: If we are being interactive enough to pop up an editor [21:23] I think it is ok to prompt for a "did you really want to commit an empty message" ? [21:24] lifeless: the mutter() seems fine [21:24] I'm trying to understand the revprops chacheng [21:24] change [21:24] oh [21:24] I had it in my commit branch; it just reduces some cruft [21:24] you are removing it from _commit, but adding it to the constructor? [21:24] shorter parameter list [21:24] its already in the constructor [21:25] its an internal handoff within the object [21:25] so just changing where it becomes an attribute, to spend less time passing it around [21:25] it would seem reasonable to do that to all the other ones as well then... [21:26] so... not a problem, but incomplete one way or another [21:26] yup [21:26] If I end up doing much more to commit I'll probably do the rest too [21:26] its a bit ugly as it stands [21:27] anyway :approve [21:28] thanks [21:28] re prompt/message [21:28] I guess I feel that its cleaner to only interrupt the user once [21:29] GUI's can check in the dialog [21:29] hmmm [21:32] jam: whats the current open bzr version on trunk ? [21:32] assign to milestone is showing about 15 things, rather than 3 [21:32] <- confused bear of small brain [21:33] lifeless: either 2.2.0 or 2.2.0b2 I believe [21:33] so commits in trunk will still be 2.2 ? [21:33] well, 2.2b2 [21:33] yes [21:33] kk [21:34] can we deactivate some of the others, do you think ? [21:47] From looking at it, they are all stuff in the future [21:47] not sure that we would disable them, but I suppose we could for now [21:47] unlikely to target them explicitly, but it can be nice to know when they are expected [21:47] (so not great in the bug UI, nice to have elsewhere) === salgado is now known as salgado-afk [22:22] Hi there, i was just reading http://wiki.bazaar-vcs.org/CherryPick [22:22] Cherrypicking is vital to our project, the idea is that other users can mirror our branch, make changes [22:23] and we can merge them if they're decent [22:23] i was just wondering what's meant by "Bazaar does not track cherrypicked revisions, although this feature is firmly on the wish list. " [22:23] ideally (currently im testing on launchpad) the person we merge from is credited with the change [22:24] I don't see why you need cherrypicking given your description of your idea. [22:27] well, we're thinking of migrating from GIT [22:27] which was hosted at github [22:27] there everyone (besides official mantainers) would open their own fork [22:27] right [22:27] andn push their changes to their own fork [22:28] and if we thought they were decent, we'd merge them over [22:28] my understanding is that it requires cherrypicking for bazaar [22:28] Bearing in mind their fork might have loads of crap we dont want [22:28] ah [22:28] we only select the commits that we actually want [22:29] so, you can use 'bzr merge -c rev' [22:29] which will do a cherrypick merge [22:29] i was just wondering what "does not track cherrypicked revisions" actually means [22:29] and commit --author "foo bar " [22:29] it means we don't track them [22:30] mind elaborating? [22:30] uhm [22:30] I'm not sure where the disconnect is [22:30] track could mean a whole lot of things [22:31] there is no record kept of the fact that a particular commit was a cherrypicked merge. [22:31] Right, okay thanks [22:31] you can do cherrypick merges easily [22:31] (merge -c) [22:31] and if you commit with --author they will be credited appropriately [22:32] (in bzr merges never create commit objects, you need to do a commit after the merge) [22:38] jelmer: ayt? [22:40] What windows GUI bzr client would you guys reccomend? [22:40] mwhudson: I am now [22:40] TortoiseSVN is the only Tortoise* i've had a good experience with, tortoisegit is a pile of rubbish, and tortoisehg is average [22:41] jelmer: have you tested the kernel import on staging since last week? [22:41] (with a new bzr-git & dulwich) [22:42] mwhudson: I don't have a way to update the dulwich/bzr-git on staging afaik :-) [22:43] jelmer: well, "caused to be tested" then :) [22:43] jelmer: you have the same mechanism as me! [22:44] jelmer: do you think it would be a useful test at this stage? [22:46] Talidan: I hear good things about bzr-explorer [22:46] Thanks, i'll give it a go [22:48] mwhudson: ah, asking a losa ? :-) [22:49] mwhudson: There's one bug I noticed that I'd like to get fixed first (unusual modes disappearing doesn't seem to be handled right) [22:49] jelmer: yeah [22:49] jelmer: ok [22:52] jelmer: any idea how long that'll be? [23:00] mwhudson: might be tonight, but there's a few other things I have to take care of first [23:00] jelmer: okidoke [23:00] * mwhudson continues applying the email shovel [23:18] i think it's an oversight to not have simple authentification given open source projects that dont need security but simplicity [23:20] or wiat, does bzr support user/pass? [23:20] i guess it's launchpad doesn't? [23:22] bzr supports user/pass [23:22] lp uses ssh and oauth [23:22] bzr doesn't support oauth yet [23:22] there is an interesting spec though [23:23] that google and others have done for oauth IMAP and SMTP [23:23] we could perhaps adapt that [23:24] lp doesn't maintain cleartext passwords though, so it wouldn't be easy to turn on plain ol password supoprt [23:28] lifeless: OAUTH over SASL, or using some different mechanism? [23:29] jelmer: I haven't read the spec for how they did it in IMAP yet