/srv/irclogs.ubuntu.com/2010/04/06/#bzr.txt

lifelessigc: I'm popping up to the doctor00:01
igclifeless: ok00:01
lifelessigc: I'm back on bzr now; if you need me in the next few hours you can SMS me00:01
igcthanks00:02
=== jfroy|work_ is now known as jfroy|work
meoblast001hi01:49
meoblast001i'm considering writing a hook for my Bazaar server01:49
meoblast001i need to know if this will work01:50
meoblast001i know a guy who is willing to help me out on one of my projects, but i don't know if i can trust him because he specifically said he would replace all tab characters will 4 spaces and commit with that...... is there a way i could make a server hook that swaps all 4-space tabs with tab characters when he pushes?01:51
meoblast001right now,i really can't trust him01:51
meoblast001or at least deny his client from pushing if he uses 4-space tabs (that way at least he can't cause damage)01:51
fullermdWell.  That would require rewriting all the revs, so he'd have to pull --overwrite afterward...01:55
meoblast001is there a pre-push hook on the server?01:56
fullermdBut if you can't trust him, why the putz would you let him push in the first place?01:56
meoblast001that would allow me to end the push01:56
meoblast001fullermd: i have no idea... i don't think i will01:56
meoblast001i just want to know if i can just incase i ever do give him write access01:56
* igc back in a few hours02:46
GungaDinis there an easy way to check what branch another branch was branch off?02:48
lifelessmeoblast001: there is a pre change hook yes02:51
lifelesssee bzr help hooks02:51
meoblast001lifeless: can i use that to completely kill the push?02:51
lifelessthat is why I mentioned it02:52
meoblast001ah, ok :)02:53
meoblast001lifeless: you're both a Bazaar and Launchpad dev right?03:38
lifelessroughly03:40
lifelessI've written code for both03:40
lifelessnot really active on lp codebase at the moment though03:40
meoblast001the code viewer in LP, that browses code in Bazaar03:41
meoblast001i'm thinking about scripting up something small like that in PHP for my server, so people can browse my sources03:41
meoblast001does Bazaar have utilities for getting file sources at certain revisions?03:41
lifelessyes03:42
lifelessbzr cat03:42
meoblast001ah, ok thanks :)03:42
lifelessbzr xmloutput might have something03:42
lifelessbut personally, I'd suggest you just install loggerhead03:42
meoblast001what does loggerhead do?03:42
spmmeoblast001: loggerhead == "the code viewer in LP, that browses code in Bazaar"03:44
meoblast001ah, is it a Python/Django script?03:44
spmdjango? not that I'm aware of. python yes. https://edge.launchpad.net/loggerhead/03:46
meoblast001ah, ok03:47
meoblast001spm: does that use bzr cat?03:47
meoblast001or is it a Bazaar plugin?03:47
spmmeoblast001: perhas if we back up a bit and you ask what you're trying to get to? else we're just playing 1001 questions here :-)03:48
meoblast001well, does Loggerhead use bzr cat, or does it actually consist of a Bazaar plugin to make it more efficient?03:49
lifelessmeoblast001: its a bzr plugin04:09
meoblast001ah, ok04:09
lifelessmeoblast001: and standalone WSGI app04:09
lifelessusing python-paste for serving04:09
lifelessI think django can speak wsgi too04:09
meoblast001what advantage does that provide over using a pipe in PHP?04:09
lifelessits already written and feature complete?04:14
lifelessit uses bzr's library, so can keep state in memory, making it ~ 1000 times faster04:14
Talidanhey there, im a little confused about what bazaar is capable of doing04:27
Talidanbtu for our project, we're basically looking for SVN with forks04:27
Talidanpeople can create their own fork of the main repo, and we (administrators) can cherry pick and merge specific changes from remote forks04:28
RAOFTalidan: bzr can do that (among other workflows)04:34
RAOFTalidan: http://wiki.bazaar-vcs.org/Workflows might help you.04:35
Talidanthanks RAOF, i'll check it out04:41
eydaimonI need merge revno 519, 521, and 522. Is there some way I can do it in a single merge?05:46
TresEquisHowdy, all06:26
lifelesshi06:26
TresEquisAny news on the nested-trees blueprint on launchpad?  Look like it blocks support for implementing svn:externals in bzr-svn06:26
TresEquishttps://blueprints.launchpad.net/bzr/+spec/nested-tree-support06:26
TresEquisBranch hasn't been updated in quite a while06:27
lifelessno news06:27
lifelessits essentially stalled06:27
TresEquis2 years worth :(06:27
lifelessthere are other projects like scmproj and bzr-externals that may interest you06:27
TresEquisI was looking at bzr-externals06:28
lifelessyou've been able to do svn-externals like stuff for about 5 years now, using config-manager06:28
TresEquiswould be cool if bzr-svn would use one of them to handle the mapping06:28
lifelesstrue06:28
lifelessthe main thing is that noone is scratching the itch in the core06:29
TresEquisIts a blocker for using bzr against "big" SVN projects, which use externals a bunch06:30
lifelesssure06:30
lifelessyou know how open source goes though :)06:30
TresEquisSo far, the only one I have stumbled over, using bzr 2.1 and bzr-svn 1.0.206:30
TresEquisyup06:30
sinasalekHi06:45
lifelesswoo: Running [ 1% 3 test(s) ] Current test: bzrlib.tests.blackbox.test_add.TestAdd.test_add_control_dir(pre-views)06:48
sinasalekI'm about to select bazaar , however there is a feature i pretty much need which it seems that bazaar does not support.06:48
sinasalekDo you the status of partical checkout/branch , i know there is a roadmap but don't know when it will be released06:49
lifelesspartial in what sense ?06:49
sinasaleklike subversion update --depth06:49
lifelessI don't know what that deos.06:50
lifelesss/deos/does/06:50
sinasalekWhen i want to checkout a large project in order to work only on an specific folder. i don't want to download the whole thing06:50
sinasalekOnly that folder.06:50
sinasalekdoes it make sense know?06:51
lifelessthere is a feature called 'views' that avoids putting the content for other folders on disk.06:51
lifelessHowever, bzr still downloads the same backend data for the repository content.06:51
sinasaleki'm aware of that, but i couldn't figure out how to use it with branch sub command06:53
sinasalekthere nothing mentioned in documentation or man pages06:54
lifelessprobably needs a patch to let you set the view when you branch06:55
sinasalekYeah, that's the problem. what do suggest me to do?06:56
lifelesssee if there is a bug open06:57
lifelessif there isn't, file one06:57
lifelessif you'd like, I can then mentor you on making the code change06:57
sinasalekThan lifeless i appreciate it. So you're saying that it's implemented ?07:00
lifelessno07:01
lifelessI'm saying that it needs to be implemented07:01
lifelessand we need a bug open to track it07:01
lifelessand its likely small, so if you want to do it, we can help you do it.07:02
lifelessspm: http://pqm.bazaar-vcs.org/ - check it out :) - thanks07:03
spmwooo!07:04
lifelessthe line 'Running [ 19% 4408 test(s) ] Current test: bzrlib ...07:04
lifelessis the UI benefit here07:04
lifelessfailures will include a subunit trace if we've done it right07:05
lifelessI'm going to send a failing merge after this07:05
sinasaleklifeless: I see07:06
sinasaleklifeless: i'll search the issue and come back. tx07:06
lifelessjelmer: ping07:30
sinasaleklifeless: i submitted a new issue.https://bugs.launchpad.net/bzr/+bug/55623607:32
ubottuUbuntu bug 556236 in bzr "branch --view feature is missing" [Undecided,New]07:32
lifelessvila: see the shiny shiny pqm progress bar ?09:59
vilalifeless: yup :)10:02
=== radoe_ is now known as radoe
* igc dinner10:35
mobbyHi, just wondering if someone could help me please? I'm getting "Unable to find old fileid for..." errors when using bzr with an SVN repository in work. Any ideas why I would be getting this problem or what the problem means?11:11
bialixcan you pastebin full traceback?11:14
bialixhave you changed anything on your svn server?11:14
bialixmaybe root id f svn server?11:14
bialixmaybe root id of svn server?11:14
mobbywell the repository recently went from 1.5 to 1.6 format but I was having the same problem before the migration11:15
mobbyroot id? sorry bit of a newbie to the finer workings of vcs :)11:16
mobbyI've put a stack trace on pastebin, here http://pastebin.com/4g9Xz0ez11:23
mobbyhopefully I've done that right :)11:23
mobbyI changed some of the path names, filenames, etc as like I said it's an svn repo in work11:24
mobbyyou'll see in the traceback I'm doing an update, but I get the same error doing a checkout (lightweight) of a different path as well (albeit due to a different file).11:28
bialixmobby: it looks like the bug in bzr-svn11:36
bialixour bzr-svn wizard is not here right now11:37
bialixcan you file a bug against bzr-svn on LP, please?11:37
bialixmobby: also, can you say how you get the working copy? with `svn co` or some bzr command?11:38
mobbyOk I'll raise a bug. I use "bzr checkout --lightweight http://....." I can't quite remember how I got hold of the one I'm doing the "update" in. I think it was using "bzr checkout ...." (I'll put this info in the bug)11:41
mobbythanks for your help11:41
bialixmobby: you can try to re-create working copy from the scratch in some empty folder11:42
bialixmobby: also there is cache of svn metadata in your bazaar config directory. maybe you need just delete it11:43
mobbyoh ok, I'll try deleting that...11:43
bialixthough this is just stab in the dark11:43
mobbyI'm trying a new checkout and if that fails I'll try deleting the metadata and try again. Fingers crossed as I've been wanting to use bzr for a while now with our repo but keep hitting brick walls, usually due to our svn server which bzr isn't liking.11:53
masklinnsmall issue with bazaar & merges: a colleague merged a branch in our trunk, but it turned out that he'd incorrectly resolved conflicts/missed some and hadn't tested the merge correctly. Except he still pushed it. So we decided not to uncommit (because it's ugly) and we reverted the merge commit instead (with merge -r{commit-revision}..{revision-before-commit}).11:53
masklinnBroken code gone, but now when I try to rebase the old branch into the reverted trunk, whatever options I provide, it tells me the trunk is a parent of my branch (technically true) and just performs a pull11:53
masklinnis there a way out of that?11:53
masklinns/into the/onto the/11:54
maxbOther than hacking the bzr-rewrite source to remove that check, I can't think of one11:55
masklinndammit11:56
masklinnmmm or maybe I can try a rebase on the revision right before the merge, it should change all revision ids, and then I rebase again to the current tip11:56
mobbybialix: Is that metadata the "subversion.conf" file or are there other cache files?12:00
jelmermasklinn: that revision is already present in the parent branch, that's why it's trying a pull12:00
masklinnjelmer: yeah I understand why it behaves as it does and why you'd usually want that, but that doesn't help me much as I'm specifically in a case where that's not what I want12:01
maxbmasklinn: I have commented out this check in the past and things have worked fine12:09
masklinnmaxb, ok, I'll try to do it in to steps first, and if that doesn't work I'll do the commenting thing12:09
masklinns/to/two/12:10
lifelessmasklinn: please make sure there is a bug erport for what you need12:13
masklinnlifeless: are you sure? it's a fairly specific (and I'd hope rare) need12:13
masklinnand I'm not sure whether it'd be a bug in bazaar (bug: impossible to revert a "merge" revision, merged commits are still considered being in the history) or rebase (wishlist: be able to force rebase even when rebased revisions are already in target branch)12:14
bialixmobby: ask jelmer12:15
mobbybialix: ok thanks :)12:15
mobbyjelmer: Hi, before you were here I asked the following question. I wonder if you could help me please? - I'm getting "Unable to find old fileid for..." errors when using bzr with an SVN repository in work. Any ideas why I would be getting this problem or what the problem means?12:16
bialixmobby: btw, are you using bzr.exe? from standalone windows installer?13:04
mobbybialix: yeah it's the standalone windows install13:12
bialixmobby: I'm not sure which libsvn version there is bundled. in the past it was 1.513:13
jelmermobby: hi13:13
mobbyjelmer: hi13:13
bialixso it's not really possible to work with 1.6 server13:13
bialixmaybe it's not true anymore13:13
jelmermobby: Can you pastebin the error?13:13
mobbyhttp://pastebin.com/4g9Xz0ez13:13
mobbybialix: from the .bzr.log the svn version is 1.6.6 in bzr-svn. (I think)13:15
jelmermobby: please file a bug, this is almost certainly a bzr-svn issue13:15
bialixmobby: ok, thx for the info13:15
mobbyjelmer, bialix: ok will do. I've cleared the cache and I'm doing a new checkout, so far it is going on longer than before so the fix for the bug might be to clear the cache. I'll let you know how it goes.13:16
mobbys/fix/workaround/13:16
jelmerI don't think clearing the cache will help, unless you've been changing existing revisions in your repository13:18
mobbyjelmer: Not sure, I know our repo was upgraded from 1.5 format to 1.6 recently but I *think* I was having the same problem before that change.13:24
lifelessmasklinn: a rebase bug13:25
lifelessthere are bugs already for cherrypick merge13:25
mobbyjelmer: hmm, after clearing the cache and doing a clean checkout using bzr I got a different error - bzr: ERROR: subvertpy.SubversionException: ("REPORT of '/PATHXXX/!svn/bc/601278': 200 OK (http://XXX)", 175002 )13:26
mobbyjelmer: I'll raise the other bug now anyway13:26
mobbyjelmer: Bug raised here - https://bugs.launchpad.net/bzr-svn/+bug/556451. Hopefully it has the information you require.13:31
ubottuUbuntu bug 556451 in bzr-svn "Unable to find old fileid error on checkout/update" [Undecided,New]13:31
jelmermobby: thanks13:33
mobbyjelmer: no problem :)13:35
Lo-lan-doHi all13:43
jelmerhi Lo-lan-do13:44
Lo-lan-dojelmer: Question about bzr-svn… apparently branches stored in /branches/dir/subdir in the SVN repo don't get recognised by "bzr branches", is that known?13:45
Lo-lan-doI'd like to tell FusionForge hackers to feel free to host their branches in /branches/people/$login/$feature13:45
Lo-lan-do(Or even without the /branches component, ie straight in /people/$login/$feature)13:46
Lo-lan-doAm I likely to run into problems?13:46
jelmerLo-lan-do: where /branches is inside of a Subversion repository?13:47
Lo-lan-doYes13:47
jelmerLo-lan-do: You won't run into problems trying to clone those branches later, but they e.g. won't be imported as separate branches by "bzr svn-import"13:48
jelmerLo-lan-do: This is all because Subversion doesn't really have a notion of what a branch is13:49
Lo-lan-doI see.  Yes, I know of that limitation, but I guess I can live with that.13:49
jammorning all14:02
jamSo vila, about history-db, you mentioned there were bits you didn't understand, care to elaborate?14:03
vilajam: So, about Caching 'dotted_revnos' and merge_sort, first, what are you caching exactly for a given revision ?14:03
vilathe same revision (in the worst case) can have a different revno in each branch14:04
jamvila: so yes, "worst case" the cache would be N revs * M branches, where M could get close to N14:05
jamhowever, I posted some results for both mysql and bzr14:05
vilayes,14:05
jamand importing the whole history, with each rev as a different "branch"14:05
vilathat's the part I don't get14:05
jamtake the history of bzr.dev14:06
jamwalk all revisions14:06
vilayou mean a branch the branch is the tip ? Something else ?14:06
jamconsider this revision as the tip14:06
jamimport and cache14:06
vilaok14:06
jamI optimize it slightly14:06
jamby walking backwards through history14:06
jamso I'll often get shared mainlines14:06
vilathat doesn't give me the revno when this branch is merged though14:06
jamand only import 1 of a series14:06
jamvila: so #1, import the merge_sorted graph14:07
jamsplitting it by mainline revno14:07
lelithi all, a tailor user asked me about this problem: http://codepad.org/ru75we4F that happens on a big CVS repo, both with bzr 2.0.1 and with 2.1.1. Anybody knows if there is some kind of "finalization" I'm missing, when dealing with open_workingtree() objects?14:07
jamwhich *does* tell you what revisions were merged into a given mainline rev14:07
jamit doesn't tell you the info for that merged revision as the tip, though14:08
jamthat comes later.14:08
jamvila: does that help?14:08
vilaRight, so that's an intermediate result,14:09
vilaand you use that to speed merge_sort by importing the "bottom" of the graph, right ?14:09
jamlelit: looking at "stat_and_sha1" the file_obj = ... is shortly followed by a "try/finally:file_obj.close()"14:10
jamSo I don't think it would be leaking *those* file handles14:10
jambut it may be leaking elsewhere?14:10
jamvila: so... I *think* I haven't finished implementing "speed merge_sort" yet14:11
jamThat's part of what I wanted to discuss, to see if my thoughts there made sense.14:11
jam*Right now* the current code is:14:11
jamfor a given branch (bzr.dev): KG.merge_sort(), import everything14:12
lelitjam: the tailor side is doing this: http://progetti.arstecnica.it/tailor/browser/vcpx/repository/bzr.py#L36514:12
vilajam: also, in your mail for mysql-6.0 --expand-all, you say: 32m import time14:12
jamwith --expand-all14:12
vila32 minutes ?14:12
jamI then walk backwards from tip14:12
bialixhi jam, vila14:12
jammorning bialix14:12
vilahey bialix14:12
bialixmorning jam, there is afternoon :-P14:12
jamand if a given revision isn't already imported as a 'tip' (as identiified by tip_revision = merge_revision), then I do another KG.merge_sort(new_tip) and cache the result14:13
jamWithout --expand-all, mysql imports in about 11s14:13
jamWith --expand-all, it finds ~17k unique branches, and takes 32 minutes, yes14:13
bialixjam, after reading your cache db mail, I've suddenly realized we need for qlog only dotted revno of the tip of merged branch14:13
bialixI'm not sure what is KnownGraph stuff you and Gary merged recently14:13
jambut 17k * 11s >> 32 minutes14:14
jambialix: not really related to what I'm doing now, I suppose tangentially14:14
vilahaaa, imports is reading all the graph *today*14:14
jamvila: so you have to read the whole graph 1 time, but after that we can do better14:14
bialixjam: do we have any way to say "this revision is mainline", or this is also should be cached?14:15
bialixsorry to cross you and vila14:15
jambialix: its ok. for an arbitrary revision?14:15
bialixjam: yes14:15
bialixjam: when you traverse merged branch you need to know where to stop, when you come back to mainline. is it correct?14:16
jamYou can call Branch.revision_id_to_revno14:16
jamand if it raises an exception, the revision isn't on the mainline14:16
vilajam: I have trouble with the numbers you mention as I can't easily say which relates to our existing code and which depend on your wip14:16
jambialix: there is quite a bit more too it, as some of the intermediate 'merged' revisions may have been merged earlier14:17
jamhttp://paste.ubuntu.com/410035/14:17
jambialix: with time going up14:17
jamyou need to know that C is merged into E when you go to expand F14:17
jambecause you don't display it14:18
bialixjam: I see14:18
jamhttp://paste.ubuntu.com/410036/14:18
jammerge_sort is how we determine what revisions go where in the graph14:18
jamand then we put dotted_revnos from that fairly easily14:19
bialixactually qlog expand entire merged branch, which sometimes is not the best thing. but I see what you mean14:19
jambialix: sure, but even then you need to know that C was merged14:19
jamsomewhere14:19
jamand to determine that14:19
jamyou currently have to walk backwards from all mainline revs14:19
jamin *case* it was merged there14:19
jamvila: so, the *counts* should be correct14:19
vilajam: I think loading the whole graph for a mysql branch < 2s, not 11s14:19
bialixjam: ok14:20
jamvila: importing the whole graph into the database is 11s14:20
jam"bzr walk-ancestry --method=bzr" was 2s14:21
jam--method=bzr-kg was 1.4s14:21
jam--method=db-dotted was 0.087s14:22
Lo-lan-dojelmer: Switching to /branches/$login_$branchname, I seem to run into "Operation denied because it would change the mainline history." when committing a merge, which I don't understand since I'm doing no such thing.14:22
jamvila: so the time to get data *out* of the db should be accurate, and the counts of objects *in* the db should be accurate for future reference14:22
jam(subject to tuning, etc :)14:22
jamthe time to put data *in* the database is still being worked on14:22
vilaok, clearer, loading the db is not the most relevant, it should occur only one and then incrementally14:23
jamvila: so I think loading the db taking 11s the first time is probably going to stay14:23
jelmerLo-lan-do: you're not pushing merges?14:23
vilahmm, may be not, well it depends, your focus is loggerhead only/first/dunno ?14:23
jamLoading an arbitrary real-world branch (today) requires grabbing get_known_graph().merge_sort()14:24
jamwhich has a similar ~1s overhead14:24
Lo-lan-dojelmer: I'm trying to commit a merge in a branch bound to SVN, so I probably am.14:24
jambut then loading the unmerged data into the db is quite fast14:24
jam--expand-all is sort of the ultimate "bring in as many branches as I can possibly find"14:24
jammeant to stress-test the system14:24
jamthough you may want to run that in the future14:24
jamIn the import code *today* I believe the bottleneck is calling KG.merge_sort() for each one14:25
Lo-lan-dojelmer: svn cp $svnrepo/trunk $svnrepo/branches/lolando_test, then a commit in a bzr branch bound to svnrepo/branches/lolando_test, then a merge of that into a checkout of $svnrepo/trunk, then commit.14:25
jamOnce I finish what I'm looking at now14:25
jamboth should be possible as an 'incremental update', and then you should be sub-second for all imports after the fisrt one14:25
jam(note that right now you are at around 44ms for --expand-all per branch, and I think 40ms of that is merge_sort)14:26
jammysql is around 100ms per expanded-branch14:26
jamvila: do those numbers make sense to you?14:26
vilajam: that sounds sensible, my first interpretation is that you're demonstrating that the IOs are killing us, so batching them is good :)14:27
vilajam: the db size is still a problem though, may be not for loggerhead... well, at least lp should make it possible to have one by project14:28
jamvila: well, I was pretty surprised to see that "walk-ancestry" using the database was still 300ms or so, until you start walking 100 mainlines at a time, and it suddenly drops by an order of magnintude14:29
vilajam: your dn can be shared across all branches of project right ?14:29
jamvila: in theory it can be shared across all branches of all projects14:29
jamthough like shared repos, you don't benefit as much14:29
vilajam: the numbers for walk_ancestry are for iterating the whole ancestry ?14:31
jamvila: yes14:31
jam80ms to get the *entire* ancestry14:31
jam30k revs14:31
jamor 68k revs for mysql14:32
jamthough for the db numbers, that isn't always in 'revision_ids'14:32
jamyou would have to probe one more table to get back to raw revision_ids14:32
Lo-lan-dojelmer: http://whiteboard.debian.net/d6d21f.wb has the log14:32
jamvila: so I'd like to discuss some of the algorithms for generating merge_sort, and see if they make sense to you14:40
=== kfogel is now known as emacs
=== emacs is now known as kfogel
vilajam: sure14:43
jamso I have some updated docs, and a start of an implementation in lp:~jameinel/+junk/bzr-history-db14:45
jam(currently on rev 45)14:45
jamthere are a few ideas in there, I didn't really get it polished14:45
jambut the general idea is to figure out what revisions need to be adedd14:46
jamadded14:46
SamB_XPaeddd14:46
jam(to the dotted_revno cache)14:46
jamand I thought of some interesting uses of gdfo and parent => child mappings14:46
jamthe idea is that if all known children of a revision can be accounted for, then you can know that a rev was/wasn't merged14:48
jamrather than having to walk in pure gdfo order14:48
vila"accounted for" ?14:49
vilaoh, and how do you handle ghosts ?14:49
jamhttp://paste.ubuntu.com/410050/14:49
jamtime going upwards for you14:49
jamsorry, missed a step, just a sec14:50
jamwell, maybe I can work with that14:51
jamthe idea is that X : B could be very long14:51
jamwhich would make E have a high gdfo14:51
jamhowever, thinking about it, it would still only be proportional to the diffs in the graph14:51
jamso that doesn't quite fit my case yet14:51
jamvila: at the moment ghosts are just treated as roots14:53
jamI plan on accounting for them by putting them in a 'ghosts' table14:53
jamsince they are rare, I don't want to add a column on the 'revision' table for all the ones that aren't ghosts14:53
jamso one possiibility: http://paste.ubuntu.com/410052/14:54
jamIn this case, B is from a very old revision, so X has a few revs to it, but can have a much lower gdfo than C14:54
jamthe idea is that the only known child of X is D14:55
jamso you know it wasn't merged before C14:55
jamand the only known child of parent_of(X) is X14:55
jamso again, you know it wasn't merged before14:55
jamwhich continues until you get to B14:55
jamat which point, you have to see if B was merged before,14:55
jamso you start walking the mainline14:55
jamhowever, I do wonder how the design works with: http://paste.ubuntu.com/410056/14:56
jam(no merge)14:57
jamIt will need to walk all the way back to A, and will load the dotted-revno info inbetween14:57
jammay be unavoidable, but I think with known_children you could instead just walk the mainline revs, and not their merged revs14:58
vilayour db provide the children ? Or you build that as you walk the graph ?14:59
jamvila: the parent table intrinsically provides children15:00
jamSELECT child FROM parent WHERE parent =?15:00
jamvs15:00
jamSELECT parent FROM parent WHERE child = ?15:01
jamI believe I have 2 indexes on the table15:01
jelmerLo-lan-do: it wants to copy from one of the merged branches I suspect15:01
jelmerLo-lan-do: and keep that as mainline15:01
jelmerLo-lan-do: please file a bug15:01
vilajam: well, then you have even more shortcuts than with gdfo :)15:02
Lo-lan-dojelmer: I've tried to do a svn commit in trunk just now, and now I can't bzr update the bound branch :-/15:04
Lo-lan-doAssertionError: Tried registering <CachingRevisionMetadata for revision 9363, path trunk in repository '9d84d37e-dcb1-4aad-b103-6f3d92f53bf6'> as parent while None already was parent for <CachingRevisionMetadata for revision 9368, path trunk in repository '9d84d37e-dcb1-4aad-b103-6f3d92f53bf6'>15:04
jamvila: right15:04
vilajam: you're still loading the whole graph anyway right ? (by whole I mean the revisions reachable from tip)15:13
jamvila: the goal is to not do so15:16
jamI believe I can compute merge_sort by only loading "enough" revisions15:16
jamwhich is, the last merge sorted parent15:16
jamof all newly merged revisions15:16
jamand possibly a bit more context to get the numbers right15:16
jamI still haven't sorted all that out, but I think I have a feel for it15:16
vilajam: the thing I like with your approach is that it makes it easier to try various new tricks, some of them could be backported with a bit of luck15:24
vilajam: can we query the bzr indices from a starting key for batch of keys ?15:24
jamvila: not trivially15:27
jamthough that is what _known_graph_ancestry tries to do15:27
IslandUsurpersorry for the interruption, but did cherrypicking ever cause a pending merge? I was surprised when it didn't just now15:32
maxbcherrypicking never does15:32
maxbIt would be illogical for it to do so15:33
jamIslandUsurper: if you give a "cherrypick" range that actually includes a full ancestry, then it will record a pending merge15:33
jamif it is 'accidentally' not actually a cherrypick15:33
IslandUsurperyeah, that makes sense15:33
jamvila: I think I've found a couple holes in my new design15:39
jamI pushed up a test case in rev 46 that might show it15:39
jamspecifically, when computing the second value in the dotted revno scheme, you pretty much *have* to load all children of the first revno15:39
jambecause there can be shortcuts that bump the 'branch' number, that are hidden by more recent merges15:40
jam:(15:40
jamvila: what were you saying about changing how we number again ? :)15:40
vilajam: hehe, don't resign too fast :)15:42
vilajam: but yeah, since there is not much added value in our actual scheme, changing for one that would give the merged-in mainline revno, can't be a loss15:43
jamso one option is still "load all the ancestry between now and the parent you want to number from"15:43
vilajam: but if merge_sort is only 40ms for a complete graph, the problem to address is really to load the graph (or the needed part) faster (and preferably with an acceptable space cost ;) no ?15:46
jamvila: 40ms * 4k is minutes to expand all of bzr.dev. With incremental merge numbering we should be able to cut that by at least an order of magnitude15:47
jamthe bigger issue15:47
jamis that I'd like to have a cache that gets updated with *commit*15:47
vila4k ?15:47
LeoNerdTiny bugreport: bzr should ignore SIGWINCH...   I got a failure of "bzr pull" with "Interrupted system call" from resizing the window15:47
jam(4k branches in the history of  bzr)15:47
jamso the 1-2s to load the ancestry of bzr.dev plays a much bigger deal there15:47
jamwhich is why I wanted an incremental algo15:48
vilaI'm talking about a *single* merge_sort, isn't it 40ms *without* the loading part ?15:48
jamright15:48
vilaright, gdfo is the easiest trick to manage at commit time :-)15:48
vilagdfo = mac(gdfo) + 1 :)15:48
vilas/mac/max/15:48
=== deryck is now known as deryck[lunch]
vilabut the actual numbering scheme is the problem since it requires loading the whole graph15:49
vilaeven if you manage to  "load all the ancestry between now and the parent you want to number from", you still have the case where some daggy-fixed from a merged branch and merge again15:50
vilas/some/someone/15:50
jamvila: true, though even daggy fixes don't go back to revno 115:50
vilajam: indeed !15:50
jamthe painful one in bzr's history is aaron's *long* lived integration branch15:50
jamhowever, I do think those cases are the exception rather than the rule15:51
jamand having an incremental algo will be good15:51
jamjust that i have to be aware of edge cases so I don't get them *wrong*15:51
jamand the algo can be fast as long as it detects when it needs to fall back15:53
jamso far, the 'detecting' part still looks expensive, which I'm unhappy about15:53
vilajam: I think that even branches that are merged multiple times could be addressed too  with a different revno scheme, they'll get dotted revnos with different first parts15:53
vilajam: and again, using gdfo will help ensuring we number each revision based on its first landing (but gdfo may not be enough here, we may need a bit more help)15:55
vilajam: so, concerning roots, you haven't addressed the filling part yet right ?15:56
jamvila: ghosts/roots, no. But wipe and restart is a valid option.15:57
jamThe parent graph is still correct in the db15:57
jamso you don't have to  go back and touch branches again15:57
jamvila: note that gary told me earlier that qlog makes use of the fact that a "branch" maintains its second number15:58
jamwell, first 2 numbers15:58
vilajam: yeah, I think it's not such a good idea, but a good shortcut because we don't have cheaper ancestry checks15:58
vilajam: and I would be delighted if he could magically find back the true branches in mysql mazes :)16:00
vilajam: I just noticed: "If a revision   X is in the ancestry of tip T, then the dotted revno for X will be the same  for all descendents of T."16:04
vilathis is true *only* if T was a tip and is *still* on the mainline16:04
vilajam: have you tried reusing your db on different mysql branches ?16:05
jamvila: well I did --expand-all, which should be approx the same as "different branches"16:21
jamvila: and for the "same for all descendents", sure, but that is how I'm caching16:22
vilajam: ok, just wanted to make sure, there is really no mainline shared between mysql branches (apart from the initial import maybe)16:23
jamvila: see my results16:26
jamthere is an average of only 68 rev divergence16:26
jamand the rest is 'shared mainline'16:26
jamvs bzr which has an average of 170 rev divergence16:26
jamput another way, even though the ancestry of mysql-6.0 is 2x that of bzr, the final size of the db was == bzr after filling out the dotted revno table16:26
vilajam: that's where I'm really surprised :)16:27
jamvila: yep, I was too16:27
jamI haven't probed enough to figure out why16:27
vilajam: as in: a bit too much to trust these numbers wit... yeah :)16:28
vilajam: what commands should I run in a given branch to get the same stats ?16:31
jamvila: generally 'bzr create-history-db --db=filename -d branch --expand-all'16:33
jamIt *might* be broken right now, let me check16:34
jamah, it should be fine if you don't use --incremental16:34
jamwith --expand-all it should spit out some stats about how many revs were expanded, etc. And how many total entries you have16:35
jamwhich is how I figured the numbers16:35
jamyou can also use "sqlite3 filename" and then do custom queries in there16:35
=== mobby is now known as mobby_
vilajam: random thought: at *merge* time it's cheap to record how "deep" the merged branch is (for external branches) or how many new revisions are merged from the other mainline16:41
=== mobby_ is now known as mobby
vilajam: some hints may help devising a new incremental revno scheme16:42
jamvila: I'm not really sure what you are proposing.16:42
jamwhen merging, all branches are 1 deep16:42
jamall *merged* branches are 1 deep16:42
jam'how many new revisions' is probably not all that easy16:43
jamgraph-difference vs just finding a common ancestor16:43
vilajam: think merge-into and the consequences on the revnos16:43
vilajam: sure, but you have the relevant graph already loaded anyway16:43
jamat least most, probably16:43
vilajam: so we are in the "40ms for merge_sort" area, hence cheap16:44
=== deryck[lunch] is now known as deryck
NfNitLoopCoworker came to bzr from git.  He wanted to get rid of the last 3 revisions, so he did a 'bzr update -r -4'.  Then did some work, and was surprised when he couldn't commit.16:55
NfNitLoopwe fixed it with shelve and 3x 'bzr uncommit',  but...16:55
NfNitLoopis there a better way to do that?16:55
NfNitLoopbzr pull?16:56
james_wyeah, that works16:56
james_wor bzr uncommit -r16:56
NfNitLoopOk.  What's the purpose of being able to do an update to a past revision, then?16:57
NfNitLoopseems weird if you can't commit to it.16:57
zookoFolks: there is a student applying for Google Summer of Code to work on the Tahoe-LAFS project and to integrate Tahoe-LAFS with a DVCS.16:59
zookobzr is a candidate.16:59
zookoWould any bzr hackers be interested in mentoring or at least consulting?16:59
=== IslandUsurper is now known as IslandUsurperAFK
jelmerNfNitLoop: you can if the branch is not bound I think17:05
jelmerzooko: Yeah, np17:05
NfNitLoopit was not a bound branch.17:06
NfNitLoop'bzr up -r -4' seemed to update the working copy and even the current revision, but when he tried to commit it said that he couldn't because the working copy was out of date.17:07
NfNitLoopeven though it's a standalone branch.  (Well, within a shared repo.)17:07
=== salgado is now known as salgado-lunch
=== beuno is now known as beuno-lunch
eydaimonI need merge revno 519, 521, and 522. Is there some way I can do it in a single merge?17:30
=== salgado-lunch is now known as salgado
vilaWhat is the english expression about a round thing into a squared other thing ?18:24
LeoNerdRound peg, square hole?18:25
LeoNerd(or vice versa, really.. I expect either way works)18:25
vilayes ! But what is the sentence used ?18:25
vilatrying to fit a round peg in a square hole ?18:26
LeoNerdOh. something to the effect of "trying to fit.." yes. that18:26
=== beuno-lunch is now known as beuno
=== IslandUsurperAFK is now known as IslandUsurper
jamvila: are you still around?18:45
jamjust to mention "square peg in a round hole" is the more common phrasing18:47
jamhttp://www.google.com/search?q=round+peg+square+hole18:47
jam(notice that I searched for round peg square hole and got the reverse)18:47
jamanyway, I had another idea about the partial graph, in case vila comes back around.18:47
vilajam: still there for a few minutes18:48
jamhttp://paste.ubuntu.com/410151/18:48
jamvila: basic observation18:48
vilajam: and another random idea: you can add gdfo into your db right ?18:48
jamvila: gdfo is already in the db18:48
jamand I'm using it now18:48
jambasic observation: you don't need to load back to revno 018:49
jamyou need to load back to revno x.y.? that you are based on18:49
jamThat can potentially be a big difference18:49
jamwhen finding a new branch number based on 1.5000.1, you don't need the data for anything 1.<5000 (maybe)18:49
vilajam: you got it18:50
vilajam: and then, if you had a bit more info in the ancestry graph you may even get there without a cache18:51
vilajam: like: next merge point in my ancestry is at distance x18:51
vilajam: which should help addressing the problem of the last part of the revno (otherwise you have to walk it until its origin to start at 1) (for various definitions of origin)18:52
vilajam: but I'm still not sold about numbering in one way or the other (M: 5.1.1, L: 5.2.1, K: 5.1.2, etc) or (M: 5.1.2, K: 5.1.1)18:57
balorGah, I can never remember what should be merged! Do I merge a THIS and OTHER on to BASE?19:03
jamvila: sure, I can see changing the numbering, but finding that while I need more-than-minimal but I need less-than-back to 1 makes it worth moving forward19:05
jamI just wanted to make sure someone else agreed with my logic19:05
IslandUsurperbalor, THIS is your working tree, OTHER came from the merged branch, and BASE is what they have in common. So for each chunk, ask which change is more important, the one you had or the one you're getting.19:10
IslandUsurperthe tricky ones have some of each19:11
balorSo merge THIS and OTHER into each other19:12
IslandUsurperright19:12
IslandUsurperthey're displayed together in the original filename19:12
balorSo what's the point of BASE?  What information does it give me?19:13
balorAs I get common chunks in the same colour in Meld19:13
IslandUsurperif you diff BASE and either THIS or OTHER, you can see exactly what they changed19:14
balorah19:14
balorthanks19:14
vilajam: sure, sure, I'm very happy you work on *that* (reducing the cases requiring loading all the history)20:16
lifelessjam: your logic ?20:24
jamfull discussion is in the text file in my lp:~jameinel/+junk/history-db branch.20:29
jambasically,20:30
jamfor simple children, you just need to know there isn't another child20:30
jamand then you can do X.Y.Z+120:30
jamfor a new sub-branch20:31
jamyou need to know if there is an X.Y+120:31
jamto know that, you need to check all descendants merged into trunk from X.Y.120:31
jamsorry20:31
jamall merges into trunk since X.Y.1 was merged20:31
jamhowever, you don't care about when X.Y-1.* was merged20:32
jamlifeless: ^^20:32
jammaybe put another way20:33
jamif you are walking backwards, you need to load X.Y.Z if it is a parent of yours, so that you can do any of X.Y+1.Z, or X.Y.Z+120:33
jamto determine which20:33
jamyou need to load the children of X.Y.Z20:33
jamif there are no other children, you can do X.Y.Z+1 and do no further loading20:33
jamif there is one20:33
jamthen you need to load from X.Y.1 to determine any X.Y+1.1 that exist20:34
jam(+N if you prefer)20:34
jamThe key is that you don't need to load from X20:34
jamsince anything landed before X.Y.1 was landed, will have have a second number < Y.20:35
lifelessvila: can I purchase a review ?20:48
lifelessjam: bzr+ssh://bazaar.launchpad.net/~jameinel/%2Bjunk/history-db/ - not a branch20:48
jamlifeless: sorry bzr-history-db20:49
jamlp:~jameinel/+junk/bzr-history-db20:49
lifelessso, 'only the right hand side *children* of X can influence revision numbers X.*.*' ?20:50
jamlifeless: I'm pretty sure only lh children20:54
jamsorry20:54
jamyou were saying X20:54
jamchildren of X with X in their LH history will be numbered based on it, yes20:55
jamthe way the data is stored in the dotted_revno table today does affect things20:57
jamyou need to know that a child of X is in the ancestry of the current branch20:57
jamjust being a child doesn't affect X.*.* for *this* branch20:57
lifelesssorry, I should have said 'rh parents of X'20:57
jamlifeless: rh parents of X have no affect20:58
jamX is the ancestor of X.*.*20:58
jamit is, specifically, a LH ancestor20:59
=== ubott2 is now known as ubottu
lifelessjam: https://code.edge.launchpad.net/~lifeless/bzr/commit/+merge/2284821:21
lifelessjam: vila: https://bugs.edge.launchpad.net/bzr/+bug/53026521:22
lifelessdo you guys have a preference between21:22
lifeless*) a line to delete21:22
ubottuLaunchpad bug 530265 in bzr "Not modifying a suggested commit message commits anyway" [Medium,Confirmed]21:22
lifeless*) a y/n prompt if the message is the unaltered template21:23
jamlifeless: If we are being interactive enough to pop up an editor21:23
jamI think it is ok to prompt for a "did you really want to commit an empty message" ?21:23
jamlifeless: the mutter() seems fine21:24
jamI'm trying to understand the revprops chacheng21:24
jamchange21:24
lifelessoh21:24
lifelessI had it in my commit branch; it just reduces some cruft21:24
jamyou are removing it from _commit, but adding it to the constructor?21:24
lifelessshorter parameter list21:24
lifelessits already in the constructor21:24
lifelessits an internal handoff within the object21:25
lifelessso just changing where it becomes an attribute, to spend less time passing it around21:25
jamit would seem reasonable to do that to all the other ones as well then...21:25
jamso... not a problem, but incomplete one way or another21:26
lifelessyup21:26
lifelessIf I end up doing much more to commit I'll probably do the rest too21:26
lifelessits a bit ugly as it stands21:26
jamanyway :approve21:27
lifelessthanks21:28
lifelessre prompt/message21:28
lifelessI guess I feel that its cleaner to only interrupt the user once21:28
lifelessGUI's can check in the dialog21:29
lifelesshmmm21:29
lifelessjam: whats the current open bzr version on trunk ?21:32
lifelessassign to milestone is showing about 15 things, rather than 321:32
lifeless<- confused bear of small brain21:32
jamlifeless: either 2.2.0 or 2.2.0b2 I believe21:33
lifelessso commits in trunk will still be 2.2 ?21:33
jamwell, 2.2b221:33
jamyes21:33
lifelesskk21:33
lifelesscan we deactivate some of the others, do you think ?21:34
jamFrom looking at it, they are all stuff in the future21:47
jamnot sure that we would disable them, but I suppose we could for now21:47
jamunlikely to target them explicitly, but it can be nice to know when they are expected21:47
jam(so not great in the bug UI, nice to have elsewhere)21:47
=== salgado is now known as salgado-afk
TalidanHi there, i was just reading http://wiki.bazaar-vcs.org/CherryPick22:22
TalidanCherrypicking is vital to our project, the idea is that other users can mirror our branch, make changes22:22
Talidanand we can merge them if they're decent22:23
Talidani was just wondering what's meant by "Bazaar does not track cherrypicked revisions, although this feature is firmly on the wish list. "22:23
Talidanideally (currently im testing on launchpad) the person we merge from is credited with the change22:23
lifelessI don't see why you need cherrypicking given your description of your idea.22:24
Talidanwell, we're thinking of migrating from GIT22:27
Talidanwhich was hosted at github22:27
Talidanthere everyone (besides official mantainers) would open their own fork22:27
lifelessright22:27
Talidanandn push their changes to their own fork22:27
Talidanand if we thought they were decent, we'd merge them over22:28
Talidanmy understanding is that it requires cherrypicking for bazaar22:28
TalidanBearing in mind their fork might have loads of crap we dont want22:28
lifelessah22:28
Talidanwe only select the commits that we actually want22:28
lifelessso, you can use 'bzr merge -c rev'22:29
lifelesswhich will do a cherrypick merge22:29
Talidani was just wondering what "does not track cherrypicked revisions" actually means22:29
lifelessand commit --author "foo bar <foo.bar@example.com>"22:29
lifelessit means we don't track them22:29
Talidanmind elaborating?22:30
lifelessuhm22:30
lifelessI'm not sure where the disconnect is22:30
Talidantrack could mean a whole lot of things22:30
lifelessthere is no record kept of the fact that a particular commit was a cherrypicked merge.22:31
TalidanRight, okay thanks22:31
lifelessyou can do cherrypick merges easily22:31
lifeless(merge -c)22:31
lifelessand if you commit with --author they will be credited appropriately22:31
lifeless(in bzr merges never create commit objects, you need to do a commit after the merge)22:32
mwhudsonjelmer: ayt?22:38
TalidanWhat windows GUI bzr client would you guys reccomend?22:40
jelmermwhudson: I am now22:40
TalidanTortoiseSVN is the only Tortoise* i've had a good experience with, tortoisegit is a pile of rubbish, and tortoisehg is average22:40
mwhudsonjelmer: have you tested the kernel import on staging since last week?22:41
mwhudson(with a new bzr-git & dulwich)22:41
jelmermwhudson: I don't have a way to update the dulwich/bzr-git on staging afaik :-)22:42
mwhudsonjelmer: well, "caused to be tested" then :)22:43
mwhudsonjelmer: you have the same mechanism as me!22:43
mwhudsonjelmer: do you think it would be a useful test at this stage?22:44
lifelessTalidan: I hear good things about bzr-explorer22:46
TalidanThanks, i'll give it a go22:46
jelmermwhudson: ah, asking a losa ? :-)22:48
jelmermwhudson: There's one bug I noticed that I'd like to get fixed first (unusual modes disappearing doesn't seem to be handled right)22:49
mwhudsonjelmer: yeah22:49
mwhudsonjelmer: ok22:49
mwhudsonjelmer: any idea how long that'll be?22:52
jelmermwhudson: might be tonight, but there's a few other things I have to take care of first23:00
mwhudsonjelmer: okidoke23:00
* mwhudson continues applying the email shovel23:00
Talidani think it's an oversight to not have simple authentification given open source projects that dont need security but simplicity23:18
Talidanor wiat, does bzr support user/pass?23:20
Talidani guess it's launchpad doesn't?23:20
lifelessbzr supports user/pass23:22
lifelesslp uses ssh and oauth23:22
lifelessbzr doesn't support oauth yet23:22
lifelessthere is an interesting spec though23:22
lifelessthat google and others have done for oauth IMAP and SMTP23:23
lifelesswe could perhaps adapt that23:23
lifelesslp doesn't maintain cleartext passwords though, so it wouldn't be easy to turn on plain ol password supoprt23:24
jelmerlifeless: OAUTH over SASL, or using some different mechanism?23:28
lifelessjelmer: I haven't read the spec for how they did it in IMAP yet23:29

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!