/srv/irclogs.ubuntu.com/2008/06/11/#bzr.txt

beunomwhudson, interesting result00:03
mwhudsonbeuno: ?00:03
beunoquick and dirty cleanup of genshi, per Toshio's recommendations00:03
beunocuts the time it takes to generate the template in half00:04
beunolet me compare that against zpt...00:04
mwhudsonah00:05
beunodamn, zpt is still faster00:06
beuno~5sec against ~3sec00:06
beunoto render NEWS annotation00:06
mwhudsonbeuno: seem to be getting somewhere with simpletal fwiw00:09
beunomwhudson, ah, cool. It'll be interesting to see how that works out00:09
mwhudsonyes :)00:09
beunoI'm trying to set aside the other options, while I wait00:10
mwhudsonbeuno: tests pass :)00:26
beunomwhudson, oh, very cool!  branch/patch?00:27
lifelessbeuno: what changes did you make to the genshi templateE?00:27
mwhudsonmy zpt-templating branch, in a few minutes00:27
beunolifeless, replaced the include tag with the actual content, and avoided match:00:28
lifelessawesome00:28
beunoyeah, amazingly, that cuts render time in half00:29
beunostill uses more ram than zpt though, aside from the extra seconds00:29
lifelessbugs ftw huh00:29
mwhudsonoops, pushed to the wrong location00:29
beunolifeless, btw, I found a bug in bzr-search, which I haven't been able to fix very nicely yet, so when you're up to it, and want to help me a bit, I'll be happy to work it out. It also may merit a test.  When you re-index a branch with no changes, md5 collision tracebacks00:30
lifelesssure00:31
lifelesstwo issues there00:31
lifelessthree in fact00:31
mwhudsonman simpletal's logging is noisy00:31
lifelessa) you are indexing already indexed revisions00:31
lifelessb) md5 collisions should result in a content-match check followed by ignore(matches) or raise(fails)00:32
mwhudsonoh and unicode issues, fun00:32
lifelessc) we need the component combining facility shortly after we get growing-index-component-counts00:33
lifelessa) is the fix needed for this scenario00:33
beunoright, my initial though was to compare revisions, but it seems like too expensive of an operation for each time you index00:34
beunoI'm wondering if we can safely assume we need to index from the last revision indexed on00:34
lifelessso there is no 'last' as such00:34
lifelessbut we can take the branch revision tip00:35
lifelessand do a breadth first search from that out00:35
lifelessand when we find a revision that has been indexed, call search.stop_searching_any([revid])00:35
beunothat sounds pretty good00:36
lifeless(bzrlib.repository has this code, in search_out_something_or_other)00:36
beunoI like that   :)00:36
lifelessit will need a new method on the Index, to test for the presence of revisions00:36
lifelesssomething like adding a parameter to indexed_revisions00:36
beunomwhudson, revision 244 is the simpletal's?00:36
lifelessso indexed_revisions(revisions) -> intersection of revisions and the index00:36
mwhudsonbeuno: 22400:37
=== bigdo1 is now known as bigdog
beunolifeless, I already have a patch to seperate getting the branches revisions, as it seemed like it would be useful for other operations00:37
beunomaybe I can tweak it to accept an optional parameter to stop at00:37
beunomwhudson, right, 224, slipped, I'll take a look now00:37
lifelessI don't think that will work ;)00:38
mwhudsonbeuno: at least one more fix coming up00:38
mwhudsonbeuno: simpletal seems a bit slower, probably around the same speed as kid00:40
mwhudsonuh00:40
mwhudson'your fixed genshi'00:40
mwhudsonis what i meant to say00:40
beunomwhudson, I'll compare them now00:41
mwhudsonbeuno: get rev 22500:42
beunolifeless, so you'd rather I leave it as-is and duplicate code?00:42
lifelessbeuno: imagine we have 60K revisions00:43
mwhudsonbeuno: memory usage seems the same or slightly better with simpletal though00:43
lifelessbeuno: anything that requires getting set(revisions) for *either* the branch or the index will perform badly00:43
lifelessbeuno: so we have to spider, breadth first search00:43
beunomwhudson, simpletal really does complain a lot...  but that may be a good thing eventually00:46
mwhudsonbeuno: it does seem a little slower than zope.pagetemplate, but the memory usage is very nice00:47
mwhudsonand yes, it's whiny :)00:47
mwhudsonwhich is probably good, it's pointing out mistakes in the templates00:47
PengI wonder how speedy hg's custom template system is?00:49
lifelessprobably very fast00:50
lifelessbeuno: does my explanation make sense?00:53
lifelessbeuno: also the index is a set, its not got an inherent ordering so there isn't 'most recently indexed' as a concept to work from00:54
beunolifeless, not to me, no. But I probably didn't explain well what I meant. Aside from this bug, I moved the logic of getting all the revision ids in a branch to a seperate function, which returns a list. So basically just changed the code for the patch I sent out earlier. On top of that, I think I may be able to add a "stop at this rev" optional argument, so we can re-use that piece of code. Doesn't really have anything to do with search indexes, as far as I00:56
beunomwhudson, seems a bit slower, yes. 6-8 seconds slower to generate my 36k diff. But it does use less memory, and, well, it's nice to depend on than zope00:58
mwhudsonthere are also bugs remaining00:58
lifelessbeuno: so that piece of code (get all revision ids) is _only_ useful in an entirely empty index00:58
mwhudsonbut yes, seems worth pursuing00:58
lifelessbeuno: and its actually not needed then, because the search approach I describe will generate the same results00:58
lifelessbeuno: also note that there is no _single_ revision to stop at - its a graph, you need the surface of the intersection between the two sets (indexed, all) to be able to provide a stop-list, but that means knowing them all a-priori so is thus more expensive00:59
lifelessbeuno: grab a whiteboard, draw a branch with merges - say 5 mainline revs, 10 in total.01:00
lifelessdraw a circle around the bottom two mainline revs, should cover 3 or 4 total revs01:01
beunomwhudson, btw, what do you think about removing full text indexing by default in trunk?  It's driving me crazy  :)01:01
mwhudsonbeuno: +101:01
lifelesswe need to identify what is outside the circle without looking inside the circle01:01
beunolifeless, I see your point. I'll give it some thought and get back to you. Meanwhile, indexing an indexed-unchanged branch == boom01:02
beunomwhudson, I think just commenting that like out would suffice for now, and you've got the push powers  :)01:03
lifelessbeuno: right. tell you what, I'll whip it up tonight, its very straight forward (I've written it before). I'm mainly trying to broaden your mind to think about these problems in a clearer way01:03
lifelessbeuno: because these problems are the heart of the performance-of-bzr issues for many things01:03
mwhudsonbeuno: we need to be careful what happens when people type something in the search box then01:04
mwhudsonthough i dunno, the naive approach only took 20s on bzr.dev01:04
lifelessmwhudson: 20s its 19 seconds too long for a web app01:05
mwhudsonlifeless: yes, indeed01:05
lifelessmwhudson: also, I suggest you guys don't limit yourself to bzr.dev for a test branch.01:05
lifelessmay I humbly present python01:05
beunolifeless, perfect. I'll look into that piece of code as soon as you push it. I understand the problem, and I have a few ideas on how to get around it, but I'd have to put them into practice. I really apreciate the explanation  :)01:06
lifelessits 60K revisions, but only a few hundred MB01:06
mwhudsonlifeless: the ~vcs-imports branch?01:06
lifelesses01:06
mwhudsonor is there a better one?01:06
lifelessno01:06
mwhudsonk01:06
lifelessbzr-svn'd01:06
mwhudsoni use launchpad sometimes :)01:06
beunomwhudson, what took 20s?  It takes a good 25 minutes to index commits here01:06
mwhudsonbeuno: if you run without caches01:06
mwhudsonthe search is very naive01:07
beunoah, right. Well, at this rate, we should be able to benefit from lifeless's efforts real soon and cut that down to under 1 sec  :)01:07
mwhudsonbeuno: https://code.edge.launchpad.net/~mwhudson/loggerhead/zpt-templating is up to rev 226 now01:08
lifelessI have to commit this soon01:10
lifeless 63 files changed, 4891 insertions(+), 6962 deletions(-)01:10
jmlhello #bzr01:10
RAOFHello jml :)01:10
jmlWe were doing some tests with RepositoryTestProviderAdapter. It seems to have disappeared from bzr.dev.01:10
mwhudsonbeuno: i notice the javascript on the revision page is broken :/01:11
lifelessit is deleted01:11
lifelessyou should now use01:11
lifelessTestScenarioApplier01:11
lifelessto apply the scenarios01:11
lifelessand peek in bzrlib.tests.repository_implementations.test_suite to see how the scenarios are created01:11
mwhudsonbeuno: and the css on the revision page seems a bit odd, though maybe that's just the new ff01:11
mwhudsonbeuno: the inventory listing also looks crappy, the 'last changed' column is too narrow and the dates are wrapping01:12
mwhudsonbeuno: other than that, i reckon this branch is probably good to land, what do you think?01:13
beunomwhudson, looks better, yes. I doing some last performance tests to convince myself we're not loosing too much, and that it's worth the tradeoff for better dependencies01:14
beunostill, 40 seconds to render a file seems too much to me01:14
mwhudsonwhere does 40 seconds to render a file come from?01:15
beunoso, this could be a good step forwards, as opposed to 1.35m and gigwats of ram01:15
beunomwhudson, my 36k dif01:15
mwhudsonoh right01:15
beunoit takes bzr ~13 seconds to generate it, and the rest is rendering01:16
* pygi wonders if anyone of the bzr folks is going to Akademy?01:16
lifelesswhen is it?01:17
mwhudsonbeuno: yes, still too slow01:17
lifelesscertainly Jonathan Riddell will be, but hes more a user :P01:17
mwhudsonbeuno: but much better01:17
pygiOctober 8 - October 13 lifeless01:17
pygipygi, October 15 that is :p01:17
pygiergh01:17
pygimy counting xD01:18
pygiAugust*01:18
mwhudsonbeuno: can you work on the things i mentioned above?01:18
beunomwhudson, yup, on it. I'll work towards sending a similar patch to zpt yesterday01:18
mwhudsonawesome01:18
beunojavascript seems fine for me01:18
beunoah, no it doesn't01:19
beunoin revision view01:19
mwhudsonit's probably worth getting trunk loggerhead and zpt-templating running side by side and comparing the look of each page01:22
mwhudsoni can help with this of course01:22
mwhudsonbut not right now as i'm going to go and get lunch01:23
lifelessjam: ping01:23
beunomwhudson, yes, I'll make sure I do that before I send the patch in. I'm going to go home, it's been to many hours at the office. Oh, and also eat something  :)01:23
pygilifeless, so I guess no one :p01:24
mwhudsonbeuno: ok, we will meet again in a few hours then :)01:24
lifelesspygi: I would raise it on the list01:24
pygilifeless, was trying to see if we could collect like 3 more people, and start a bzr group (share a room)01:24
pygithen we could hack a bit yay :D01:24
lifelessthat sounds like an excellent idea01:25
lifelessthere are more people that read the list than hang out in the IRC channel :P01:25
pygilifeless, perhaps but you know, rooms are limited (there are like 6 more beds or so :-/)01:25
lifelesspygi: so mail _now_ ? :P01:26
beunolifeless, FWIW, I sent you the patch for bzr-search, just because, as it is now, that's cleaner  :)01:28
beunoyou can discard it silently, just felt better to send before deleting01:28
pygilifeless, done :p01:28
* beuno -> home01:28
lifelessbeuno: thanks01:31
pooliehello01:32
pygihi hi poolie01:33
jmllifeless: would it be a good idea for TestScenarioApplier to have a method that takes a suite of tests and returns the multiplied suite?01:33
lifelessisn't multiply_* already what you want?01:34
jmllifeless: not quite. adapt_tests is close.01:34
jmllifeless: but it uses an out parameter rather than returning.01:35
lifelessjml: (No, not really, we want to decouple: the act of application, the act of accessing and deciding, and the act of making scenarios)01:35
lifelessjml: also, EOVERFLOW01:35
lifelessI have a 10K patch in my head. space is reduced01:35
lifelessseriously:  63 files changed, 4935 insertions(+), 7037 deletions(-)01:35
lifelessdon't expect arbitrary topics to work until this lands01:35
jmllifeless: ok. I'll disagree with you later in more detail :)01:35
lifelessthat sounds fine01:41
pooliejml, i can talk to you about this01:43
jmlpoolie: if you'd like. I've figured out how to do what I want, I just don't yet understand why the API is as it is.01:45
pooliei think the api needs to be cleaned up01:49
poolieand the various users made consistent01:49
pooliei posted about it on friday iirc, you can reply to that if you have any other thoughts01:49
lifelesswheee02:11
lifeless 63 files changed, 4891 insertions(+), 7387 deletions(-)02:11
bob2vf?02:12
lifelessyah02:12
lifelessmigrating the very low level tests across now, so deleting all the old classes02:13
lifelessFEELS GOOD02:13
pooliejml, is there any meaningful difference between a TestSuite and just a container full of tests?02:50
jmlpoolie: it's a subject of scholarly debate.02:51
poolieheh02:51
jmlpoolie: the big difference is you can run() a TestSuite.02:51
jmlpoolie: mostly I don't get why adapt_tests takes a suite as a parameter, rather than returning one.02:52
jmlpoolie: actually, another important difference between suites and generic containers is that TestSuite().addTest(some_suite) works.02:54
pooliehm02:54
pooliebut list.append(a_list) works too...02:54
jmlpoolie: yeah.02:55
jmlpoolie: if TestSuite.run() were implemented with iter_tests, then what I said about addTest wouldn't matter.02:56
lifelesspoolie: test suites can add capabilities03:00
lifelessbecause they can do things before and after tests03:01
lifelesslike IsolatedTestSuite for instance03:01
poolieso if they can be subclassed then we should presumably not be creating them ourselves03:02
poolieas we don't really know which one to create03:02
pooliei'm not sure whose decision it _is_ as to which one is used03:02
lifelessits the loaders03:02
poolieand it may not matter, because the tests can be copied across, or the suites composed03:02
lifelessand load_tests is designed to support this03:02
lifelessso TSA should not create a suite03:03
lifelessblink, I've been sucked in03:03
* lifeless goes back to code03:03
beunomwhudson, back (and thanks for disabling the search cache in trunk)03:11
mwhudsonbeuno: hi03:12
mwhudsoni'm working on changing the modified file links on the changelog view to match what trunk does03:12
beunomwhudson, ok, cool. I'll fix javascript and see what else is off03:14
mwhudsongrr03:17
jamlifeless: I'm around for a short time, what did you need?03:22
lifelessjam: I mailed you03:22
lifelessjam: basically, I have all tests passing but am not setting modes on knit files03:23
mwhudsonbeuno: ah, the javascript problems are '>'s getting turned into '&gt's in the output03:23
lifelessI want to know whether you think I need to (fixthetestsANDputmodesupportintothenewcode)03:23
jamlifeless: I've often thought about giving up as well, and just telling users to set their umask for their bzr connection03:23
lifelessjam: I haven't broken those tests or anything, they just don't seem to hit this code03:24
lifelessthe patch is quite large now:03:24
mwhudsonbeuno: maybe it's a simpletal buglet, but it's easy enough to work around (put the javascript in a js file...(03:24
lifeless 63 files changed, 4937 insertions(+), 7958 deletions(-)03:24
jamI'm pretty sure we had them at one point, I don't know at what level03:24
lifelesswe had them at a couple of levels03:24
jamand certainly they could have been removed without realizing03:24
lifelessthey just aren't triggering for this for some reason.03:24
jamI know poolie did some work on moving around where "_dir_mode" was being accessed03:24
beunomwhudson, from what I saw, it removes anything within an HTML comment  <!-- -->03:24
beunoat least for the revision view03:25
pooliehello jam03:25
mwhudsonbeuno: yeah, that too03:25
jamanyway, it is fairly moot for packs, and anyone on knits should be using bzr < 1.3 or so03:25
mwhudsonif you take out the <!-- --> you get the problem i mentioned03:25
lifelessjam: so, you're cool with this? we can always add it in later..03:25
poolielifeless: is it hard to set the modes?03:26
jamlifeless: *I'm* okay with it, but I've approved some of your recent changes and *other* people stumble over them03:26
pooliewith my recent landing it should be easy03:26
jamlike get_revision_graph03:26
beunomwhudson, ah, yes, you're one step ahead of me. So, move all javascript to .js file, where it should actually be.03:26
jami don't really know whether people are really depending on it or not.03:26
jamIt seems broken for sftp which would be one of the primary use cases03:26
mwhudsonrevision 229 heading to lp...03:26
lifelesspoolie: I haven't merged bzr.dev for a bit; I still have to deal with the conflicts there03:27
lifelesspoolie: I wouldn't say its hard, just not done03:27
lifelesspoolie: and as I count I have 8 days to land this and stacked03:27
lifeless><03:27
pooliesure03:27
lifelessI pity the reviewer03:28
lifeless(say it like mr T)03:30
beunomwhudson, does the code look small to you to?03:31
mwhudsonbeuno: yes03:31
beunogood03:31
* beuno pops the CSS open03:31
mwhudsoni probably bungled the css somewhere03:31
beunoall fonts seem different03:35
beunothe revision info block is bigger03:35
=== thumper_laptop is now known as thumper
mwhudsonbeuno: yeah, it's a bit odd03:38
pooliespiv, you should tweak and merge http://bundlebuggy.aaronbentley.com/request/%3C20080602011607.GA27779@steerpike.home.puzzling.org%3E03:39
mwhudsonoh man, how do i check that these six incomprehensible urls are the same on the two pages03:44
mwhudsonbeuno: how are you enjoying the css?03:54
beunomwhudson, I'm crying03:55
mwhudsonoh dear03:55
beunomwhudson, I've went ahead and started to convert the new theme to HTML, even though we're changing it a bit later on03:57
beunojust so we can use it ASAP03:57
beunothat's how bad it is  :)03:57
mwhudsonheh03:57
mwhudsoni would _hope_ we can fix the differences in zpt-templating with less effort than that03:57
mwhudsonafk for a few03:58
beunoalrighty, I'm back to making sense out of the CSS03:58
lifelessFINALLY04:01
mwhudsonbeuno: would you like to skype about this?04:01
lifelessold knit code deleted, full test run going04:01
mwhudsonlifeless: hurrah04:01
beunomwhudson, sure, opening it up now04:02
mwhudsonbeuno: read and weep http://bazaar.launchpad.net/~mwhudson/loggerhead/zpt-templating/revision/23204:07
bud3030Just had a pc to break couple of wires in the box was little hot04:23
=== bud3030 is now known as flint_dude
flint_dudenot able to fix it this late04:25
=== flint_dude is now known as bud3030
beunomwhudson, fixed the diff view, trying to figure out the revision info04:39
mwhudsonbeuno: great04:39
pooliemwhudson: could you sometime send a patch to add a pydoctor makefile target for bzr?04:44
mwhudsonpoolie: sure04:45
poolieis "pydoctor --make-html bzrlib --add-package=bzrlib" about right?04:45
mwhudson--make-html doesn't take an argument04:46
poolieoh right04:46
mwhudsonit's --make-html --html-output-dir bzrlib or something04:46
mwhudsonthese days i mostly use pydoctor --server --add-package bzrlib04:46
beunomwhudson, emailed you the patch to fix all the CSS oddness I could find04:52
mwhudsonbeuno: looks good to me04:54
beuno:)04:55
mwhudsoni mean, the css looks pretty nasty, but that's nothing new04:55
beunomy head will work for about an hour more. Anything else I missed to get this into trunk?04:55
mwhudsonnope, don't think so04:56
mwhudsoni'm going to do a good chunk of diff reading and then merge it04:57
beunogood, I'll work on the nicer-urls branch, and stick around if you run into anything you need me to fix04:58
mwhudsonok04:58
pooliemwhudson: there doesn't seem to be a --server option04:59
poolieat least in the version in hardy04:59
beunomwhudson, and just sent you an email with what we'll be working with04:59
mwhudsonpoolie: well, i only made the "release" that that package is based on under mild protest04:59
mwhudsonpoolie: bzr get lp:pydoctor05:00
poolieok05:00
mwhudsoni'm probably not that far off making a release that i'm actually happy with, come to think of it...05:00
lifeless7K passing05:06
lifelesskeep thy fingers crossewd05:06
pooliemwhudson: is there a way to make it just print the warnings?05:07
mwhudsonpoolie: maybe, the whole 'what gets printed' stuff is a bit over engineered in pydoctor05:07
mwhudsonpoolie: which warnings do you mean?05:07
poolierest syntax errors etc05:08
mwhudsonif you add a -v do the command line you'll get more information about them05:09
mwhudsonadd two and you'll get even more05:09
mwhudson(and a lot of other stuff i expect)05:09
poolieyeah it's all the other stuff about every module being imported that is undesirable05:09
mwhudsontry --verbose-about epydoc2stan05:10
poolieah05:10
pooliethere is also the issue that our docs are mostly in rest not epydoc05:10
mwhudson--docformat restructuredtext05:10
poolieoh great05:10
poolie:)05:11
mwhudsonyou can make a config file with lots of this stuff in05:11
mwhudsonthis is the one used for the online docs: http://python.net/~mwh/mybzrlib.cfg05:11
mwhudsonwarning: restructuredtext parsing is about 5 times slower than epytext parsing...05:12
pooliemm, so i see!05:14
lifelessyes yes yes yes yes yes yes yes yes yes yes05:18
lifelessRan 11009 tests in 1359.977s05:18
lifelessOK (known_failures=10)05:18
lifeless681 tests skipped05:18
poolieway to go05:19
pooliebtw is spiv around today? maybe just quiet?05:19
lifelesshe's around05:20
lifelesstime for bzr send05:20
lifelesshahhaha 800K bundle05:20
spivI'm around.05:20
pooliehello spivvo05:23
lifelessI'm gonig to take a short break (lunch, and do something to take my mind out of the details)05:24
lifelessthen I'll merge bzr.dev05:24
mwhudsonyay for bugs in loggerhead trunk05:32
pooliedown to 8 pending reviews05:33
pooliei'm going to take a break too05:33
beunomwhudson, I python2.5 slipped into your branch too, in start-loggerheat.py05:43
beunoit seems I committed my testing values at some point, sorry  (commenting out the daemon was the other)05:43
mwhudsonah, right05:44
beunoso, we can go with plain python05:44
beunoor stick to 2.5/2.4 in both05:44
mwhudsonlet's not make this change in this branch05:44
beunofair enough05:45
beunoI'm still resolving the remaining 4 conflicts out of 22 from merging my url-cleanup branch to trunk05:49
mwhudsoni'm still fighting url details a bit05:56
beunowhat do you mean?05:56
mwhudsonbeuno: click one of the 'changes' links in the table on http://bazaar.launchpad.net/~bzr/bzr/trunk/files05:58
beunomwhudson, right, I reported that as bug #23847705:59
ubottuLaunchpad bug 238477 in loggerhead "View revisions that affect file doesn't work" [Undecided,Confirmed] https://launchpad.net/bugs/23847705:59
mwhudsonoh ok :)05:59
mwhudsonwell, that's fixed in zpt-templating now06:00
mwhudsonbut checking all sorts of stuff like that06:00
beunomwhudson, that was just a problem with the URL?06:06
mwhudsonyeah06:06
mwhudsonbeuno: i merged the branch and am pushing to trunk06:08
beunomwhudson, great!06:09
beunoI'll merge my clean-url branch with that, and send a bundle for it. It seems pretty decent now06:10
mwhudsonsounds good06:11
lifelessbeuno: bzr-search can now index bzr.dev06:16
beunolifeless, time?06:17
lifelesssame, 20s06:17
lifelessbut it can report on the results now :P06:17
beunohahah06:18
lifeless0.7 seconds to query06:18
lifelessin theory it will scale better06:18
beunowell, that's more than usable today06:18
lifelesslunch is nearly over, I'm going to test indexing python now06:18
beunoactually, in loggerhead's terms, it's amazingly fast06:19
lifeless60K commits06:19
lifelesspython-packs$ time bzr index06:19
lifelessreal    1m9.116s06:19
lifelessuser    0m51.147s06:19
lifelesssys     0m6.468s06:19
lifelessgive me a search query06:19
beunobzr search the06:19
lifeless time bzr search Robert Collins06:19
lifelessbzr: ERROR: No matches were found for the search [u'Robert', u'Collins'].06:19
lifelessreal    0m2.536s06:19
lifelessuser    0m0.368s06:19
lifelesssys     0m0.084s06:19
beuno:)06:19
lifelessso, the turns up a lot06:20
lifelessmy screen is paging and paging :P06:20
lifelessI've stopped it06:20
beunohahaha, well, it works!06:20
lifelesstime to first line is more interesting06:20
lifelesstime bzr search the | head -n106:20
lifelessRevision id 'svn-v3-trunk1:6015fed2-1504-0410-9fe1-9d1591cc4771:python%2Ftrunk:43495'. Summary: 'Bug #1445068: getpass.getpass() can now be given an explicit stream'06:20
lifelessbzr: broken pipe06:20
lifelessreal    0m6.195s06:21
lifelessuser    0m5.840s06:21
lifelesssys     0m0.140s06:21
lifelessI can tell you why that is as well06:21
lifelessthe 'the' posting list is going to be freaking long06:21
lifelessso the shortest-list it can give results from it reads fully at the moment06:21
lifelessthis will get worse when documents are indexed06:21
lifelessit probably wants to iterate there06:21
lifelessand use concurrent iterators across all keys or some such06:22
lifelessthis needs bzrlib core changes, which I posted about in the context of pack performance today06:22
lifelessstill:06:22
lifeless time bzr search "Lange"06:23
lifelessRevision id 'svn-v3-trunk1:6015fed2-1504-0410-9fe1-9d1591cc4771:python%2Ftrunk:36422'. Summary: '[Patch #945642] Fix non-blocking SSL sockets, which blocked on reads/writes in Python 2.3.'06:23
lifelessRevision id 'svn-v3-trunk1:6015fed2-1504-0410-9fe1-9d1591cc4771:python%2Ftrunk:25536'. Summary: 'add SSL class submitted by Tino Lange'06:23
lifelessreal    0m0.538s06:23
lifelessuser    0m0.332s06:23
lifelesssys     0m0.108s06:23
lifelessthis is pretty fun06:23
lifelessI might blog about it tonight06:23
beunoI'm going to try and make it work with Loggerhead tomorrow06:24
lifelessawesome06:24
lifelessk, thats my hour, time to merge bzr.dev :P06:24
beunohave fun06:24
lifelessindices are fairly large btw:06:32
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh .bzr/bzr-search/06:32
lifeless55M     .bzr/bzr-search/06:32
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh ../.bzr/repository/06:32
lifeless99M     ../.bzr/repository/06:32
beunoI'd check to see how much loggerhead's are, but I need my cpu for the next hour06:34
lifeless:P06:34
beuno55M doesn't seem that bad for 16k revision text indexed, especially if it's that fast06:34
lifelessthats better06:34
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh .bzr/bzr-search/ --apparent06:34
lifeless6.1M    .bzr/bzr-search/06:34
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh ../.bzr/repository/ --apparent06:34
lifeless99M     ../.bzr/repository/06:34
lifelessit was all allocation overhead06:34
lifelesswhen I shove it into packs it will be that big06:35
beuno:)06:36
lifelesspoolie: is API CHANGES replacement for API BREAKS ?06:37
gourwhat is this 'bzr search' stuff? something new in 1.6?06:39
lifelessgour: its a plugin I wrote on the weekend06:41
gourlifeless: it looks cool. something like 'darcs changes' or ?06:44
lifelessI don't know what darcs changes is06:44
lifelessis darcs changes something like bzr search?06:44
Penglifeless: FWIW, your mailing list message went through.06:44
lifelessPeng: thanks06:44
gourlifeless: "Gives a changelog-style summary of the repository history" with ability to 'grep' on tags, patches...06:45
lifelessgour: no, not really then. its a full text index, (though it currently only indexes the commit messages)06:46
* gour is inspecting bzr-search on LP06:47
lifelessit won't eat your data :P you can install it and play :)06:48
gouri just wanted to ask if it eats data ;)06:49
lifelessman I hate conflicts. bzr.DWIM06:50
beunomwhudson_, any reason for "kw['start_revid'] = start_revid" in controllers/revision_ui.py?06:50
beunoI think it has something to do with what you fixed06:51
beunobut it broke my clean-urls branch06:51
mwhudson_uh, i forget06:51
mwhudson_it's probably not essential06:51
beunoI'll triple check06:51
beunojust wanted to know if I was missing something obvious06:51
beunorunning start-loggerhead.py twice makes you loose control of it, and you have to kill -9 it06:53
beunowe should probably do something more sensible there06:53
gourlifeless: what support do you plan for TERM? regex?06:54
lifelesswords06:54
lifelesspossibly phrases06:54
gourwildcards?06:54
lifelessI have some literature searches to do (though I have some probably naive ideas on that)06:54
beunomwhudson_, everything seems to work fine. I'm going to send my patch with that removed. Just FYI06:54
mwhudson_beuno: cool06:55
lifelessuhm, wild cards, like stemming (and regexs-to-match-one-term) can be done if someone wants to06:55
gourlifeless: boolean searches?06:55
mwhudson_what does annotate say about that line?06:55
lifelessgour: yah, it is a simple boolean at the moment, every term is AND06:55
lifelessto do wild cards, its basically search the term list for matching terms with the wildcards, then do a boolean on that06:55
gourlifeless: it just needs some good parser and then it can search everything ;)06:56
lifelessgour: do you mean 'it needs to get terms from file texts and so on' ?06:56
lifelessgour: because if thats what you mean, its definitely planned, see DESIGN.06:56
lifelessthere are some complications the more you think about it - given we're dealing with versioned data, what hits are most useful and so on06:57
gourlifeless: no, i meant something like mairix's syntax which is, afaik, used in e.g. claws-mail06:57
gourmairix is quite fast06:58
lifelessoh06:58
mwhudson_beuno: that kw stuff looks like super old code06:58
gouralthough, afaik, no utf-8 support06:59
beunomwhudson_, 167.2.1206:59
lifelessso the syntax you can support is driven by your back end06:59
beuno2007-08-2306:59
lifelessI don't have big plans for this, I wrote it because it seemed like an interesting thing to do06:59
mwhudson_beuno: that just changed the indentation07:00
lifelessI'll happily hand it over to someone, or review patches that improve it07:00
gourlifeless: it could be quite useful07:00
lifelessbut i'm not sure I'm going to spend hours on it07:00
beunomwhudson_, well, than yes, it's *super* old  :)07:00
lifelessgour: sure, it is already :P07:00
* gour thinks that using mairix could be interesting07:00
lifelessdoesn't look like a good fit TBH07:01
lifelesswe have very specific constraints on same management of data to work with FTP etc servers07:01
gour'marix-like' i wanted to say...mairix is for email07:01
lifelessoh well, feel free to dive in :P or file detailed feature requests.07:02
goursure, let me explore present solution 1st07:02
lifeless('be mairix like' means nothing to me. "Support '-foo' to exclude documents matching foo" does)07:02
lifelessok merge done, running tests to see what broke07:04
gourlifeless: right, let's say, using syntax similar to the one in mairix, explained under "Match words" in man page07:05
lifelessgour: I don't have mairix, I don't plan to install it. I'm not trying to be stubborn or anything but referencing something I don't have and don't use *doesn't help*07:05
beunomwhudson_, I have a few more quirks to work out on my branch since the merge with trunk. I'm going to defer sending til tomorrow, when I'm more awake07:06
mwhudson_beuno: sounds good, i'm not awake enough to read it today anyway :)07:06
lifeless(because everyone can point at their favourite and say 'be like that'). I could spend ages just reading them all. But there are more users than authors for bzr-search (already :P)07:06
lifelessso I think it makes sense for the larger group, which has the requests, to do the effort of creating specific suggestions07:07
lifelessin particular, things like 'what is it about mairix's search language you like'07:07
gourthe biggest win is that it's very fast07:11
lifelessindeed, that is a useful thing :)07:12
gourhere is some simple example of the syntax http://rafb.net/p/4kdE6G75.html i'm just saying as idea, not that it has to be like that at all...i.e. support for boolean (and, or, not) and some wildcarding would be enough07:12
lifelessI think I prefer the google style language07:13
lifelesswhere everything is AND07:13
lifelessto do not you do -TERM07:14
lifeless(giving AND NOT)07:14
beunoI'm off to bed07:14
lifelessanyhow, it sounds like you are really saying "I want more than exact word searches"07:14
beunomwhudson_, great marathon today. Congrats on getting trunk in better shape  :)07:14
gourhow do you handle rebuilding of the index?07:15
lifelesswhich is fair enough - but like I said, I'm doing this to scratch my own itches at the moment, which is mainly about the challenge of the thing07:15
lifelesswhat rebuild07:15
lifeless:P07:15
lifelessmore seriously, the disk format is logically done now, but the individual elements of an 'index component' are in separate disk files rather than a single pack file07:16
gourwhat about OR? or it is done by running two queries one after another?07:16
beunoaaaaand another reason why commit messages are immutable: it wouls break bzr-search  :p07:16
beunos/wouls/would07:16
lifelessincremental index operations generate additional components07:16
lifelessand I'll use a bog standard exponential backoff to combine components to amortise time-to-insert and time-to-query07:17
gourthat's good07:17
lifelessit has no OR support today, but yes, to do or I would expect serial queries inside the engine with a union07:17
gourbtw, what follows after bzr-1.9? 1.10?07:18
lifelessyah07:20
lifelessits just a serial07:20
gourwill 1.6 release consider guadec conference and gnome needs?07:22
lifelessI sure hope so07:24
gouryesterday i was reading about imendio plans to push gtk-3.0 out...too bad, they're on git already07:24
lifelesspoolie: ping07:29
poolielifeless: pong07:32
lifelesswant a quick call before EOD?07:32
pooliesure07:34
lifelesspoolie: if you are still around the updated patch is in the moderator queue08:56
lifelessholy cow09:16
lifelessI strongly suspect this is as long a slope as bzr itself...09:16
mwhudson?09:19
lifelessbzr.dev$ time bzr search VersionedFiles09:20
lifelessFile id 'versionedfile.py-20060222045106-5039c71ee3b65490', revision 'robertc@robertcollins.net-20070921013648-i9w180g6ea73w9mf'. Summary: 'No summaries yet.'09:20
lifelessFile id 'fetch.py-20050818234941-26fea6105696365d', revision 'andrew.bennetts@canonical.com-20070821235535-37okm0uaprwku9cu'. Summary: 'No summaries yet.'09:20
lifeless...09:20
lifelessreal    0m0.501s09:20
lifelessuser    0m0.420s09:20
lifelesssys     0m0.076s09:20
mwhudsonnice09:22
lifelessmwhudson: that is, I just implemented a crude index-file-texts patch for bzr-search09:22
mwhudsonthat must bump up the indexing time quite a lot?09:23
lifeless$ du -sh .bzr/bzr-search/ --apparent09:23
lifeless44M     .bzr/bzr-search/09:23
lifelessreal    5m53.305s09:23
lifelessuser    4m2.547s09:23
lifelesssys     0m14.733s09:23
lifelessand 545MB of memory at peak09:23
mwhudsonhuh, not bad09:24
lifelessthanks09:24
lifelesswithout --apparent, 316M of disk in the index :P. Really must put them into packs.09:25
lifelessprobably can improve things by doing a two-phase task09:25
lifeless40K terms, or so09:26
lifelessmake that 73K09:27
lifelessrevno 21 if you feel tempted to, say, index launchpad ;P09:30
lifeless"boom"09:30
mwhudsoni think i'd like to be able to use my machine for a little while longer :)09:31
lifelessyou could index, say, twisted. for fun.09:34
mtaylorI don't know if this is a bzr or launchpad thing - but I think bzr since the output looks like it's just bzr log reworked...09:53
mtaylorif you look at https://code.launchpad.net/~mysql/mysql/mysql-5.1-telco-6.309:53
lifelessgo on09:53
mtaylorall you see are a bunch of merge changesets, without any idea of what went on beneath them...09:53
pygilifeless, your plan to conquer the world by sending a mail to m-l did not work :p09:54
lifelesspygi: I know, so I sent another09:54
lifelessmtaylor: thats an lp thing, because bzr log does know and shows the indented merge09:54
mtaylorhm. oh you're right... it's missing that doesn't show them, right?09:55
lifelessyah09:55
mtaylork. thanks.09:55
* mtaylor goes to bug launchpad09:55
mwhudsonit's known09:55
* mtaylor aborts mission to annoy launchpad09:55
* mwhudson clicks browse code and wonders if that was a good idea09:55
mtaylorany time anybody wants a good will-this-work-quickly? testcase, doing stuff with the mysql trees seems to be a good place to start :)09:56
lifelessmtaylor: yah09:56
mwhudsonhow big is the tree?09:56
lifelessmtaylor: how much ram do you have ?09:57
mtaylorbig tree09:57
mtaylorI have 3G09:57
lifelessyou might like to try bzr-search out then09:57
lifelessI'd be interested to know how badly it fars09:57
mtaylorooh. yes, I might.09:57
* mtaylor goes to try it09:58
mtayloranything specific I should try to give you good feedback ?09:59
lifelesswell09:59
lifelessrev 21 adds full text indexing09:59
mtaylorso "bzr index" should make the indexes, right?09:59
mwhudsonmtaylor: roughly how many files?09:59
lifelessso try with rev 20, (time bzr index), and them rm -rf .bzr/bzr-search, and try with rev 2109:59
mtaylorok09:59
lifelessbzr search term [term ...] performs a search09:59
lifelesscurrently case sensitive10:00
mtaylorcan you do bzr search "term with spaces" ?10:00
lifelessnot yet10:00
lifelessI need to do some reading/thinking10:00
lifelessI imagine there are canned answers for that in the field10:00
mtaylorprobably so10:01
mtaylorrunning index using r20 now10:01
mtaylorwithout fulltext - real0m59.664s10:02
mtaylormwhudson: 960010:02
mtaylormwhudson: but around 60k revisions10:02
mwhudsonmtaylor: so pretty big but not huge10:03
mtaylorcorrect10:03
mwhudson~3 times the size of launchpad10:03
lifelessI should note10:03
mwhudsonnothing insane like OOo :)10:03
lifelessthis is not in any way optimised today10:03
mtaylormwhudson: a fresh branch normally takes around 25 minutes10:03
mtaylorlifeless: that's fine :)10:03
mwhudsonmtaylor: in a repo?10:03
lifelessits not completely-stupidly-naive10:03
mtaylormwhudson: well, the first time10:03
lifelessdoes mysql have fulltext-and-phrases search indices?10:04
mtaylorwell, fulltext indexes10:05
mwhudson_grr10:05
mtaylorI'm not sure it does phrases10:05
mwhudson_mtaylor: in a repo? <- last thing i said, dunno if it got through10:05
mtaylormwhudson_: yeah - but the first time - subsequent branches, of course, are not bad10:05
lifelessso my first idea for phrases is an adjaceny graph10:05
mwhudson_ok10:06
lifelessstart by getting candidates for simple AND10:06
mtaylormwhudson_: I think the biggest problem with that is the non-updating status bar bug10:06
lifelessthen look for term, term in an adjacency graph for that10:06
mtaylormm10:06
mtaylorthat sounds reasonable10:07
lifelesstats about 40 seconds worth of thought in the weekend10:07
mtaylorhehe10:07
mtaylorwell, if you want to look at other work in the field, definitely check out sphinx search10:08
lifelessyeah10:08
lifelessI was pointed at that10:08
lifelessseems interesting, but only the search api seems to be python accessible10:08
mtaylorhm. I wonder how hard it would be to embed...10:08
mtaylordang it10:09
lifeless:)10:09
mtaylornow you've got me thinking about YET ONE MORE project10:09
lifelessthing for me is I want this to run over the bzr vfs10:09
lifelesswhich means being safe on a small set of IO primitives10:09
lifelessand having their back end talk back to python10:09
lifelessit all seemed rather uninteresting, so I wrote an engine instead :P10:11
lifelessmtaylor: so, whats the memory use up to ?10:16
lifelessso mwhudson when are you getting a internet connection ?10:28
lifelessmtaylor: has the fulltext run started thrashing yet ?10:28
radixlifeless: have you written than indexer yet10:31
radixis that what mtaylor is testing10:31
radixs/than/that/10:31
lifelessradix: https://launchpad.net/bzr-search/10:33
=== mwhudson_ is now known as mwhudson
awilkinsHow's the XMLRPC server thnig going?11:36
=== kiko__ is now known as kiko-afk
lifelessawilkins: verterok might know12:09
awilkinsI just reverted two revisions and commited them as one but I still have one hanging around as a pending merge, how can I get rid of it?12:26
mwhudsonrevert has a --forget-merges option or something, but i don't really undestand the question12:29
awilkinsmwhudson: That was the right answer though :-)12:29
awilkinsI uncommitted two revisions, then committed the changes from those two revisions as one revision ; it retained the merge marker for one of those revisions which caused problems12:30
awilkinsGah, still got problems though12:31
fullermdEr.  That really shouldn't happen...12:31
awilkinsIt seems stuck now... it won't push to a branch it really should be pushing to without complaint "have diverged"12:32
awilkinsI don;'t hink I did anything bad....12:32
awilkins\All I did was hand-merge  some changes from the bzr-gtk plugin windows install and commit them12:33
mtaylorlifeless: it wound up eating all my RAMs12:41
lifelessmtaylor: yah12:42
lifelessmtaylor: I suspected it might :P12:42
mtaylorhehe12:42
lifelesslet me put a quick 'batch by 5000' in there12:42
mtaylorwell, you were correct!12:42
spivawilkins: perhaps "bzr missing" will help12:47
=== weigon__ is now known as weigon
lifelessmtaylor: I'm just acceptance testing this patch13:04
lifeless225MB peak13:04
lifelessthats much more tolerable13:04
mtayloryes. much more!13:04
lifelesswhen it finishes I'll see if the performance blows chunks or not (it doesn't have the grouping-logic done yet13:05
lifelessalso, the heuristic I'm using - 5K commit - is not that suitable for very wide trees13:05
lifelesspossibly I should instead use 200 or something ;P13:05
lifeless273 mb resident now13:06
lifelessok, 4m47 to index13:08
lifelessnow to search13:08
lifelessyeah, performance has been hit, largely disk cache I think, but all the same -13:08
lifeless133505 files13:09
lifelessI have a plan for that - incremental development FTW13:09
mtaylorhehe.13:10
mtaylorI just told someone yesterday "oh yeah, you need this patch I'm working on right now to do that..."13:10
lifelessok, pull, rev 2213:11
mtaylork13:11
mtaylorrunning13:11
lifelessplease keep an eye on top13:12
mtayloryup13:12
lifelessI'm very interested in memory13:12
mtaylorup to 500m resident13:12
mtaylorseems to be holding steady at 506m13:13
mtaylornope13:13
mtaylor60213:13
lifelessif you want to tweak this, in index.py look for 500013:14
mtaylor65013:14
mtaylorok13:14
lifelessbut see how it goes first13:14
lifelesslarge trees do imply large developer machines :P13:15
mtayloryou'd think :)13:15
mtaylor72513:15
matkorHmm how can I get info about given revision (342.29.1) who did it ? when etc ?13:16
lifelessmatkor: bzr log -r 342.29.113:16
lifelessmtaylor: once it starts the progress bar 0...5000 again it will be roughly stable13:16
mtaylor100013:18
mtaylorit's going to start another status bar?13:18
lifelessyou have 3 :P13:18
lifelessit does a progress bar for each group of 5K13:18
lifelessI haven't really done a UI for this, beyond proof-of-capability13:18
mtaylorso, what's it doing before that status bar is displayed?13:19
lifelesswell13:20
lifelessat the start it finds what revisions to index13:20
lifelessthen it topo sorts them13:20
lifelessgroups by <size>13:21
lifelessthen for each group it makes a new component index13:21
lifelessreads the inventories (this shows the progress bar) for those commits13:21
lifelessreads the lines introduced (roughly) by those commits13:21
lifelessand then reads the commit messages13:22
lifelesswrites that component out13:22
lifelessand loop13:22
mtaylorI seem to be stable at around a gig so far13:22
mtaylorand it's just making kcryptd work now13:23
mtaylorwhich, I suppose, one would expect with an encrypted root partition13:23
lifeless:P13:23
lifelessso its flushing an index probably13:23
lifelessmy entire drive is enrypted13:23
lifelessnow - the posting lists for each term are a separate index13:23
lifelessthis creates hundreds of thousands of files for me with the 5K group13:24
lifelessI hope you have btrees enabled for your file system :)13:24
mtaylorhehe.13:28
mtaylornope13:28
mtaylorplain-ol ext313:28
lifelessext3 has btrees13:28
lifelessuhm dirindex or something the option is called13:28
mtaylorbut not unless I've enabled them, right?13:28
lifelesshow old isyour install, what distro13:28
lifeless(as in, what distro install CD created your file system)13:29
mtaylorubuntu. hardy - reinstalled from scratch  a couple of months ago to get the encrypted root13:29
lifelessyou'll be fine :)13:29
mtaylorso, hardy alt cd13:29
mtaylorok13:29
lifelessstable ?13:37
lifeless(I mean, is it stable at 1G ?)13:38
jelmerPeng, do you know if mercurial has something like "bzr send" ?13:40
spivtune2fs can tell you what options are set on your ext2/3 partition, e.g. "tune2fs -l /dev/sda3 | grep features"13:41
lifelessmtaylor: (I mean, is it stable at 1G ?)13:42
mtaylorlifeless: yes. it seems to be stable13:43
mtaylorooh!13:43
mtaylorand I've got status bar now13:43
lifelessright, its doing another group13:43
mtaylorwell, I had a status bar for a short time13:43
lifelessits almost certainly done that quite a few times13:43
mtaylorthe bit with the status bar seems to be the quick bit. :)13:43
lifeless:P13:43
lifelesshow many mb is the .bzr/repository ?13:50
mtaylorlifeless: 635M13:50
toyto1hello lifeless, this is possible 'bzr commit -m "My Message" > filename.txt'?13:50
lifelessso, I would geuss at 1 hour or so to index13:50
lifelesstoyto1: what would you like that to do ?13:52
toyto1lifeless: let's say viewing the files that you have commited, like if you'll notice the 'bzr commit' it will open the default editor right?13:53
lifelessyes13:54
toyto1instead of viewing that editor i want to make it a file instead of a log file or let's say i use 'bzr commit -m "my message" > file.txt so that file.txt can be used as my backup for files that had been in the trunk13:54
toyto1or is there a way for bzr commands to list the files inside a trunk?13:55
toyto1let's say 'bzr filelist' will list the files or it can be 'bzr filelist > file.txt'13:55
lifelessbzr ls13:57
lifelessbzr ls URL13:57
lifelessbzr ls -r X URL13:57
toyto1thank you lifeless, may i kow what's the -r means?13:59
lifelesstoyto1: its used to specify revisions13:59
lifelesstoyto1: I recommend you read our user guide13:59
toyto1lifeless: ah thanks, yeah i was reading the user guide but I didn't notice reading it :( never had that idea yet14:00
lifelessmtaylor: can you do find .bzr/bzr-search/indices -name "*.rix"14:09
Pieterdoes bzr search work with large repositories yet?14:09
mtaylorPieter: that's what we're testing :)14:10
Pieterah14:10
mtaylorlifeless: I get 6 results14:10
Pieteris it in the branch now? or do you need changes in bzr.dev?14:10
mtaylorPieter: branch. I just pulled it off launchpad14:11
Pieterlet's try it then :)14:11
mtaylorif you have a large repo - it's not quick to index :)14:11
PieterI'll try on emacs14:11
mtaylorPieter: lp:bzr-search14:12
mtaylorPieter: emacs is in bzr now?14:12
Pietermtaylor: yeah, I already have it14:12
toyto1lifeless: I have found a good user guide it's 'bzr help commands' :D14:12
Pietermtaylor: no, they're switching14:12
mtaylorsweet!14:12
Pietermtaylor: but someone imported their cvs in bzr and made it public14:12
lifelessmtaylor: its completed 30K revisions then14:13
mtaylorwell, then I shoujld be done in another hour or so then14:13
lifelessPieter: its /functional/ on larger repos now14:13
lifelessPieter: how much ram do you have, and how many files are in the emacs repo ?14:13
Pieterlifeless: 2GB RAM14:14
Pieterlet's see how many files14:14
Pieter3000 files, 89000 commits14:14
lifelessK14:16
lifelessyou should be fine14:16
Pieterit's still cloning though14:16
lifelessyou don't have a copy alrady ?14:18
Pieteryeah, but I didn't want to use my pristine clone :)14:19
lifelessit writes to .bzr/bzr-search only14:19
lifelesscheck BUGS out for current caveats14:20
* mgedmin hugs everyone14:20
mgedminbzr is reading my mind again14:20
mgedminand working just the way I want it to work14:20
lifelesscool14:21
* mgedmin discovered that "bzr push" and "bzr merge" remember the last location separately14:21
Pieterlifeless: what does it currently index14:21
Pieter?14:21
lifelesscommit messages and file contents14:24
lifelessits pretty basic stuff14:26
lifelesslots of obvious room for improvement14:26
lifelessthe biggest actual caveat today is that it creates hundreds of thousands of files14:26
lifelessbecause I haven't written a transport adapter to allow the index layer to read an index from within a .pack file14:27
lifelesswhich I will get around to pretty soon14:27
lifelessthe main impact of that is that you have to have dirindex on on ext3 to have it work well :)14:28
Pieter:)14:28
lifelessit also badly needs some ui love14:28
Pieterwon't do wonders on my hfs then :P14:28
lifelesslike overall progress during indexing14:28
lifelessuhm, should be ok I imagine. Well - even though its a known defect feel free to report a bug if it blows horrible chunks14:28
lifelessif you want a teaser without the file text content searching, revert to revision 20, which only indexes commit messages14:29
lifeless(it has to do less, so obviously is a lot faster)14:29
PieterI tried that some time ago, but then it crashed somewher in bisect in bzr14:31
lifeless'bzr revert -r XXX' ?14:31
PieterI meant the index14:31
lifelessoh right14:31
lifelessrevno 20:14:32
lifelessmessage:14:32
lifeless  New disk format, one step closer to being in packs, and fixes the issue with overflowing bzrlib's index bisection capabilities.14:32
Pieter:)14:32
lifeless(I knew going it would fail on big environments till I built up additional capabilities, but I wanted early-results so took an incremental approach)14:34
lifelessall the mundane framework stuff like basic ui, format marker, open/close/create etc14:34
lifelessthats all in place, so actual changes now can deliver real improvements (like the full text stuff, which was about 40 minutes to do)14:35
lifelesss/full text/file text/14:35
lifelessanyhow, feedback appreciated14:40
lifelessif it chews ram (I would expect about 600MB stable for your tree) let me know - file a bug14:41
lifelessperformance too14:41
lifelessmtaylor: welcome back14:45
lifelessmtaylor: how many .rix now ?14:45
lifelessPieter: I'm just pushing a patch to drop the batch size to 2500 revisions, you may prefer that - less memory pressure14:46
lifelessit will degrade search performance until I write the batch-combining logic14:46
lifeless(or someone sends me a patch :))14:47
mtaylor_lifeless: my machine is taking a very long time to answer that question14:48
mtaylor_oh, and bzr is up to 1.2G14:49
lifelessmtaylor_: dentry cache :)14:49
lifelessmtaylor_: you will have _many_ files :)14:49
mtaylor_hehe14:49
lifelessactually14:50
lifelessI was being nub14:50
lifelesscat .bzr/bzr-search/names14:50
lifelessthats the root node14:50
lifelesseach line is a block of batch_size (e.g. 5K) revisions14:51
lifeless(actually its arbitrary size revisions, but your version only writes in one group size)14:51
Pieterlifeless: memory pressure is fine so far -- no more than 350MB14:55
lifelessPieter: ok cool; its proportional to (term_count * document_references) per batch14:55
lifelesswell, I say proportional, thats the key driver14:55
PieterI wonder how big the index will be14:56
lifelessso lots of small changes will use less memory than lots of big changes14:56
Pieterthe repo itself is 300MB14:56
lifelessuse du --apparent14:56
lifelessthe next stage will combine all the little files14:56
lifelessby writing them end to end in a big single file14:56
lifeless(next stage of development)14:56
lifelessI would expect, 200MB or something14:57
lifelessheres the one for bzr14:57
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ find .bzr/bzr-search/ | wc -l14:58
lifeless13350514:58
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh .bzr/bzr-search/14:58
lifeless549M    .bzr/bzr-search/14:58
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh .bzr/bzr-search/  --apparent14:58
lifeless50M     .bzr/bzr-search/14:58
lifelessrobertc@lifeless-64:~/source/baz/bzr.dev$ du -sh ../.bzr/repository/ --apparent14:58
lifeless99M     ../.bzr/repository/14:58
lifeless(../.bzr/repository is the shared repository containing all my branches)14:58
Pietermy du doesn't have apparent14:59
Pieterit's at 400k files so far14:59
lifelessoh14:59
lifelesswell14:59
lifelessthe posting lists are generally very small14:59
lifelessI forget the distribution name, its the one for use of words in programming languages :)15:00
lifelessPieter: mtaylor_: gnight15:04
Pietersee ya15:04
lifelessplease do let me know (email to the list or a bug report in launchpad)15:04
lifelesshow it goes15:04
lifelessoh15:04
lifelessfor reference15:04
lifelessits subsecond to start outputting hits for most every search I do on bzr.dev.15:05
PieterI think I'm going to stop it and wait until it's a single-file thing15:06
Pieterit's just doing disk trashing now15:07
lifelessdo you lurk here?15:08
lifelessI can ping you when it is15:08
Pieteryes15:08
lifelessk, I will do that then if you like15:08
Pieteryeah, thanks15:08
lifelessno problems15:09
lifelessgood night15:09
mtaylor_I know this is off-tope15:27
mtaylor_I know this is off-topic15:27
mtaylor_nm15:28
ricardokirknerhi, I am having an issue with bzr running as a smart server under wsgi. the error I am getting is: 'module' has no 'ElementTree' attribute.15:28
ricardokirknerthis is happening when trying to open the bzr shared repository15:29
ricardokirknerthe server is using bzr 1.5 while the client is using 1.315:29
ricardokirknerany ideas? I managed to trace a possible location for that problem to the xml_serializer.py file, where there is an import elementtree which falls back to the elementtree implementation of python2.4 which does not have the ElementTree attribute, but after changing that specific line, the problem is still there, so I am still trying to figure out where the problem is15:30
maanskyndoes anyone here have any experience with loggerhead?16:15
trepcahey16:18
trepcais there any support for attribute insertion as in subversion where you inserted "$Revision: $" which was then replaced with actual revision on commit ?16:19
statikjam: do we have a bzr logo like the one on bazaar-vcs.org but without the blue background?16:27
jamstatik: http://bazaar-vcs.org/LogoOptions16:27
jamIt comes in an SVG16:27
jamor a bunch of .png sizes16:27
statikjam: thanks!16:27
jamstatik: btw, looking forward to what comes out of that :)16:27
statikme too16:28
ricardokirknerhas anyone successfully integrated bzr with trac for multiple projects using http authentication for both read and write? (I am facing several difficulties here)16:38
=== mario__ is now known as pygi
jelmerricardokirkner: read/write in trac is unrelated to trac-bzr - or do you mean something else?16:40
ricardokirknersorry, yes16:41
ricardokirknerI think the trac part is irrelevant here16:41
ricardokirknerwhat I am trying to achieve is to have bzr use apache's authentication plugins (like for example ldap, which I already have working)16:41
ricardokirknerthe thing is: authentication works ok (it doesnt depend on bzr)16:41
ricardokirknerbut the integration through mod_<something> is not working correctly16:42
jelmerahh, ok16:42
ricardokirknerfirst I tried mod_python, but I got some issues that according to my research were due to mod_python16:42
* jelmer doesn't have much experience with bzr and apache16:42
ricardokirknerso I tried to use mod_wsgi, and I am still having issues (very strange ones)16:42
ricardokirknerregarding 'module' has no 'ElemenTree' attribute, when trying to open a repository16:43
ricardokirknerbut this does only happen when I try to open the second project's repository16:43
ricardokirknerthe first open works ok16:43
ricardokirknerand if I restart apache and try the other way around, the same thing happens, but with the projects reversed16:44
=== mario__ is now known as pygi
=== mtaylor_ is now known as mtaylor
emilis_infois it possible to commit a file to an older revision that has already been committed?17:32
mtayloremilis_info: no17:37
emilis_infoheh :/17:37
mtayloremilis_info: you can uncommit, and then redo the commit17:37
mtaylorbut a commit is a commit17:37
emilis_infoI understand that, but that would require me to uncommit a few revisions back...17:38
mtayloryeah. then you're pretty much screwed. :)17:38
emilis_infothe files in these revisions are completely independent17:38
=== kiko-afk is now known as kiko
emilis_infodoh, I guess I'll just do another commit17:38
emilis_info:)17:38
mtayloremilis_info: well... you could branch your current branch into a different place up until that revision17:39
mtayloremilis_info: then in the new branch, you could uncommit the last rev, then recommit adding in your new file17:39
mtaylorthen you could merge your other changes in maybe...17:39
mtaylorbut it might get ugly17:39
mtayloryou probably should not do ^^17:39
emilis_infoseems really ugly from your description :)17:40
fredreichbieranybody here who uses bzrweb from http://vmlinux.org/jocke/bzr/index.py?17:41
gouri need some advice what kind of workflow to use with bzr17:50
gouratm i used darcs for one project - study notes...17:50
gourthere are few courses, each has several written assignments with some questions. we could say project with few componenents, each having few features17:51
gouri plan to create shared repo and would like to work on specific 'component' and then merge to the main 'trunk' by having clear history17:52
gourdo you suggest to create branch for each 'feature' (question) and when i finish with it, then to merge back to main trunk and discard 'working' branch?17:53
pickscrapeSounds like you're talking about partial tree branches, or whatever it's called17:53
gourhmm, dunno17:53
pickscrapeCurrently when you branch you branch the whole tree, and your working copy contains that whole tree17:54
gourin darcs i can cherrypick patches and push them to the main trunk keeping the history clear and after finishing with the 'feature' i can easily dismiss unwanted branch.17:54
pickscrapeSounds a bit like rebase?17:55
gouri do not mind having the whole tree, but to have somewhat clean history in the 'main'17:55
gourhmm, no experience with git, but can take look at bzr's rebase17:55
fullermdThat ["do you suggest..."] sounds like what I'd do...18:01
gourfullermd: thanks18:03
Leonidasgour: what do you mean by "clear history"?18:14
=== cprov-lunch is now known as cprov
=== cprov is now known as cprov-out
=== kiko is now known as kiko-afk
=== mw is now known as mw|food
=== kiko-afk is now known as kiko
=== mw|food is now known as mw
Verterokmoin21:23
james_whi Verterok21:27
Verterokhi james_w21:27
fdvHi. Does anybody know whether it's possible to push a set of revisions as one to an svn repo?22:05
Odd_Blokefdv: As a single revision, or all at the same time?22:06
Odd_BlokeBoth are possible, but require different methods.22:06
fdvOdd_Bloke: as a single revision22:06
fdvthen I don't necessarily have to have my code in a completely good state for small commits22:07
Odd_Blokefdv: You should merge the branch with those revisions into a pristine checkout/branch of the SVN repo.22:07
fdvah, so that's possible, too?22:08
fdvI don't necessesarily have to talk directly to the svn repo?22:08
Odd_Blokefdv: Well, you'll merge into the checkout of the SVN repo and commit it having first sorted out conflicts/tested it is working properly.22:08
Odd_BlokeThis creates a single mainline revision, which is either in the SVN repo (a checkout) or can be pushed there (a branch).22:09
fdvOdd_Bloke: right, that seems fine22:09
fdvso, basically, I have two trees, then, a working tree and a 'commit' tree, and I merge my changes into the commit tree everytime I wish to push to svn22:10
fdvOdd_Bloke: sounds feasible. Thanks :)22:10
Odd_Blokefdv: Yup, that's it.  No worries. :)22:10
fdv(well, apart from that the repo will probably be about 7-8GB.. :p)22:11
LarstiQif you don't really care about the bzr history in svn, or roundtripping, you could also coversion22:12
fdvroundtripping and coversion?22:18
fdvsorry, just not familiar with the terminology22:18
LarstiQfdv: roundtripping would be doing a commit with bzr, send it over to svn, and pull with bzr from it again, getting the equivalent revision.22:19
Odd_Blokefdv: Coversioning would be where you just init a bzr branch in your SVN checkout, do intermediate work in bzr and then commit normally in SVN.22:19
LarstiQfdv: coversioning is when you use two seperate versioning control systems to version one tree, without them knowing about the other22:19
LarstiQfdv: so no sharing of history, but sometimes that is what one wants22:20
fdvah, right22:22
fdvthanks for great help, once again :)22:26
fdvI think I have to work with this for a while to find out how it's best to incorporate into our workflow22:27
fdv(or what kind of workflow one can accomplish)22:27
james_wstatik: I was just today trying to put together a screencast about bzr.22:39
james_wI got stuck trying to find something that I could use to edit the video without crashing.22:40
Odd_Blokestatik: You're emurphy on Twitter?22:40
james_whi Odd_Bloke22:42
=== howari_ is now known as howari
Odd_Blokejames_w: o/22:44
lifelessmtaylor: hi22:57
lifelessbeuno: also, hi22:57
jelmer'morning lifeless22:59
beunomornin' lifeless. I see you've managed to index files as well  :)23:00
lifelessyeah23:01
lifelesspeople kept bitching about it23:01
lifelessso I'm like fuck-it, I'll do it23:01
james_wmoanin' lifeless23:01
beunohahah23:02
beunoso that's the secret...23:02
lifelesshi james_w, jelmer23:03
mwhudsonbeuno: good morning23:39
beunogood morning to you mwhudson23:40
mwhudsonso i looked at pylons a bit last night23:43
beunomwhudson, I've been thinking about what would be enough to make a Loggerhead release. To get things rolling again. Maybe we can add nicer-urls, clean up setup.py, and... well, the templating change is already a big one by itself23:43
mwhudsoni wasn't very impressed :(23:43
beunoreally? Hard to use? Incomplete?23:44
beuno"just ugly"?23:44
mwhudsonugly, from my personal pov, and not really appropriate for loggerhead23:45
mwhudsoni think i'm quite stuck in my ways in some things23:45
mwhudsoni like xml templating and i like object publishing23:45
mwhudson(as opposed to routes-type things)23:45
beunofair enough. Did that include looking at Mako as well?23:46
mwhudsonalso pylons seems to have and encourage quite a lot of magic23:46
mwhudsonno23:46
beunook, well, a better look at Genshi may be worth a try instead23:47
beunoeither way, if pylons is that ugly, we may as well concentrate on the core instead, now that the templating engine is saner23:50
mwhudsonwrt a release, if we could get away from cherrypy, i'd say we were pretty close to a release-candidate23:51
mwhudsoncherrypy3 is an option i guess, but it conflicts with cherrypy 2 :(23:52
pooliegood morning23:54
beunohow about paste?23:54
beunomorning poolie23:54
mwhudsonbeuno: i'm still not really sure what paste is :)23:55
mwhudsonhi poolie23:55
mwhudsonbeuno: are you going to bzr send your cleaner urls branch?23:57
beunomwhudson, http://pythonpaste.org/23:58
mwhudsonbeuno: yes23:58
beunomwhudson, yeah, I just need to cleanup the latest quirks introduced when merging from trunk23:58
mwhudsonbeuno: that site is very bad at explaining what paste is :)23:58
pooliebeuno, hi23:59
beunoheheh, AFAIK, it can replace cherrypy, which is why I have that in my head23:59
mwhudsonah, http://pythonpaste.org/do-it-yourself-framework.html seems like the page i was looking for23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!