[00:31] <wgrant> If I have a remote branch with some ghosts, how do I fix them? If I fetch-ghosts on a local copy and then push, it does nothing.
[00:46] <lifeless> jml: what bug number is your stacked performance # ?
[00:46] <lifeless> wgrant: you need to fetch-ghosts directly on the remote branch
[00:46] <lifeless> I;m not sure if the cli supports that or if you'll need to [trivially] patch the command first
[00:46] <jml> lifeless: bug 331823.
[00:47] <garyvdm> wgrant: Hmmm - fetch-ghosts does not have a -d option. It should have...
[00:47] <wgrant> lifeless: Thanks.
[00:47] <lifeless> jml: its almost certainly the interface fixes for hooks
[00:47] <wgrant> garyvdm: What would -d do?
[00:47] <jml> lifeless: ok.
[00:48] <garyvdm> wgrant: normaly a -d option lets you specify the branch/directory to perform the action on
[00:48] <wgrant> garyvdm: Ah, right.
[00:49] <garyvdm> a> bzr push b == bzr pull -b a
[00:49] <garyvdm> Err
[00:49] <garyvdm> a> bzr push b == bzr pull -d b a
[01:02] <lifeless> my next patch drops push-trivial-branch to 44 hpss calls
[01:03] <bob2> spiffy
[01:14] <garyvdm> What does hpss stand for?
[01:14] <mwhudson> high performance smart server
[01:25] <jml> is loom included in the OS X dmg file?
[02:20] <thumper> lifeless: 44 seems high, what was it? and what is a trivial branch for this example?
[02:41] <lifeless> thumper: 107
[02:41] <lifeless> thumper: a trivial branch is a trivial branch
[02:41] <lifeless> current bzr.dev is 64
[02:42] <thumper> lifeless: like a single revision?
[02:42] <lifeless> yah
[02:42] <thumper> lifeless: to a stacked branch?
[02:42] <lifeless> no
[02:43] <lifeless> current bzr.dev stacked is 88
[02:43] <lifeless> for the equivalent test case
[02:46] <lifeless> my branch drops that to 68
[03:14] <rockstar> beuno, http://www.ericsink.com/entries/git_immutability.html
[03:15] <Peng_> The streaming push stuff should help "push a couple new revisions to an existing branch", right?
[03:16] <lifeless> Peng_: modulo bug 331823
[03:18] <lifeless> rockstar: nice
[03:18] <Peng_> Well, I don't use stacking. For my usual "bzr push-and-update" case, I think it's helped nicely.
[03:18] <rockstar> lifeless, I thought so.  This is something that's always bothered me about a DVCS in general.
[03:18] <Peng_> Seems to be about 7-10 seconds now, when it used to be maybe twice that.
[03:18] <rockstar> I still don't really like `bzr uncommit` either.
[03:19] <lifeless> Peng_: cool
[03:19] <spiv> Peng_: sweet
[03:20] <fullermd> "Some folks say that Git's killer feature is its index.  That's like saying that C's killer feature is the ability to cast an int to a pointer."
[03:20] <spiv> Peng_: http://bazaar-vcs.org/SmartPushAnalysis1.13 has some early numbers, FWIW.  1.13 final is likely to be significantly better than those numbers.
[03:21] <spiv> Peng_: glad to hear the improvements exist for real users :)
[03:22] <Peng_> spiv: Oh, thanks for the link.
[03:22] <Peng_> Thank you for your work! :)
[03:24] <RAOF> Faster push?  Sweeeeet!
[03:42] <mlh> fullermd: ... but that's true :-)
[03:43] <spm> spiv: any truth to the rumour that v1.13 will be release on the 2009-04-01? ;-)
[03:48] <spiv> spm: you'd have to ask the release manager ;)
[03:54] <rockstar> Hm.  `bzr help commands` seems to see the new plugin I'm writing, but actually invoking tho command throws a NotImplementedError
[03:54] <rockstar> Any idea why that might be happening?
[03:56] <spiv> rockstar: what's the traceback?
[03:56] <spiv> rockstar: I bet something in your command is triggering a raise of NotImplementedError ;)
[03:57] <rockstar> spiv, the only thing in the command is a docstring and a pass
[03:57] <rockstar> spiv, http://pastebin.ubuntu.com/122179/
[03:58] <spiv> rockstar: so you don't implement a run method on your command object?
[03:58]  * rockstar facepalms
[03:59] <rockstar> Was doing that in the __init__  Thanks for the extra set of eyes there.
[04:09] <jml> abentley: ping
[04:10] <abentley> jml: pong
[04:10] <jml> abentley: I'm looking at the lp-open patch...
[04:10] <jml> abentley: I would like to be very careful that Branch.open doesn't do network operations in the tests.
[04:12] <jml> abentley: sorry, I'm trying to think about how to formulate an actual _question_
[04:12] <abentley> jml: Seems ironic, considering what you're testing...
[04:12] <jml> abentley: yeah, I know. but I'd like the tests to not suck :)
[04:14] <abentley> jml: The approach of using Branch.base was a suggestion, not a requirement.  If you want to land the current patch first, that's fine with me.
[04:14] <jml> abentley: ok. I'll leave that bit as-is.
[04:14] <jml>         self.assertEqual(
[04:14] <jml>             ['Opening https://code.edge.launchpad.net/~foo/bar/baz in web '
[04:14] <jml>              'browser'],
[04:14] <jml>             self.run_open('bzr+ssh://bazaar.launchpad.dev/~foo/bar/baz'))
[04:14] <jml> abentley: ^^ that's the thing I'm not sure how to test.
[04:16] <abentley> jml: I guess you could register your own https handler.  Seems a bit overkill
[04:16] <jml> abentley: yeah.
[04:17] <abentley> Sorry, I meant bzr+ssh handler
[04:18] <abentley> jml: So this looks like a blackbox test.  Is it?
[04:18] <jml> yeah.
[04:18] <abentley> Does it need to be?
[04:18] <jml> umm.
[04:19] <jml> no. It was the easiest way to do it when the command was smaller.
[04:19] <jml> there are also unit tests for the feature.
[04:20] <abentley> It sounds like this test is redundant.  Is that right?
[04:21] <jml> not quite
[04:21] <jml> I tell you what the Right Thing is
[04:22] <jml> _possible_locations and _get_web_url ought to be unit tested
[04:23] <jml> and then that test would reduce to 'attempt to look up whatever is passed on the command line'
[04:24] <abentley> Sounds good.
[04:26] <jml> anyway, I've filed https://bugs.edge.launchpad.net/bzr/+bug/333681 and replied to your review
[04:26] <jml> abentley: do I need to send another bundle or a progress diff or something?
[04:28] <abentley> jml: Yes, you should submit a new merge directive, not an incremental diff.
[04:32] <jml> thanks.
[04:38] <jml> why would 'info' not print anything.
[04:39] <lifeless> if it crashed?
[04:39] <jml> lifeless: return code 0, say the logs.
[04:40] <jml> yeah, looks like non-existent branch.
[04:40] <mwhudson> bzr info in a bzrdir with no branch or repo says nothing silently
[04:40] <mwhudson> i meant to file a bug about that...
[04:41] <mwhudson> for example:
[04:41] <mwhudson> mwh@grond:trunk$ bzr info sftp://bazaar.launchpad.net/~mwhudson/bzr
[04:41] <mwhudson> mwh@grond:trunk$
[04:41] <lifeless> mwhudson: please do
[04:41] <jml> mwhudson: yeah, that's what I'm seeing.
[04:41] <jml> I'll do it
[04:42] <mwhudson> jml: thanks
[04:42] <jml> mwhudson, lifeless: https://bugs.edge.launchpad.net/bzr/+bug/333684
[04:46] <jml> incidentally, info -v does *way* too much.
[04:51] <jml> spiv: in this comment -- https://bugs.edge.launchpad.net/bzr/+bug/255292/comments/22 -- you say "capture the SFTP conversation"
[04:51] <jml> spiv: presumably you want some sort of unencrypted version?, rather than the data sent on the wire?
[04:51] <lifeless> gotta say the whole green flash when changing a bug title is ... odd
[04:53] <jml> lifeless: file bugs about it on Launchpad if you want to.
[05:06] <mwhudson> jml: fwiw, the copying of .bzr to backup.bzr on upgrading a launchpad branch seems to trigger that bug a pretty large fraction of the time
[05:06] <lifeless> jml: oh, I do
[05:06] <mwhudson> jml: if you want a test case
[05:06] <jml> mwhudson: that's good to know.
[05:07] <jml> mwhudson: right now, I just want to make some movement on it.
[05:41] <spiv> jml: yes please
[05:42] <spiv> jml: unless you want to provide me with tools to decrypt your SSH conversations ;)
[05:42] <jml> spiv: do you want to provide me with tools to record a decrypted SFTP session?
[05:42] <mwhudson> i imagine hacking paramiko is going to be required :/
[05:46] <jml> mwhudson: gosh, I haven't hacked paramiko to get debugging information in _months_
[05:52] <spiv> jml: I *want* to
[05:52] <spiv> jml: that unfortunately doesn't mean that I *will*...
[05:52] <spiv> Basically, what mwhudson said sounds like the most likely route.
[05:54] <spiv> Someone should write a pipedump util, like tcpdump for interprocess pipes ;)
[06:08] <spiv> jml: btw, I've just sent a probable fix for your bug to PQM
[06:08] <jml> spiv: the hour-long push bug?
[06:08] <jml> spiv: thanks.
[06:27] <spiv> jml: yeah
[06:27] <igc> igc
[06:27] <igc> is back
[06:27] <jml> welcome back!
[07:00] <vila> hi all
[07:26] <igc> hi vila
[07:26] <vila> hey Ian !
[07:54] <igc> vila: can we mark http://bundlebuggy.aaronbentley.com/project/bzr/request/%3Cm2prhqzm09.fsf%40free.fr%3E as merged now?
[07:54] <vila> igc: man, why are you *all* looking above my shoulder ? :-)
[07:55] <vila> I think it's merged, I'm trying to understand why it isn't :-)
[07:56] <fullermd> Would it help if I looked at your knee instead?
[07:56] <vila> igc: I think what happened is that poolie merged it more or less manually, I'm checking that
[07:56] <vila> fullermd: leave my knees alone too, please !
[07:57] <vila> fullermd: I mean, I'm an honest person, how dare you young man looking at my knees
[07:58] <poolie> hello vila
[07:58] <igc> vila: I assume you mean "over my shoulder" ..
[07:58] <fullermd> YOU claim you're honest.  I think you have shifty knees.
[07:58] <igc> vila: I'm just cleaning out the merge Q - you're patch just happened to (still) be in it
[07:59] <vila> igc: damn, over my shoulder, yes. Remember to tell you the favorite expression of my gf grand-mother  one day :)
[07:59] <vila> igc: damn, over my shoulder, yes. Remember me to tell you the favorite expression of my gf grand-mother  one day :)
[07:59] <poolie> vila, i'm pretty sure i did merge that, maybe
[07:59] <poolie> probably manually because all bb has is a patch not a bundle
[07:59] <vila> poolie: we are all pretty sure :)
[08:00]  * fullermd ponders.
[08:00] <fullermd> As more people are pretty sure, does that make the total likelihood higher or lower?
[08:00] <fullermd> I mean, a bunch of numbers [certainty] less than 1, multiplied together, get smaller and smaller...
[08:07] <vila> poolie, igc: right, the only missing bit is a depreceation test, I'll send it to PQM and BB should catch up
[08:09] <vila> poolie, igc: err, not even that much in fact, only some review feedback is missing regarding adding some try/finally around some f.write() calls
[08:09] <poolie> yes i was merging it for 1.12 so i didn't actually do the tweaks
[08:09] <vila> and the deprecation mention in NEWS
[08:09] <poolie> would be good to add them if you want to tidy up though
[08:10] <vila> poolie: doidn it right now
[08:10] <vila> poolie: doing it right now
[08:15] <igc> thanks vila
[08:19]  * vila slaps his knee: *Don't* run selftest in a bound branch, you know that damn it
[08:20] <poolie> vila, what happens?
[08:21] <vila> poolie: I get prompted for my id_dsa password since the test suite is isolated from my ssh agent
[08:21] <vila> this is due to 'bzr info' being used by some tests as a 'do something but not that much' command
[08:22] <vila> it's a known problem related to the branch nick
[08:22] <lifeless> vila: sounds like they are not properly isolated; which tests
[08:23] <poolie> it may be that we're trying to open the branch to print the revision of the source tere
[08:23] <poolie> tree*
[08:23] <vila> err wait, not bzr info, bzr version, sorry for the confusion
[08:26] <vila> lifeless: they *are* isolated ans SSH_AUTH_SOCK is cleared, that's the problem :)
[08:27] <lifeless> vila: if htey are isolated the fact that your branch is bound should be irrelevant
[08:28] <vila> well, not really *the* problem, that's only one way to look at it, it's just that bzr version get some information from the branch for the bzr it is running, which IMHO, makes it an unfortunate choice for that kind of tests
[08:28]  * vila searching for those tests again
[08:29] <lifeless> vila: the gathering of data of the bzr running the tests should be outside the test isolation
[08:32] <vila> lifeless: have a look at bzrlib.tests.blackbox.test_selftest.TestRunBzrSubprocess.test_run_bzr_subprocess and tell me if you will try your approach or choose another command :)
[08:35] <lifeless> vila: it looks like an isolation issue to me
[08:35] <lifeless> vila: I *bet* the cwd in one of tise is wrong
[08:36] <lifeless> vila: and in fact it is, its running the bzr under test rather than [e.g.] an exported copy or some such
[08:37] <vila> lifeless: I *agree*, but do you you really want to export bzr to run these tests ?
[08:37] <vila> I meant, I agree that's an isolation issue, but the cure sounds a bit excessive
[08:38] <lifeless> vila: perhaps --version shouldn't connect to bound branches in this way
[08:38] <vila> lifeless: I think I remember tracking the problem until an access is made to the branch *nick*, so not really a --version directly related issue
[08:39] <lifeless> yes, nick ill be the trigger
[08:41] <vila> and still from memory there is a default value which says get the remote one when we access the nick, but changing that wasn't an option, so I back-tracked and finally thought that the right fix was to not *use* --version when possible because that was not worth the effort, and since this was introduced by yet another totally unrelated patch, I just went unbinding my branches when needed :-)
[08:42] <vila> I mean, I mentioned the problem just after the patch was merged but... there was more important issues around that patch
[08:46] <vila> In other news, I can confirm that our ftp support hasn't regress under python-2.6 and that all the failing ftp tests were indeed caused by medusa not being compatible
[08:50] <poolie> http://benchmark.bazaar-vcs.org/rrd/runtime-common-suite-all.png
[08:54] <mtaylor> lifeless: what was that thing ( I think it was you that showed me ) that lets me manage multiple deb packaging branches from a single thing?
[09:01] <poolie> bzr-loom?
[09:01] <poolie> nightall
[09:09] <lifeless> mtaylor: autoppa
[09:10] <mtaylor> lifeless: hell yes! thanks
[10:33] <lifeless> spiv: it landed!
[10:33] <lifeless> spiv: if you have a script to record the timings, it could run overnight :P
[10:36] <lifeless> spiv: also, train ride home == VF decorator fully tested.
[12:07] <willwill> hello, I got bzr: ERROR: Invalid http response for http://bazaar.launchpad.net/%7Enotify-osd-developers/notify-osd/main/.bzr/repository/packs/535754cb35e490c69b794a3cc65ffee4.pack: Expected a boundary ()yo,4lJD726(aEMWnFZV) line, got ''
[12:08] <willwill> bzr 1.11 on Fedora 10 , bzr branch lp:notify-osd , I does not supply my lp login yet
[12:13] <awilkins> Sounds like the "defective squid" error
[12:43] <willwill> I directly connected to the internet, no proxy is used except for DNS proxy
[12:55] <igc> night all
[12:55] <beuno> good night igc
[12:55] <beuno> it's great to have you back
[12:55] <beuno> have I said that already?  :)
[12:57] <james_w> night igc, the document looks great, I'll review fully later
[12:57] <james_w> hey beuno
[12:58] <beuno> hi james_w
[12:58] <beuno> how's it going?
[12:58] <james_w> good thanks, you?
[12:58] <igc> thanks beuno
[12:58] <igc> thanks james_w
[12:59] <beuno> pretty good, trying to catch up on work, as always  :)
[15:16] <bialix> hi, I'm looking for some (wise) advice about some almost standard options like --include(-i) and --exclude(-x)
[15:16] <bialix> does --include implies that its argument (list of an items) should be used instead of default one?
[15:17] <bialix> or it can be used as "add this items to default list"?
[15:17] <bialix> fast-import-filter command e.g. used first variant: --include list used as default processing list
[15:18] <bialix> any suggestions?
[15:20] <vila> bialix: I don't remember having ever encountered the second variant, so go with your first variant
[15:21] <bialix> my question related to my scmproj plugin. I have several ways to select components for processing. Yep, the first variant is most logical, and I hope "standard" way to doing this
[15:21] <bialix> vila: thanks
[15:21] <vila> I'm not sure you need a --include option as opposed as accepting arguments to define your list
[15:22] <vila> then, --exclude applies to your arguments *or* the default value for your arguments
[15:22] <bialix> I need --include because I'm using arguments for other purposes
[15:22] <vila> s/as opposed as/as opposed to/
[15:22] <bialix> -i/-x pair should work just fine
[15:24] <bialix> but... I wonder, why bzr uses -i/-x form as short names for include/exclude, while hg uses -I/-X
[15:24] <bialix> IIRC
[15:24] <vila> tar defines: --files-from (files listed into a file) and --exclude-from (files listed into a file too), zip defines -x files and includes files specified as arguments
[15:24] <rexbron> bialix: dumb luck :P
[15:25] <vila> bialix: tar defines -X as an alias for --exclude-from, zip defines -x files...
[15:26] <bialix> rexbron: I don't understand your joke
[15:26] <vila> bialix: And this comes from a case sensitive OS :-)
[15:26] <bialix> vila: I like zip. -x ftw
[15:27] <bialix> no need to hold shift in the end
[15:27] <bialix> ok, bzr is simpler
[15:27] <awilkins> reduced wear on the little-finger tendons
[15:27] <bialix> :-)
[15:27] <vila> bialix: diff defines -x pattern and -X file (containing file names) :-)
[15:27] <bialix> OMG
[15:28] <awilkins> How people use Emacs without getting swingeing tendonitis I'll never know
[15:28] <vila> bialix: on unix, man(1) is your friend :-)
[15:28] <bialix> but in my case it's `diif -x`
[15:28] <awilkins> ctrl-alt-meta-ow-shit-my-hand-is-now-a-claw
[15:28] <vila> awilkins: because they started young and are trained :)
[15:28] <vila> Escape-Meta-Alternate-Control-Shift ftw :)
[15:29] <awilkins> Ah, so you're saying Emacs is the editor for chinese acrobat children?
[15:29] <bialix> and del?
[15:29] <vila> awilkins: You didn't know *that* ?
[15:29] <vila> bialix: real programmers don't need del :-)
[15:30] <bialix> oh
[15:30] <bialix> vila: Putty screwed up Backspace when I'm editing files in nfte on Linux, and have to use Del. This always make my crying
[15:31]  * awilkins gets around this by using Windows via rdesktop on Linux
[15:31]  * bialix has no such luxury
[15:31] <awilkins> Especially since they installed some kind of security product that stops the Windows Vista RDP client working properly
[15:32] <vila> bialix: besides del is achived with Ctrl-d under emacs :)
[15:32] <awilkins> It's really good when you try to open a \\tsclient\ drive and the server BSODs
[15:32] <bialix> ctrl-d deletes selected region for me in fte
[15:33] <bialix> ok, I have answer I've looked for. thanks
[15:34] <vila> bialix: nah, delete-region is C-w :)
[15:34] <bialix> :-D
[15:34] <bialix> I'd better to do not continue this finger battle
[15:35] <vila> hehe, I think being comfortable with emacs *requires* to know at least ~50 key bindings, efficient ~100, hacker mode ~150, after that you'd better learn the commands themselves in case you forget the key bindings :)
[15:36] <cody-somerville> How do I disable a plugin?
[15:36] <vila> cd ~/.bazaar/plugins ; mkdir DISABLED; mv bogus-plugin DISABLED
[15:37] <vila> cody-somerville: why do you need to disable it and what plugin is it ?
[15:37] <cody-somerville> vila, gtk
[15:37] <cody-somerville> vila, it prevents me from using bzr when X isn't available
[15:37] <vila> urgh, is it dbus related *again* ?
[15:38] <cody-somerville> yup
[15:39] <vila> bialix: by the way, qbzr still blows out when I try to run selftest with python-2.6 (for which I don't have pyqt4)
[15:39] <bialix> file a bug please
[15:39] <vila> cody-somerville: I think this has been fixed at least a dozen of times already :-/
[15:40] <bialix> vila: IIRC I dont try to fix selftest w/o PyQt4 yet
[15:40] <vila> bialix: I did and I thought you fixed pretty quickly, only to discover later that the problem was still there
[15:40] <bialix> vila: this is different problem
[15:41] <bialix> I've fixed bug-url command, but you now talks about selftest
[15:41] <bialix> why you run selftest for qbzr if you don;t have qt?
[15:42] <vila> I run selftest and that includes qbzr because it's installed
[15:42] <vila> the problem is that qbzr *aborts* at load time instead of raising some import error or something that disables it without aborting the current command
[15:43] <bialix> please, file a bug about selftest
[15:43] <bialix> I'll fix it shortly
[15:44] <bialix> qbzr has a lot places where we import pyqt4 lazily, but some internal modules are not so lazy
[15:46] <cody-somerville> hmm... I guess it wasn't bzr-gtk that was causing the dbus issue
[15:46]  * cody-somerville tries removing bzr-dbus
[15:47] <bialix> vila: btw my fix is not released yet
[15:47] <vila> bialix: and you think I'm not running qbzr trunk ? :-)
[15:48] <vila> bialix: no time to file bug, here is the fix: http://paste.ubuntu.com/122430/
[15:48] <vila> bialix: :)
[15:48] <bialix> vial: you said about "installed" version
[15:48] <bialix> sorry, vila ^
[15:48] <vila> bialix: tested against 2.5 with pyqt4 and 2.5 without :)
[15:48] <vila> bialix: sorry about what ? Providing qbzr ? You should be proud instead :)
[15:49] <bialix> sorry about typo in your nickname (I wrote vial)
[15:49] <bialix> the main author is luks
[15:49] <vila> bialix: I'm running lp:~qbzr-dev/qbzr/trunk@revno 623
[15:49] <bialix> I'm just hacking it and maintain it
[15:50] <bialix> vila: your fix is quick hack, I'll try to investigate the problem a bit deeper
[15:52] <vila> bialix: sure
[15:52] <vila> bialix: and thanks
[15:52] <bialix> for what?
[15:52] <bialix> if you don't have qt you can't use qbzr
[15:56] <bialix> vila: bug #33389
[15:57] <bialix> no Bug #333892
[15:58] <vila> bialix: thanks ! And blame launchpad for eating the spaces in the patch :-)
[15:58] <bialix> bad lp, bad lp. go to the corner and think about it
[15:59] <beuno> vila, it's the world crisis, we had to cut down on food for servers, so now they're after characters. Just wait until they find out how yummy a's are
[15:59]  * bialix waves
[16:02] <jelmer> whoa
[16:02]  * jelmer is running bzr log on a 190k ooo bzr branch
[16:03] <jelmer> bzr sure doesn't like that
[16:03] <vila> jelmer: 190k revisions I presume ? Having fun ? :-)
[16:03] <jelmer> vila: yeah
[16:03] <jelmer> vila: brisbane-core rocks btw
[16:03] <jelmer> vila, without it it would've taken me days to get this import
[16:03] <jelmer> now it took me around 14 hours
[16:05] <vila> jelmer: good to know, did you talk with igc about it ?
[16:05] <jelmer> vila, no, not yet
[16:05] <jelmer> it takes about 100 seconds before "bzr log" even gives any output..
[16:06] <vila> jelmer: a bare 'bzr log' ? Without any options ? No hidden alias or something ?
[16:06] <jelmer> vila: yep
[16:07] <jelmer> running with lsprof now
[16:08] <vila> jelmer: how many packs ?
[16:08] <jelmer> vila, 30
[16:08] <jelmer> the largest is 640 Mb
[16:09] <NfNitLoop> ... whoah.
[16:10] <jelmer> vila, the problem seems to be that it has to walk the graph all the way down to the bottom
[16:10] <NfNitLoop> Do they have lots of binary files?  (Clip art, logos, etc?)
[16:10] <jelmer> NfNitLoop, they mainly have a big history (270k revisions)
[16:10] <jelmer> vila, and most revisions are on the mainline
[16:11] <vila> jelmer: it shouldn't need to walk the whole graph for a bare 'bzr log'... at the least in common cases...
[16:11] <jelmer> vila: it has to determine the revno's
[16:13] <vila> jelmer: yeah, nevermind, I certainly know that and I mixing potential optimizations with actual code ;)
[16:13] <jelmer> vila: :-)
[16:13] <jelmer> vila: I'll certainly talk to Ian about it
[16:14] <vila> Ian's revno cache plugin could certainly help there, until we implement some lazy revno tricks
[16:14] <jelmer> vila: after the initial wait, it's pretty decent
[16:15] <jelmer> I haven't tried his revno cache plugin yet, I'll give it a try
[16:15] <vila> jelmer: yeah, once we get the graph loaded, log is really streamed
[16:15] <vila> jelmer: me neither but I saw the code passing around :-)
[16:17] <jelmer> vila: same story for "bzr viz" btw, it Just Works except that the initial ancestry browsing is a bit slooow
[16:17] <vila> jam: http://paste.ubuntu.com/122440/ can fix two more failing tests but I'm affraid the approach can be rude for performances, thoughts ?
[16:17] <vila> jelmer: bzr viz lacks the nifty +/- of qlog :-)
[16:18] <jelmer> vila: To quote lifeless: Patches welcome (-:
[16:18] <vila> jelmer: not that it will solve the problem you're referring to :-)
[16:19] <vila> jelmer: hehe, I looked at it once,... and had to work on other things...
[16:19] <vila> jam: come back  ! :)
[16:25] <jelmer> vila: do you know why a push from development5-subtree into 1.9-rich-root would use a lot (1Gb) of memory?
[16:26] <vila> jelmer: in bbc ?
[16:26] <jelmer> vila: yeah
[16:26] <vila> revno 3834 may be ?
[16:27] <jelmer> vila: is that the cause or the solution ?:-)
[16:27] <vila> jelmer: I think it's the cause, but if you don't use it already, let see if it's a solution :-)
[16:28] <vila> jelmer: and it's the cause, we really like to know as it's part of a streamed push whose purpose is to avoid consuming too much memory :-)
[16:29] <vila> jelmer: and *if*it's the cause, we really like to know as it's part of a streamed push whose purpose is to avoid consuming too much memory :-)
[16:36] <jelmer> vila, is it right that that no longer gives any progress indication?
[16:37] <vila> jelmer: right ? no. Known ? I think so. IIUC part of it was lost during the last big merge made by jam
[16:40] <jelmer> thrope, fwiw this is wrong: http://projects.scipy.org/pipermail/scipy-dev/2009-February/011210.html
[16:40] <jelmer> thrope, clones *are* the same, and the bzr metadata is not visible as long as you're running svn >= 1.5
[16:41] <latexer> morning all.
[16:42] <latexer> if bzr blame shows a revision number like "297.2.3" how do I find the "main" revision that introduced the change?
[16:42] <awilkins> jelmer: You don't even need to run svn > 1.5  repository
[16:42] <awilkins> jelmer: Works fine with a 1.4 format repo and 1.5 libraries
[16:43] <awilkins> latexer: You'd need to know which branch that revision came from to find the "main" revision.... or you could just branch it at that revision, and hey presto, you have theat revision
[16:44] <latexer> awilkins: can I at least find the "merge point" that introduced the change?
[16:44] <awilkins> latexer: You have qbzr or bzr-gtk?
[16:45] <latexer> awilkins: I think i've got bzr-gtk installed on this box.
[16:46] <latexer> hrm... nope, I removed it cause it kept spitting warning about not working with my bzr version.
[16:46]  * latexer re-installs.
[16:48] <jelmer> hi enigma42
[16:48] <jelmer> awilkins: Ah, I thought that was specific to the collabnet server
[16:49] <awilkins> jelmer: I've been using file:/// to push branches into a 1.4 repo locally
[16:50] <awilkins> The repo has to be compatible with our server and since I'm the server admin I can't be bothered to upgrade it to 1.5.x
[16:50] <jelmer> awilkins, but does that use file proeprties or revision properties?
[16:50] <awilkins> jelmer: I'll take a look at the repo
[16:50] <awilkins> If you want a copy it's only a couple f meg
[16:51] <latexer> awilkins: ok.... now what? I have a *lot* of revisions in this branch...
[16:51] <latexer> awilkins: the log view doesn't show the revision numbers...
[16:52] <awilkins> jelmer: bzr:file-ids? Seems to be using revprops
[16:54] <jelmer> awilkins: So nothing shows up in "svn diff" wrt properties?
[16:54] <awilkins> On a file across two revisions?
[16:54]  * awilkins looks
[16:59] <enigma42> jelmer: Hello
[16:59] <enigma42> jelmer: Not ignoring you...I had just stepped away from my desk.
[17:00] <jelmer> awilkins, any kind at all (but bzr-specific, of course)
[17:00] <awilkins> Ok, I'm having trouble with the CLI (used to tortoise), and I have to cath my train now, I'll check later
[17:15] <jam> vila: just seeing how things are going for you
[17:15] <enigma42> jelmer: Oh...one thing comes to mind about 0.5...
[17:15] <vila> jam: hi !
[17:15] <enigma42> jelmer: Sometimes, a push or a pull takes *forever*
[17:16] <LarstiQ> latexer: I was hoping to get close with 'bzr log -r 297.2.3 -n1 ', but it doesn't do exactly what I'm looking for
[17:16] <enigma42> When I turn on transport logging, it looks like it is fetching revision information on rev at a time.
[17:16] <enigma42> And for my project, that's several hundred revisions.
[17:17] <enigma42> jelmer: Maybe this is due to having a mix of file properties and revision properties?
[17:17] <jelmer> enigma42, 0.5.2 should be a lot faster
[17:17] <enigma42> (The total push time is about 9 minutes for no changes.)
[17:17] <enigma42> jelmer: OK. Let me grab it and try it out. I didn't realize there was a new release.
[17:17] <enigma42> jelmer: I guess I haven't been checking it daily recently. ;-)
[17:18] <vila> jam: I'm back on bbc fixing tests
[17:20] <jam> vila: sounds good.
[17:21] <jam> How is the current state for bbc?
[17:21] <jam> (Just curious if merging bzr.dev made it a lot better or worse)
[17:22] <vila> jam: Currently working on 'CHKInventory' object has no attribute 'apply_delta'
[17:23] <vila> jam: merging made it a bit worse, but I've already fixed some, we are not *that* far the previous point
[17:23] <jam> vila: k. Just to mention I don't think you want to implement 'apply_delta'
[17:23] <jam> CHKInventory has "create_by_apply_delta" which is what we should be switching to
[17:23] <jam> CHKInventory is meant to be immutable
[17:24] <jam> and you have to create a new one to modify it
[17:24] <vila> jam: I know: http://paste.ubuntu.com/122440/
[17:25] <vila> now it's failing with   p_offset, p_length = self._container_writer.add_bytes_record(
[17:25] <vila> AttributeError: 'NoneType' object has no attribute 'add_bytes_record'
[17:25] <jam> vila: you need to be in "start_write_group()"
[17:26] <jam> aka, repository.lock_write(), repository.start_write_group(), *create_by_apply_delta*, repository.commit_write_group(), repository.unlock()
[17:26] <jam> CHKInventory.create_by_apply_delta() writes bytes down to disk
[17:26] <jam> We don't have a memory-only implementation
[17:26] <jam> In theory we could
[17:26] <jam> but we pun the "finalized form" with "written to disk"
[17:27] <jam> so that we know what pages need to be written, and what not
[17:28] <vila> hmm, the test use branch_builder, does that imply a memory only impl. ?
[17:28] <jam> vila: A plain Inventory can exist nowhere but in memory objects (InventoryFile, etc)
[17:28] <jam> CHKInventory only exists when backed by a repository
[17:28] <jam> whether that repo's bytes are stoored in a MemoryTransport or on actual disk
[17:29] <jam> vila: does that make more sense?
[17:29] <vila> yes
[17:30] <jam> lifeless: when you get online... it looks like git uses 'libxdiff' not 'libxdelta', minor difference, but xdiff seems to be about "Rabin's Polynomial", so I would guess they cribbed the implementations from there.
[17:33] <jam> ah, the thread seems to be here: http://kerneltrap.org/index.php?q=mailarchive/git/2006/4/21/203883/thread
[17:54] <thrope> jelmer: ping
[17:54] <jelmer> thrope, pong
[17:55] <thrope> hello - I saw your point earlier - I am replying to that guy and jsut wanted to check something
[17:55] <thrope> for me bzr branching etc. works fine without any bzr metadata in the svn repo
[17:55] <chx> I read http://bazaar-vcs.org/Documentation/LoomAsSmarterQuilt and I like it. "Thus, if a partial commit is performed, I first shelve any remaining changes before going up-thread: " -- what does it mean by partial commit?
[17:55] <jelmer> thrope, yes, that's correct - and will always create the same bzr repository
[17:55] <thrope> so when is that needed? bzr stuff in the svn repo is only needed when you are pushing branches back?
[17:56] <thrope> or you want to keep track of bzr side histroy of commits you are pushing to svn
[17:56] <jelmer> thrope, yes, and that's necessary for the 1-to-1 mapping he mentions as not existing
[17:57] <jelmer> thrope, the metadata is *optional* when pushing back into svn, but mandatory if you would like to push the exact same revision, including all bzr metadata
[17:57] <thrope> right
[17:57] <thrope> is this correct: for example, one can checkout from subversion using bzr-svn, branch using bazaar, modify, and push back to a different location in svn and it will be a svn branch
[18:00] <jelmer> yes, that's correct
[18:02] <thrope> ie svn copies
[18:02] <thrope> with history preserve
[18:03] <rawler> hey people..
[18:03] <Peng_> Hi
[18:03] <rawler> we're a bunch of people using bzr for various projects, with mostly very good experiences..
[18:04] <Peng_> That sounds kind of secretive.
[18:04] <rawler> now, however we've seen our first challenge.. we're starting up a new project, with a bunch of new people, who have never used version control, or even linux command-line before..
[18:04] <Peng_> Sorry, Peng_ watches too much TV. Anyway..
[18:04] <jelmer> thrope, yes
[18:05] <rawler> Peng_: heh.. not really a secret at all.. just trying to not having to list the 10-15 project dirs controlled by BZR.. ;)
[18:06] <rawler> so, my question is: is there any really good book or booklet (preferably in printed form), that goes through the very basics of bzr source control, without requiring too much previous knowledge on POSIX shells and such?
[18:07] <rawler> I.E. if someone expects them to know in advance what a unified diff is, they will probably run for the hills.. ;)
[18:09] <jelmer> vila, not much better :-(
[18:09] <vila> jelmer: context ?
[18:09] <jelmer> vila: push from bbc to 1.9-rr
[18:10] <jelmer> vila, still uses about 1g of RAM and takes around 5 minutes to push 150 revisions
[18:11] <vila> jelmer: both repos are local or ?
[18:11] <jelmer> vila: yes, both local
[18:12] <thrope> jelmer: would  you say this is accurate ? http://pastebin.com/m7e84dfb7
[18:13] <jam> jelmer: what is the repository?
[18:13] <jam> Also, last I checked bbc isn't always rich-root
[18:13] <jam> (--dev5 is *not* IIRC)
[18:13] <jam> and neither are the current hash forms
[18:13] <jam> but regardless, pushing from bbc => 1.9* is going to have to do a conversion
[18:13] <jelmer> jam, dev5-subtree
[18:14] <jam> k
[18:14] <jelmer> jam, the repository is here: http://people.samba.org/bzr/jelmer/ooo/trunk
[18:14] <jelmer> http://people.samba.org/bzr/jelmer/ooo/ooo/trunk/
[18:14] <jam> jelmer: so did it convert in ~20hrs ?
[18:15] <jelmer> jam: it got up to 190k in 12 hours, after that I stopped it since I wanted to do actual work :-)
[18:15] <jam> out of?
[18:15] <jelmer> jam: out of a little less than 270k
[18:23] <jelmer> thekorn, I'm not sure what you mean exactly by "it will be a svn branch"
[18:23] <jelmer> s/thekorn/thrope/
[18:24] <jelmer> thrope, also, it's possible to push without those bzr properties, it just means you don't get the 1-to-1 mapping
[18:25] <thrope> jelmer: i mean the files will be svn copies - ie preserve history from the originals
[18:26] <thrope> thanks for your help
[18:26] <jelmer> thrope, other than that, seems right to me
[18:29] <enigma42> jelmer: About push and pull slowness....
[18:29] <enigma42> jelmer: 0.5.0 took about 9 minutes, 0.5.2 took about 2m15s.
[18:29] <enigma42> jelmer: So, it's much faster.
[18:30] <jelmer> enigma42: great :-) 0.5.3 should be significantly faster again, once it's released
[18:31] <enigma42> Cool!
[18:36] <NfNitLoop> yay!
[18:40] <ronny> jelmer: is dulwich already able to generate commits (preferable without the need of a workdir)
[18:40] <jelmer> ronny, yes
[18:40] <jelmer> ronny, and without the need for a workdir; in fact, we can't generate a commit for a workdir yet
[18:41] <ronny> oh, thats good
[18:41] <ronny> i need some kind of backup app, and a git repo will do fine
[18:46] <jelmer> vila: whoa, bzr-revnocache reduces the "bzr log" startup time on OOo from >100s to 0s
[18:47] <beuno> yeah, igc's a genius
[18:47] <vila> jelmer: told you ! :-) Seriously, I had no idea, that's great !
[18:47] <beuno> it does magic with loggerhead
[18:48] <beuno> haven't been able to work in the changes needed to use it
[18:48] <beuno> but experiments show amazing improvements
[18:48] <jelmer> would also be nice if bzr-gtk could use it..
[18:48] <Peng_> bzr-revnocache stores data in .bzr, doesn't it?
[18:48] <beuno> yes
[18:49] <Peng_> Does it use a significant amount of disk space?
[18:49] <Peng_> (Although I have lots of bzr-search indexes anyway, so who am I to care about that? :P )
[18:49] <jelmer> Peng_, 12Mb for 190000 OOo revisions
[18:49] <beuno> jelmer, it needs to be updated to use the new API. I guess if I manage to get to it for LH, I can do the same for bzr-gtk
[18:51] <jelmer> beuno, that would be awesome
[18:51] <Peng_> I guess I might as well install it, then?
[18:52] <beuno> jelmer, I peaked at the code briefly, and it's roughly the same changes for both. Maybe bzr-gtk is easier
[18:52] <beuno> Peng_, go for it, although LH will probably not use it
[18:52] <beuno> it calls the old APIs, etc
[18:52] <Peng_> I don't think my Loggerhead install has permission to write to .bzr anyway.
[18:53] <beuno> right, that's one of the changes I need to make
[18:53] <beuno> configurable caching dir
[18:54] <jelmer> bzr-search could use that as well
[18:54] <beuno> yeah, also on my ToDo list
[18:54] <beuno> once I get that, and auto-scan-new-branches
[18:54] <beuno> I want to make bzr-search a dependency for LH  :)
[18:54] <beuno> well, that and allowing you to turn it off, of course
[18:56] <ronny> hmm
[18:57] <Peng_> FWIW, I don't want LH to do automatic indexing, but I do want it to use bzr-search indexes when they're available.
[18:58] <ronny> i'll have to steal some ideas from lh later
[18:58] <beuno> Peng_, the current behavior
[18:59] <beuno> ronny, that's what open source is about  ;)
[18:59] <ronny> im still not at the point where anyvc is ready to power a web interface
[18:59] <Peng_> beuno: Right. If you add automatic indexing, please don't remove the current behavior. :)
[18:59] <beuno> Peng_, of course not, I will make it a non-default optional setting
[19:01] <Peng_> beuno: Thank you.
[19:01] <ronny> hmm
[19:02] <ronny> it kinda sucks that git lacks a smart server one could mimic over wsgi
[19:13] <jam> jelmer: I'm repackaging 1.12 so that I can remove the ugly dll that is causing problems
[19:14] <jam> Should I include svn 0.5.2 ?
[19:16] <jam> also, jelmer, why are your branches no longer mirrored on Launchpad?
[20:02] <Peng_> jam: So that people can't do "bzr branch lp:bzr-svn" and get an out-of-date copy.
[20:02] <Peng_> jam: Last I checked, the mirrors still exist, under names like lp:~jelmer/bzr-svn/0.5-mirrored.
[20:03] <jam> Peng_: of course it breaks all my build scripts...
[20:04] <jam> which don't really care about getting the *latest*, just want to get an older tag
[20:04] <jam> not to mention that people.samba.org was broken a couple times recently...
[20:04] <kfogel> are revprops a new feature?  the string "revprop" does not appear in the Users Guide.
[20:04] <jam> kfogel: possibly under "revision properties". Not new, but not really exposed to users, either.
[20:04] <jam> I wouldn't expect to see them in a "Users Guide"
[20:04] <jam> (--author happens to be implemented using it, though)
[20:05] <kfogel> jam: not under "revision properties" either, but okay, I didn't realize they weren't exposed.
[20:05] <jelmer> jam, yes, 0.5.2 would be nice
[20:05] <jelmer> jam, sorry about the branches, I'm surprised the remote branches stuff doesn't allow -r..
[20:06] <jam> jelmer: well, it would probably have worked, but by checkouts were from "bazaar.launchpad.net" and those branches disappeared
[20:06] <jam> plus that machine has bad DNS, so I have to manually enter each ip address
[20:06] <jelmer> jam, Yeah, it seems to work with just "lp:bzr-svn"
[20:06] <jam> for now I just rebound to 0.5-mirrored
[20:18] <kfogel> LarstiQ: hey, did you see that mail about bzr-email-notifier on the list?  The project that appears to want to be a superset of bzr_hookless_email?  Probably good if you respond and advise the developer whether to try to merge or to do an independent project.
[20:25] <LarstiQ> kfogel: I've not been reading my email for the last ~2 weeks (been away from home), thanks for the heads up
[20:26] <kfogel> LarstiQ: want me to follow up in the thread saying your attention has been drawn and you will be replying?
[20:26] <LarstiQ> kfogel: that'd be swell :)
[20:26] <kfogel> LarstiQ: doing now, np
[20:29] <lifeless> spiv: when you see this; sometimes sleeping on the problem helps :)
[20:29] <lifeless> http://paste.ubuntu.com/122552/
[20:32] <jam> hi lifeless
[20:35] <lifeless> hi jam
[20:37] <lifeless> jam: I have a full test running; if you want to chat we could do so no
[20:37] <lifeless> w
[20:38] <jam> lifeless: I didn't get much done on compression today.
[20:38] <jam> Today was packaging win32 installers, looking into python.org questions
[20:38] <jam> etc
[20:38] <jam> Like changing "time bzr branch --stacked" from 67m down to 4m30s...
[20:38] <Peng_> 67m?
[20:39] <jam> Peng_: we currently build the working tree by requesting each file's content separately
[20:39] <lifeless> Peng_: get_record_stream was being overly conservative on memory
[20:39] <Peng_> Ah.
[20:39] <jam> so 4k files == 4k ish round trips at 100ms each
[20:39] <lifeless> jam: thank you! :)
[20:39] <jam> lifeless: Well, I did it by cheating, and I don't have great answer as to how to stream the content off correctly.
[20:39] <jam> I do have 1 patch which is obvious for bzr.dev
[20:40] <lifeless> jam: cheating - back to one big request?
[20:40] <jam> lifeless: yeah
[20:40] <jam> I just wanted to see where we would get to
[20:40] <lifeless> jam: components_positions has the sizes
[20:40] <lifeless> I suggest summing stuff from in there, or something
[20:41] <jam> lifeless: 'bzr branch --stacked bzr+ssh://" is 3m39s' which is nice
[20:41] <jam> I'm not sure if it is the max 200-range request issue
[20:41] <jam> or if it is because of the new streaming
[20:43] <lifeless> new streaming ?
[20:43] <jam> lifeless: One is a smart server, rather than a dumb request
[20:44] <jam> lifeless: so I don't know if the 4m40s => 3m40s is because of the smart server
[20:44] <jam> or because of better readv() handling
[20:44] <lifeless> oh
[20:44] <jam> Our HTTP code has a hard-cap of 200 Range requests
[20:44] <lifeless> readv() I would suspect
[20:44] <jam> because Apache gives errors at 400 Range requests
[20:44] <jam> our bzr+ssh code just has a hard-cap at ~5MB
[20:44] <jam> but 5MB >> .5MB I was seeing for HTTP
[20:48] <jam> lifeless: I'm seeing the "get_parent_map()" calls of death on a 'bzr push' for a bzr branch
[20:48] <jam> (to a stacked branch on Launchpad)
[20:48] <jam> so it isn't specific to lp trees
[20:48] <jam> this is with bzr.dev tip
[20:49] <jam> well 4039, close enough
[20:52] <lifeless> jam: spivs fix hasn't landed
[20:52] <lifeless> jam: I imagine it failed in some way
[20:52] <jam> lifeless: k. I thought I saw him mention he sent in a fix
[20:52] <jam> lifeless: can you point me to the patch?
[20:52] <lifeless> yeah, its not in the log though
[20:53] <jam> I haven't seen a specific patch for it on the bzr mailing list
[20:53] <jam> though maybe I just haven't found it
[20:53] <lifeless> yah I reviewed in person
[20:53] <lifeless> http://people.ubuntu.com/~andrew/bzr/bug-331823/
[20:57] <jam> so is the actual fix just getting rid of "update_revisions" ?
[20:58] <lifeless> yes
[20:59] <jam> anyway, it works here :)
[20:59] <jam> stopped a process after 11min, new one finished in 17s
[20:59] <lifeless> yeah
[20:59] <mwhudson> gaaaaar, depending on other people's software is terrible
[20:59] <jam> jml: so it looks like spiv's fix will probably work for your launchpad pushing woes, as long as we can actually get it merged :)
[21:00] <lifeless> it was intended to land yesterday
[21:02] <jam> lifeless: I did manage to spend a little bit of time looking through the git 'repack' code
[21:02] <jam> some interesting bits
[21:03] <jam> like the max allowed delta size changes based on how long the delta chain is
[21:04] <jam> it is a bit hard to understand how it is defined, but they will stop generating a delta early
[21:04] <jam> if it exceeds the expected length
[21:18] <jelmer> hi davidstrauss
[21:18] <davidstrauss> jelmer: hi
[21:20] <jml> jam: sweet
[21:23] <Tak> mwhudson: s/depending on // :-P
[21:23] <mwhudson> Tak: if i don't depend on it, i can blissfully ignore it :)
[21:24] <tdomhan> can I somehow delete all revisions before a specified revision in bzr?
[21:26] <jam> lifeless: are we doing the conference call via phone again today?
[21:27] <lifeless> jam: I hope not
[21:28] <jam> lifeless: I thought other people said they preferred it
[21:29] <lifeless> jam: opinions vary; I find it less convenient and worse audio
[21:31] <mwhudson> skype has better audio when you can hear anything at all
[22:08] <Peng_> Bazaar is supposed to work right when a local branch is read-only, right?
[22:09] <Odd_Bloke> Peng_: Locking might be a problem.
[22:09] <lifeless> Peng_: yes, it should
[22:09] <lifeless> Odd_Bloke: if it is is a bug
[22:10] <Odd_Bloke> Yeah, I was about to say I don't know about the 'supposed to'. :)
[22:10] <Peng_> Just checking. Thanks.
[22:10] <Peng_> (I'm not having any problems with bzr itself, just bzr-revnocache.)
[22:29] <james_w> poolie: confirmed today
[22:30] <poolie> yay
[22:30] <jelmer> igc, 'morning
[22:31] <poolie> james_w, put up your details when you get a chance please in https://wiki.canonical.com/Bazaar/Sprints/Brisbane09?action=diff&rev1=13&rev2=14
[22:31] <garyvdm> Hi - I'm trying to use rebase for the first time. I'm trying rebase the revisions 631..624 on to the tip of trunk
[22:31] <james_w> poolie: done
[22:31] <james_w> hey jelmer
[22:31] <garyvdm> I did bzr rebase -r 624 ../../trunk/
[22:31] <jelmer> hey James
[22:31] <garyvdm> It did nothing.
[22:32] <garyvdm> What am I doing wrong
[22:32] <igc> hi jelmer
[22:32] <igc> nice to hear revnocache is helping OOo log speed
[22:32] <jelmer> igc: Yeah, I was about to mention that :-)
[22:32] <jelmer> igc, revnocache rocks, and so does the development5-subtree format :-)
[22:33] <igc> I still feel making --short the default is the right thing from both a usability and performance perspective but I'll leave that battle till later :-)
[22:33] <jelmer> igc: I hope to have a full import of OOo tomorrow, but that will be in development5-subtree
[22:33] <igc> jelmer, so tell me more about why development5-subtree was so much faster at importing
[22:34] <igc> I'm not seeing big gains in fastimport yet
[22:34] <jelmer> igc: Well, the original comparison was against rich-root-pack, not 1.9-rich-root
[22:34] <jelmer> igc: I'm also using create_by_inventory_delta
[22:35] <igc> jelmer:that's good but you must be doing other stuff too
[22:35] <igc> create_by_inventory_delta is typically wrapped by higher levels repository APIs isn't it?
[22:35]  * igc looks
[22:36] <jelmer> igc: previously I was using apply_delta()
[22:37] <jelmer> which means a lot of time is spent copying and serializing inventories
[22:38] <jelmer> igc: In fact, pushing development5-subtree back into 1.9-rich-root is probably about as slow as importing into 1.9-rich-root directly
[22:41] <igc> jelmer: I'll get back to fastimport in a day or two and take another look at this
[22:42] <lifeless> igc: create_inventory_by_delta is a huge win
[22:42] <igc> jelmer: add_inventory_by_delta *looks* like its building a full new inventory but I suspect it's being smarter than that down in the ChkInventory, in which case the serialisation time should be much better
[22:43] <jelmer> igc: yeah, it's a lot smarter than that
[22:44] <igc> jelmer: thanks so much for working on the OOo import and keeping the conversation going with Jens
[22:44] <igc> jelmer: back to revnocache, it's worth mentioning that's its still pretty raw
[22:45] <igc> there's a TODO file in the plugin that I'm hoping mwhudson, beuno and others can use to get inspired and hack on it further :-)
[22:46] <jelmer> igc: ah
[22:46] <jelmer> igc: but are there plans to eventually have some of this in the cocore?
[22:46] <jelmer> s/cocore/core/
[22:46] <mwhudson> i want to work on incremental update,
[22:46] <igc> jelmer: the trick to using revnocache is to update code to use Branch.iter_merge_sorted_revisions()
[22:47] <igc> jelmer: I do want put of revnocache in the core but it's a part I haven't written yet ...
[22:47] <igc> namely fast mapping of dotted-revno to revision-id by using a mergeline-cache
[22:48] <jelmer> igc: looking forward to it :-)
[22:48] <igc> unlike the full sorted merge graph, that cache can be small and (hopefully) cheap to update
[22:49] <igc> jelmer: the biggest problem with revnocache is that it's a space pig - it stores the sorted merge cache *per* branch
[22:50] <igc> if and when we can start chaining caches so that most the data is only cached in the local mirror/trunk branch with
[22:50] <igc> incremental changes stored per branch, that issue will largely go away
[22:51] <mwhudson> also, btrees will be a lot smaller i guess
[22:51] <igc> mwhudson: right
[22:57] <jelmer> igc: ah, hmm
[22:57] <jelmer> igc: for OOo it's managable but that's because they have a huuuuge mainline and not a lot of branches
[22:58] <dirkD> i just installed bzr-gtk from EPEL (on Centos 5), but how do i enable the Bazaar integration in Nautilus?
[22:58] <garyvdm> igc: what is the difference between mergeline-cache and  full sorted merge graph?
[22:58] <lifeless> jml: spiv: the fix for the bug is on its way now
[22:58] <jml> lifeless: cool.
[22:59] <garyvdm> igc: I assume that full sorted merge graph is what you get from merge_sorted_revisions()?
[23:01] <jelmer> dirkD, install python-nautilus
[23:01] <jelmer> dirkD, if it's doesn't work after restarting nautilus, then it probably is a bug in the RPM
[23:02] <garyvdm> jelmer: Do you think a similar architecture to tbzr help the Nautilus integration perf problems?
[23:02] <dirkD> jelmer: what's the name of python-nautilus in EPEL or Centos?
[23:03] <jelmer> dirkD, no idea, sorry - the bzr-gtk package should already depend on it, or suggest it
[23:03] <dirkD> gnome-python2 maybe?
[23:03] <jelmer> garyvdm, perhaps
[23:03] <jelmer> dirkD, if the bzr-gtk package doesn't depend on it, it probably doesn't install nautilus-bzr
[23:05] <dirkD> jelmer: i think it's installed: i have "/usr/lib64/python2.4/site-packages/bzrlib/plugins/gtk/nautilus-bzr.py"
[23:06] <jelmer> dirkD, it needs to be installed in the nautilus extensions directory
[23:07] <jelmer> something like /usr/lib/nautilus/python
[23:09] <dirkD> now the monkey comes out of the sleeve, i only have  /usr/lib64/nautilus/extensions-1.0
[23:15] <jelmer> ah
[23:19] <lifeless> argh
[23:19] <lifeless> this test_source is killing me
[23:20] <lifeless> honestly, who the hell cares
[23:23] <lifeless> we've wanted to get to fast deltas between trees for a couple years now
[23:24] <lifeless> and the branch to do that is only now closing in on completeness
[23:24] <lifeless> bah, ECHANNEL
[23:27] <dirkD> jelmer: should i put the *contents* of the gtk folder (/usr/lib64/python2.4/site-packages/bzrlib/plugins/gtk/nautilus-bzr.py etc.) or the whole gtk folder in ' /usr/lib64/nautilus-python/'?
[23:28] <jelmer> dirkD, just nautilus-bzr.py
[23:28] <jelmer> dirkD, but the package should be doing that for you..
[23:28] <dirkD> ok
[23:29] <dirkD> well, it doesn't (i even removed, and re-installed bzr-gtk after installing nautilus-python)
[23:30] <jelmer> dirkD, sounds like a bug in the packaging
[23:31] <dirkD> yes
[23:31] <dirkD> jelmer: how would i know if the extension works?
[23:31] <jelmer> dirkD, it'll show extra icons on bzr-managed files
[23:31] <jelmer> hmm, autopack now seems to be taking 20% of the time of a svn-import now :-/
[23:32] <lifeless> jelmer: cool
[23:33] <ronny> is it possible to disable autopack?
[23:34] <jelmer> ronny, afaik, yes
[23:34] <jelmer> lifeless, so I'm wondering what the best approach is; I write a pack (commit_write_group) every 500 revisions at the moment
[23:34] <jelmer> lifeless, however, that doesn't seem optimal anymore once the pack files are close to a Gb in size
[23:34] <lifeless> ronny: its not a good idea, but for some specific applications [imports] it makes sense, so there is no UI, but there is a facility
[23:35] <ronny> lifeless: thats why i asked
[23:35] <jelmer> lifeless, would writing a pack file every 500 revisions and then later on triggering a single autopack have any negative influence on performance?
[23:35] <lifeless> jelmer: its unlikely too as our delta chains are hard capped at 200 anyway
[23:36] <dirkD> jelmer: the extension still doesn't work :( is there a way to debug it?
[23:36] <lifeless> jelmer: btw, write 999 revisions
[23:36] <lifeless> the pack count is log10 based
[23:36] <jelmer> lifeless, what's unlikely /too/ ?
[23:36] <lifeless> packs of 1 for 1-9
[23:37] <jelmer> dirkD, it should print something to stdout when you load it in nautilus
[23:37] <lifeless> its unlikely to have a negative performance impact doing a single autopack with that many revisions in a pack
[23:37] <igc> jelmer: right, but the moment they start feature branching (which is a core part of their "cws" workflow), revnocache will grab the disk space for that branch (once a revno off the mainline is needed)
[23:37] <lifeless> packs of 10 for 10, 20, ... 90
[23:37] <lifeless> jelmer: so 999 revisions are permitted 27 pack files; you'll trigger less autopacks simply by doing that
[23:38] <jelmer> lifeless, ok, will change that
[23:38] <lifeless> jelmer: 500 revisions will be autopacking every 2 packs
[23:38] <lifeless> (5, 10 -> 1 pack)
[23:38] <igc> garyvdm: yep, the mergegraph-cache is the serialised output from merge_sorted_revisions()
[23:38] <lifeless> 15, 20 -> 2 packs
[23:38] <lifeless> 25, 30 -> 3 packs
[23:38] <dirkD> jelmer: wait.... *i* should load it?
[23:38] <lifeless> jelmer: changing to 499 would work too
[23:39] <jelmer> dirkD, no, nautilus should load it
[23:39] <bob2> lifeless: why 200? robustness?
[23:39] <dirkD> jelmer: ok, it doesn't (just 'Initializing nautilus-open-terminal extension')
[23:40] <jelmer> dirkD, my guess would be that it's not in the right directory
[23:40] <dirkD> /usr/lib64/nautilus-python/nautilus-bzr.py
[23:40] <jelmer> lifeless, ok, I've changed it to 999
[23:40] <garyvdm> dirkD: run nautilus from the command line and see if it prints anything.
[23:40] <lifeless> bob2: arbitrary figure
[23:40] <dirkD> garyvdm: yes, i did, "[root@server1 ~]# nautilus -q"       "Initializing nautilus-open-terminal extension"
[23:41] <jelmer> dirkD, In my case it's in /usr/lib/nautilus/extensions-2.0/python
[23:41] <igc> garyvdm: the proposed mergeline cache is basically (base-revno, length, tip-revision) ...
[23:41] <jelmer> lifeless, so, just to double-check:
[23:41] <igc> e.g. 1.23, 12, xxxx
[23:42] <jelmer> lifeless, there's no significant performance influence in turning off autopack during imports and autopacking only once at the end of the import ?
[23:42] <igc> so finding the revision for 1.12.11 (say) will have much the same speed as finding revno:-2
[23:42] <lifeless> jelmer: as long as you don't end up with massive scatter-gather IO on getting basis texts, it is fine
[23:42] <garyvdm> igc: ic
[23:42] <lifeless> jelmer: if you have delta chains spread across very many packs, its a problem.
[23:42] <igc> i.e. we can match the base revno (1.12), find it's tip and search backwards from there
[23:43] <jelmer> lifeless, hmm
[23:43] <garyvdm> igc: I'm interested in this cause I can maybe use it to speed up qlog
[23:43] <igc> that will be much faster that building/loading the whole revision graph and doing a map lookup
[23:43] <jelmer> lifeless, thanks
[23:43] <ronny> jelmer: are there any higher level apis for building trees/commits for dulwich?
[23:44] <jelmer> ronny, other than bzr-git, not yet
[23:44] <igc> garyvdm: well, I think qlog ought to call a new log api I'm working on called iter_log_revisions()
[23:44] <igc> See my patch on logging multiple directories for the details
[23:45] <dirkD> jelmer: a strace gave some nice info: "open("/usr/lib64/nautilus/extensions-1.0/python", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOENT (No such file or directory)"
[23:45] <igc> Until that lands - I need to resubmit it following vila's review - the trick is to call Branch.iter_merge_sorted_revisions()
[23:45] <ronny> jelmer: so i would have to use bzr-git to build things like commits for dulwich?
[23:45] <igc> garyvdm: ^^^
[23:45] <jelmer> ronny, well, what do you mean by a higher-level API exactly?
[23:45] <james_w> dirkD: what version of nautilus are you using?
[23:46] <jelmer> dirkD, looks like you want that dir then
[23:46] <garyvdm> igc: Unfortunately qlog relies it's copy of graph_parents to work out what lines it needs
[23:46] <igc> garyvdm: the mergeline cache will only speed up "log -rx.y.z"; otherwise you'll want what revnocache does now
[23:46] <dirkD> james_w: 2.16.2
[23:46] <lifeless> fooding
[23:46] <lifeless> spiv: eta?
[23:47] <james_w> dirkD: and how did you install the nautilus-python extension?
[23:47] <garyvdm> igc: I'll have to make that lazy before I'll be able to take advantage of revnocache
[23:47] <ronny> jelmer: 2 things i need to do- 2. iterate the paths a tree builds to blobs, so i can infer a sha1 sums file, 2. updating the current tree from a tarball that only has the changed blobs
[23:47] <dirkD> james_w: from source
[23:47] <spiv> lifeless: not sure, have been fighting the suddenly flaky wireless for a chunk of the morning.  Will SMS.
[23:47] <lifeless> spiv: ok
[23:48] <dirkD> ah, got it: http://pastebin.com/m78634a99
[23:48] <lifeless> spiv: fwiw my wireless still works :)
[23:48] <ronny> hmm
[23:48] <ronny> i really love that content-addressing stuff
[23:48] <james_w> dirkD: it seems to have picked up the wrong nautilus extension dir, is it hardcoded in the build system?
[23:50] <dirkD> james_w: of bzr-gtk?
[23:50] <james_w> dirkD: no, python
[23:51] <jelmer> ronny, There's no functions for that at the moment, but the functions required to implement them are there
[23:51] <dirkD> james_w: it's in the makefile
[23:53] <james_w> dirkD: is there any mention of the -1.0 path?
[23:53] <james_w> if not then you will need an older nautilus python
[23:53] <dirkD> yes, it is :)
[23:55] <igc> poolie: the benchdata directory on orcadas now has pack & btree archives for mysql & bzr
[23:55] <dirkD> james_w: could this be relevant? http://pastebin.com/m30f48386
[23:55] <igc> I've added bzr.ini and mysql.ini files as well showing how the indirection works
[23:55] <igc> poolie:^^
[23:56] <lifeless> igc: indirection ?
[23:57] <igc> lifeless: picking the best pre-upgraded (branch) archive based on the tool name usertest is benchmarking
[23:57] <james_w> dirkD: maybe
[23:58] <james_w> dirkD: your strace shows that you haven't got nautilus-python installed for the right nautilus ABI, you need to fix that to have any chance of this working
[23:58] <dirkD> PYTHON_LIBS = -L/usr/lib -lpython2.4
[23:58] <dirkD> PYTHON_LIB_LOC = /usr/lib
[23:58] <igc> lifeless: that way, usertest benchmarks the operations without taking 10-20 hours to upgrade to the right format first
[23:58] <igc> (well, for development5 formats on big projects at least)