[00:08] <jml>  Launchpad uses Bazaar  1.17.
[00:08] <lifeless> grats
[00:13] <mwhudson> jml: yay
[00:18] <jelmer> lifeless: I think I may've missed something - where would that be necessray?
[00:19] <jelmer> mwhudson: hi
[00:19] <lifeless> jelmer: test suites ask the test result whether to stop
[00:19] <lifeless> to allow aborting a run early
[00:20] <jelmer> mwhudson: It was interesting to finally see the lp code :-)
[00:20] <jelmer> mwhudson: I noticed a trivial thing that could be fixed, lp:~jelmer/launchpad/default-rich-root
[00:20]  * jelmer looks again
[00:21] <mwhudson> jelmer: oh i suspect i didn't fix that because it would actually downgrade all the bzr-git branches now
[00:21] <jelmer> mwhudson: it would only affect new ones, but that's a good point..
[00:22] <mwhudson> jelmer: if it's the change i think it is, not true
[00:22] <mwhudson> launchpad rollout issues, full attention in a bit :)
[00:23]  * SamB wonders where launchpad is rolling to
[00:25] <spiv> mwhudson: it's this change: http://bazaar.launchpad.net/~jelmer/launchpad/default-rich-root/revision/8303 :)
[00:26] <mwhudson> jelmer: http://bazaar.launchpad.net/~jelmer/launchpad/default-rich-root/annotate/8303/lib/lp/codehosting/codeimport/worker.py#L65
[00:26] <spiv> jelmer: also, congrats, that's probably the first launchpad branch from a non-canonical LP dev that I've seen :)
[00:28] <jelmer> mwhudson: ahh, sorry
[00:28] <mwhudson> jelmer: np, the comment is misleading
[00:28] <mwhudson> jelmer: also the code should probably be changed to do a reimport rather than an upgrade for the brisbane-core transition
[00:29] <jelmer> mwhudson: yeah
[00:36] <igc> morning all
[00:36] <jelmer> hi Ian
[00:40] <lifeless> mwhudson: why reimport?
[00:40] <lifeless> mtaylor: perl!
[00:41] <mwhudson> lifeless: (a) quicker probably (b) the packs imports have squashed commit messages
[00:41] <mtaylor> lifeless: muah!
[00:44] <mtaylor> lifeless: question on "best" way to figure out a specific chunk of bzr data...    I have a local branch... how do I find the GCA between the local branch and the submit branch? (like, the rev from which a revision bundle would start)
[00:45] <mwhudson> there's the ancestor: revspec
[00:47] <lifeless> mwhudson: will it get the same revids? if so, and the root conversion isn't identical to the conversion logic, then its a problem
[00:47] <mwhudson> lifeless: yes
[00:48] <mwhudson> lifeless: they're in rich root already
[00:48] <lifeless> mtaylor: as mwhudson says. do you want command line or python?
[00:48] <lifeless> mwhudson: ok, if they are already rich root its fine
[00:48] <mwhudson> lifeless: i'm only talking about bzr-git imports here
[00:48] <mwhudson> lifeless: not cscvs ones
[00:48] <lifeless> mwhudson: converting from packs should be faster than a reimport btw
[00:48] <mwhudson> for bzr-git
[00:48] <mwhudson> ?
[00:48] <lifeless> I'd expect so
[00:48] <mwhudson> anyway, the fidelity is the real reason
[00:48] <mtaylor> mwhudson, lifeless: ah - sorry - irc client misbehaved. command line
[00:49] <jelmer> lifeless: last I checked an import from git is faster than an upgrade
[00:49] <lifeless> jelmer: thats surprising
[00:49] <lifeless> mtaylor: do you want the revid or revno?
[00:50] <lifeless> mtaylor: there is --show-base to some commands
[00:50] <mtaylor> lifeless: well, I think both would eventually be nice ... but revid would be probably more useful
[00:54] <mtaylor> lifeless: on the other hand, if I need to use bzrlib to do this, I can live with that
[00:54] <lifeless> mtaylor: so, for the specific case of where a bundle will start at
[00:54] <lifeless> I think its
[00:54] <lifeless> bzr revision-info -r submit:
[00:55] <lifeless> mtaylor: also, perl!
[00:56] <jelmer> lifeless: upgrade seems to be a lot better now
[00:57] <jelmer> lifeless: unscientfic tests: bzr-git 8s, upgrade 5s
[00:58] <mtaylor> lifeless: that gives me ... hrm.
[00:59] <mtaylor> lifeless: that just gives me my head revision
[00:59]  * mtaylor looking further...
[01:01] <spiv> poolie: good morning.
[01:01] <poolie> hello spiv :)
[01:01] <poolie> hello lifeless, all
[01:02] <jelmer> hi poolie
[01:02] <garyvdm> Hi poolie
[01:04] <mtaylor> lifeless: so, I think revision-info is giving me the right thing from one perspective- I think it's giving me the GCA of the local branch that it shares with the remote...
[01:04] <mtaylor> lifeless: but what I want is the GCA in the context of the remote branch...
[01:04] <mtaylor> oh wait
[01:04] <mtaylor> lifeless: nevermind... /me smacks self in head
[01:06] <spiv> jam: ping -- want to review my inventory-delta branch?
[01:18] <garyvdm> When you do a merge, is the revision that is used to get .BASE the same for all files, or can it be different?
[01:18] <lifeless> by default the same
[01:18] <lifeless> but there are options that can make it different
[01:19] <garyvdm> Like a different merge type?
[01:19] <lifeless> yah
[01:20] <garyvdm> Is it the same with --lca?
[01:26] <SamB> so ... how come "bzr upgrade" to --rich-root-pack or --2a seems to go nearly to the end of the progress bar pretty quickly, but then stay near the end for a long time ?
[01:26] <lifeless> I'm not sure garyvdm
[01:26] <lifeless> SamB: Possibly incorrect phases, or that a single logical step has more work to do
[01:27] <garyvdm> lifeless: I need to investigate that for something I'm doing for qbzr, and figure how I'm going to handle --weave...
[01:27] <SamB> [#################\  ] Copying content into repository.:Transferring revisions
[01:28] <garyvdm> lifeless: As allways - Thanks for the pointers
[01:28]  * SamB wonders what causes the bar to stop spinning
[01:28] <lifeless> ask instead what makes it spin at all
[01:34] <poolie> hello lifeless
[01:36] <SamB> 6611.394  Auto-packing repository <bzrlib.repofmt.pack_repo.RepositoryPackCollection object at 0x8cf81cc>, which has 15 pack files, containing 15000 revisions. Packing 10 files into 1 affecting 1000 revisions
[01:42] <igc> lifeless: thanks. Raised as bug 403272
[02:11]  * garyvdm is writing a reply to "Nubee question"
[02:11] <SamB> what was that Pyrex alternative?
[02:11] <SamB> Cython?
[02:11] <lifeless> yes
[02:15] <JoaoJoao> hello
[02:16] <JoaoJoao> is there a command to rename a bzr branch
[02:16] <JoaoJoao> ?
[02:17] <spiv> JoaoJoao: typically "mv oldname newname" on your filesystem does the trick :)
[02:18] <spiv> JoaoJoao: or is this a branch on launchpad?
[02:25] <JoaoJoao> spiv, that doesn't work when I'm using shared repos + lightweight checkouts
[02:36] <jml> JoaoJoao, there isn't a command for that: it's a bit of a pain
[02:37] <JoaoJoao> hmmm looks like mv + bzr switch --force could do the trick
[02:38] <jml> JoaoJoao, give it a try.
[02:38] <jml> JoaoJoao, what I do is: mv ~/repos/project/old ~/repos/project/new; echo -n 'file:///home/jml/repos/project/new/' > ~/src/project/old/.bzr/branch/location; mv ~/src/project/old ~/src/project/new
[02:38] <mwhudson> yes, switch --force should help
[02:38] <JoaoJoao> thanks jml, mwhudson
[02:40] <JoaoJoao> btw, does bzr support or has some plugin for interactive commits?
[02:40] <lifeless> theres a plugin I believe
[02:40] <lifeless> also the qbzr commit dialog can select bits
[02:49] <JoaoJoao> yup, bzr-interactive works fine :)
[02:49] <JoaoJoao> bzr plugins rock
[02:50] <garyvdm> If you have 2 branches, with the same file-id's, but no common ancestor, how do you merge?
[02:51] <garyvdm> nvm - bzr merge ../dev2 -r -1..0
[02:52] <lifeless> garyvdm: uhm, probably you want 0..-1
[02:52] <garyvdm> ah
[02:53] <JoaoJoao> any estimated release date for bzr2?
[02:53]  * fullermd admits to mild curiosity as to what -1..0 will DO...
[02:56]  * igc lunch
[03:15] <garyvdm> Please can someone read this:http://paste.ubuntu.com/225619/ I want to make sure that the advice I'm giving this guy is good.
[03:15] <garyvdm> http://paste.ubuntu.com/225619/
[03:16] <lifeless> abentley: around?
[03:22]  * garyvdm sends mail
[04:04]  * SamB wonders how jam runs the tests for pymemdump
[04:13] <GungaDin> hi
[04:13] <GungaDin> How do I tell bzr to ignore committing a file?
[04:13] <GungaDin> I mean... not to commit a file that was changed.
[04:15] <SuperMMX> GungaDin: there is an option "-x" or "--exclude" for commit
[04:15] <mwhudson> bzr ci -x
[04:15] <SuperMMX> or just specify files that you wanna commit
[04:20] <GungaDin> sure thx
[04:56] <abentley> lifeless: kinda around.
[04:57]  * SamB wanted to file a bug against pymemdump for not having a name and then realized it'd need to have one for him to do that ... jam!
[04:57] <lifeless> abentley: I mailed the list
[04:57] <lifeless> abentley: copied to you
[04:57] <abentley> lifeless: I see it.
[05:01] <abentley> lifeless: I'm not deeply opposed to PreviewTree.inventory, but I thought omitting it would help us explore what's needed to go inventory-free.
[05:02] <lifeless> abentley: I think it has helped us do that
[05:05] <poolie> http://fishbowl.pastiche.org/2009/07/22/a_letter_to_my_younger_self/
[05:06] <lifeless> nice
[07:28] <lifeless> garyvdm: if you're around, and jelmer: too; I'm looking [well, I *will be*] looking for a GUI progress bar
[07:29] <lifeless> I want to pop up a little progress bar in the bottom right of my desktop.
[07:29] <lifeless> pointers appreciated
[07:32] <poolie> lifeless: http://bzr.pastebin.com/d7c92d84a <- just adding a plain Reconfigure class
[07:33] <poolie> i don't want to tease apart the existing code for now
[07:36] <lifeless> poolie: sure; I don't think you should tease it apart. The only concern I have is that this makes the module be mixed in style, which muddies things more.
[07:36] <lifeless> poolie: I will let you be the judge of how important that i.s
[07:43] <spiv> Incremental steps to a better style is probably better than adding more code to an unwanted style.
[07:50] <lifeless> spiv: as long as its really clear to other folk coming along that this is the trend
[07:51] <lifeless> spiv: without clarity its just unclear ;)
[07:53] <spiv> lifeless: agreed
[07:53] <poolie> lifeless: pydoc debug is documented to 'Run the test without collecting errors in a TestResult'
[07:54] <poolie> that might be something you'd want to call from inside a debugger i guess...
[07:54] <lifeless> poolie: yah
[07:54] <poolie> i'm disappointed that it doesn't remove my bugs :)
[07:54] <lifeless> poolie: its a underdeveloped aspect of te interface
[07:54] <garyvdm> lifeless: lp:~jameinel/qbzr/progress
[07:56] <garyvdm> lifeless: If you want, I can hack up a minimal plugin.
[07:56] <garyvdm> bbl
[07:57] <lifeless> garyvdm: oh, its not bzr
[07:57] <lifeless> garyvdm: its for subunit
[07:59] <garyvdm> lifeless: Then this diff is a good sample: http://bazaar.launchpad.net/~jameinel/qbzr/progress/revision/813
[07:59] <lifeless> thanks
[08:00] <GungaDin> How do I check a log given a rev id?
[08:13] <poolie> spiv, btw lifeless said you might have fixed something recently to make tests rm their working directory as soon as they finish?
[08:13] <poolie> GungaDin: bzr log -r revid:blah
[08:14] <spiv> Hmm, I don't remember fixing anything like that very recently.  I did make some change about six months ago maybe.
[08:16] <poolie> spiv, also it should have a smiley when reporting 0 vfs :)
[08:16] <poolie> as this pull just did :)
[08:16] <spiv> :)
[08:45] <lifeless> jml: jelmer: http://paste.ubuntu.com/226322/ feedback sought
[09:15]  * igc dinner
[10:31] <fullermd> I love comments like "# XXX: Update this after 0.10 is released"...
[11:36] <LarstiQ> igc: I have time to look at the sphinx users docs in ~1.5 hours if that's ok
[12:20] <igc> LarstiQ: cool
[12:23] <jelmer> 'evening Ian
[13:23] <LarstiQ> igc: done
[13:23] <LarstiQ> LenZGr_: did you see the bzr-svn 0.6.3 announcement, and was that helpful for your packaging?
[13:31] <LenZGr_> LarstiQ: Yes, thank you! 0.6.3 should be available shortly (I commited it already, but made a mistake which caused the build to fail)
[13:34] <DaffyDuck_> I'm looking for a good way to create archived backups of my repositories. I don't want to backup unless I need to, so I was thinking about parsing "bzr info -v" and using the "revisions" under "Repository:", and use the revision number in the file name. That way I can check if the revision is backed up already. Is there a problem with this?
[13:44] <LarstiQ> LenZGr_: good, that makes me happy :)
[13:46] <LarstiQ> DaffyDuck_: there shouldn't be that many files, so why not let rsync do it's job?
[14:13] <DaffyDuck_> LarstiQ: The data needs to be encrypted and stored on external media. We're not fooling around when it comes to backups.
[14:15] <Raim> even if you encrypt it, it should still be stored efficiently by your backup software. so backing up an unchanged bzr repo again should not be a problem...
[15:17] <LarstiQ> DaffyDuck_: as Raim said, I'd expect the backup software to be able to detect changes. Other than that -
[15:18] <LarstiQ> DaffyDuck_: in theory revisions could be removed, in practice they probably only get added. How serious about the backups are you?
[16:12] <bialix> igc: hi
[16:12] <igc> hi bialix
[16:13] <bialix> igc: can we disable bugmails in bzr-explorer-dev mailing lists?
[16:13] <bialix> I've got 2 copies of those mails
[16:13] <bialix> one from LP and another from ML
[16:14] <igc> bialix: you might be subscribed separately?
[16:14] <bialix> I don't understand
[16:14] <igc> the people notified list on some of them was 'bialix + Bzr Explorer Devs'
[16:15] <bialix> on all of them actually
[16:15] <bialix> I'm trying to disable bugmails on bugs page
[16:16] <igc> bialix: go for it
[16:16] <bialix> ok, will see how it will going
[16:16] <igc> bialix: I'm not sure how to do it fwiw
[16:16] <bialix> https://bugs.launchpad.net/bzr-explorer/+subscribe
[16:17] <bialix> I was under impression you explicitly subscribe ML to receive bugmails, no?
[16:22] <igc> bialix: I can't see anything in the ml or team configuration re that. Ask on #launchpad maybe
[16:24] <bialix> igc: can you explain your comment: https://bugs.launchpad.net/bzr-explorer/+bug/393615/comments/6 please?
[16:24] <bialix> what's fixed in 1.18?
[16:28] <spiv> jam: hey
[16:31] <igc> bialix: bug 394227
[16:32] <jam> hi spiv
[16:34] <spiv> jam: If you are inclined to review my updated inventory-delta patch, that would be lovely :)
[16:35] <spiv> jam: I also heard that you were surprised to hear that inv-delta streaming was much better than IDS over localhost HPSS?
[16:36] <spiv> jam: try comparing bzr push -r2000 of bzr.dev into a "bzr init bzr://localhost/ --1.9-rich-root" -- I found ~7.5 min real time vs. 31 min, and much less memory consumption and IO too.
[16:36] <jam> spiv: I was, though there are ways to explain it.
[16:36] <jam> mostly in the marshalling/unmarshalling overhead
[16:37] <spiv> As in, 30M transferred vs. ~740M.
[16:37] <jam> spiv: Not a 2a format?
[16:37] <jam> spiv: also, I've always focused on "pull/upgrade" and not "push"
[16:37] <jam> not sure if that matters here
[16:37] <spiv> jam: that would be interesting too, but I thought I'd measure something simpler.
[16:38] <spiv> I wouldn't expect it to affect IDS, it might affect streaming but probably not.
[16:38] <spiv> (the difference between pull/upgrade vs. push, that is)
[16:38] <spiv> Anyway, the current patch reinstates IDS for local fetches.
[16:39] <jam> my bigger concern ATM is why things are breaking
[16:39] <spiv> Given that IDS is known to have pretty bad performance on the network that seems like the best compromise to me.
[16:39] <jam> is it a bug that already existed that you are now testing
[16:39] <jam> or did something break with the new changes
[16:39] <spiv> The test failure I reported?
[16:39] <jam> yes
[16:39] <spiv> I'm pretty sure it's an existing bug.
[16:40] <spiv> I extended the test coverage significantly.
[16:40] <spiv> That particular test is more precise in verifying that it can actually stream the revisions now, and I also added several new scenarios to the parameterisation.
[16:41] <spiv> It may be that IDS is actually broken for local stacked branches, but even if so my branch isn't a regression there.
[16:41] <spiv> It *might* shed some light on some of our bug reports, though.
[16:42] <spiv> It may be worthwhile extracting the part of the patch that extends per_interrepository's scenarios, and it's test_fetch module, trying it on bzr.dev, and perhaps landing it directly.
[16:43] <spiv> Maybe with a KnownFailure if it really does uncover an existing bug.
[16:44] <spiv> I'm pretty confident that my changes would not have introduced it, the refactorings I did to IDS to make its code reusable by the streaming path were fairly simple... but I've been wrong before :)
[16:45] <spiv> jam: I *think* the issue may be that the tests in bzr.dev don't test any rich-root -> rich-root IDS paths.
[16:45] <spiv> jam: only non-rr -> rr.
[16:46] <wadesworld> just to verify, the only way to run bzr serve and allow only some users write access is to leave it read-only but have the ones with write access use bzr+ssh, correct?
[16:46] <jam> spiv: weird
[16:46] <jam> your review request is in the "other" category
[16:46] <jam> Maybe because of how you resubmitted it?
[16:46] <jam> (I was having trouble finding it)
[16:47] <spiv> jam: oh, weird.
[16:47] <jam> It didn't have any "requested reviews"
[16:47] <spiv> jam: I guess that's possible, just changed the "status" to "resubmit".
[16:47] <jam> especially not of 'bzr-core' like the other ones
[16:47] <jam> I fixed that
[16:47] <jam> Probably a UI bug for launchpad-code reviews
[16:47] <spiv> Which I thought was the way to resubmit, but perhaps not.
[16:47] <spiv> Yeah.
[16:48] <jam> spiv: I think it *is*, with the caveat of about 3-4 bugs I've opened on it so far :)
[16:48] <spiv> jam: so, I see this bug in bzr.dev
[16:49] <spiv> jam: if I add 6RichRoot->2a to the interrepo scenarios and run interrepo.*test_fetch.*stacking.*2a I see the failure.
[16:49] <spiv> jam: needless to say, my inv-delta streaming code passes that test ;)
[16:50] <spiv> Hmm, or rather, I see a failure, not necessarily a valid failure now I look at it.
[16:50] <spiv> This failure is why I changed the assertion to "can stream these revisions" rather than directly checking for inv parents in the repo.
[16:51] <spiv> Because things seem a bit different in 2a...
[16:52] <jam> spiv: so the inventory roots should be present in 2a formats
[16:52] <jam> (as should all the chk pages underneath)
[16:52] <jam> which is what we had worked on with the "get_stream_for_missing_keys()" a while back
[16:52] <jam> "can stream" might be a stronger assertion because of the need for child chk pages...
[16:53] <spiv> jam: ah, but it does still fail the "can stream" check.
[16:53] <jam> spiv: so it is very likely that IDS doesn't fill in parent inventories into a stacked branch
[16:53] <jam> as the only code that does 'get_stream_for_missing_keys()' is the. .... streaming code :)
[16:53] <jam> stacking
[16:53] <jam> the bane of my last few months
[16:54] <jam> really, it breaks all sorts of things. Especially when coupled with the "RemoteRepository" are not actually stacked anymore...
[16:54] <spiv> Yeah, I know what you mean!
[16:54] <jam> so the abstraction we put in, is removed, and then everything is exposed again
[16:55] <spiv> "AbsentContentFactory" bugs etc.
[16:55] <jam> yep
[16:56] <spiv> So, the bug is in bzr.dev; I have fix ready to go, just merge my patch and disable IDS ;)
[16:56] <Kinnison> jelmer: didja file the bug in the end?
[16:57] <jam> spiv: I actually thought IDS was disabled when stacking was involved
[16:57] <jam> but I guess that is more it is disabled when the formats are identical
[16:57] <spiv> In doing the timings of streaming vs. IDS over localhost I did fix one performance bug, btw.
[16:57] <spiv> (in streaming, that is)
[16:57] <jam> spiv: btw, my loopback can stream 700MB in about... 1s
[16:57] <spiv> There's almost certainly a fair bit of low hanging fruit there, performance-wise.
[16:58] <jam> so I doubt that is a specific cause of slowdown
[16:58] <jam> probably the serialization/deserialization is, though
[16:58] <spiv> Right, but my disk can't seek back and forth that fast, particularly when bzr is pushing the disk cache out of memory...
[16:58] <spiv> With IDS, -Dmemory reported VmPeak of 333840.
[16:59] <spiv> vs. 126328 for inv-delta.
[16:59] <spiv> (and VmHWM and VmRSS had similar ratios)
[16:59] <jam> spiv: I'm curious if you just tweaked the size of the internal cache, how much effect that would have
[16:59] <jam> (batch_size, and one other flag IIRC)
[17:00] <spiv> So, if you can bzr push -r2000 of bzr.dev over localhost with IDS in 1s I'll be impressed :)
[17:01] <spiv> Possibly the overhead of 103958 HPSS calls started to matter?
[17:01] <spiv> It was certainly reading back a lot of the data it had already written (in addition to the regular repacking over the wire).
[17:01] <jam> spiv: Right, I'm saying the cost of marshalling/serializing 700MB was important
[17:01] <jam> not the fact that it was 700MB
[17:02] <jam> over the loopback
[17:02] <spiv> Sure.  700MB over any other network interface is pretty important, though :)
[17:03] <jam> spiv: so 4519 is your latest, right?
[17:03] <spiv> That inv-delta was so much faster, even though it's paying CPU cost on both sides (and I don't have dual core), is a good sign for streaming I think.
[17:03] <spiv> jam: yep
[17:04] <jam> spiv: but time 'bzr branch x y' and compare
[17:04] <spiv> jam: interesting, IDS and inv-delta were much much closer in time on -r1000
[17:04] <jam> spiv: very little interesting in bzr.dev pre 1500
[17:04] <jam> that is where we got merging support
[17:04] <spiv> (-r1000 is only 1102 revs, -r2000, is 7410 revs)
[17:04] <jam> 1-~1500 are all just simple commits by martin
[17:05] <jam> hm... maybe that changed with some of our history mutations
[17:05] <spiv> As in, within 20% rather than over 4x slower.
[17:05] <jam> there is a mainline history switch with robert as a committer
[17:05] <spiv> Obviously it's many more revs and larger trees, but it's still seems disproportionate.
[17:06] <spiv> (hmm, about 10x more data in knit pack format)
[17:06] <jam> spiv: computing inventory deltas is generally O(N^2)
[17:07] <jam> spiv: so what to do about the 'bloat' factor?
[17:07] <jam> as in, taking 400+MB on disk, rather than <100MB?
[17:07] <spiv> I think the 2a sink needs to be smarter.
[17:08] <spiv> (possibly the smarts should be builtin to the generic sink, rather than making a 2a specific sink...)
[17:09] <spiv> i.e. do a pack midstream if a certain volume of records has been received, on the assumption that can probably save significant disk space.
[17:09] <jelmer> Kinnison: not yet, I forgot about it :-)
[17:09] <jam> spiv: I'lll at least argue that perhaps the source can stream the content in a more efficient manner
[17:09] <spiv> Currently the StreamSink is smart enough to do that at the end, but that's a bit late.
[17:09] <jelmer> jam: Is there any situation in which an InventoryEntry could have a symlink_target that is None ?
[17:09] <Kinnison> jelmer: *pout*
[17:10] <Kinnison> jelmer: How can I be anything less than the most important person in your life?
[17:10] <spiv> Yeah, that seems likely too, but tweaking StreamSink is likely to be easier.
[17:10]  * Kinnison sobs
[17:10] <jam> jelmer: I believe InventoryEntries from a WorkingTree.inventory are set to None, because we want you to go read the symlink directly
[17:10] <spiv> I'd also love to make the stream resumable while we're on the topic ;)
[17:10] <spiv> But the branch is already pretty large!
[17:10] <jelmer> Kinnison: :-)
[17:11] <jelmer> Kinnison: I'll do it now, while I have my mind on bzr
[17:11] <jelmer> jam: ah, thanks
[17:11] <jam> spiv: so IDS does a repack every 1k revs or so, is that possibly the extra data you were seeing over the wire?
[17:11] <jelmer> jam: In that case bzrlib/export/dir_exporter.py should possibly special case that situation?
[17:12] <jam> jelmer: perhaps. I don't really know what export w/ a workingtree as a source is really meant to mean
[17:12] <spiv> jam: it's some, but not all of it.
[17:13] <spiv> jam: even when it's not repacking I see an enormous number of readvs.
[17:13] <spiv> In fact, I suspect repacking isn't even a majority of the traffic.
[17:13] <jelmer> jam: it's done in bzr builddeb, which allows you to build based on either the current working tree or a historic revision.
[17:14] <jam> jelmer: I'm not really 100% familar with the code. But building from an uncommitted tree seems foolish
[17:14] <jelmer> jam: Since it has a tree object around as well I'll change it to use tree.get_symlink_target
[17:14] <jam> jelmer: tree.get_symlink_target would be the safest bet
[17:14] <spiv> ~740M of traffic for 27M of packs isn't accounted for by repacks every 1k when there's only 7410 revs.
[17:14] <jelmer> jam: It's pretty common to not commit until you've tested that building the package works
[17:15] <jam> spiv: except the size on disk *before* repacking is probably a bit larger?
[17:15] <jam> I guess you are testing with --1.9-rich-root
[17:15] <spiv> jam: this target is 1.9-rich-root, though.
[17:15] <jam> which doesn't change much
[17:16] <spiv> Right.
[17:16] <jam> so... currently testing your code and it is "hung"
[17:16] <jam> as expected
[17:16] <spiv> Yeah, it has some long pauses :(
[17:16] <jam> and, as someone else mentioned
[17:16] <jam> it pauses at:
[17:16] <jam> [###################/]  14218KB    94KB/s
[17:16] <jam> which looks like it is almost done
[17:16] <jam> when in reality, it has barely started
[17:16] <spiv> Two parts where it has long pauses, I think.
[17:17] <jam> spiv: any idea why we don't even get transport progress updates?
[17:17] <spiv> One where it's busy doing lots of generate_root_texts » _find_root_ids » iter_rev_trees (judging from sampling backtraces with SIGQUIT)
[17:17] <jam> are you computing all deltas at once?
[17:17] <jam> (if only SIGQUIT worked on windows...)
[17:18] <spiv> Then another pause after it's sent the full stream and is waiting for the server to respond.  Apparently the server does some lengthy work at end of stream.
[17:18] <spiv> (check references, find missing keys?)
[17:18] <jam> spiv: neither of those should be minutes...
[17:18] <jam> repacking?
[17:18] <spiv> Although in fact I noticed during that pause for the server that it the pack file in the upload dir was slowly growing.
[17:19] <spiv> Which seems odd, I would have expected it to have spooled out to disk as it was received.
[17:19] <jam> spiv: converting inventory deltas to full inventories?
[17:19] <jam> adding them to the repository as fulltext
[17:19] <jam> and recomputing a "knit-delta"
[17:19] <spiv> Hmm, not a repack on after a single insert into 1.0-rich-root I don't think.
[17:19] <jam> would be my guess
[17:19] <spiv> Yeah, that could well be it.
[17:20] <spiv> I assume applying deltas is faster on 2a?
[17:20] <jam> spiv: faster than --1.9-rich-root, most likely
[17:20] <jam> fast as you would think... not always
[17:20] <jam> finding minimal deltas helped a lot here
[17:20] <spiv> Well, I'm used to that :)
[17:21] <spiv> The streaming code currently makes a delta against every parent and chooses the shortest to send.
[17:21] <jam> spiv: so I noticed that you are still using Inter1and2Helper... which means we have to re-read all the inventories. is that correct?
[17:22] <spiv> Yes, probably.
[17:22] <spiv> I think that maybe we can improve that...
[17:22] <jam> spiv: why is the api:
[17:22] <jam> _stream_inventories_as_deltas(..., fulltexts=??)
[17:23] <jam> seems a bit strange to ask for fulltexts when you want deltas
[17:23] <jam> spiv: so here is *my* summary with mysql-525
[17:23] <spiv> jam: get_stream_for_missing_keys also generates inv-deltas.
[17:23] <jam> "time bzr branch"
[17:23] <jam> real    3m20.731s
[17:23] <jam> "time bzr IDS push bzr:///"
[17:23] <jam> real    3m45.805s
[17:23] <spiv> jam: but we don't want missing "compression" parents, we want "fulltexts" at that point.
[17:23] <jam> "time bzr IDelta push bzr:///"
[17:23] <jam> real    9m40.311s
[17:24] <spiv> jam: so that flag causes it to generate deltas from an empty revision.
[17:24] <spiv> jam: an alternative that doesn't work would be to send a fulltext inv in the native format
[17:25] <spiv> jam: that doesn't work because the sink expects the inv in the *source* serialisation, but gc formats don't have a functional inv serialisation.
[17:25] <jam> spiv: well they do after I finish my patch to the bundle code
[17:25] <spiv> jam: (they actually try to serialise as xml5, which doesn't support rich-roots, but looks like they do if the rich-root version is the revision...)
[17:25] <jam> but stuff like accidentally inheriting from a non-rich-root capable xml inventory serializer needs to be fixed
[17:26] <spiv> So I thought that given that I just defined a format-independent serialisation I should just use that :)
[17:26] <jam> spiv: I have a patch which fixes that sort of thing :0
[17:26] <jam> I agree
[17:26] <jam> just can't do it for bundles just yet
[17:26] <spiv> Right.
[17:26] <jam> spiv: I would probably recommend not using 'fulltexts' as the flag, as I find it confusing, I'll see if I can come up with a better name.
[17:26] <spiv> I agree that part of the code is a little unclear, feel free to mention that in your review :)
[17:26] <spiv> Thanks!
[17:27] <jam> spiv: and I would recommend retrying with --2a as the target. I think that may be a major difference between what we are seeing.
[17:27] <spiv> Yeah, I will.
[17:27] <jam> spiv: I noticed that you don't delta against revisions you haven't sent... I'm wondering if that is optimal
[17:27] <spiv> (I also have high on my todo list trying the blackbox hpss acceptance tests with 2a to make sure the hpss call counts don't regress)
[17:27] <jam> consider "bzr push" is just adding 1 rev to a branch
[17:28] <jam> it will have to extract and send the whole inventory
[17:28] <spiv> Yeah.  It was a cheap way to avoid having to figure out what inventories I could depend on the remote having.
[17:28] <jam> spiv: code like this is der very bad:     parent_inv = from_repo.get_inventory(parent_id)
[17:29] <jam> as in, you re-extract every inventory at least 2x
[17:29] <jam> actually 3x for rich-root conversion
[17:29] <jam> which is probably the bulk of 3min => 9min
[17:29] <spiv> Probably now that I have the get missing keys working probably I don't need to be so pessimistic...
[17:29] <jam> pushing bzr doesn't show it
[17:29] <jam> because they are small
[17:30] <jam> spiv: depending on how your StreamSink works, hard to say
[17:30] <jam> would it buffer an inventory delta it can't extract
[17:30] <jam> until it fills in a missing parent
[17:30] <jam> if so
[17:30] <jam> what about all the other deltas chained on top of it
[17:30] <spiv> Oh right, yes, that was why.
[17:30] <jam> and where are those buffered
[17:31] <jam> spiv: now I would imagine *not* caching any inventories would be the reason your peak memory consumption went tdown
[17:31] <jam> but at a fairly high cost
[17:31] <spiv> It would potentially be fine, except that we don't have a way to store format-neutral inv-deltas between HPSS calls.
[17:31] <jam> "time bzr branch mysql mysql-19-rich-root" real    6m2.345s
[17:31] <jam> so it is 2x slower to branch 1.9 => 1.9-rich-root than to branch 1.9 => 2a
[17:32] <spiv> Whereas we can store arbitrary pack knit-delta records with missing compression parents (by not inserting that pack, and reporting the missing keys)
[17:32] <jam> spiv: which brings up some design questions
[17:32] <jam> of doing this
[17:32] <jam> though I guess we still have "optimized" paths when the formats are the same...
[17:32] <spiv> We can fudge it -- make a new pack to hold dummy records that store the inv-deltas verbatim.
[17:33] <spiv> And then don't insert it in the final commit, obviously.
[17:33] <spiv> Yeah, that was basically my thinking... we don't really need to tie ourselves in knots to get maximal cross-format performance.
[17:34] <spiv> The main thing is avoiding the pathological performance we currently get, while moving towards a unified code path.
[17:35] <spiv> I'm finally going to go sleep shortly.
[17:35] <spiv> Anything else I can do for you before I do?
[17:35] <jam> spiv: well, in my trivial loopback test, I saw a 3x degradation still... but yeah, I'd like more unified code paths.
[17:35] <jam> and I like the concept of using inventory deltas more
[17:36] <jam> spiv: ah, I bet that with --1.9-rr you are seeing such a difference because of "add_inventory_by_delta"
[17:37] <jam> spiv: RemoteRepository.add_inventory_by_delta is not a smart method
[17:37] <jam> it is an _ensure_real one
[17:37] <jam> which means for --1.9-rr we have to read back the whole inventory we just wrote out
[17:37] <spiv> jam: perhaps I should try with CHK1->2a :)
[17:37] <jam> spiv: 1.9 => 2a would be reasonable as I think that is a common case we will be hitting
[17:38] <spiv> Yeah.  I will definitely try that out.
[17:38] <spiv> I should grab a mysql tree at some point, I guess...
[17:38] <jam> spiv: so clearly what we need to do is leave IDS alone
[17:38] <jam> and just have add_inventory_by_delta use the serialized inventory deltas
[17:38] <jam> and apply them on the other side :)
[17:38] <spiv> Hah.
[17:39] <jam> honestly, I think IDS as a *source* is pretty optimal
[17:39] <jam> as a Sink, obviously we need some work
[17:39] <spiv> Well, I've borrowed large chunks of its logic for the source already.
[17:40] <spiv> We can conceivably try borrowing more...
[17:40] <spiv> But IIRC we start running into issues with different expectations about when things happen in IDS vs. streaming.
[17:40] <spiv> Perhaps not insurmountable.
[17:41]  * SamB wishes the bzr .debs would display their Debian versions as well as their internalized versions
[17:41] <jam> time bzr inter-delta push localhost/mysql-19rr real    4m28.411s
[17:41]  * SamB doesn't know where the PPA gets it's debian dir from, though :-(
[17:41] <jam> spiv: So interestingly, for *1.9-rich-root* your delta code is faster than IDS locally
[17:41] <spiv> jam: so, I am a bit puzzled about parts of Inter1and2Helper.generate_root_texts
[17:42] <jam> but it is still 50+% slower than IDS to 2a formats
[17:42] <jam> though
[17:42] <jam> I *do* have dual cores here
[17:42] <SamB> jam: can you tell the kernel to stop using one for a while ?
[17:42] <SamB> hmm
[17:42] <spiv> I get the feeling it is doing more work, or at least more work up-front, than is strictly necessary.
[17:43] <SamB> I think there's some kind of CPU mask you can set on processes?
[17:43] <spiv> If I were to keep using more of IDS's code I'd probably see if I can replace that part.
[17:44] <jam> spiv: from what I understand
[17:44] <jam> generate_root_texts reads all the inventotries
[17:44] <jam> and then compares the root to see how it should create the per-file graph for the root key
[17:44]  * SamB wishes the progress text would stop getting cut off just before the really useful part
[17:44] <jam> it does it in a bass-ackwards way
[17:44] <jam> but that was how we do rich-root conversions
[17:44] <SamB> hmm
[17:44] <jam> was/is
[17:44] <SamB> bzr doesn't seem to handle SIGWINCH correctly
[17:45] <spiv> jam: and in fact it makes actually revision trees, not just reads the inventories.
[17:45] <spiv> Which seems like overkill to me.
[17:45] <jam> spiv: rev trees are trivial overhead over an inventory
[17:45] <jam> the problem is the inventory reading
[17:45] <spiv> Sure.
[17:45] <jam> spiv: so one thing from you before you go
[17:46] <jam> do you have any idea *how* to share more code between IDS and the streaming stuff?
[17:46] <SamB> so what is rich about the root in rich-root?
[17:46] <jam> Is it to put the members onto repo
[17:46] <jam> or what?
[17:46] <SamB> and where is this documented ?
[17:47] <spiv> jam: my vague thinking was "replace Inter1and2Helper.generate_root_texts with an equivalent based on IDS' code."
[17:47] <jam> spiv: what surprises me the most is how much worse the push to 2a is versus push to 1.9-rich-root
[17:47] <jam> 4.5min => 9+min
[17:47] <spiv> Yeah, that surprises me too.
[17:47] <spiv> jam: btw, regarding SIGQUIT...
[17:47] <spiv> jam: windows has SIGBREAK
[17:48] <jam> spiv: and how would one generate that :)
[17:48] <jam> ?
[17:48] <spiv> jam: perhaps you could hook the pdb hack into that?
[17:48] <spiv> jam: the Break key (Alt-Pause or something)
[17:48] <spiv> Fn-Pause on this laptop keyboard, apparently.
[17:49] <spiv> Or rather Python on windows has SIGBREAK, it's not a real posix signal of course...
[17:49] <jam> spiv: Ctrl+Pause on this keyboard
[17:49] <jam> Fn+Pause on my laptop as well
[17:49] <spiv> But it should work well enough for this, I think.
[17:52] <spiv> jam: any luck with SIGBREAK?
[17:52] <jam> spiv: i can get it to stop the process
[17:52] <jam> checking on whether I can trap it
[17:54] <jam> spiv: so far, doesn't look like I can actually trap SIGBREAK
[17:54] <spiv> Oh, that's a shame.  ISTR I could, but it's been a long time since I tried...
[17:55] <jam> spiv: I see stuff online that claims you can
[17:55] <jam> I'm checking
[17:57] <spiv> (weird that the stdlib docs don't seem to mention it)
[17:57] <jam> ah, just doing it wrong
[17:58] <jam> It seems that win32 signal doesn't have the *attribute* SIGQUIT
[17:58] <jam> so it wasn't installing the hook
[17:58] <jam> trying again
[17:58] <spiv> Right, it wouldn't.
[17:58] <jam> yep, seems to work
[17:58] <spiv> It probably only has SIGINT and SIGBREAK.
[17:58] <jam> need to poke a bit more
[17:59] <spiv> A hasattr-style check in breakin.py is probably appropriate.
[17:59] <jam> yeah, we have that now for SIGQUIT
[17:59] <jam> doing the same for SIGBREAK
[17:59] <spiv> Cool, glad you don't have to miss out :)
[17:59] <spiv> jam: and update the boilerplate stderr text about SIGQUIT to be parameterised I guess...
[17:59] <jam> yep
[17:59]  * SamB finds http://msdn.microsoft.com/en-us/library/ms686016(VS.85).aspx from http://twistedmatrix.com/pipermail/twisted-python/2006-September/014113.html
[18:00] <spiv> jam: Ctrl-Break is in some ways more appropriate
[18:00] <jam> sure
[18:00] <spiv> jam: "give me a breakpoint where you are now" ;)
[18:01] <spiv> And of course the module name is "breakin"
[18:01] <spiv> Anyway, -> zzz time.
[18:01] <spiv> jam: thanks for taking a look at the inv-delta patch.
[18:01] <spiv> And for the timings.
[18:02] <jam> sleep well
[18:13] <dvheumen> hi everyone. I've got a branch ready that fixes one of the 'easy' marked bugs (I'm trying to become familiar with bzr and python), and know I'm wondering how I should push this branch to launchpad/bzr? (So I can request a merge)
[18:17] <dvheumen> isn't there anyone who knows how to push a branch to launchpad?
[18:17] <jelmer> dvheumen: bzr push lp:~userorteamname/projectname/branchname
[18:18] <SamB> dvheumen: well, you could push it to lp:~<insertyourusernamehere>/bzr/<bugno>-descriptive-text
[18:18] <dvheumen> okay, so I don't need to be part of a team or something to push there ... tnx guys, I'll do that :)
[18:57] <jetole> hey guys. I just pushed a repo into bazaar on launchpad and then realized after the fact that it had more files then I had thoght. Is there a way to delete them from bazaar on launchpad so that you can't see past revisions etc
[18:57] <jetole> ?
[19:00] <beuno> jetole, yes
[19:00] <beuno> on the branch page
[19:00] <beuno> next to the title
[19:00] <beuno> there's a trash can icon
[19:01] <beuno> that will delete the branch completely
[19:01] <Jesse> hi
[19:01] <jetole> not sure what the branch page is but I don't see a trash can. I am in loghead I think it's called
[19:01] <jetole> loggerhead
[19:02] <jetole> how do I get to the branch page?
[19:02] <beuno> jetole, right, you need to go to the branch page in launchpad
[19:02] <beuno> there's a few ways
[19:02] <beuno> one of them is running, from the branch:  bzr lp-open
[19:02] <Jesse> hi folks, i know this might be 50% off-topic, but i just saw the screen shots http://bazaar-vcs.org/BzrScreenshots?action=AttachFile&do=get&target=bzrk.png
[19:02] <beuno> the other is going to your profile page, and clicking the "Code" tab
[19:02] <jetole> ok, one sec
[19:03] <Jesse> and i'm wondering what those panels in the bottom are
[19:03] <jetole> beuno: bzr: ERROR: bzr+ssh://jetole@bazaar.launchpad.net/~jtole/+abc/def is not registered on Launchpad.
[19:04] <Jesse> i mean i can't get gnome-panel to work like those
[19:05] <jetole> beuno: does that mean much to you? I created the launchpad base by simply doing a bzr push with my user name after I uploaded my ssh keys so I didn't register a project per se
[19:05] <beuno> jetole, +abc?
[19:06] <beuno> jetole, what's your launchpad username?
[19:06] <Jesse> @jetole you can push to your +junk folder
[19:06] <Jesse> @jetole like bzr push lp:~jetole/+junk/myawesomebranch
[19:06] <beuno> jetole, https://code.edge.launchpad.net/~jetole
[19:06] <beuno> according to Launchoad
[19:07] <beuno> you have not pushed any branches to it
[19:07] <jetole> beuno: I just found the branch page and deleted it, I did use +junk but I didn't know that was a standard name and didn't really want to say too much as some content was personal
[19:07] <beuno> jetole, gotcha
[19:07] <Jesse> so anybody knows about the panels in the screen shot?
[19:07] <jetole> beuno: last quick question, on my own personal branch on my computer, do you know how I can remove a file so when I push the branch again there is no revision included?
[19:08] <jetole> I want to keep the details of the files I want to push i.e. history etc
[19:08] <beuno> jetole, if you added a file a few revisions back
[19:08] <beuno> it's in the history, and the only real way to get rid of it
[19:08] <beuno> is to get rid of that history
[19:09] <jetole> ah fine, I am just going to create a new repo
[19:09] <Jesse> i once wrote a script to sanitize history like you wanted
[19:10] <Jesse> that was before i pushed code on to lp
[19:11] <jetole> it's actually all right. One I find out how to remove a directory from bzr I will create a subdir, move the files in and re init
[19:12] <jetole> or wait, can I create a subdir as part of this repo and push just the subdir?
[19:14] <garyvdm> Hi all
[19:20] <Jesse> "as part of this repo and just push subdir" no
[19:20] <Jesse> if you want to sanitize your repo, you can only start clean and import it with some cleaning routine
[19:23] <jetole> Jesse, Thanks for the info. I already created a new repo and published it
[19:23] <Jesse> remember to sanitize stuffs when you import
[19:23] <jetole> btw, does my folder have to be +junk? Can I use anything else I want?
[19:24] <Jesse> i'm personally ok with the name
[19:24] <jetole> heh
[19:24] <jetole> I'm taking a smoke brake. bbiab
[19:24] <Jesse> funny that i 'mkdir junk' in my own $HOME
[19:24] <Jesse> ciao
[19:25] <Jesse> i don't know if this is rude, but anybody (either from bzr team or not) use gnome?
[19:26] <jelmer> sure, there are quite a few people here using gnome
[19:26] <Jesse> http://bazaar-vcs.org/BzrScreenshots
[19:26] <Jesse> i'm wondering what that panel is
[19:26] <Jesse> i mean the two at the bottom
[19:27] <Jesse> hmmm seems john added those
[19:28] <Jesse> i mean the two bzrk pics
[19:33] <jelmer> Jesse: no idea, sorry
[19:33] <Jesse> thx jelmer anyway
[19:34] <Jesse> maybe i'll need to raise it to john when he's online
[19:34] <Jesse> arr sorry to go off-topic
[19:34] <jelmer> Jesse: I think that screenshot was made by Keybuk
[19:35] <Jesse> really?
[19:35] <Jesse> i got that info from http://bazaar-vcs.org/BzrScreenshots?action=info
[19:35] <jelmer> Jesse: yeah, he originally wrote bzrk and his username (scott) can be seen in the screenshot
[19:37] <jetole> Jesse: I use gnome
[19:37] <Jesse> jelmer can it be that john took the screenshot when he was looking at a commit by scott? you know bzr is distributed :p
[19:37] <jetole> although I don't use bzr in gnome
[19:37] <jetole> I do that from the terminal
[19:38] <Jesse> Jelmer: I meant I could only find "scott" in the bzrk history window
[19:38] <Jesse> jetole: me too
[19:39] <jelmer> Jesse: scott == keybuk
[19:39] <Jesse> jetole: i use qbzr for diff and log though, it's a great piece
[19:39] <jetole> do you know how or if it's possible to set a push to launchpad as private?
[19:39] <jelmer> jetole: I don't think you can unless you have a commercial subscription
[19:39] <Jesse> jelmer: a silly question, where can i see the user name other than in the history window
[19:40] <jetole> I haven't had to do a diff in bzr yet but I probably will still use the term since I used that back when I used svn and have used diff -u a million times
[19:40] <jetole> plus I use vimdiff from time to time
[19:41] <Jesse> jetole: i have an incredibly short attention span so when i get back to work i usually "bzr diff -r -1" to see where i was
[19:41] <jelmer> Jesse: the shell
[19:42] <Jesse> jelmer: oh my bad. i shoulda noticed that ...
[19:42] <jetole> Jesse: makes sense although I would use a more current revision then again I am not a programmer but a network admin so I only program occasionally
[19:43] <Jesse> jetole: -1 is at the tip
[19:43] <Jesse> it's kinda pythonic notation
[19:44] <jetole> Jesse: oh I thought that was implying the first revision
[19:45] <jetole> I guess that would be -r 1
[19:45] <Jesse> yeah
[19:46] <Jesse> and if your working tree is clean, then bzr diff -c -1 would also be helpful
[19:56] <LarstiQ> evening
[19:56] <bialix> heya
[19:57] <pygi> hi LarstiQ
[19:57] <pygi> bialix, too
[19:57] <pygi> :)
[19:57] <garyvdm> Hi bialix, pygi, LarstiQ
[19:57] <bialix> hi
[19:57] <bialix> heya!
[19:57] <pygi> oh noes, garyvdm !
[19:58]  * pygi hides
[19:58] <LarstiQ> hi bialix, pygi, garyvdm :)
[19:58] <bialix> lol
[19:58] <garyvdm> Hi jelmeer
[19:58] <bialix> beuno?
[19:58] <garyvdm> err s/jelmeer/jelmer
[19:59] <jelmer> hi garyvdm
[20:18] <garyvdm> bialix: How are the changes for qcommit/qadd/qrevert working for you?
[20:26] <bialix> bbl
[21:00] <jetole> is it possible to have bzr do a push after every commit?
[21:02] <Tak> you could use a bound branch
[21:06] <jelmer> hi Tak
[21:06] <Tak> hello jelmer
[21:06] <Tak> what's up?
[21:14] <LarstiQ> abentley: is http://bundlebuggy.aaronbentley.com/project/bzr/ still supposed to work? I get a Proxy Error
[21:15] <abentley> LarstiQ: Yes.  Presumably, it's hung.
[21:15] <LarstiQ> right
[21:16] <abentley> LarstiQ: restarted.
[21:16]  * LarstiQ hasn't looked at reviewing for a long time, so wasn't sure if bb was supposed to be up
[21:16] <LarstiQ> abentley: thanks
[21:16] <LarstiQ> !
[21:58] <LarstiQ> jam: yeah, will have to go do the development focus dance. Would be nice to have LP ui to mass upgrade branches
[22:02] <LarstiQ> feh, sleep
[22:02]  * LarstiQ will attach his plan for action to #390502 in 10 hours
[22:08] <Kinnison> jelmer: bug number
[22:08] <Kinnison> jelmer: ?
[22:08] <bialix> garyvdm: hi again
[22:08] <garyvdm> Hi bialix
[22:08] <bialix> ask me again
[22:09] <garyvdm> bialix: How are the changes for qcommit/qadd/qrevert working for you?
[22:09] <bialix> I have not used qadd/qrevert recently -- can't say
[22:10] <bialix> I'll try to try them in next days
[22:10] <bialix> qci another thing
[22:10] <bialix> recently I'e made a lot of merges
[22:10] <bialix> and a lot of qci for renamed files/dirs
[22:11] <garyvdm> There was a bug for renamed files which I've fixed.
[22:11] <bialix> I can't say new reports for renamed files/folders is very cool, sometimes I have strange feeling
[22:11] <bialix> nothing specific, but mostly unintuitive
[22:12] <bialix> maybe it's because I've used to old way
[22:12] <bialix> I need more testing
[22:12] <garyvdm> Yes.
[22:12] <bialix> in #lp channel one man said: qbzr is ideal
[22:13] <bialix> err, looks ideal
[22:13] <bialix> garyvdm: so what is your question about? user feedback?
[22:13] <garyvdm> how long ago?
[22:13] <bialix> just
[22:14] <garyvdm> bialix: yes - just wanted some feedback
[22:14] <garyvdm> Maybe I need to relook at renames in qcommit.
[22:14] <bialix> I feel maybe I'd did something differently for new wt view, but you did incredible work
[22:15] <garyvdm> The change makes senses when you look at renames in qbrowse.
[22:15] <garyvdm> Thanks
[22:16] <bialix> unfortunately I have a lot of urgent tasks on my real job
[22:16] <garyvdm> np
[22:16] <bialix> I really need to finish redesign of saved commit/uncommit data
[22:17] <bialix> recently there was bug about qbzr and changes in core cmd_merge
[22:17] <garyvdm> Yes - I should take a look at that.
[22:17] <bialix> maybe we really need to drop merge decoration?
[22:18] <bialix> we can move qpreview support in qmerge
[22:18] <bialix> maybe it will be more intuitive
[22:18] <garyvdm> I don't think that would be better.
[22:18] <bialix> qpreview could be our diff counterpart for qmerge
[22:19] <garyvdm> Yes - but qmerge should just have a ui to append --qpreview to the args
[22:19] <bialix> mmm, no
[22:20] <bialix> I think we need Preview button in left bottom corner, as Diff button in qlog and qci
[22:20] <bialix> so user can press Preview and got the result
[22:20] <bialix> and then actually do merge or not
[22:21] <garyvdm> Ok - I see your point.
[22:23] <bialix> I'm trying to use more q-commands now, esp.with explorer
[22:23] <garyvdm> A question we need to look at then is - for the revisions that are needed to be fetched - do we do a fetched, so that they are then stored in the local repo, or do we just retrieve them temporarily.
[22:23] <bialix> one thing that all the time bother me is selective qdiff
[22:24] <garyvdm> Yes - we don't have a nice solution there.
[22:24] <bialix> what's about at least provide --exclude option for qdiff?
[22:25] <garyvdm> what would that do?
[22:25] <bialix> use case: I have big tree with separate folder with tests data
[22:25] <bialix> very often I want to see changes w/o changes in that folder
[22:25] <garyvdm> Ok - I see
[22:26] <bialix> so I can at least : bzr qdiff -x tests
[22:26] <bialix> are you using -A -M -R -D options?
[22:26] <garyvdm> Nope
[22:27] <bialix> I found them useful, esp. -M
[22:27] <garyvdm> bialix: This is what I would like for you to be able to do (not possible right now):
[22:27] <bialix> so back to your question about fetching
[22:27] <garyvdm> go into qlog
[22:27] <garyvdm> highlight a revision range
[22:27]  * bialix listen
[22:28] <garyvdm> the file list in qlog should show all changes for the revision range.
[22:28] <bialix> yes, it will be nice
[22:28] <garyvdm> At the moment - it only shows for the top revision selected.
[22:28] <bialix> yeah, it's a bit wrong
[22:29] <garyvdm> Then you should be able to select a number of files in that list, and do a diff on them
[22:29] <bialix> and what should be shown in rev details view then?
[22:29] <garyvdm> At the moment, you can only select 1
[22:29] <garyvdm> Maybe show all revisions selected.
[22:29] <garyvdm> One under the other
[22:30] <bialix> maybe
[22:31] <garyvdm> I think that would that give you the ability to do what you want?
[22:31] <bialix> not really
[22:31]  * garyvdm is cold - and going to find a heater.
[22:32] <bialix> our weather tomorrow +37 probably
[22:32] <garyvdm> The command line option --exclude is something we should do anyway.
[22:32] <bialix> there is still problem with qdiff before commit
[22:33] <bialix> maybe new qbrowse with real wt view can help here
[22:33] <garyvdm> What is that problem
[22:33] <bialix> can it be used and qstatus?
[22:34] <bialix> ?
[22:34] <bialix> s/and/as/
[22:36] <bialix> btw about qci and wt view
[22:36] <garyvdm> In qbrowse and qci, qadd, qrevert - you can now highlight a selection of file, and do a diff for that.
[22:37] <bialix> if there is changed empty folder (renamed or added) it shown with + sign to expand the subtree, but there is no subtree
[22:37] <garyvdm> I allways show the + if the item is a directory
[22:37] <garyvdm> That would be easy to change though
[22:38] <bialix> qbrowse is fine, but maybe we can provide qstatus as qbrowse of wt with only changed filter applied?
[22:38] <garyvdm> Like we show in qcommit
[22:39] <bialix> I'd better to not run qci without reason to actually commit
[22:39] <bialix> yes, like we show in qci
[22:39] <garyvdm> Would be simple to do.
[22:39] <garyvdm> Do you think fixing bug 390918 is important?
[22:40]  * bialix looks
[22:41] <bialix> this is the same problem like in qadd?
[22:41] <bialix> we can fix it for qci and built-in qdiff
[22:41] <garyvdm> Well qadd does not have a diff button...
[22:41] <bialix> even w/o adding files to inventory
[22:43]  * bialix checks
[22:43] <garyvdm> Yes - When I filed the bug, I was thinking that we would need to create a temporary inventory, but I have since realized that that is not nessesery.
[22:43] <bialix> hmm
[22:43] <bialix> qadd has Open acion
[22:44] <bialix> why not using qviewer for this instead?
[22:44] <bialix> it runs python scripts
[22:45] <dazjorz> guys, I'd like to drop you a little note
[22:45] <dazjorz> bzr-svn: PERFECT
[22:45] <dazjorz> after reading about Hg-Svn - bzr-svn is a breeze!
[22:45] <david__> Hey all... Trying to get a newer version of bzr working on ubuntu 9.04 so I downloaded the bzr-1.17.tar.gz for jaunty from https://launchpad.net/~bzr/+archive/ppa ....  and I am getting a lot of errors .. can anyone point me in the right direction or help me debug the problem ?
[22:45] <jelmer> dazjorz: thanks :-)
[22:45] <garyvdm> jelmer ^^^
[22:46] <dazjorz> jelmer: I take it you're the developer for bzr-svn or so? :)
[22:46] <bialix> jelmer is our wizard of svn
[22:46] <jelmer> dazjorz: yeah :-)
[22:46] <dazjorz> let me thank you personally jelmer ;)
[22:46] <dazjorz> very, very nice work
[22:46] <bialix> garyvdm: I suppose you're using some std icons for file items in wt view?
[22:47] <bialix> garyvdm: folders are great, but plain files have terrible look on windows
[22:47] <garyvdm> Yes - qt  get the os desktop std icons for folders and files.
[22:48] <garyvdm> I would like to be able to show the icon of the file type.
[22:48] <bialix> QDirModel can use accosiated icons
[22:48] <garyvdm> It can???!!!???
[22:48]  * garyvdm looks
[22:48] <bialix> in explorer
[22:48] <bialix> it can
[22:49] <dazjorz> jelmer: could you explain to me why "svn up" says I'm at rev 5129, and "bzr up" says I'm at rev 4297? :)
[22:50] <jelmer> dazjorz: bzr revision numbers are per-branch, svn revision nunbers are per-repository
[22:50]  * bialix tries to imagebn screenshot for garyvdm
[22:50] <garyvdm> bialix: not in gnome - Must be just windows.
[22:50] <dazjorz> jelmer: ah, so there were 5129-4297 commits outside of my current working directory? :)
[22:51] <garyvdm> I take a look to see how they do it, and reproduce in in TreeWidget
[22:51] <jelmer> dazjorz: yep
[22:51] <dazjorz> thanks
[22:51] <jelmer> dazjorz: "bzr info" should also show you the svn revision number
[22:51] <bialix> garyvdm: http://imagebin.ca/view/tqiRJKJ.html
[22:51] <jelmer> dazjorz: and "bzr log" should show both the bzr and the svn revision numbers
[22:52] <dazjorz> jelmer: I don't see it, but it does tell me the checkout of this branch is at the SVN URL
[22:52] <garyvdm> bialix: Thats cool - I must make TreeModel be able to do that.
[22:52] <dazjorz> jelmer: bzr 1.17rc1, bzr-svn 0.6.3-1
[22:52] <lifeless> moin
[22:52] <garyvdm> Hi lifeless
[22:52] <jelmer> dazjorz: bzr info will only tell you if you run it against the svn URL directly
[22:52] <bialix> garyvdm: QDirModel has fileIcon method
[22:52] <jelmer> moin lifeless
[22:53] <dazjorz> bzr info tells me "Lightweight checkout (format: subversion-wc), under Location, light checkout root and shared repository have ., checkout of branch has the correct checkout URL
[22:53] <lifeless> dazjorz: what sort of problem?
[22:53] <dazjorz> lifeless: none, really - just checking out bzr-svn :)
[22:53] <bialix> garyvdm: I suspect it can provide access to native icos
[22:53] <lifeless> david__: what sort of errors are you getting?
[22:54] <dazjorz> jelmer: you mean in the repository root, or with the URL as an argument? 'cause I don't see the svn rev when I run bzr info https://... too
[22:54] <jelmer> dazjorz: my bad - try with -v
[22:55] <david__> Cannot build extension "bzrlib._chk_map_pyx".
[22:55] <lifeless> david__: any reason you aren't using the prebuilt versions?
[22:55] <garyvdm> bialix: yes - would be nice to be able to use qviewer for unversioned files (like in qadd)
[22:55] <dazjorz> jelmer: ah, yes! it syas Branch history: 4297 revisions, 838 days old; Repository: 5130 revisions
[22:55] <dazjorz> says*
[22:56] <david__> lifeless: the latest prebuilt version I could get working was 1.13
[22:56] <dazjorz> jelmer: it does say, by the way, that the first revision was in 2007, while when I run svn log -r1 I see the first commit being in 2003
[22:56] <lifeless> david__: if you at the apt url to your sources, you should be able to do apt-get update; apt-get install bzr and have it work
[22:56] <david__> I need 1.16 +
[22:56] <lifeless> david__: before we debug why building it is failing, I'd like to understand why the built version isn't working
[22:57] <jelmer> dazjorz: But this particular branch was perhaps created in 2007 ?
[22:58] <dazjorz> jelmer: my workout directory was changed in commits r1, r2, r6, r7, r8, all february 2003
[22:58] <david__> lifeless: apt-get install 1.13 .. i am a little new to the deb way of doing things .. always been a redhat guy until recently so I am a little out of sorts with ubuntu
[22:58] <dazjorz> jelmer: this is trunk :) from the repository root, it's /trunk/kmess
[22:58] <david__> lifeless: its probably user error
[22:58] <lifeless> david__: ok; try this then
[22:59] <lifeless> as root, make a file /etc/apt/sources.list.d/bzr-ubuntu.list containing:
[22:59] <lifeless> deb http://ppa.launchpad.net/bzr/ppa/ubuntu jaunty main
[22:59] <lifeless> deb http://ppa.launchpad.net/subvertpy/ppa/ubuntu jaunty main
[22:59] <lifeless> those two lines contain our upstream builds of bzr for ubuntu
[22:59] <lifeless> oh, what version did you say you have of ubuntu ? 9.04?
[23:00] <david__> yes
[23:00] <SamB> hmm ... would it be extremely bad style to monkey-patch the traceback module to print variable bindings ?
[23:00] <david__> 9.04
[23:00] <lifeless> where it says jaunty, put intrepid, in those lines
[23:00] <david__> I thought I had jaunty
[23:00] <lifeless> 9.04 is intrepid
[23:01] <david__> are you sure ?
[23:01] <lifeless> lsb_release -a
[23:01] <lifeless> what does that output
[23:02] <david__> No LSB modules are available.
[23:02] <david__> Distributor ID:	Ubuntu
[23:02] <david__> Description:	Ubuntu 9.04
[23:02] <david__> Release:	9.04
[23:02] <david__> Codename:	jaunty
[23:02] <david__> I think 8.x in intrepid
[23:02] <lifeless> oh, man,
[23:02] <lifeless> <- E NO MORNING CAFFEINE
[23:02] <jelmer> abentley: hi
[23:02] <abentley> jelmer: hi.  On a call.
[23:02] <lifeless> ok, you have jaunty, and i comes before j
[23:03] <david__> it happens .. thanks for your help .. so will this make apt-get install the newer version ?
[23:03] <jelmer> abentley: ok
[23:03] <lifeless> its one step of four to do that
[23:03] <david__> oh
[23:03] <SamB> why are you using apt-get?
[23:03] <lifeless> the next step is to tell your system about the gpg keys that are used by these repositories
[23:03] <SamB> get with the times -- aptitude is the thing ;-P
[23:03] <dazjorz> jelmer: do you want me to report failing tests in "bzr selftest svn" or do you already know? :)
[23:04] <jelmer> dazjorz: please report them
[23:04] <david__> lifeless: how does that work
[23:04] <lifeless> to do this,  run "sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXXXXXX" with the XXXXXX bit replaced by the key ids for the two repositories we listed
[23:05] <lifeless> the key id is found on the web page - https://edge.launchpad.net/~subvertpy/+archive/ppa and https://edge.launchpad.net/~bzr/+archive/ppa
[23:05] <lifeless> (take thy bit after the 1024R/
[23:05] <dazjorz> jelmer: all three in one bug or one bug for every failure? :)
[23:06] <dazjorz> correction, all six, up til now
[23:07] <david__> lifeless: thanks thats done
[23:07] <Kinnison> lifeless: has jelmer filed the interesting bug about stacked branches and ghosts?
[23:07] <dazjorz> seven :)
[23:07] <lifeless> david__: now, sudo apt-get update
[23:07] <lifeless> this should complete without errors
[23:07] <SamB> okay ... I want to MONKEY-PATCH the traceback MODULE!
[23:07] <david__> it dfid
[23:07] <david__> did
[23:07] <lifeless> and you can then sudo apt-get install bzr
[23:07]  * SamB is zippyizing that to attract attention
[23:08] <david__> lifeless: perfect .. thank you so much
[23:08] <jelmer> dazjorz: depends on the error message
[23:08] <jelmer> dazjorz: what platform are you on?
[23:08] <lifeless> david__: no problem
[23:08] <dazjorz> jelmer: kubuntu karmic, normal linux (2.6.31), amd64, with bzr 1.17rc1 and python 2.6.2
[23:09] <dazjorz> jelmer: after 1100 tests, I get 3 err, 2 fail, 2 xfail
[23:09] <dazjorz> the err are all "delete() takes no keyword arguments"
[23:09] <lifeless> Kinnison: I don't know
[23:09] <lifeless> Kinnison: ask him :P
[23:10] <Kinnison> jelmer: didja? huh? didja? :-)
[23:10] <Kinnison> jelmer: I'm only bugging you so that it doesn't get forgotten and lost
[23:10] <Kinnison> jelmer: esp. if it can be fixed before 2.0
[23:11] <jelmer> Kinnison: No, I didn't yet.
[23:11] <jelmer> lifeless: Actually, now that you're around
[23:12] <jelmer> lifeless: Kinnison was getting NoSuchRevision errors trying to pull from a branch with ghosts into a branch stacked onto lp:launchpad
[23:12] <lifeless> SamB: if you need more details, inspect is easy to work with
[23:12] <lifeless> both in 2a format?
[23:12] <jelmer> lifeless: it looked like the code path that dealt with pullling into a repository with fallback repositories didn't consider ghosts but assumed that everything that wasn't present was going to be brought in by the fetch.
[23:13] <lifeless> jelmer: it deals with ghosts by looping
[23:13] <lifeless> if it doesn't get every revisions, ce la vie; but if it misses texts or inventories the alarms go off
[23:13] <jelmer> Kinnison: Do you still have the backtrace handy?
[23:13] <Kinnison> nope, someone decided to unplug my machine overnight
[23:13] <Kinnison> and the vmware stopped it suspending
[23:14] <dazjorz> jelmer: I'll file them as one bug and if one of them requires more attention, I can re-file it or so.
[23:14] <lifeless> Kinnison: ~/.bzr.log
[23:14]  * Kinnison is remaking it
[23:14] <Kinnison> please wait
[23:14] <lifeless> also, in cases like this, feel free to just file a bug
[23:14] <lifeless> we can mark as dup if it is
[23:14] <Kinnison> nod.
[23:15] <Kinnison> It's a dumb sequence of commands which triggers it
[23:15] <lifeless> speaking of stuff over night
[23:15] <Kinnison> rocketfuel-get decided that it was a good idea to use "bzr branch --no-tree --stacked /path/to/lp-branches/devel foo" and then "bzr pull --overwrite lp:...."
[23:15] <jelmer> lifeless: I promised to file one when we looked at this together, including details, but then never got round to it
[23:15] <lifeless> jelmer: did you see my proposed progress stuff for subunit?
[23:16] <lifeless> Kinnison: thats uhm, an interesting choice
[23:16] <Kinnison> lifeless: I have no idea why it did
[23:16] <Kinnison> http://paste.debian.net/42539/
[23:16] <Kinnison> there's an example
[23:18] <jelmer> Kinnison: yeah
[23:18] <jelmer> s/kinnison/lifeless/
[23:18] <dazjorz> jelmer: http://bugs.launchpad.net/bzr-svn/+bug/403783
[23:19] <Kinnison> lifeless: is that enough for you to reproduce?
[23:19] <lifeless> jelmer: what did you think of it? :)
[23:20] <lifeless> Kinnison: its a dupe, but its an insane command anyway
[23:20] <lifeless> Kinnison: its pulling cscvs on top of lp
[23:20] <Kinnison> lifeless: blame rocketfuel-get
[23:20] <lifeless> file a bug on that
[23:20] <lifeless> or seek help in #launchpad-dev
[23:20] <Kinnison> meh, I fixed it manually by branching them sanely
[23:20] <Kinnison> I assume rocketfuel-update will just cd in and do bzr pull
[23:21] <Kinnison> Mostly I only wanted to branch lp to take a look at how much of my crap was left in it :-)
[23:21] <Kinnison> lifeless: what bug is it a dupe of, so I can subscribe to it?
[23:21] <jelmer> lifeless: So
[23:22] <jelmer> lifeless: I think the + / - bit is clever
[23:22] <lifeless> 402778
[23:23] <jelmer> lifeless: I'd like to be able to use this for Samba as well though, and wouldn't be able to do so in the current situation
[23:23] <jelmer> lifeless: since samba uses testsuites and can only report the number of testsuites that will follow rather than the number of individual tests
[23:23] <Kinnison> lifeless: ta
[23:24] <jelmer> lifeless: It would be nice if the progress indication could take care of that somehow, but I haven't figured out how
[23:24] <lifeless> jelmer: you should definitely reply to the list ;)
[23:24] <jelmer> lifeless: perhaps by somehow indicating how long the rest of the test run will take compared to how long it's already been running?
[23:24] <jelmer> lifeless: good point (-:
[23:25] <lifeless> so, I did think about that
[23:25] <lifeless> but I don't have great answers
[23:26] <lifeless> one way you could do it is to placehold every remaining suite as being 1 test long
[23:27] <lifeless> and issue +suitecount -1 as you move forward
[23:27] <lifeless> the basic question is 'how do you do a global fraction with local data
[23:30] <jelmer> the problem with placeholding every remaining suite as being 1 test long is that it makes it hard to e.g. give a sane progress bar
[23:30] <jelmer> lifeless: perhaps we can allow a fraction as well as a number?
[23:31] <lifeless> how does that avoid the global/local problem
[23:32] <jelmer> lifeless: it allows the test runner to pick whether it'll report using global or local data
[23:33] <lifeless> jelmer: so, say I run two separate test programs
[23:34] <lifeless> how does the first know to adjust for the second
[23:34] <lifeless> or if the tests are on multiple ec2 machines
[23:38] <poolie> hello
[23:38] <lifeless> jelmer: separately, by fractions, how would it work?
[23:38] <lifeless> jelmer: would it essentially set an upper cap on the current set of tests?
[23:44] <jelmer> re
[23:45] <jelmer> (where did the coffee go ??)
[23:45] <jelmer> lifeless: that wouldn't work - only the top-level test runner would be able to use this indication
[23:46] <jelmer> at least, I can't think of a way to make this work properly with multiple test runners
[23:46] <jelmer> you would have to have some way of identifying them to track their progress
[23:48] <lifeless> right
[23:48] <SamB> lifeless: oh, doing the monkeypatching will be the easy part ... the part I'm having trouble with is where to put the code to do the monkeypatching
[23:48] <lifeless> so this is why I think it reduces to just global/local
[23:50] <lifeless> jelmer: I think this is how I'd summarise it:
[23:50] <lifeless> if you have a test runner with more info, it can output that info
[23:51] <lifeless> knowing that there are 50 suites tells you little about the relative time of each suite
[23:51] <SamB> I think I want to monkeypatch traceback.format_tb to do what I want ...
[23:52] <lifeless> While it would be great to get an accurately scaled progress bar, I don't see how that can be done simply.
[23:52] <lifeless> doing a stacked approach like bzr does allows hierarchical knowledge
[23:52] <SamB> but can bzr display the stack of progress bars?
[23:52] <jelmer> lifeless: atm we get a pretty good progress bar using the testsuites, mostly because the run time of the testsuites doesn't vary too wildly
[23:53] <lifeless> which in a large subset of cases permits forward-only progress bars
[23:53] <lifeless> SamB: nothing to do with bzr; and yes it can
[23:53] <SamB> on the terminal?
[23:53] <lifeless> SamB: or a gui
[23:53] <SamB> why isn't it doing that, then?
[23:53] <lifeless> SamB: it does.
[23:54] <SamB> I only ever see one line of progress bars on *my* terminal
[23:54] <lifeless> thats right
[23:54] <lifeless> thats how we render the stack
[23:54] <SamB> except when things screw up
[23:54] <SamB> oh, I meant why not actually display all of the bars?
[23:54] <SamB> or rather ... I wish it could be done
[23:54] <SamB> but that needs termcap/terminfo etc.
[23:54] <lifeless> indeed, it also needs more VWS
[23:55] <SamB> VWS?
[23:55] <lifeless> vertival space
[23:55] <lifeless> erroneous W
[23:55] <SamB> ah
[23:56] <lifeless> so we render it collapsed onto one - the higher tasks take up a smaller fraction of the bar, constrained within the position of the next one down of the stack
[23:56] <lifeless> anyhow
[23:56] <lifeless> jelmer: so
[23:56] <SamB> I don't expect that gets you very far down the stack often ...
[23:56] <lifeless> SamB: you should experiment a little
[23:57] <SamB> I mean, how many characters wide is the progress bar, usually?
[23:57] <lifeless> bzrlib.ui.ui_factory.get_nested_progressbar()
[23:57] <SamB> not that many!
[23:57] <lifeless> doesn't need to be
[23:57] <jelmer> lifeless: so
[23:57] <lifeless> anyhow, I'm switching my conversation back. pop()
[23:57] <SamB> oh ... anyway ...
[23:57] <jelmer> lifeless: I think your proposal is definitely a step forward
[23:58] <jelmer> lifeless: I might propose an extension in the future to make things easier for Samba
[23:59] <lifeless> jelmer: my goals are: streams remain concatable; smarter code can do more than concat; runners with more data can do better.