[12:03] <NfNitLoop> Yeah, IIRC bzr-svn will create consistent changeset IDs.
[12:03] <NfNitLoop> so you could even each do your own checkout from the svn source.
[12:03] <NfNitLoop> (Not sure how one handles sharing between svk instances... but that's probably just due to ignorance on my part.)
[12:19] <NfNitLoop> Hmm.  It looks like bzr-svn can't handle HTTP authentication for a svn repo.
[12:19] <NfNitLoop> Unless I'm missing something.    Why do I always have to be the corner case?  :p
[12:20] <pygi> :D
[12:23] <NfNitLoop> Ooh, nope, found a bzr bug.
[12:24] <beuno> NfNitLoop, 0.90 has that problem, 0.18 should work fine
[12:24] <beuno> at least I've used http auth since 0.12
[12:28] <NfNitLoop> Hmm, 0.90 was using urllib, so I installed pycurl.
[12:28] <NfNitLoop> Now it's complaining that my cert is invalid.  :p
[12:28] <NfNitLoop> (self-signed)
[12:29] <lifeless> urllib is fine
[12:29] <lifeless> NfNitLoop: for svn
[12:29] <lifeless> NfNitLoop: use the svn client to authenticate
[12:29] <lifeless> NfNitLoop: then it will work, nothing to do with urllib/pycurl
[12:29] <lifeless> beuno: ^ FYI
[12:30] <NfNitLoop> Hmm.   how so?
[12:30] <NfNitLoop> I'm doing bzr branch, how would I use the svn client to auth?
[12:32] <beuno> aaah, right, I use http auth directly to bzr, don't use svn at all  :D    thanks lifeless for the info though
[12:32] <NfNitLoop> beuno: do you know what he's talking about?
[12:33] <NfNitLoop> How do I auth with the client?
[12:33] <NfNitLoop> lifeless: or do you mean do 'svn co', then use bzr on the checked-out copy?
[12:33] <NfNitLoop> that's not really what I want.  I want a native bzr branch. :)
[12:33] <lifeless> NfNitLoop: right
[12:34] <lifeless> NfNitLoop: 'svn ls URLYOUWANTTOUSEWITHBZR'
[12:35] <NfNitLoop> still not quite following.
[12:35] <NfNitLoop> svn list isn't going to get me a checkout. :p
[12:35] <NfNitLoop> (or branch)
[12:40] <lifeless> NfNitLoop: it will cause svn to authenticate
[12:40] <lifeless> and cache the credentials for libsvn to use
[12:40] <lifeless> which bzr-svn uses
[12:40] <lifeless> you could actually try doing what I suggest you know
[12:41] <lifeless> hmm, I'm grumpy today; excuse me and I'll get breakfast - blood sugar++
[12:42] <beuno> irc is so cheap...  :p
[12:48] <NfNitLoop> lifeless: aah.  well I suspect it still wouldn't work since it's barfing on my SSL cert at the moment.
[12:48] <NfNitLoop> but if authentication fails I'll give that a try, thanks.
[01:17] <NfNitLoop> lifeless: Hmm.  svn ls <url> works and has saved my auth.   bzr branch <url>  still fails saying:  Unable to handle http code 401: expected 200 or 404 for full response.
[01:17] <NfNitLoop> Oh, 401.  *doh*
[01:17] <NfNitLoop> No, yeah, that's authentication failed.
[01:17] <NfNitLoop> I thought it was 404 for a sec.  :p
[01:24] <lifeless> jelmer: ^
[01:28] <jelmer> NfNitLoop: please try prefixing with svn+
[01:29] <jelmer> bzr branch svn+<URL>
[01:31] <NfNitLoop> I did that and got the same error...
[01:31] <NfNitLoop> though I jsut tried  bzr svn-imoprt svn+https://...
[01:31] <NfNitLoop> and it's... doing something.
[01:31] <NfNitLoop> Oh, nope, failed.  "Undefined tunnel scheme: https"
[01:32] <NfNitLoop> (This is all with bzr 0.90 and bzr-svn 0.41, btw.)
[01:33] <jelmer> please try the current 0.4 brancxh
[01:33] <jelmer> is this a public svn repository?
[01:34] <NfNitLoop> No.
[01:34] <NfNitLoop> It's using HTTP Auth & SSL.
[01:34] <jelmer> that should work ok in current 0.4
[01:35] <NfNitLoop> 0.41?  Or the dev branch?
[01:35] <jelmer> the dev branch
[01:35] <NfNitLoop> ok, I'll download that and give it a shot.
[01:35] <jelmer> I've fixed the handling of svn+https://
[01:35] <NfNitLoop> aaah.
[01:35] <NfNitLoop> cool. Thanks!
[01:44] <NfNitLoop> jelmer: It's doing something now.  It didn't like that I had it in a shared repo directory, so I'm doing it outside of there.
[01:44] <NfNitLoop> Is it not able to use shared repos?
[01:44] <lifeless> it uses an unsupported repository format
[01:45] <lifeless> the nested trees one
[01:45] <lifeless> if your repo format is in that format it will work, but you can't interoperate with most bzr trees
[01:46] <NfNitLoop> ah, I could create a shared repo in that format?
[01:46] <NfNitLoop> That's good.  This could get large.  :)
[01:46] <jelmer> yes, 'bzr init-repo --dirstate-with-subtree'
[01:49] <jelmer> hey, looks like tortoisehg is based on tortoisebzr
[02:00] <spiv> Good morning.
[02:01] <jelmer> 'morning
[02:03] <NfNitLoop> Hmmm.  I use the typical svn trunk/ branches/ tags/ root.  Should I be branching the root, or trunk?
[02:05] <lifeless> igc: morning
[02:05] <lifeless> spiv: morning
[02:05] <lifeless> spiv: visiting?
[02:05] <lifeless> igc: ring my mobile if you would, lets talk commit
[02:06] <igc> shall do - give me a few more minutes to scan emails
[02:06] <jelmer> NfNitLoop: You should be branching /trunk or use svn-import to import the repository (which will splitup into multiple branches)
[02:08] <spiv> lifeless: yeah, I'll head off fairly soon.
[02:08] <NfNitLoop> Ooooh.  *lightbulb*    :)
[02:16] <NfNitLoop> So it's plugging away at my very large svn repo.   and in the meantime, in another dir, I'm trying to check out a tiny test repository.
[02:16] <NfNitLoop> if I try to branch /trunk, I get ...
[02:16] <NfNitLoop> KeyError: 'readme.txt'
[02:17] <NfNitLoop> and if I try to svn-import the root, I get ...
[02:17] <NfNitLoop> bzr: ERROR: Invalid revision-id {None} in SvnRepository(my repo URL)
[02:19] <jelmer> which bzr-svn branch are you using?
[02:19] <NfNitLoop> the 0.4 branch.
[02:19] <NfNitLoop>   parent branch: http://people.samba.org/bzr/jelmer/bzr-svn/0.4/
[02:20] <jelmer> hmm, that should be ok
[02:21] <jelmer> hey, find_previous_heads() has been deprecated?
[02:21] <NfNitLoop> I wonder if my little test repo is somehow a "special" (ie: foobar'd) case.
[02:22] <jelmer> bzr-svn has worked ok against a large number of repositories with all sorts of weird changes in them
[02:23] <jelmer> it's been a while since I've come across a pull bug
[02:24] <NfNitLoop> the 'readme.txt' it's complaining about was checked in in the first revision, along with the dirs trunk/ branches/ and tags/
[02:24] <NfNitLoop> (I notice that is not the case in my larger... seemingly working repo)
[02:25] <poolie> hi jelmer
[02:25] <poolie> lifeless, i have no objection to removing get_deltas
[02:25] <jelmer> NfNitLoop: if the test repo doesn't contain any private data, can you perhaps send me a svn dump of it?
[02:25] <jelmer> 'morning poolie
[02:26] <igc> morning poolie
[02:26] <NfNitLoop> jelmer: Sure.
[02:27] <NfNitLoop> better yet, I'll try to reproduce it.
[02:27] <NfNitLoop> and send you that, for a smaller case.
[02:27] <poolie> igc, hi
[02:27] <jelmer> NfNitLoop: cool, thanks! My address is jelmer@samba.org
[03:07] <lifeless> poolie: then send a bb:approve :)
[03:07] <poolie> i thought i read an rfc about it not a patch
[03:07] <lifeless> keep reading
[03:07] <lifeless> :)
[03:08] <poolie> this is your thread about knits and get_deltas()?_
[03:09] <poolie> i see
[03:13] <poolie> i'm reading it
[03:15] <wedderburn> hello all, i have a rather cryptic message spat out when i pushed a new revision of something - "ERROR: Can't rename /srv/sm-ng/push-branches/00/00/0c/bf/.bzr/repository/lock/wbei8a3n94.tmp to /srv/sm-ng/push-branches/00/00/0c/bf/.bzr/repository/lock/held: /srv/sm-ng/push-branches/00/00/0c/bf/.bzr/repository/lock/held already exists" does anyone have a idea what it means, thanks
[03:16] <poolie> wedderburn, it's failing to acquire a lock
[03:16] <poolie> um, it's strange that you're getting this rather than just 'waiting for  lock'
[03:16] <poolie> jml, spiv: ping?
[03:17] <jml> poolie: hi
[03:17] <jml> poolie: wassup?
[03:17] <poolie> jml, can you look on the sm at the directory wedderburn talks about
[03:17] <wedderburn> thanks
[03:17] <poolie> wedderburn, did bzr just stop after giving that error?
[03:18] <poolie> can you try it again with
[03:18] <poolie> bzr -Derror push .....
[03:18] <jml> poolie: alas no, I don' t have access to that yet.
[03:18] <poolie> really
[03:18] <wedderburn> poolie: yes bzr stopped after the erro
[03:18] <wedderburn> r
[03:19] <wedderburn> i'll try the command now and tell you what it spits out
[03:21] <jml> poolie: yeah, it's in the works though.
[03:22] <wedderburn> poolie: heres what it spat out http://andrew.wedderburn.googlepages.com/bzr-error.txt
[03:28] <poolie> thanks
[03:32] <poolie> spiv, the traceback shows we're raising this because we don't understand what the sftp error means
[03:32] <poolie> did anything change with the sftp server there?
[03:32] <jml> yes.
[03:33] <poolie> jml, does that error message look like something you wrote?
[03:34] <jml> poolie: it could well be.
[03:34] <spiv> I guess bzr+ssh:// instead of sftp:// might dodge this error.
[03:36] <spiv> That error message comes straight out of twisted.vfs.backends.osfs I think.
[03:36] <spiv> Where it raises AlreadyExistsError
[03:37] <spiv> Which the sftp adapter should turn into a FX_FILE_ALREADY_EXISTS in the SFTP protocol.
[03:40] <poolie> wedderburn, could you try substituting bzr+ssh for sftp in the url please
[03:40] <lifeless> I thought we had acceptance tests for bzr's treatmnet of the sftp behaviour
[03:40] <lifeless> in lp
[03:40] <poolie> yes, but apparently the lp behaviour has changed
[03:40] <poolie> this is a different misbehaviour to the one you might be thinking of
[03:41] <jml> lifeless: we do.
[03:41] <wedderburn> poolie: i'll do that now
[03:41] <lifeless> poolie: the ExistingContent review comment
[03:42] <lifeless> poolie: I'm confused, when I look at the patch without the trimming the fmt string is quite clear to me
[03:42] <lifeless> poolie: so I'm not sure - are you asking me to duplicate the format string into the doctstring?
[03:42] <jml> poolie: the lp behaviour changing should be caught by lp acceptance tests.
[03:42] <lifeless> poolie: I mean, what comes to mind is 'This is an exception raised when the content being inserted is already present'
[03:42] <lifeless> poolie: and that doesn't seem helpful.
[03:43] <poolie> you could change the docstring to just a comment
[03:44] <poolie> it seems like a very strange docstring at presentt
[03:44] <wedderburn> poolie: Unable to obtain lock lp--1216787156:///lock
[03:44] <wedderburn> held by andrew.wederburn@gmail.com on host andrew-laptop [process #10620] 
[03:44] <wedderburn> locked 22 hours, 50 minutes ago
[03:44] <wedderburn> Will continue to try until 02:47:26
[03:44] <poolie> wedderburn, thanks
[03:44] <poolie> assuming that process is not still running
[03:44] <poolie> just use bzr break-lock $url
[03:44] <wedderburn> ok
[03:44] <lifeless> poolie: I'll make it a comment then. Though I thought you wanted introductory-api notes in docstrings, which is why I added it (most exceptions dont have docstrings at all)
[03:45] <poolie> on either sftp or bzr+ssh
[03:45] <poolie> iirc the thing i said we should document the introduction of was rpcs
[03:46] <lifeless> oh, I took it to be all api's
[03:47] <wedderburn> poolie: and now?
[03:48] <poolie> now you should be fine to push
[03:49] <wedderburn> k
[03:50] <wedderburn> poolie: using the sfpt://... ?
[03:50] <poolie> either way
[03:50] <poolie> bzr+ssh will be a bit faster
[03:51] <wedderburn> poolie: :) it works now thanks
[03:51] <poolie> thanks for reporting it
[03:52] <wedderburn> :)
[03:52] <wedderburn> well im off thanks for all the help everyone
[03:59] <spiv> poolie: is it ok with you if I merge http://bundlebuggy.aaronbentley.com/request/%3C20070903164726.GA22003@steerpike.home.puzzling.org%3E ?
[04:00] <lifeless> does it fix the bugs with tag branhces on initial pull
[04:00] <lifeless> that is does it stop using the tarball hack ?
[04:00] <spiv> It stops using the tarball hack.
[04:00] <lifeless> then I'm pro it going in noweth
[04:00] <lifeless> but today would be the latest I think
[04:00] <spiv> I don't know about the tags issue; should there be an automated test written for that?
[04:01] <spiv> (It stops using the tarball hack on the client entirely; it will fallback to VFS operations)
[04:02] <mkanat> Is there a machine-readable output that would consistently allow me to tell that a plugin is installed?
[04:02] <lifeless> bug 129717 I think
[04:02] <ubotu> Launchpad bug 129717 in bzr "Bizarre error when checking out from bzr server mode" [High,Triaged]  https://launchpad.net/bugs/129717
[04:02] <mkanat> I don't necessarily want to rely on "bzr plugins" since that text could change.
[04:02] <lifeless> mkanat: python -c 'import bzrlib.plugins.pluginname'
[04:02] <mkanat> lifeless: Thank you. :-)
[04:02] <poolie> spiv, thanks for that
[04:03] <igc> lifeless: looking at http://bundlebuggy.aaronbentley.com/?selected=pending&unreviewed=n, have the manually tracked ones been merged yet? If not, do we want them in 0.91?
[04:04] <lifeless> igc: we're in freeze
[04:04] <lifeless> igc: so everything needs poolies magic pixie dust
[04:04] <NfNitLoop> jelmer: Hmmm.  checking out my big repository fails.   It locks up my box for a while, while the HD thrashes, then it just says "Killed" and stops.   The box only has 512M RAM.  How RAM intensive is it?
[04:04] <lifeless> poolie: can I merge the one you approved for 0.91 ?
[04:04] <poolie> i approved a few...
[04:04] <igc> sure. I'll take that as a "no" to not yet merged.
[04:05] <igc> all up, we have 10 changes approved or conditionally approved
[04:06] <jelmer> NfNitLoop: yes, there's a leak in the python-subversion bindings (see bug 54253)
[04:06] <ubotu> Launchpad bug 54253 in bzr-svn "Excessive memory usage in bzr-svn" [Undecided,Confirmed]  https://launchpad.net/bugs/54253
[04:06] <jelmer> NfNitLoop: but you can restart the process if you're importing to a shared repository
[04:06] <mkanat> lifeless: And is there a way to get from bzr the path to its default python interpreter?
[04:06] <poolie> for http://bundlebuggy.aaronbentley.com/request/%3C1187769804.17572.104.camel@localhost.localdomain%3E,
[04:06] <lifeless> mkanat: I think bzr --version will show that
[04:07] <poolie> i agree with john's comments
[04:07] <mkanat> lifeless: Yes, it does! :-)
[04:07] <mkanat> lifeless: Thank you. :-)
[04:07] <poolie> i guess there is no small way to address them?
[04:10] <igc> poolie: fyi, one change I had hoped we'd see in 0.91 is this one: http://bundlebuggy.aaronbentley.com/request/%3Cfaekm0$qvu$1@sea.gmane.org%3E
[04:11] <lifeless> poolie: I will fix that one, haven't yet.
[04:11] <ubotu> New bug: #137668 in bzr "bzr doesn't understand new 'directory already exists' message from codehosting sftp server" [Undecided,New]  https://launchpad.net/bugs/137668
[04:11] <NfNitLoop> jelmer: is there any way to do it in small chunks?
[04:11] <poolie> http://bundlebuggy.aaronbentley.com/request/%3C1187769900.17572.106.camel@localhost.localdomain%3E
[04:11] <poolie> same, ok to merge with john's change
[04:11] <NfNitLoop> jelmer: instead of waiting for it to get killed?  :p
[04:11] <igc> abentley approved it but I wanted some changes ...
[04:13] <igc> we haven't heard back from Fabio though and it would be a shame to see it fall into limbo because no one was driving it
[04:13] <igc> poolie: any thoughts?
[04:13] <lifeless> igc: that patch is broken
[04:13] <lifeless> +                filtered_list.append(f)
[04:13] <lifeless> +        except:
[04:13] <lifeless> +            invalid_filenames.append(f)
[04:13] <lifeless> bare except: it will stop ctrl-C
[04:14] <igc> lifeless: can you explain further?
[04:15] <lifeless> ctrl-C at the right time will make the path being sorted into an invalid path, even if its valid, and bzr will continue rather than exiting
[04:15] <lifeless> 'except:' is verboten
[04:15] <igc> Ahhh
[04:15] <lifeless> its only *ever* ok when there is a 'raise' immediately after it.
[04:15] <poolie> interesting:
[04:15] <poolie> mbp@grace% bzr merge http://bundlebuggy.aaronbentley.com/download_patch/%3C1188888611.6248.3.camel@nemo%3E
[04:15] <poolie>  M  NEWS
[04:15] <poolie> All changes applied successfully.
[04:15] <poolie> bzr merge   1.08s user 0.09s system 35% cpu 3.282 total
[04:15] <poolie> mbp@grace% bzr di
[04:22] <lifeless> poolie: can I merge http://bundlebuggy.aaronbentley.com/request/%3C1189031297.27465.2.camel@lifeless-64%3E for 0.91
[04:23] <poolie> with those doc changes
[04:23] <poolie> yes
[04:23] <lifeless> thanks
[04:23] <poolie> thanks for writing it :)
[04:29] <mkanat> lifeless: Ah, I have a proooooblem. :-)
[04:29] <mkanat> lifeless: I need to know if a plugin is installed by the functionality it offers, not by its name.
[04:29] <mkanat> lifeless: That is, because the name can be anything the directory is called, and I need some reliably way of determining if the functionality is available.
[04:30] <mkanat> *reliable
[04:30] <lifeless> mkanat: no
[04:30] <lifeless> mkanat: plugins have canonical names
[04:30] <lifeless> plugins depend on other plugins by name
[04:30] <lifeless> its not installed if its not under its official name
[04:30] <mkanat> lifeless: Hrm.
[04:31] <mkanat> lifeless: And that's true for xmloutput?
[04:31] <mkanat> lifeless: bzrtools I have no problem with. :-)
[04:31] <mkanat> That will always be called bzrtools. There's almost always a distro package for it, too.
[04:32] <lifeless> mkanat: this is the contract for bzr pluigns
[04:32] <lifeless> if someone does something different they haven't isntalled the plugin correctly
[04:33] <mkanat> lifeless: Okay.
[04:33] <lifeless> I mean, if I put bzr on my path as 'bzr-temp-copy'
[04:33] <lifeless> you can't be expected to figure that out
[04:34] <lifeless> this is the same problem basically
[04:34] <mkanat> Hahaha. :-) Okay. :-)
[04:35] <lifeless> given you want to be able to use a command
[04:35] <lifeless> just run 'bzr xmloutput'
[04:35] <lifeless> and if you get an error show the user the output from bzr
[04:35] <mkanat> Well, that particular plugin doesn't provide a command.
[04:35] <lifeless> it provides options though
[04:35] <mkanat> But I'm using the import method, that works fine. :-)
[04:35] <mkanat> Yeah, it does.
[04:35] <lifeless> and if its not installed the option will error
[04:36] <mkanat> Just so you know what I'm doing, I'm writing tests that I have to skip if bzrtools and xmloutput aren't installed.
[04:37] <mkanat> (But from another language.)
[04:38] <lifeless> ok
[05:24] <mkanat> BTW, <3 bzr, guys. :-)
[05:25] <mkanat> Every time I have to use CVS or some other crazy VCS, I wish it was just bzr.
[05:25] <NfNitLoop> mkanat: y'know, bzr can act as a front-end to other VCSes.  :)
[05:26] <NfNitLoop> (I'm playing with bzr-svn this evening.)
[05:26] <mkanat> NfNitLoop: Cool. :-)
[05:26] <mkanat> NfNitLoop: Well, see...I'm the author of a library that interfaces with VCSes. :-)
[05:26] <mkanat> So I have to actually *use* them. :-)
[05:26] <NfNitLoop> Doh.  :p
[05:27] <mkanat> I thought bzr was kind of hard to understand for newbies, until I tried git. :-)
[05:28] <NfNitLoop> mkanat: Yeah...  I tried out several DVCSes a while back...
[05:28] <NfNitLoop> I think I started with gnu arch.
[05:28] <NfNitLoop> it seemed...  tedious.
[05:28] <mkanat> Wow. :-)
[05:28] <mkanat> Yeah, I'd imagine.
[05:28] <mkanat> I don't anybody really uses Arch anymore.
[05:28] <NfNitLoop> Yeah.  It was just the one I'd heard of.
[05:29] <NfNitLoop> so I started with it.
[05:29] <mkanat> Yeah, as far as ease of use and sensible, complete implementation of features, bzr is the best one I've used.
[05:29] <NfNitLoop> anyone:  How do I remove a lock on a shared repository if the process that created the lock has died?   is it safe to rm -r .bzr/repository/lock/  ?
[05:30] <jml> NfNitLoop: bzr break-lock
[05:30] <NfNitLoop> jml: thanks.  I figured there'd be some command.  :)
[05:33] <NfNitLoop> jelmer: Doh.   Yeah, resuming from a shared repo didn't work too well.   :p
[05:33] <NfNitLoop> jelmer: bzr: ERROR: exceptions.KeyError: 'design'  (the name of a folder in the repo)
[05:48] <abentley> lifeless: pong
[05:50] <abentley> spiv: I'd prefer to merge the reconcile fix first.  I don't want to break fetch without providing a way to fix fetch.
[06:05] <lifeless> abentley: ok, I will do a revision-index cross check patch
[06:05] <abentley> Wow, you really have a hate on for RevisionParentsProvider.
[06:06] <lifeless> heh, I don't think I would have put it that way
[06:17] <lifeless> but I don't think we should add additional code to work around other code that we should have being absent
[06:18] <lifeless> and for things like check/reconcile on a remote repository this is clearly going to be doing much more work than we want
[06:24] <lifeless> a$ rm -rf .bzr && bzr --no-plugins  init --experimental && bzr --no-plugins add > /dev/null && time ~/source/baz/repository/bzr --no-plugins  commit -m "Initial commit"  -q
[06:24] <lifeless> 
[06:24] <lifeless> real    2m12.203s
[06:24] <lifeless> thats annotation-less packs
[06:24] <lifeless> user    1m58.647s
[06:24] <lifeless> sys     0m6.256s
[06:24] <lifeless> I'm aiming for 1m12
[06:24] <lifeless> so 60 seconds to go
[06:25] <abentley> This is for moz?
[06:25] <lifeless> yup
[06:25] <lifeless> 550MB of source, 55K files
[06:26] <abentley> I suppose delta generation time doesn't come into play at all for the initial commit?
[06:26] <lifeless> not at all
[06:26] <lifeless> unless we were to do deltas-against-NULL
[06:26] <abentley> Which should be fast-pathed.
[06:27] <lifeless> so initial commit is great for finding constant factors that will affect all commits
[06:27] <lifeless> like zip overhead
[06:27] <lifeless> and api friction
[06:27] <lifeless> in should be only slightly slower than for path in paths: compress(path)
[06:27] <keir> lifeless, i'm pretty sure i know how to build a graphindex nicely now. compressed and tightly encoded and spatially coherent.
[06:27] <lifeless> s/in/it/
[06:28] <lifeless> keir: thats excellent news. Care to describe it in a mail to me/the list ?
[06:28] <keir> yes
[06:28] <abentley> Yeah, I was a little worried about your earlier talk of removing get_deltas, but then I realized it's not what I'm using to get matching blocks.
[06:28] <lifeless> abentley: yah, that why I was pinging in fact, but Ifigured it out
[06:28] <keir> lifeless, it's going to take me some time to write up, but absolutely. there are a couple little things i have to think about, but hopefully i'll have a description ready before tmrw night
[06:29] <lifeless> keir: cool. Don;t worry about those things- I mean do worry, but don't block on documenting your thoughts, because perhaps I can offer some feedback that will help you with them
[06:29] <abentley> I was expecting Xdelta was going to go into pack repos, but I haven't heard anything on that front for some time.
[06:30] <lifeless> keir: also tomorrow is a public holiday for me, so you may not hear back till my monday - but it won't be me ignoring you :)
[06:30] <lifeless> abentley: IIRC there was some performance issue there.
[06:30] <abentley> Oreilly?
[06:30] <lifeless> abentley: and open questions about bytes vs lines as the delta form
[06:30] <lifeless> didn't john say xdelta application was slower due to bytecode interpretation or something?
[06:31] <abentley> I thought it was creaming MPdelta, which was creaming knit in turn.
[06:32] <lifeless> anyhow, I'm happy to support a few pack repository versions if we can deliver something moderately high performing in the short term and rev it - getting past the current 'you are too slow kthxbye' is the immediate priority
[06:32] <lifeless> not that I want to do that
[06:32] <lifeless> just that I think its ok
[06:33] <abentley> I am okay with that also.
[06:33] <abentley> But I would rather have Xdelta or MPdiff than knit.
[06:34] <abentley> I'll talk to John.
[06:35] <lifeless> why is mpdiff faster than knits ?
[06:36] <lifeless> is it faster in both directions, and does it need more or less data on average to construct a text (how do full-texts fit in etc)
[06:36] <lifeless> if this is already on-list thats fine
[06:41] <abentley> lifeless: the mpdiff text reconstruction algorithm doesn't generate intermediate texts, and never inserts in the line list, only appends.
[06:42] <abentley> I don't know whether it's faster or slower for creating the mpdiff.
[06:43] <abentley> "more or less data" is tricky to answer.  More revisions, less data in each revision.
[06:43] <poolie> lifeless, i'm confused - the 'implement chunked encoding' patch is not yet approved
[06:43] <lifeless> poolie: thats not he patch under discussion
[06:44] <lifeless> poolie: so indeed, you are confused :)
[06:45] <lifeless> abentley: ok; and presumably this no-insertion behviour is hard to do for knits
[06:45] <keir> lifeless, ok so here's the short version of what i'm thinking
[06:45] <abentley> lifeless: You just have to massage the offsets more.
[06:46] <ubotu> New bug: #137681 in bzr "bzr: ERROR: Unable to delete transform temporary directory" [Undecided,New]  https://launchpad.net/bugs/137681
[06:46] <keir> lifeless, there are three parts to a graphindex. 1) index preamble (an index to the index) 2) the index, which lets you map from string key -> integer hash, and also contains the values 3) the graph store, which is just like the current storage scheme, but the keys are entirely ommitted.
[06:46] <abentley> But what I really dislike about knits is they don't properly delta the final newline, and screw up attempts to extract matching blocks.
[06:47] <abentley> You have to extract the fulltext to get the matching blocks out.
[06:47] <keir> 1) is a lzw'd or other streaming compressed index (just like in a book) into the 'chunks' of the index.
[06:48] <abentley> They also don't provide an indication of the number of lines in the file.
[06:48] <keir> 2) is a series of 'chunks' each indepently lzw'd,  which contain key->hash,val_len,val ...
[06:48] <keir> each 'chunk' in 2) is the same size. because of how lzw works, we can still make each chunk mostly the same size
[06:49] <keir> then 3) is just a list of the edges as straight up offsets; the id of a key is just the byte offset in section 3) of that node
[06:49] <keir> section 3 is toposorted, such that the enumeration of the hashes corresponds to the toposort of the keys
[06:50] <keir> this has a variety of nice properties for traversing the graph
[06:50] <lifeless> abentley: these are all good points.
[06:51] <keir> there is some debate on whether to include the values in 2) or 3). in 2), the keys will be compressed; but if they are in 3), then if you only care about the values it traversal, you get them for free without having to re-query the index for each key
[06:51] <lifeless> abentley: I'd be happy to see packs using mpdiffs given them. Its not top of my priority list (my time on bzr at the moment is really focused on helping out with perf - I'm not in the bzr department @ canonical these days)
[06:52] <abentley> lifeless: I am happy to ride my hobbyhorse into pack land :-)
[06:52] <lifeless> abentley: and while they will clearly help with performance, I'm currently tackling things surrounding the delta layer rather than the deltas themselves :)
[06:52] <keir> then there is the debate on how big to make the chunks.
[06:53] <abentley> lifeless: is revno 2744 current?
[06:53] <lifeless> keir: this is sounding very interesting; An email would be much easier for me to digest as I'm deep in commit's guts at the moment and I'll need to look at this later
[06:54] <keir> lifeless, sure
[06:54] <lifeless> abentley: pushing
[06:54] <keir> lifeless, there is some measurements i need to do regarding lzw tradeoffs; i'll do those now then write it all up
[06:54] <lifeless> abentley: (its not, but the changes are mainly merges of other branches)
[06:55] <lifeless>  rm -rf .bzr && bzr --no-plugins  init --experimental && bzr --no-plugins add > /dev/null && time ~/source/baz/repository/bzr --no-plugins  commit -m "Initial commit"  -q
[06:55] <lifeless> real    2m2.092s
[06:55] <lifeless> user    1m48.035s
[06:55] <lifeless> sys     0m5.564s
[06:56] <keir> lifeless, one last question: how big do you think it's reasonable to make the 'minimum size read chunk'?
[06:59] <lifeless> 4K? thats a disk cluster isn't it ?
[07:00] <keir> ok, but if you're roundtripping, is't something like 32k more reasonable? if we assume people are on broadband...
[07:00] <lifeless> well
[07:00] <lifeless> transport.get_recommended_page_size() might be your friend :)
[07:01] <keir> hmm, but the indexes will have a block structure which is independent of the transport
[07:01] <keir> so we'll need to find a reasonable middleground.
[07:01] <lifeless> so
[07:01] <lifeless> more data -> more processing
[07:01] <lifeless> less latency impact
[07:02] <lifeless> so when there is low latency you want low processing
[07:02] <lifeless> and when there is huge latency we're willing to spend more cpu
[07:02] <lifeless> if you were to (say) have a 4K page in the index, which must be entirely read if read at all
[07:03] <lifeless> then you can do ceiling(recommended_page_size/4k) * 4k
[07:03] <lifeless> to get the amount to read
[07:03] <keir> yeah, ok
[07:03] <keir> the problem is that if you're compressing blocks, you don't win much on 4k
[07:03] <keir> you really want > 16k
[07:03] <lifeless> we win plenty on 200 bytes in knits
[07:03] <keir> really!
[07:03] <keir> ok
[07:04] <keir> nevermind then, i'll make it a knob
[07:04] <lifeless> 44 bytes is the break-even for libz
[07:04] <keir> and libz is lzw?
[07:04] <lifeless> lzw78 IIRC
[07:04] <lifeless> might be 77. I dunno offhand, check the library :)
[07:05] <lifeless> so for compression we win on entropy in the data
[07:05] <keir> ok. i need the following thing for implementation: given a large string, compress until your compressed size is N bytes, then terminate and tell me how much of the source string you chomped, and give me the compressed string
[07:05] <keir> lifeless, you mean we win on lack of entropy
[07:05] <lifeless> the amount we win and the entropy are lnked :)
[07:06] <lifeless> anyhow, that API is a problem for all compressors I can think of
[07:07] <keir> because i'm a C loser, i want to go off and code it in C by using an existing lzw algo
[07:07] <lifeless> every compressor API I've seen has internal state for the current symbols that have not yet selected a terminal representation
[07:08] <lifeless> this state is cleared by calling flush()
[07:08] <lifeless> which in pathological cases can output 1/2 the already processed input IIRC
[07:08] <lifeless> (as the longest match-to-date)
[07:08] <lifeless> particularly for dictionary compressors
[07:08] <keir> wait, but doesn't lzw's dictionary get built implicitcly?
[07:09] <keir> so it's never put directly in the output stream?
[07:09] <keir> so you can just 'stop' compressing at any stage?
[07:09] <keir> (i haven't implemented it for some years...)
[07:09] <lifeless> well. It would be really nice not to require C; that keeps us more portable. C as optimisation - Great!. C as requirement - if we have too we have too, but do we have too ?
[07:09] <lifeless> keir: the dictionary is built from the symbols processed
[07:09] <lifeless> keir: say we compress AAAAAA
[07:09] <lifeless> first symbol A - its a terminal, output it
[07:10] <abentley> keir: We were all C losers at one point or another.  Remember tla, lifeless?
[07:10] <lifeless> next symbol A - its a hit for A, keep it, next symbol A, output reference-to-first A, then A.
[07:10] <lifeless> next symbol A, hit for A, next symbol A, hit for AA, next symbol A, output reference to AA, then A
[07:11] <lifeless> this ummary is wrong, I'm just demonstrating that there are things that are not yet output, and the number of bits that have tobe output to represent them is determined by the compressor state
[07:11] <lifeless> that is - its eaten the input, but not output it yet.
[07:12] <keir> ah yes
[07:12] <lifeless> abentley: oh yes.
[07:12] <lifeless> abentley: 2753 is pushed btw
[07:12] <keir> ok, so if i hack lzw itself, it's fine because i can estimate the current compressed size if we were to terminate
[07:12] <abentley> lifeless: tx
[07:14] <keir> libz is zlib? i'm not sure how to get at a lzw compressor in python
[07:14] <abentley> Um, is it pushed to the packs version of your branch?
[07:16] <lifeless> abentley: yes, not to the knits version yet
[07:17] <lifeless> I'll ssh in and clone it across to knits, one sec
[07:17] <spiv> keir: Yes. http://docs.python.org/lib/module-zlib.html.
[07:20] <lifeless> keir: yes, but see above - requiring C is undesirable. You can achieve a version of this without requiring C though, (just add until a page size is within some figure, then flush, and accept slack space)
[07:25] <keir> yeah, that's the plan
[07:26] <keir> so it looks like you get smack on 0.2 compression ratio for 4k, 16k, 32k blocks
[07:26] <keir> at least for rev id's
[07:26] <keir> i'm out for tonight but i'll write this up tmrw morning
[07:27] <lifeless> http://www.smh.com.au/news/national/chaser-duo-held-over-motorcade-stunt/2007/09/06/1188783378804.html
[07:27] <lifeless> keir: thanks!
[07:48] <igc> lifeless: re your changes to test_versionedfile.py just merged, it looks like you have a duplicate routine name - test_add_lines_nostoresha. I gather that's the one test and it got duplicated by a merge maybe?
[07:49] <igc> my merge of your 0.92 patch has the routine name 4 times. :-)
[07:55] <lifeless> abentley: oh I've pulled it to the knit copy of packs
[07:55] <spiv> The current version of pyflakes notices that sort of mistake.
[07:56] <lifeless> igc: .dev$ grep test_add_lines_nostoresha bzrlib/*/*.py
[07:56] <lifeless> bzrlib/tests/test_versionedfile.py:    def test_add_lines_nostoresha(self):
[07:56] <lifeless> bzrlib/tests/test_versionedfile.py:    def test_add_lines_nostoresha(self):
[07:56] <lifeless> I see it twice
[07:57] <igc> yup - twice in bzr.dev
[07:57] <igc> when I merge your patch, I get it 4 times
[07:57] <lifeless> igc: and it should be twice with a slight change to the name
[07:58] <lifeless> fixed and sent to pqm
[07:58] <igc> thanks
[07:58] <lifeless> igc: I am down to m48
[07:58] <igc> neat
[07:58] <lifeless> 1m48 I mean lol
[07:59] <igc> i knew that :-)
[07:59] <lifeless> 2m2 wall clock
[08:02] <igc> lifeless: does merging my "improved commit reporting" patch help as well?
[08:03] <lifeless> I'm running with -q
[08:03] <lifeless> so I'd hope not :)
[08:03] <igc> even with -q, show_change (or whatever it's called) is still being called unnecessarily without my patch
[08:04] <igc> in bzr.dev at least
[08:08] <lifeless> igc: ok
[08:21] <lifeless> igc: did you see my /query?
[08:22] <igc> I saw the times, yes
[08:24] <lifeless> hard to tell when you dont reply :)
[08:25] <igc> I did
[08:25] <lifeless> I bet you are not logged in
[08:26] <igc> hmm
[08:26] <lifeless> tomorrow I'm going to make clean patches for several of these things for bzr.dev
[08:26] <igc> cool. I've got a quick review if you want it ..
[08:26] <lifeless> on bb ?
[08:26] <igc> just on paper here so far :-)
[08:26] <lifeless> :)
[08:27] <igc> want a quick email now or a more complete one
[08:27] <lifeless> I'm calling it a day, or really a week
[08:27] <igc> ok - I'll go with the complete one in the next few hours
[08:27] <igc> gotta luv APEC :-)
[08:28] <igc> lifeless: do you have a branch I can pull?
[08:29] <igc> lifeless: just emailed you my gaim session - not sure why you didn't see my times
[08:42] <lifeless> heh
[08:42] <lifeless> no, I don't have everything committed, I'm do sketch-tests
[08:42] <lifeless> then I take the uncommitted change and test in isolation to make a real branch for it
[08:42] <lifeless> then submit that
[08:43] <igc> ok
[08:43] <lifeless> if you merge all my pending bundles
[08:43] <lifeless> and disable annotations as previously documented
[08:43] <lifeless> and disable the _check_lines and _check_present_parents in knit.py's _add_lines method
[08:43] <lifeless> then I think you will have my tree
[08:44] <igc> so your times are with packs, not knits, right
[08:44] <lifeless> rm -rf .bzr && bzr --no-plugins  init --experimental && bzr --no-plugins add > /dev/null && time ~/source/baz/repository/bzr --no-plugins  commit -m "Initial commit"  -q
[08:45] <igc> cool
[08:45] <igc> enjoy your day off
[08:45] <lifeless> okies :)
[10:32] <sabdfl> hey guys, when last was lifeless around?
[10:32] <pygi> 3:07 my time
[10:32] <pygi> was the last time he said something :)
[10:33] <dato> sabdfl: 1h45m ago
[10:34] <sabdfl> thanks dato
[11:34] <Peng> Woah. _patiencediff_c.c is real C, not Pyrex.
[12:00] <Peng> Hmm, easy way to run patiencediff on a directory? python -m bzrlib.patiencediff only accepts files.
[12:00] <Peng> (as of 0.90, at least)
[12:29] <Ng> are there ubuntu packages of 0.18 for dapper lurking anywhere?
[12:32] <Peng> What about 0.90?
[12:32] <Ng> oh yeah, I'd forgotten that was out. 0.90 then :)
[12:33] <Peng> Well, I don't know.
[12:33] <mwhudson> http://bazaar-vcs.org/releases/debs/feisty/
[12:34] <mwhudson> oh sorry
[12:34] <mwhudson> bazaar-vcs.org only seems to have 0.17
[12:34] <Ng> yeah and the feisty debs for 0.90 seem to be arch dependent and not happy with dapper's libc
[12:35] <mwhudson> Ng: i would _hope_ that the debs would build easily enough on dapper
[12:36] <Ng> mwhudson: I expect so, I was just hoping someone was continuing the fine bazaar tradition of doing it for me :)
[12:36] <mwhudson> unless the build-deps are hard to satisfy, e.g. missing a new enough pyrex
[12:39] <mwhudson> Ng: fair enough
[12:39] <mwhudson> i'm not the one to help here though :)
[01:08] <lifeless> Ng: we will
[01:09] <lifeless> Ng: need to backport the debs and python build rules
[01:09] <lifeless> because dapper has that very different python setup
[01:09] <Ng> lifeless: ok, thanks :)
[01:14] <abentley> lifeless: up?
[02:00] <fullermd> Hm.  How odd.
[02:00] <fullermd> Where does 'info -v' get its "first revision" from?  'cuz it's wacky.
[02:01] <fullermd> I've got a branch with 3 initial revs, and the one it picks for 'first' isn't the left-path initial.  Nor is it the oldest of the initials.
[02:17] <jelmer> NfNitLoop: that appears to be a recent regression, working on it at the moment
[02:18] <jelmer> NfNitLoop: looks like a http-specific bug
[02:49] <effbiae> hi all, bzr's great.  how do i turn a local branch into a sftp branch, retaining all the history?
[02:51] <gabe_> effbiae: it already is
[02:52] <effbiae> so just sftp to the dir, yeah?
[02:53] <luks> or `bzr push sftp://host/path/to/branch`
[02:57] <gabe_> push doesn't work well for me via sftp
[02:57] <gabe_> it doesn't update the trees
[02:57] <effbiae> my branch is in ~/mysql.5.bak  ,  then bzr clone sftp://jack@192.168.0.4/mysql.5.bak results in ...
[02:57] <gabe_> because it is too expensive via sftp
[02:57] <effbiae> bzr: ERROR: bzrlib.errors.NotBranchError: Not a branch: sftp://jack@192.168.0.4/mysql.5.bak/
[02:58] <gabe_> does   mysql.5.bak have   a  .bzr   directory?
[02:58] <effbiae> yup
[03:02] <effbiae> bzr check returns clean bill of health
[03:04] <fullermd> effbiae: You probably mean "sftp://jack@192.168.0.4/~/mysql.5.bak
[03:05] <effbiae> fullermd: brilliant.
[03:05] <gabe_> fullermd: yeah
[03:06] <effbiae> now is a local commit the same as an sftp commit?
[03:06] <gabe_> didn't know you could do a sftp commit!
[03:06] <fullermd> ETERMINOLOGY
[03:07] <effbiae> EAGAIN?
[03:07] <fullermd> It's hard to answer without knowing what you mean by 'local commit' and 'sftp commit'   :)
[03:10] <effbiae> hmmm, i think i'm confused.   i want to run a server, like CVS.
[03:13] <effbiae> well, i want a central repository that i can get and put revisions.  don't want more than one branch at this stage.  which man page should i look at?
[03:15] <fullermd> Well, in that use case, you want to use checkouts of a single branch.
[03:15] <fullermd> So you'd have a branch "somewhere" (whether it's local, or via sftp, or bzr+ssh, doesn't matter), and you'd "bzr co" it wherever you want to work.
[03:19] <effbiae> so the key is to 'checkout' and then future commits will update the .bzr directory from where i checkedout, yeah?
[03:21] <fullermd> Right.
[03:22] <effbiae> thanks for all your help.  good night.
[05:00] <joe99> Anyone have any suggestions for how to setup a repository on OS X that more than one person can commit to?
[05:01] <bwinton> Give all the contributors accounts?
[05:01] <bwinton> (That's what I did, but I trusted the other person with an account.)
[05:01] <bwinton> (And set his shell to /bin/false just in case. ;) )
[05:01] <joe99> Everyone has accounts.  But how do you handle the file permissions?
[05:01] <fullermd> That rather cramps the ability to use bzr+ssh   :p
[05:01] <fullermd> Group writable.
[05:02] <bwinton> It does, but he could push over https.
[05:02] <bwinton> Yeah, I did the group-writable thing.
[05:02] <bwinton> Or I set them up to be owned by him, and sudo-ed everything I needed to.
[05:02] <fullermd> Well, if you go over https (bzr+, I presume), you don't need accounts period.
[05:02] <joe99> Yes, but new files created don't seem to keep the permissions.
[05:03] <fullermd> They should.  I've tried a couple times, and bzr always DTRT.
[05:04] <joe99> If I create something as root & make everything writable new files created by others are not compatible.   Is there a simple solution to this?
[05:04] <mwhudson> do you need g=ws or however it's spelt?
[05:04] <fullermd> How are the perms showing up wrong.
[05:04] <fullermd> I'm pretty sure OS X doesn't have that SysV brain damage...
[05:05] <fullermd> (Er, that first was supposed to have a '?' at the end...)
[05:05] <joe99> New files reflect the ownership of the creator and -rw-r--r-- permissions.
[05:05] <fullermd> The dir they're in is 77x?
[05:06] <joe99> Yes.
[05:09] <fullermd> Mmm.  Works for me.
[05:09] <fullermd> dir is 770, files end up like -rw-rw----  1 fullermd  wheel   94 Sep  6 10:08 a-20070906150753-pzfx8evgly2b3tep-1.kndx
[05:09] <joe99> Let me try again.
[05:09] <fullermd> Make sure it's all set to start out.
[05:09] <fullermd> find .bzr -type d -print | xargs chmod 770   ; find .bzr -type f -print | xargs chmod 660
[05:11] <joe99> sudo chmod 770 foo;       cd foo;       touch bar;     ls -l;     results in -rw-r--r--
[05:13] <fullermd> Yes, but that's touch, not bzr.
[05:13] <joe99> Also,  I tried bzr serve --allow-writes as an alternative, but it didn't work, and I'm not sure of the security model for it.
[05:13] <joe99> Sure, but why would bzr be different?
[05:13] <fullermd> You only care about the files bzr makes.
[05:14] <fullermd> Because bzr checks the perms and intentionally preserves g+w if it's around.
[05:14] <joe99> But bzr is run by the users ssh-ing.
[05:14] <joe99> Well, maybe my experiment is flawed.
[05:14] <fullermd> touch just does an open().  bzr looks at the perm of the parent dir (I think; maybe the perm of the .bzr/ at the root) and uses that as a template for the new file.
[05:15] <joe99> So as long as nobody actually does anything by hand in that directory it should work fine?
[05:16] <fullermd> Well, anybody doing something in .bzr/ by hand needs a good thwapping just on general principles   :] 
[05:17] <joe99> Not in .bzr,  I meant creating, adding, commiting, etc in the branch in the shared directory.
[05:17] <fullermd> What, multiple people in one working tree?  That's a nightmare waiting to happen.
[05:18] <fullermd> I wouldn't consider it except in very specialized cases (like /etc, where there's only one uid that's ever there)
[05:18] <joe99> Doesn't somebody have to create the initial tree?
[05:19] <fullermd> Yes, but everybody would have their own WT (either by having their own branches in a dist-like setup, or their own checkouts in a central-like setup)
[05:19] <joe99> How do I put a tree there in the first place?
[05:20] <joe99> I mean, I have one.   Can I branch it there and have that branch be the branch acts as the central branch that everyone can push to?
[05:21] <fullermd> Yes, that'll work.  Push it there (one way or another), then do the chmod's on that "central" location, then have everybody else branch from and push to that, or checkout it, or whatever.
[05:22] <joe99> And bzr will DTRT with respect to file permissions regardless of who is ssh-ing?
[05:23] <fullermd> Should (and in all my testing, does), yah.
[05:23] <joe99> I'll give it a try.  Thanks.
[05:23] <fullermd> If OS X uses SysV filesystem semantics, you'd need g+S on the dirs to properly propogate the group ownership to new files.
[05:23] <fullermd> I don't _think_ it does.  But if new .knit's and the like show up with the wrong group, that's the thing to try.
[05:25] <joe99> How do you set g+S?
[05:25] <joe99> Do you mean g+s?
[05:25] <fullermd> Yah.
[05:27] <cory> Hi, is there a way to cleanly remove a branch from a shared repository (and any cruft it might have left behind)?
[05:27] <fullermd> Not really.
[05:30] <cory> Ok, I suppose I can always just branch everything I want to a different shared repository and let it sort itself out if I really care.  I was just curious and having trouble searching, thanks.
[05:30] <fullermd> Yeah, that's the usual way.
[05:31] <fullermd> It's a fair bit of work, and it loses branch-local config (like parent/push/etc branches).
[05:31] <fullermd> There was a proof-of-concept 'cleanup' plugin somewhere once.  It's certainly on the list to add, just nowhere near the top of anybody's todo list.
[05:32] <fullermd> Mostly, disk space is cheap enough that we all just ignore it for now.
[05:32] <bwinton> Mental note: Don't put my video collection under bzr...  :)
[05:33] <cory> Oh, we have lots of big binary files. :S
[05:34] <Lo-lan-do> Disk space is rather cheap, but bandwidth (and time) isn't.
[05:34] <fullermd> Well, most of any given branch will be common with other branches in the project, so it's not like you'd save much in most cases.
[05:34] <fullermd> Sure, but if it's not referenced, it doesn't cost you any bandwidth.
[05:34] <Lo-lan-do> So remote backups of obsolete data is less than optimal.
[05:34] <fullermd> (well, maybe a little in throwing around inventories or something, but...)
[05:34] <fullermd> Oh, non-bzr?  Yeah, it'll hit you there.
[05:35] <Lo-lan-do> Yeah, like, you know, backups :-)  rsync, or amanda, or tar+scp, or whatever
[05:35] <fullermd> Backups are for pessimists   ;)
[05:36] <Lo-lan-do> Backups are also for fat-fingered people and (maybe more importantly) their sysadmins.
[05:38] <cory> It was more a policy thing.  I'm trying to get some people to switch from p4, and I'm inclined to push lightweight checkouts and lots of branching in a centralized repo, and it was the difference of whether I say, "it mostly doesn't matter to leave it sitting around," or, "you can push this button to clean up an old branch, and it will be obliterated."
[05:39] <fullermd> Unrelated to the issue at hand, but you probably want to look at something like bzrtools' 'cbranch' with that workflow, too.
[05:40] <cory> Interesting, just to set the default branch root?
[05:41] <fullermd> Well, it lets you remotely 'branch' and create a local checkout in one step (AIUI; I've never done more than glance at it)
[05:42] <fullermd> I know that abentley uses the separate repo/checkout workflow more than most people here, so there are a few things (cbranch, switch, at least) in bzrtools aimed at supporting those workflows.
[05:44] <cory> Yes, switch is key.  Thanks.
[06:02] <NfNitLoop> jelmer: Hmm.  I just realized that the BzrForeignBranches/Subversion says that bzr-svn needs some fixes from svn 1.5 trunk.
[06:02] <NfNitLoop> jelmer: Is that still the case?  I hope that's not the problem I've been having and pestering you about.  :p
[06:02] <fullermd> Actually, last I heard, it still needed some fixes that weren't even in svn trunk...
[06:03] <NfNitLoop> fullermd: Heh, well, there is that nasty memory leak...  :)
[06:11] <samurai> morning all
[06:17] <joe99> Is sftp access safe for concurrent users?  Or do you need to use bzr+ssh?
[06:18] <mwhudson> both create lockfiles
[06:18] <fullermd> sftp should be as safe.
[06:18] <fullermd> (of course, if you can use bzr+ssh, it's probably better performing etc. anyway)
[06:19] <joe99> I'm not sure how to setup bzr+ssh on OS X.   The instructions are for inet.d, which OS X apparently doesn't use anymore.   And I don't really have a handle on launchd.
[06:20] <fullermd> bzr+ssh doesn't need any setup (other than having bzr in $PATH)
[06:20] <fullermd> If you're looking at inetd stuff, that's for bzr://, not bzr+ssh://
[06:22] <joe99> What's the difference between bzr & bzr+ssh?
[06:22] <fullermd> bzr+ssh ssh's in and runs bzr, and chats with it over the ssh connection (like :ext: CVS does)
[06:23] <joe99> So does that make it more secure?
[06:23] <fullermd> bzr:// talks over TCP to a server talking the bzr protocol, either via 'bzr serve' (which runs it as a daemon), or through something like inetd (that accepts the connections and passes them to bzr)
[06:23] <fullermd> Well, it means that ssh handles all the authentication and such.  With bzr://, you get no authentication (at least, not currently)
[06:26] <joe99> Does anyone happen to remember which file on OS X is the right one for setting the path to bzr?
[06:27] <fullermd> Depends on your shell, probably.  Something like /etc/profile or /etc/csh.cshrc or something else for zsh, etc.
[06:37] <NfNitLoop> Heh, I'm reading the "svk book".   It looks like someone just took the subversion book and did s/svn/svk/g
[06:37] <joe99> If I branch with sftp that seems to work.   If I use bzr+ssh I get an error that says bash: line 1: bzr: command not found  bzr: ERROR... and then this long traceback.
[06:38] <NfNitLoop> joe99: That probably means that bzr isn't found on the remote path.
[06:38] <joe99> Yes.  But wouldn't have been found by sftp?
[06:38] <NfNitLoop> does the remote machine have bzr installed?   Is it accessable by ssh's default path?
[06:38] <fullermd> sftp doesn't use bzr on the remote end.
[06:38] <NfNitLoop> sftp does not use bzr on the remot...
[06:39] <NfNitLoop> :)
[06:40] <joe99> That may be it then.  I added /opt/local/bin to path in /etc/profile.  So it's there when I ssh in.   But maybe that's only for login shells.  It must be one of those other files I can't remember the name of.
[06:40] <NfNitLoop> Yeah... I never remember either.   I'm glad bzr doesen't require a remote copy to function.  :)
[06:41] <fullermd> Avoiding Bourne shells does the trick for me   ;p
[06:42] <joe99> If the sftp command fails do bad things happen?
[06:43] <NfNitLoop> God kills a kitten.
[06:43] <NfNitLoop> (Though I'm not sure what happens to your branch.)
[06:43] <joe99> Does it keep the kitten company?
[06:44] <NfNitLoop> that or it just lands in intensive care.
[06:44] <NfNitLoop> I've not had luck with partially-completed bzr-svn branches.
[06:44] <NfNitLoop> Though that may be a special case.
[06:45] <joe99> I'm not concerned that much about the branch as the push.
[06:47] <NfNitLoop> Heh.  Good point.  I would hope they would be atomic.
[06:47] <NfNitLoop> there are a couple commands:  bzr break-lock and bzr check, that would probably help with recovery.
[06:48] <mwhudson> yeah, you'll often end up with dangling locks
[06:48] <mwhudson> but that should be the worst of it, afaik
[06:49] <jelmer> NfNitLoop: well, the python-subversion package in recent ubuntu and debian contains a backport of those changes from subversion 1.5
[06:51] <NfNitLoop> jelmer: aah.  I'm running Debian etch.
[06:52] <NfNitLoop> jelmer: you got my e-mail with the sample svn dump?
[07:19] <joe99> sftp doesn't update the working tree of a remote branch.  Is this of any concern if nobody ever works on that branch directly?   Does it require any special considerations for conflicts or anything that are pushed to that remote branch?
[07:20] <fullermd> Only if you want the files there and up to date for some reason.  bzr doesn't need them.
[07:21] <fullermd> Unless you need 'em for some other reason (and thus go through other steps to keep them up to date), you can just as easily get rid of the working tree altogether.
[07:22] <joe99> At some point I might want to run automated tests.  But for now I was hoping to use a remote branch as a central repository for things everyone was happy.
[07:22] <joe99> ..with.
[07:23] <fullermd> I'd probably run automated test in their own checkout anyway.
[07:35] <joe99> Yes.  Assuming everybody actually runs them.   But some live by the motto "If it compiles, ship it."   And with interpreted code there are those who don't even compile.
[07:35] <fullermd> No, I mean automated tests would run as an automated job, but with its own checkout that it updates and works with.
[07:36] <joe99> That makes sense.
[07:47] <silwol> somebody here who can help me about how to best package my piece software that is checked in to bzr?
[08:13] <NfNitLoop> silwol: Just like you would package any other software.
[08:13] <NfNitLoop> all of bazaar's files are in .bzr
[08:13] <NfNitLoop> so just exclude that directory.
[08:15] <silwol> NfNitLoop: i remember chatting with pitti (from ubuntu) and he told me it was best to keep a separate bzr branch for packaging
[08:15] <silwol> so the main branch has no debian directory, and is always merged into the packaging branch where the debian directory is maintained
[08:16] <silwol> but packaging requires a original .tar.gz so i can not directly pull from the main branch
[08:16] <silwol> ...i hope you understand what i mean
[08:19] <NfNitLoop> aah.   Yeah, I'm not that famailiar with Ubuntu's packging process.
[08:19] <NfNitLoop> keeping a separate branch for debian/ sounds like a good idea, since that doesn't have much to do with the original source.
[08:20] <NfNitLoop> I vaguely recall that debian sources are packaged as the original.tar.gz, plus a debian.diff.tgz or something like that.
[08:21] <dato> diff.gz, yes
[08:21] <NfNitLoop> so you would zip up the source repo.   then diff your 'debian' branch to create the diff.
[08:21] <NfNitLoop> beyond that, I'm afraid I won't be much help.
[08:21] <silwol> okay, but thanks anyway, NfNitLoop
[08:22] <silwol> i think i'll try to catch pitti when he is online
[08:22] <dato> silwol: so you are the upstream author of the software?
[08:23] <silwol> dato: yes, but it is not very advanced yet.
[08:23] <silwol> just trying to package in order to get some people to try it out
[08:23] <dato> silwol: and you will be maintaining the packages as well? or you intend for a separate person to do that?
[08:24] <silwol> actually, i don't mind if it is me or somebody else on the long term. for now i just wanted to create packages, but if somebody else wants to maintain them, why not
[08:25] <silwol> although i think i will in future need the knowledge about how to more often
[08:25] <dato> well, then my advice would be:
[08:25] <silwol> should mean about how to package
[08:26] <dato> (1) when you create the upstream tarball for a release, never include the debian/ directory in it, if one exists in your repository
[08:27] <silwol> that's what pitti told me too
[08:27] <dato> and for where to keep debian/, you have several options
[08:28] <dato> (2a) keep debian/ out of trunk, and have a packaging branch; you `bzr branch trunk packaging`, and `bzr add debian/` in packaging. when an upstream release happens, you `bzr merge ../trunk` inside packaging, and make any packaging changes as well there
[08:29] <dato> I think that's the one prefered in Ubuntu
[08:29] <dato> another one, (2b) would be to keep the debian/ subdirectory in its own branch, one that is not a full branch of the whole source
[08:31] <dato> and then there is (2c), the one I use for my projects (since I'm also the maintainer), keep debian/ in trunk, at release time build the package from trunk, and if there is another upload to make while trunk has diverged, make the debian packaging changes in a new branch created from the release tag, and merge that back to trunk. it may look a bit messier, but I like it.
[08:31] <dato> silwol: I guess I recommend (2a). do you have any questions about it, or about something else?
[08:32] <keir> anyone other than lifeless familiar with the uses of the graphindex code?
[08:33] <keir> i'm wondering if there's cases where you need to do a DFS type search, where you only care about the values and the ordering, not the actual keys encountered on the way
[08:33] <silwol> dato: until here it seems okay for me, and i would prefer the (2a) way.
[08:34] <silwol> dato: what confuses me a little bit is that the ubuntu packaging guide talks about having the source in a package-x.y.z directory
[08:34] <silwol> dato: this means that i would have to rename my packaging branch each time
[08:34] <silwol> dato: well, the directory where it lives
[08:35] <dato> well, the real truth is it doesn't need to be named srcname-x.y.z (you'll just get a warning from dpkg-source if not), but it's good practice
[08:36] <dato> do you see a problem in renaming it?
[08:36] <dato> (you can rename it, or do `bzr branch packaging srcname-x.y.z` instead, commit in srcname-x.y.z and then `bzr push ../packaging` when finished)
[08:37] <dato> (the latter approach seems a bit cleaner, because you can throw away the directory you built the package on after pushing)
[08:38] <silwol> okay, that seems to make more sense to me.
[08:39] <silwol> ...here some of the real benefits of a distributen versioning system seem to open up
[09:04] <jelmer> NfNitLoop: yep, thanks
[09:23] <NfNitLoop> Heh.  playing with svk....   it's got one central store that's based on svn.   If you're done with a project, there's no way to get rid of just that project's data without blowing away data for other projects.
[09:28] <NfNitLoop> (afaik)
[09:29] <keir> aren't bzr shared repos no different?
[09:29] <keir> or would 'repack-dropping-empty-refs' take care of that?
[09:30] <keir> my graphindex is going to be sweeeeeet!
[09:33] <Treeform> hey can any one help me!  Is there a difference they way bzr handles text and binary files?  I think it handling one of my 100mb files as a text file and cosing massive mem usage
[10:02] <NfNitLoop> keir: with bzr, you can easily create a shared repo for each project, then delete the shared repo & project all at once, not affecting your other shared repos.
[10:02] <keir> oh
[10:02] <NfNitLoop> keir: and even if you just used one central shared repo, I think I've read in here that there are tools to prune unreferenced changesets.
[10:03] <keir> NfNitLoop, what i mean is, if you have a shared repo, then you decide you don't care about 2 of the branches, deleting them  doesn't remove the stuff in the shared repos which was only ref'd by those branches.
[10:03] <NfNitLoop> ^-- see my last comment. :)
[10:03] <keir> i was 90% of the way through mine when i saw yours so i finished and hit enter anyway :P
[10:04] <NfNitLoop> I forget what they are.... but in the worst case you can create a new shared repo and migrate into there, then delete the old shared repo.
[10:04] <NfNitLoop> inefficient, but at least *possible*.
[10:04] <NfNitLoop> I mean, disk space may be cheap, but endlessly growing files still aren't a good idea, IMO.  :p
[10:05] <keir> yeah
[10:06] <ubotu> New bug: #137823 in bzr "bzr selftest failures and haning unit tests?" [Undecided,New]  https://launchpad.net/bugs/137823
[10:38] <dato> aw, this new 'Committing revision x to "/path"' line that commits outputs is unexplainably getting on my nerves.
[10:39] <keir> lifeless, are you around?
[10:43] <keir> i'm confident i can make repacking the GraphIndex's super fast because AFAICT, old nodes are never changed; new nodes only refer to old ones.
[10:44] <keir> because my new format is toposorted, repacking sould be 2x linear in len(pack1 + pack2 + ...)
[10:44] <fullermd> Mmph.  I think "ghost filling" is the usual answer to that sort of thing...
[10:44] <keir> ghost filling?
[10:46] <fullermd> If you've got revs referencing ghosts, the graph cuts off at that point, but if the ghosts get filled that leg suddenly gets extended.
[10:47] <keir> yes, that's exactly the algorithm i imagined in my head
[10:48] <keir> is the pack format described somewhere? the parts that aren't the graph index
[10:48] <fullermd> lifeless would be your best source for that I imagine, and I think he's mostly afk until next week.
[10:49] <keir> yeah, ok
[10:49] <keir> i'm tempted to go ahead and implement my new format in C
[10:49] <keir> mercurial is beating us in popularity, and they have C
[10:50] <keir> besides, the code i'm going to write is very general; it stores highly packed DAG's which are k->v and k->k*
[10:50] <keir> it so happens that it's a perfect format for the ways bzr uses it