[00:33] <Lor> Is there some estimate of when nested (by-reference) trees will become a supported feature?
[00:41] <fryguy_> is there any way to get a list of files that would be applied in a merge without applying it? `bzr merge --preview` shows a diff, but I'd like to see the files (more like a --pretend?)
[00:42] <idnar> as a quick fix, you could pipe the diff through diffstat
[00:47] <fryguy_> is that a unix command?
[00:48] <fryguy_> i ended up just applying the merge and piping stderr to a file, and then reverting it
[00:58] <AfC> fryguy_: yes, it's in a package you can install. The "diffstat" package, as it happens.
[00:59] <fryguy_> i'm on windows :(
[01:00] <Kilroo> I feel your pain
[01:01] <fryguy_> well, I do like developing on windows. c# is a lot of fun.. so it's not all bad
[01:03] <Kilroo> I did like C# okay.
[01:03] <idnar> ah, might be harder to get diffstat on windows
[01:03] <Kilroo> Didn't much care for codebehind, but C# was good.
[01:27] <igc> morning
[01:46] <Kilroo> Random thought of the moment: I think the decentralized-but-not-quite-distributed development model is far more easily supported by bzr than by any other vcs I've encountered.
[01:47] <Kilroo> Or to put it another way, I like bound branches.
[01:48] <mneptok> take up bonsai as a hobby.
[01:48] <mneptok> ;)
[01:49] <Kilroo> Lulz.
[02:02] <poolie> Kilroo, it's true
[02:23] <spiv> lifeless: yeah, I saw that GIL message.  My guess is it doesn't have much relevance for us
[02:24] <spiv> lifeless: because our common case atm is pull from bzr+ssh, which shouldn't involve any threads.
[02:24] <lifeless> beep, wrong
[02:25] <lifeless> or perhaps
[02:25] <lifeless> I was thinking paramiko for a second
[02:25] <lifeless> but then I remembered that paramiko isn't used for bzr+ssh
[02:25] <lifeless> however
[02:25] <spiv> Right.
[02:25] <lifeless> we do have a threading handshake-thunk in the server streaming and client streaming code
[02:25] <lifeless> don't we ?
[02:25] <lifeless> converting reads to generators
[02:25] <spiv> We do use threads for insert
[02:25] <lifeless> and generators to writes
[02:25] <spiv> And for builtin TCP serve.
[02:26] <lifeless> right, TCP serve very obviously has threads
[02:26] <spiv> No thread for get_stream*, just insert_stream*
[02:26] <lifeless> so
[02:26] <lifeless> if the client doesn't read its buffer enough
[02:26] <lifeless> client side buffer fills up
[02:27] <spiv> But we've been observing apparent oddness with get_stream
[02:27] <lifeless> os socket buffer is full - network will back off
[02:27] <lifeless> yes, oddness to
[02:27] <lifeless> too
[02:27] <lifeless> I filed a couple of bugs I want to chase down, as branch time is hurting the distro
[02:27] <spiv> Even in the inserter case, there's no fixed size buffer.
[02:28] <spiv> As the network thread receives bytes it unwraps them from the HPSS layer and pushes them onto a Queue.Queue
[02:29] <spiv> Which "should" be fast, assuming the other thread(s) aren't starving the network thread of CPU time.
[02:29] <spiv> Yeah, I saw the bugs, they are marked Critical after all ;)
[02:29] <lifeless> oh good, for some reason I thought we had a fixedish queue
[02:30] <lifeless> the reason the GIL thing stood out to me was that it describes networking threads getting starved out of proportion to the cpu load - AIUI
[02:30] <spiv> Well, from reading from a pipe, we are a bit pessimistic, lots of small reads to avoid blocking.
[02:31] <spiv> When reading from a socket we just always grab 4k or something.
[02:31] <lifeless> is bzr+ssh considered to be a pipe or socket ?
[02:31] <spiv> A pipe.
[02:31] <spiv> Anyway, for the bugs you filed, they are streaming from LP, so no threads in the server bzr involved.
[02:31] <lifeless> argh
[02:32] <lifeless> maybe I should fix
[02:32] <lifeless> Conflict: can't delete bzrlib/plugins/bash_completion because it is not empty.  Not deleting.
[02:32] <lifeless> first
[02:32] <lifeless> as its breaking the workflow for 2.0->2.1->trunk merges
[02:32] <spiv> lifeless: you would make many people happy
[02:32]  * lifeless gets out the economy sized yak shaver
[02:32] <spiv> lifeless: I suggest "moving bzrlib/plugins/bash_completion to lost+found/" ;)
[02:35] <nDuff> Hmm. I have a project where I'm interested in determining if any files in a large tree revert to prior states, and being able to scan said directory for changes quickly/efficiently; however, I _don't_ need to be able to retrieve content associated with said former states, just enough metadata to identify them.
[02:35]  * nDuff is pondering whether bzrlib can be pervented to the purpose. :)
[02:35] <lifeless> so, you're happy with a lost+found directory approach ?
[02:35] <lifeless> perhaps bzr-lost+found, so as not to collide with the fs one
[02:35] <lifeless> nDuff: sure, you could use the dirstate module, or perhaps even working tree 4, directly.
[02:36] <lifeless> you may find some areas where we are a bit fuzzy on layers - but we'd be delighted to sharpen them up to serve your purpose
[02:36] <mkanat> lifeless: You could just do .saved or something, too.
[02:36] <mkanat> lifeless: Somewhat like rpm does.
[02:38] <spiv> lifeless: I'm happy with any improvement
[02:38] <nDuff> lifeless, thanks for the encouragement -- I was already reading through the pydoc for bzrlib.dirstate, and will look into the API to the working tree code as well.
[02:39] <spiv> lifeless: I think in practice most people will routinely delete everything in lost+found, and maybe rarely rescue one or two files from it
[02:39] <lifeless> mkanat: are there docs on precisely what rpm does ?
[02:39] <mkanat> lifeless: Good question.
[02:40] <spiv> lifeless: but "rm -r lost+found" is a much easier way to recover a sane tree than the current situation
[02:40] <mkanat> lifeless: Brief search finds an email mention briefly: http://www.redhat.com/archives/rpm-list/2001-July/msg00213.html
[02:40] <spiv> So, I guess I am happy with the lost+found approach :)
[02:41] <spiv> mkanat: that doesn't seem to address this situation (that a managed directory is being deleted; what should happen if there are unmanaged files in it?)
[02:41] <lifeless> mkanat: so, we do ok on single files
[02:41] <thumper> hey
[02:41] <mkanat> lifeless: Yeah.
[02:41] <thumper> I just noticed that kdiff3 is back
[02:41] <thumper> how can I use it to resolve merge conflicts?
[02:41] <lifeless> mkanat: we're talking about the case where we want to delete a directory X containing a file X/Y that is not versioned
[02:42] <thumper> is it easy?
[02:42] <lifeless> mkanat: or is versioned and is modified
[02:42] <lifeless> mkanat: the proposal is to move the entire path under a top level bzr-lost+found directory
[02:42] <mkanat> lifeless: Well, if it only contains unversioned files and is being deleted, then it's fairly easy to just say "make it .moved or .saved" or something.
[02:42] <lifeless> mkanat: and say that we've done that
[02:43] <mkanat> lifeless: I imagine that's the most common case--that's the one I've hit.
[02:43] <lifeless> mkanat: 99% of the time [NotAMetric statistic] its just a .pyc file so the entire thing should be deleted
[02:43] <lifeless> at the moment though, we leave the directory in place, and it gets in the way when you switch back
[02:43] <mkanat> lifeless: Unless it's some configuration file that's important to the functioning of some customization on a branch that you're merging into, or something like that.
[02:44] <lifeless> mkanat: which is why moving it under a different directory is better - the common case is to nuke all the things; the uncommon case is to want to add the containing directory again and put the file back
[02:44] <lifeless> mkanat: sure thats why we can't delete the file outright
[02:45] <mkanat> lifeless: Okay. Might want to call it bzr-deleted or something like that, for clarity.
[02:45] <mkanat> Or -removed.
[02:45] <mkanat> lifeless: I think it will be somewhat surprising behavior, though.
[02:45] <lifeless> I'm not sure thats more or less clear
[02:45] <lifeless> mkanat: the current behaviour sucks :)
[02:46] <mkanat> lifeless: I think lost+found definitely won't be clear to people who don't have that on their OS.
[02:47] <mkanat> lifeless: I think I'd find it somewhat more intuitive to rename the directory in place.
[02:47] <lifeless> what do you mean ?
[02:48] <mkanat> lifeless: I mean, if I were looking for where my files went, and I actually needed them.
[02:48] <mkanat> lifeless: I could also nuke them with clean-tree, which is how I nuke things now anyhow.
[02:48] <lifeless> so we print out whats happening
[02:48] <mkanat> lifeless: But bzr prints out so much stuff when merging, it might get lost.
[02:48] <lifeless> mkanat: it doesn't on switch, IME
[02:49] <mkanat> lifeless: Oh, that's true.
[02:49] <lifeless> mkanat: this is primarily an issue for switch, because you're often changing between radically different code bases.
[02:50] <mkanat> lifeless: I think I encountered it with "up" once, maybe, when I was updating with local commits.
[02:50] <lifeless> we could introduce the concept of precious files and say 'ignored files can be deleted willynilly', and treat unversioned unignored files as precious [thereby avoiding an initial config file], but we'd still have an issue with that config file story, for intance.
[02:50] <mkanat> lifeless: Yeah.
[02:51] <mkanat> lifeless: I'm just thinking about the case where, let's say I have two or three of those directories, I switch, and then I do "bzr status", and I see only one directory that I never added.
[02:52] <mkanat> lifeless: I think treating it like a contents conflict and making it .moved would be more in line with what I normally expect bzr to do.
[02:53] <lifeless> do this
[02:53] <lifeless> sit in bzr.dev
[02:53] <lifeless> bzr switch 2.1
[02:53] <lifeless> bzr switch 2.0
[02:53] <lifeless> [using whatever layout you want :)]
[02:54] <lifeless> oh, but dp a 'bzr selftest nothingtodo' at each step, to compile pyc files
[02:54] <mkanat> Okay.
[02:55] <lifeless> spiv: the 2.0->2.1 merge could use a NEWS merge magic to move the new items in 2.0.unreleased to 2.1.unreleased
[02:55] <lifeless> spiv: any ideas how? file a bug ?
[02:57] <lifeless> we should squelch /usr/lib/pymodules/python2.6/lazr/uri/__init__.py:19: UserWarning: Module launchpadlib was already imported from /usr/lib/pymodules/python2.6/launchpadlib/__init__.py, but /usr/lib/pymodules/python2.6 is being added to sys.path
[02:57] <lifeless>   import pkg_resources
[02:58] <lifeless>  too
[02:58] <lifeless> actualy, lazr should
[03:02] <spiv> lifeless: that would be good, file a bug
[03:02] <lifeless> I did already, weeks back
[03:02] <lifeless> now to find it
[03:02] <mkanat> lifeless: Ah, okay.
[03:02] <mkanat> lifeless: (I just went through the process.)
[03:02] <spiv> lifeless: there's the related case of "new item in feature branch, when merged to trunk (or stable branch) move to right place"
[03:03] <mkanat> lifeless: See, I think that if there were anything important in there, and they were in a deep directory structure, trying to sync them back into the right place from a lost+found directory would be difficult.
[03:04] <lifeless> mkanat: cp -r lost+found/foo ./ would do it quite well
[03:04] <spiv> lifeless: you're possibly thinking of bug 517490?
[03:04] <lifeless> mkanat: where foo is a top level dir there, not the whole path
[03:04] <mkanat> lifeless: On *nix, if you're familiar enough with the command line.
[03:05] <lifeless> mkanat: explorer will permit copying to merge things too
[03:05] <mkanat> lifeless: True. Does the OS X interface, also?
[03:05] <lifeless> mkanat: I believe so
[03:05] <mkanat> lifeless: Okay.
[03:05] <mkanat> lifeless: Well, why not just not consider them conflicts?
[03:06] <mkanat> lifeless: That is, just leave them where they are, unversioned, and don't add a conflict.
[03:06] <lifeless> mkanat: thats what we do at the moment, and that is annoying us
[03:06] <lifeless> spiv: no
[03:06] <mkanat> lifeless: Is that new since 2.0? (I see conflicts in bzr status.)
[03:06] <lifeless> spiv: grab 2.1 from lp, merge in 2.0. there is a small conflict - resolve that [its not what I'm talking about]. Then compare the result to what I committed in lifeless/bzr/2.1
[03:07] <mkanat> lifeless: I see how it gets annoying when you move back to bzr.dev.
[03:07] <mkanat> lifeless: Renaming them to .moved or .saved would work too, though, wouldn't it? You could do .saved.1 if there was already a .saved there.
[03:08] <spiv> lifeless: I agree that isn't 517490
[03:08] <spiv> lifeless: just that it's the closest I remember you filing
[03:09] <lifeless> mkanat: so if we renamed them to .moved or .saved, (the dir that is), we'd have less issues with successive switching back
[03:09] <lifeless> but we'd still clock up an impressive amount of cruft
[03:09] <mkanat> lifeless: Yeah, true.
[03:10] <mkanat> lifeless: And in the switch case, you probably don't want that stuff sticking around, usually.
[03:10] <mkanat> lifeless: What if switching puts that directory in .bzrignore though?
[03:10] <lifeless> ideally we'd just delete the dir - but that needs more info than we have [today], or a change in the definition of 'what bzr does to ignored files'
[03:11] <lifeless> mkanat: so in branch A its versioned and B its unversioned and ignored ? we still promise not to delete files we can't recreate
[03:11] <mkanat> Okay.
[03:11] <mkanat> So we use the versioning status of it in branch B to decide whether it goes in lost+found?
[03:11] <mkanat> I understand the convenience of it. I still think that it will be surprising to people who don't know the rationale, though.
[03:13] <spiv> mkanat: I think some surprise is inevitable, given the constraints.
[03:13] <lifeless> so the current concept I have in my head is:
[03:13] <lifeless> when a directory is deleted by a transform and it has children we can't dispose of gracefully, we move the directory <somewhere>
[03:14] <lifeless> unlike files *most* conflicts of this nature will be because of build products / editor temporary files (including ones bzr makes) and so on.
[03:14] <lifeless> e.g. __init__.py~1~
[03:15] <lifeless> I think Tom may have had it right when he had precious files, in arch.
[03:15] <lifeless> so a question here is, is it better to do this now, or do precious files first, or just one of the two things
[03:16] <lifeless> if we just do precious files (or junk files - they are symmetrical enough), we could gracefully dispose of more directories.
[03:16] <lifeless> and the thing wouldn't be as fantastically annoying
[03:17] <lifeless> if we just move directories we can't dispose of further than we do today, we won't get conflicts switching back and forth, but we will still pile up the cruft
[03:17] <lifeless> if we do both, we won't get conflicts, and we won't pile up as much cruft
[03:18] <spiv> Avoiding cruft is less important than avoiding conflicts.
[03:18] <lifeless> so, I think, I'm going to just make these conflicts a little harsher - to dir.deleted
[03:18] <lifeless> if thats not enough we can go for a single place to move the cruft, and-or junk|precious file categorisation
[03:19] <lifeless> mkanat: thanks for asking all those questions
[03:19] <mkanat> lifeless: Thanks for having the conversation with me. :-)
[03:19] <spiv> If you do dir.deleted, make sure it still solves the problem with multiple levels of deleted dir, each with unmanaged files.
[03:21] <lifeless> spiv: naturally - I already had that on my mental tests to write list
[03:21] <lifeless> spiv: so - did you see what I mean with the 2.0->2.1 merge?
[03:23] <spiv> lifeless: I haven't yet run those commands, but the description made sense.
[03:23] <lifeless> spiv: so, I should file a bug ? Basically I want to copy upwards new things to show where they arrive in the series
[03:24] <spiv> lifeless: yes
[03:24] <lifeless> is there a tag for this
[03:25] <spiv> I thought there was, but it doesn't seem to exist anymore.
[03:34] <lifeless> https://bugs.edge.launchpad.net/bzr/+bug/583630
[03:37] <lifeless> spiv: https://bugs.edge.launchpad.net/launchpadlib/+bug/552419 is the warning to squelch bug
[03:38] <spiv> lifeless: ?
[03:38] <lifeless> spiv: the launchpadlib already imported thing I was saying we should suppress
[03:39] <lifeless> just FYI - gary closed it as invalid, need to find out why
[03:39] <spiv> I'm still lacking context, I don't recall noticing that bug myself.
[03:39] <lifeless> oh
[03:39] <lifeless> are you running lucid ?
[03:39] <spiv> Yes.
[03:40] <lifeless> can you please run feed-pqm bzr (from trunk)
[03:40] <lifeless> I see this:
[03:40] <spiv> When would I notice it?  Using hydrazine?  Or just lp:... lookups?
[03:40] <lifeless> ./feed-pqm bzr
[03:40] <lifeless> /usr/lib/pymodules/python2.6/lazr/restfulclient/__init__.py:19: UserWarning: Module launchpadlib was already imported from /usr/lib/pymodules/python2.6/launchpadlib/__init__.py, but /usr/lib/pymodules/python2.6 is being added to sys.path
[03:40] <lifeless>   import pkg_resources
[03:40] <lifeless> loaded existing credentials
[03:40] <lifeless> bzr lp-propose
[03:40] <lifeless> etc
[03:40] <lifeless> lp: lookup doesn't use launchpad apis.
[03:40] <lifeless> ./feed-pqm bzr
[03:40] <spiv> Yeah, that's what I thought.
[03:40] <lifeless> /usr/lib/pymodules/python2.6/lazr/restfulclient/__init__.py:19: UserWarning: Module launchpadlib was already imported from /usr/lib/pymodules/python2.6/launchpadlib/__init__.py, but /usr/lib/pymodules/python2.6 is being added to sys.path
[03:40] <lifeless>   import pkg_resources
[03:40] <lifeless> bah pastefail
[03:40] <lifeless> loaded existing credentials
[03:42] <spiv> I don't see that with hydrazine trunk.
[03:42] <lifeless> ok, thats *interesting*
[03:42] <lifeless> fresh lucid install ?
[03:42] <lifeless> what version of setuptools? do you have the launchpad ppa ?
[03:42] <spiv> Depends on what you mean by fresh ;)
[03:43] <spiv> Recently upgraded to lucid (< 1 month), but the system as a whole was originally installed >3 years ago, IIRC.
[03:43] <lifeless> ok, not fresh :)
[03:43] <lifeless> well, will need to dig deeper
[03:44] <spiv> python-setuptools 0.6.10-4ubuntu1
[03:44] <spiv> I don't think I have the launchpad PPA
[03:53] <lifeless> and pkg-resources?
[03:53] <lifeless> hmm
[03:53] <lifeless> possibly pycentral or whatever to blame to
[03:53] <lifeless> too
[03:54] <spiv> Same version of pkg-resources
[03:54] <lifeless> ok
[03:54] <lifeless> something to track down later
[03:55] <lifeless> for now; time to go pickup bank card, fuel the car - do the stuff that got truncated before
[03:55] <lifeless> won't be long
[04:10]  * igc lunch
[04:46] <lifeless> I need a teddy bear
[04:46] <lifeless> spiv: got a minute
[04:47] <lifeless> actually scratch that, XXX time.
[05:55] <lifeless> spiv: so we might want to move to 64K reads on pipes and sockets when we're in streaming mode
[05:55] <lifeless> keep the buffers as empty as possible
[05:55] <lifeless> or for cleverness value, figure out the current buffer size regularly and start reading in that size
[05:55] <lifeless> can scale up quite large
[05:55] <spiv> Yeah.
[05:55] <lifeless> MBs
[05:56] <spiv> Although at some point large reads may start hitting perf issues in our code
[05:56] <spm> ziggy bytes!
[05:56] <spiv> Due to relatively naïve string splitting, etc.
[05:56] <lifeless> \o/ partial branch fail -> uploading entire bzr now.
[05:56] <lifeless> spiv: yeah. OTOH if we find those we can fix em
[05:57] <spiv> Right.  Just something we should be aware of.
[06:20] <lifeless> :(
[06:20] <lifeless> :!bzr push lp://staging/~lifeless/bzr/propose-accepted
[06:20] <lifeless> 119387kB    74kB/s | Fetching revisions:Inserting stream
[06:34] <lifeless> spiv: another data point, we're doing many small writes in that push ^
[06:34] <lifeless> write(7, "b\0\0\f\346B3289\ntexts\n\ngroupcompress-"..., 3307) = 3307
[06:37] <spiv> lifeless: Hmm :(
[06:38] <spiv> I thought we'd fixed that :(
[06:40] <lifeless> - 207016kB     0kB/s | Fetching revisions:Inserting stream
[06:40] <lifeless> a tad excessive, methinks
[06:41] <spiv> For bzr.dev?  Oof.
[06:41] <lifeless> yeas
[06:42] <lifeless> now I get to test my patch
[06:42] <lifeless> YAK SHAVING
[06:42] <spiv> I hear you have particularly woolly yaks in NZ.
[06:44] <lifeless> hmm, trunk is unhappy for lp-propose
[06:44] <lifeless> however, dinner, then digging.
[06:44] <lifeless> NotFound: Object: <canonical.launchpad.systemhomes.WebServiceApplication object at 0x6d90d90>, name: u'1.0'
[06:45] <lifeless>  <>
[07:53] <lifeless> back
[07:54] <lifeless> ok, so lp-propose in trunk is failing to get an OAuth token
[08:02] <lifeless> mgz: did you do the edge->production mass change?
[08:02]  * lifeless needs to find it again
[08:02] <mgz> http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/revision/5244/bzrlib/plugins/launchpad/lp_propose.py
[08:02] <mgz> revert that, see if it magically fixes it
[08:02] <lifeless> mgz: it does
[08:02] <lifeless> mgz: or rather, reverting the entire patch fixed it
[08:03] <mgz> okay, so, either I didn't know what I was doing and broke something, or a change is needed elsewhere as well
[08:03] <lifeless> mgz: possibly/probably the latter. Or cached credentials may be at fault.
[08:03] <lifeless> mgz: I propose to:
[08:03] <lifeless>  - back it out now [so dailies don't get broken etc]
[08:04] <mgz> can you see if just backing out that line is sufficient?
[08:04] <lifeless> sure
[08:04] <mgz> narrows down what needs looking at. if it's just that, nix it.
[08:06] <lifeless> yeah, that makes it work for me
[08:06] <lifeless> https://code.edge.launchpad.net/~lifeless/bzr/propose-accepted/+merge/25747
[08:09] <lifeless> mgz: so, roll back that one change and you'll inwestigate ?
[08:10] <mgz> yeah, revert that file then, and maybe someone who knows launchpadlib can say what was wrong
[08:10] <lifeless> I think its probably something to do with the beta->1.0 change. Or something.
[08:10] <mgz> or yeah, I can poke further
[08:10] <lifeless> so, I think the key elements are:
[08:10] <lifeless> lets open a bug on this
[08:10] <lifeless> so we can close the one about xmlrpc
[08:11] <mgz> right, makes sense.
[08:11] <lifeless> I'll do that
[08:11] <mgz> are there any elements not done for the current bug? it's filed against three projects.
[08:11] <lifeless> whats the current bug # ?
[08:12] <lifeless> 581670
[08:12] <mgz> ...I just spent fifteen seconds failing to copy that from my address bar >_<
[08:12] <mgz> mouse/keyboard coordination lacking
[08:13] <lifeless> :)
[08:13] <lifeless> https://bugs.edge.launchpad.net/bzr/+bug/583667
[08:14]  * mgz unedges the url to subscribe
[08:14] <lifeless> also, naughty, the 581760 change didn't have NEWS entry.
[08:14] <lifeless> please always put stuff in NEWS if it is going to matter to anyone.
[08:14] <mgz> didn't really seem user-facing to me, but maybe it is
[08:15] <lifeless> new cached credentials
[08:15] <lifeless> every user having to re-auth against lp is fairly user facing :)
[08:15] <mgz> ah.
[08:15] <lifeless> now, arguably we should use the same token file on edge and prod
[08:15] <lifeless> I'll file a bug on that in a sec.
[08:22] <lifeless> mgz: https://code.edge.launchpad.net/~lifeless/bzr/lp-propose-host/+merge/25749 review kplease
[08:27] <mgz> done.
[08:31] <lifeless> oh also - for judging whether to put something in NEWS
[08:31] <lifeless> we have a programming userbase too; so 'user facing' really means 'end user or plugin/api consumer or devs-should-know'
[08:31] <lifeless> if that makes sense
[08:34] <mgz> hm, sure.
[08:42] <spiv> We even have an "Internals" section in NEWS
[08:43] <spiv> For changes likely to be significant for other bzrlib devs, I guess, even if it's not strictly meant to affect end users or 3rd party code.
[08:54] <lifeless> mgz: anyhow, enough said about it - I think you need to dial up the 'tell people about it' sensitivity a bit and its all good.
[09:13] <igc> night all
[09:19] <lifeless> gnight
[10:04] <mgz> hmpf, need to get some of my harder stuff to a landable state, not just doing these ten-minute changes
[10:17] <lifeless> true
[10:17] <lifeless> and I'm going to encourage you on the testtools exception encoding thing too.
[10:18] <lifeless> though I think its temporarily under control
[10:18] <mgz> I'm struggling to write that in a non-black magic way
[10:18] <lifeless> if you want a teddy bear, tell me your thoughts
[10:19] <lifeless> I'm utterly failing to do $fridaynight
[10:19] <mgz> well, to change how testtools formats the traceback, I'm adding a method _exc_info_to_unicode to TestResult
[10:20] <mgz> which is basically just _exc_info_to_str but with traceback.format_exception swapped out
[10:21] <mgz> but means I need to to do one of 1) duplicate the function, 2) monkey-patch, 3) crazy low level stuff
[10:22] <mgz> and it only gets more squiggly from there
[10:24] <mgz> the bits that need to be change are scattered across unittest, traceback, and linecache and not in any coherant manner
[10:24] <lifeless> ok
[10:24] <lifeless> so lets start at the bottom
[10:24] <lifeless> we know we want a fixed linecache ?
[10:24] <lifeless> lets say we have a fixed_linecache module
[10:24] <lifeless> which we use and wraps linecache
[10:25] <lifeless> ignoring how we choose to use it, it can be tested well there, yes ?
[10:25] <mgz> hm, yeah, the other option I was considering there was giving it promoted tuples, which also store the file encoding
[10:25] <lifeless> sure, that seems orthogonal to where you put the adaption stuff
[10:25] <lifeless> somewhere we need the code that fixes linecache
[10:25] <lifeless> so lets get that solid with tests.
[10:26] <lifeless> then a fixed traceback, with tests. We could use mocking or dependency injection, or just integration tests for that.
[10:26] <mgz> was it selftest you suggested as a template for whole-source-module tests?
[10:26] <mgz> test_selftest that is
[10:26] <lifeless> oh
[10:26] <lifeless> bzrlib's test_plugins does something like that
[10:27] <lifeless> as does testrepository
[10:27] <lifeless> testrepository uses testresources... hmm
[10:27] <mgz> a, testrepository, that was it
[10:27] <lifeless> jml: how would you feel about testtools growing a make check dep on testresources?
[10:27] <mgz> but yeah, the test_plugins way makes sense to me
[10:28] <jml> lifeless: fine by me.
[10:28] <mgz> this is going to be 90% corner cases, but should all end up making sense, and cover the common(ish) case of locale'd os messages
[10:30] <jml> lifeless: I kind of half-think they should be one project anyway
[10:30] <lifeless> mgz: ok, so you could use either - have a look at both. If you want to use testresources, I'll happily have the generic resources from testrepository lifted up into testresources in a new resources package, or something.
[10:30] <lifeless> jml: possibly
[10:30] <lifeless> man, today was a bug filing day
[10:30] <mgz> I see you've done an impressive number
[10:31] <mgz> five or so just from lp-propose investigations
[10:34] <lifeless> GaryvdM: actually bzr-search wasn't quite fixed; Its fixed now.
[10:42] <GaryvdM> lifeless: cool
[11:12] <lifeless> spiv: apparently resolve --take-other will delete the dir and its contents
[11:13] <lifeless> this is tolerable but still ... icky, I think
[11:17]  * lifeless pops up a level
[13:15] <lifeless> jam: gmorning
[14:55] <GaryvdM> amanica: Mow about hacking together this Sunday?
[14:55] <GaryvdM> amanica: Is your wife busy?
[14:57] <bialix> mow?
[14:57] <GaryvdM> Hi bialix
[14:57] <bialix> hey Gary :-)
[15:09] <jam> morning lifeless
[15:09] <jam> Peng__: history-db just landed in trunk, if you want to check it out :)
[15:10] <amanica> hi GaryvdM :-) , I'll have to check with her, but I'm keen to get some things done this weekend. will have to let you know
[15:11] <bialix> hey amanica! how's your trip back?
[15:11] <GaryvdM> amanica: Ack
[15:11] <amanica> hi bialix, I slept like a baby in the plane. it still took me a couple of days to recover
[15:12] <amanica> bialix, I used some of the russian you tought me today
[15:12] <bialix> yep, the same here
[15:12] <bialix> oh, nice
[15:12] <amanica> good to here
[15:12] <bialix> you make your coleage shy?
[15:12] <amanica> no :-) , but she said she "isn't lazy!"
[15:12] <bialix> work hard, don't be lazy!
[15:12] <amanica> yep :)
[15:12] <bialix> lol
[15:13] <bialix> and what you said?
[15:14] <amanica> I told her the story about why I thought about learning that in russian
[15:14] <amanica> her russian is a bit rusted though.
[15:15] <amanica> bialix, you never tought me how to say goodbye
[15:16] <bialix> bye == poka
[15:16] <amanica> wow
[15:16] <bialix> or even paka
[15:16] <amanica> cool
[15:16] <bialix> goodbye is longer
[15:16] <bialix> do svidanya
[15:17] <bialix> direct translation: for next meeting
[15:17] <amanica> cool thanks
[15:19] <GaryvdM> bialix, amanica: qannotate now has Stable Scroll®
[15:20] <amanica> GaryvdM sweet!
[15:21] <GaryvdM> bialix, amanica : It also trys to keep the selection stable, but is a bit iffy if the selection is inside a modified hunk
[15:22] <bialix> Stable Scroll?
[15:22] <GaryvdM> bialix: If you change revision/refresh/change encoding, it keeps you scroll position.
[15:23] <GaryvdM> *your
[15:23] <bialix> COOOOOOOOOL!
[15:25] <beuno> removed:
[15:25] <beuno>  loggerhead/wholehistory.py
[15:25] <beuno> \o/
[15:26]  * GaryvdM must take a look at history-db to see if it can be used in qlog
[15:27] <GaryvdM> jam: You said earlier that history-db landed in trunk. Is that in loggerhead?
[15:27] <jam> GaryvdM: yes
[15:38] <nigelb> If I want to import a cvs repository to bzr to dissect commits, how do I go about it?
[15:38] <jam> nigelb: cvs2bzr would be my suggestion
[15:38] <jam> in the cvs2svn project
[15:39] <nigelb> jam: is there documentation somwhere I can refer to while it installs??
[15:39] <jam> Not sure offhand
[15:39] <nigelb> no problem,s I'll look at man :)
[15:40] <jam> also, you probably want bzr-fastimport
[15:43] <nigelb> well, now the problem is cvs2bzr fails because I dont' have a cvsroot directory, sigh
[15:44] <jam> it expects you have the raw repo, I believe
[15:46] <nigelb> great.  I only have an annonymous checkout
[15:46] <nigelb> Nothing I can do in that case?
[15:48] <jam> ask launchpad to import it for you?
[15:49] <nigelb> just for dissecting a commit, wouldn't that be abuse? ;)
[15:49] <jam> how much history do you want?
[15:49] <jam> lp doesn't really mind
[15:49] <jam> the project might even already be there :)
[15:50] <nigelb> there is a bug fixed between 1.2.2 and 1.2.3 release and cvs isn't friendly enough to examine the thing
[15:50] <nigelb> the project uses source forge
[16:14] <nigelb> jam: as you said, there is a project
[16:14] <nigelb> and I just requested code import. :)
[16:51] <bialix>   File "C:\Python25\lib\site-packages\testtools\content.py", line 91, in <lambda>
[16:51] <bialix>     content_type, lambda: [value.encode("utf8")])
[16:51] <bialix> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd2 in position 425: ordinal not in range(128)
[16:52] <bialix> and what I should do?
[16:56] <jelmer> bialix: it looks like value is a plain string and not a unicode string
[16:56] <vila> bialix: ping mgz ;-)
[16:56] <bialix> yes, it is
[16:57] <bialix> I'm trying to write test for diff
[16:57] <bialix> there is only plain strings
[16:57] <bialix> but with non-ascii characters
[16:57] <bialix> mgz: ping!
[17:05] <bialix> vila: what I can do about this problem?
[17:06] <bialix> I'm sure my test is correct now, but I can't go further
[17:08] <vila> bialix: use pdb to identify the string
[17:08] <bialix> which string?
[17:08] <vila> I think there is an env variable to make selftest call pdb on execptions
[17:08] <vila> bialix: value
[17:08] <vila> bialix: the one that is failing
[17:10] <bialix> just print was enough
[17:10] <bialix> found
[17:32]  * mtaylor thinks bzr launchpad-login should grab info from launchpad and set bzr whoami... would make getting set up on a new box quicker
[17:36] <jelmer> mtaylor: or the other way around perhaps
[17:37] <jelmer> mtaylor: it could just ask launchpad for the credentials associated with the current users id
[17:37] <jelmer> not the credentials but the username
[17:37] <jelmer> having launchpad provide your credentials for you is probably not such a good idea :-)
[17:37] <mtaylor> good point
[18:17] <GaryvdM> jam: Hi
[18:17] <GaryvdM> jam: I'm trying to grork history-db
[18:17] <jam> hi GaryvdM
[18:17] <GaryvdM> jam: Is there a README?
[18:21] <GaryvdM> jam: I have some questions, I don't want bug you if there are docs I should read first. The only doc I could find is dotted_revno_caching.txt
[18:21] <jam> GaryvdM: that's the only doc I know of :). What are you looking for?
[18:21] <jam> Querier is what you are going to want to be using
[18:22] <GaryvdM> jam: How do I get it to create and populate a db?
[18:22] <GaryvdM> jam: Does it update on tip change or read?
[18:23] <GaryvdM> jam: or revision fetch?
[18:23] <jam> GaryvdM: history-db has hooks to auto-update on tip changed
[18:24] <jam> however, anytime you use Querier, it also ensures that the data has been imported
[18:24] <jam> (Querier.ensure_branch_tip() will import if it hasn't already)
[18:25] <jam> You might want to use "bzrlib.plugins.history_db._get_querier" which will cache a Querier object on the Branch object
[18:25] <GaryvdM> jam: Cool, so if you do a tip change with out the plugin, things will be ok.
[18:25] <jam> yep
[18:25] <GaryvdM> jam: How do I get it to create and populate a db?
[18:26] <jam> The way the plugin is written, it will manage a db if you set "history_db_path=XXX"
[18:26] <jam> If you want qbzr to depend on it existing, then we can look at having a custom field (loggerhead does this)
[18:27] <GaryvdM> jam: enviroment var, or config?
[18:27] <jam> config
[18:27] <nigelb> ok, so LP cannot import from sourcforge? running into issues because anonoymous access still requires a password :/
[18:27] <GaryvdM> jam: I plan to make qlog use it if availible, else work the old way
[18:29] <jam> nigelb: I'm pretty sure you can import from the https:// anon access
[18:29] <jam> At least, google shows other projects being imported from there
[18:29] <jam> GaryvdM: so one option is to use Branch apis that history-db interacts with
[18:29] <jam> such as Branch.iter_merged_revisions()
[18:29] <nigelb> jam: hm, I'll read that up
[18:29] <jam> (iter_merge_sorted() ?)
[18:30] <GaryvdM> jam: ack
[18:30] <GaryvdM> jam: can history db's be shared accross branches and repos?
[18:38] <nigelb> jam: um, ok I can't find anything for that :(
[18:38] <jam> GaryvdM: yes, and it is intended to be used that way
[18:39] <jam> (it was intentionally designed to share history between branches)
[18:41] <GaryvdM> jam: You have a cmd_history_db_create, but it's not registered.
[18:42] <jam> GaryvdM: it should be, as I use it
[18:42] <jam> hmm. I wonder if I deleted the registration by accident when I removed some other stuff
[18:42] <jam> probably
 moin
[18:43] <GaryvdM> Hi lifeless, wow, very early.
[18:43] <jam> GaryvdM: restored
[18:43] <GaryvdM> jam: I'll check for you
[18:43] <GaryvdM> oh - og
[18:43] <GaryvdM> *ok
[18:43] <jam> I got rid of a bunch of them, accidentally included this one
[18:47] <lifeless> GaryvdM: yes, sleep fail.
[18:47] <lifeless> tail end of jetlag getting its revenge, I think.
[18:47] <GaryvdM> ~/bzr/bzr.dev $ bzr history-db-create
[18:47] <GaryvdM> bzr: ERROR: sqlite3.OperationalError: unable to open database file
[18:47] <GaryvdM> jam: Must I create it my self with sqlite?
[18:48] <GaryvdM> lifeless: sorry to hear that
[18:48] <jam> GaryvdM: no, it should create it itself, but you have to tell it where with --db or by setting history_db_path=XXX
[18:49] <GaryvdM> I'm like jelmer - no fail once I get to bed, but I fail alot in getting to bed
[18:49] <GaryvdM> jam: I did
[18:49] <jam> bzr history-db-create --db=test.db -d .
[18:49] <jam> try that
[18:49] <GaryvdM> jam: global config in ~/.bazaar/bazaar.conf
[18:49] <jelmer> GaryvdM: lol :-)
[18:49] <GaryvdM> ok
[18:49] <jelmer> that's a nice way of putting it
[18:50] <jam> hmm... still should have worked
[18:50] <jam> does the *directory* where you wanted to create the db exist?
[18:50] <jam> I only create the db itself
[18:50] <GaryvdM> jam: command worke
[18:50] <GaryvdM> jam: let me check
[18:51] <GaryvdM> jam: sorry - I had a typo in my .conf
[18:51] <jam> :)
[18:52] <lifeless> jam: the brain in 2 days ;P
[18:54] <jam> lifeless: nice. I should have mentioned that a lot of the mobs don't see a lot of use.
[18:54] <lifeless> :>
[18:54] <jam> At high level, it tends to be ichi swarm because of all the towers
[18:54] <jam> and fast rebuild time
[18:55] <jam> and 2 ichi == 1 crabatron, but they build in <1/2 the time
[18:55] <lifeless> ah
[18:55] <lifeless> designing games is hard
[18:55] <lifeless> designing is hard
[18:55]  * GaryvdM curses backyard monsters..... (I'm addicted...)
[18:56] <lifeless> thats it, friend request for you.
[18:56] <jam> I'm just waiting for Robert to come out of protection so I can raise his town :)
[18:57] <lifeless> raise or raze :P
[19:00] <GaryvdM> dam - just got the trogen, and ackpected...
[19:00] <GaryvdM> *accepted
[19:04] <GaryvdM> lifeless: Your yard is very neat - I wish I could restart...
[19:05] <lifeless> GaryvdM: you can move things
[19:05] <lifeless> GaryvdM: long click on something
[19:06] <GaryvdM> Ah - that is very cool
[19:12] <lifeless> my yard is probably sillyly laid out and jam will toast me
[19:12] <lifeless> but perhaps not
[19:12] <jam> lifeless: well, it depends what level your towers are :)
[19:13] <lifeless> 5 ?
[19:13] <jam> but generally, you need overlapping fire on towers
[19:13] <jam> or i can send 20+ mobs to take the towers down one-at-a-time
[19:13] <lifeless> I wish the fire zone showed
[19:13] <lifeless> I *think* my snipes are all overlapping
[19:13] <jam> absolutely, I really miss that from DD
[19:14] <jam> snipers are excellent at taking out non tower attackers
[19:14] <jam> usually only 1-shot 1-kill
[19:15] <Meths> Which game is this?
[19:15] <jam> but it takes 4 shots to kill an ichi with a level 5 tower
[19:15] <jam> Meths: Backyard Monsters, (a facebook game)
[19:15] <jam> and I can build something like 40 ichis for one go
[19:15] <jam> oh, and upgrading the flinger should take low priority
[19:15] <jam> you can fling multiple times
[19:16] <jam> lifeless: on the flip side, a full aoe tower takes 20 hits, but can kill all 40 ichis at once
[19:16] <jam> anyway, nothing really stands against multiple waves
[20:08] <GaryvdM> jam: Could we add a branch lines table, or is there an easy to query for it?
[20:09] <jam> doesn't that only matter when you have them expanded?
[20:09] <GaryvdM> jam: If the rev no in dotted_revno in 3 fields, we could SELECT UNIQUE on the first 2. Would that be fast?
[20:09] <GaryvdM> jam: hmm - intresting point
[20:10] <jam> so certainly we should think about what you need
[20:10] <jam> But if you expand, and see that there is 1.2.100, you can certainly send another query for 1.2.1-99
[20:10] <jam> which should be pretty cheap given the existing apis
[20:11] <jam> they can all be done in parallel
[20:11] <GaryvdM> jam: could I query for 1.2.* ?
[20:12] <jam> not going through Querier
[20:12] <jam> probably could from an SQL perspective, would have to think about it
[20:12] <jam> I think so
[20:13] <jam> but until you expand, you don't know if you want 1.2 or 10.50
[20:13] <GaryvdM> jam: with current schema a LIKE, but 3 fields would be cheaper
[20:13] <GaryvdM> ^ query for 1.2.*
[20:14] <GaryvdM> jam: logic would be like this:
[20:14] <jam> GaryvdM: the cost is going to be in walking the mainline, since you need to exclude other branches, etc.
[20:14] <jam> I don't think SELECT UNIQUE vs LIKE is going to be a big deal
[20:16] <GaryvdM> jam: I was thinking of to queries: select unique was to get a list of branch lines (which I might not need)
[20:17] <GaryvdM> like was for getting the revisions in a branch line.
[20:18] <GaryvdM> and LIKE would not be needed for that if the rev no was stored as 3 fields
[20:19] <GaryvdM> I'm trying to think about what we load in to mem, and when....
[20:19] <jam> GaryvdM: so we can certainly discuss how qbzr could look
[20:20] <jam> like just grabbing the mainline to fill a page to start
[20:20] <jam> and then using 'iter_merge_sorted' when expanding a node
[20:20] <jam> to get just what was merged into that rev
[20:20] <jam> and possibly filling out the rest of the branch line
[20:20] <jam> I'm not sure if the latter is needed or not
[20:20] <jam> vs just having a 'tail' pointer that could be expanded
[20:20] <GaryvdM> jam: yes - was just about to say that
[20:21] <GaryvdM> for branches like emacs and OOo
[20:21] <jam> And iter_merge_sorted() is available on Branch and optimized today by bzr-history-db
[20:21] <jam> so it would allow you not to directly think about bzr-history-db
[20:21] <jam> branch already caches its merge sorted result
[20:21] <jam> so it will be cheaper if you have it, but okay if you don't
[20:22] <jam> ok== no worse than today I tihnk
[20:22] <jam> loggerhead couldn't because it doesn't hold onto a branch between requests
[20:23] <GaryvdM> jam: I create a fake merge rev for viewing multiple branches, and branches with pending merges. Could we add a revision and its merge sort, with the revid a hash of the parnent ids?
[20:24] <jam> GaryvdM: you could certainly do something like that
[20:25] <jam> not sure how to inject that into Importer
[20:25] <jam> but i'm sure we could find a way
[20:25] <GaryvdM> jam: Cool - I think this could work well.....
[23:09] <exarkun> Are repositories safe for concurrent access?
[23:10] <lifeless> yes
[23:11] <exarkun> Hooray.  Thanks.
[23:15] <lifeless> why do you ask?
[23:23] <exarkun> Coz I'm about to have a repo shared between multiple mostly-uncoordinated actors doing checkouts of various things
[23:23] <exarkun> So I'd like for the repo to not end up corrupted by this. :)
[23:24] <exarkun> Do you know if the same goes for the stuff in ~/.cache/bazaar/svn?  Coz I'm checking out from an svn repo.
[23:27]  * exarkun waits ages for the initial 'bzr checkout' to complete so he can see what happens next
[23:29] <lifeless> less so but enough
[23:29] <lifeless> one particular bit that isn't safe is ~/.bazaar/locations.conf
[23:29] <lifeless> which bzr-svn writes to (once per repo I think)
[23:29] <lifeless> there is a bug on this
[23:32] <exarkun> yea, I filed that one
[23:36] <exarkun> Humm
[23:37] <exarkun> Is it okay if my 'bzr checkout' gets SIGKILL'd?  I mean, if I start it over again, does it pick up where it left off, or does it have to start from scratch?
[23:37] <exarkun> It was still working through the write-to-.bzr/repository/upload/ step when it got killed
[23:38] <exarkun> (my build system is configured to think 1200 seconds without output means something is broken :/)
[23:38] <lifeless> exarkun: the various components are created in a safe order but generally you'll start over. If you're using bzr-svn it does repository insertions in batches
[23:38] <exarkun> I'm just doing "bzr checkout svn://...", I'm not sure if that's bzr-svn or not, but I guess so?
[23:38] <lifeless> if you're using a shared repositroy and bzr-svn, it will resume from ~ where it left off
[23:38] <lifeless> thats bzr-svn
[23:39] <lifeless> you're using a shared repository if you've done 'bzr init-repo' somewhere above the directory you're checking out into
[23:39] <exarkun> Yea, that's what I've done
[23:43] <lifeless> in which case it should resume for you
[23:43] <lifeless> if this is twisted... you can also seed repo with an already converted branch by pulling that into it anywhere