[00:06] <spiv> Good morning.
[00:11] <poolie> hi there spiv
[00:14] <spiv> The cold seems to have mostly subsided today, thankfully.
[00:14] <peitschie> poolie: do you know much about bzr reconcile behaviour?
[00:14] <poolie> that's great
[00:14] <poolie> peitschie: a bit, what do you want to know?
[00:16] <peitschie> poolie: i've been running into issues where the canonicalize-checks take around 20hrs to run over my repository at work
[00:16] <peitschie> i was trying to come up with some ways of speeding things up a bit
[00:16] <peitschie> i did notice that the process seems to be very heavily cpu bound... but it's only single-threaded
[00:18] <peitschie> as far as you know, does reconcile require sequential repair work (i.e., the next revision requires all previous revs to be fixed) or are chunks of it disconnected (i.e., multiple revs could be farmed out to be repaired by each core and then updates sequentialized back to the main thread)?
[00:21] <poolie> normally you should only need to run it to recover from a previous problem?
[00:21] <poolie> did someone or some program actually tell you to run it?
[00:21] <poolie> i think it probably could be parallelized
[00:34] <peitschie> poolie: at the moment I'm still trying to recover from https://bugs.launchpad.net/bzr/+bug/522637 and https://bugs.launchpad.net/bzr/+bug/485601
[00:35] <peitschie> poolie: I ran reconcile last weekend, but found someone checked in on monday without getting the "reconciled" repo... which broke things again
[00:35] <peitschie> poolie: because of the amount of time it takes to run things, I don't have a big enough space to really do a full reconcile during the week, as it takes a full day to do it :)
[00:36] <peitschie> it's one of the things I think is important to a corporate environment... any repair job needs to be able to happen over night
[00:36] <peitschie> because I know programmers a not infallible, and there is always a chance there will be another repo-breaking bug...
[00:37] <peitschie> i've been considering spending some time trying to speed up reconcile, so recovery from future bugs of this nature can be achieved during the working week without stopping all dev work during the day
[00:37] <poolie> :?
[00:37] <poolie> yuck
[00:37] <poolie> one good way to start might be to be able to select particular types of reconciliation to do
[00:38] <peitschie> yep... those were some of the other things i was going to look at
[00:38] <poolie> similarly for check for that matter
[00:39] <peitschie> i figured a lot of time is likely spent calculating sha's and similar tho as well... so if I can potentially just multi-thread *that* part slightly, I think there might be a reasonable gain to be made
[00:40] <peitschie> I started playing with some profiling last night to see, so I don't really have any clear ideas on it yet :)
[00:40] <peitschie> but just thought i'd ask in case you were aware of much around it :)
[00:42] <peitschie> the other thing i'm considering is how we might potentially be able to generate more useful debug information for the previously listed bugs as well
[00:42] <peitschie> in the case of both of those, they have proved exceedingly hard to track down and verify because of the requirement of having it occur on a publically available repo
[00:43] <peitschie> again... i have no real plans at this point... just pondering some things that would make my adoption here a little less risky a prospect
[00:44] <poolie> mm, again maybe having a task-specific check output some non-sensitive debug data would be good
[00:44] <poolie> it's a bit hard to predict ahead of time what would be useful
[00:44] <poolie> but it is possible
[00:44] <peitschie> i figured that there is always some things taht would be missed
[00:45] <peitschie> i'm wondering if perhaps dumping things like revision history, sha's, ancestry... i.e., mainly the repo and branch histories might be sufficient to get a head-start
[00:46] <peitschie> my thought is to try and work out what info would have helped track down the previously listed bugs... see how hard it is to produce... then potentially wrap that up in a command/tool of some sort (potentially even as part of check)
[00:46] <poolie> that would be good
[00:46] <peitschie> so even if we don't get everything next time, *hopefully* future issues like these (that seem to revolve a bit around shared repo => shared repo) might be a bit easier to track
[01:45] <dOxxx> Good evening.
[01:46] <dOxxx> Anybody here know about bzr-explorer internals?
[01:46] <poolie> hi there
[01:46] <dOxxx> In particular, the way it invokes qbzr commands?
[01:46] <poolie> not a lot, but what's up?
[01:46] <dOxxx> I'm working on my mergetools thing and it looks like explorer is using my system installed bzr to run qbzr commands instead of the dev version of bzr that I invoked explorer with.
[01:47] <dOxxx> So I get an ImportError because my qbzr references a module (mergetools) which doesn't exist in bzr 2.2.0
[01:48] <dOxxx> but which does exist in my dev bzr
[01:48] <poolie> hm, interesting
[01:48] <poolie> that sounds right
[01:48] <poolie> hi, spiv, so what are you up to today?
[01:48] <dOxxx> poolie: what sounds right? that it uses the system bzr to invoke qbzr?
[01:49] <poolie> i mean, that sounds like a correct diagnosis, but i agree it's a bug
[01:49] <poolie> it should use the matching version, i would say
[01:49] <dOxxx> poolie: http://paste.ubuntu.com/507655/ is the relevant section of bzr.log if you could double-check my diagnosis?
[01:51] <dOxxx> I'll be damned if I can think of how to search for this bug to see if it already is known :)
[01:55] <poolie> i don't recall hearing of it
[01:56] <poolie> dOxxx: i think there is a bzr symbol to find the right executable
[01:56] <poolie> because the tests do the same thing
[01:56] <poolie> i'd just file a new bug; we can always dedupe later
[01:56] <dOxxx> I thought I remembered some discussion about it on the bzr mailing list in the last month or two.
[01:56] <dOxxx> yeah I'm typing up the bug now
[01:56] <spiv> poolie: so far mainly catching up on a week's worth of email
[01:57] <poolie> oh, wow
[01:58] <poolie> it has been a while
[01:58] <poolie> good luck :)
[02:00] <poolie> dOxxx: if you want to talk about where the bug is or how to fix it just ask, otherwise i'll leave it with you
[02:01] <dOxxx> poolie: no I'm good, just writing it up. :)
[02:01] <spiv> Thanks :)
[02:34]  * spiv -> food
[03:16] <mkanat> Is there a plugin that will just run a particular external command before "commit"?
[03:20] <lifeless> sure there is a externals hook plugin
[03:20] <lifeless> or you can just trivially do a custom one yourself.
[03:31] <mkanat> lifeless: Yeah, I figured it would be trivial, I just wanted to know if it already existed.
[06:43] <poolie> spiv, what do you think we should do about news?
[06:43] <poolie> one news file per series?
[06:44] <spiv> I think so.
[06:49] <poolie> do you want to split it up then, while you're there?
[06:50] <spiv> I can do that, what's the exact path names you're after?
[06:50] <poolie> hm, possibly we should just move it into doc/en now, and perhaps just leave a pointer in news?
[06:50] <poolie> that would be similar to whatsnew
[06:50] <poolie> and everything else
[06:50] <poolie> s/in news/in NEWS in the root
[06:51] <poolie> whatever we do, we'll need to update the doc builds rules
[06:52] <spiv> Right.
[06:52] <spiv> It would be nice to have something fairly visible in the root.
[06:52] <spiv> But not nice to accumulate clutter for every historical release in the root.
[06:53] <spiv> I suppose we could have the current series in /NEWS, and everything else in doc/en/old-news/*, but that feels a bit weird.
[06:53] <poolie> istm just a pointer there would be ok
[06:53] <spiv> Ok, phew :)
[06:54] <spiv> I'll do that.
[07:05] <vila> hi all !
[07:05] <spiv> Hey vila!
[07:05] <vila> spiv: Welcome back !
[07:06] <vila> spiv: can I have paramiko package from your ppa for maverick with my pony ? Pretty please :)
[07:06] <vila> *a paramiko package
[07:10] <spiv> vila: hmm
[07:10] <spiv> vila: I'm not sure at a glance why my last attempt to do so failed: http://launchpadlibrarian.net/52203585/buildlog.txt.gz
[07:11] <spiv> (https://code.edge.launchpad.net/~spiv/+recipe/paramiko-test/+build/340)
[07:11] <vila> spiv: the point is that there is only one test failing on babune for the maverick slave
[07:11] <vila> ha, yeah, I forgot that, you mentioned it at the time though
[07:11] <vila> spiv: sudo build-my-package ?
[07:13] <vila> spiv: hmm, rather obscure... "they" did manage to build it for maverick so there should be a way...
[07:13] <spiv> I'll hit the button again and we'll see what happens.
[07:13] <vila> aptitude command not found ??
[07:13] <poolie> hello vila
[07:14] <vila> hey poolie, hopefully I'd feel better today :-/
[07:14] <poolie> and do you?
[07:14] <vila> well, so far, yes
[07:14] <vila> but yesterday and the day before I went down without notice :-/
[07:16] <fullermd> Maybe the universe is corrupted.  Look at the checksum?
[07:17] <vila> quantic checksums make my head hurt, bad idea
[07:18] <fullermd> Just pick a worldline where they don't.
[07:18]  * vila facepalms
[07:18] <vila> ouch
[07:18] <fullermd> Also, don't smack yourself.  It hurts less that way   8-}
[07:20] <spiv> vila: apparently it built and uploaded okay this time
[07:20] <vila> fullermd: you should have said so earlier
[07:20] <vila> spiv: yeah just noticed !
[07:21] <fullermd> Well, I did.  You just heard me later.  Speed of light and all that, y'know.
[07:24] <vila> fullermd: yeah sound travels slower than light around here
[07:25] <fullermd> Naturally.  Sound is heavy.  Light is...  well, you know.
[07:26] <vila> spiv: http://ppa.launchpad.net/spiv/packaging-test/ubuntu/dists/ not updated yet, I suppose I should wait a bit
[07:27]  * fullermd cranks up his sound to compensate.
[07:27] <fullermd> Luckily, nobody around minds if I do that at 0130   :p
[07:27] <vila> spiv: ha right, the details at https://code.edge.launchpad.net/~spiv/+archive/packaging-test/+packages shows that it's still pending publication
[07:28] <vila> fullermd: :)
[07:32] <fullermd> (on that topic, I wish Youtube would sort out their 34/18 formats, so that one was consistently the better, rather than random, or even more often, one having sucky audio and decent video, and the other being the other way around :|  )
[07:46] <mgz> hm, not sure whether to send this to the list or file a bug
[07:46] <mgz> vila: can I request a full babune run on windows of a branch?
[07:46] <vila> mgz: sure, just name it
[07:47] <mgz> http://float.endofinternet.org/bazaar/bzr/hack_get_transport
[07:48] <mgz> sorted out what from your leaks branch was causing me problems, but I'm not sure why it's not broken for other people, or the best way to fix it.
[07:50] <vila> mgz: hmm, looks like your bandwith will suffer :)
[07:50] <vila> mgz: push to lp next time ;)
[07:51] <mgz> it's a five line diff, how badly can bzr... oh, are you not stacking?
[07:52] <vila> mgz: nah, it's ok, I did branch it locally to look at it...
[07:53] <vila> mgz: err, this fix changes what exactly (in behaviour) ? It looks like you're working around some import aliasing bug here
[07:54] <mgz> ha, good guess. I'll post to the list.
[07:55] <vila> mgz: note that the current approach is hackish, we should define a hook triggered when a transport *connects*
[07:56] <mgz> that sounds like a good suggestion.
[07:56] <vila> mgz: running: http://babune.ladeuil.net:24842/job/selftest-subset-windows/8/
[07:57] <mgz> thanks!
[08:03] <mgz> okay, sent email.
[08:17] <poolie> mgz, nice mail
[08:17] <poolie> i thought there already was a way to disable laziness but imbw
[08:17] <poolie> that babune report is great!
[08:18] <mgz> to be fair on that front, by the time I realised it was lazy import causing issues not something else, I didn't need to test anything else
[08:18] <mgz> it was the getting to that point which was the hard bit :)
[08:19] <mgz> (so it may well be that I could have found a way to disable lazy import, if I'd thought to(
[08:20] <mgz> ))
[08:26] <mgz> poolie: can mgordan be added to the contrib agreement group? I landed his first branch already as I got cced.
[08:28] <poolie> will do
[08:28] <bialix> heya poolie, mgz
[08:28] <poolie> hi there
[08:44] <vila> mgz: you run finished: FAILED (failures=2, errors=3)
[08:45] <mgz> not bad for a hack, and 45 mins rather than 77 is the great bit :)
[08:47] <vila> mgz: I see to... what's the message... this test would have hang or something, right ?
[08:47] <vila> s/to/no/
[08:47] <bialix> bonjour vila
[08:48] <vila> which means the leak fixes were correct except they didn't catch some import aliases right ? (I confess I focused on the ones in tests...)
[08:48] <mgz> http://babune.ladeuil.net:24842/job/selftest-windows/190/ <- Took 1 hr 32 min on xp-32bits.local
[08:48] <vila> bialix: hey !
[08:48] <mgz> http://babune.ladeuil.net:24842/job/selftest-subset-windows/8/ <- Took 45 min on xp-32bits.local
[08:48] <vila> right, most probably the cause is that we don't wait for threads to finish anymore
[08:49] <mgz> basically yeah, complete for fix and all the "thread ('127.0.0.1', 2265) -> ('127.0.0.1', 2263) hung" goes away
[08:49] <vila> shudder
[08:49] <vila> care to comment on https://code.edge.launchpad.net/~vila/bzr/imports/+merge/36324 ?
[08:50] <mgz> no. :P
[08:50] <vila> I think I failed to convey some hard numbers about time spent to debug such obscure crazyness
[08:51] <vila> anyway, thanks for pointing out the obvious fix, but that's still a hack on top of a hack, so we should implement a hook
[08:52] <vila> I went out of steam on the subject at the time, may be we can pair on this ?
[08:53] <vila> mgz: A most probably related topic being the test suite still can't be run on OSX because some sockets are still leaking, I will try your branch there
[08:53] <poolie> vila can you try to help a bit with reviews/landings?
[08:53] <poolie> i realize i'm pp and i've done some but it's still queued
[08:53] <vila> poolie: yup
[08:53] <vila> poolie: I think it's fine that PP ask for help when needed
[08:55] <mgz> ^I agree merging namespaces can cause bugs like this, but it's also useful. I don't think we can coding style it away.
[08:55] <poolie> hm, i just did a parallel test and got ebadf again
[08:55] <vila> poolie: and I wonder if we should not *all* focus on getting the queue back into control, it's awesome to have so many contributions but it will collapse fast if we don't address them
[08:55] <vila> poolie: for an https one ?
[08:56] <poolie> vila, sftp actually, inside paramiko
[08:56] <mgz> I'll happily have a crack at fixing this the right way, especially if you're offering help/
[08:56] <poolie> running parallel=fork seems to be the only time my machine's fan spins all the way up :)
[08:56] <vila> hehe
[08:56] <poolie> it would probably be an even stronger effect on apple hardware
[08:56] <mgz> vila's the one most in need of reviewing I think, you'll need to press gang someone else as well :)
[08:56] <vila> poolie: sry I can't remember without a tracback, what's the failing test ?
[08:57] <poolie> i appreciate yours
[08:57] <spiv> I have a pselftest='selftest --parallel=fork' alias
[08:57] <vila> spiv: tss
[08:57] <vila> :)
[08:57] <poolie> vila, for the ebadf thing?
[08:57] <mgz> though neil's on a roll too.
[08:57] <vila> spiv: bzr alias tss="selftest --parallel=fork" :-D
[08:58] <vila> poolie: yup
[08:58] <spiv> Hopefully now I'm back from a week of leave + virus I'll be able to help with the queue too.
[08:58] <spiv> Help reduce it, rather than grow it, that is ;)
[08:58] <vila> mgz: and you've helped a lot too, thanks for that
[08:58]  * spiv -> dinner
[08:59] <mgz> I still need a lazr.restfulclient reviewer if anyone can dig one of those up.
[08:59] <vila> poolie: on a related note, we have an RT asking for an upgraded testtools on pqm that seem to be blocked by a desire to upgrade to lucid, I'm a bit lost about the whys here, can you summarize ?
[09:00] <poolie> https://bugs.edge.launchpad.net/bzr/+bug/656170
[09:00] <poolie> vila, sorry, i don't know many specifics
[09:00] <poolie> i know the GSAs are in the process of upgrading some boxes
[09:00] <poolie> suggest you ask in canonical is
[09:00] <vila> BIG RED FLASHING WARNING: lp read-only
[09:01] <poolie> no warning?
[09:01] <vila> poolie: ha, that one, but it doesn't make the test fail right ?
[09:01] <spiv> IIRC when I asked spm last week the RT wasn't blocked, merely some way down the queue?
[09:02] <vila> it's been announced, but I lost a bug report yesterday, so I'm warning others ;)
[09:03] <vila> poolie: I'll comment on the bug, but roughly, yes, it's a bad timing, but no it's really a bug
[09:05] <vila> poolie: the fix would be to catch the exception like we do for our threads, but this one is private to paramiko and injecting our class here can't be done cleanly, so it's way down my TODO list and I would even make the bug importance a wishlist, the test doesn't fail
[09:08] <spiv> vila: i.e. the bug is (partly?) in paramiko?
[09:12] <vila> spiv: hmmm, IMHO, the bug is partly in python: there should be a simpler way to handle exceptions in threads than redefining the Thread class
[09:12] <vila> spiv: and a way to kill a thread by throwing an exception too (and a pony)
[09:14] <vila> spiv: that won't give us a direct way to tell paramiko to ignore EBADF for its threads either though, but that will become easier
[09:15] <vila> the problem here is that we want to ignore this exception before stopping the thread but *not* when starting it
[09:19] <vila> mgz: so the idea for the hook is that we want to replace the hackish overideAttr with a hook that will do roughly the same thing: keep track of the transport and disconnect it when tearing down the test
[09:20] <spiv> vila: you don't need to subclass Thread
[09:20] <spiv> vila: you just need to wrap the function you pass as the target
[09:20] <vila> there is more than one way to do it :)
[09:21] <spiv> vila: unless you mean monkey-patching the threading.Thread globally to change the behaviour of *all* threads... which seems risky.
[09:21] <vila> spiv: the most convenient is to be able to say to a running thread: from now on, swallow these exceptions
[09:22] <spiv> I mean, it is a bit weird to me that the main thread has sys.excepthook, and the other threads don't.
[09:22] <spiv> Just because it's inconsistent.
[09:22] <spiv> (Or does sys.excepthook intercept unhandled exceptions from all threads...?)
[09:22] <vila> interesting approach...
[09:23] <spiv> (Nope, it doesn't.)
[09:23] <vila> too bad :-/
[09:24] <spiv> But anyway, in principle you don't need any special facility to deal with top-level exceptions.  You can already write "try: ...; except: top_level_exc_hander()" at the top-level of the code.
[09:25] <vila> spiv: for a third-party library ?
[09:25] <spiv> Well, how a 3rd-party library runs its threads is ideally its own business.
[09:26] <vila> hehe
[09:26] <spiv> It's like how you can't really add logging to a 3rd-party library from the outside
[09:27] <spiv> Either it already does that (and ideally allows you to have some control over where/how much it logs), or it doesn't and you need to get it fixed upstream.
[09:36] <vila> spiv: right, in this sense, yes, this could be seen as a paramiko bug, but given the time it takes for your fix about the wrong address families to be taken into account, I'm not holding my breath for something that impact only tests :-(
[09:38] <spiv> vila: well, being clear about the causes and deciding what to do about the problem given the circumstances are different issues.
[09:38] <spiv> vila: maybe there's a workaround we ought to do, but that doesn't magically make it not paramiko's fault :)
[09:41] <vila> well, paramiko rightly raises the EBADF exception, *we* want to be able to ignore it in *some* cases and even there, we change our mind *after* asking paramiko to start its thread
[09:41] <vila> that's the cause
[09:41] <vila> there is indeed more than one way to deal with it
[09:43] <vila> and like the other problems encountered with the leaks, the fact that we run the server and the client in the same process is something that hasn't been taken into account in the design
[09:44] <vila> it works better for paramiko than for SocketServer though
[09:45] <spiv> vila: the bug isn't that paramiko raises EBADF, it's that it spews it to stderr rather than giving the app a chance to know about it.
[09:46] <spiv> Probably the obvious thing for paramiko to do would be log the exception.
[09:46] <vila> spiv: well, if I was the paramiko author I could answer: hey, that's Thread that is spewing on stderr :-P
[09:46] <spiv> Seeing as it already uses the logging module.
[09:46] <spiv> Sure, that's the mechanism.
[09:46] <spiv> But that doesn't make "unconditional spew on stderr" the right behaviour for a library.
[09:47] <vila> and whatever happens in this thread is supposed to be conveyed to the user via the channel object (or whatever, don't remember the details)
[09:48] <spiv> (Especially given that in general paramiko takes a fair bit of care to use logging rather than unconfigurable spew on stderr)
[09:48] <vila> spiv: my suspicion is that reproducing this without running the server and the client in the same process is non trivial (if possible)
[09:49] <spiv> Sure, I'm not arguing about that point.
[09:50] <Glenjamin> do you know where abouts in the paramiko code it's printing directly to stderr?
[09:50] <spiv> (In fact, if robey had encountered it himself already I bet he'd have fixed it to be suppressed or logged, rather than let the Python threading bits spew it to stderr)
[09:51] <vila> Glenjamin: lol, nowhere in this specific case, it's a side effect of using a thread and having an uncaught exception bubbling up to the top level
[09:51] <Glenjamin> that sounds broken
[09:53] <Glenjamin> the thread's exception has nowhere to bubble up to, and can't kill the master process i guess?
[09:55] <spiv> It doesn't kill the main thread, no.
[09:55] <spiv> The Python VM just writes a traceback to stderr when an unhandled exception happens in a thread.
[09:56] <spiv> So if you don't want that, make sure you never have unhandled exceptions in your thread.
[09:56] <vila> well, that's my main grief against python threads, the interpreter takes a great care to handle them but the exception handling is left to the user, requiring him to be ready for it in the function given to the thread
[09:56] <spiv> Which isn't that hard, really.
[09:56] <Glenjamin> So the only fix is to wrap the thread contents in an except block
[09:56] <spiv> vila: I really don't see the difference with the main thread
[09:57] <vila> well, the exception in the thread is not propagated there for one
[09:57] <vila> so if you change your design to use threads your callers are suddenly doomed
[10:01] <spiv> Propagated where?
[10:01] <spiv> They have a separate call stack, that's kinda the point ;)
[10:06] <vila> I started a thread, I want a simple mean to know that something bad occurred
[10:07] <spiv> vila: you started the thread, so you can
[10:07] <spiv> vila: just pass in a function that reports that
[10:07] <vila> mgz: wooohhoooo, yeeeehaaa, test suite passes on OSX ! Pfew, my last attempt was depressing, I thought there was some more unknown leaks...
[10:08] <vila> spiv: that's not simple enough, see my previous point: callers have to care now
[10:08] <spiv> It's not hard to write a generic wrapper ("try: foo(); except: report_error()") to help you
[10:08] <mgz> vila: you're better than you thought you were :)
[10:09] <vila> mgz: yeah, something like that ;)
[10:09] <mgz> spiv: isn't the problem that it's not "you" it's paramiko, and we can't raise the maintainer currently anyway
[10:09] <spiv> So that your start thread call becomes Thread(target=err_wrapper, args=(func,)) rather than Thread(target=func)
[10:09] <spiv> Which isn't exactly onerous.
[10:09] <vila> spiv: and you also need to provide this to the caller which itself, etc
[10:09] <spiv> I mean, it would be nice to have it more convenient.
[10:10] <vila> exactly
[10:10] <spiv> Perhaps even have Thread.get_result that blocks (i.e. calls join), and then either returns the value the target func returned, or reraises the exception it terminated with.
[10:10] <spiv> (And in Twisted, you can use blockingCallFromThread for something like that)
[10:11] <mgz> ..or are we on testing now, not the EBADF bug?
[10:11]  * mgz went off to get food
[10:11] <spiv> But it's really not hard to roll your own :)
[10:11] <spiv> mgz: right, the problem is in paramiko, that's my point :)
[10:12] <vila> mgz: and I argue that paramiko can point the finger at python :)
[10:12] <vila> ThreadWithException was my answer for bzr :)
[10:12] <spiv> Eh, it can point the finger at python for making it merely simple, rather than trivially easy.
[10:13] <vila> ..where the exception is re-raised by join()
[10:13] <spiv> So pardon me if I don't consider that a great excuse :)
[10:15] <vila> spiv: well, if you require all thread users to implement such a mechanism, why not implementing it in thread ?
[10:16] <vila> spiv: the try/except/finally is not always enough, you also need a guarding event in most cases and this event should be set to ensure the caller is not blocked
[10:17] <hrw> hi
[10:18] <vila> mgz: to be precise, there are still 3 failures on OSX (plus some tweaks of LC_ALL) but at least it *completes*
[10:18] <vila> mgz: which is *huge* from my POV :)
[10:18] <spiv> vila: the standard library has many deficiencies
[10:18] <spiv> vila: this is just one of them :)
[10:19] <hrw> I have been trying to use git-bzr-ng plugin to get two way git<>bzr bridge so I will not have to learn bzr and can use git for it. clone works but push works only if there were no changes done. push with change ends with http://pastebin.com/XA8LJgVv reported.
[10:19] <hrw> can someone look at pastebin and comment?
[10:23] <hrw> it works when changes are pushed to new bzr branch but not when pushed to existing one
[10:25] <vila> hrw: by git-bzr-ng you mean bzr-git ?
[10:25] <hrw> no
[10:26] <vila> hrw: and you seem to be using bzr-fast-import...
[10:26] <hrw> http://github.com/termie/git-bzr-ng/
[10:26] <hrw> git-bzr-ng uses bzr-fast-import in backend
[10:26] <hrw> bzr-git is to get git repo in bzr. git-bzr-ng is to get bzr repo in git
[10:27] <vila> hrw: why don't you try to use bzr-git instead ? I never heard about git-bzr-ng so I'm not sure you get a lot of answers in *this* channel...
[10:27] <hrw> vila: http://marcin.juszkiewicz.com.pl/2010/10/06/bazaar-what-is-wrong-with-it/ summaries my feelings about bzr
[10:28] <hrw> vila: in short: I do not like bzr.
[10:28] <vila> hrw: I'm not discussing whether you like bzr or not, just trying to point you to the right place to get the right answers
[10:28] <hrw> sure
[10:29] <vila> but coming here saying that don't like what we do doesn't sound like the best start :) Welcome anyway
[10:31] <hrw> I know - just wanted to ask for help to understand where from error comes.
[10:33] <vila> the traceback tells it's from fast-import, try adding BZR_PDB=1 in front or your command and poke at parts and self_git_to_bzr_name for a start
[10:33] <hrw> thx, will check in few minutes
[10:34] <mgz> vila: to be clear, mac osx needs hack_get_transport to complete, or have you just not tried since your previous fixes?
[10:35] <vila> mgz: I tried with my fixes, the hack is needed, so the remaining leaks were caused by calls to the original get_transport, defeating the override
[10:36] <mgz> ha.
[10:36] <vila> mgz: I ran with the hack and a live open-files windows and I never seen such a clean run :)
[10:36] <mgz> yeah, we're getting there!
[10:37] <mgz> a bit more chipping away and we'll have a clean suite on all the platforms we try and support
[10:37] <vila> your hack ensures get_transport is properly overridden so it validates all the fixes, I just wished I thought about it myself ;)
[10:38] <vila> the bit I don't get though, is why this made such a huge difference between windows/OSX on one side and all the other on the other side...
[10:39] <mgz> I'm just answering poolie's question on the list, my understanding is it's how the different socket libraries work
[10:39] <vila> and no cheap joke about proprietary softwares leaking resources :)
[10:40] <vila> mgz: oooh, let's try it on FreeBSD
[10:40] <mgz> and naturally that the tests are written by people on nix, so when there are problems there it's fixed before it ever lands
[10:41] <vila> in the *socket* library...
[10:41] <mgz> windows at least has diverged rather from its start, not sure what osx uses.
[10:42] <mgz> but still worth seeing if we get a speed up on freebsd
[10:43] <txdv> awesume
[10:44] <txdv> git-bzr-ng - ill have to check it out
[10:44] <vila> mgz: yup, that's the idea
[10:45] <mgz> I want to know what the ng is for. Does it do time travel into the victorian age as well?
[10:46] <txdv> there is already some git-bzr hack, so he wanted to make it sound new and cool
[10:46] <vila> hrw: Your post is not very specific about what is missing in bzr, filing bugs at https://launchpad.net/bzr/+filebug would get you more attention and better hope that we address your needs
[10:46] <hrw> ok
[10:47] <txdv> it's missing a proper remote managing interface
[10:47] <mgz> the stuff you said in here the other day was actually more enlightening
[10:48] <txdv> those half backed branches which are tied to remote branches are buggy as shit
[10:48] <mgz> you were trying to adapt your git-y workflow to bzr and struggling
[10:49] <txdv> i can adapt to new bzr'ish workflows
[10:49] <txdv> no problem
[10:49] <mgz> that was aimed at hrw, but either way.
[10:50] <mgz> hacking up a giant change then splitting it into little bits to merge on the current trunk isn't something bzr supports particularly well, but there are ways
[10:57] <Glenjamin> hrw: re: your blog post - the first link on related is a similarly exasperated post about git from 3 years ago (which i presume is when you started using it)
[10:58] <hrw> Glenjamin: yes, related posts are often crazy
[10:58] <Glenjamin> my point being, maybe you'll get used to it :)
[10:58] <hrw> who knows.. maybe in 2014 I will use bzr more often
[11:00] <mgz> heh, good point Glenjamin. I do "graaa, new thing I don't want to adapt to" all the time.
[11:02] <Glenjamin> for instance, I wouldn't generally try and re-organise a patch before trying to get it upstream, i'd just supply a mergable branch
[11:02] <Glenjamin> but presumably cleaning up a patch like this is common in git
[11:02] <mgz> yeah, it's standard workflow for a bunch of projects.
[11:03] <Glenjamin> my assumption is the reviewer would look at the final diff
[11:11] <vila> mgz: 22min on FreeBSD, nothing spectacular nor unseen
[11:12] <vila> unless
[11:12] <vila> no, parallel=fork in both cases
[11:19] <DaffyDuck_`> Hello
[11:19] <mgz> the slowness there does seem to be a little all over from disk access rather than lots in smart server tests, so shouldn't be suprised
[11:19] <DaffyDuck_`> I had some breakage in a repository earlier today, so I thought I'd take the opportunity to upgrade bzr to 2.1.2 (on Windows).
[11:20] <DaffyDuck_`> With the previous (2.0.0) version I got a bazaar directory with a bzr.exe file, but with this new version, I have not bzr.exe.
[11:20] <DaffyDuck_`> ..and not bzr.py along the path.
[11:20] <DaffyDuck_`> Do I need to do some post-installation set up to get a command line "bzr" in Windows?
[11:21] <mgz> okay, #1 did you use the right installer? and #2 were there any error messages during the install process?
[11:21] <DaffyDuck_`> No error -- and I'm certain I used the correct installer.
[11:22] <DaffyDuck_`> Oh, just found it. I think. There's no dedicated bazaar directory any longer, but it has placed a bzr.bat under c:\python26\scripts.
[11:22] <DaffyDuck_`> I assume that's what I'm supposed to use instead.
[11:22] <mgz> called -setup.exe rather than .win32-py2.6.exe right?
[11:22] <mgz> okay, so you did use a different installer.
[11:23] <DaffyDuck_`> Hmm.. I wasn't aware there was more than one. :)
[11:23] <mgz> that's fine (and often actually preferable if you have python installed anyway), but you do need to add the scripts dir to your path manually for some reason
[11:23] <mgz> the actual python installer should do it for you really.
[11:24] <DaffyDuck_`> So the *-setup.exe contains a full python installation too?
[11:24] <mgz> it does.
[11:24] <mgz> and various bundled plugins as well.
[11:24] <DaffyDuck_`> Hm.. Seems overkill since I already had it. Well, that's one mystery less.
[11:25] <DaffyDuck_`> Well, time to get back to work. Thanks for the clarification.
[11:27] <vila> meh, I meant 2.2.1 not 2.1.2 right ?
[11:28] <mgz> you typoing release notices still vila? :)
[11:28] <vila> errr, *he* meant, DaffyDuck_
[11:30] <mgz> nope, I think they really meant 2.1.2 as was upgrading from 2.0
[11:30] <mgz> and had done it long enough ago not to remember the installer looked different last time.
[11:31] <vila> shudder
[11:32] <mgz> okay, so about this hook.
[11:32] <mgz> we want something that runs just before an actual connection is established?
[11:33] <vila> or just after ?
[11:33] <mgz> or a generic transport setup thingy?
[11:33] <vila> no, I think at connection, so that it gets called if we re-connect too
[11:34] <vila> both pre and post can be useful, but I think we want post first
[11:35] <mgz> currently I see there's no one connect method like the disconnect you added, instead it's all connect_something
[11:37] <vila> right, unifying this may come later, since we're talking about connection, the hook may be specific to ConnectedTransport, but generally adding stuff there is not worth avoiding adding it as unimplemented as the Transport level
[11:38] <vila> I mean, define the hook at the Transport level even if it's called only by ConnectedTransport
[11:38] <vila> And ConnectedTransport defines _set_connection that should be used by all relevant classes as part of their connect_something
[11:42] <mgz> hm, yeah, after the connection has been established
[11:42] <vila> yes, post_connect
[12:01] <vila> spiv: ......and lp finally made your maverick package available, I installed it on the babune slave which turned blue for the first time. Pfew, just in time ;)
[12:48] <sender> bzr-svn is giving me problems since a day, bzr pull https+urllib://svn.uitburo.nl/Website/trunk barfs at me: http://pastebin.com/PZUuaHyU
[12:49] <sender> Bazaar (bzr) 2.2.1 / svn 1.0.4
[12:50] <sender> Ideas are very welcome
[12:50] <sender> :)
[13:08] <maxb> hmm
[13:08]  * maxb tries
[13:09] <maxb> oh, private repo
[13:09]  * maxb shrugs
[13:09] <maxb> sender: Your subvertpy version is what?
[13:10] <maxb> (dpkg -l python-subvertpy)
[13:11] <sender> maxb: thanks! 0.7.3-1ubuntu0~bazaar1~lucid1
[13:11] <maxb> hmm. That's recent
[13:12] <sender> It broke yesterday 'all of a sudden', I was using svn 1.0.2, and upgraded.
[13:12] <maxb> In that case, there's a fair chance you've found a bug somewhere
[13:12] <maxb> Is this repository proprietary/confidential?
[13:28] <sender> maxb: well, yes. nothing changed on the repository side...
[13:28] <maxb> nothing, other than new revisions?
[13:32] <sender> maxb: that's right
[13:43] <jelmer> sender: can you try a clone into a fresh repository ?
[13:44] <sender> jelmer: so you mean locally, using bzr branch? or using svn?
[13:50] <jelmer> sender: using bzr branch, but cloning into a fresh repository, not reusing your existing local branch.
[13:53] <sender> jelmer: ok, understood. Trying now
[14:02] <sender> jelmer: taking a while...
[14:02] <sender> jelmer: do you think the local copy has corrupted in a way?
[14:02] <sender> jelmer: bzr check dies with an error aswell..
[14:06] <txdv> muahahahaha die bzr die
[14:07] <sender> txdv: *grin*
[14:07] <jelmer> sender: yes, older versions of bzr-svn had a bug
[14:07] <sender> jelmer: too bad, error (dies) too
[14:07] <jelmer> sender: same error?
[14:09] <sender> jelmer: http://pastebin.com/nNebXdHE
[14:10] <jelmer> sender: hmm
[14:10] <jelmer> sender: can you try again but remove the cache for your svn repository first ? It should be in ~/.cache/bazaar/svn
[14:11] <sender> jelmer: try the branch again?
[14:11] <sender> jelmer:  rm ~/.cache/bazaar/svn/* -Rf ?
[14:12] <sender> jelmer: ok, here goes again
[14:12] <sender> jelmer: thanks for helping me out :)
[14:24] <sender> jelmer: darn, it still crashes after a long time
[14:25] <sender> jelmer: it even throws an ubuntu report the problem thingy, never seen that before. Oh and it complains that the problem cant be reported (typical...).
[14:26] <sender> arg! how am I going to work with this repo again :S
[14:26] <jelmer> sender: what error do you get this time?
[14:27] <jelmer> still the same?
[14:27] <sender> jelmer: what do you exactly want to know? bzr: ERROR: exceptions.AssertionError: 408 != 619
[14:34] <jelmer> sender: so, the last resort is to try lp:bzr-svn and do the fresh clone again (with cache removed). If you can still reproduce it with that, plesae file a bug and I'll have a look at it tonight.
[14:35] <sender> jelmer: ok, thanks. So I go: sander@ubuntu:~/.bazaar/plugins$ bzr branch lp:bzr-svn
[14:35] <sender> Do I need to remove the package bzr-svn?
[14:35] <sender> What version am I pulling in right now? So I can check with bzr plugins?
[14:36] <sender> jelmer: are you Dutch :)
[14:36] <jelmer> sender: bzr branch lp:bzr-svn ~/.bazaar/plugins/svn
[14:36] <jelmer> sender: there's no need to remove the package
[14:37] <jelmer> sender: jup, ik ben ook Nederlands :-)
[14:37] <sender> jelmer: great, now bzr plugins gives me: svn 1.0.5dev :)
[14:37] <sender> jelmer: locale hulp, fantastisch, zeer bedankt!
[14:37] <sender> jelmer: dan zeggen die url's in pastebin je iets meer :)
[14:39] <sender> jelmer: ok here goes again: bzr branch https+urllib://svn.uitburo.nl/Website/trunk uit-website
[14:56] <sender> jelmer: unfortunately, an error again: bzr: ERROR: exceptions.AssertionError: 408 != 619
[14:56] <sender> jelmer: :(
[14:56] <Glenjamin> hrm, what's 619 in base 8?
[14:59] <jelmer> sender: :-(
[14:59] <jelmer> sender: can you try the whole process again after applying http://samba.org/~jelmer/fileids.diff ?
[15:00] <sender> sander@ubuntu:~/.bazaar/plugins/svn$ patch -p0 < fileids.diff \ patching file revmeta.py
[15:01] <sender> jelmer: ~/.bazaar/subversion.conf of any influence?
[15:02] <jelmer> sender: no, that shouldn't be relevant - the cache is, though
[15:03] <sender> jelmer: ok, applied patch, cleared cache, branching again.
[15:30] <sender> jelmer: too bad. bzr: ERROR: exceptions.AssertionError: 408 != 619
[15:30] <sender> jelmer: I don't know what could have changed..
[15:32] <sender> jelmer: can I give you more information to work with in any way?
[15:38] <jelmer> sender: can you at least file a bug about it on launchpad? It's clearly a bug that's still present in current bzr-svn.
[15:38] <jelmer> sender: it would be interesting to have a look at the "svn log --xml --with-all-revprops -v" output of the revision that triggers this issue
[15:38] <sender> jelmer: hope it's not a problem caused by corruption on the repository side... svn checkout seems to work fine
[15:39] <sender> jelmer: how do I proceed to find the revision?
[15:39] <jelmer> sender: I doubt it's a problem on the svn side
[15:39] <jelmer> sender: if you run the command again but set BZR_PDB=1 in the environment it should put you into a python debugger
[15:51] <sender> jelmer: I'm waiting for the svn checkout to finish, it's taking extremely long.
[15:55] <sender> jelmer: I might as well start the branch while it's still running. So: export BZR_PDB=1; bzr branch https+url://svn.uitburo.nl/Website/trunk uitwebsite
[15:57] <sender> jelmer: und jetzt?
[15:57] <jelmer> sender: yep
[15:58] <jelmer> sender: how much python experience do you have?
[15:58] <sender> jelmer: virtually zero :|
[15:59] <lvh> Hello :-)
[15:59] <jelmer> hi lvh
[15:59] <lvh> Can someon reccomend a code review tool for use with bzr?
[15:59] <lvh> So far I've come across lp merge requests, which are super awesome.
[15:59] <lvh> (I'm just scoping out what's available.)
[16:00] <lvh> I've thought about doing code review as commits, since that makes it easy to programmatically verify as well
[16:01] <lvh> (ie, stuff gets proposed for review, I add # XXX: No, fix this in the code)
[16:02] <lvh> the only thing I can think of that would make merge proposals better is a side-by-side view like bzr qdiff or rietveld, but I suppose this is not the right place to suggest that
[16:10] <jelmer> lvh: I don't have experience with the integration in other tools, but I know there's been work done to support bzr in rietveld and reviewboard
[16:11] <lvh> jelmer: Yeah, bzr is supported in rietveld
[17:21] <sender> jelmer: output is getting throttled
[17:22] <sender> jelmer: zie je de output?
[22:00] <poolie> hi all
[22:35] <bzrnewb> hello. i'm planning on using bazaar for a single-user project but I have some questions about how it actually works. Anyone able to enlighten me?
[22:35] <GaryvdM> bzrnewb: sure.
[22:36] <GaryvdM> bzrnewb: Please don't ask to ask, just ask :-)
[22:36] <GaryvdM> Hi poolie
[22:37] <bzrnewb> cheers. I want to edit and maybe create my own scripts in Unrealscript for UDK. Everything is under a folder named src in UDK main folder and if I move it somewhere else the engine won't read them.
[22:37] <poolie> hi gary
[22:37] <bzrnewb> so if I set up a repository somewhere else how does it work with these files? are the files stored in repository or remain at the original folder but changes,  etc go to repositroy?
[22:38] <peitschie> mornin all :)
[22:38] <bzrnewb> repository*
[22:38] <poolie> bzrnewb: (nice nick) there are basically two directories you need to know about
[22:38] <poolie> there's the repository, which holds a bunch of pack and index files etc, which stores the history of the project
[22:39] <poolie> and then there's the working tree, which holds the actual checked out copies that you can edit, run, etc
[22:39] <poolie> in your case you just need the working tree to be under UDK
[22:39] <poolie> perhaps the easiest thing is to just do 'bzr init' in that directory then the repo will be there too
[22:39] <GaryvdM> bzrnewb: I would just "bzr init" in that dir. That will create the working tree and repo there
[22:40] <poolie> then if you want to back it up, you can 'bzr push' from there to somewhere else
[22:41] <bzrnewb> hmm can I select the entire UDK as the worktree and ignore the files/folders I won't need so I can only work with src files I want to edit.
[22:41] <bzrnewb> but also include assets etc.
[22:41] <poolie> yes
[22:42] <GaryvdM> bzrnewb: Yes, with bzr ignore
[22:42] <GaryvdM> or just add the files you do want.
[22:43] <poolie> if you want to ignore most of it and just version a few files, i'd do
[22:43] <bzrnewb> awesome. When I make a change in code does it store the original file somewhere else or does it just save a change history which I can do rollback. Just trying to understand if it will create loads of files for each change and pollute the UDK folder
[22:43] <poolie> 'bzr ignore *'
[22:43] <poolie> (or the equivalent in the gui) then add the rest
[22:43] <bzrnewb> yeah I'll probably stick to gui for now :)
[22:44] <poolie> it stores it into the repo dir, which in your case will be .bzr/repository/....
[22:44] <poolie> in a database
[22:44] <bzrnewb> I tried svn before bazaar and it was a bit complicated which is why I'm trying bazaar, but bazaar sounds a lot easier for single-user use.
[22:45] <mwhudson> i would certainly agree with that
[22:45] <bzrnewb> the whole server thing was overkill for what I'm intenting to do
[22:46] <bzrnewb> does bazaar only support code comparison or can it preview/compare doc, excel and image files?
[22:46] <bzrnewb> like perforce
[22:47] <bzrnewb> or alienbrain
[22:47] <GaryvdM> bzrnewb: bzr qdiff does compare images side by side
[22:49] <bzrnewb> sounds like it's perfect for me. I'll set it up and see how it goes. Thanks for clearing things up for me
[22:49] <GaryvdM> bzrnewb: You can use any diff tool that accepts 2 files with bzr diff --using alienbrain-diff
[22:50] <bzrnewb> cool. I might come back if I run into any problems :) Cheers guys
[22:56] <jam1> poolie: I'm heading out now, but I figured I would at least /wave at you
[22:56] <poolie> hi jam1, thanks!
[22:56] <poolie> have a good weekend
[23:00] <jam> poolie: well, only Thurs here, but thanks for the day off :)
[23:00] <jam> (yes, I plan on working tomorrow)
[23:00] <poolie> i probably won't see you though :)
[23:00] <poolie> so i'm just queueing up the good wishes
[23:01] <poolie> i'm going for a ride tomorrow
[23:02] <mwhudson> jam: hey, btw, can you run 'make lint' in lp-service?
[23:02] <jam> mwhudson: I can before I push again, but not tonight.
[23:02] <mwhudson> there's a bunch of irrelevant stuff, but the unused imports can be fixed at least
[23:02] <mwhudson> jam: fair enough
[23:03] <jam> (spent a fair amount of time getting pyflakes-vim working, since it seems to require a new python which requires a new vim, ...)
[23:04] <mwhudson> argh
[23:05] <jam> mwhudson: I'll probably be back around in another hour or so, would it be possible to work with you a bit and tie up any loose ends here?
[23:05]  * jam really leaves for now
[23:05] <mwhudson> jam: should be, give me a ping
[23:09] <GaryvdM> mwhudson: Is there a way to get pydoctor to syntax a bit of example code? The rst docs suggest .. python::, but that does not work in pydoctor
[23:10] <GaryvdM> *syntax highlight
[23:10] <mwhudson> GaryvdM: uh, don't know off hand
[23:10] <mwhudson> GaryvdM: you are using --docformat restructuredtext i assume?
[23:11] <GaryvdM> mwhudson: Yes
[23:11] <mwhudson> GaryvdM: and what does 'does not work' mean, doesn't highlight or errors?
[23:12] <GaryvdM> mwhudson:  I assume that  --docformat restructuredtext is what is used for http://people.canonical.com/~mwh/bzrlibapi/bzrlib.html
[23:12] <mwhudson> GaryvdM: it is
[23:12] <GaryvdM> mwhudson: Does not highlight
[23:14] <GaryvdM> mwhudson: Ok, don't worry. I'll dig a bit myself.
[23:14] <mwhudson> GaryvdM: if it's doing the highlighting with css it might be generating the appropriate output but pydoctor isn't including the necessary css?
[23:14] <mwhudson> GaryvdM: view source should make that pretty clear
[23:14]  * GaryvdM checks
[23:16] <GaryvdM> mwhudson: nope.
[23:16] <mwhudson> hm
[23:16] <GaryvdM> mwhudson: I think that .. python:: is maybe a feature of spinx
[23:17] <mwhudson> ah, that could be
[23:17]  * GaryvdM tries to find where me read that
[23:17] <mwhudson> but i'd really expect it to blow up in that case
[23:24] <GaryvdM> mwhudson: Oh - It was in the eypdoc docs: http://epydoc.sourceforge.net/manual-othermarkup.html#colorized-snippets-directive
[23:25] <mwhudson> GaryvdM: well pydoctor uses the epydoc routines to do the formatting, so i've no idea why that wouldn't work
[23:25]  * GaryvdM tests if it works in eypdocs
[23:35] <GaryvdM> mwhudson: Ah - It is working, but it dose not highlight much, and css missing. I try submit a patch for the css.
[23:36] <mwhudson> GaryvdM: ah