[00:00] <poolie> hi all
[00:00] <poolie> our wiki software is going to be upgraded today, there may be glitches
[00:04] <jelmer> hi poolie
[01:09] <Noldorin_> hi jelmer
[01:09] <Noldorin_> you still around?
[01:09] <Noldorin_> i've done a bit more research...
[01:11] <Noldorin_> jelmer, the problem is that branching and uncomitting back to r46 makes the branches diverge...
[01:11] <Noldorin_> (no common ancestor, it days)
[01:11] <Noldorin_> so i can't merge in :-S
[01:11] <Noldorin_> from r47
[01:11] <Noldorin_> how eird
[02:41] <cody-somerville> Is there a function I can call that'll basically parse what I might pass to bzr at the command line and transform it into whatever Branch.open will be looking for?
[02:42] <lifeless> directory service lookup is the difference there I think
[02:47] <cody-somerville> lifeless, How do I use that? I assume there must be some function I need to call to get things initialized and get a DirectoryServiceRegistry object?
[02:48] <lifeless> there is probably a module level instance you can use
[02:48] <lifeless> you'll need a library context as usual
[02:53] <cody-somerville> Figured it out.
[02:53] <cody-somerville> Branch.open and stuff automatically do directory service lookup
[02:53] <cody-somerville> I had to call bzrlib.initialize() and bzrlib.plugin.load_plugins() first though
[03:09] <lifeless> yes :)
[04:28] <poolie> wgrant, lifeless, i was going to say, people are getting things done at a good rate but it may feel a bit like one thing after another
[04:28] <poolie> without them so much adding up
[04:28] <poolie> john commented on something like this
[04:28] <poolie> one idea is to borrow the rotation idea from lp, but for the whole team:
[04:29] <poolie> for say 2 weeks all focus on one area, eg bfbia or lp bzr-git integration, or whatever
[04:29] <poolie> probably for as long as it takes to get that area adequately complete
[04:29] <wgrant> I think that's a good strategy, particularly for something like bzr.
[04:29] <poolie> interrupted only by actually critical bugs
[04:29] <wgrant> As long as the thing is of reasonable scope.
[04:30] <poolie> (in the bzr definition, meaning data loss or total use blockage or whatever)
[04:30] <poolie> and then do say 2 weeks of reactive work on general bugs
[04:30] <mwhudson> the tricky part of this, from experience and observation is getting to the point where you can stop
[04:30] <poolie> we would still continue with piloting but people would have less discretion to pick their day-to-day work
[04:30] <poolie> yes
[04:30] <lifeless> whats your average cycle time ?
[04:31] <mwhudson> james_w can tell you lots about this too :)
[04:31] <poolie> the feature you pick could easily turn out to be 3 days or 3 months instead
[04:31] <poolie> i don't know, where does lp show that? :-7
[04:31] <poolie> but, the mean is probably about a week
[04:31] <lifeless> lpkanban should be emitting that :)
[04:32] <lifeless> so, that means 2 complete patches end to end for each person (on a 2 week cycle), and if you're doing concurrent work, whatever your concurrency is in WIP when you come to change.
[04:32] <lifeless> do you have a few ideas you want to run up the flag pole, or just this particular one ?
[04:33] <poolie> i don't think it follows it would be 2 patches per person, because i'm including stalls there, which this could reduce
[04:33] <poolie> for example its skewed by 660264 being something like 18 months old
[04:33] <lifeless> k
[04:33] <poolie> this is my central idea
[04:34] <poolie> i think the 'typo fix' time is somewhere around 24h or less
[04:34] <poolie> so, a fairly sized feature could get done in 2w
[04:34] <poolie> there are some variations
[04:34] <poolie> 0- change nothing
[04:34] <poolie> 1- put one person (or two) on bugs, and have others ignore bugs and work on a feature
[04:35] <poolie> 2- anyone can do either bugs or features on any day, but there are fewer features in play to choose from
[04:35] <lifeless> how much does a [human] context switch cost in bzr?
[04:36] <poolie> 3- just generally shift the bar of which bugs ought to come ahead of feature type work
[04:36] <poolie> (obviously i'm using the word 'feature' here to mean proactive work rather than reactive, it may have a bug number)
[04:36] <poolie> lifeless, as in, switching from one bug to another, or one area of the code to another?
[04:37] <poolie> hm it seems a bit hard to generalize
[04:37] <poolie> the harder issues might need a bit of mental state
[04:37] <lifeless> poolie: bit of A, bit of B
[04:37] <lifeless> like, if you said 'lets spend 2 weeks on looms'
[04:38] <lifeless> how long before things started flowing?
[04:38] <poolie> i think generally speaking if I did an hpss bug today and a udd bug tomorrow it would not hurt
[04:38] <poolie> great example; i think the costs are going to be
[04:38] <poolie> - i already have these other mps underway and i need to finish them off (kind of queue-draining)
[04:39] <poolie> - deciding what in particular we're going to do
[04:39] <poolie> - to some extent, getting familiar with the code, though .. perhaps that's not enormous, and having a specific impetus to get into eg the loom code could be beneficial
[04:40] <poolie> perhaps the biggest cost of timeslicing is going to be queue draining/filling
[04:40] <lifeless> a few random thoughts
[04:40] <lifeless> having > 1 person on a task is enjoyable
[04:41] <lifeless> jelmer: bzr-search trunk is now owned by ~bzr
[04:41] <lifeless> jelmer: I've left a branch reference behind for convenience
[04:41] <poolie> (yes, that's one thing i'd like to have happen more often)
[04:42] <lifeless> waste [in the lean sense: time in the production of something that isn't actively moving a product through to the customer] can make things really drag out
[04:42] <lifeless> and for me at least that draggy feeling can numb the sense of variety that one would otherwise get
[04:43] <lifeless> e.g. exec rather than fork on lp :)
[04:43] <poolie> yes, that's very true
[04:44] <lifeless> so, you can either switch while you have stuff to drain, or you can switch 'new stuff' but continue to drain.
[04:44] <lifeless> something that makes these behaviours closer is reducing the time to drain/time to fill figures
[04:45] <poolie> mm
[04:45] <lifeless> I suspect you're already pretty vigorous on that
[04:45] <poolie> i think our average time from an mp to getting it merged is moderately low now
[04:45] <lifeless> but its worth having in mind when considering new scheduling approaches
[04:45] <poolie> i think having more people on one project tends to reduce it more, because there are informed reviewers
[04:46] <lifeless> e.g. doing something that will increase the time-in-queue (like less piloting) may have a negative impact
[04:46] <poolie> yes, very much
[04:46] <poolie> i wouldn't cut back on piloting for this
[04:46] <poolie> this is all within the slice of non-piloting work
[04:46] <lifeless> sure, I'm just thinking through it all
[04:46] <lifeless> releases will interact with this.
[04:47] <lifeless> unless you can automate them totally out of existence.
[04:47] <poolie> yep, they too are a kind of 'hygene' task
[04:48] <poolie> as are srus (kind of related)
[04:48] <poolie> and answering user questions
[04:48] <lifeless> yah
[04:48] <poolie> very likely we can or should automate them more
[04:50] <lifeless> I don't know if whole-team rotations make sense, I suspect you may pay in context switches
[04:50] <lifeless> OTOH if everyone is looking at the same stuff you should get some network effect.
[04:50] <lifeless> my 2c: give it a go for 3 cycles.
[04:50] <lifeless> measure it,
[04:50] <lifeless> return to what you have now
[04:50] <lifeless> measure that.
[04:50] <lifeless> compare.
[04:50] <lifeless> (return because otherwise the act of changing may be a placebo)
[04:51] <poolie> you say placebo like it's a bad thing :)
[04:51] <lifeless> it is if you want things to -stay- better ;_
[04:57] <poolie> wgrant, any other thoughts?
[04:58] <wgrant> Mmm.
[04:58] <wgrant> It's hard to say how well it would work for bzr. I think you probably have less interrupts, so it would work well.
[04:59] <wgrant> And, as you say, it would be nice to have multiple people working together on one thing.
[06:46] <vila> hi all !
[07:48] <poolie> hi vila
[07:48] <vila> poolie: hey !
[07:49] <poolie> hi there
[07:50] <vila> warming up for the standup ?
[07:50] <poolie> yeah
[07:50] <poolie> apparently it was still owned by spiv so i was just trying to fix that
[07:50] <poolie> he's still pretty much welcome to come but i doubt he wants calendar remindres
[07:51] <vila> :)
[07:57] <babune> Riddell: thanks :)
[07:58] <poolie> for a second there i thought you set up a bot
[07:58] <vila> hehe, not yet...
[07:58] <vila> acting bot only so far ;)
[08:02] <poolie> jam, jelmer, hi?
[08:05] <poolie> o/ mrevell
[08:07] <mrevell> Hi!
[08:37] <vila> lp is down ? At least the high failure rate on the package importer suggests so
[08:38] <vila> requeued
[08:44] <poolie> not as far as i know
[08:46] <vila> ~600 spurious failures
[09:19] <vila> poolie, jam, jelmer, Riddell : Did one of you want to post the standup summary or should I ?
[09:20] <jam> vila: Fine with me if you want to do it.
[09:21] <vila> poolie, jam, jelmer, Riddell : I'll do it then, still 5 secs to shout otherwise 4... 3
[09:21] <Riddell> go ahead
[09:25] <vila> done
[09:31] <poolie> jelmer, what's the tag for bfbia bugs?
[09:32] <poolie> i know i've seen them but the link from https://dev.launchpad.net/LEP/BuildFromBranchIntoArchive has nothing
[09:32] <jam> jelmer: maybe that just means the code is perfect? (*crosses fingers*)
[09:35] <poolie> jam, https://code.launchpad.net/~jelmer/launchpad/bfbia-db/+merge/68990
[09:37] <poolie> jelmer, what's actually up with that? it looks like it's mostly merged?
[09:44] <jelmer> poolie: bfbia is the relevant tag
[09:44] <poolie> so there are no bugs?
[09:44] <jam> jelmer: k, there just doesn't seem to be any matches
[09:44] <jelmer> poolie: though I think most of the bugs that have been closed were actually in bzr-builder/bzr-builddeb
[09:44] <poolie> just the db proposal?
[09:44] <poolie> oh ok
[09:44] <poolie> no bugs in lp
[09:44] <jelmer> poolie: there's one, but it lacks the tag :-)
[09:44] <poolie> nice :)
[09:44] <jelmer> bug 313602
[09:45] <jelmer> the relevant changes for bfbia are that db branch, which is trivial. I haven't looked at landing because of lp's changing of its deployment process.
[09:46] <jelmer> and there's the proof of concept branch at https://code.launchpad.net/~jelmer/launchpad/bfbia/+merge/64036 which works but had too many database changes. I've been working on getting it to use the existing recipe-based infrastructure instead.
[09:49] <jelmer> my guess is that it's fairly close now that the other Launchpad related stuff has landed
[09:51] <poolie> jam, https://bugs.launchpad.net/udd/+bug/724890
[09:58] <jam> poolie: https://bugs.launchpad.net/bugs/819604 ?
[10:02] <jelmer> Riddell: hi
[10:02] <jelmer> Riddell: just followed up to your i18n patch
[10:03] <jelmer> Riddell: I'm also wondering more generally about i18n what the best way is to translate plugins.
[10:09] <Riddell> I haven't really thought about that so far
[10:09] <Riddell> but they should load a separate gettext domain with their translations
[10:14] <poolie> could someone look into https://bugs.launchpad.net/bzr/+bug/848064 ?
[10:15] <poolie> if it is similar to the earlier bugs max dealt with, it should be able to be fixed by clearing and requeuing the imports
[10:15] <poolie> or something like that
[10:19] <poolie> night all
[10:23] <poolie> jelmer, so does that small db patch still make sense to merge?
[10:23] <poolie> just +ALTER TABLE SourcePackageRecipeData ALTER COLUMN deb_version_template DROP NOT NULL;
[10:23] <poolie> if so, can i remind you to push for a review, or for clarification on how/where/when to merge it?
[10:27] <jelmer> poolie: I think it still makes sense, but I'd like to finish my other branch first to be sure.
[10:29] <jelmer> Riddell: it would be nice to have some convenience functions in bzrlib.i18n for that
[10:29] <Riddell> jelmer: for plugins?
[10:29] <jelmer> Riddell: yeah
[10:30] <Riddell> yes I agree, I'd like to do that
[10:58] <Riddell> vila: at what stage in the release process do we make a new stable branch?
[10:59] <vila> Riddell: I'd prefer an easier question before my lunch
[10:59] <vila> :)
[10:59] <vila> Riddell: you mean on pqm ?
[10:59] <Riddell> vila: I mean when will lp:bzr/2.5  be separate from lp:bzr
[10:59] <vila> or rather you mean a branch associated with a stable series
[10:59] <vila> yeah
[11:00] <vila> so, that should be in releasing.txt, but from memory, the branch is created to release 2.x.0
[11:00] <vila> aka when we create the first stable release in a series
[11:01] <vila> it shouldn't be hard to do that for the *last* beta before the first stable if that makes it easier for translations
[11:01] <vila> we freeze the API after the last beta already
[11:02] <Riddell> ok, I'll write that up
[11:04] <vila> Riddell: search for 'Kick off the next cycle' in releasing.txt
[11:04]  * vila off for lunch
[11:59] <Riddell> vila: for your release manager eyes https://code.launchpad.net/~jr/bzr/i18n-release-process-doc/+merge/75161
[12:00] <vila> maxb, jam : Any issue blocking 2.4.1 ?
[12:00] <vila> jelmer: oops ^ :)
[12:04] <jam> vila: anything blocking me from building it? (just not having done it for me)
[12:05] <vila> jam: yeah, I mean, any reason to delay the announce
[12:05] <jam> well, windows installers aren't built yet...
[12:06] <vila> jam: discussed numerous times, unless critical bugs block building, availability should not block announce
[12:06] <vila> lack of availability for that matter
[12:06] <vila> :)
[12:09] <jelmer> vila: not afaik
[12:14] <vila> jelmer: IIUC you're first to shoot by releasing on debian right ?
[12:14] <vila> sid even
[12:14] <jelmer> not sure what you mean by first to shoot :)
[12:15] <vila> jelmer: once it's released on debian, the ubuntu releases can be done
[12:15] <jelmer> ah, yeah
[12:16] <jelmer> vila: I usually upload to sid and ubuntu at the same time
[12:16] <vila> ok
[12:16] <vila> jelmer: any ETA ? :D
[12:18] <vila> Riddell: reviewing, can we discuss it a bit ?
[12:18] <jelmer> vila: I'll have a look now, if the tarball is up
[12:19] <jelmer> vila: looks like the 2.4.1 tag is missing
[12:20] <vila> jelmer: it's up since last Thursday. Did my '[ANN] 2.4.1 has gone gold' mail got lost ? Ha crap
[12:20] <vila> it's here :-/
[12:21] <vila> jelmer: oh, may be you mean not on trunk ?
[12:21] <jelmer> vila: not on lp:bzr/2.4
[12:21] <jam> jelmer: bzr-2.4.1            6042.1.4
[12:21] <jam> looks like it is here
[12:22] <jam> jelmer: also in lp:bzr/2.4 (I just confirmed)
[12:22] <vila> pfew, thanks jam, I was starting to suspect pqm (I *did* sent 2 submissions with this tag (AFAIK) but just when we migrated)
[12:22] <vila> and there has been some weird branch shuffling during the migration
[12:23] <jelmer> sorry, my bad - we don't have the right magic in the 2.4 packaing branch yet
[12:23] <vila> np np
[12:24] <vila> (and I just pulled lp:bzr/2.4 and didn't get any updated tags message :-D)
[12:25] <vila> and 2.4 hasn't been merged on trunk yet so trunk doesn't have the bzr-2.4.1 tag, all is well :)
[12:35] <Riddell> vila: of course
[12:35] <vila> Riddell: reviewed locally, I was surprised by the 27	+The final beta - branching and translations
[12:35] <vila> Riddell: but it's correct
[12:36] <vila> Riddell: you used (AT) in emails, why ?
[12:36] <vila> Riddell: I mean, we didn't do this for the others, should we change ?
[12:36] <Riddell> there's enough special casing for the last beta that I think it needs its own section
[12:36] <vila> yeah, yeah, good idea, I agree
[12:36] <Riddell> vila: the e-mails were just copied off what dpm gave me, I can change them to @
[12:36] <vila> and enclose them in `` ``
[12:37] <vila> i.e. only tweaks, I'll vote
[12:38] <Riddell> updated
[12:39] <vila> Riddell: generally when we say tweak, it means, no need for a second review, go :)
[12:40] <Riddell> formidable
[12:42] <vila> :)
[12:49] <jam> vila: poke for great justice!
[12:49] <jam> I have a question about some interactions with threading and socket receiving
[12:50] <vila> jam: la la la , I can't hear you
[12:50] <jam> specifically, I'm trying to add a 'select()' call before we start reading the next data stream
[12:50] <vila> kidding :)
[12:50] <jam> so that I can timeout if a connection goes idle
[12:50] <jam> however, in doing so, some tests start getting "hung threads"
[12:50] <vila> yeah
[12:50] <jam> however, when I run the test it claims is a problem
[12:51] <jam> it passes in 0.1s
[12:51] <vila> don't mix timeouts and blocking calls
[12:51] <jam> vila: specifically, the socket is set to blocking, and I just select.select(... timeout)
[12:51] <jam> which generally seems to work
[12:51] <vila> yeah, generally
[12:51] <vila> but, just no
[12:51] <vila> it's in the docs, let me find it again
[12:53] <vila> http://docs.python.org/library/socket.html
[12:54] <vila> jam: re-reading as the wording seems different from what I remember and things may have changed
[12:54] <vila> ha, yeah, I think it's even more forbidden when using sockets *and* makefile() objects(which we do)
[12:55] <vila> ile objects returned by the makefile() method must only be used when the socket is in blocking mode;
[12:55] <vila> quteo: "File objects returned by the makefile() method must only be used when the socket is in blocking mode"
[12:55] <vila> quote
[12:55] <vila> so, don't do that
[12:56] <vila> most of the issues in the socket/thread leaks in selftest saga (part I) were caused by timeouts
[12:57] <vila> jam: IIRC, the bzr server itself cheats a bit by using a timeout but it does so on the listening socket where we don't use makefile() objects
[13:01] <vila> jam: ping ?
[13:01] <jam> vila: there is a difference from setting a timeout on the socket, and using select with a timeout
[13:01] <jam> I'm not setting a timeout on the socket
[13:01] <jam> (and, unfortunately, empathy still refuses to make noise when I get mentioned.. :(
[13:01] <vila> ha, ok (empathy)
[13:01] <jam> I'll probably switch to Pidgin soon
[13:01] <jam> It used to work
[13:02] <jam> But  I think it stopped when I installed natty
[13:02] <jam> I have to see the little blue icon in the corner.
[13:02] <vila> you may be right about select, but the symptom you describe reminded me of so many issues I encountered
[13:03] <vila> so, back tracking,
[13:03] <vila> what do you mean by "before we start reading the next data stream" ?
[13:04] <jam> ugh, still so much pain. If I run the test suite, I can see what test is failing, if I run *just* those tests, no failures
[13:04] <jam> vila: In SmartServerStreamMedium.serve() we loop over _build_protocol() _serve_one_request(protocol)
[13:05] <jam> so for every message that comes in from the client
[13:05] <jam> we read the first line, then dispatch the request and process it in 'blocking' mode
[13:05] <jam> I'm trying to add a "select(..., timeout)" just before reading the first line
[13:05] <jam> so if a client connects, chats with us a bit, and then never hangs up
[13:05] <jam> we'll hang up after a time
[13:06] <jam> as long as they *initiate* an RPC, we won't timeout on them (until that RPC has been fully responded to, and they haven't started a new one)
[13:07] <jam> funny enough, I can set the select timeout to 4.0s, which is shorter than the 'hung_thread' timeout of 5s.
[13:07] <jam> And then I get the exception in my code.
[13:07] <jam> Running a bunch of tests, seems to deterministically pick what test will fail
[13:07] <jam> running just that test, no failure
[13:07] <vila> jam: hmm, my gut feeling is that this won't mix up well with the rest (and the test failure kind of confirm that, but well, easy )
[13:08] <vila> Does the serve actually capture the connections or is it only the test server that does that for now ?
[13:08] <vila> s/serve/server/
[13:09] <vila> jam: and do you include errors in your select ?
[13:10] <vila> during tests we close the sockets with prejudice so if you don't include errors in your select, you may miss that
[13:10] <vila> alternatively, if the connections are captured you can update a stamp and let the main thread close/kill the idle connections
[13:11] <vila> whether you want to kill a connection taking too long to process a request or not may be worth considering too
[13:24] <jam> vila: currently I'm just doing select([r], [], []) however I get EBADF if the socket was closed on the server side.
[13:24] <jam> I don't do any special capturing, etc.
[13:24] <jam> vila: i don't *think* bzr serve --inet runs any threads
[13:24] <jam> I could be wrong
[13:26] <jam> (that particular case is also a PipeStreamMedium, which is not the same code path)
[13:27] <jam> plain 'bzr serve' creates a SmartTCPServer that dispatches to threads
[13:27] <jam> but "bzr serve --inet" just hooks up a SmartServerPipeStreamMedium to stdin/stdout
[13:27] <jam> so there isn't a master process monitoring the 'connections'
[13:53] <jam> further, it only fails during teardown parts of the test suite. Never in 'live' code.
[13:53] <jam> so i'm tempted to say it is something incomplete with out test infrastructure, not sure what yet
[13:55] <vila> I don't remember issues with SmartServerPipe, may be because I didn't work much with it
[13:56] <vila> jam: which tests are failing ?
[13:57] <jam> A random selection (different each run) from: ugh, still so much pain. If I run the test suite, I can see what test is failing
[13:57] <jam> sorry bad paste
[13:57] <jam> bzr selftest -s bt.per_branch.test_branch Remote -v
[13:58] <jam> Note, I did just find:         self.clients.pop()
[13:58] <jam>         self.clients.append((request, client_address, t))
[13:58] <jam> which doesn't actually check that the thing popped off is the same thing we are appending
[13:58] <jam> so there is at least the possibility that between 'get_request' and 'process_request' the order changes
[13:58] <jam> and we close the same object 2 times, but never close another object, etc.
[14:00] <vila> but why is that involved if you focus on SSPipe ?
[14:00] <jam> I'm not focussing on SSPipe
[14:00] <jam> since most of our test suite uses the other
[14:00] <jam> and I'd *like* them to act roughly the same
[14:01] <jam> *Launchpad* wants us to support the TCP stuff as well
[14:01] <jam> so that stale connections get reaped
[14:01] <jam> yes, we *could* do it in 2 different ways
[14:02] <jam> even weirder. sometimes you clearly get a 2s timeout, but the test still passes
[14:05] <jam> ugh. it seems to pass if I add a 'print' statement
[14:06] <jam> sys.stderr.write is enough
[14:07] <vila> timing issues are generally caused by a lack of synchro somewhere, addind timeouts in the mix.... shouldn't help
[14:07] <jam> I guess that gives the other thread a chance to say "oh hey, he must be shutting down"
[14:08] <vila> yeah, kind of adding sleep(abit)
[14:12] <jam> vila: oddly enough adding time.sleep(0.1) does *not* fix it. but print "data" *does*
[14:13] <blackarchon> hi guys :)
[14:13] <blackarchon> I don't see a windows build for bazaar 2.4.1 yet
[14:13] <jam> I have the feeling it is just the test suite failing to see the exception.
[14:14] <jam> blackarchon: it isn't built yet
[14:14] <blackarchon> is there an ETA?
[14:19] <jam> blackarchon: nothing concrete. I have it building now, but if there are any failures I probably won't have time to address them
[14:28] <blackarchon> jam: ok, thx :)
[14:44] <jam> blackarchon: looks like the build went fine. Uploaded
[14:52]  * Riddell nudges jelmer into reviewing https://code.launchpad.net/~jr/bzr/i18n-enable-translations-everywhere/+merge/75040 again before the end of the day
[14:52] <jelmer> Riddell: I'll have a look
[14:53] <jelmer> oh, actually
[14:53] <jelmer> jam, vila: do you have an opinion on whether installation of gettext should happen by bzrlib.initialize() or lazily in bzrlib.i18n.gettext()?
[14:54] <vila> both :)
[14:55] <vila> it sounds like something that should be done in bzrlib.initialize() but if the cost is too high to be done unconditionally, then it should be lazy
[14:56] <vila> the test suite itself doesn't really respect bzrlib.initialize though
[14:56] <jelmer> Riddell: is there a way to disable i18n explicitly?
[14:57] <jelmer> Riddell: I don't really want to bikeshed on this, as I'm not sure how important it is
[14:57] <Riddell> jelmer: currently unset LANGUAGE and the other envirnment variables
[14:58] <jelmer> Riddell: but it would be nice to give 3rd party users of bzrlib the ability to disable i18n
[14:59] <Riddell> a function in i18n.py to do that would be easy enough
[14:59] <jam> Riddell: you said it was "slightly less than 0.1s to slightly more than". Anything more quantifiable?
[14:59] <jam> Like 0.01s or 0.08s, etc?
[14:59] <jam> I do like the idea that if a command has nothing to say, it doesn't have to load the mappings
[15:00] <jam> I have the feeling that the mappings will get more expensive to load when
[15:00] <jam> a) they aren't just english
[15:00] <jam> b) we have more stuff that gets translated
[15:00] <jam> I don't know if gettext() loads all the mappings preemptively, or if it has an index and can only load the ones that are actually
[15:00] <jam> translated.
[15:01] <jam> I would hope the latter
[15:02] <jelmer> I think it has the latter because it has some kind of binary file that it loads rather than the plain text
[15:02] <blackarchon> jam: thank you, I will test it! :)
[15:13] <Riddell> jam: for example http://paste.kde.org/121057/
[15:21] <nigelb> Riddell: Interesting post re:OpenSuse :)
[15:30] <jelmer> vila: bzr-2.4.1-1 uploaded to debian, 2.4.1-1ubuntu1 now building..
[15:31] <vila> \o/
[15:35] <Riddell> jelmer: i18n.disable_i18n() added
[15:44] <jelmer> \o/
[15:47] <jelmer> Riddell: r=me
[15:47] <Riddell> pardon?
[15:48] <jelmer> sorry, Launchpad jargon
[15:48] <jelmer> Riddell: I've approved your merge proposal.
[15:48] <Riddell> awooga
[16:39] <Noldorin> hi again jelmer
[16:41] <jelmer> hey Noldorin
[16:58] <jam> vila: I think I've tracked it down. My guess is that we enter select.select([socket]), and then the TestCase teardown code does socket.close()
[16:59] <jam> If the teardown happened *first* we would get EBADF
[16:59] <jam> because it happens second, we don't see it until after the timeout.
[16:59] <jam> I changed it to a loop, and now I get socket.error(EBADF) often
[16:59] <jam> About 1 time in 500 or so, I get something that looks just like socket.error
[16:59] <jam> but doesn't catch "exception socket.error"
[16:59] <jam> but it gets caught by a generic "except Exception"
[17:00] <jam> Now I have something that gets EBADF about 1-in-3 tests, but never causes the test suite to hang or crash
[17:00] <vila> ...an socket.error that is not a socket.error... rings a bell, windows ?
[17:00] <jam> vila: this is on Ubuntu Natty
[17:00] <vila> no ssl involved ?
[17:01] <jam> It *sounds* like if someone re-imported socket, so you had 2 real modules involved
[17:01] <jam> vila: This is mostly loop-back stuff.
[17:01] <jam> It just str(e) says "error"
[17:01] <jam> well
[17:01] <vila> type(e)
[17:01] <jam> error(9, "bad file descriptor")
[17:01] <jam> I'll check, just a sec
[17:01] <vila> but first things first
[17:01] <vila> your enter the teardown while in select() ?
[17:02] <vila> you have a thread running and you don't join it ?
[17:02] <jam> vila: I think it is your "issue a connect but don't write anything" connection
[17:02] <jam> which starts a new "oh, someone wants to talk to me" thread
[17:02] <jam> I'm not positive yet
[17:03] <vila> oh, the hackish 'last connection to unblock the server in listen' ?
[17:04] <vila> and I'll stop asking more questions, 3 pending ones is already a lot...
[17:05] <jam> vila: Right, racing threads. If the select() thread happens first, it doesn't see EBADF, if it happens second, it does
[17:05] <jam> if it doesn't see EBADF, it waits until timeout
[17:05] <jam> before it fails
[17:05] <jam> with nothing to read from
[17:05] <jam> we *are* joining from it, but it fails in the mean-time with a timeout
[17:05] <jam> joining to it
[17:05] <jam> And maybe the hackish last connection
[17:06] <jam> maybe just a connection that is still open, but gets randomly closed by the test suite
[17:06] <jam> vila: Exception: <class 'socket.error'> socket [Errno 9] Bad file descriptor
[17:06] <jam> thats "type(e), getattr(e, '__module__'), e"
[17:06] <jam> so it is completely "socket.error", just not *this* socket.error
[17:07] <jam> If you get someone who, for example, reload(socket)
[17:07]  * vila puke
[17:07] <jam> vila: I don't think that is the case here. But I've seen cases where a module gets loaded twice.
[17:07] <jam> only times I know of are relative vs absolute imports
[17:08] <jam> but there should be only one "socket" module
[17:08] <jam> I suppose someone could do: "sys.modules.pop('socket'); import socket"
[17:08] <jam> or something like "during teardown we  "import socket; raise socket.error()"
[17:08] <jam> so socket got deleted by the garbage collector
[17:08] <jam> and just after that
[17:08] <jam> we re-import it creating a new socket module with all new classes.
[17:09] <jam> vila: I know we have other code that does "socket_error = socket.error" with a comment
[17:09] <jam> about socket.error not being around during teardown time
[17:09] <jam> oh, 1 sec, I just realized I was logging all errors, not just the ones that miss "socket.error". Trying again. It takes a while to trigger.
[17:10] <vila> only in the case of the threaded smart server for threads finishing after the main thread AIUI, but the explanation wasn't very clear
[17:10] <vila> when you say teardown, you're referring to the server teardown shutting down the client sockets ?
[17:11] <jam> vila: ah, not socket.error: select.error
[17:11] <jam> <class 'select.error'> select (9, 'Bad file descriptor')
[17:11] <jam> so 99/100 times it is a socket.error
[17:11] <jam> and less than 1:100 times it is a select.error
[17:12] <vila> haaa, makes more sense
[17:12] <vila> you may need to add select.error to the  expected exception list for the test server then
[17:14] <jam> vila: well, in the timeout code, I'm treating EBADF as "yeah, stop what you're doing". So no problem there. It shouldn't have to propagate to the test suite.
[17:14] <jam> vila: thanks for the hint on type(e)
[17:14] <jam> I *really* don't like the generic "error" classes
[17:15] <vila> especially when they create aliasing problems like this
[17:15] <jam> vila: yeah, I guess if __repr__ included __module__, that might be sufficient
[17:16] <vila> so you're catching both select.error and socket.error and they both specify EBADF ?
[17:24] <Noldorin> jelmer: hey
[17:28] <jam> vila: bonus points. select.error doesn't have an 'errno' attribute, instead it just has e.args[0] == EBADF.
[17:28] <mgz> is that still the case in 2.6?
[17:28] <jam> mgz: python 2.7.1
[17:28] <vila> jam: isn't that nice :)
[17:28] <mgz> I was under the impression it became a IOError subclass
[17:28] <mgz> but that may well be py3k only.
[17:28] <vila> jam: and forget that windows will probably a different idiom
[17:29] <vila> s//don't/ don't forget !
[17:29] <jam> vila: certainly. That gets tested before this gets submitted.
[17:29] <jam> I probably *can't* select.select() at all for Pipes
[17:29] <jam> but at least we can do select(socket )
[17:29] <vila> jam: yeah, sorry :)
[17:30] <vila> jam: I think the required interface is fileno() so pipes should work (well, at least on posix)
[17:30] <vila> . o O (I't be so nice if all this crap was hidden in osutils once and for all...)
[18:04] <JonOomph> Hi! I am trying to test a LaunchPad recipe using the command "bzr dailydeb openshot.recipe working-dir".  It generates a folder inside the working-dir with "{" and "}" characters.  Is there a way to configure the name of the "build" folder?  Thanks!
[18:04] <JonOomph> The name of the folder it creates is "openshot-{debupstream}+{revno}+{revno:packaging}"
[18:05] <JonOomph> This breaks one of my build steps... which call the "xml2po" command.  Apparently, xml2po hates the { and } characters
[19:24] <exarkun> http://codepad.org/lwI05OCH
[19:26] <AuroraBorealis> seems that launchpad is overloaded or somethng
[20:44] <Noldorin> hi jelmer
[20:44] <Noldorin> guess i missed you earlier
[20:44] <Noldorin> jelmer, having trouble testing here a bit..
[20:44] <jelmer> hi Noldorin
[20:44] <Noldorin> jelmer, the problem is that once i uncommit, i cann't merge into the testing branch :-S
[20:45] <jelmer> Noldorin: how do you mean?
[20:45] <Noldorin> PS C:\Users\Alex\Documents\Visual Studio 2010\Projects\IRC.NET\0.4-test> bzr merge ..\0.4\tests -r 47
[20:45] <Noldorin> bzr: ERROR: Branches have no common ancestor, and no merge base revision was specified.
[20:45] <jelmer> what is in the testing branch?
[20:46] <Noldorin> the testing branch is the 0.4 branch, uncommited to r46, then reverted
[20:47] <jelmer> Noldorin: what does "bzr missing ..\0.4\tests" say?
[20:49] <Noldorin> jelmer, ERROR: not a branch
[20:50] <jelmer> sorry, what about ..\0.4 ?
[20:50] <Noldorin> jelmer, "you are missing 49 revision:" etc. etc.
[20:53] <jelmer> Noldorin: ah, but one of them has been dpushed?
[20:53] <Noldorin> yes
[20:53] <Noldorin> i think so
[20:53] <Noldorin> the 0.4-test branch
[20:53] <Noldorin> (successfully)
[20:53] <Noldorin> since it's on r46
[20:53] <jelmer> in that case you indeed won't be able to merge between them
[20:53] <jelmer> dpush changes the history
[20:54] <Noldorin> oh i see
[20:54] <Noldorin> hrmm
[20:54]  * jelmer ponders having a look
[20:55] <jelmer> I'll bbiab, give me a minute or two
[20:55] <Noldorin> sure
[20:55] <Noldorin> :-)
[21:07] <jelmer_> Noldorin, back
[21:07] <Noldorin> hi
[21:08] <jelmer_> Noldorin, so, the easiest way to reproduce the issue is to dpush r47 from your ircnet branch/
[21:08] <Noldorin> yes
[21:08] <Noldorin> i know this much
[21:09] <Noldorin> it's testing it that i've been trying to do the past 3/4 days
[21:10] <jelmer_> Noldorin, where is the bzr version of your ircnet branch again?
[21:11] <jelmer_> (Sorry, I know you mentioned it earlier but I don't have the link here)
[21:11] <Noldorin> jelmer: lp:ircdotnet/0.4 should demonstrate the issue
[21:11] <Noldorin> no prob
[21:21] <jelmer_> ok, reproduced
[21:22] <Noldorin> jelmer, btw, writing a little script for testing here... how can i run bzr commands *without* it prompting me?
[21:22] <Noldorin> i.e. no confirmation for bzr commands
[21:22] <jelmer_> I think the only command that prompts is uncommit
[21:22] <jelmer_> it has a --force option
[21:23] <Noldorin> ok cool
[21:58] <Noldorin> jelmer, written a test powershell script now, so i'm getting closer!
[22:00] <Noldorin> jelmer, do i have to do one commit per merge?
[22:05] <jelmer_> Noldorin, not sure I follow
[22:06] <Noldorin> jelmer, well if i try to do two merges in a row, it gives ERROR: Working tree "..." has uncommitted changes.
[22:06] <Noldorin> ....no away around this?
[22:06] <Noldorin> or do i have to do merge, commit, merge, commit, merge, commit, ... ?
[22:08] <maxb> You can --force it, but it's recommended to commit each merge individually
[22:12] <jelmer_> Noldorin, you probably want to commit each merge to be able to actually see where things break
[22:12] <Noldorin> jelmer, well i have to do them each at once...because as soon as i do dpush, bam, the branch is fucked
[22:12] <Noldorin> can't merge in..
[22:12] <Noldorin> (history has changed)
[22:16] <jelmer_> Noldorin, you can push an entire branch with all the changes split up into individual commits though, and then see which one has problems
[22:16] <Noldorin> yep
[22:16] <Noldorin> that's my plan
[22:27] <Noldorin> jelmer, okay, i have interesting results here..
[22:33] <jelmer_> Noldorin, cool
[22:33] <jelmer_> Noldorin, what are they?
[22:34] <Noldorin> jelmer, if you look at the log for -r 47, it is specifically one line that causes the problem
[22:34] <Noldorin> that is, tests/
[22:40] <Noldorin> jelmer, you thought src/ i guess...but i don't think so
[22:47] <Noldorin> specifically
[22:47] <Noldorin> it seems to be hte line:
[22:47] <Noldorin> RM  src/IrcDotNet.Tests/README.md => tests/README.md
[23:05] <Noldorin> jelmer, yo uthere?
[23:35] <jelmer_> Noldorin: sorry, yep
[23:35] <Noldorin> cool
[23:35] <Noldorin> jelmer, okay, more news if you're ready!
[23:36] <jelmer_> I am :)
[23:37] <Noldorin> right, so firstly i took all the changes from the r47 log individually and applied them in a series of merges to r46
[23:37] <jelmer_> Noldorin, so if you reduce the changes in r47 to just that change the bug occurs?
[23:37] <Noldorin> jelmer, if i then force merge (and only one commit at the end), it *works* fine
[23:37] <Noldorin> however, if i commit after each merge, it fails as usual
[23:39] <Noldorin> however, even in the case with force merges/single commit, it is necessary for both tests/ and samples/ (or neither) to me merged -- otherwise i get the error
[23:41] <Noldorin> jelmer, beyond that, i'm struggling to find the ultimate cause
[23:41] <jelmer_> Noldorin, hmm, that's interesting
[23:42] <Noldorin> jelmer, but then, if the "tests/" merge is not present, everything works fine, regardless of how it's merged it :-S
[23:43] <Noldorin> so evidently the tests/ change is the root of the issue
[23:43] <Noldorin> but something about the samples/ change resolves it, *if* done simultaneously.
[23:43] <Noldorin> (i.e. 2 forced merges in the same commit)
[23:44] <Noldorin> jelmer, you see what i'm getting at?
[23:44] <jelmer_> Noldorin, yep, I do
[23:44] <jelmer_> Noldorin, we're getting somewhere
[23:44] <Noldorin> good good
[23:45] <jelmer_> I suspect the order in which we process the changes in bzr-git might be relevant
[23:45] <Noldorin> indeed
[23:50] <Noldorin> bzr init --git
[23:50] <Noldorin> bzr: ERROR: No module named posix
[23:51] <Noldorin> jelmer, any idea why that is happening? ^
[23:51] <jelmer_> Noldorin, that should be fixed in newer versions of bzr-git
[23:51] <Noldorin> oh ok cool
[23:51] <Noldorin> i should update first indeed