[00:00] hi all [00:00] our wiki software is going to be upgraded today, there may be glitches [00:04] hi poolie [01:09] hi jelmer [01:09] you still around? [01:09] i've done a bit more research... [01:11] jelmer, the problem is that branching and uncomitting back to r46 makes the branches diverge... [01:11] (no common ancestor, it days) [01:11] so i can't merge in :-S [01:11] from r47 [01:11] how eird [02:41] Is there a function I can call that'll basically parse what I might pass to bzr at the command line and transform it into whatever Branch.open will be looking for? [02:42] directory service lookup is the difference there I think [02:47] lifeless, How do I use that? I assume there must be some function I need to call to get things initialized and get a DirectoryServiceRegistry object? [02:48] there is probably a module level instance you can use [02:48] you'll need a library context as usual [02:53] Figured it out. [02:53] Branch.open and stuff automatically do directory service lookup [02:53] I had to call bzrlib.initialize() and bzrlib.plugin.load_plugins() first though [03:09] yes :) [04:28] wgrant, lifeless, i was going to say, people are getting things done at a good rate but it may feel a bit like one thing after another [04:28] without them so much adding up [04:28] john commented on something like this [04:28] one idea is to borrow the rotation idea from lp, but for the whole team: [04:29] for say 2 weeks all focus on one area, eg bfbia or lp bzr-git integration, or whatever [04:29] probably for as long as it takes to get that area adequately complete [04:29] I think that's a good strategy, particularly for something like bzr. [04:29] interrupted only by actually critical bugs [04:29] As long as the thing is of reasonable scope. [04:30] (in the bzr definition, meaning data loss or total use blockage or whatever) [04:30] and then do say 2 weeks of reactive work on general bugs [04:30] the tricky part of this, from experience and observation is getting to the point where you can stop [04:30] we would still continue with piloting but people would have less discretion to pick their day-to-day work [04:30] yes [04:30] whats your average cycle time ? [04:31] james_w can tell you lots about this too :) [04:31] the feature you pick could easily turn out to be 3 days or 3 months instead [04:31] i don't know, where does lp show that? :-7 [04:31] but, the mean is probably about a week [04:31] lpkanban should be emitting that :) [04:32] so, that means 2 complete patches end to end for each person (on a 2 week cycle), and if you're doing concurrent work, whatever your concurrency is in WIP when you come to change. [04:32] do you have a few ideas you want to run up the flag pole, or just this particular one ? [04:33] i don't think it follows it would be 2 patches per person, because i'm including stalls there, which this could reduce [04:33] for example its skewed by 660264 being something like 18 months old [04:33] k [04:33] this is my central idea [04:34] i think the 'typo fix' time is somewhere around 24h or less [04:34] so, a fairly sized feature could get done in 2w [04:34] there are some variations [04:34] 0- change nothing [04:34] 1- put one person (or two) on bugs, and have others ignore bugs and work on a feature [04:35] 2- anyone can do either bugs or features on any day, but there are fewer features in play to choose from [04:35] how much does a [human] context switch cost in bzr? [04:36] 3- just generally shift the bar of which bugs ought to come ahead of feature type work [04:36] (obviously i'm using the word 'feature' here to mean proactive work rather than reactive, it may have a bug number) [04:36] lifeless, as in, switching from one bug to another, or one area of the code to another? [04:37] hm it seems a bit hard to generalize [04:37] the harder issues might need a bit of mental state [04:37] poolie: bit of A, bit of B [04:37] like, if you said 'lets spend 2 weeks on looms' [04:38] how long before things started flowing? [04:38] i think generally speaking if I did an hpss bug today and a udd bug tomorrow it would not hurt [04:38] great example; i think the costs are going to be [04:38] - i already have these other mps underway and i need to finish them off (kind of queue-draining) [04:39] - deciding what in particular we're going to do [04:39] - to some extent, getting familiar with the code, though .. perhaps that's not enormous, and having a specific impetus to get into eg the loom code could be beneficial [04:40] perhaps the biggest cost of timeslicing is going to be queue draining/filling [04:40] a few random thoughts [04:40] having > 1 person on a task is enjoyable [04:41] jelmer: bzr-search trunk is now owned by ~bzr [04:41] jelmer: I've left a branch reference behind for convenience [04:41] (yes, that's one thing i'd like to have happen more often) [04:42] waste [in the lean sense: time in the production of something that isn't actively moving a product through to the customer] can make things really drag out [04:42] and for me at least that draggy feeling can numb the sense of variety that one would otherwise get [04:43] e.g. exec rather than fork on lp :) [04:43] yes, that's very true [04:44] so, you can either switch while you have stuff to drain, or you can switch 'new stuff' but continue to drain. [04:44] something that makes these behaviours closer is reducing the time to drain/time to fill figures [04:45] mm [04:45] I suspect you're already pretty vigorous on that [04:45] i think our average time from an mp to getting it merged is moderately low now [04:45] but its worth having in mind when considering new scheduling approaches [04:45] i think having more people on one project tends to reduce it more, because there are informed reviewers [04:46] e.g. doing something that will increase the time-in-queue (like less piloting) may have a negative impact [04:46] yes, very much [04:46] i wouldn't cut back on piloting for this [04:46] this is all within the slice of non-piloting work [04:46] sure, I'm just thinking through it all [04:46] releases will interact with this. [04:47] unless you can automate them totally out of existence. [04:47] yep, they too are a kind of 'hygene' task [04:48] as are srus (kind of related) [04:48] and answering user questions [04:48] yah [04:48] very likely we can or should automate them more [04:50] I don't know if whole-team rotations make sense, I suspect you may pay in context switches [04:50] OTOH if everyone is looking at the same stuff you should get some network effect. [04:50] my 2c: give it a go for 3 cycles. [04:50] measure it, [04:50] return to what you have now [04:50] measure that. [04:50] compare. [04:50] (return because otherwise the act of changing may be a placebo) [04:51] you say placebo like it's a bad thing :) [04:51] it is if you want things to -stay- better ;_ [04:57] wgrant, any other thoughts? [04:58] Mmm. [04:58] It's hard to say how well it would work for bzr. I think you probably have less interrupts, so it would work well. [04:59] And, as you say, it would be nice to have multiple people working together on one thing. [06:46] hi all ! === _nyuszika7h_ is now known as nyuszik7h === nyuszik7h is now known as nyuszika7h [07:48] hi vila [07:48] poolie: hey ! [07:49] hi there [07:50] warming up for the standup ? [07:50] yeah [07:50] apparently it was still owned by spiv so i was just trying to fix that [07:50] he's still pretty much welcome to come but i doubt he wants calendar remindres [07:51] :) === vila is now known as babune [07:57] Riddell: thanks :) === babune is now known as vila [07:58] for a second there i thought you set up a bot [07:58] hehe, not yet... [07:58] acting bot only so far ;) [08:02] jam, jelmer, hi? [08:05] o/ mrevell [08:07] Hi! [08:37] lp is down ? At least the high failure rate on the package importer suggests so [08:38] requeued [08:44] not as far as i know [08:46] ~600 spurious failures [09:19] poolie, jam, jelmer, Riddell : Did one of you want to post the standup summary or should I ? [09:20] vila: Fine with me if you want to do it. [09:21] poolie, jam, jelmer, Riddell : I'll do it then, still 5 secs to shout otherwise 4... 3 [09:21] go ahead [09:25] done [09:31] jelmer, what's the tag for bfbia bugs? [09:32] i know i've seen them but the link from https://dev.launchpad.net/LEP/BuildFromBranchIntoArchive has nothing [09:32] jelmer: maybe that just means the code is perfect? (*crosses fingers*) [09:35] jam, https://code.launchpad.net/~jelmer/launchpad/bfbia-db/+merge/68990 [09:37] jelmer, what's actually up with that? it looks like it's mostly merged? [09:44] poolie: bfbia is the relevant tag [09:44] so there are no bugs? [09:44] jelmer: k, there just doesn't seem to be any matches [09:44] poolie: though I think most of the bugs that have been closed were actually in bzr-builder/bzr-builddeb [09:44] just the db proposal? [09:44] oh ok [09:44] no bugs in lp [09:44] poolie: there's one, but it lacks the tag :-) [09:44] nice :) [09:44] bug 313602 [09:44] Launchpad bug 313602 in Launchpad itself "Please add "build this branch" option" [High,In progress] https://launchpad.net/bugs/313602 [09:45] the relevant changes for bfbia are that db branch, which is trivial. I haven't looked at landing because of lp's changing of its deployment process. [09:46] and there's the proof of concept branch at https://code.launchpad.net/~jelmer/launchpad/bfbia/+merge/64036 which works but had too many database changes. I've been working on getting it to use the existing recipe-based infrastructure instead. [09:49] my guess is that it's fairly close now that the other Launchpad related stuff has landed [09:51] jam, https://bugs.launchpad.net/udd/+bug/724890 [09:51] Ubuntu bug 724890 in Ubuntu Distributed Development "excessive memory consumption for nexuiz-data" [Critical,Confirmed] [09:58] poolie: https://bugs.launchpad.net/bugs/819604 ? [09:58] Ubuntu bug 819604 in Bazaar 2.4 "when an idle ssh transport is interrupted, bzrlib errors; should reconnect instead" [High,Confirmed] === zyga is now known as zyga-afk [10:02] Riddell: hi [10:02] Riddell: just followed up to your i18n patch [10:03] Riddell: I'm also wondering more generally about i18n what the best way is to translate plugins. [10:09] I haven't really thought about that so far [10:09] but they should load a separate gettext domain with their translations [10:14] could someone look into https://bugs.launchpad.net/bzr/+bug/848064 ? [10:14] Ubuntu bug 848064 in Ubuntu Distributed Development "Revision not present branching from LP" [Critical,Confirmed] [10:15] if it is similar to the earlier bugs max dealt with, it should be able to be fixed by clearing and requeuing the imports [10:15] or something like that [10:19] night all [10:23] jelmer, so does that small db patch still make sense to merge? [10:23] just +ALTER TABLE SourcePackageRecipeData ALTER COLUMN deb_version_template DROP NOT NULL; [10:23] if so, can i remind you to push for a review, or for clarification on how/where/when to merge it? [10:27] poolie: I think it still makes sense, but I'd like to finish my other branch first to be sure. [10:29] Riddell: it would be nice to have some convenience functions in bzrlib.i18n for that [10:29] jelmer: for plugins? [10:29] Riddell: yeah [10:30] yes I agree, I'd like to do that [10:58] vila: at what stage in the release process do we make a new stable branch? [10:59] Riddell: I'd prefer an easier question before my lunch [10:59] :) [10:59] Riddell: you mean on pqm ? [10:59] vila: I mean when will lp:bzr/2.5 be separate from lp:bzr [10:59] or rather you mean a branch associated with a stable series [10:59] yeah [11:00] so, that should be in releasing.txt, but from memory, the branch is created to release 2.x.0 [11:00] aka when we create the first stable release in a series [11:01] it shouldn't be hard to do that for the *last* beta before the first stable if that makes it easier for translations [11:01] we freeze the API after the last beta already [11:02] ok, I'll write that up [11:04] Riddell: search for 'Kick off the next cycle' in releasing.txt [11:04] * vila off for lunch === zyga-afk is now known as zyga [11:59] vila: for your release manager eyes https://code.launchpad.net/~jr/bzr/i18n-release-process-doc/+merge/75161 [12:00] maxb, jam : Any issue blocking 2.4.1 ? [12:00] jelmer: oops ^ :) [12:04] vila: anything blocking me from building it? (just not having done it for me) [12:05] jam: yeah, I mean, any reason to delay the announce [12:05] well, windows installers aren't built yet... [12:06] jam: discussed numerous times, unless critical bugs block building, availability should not block announce [12:06] lack of availability for that matter [12:06] :) [12:09] vila: not afaik [12:14] jelmer: IIUC you're first to shoot by releasing on debian right ? [12:14] sid even [12:14] not sure what you mean by first to shoot :) [12:15] jelmer: once it's released on debian, the ubuntu releases can be done [12:15] ah, yeah [12:16] vila: I usually upload to sid and ubuntu at the same time [12:16] ok [12:16] jelmer: any ETA ? :D [12:18] Riddell: reviewing, can we discuss it a bit ? [12:18] vila: I'll have a look now, if the tarball is up [12:19] vila: looks like the 2.4.1 tag is missing [12:20] jelmer: it's up since last Thursday. Did my '[ANN] 2.4.1 has gone gold' mail got lost ? Ha crap [12:20] it's here :-/ [12:21] jelmer: oh, may be you mean not on trunk ? [12:21] vila: not on lp:bzr/2.4 [12:21] jelmer: bzr-2.4.1 6042.1.4 [12:21] looks like it is here [12:22] jelmer: also in lp:bzr/2.4 (I just confirmed) [12:22] pfew, thanks jam, I was starting to suspect pqm (I *did* sent 2 submissions with this tag (AFAIK) but just when we migrated) [12:22] and there has been some weird branch shuffling during the migration [12:23] sorry, my bad - we don't have the right magic in the 2.4 packaing branch yet [12:23] np np [12:24] (and I just pulled lp:bzr/2.4 and didn't get any updated tags message :-D) [12:25] and 2.4 hasn't been merged on trunk yet so trunk doesn't have the bzr-2.4.1 tag, all is well :) [12:35] vila: of course === zyga is now known as zyga-afk [12:35] Riddell: reviewed locally, I was surprised by the 27 +The final beta - branching and translations [12:35] Riddell: but it's correct [12:36] Riddell: you used (AT) in emails, why ? [12:36] Riddell: I mean, we didn't do this for the others, should we change ? [12:36] there's enough special casing for the last beta that I think it needs its own section [12:36] yeah, yeah, good idea, I agree [12:36] vila: the e-mails were just copied off what dpm gave me, I can change them to @ [12:36] and enclose them in `` `` [12:37] i.e. only tweaks, I'll vote [12:38] updated [12:39] Riddell: generally when we say tweak, it means, no need for a second review, go :) [12:40] formidable [12:42] :) [12:49] vila: poke for great justice! [12:49] I have a question about some interactions with threading and socket receiving [12:50] jam: la la la , I can't hear you [12:50] specifically, I'm trying to add a 'select()' call before we start reading the next data stream [12:50] kidding :) [12:50] so that I can timeout if a connection goes idle [12:50] however, in doing so, some tests start getting "hung threads" [12:50] yeah [12:50] however, when I run the test it claims is a problem [12:51] it passes in 0.1s [12:51] don't mix timeouts and blocking calls [12:51] vila: specifically, the socket is set to blocking, and I just select.select(... timeout) [12:51] which generally seems to work [12:51] yeah, generally [12:51] but, just no [12:51] it's in the docs, let me find it again [12:53] http://docs.python.org/library/socket.html [12:54] jam: re-reading as the wording seems different from what I remember and things may have changed [12:54] ha, yeah, I think it's even more forbidden when using sockets *and* makefile() objects(which we do) [12:55] ile objects returned by the makefile() method must only be used when the socket is in blocking mode; [12:55] quteo: "File objects returned by the makefile() method must only be used when the socket is in blocking mode" [12:55] quote [12:55] so, don't do that [12:56] most of the issues in the socket/thread leaks in selftest saga (part I) were caused by timeouts [12:57] jam: IIRC, the bzr server itself cheats a bit by using a timeout but it does so on the listening socket where we don't use makefile() objects [13:01] jam: ping ? [13:01] vila: there is a difference from setting a timeout on the socket, and using select with a timeout [13:01] I'm not setting a timeout on the socket [13:01] (and, unfortunately, empathy still refuses to make noise when I get mentioned.. :( [13:01] ha, ok (empathy) [13:01] I'll probably switch to Pidgin soon [13:01] It used to work [13:02] But I think it stopped when I installed natty [13:02] I have to see the little blue icon in the corner. [13:02] you may be right about select, but the symptom you describe reminded me of so many issues I encountered [13:03] so, back tracking, [13:03] what do you mean by "before we start reading the next data stream" ? [13:04] ugh, still so much pain. If I run the test suite, I can see what test is failing, if I run *just* those tests, no failures [13:04] vila: In SmartServerStreamMedium.serve() we loop over _build_protocol() _serve_one_request(protocol) [13:05] so for every message that comes in from the client [13:05] we read the first line, then dispatch the request and process it in 'blocking' mode [13:05] I'm trying to add a "select(..., timeout)" just before reading the first line [13:05] so if a client connects, chats with us a bit, and then never hangs up [13:05] we'll hang up after a time [13:06] as long as they *initiate* an RPC, we won't timeout on them (until that RPC has been fully responded to, and they haven't started a new one) [13:07] funny enough, I can set the select timeout to 4.0s, which is shorter than the 'hung_thread' timeout of 5s. [13:07] And then I get the exception in my code. [13:07] Running a bunch of tests, seems to deterministically pick what test will fail [13:07] running just that test, no failure [13:07] jam: hmm, my gut feeling is that this won't mix up well with the rest (and the test failure kind of confirm that, but well, easy ) [13:08] Does the serve actually capture the connections or is it only the test server that does that for now ? [13:08] s/serve/server/ [13:09] jam: and do you include errors in your select ? [13:10] during tests we close the sockets with prejudice so if you don't include errors in your select, you may miss that [13:10] alternatively, if the connections are captured you can update a stamp and let the main thread close/kill the idle connections [13:11] whether you want to kill a connection taking too long to process a request or not may be worth considering too [13:24] vila: currently I'm just doing select([r], [], []) however I get EBADF if the socket was closed on the server side. [13:24] I don't do any special capturing, etc. [13:24] vila: i don't *think* bzr serve --inet runs any threads [13:24] I could be wrong [13:26] (that particular case is also a PipeStreamMedium, which is not the same code path) [13:27] plain 'bzr serve' creates a SmartTCPServer that dispatches to threads [13:27] but "bzr serve --inet" just hooks up a SmartServerPipeStreamMedium to stdin/stdout [13:27] so there isn't a master process monitoring the 'connections' === zyga-afk is now known as zyga [13:53] further, it only fails during teardown parts of the test suite. Never in 'live' code. [13:53] so i'm tempted to say it is something incomplete with out test infrastructure, not sure what yet [13:55] I don't remember issues with SmartServerPipe, may be because I didn't work much with it [13:56] jam: which tests are failing ? [13:57] A random selection (different each run) from: ugh, still so much pain. If I run the test suite, I can see what test is failing [13:57] sorry bad paste [13:57] bzr selftest -s bt.per_branch.test_branch Remote -v [13:58] Note, I did just find: self.clients.pop() [13:58] self.clients.append((request, client_address, t)) [13:58] which doesn't actually check that the thing popped off is the same thing we are appending === zyga is now known as zyga-afk [13:58] so there is at least the possibility that between 'get_request' and 'process_request' the order changes [13:58] and we close the same object 2 times, but never close another object, etc. [14:00] but why is that involved if you focus on SSPipe ? [14:00] I'm not focussing on SSPipe [14:00] since most of our test suite uses the other [14:00] and I'd *like* them to act roughly the same [14:01] *Launchpad* wants us to support the TCP stuff as well [14:01] so that stale connections get reaped [14:01] yes, we *could* do it in 2 different ways [14:02] even weirder. sometimes you clearly get a 2s timeout, but the test still passes [14:05] ugh. it seems to pass if I add a 'print' statement [14:06] sys.stderr.write is enough [14:07] timing issues are generally caused by a lack of synchro somewhere, addind timeouts in the mix.... shouldn't help [14:07] I guess that gives the other thread a chance to say "oh hey, he must be shutting down" [14:08] yeah, kind of adding sleep(abit) [14:12] vila: oddly enough adding time.sleep(0.1) does *not* fix it. but print "data" *does* [14:13] hi guys :) [14:13] I don't see a windows build for bazaar 2.4.1 yet [14:13] I have the feeling it is just the test suite failing to see the exception. [14:14] blackarchon: it isn't built yet [14:14] is there an ETA? [14:19] blackarchon: nothing concrete. I have it building now, but if there are any failures I probably won't have time to address them [14:28] jam: ok, thx :) [14:44] blackarchon: looks like the build went fine. Uploaded [14:52] * Riddell nudges jelmer into reviewing https://code.launchpad.net/~jr/bzr/i18n-enable-translations-everywhere/+merge/75040 again before the end of the day [14:52] Riddell: I'll have a look [14:53] oh, actually [14:53] jam, vila: do you have an opinion on whether installation of gettext should happen by bzrlib.initialize() or lazily in bzrlib.i18n.gettext()? [14:54] both :) [14:55] it sounds like something that should be done in bzrlib.initialize() but if the cost is too high to be done unconditionally, then it should be lazy [14:56] the test suite itself doesn't really respect bzrlib.initialize though [14:56] Riddell: is there a way to disable i18n explicitly? [14:57] Riddell: I don't really want to bikeshed on this, as I'm not sure how important it is [14:57] jelmer: currently unset LANGUAGE and the other envirnment variables [14:58] Riddell: but it would be nice to give 3rd party users of bzrlib the ability to disable i18n [14:59] a function in i18n.py to do that would be easy enough [14:59] Riddell: you said it was "slightly less than 0.1s to slightly more than". Anything more quantifiable? [14:59] Like 0.01s or 0.08s, etc? [14:59] I do like the idea that if a command has nothing to say, it doesn't have to load the mappings [15:00] I have the feeling that the mappings will get more expensive to load when [15:00] a) they aren't just english [15:00] b) we have more stuff that gets translated [15:00] I don't know if gettext() loads all the mappings preemptively, or if it has an index and can only load the ones that are actually [15:00] translated. [15:01] I would hope the latter [15:02] I think it has the latter because it has some kind of binary file that it loads rather than the plain text [15:02] jam: thank you, I will test it! :) [15:13] jam: for example http://paste.kde.org/121057/ === bigjools is now known as bigjools-otp [15:21] Riddell: Interesting post re:OpenSuse :) [15:30] vila: bzr-2.4.1-1 uploaded to debian, 2.4.1-1ubuntu1 now building.. [15:31] \o/ [15:35] jelmer: i18n.disable_i18n() added [15:44] \o/ [15:47] Riddell: r=me [15:47] pardon? [15:48] sorry, Launchpad jargon [15:48] Riddell: I've approved your merge proposal. [15:48] awooga === beuno is now known as beuno-lunch === bigjools-otp is now known as bigjools === zyga-afk is now known as zyga === deryck is now known as deryck[lunch] [16:39] hi again jelmer [16:41] hey Noldorin [16:58] vila: I think I've tracked it down. My guess is that we enter select.select([socket]), and then the TestCase teardown code does socket.close() [16:59] If the teardown happened *first* we would get EBADF [16:59] because it happens second, we don't see it until after the timeout. [16:59] I changed it to a loop, and now I get socket.error(EBADF) often [16:59] About 1 time in 500 or so, I get something that looks just like socket.error [16:59] but doesn't catch "exception socket.error" [16:59] but it gets caught by a generic "except Exception" [17:00] Now I have something that gets EBADF about 1-in-3 tests, but never causes the test suite to hang or crash [17:00] ...an socket.error that is not a socket.error... rings a bell, windows ? [17:00] vila: this is on Ubuntu Natty [17:00] no ssl involved ? [17:01] It *sounds* like if someone re-imported socket, so you had 2 real modules involved [17:01] vila: This is mostly loop-back stuff. [17:01] It just str(e) says "error" [17:01] well [17:01] type(e) [17:01] error(9, "bad file descriptor") [17:01] I'll check, just a sec [17:01] but first things first [17:01] your enter the teardown while in select() ? [17:02] you have a thread running and you don't join it ? [17:02] vila: I think it is your "issue a connect but don't write anything" connection [17:02] which starts a new "oh, someone wants to talk to me" thread [17:02] I'm not positive yet [17:03] oh, the hackish 'last connection to unblock the server in listen' ? [17:04] and I'll stop asking more questions, 3 pending ones is already a lot... [17:05] vila: Right, racing threads. If the select() thread happens first, it doesn't see EBADF, if it happens second, it does [17:05] if it doesn't see EBADF, it waits until timeout [17:05] before it fails [17:05] with nothing to read from [17:05] we *are* joining from it, but it fails in the mean-time with a timeout [17:05]