#launchpad-dev 2009-08-31
<lifeless> does anyone know the rdf ontology used for our project metadata?
<lifeless> found it, doap.
<jml> mwhudson, fwiw, I have an ec2test run going that runs the code doctests in the DBFunctionalLayer
<mwhudson> jml: which layer is that?
<jml> mwhudson, the DatabaseFunctionalLayer
<mwhudson> jml: is that the one that doesn't start the librarian?
<jml> mwhudson, the one with zcml & the db and not much else
<jml> mwhudson, yeah
<mwhudson> jml: some code import tests may well fail
<mwhudson> i guess you'll find out :)
<jml> mwhudson, I expect so. :)
<mwhudson> some diff related ones too, perhaps
<jml> mwhudson, I'm also running tests that delete a lot of columns from ProductSeries
<mwhudson> jml: \o/ for that
<jml> mwhudson, all of which are related to vcs-imports, afaict.
<mwhudson> yeah
<mwhudson> they're long dead
<jml> mwhudson, if the tests pass, can you please review the patch?
<mwhudson> jml: sure
<mwhudson> thumper: i just wrote https://bugs.edge.launchpad.net/launchpad-code/+bug/118625/comments/12 btw
<mup> Bug #118625: codebrowse sometimes hangs <codebrowse> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/118625>
<spm> mwhudson: fwiw, sounds sane to me; and would seem match observation. logs tho... hrm.
<lifeless> wget -O- https://edge.launchpad.net/launchpad/+rdf | wc -l
<lifeless> 1634
<jml> thumper, btw, I have your 'active-branch-counts' still in my active reviews page
<thumper> jml: yes, I've not done anything with it recently
 * mwhudson lunches
<jml> thumper, write better cover letters.
<thumper> jml: yeah, I suppose
<thumper> normally do
<thumper> just frustrated
<jml> thumper, page migration getting you down?
<thumper> just a bit...
<thumper> given that the breadcrumbs were broken for branches
<thumper> again
<thumper> s/were/are
<jml> mwhudson, ping
<mwhudson> jml: hi
<jml> mwhudson, I'd like a second opinion on something. Got a moment for a short skype call?
<mwhudson> jml: sure thing
<jml> mwhudson,   lib/lp/code/tests/../doc/branch-merge-proposal-notifications.txt
<jml>   lib/lp/code/tests/../doc/codeimport-result.txt
<jml> mwhudson, those are the only ones that failed.
<mwhudson> jml: my predictive powers are clearly working reasonably well today
<jml> mwhudson, don't jinx it :)
<mwhudson> (i just wish they were predicting more success with buildbot :/)
<jml> haha
<lifeless> mwhudson: you're having op trouble?
<mwhudson> lifeless: op ?
<lifeless> operational
<mwhudson> ah
<mwhudson> no, not really
<mwhudson> lifeless: mostly trying to bash the model buildbot uses into my brain
<lifeless> mwhudson: pb jelly + bidirectional callbacks around steps of activity.
<lifeless> mwhudson: If you need a clue at any point gimme a shout
<lifeless> mwhudson: I doubt its changed radically since I climbed all through it years ago
<mwhudson> lifeless: well, it has changed somewhat because gary added 'latent builders' to it
<lifeless> they being ec2 slaves
<lifeless> which it can activate
<lifeless> ?
<mwhudson> yes
<lifeless> Arbitrary number, or a slave-factory?
<lifeless> [does this make buildbot a slave owner?]
<mwhudson> the 'force build' button doesn't appear for them unless a build is running
<lifeless> and thats the bug you're working on?
<mwhudson> because the slave isn't 'connected' according to buildbot in this state
<mwhudson> well
<lifeless> yes
<mwhudson> i have this niggling suspicion that if i can make it appear it then won't work
<lifeless> thats to be expected. Uhm, I'd decouple that assumption [yes I realise thats what you're looking for the knobs of]
<mwhudson> yes
<lifeless> are the latent slaves all manuall configured?
<lifeless> or does it act as a factory for them?
<mwhudson> lifeless: i don't understand the question
<mwhudson> lifeless: it starts an ec2 instance for each build
<lifeless> so does the config say 'slave FOO; is latent; <various details>', 'slave BAR; is latent...', and then you can have 1 and only 1 build on FOO at a time, ditto BAR
<lifeless> -or-
<mwhudson> slaves are ~always configured manually in buildbot aren't they?
<mwhudson> lifeless: yes
<mwhudson> this is part of the terminology thing
<lifeless> does the config say 'slave FOO is latent <factory details>', and then new slaves appear each time ec2 is prodded to do stuff, called FOO-<instanceid>
<mwhudson> i have a hard time remembering the difference between builders and slaves
<mwhudson> and build factories
<lifeless> mwhudson: IIRC builders are 'code that knows how to perform an operation'
<lifeless> build factories are 'code that parameterises a builder for a job'
<jml> wuu, -1 actually seems to work these days.
<jml> (it shows only the first failure in a doctest)
<lifeless> mwhudson: so if its the first case, then all the slaves are static, the only interesting thing is that they connect on-demand.
<lifeless> mwhudson: I'd consider two things;
<lifeless>  - a 'connect this slave' button.
<lifeless> this would work, unless the slave auto-disconnects instantly
<lifeless> (and a connecting slave would want 'force build' to be available as a callback on connect completing)
<lifeless>  - making force build trigger a connect implicitly
<jml> hmm. the lp-dev-utils kerfuffle is stopping me from opportunistically coding.
<jml> something to deal with when I return.
<lifeless> jml: comefrom?
<jml> lifeless, return from running a bunch of errands, which is what I'm about to do.
<wgrant> mwhudson, thumper: Is lp:~wgrant/launchpad/structural-subscription-security cursed, or just unlucky?
<wgrant> Three have tried to land it so far.
<wgrant> I never heard back from the third attempt.
<jml> wgrant, hmm.
<jml> wow, the "Request another review" widget is broken as.
<jml> wgrant, commit message?
<jml> (I'll update the field on the MP page to have the commit message so you don't get asked again)
 * jml writes out, "[r=gary][ui=none] (wgrant) Move structural subscription security to the model, rather than the view."
<wgrant> jml: That sounds about right.
<wgrant> Thanks.
<wgrant> Hopefully this time it will work.
<wgrant> You know, I really hate tests that fail due to obscure errors in a three-level-deep exception handler.
<wgrant> I also really hate three-level-deep exception handlers, but particularly when they crash.
 * jml sympathises
<jml> sometimes it's a sign that there need to be unit tests for that third level layer.
<jml> particularly in the Launchpad code base.
<jml> mwhudson, thanks for replying to the RFC about package branchse.
 * jml needs to reply to slangasek's email.
<mwhudson> jml: np
 * mwhudson doesn't think he's getting all the mails on this topic
<jml> mwhudson, slangasek's mail was sent to launchpad-dev on the 29th.
<mwhudson> jml: subject?
<mwhudson> i don't seem to have it
<jml> mwhudson, Re: [RFC] Taking source package branches to the next release
<mwhudson> jml: nope, nothing from him
<jml> mwhudson, hmm.
<wgrant> It wasn't on the list.
<wgrant> Or is stuck in moderation.
<jml> stuck in moderation, I'll bet
 * jml moderates in anger
 * mwhudson receives lots of fairly old mail\
<wgrant> Ah. That's filling in the gaps.
<spm> mwhudson: jml: which list? I did the int one this morning, nought but spam.
<jml> spm, the launchpad-dev list
<jml> spm, as found on http://launchpad.net/~launchpad-dev
<spm> hrm. nothing there either.
<jml> spm, that's because I just moderated them all :)
<spm> awesome. my problems just went away. ;-)
<spm> heh. just noticed (and fixed) the lp-eng team membership of same was set to expire later this year. that would have been amusing to watch.
<wgrant> spm: You seem to want https://launchpad.net/+announcements.
<wgrant> That page was almost removed a week or two ago.
<spm> wgrant: rofl. which just goes to show how well... advertised(?) the service that provides is :-)
<wgrant> spm: I've never seen it linked.
<wgrant> But it has a useful Atom feed too.
<wgrant> (in fact, I only discovered that page by guessing the URL when I saw people pondering its removal)
<spm> lol
<mwhudson> now why the bleep isn't db_lp building??
<wgrant> mwhudson: buildbot hates you.
<mwhudson> wgrant: i hate it more
<wgrant> alright. I am very confused.
<wgrant> I have a test here that really should fail in python2.4, but only fails in python2.5.
<wgrant> AFAICT.
<wgrant> Basically, it's logging an exception.
<wgrant> Said exception contains '%s' in the traceback.
<mwhudson> exceptions are new style in 2.5, could it be that?
<wgrant> So when the logger function attempts to use % on the string, it crashes.
<wgrant> Possibly. Let's see what 2.4 exceptions look like.
<mwhudson> python2.4 exceptions might look a little bit more like sequences to some parts of the code
<wgrant> Hmmm.
<mwhudson> or maybe not
<mwhudson> wgrant: can you pastebin come code?
<wgrant> mwhudson: http://paste.ubuntu.com/262311/
<wgrant> The problematic exception is http://paste.ubuntu.com/262312/.
<mwhudson> wgrant: oh gosh, mocklogger is terrible
<wgrant> mwhudson: Heh, there is a comment in one of the callsites saying that it should be merged with FakeLogger.
 * wgrant looks for any other callsites.
<spm> FakeLogger? is that like an alias to /dev/null?
<wgrant> Mm, there are a few.
<mwhudson> there are so many dummy loggers lying around
<mwhudson> it's really cruddy
<spm> *why*!?!?!?
<mwhudson> wgrant: i don't see why that would behave differently 2.4/2.5
<wgrant> mwhudson: Neither.
<wgrant> mwhudson: But the same exception is raised in both.
<mwhudson> wgrant: but the mocklogger code is broken, please fix it :)
<wgrant> And somehow crashes on 2.5...
<mwhudson> wgrant: maybe format_exception does different things?
<mwhudson> seems unlikely though
<wgrant> mwhudson: Possibly. Let's see..
<mwhudson> pdb ftw
<wgrant> Oh.
<wgrant> No.
<wgrant> Surely not.
<wgrant> mwhudson: I now know why.
<wgrant> mwhudson: 2.4 prints the first line of the statement.
<wgrant> 2.5 prints the last.
<mwhudson> wgrant: ha ha
<wgrant> How utterly stupid.
<wgrant> I almost didn't bother to check that, it was such a silly idea.
<lifeless> its bytecode data I think
<wgrant> So, I think that was the only mysterious Python 2.5 failure.
<wgrant> Anybody want to review https://code.edge.launchpad.net/~wgrant/launchpad/fix-mocklogger-format-string-crash/+merge/10902?
<jml> wgrant, has anyone answered yet?
 * mwhudson eods
<wgrant> jml: No.
<wgrant> jml: (has my accursed branch killed ec2test yet?)
<jml> wgrant, ec2test went headless, still waiting for a reply
<wgrant> jml: Great, thanks.
<jml> wgrant, I don't fully grok your patch yet...
<jml> wgrant,
 * jml kicks xchat
<jml> wgrant, specifically, I don't know what "2.4 seems to render the first line of each statement in tracebacks, while 2.5 renders the last" has to do with only doing string interpolation if arguments are provided.
<wgrant> jml: Well, I half expected the reviewer to be somebody who had read the earlier discussion.
<wgrant> jml: Basically, in the case of that test the last line of the statement contains '%s'.
<wgrant> So MockLogger gets a string with a '%s' in it, without any arguments.
<wgrant> Boom.
<jml> ahh ok.
<jml> (also, I'm extremely hungry)
<wgrant> It is approximately way past lunchtime.
<wgrant> jml: How would I best go about testing it? I can't exactly just use a mock logger and check the output for this one...
<jml> wgrant, construct a MockLogger, temporarily replace sys.stdout with a StringIO object, call log() without args, check the contents of the StringIO, replace sys.stdout
<wgrant> jml: OK. I was hoping there was some less bad way than replacing sys.stdout.
<jml> by 'without args', I really mean "log('%s')" or something similar.
<wgrant> Yep.
<jml> wgrant, well I was about to get to that :)
<jml> wgrant, the nicer way is to give MockLogger a constructor that takes an optional file-like object
<jml> wgrant, if provided, use that; if not, use sys.stdout
<wgrant> jml: Ah, of course.
<jml> that would mean changing the print call to self._output.write or whatever, naturally.
<wgrant> Yep.
<wgrant> Thanks.
<jml> np
<spiv> wgrant: presumably the caller is buggy, and should be doing "log('%s', msg)" instead of "log(msg)" ?
<jml> hmm...
<spiv> (Making the logger robust against this misuse is still a good idea, though)
<wgrant> spiv: stdlib logging says otherwise.
<spiv> Huh?  Ok.
<wgrant> spiv: Anyway, the problematic caller is actually the logger itself.
<wgrant> spiv: The standard library logging module allows that kind of call.
<lifeless> loggers are meant to lazy evaluate
<lifeless> out of a sense of cheapness
<lifeless> this is arguable
 * jml jfdis
<noodles775> Howdy
<jtv> hi noodles775!
 * noodles775 waves at jtv
<jtv> noodles775: and since it's Monday morning for you... the "this site is running demo code" bar seems to be horribly broken on both 3.0 and pre-3.0 pages, but differently on dev and edge.  Saw something about it on the reviews list; is it being worked on?
<noodles775> jtv: https://bugs.launchpad.net/bugs/421454
<mup> Bug #421454: Filebug link on edge/staging header needs structure <trivial> <ui> <Launchpad Foundations:Fix Committed by michael.nelson> <https://launchpad.net/bugs/421454>
<noodles775> yep, it's landed in devel... should be fixed on edge on the next rollout (hopefully soon).
<jtv> noodles775: does that also cover the bar not showing up at all on dev?
<noodles775> jtv: huh? it's never shown up on dev afaik?
<jtv> noodles775: oh, that might well be it...  It's just that dev seemed to reserve space for it.
<noodles775> (ie. you need to add the site_message config option to dev.
<jtv> noodles775: sorry for the false alarms, but as I said, this _is_ Monday morning so bad things are supposed to happen.  :)
<noodles775> jtv: np! fwiw, it was a case of a missing test (we'd never had a test for markup in the site_message, although we'd always used it on edge). When the site_message was moved to a template it was auto-escaped.
<jtv> noodles775: tricky!
<jml> noodles775, hello
<jml> noodles775, did you manage to get that branch reviewed?
<noodles775> Hi jml
<noodles775> Yep, it's been through buildbot and waiting for the edge update.
<jml> cool.
<noodles775> jml: sorry, I was assuming that you were talking about the branch above - rather than your permissions branch?
<jml> noodles775, the branch above.
<jml> noodles775, bigjools got to my permissions branch.
<noodles775> great.
 * jml is eagerly awaiting the British morning.
<noodles775> jml :(
<noodles775> it's a public holiday there.
<jml> they are always having public holidays
<jml> I shall write a sternly worded letter to the Daily Mail.
<noodles775> lol
<adeuring> good morning
<lifeless> jml: whats the name of the plugin/command you wrote to find merge, no-shelve, no-edit local branch|trees ?
<jml> lifeless, bzr-removable
<lifeless> is it packaged?
<jml> lifeless, it probably doesn't even work any more
<jml> lifeless, no, it's not.
<lifeless> lp:bzr-removable?
<jml> lifeless, yeah, I think so.
<lifeless> thats doing something
<lifeless> you should upgrade it to 2a
<noodles775> Does anyone know if there is something similar to https://staging.launchpad.net/successful-updates.txt but for edge?
<noodles775> spm: I don't suppose you could check why the daily update of edge hasn't happened yet... or am I too early?
<spiv> I don't think bzr-removable has been updated for the new shelf in bzr.
<jml> lifeless, I should certainly do _something_ to it.
 * wgrant returns.
<wgrant> jml: If you haven't EODed yet, can you have a look at my branch with added tests once I push it?
<jml> sure
<noodles775> spm: Either thanks or don't worry - edge just updated :)
 * wgrant finds the new header a bit boring and colourless.
<wgrant> I stumbled upon some UI 1.0 screenshots yesterday, and they are much more inviting than I remembered or 2.0/3.0 are.
<noodles775> wgrant: I *think* the idea is that projects themselves can emphasize their own branding, like https://edge.launchpad.net/firefox
<wgrant> noodles775: Right, and that's a good thing. But the facet links are all the same. They're boring, and hard to look through.
<noodles775> Is it the lack of colour on the app-buttons specifically that you think is missing?
<noodles775> yep.
<wgrant> Right.
<wgrant> I thought maybe facet colours were being eliminated, but the involvement portlet and titles suggest otherwise.
<lifeless> jml: well, something yes. Anyhow I've just sent you a merge request.
<noodles775> wgrant: yeah, I'm not sure what the reasoning is for removing them in the app-buttons, but guessing it's to avoid clashing with the product branding...
<wgrant> noodles775: I thought that might be it.
<noodles775> I'll check with beuno later - (or maybe he's still around)?
<wgrant> noodles775: But it makes it all really really boring.
<wgrant> noodles775: The bottom has that nice blue fade.
<wgrant> s/fade/gradient/
<noodles775> ... and a nice colourful lp branding (which isn't linked :/ ).
<wgrant> My only other complaint about the branding/watermark is the alignment of the heading wrt. a selected Overview tab.
<noodles775> Heh... so I know about that one - that's a specific request from Martin... that the first-letters line up (rather than the outline).
<wgrant> noodles775: It looks pretty strange.
<wgrant> Hm. although if the -0.5em margin is removed, it looks slightly strange when deselected.
<wgrant> How unfortunate.
<noodles775> wgrant: I agree - when the overview tab is selected - and for that reason the initial version had the background aligned.
<wgrant> noodles775: I guess it will become less of a concern when the Overview tab dies.
<noodles775> wgrant: well, the same issue will then hit the Code tab.
<wgrant> noodles775: right, but that's not the default view.
<noodles775> Right.
<wgrant> And first impressions do count, a lot.
<noodles775> I actually had the -0.5em margin removed only when the first li was selected - so it's possible to have both, but the decision was for the first letters to always align (and I just do what I'm told there ;) ).
<lifeless> micro freshmeat
<lifeless> http://simal.oss-watch.ac.uk/index.html
<jml> lifeless, thanks for the patch.
<jml> lifeless, the shape of it looks about right :)
 * jml needs to figure out what all the words mean though ...
<lifeless> jml: it works. Use it. Love it. Commit it.
<lifeless> oh,and I didn't reduce test coverage.
<jml> lifeless, heh heh
<jml> lifeless, in terms of percentage, no.
<lifeless> \o/
<wgrant> noodles775: Any ideas on how PPA download stats could be shown?
<noodles775> wgrant: sure - let's collect them at https://dev.launchpad.net/SoyuzPPAIndexPage
<noodles775> wgrant: they'll be useful immediately to both the owner and potential users, so we'll probably want some general stats on the index page, and then developer/detailed stats on the /+packages page?
<wgrant> noodles775: I suspect so.
<noodles775> wgrant: what are your thoughts?
<wgrant> noodles775: Not sure. It gets rather complicated, because there are different semantics associated with different sets of binaries. Some sources will produce binaries that will be installed disjointly, while others will always be installed together.
<wgrant> So numbers are probably not meaningful except at the binary level.
<noodles775> wgrant: maybe it would be helpful to identify (on the wiki page - or a separate one even, or even it's own blueprint spec) what we think the target users will want to see, and then adjust that with what's actually possible/meaningful.
<noodles775> Once we've got that info, a lot more people would be able to comment too.
<jml> wgrant, started the run on the mystery meat branch
<noodles775> I'm not sure I caught how you'd get around the download count simply reflecting how often people update a package (without storing ip's etc.)
<noodles775> s/you'd get/you got/
<wgrant> jml: Thanks... I can't see anything even mildly suspicious
<jml> wgrant, me neither.
<jml> wgrant, not being able to reliably land code bothers me quite a lot, so I hope we can get to the bottom of this.
<wgrant> noodles775: Except for daily builds, it should be fairly good, as every user should get every version if it's there for more than a few days.
<wgrant> noodles775: And for daily builds, it is at least a reasonable way to judge between other PPAs and packages.
<wgrant> noodles775: The raw numbers are probably not useful on their own.
<noodles775> wgrant: yes, but what I meant is, I'm assuming a user of a PPA is most interested in how many downloads the ff package in PPA X has - irrespective of the version.
<jml> tee hee
<jml> I don't even have bzr-removable on my laptop :)
<jml> oh wait, it's in my dropbox
<wgrant> noodles775: Right. It should be possible to do that quite reliably for packages updated less frequently than weekly.
<noodles775> Great.
<wgrant> I have most of the backend code done, it just need to be split into threeish branches and tested properly.
<noodles775> whoohoo! great stuff :)
<wgrant> And then needs to be poked into the UI.
<jml> lifeless, also, https://code.edge.launchpad.net/~lifeless/bzr-removable/merge hasn't got branch data :(
<noodles775> wgrant: have you tried bzr pipeline? It's great for multiple-branch-spanning-functionality.
<wgrant> noodles775: I started using it about a week ago for my latest four-branch effort.
<wgrant> It is excellent.
<noodles775> :)
<lifeless> jml: code review merge-by-mail bug I guess.
<lifeless> jml: I replied with the url i did push to
<lifeless> which is ...treeless
<jml> gah pycs and directory removal and bzr
<wgrant> jml: +1
<wgrant> maxb: So, the mysterious soyuz-set-of-uploads.txt failure is pretty much resolved.
<wgrant> maxb: That just leaves the rest of the obvious (largely benign) failures.
<ddaa> lifeless: ping
<lifeless> hi
<ddaa> lifeless: re twitter
<lifeless> hai yes,
<ddaa> the thing that does not work was bin/jstest
<ddaa> as in "the test script freeze to start with, unless I fix an import bug, then it actually runs to completion but reports 3 failed test suites"
<lifeless> ah, nifty. In a sad sad way.
<lifeless> I thought you were referring to something in the web app :)
<ddaa> fix for the import bug: http://python.pastebin.com/m34a9f134
<lifeless> what python version?
<ddaa> does not matter
<lifeless> indeed, I see
<lifeless> uhm, care to file a bug, or would you like meto?
<ddaa> if I were you, I'd just trivial it
<lifeless> Lp
<lifeless> LP's landing process is a bit byzantine
<lifeless> its easier for me to hand it off
<lifeless> :(
<ddaa> wow
<ddaa> sounds to me like the process went a bit wrong if you can no longer trivial one-liners like that
<lifeless> one can
<ddaa> anyway, I do not care about the launchpad bug in itself
<lifeless> I need to do a bunch of setup to be able to
<lifeless> my laptop overflowed, and i've not cut code for lp in a couple years
<ddaa> I was only checking this because I wanted to learn from how launchpad integrated with windmill
<lifeless> ah :)
<lifeless> and have you learnt?
<ddaa> "not very well"
<ddaa> "does not run headless, so is not part of the test suite run by pqm"
<lifeless> anyhow, bug files.
<ddaa> "still requires painful hard-coded waits in some places"
<lifeless> [once cron takes care of mail]
<wgrant> ddaa: Somebody here had it running headless.
<wgrant> ddaa: It's close to buildbot integration.
<wgrant> And hard-coded waits are meant to be removed.
<wgrant> Lots were two weeks ago.
<ddaa> wgrant: THAT sounds very interesting
<ddaa> wgrant: anything else of the same kind that you think I might be interested in knowing about?
<wgrant> ddaa: Mmm. Not sure.
<wgrant> I don't know much more about that stuff.
<ddaa> Fine then. I was just probing.
<maxb> wgrant: Yay! And congratulations on your amazing detective work :-)
<wgrant> maxb: It was more than a bit of a puzzle.
<wgrant> maxb: Particularly as it was buried inside unlogged exception handlers...
<maxb> It does strike me that the LP tests seem to have a tendency to throw away useful logging
<wgrant> It's normally easy to get it back.
<wgrant> Just not in this case.
<lifeless> bug 421840
<mup> Bug #421840: trivial bug in test support <Launchpad itself:New> <https://launchpad.net/bugs/421840>
<lifeless> jtv: ^ if you're around, care to peek?
<jtv> lifeless: peeking...
<jtv> lifeless: greek to me
 * thumper taps fingers while 'make build' runs
<jtv> well, not greek, but...
<lifeless> oh look thumper
<lifeless> that bug is in your tests ;0
<thumper> where?
<jtv> lifeless: afaik we're still not running these tests as part of the regular test suite; mars was working on that
<lifeless> https://launchpad.net/bugs/421840
<mup> Bug #421840: trivial bug in test support <Launchpad itself:New> <https://launchpad.net/bugs/421840>
<thumper> lifeless: I'll add it to my branch
<lifeless> ddaa: ^
<lifeless> thanks thumper
<thumper> lifeless: I even added (ddaa) to the revision message :)
<ddaa> thumper: thanks pal
<thumper> ddaa: np
<jtv> hi carlos!
<jtv> hi ddaa!
<carlos> jtv: morning
<ddaa> jtv: hey buddy, how's life?
<jtv> ddaa: overrated  :-P
<ddaa> jtv: watch you for your attitude pal ;-)
<ddaa> s/watch you/watch out/
<jtv> oh, did it bite someone while I wasn't looking?
<lifeless> thumper: https://edge.launchpad.net/pyjunitxml/+download
<lifeless> search on that page for zz
<thumper> lifeless: heh
<jml> ok, now I can't download keys from the GPG server :(
<lifeless> thumper: I figured you'd like it
<lifeless> in a head slapping kinda way
<deryck> Morning, all.
<thumper> deryck: hi
<thumper> deryck: I'd like to organise a short weekly call with you sometime
<thumper> deryck: do you ever work *normal* hours for your TZ?
<thumper> deryck: 'cause if you did, we could talk during the day time :)
<deryck> thumper, a call would be cool.  And sorry, but I'm always 10-7 GMT.  But I could hang around later one day for a call.
<thumper> may be a plan
<thumper> deryck: lets organise this later though
<thumper> I'm done
<deryck> thumper, ok, cool.  have a good night.
<wgrant> Ooh. lp-dev-utils stuff.
 * jml looks all innocent
<wgrant> Thanks jml.
<jml> np.
<wgrant> jml: Are you running that accursed branch without going headless, or should I try?
<jml> wgrant, I am.
<wgrant> jml: Great, thanks.
<jml> wgrant, it's still going...
<wgrant> jml: Damn.
 * jml stops
<jml> g'night all
<jml> wgrant, I'll let you know how the run goes
<wgrant> jml: Thanks.
<wgrant> jml: I guess ec2test won't really work for non-Canonicalites without some hacking...
<wgrant> It seems to use devpad a bit.
<wgrant> RuntimeError: You don't have access to a test-runner image.
<wgrant> Sad.
<maxb> I have branches of meta-lp-deps ready for pushing. Any thoughts on the preferred bzr format for new stuff?
<wgrant> maxb: 2a
<maxb> I guess the chances of anyone who isn't already branching launchpad itself wanting to hack on meta-lp-deps are vanishingly remote, so makes sense
<wgrant> Right.
<wgrant> And 2.0 is coming RSN.
 * maxb runs some bzr upgrades
<wgrant> Who owns mailing lists? Both https://edge.launchpad.net/launchpad-foundations and https://edge.launchpad.net/launchpad-registry claim to.
<salgado> wgrant, -registry
<wgrant> salgado: That's what I suspected. I see that Foundations lays claim to Registry itself, too.
<abentley> wgrant: The registry team is like a subteam of the foundations team.
<wgrant> abentley: I see.
<wgrant> salgado: You are lord of the Apache log parser, aren't you?
<salgado> wgrant, I wrote the log parser that counts downloads of LibraryFileAliases
<wgrant> salgado: Right. I've refactored and split it so it can be used to count PPA downloads too. Do you want to look at that, or will anybody do?
<salgado> wgrant, I'd be happy to have a look
<salgado> wgrant, how are you going to count PPA downloads? are PPAs served by the librarian now?
<wgrant> salgado: No. There will be (well, I sort of have it mostly working already) an alternate script which works out the archive and package. It uses the same infrastructure as the librarian one, but uses different database stuff and path format.
<salgado> I see
<wgrant> salgado: It would actually be much harder if PPAs were served by the librarian.
<wgrant> salgado: So, https://code.edge.launchpad.net/~wgrant/launchpad/refactor-librarian-log-parser and https://code.edge.launchpad.net/~wgrant/launchpad/refactor-librarian-log-parser-part-2 are what I have now. Can you advise me if I'm on the right track or have done something utterly stupid, as seems likely?
<salgado> wgrant, can you give me an overview of the changes in each branch?
<wgrant> salgado: Sure. Give me a sec.
<wgrant> salgado: So, basically, all the non-LFA-specific code has moved into lp.services.apachelogparser. The first branch does everything except stuff from cronscripts/parse-librarian-apache-logs.py, while the second branch includes that.
<wgrant> salgado: This leaves a fairly tiny amount of code in canonical.launchpad.scripts.librarian_apache_log_parser and cronscripts/parse-librarian-apache-logs.py, and a similarly small amount for PPAs later on.
<sinzui> salgado: can you mark your bugs on https://edge.launchpad.net/launchpad-registry/+milestone/2.2.8 as FIX COMMITTED or move them to 3.0? I want to close the milestone
<salgado> sinzui, done
<sinzui> thanks
<salgado> wgrant, the refactorings in the first branch look good. for the second I'll need more time
<salgado> wgrant, why do you think you've done something stupid there?  are you seeing any test failures or unexpected behaviour?
<wgrant> salgado: The second one has just one unique revision, with a 200 line diff.
<wgrant> salgado: Oh, no, just guessing I've done something horribly wrong.
<salgado> yeah, I'm looking at the 200-line diff here, but it seems to touch all the code of ParseLibrarianApacheLogs.main(), which I don't remember anything about
<wgrant> salgado: Ah, right, that one is a bit big.
<salgado> wgrant, anyway, I'll have a closer look at that.  will get back to you soon
<wgrant> salgado: Thanks. The first one is sane enough to propose a merge?
<salgado> wgrant, I think so
<wgrant> salgado: Lovely. Thanks
<sinzui> barry: bac: I'm going to convert the list of simple page conversions to bugs that the we can assign to ourselves at our own pace.
<barry> sinzui: grat
<barry> sinzui: er, great!
<barry> danilos: when you have a chance, please set up the gnu mailman translation group
<danilos> barry: done, https://translations.edge.launchpad.net/+groups/mailman-translators
<danilos> barry: I've made you the owner (you can reassign to a team or whatever) so you can assign each per-language team
<sinzui> barry: bac: You can see the UI 3.0 update bugs at https://edge.launchpad.net/launchpad-registry/+milestone/3.0
<danilos> barry: we usually have people register teams like mailman-l10n-XX
<danilos> (XX is the language code)
<bac>  sinzui: great, thanks!
<barry> danilos: fantastic, thanks
<barry> sinzui: looking
<barry> sinzui: so i will just claim some bugs and start working on them
<sinzui> barry: yep. I think the forms are just change the layout and add the page_title to the view.
<barry> +1
<flacoste> abentley: say i use bzr co -r '2009-07-27' to create a working tree at a particular point in the past, how do I update the working tree to say its state one week later?
<flacoste> bzr co - r '2009-08-03' fails with .bzr already exists
<maxb> I want to give ownership of some branches to ~launchpad. If I post to launchpad-dev@, I guess there are some people on the launchpad dev team with the relevant super admin powers to take the branches?
<flacoste> maxb: yep, you can't give it away yourself?
<maxb> You can't give to a team you're not a member of
<maxb> Which is fair enough, if inconvenient in this instance
<flacoste> maxb: ok, we need to ask a LOSA to do it then, what branches?
<maxb> I was thinking that perhaps being a ~bazaar-expert would be enough
<maxb> abentley could tell us :-)
<abentley> flacoste: there isn't a way to do that.  update -r is a missing feature.
<flacoste> great
<wgrant> abentley: revert -r?
<maxb> not quite the same
<maxb> that would change the file text, but not what revision the tree thought it was related to
<sinzui> bac: I see you changes a bug to Medium. What does that mean?
<wgrant> True.
<flacoste> revert -r seems to work
<maxb> flacoste: but now try bzr status
<flacoste> well, for my purpose, i simply care about the content of the tree
<bac> sinzui: that it's higher than a low.  sorry, i forgot our ban on mediums
<maxb> I recall having a long discussion on #bzr about why update -r was missing, and it being something to do with the confusion about what it should mean for bound branches
<sinzui> bac: you you are committing us to fixing the bug by a certain time, it is High. otherwise it is low
<bac> sinzui: i can change it to high if you want, but it's in review now and will be fix committed soon
<bac> high it is
<flacoste> revert will work fine for my purpose here
<maxb> abentley: does being a ~bazaar-expert give you the powers to reassign some branches away from me to ~launchpad?
<sinzui> bac: Bugs uses Medium to mean we want to fix it bug we wont commit to it. I think that stinks because it implies Low is "Someone wants my to fix their bug but I do not wan to."
<gary_poster> cprov: hey.  is poppy soyuz-related?  If so, am I right in assuming we still use it?
<maxb> gary_poster: I thought it was the upload ftp server?
<cprov> gary_poster: yes, it is.
<wgrant> Yes, it's... rather critical.
<gary_poster> :-) ok
<gary_poster> ok thanks all
<abentley> maxb: I am not sure.
<wgrant> gary_poster: Porting it off ThreadedAsync?
<gary_poster> yeah
<gary_poster> in spare cycles ;-)
<henninge> What's wrong with buildbot? The page won't load ...
<deryck> danilos, ping.
<danilos> deryck: hi
<bac> sinzui: on the product index page, the in-line programming languages picker is broken.
<sinzui> bac: I saw
<bac> ok
<barry> danilos: ping
<danilos> barry: pong
<barry> bac: there's a bug open on lazr-js about that
<barry> it's a lazr-js bug
<bac> barry: ok.
<barry> danilos: hi.  will you be around for a little while to talk about your page titles email?  i am tasked with trying to figure it out and would like to pair with you on this
<danilos> barry: not for a lot longer, and I believe I should have a call with flacoste in a few minutes
<danilos> barry: let me see if flacoste is around for a call (I know he was in the UI call), and if we are not having a call, we can do it instead
<barry> danilos: okay, it can fill you in with the basics.  i'll figure out what i can and we can chat tomorrow.
<barry> s/it/he
<danilos> barry: ok, cool
<flacoste> danilos: i'm free!
<danilos> flacoste: cool :)
<barry> danilos: i am going to get some lunch, flacoste and you can fill me in on anything you decide and i'll read the scrollback when i return
<danilos> barry: ok, though it'll mostly be phone call :)
<barry> cool
<sinzui> noodles775: ping
<rockstar> sinzui, jtv, deryck - Could I get your 3.0 conversion progress reports by email?
<rockstar> deryck, I picked you because intellectronica is on leave (and I'm sending the email out for him)
<deryck> rockstar, np.
<jtv> rockstar: mwhmyumsoonmhwmsI'mthrough<smack>dinnermyom
<rockstar> jtv, :)
<flacoste> abentley, rockstar: as members of ~bazaar-experts can you reassign ownership of branches to arbitrary team?
<abentley> flacoste: I don't know.
<flacoste> ok, i'll ask a LOSA anyway
<maxb> flacoste: they can, I checked in a dev launchpad instance
<jtv> rockstar: yhm
<rockstar> abentley, hey - does hitchhiker have a way of killing and recreating the .bzr directory of a branch?
<rockstar> abentley, I need to restart an import branch, but the user has made it a development focus, so I can't delete and try again.  I was thinking maybe I'd use hitchhiker to be clever.
<abentley> rockstar: It can delete the .bzr directory, it can't create a new one.
<abentley> rockstar: but bzr can.
<rockstar> abentley, hm, as a bzr-expert, can I just init the path then?
<abentley> rockstar: Yes.
<rockstar> abentley, hm, when I try to get into the branch, it tells me the dir doesn't exist...  wtf?
<rockstar> Oh wait, I think it's trying to get into the mirrored area.
<rockstar> Nope that wasn't it.
<abentley> rockstar: This is an import branch?  I don't think you have access to those.
<rockstar> abentley, yeah, that might be it.
<rockstar> abentley, although that seems kinda odd.  I can mess with people's owned branches, but I can't mess with the ones that are owned by a team I'm a member of.  :)
<rockstar> Anyway, I'll just have the user remove the series link, and then do the change through the UI.
<salgado> sinzui, I'm thinking of doing the person-edit* pages now.  is that ok or has someone already taken them?
<sinzui> salgado: take them all
<sinzui> salgado: I broke it into smaller bugs: bug 421975, bug 421976
<mup> Bug #421975: Update identity/location pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421975>
<mup> Bug #421976: Update keys/wikiname pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421976>
<salgado> sinzui, cool, I'll assign them to me
<sinzui> salgado: we can avoid creating links between the edit pages if we add the edit icon to the information on the +index page.
<sinzui> salgado: beuno really wants us to do that. It might be awkward if we cannot add it to the 2.0 +index page
<salgado> sinzui, ok, I'll give that a try, but '2.0'?
<sinzui> salgado: the user page is still 2.0
<sinzui> salgado: I see we have links to the edit pages in now...maybe we should keep them. When we update the +index to 3.0, we can remove them from the edit pages.
 * sinzui thinks that is a fastest way to update the pages
<salgado> that sounds like a plan
 * barry is doing the /people/+new{person,team} pages
<salgado> sinzui, currently it's only the +edit page that has links to other pages. should I add them to +edit[irc/jabber/etc]?
<sinzui> salgado: we could create a menu, but I think that is work we should avoid. We use a related pages menu on an edit page if it is not possible to get to the form from the +index page
<sinzui> salgado: branding is only form that we cannot make a link to from an +index page.
<sinzui> salgado: that is why is suggested that we may want to update the 2.0 +index page
<sinzui> salgado: You could pick something else from https://edge.launchpad.net/launchpad-registry/+milestone/3.0 and we take take up the person edit pages /after/ the user page is 3.0
<salgado> that might be a better idea
<salgado> barry, I think you'll want to assign bug 421972 for you, then ;)
<mup> Bug #421972: Update merge pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421972>
<salgado> sinzui, I'll take bug 421966, then
<mup> Bug #421966: Update contact, annoucement, driver pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421966>
<sinzui> fab, you'll be done is 2 hours
<sinzui> maybe less
<barry> salgado: i grabbed bug 421974
<mup> Bug #421974: Update new person/team pages to UI 3.0 <story-ui-3> <Launchpad Registry:In Progress by barry> <https://launchpad.net/bugs/421974>
<salgado> barry, duh. I misunderstood you and were thinking you'd be doing the /people/+*merge pages. nevermind me
<barry> :)
<salgado> sinzui, are all form pages supposed to have the <h2 content="context/title"> before the page's main heading, like in http://people.canonical.com/~salgado/double-heading.png ?
<sinzui> salgado: That is a header issue that barry is going to work on, and it related to bug 417089
<mup> Bug #417089: the base-layout heading-slot should not render any heading for IPrimaryContext <story-ui-3> <Launchpad Foundations:Triaged by sinzui> <https://launchpad.net/bugs/417089>
<sinzui> salgado: ignore it since we have more than 50 pages that do exactly what you are seeing
<salgado> ok, cool
<barry> right.  i'm going to work on that as soon as i get this other bug into review (should be within the hour)
<barry> danilos: bug 422150 describes the other problem i mentioned in our skype
<mup> Bug #422150: Not all series are displayed when setting up translation branches <Launchpad itself:New> <https://launchpad.net/bugs/422150>
<danilos> barry: cool, thanks!
<danilos> barry: don't forget the "john is the uploader" bug as well :)
<barry> danilos: that one's in the email i just sent, following up to that thread
<barry> hey is anybody having trouble resolving the dns for launchpad.net?
<danilos> barry: excellent, always a step ahead, thanks!
<barry> :)
<bac> hi sinzui.  can i have a mid-imp chat with you re: bug 422128?
<mup> Bug #422128: Making a private team the owner of a project fails if the project has a ProductRelease <Launchpad Registry:Triaged by bac> <https://launchpad.net/bugs/422128>
<barry> hmm.  it seems happy again.
<barry> sinzui: i guess i should steal the assignment of bug 417089
<mup> Bug #417089: the base-layout heading-slot should not render any heading for IPrimaryContext <story-ui-3> <Launchpad Foundations:Triaged by sinzui> <https://launchpad.net/bugs/417089>
<sinzui> barry: I'll paste a conversation into the bug
<sinzui> barry: I updated the bug
<barry> sinzui: thanks
<sinzui> bac:  sorry, I got distracted. Yes talk about the bug now
<bac> hi sinzui
<sinzui> bac: skype?
<bac> ok
<EdwinGrubbs> cprov: ping
<cprov> EdwinGrubbs: pong
<EdwinGrubbs> cprov: I'm trying to make a team the maintainer of a sourcepackagerelease, so that I can test the showing/hiding of the Maintained Packages link in the portlet. Is there a test helper function for that?
<cprov> EdwinGrubbs: not specifically, but you can create new SPRs using SoyuzTestPublisher.getPubSource(maintainer=a_person)
<EdwinGrubbs> thanks
<cprov> EdwinGrubbs: np
<beuno> EdwinGrubbs, has the team page landed yet?
<EdwinGrubbs> beuno: no, I'm working on tests for it.
<beuno> EdwinGrubbs, super
<gary_poster> flacoste: someone looked at lp this weekend and was a bit horrified to discover that not only did our JS tests not pass, they didn't even run because of a trivial bug (http://python.pastebin.com/m34a9f134).  This highlighted to me that mars' absence is kind of a big deal, especially with 3.0's focus.
<gary_poster> AFAIU, we don't have automated tests, and we don't have any chance of getting a non-red buildbot for this.  I don't think I have anybody on foundations that can work on this, but it feels a bit urgent.  Do you have any thoughts?
<gary_poster> (we don't have automated tests that are run, I mean, apparently)
<flacoste> gary_poster: i don't unfortunately
<gary_poster> flacoste: ok.  are we getting further and further in debt because of that right now, or is custom JS halted ATM because of the template changes?
<flacoste> gary_poster: kind of the latter
<flacoste> everyone is working on template conversion
<flacoste> so we shouldn't be adding JS much at this point
<gary_poster> flacoste: ok.  I guess I'll bring up with team leads that problem, and ask that if any JS-aware dev starts to move from templates to new JS things, they move instead to fixing the tests.  I think that should be the next JS thing to happen, with or wothout mars
<flacoste> i agree
<gary_poster> ok cool
<beuno> flacoste, what are you using to create the burndown chart for the UI conversion?  I was thinking about doing a per-week checkout and creating a report per week
<beuno> thought about all kinds of fancy things, but considering we're 3 weeks to go...
<flacoste> beuno: it's all taken care of
<flacoste> beuno: i have the data
<beuno> flacoste, super
<flacoste> and updating the script and HTML template to show it
<flacoste> beuno: i'll send you an email with update instructions
<beuno> flacoste, perfect, looking forward to seeing what it looks like  :)
<flacoste> beuno: actually, if you are not going away soon, i'll ask you for feedback on the actual charet
<beuno> flacoste, I am not
<beuno> have TONS of things to do
<beuno> *TONS*
<beuno> caps and bold
<flacoste> which i think means you are locked to your computer for a while...
<maxb> So, now we have branches for launchpad-dependencies, I want to submit some changes - what's the right thing to do with debian/changelog here - completely update it ready to build the new version? Leave it saying UNRELEASED?
<mwhudson> maxb: are you asking for policy or advice?
<mwhudson> i'm pretty sure there's no policy yet...
<maxb> either? both?
<thumper> beuno: I really need to talk to you about breadcrumbs soon
<thumper> beuno: I have a call with abentley now
<thumper> beuno: perhaps in an hour and a half maybe?
<beuno> thumper, sure. One hour sounds better, but we can see what happens  :)
<flacoste> beuno: http://people.canonical.com/~flacoste/conversion.html
<thumper> flacoste: nice!
<flacoste> thumper: well, arguably...
<flacoste> kind of show in our face that we aren't going to make it
<thumper> flacoste: I've submitted a CP request for the branch listing timeouts in production
<flacoste> ok, i'll look at it in an instant
<thumper> flacoste: ta
 * thumper waits for abentley to call
<flacoste> maxb: i'd vote for "ready to build"
<flacoste> beuno: i sent you an email with the update instructions
<maxb> flacoste: ok, and "bzr tag" the new revision too?
<maxb> I guess I could even put the package to my PPA, but then if the changes are not accepted as-is, I have a version in my PPA which I can't delete and revise
<rockstar> flacoste, your page is great.  I fear we have too many people setting up this graph.  It might be nice to have one canonical page (no pun intended)
<flacoste> rockstar: beuno is the canonical source now
<rockstar> flacoste, he doesn't have your super sexy graph.
<flacoste> rockstar: he will, my copy is just a demo, it's not updated
<flacoste> maxb: about bzr tag, i have no idea how this works, so whatever you feel makes sense
<flacoste> maxb: regarding you publishing some version, i don't think it's a big deal. if changes are not accepted, we'll see them in the changelog
<rockstar> flacoste, okay.  We should maybe think about putting it somewhere official looking.
<flacoste> maxb: or we can use UNRELEASED until it's merged approved
<barry> jml: please make --headless go faster :(
<maxb> That might be best ... I put the branch up as UNRELEASED, file a merge proposal, wait for approval, then do one more commit changing UNRELEASED->karmic, do the bzr tag, upload the source package to my PPA, and then a ~launchpad member just has to pull/push the changes and copy the packages between PPAs
<beuno> flacoste, cool, thanks
<beuno> I will update now
<flacoste> maxb: that sounds good
<barry> sinzui: ec2 --headless is too slow so i'm crossing my fingers and jfdi
<rockstar> barry, I think you meant to direct your statement about --headless to mwhudson
<lifeless> mwhudson is not headless!
<sinzui> barry: your changes should have only affected the doc/browser/stories tests. if something fails, I think you have justification to remove the offender
<barry> i thought i saw a bug from jml on --headless, but maybe it was mwhudson (or maybe it's assigned to him)
<barry> sinzui: yep.  worst case, backing out the button change is easy
<mwhudson> yes, --headless should go faster
<mwhudson> unfucking buildbot seems higher priority right now though
<beuno> flacoste, looks like we're doing a great job at deleting templates
<flacoste> beuno: well, not enough i think, we still have more than 60% to go and 3 weeks left
<flacoste> we need to make a serious dent in that curve this week
<mars> :)
<mars> nice burndown chart.  it really helps visualize the progress.
<gary_poster> mars: whoa! hey man!  how goes it?
<mars> gary_poster, busy :)
<gary_poster> heh, I bet
<mars> nothing much else to say, really.  Just normal newborn stuff, but going through 30 nappy changes a day, instead of the usual 12.
<gary_poster> lol
<gary_poster> cool, well, congratulations.  hope you send some pics to warthogs eventually
<mars> names first, then pictures :)
<gary_poster> heh, ok, I'll agree to that ordering
<beuno> flacoste, http://people.canonical.com/~beuno/conversions.html
<mars> everyone is happy and healthy, btw.
<gary_poster> excellent
<jml> :(
<jml> my network connection went down while running tests on an instance.
<gary_poster> maxb: btw, I have a branch that cleans out the last traces of the old zope branch, ~gary/launchpad/poppyasync.  We've had more than the usual share of spurious failures today, so I'm retrying an ec2test run.  Should be in either within the next few hours, or the next ~16 or so.  I tried to make some progress on the karmic/buildout problem but getting things running on karmic is still not done, and I couldn't focus on it exclusively.
<gary_poster> But anyway, I'm working on these things.
<gary_poster> bye all
<rockstar> beuno, WOOT!
<thumper> rockstar: skype?
<rockstar> thumper, here
<thumper> rockstar: skype doesn't think so
<rockstar> Skype == retarded
<Ursinha> lol
<barry> looks like we're in testfix
<mwhudson> barry: shouldn't be
<rockstar> We never SHOULD be in testfix
<barry> mwhudson: hmm. i just had a branch fail because it didn't match the testfix regexp
<mwhudson> barry: hmm
<jml> hello
<barry> mwhudson: otoh, r9280 seems happy
<barry> (in buildbot)
<mwhudson> barry: jtv's branch seems to be being processed
<mwhudson> barry: regexps are a notoriously wonderful user interface component, are you sure pqm is telling you it's in testfix?
<barry> mwhudson: i just deleted the email :(  but i resubmitted :/  the regexp had no ui= or r= match that i could tell
<barry> mwhudson: let's see what happens with jtv's and my branches
<mwhudson> The size of the diff (15093 lines) is larger than your specified limit of 1000 lines
<mwhudson> oof
<mwhudson> barry: can you say [ui=edwin, sinzui] yet?
<sinzui> I do not think you can
<sinzui> ui=<beuno|none|rs>
<barry> really?  istr doing that in a previous branch
<barry> it would be crazy if we can't since we have ui mentats
<sinzui> barry: without a request to a losa to update the *provided* RE, it wont happen
<barry> are there any losas around right now?
<spm> barry: no we're all hiding. sorry. :-P
<sinzui> barry: first we need to get the current RE
<mbarnett> barry: none, none whatsoever!
<barry> spm: can you give us the current regexp for pqm?
<barry> you guys... :)
<spm> arrrgghhh!!! stop typing 'bzr' when you want 'barry'
<spm> barry: sure. db-devel? devel?
<barry> spm: devel
<spm> barry: commit_re=(?is)^\s*(:?\[testfix\])?\[(?:release-critical|rs?=[^\]]+)\]\[ui=(?:.+)\]
<barry> mwhudson, sinzui ^^ see?  ui=.+
 * jml tries running the tests for this branch again.
<mwhudson> barry: cool
<barry> yay!  well, let's see what happens
<mwhudson> jtv's change landed
<barry> cool
 * jml files bug 422274
<mup> Bug #422274: PQM regex rejection emails are confusing <build-infrastructure> <Launchpad Foundations:New> <https://launchpad.net/bugs/422274>
<thumper> beuno: ping
<beuno> thumper, pong
<thumper> beuno: call?
<beuno> thumper, in 10
<thumper> ok
<thumper> flacoste_afk: how close are we to python 2.5 now?
<beuno> thumper, ready
<maxb> thumper: http://dev.launchpad.net/LaunchpadOnKarmic has a list of known test failures
<barry> success this time
<beuno> thumper, https://wiki.canonical.com/Launchpad/UI/Navigation
<mwhudson> barry: oh, before, it's possible that the buildbot-poller hadn't noticed that we were out of testfix yet
<mwhudson> spm: how often does the buildbot-poll.py script run?
<barry> mwhudson: that's what i was thinking
<spm> mwhudson: every 5
<mwhudson> spm: thanks
<mwhudson> MOAR EVENTS PLS
<lifeless> mwhudson: so
<lifeless> mwhudson: there is a fix branch script; that I've been nagging you to run :)
<lifeless> mwhudson: and as Andrew points out, users cannot fix their mirrored branches themselves - we overlooked that in the initial work.
<mwhudson> lifeless: on which branches?
<lifeless> bug 354036
<mup> Bug #354036: ErrorFromSmartServer - AbsentContentFactory (unfixable by users) error when  pulling a branch from the mirrored area <hpss> <lp-needs> <Bazaar:Fix Released by spiv> <Bazaar 1.13:Fix Released by tanner> <Launchpad Bazaar Integration:Fix Committed> <bzr (Ubuntu):Confirmed> <https://launchpad.net/bugs/354036>
<mwhudson> lifeless: it sounds like it's going to take a LOSA to run this script
<mwhudson> lifeless: is there much value in going through me?
 * mwhudson puts his build engineer hat on so hard it cover his eyes
<lifeless> you could add the ability for people to say 'please remirror'
<lifeless> which would be generally useful as a in-extremis knob.
<mwhudson> yes, there's a bug about that already
<mwhudson> which i am also not going to work on this month
<lifeless> ok
<lifeless> then its an rt tissue
<mwhudson> yeah
<beuno> barry, still around?
#launchpad-dev 2009-09-01
<barry> beuno: i am
<beuno> barry, want to have that heading call?
<barry> beuno: sure!
<barry> beuno: give me 2m
<beuno> barry, give me 10   :)
<barry> beuno: okay, but if jane gets back i'll have to make dinner :)  i'm on skype now, ring me when you're ready
<Ursinha> hey rockstar, do you have half a minute?
<rockstar> Ursinha, sure, what's up?
<beuno> barry, now?
<barry> beuno: yep
<beuno> barry, https://code.edge.launchpad.net/~sinzui/launchpad/official-portlets/+merge/10946/+review?claim=launchpad&review_type
<beuno> barry, https://edge.launchpad.net/bzr
<beuno> barry, https://wiki.canonical.com/Launchpad/UI/Navigation
<beuno> barry, https://bugs.edge.launchpad.net/bzr/+bug/165293
<mup> Bug #165293: collisions through uploading same-named .pack files not handled correctly <packs> <Bazaar:Confirmed> <https://launchpad.net/bugs/165293>
<lifeless> beuno: ?
<beuno> lifeless, it's my example bug  :)
<lifeless> oh kk
<barry> beuno: https://launchpad.dev/people/+newteam
<barry> beuno: http://people.canonical.com/~barry/screen.png
 * barry -> dinner
<flacoste> thumper: actually, we should be pretty close
<flacoste> thumper: zope is upgraded as is all of our lazr deps
<flacoste> thumper: gary wants to tackle the chameleon branch before finishing the 2.5 upgrade
<flacoste> thumper: but you might try it, it might actually just work
<flacoste> thumper: python2.5 bootstrap.py && ./bin/buildout
<thumper> flacoste: I'm happy that we're close
<flacoste> yeah, gary wanted to tackle the chameleon integration before it bit rots more
<flacoste> and didn't want to risk having it break more because of other deps upgrade
<flacoste> besides, a nice performance boost on rendering would be nice for 3.0
<beuno> flacoste, any udeas why I'd get this in karmic?  https://pastebin.canonical.com/21664/
<bac> salgado-afk: i submitted a patch for the typo poolie reopened as bug 156919
<mup> Bug #156919: More information on where to download files needed <Launchpad Registry:Confirmed for salgado> <https://launchpad.net/bugs/156919>
<bac> beuno: no python-psycopg2 installed or installed against the wrong python?
<beuno> bac, it's ssntalled
<beuno> installed
<beuno> how do I know against which version of python?
<beuno> ah, I think karmic's version is newer, so it's probably taking over the PPA one
<bac> beuno: you can dpkg -L python-psycopg2 to see
<bac> beuno:  i've got 2.0.8-0ubuntu3~launchpad1
<beuno> bac, you are tight
<beuno> 2.5 and 2.6
<beuno> DAMN
<beuno> you are *right*
<bac> i *am* cheap, if that's what you meant
<beuno> heh
<bac> beuno:  this early adoption thing -- is it serving you well?  didn't you go through this about 6 months ago?
<beuno> bac, it is not. But I'm 50% massochist.
<flacoste> beuno: maxb should have this fixed soon, he's really on top of the dependencies
<flacoste> thanks to him, we can expect the karmic updates to be a lot much smoother than Jaunty
<flacoste> bac: what are you doing on karmic?
<flacoste> already, i mean?
<bac> flacoste: dude, i'd be on feisty if i could
<bac> i'm *not* on karmic -- i'm still all jaunty
<flacoste> ah, i see
<flacoste> you were speculating
<bac> i was just poking fun at beuno
<flacoste> good guess!
<beuno> flacoste, it's fixed
<beuno> but something else is broken now
 * jml frowns
<jml> so here's the thing
<jml> pocket-based upload permissions have nothing to do with the uploader.
<lifeless> who do they have to do with
<lifeless> also, you're @ cafÃ©?
<jml> I am.
<jml> no one. they are a function of the archive and the distroseries
 * jml grumps at soyuz and timezones
<lifeless> jml: why is this? surely fine grained permissions will give individual users privilege via team-or-something inheritance?
<jml> lifeless, I have no idea what you mean, I'm sorry.
<lifeless> jml: ok
<lifeless> well I'll be there about 10 past 12
<jml> heh
<lifeless> if you want to discuss in more detail before ring me
<jml> thanks
<wgrant> jml: Is canUploadToPocket so hard to use?
<jml> wgrant, it's not hard to use
<jml> wgrant, the question is whether it's relevant.
<wgrant> jml: Not for normal permission checking. I think it's only relevant for whether the branch should be completely immutable.
<mwhudson> time for nommage
<wgrant> Which goes beyond 'does this user have additional privileges', as it affects even the branch owner.
<jml> wgrant, yes, this is what I'm thinking.
<wgrant> abentley: Can you forward me the failure email?
<jml> wgrant, also, while james_w is still uploading branches, I can see a pocket check getting in the way
<wgrant> jml: Crap, you are right.
<wgrant> How awkward.
<jml> wgrant, yes.
<wgrant> Sounds like you need a new distroseries or even distribution column.
<jml> well, at the least, I think I want to talk to james_w
<wgrant> jml: Actually, maybe you don't need to.
<wgrant> jml: Because the branches are probably only made official at the end.
<wgrant> So it might not matter that they become immutable.
<jml> good point.
<jml> but even that means changing my plans a little
<wgrant> Why?
<jml> since I was going to change the permissions for "make official" from what they are now to "own the branch & can upload"
<wgrant> 'own the branch' is probably wrong.
<jml> now, it would mean "(own the branch & can upload) OR are local deity"
<jml> wgrant, why?
<wgrant> jml: Mmm, I guess it's a good idea to require that if it means that privileges will be revoked.
<jml> wgrant, well, the thing is that it's unacceptable to allow random uploaders to expand branch permissions
<wgrant> But ISTM that launchpad.Edit in general shouldn't be revoked based on IDS.canUploadToPocket. You probably want to still allow editing of the description and all that.
<abentley> wgrant: There wasn't a failure email.  I ran it on my own box before submitting it.
<wgrant> And I don't think the officialness setting should be forbidden based on that either.
<wgrant> abentley: Ah. Can you then advise me which tests failed?
<wgrant> Or shall I run it myself?
<abentley> wgrant: I don't remember which tests failed.
<wgrant> abentley: OK.
<jml> wgrant, in an ideal world, I think officialness should be set through a handshake
<wgrant> jml: Probably. But that's overcomplex.
<jml> I agree :)
<wgrant> jml: (owner & uploader) | ~ubuntu-branches is probably the best you can do.
<jml> wgrant, yeah, that's what I'm leaning toward.
<jml> (where ubuntu-branch == local deity)
<wgrant> jml: But it would have been reallly nice to kill off ~ubuntu-branches.
<jml> yes, it would have been nice
<jml> maybe we still can.
<jml> but one step at a time.
<jml> and I think the best next step is to change the bug nomination logic to use my newly extracted permissions function
<jml> since neither cares about pockets
<flacoste> hey jml!
<jml> flacoste, hello
<jml> sorry I missed your privmsg
<jml> xchat hates me
<EdwinGrubbs> sinzui: ping
<sinzui> Hi EdwinGrubbs
<wgrant> jml: Oh! Did anything come of my branch?
<EdwinGrubbs> sinzui: it looks like all the story-ui-3 registry bugs are triaged. Do I just steal one and hope that they haven't started on it? I was going to work on it some tonight so I can be ready for reviews in the morning.
<sinzui> EdwinGrubbs: Triaged means it is acknowledged an waiting for someone like your self to take and mark in progress.
<sinzui> EdwinGrubbs: All the Update...UI 3.0 that are triaged are available: https://edge.launchpad.net/launchpad-registry/+milestone/3.0
<wgrant> jml: Was that the instance during which the network died?
<jml> wgrant, yes. it was
<jml> wgrant, sorry, I'm on the phone right now
<jml> wgrant, I've fired off another ec2test instance for merging
<wgrant> jml: Isn't that just going to die silently again?
<jml> wgrant, who knows!
<wgrant> We shall see. Thanks.
<wgrant> Can somebody please ec2test lp:~wgrant/launchpad/refactor-librarian-log-parser? Apparently there are failures, but I can't run the whole test suite sufficiently reliably here at the moment.
<jml> wgrant, hey
<jml> wgrant, I can do that.
<wgrant> jml: Thanks.
<jml> wgrant, have you got any email about the structural subscription branch?
<wgrant> jml: No :(
<jml> ok
<jml> the instance isn't running
<jml> this is obviously a critical bug in the test suite
<jml> ec2test, rather
<wgrant> Well, ec2test is meant to terminate the instance in situations where the test suite dies.
<wgrant> So it's doing exactly what its meant to, however useless that may be.
<mwhudson> it's also meant to tell you what happened before that
<wgrant> Hm, so it is.
<wgrant> Hadn't looked at ec2test-remote in detail.
<wgrant> So it is being bad after all.
<jml> wgrant, any situation where you can run tests and not get results reliably is critical, imo
<jml> wgrant, otherwise, wtf does 'test' mean?
<wgrant> jml: 'no results' == 'uberfail'
<wgrant> jml: Do you have any idea how far through it died?
<jml> wgrant, no, none at all.
<jml> wgrant, basically, I threw it over the wall, and nothing came back.
<wgrant> jml: Right. Damn.
<wgrant> jml: There's nothing that can really be done besides running it headless and hoping the connection survives, is there?
<jml> wgrant, running headful, itym
<wgrant> jml: That one.
<thumper> For anyone who is interested: https://dev.launchpad.net/Code/Review/Plans
<jml> thumper, thanks!
<jml> mwhudson, how much pain would be involved in setting up a buildbot that ran with --coverage
<wgrant> Wouldn't that take like forever to run?
<jml> wgrant, yes.
 * jml files some bugs
<mwhudson> i think a rule of thumb is that --coverage takes 5x as long as normal
<mwhudson> jml: setup, not much pain
<spiv> Fortunately, every test that uses subprocesses will seem 5x as fast with --coverage ;)
<mwhudson> spiv has a point
<mwhudson> you could probably get around that by running the tests with a mangled ./bin/py
<spiv> Or just change site.py in the ec2 image...
<jml> well, I'm happy to have an incomplete coverage report to start with
<jml> the report is still interesting even if it ignores coverage via subprocess execution
<spiv> Yes.
<spiv> Plus, it adds motivation to fix tests that use subprocesses :)
<jml> or delete code only used in subprocesses :)
 * mwhudson is more or less done for the day
 * jml is off
<lifeless> so coverage times are distorted
<lifeless> measure time and coverage separately.  KThanks.
<lifeless> mwhudson: you can get some rad coverage using lspro
<lifeless> f
<jml> mwhudson, you've probably EODd, but https://bugs.edge.launchpad.net/launchpad-foundations/+bug/5814 is interesting
<mup> Bug #5814: want to know breakdown of test run time by area of development <build-infrastructure> <test-system> <Launchpad Foundations:Triaged by flacoste> <https://launchpad.net/bugs/5814>
<mwhudson> jml: four figure bug!
 * mwhudson is not here
<jml> mwhudson, yes!
<henninge> Hallo noodles775!
<henninge> Moin al-maisan!
<noodles775> Hi henninge and al-maisan :)
<al-maisan> good morning noodles775 and henninge :)
<henninge> noodles775: I am for the first time trying to use a sub-view. I see two different approaches.
<henninge> noodles775: Register it as a page in zcml, including the template, and call that page from the enclosing template (@@,,,)
<henninge> noodles775: or not register in zcml and tell the sub-view in-line about its template using ViewTemplateFile (I think it is called) and then just calling the view from TAL (structure ...).
<henninge> noodles775: are you with me? ;_)
<henninge> ;)
<noodles775> yep
<henninge> noodles775: is there any preference on one of the approaches?
<noodles775> So, afaik, at least in soyuz code we usually go for the former (ie. keeping the configuration in the zcml - not that we like zcml, but for consistency ;) ).
<al-maisan> hmm .. zcml .. my love ;)
<noodles775> I'm not sure of other advantages/disadvantages, wgrant, what are your thoughts?
<henninge> I was wondering if using the @@ approach engages more zope machinery than necessary and there might be a performance penalty.
<henninge> I am using this in a repeat for a list of translatable strings.
<noodles775> yeah, that would be very interesting to know.
<noodles775> one tic
<noodles775> henninge: why isn't a simple tal:repeat with a template-snippet/macro enough in that case?
<wgrant> henninge: I think keeping it in ZCML is better, but I'm not thoroughly opinionated on that yet.
<wgrant> Er, noodles775 ^^
<noodles775> Why do you need a separate view for each repetition?
<noodles775> wgrant: thanks.
 * noodles775 looks for similar example for henninge 
<wgrant> noodles775: Bug comments, maybe.
<henninge> noodles775: well, the display is a bit more complex than just a string.
 * wgrant looks.
<wgrant> henninge: A tal:repeat can contain anything, though.
<henninge> true
<noodles775> henninge: sure, I was thinking more of printing a table row for each repetition...
<noodles775> henninge: but if you need some complicated manipulation of the repeated item's attributes, then yeah, that's a pain in the template (and is usually where we'd use a view).
<henninge> noodles775: yes, there is more to it.
<adeuring> good morning
<henninge> Hi adeuring!
<noodles775> henninge: so in templates/archive-index.pt
<adeuring> hi henninge!
<noodles775> hi adeuring :)
<adeuring> hi noodles775!
<henninge> noodles775: I am not sure I want to put this all into the enclosing view
 * henninge looks at that
<noodles775> henninge: you'll see at the bottom <metal:package-list use-macro="context/@@+macros/source-package-list" />
<wgrant> See also bugs/templates/bugtask-index.pt just after line 300.
<wgrant> That might be more like what you want.
<noodles775> yeah, that's a good example, so the original view is providing an enum of structured information that can be easily repeated.
<henninge> wgrant: yes, thanks
<wgrant> Great.
<noodles775> s/enum/iterable
<henninge> noodles775: thanks, too
<noodles775> np! /me loves learning too :)
<henninge> ;)
<noodles775> henninge: jfyi, the reason we use a macro on that archive template is because we re-use the source-package-list in other places too.
<noodles775> I'm not sure how expensive the context/@@+macros/macro-name is though.
<noodles775> It's not creating a new view though - it just uses the current view (similar to the new navigation menus, afaics)
<henninge> noodles775: yes, I was just trying to figure out how that works (from bugtask-index.pt, though).
<henninge> noodles775: but I'll go with an extra page and view for now.
<noodles775> yep.
<henninge> sub-page and sub-view
 * henninge needs to get results ... ;)
<noodles775> Hmm... I should have said dis-similar to the new menus, they do use their own sub-views too (NavigationMenuTabs).
<wgrant> jml: ec2test won't have submitted that branch, will it?
<wgrant> jml: (it succeeded -- abentley must have caught devel at a bad time)
<jml> wgrant, it doesn't say it submitted
<jml> hmm.
<jml> I must have forgot the -s
<wgrant> jml: Well, I didn't ask you to submit.
<wgrant> jml: Because I believed it would fail.
<jml> oh right.
<jml> wgrant, do you want me to submit?
<wgrant> jml: So, can you submit it please? "[r=abentley][ui=none] (wgrant) Refactor the librarian apache log parser so pieces can be reused for PPA download counts."
<jml> wgrant, can do
<wgrant> jml: Thanks.
<jtv> Why does our CSS make labels non-wrappable?  It wreaks total havoc with our radio buttons!
<noodles775> jtv: I assume it makes sense in the general case - it's been in the style.css for a long time (r6771). But you can always provide a more specific rule for your exception?
<jtv> noodles775: I'm not sure how I'd do that cleanly though, since these are LaunchpadForm-generated.
<jml> wgrant, it's in PQM now
<jml> statik, are you actually around?
<wgrant> jml: Thanks.
<noodles775> jtv: in your template, if the call to the form macro is within a div with a class of your choice, you can then style it easily. If you're using generic-edit etc., then I'm not sure :/
<noodles775> Hrm, but even in the first case, it would apply to all labels on that form.
<jtv> noodles775: it's in there, but CSS says "label { white-space: ... }"
<jtv> I wouldn't even mind if I could override it for all labels on that form, except that the problem happens elsewhere as well.
<jml> bigjools, I don't have time this evening, but I'd really appreciate the chance to chat with you or cprov about package permissions.
<bigjools> jml: of course
<bigjools> maybe tomorrow, I can get up early
<jml> bigjools, that'd be great
<jtv> noodles775: maybe the solution is to give these labels an extra class that doesn't have this setting.  Right now it's just "label," and all labels have wrapping prohibited.
<jml> bigjools, thanks.
<noodles775> jtv: I'm not sure I'm understanding... if your form is rendered within a <div class="foo">... then you can add a style rule div.foo label {white-space: normal} and it will override the other one (as it's more specific).
<jtv> noodles775: oh, hadn't tried it that way...  I just tried setting it directly on the div, but that would be overridden by the label style of course.
<jtv> noodles775: nope, that doesn't work either.  :(  I can override label locally, but that's probably not the way Mommy Would Want.
<noodles775> jtv: push your branch and I'll try it.
<jtv> noodles775: Thanks.  Pushing to lp:~jtv/launchpad/translations-person-3.0-mechanical
<jtv> noodles775: will advise when it's pushed in all the way.
<noodles775> jtv: great, thanks! And the demo url you're using.
<jtv> noodles775: the home page for whomever you're logged in as; Translations tab.
<noodles775> Perfect, thanks.
<jtv> There, go to "Translations licensing."
<jtv> Then make your window narrow enough that at least the text for the bottom radio button want to wrap.
<jtv> *wants
<jtv> noodles775: pushed!
<noodles775> jtv: http://pastebin.ubuntu.com/262973/
<noodles775> That works, but I'd not really recommend doing it...
<jtv> *facepalm*
<jtv> noodles775: I agree, but what's the alternative?
<noodles775> jtv: just doing it with the current style and creating a bug explaining the problem so it can be solved generally - there might be other pages that would benefit too.
<jtv> noodles775: can't argue with you there; I want this solved for other templates.
<maxb> beuno: I'm a little confused by your problem earlier, since python-psycopg2 has the same version in jaunty and karmic. (And I can't see your paste)
<noodles775> Great. The less work-arounds we have, the easier it will be for us to fix/maintain etc.
<maxb> Whoa, new /builders page.
<maxb> Overall, quite shiny, though I don't like the way all the builders are now squished into one table, instead of being visually demarcated by architecture and official/ppa status
<maxb> The summary side portlet is neat, though
<noodles775> maxb: you can click on the table headers to sort by arch/status etc.
<maxb> that's simultaneously neat yet kinda pointless, as it disrupts the default sort that separates official/ppa, and I can't imagine any use case where you want the two mixed
<bigjools> maxb: bug 225738 and bug 245459
<mup> Bug #225738: new build farm page UI is a regression for buildd admins <ui> <Soyuz:Fix Committed by cprov> <https://launchpad.net/bugs/225738>
<mup> Bug #245459: Sort the list of builders by architecture then status, not by hostname <trivial> <ui> <Soyuz:Fix Committed by cprov> <https://launchpad.net/bugs/245459>
<noodles775> maxb: the default ordering has all the official first and then all the ppa builders (see the first column). It was a hard problem to solve - the input that cprov got was that it *needed* to be sortable.
<maxb> hmm, why does it suddenly need to be sortable when the 2.x page is not?
<noodles775> maxb: I think the 1.0 was, and the 2.0 was considered a regression, as per the bug above (I wasn't around, but am just guessing).
<maxb> Hmm. I think a good start at fixing the 3.0 layout would be to put ppa and official in separate tables
<noodles775> maxb: you can read a bit about the purpose and target audiences that cprov used for the current 3-0 design at https://dev.launchpad.net/SoyuzBuildersIndexPage
<noodles775> and chat with him later when he's around.
<wgrant> maxb: Also, the distinction between official and PPA, and between the architectures, is going to become much more hazy soon.
<wgrant> maxb: This new layout makes that much easier.
<deryck> Morning.
<noodles775> Hi deryck :)
<noodles775> deryck: btw, thanks for the TextAreaEditorWidget - it couldn't have been easier to integrate :)
<deryck> noodles775, excellent!  good to hear.
<bigjools> wgrant: around?
<wgrant> bigjools: I am.
<bigjools> wgrant: do you know who's the LP MOTU contact?
<wgrant> bigjools: mok0 and myself.
<bigjools> fabulous
<bigjools> wgrant: I grant you three wishes for Soyuz improvements
<noodles775> heh, and you're not allowed to choose things that you're implementing yourself ;)
 * wgrant shall put it to the list.
<bigjools> wgrant: https://dev.launchpad.net/VersionFourDotO/Soyuz
<wgrant> bigjools: Thanks.
<bigjools> wgrant: I kept the items from the 3.0 pages on there as suggestions, feel free to re-use them
<bigjools> and now, food
<gary_poster> maxb: ping?
<gary_poster> abentley: btw, I talked with Francis about the approach we talked about for bug 113993.  He thought it sounded good, and had some suggested refinements.  My focus right now is on a couple of other things, but I can maybe get to it next week or the week after.  Alternatively, I can share what Francis told me with you, and you can do it.  Let me know.
<gary_poster> uh, mup?  bug 113993 ?
<gary_poster> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/113993
<abentley> gary_poster: Thanks for following up on that.
<gary_poster> np
<abentley> gary_poster: For right now, I don't think I'll tackle it, but I'll let you know.
<gary_poster> ok cool
<gary_poster> I'll keep it on my medium-term to do list
<beuno> maxb, it was because I was installing the karmic version, which dos not install in python2.4
<sinzui> salgado: Are you still working on bug 419742? Do you want to drop it for 3.0.
<mup> Bug #419742: downloads page should include some data about the releases <Launchpad Registry:In Progress by salgado> <https://launchpad.net/bugs/419742>
<salgado> sinzui, I've fixed that too; just need to do a couple minor tweaks that martin requested
<sinzui> salgado: okay
<beuno> anyone know why I could be getting this error?  http://paste.ubuntu.com/263176/
<beuno> danilos, looks like something to do with translations  ^
<jtv> beuno: libgettextpo isn't building...  Problem with your rf setup.
 * jtv is not being very helpful
<beuno> jtv, care to give me any tips?
<beuno> :)
<jtv> beuno: slow but easy to try: move your lp-sourcedeps/sourcecode out of the way and run rocketfuel-setup
<beuno> jtv, I can do slow and easy (right after I finish with my skype calls)
<beuno> thanks
<deryck> rockstar, ping
<rockstar> deryck, yo!
<barry> beuno: ping
<beuno> barry, hi
<barry> beuno: hi!  Please verify the rules here https://dev.launchpad.net/VersionThreeDotO/UI/Conversion#Heading%20rules
<barry> beuno: the coding guidelines haven't yet been updated, but please let me know if i got any of the ui rules wrong
 * beuno looks
<beuno> barry, I'm secretly on a call, but will get back to you about this
<barry> beuno: no rush.  i have to run up to the local computer store.  my wireless ap died last night.  i'm going to combine that with lunch so will be back in 60 minutes or so
<barry> beuno: just pvtmsg me with your feedback and we can chat when i get back
<beuno> barry, will do. Have fun computer-shopping
<barry> beuno: it's always fun shopping, never fun paying :)
<beuno> barry, you can make it more exicting if you don't...
<barry> beuno: just be sure to respond to that facebook "bail me out" app request
<beuno> :)
<danilos> beuno: not sure why you'd get that, it's about compiling gettext python library... is this a new system?
<beuno> danilos, upgraded to karmic
<danilos> beuno: hum, haven't tried that yet, perhaps I will as soon as the new laptop gets here
<salgado> beuno, have you given any thoughts on breadcrumbs for https://code.launchpad.dev/~name12/firefox/main/+bug/5/+delete ?
<sinzui> beuno: I think I need to design the product and distro series pages at the same time to ensure they are consistent
<beuno> salgado, same as the bug, but an extra "Delete this bug"?
<beuno> sinzui, I agree
<salgado> beuno, that page is for deleting a bug-to-branch link
<beuno> salgado, ah (I don't have a running rf)
<beuno> salgado, than "delete link to branch"?
<salgado> beuno, right, but the previous items should be identical to the bug's breadcrumbs?  nothing pointing to the branch(es)?
<beuno> salgado, I don't think so, no
<beuno> that should be an ajax action anyway
<salgado> fair enough
<maxb> beuno: huh. How did you end up with the karmic version, not the karmic-ppa version?
<beuno> maxb, by being stupid  :)
<maxb> oh, that reminds me, does a member of ~launchpad fancy copying subversion from ~maxb/+archive/launchpad to ~launchpad/+archive/ppa ?
<maxb> gary_poster: Just noticed your ping of 4 hours ago - I'm semi-around now
<gary_poster> maxb: np.  hey.  had two checkins of interest for you yesterday
<gary_poster> first should get rid of zope branch completely
<gary_poster> second switches to lazr.* distributions
<maxb> oh that's excellent, I shall be sure to get another testrun going overnight then
<gary_poster> that *might* fix the karmic buildout
<gary_poster> I'd love to know if it does
<maxb> gary_poster: hmm... no luck on the "caused the installation of lazr.uri 1.0" buildout issue.
<gary_poster> maxb: ok, thank you very much for checking.  Will try to finish getting lp on karmic up today
<maxb> In theory it should be just the same as getting it running on Jaunty, if you add my launchpad ppa
<gary_poster> maxb: except the problem is lazr.uri on karmic, right?
<maxb> ah, right, well you just temporarily uninstall all the python-lazr-* system packages to dodge around that temporarily
<gary_poster> I already have it running on jaunty generally, of course (and squashed various similar buildout-with-system-python problems)
<maxb> but then, if the problem you're trying to investigate *is* that problem.... :-)
<gary_poster> righht :-)
<gary_poster> My zc.buildout changes are about to be reviewed upstream so I'd like them to also address whatever it is you found
<gary_poster> so we can be using an official release rather than the local dev release I made
<maxb> That would be nice :-)
 * maxb has to go afk now
<gary_poster> ok thanks maxb
<maxb> Most unhelpful test failure EVER!:
<maxb> Failure in test testScanRescuesJobFromBrokenBuilder (lp.buildmaster.tests.test_manager.TestBuilddManagerScan)
<maxb> that's  it. nothing more
<barry> beuno: how'd that page look?
<beuno> barry, I... forgot
<beuno> looking now
<barry> beuno: no worries
<beuno> barry, perfect
<barry> beuno: sweet.  now let's see if i can implement it :)
<beuno> sinzui, "yes"
<beuno> let me know if you need anything else from me for that bug
<beuno> tomorrow is my last day before I start my vacations
<sinzui> beuno: I think salgado has the talent to make that happen after he lands the person page
<beuno> sinzui, I think salgado has the talent for everything
<salgado> not for snowboarding
<beuno> sinzui, who has the "stop using green titles for registry and soyuz" bit on their plate?
<beuno> do you know?
<beuno> or should I just file a bug?
<sinzui> beuno: no one
<sinzui> beuno: That is just colour
<beuno> sinzui, so can I file a bug and assign to you?
<beuno> so I can sleep well while on leave?   :)
<sinzui> beuno: I can fix that at the same time I make the download links blue in the sidebar
<beuno> sinzui, thank you
<beuno> and, the green link is my fault
<beuno> I let it leak through the design
<beuno> I'm sorry about that
<sinzui> beuno: I really think I need to move the RDF to the infomation portlet. Then it will be in the same place for all objects that have it.
<beuno> sinzui, I'm almost certain I agree
<barry> sinzui: it's a thought, though we do not currently use tags.  in the meantime, i want to change that to read it out of a version.txt file at the top level.  it'll still need a manual update, but it will be /much/ cleaner to change that file than to hack the base-layout.pt.  i just want to be sure i'm not messing with some existing scripts or something
<sinzui> barry: You'll need to repeat that...I closed the tab trying to view it
<barry> <barry> sinzui: it's a thought, though we do not currently use tags.  in the
<barry>         meantime, i want to change that to read it out of a version.txt file
<barry>         at the top level.  it'll still need a manual update, but it will be
<barry>         /much/ cleaner to change that file than to hack the base-layout.pt.  i
<barry>         just want to be sure i'm not messing with some existing scripts or
<barry>         something  [15:57]
<barry>  
<barry> (oh that looks horrible)
<sinzui> I agree
<barry> cool.  i'll do it in this branch then, since i'm already looking at it
<sinzui> barry: I am available now. I got distracted by a test I took a disliking to.
<rockstar> sinzui, hi
<sinzui> Hi rockstar
<rockstar> sinzui, so, do we already have existing styles for 3.0 for something like the branch index pages details area (the one that shows the branch path, how to push/pull, the branch formats, etc.)
<sinzui> rockstar: yes
<rockstar> It's currently a table, and in 3.0 CSS its not styled at all.  I'm wondering if I should restyle it, or implement it differently.
<rockstar> sinzui, where can I find an example?
<sinzui> rockstar: it is the <object> Information format
<rockstar> sinzui, I'm not sure what that means.
<sinzui> Look at the product-index.pt
<rockstar> sinzui, nice, thanks.
<sinzui> rockstar: The info should follow any narrative (it if exists) and be the first left portlet,
<rockstar> sinzui, great.  I'm glad to see we're using dl/dt/dd now too.
<sinzui> rockstar: the magic is:
<sinzui>             <div class="two-column-list">
<sinzui>               <dl id="partof" tal:condition="context/project">
<rockstar> sinzui, great, thanks.
<rockstar> sinzui, where can I find a demo of the markup in product-index.pt on launchpad.dev?
 * rockstar suspects CSS foolery in his implementation
<sinzui> rockstar: firefox
<sinzui> rockstar: or every project on edge
<sinzui> And ever projectgroup and every distro
<rockstar> sinzui, ah, okay.  Just as I thought - I need do a little more redesign than previously suspected.
<kfogel> sinzui: if someone needs to merge their N launchpad accounts into one account, how do they do that?  Ask us?
<sinzui> rockstar: if you need to add something after the <dl>s, do it after the </div> so that the content clears the floats.
<kfogel> ~sullivan and ~latchkeyed want to merge into ~sullivan (it's Ian Sullivan, and I just talked to him in person about it)
<sinzui> kfogel: If the user cannot do it himself, (he has not email for the old account), he asks a question for a LOSA to do it
<kfogel> sinzui: ^^
<kfogel> sinzui: no, he should have email.  I'll go talk to him.
<sinzui> kfogel: https://edge.launchpad.net/people/+requestmerge will work if the user has email access to the old account.
<kfogel> sinzui: thank you.  We just did it.  I'm filing a bug now (or finding a dup) for the fact that the way you merge two accounts is: log in under the account you want to *keep*, go to "deactivate account", then find the link about merging accounts, click on it, and follow the instructions.  Not exactly intuitive to click on deactivate in the account you don't want to deactivate :-).
<sinzui> kfogel: This problem will never be intuitive. Having more than one account is the start of non-intuitiveness
<kfogel> sinzui: I can think of a more intuitive interface than what we've got right now: on the same page where we offer to deactivate an account, offer right next to it to merge another account into this one.
<mwhudson> good morning launchpad!
<mwhudson> gary_poster: particularly you :)
<launchpad> morning, mwhudson!
<sinzui> kfogel: There is a bug that the deactivate the user in non-intuitive. I agree they should be in the same place though.
<gary_poster> mwhudson: lol :-) on call; when done will ping you
<mwhudson> gary_poster: ok :)
<kfogel> sinzui: "let not the perfect be the enemy of the clichÃ©", or something like that
<sinzui> kfogel: bug 165148
<kfogel> sinzui: thx
<mup> Bug #165148: "Deactivate my account" link is hard to find <registry-people> <ui> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/165148>
<rockstar> beuno, I has a question for you.
<beuno> rockstar, I can take it
<rockstar> beuno, so, on branch merge proposals, do we really need the subscribers portlet?
<rockstar> (I tend to think the answer is no, because you're technically not subscribed to the bmp)
<beuno> I agree
<beuno> what I do think we need to convey
<beuno> is who's being notified of the MP
<beuno> is that not the same list?
<rockstar> Maybe.
 * maxb wonders why bin/test --list-tests doesn't give the tests in the same order that they are actually running
<kfogel> sinzui: filed bug #422837
<mup> Bug #422837: UI for merging accounts is unintuitive <ui> <Ubuntu:New> <https://launchpad.net/bugs/422837>
<sinzui> kfogel: why in ubuntu?
<kfogel> sinzui: huh?
<kfogel> lost sinzui
<gary_poster> mwhudson: yo. :-)
<mwhudson> gary_poster: actually, i didn't have anything super urgent to say over the email i just sent you
<gary_poster> heh, oh, didn't see, will look
<mwhudson> gary_poster: we can supply our own html for the root resource
<mwhudson> gary_poster: so we can stick a couple of buttons on there
<gary_poster> mwhudson: cool.  yeah.  sounds good to me. <shrug> :-)
<gary_poster> Agree that maybe the FAQ stuff doesn't belong in glue.
<gary_poster> at least not in that format
<gary_poster> I can add what I wrote really quickly to /Trunk if you are +1
<gary_poster> IRT the bzr test stuff, if you think it is working, I think it is working. ;-)
<gary_poster> I'm happy to look at it or not, as you think is helpful.
<mwhudson> gary_poster: +1 for a FAQ on Trunk
<gary_poster> mwhudson: k, on it
<mwhudson> argh
<gary_poster> listen! the sound of someone hacking on buildbot!
<mwhudson> it looks like the bzr builder isn't the latest one that was running on devpad.buildbot.info :(
<gary_poster> mwhudson: oh :-(
 * mwhudson really should pay attention to the standup call he's on...
<gary_poster> mwhudson: (when you get a chance) do you want me to fire up the image of the old ec2 master I made before I shut it down?
<gary_poster> and send you a copy of the master.cfg?
<mwhudson> gary_poster: would be good, yeah
<gary_poster> ok
<gary_poster> mwhudson: hey.  just sent you master.cfg.  Could you verify that it is what you need so I can safely shut down the instance?
<gary_poster> maxb: we need to tell upstream about the zope.sendmail problem.  Did you get a chance to get more details?  /me looks for bug...
<mwhudson> gary_poster: yeah, that's more what i expected thanks
<gary_poster> cool, ok, shutting it down
<gary_poster> maxb: got it, you put it in the bug report thank you.  I'll follow up with upstream.
<mwhudson> gary_poster: btw, i filed http://buildbot.net/trac/ticket/613
<gary_poster> Cool.  I strongly suspect that the maintainers will applaud a JFDI approach.
<gary_poster> I need to run.  Have a good day
#launchpad-dev 2009-09-02
<jml> so hungry
<lifeless> jml: so eat
<lifeless> mwhudson: hi
<lifeless> mwhudson: lp is running <bzr> ?
<jml> lifeless, https://code.launchpad.net/, bottom left of the page.
<lifeless> spm: ping
<spm> lifeless: heyo
<lifeless> wow, slow page day :(
<lifeless> spm: bug 422849
<mup> Bug #422849: Incorrect conversions in 2.0rc1 and bzr.dev <Bazaar:In Progress by lifeless> <https://launchpad.net/bugs/422849>
<spm> what we discovered last night?
<lifeless> spm: if we have 2.0rc1 in use anywhere, don't do conversions with it.
<lifeless> spm: no, new one found this morning. Next up is checking to see if last nights one is the same cause.
<spm> ahhh. nice. oki; afai aware, we don't.
<spm> righto
<lifeless> also bzr.dev is bust i nthe same way.
<spm> the rc1 we tried last night was a special build inside the chroot jjust as a dogfooding exercise purely for bzr itself.
<lifeless> spm: inside the chroot?
<lifeless> spm: pqm doesn't use the bzr inside the chroot for anything...
<spm> pqm on balleny - we don't have pyrex installed outside the chroot.
<spm> lifeless: yes; this was a special build used explicitly to do (or not...) the check/upgrade
<lifeless> ah right
<spm> made for seriously ikky pathing on the cmd line :-)
<wgrant> jml: So, about that branch... anything I can do? Anybody who should be poked in preference to you? Any public AMIs available so I can debug it myself?
<jml> wgrant, all good questions
<jml> wgrant, I'll think while I chew :)
<jml> wgrant, I've filed https://bugs.edge.launchpad.net/launchpad-foundations/+bug/422908
<mup> Bug #422908: Some branches crash ec2test <build-infrastructure> <test-system> <Launchpad Foundations:New> <https://launchpad.net/bugs/422908>
<jml> wgrant, I'll also kick off an attached run
<jml> wgrant, I'm happy to be poked. You could argue it's a build engineer task, but I find that particular flavour of delegation distasteful
<jml> wgrant, as for public AMIs, that is most definitely a build engineer task :) there aren't any that I know of, and we should change that.
<wgrant> jml: It's a bit sad that completely reliable community ec2testing will not be possible, due to c-i-p and shipit. But I guess they're mostly isolated.
 * thumper lunches
<mwhudson> jml: have you tried to land that branch without --headless?
<lifeless> how do you supersede a review?
<jml> mwhudson, yes, but my network connection died
<wgrant> mwhudson: You may recognise this as the branch that I handed to you from gary, and you handed to thumper.
<mwhudson> wgrant: indeed
 * jml tries again.
<jml> mwhudson, actually, could you also try landing the branch without headless?
<mwhudson> jml: i am already actually
<jml> mwhudson, thanks :)
<wgrant> That sounds like a bad idea. The moment you give it the chance to do something bad (ie. land twice), it will succeed.
<wgrant> Aanyway, thanks jml and mwhudson. I need to go to uni.
<mwhudson> mine died in the usual router hating way :(
<jml> mwhudson, :(
<jml> mwhudson, mine is still going...
<mwhudson> jml: hooray
<jml>  test_bugwatch_added_from_comment (lp.bugs.tests.test_bugchanges.TestBugChanges)
<jml> cprov, hello
<jml> cprov, are you actually around?
<cprov> jml: yes
<cprov> jml: what's up ?
<jml> cprov, I would like to talk about pockets
<cprov> jml: sure
<jml> cprov, please give me a second, I'm filling out a form that requires complete concentration
<cprov> jml: okay, np
<jml> done
<jml> cprov, so, we've been talking about package-permission-love
<jml> and extracting out the pocket check from the uploadpolicy and putting them into the new verify_upload function
<cprov> jml: right, what did you think about my comments ?
<jml> (which should have its name changed, but that's a different discussion)
 * jml rereads
<jml> regarding the control variable in getPubSource
<jml> I added that because at one time I wanted to have a makeSPPH() equivalent
<jml> and bigjools said that was the best place to do it.
<cprov> jml: I like makeSPPH() better and STP could also use it from the factory, so no duplications.
<jml> hmmm.
<jml> that smells too much like work for this branch
<jml> cprov, but I'll probably need it in the next branch anyway
<cprov> jml: a good thing is that you don't need it in this branch
<jml> cprov, so I'll back out the change from this branch, and put it in my notes for the next one
<cprov> perfect!
<jml> and I'll fix makeGPGKey.
<jml> cprov, next up, I want to change the bug nomination code to use this checker
<jml> cprov, and (in parallel, I guess), I want to sort out the pocket stuff
<jml> cprov, the thing with pockets is that there are no signer-based checks related to pockets
<jml> it's purely a function of distroseries, pocket
<cprov> no, it's strictly related to the series status & the pocket
<jml> right.
<jml> I'm wondering whether I care about this wrt pushing to official package branches
<jml> and whether bug nomination cares about this.
<cprov> jml: I'm no familiar with the bug nomination code, but I'm almost sure it doesn't depend on pockets
<jml> it doesn't
<jml> but should it!
<cprov> jml: rarely new packages show up in the post-release pockets (-updates, -security, etc)
<cprov> jml: it happened only once in ubuntu with the SSH-incident (edgy, I guess)
<jml> cprov, hmm
<jml> cprov, this is making me think that the current function should stay as-is
<jml> and that there should be another one that also checks the pocket.
<cprov> jml: yes, probably, since they answer different questions ... 1) has upload rights? 2) is this suite open to new uploads?
<jml> cprov, "new" uploads
<jml> it seems to me to be used for whether it's open to uploads at all
<jml> but we're on the same page
<cprov> you may have rights to upload 'foo' source to jaunty-updates and karmic, but you cannot upload it to jaunty (because it's already closed)
<jml> also, it's about time Launchpad grew an object that represented suite
<cprov> agreed
<jml> ok, I think I know what I want to do now.
<jml> I still need to talk to james_w sometime to go over a few edge cases.
<jml> cprov, thanks
<cprov> jml: you're welcome.
<jml> how did it get so late so fast :(
<cprov> you tell me ...
<jml> mwhudson, I guess we are in testfix mode...
<mwhudson> jml: yeah, that odd db_lp failure
 * jml pulls db-devel
 * jml considers registering whyareweintestfix.com
<wgrant> jml: Bug nominations *must not* respect pockets.
<wgrant> jml: Or everything breaks.
<jml> wgrant, thanks
<wgrant> And people start to stab you.
 * jml frowns
<wgrant> It's rather annoying when people break nomination permissions.
<jml> it would be very difficult to have introduced this test failure if you used pyflakes + flymake
<wgrant> jml: Oh, so you have the failure?
<jml> wgrant, I was speaking of the failure that broke db-devel
<wgrant> jml: Phew.
<jml> the 'you' should have been 'one'
<wgrant> Yep.
<jml> we're up to 'lib/lp/bugs/tests/../stories/bugtracker/bugtrackers-index.txt' on the ec2test run for your structural subscription branch
<wgrant> jml: OK, so it does get somewhere...
<jml> I can't wait to be able to do Launchpad development in karmic
<wgrant> You can.
<jml> chroots are a pita
<wgrant> It works fine.
<wgrant> You just need...
<wgrant> to remove python-lazr-uri
<jml> wgrant, but some of the tests still fail, right?
<wgrant> Then grab python-xml and launchpad-(developer-)dependencies from maxb's PPA.
<wgrant> jml: No, I fixed all the 2.4 Karmic failures.
<wgrant> 2.5 fails, sure.
<jml> wgrant, cool
<jml> I might make the switch today than.
<jml> then.
<jml> but first, fix the failure in db-devel
<wgrant> Indeed.
<jml> mwhudson, can I get your approval for http://pastebin.ubuntu.com/263535/
<mwhudson> jml: yes
<mwhudson> jml: is this a mis-merge or something?
<jml> mwhudson, I don't know.
 * mwhudson is having on of those "aaah 300 things to do *now*" hours
 * jml too
<jml> swap you :)
<mwhudson> it must be a mis-merge, no manual landings to db-devel for ages
<mwhudson> jml: i don't think that would help :)
<mwhudson> "argh, now i still have 300 things to do, and i don't know anything about any of them!"
<jml> mwhudson, I'm actually in the process of paying someone to take care of about 50 of the things I need to do.
<jml> which is pretty exciting
<jml> but it's still something to do
<mwhudson> jml: moving related in some way or other i presume?
<jml> yeah.
<jml> I wonder what the 'go out of testfix' time period is
<mwhudson> jml: */5
<mwhudson> jml: your branch is still in pqm though
<jml> so, ~5 mins after it's finished in pqm?
<mwhudson> jml: yeah
<jml> a queueing system would be nice
<mwhudson> it would be nice if testfix meant the queue stopped rather than rejecting things
 * thumper snorts
<thumper> mwhudson: see earlier talk about a sprint for landing queues :)
 * jml files a bug
<jml> BUILD FAILED: failed failed slave lost
<jml> I guess that means it failed.
<mwhudson> jml: my fault
<mwhudson> oh joy, that means we'll be back in testfix i guess
<jml> so tim's patch will get rejected
<jml> and maybe mine will too
<mwhudson> jml: yours will be ok i think
<jml> yay
 * jml sneaks actually working changes in between breakages
<thumper> jml: how does you patch fix it?
<jml> thumper, which one?
<thumper> jml: db-devel Rev 8426
<thumper> jml: it looks like you are just assigning the result
<jml> thumper, that's right
<thumper> jml: ok, lack of visual context doesn't make the fix apparent
<jml> thumper, below the visible diff, you'll see that 'new_rev_id' was being returned from the function
<thumper> ok
<thumper> jml: I'm adding an ownerCounts method to branch collection
<jml> thumper, it should be clear if you look at the actual failures.
<thumper> jml: have I mentioned recently how much I like branch collection?
<jml> thumper, heh
<thumper> jml: especially with adapters, it makes some things really easy
<jml> thumper, in my infinite spare time, I'm working with jkakar on a paper or something about it
<thumper> jml: that'd be pretty interesting
<thumper> jml: I'll be talking about it during the kiwipycon keynote as something that works well
 * thumper wants named tuples
<thumper> bring on python 2.6
<jml> keynote?
<jml> wow.
<mwhudson> jml: here's some extra letters for you "simal"
<mwhudson> (they go after "infinite")
<thumper> jml: yeah, I get the Sunday keynote spot :)
<thumper> jml: for some reason they got the opinion that I'm a person of some substance :)
<thumper> mwhudson: it seems that my branch will go through
<mwhudson> thumper: hooray
<wgrant> jml: OMG!
<wgrant> jml: I have email!
<wgrant> And that is quite a reasonable failure, too.
<jml> wgrant, I have email too.
<jml> wgrant, but I have no idea at all as to why previous email attempts failed.
<jml> mwhudson, heh heh
<wgrant> jml: Was this the unheadless run?
<jml> wgrant, it was.
<wgrant> jml: Interesting.
<wgrant> jml: Thanks, I'll fix that failure soonish.
<jml> wgrant, np.
<jml> with any luck, we'll be out of testfix then.
<wgrant> I hope so, as I'll be gone for two hours.
 * wgrant vanishes.
<thumper> mwhudson: there isn't any point in the lp builder waiting 10 minutes if pqm takes 15 minutes to land a branch
<thumper> :(
<mwhudson> thumper: this is true
<mwhudson> thumper: do you know if the pqm config we use for launchpad is in a bzr branch anywhere?
<thumper> mwhudson: no
<thumper> mwhudson: I don't know
<thumper> mwhudson: spm would
<mwhudson> thumper: ok thanks
<mwhudson> thumper: yeah, got his attention in another channel now
<spm> short answer is - yes in a bzr branch; but not one you guys have access to
<thumper> I have some family stuff to deal with, school show tonight, I'll be back after 8:30
<spm> later thumper-afk
<jml> james_w, can you please ping me when you come online.
<jml> mwhudson, up for a quick call?
<mwhudson> jml: if it's very quick
<jml> ok
<jml> mwhudson, https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=test-system
<jtv> hi noodles775
<noodles775> Hiya jtv :)
 * mwhudson is no longer here, btw
<spm> hey jtv, noodles
<jtv> g'day spm
<noodles775> Evening spm.
<spm> jtv: you may wish to have a look at https://bugs.edge.launchpad.net/oops-tools/+bug/422960 I've put against oops-tools as a starter; but it may have translation team implications? or not... :-/
<mup> Bug #422960: appear to be failing to record oops for all +translate HTTP 503 errors <OOPS Tools:New> <https://launchpad.net/bugs/422960>
<jtv> spm: what's 503?
<spm> bad stuff happened. server fail.
<jtv> Oh, the IIS one?
<jtv> (That's a joke.  But not entirely untrue, in my experience.)
<spm> iwas like "IIS? we don't use IIS" and then the penny dropped and rofl followed.
<jtv> We don't use IIS.
<jtv> *We don't use IIS*
<spm> rfc 2616, 10.5.4 503 Service Unavailable
<jtv> *OF COURSE* we don't use IIS!
<spm> man I haven't been near an IIS box in anger since the 90's. <== pride
<jtv> spm: will the oops tools be fixed soon to report these?  If so, best course of action is probably to warn my team of the coming tidal wave.
<spm> now i thin on it, in anger was the only way too.
 * jtv has never worked with IIS
<spm> jtv: dunno. I'm not even sure what the problem is. afaict, those errors aren't being oopsed - which is part of the problem. :-/
<jtv> Just visiting IIS-based sites is enough to make you think of 503 as "the IIS notice"
<spm> heh
<spm> jtv: consider - we had ~ 11,600 oops-able page requests, yet only recorded ~ 1000. Begs the question: is this just vs +translate, or the fault is more widespread. if the latter.... ow.
<jtv> If not... still ow
<wgrant> Isn't a 503 a timeout?
<wgrant> Which should certainly be loged...
<spm> wgrant: logged or oopsed?
<wgrant> spm: OOPSed, sorry.
<wgrant> Unless it's the Apache or Pound sort of 503.
<spm> right - I assumed that's what you meant.
<spm> my concern is that if "X" (DB, app server whatever) is overloaded; and an OOPS requires even some minimal App/DB resources it also fails; and contributes to making things worse. :-(
<wgrant> I've never received a 503 from LP except during the edge restart or a timeout.
<spm> heh. start requesting +translate pages ;-)
<spm> anyways, now I've cheered you all up with that wonderful news, outa here. time to cook dinner :-)
<bigjools> jml: good morning/afternoon/evening
<jml> bigjools, hi
<jml> bigjools, I'm on a skype call atm
<bigjools> jml: ok, I got up early just for you, I feel upset now :)
<jml> bigjools, it'll finish soon :)
<bigjools> heh no worries
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv
<jml> bigjools, hi
<jml> bigjools, sorry that took so long.
 * bigjools wakes up
<jml> bigjools, I've been thinking about pockets and permissions
<bigjools> ok
<jml> bigjools, pocket has nothing to do with whether or not any given individual can upload a package
<jml> bigjools, it's across all individuals
<bigjools> yes
<jml> a function of pocket and distroseries status.
<jml> I think this makes it conceptually separate.
<bigjools> right, it's not an ACL check, it's a state check
<jml> I'd like to re-use the function I've abstracted out to handle the bug nomination permission check
<jml> but that particular check should not care about pockets at all
<bigjools> however, verify_acl might say yes, and then you get rejected further down the road
<bigjools> so maybe we need a separate place to ask "can this be uploaded?"
<jml> for write access to official branches, I think the question about whether to reject based on distroseries status + pocket
<jml> is a bit more open
<jml> bigjools, I think we need one function, "barring release policy, can I upload", and another one saying "can I actually upload". The second does the pocket check, the first does not.
<bigjools> jml: I agree
<jml> bigjools, for official branches, I really want to talk with james_w about our our use cases.
<jml> (also, Soyuz needs a Suite object already.)
<bigjools> heh
<bigjools> jml: abstracting out the latter will be very hard
<jml> bigjools, a Suite object?
<bigjools> it's basically what nascentupload does
<jml> bigjools, oh, the pocket thing
<bigjools> upload checks
<jml> well, it won't be substantially harder than what I've already done, I don't think
<jml> (famous last words, perhaps)
<bigjools> it needs to have its state checks separated from the package consistency checks
<bigjools> they're very much mingled right now
<jml> I might not have to do the full decoupling in order to make official branches work correctly
<bigjools> indeed, you only need the pocket check
<jml> cool :)
<jml> given the constraints on my time right now, that's all I will do.
<jml> probably
<bigjools> jml: is there a deadline for this?
<jml> bigjools, yesterday
<bigjools> oh, cock
<jml> bigjools, it should have actually been done weeks ago
<jml> open sourcing + my vacation + other things has made this take way too long.
<jml> bigjools, that said, I had expected it to be less work -- I had thought there was already an easy way of asking about package upload permission :)
<bigjools> jml: assumption is ther mother of all fuckups
<jml> bigjools, quite
<bigjools> jml: ok, it should be relatively easy to hack out the pocket check and then we can improve that as time goes by
<bigjools> it would be really nice to improve nascentupload, it's a bit monolithic
<jml> cool.
<jml> yes, it's ripe for improvement
<jml> more functions, more parameters, less state, more unit tests
 * bigjools nods vigorously
<jml> a good day's work for a good day's pay.
<jml> Freedom and dignity for Britain's swans.
<bigjools> you're bonkers mate :)
<jml> and loving it.
<bigjools> are you coming to London later this month?
<jml> bigjools, I am.
<bigjools> cool
<jml> bigjools, I very much want to have this done before then
<bigjools> your airmiles total must be bigger than the combined G8 national deficits
<jml> bigjools, not quite. :)
<jml> any bugs people around?
<jml> why does the count on a tag say '16' but only 9 bugs show up when I click on it?
<wgrant> jml: Duplicates count in the counts.
<wgrant> jml: 'tis a bug.
<jml> wgrant, thanks.
 * wgrant too whines about nascentupload.
<bigjools> al-maisan: I messed up, your meeting with me is in 45m :)
<bigjools> but your TZ is still borked
<al-maisan> bigjools: no problem, let's have the call in 45 minutes then :)
<wgrant> jml: So, what should I do about my branch? The failure is simple to fix, but I somehow don't think it's a good idea to land it.
<wgrant> Hmm. I'm not sure I like the way the black links in the corner of the main content portlets are now blue.
<deryck> Good morning, all.
<wgrant> bigjools: Very nice new DSP page.
<bigjools> wgrant: cool, thanks
<wgrant> Needs some more links, but otherwise very very nice.
<bigjools> where to?
<wgrant> bigjools: I tried to click wildly on the 'Upstream' section; nothing happened.
<wgrant> bigjools: The series names in the distroseries listing also deserve to be links.
<bigjools> hmmm yeah we didn't think of that
<bigjools> er, they are?
<wgrant> bigjools: Sorry, *product*series names.
<bigjools> wgrant: can you file bugs - and ideally submit a patch :)
<wgrant> bigjools: Currently filing bugs about the team page. Will do soon.
<wgrant> bigjools: DSP bugs live in -registry these days, right?
<bigjools> wgrant: yep
<noodles775> allenap: when you tried the two-column dl list, did you make sure that each dt/dd-pair was a *separate* dl? It's unexpected, but apparently how it's currently done in LP (looking at the product template.)
<allenap> noodles775: Yeah, I followed the way that cprov had done it.
<allenap> noodles775: I was thinking maybe there wasn't enough horizontal space. I should have tried widening my window.
<noodles775> yeah, worth trying.
<salgado> barry, product-new.pt has the following
<salgado>         <tal:comment condition="nothing">
<salgado>           XXX BarryWarsaw 28-Apr-2009 bug 368910
<salgado>           REMOVE THE FOLLOWING BEFORE 2.2.5 PRODUCTION ROLLOUT
<salgado>         </tal:comment>
<salgado>         <a href="mailto:feedback@launchpad.net">Contact us</a>
<mup> Bug #368910: Remove "Contact us" from new projects/+new page <story-guided-project-registration> <Launchpad Registry:Won't Fix by barry> <https://launchpad.net/bugs/368910>
<salgado> thanks mup
<salgado> barry, I guess I should just remove the comment from there?
<barry> salgado: that can be removed.  we decided to just keep the text forever <wink>
<barry> salgado: +1
<salgado> cool
<salgado> barry, do you have a couple minutes to help me with a template I'm converting?
<barry> salgado: sure
<salgado> barry, actually, I think I've sorted it out myself, but now I have another question. :)
<salgado> # There should be an H2 above the app tabs which contains a description of the context of the page, e.g. "Launchpad itself"
<salgado> barry, that's from https://dev.launchpad.net/VersionThreeDotO/UI/Conversion#Heading rules
<salgado> for non-default pages
<salgado> but what I see is an H1 (not an H2) above the app tabs
<barry> salgado: for which page?
<salgado> barry, https://launchpad.dev/~name16/+vouchers
 * barry looks
<salgado> after converting it, that is
<barry> salgado: right.  i think that h1 goes away in favor of the h2 above the app tabs
<barry> salgado: if you have a branch w/ partial conversion i could take a look at it
<salgado> barry, would it be ok if I gave you a patch?
<salgado>         if IRootContext.providedBy(self._context):
<salgado>             heading = 'h1'
<salgado>         else:
<salgado>             heading = 'h2'
<barry> salgado: i'm hacking in that code in my current branch
<salgado> barry, that's why that pages uses an H1; its context (IPerson) is an IRootContext
<salgado> barry, is this the correct behaviour or are you changing that?
<barry> salgado: hmm, yeah
<barry> salgado: i probably will have to.  i was actually trying to work in an interface check to see if i could make that label editable, but the editable widget is on the view not the context and the view isn't available to the WatermarkTalesAdapter
<barry> salgado: other than IRootContext, what's the determining factor here?  it's that the page is not an index page on the root context, so once again, i think we need access to the view here
<barry> sinzui: please try me again
<sinzui> barry: we hear your away message
<barry> sinzui: i just restarted skype
<sinzui> bac: https://launchpad.net/asword
<salgado> barry, I'm not sure actually.  I just stumbled upon that code after seeing the context/watermark:heading in base-layout.pt
<barry> reviewers, beuno -> #launchpad-meeting in 3m
<barry> salgado: yeah, we'll have to see.  i think that should be changed to view/watermark:heading, but i'm not positive
<barry> reviewers, beuno -> #launchpad-meeting
<bac> sinzui: that license is quite a piece of work
<sinzui> bac: Yes. I read it three times. It does not discriminate who or how it is used. I am still puzzled by it though
<bac> sinzui: i can't get my head around it
<bac> sinzui: and it is so poorly written i can't imagine it is binding
<bac> hackerz are not lawyerz and should quit pretending to be
<sinzui> bac: we could say we do not host android and end the madness
<bac> sinzui: yeah, it does approach a dual license
<bac> sinzui: tell them to slap affero on it for LP and be done
<sinzui> bac: yes. I think that is the right way to think about it. I will approve it.
<bac> ok
<jtv> noodles775, as promised: bug 423212
<mup> Bug #423212: Not enough padding around headings and listings <Launchpad itself:New> <https://launchpad.net/bugs/423212>
<noodles775> jtv: taa.
<bigjools> salgado: hey dude, around?
<salgado> gary_poster, given a context, a request and a page name, can I look up what page template would be used to render that?
<salgado> bigjools, hi
<bigjools> salgado: hi there, I've got a page with incomplete breadcrumbs: https://dogfood.launchpad.net/ubuntu/karmic/i386/amarok/2:2.1mysql5.1.30-0ubuntu3
<bigjools> salgado: how do I fix it?
<gary_poster> salgado: yes.  it's not exposed very well.  You have to get the component registry's underlying registry and use the "lookup" method.
<salgado> bigjools, you need to define IBreadcrumb adapters for the objects traversed there. (e.g. karmic i386, amarok, amarok version 2:2.1)
<bigjools> salgado: ok sounds simple enough
<bigjools> salgado: what's the usual way of testing that?
<bigjools> point me at an example if you like
<salgado> bigjools, lp/bugs/browser/tests/test_breadcrumbs.py
<salgado> gary_poster, cool, but how do I get the component registry's underlying registry?
<bigjools> salgado: excellent, thanks
<gary_poster> salgado: :-) yeah, looking
<gary_poster> salgado: (registry).adapters
<gary_poster> is the underlying one I thing
<gary_poster> think
<gary_poster> that you want
<gary_poster> (just looking at code)
<gary_poster> salgado: (registry).adapters.lookup((tuple of what you have), interface you want, name) it looks like.  checking out interface
<gary_poster> tuple of what you have may have to be specs...
<bac> sinzui: chat?
<gary_poster> (this is one of the APIs that I want to make more obvious)
<sinzui> bac: yes, I am getting my self ready
<salgado> gary_poster, what are specs?
<gary_poster> salgado: the collected interfaces provided by an object.  Trying to check.  not clear in iface...
<gary_poster> salgado: yeah, ok I have it, I think
<gary_poster> (registry).adapters.lookup([zope.interface.providedBy(o) for o in provided_objects], required_interface, name[, default])
<gary_poster> salgado: that make sense?
<sinzui> bac: I am ready, are you?
<bac> yep
<salgado> gary_poster, it would if I knew how to get the component registry and what name[, default] means
<gary_poster> salgado: ok we're close then
<gary_poster> salgado: name: the name of the page in this case
<gary_poster> [, default] : if you provide a default, and there is no match, the default will be returned.  Otherwise, you'll get a lookup error
<salgado> oh, I see
<gary_poster> registry: zope.component.getSiteManager() should be what I expect here.  If not, let me know and I'll dig a tiny bit more.
<gary_poster> sm = zope.component.getSiteManager()
<gary_poster> sm.adapters.lookup([zope.interface.providedBy(o) for o in provided_objects], required_interface, name)
<gary_poster> I think :-D
<salgado> gary_poster, great, thanks.  I'll give that a try. :)
<gary_poster> cool :-)
<gary_poster> lemme know
<beuno> \o/  finally got ohloh to start importing Launchpad  https://www.ohloh.net/p/launchpad/enlistments
<beuno> jml, ^
 * beuno waits to see if it chokes to death half way through
<sinzui> bac: http://people.canonical.com/~beuno/conversions.html
<salgado> gary_poster, I tried that using IPageTemplate for required_interfaces (https://pastebin.canonical.com/21721/) but got an empty tuple back from lookupAll() and an error from lookup(..., name='foo').  am I doing something wrong?
<salgado> also, the signature of the lookup() family seems to be lookup*(required, provided), so I tried changing the order of the arguments, but that didn't help either
<gary_poster> looking
<gary_poster> salgado: last line should be  (zope.interface.providedBy(o) for o in [ob, request]), zope.interface.Interface)
<gary_poster> or use lookup with zope.interface.Interface and real name
<salgado> a ha
<salgado> gary_poster, why do I need zope.interface.Interface?
<salgado> the lookupAll() call gives me back a tuple with all the views, but the lookup() one with a name fails:
<gary_poster> salgado: short answer: because zope's view registration has sucked for a long time, so long that it is unlikely to change for the core libraries unless things are ported to Python 3 and pain is already expected.  views are registered and looked up with the null interface.
<gary_poster>  (Even if it didn't suck, the interface would essentially just define __call__: it doesn't have to be a page template, and in fact the views we have use page templates internally, but do not themselves provide the interface)
<gary_poster> it has sucked from a theoretical perspective, I should say ;-)
<salgado> gary_poster, https://pastebin.canonical.com/21723/ is what I got when I tried lookup(name)
<gary_poster> salgado: is +edit a page or a traversal step?  also, what did lookupAll give?
<salgado> gary_poster, it's a page.  lookupAll() gave me a tuple of SimpleViewClass objects
<gary_poster> salgado: oh, but without names?  bleh
<salgado> two-tuples, sorry
<salgado> (template-name, SimpleViewClass)
<gary_poster> salgado: oh, was +edit in there?
<gary_poster> I think our +* stuff uses some Launchpad specific trick.  Can go try to find
<salgado> gary_poster, yep, it's there
<gary_poster> oh...
<gary_poster> salgado: when you used lookupAll, were you using a generator comprehension or a list comprehension?  Maybe the code doesn't like generators.  You used a generator comprehension in the pastebin.  Going to look at code to see if it is lame like that
<gary_poster> salgado: yes it is
<gary_poster> salgado: that's where your len error is coming in
<gary_poster> salgado: change that to a list comprehension (or your favorite flavor of len-able) and it will be good
<salgado> indeed
<salgado> but that gives me just the SimpleViewClass, without the template name. :(
<gary_poster> salgado: but...you provided the template name?  so you already know it?
<salgado> I provided the page name, and I wanted the template file
<salgado> but that's the same thing that lookupAll() gives me. for some reason I thought it was giving the file names, but they were actually the page names
<gary_poster> salgado: oh.  I see. :-(  I think I led you on a goose chase, because I misunderstood the goal.  :-(  I'm sorry.  (1) get the view.  (2) look at the .template attribute, IIRC.  I think if you dir that or look at the dict you will see what you want (or look at the class, look at what it implements, and that will show you an API to use)
<gary_poster> The only possible advantage of the goose chase in this case is that you don't instantiate the view.
<gary_poster> which is usually not important
<salgado> gary_poster, but to have access to view.template I need to instantiate it, no?
<gary_poster> salgado: I'd expect it to be on the class initially, but I don't remember the machinery.  If you want to just get the instantiated view, forget everything I've told you, and do the following:
<gary_poster> sm = zope.component.getSiteManager()
<gary_poster> view = zope.component.getMultiAdapter((context, request), name='+edit')
<gary_poster> that's from memory, double-checking...
<salgado> gary_poster, yeah, that works
<gary_poster> k
<gary_poster> actually you can forget the first line
<salgado> yep
<salgado> gary_poster, ok, now I have all that I need, I think. thanks for holding my hands during that trip down zope's internal bits. :)
<gary_poster> salgado: np, sorry for the goose chase again
<salgado> no worries, it was fun
<gary_poster> heh, cool
<rockstar> beuno, ping
<beuno> rockstar, yo
<rockstar> beuno, so, I just made the mechanical changes to the branch index page, and have a test failure requiring some thought.
 * beuno checks to see if flacoste is around
<beuno> no, good
<rockstar> beuno, basically, the sub-navigation links on the branch index page need sorting.  I know those "sub-tab" links will go away, but I wonder if having the link to the merge proposals is important anyway.
<beuno> rockstar, just delete the test
<beuno> :)
<rockstar> beuno, cool.  That's all I needed.
<beuno> I'm sort of kidding
<rockstar> The lost functionality will be returning in a bit.
<rockstar> beuno, dammit!
<beuno> well, those things should be inline links
<beuno> rather than sub-menus
<rockstar> beuno, well, yes, but the link to "merge proposals" is not at all necessary.  We already have that link available elsewhere.
<beuno> rockstar, where?
<beuno> I agree it shouldn't be on that page
<beuno> but I'd like to be sure that it's easy to find
<rockstar> beuno, if there are merge proposals, it's the link that says "X branches proposed for merging into this one"
<rockstar> beuno, also, that page is really going to get hammered (probably deleted and re-invented).  It sucks bad.
<beuno> rockstar, did you see my mockup for it?
<rockstar> beuno, I'm not doing anything with mockups right now.  I just made the mechanical changes to get it to the new template.
<beuno> rockstar, drop it then
<beuno> what about source code?
<rockstar> beuno, linked by revision.
<beuno> rockstar, one of the problems we have is that people think that Launchpad doesn't actually have the code
<beuno> that is one of the main problems that we need to solve with this page
<rockstar> beuno, yeah, so what I'm doing now is making those mechanical changes, then starting mockups and sending them out.
<beuno> rockstar, it sounds like cheating for the stats
<beuno> but, ok
<beuno> I'm on leave in about 4 hours
<beuno> not much I can do  :)
<jtv> beuno: I posted those screenshots you wanted for my review btw
<rockstar> beuno, it's actually cheating for breadcrumb's sake (they are broken for branches currently, we'd like to make that more obvious)
<beuno> jtv, I saw them
<rockstar> beuno, at this point, the stats are less important.
<jtv> beuno: it's almost midnight here though, and high time for me to scram
<beuno> jtv, it's like 1200 of them, and I can see a lot of issued with many. Is this a mechanical change?
<rockstar> beuno, I think having a list of problems to solve with the page is absolutely helpful.
<beuno> rockstar, so, I'd focus on the re-design, and not on the conversion
<beuno> like was done with the project page
<rockstar> beuno, yeah, so we're getting the conversion out of the way.
<beuno> and the DSP in soyuz
<beuno> well, the thing is, it's twice the work IMHO
<jtv> beuno: it's 5 "before" and 5 "after" IIRC.  Yes, mechanical change, nothing exciting in the layout, but not sure the nav menus are what you want.
<bigjools> beuno: the worst thing is, that's not a Soyuz page so I just ticked one off for registry!
<beuno> jtv, cool. So I know what to look at. Go to bed, you'll have a review in the morning
<rockstar> beuno, I disagree.  Also, if we didn't move these pages over, we're putting off the need to fix things.
<jtv> beuno: thanks
<beuno> rockstar, aiight. You're the one actually doing the work  :)
<beuno> rockstar, so, drop those menu items, but make sure source code is easy to find
<beuno> small numbers with links is not easy
<rockstar> beuno, yeah, but that's kinda the idea behind this branch: find out what issues we really have with the branch page.
<beuno> rockstar, gotcha. Do you want the original file for my mockup so you can play around with it in balsamiq?
<rockstar> beuno, that'd be nice, but I don't know what balsamiq is.
<beuno> rockstar, have you seen my mockup?
<rockstar> beuno, I have.  I've also seen the mass amount of bikeshedding as a result.  I'm trying to get that churn out of the way so we have definitive answers.
<beuno> rockstar, it's the software I used to create that mockup: http://www.balsamiq.com/products/mockups
<beuno> ignore the bikeshedding
<beuno> you have something to work on
<beuno> go for best effort
<beuno> the mockup fixes a lot of the current problems
<beuno> go with that, and try to address as many problemas as you can, without making the page suck in the end
<beuno> it's better to not address some people's concerns, than to have 400k unhappy users
<rockstar> beuno, yeah, that's the idea.
<rockstar> beuno, yeah, but some of the concerns are valid.  I'm aggregating those concerns as we speak, and will be talking this over with thumper today.
<beuno> rockstar, of course the concerns are valid
<beuno> in the end, we need a better page
<beuno> so make hard choiced!
<beuno> whatever you do
<rockstar> beuno, yeah, in the process.
<beuno> look at the end result, and be happy with it
<beuno> ignore what was or was not addressed
<beuno> otherwise, revert to a step where you where happy
<rockstar> beuno, cool. thanks for the pep talk.
<beuno> :)
 * beuno passes the baton to rockstar
<rockstar> beuno, so how do I get to this balsamique stuff?
<beuno> rockstar, either use the online version
<beuno> or
<beuno> download it
<beuno> install it
<beuno> and check your email
<rockstar> beuno, it's software that wants me to pay for it though.  I stopped paying for software 10 years ago!  :)
<beuno> rockstar, sorry
<beuno> I forgot the last step
<beuno> read the email  :)
<rockstar> beuno, okay.
<salgado-lunch> sim
<flacoste> beuno: what's up?
<beuno> flacoste, I was joking about removing tests
<flacoste> lol
<beuno> :)
<barry> beuno: nice! i have editable h1 headings working for root context pages
<beuno> barry, WOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
<beuno> screenshot?
<barry> beuno: sure. sec...
<barry> gnome-do, why are you gnome-dont today?
<barry> beuno: https://people.canonical.com/~barry/screen.png
<beuno> barry, not loading...  :/
<beuno> bigjools, cprov, https://edge.launchpad.net/ubuntu/+source/gnome-disk-utility
<beuno> *love* it
<cprov> beuno: bigjools merit, I wasn't involved ;)
<bigjools> beuno: check out https://edge.launchpad.net/ubuntu/+source/amarok
<bigjools> far more interesting
<barry> beuno: i guess rookery isn't feeling well.  try https://devpad.canonical.com/~barry/screen.png
<bigjools> beuno: oh, and thanks :)
<beuno> bigjools, super awesome
<beuno> bigjools, VERY NAIS
<bigjools> verrah nahce
<beuno> abentley, rockstar, thumper, my #1 wish for the MP pages is that subscribers stop pushing down the content  :)
<bigjools> beuno: did you play with the expanders?
<thumper> beuno: ok
<barry> beuno: i have to say though, that the project index page is a bit busy :/
<beuno> bigjools, yes. The content still needs some vertical spacing between sections
<beuno> but, all in all, fantastic work bigjools
<beuno> it's a very hard problem
<beuno> and the result is very nice
<bigjools> beuno: the same template renders with spacing on the PPA expanders, dunno why it's borked on +source
<beuno> barry, yes. Talk to noodles775 about global padding
<beuno> bigjools, true that. Bug?
<bigjools> prob some stylesheet funkiness making assumptions about things
<bigjools> I'll play with firebug tomorrow and see
<beuno> super
<beuno> that page made me very happy
<bigjools> rawk
<abentley> beuno: You mean reviewers?  MPs don't have subscribers.
<beuno> abentley, well, subscribers to the branch
<beuno> https://code.edge.launchpad.net/~rockstar/launchpad/subscription-widget/+merge/7643
<abentley> Okay, I'm not sure what you mean by "pushing down the content".
<beuno> see the subscribers on the right
<barry> beuno: were you able to snag the devpad png?
<beuno> abentley, it pushes down comments
<beuno> barry, yes. Super nice
<bigjools> beuno: oh also, see https://edge.launchpad.net/ubuntu/+source/gnome-disk-utility again, it appears that the slightly larger icon on the maintainer screws up the 2-column dl
<abentley> beuno: Oh, that portlet is controlling the position of the comments?  Yes, that would be bad.
<beuno> abentley, correct. It is now, so when we do the conversion, it would be great to solve that issue
<beuno> barry, it is a busy page, padding should help. noodles775 will work on improving it
<beuno> if he doesn't prod him  ;)
<bigjools> beuno: noodles is busy right now, please leave a message at the beep
<bigjools> *BEEP*
<barry> beuno: coolio
 * beuno curses in spanish
<beuno> bigjools, I don't understand the issue with the icon. I don't see it
<bigjools> your 10 minute job took 3 days ;)
<beuno> bigjools, have you not been to south america?
<bigjools> beuno: "Architectures" has a weird position
<beuno> you should really have a conversion table handy
<beuno> bigjools, ah. It does.
<bigjools> even my wildest time mark-ups didn't account for that :)
<beuno> abentley, rockstar,  the problem also occurs on the branch index: https://code.edge.launchpad.net/~bzr/bzr/trunk
<beuno> pushes down commits
<barry> does anybody know if we have any merge proposals in sampledata?
<beuno> barry, AFAIK, no
<barry> beuno: k, thanks.  shouldn't be too hard to make one up
<beuno> barry, I use firefox
<beuno> has multiple branches. etc
<barry> beuno: thanks
<beuno> QUICK! someone convert 17 templates so I can go on leave with less than 200 to go
<statik> any of you crazy launchpad developers with iphones figured out how to gpg sign emails from your iphone to make it possible to report bugs via mail from the phone?
<barry> beuno: http://devpad.canonical.com/~barry/mp.png
<beuno> barry, lovely. Lacks breadcrumbs, but that's probably not your fault
<beuno> but, LOVELY
<barry> beuno: yeah. i'm going to look at breadcrumbs next, as well as reverse breadcrumbs (boy, that sounds dirty) in the <title>.  mostly working on the h1/h2 header bits right now
<barry> beuno: excellent!
<salgado> sinzui, what are the templates that should be converted as part of https://bugs.edge.launchpad.net/launchpad-registry/+bug/421986 ?
<mup> Bug #421986: Update team list pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421986>
<beuno> barry, you rock so hard, I can barely read text
<sinzui> Sorry. I missed he message because the window closed instead of coming forward
<barry> :-D thanks man!
<salgado> sinzui, what are the templates that should be converted as part of https://bugs.edge.launchpad.net/launchpad-registry/+bug/421986 ?
<mup> Bug #421986: Update team list pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421986>
 * sinzui looks
<sinzui> salgado:
<sinzui>     teammembership-invitation.pt
<sinzui>     team-mailinglist-moderate.pt
<sinzui>     team-mailinglist-subscribers.pt
 * sinzui updated the bug
<salgado> thanks sinzui
 * barry -> short walk
<salgado> gary_poster, can I pick your brain once again?
<gary_poster> salgado: sure, but be warned, it's kinda gray and gooey, so I hear.
<salgado> heh
<salgado> gary_poster, so, I got a view with getMultiAdapter((ProductSet(), LaunchpadTestRequest()), '+new')
<salgado> that gives me a SimpleViewClass wrapping ProjectAddStepOne
<salgado> AIUI
<gary_poster> (name='+new' I suspect, but gotcha)  yeah sounds good
<gary_poster> (Whatever ProjectAddStepOne is)
<salgado> yeah, with name='+new')
<gary_poster> cool
<salgado> but then I don't have access to most of the view's attributes
<salgado> .template works
<salgado> but .heading (a class attribute) raises an AttributeError
<gary_poster> a class attribute!  A property?
<salgado> just a regular class variable, as opposed to an instance variable
<gary_poster> salgado: ...weird!  I wouldn't be surprised that a lot of instance bits are not there--__call__ (and initialize often) does a lot of stuff of course
<gary_poster> but a missing class variable is bizarre.
<gary_poster> I'm not sure how stable my LP is right now--I was debugging a build problem in Karmic--but I can try make harness and some experiments if you like
<salgado> actually, it looks like all class variables are not there. .schema isn't there either
<gary_poster> salgado: this is not security wrapped, is it?
<gary_poster> I bet it is "trusted"
<salgado> gary_poster, it doesn't look like it is. trying to removeSecurityProxy() returns the same thing
<gary_poster> oh darn, that would have made sense :-P
<gary_poster> salgado: if you walk up the __class__.__bases__ do you eventually see the class you expect?
<gary_poster> or alternatively does isinstance tell you what you expect?
<rockstar> beuno, I think I fixed that already.
<beuno> rockstar, super awesomeness
<salgado> gary_poster, it doesn't, and there we go.  this is probably because these views are part of a wizard thing
<gary_poster> salgado: ok, good to have an explanation at least.  It's been a while since I looked at the wizards.  Can I help with that, or should I leave you to attack the wizard yourself? :-)
<salgado> gary_poster, hahah. I think I know what's going on now, but I'm not yet sure I can workaround it to achieve what I want. damn wizards
<salgado> gary_poster, anyway, thanks again for the help.  I'll bug you again if I get stuck again. :)
<gary_poster> ok, np, cool :-)
<salgado> that's a lot of agains
<sinzui> beuno: My first draft of the series page: http://curtis.hovey.name/sinzui-assets/productseries.png
<sinzui> ^ The bug and blueprint portlets are not showing up...this is an issue in the view and I suspected this was an issue when I worked on this page a few months ago
<sinzui> ^ For the distroseries, I would replace the  timeline with the package search. I do not know what to put in the code area, uploads is the only candidate, but I want something that a potential contributor can act on.
<beuno> sinzui, or have a timeline for distros?  :O
<sinzui> beuno: I think that is the right answer, but I cannot commit to it for 3.0
<beuno> sinzui, I think you are a wise man
<sinzui> beuno: since package search is above the portlets in the distro page, I wanted to keep it in the same place
<beuno> sinzui, s/Set/Set date/?
<sinzui> yes, damn, there is a bug about that too
<beuno> sinzui, page is lovely
<sinzui> beuno: my thinking is that on the pillar page, you see the details, then you see where development is happening (series), on the series pages you see the detail, and how to get involved in development. So I think the distro series needs something that says what packages need looking help. may info about how to get the alpha or the build.
<beuno> sinzui, yes!  it suddenly makes serieseses SO much more interesting
<beuno> sinzui, the last piece remaining for the global 3.0 change
<beuno> is removing the edge redirect message
<beuno> and move it to the footer
<beuno> make it a green link
<beuno> "Stop redirecting to edge"
<beuno> or something like that
<beuno> kiko and matsubara know all about this
<sinzui> I can do thatr
<beuno> it was my job, and I failed them
<sinzui> That would make testing a lot easier
<kiko> beuno, what about the "This site is running pre-release code" edge bar? we need to get rid of that as well.
<beuno> kiko, that's what I was talking about
<sinzui> beuno: distroseries has https://edge.launchpad.net/ubuntu/karmic/+packaging that has some underlying call for action
<beuno> OMG
<beuno> sinzui, how about:  not showing the linked ones
<beuno> and show the unlined ones in a table, like the current ones
<beuno> with a call to action button on the right column?
<beuno> it's kinda crazy that there's no paging there
<sinzui> The page is crazy. I had to use find to locate packages in the list. I will batch it
<beuno> thank you
<sinzui> beuno: sprints is missing from the series page. It should show up because this is the natural place for a release manager to register a sprint to get work done
<beuno> sinzui, for ubuntu, I guess that's the natural place, yes
<beuno> I don't think so for anyone else
<sinzui> beuno: a driver/release manager for a project is the only person who can set an agenda. The series is what drives an agenda and is owned by drivers
<sinzui> beuno: We can restrict the link to just the drivers
<beuno> well, series don't *have* to drive the agenda
<sinzui> but the fact is anyone can create a sprint (if they can find the collection...there is no link to create a sprint there anymore)
<beuno> right, do it
<sinzui> did bigjools land code that will ellipseise long file name?
<beuno> no, he cheated on the mockups
<beuno> there is no code
 * bigjools did not
<sinzui> bigjools: what do you think the rule is? truncate the access characters before the first dot? my-long-pa...tar.gz
<bigjools> sinzui: possibly.  it can be easier if we can pass an available width though
<beuno> sinzui, I think the rule is truncate in the middle
<beuno> count the number of characters, and cut off the middle characters the exceed X
<bigjools> that was my thought
<bigjools> but you need to know how many you have to cut out
<sinzui> bigjools: I think we extend fmt:length to fmt:ellipse/32
<kiko> bigjools, beuno: why not cut out 5 ([...])?
<kiko> so
<kiko> middle = len(str)/2
<beuno> yes
<kiko> return s[:middle - 2] + "[...]" + s[middle+2]
<kiko> or something like that
<kiko> taking care of the corner cases (len(s) < middle * 2, etc)
<barry> sinzui: do we have a bug in story-ui-3 for ~person/+related-software page?
<sinzui> barry: no
<sinzui> barry: do you want to talk about it? I think we need wgrant and persia's assistance to get the use cases for the page
<barry> sinzui: no.  i'm actually looking for a simple page that i can mechanically convert, but which also should have breadcrumbs, in registry.  i actually don't care what page it is, just looking for something simple, so if you have suggestions i'm all ears
<sinzui> barry: nothing about that page is simple. it is number one on the nightmare list
 * sinzui suspects it is three pages mascaraing as one.
<barry> sinzui: i do not want a nightmare, i want a dream with unicorns and kittens :)
<sinzui> barry: I'll get you once form the list
<barry> sinzui: beauty, thanks!
<sinzui> barry: consider person-oauth-tokens.pt person-participation.pt or teammembership-index.pt
<barry> sinzui: are there bugs open on those?
<sinzui> barry: no. I thought we could discuss these tomorrow. take and report a bug as you like
<barry> sinzui: perfect, thanks!
<bac> hi sinzui, could you look at http://people.canonical.com/~bac/
<bac> sinzui: the may and maynot screen shots
<sinzui> How do I get a people URL. I am a people too. not just a monkey that is strategically shaved and put into a suit
<bac> sinzui: the question is what to do when the person cannot join.  Provide a 'Back' link, a 'Cancel' link, do nothing
<bac> sinzui: login in to rookery just like you did devpad
<bac> add a stanza to .ssh/config for rookery to go through chinstrap
<sinzui> I will try that
<bac> elmo sent mail about it awhile ago
<bac> hey!  look at my screenshots first!
<thumper> morning morning
<sinzui> Why do we have such a scary notification for mayjoin? I don't think anything extraordinary happened.
<sinzui> bac: You found a translations page~
<bac> sinzui: ignore that
<bac> sinzui: what scary notice are you talking about on http://people.canonical.com/~bac/mayjoin.png
<sinzui> bac the /!\ There is no error. I think an (i) will suffice
<sinzui> bac: is that message in the template or in the response?
<barry> beuno: ping
<beuno> barry, pongz
<barry> beuno: i have a page i'd like to talk about briefly.  making screenshot now...
<barry> beuno: https://people.canonical.com/~barry/karma.png
<beuno> barry, still no p.c.c
<barry> beuno: craptastic.  se...
<barry> beuno: https://devpad.canonical.com/~barry/karma.png
<barry> beuno: this is a purely mechanical change to +karma using my header branch.  ping me when you've got it
<beuno> barry, got it
<barry> beuno: so, first off, that H1 needs to be an H2
<barry> but then, what do you think of the breadcrumb placement?
<barry> beuno: (and then... what about the rest of the page?)
<beuno> yes
<beuno> breadcrumbs below the h1
<beuno> teh top table are totals, right?
<barry> beuno: so, breadcrumbs below the "Launchpad karma" heading?
<beuno> yes
<barry> beuno: yes, i believe so; they are totals
<barry> beuno: +1 on breadcrumb placement
<beuno> so it feels like you don't need a table for totals
<beuno> and, the table below could maybe invert the columns?
<barry> beuno: what would you use instead of a table for the totals?
<barry> beuno: for the second table, you mean Date, then Action?
<beuno> barry, make use of the apps colors, not have a heading for the table, and make the table smaller
<beuno> so mayb I do want a table
<beuno> just not like that  :)
<beuno> yes, I mean date and action
<barry> +1 on the second table
<barry> beuno: i like using the app colors
<barry> beuno: and i think you're right about no header necessary for the first table
<barry> beuno: and by smaller, you mean horizontally smaller, right?
<beuno> yes
<beuno> with the total beneat the actual numbers
<barry> beuno: fab. thanks!
<barry> beuno: this is my test page for sub-page-with-breadcrumbs :)
<beuno> barry, yay
<barry> beuno: have a good time! you will not recognize the place when you return :)
<beuno> barry, thank you, and I'll take your word for it
<beuno> I'm off home to pack! woooo
<barry> \o/
<rockstar> thumper, from my bazaar.conf aliases -> send-pipe = send -r branch::prev..
<mwhudson> thumper, abentley, rockstar: back
<mwhudson> thumper, abentley, rockstar: skype?
 * maxb blinks and wonders why 'bin/test -vv --layer=SSHServerLayer' is executing BaseLayer tests
 * gary_poster pretends he doesn't see that.
<maxb> heh
<gary_poster> maxb, fwiw, I identified why lazr.uri was causing problems.  I'll have something checked in tomorrow if I'm lucky, this week if things proceed apace.
<mwhudson> gary_poster: evening
<gary_poster> mwhudson: howdy :-)
<maxb> sounds like its high time that I started herding some of my miscellanea of tweaks into branches for landing
<gary_poster> heh, that would be good :-)
<mwhudson> gary_poster: you've got my usual morning reply emails, nothing to surprising though
<gary_poster> oh ok, looking
<mwhudson> gary_poster: who should be the default reviewer for lp:lpbuildbot do you think?
<mwhudson> gary_poster: i guess a new team that's losas + you and i
<gary_poster> mwhudson: ah yes, I had read them, but didn't have much to add.  I'm not blocking anything ATM in this regard am I?
<mwhudson> gary_poster: nope
<mwhudson> gary_poster: did you look at the code at all in my force build branch?
<spm> mwhudson: name suggestion: "legendary-osas-plus-some-orrible-dev-types"
<mwhudson> spm: lopsodt ?
<gary_poster> mwhudson: cool.  default reviewer... well, I suppose you can come and go like the wind once you leave the BE position, so sure
<mwhudson> actually i like lopsod as an acronym
<gary_poster> and then it can just be "developers"
<mwhudson> right
<spm> mwhudson: damnnit! you aren't supposed to take it seriously!
<gary_poster> lol
<mwhudson> spm: the o can be 'operationally-minded' if you like :)
<mwhudson> gary_poster: i also notice bzrbuildbot doesn't have any tests...
<gary_poster> mwhudson: didn't look at code, since I didn't have explicit request to. ;-)   happy if you wish.  probably should.
<gary_poster> if you request a review I will ;-)
<gary_poster> mwhudson: no tests:  nope.  :-/  that was among the first buildbot hacking I did.  I ended up writing tests, ever-so-painfully, for the latent slave stuff, though not for the EC2-specific bits (ran away from stubbing boto).
<mwhudson> gary_poster: this is the diff so far https://pastebin.canonical.com/21760/
<mwhudson> but it's not ready for review yet
<gary_poster> mwhudson: ok.  I actually need to run for family bits, but will spend a minute or two looking at it in a few hours
<mwhudson> gary_poster: cool
<gary_poster> bye all
<sinzui> gmb: intellectronica ping
<wgrant> sinzui: It is in fact three and a bit pages masquerading as one, while each page also exists separately, yes. The use cases for that are complex and not well defined at all.
<wgrant> It will require some thought.
<intellectronica> sinzui: yo
<sinzui> wgrant: can you send an email to the list outlining the use cases you know of? I don't think the registry team can design this page without your help
<wgrant> sinzui: I will try.
<sinzui> wgrant: thanks.
<sinzui> intellectronica:  I added the latest bugs portlet to productseries. It only shows bugs explicitly targeted to the series. I think this will be surprising to most users who know that a bug  targeted to a milestone is implicitly targeted to the series
<wgrant> sinzui: I don't think that page itself has any real use cases apart from being able to see at a glance what a person is involved in, but some of the constituent pieces have important use-cases on their own pages.
<mwhudson> man
<mwhudson> buildbot would be easier to understand if it was written in a statically typed language
<mwhudson>         self.status = req.site.buildbot_service.getStatus()
<mwhudson> aararasdsaadasfafa what sort of object is that?
<sinzui> intellectronica: I am contemplating adding another Latest...Bugs method to the view that includes the explicit and implicit series bugs. Can you think of any reason we would not want to show the bugs targeted to a series milestone on the series index page?
<wgrant> IMHO you should leave it to just show the directly targetted bugs, until somebody works out how they are meant to be used.
<intellectronica> sinzui: i don't know how useful that is (i've come to think that this sort of lists aren't very useful) but there's certainly no harm in doing that, this is certainly relevant information
<sinzui> I think milestone vs series targeting makes work complex. I once contemplated an adapter to make a series act as a milestone to make it possible to target a fix to series from inside the milestone dropdown.
<sinzui> intellectronica: right. latestbugs is really about new bugs against a series. But latestChangedBugs seems to be about bugs that are being worked on in the series
<sinzui> intellectronica: I think these two pages contradict eachother: https://bugs.edge.launchpad.net/malone/3.1 and https://edge.launchpad.net/malone/+milestone/3.0
<intellectronica> sinzui: i'm not sure i understand what you mean by contradict each other
<sinzui> The milestone page shows that theses bugs are being fixed in this series, but the series report says there are no bugs.
<maxb> I have a merge proposal for launchpad-dependencies - https://code.edge.launchpad.net/~maxb/meta-lp-deps/ready-for-karmic/+merge/11094 - is there anyone I can request a review from?
<intellectronica> sinzui: oh right, i didn't realise the 3.0 milestone is part of the 3.1 series. indeed it seems there's a contradiction
<sinzui> intellectronica: look at https://edge.launchpad.net/malone/+series
<sinzui> There are clearly bugs in the series. I am also certain that if we were to target a bug in the series the counts will be off because the code is not looking at explicit targets
<sinzui> intellectronica: This is also true for distroseries, but their workflow hides part of the problem. Since a release manager uses the serie page to plan, I think I need to explain how bugs can come from direct and indirect targets
<wgrant> Wasn't the plan to allow use of a milestone as a series, and use series tasks only for backports?
<wgrant> Er, use of a series as a milestone.
<sinzui> wgrant: yes I contemplated it. I still do.
<wgrant> sinzui: It really makes sense and would make it so much more consistent.
<sinzui> wgrant: For the people planning and working, they just need a way to say here is a point I want this bug fixed
<sinzui> they do not care about how the implementation does it
<sinzui> maxb: did you test these changes on Hardy?
 * sinzui recalls Hardy needed the package
<maxb> sinzui: No, but hardy has already diverged from the versions being used in intrepid/jaunty anyway
<maxb> hardy is using something branched off of trunk 0.47, intrepid has trunk 0.50 and jaunty has 0.53
 * sinzui really need to get pylint and tidy out of the deps
<sinzui> maxb: I do not think tidy is required by anything.
<sinzui> spm: ping
<maxb> Quite possibly. My aim here was just to prune the deps that it was extra work to get for karmic
<spm> sinzui: heyo, how's it giong?
<sinzui> spm: where does python2.4-lxml come from on our staging or lpnet? or maybe I mean to ask, where does lxml come from on those machines?
<maxb> Does it come from anywhere? I don't see it being used at all anywhere in launchpad
<sinzui> maxb: I am concerned about the XML changes. I recall a lot of pain during Ubuntu upgrades where we need to to add hacks to the code to make launchpad run on LTS and current
<spm> sinzui: v1.3.6-1 , straight from where ever default packages come
<sinzui> spm thanks.
<sinzui> maxb: lxml was a problem for the hardware database. We needed to tinker with the version used by the hardy servers and our own envs for a time
<maxb> sinzui: The XML changes? Well, lxml does not appear to be used anywhere in launchpad (or so says grep), and python-xml is the same project as sourcecode/old_xmlplus, and I've run the tests without it installed, if it's needed at all, it seems to be getting it from sourcecode/
<sinzui> ??
<sinzui> maxb: I count 6 files that use lxml
<maxb> sinzui: pastebin? because I really can't see any
<sinzui> If only I implemented that feature in my editor
<sinzui> maxb: I see that the tree talks about it, but there is no real code using it
<maxb> yup
<maxb> but I still only count 2 files of which that is true
<sinzui> maxb: ha ha
<sinzui> maxb the others are false matches
<jml> beuno-on-vacatio, sweet
<sinzui> maxb: oh old xml I pushed that up as dtdvalidator
<jml> kfogel, did I stand you up?
<maxb> sinzui: ?
<sinzui> maxb: We only used python-xml for DTD validation, and it is not supported in the modern version. I extracted the lib and put it at https://edge.launchpad.net/dtdparser
<maxb> huh. I wonder what the copy in sourcecode/ is used for, if anything
<sinzui> maxb: so we definitely do not use that package
<sinzui> maxb: We need to parse the dtd when we import mozilla translations
<maxb> oh! old_xmlplus *is* dtdparser
<sinzui> maxb your work is enlightening. r=me
<sinzui> bugger, I cannot claim the review
<wgrant> Why have breadcrumbs like "Ubuntu Â» +ppas" just sprung up?
#launchpad-dev 2009-09-03
<jml> cprov, wuu Code team
<sinzui> wgrant: beuno-on-vacatio and barry may  be able to explain. I think some more work is needed. Barry has bravely undertaken the issue
<maxb> sinzui: What do you think I should do next for this? flacoste and I conjectured that the appropriate thing to do next would be for me to commit one more revision to the branch, releasing and tagging the new version, upload the source to my PPA, and then a ~launchpad member could pull/push my branch into trunk and copy the source+binaries to ~launchpad/ppa
<sinzui> maxb: I will leave that to foundations who makes the deps.
<sinzui> maxb: I do not think we need tidy, though that is not a karmi problem
<maxb> Is there a list of who is the foundations team anywhere?
<sinzui> maxb: I wrote the frankenscript that we use for linting. it breaks with most Ubuntu releases. I have replaced it in my editor. I plan to replace the entire lint checking with a script that does not need packages.
<sinzui> maxb: flacoste, gary poster, and stub are the three people who manage that
<maxb> ok, I'll catch flacoste on irc some time, since I've been working with him for the rest of the .deb stuff
<sinzui> I see they did not organise themselves into a team
<sinzui> I see the team page has been hit by the ugly stick.
 * sinzui postpones writing a bug report because it will ruin his evening
<wgrant> What's wrong with it?
<sinzui> The details portlet should be the first one in the left column; contact the team should be a link in action menu. owner, created should be first in the details
<wgrant> I also think the map could probably go in the right column, as it would often fit alongside the very long participation list.
<sinzui> polls should not appear if I cannot create one
<wgrant> I filed that one last night.
<sinzui> wgrant: !! I suggested the same thing and continue to suggest it for the profile page
<wgrant> sinzui: Well, I've no idea what the 3.0 Person index looks like, as it seems to be hiding on devpad.
<sinzui> can you see people.canonical.com...I set myself up on that today
<wgrant> Ah, it's up again.
<wgrant> Yes, that one is public.
<sinzui> http://people.canonical.com/~curtis/
<sinzui> ^ The user pages are very old, They do show the content we want. When we started the team page, we had made big changes to the header, so I proposed that the details go to the left like pillars, and place the map on the right.
<wgrant> Ah, I see.
<wgrant> It is tempting to un-break the Packaging-related views now that it's used for something visible.
<sinzui> yes it is
<wgrant> sinzui: What do you think of the "This is a Restricted Team.(?)" thing? Doesn't HTML have a nice tag for doing that in a less confusing way?
<thumper> rockstar: ping
<rockstar> thumper, pong
<thumper> rockstar: call?
<rockstar> thumper, yea, lemme bring the dogs in the house really quick.
<rockstar> thumper, call when you're ready.
<sinzui> wgrant: Thanks for point that out. That is in the wrong place. I have no idea what the icon means. That belongs in the details...which I think we decide would be names Team information
<wgrant> sinzui: The icon has an explanation of "restricted team" as a title.
<sinzui> I see the related projects portlet did not get fixed
<sinzui> I believe it should be inline help
<wgrant> What's broken about it?
<Ursinha> rockstar, you there?
<rockstar> Ursinha, yes, but on the phone.
<Ursinha> oops.
<jml> sinzui, are you available for a call?
<Ursinha> rockstar, can we chat briefly after that about two code imports that are failing?
<sinzui> jml: I will be in a few hours. My children want me to feed them
<jml> sinzui, cool, thanks.
<rockstar> thumper, http://people.canonical.com/~beuno/branch-index.png
<rockstar> Ursinha, yes ma'am
<jml> oooh, mockups
<Ursinha> rockstar, there are two imports from sf's svn failing similarly
<Ursinha> rockstar, http://launchpadlibrarian.net/31252118/timer-applet-trunk-log.txt and http://launchpadlibrarian.net/31258356/whirlwind-sf-trunk-log.txt
<Ursinha> rockstar, is it our problem or sf's?
<Ursinha> or the branches..
<rockstar> Ursinha, can you change the import path to not use https, opting for http?
<kfogel> jml: I see what happened.
<jml> kfogel, are you available for a call now
<kfogel> jml: two things in combination: 1) we didn't put it in the online calendar, and 2) Jono *Bacon* and I did put a call in the calender, for this Friday.  He did the invite title as "Karl/Jono".  Today I looked at my calender, thinking "Oh, don't Jono (Lange) and I have a call today?", and I saw "Karl/Jono" listed for Friday, and nothing today, so I thought "Guess I must have been wrong."
<kfogel> jml: I can't -- I accepted a friend's invitation for dinner.
<jml> kfogel, ok
<kfogel> jml: let me get my cal out
<jml> kfogel, UTC 2300 tomorrow then?
<Ursinha> rockstar, sure
<kfogel> jml: tomorrow night completely booked (http://thechangeyouwanttosee.com/event/copyright-and-creative-practice-jamaica-and-beyond-dj-dance-party).  How about 2300 Friday?  That's 6pm EDT, which is right after my call with OtherJono.
 * jml considers
<jml> kfogel, sure.
<jml> 'jml' for reduced ambiguity :)
<kfogel> jml: you should have an invite now
<kfogel> jml: I did it as "Karl/Jono" again, finding that I cannot resist tempting fate.
<jml> :)
<barry> jml: i forgot about you guys tonight didn't i :(
<jml> barry, you did, I think.
<jml> barry, but that's probably ok :)
<barry> phew :)
<barry> jml: we had a bit of late night excitement today.  we got a cat
<jml> barry, I saw
<jml> barry, yay for cats :)
<barry> \o/
<Ursinha> rockstar, I've changed them both and asked to reimport, let's see
<barry> oh well, we'll *cat*ch up next week
<Ursinha> rockstar, I have another question, unrelated :) you are the guy working on answers templates, right?
<Ursinha> barry, hahahaha
<rockstar> Ursinha, yes.  I saw the bug, will work on it as soon as thumper stops talking to me.  :)
<Ursinha> rockstar, actually was more a question than a bug.. where is the assignee of the question? am I blind or is it really missing
<jml> barry, ...
<jml> thumper, I accidentally submitted a merge proposal just before the cron job was scheduled to run.
<jml> thumper, man it would be nice if it always was that quick
<thumper> :)
<rockstar> RabbitMQ ftw
<thumper> I was just talking with rockstar about message queues in that position
<mwhudson> it's totally what we should do first with message queues
<jml> is there anything actually stopping us from trying that today?
<jml> other than having other non-related things to do instead
<mwhudson> well, that last point is what stops us doing all the fun things, isn't it?
<mwhudson> we could do landing queues today, if we didn't have other things to do
<mwhudson> but i guess there's some infrastructural stuff to make decisions on (or at least find out if decisions have already been made)
<jml> mwhudson, hmm. sounds very mysterious.
<mwhudson> jml: i wasn't trying to be
<mwhudson> jml: but before we can use message queues, we need a message queue set up and running
<mwhudson> i don't know if that's been done yet
<jml> mwhudson, sorry, I didn't mean to say you were. Just that the whole process isn't particularly easy to inspect.
<mwhudson> jml: true
<mwhudson> jml: there may be a wiki page about this (ha ha!)
 * jml looks
<jml> not on the public wiki
<thumper> jml: I believe that the only thing stopping us using rabbit mq is resources to work on the problem
<jml> thumper, I was just thinking about how long it would take to get the merge proposal use-case working on a local laptop
<jml> thumper, pretending we had a magic wand to wave away all the production deployment issues.
<thumper> :)
<jml> thumper, I'm serious!
<sidnei> jml, thumper, mwhudson: we are using message queues in landscape. we should probably have a pow-wow on that. maybe on softarch?
<thumper> sidnei: what is softarch?
<jml> heh heh
<sidnei> thumper: the softarch CoP i mean
<mwhudson> i'm not in softarch yet
<mwhudson> i need to pummel gustavo into adding me i guess
<thumper> sidnei: I'm still not clear what you're talking about
 * thumper has vague recollections of an email from gustavo
<sidnei> yeah, that /me does jedi handwave
 * thumper can't find the email
 * thumper has to dash, back later
<thumper> w00t
<thumper> LCA2010 paper accepted: Using Launchpad for Code Reviews
<spm> \o/
<spm> congrats thumper, great news!
<thumper> school run, bbs
<lifeless> thumper: nice, will be seeing you there
<mwhudson> thumper: yay
<jml> thumper, woot
<jml> thumper, do you want to talk at some point about Code team hand off?
<thumper> jml: sure
<lifeless> jml: please rephrase :P
<thumper> 20m now?
<thumper> lifeless: he said hand off, not hand job
<jml> thumper, 20m from now, or 20m starting now?
<thumper> now for 20m
<jml> cool.
<jml> lets'
 * jml off for a bit
<mwhudson> for most of the older bugs tagged build-infrastructure, i have no idea what a fix might look like
 * thumper EODs
<jml> netsplit :(
<adeuring> good morning
 * wgrant plays whackamole with sprites.
 * jml is finishing up package-permission-lovwe
<jml> love, rather.
<bigjools> jml is my hero
<jml> aww shucks.
<bigjools> jml: I think our review thread has finished, unless I take it way OT :)
<jml> bigjools, heh, yeah, I think so :)
<bigjools> ooo and see lifeless's reply
<jml> yeah, just did :)
<bigjools> there ya go then, exceptions suck :)
<jml> let's use return codes for everything :)
<jml> there's one more thing I need to do for the review... move makeGPGKey
<jml> (and fix it)
<deryck> Morning.
<lifeless> bigjools: it actually sucked a lot
<lifeless> bigjools: there was a place we really wanted a catch to transform an underlying failure into a clean message, without catching too much
<bigjools> lifeless: I used to work somewhere where they were banned in our near-realtime env
<lifeless> bigjools: poolie put it in... and we had to take it out again
<lifeless> bigjools: c++ or python?
<bigjools> c++
<lifeless> yeah
<lifeless> c++ ones aren't that cheap either
<bigjools> python and realtime don't go together to well
<lifeless> bigjools: they do so; the end of the universe requires entropy!
<bigjools> arf :)
<jml> ok, added a getUniqueHexString function to the factory
<jml> and submitting to ec2test.
<noodles775> Hi deryck, I've got two questions about the inlineedit widget. It has a margin-top: -10px; - but this causes some issues for me when integrating it - do you remember what the reason for it was?
<noodles775> Secondly, the width seems to be fixed, so when the browser is made narrow, the edit-button/tab overlaps with other content... is that known?
 * noodles775 checks to see if there's a bug for it.
<deryck> noodles775, I think the top margin is to get the tab and edit icon at the top of the widget to line up correctly against the heading, i.e. "Bug description" in my case
<deryck> noodles775, and yeah, the fixed width is known and problematic
<deryck> noodles775, but not simple to fix
<noodles775> deryck: great, as long as it's known. Thanks!
<simon-o> Hi, can I run rocketfuel-setup twice or will this break things?
<salgado> simon-o, it shouldn't break anything. if it does, it's a bug that we need to fix
<simon-o> salgado: ok, I thought that. I'll report back if it worked :)
<simon-o> salgado: it doesn't work if the folder $LP_TRUNK_NAME exists, but is incomplete
<simon-o> the bzr branch command failed and it was incomplete
<simon-o> maybe you could check first if it's a valid bzr branch and nothing is missing
<salgado> simon-o, indeed. care to file a bug about that?
<simon-o> salgado: sure, I'll try to attach a merge proposal too, if that's ok ;)
<salgado> simon-o, that'd be perfect!
<flacoste> morning launchpad
<jamone1313> I have a clean Ubuntu install, I've followed the guide exactly and in the "make run" stage I get "psycopg2.OperationalError: FATAL:  Ident authentication failed for user "launchpad_main"" after that it quits, I've tried doing a "make clean.... make run" but same deal
<wgrant> jamone1313: You forgot to run launchpad-database-setup?
<wgrant> Were you following https://dev.launchpad.net/Running?
<jamone1313> wgrand: No, I ran it
<barry`> //help
<barry`> erg
 * wgrant doesn't know, then.
<jamone1313> wgrant: Just to be sure I just reran the db-setup and that fixed it. Maby it failed last time
<wgrant> https://code.edge.launchpad.net/launchpad/+activereviews is 403ing for me ATM.
<wgrant> That shouldn't happen.
<sidnei> salgado: do you know of any existing script to automate importing a bunch of old series/releases and their respective download files into launchpad?
<salgado> sidnei, I don't think we have one, but let's see what sinzui says about it.
<sinzui> sidnei, salgado : no, script, but the product-release-finder will create the milestones and releases for each series it is setup to find
<sidnei> i guess it's time to give it a try again :)
<sinzui> sidnei: there is a bug that a user claims he will fix that will ensure all the release files are downloaded.
<bigjools-afk> allenap: will your formatter work for compound file endings, like .tar.gz?
<allenap> bigjools-afk: Yeah, should do. mimetypes.suffix_map contains some .tar.gz-like ones.
<bigjools-afk> allenap: awesome!
<matsubara> Chex, gary_poster, rockstar, bigjools, henninge, sinzui, intellectronica, Ursinha: LP production meeting in 22 minutes at #launchpad-meeting
<Ursinha> matsubara, roger
<intellectronica> roger, matsubara
<simon-o> Hi, I fixed a small bug in rocketfuel-setup. I linked a branch in bug 402187. What to do next? Propose for merge or find a reviewer?
<mup> Bug #402187: rocketfuel-setup can't resume if fails to complete <Launchpad itself:Confirmed> <https://launchpad.net/bugs/402187>
<salgado> simon-o, that was quick!  you can propose a merge and then poke the on call reviewer on #launchpad-reviews
<simon-o> salgado: It was only a small fix. I have to merge into devel not db-devel, right?
<salgado> simon-o, right
<simon-o> salgado: Thanks for your help. Do I also need to sign the Contributor Agreement Form, or is this only for larger contributions?
<salgado> simon-o, I think you have to sign it for all contributions. let's confirm with kfogel
<kfogel> salgado, simon-o: the contributor agreement is something you sign once and once only.
<kfogel> salgado, simon-o: should happen with the first contribution to any Canonical project.
<simon-o> kfogel: ok, than I have to sign it now.
<kfogel> Even if it's a small fix, I'd recommend just doing it now, to get it out of the way for the future, and because (due to international copyright law being inconsistent) what counts as a "small fix" in one place maybe be considered copyrightable somewhere else.
<kfogel> simon-o: thanks
<kfogel> salgado: you know the process?  (http://canonical.com/contributors)
<simon-o> kfogel: Thanks for your explanation. I'll send it right now
<kfogel> great
<salgado> kfogel, yes, I do.  I was just wondering whether or not this applied to small contributions as well, just failed to make that clear. ;)
<kfogel> salgado: yup, it does -- see http://canonical.com/contributors/faq (which is a relatively new document)
<salgado> kfogel, ah, nice to know about that. thanks
<kfogel> simon-o: (above URL might be of interest to you too)
<barry> bac: i've made some changes to my branch which will affect the headers.  i'm going to get some lunch, but i'd love to see how your pages look with my branch.  maybe we can try some things afterward?
<bac> barry: ok.  mired in test failures ATM
<simon-o> kfogel, salgado: just saw that too. Do I need to send this to Francis Lacoste or to whom?
<barry> bac: y'know, that's a good idea.  i think i'll run my branch through ec2 while i eat
 * barry fears the output
<bac> barry: wow, long lunch!
<barry> bac:  ;)
<kfogel> simon-o: fcontributor-agreement@canonical.com, francis lacoste, and CC'ing salgado too
<kfogel> simon-o: oops, paste error, ignore the "f" on the front there :-)
<barry> bac: if i can't run the full suite on my new machine in under 15m i'm taking it back :)
<simon-o> kfogel, salgado: Thanks to both of you for your help :)
<bac> barry: if so i think elmo would be beating a  door to costco
<awilkins> Anyone know of any proposals or just requests to support Gantt charts?
<bigjools> awilkins: https://dev.launchpad.net/Wishes
<awilkins> Guess that fits in https://dev.launchpad.net/Wishes/ProjectManagementTools then
<awilkins> Thanks :-)(
<bigjools> yep!
<kfogel> maxb: are you a moin syntax guru?
<maxb> moderately so, yes
<kfogel> maxb: I'm trying to get external hyperlinks to work inside a {{{...}}} block (see https://dev.launchpad.net/Contributions/Draft -- I'm trying to make those bug numbers into links in the commit messages).
<kfogel> maxb: I've got the code to linkify the bug numbers all done, but figuring out how to make something inside {{{...}}} into a live link is harder.
<maxb> hmm
<maxb> I have this nasty feeling that this is not going to work without a custom plugin
<maxb> afaik plain {{{ }}} is "ignore all other markup inside me"
<kfogel> maxb: that's what I was afraid of.  If I knew some other way to preserve the formatting of the log messages without using {{{...}}}, I'd do that (of course, I'm fickle and don't want any *other* wiki syntax besides my hyperlinks to be live in there).
<maxb> hm. Sorry, short of writing a custom commit message formatter, I don't see any easy way of making this wkr
<maxb> *work
<kfogel> maxb: np, thanks for checking
<kfogel> s/checking/reading/
<kfogel> s/reading/commiserating/
<salgado> flacoste, ping. currently, it's Navigation._publishTraverse() that appends objects to request.traversed_objects, but that means only instances of a class that has a Navigation will be appended to traversed_objects (e.g. DistroMirror and Announcements are not). I was wondering if there's any reason why we do that in Navigation or if it would be OK to try and find another place for doing that
<flacoste> salgado: no particular reason, might be a good idea to put it into the request/publiation traversal hook
<salgado> flacoste, cool, thanks.  I'll see if I can do that later today
<sinzui> barry, bac, salgado: release planning meeting in 2 minutes
<barry> sinzui: did you send an email about it?
<sinzui> yes, the standup minutes as I said on the call this moring
 * salgado tests skype
<sinzui> barry: salgado: bac: we are looking at http://people.canonical.com/~beuno/conversions.html#registry
<barry> bac, sinzui, salgado: http://pastebin.ubuntu.com/264578/
<barry> bac: also, the person.py conflicts were not too hard to resolve, so don't worry about it.  i'll do that in my branch once yours lands
<bac> yeah, they did turn out to be easy
<barry> cool.  smerge makes it /really/ easy :)
<salgado> Entity-body was not a well-formed JSON document.
<salgado> just got that when changing the description of a bug
<salgado> deryck, have you seen that before? (^)
<deryck> salgado, no, I haven't.
 * deryck looks back through description bugs
<salgado> looks similar to bug 331990
<mup> Bug #331990: The inline editor widget reports a JSON error when saving non-ASCII characters <javascript> <Launchpad Foundations:Fix Released by intellectronica> <lazr.restful:Fix Released by intellectronica> <https://launchpad.net/bugs/331990>
<salgado> but I tried again with the same description and didn't get the error
<deryck> salgado, that's weird.
<deryck> it does sound like the same thing Tom already fixed, though.
<deryck> the multi-line editor is based of the inline editor.
<salgado> indeed, but I used the same description, so I'd expect it to keep failing
<salgado> deryck, I've filed bug 331990 just in case
<mup> Bug #331990: The inline editor widget reports a JSON error when saving non-ASCII characters <javascript> <Launchpad Foundations:Fix Released by intellectronica> <lazr.restful:Fix Released by intellectronica> <https://launchpad.net/bugs/331990>
<salgado> and attached a screenshot
<salgado> I've filed bug 423924, that is
<mup> Bug #423924: Entity-body was not a well-formed JSON document when updating bug description <Launchpad Bugs:New> <https://launchpad.net/bugs/423924>
<deryck> salgado, excellent, thanks.
<kfogel> gmb: https://dev.launchpad.net/GettingStartedIRCSession
<kfogel> gmb: (I reformatted it a bit to be more readable; I'm trying to think of where best to link to it from now.)
<bac> sinzui: do you have a preference for which bug i grab next?
 * sinzui looks
<bac> sinzui: bug 421976
<mup> Bug #421976: Update keys/wikiname pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/421976>
<bac> or bug 422974 ?
<mup> Bug #422974: Update productrelease form pages to UI 3.0 <story-ui-3> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/422974>
<sinzui> bac: 422974 should be sane lpfv
<sinzui> yes, the last one may be the easiest
<bac> sinzui: ok, i'll take it and up my velocity
<sinzui> bac: there are other bugs about the view...I will fix them over the weekend.
<bac> sheez, that bug has 5 templates.
<sinzui> bac: page_title, label main_only pagetitle deletes. It will be easy. I think all the views are alreayd tested
<bac> cool
<sinzui> edit and add are minor test changes
 * bac sets time box countdown timer to 75 minutes
<sinzui> bac: The delete release rules are tested in milestone and series (it's a mixin) so you need to add a short test to create_view() and verify the properties
<bac> ok
<sinzui> bac: I guess that means you will not see that the delete rules are broken (the series and milestones do no unsubscribe structural subscriptions)
<bac> sinzui: i'm confused.  is that brokenness filed as another bug or part of my task?
<sinzui> bac files in 3 other bugs. I just didn't want you to see one of the problems and try to fix only one.
<sinzui> I will fix all three
<bac> ok
<rockstar> sinzui, oh noes ping.
<sinzui> rockstar: oh no?
<rockstar> sinzui, yeah, I can't figure out how to link a branch to a series (setting a development focus)
<rockstar> sinzui, nevermind, the user found it.
<rockstar> sinzui, I thought we had some royal breakage there for a second.
<sinzui> rockstar: yes there are really two problems. One setting a branch to the series, and the other is "This is the development focus". I started working on this by making it more prominent: http://people.canonical.com/~curtis/productseries.png
<sinzui> Code for this series will get both actions!
<rockstar> sinzui, awesome.  I have three users that had no idea what the development focus was.
<sinzui> rockstar: Series are an enigma, and the can be used for forward planning and backward ass covering. I think we need a tutorial about them
<barry> salgado: ping
<salgado> hi barry
<barry> salgado: hi.  i'm so close i can taste it, but i'm having a small problem with breadcrumbs that i'm stuck on
<barry> salgado: have a few minutes?
<salgado> barry, sure, what's up?
<barry> salgado: so, this page: https://code.launchpad.dev/~mark/firefox/release-0.8/+merge/1/+review?claim=name16&review_type=
<barry> salgado: the breadcrumbs under the title say "Mozilla Firefox >> +review"
<barry> salgado: but i've added code to class Hierarchy to put the reversed breadcrumbs in the <title>
<barry>     @cachedproperty
<barry>     def page_title(self):
<barry>         """The page title, constructed from the reversed breadcrumbs."""
<barry>         return COLON.join(
<barry>             breadcrumb.text for breadcrumb in reversed(self.items))
<barry>  
<barry> salgado: but the title says "+review : Mozilla Firefox : Mark Shuttleworth"
<barry> salgado: i can't figure out why the breadcrumbs doesn't have mark as the first component but the page_title does
<barry> salgado: does this ring any bells?
<barry> salgado: i'm looking at launchpad-hierarchy.pt and can't see where that might be conditional, and i'm using the same Hierarchy.items property (i think anyway)
<barry> salgado: first question i guess is which would you consider right?
<barry> salgado: i.e. maybe the in-page breadcrumbs really should be mark >> firefox >> +review
<barry> salgado: oh, and this page: https://launchpad.dev/~name16/+karma
<salgado> barry, that page uses a custom hierarchy view (see BranchHierarchy in lib/lp/code/browser/branch.py), and I guess you're using the standard Hierarchy view to generate the title
<barry> salgado: has Foo Bar >> +karma as the breadcrumbs and +karma : Foo Bar as the title
<barry> salgado: gah! that must be it
<salgado> the breadcrumbs on that page are broken, though.  someone has to fix that custom Hierarchy view
<barry> salgado: thanks.  i'd seen that but didn't but 2 and 2 together
<barry> salgado: i'm not going to worry about that :)  when they fix that, both will be fixed.  i just need to add another adapter for @@+page-title
<barry> salgado: let me try that... thanks!
<salgado> barry, you're welcome. :)
<salgado> barry, I don't know how your adapter works, but you might want to use getMultiAdapter((obj, request), name='+hierarchy') to get the correct Hierarchy view instead of hardcoding the standard one there and redefining it for the objects that use a custom Hierarchy view
<barry> salgado: yeah, dang. just adding the browser:page to configure.zcml did not work
<mwhudson> morning
<barry> salgado: i didn't add an adapter, i just added a page_title property to Hierarchy, and then added the browser:page zcml
<thumper> morning
<thumper> can someone explain to my why the build has been broken for over 10 hours?
<mwhudson> gary_poster: did you see my buildbot merge request?
<barry> salgado: i would have thought BranchHierarchy.objects would be enough to make it work
<gary_poster> mwhudson: probably.  on call, one sec
<mwhudson> ok
<mwhudson> thumper: "psycopg2.OperationalError: could not write to file "base/195483/2667": No space left on device"
<mwhudson> slaves seem to be running out of space!?
<barry> salgado: ah, there are more browser:pages to add...
<salgado> barry, it does, but I think the problem is that the Hierarchy view is hardcoded somewhere, so it's being used to generate the title of that page instead of BranchHierarchy
<gary_poster> mwhudson, yes, I got it all nice and queued up for me to read this morning and have not looked at it since. :-P  looking now
<mwhudson> gary_poster: :)
<barry> salgado: joy!
<gary_poster> I also looked at the draft last night
<thumper> mwhudson: oh arse
<thumper> barry: on the new active reviews page we have approved merges shown too
<thumper> barry: you have some old ones showing
<gary_poster> mwhudson: fwiw, I contributed some bzr bits to the "contrib" directory (or equivalent, I forget the exact name) of twisted.  They included a bzr push plugin, and a poller that at one time bore some resemblance to the one we are using.  IIRC, neither of those had tests either.
<thumper> barry: which I think are just artifacts due to looms
<gary_poster> It might be nice in a theoretical sort of way to sometime get to the point that we are using the contributed versions for some of these things--so that our changes continue to enrich upstream, for instance.  Probably obviously, I didn't do that because of ease and expedience, and that may continue to be a driving factor.
<thumper> barry: can you mark them merged if they are actually merged?
<gary_poster> mwhudson: s/twisted/buildbot/
<gary_poster> that is, I contributed to buildbot, as you'd expect
<mwhudson> gary_poster: yes, that's a good goal
<mwhudson> gary_poster: our testfix mode voodoo seems a little special cased to us, perhaps, but maybe not...
<gary_poster> ah yes
<mwhudson> gary_poster: in any case, it needs testing in our environment before we do that :)
<gary_poster> agree
<barry> thumper: will do
<mwhudson> gary_poster: also, does this out of disk space error make any sense to you>
<mwhudson> ?
<gary_poster> mwhudson: "Carry on" link: +1 :-)
<thumper> ta
<mwhudson> gary_poster: yeah, that's hideous :)
<gary_poster> mwhudson: don't see out of disk space error--maybe started with a slash and was swallowed by IRC gods? Looked on lpbuildbot and didn't see anything pertinent at a quick glance.  We have had things like that before.  They usually seemed to be spurious random EC2 things.  Sometimes we did identify some way that we were not cleaning up our logs, but most of the time it seemed unlikely that we were at fault
<barry> thumper: done
<thumper> barry: awesome, thanks
<thumper> barry: in which case you can reply to my email to encourage the others :)
<barry> :)
<gary_poster> mwhudson: r=gary.  ship it.
<thumper> barry: easy _and_ fun? :-)
<gary_poster> mwhudson: thumper may have already spoken to you.  the next buildbot I think we will want is a JS buildbot, running once daily like the bzr buildbot.  flacoste is the keeper of the keys of knowledge here, while mars is out.  Please ping me about it when you have a moment, or go to thumper or flacoste.  Note that the js tests are currently failing; having a reminder of this is the point.
<mwhudson> gary_poster: sounds like an excuse to tidy up master.cfg
<gary_poster> mwhudson: +1
<thumper> mwhudson: skype?
<thumper> rockstar: skype?
<mwhudson> thumper: yeah, skype works better when it's running
<kfogel> jml: my other call tomorrow got moved; I can (& would like to) do our call earlier if possible.  How much earlier is good for you?
<barry> thumper: okay.  easy :)
<thumper> sinzui: IRevisionCache(self.context).getRevisions.order_by(revision_date).config(limit=5)
<thumper> sinzui: getUtility(IAllBranches).getMergeProposalsForReviewer(person)
<mwhudson> gary_poster: in general, shouldn't it be possible for buildbot-poll to be replaced with something that listens to a statustarget we add to buildbot?
 * mwhudson dislikes polling
<mwhudson> flacoste: you here?
<flacoste> mwhudson: still
<mwhudson> flacoste: do you want to talk about a windmill buildbot?
<flacoste> mwhudson: sure, i'll forward you the email from mars on this
<mwhudson> flacoste: how much longer are you going to be around for?
<mwhudson> thanks
<flacoste> mwhudson: i need to leave for an external dinner very soon
<flacoste> so not much actually
<mwhudson> flacoste: ok
<flacoste> mwhudson: i'll pop-in when i'm back later though
<mwhudson> flacoste: so if the mail doesn't make enough sense, i'll talk to you next week about it i guess
<mwhudson> flacoste: oh, ok
<flacoste> mwhudson: it's actually three emails
<flacoste> all sent
<mwhudson> flacoste: thanks
<flacoste> ok, i need to run
<mwhudson> bye for now
 * mwhudson decamping to a cafe in town
<bac> hi sinzui
<bac> sinzui: i'm blocked on this test:  http://pastebin.ubuntu.com/264677/
<bac> sinzui: in the 'Adding a download file' i expected to get some view.errors since some required input is missing.  i don't care to demonstrate uploading a file, just checking label and page_title, but i'm confused why the error processing isn't working
<bac> sinzui: the branch with the broken test is at lp:~bac/launchpad/bug-422974-productrelease
<mwhudson> biab
 * bac -> dinner
<wgrant> I still can't access launchpad's +activereviews.
#launchpad-dev 2009-09-04
<sinzui> bac: do you need to move the test the LaunchpadpadFunctionalLayer because the file need the librarian?
 * sinzui looks
<bac> sinzui: no, b/c i'm not really hitting the librarian.  the form validation machinery should kick in and flag the missing input
<bac> wgrant: is that a permission problem?
<sinzui> didn't you say "Adding a file"...That is librarian
<wgrant> bac: Yes.
<bac> sinzui: but the form should never get there.
<wgrant> bac: It was accessible a couple of days ago, but wasn't last night and still isn't now.
<bac> wgrant: is there an open bug?
<wgrant> bac: No. Is it one?
 * bac looks at permission settings
<bac> wgrant: that make no sense
<wgrant> bac: No private branches ATM?
<wgrant> And no easy way to track down the OOPS?
<bac> no
<wgrant> Ah, it's only a problem on edge.
<sinzui> bac: I do not think the cgi.escape is needed. base-layout escapes the page title using standard tales
<bac> sinzui: ok.  i was following barry's example
<sinzui> bac: there is no structural call...and if there was a problem., all the labels in all the forms would be bad.
<sinzui> I think the form label and the page_title should be identical... what does one say file and the other say download
<bac> sinzui: i was being all mechanical. i didn't give it much thought.  i'll change that.
<bac> i don't understand your "no structural call" comment.
<sinzui> I don't think about it either. I set the label to the page title, the page title is the label
<sinzui>     page_title = label
<bac> wgrant: the page should be public.  i don't see anything referenced on the page that is private.  stumped.
<sinzui> launchpadFormView make it easy
<bac> sinzui: i think the wiki says some should append "in launchpad" in the page title
<wgrant> bac: There was an 1800 line revision making large changes to that page yesterday. I'm going to see if I can reproduce it locally.
<sinzui> bac: wgrant: This may related to the changes by thumper
<sinzui> bac: that is new then. Sorry
<wgrant> sinzui: Right, that's the merge I'm looking at.
<bac> sinzui: no, it needs to be sorted out.
<sinzui> bac: actually, if that is required, we should not change any pages...that must be done by base-layout
<bac> sinzui: so you have no ideas on the test failure?
 * sinzui is not change 300 pages again
<sinzui> no, I looked at the request and see nothing happened. I do not thine the action is firing
<bac> sinzui: sorry, i'm being called to dinner.  let me know if you have any thoughts.
<bac> sinzui: it was firing, because i got a code error earlier when i omitted the contenttype
<bac> it must be there, we assume it is, and so i got a keyerror
 * bac off to eat.  bbiab.
<jml> now?
 * jml apologises
<jml> kfogel, earlier as in, now?
<sinzui> bac: you did not include all the information requires to get the error. You passed in some field, but none are in a bad state. Make the description an empty string to get an error
<wgrant> thumper: Would the 403ing of launchpad's +activereviews to unprivileged users (eg. me, or the unauthenticated user) be due to your super-branch yesterday?
<thumper> wgrant: almost certainly
<sinzui> bac:remember that the web form is passing more fields, which is an opportunity for bad data. Consider playing with 'filecontent' if you really want a disaster. I think you can skip this though since the form is being tested else where (a unittest and story I think)
<thumper> wgrant: file a bug
<wgrant> thumper: Thanks.
<jml> hello
<jml> are we out of testfix yet?
<mwhudson> jml: should be
<jml> mwhudson, thanks.
<mwhudson> jml: is there a bug "it's impossible to find out if we're in testfix" yet?
<jml> mwhudson, I'll file one
<jml> and link from the Glue page
<mwhudson> thanks
<jml> done
<bac> sinzui: i expected to get an error because the file is not being passed
<bac> sinzui: but i will happily forge on
<sinzui> but the form clearly does not require it
<sinzui> please do.
<bac> sinzui: in the schema it is marked as required
<bac> in the web ui it is required
<sinzui> and what does the widget do.. or make it
<sinzui> s/or//
<mwhudson> jml: woo for landing that branch!
<mwhudson> sinzui: are we still using pagetitles.py in the 3.0 layout?
<thumper> mwhudson: no
<thumper> mwhudson: only as a fallback
<mwhudson> thumper: hooray
<thumper> mwhudson: prefer page_title in the view
<mwhudson> thumper: when can we kill it?
<thumper> mwhudson: we are slowly killing it
<mwhudson> yay
<thumper> mwhudson: people should be removing stuff as they migrate pages
<thumper> at least I am
<mwhudson> there is a bug that basically says "pagetitles.py is full of obsolete cruft"
<thumper> probably not
<mwhudson> my preferred option for fixing it is deleting it
<thumper> but I added a comment to the top of the file
<mwhudson> thumper: an oooold bug
<thumper> that says DON'T ADD SHIT HERE
<mwhudson> :)
 * thumper heads for lunch an dcoffee
<mwhudson> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/1584 fwiw
<mup> Bug #1584: Tests don't check for titles of non-existent pages <build-infrastructure> <Launchpad Foundations:Triaged by stevea> <https://launchpad.net/bugs/1584>
<sinzui> mwhudson: Not officially. we want to remove it next cycle. base-layout  preferred the view.page_title. once we have converted all pages, we need to check that pagetitles.py is empty
<mwhudson> sinzui: cool, sounds like a plan
<mwhudson> OOPS-1342ED479
<mwhudson> fine, i'll make my own url then
 * mwhudson lunching
<mwhudson> biab
<flacoste> hi mwhudson
<mwhudson> flacoste: hello
<flacoste> mwhudson: any questions regarding the JS builder?
<mwhudson> flacoste: the js builder seems simple enough, i just need to torture the OSAs into making suitable images
<flacoste> ok, cool
<mwhudson> flacoste: we can't use mars generate script directly, root ssh login to the slaves is turned off
<flacoste> what is the workaround?
<mwhudson> flacoste: the osas have accounts in sudoers
<mwhudson> flacoste: so i'll just make them do it by hand
<flacoste> can't we update the script to work with sudo?
<flacoste> so that the OSA simply have to run the script?
<mwhudson> flacoste: it seems all the script really does is apt-get -y install xvfb firefox xfonts-base
<mars> mwhudson, yep, that's about all that is needed
<mwhudson> mars: oh hello
<mars> hi
<mwhudson> mars: shouldn't you be changing nappies or something?
<mars> just passing by
<mwhudson> :)
<flacoste> hmm, ok, then it should be fine
<mars> I just changed two
<mwhudson> flacoste, mars: should we run the jscheck builder after every landing or nightly?
<flacoste> mwhudson: nightly for now
<mars> mwhudson, be sure to have the slave monitored for memory leaks, too
<mars> part of the suite crashes in bad ways
<mwhudson> mars: it'll run on ec2
<mwhudson> mars: i'm not sure what you mean?
<mars> by crashes?  or 'our JS test suite may have memory leaks'?
<mwhudson> mars: "be sure to have the slave monitored for memory leaks"
<mars> mwhudson, when I was running 'make jscheck' on my local system, part of the suite appeared to be segfaulting, or at least prematurely terminating Firefox and Windmill
<mwhudson> mars: well, that will certainly cause build failures
<mars> heh
 * thumper is fiddling with loggers for the base mailer
<mars> mwhudson, basically, someone needs to keep an eye on the build slave, to see that it doesn't die mysteriously.  I would hate to see someone's time wasted tracking down issues, when a bit of forewarning would have saved them.
<mwhudson> mars: ok
<mwhudson> mars: otoh, i'm worrying about making it run
<mwhudson> mars: making it pass gets the SEP treatment
<mars> SEP?
<mars> mwhudson, that will be great, I'm looking forward to it :)
<flacoste> mwhudson: any more questions, i'm about to head off afk again?
<sidnei> hey mars!
<mwhudson> mars: Someone Else's Problem
<mwhudson> flacoste: nope, go have an evening
<flacoste> thanks, talk to you next week probably
<mars> hi sidnei
<mwhudson> mars: have you not read the hitch-hiker's guide to the galay books?
<mwhudson> +x
<mars> mwhudson, I have
<deryck> oh look, there's mars
<mars> heya deryck
<deryck> mars, hey hey
<mwhudson> mars: the spaceship powered by bistromaths in life, the universe and everything is hidden using a SEP field
<mars> ah!
 * deryck is out for the night.  Until tomorrow y'all...
<bac> hi mars!
<mars> hi bac
<bac> how's it going?
<bac> mars:  when are you back?
<mars> pretty good.  Had a moment, so decided to drop by
<mars> bac, not for a month yet :)
<bac> :(
<bac> good for you, though
<mars> I'm using it to my best advantage
<mars> it still takes hours to get the day rolling
<bac> great.  i'll talk to you later.  was on my way out.
<mars> we have relatives over, playing with the boys, so I can actually sit at the computer for more than 5 minutes :)
<mars> good night!
<thumper> mwhudson: I need some suggestions for testing my email problem
<thumper> mwhudson: got a few minutes?
<mwhudson> thumper: yes, but i'm in a cafe
<mwhudson> thumper: i'll move to the library and skype you?
<thumper> hmm...
<thumper> ok
<mwhudson> thumper: talk to you in a few minutes
 * thumper nods
 * mwhudson fights with skype a bit
<mwhudson> ffs
<mwhudson> thumper: have you used the new pulseaudio skype with a headset yet?
<thumper> no
<rockstar> thumper, I has returned.  Can we talk on the phone now?
<thumper> oh ffs
<rockstar> mwhudson, abentley said you can't redirect the ringing the computer speakers and the actual phone call to the headset.
<rockstar> thumper, sorry.  We don't have to have the call.
<thumper> rockstar: just about to have a quick call with mwhudson
<thumper> rockstar: that wasn't directed at you
<mwhudson> rockstar: i'd be happy to have everything going through the headset
<rockstar> thumper, :)  I figured it wasn't.  Just wanted to be slightly comical
<mwhudson> thumper: :/
<thumper> rockstar: doesn't translate too well over irc :)
<rockstar> thumper, yeah, but I was going for quotes page material, so it had to be taken entirely out of context.  :)
 * thumper stabs rockstar
<mwhudson> thumper: i win!
<mwhudson> thumper: still want to skype?
<thumper> mwhudson: yes
<mwhudson> thumper: sorry
<abentley> thumper: Also, I think the faint vocals you were complaining about yesterday were because PulseAudio was routing from my laptop mic, not my headset mic.
<mwhudson> thumper: it's gone quite
<mwhudson> quiet even
<thumper> :)
<mwhudson> thumper: i could call your cell phone...
<thumper> paramatise from above
<thumper> mwhudson: I think I have enough now
<mwhudson> thumper: ok
<thumper> mwhudson: thanks
<mwhudson> thumper: np
<thumper> rockstar: call now?
<rockstar> thumper, yes
<mwhudson> abentley: it sounded like that, actually
 * jml is back
<kfogel> jml: ping
<jml> kfogel, pong
<kfogel> jml: hey, so re tomorrow's meeting (uh, trying to think if it's also tomorrow for you...)
<jml> kfogel, it would be tomorrow for me
<kfogel> jml: yeah, tomorrow morning
<jml> kfogel, Sat 8am.
<kfogel> jml: it's already 8am for you?
<jml> kfogel, no, that's the scheduled time for the meeting
<kfogel> jml: sorry, let's talk in absolute times.  that was what I meant: "the meeting time we have set is already 8am for you?"
<kfogel> so, moving it earlier doesn't work unless we move it *much* earlier
<jml> kfogel, that's right.
<kfogel> jml: let me get my timeanddate.com on
<kfogel> jml: ok, you are 14 hours ahead of me
<kfogel> jml: it's 5:11am UTC right now, 12:11am for me, and 2:11pm (fri) for you.  Right?
<jml> kfogel, yes.
 * mwhudson should move to the cook islands or tahiti for extra confusion of this sort
<kfogel> jml: whew, wow.  Okay, I guess we keep the original meeting time for this one.
<kfogel> mwhudson: don't you dare
<kfogel> mwhudson: but let's be glad we have no interplanetary developers
<jml> kfogel, yeah. thanks :)
<jml> oi, the lag :)
<kfogel> jml: you're thanking me for a Saturday 8am meeting?  I don't even have to train you :-).
<jml> kfogel, heh
<thumper> kfogel: imagine the lag on skype for someone in orbit let alone another planet :)
<mwhudson> or someone on satellite internet, i guess
<mwhudson> (which you would be on the cooks)
<jml> hmm
<jml> anyone around who does LP dev on karmic?
<wgrant> jml: me
<jml> wgrant, I'm now trying to get my laptop set up for karmic dev, rather than inside a chroot
<jml> wgrant, I hit a bug, but solved it by RTFMing :)
<wgrant> jml: What was the bug? launchpad-dependencies not installing due to a missing python-xml?
<jml> wgrant, I didn't have https://edge.launchpad.net/~launchpad/+archive/ppa in my sources.list
<wgrant> jml: Doesn't rocketfuel-setup do that for you?
<jml> wgrant, what makes you think I use rocketfuel-setup? :P
<wgrant> jml: Ah, I see.
<jml> gah, now postgres won't install
 * jml reboots ... maybe that helps
<jml> ok that proved to be unnecessary
<jml> it was because I had postgresql on a ramdisk
<jml> hey
<jml> how do I get rid of a hardy chroot without blowing away my home dir
<jml> wgrant, have you got errors about not being able to import from Crypto.Cipher.
<mwhudson> how do you escape $ in makefiles?
<thumper> $$
 * thumper thinks
<mwhudson> ah
 * thumper is done
 * mwhudson vanishes in a puff of hating EC2
<wgrant> jml: I don't get such errors, no. Do you have a 2.4-compatible python-crypto?
<wgrant> jml: Karmic's isn't sufficient.
<jml> wgrant, I do now!
<jml> wgrant, I wonder why lp-dev-dependencies doesn't depend on it.
<wgrant> jml: It looks to me like python-crypto in ~launchpad/ppa is sufficiently 2.4-enabled. Did lp-dev-deps not pull that one in?
<jml> wgrant, apparently not.
<wgrant> or did you not run an upgrade after activating the PPA, just installing the new package?
<jml> it might have been that.
<jml> I just installed lp-dev-deps
<wgrant> Ah.
<wgrant> If you'd upgraded, it would have pulled the right one in.
<jml> I'll amend the wiki page.
<wgrant> rf-setup does it all for you.
<jml> wgrant, yeah, but...
 * jml blogoblags
<wgrant> What aboot?
<jml> wgrant, activereviews
<jml> wgrant, I have a queue of other things I'd like to publish, but I think I'll spread them out :)
<wgrant> jml: Bad choice of day.
<jml> wgrant, why so?
<wgrant> jml: launchpad's activereviews is borked.
<jml> on edge?
<jml> really?
<wgrant> Yes. 403 for unprivileged users.
<jml> grunk
<jml> I'll pull that one, and post the cranky one about layers then.
<wgrant> Oh. Fixed now.
<jml> \o/
<maxb> jml: hiya. Do let me know where your python-xml came from if you figure it out :-)
<jml> maxb, I lack apt-fu -- how can I find out?
<maxb> well, dpkg -l python-xml will tell you the version
<jml> (10 years of using debian-based systems and I'm still a noob)
<jml> maxb, 0.8.4-10.1ubun
<maxb> oh, it has chopped it to fit the column
<maxb> dpkg -l python-xml | cat
<maxb> will bypass the auto chopping to your terminal width
<jml> 0.8.4-10.1ubuntu2
<maxb> well that's the jaunty package.
<maxb> I've no idea how you convinced a karmic system to install it without specific extra steps
<jml> umm
<maxb> unless you still had it installed from upgrading from jaunty, and didn't let update-manager remove obsolete packages
<adeuring> good morning
<jml> maxb, well, I did some crazy things when I first upgraded to karmic
<noodles775> Hi adeuring
<adeuring> hi noodles775!
<wgrant> jml: I didn't actually say you needed python-xml; that was just my memory of why rocketfuel-setup failed for me on karmic.
<jml> wgrant, oh, my bad.
<wgrant> Was one of the 3.0 design goals actually 'eliminate absolutely all padding'?
<jml> :\
<jml> I hope not.
<wgrant> Looking at https://code.edge.launchpad.net/~bac/launchpad/bug-421983, I think it was.
<wgrant> Look at that block of fields near the top.
<jml> wgrant, I've noticed the trend
<wgrant> jml: Ah, good.
<wgrant> Does LP's copy of devel suck, or does bzr?
<jml> wgrant, why do you ask?
<wgrant> jml: A checkout downloads around 250MB, and writes around 250MB to disk. A bzr pack takes it down to 110MB.
<spiv> There's a couple of bzr bugs that were fixed very recently that might contribute to that.
<wgrant> The packs on bazaar.lp.net aren't quite so obscenely oversized, but they're still almost twice as big as they need to be.
<jml> it'd be nice to have a 'pack' button on Launchpad
<jml> (as well as an 'upgrade' button)
<wgrant> And a 'seriously fully remirror' button
<jml> oh yeah, that too.
<wgrant> Does LP's mailman do something silly like not send me a copy through the list if I'm in the To?
<maxb> it would seeme so
<maxb> annoying, that
<wgrant> Yes.
 * wgrant files a bug.
<jml> http://paste.ubuntu.com/264827/
<jml> maxb, do you know anything about that particular error?
<maxb> ahhhh
<wgrant> jml: Just grab the right version from ~launchpad's PPA.
<wgrant> jml: You'll have to manually downgrade.
<jml> ahh ok.
<wgrant> It wants 1.5c IIRC.
<wgrant> Then it's happy.
<wgrant> Until it tells me that I don't have an image available.
<maxb> Also I didn't bother repacking things for karmic that I couldn't find what they were needed for - and being that I can't ec2test, I didn't do anything with python-boto
<jml> wgrant, that works, thanks.
<jml> wgrant, have I filed a bug about getting a public image
<wgrant> jml: You haven't, I don't think.
<maxb> you may want to copy the python-boto in the LP PPA to karmic series
<jml> maxb, I don't know how to do that? :)
<wgrant> jml: No, you haven't.
 * jml does so
<wgrant> jml: Thanks.
<wgrant> maxb: That's not entirely useful, as it's an *older* version, so it would need to be explicitly installed. Better is to convince ec2test to work with 1.8.
<jml> +1
<jml> how about I file a bug about that too
<wgrant> jml: A grand idea.
<jml> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/424197
<mup> Bug #424197: ec2test doesn't work with python-boto 1.8 <build-infrastructure> <ec2test> <Launchpad Foundations:New> <https://launchpad.net/bugs/424197>
<jml> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/424197
<mup> Bug #424197: ec2test doesn't work with python-boto 1.8 <build-infrastructure> <ec2test> <Launchpad Foundations:New> <https://launchpad.net/bugs/424197>
<jml> *ahem*
<jml> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/424198
<mup> Bug #424198: Public EC2 image for testing Launchpad <build-infrastructure> <ec2test> <Launchpad Foundations:New> <https://launchpad.net/bugs/424198>
<wgrant> jml: Thanks.
<jml> 51 build-infrastructure bugs :(
<wgrant> mwhudson has quite a job.
<jml> we all do!
<jml> he's only BE for this month
<wgrant> True.
<jml> and personally, I don't see the role as an excuse to not fix the build system
<maxb> argh. So remember I got a bunch of meta-lp-deps branches reassigned from ~maxb to ~launchpad? Well it looks like LP doesn't update the stacked-on URLs in the branches......
<maxb> fail
<jml> maxb, yeah, that's a known bug.
<jml> maxb, I can fix the broken branches for you, if you'd like.
<jml> (although my mana is running pretty low right now)
<wgrant> jml: Is ec2test happy now? It has probably not been run on Karmic before.
<maxb> jml: lp:~launchpad/meta-lp-deps/{all but trunk} if you have time
<jml> wgrant, looks like it.
<maxb> all of which are supposed to be stacked on trunk
<jml> wgrant, but I have a conflict
<jml> maxb, hmm. the easiest way might be to make a copy of trunk at the old location
<jml> there's only three though
<gmb> Can anyone remind me of the runes for running a single story with bin/test
<gmb> ?
<gmb> <-- Person who has been doing lots of back-end work.
<intellectronica> gmb: bin/test -vv -t the-test.txt
<jml> gmb, ./bin/test -cvvt foo.txt
<jml> in fact, I always use ./bin/test -1cvvt foo.txt
<jml> because the second failure never helps.
<wgrant> -1? -c?
<jml> wgrant, -c == color
 * intellectronica doesn't even know what -c does
<wgrant> -1 I can now guess.
<jml> wgrant, -1 == show only the first failure for any given doctest.
<wgrant> Right.
<jml> (the other failures still exist, they are just hidden)
<intellectronica> nice
<jml> I'm pretty sure it didn't work with the old zope.testing
<gmb> jml, intellectronica Thanks.
<gmb> That'll do nicely.
 * gmb thought it was still different from running doctests
<gmb> (e.g bin/test -vvf canonical $test
<gmb> )
<jml> gmb, I don't think you need -f ever.
<gmb> Right.
<jml> I really want to see if I can use trial or nose or bzr selftest or something to run LP tests now that we've upgraded our zope.testing
<jml> or I should fix tribunal & double check that our subunit output works
<jml> that'd be fun
<bigjools> jml: would using those bring any advantages?
<jml> bigjools, yes.
<jml> bigjools, most of them are nicer, basically
<jml> bigjools, better error reporting or more fun or more extensible in some way
<bigjools> faster? :)
<jml> bigjools, unlikely
 * bigjools thinks about porcine aviation
<jml> bigjools, maybe one of them is smarter about test discovery & searching...
<jml> bigjools, that'd save some time for local red-green-refactor cycles
<bigjools> indeed - the startup time for doctests gives me an earache
<wgrant> The world would be a better place if tests didn't depend on LibrarianLayer unnecessarily.
<jml> wgrant, earlier today I mentioned a cranky blog post about layers...
<wgrant> jml: So you did.
<jml> wgrant, basically, the bug is that our tests should declare what they actually need one by one, rather than name a layer that they hope contains everything they need.
<wgrant> jml: Right.
<bigjools> that'd be nice
<wgrant> Can the testing machinery do that?
<jml> if we weren't using zope.testing, it would be pretty straightforward
<jml> we'd use testresources & that would be that.
<jml> but zope.testing kind of craps all over any unittest extensions you might want to use
<jml> so it requires a little cleverness to get things in
<jml> also, there'd be a substantial amount of work in migrating
<jml> I haven't looked at the code for the more recent versions of zope.testing. It might be a lot better (I'm told it is)
<jml> oh, there's also the thing that some layers are unteardownable
<jml> so you have to run all the tests that use them in a subprocess.
<wgrant> Ah, I wondered why it did that.
<jml> zope.testing has some amazing contortions to make this work
<jml> but you can do it nicely with subunit
 * jml really should JFDI some time
<jml> g'night all
<deryck> Morning, all.
 * wgrant is amused at spammers who spam bugs filed by LOSAs.
<bac> hi deryck
<bac> danilos: can you follow up with https://answers.edge.launchpad.net/rosetta/+question/81816
<henninge> danilos: I can take it, danilos is vey busy today.
<bac> thanks henninge
<henninge> bac: done
<bac> thanks!
<flacoste> barry: nice update on the h1/h2 rules!
<flacoste> barry: has the branch landed?
<barry> flacoste: not yet.  hope to get it into review today, but i have test failures :/
<barry> flacoste: i may ask for a review anyway, fix the tests in parallel and ask for an updated review if the test repairs call for it
<barry> flacoste: if you're up for looking at it, let me know!
<barry> flacoste: otherwise i'll ask ocr salgado
<flacoste> barry: how big is it?
<barry> flacoste: 1035 lines
<barry> so yeah, i suck
<flacoste> i'll pass :-)
<barry> k, np :)
<fjlacoste> ba, sinzui: do you know how to control the 'Featured project list?'
<sinzui> fjlacoste: There is a hidden page that joey can see to set it. We do not have permission
<fjlacoste> sinzui: launchpad.Admin?
<fjlacoste> i don't think joey has that anymore
<fjlacoste> but kiko and the losa will
<fjlacoste> sinzui: if you tell me the secret page, i'll ask a losa to make some changes
 * sinzui is still looking
<sinzui> I is Admin on /+featuredproject
<sinzui> ^ they can see the "Manage featured project list" on /
<sinzui> flacoste: Should we change the permission to registry-experts?
<flacoste> sinzui: maybe, it's not that common though
<c0rtis0n> Hi; just a question (#lauchpad guided me to you): when I want to upload a patch to solve a bug reported on launchpad which files are needed
<maxb> A bug in Launchpad itself, or a bug in a project which uses Launchpad for bug tracking?
<c0rtis0n> the second
<maxb> In that case, this is the wrong channel for you (and so is #launchpad). You should contact the project concerned.
<c0rtis0n> ok thank you very much
<kfogel> wgrant: https://wiki.ubuntu.com/MeetingLogs/devweek0909/HackSoyuz doesn't say what server & channel, unless I'm missing something.
<kfogel> ah, #ubuntu-classroom
<kfogel> it's on parent page
<rockstar> matsubara, your tarmac branch is probably also going to conflict with a branch that Tarmac is chewing right now.
<matsubara> hey rockstar
<rockstar> matsubara, hi.
<matsubara> rockstar, thanks for letting me know. once that's landed on trunk I'll update mine and solve any conflicts
<rockstar> matsubara, CIA says it's landed.
<matsubara> rockstar, I'm getting a bzr: ERROR: Invalid format version '5' when I try to bzr merge lp:tarmac. do you know how can I fix that?
<rockstar> matsubara, bzr upgrade --2a
<rockstar> matsubara, I did this before, but something broke it.
<rockstar> Also, you'll want to upgrade the push location, which you can do with hitchhiker
<matsubara> rockstar, https://pastebin.canonical.com/21846/
<rockstar> (abentley makes the coolest tools)
<rockstar> matsubara, eep.  Um. try a bzr reconcile.
<matsubara> rockstar, https://pastebin.canonical.com/21848/
<rockstar> matsubara, what if you create a new branch and merge your branch into it?
<matsubara> rockstar, got the same bzr: ERROR: Invalid format version '5' error
<matsubara> rockstar, hmm I need to go for lunch. I'll ping you when I get back
<rockstar> matsubara, when did you get the error?  Branching or merging?
<matsubara> merging
<rockstar> matsubara, okay.  I'll chat when you get back.
<matsubara> https://pastebin.canonical.com/21849/
<matsubara> rockstar, ^
<maxb> Where does the CVE XML data come from?
<matsubara> rockstar, sorted out the error by upgrading the remote branch first
<rockstar> matsubara, awesome.  it's ready to land then?
<matsubara> rockstar, yes, just pushed the changes to https://code.edge.launchpad.net/~matsubara/tarmac/tarmac-bug-418823 with your review comments. I'll reply in the MP
<rockstar> matsubara, great, thanks.
<matsubara> rockstar, done. MP updated with commit message
<rockstar> matsubara, ack.
<matsubara> rockstar, argh, https://code.edge.launchpad.net/~matsubara/tarmac/tarmac-bug-418823 now shows one of those yellow boxes
<matsubara> man this format changes in bzr are pretty annoying
<rockstar> matsubara, yeah, they're being sorted.  I'll get it figured out.
<dhillon-v101> gmb: hi how are you
<dhillon-v101> hi everyone
<sinzui> What do I run in dev to put uploads in the queue. I am trying the see the Latest uploads portlet
<wgrant> sinzui: You go to #ubuntu-classroom 10 minutes ago.
<sinzui> bugger
<wgrant> sinzui: Does the portlet use PackageUploads or something else?
<sinzui> getLatestUploads returns looks like PackageUploads
<wgrant> So it does.
<wgrant> So it looks like you just need to get a SoyuzTestPublisher and run addPackageUpload a few times.
<wgrant> noodles775 has a nice screencast for how to use STP.
<bigjools> getPubSource will call that for you
<wgrant> Ah, useful.
<bigjools> PU is not much use on its own
<noodles775> sinzui: out of interest, what page are you doing the latest uploads portlet for? I might be able to re-use it :)
<sinzui> distroseries
<sinzui> noodles775: This is what I have so far: http://people.canonical.com/~curtis/distroseries.png
 * sinzui is tuning the portlets for the series
<noodles775> sinzui: wow... looks great!
<noodles775> sinzui: do you have a mockup handy showing what your latest uploads portlet will look like?
<noodles775> Ah, I was looking in the wrong place :)
<sinzui> noodles775: that is it in the lower left
<sinzui> I want to see some more to be certain it works. I also underestimated the labour to adapt the milestone table.
<sinzui> oh, the link is missing
<noodles775> sinzui: Did you try out the SoyuzTestPublisher?
<sinzui> not yet. I decided to do the easy template pieces
<noodles775> aha, the screencast that wgrant meantioned is here:
<noodles775> http://micknelson.wordpress.com/2009/09/04/testing-launchpad-soyuz-features/
<sinzui> thanks!
<noodles775> Night all.
<maxb> Oh, *GAH* I forgot about that classroom session
<noodles775> maxb: heh, well, I hope the above is useful :), and the log of the whole session should be available soon :)
<maxb> Also, I am the author of the RemoteAccess page in its current form, so I can assure you that it does work, though it assumes you're confident with understanding Apache vhosts
<maxb> gah
<maxb> that was meant for #launchpad
<wgrant> WTF
<wgrant> Bug #17 just got attacked.
<maxb> Can someone peek at buildbot for me and tell me why stable is currently so far behind devel?
<jml> how do I submit to launchpadlib?
<jml> maxb, it went offline
<maxb> :-/
<kfogel> jml: getting my skype on
<maxb> At this point, should I expect it to be dead until Monday?
<jml> maxb, yes. I think this is pretty lame, and hope we can do something about it
<jml> kfogel, skype is being recalcitrant. gimme a few moments.
<kfogel> jml: no problem, I got stuffs to do
 * wgrant hopes Skype will die die die die die once Empathy is a little better.
<kfogel> wgrant: is Empathy some kind of Skype clone?  Open source?  Company making money on the service, just like Skype, or something like that?
<wgrant> kfogel: Empathy is the new GNOME IM client.
<wgrant> kfogel: It does Jingle, which is the audio/video protocal that Google puts on top of XMPP.
<kfogel> wgrant: *nod*  but does it do the "dial ordinary phones" thing Skype does?
<wgrant> kfogel: It doesn't.
<wgrant> Although I suppose an XMPP gateway could be written.
<jml> kfogel, ok
<jml> kfogel, let's go :)
<kfogel> jml: hey
<kfogel> let's retry, BUT
<jml> kfogel, I could hear you just fine...
<kfogel> let me first send this time-sensitive mail to atlas travel re london trip -- their response just came in with flights, and I should respond now if I can
<jml> kfogel, ok :)
<jml> aouheoastisn
<kfogel> jml: ok, you try me this time I guess
<kfogel> maybe that makes some kind of difference
<kfogel> maxb: are you coming to that meetup on 28 Sep evening in London?
<maxb> I am
<kfogel> ah great!
<kfogel> see you there!
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 2.2.8 | https://dev.launchpad.net/ |  Please use #launchpad for support. | Get it: https://dev.launchpad.net/Getting | http://paste.ubuntu.com/ | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html | Buildbot is down
#launchpad-dev 2009-09-05
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 2.2.8 | https://dev.launchpad.net/ |  Please use #launchpad for support. | Get it: https://dev.launchpad.net/Getting | http://paste.ubuntu.com/ | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html | Buildbot is coming back up
<maxb> https://dev.launchpad.net/DetailedCoverLetterTemplate is quite daunting
<maxb> does it really all apply for a 3-line change?
<maxb> e.g. http://bazaar.launchpad.net/~maxb/launchpad/py2.5-buildmailman/revision/9329
<mthaddon> maxb: I imagine (but am definitely not really that qualified) that you'd want to include all those section, but if they're not relevant, very briefly explain why
<jml> maxb, I almost never follow the template & I've yet to get in trouble :)
<maxb> :-)
<jml> maxb, I find it helpful to think of the headings as a checklist
<jml> maxb, so that when I'm writing a cover letter, I think "Have I explained the implementation? Have I mentioned how to do QA?" etc.
<jml> flacoste, do you know how to submit a branch to launchpadlib?
 * jml keeps getting rejected by PQM
<wgrant> jml: Merge to trunk!
<wgrant> It's not PQM-owned.
<jml> wgrant, ahh, ok
<jml> my local trunk branch is branched from pqm
<jml> hmm
 * jml corrects
<wgrant> jml: Is there a DEV_SERVICE_ROOT yet?
<jml> wgrant, no, how about I sneak one in with this change
<wgrant> jml: Thanks.
<jml> hmm
<jml> we should set lp:launchpad to append_revisions_only
<wgrant> lp:launchpadlib, you mean?
<jml> no :)
<jml> although that too
<wgrant> But lp:launchpad isn't pushable to.
<jml> I almost typed 'bzr push lp:launchpad' from my launchpadlib branch.
<wgrant> So why does it matter?
<jml> wgrant, it is by me :)
<wgrant> Oh.
<wgrant> Of course.
<jml> I don't want permissions to do so, but I have them.
<wgrant> Wasn't that meant to go away?
<jml> I couldn't persuade thumper to do so.
<jml> well, not in the short term
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 2.2.8 | https://dev.launchpad.net/ |  Please use #launchpad for support. | Get it: https://dev.launchpad.net/Getting | http://paste.ubuntu.com/ | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html | Buildbot is down
<maxb> When a test reports failure, without any error messages whatsoever, how do you debug?
<wgrant> maxb: Pastebin?
<maxb> preparing
<maxb> http://paste.ubuntu.com/265476/
<wgrant> maxb: Odd. But my guess is that you don't have the right python-sqlite.
<maxb> hmm
<wgrant> Or sqlite2 or whatever it is.
<maxb> I seem to have both
<wgrant> That's a pretty wild guess, though.
<wgrant> Does it say anything more if you run just that test with lots of '-v's?
<maxb> no :-(
<wgrant> maxb: Does -D drop you into anything useful?
<maxb>     self.assertFalse(broken_builder.builderok)
<maxb> hm
<maxb> eww
<maxb> we really ought to disallow assertions without messages, or make them show a traceback
<maxb> It's throwing a twisted.trial.unittest.FailTest and that's disappearing into the voids
<maxb> *void
<wgrant> Ah, yuck.
<wgrant> Ah, that's from the top failure which I had no idea about.
<wgrant> In what environment is this failing?
<maxb>  karmic/py2.4 + a few innocuous patches which I can't believe are related
<maxb> but I'll revert them anyway and try again
<wgrant> Hmm
 * wgrant tests.
<maxb> on lp:launchpad/stable
<wgrant> maxb: Um, it fails here too, but doesn't contribute to the failure counts.
<maxb> !
<wgrant> How odd.
<wgrant> maxb: Ah. Sampledata change.
<maxb> ooh
<maxb> right
<wgrant> maxb: But not in the direction that I would expect...
<maxb> but....... doesn't make check reload it all every time?
<wgrant> Yes.
<wgrant> There's something strange here.
<wgrant> The test should fail.
<wgrant> And I suspect buildbot logs will say it does.
<wgrant> But it's not counting in the totals.
<wgrant> This is very very bad, or we are very confusing.
<wgrant> (the change was to make 'bob' in the ftest sampledata OK, so this test should obviously faily)
<wgrant> But if it's in stable, the test never failed.
<wgrant> But we know it did.
<maxb> unpossible! :-)
<wgrant> And:
<wgrant> Failure in test testScanRescuesJobFromBrokenBuilder (lp.buildmaster.tests.test_manager.TestBuilddManagerScan)
<wgrant>  testScanUpdatesBuildingJobs (lp.buildmaster.tests.test_manager.TestBuilddManagerScan) (1.593 s)
<wgrant> That makes me very scared.
<wgrant>   Ran 4 tests with 0 failures and 0 errors in 9.342 seconds.
 * wgrant retries the test with old and new clean sample data.
 * maxb kicks off a testrun of devel/jaunty/py2.4 and goes out shopping
<wgrant> security.py is not being friendly.
<maxb> and after shopping, I shall get stuck into writing merge cover letters
<wgrant> Fun fun.
<wgrant> maxb: So, that test has been failing. I thought to check an ec2test log from a couple of days ago, and it's shown there (the only failure).
<wgrant> But the test suite completes successfully.
<maxb> riiigh
<maxb> t
 * wgrant files a bug.
<wgrant> Two bugs
<maxb> well, that sucks, and I have more weird failures too
<maxb> For example a couple of launchpadlib-1.5.1's doctests are exploding because I'm getting bytestrings and the doctest expects unicode strings
<wgrant> Same.
<james_w> maxb: I landed a fix for that a few weeks ago I thought
<james_w> ah, that was wadllib
<james_w> http://bazaar.launchpad.net/~lazr-developers/wadllib/trunk/revision/11
<james_w> might help though
<maxb> Data point: running make check on an AAO takes 9Â¼ hours
<wgrant> maxb: *Ouch.*
#launchpad-dev 2009-09-06
<MTeck> I need to do a project for my university. My first thought was making dd have a hasning function so you could say dd if=/from of=/to hash=in ; then it would hash the file as it's being read so the file is only read from disk once. Apprently dd can do the same thing. So... You guys have any ideas for how my group could help LP? We do need a clear goal to accomplish.
<jml> umm
<jml> MTeck, you want to contribute patches to LP?
<jml> MTeck, how much time does your group have? :)
<MTeck> jml: You think somehow we could develop a clear goal? Is there any feature that you guys want added that we could do
<MTeck> ~5mo
<jml> MTeck, we could definitely come up with something.
<MTeck> :)
<jml> MTeck, is there anything on LP that interests you in particular?
<MTeck> branches - loggerhead - bugs - teams
<jml> MTeck, cool.
<MTeck> I think those are my favorite part :P
<MTeck> I did like the tabs better for the UI - but I can see where removing images is a good idea
 * jml nods
<jml> MTeck, I'm definitely sure that we can find something in one of those that would be suitable
<MTeck> Was it a bandwidth decision?
<jml> MTeck, also, what TZ are you guys in?
<MTeck> -6
<jml> MTeck, It was partly a bandwidth decision, and partly a communication-of-structure thing
<jml> MTeck, I'll chat to a couple of the team leads & get back to you.
<MTeck> :)
<jml> logging out to test a gnome thing
<jml> brb
<jml> time to repeat the ipod experiment
<jml> back soon.
<jml> how do I get the exit code of the last command?
<MTeck> All I know about getting an exist status is either an error message handler, %d, or echo $?
<wgrant> jml: $? is the shell variable, as MTeck suggested.
<jml> thanks.
 * jml is giving up on Banshee, and trying gtkpod for syncing
 * wgrant gave up on Banshee when he couldn't work out how to use its DAAP plugin and Launchpad at the same time.
<wgrant> jml: LH wants to be kicked very much.
<jml> I can't really do anything about it, I'm afraid.
<wgrant> Damn.
 * MTeck uses totem when he really needs audio/video - but much of the sound type
<jml> I guess I should use gstreamer to convert all my flacs to mp3s and then try gtkpod again
<jml> transcoding is not its strength
<jml> I wish banshee didn't rely so much on HAL, dammit.
<jml> and now I can't even figure out gstreamer
 * jml takes a break
<MTeck> This is how I like to start coding (same as when I started usling Linux) - http://img196.imageshack.us/img196/6990/arcticfoxhunting.gif
<jml> heh
<MTeck> jml: head first :D
<wgrant> jml: What's broken with HAL?
<jml> wgrant, podsleuth uses HAL to find iPods. There's some crazy interaction between it and gnome's auto mounting stuff.
<wgrant> jml: Possibly related to the HALectomy? Or was this a problem pre-Karmic as well?
<jml> wgrant, HALectomy, I think. Haven't tested pre-Karmic, it's a new ipod
<jml> wgrant, F11 users are reporting the same thing
<jml> http://bugzilla.gnome.org/show_bug.cgi?id=586508
<wgrant> jml: Yeah, that looks like it.
<wgrant> Nautilus has used DeviceKit-disks for some time now.
<jml> I don't really know what to do about it, other than change banshee / podsleuth to use DeviceKit / udev / whatever
<lifeless> ooh look people
<jml> lifeless, what?
<jml> lifeless, btw, tribunal is now in 2a & works with testtools.
<wgrant> Is 2a stable now?
<lifeless> jml: cool
<lifeless> jml: subunitification?
<jml> lifeless, writing an email about that now :)
<lifeless> wgrant: its been supported since 1.16
<lifeless> I'm trying to decide how best to plugin in a SubunitLogObserver to arbitrary buidl steps in buildbot
<wgrant> lifeless: But it had some big issues back then. Are they all gone now?
<lifeless> wgrant: 2.0rc2 should have no operational issues using it
<wgrant> lifeless: OK, thanks.
<jml> lifeless, .... and sent
<lifeless> jml: [replied]
<jml> make build is broken in stable :(
<jml> oh wait, that's just karmic
<wgrant> jml: There are also test failures on stable.
<jml> wgrant, test failures?
<wgrant> jml: yeah, one test is failing without counting towards the failure counts, so the suite passes.
<jml> wgrant, sheesh.
<wgrant> jml: 'make build' works fine here on karmic/2.4. What's wrong with yours?
<jml> wgrant, I'll try again & paste.
<jml> it fails on wadllib generation
<jml> http://paste.ubuntu.com/265972/
<wgrant> jml: WFM :(
<jml> wgrant, up to date with download-cache?
<wgrant> Wow.
<wgrant> That's impressive.
<wgrant> jml: No.
 * wgrant updates
<wgrant> I probably should get onto those remaining formatting-related 2.5 failures at some point.
<jml> I was going to fix ec2test on karmic, but...
<wgrant> It needs make build?
<jml> hmm, I guess it doesn't.
<jml> but 'make build' is pretty essential to almost everything else
<wgrant> It works for me with up-to-date devel and download-cache.
<jml> I'll give it a try with devel
 * jml runs branch-flow
<wgrant> branch-flow?
<jml> wgrant, it's a utility I wrote that lives in utilities/.
<jml> wgrant, give it a burl :)
<lifeless> I always forget how ugly unittest.py is
<lifeless>         self.testsRun = self.testsRun + 1
<lifeless> _why_
<jml> lifeless, it predates +=, iirc.
<lifeless> all the same
<lifeless> its had lots of time to be cared for
<wgrant> jml: Ah, that's very nice.
<jml> wgrant, thanks.
<jml> it's not so nice that we have 26 untested revisions in devel.
<wgrant> jml: No. But I guess that'll be fixed in less than 24 hours.
 * jml hopes so.
<wgrant> But is make build still broken?
<jml> yeah.
<jml> I think it's a python-xml thing.
<jml> I recall someone saying something about xmlplus at some point.
 * jml uses a proprietary web service to find out where.
<wgrant> Remove python-xml?
<wgrant> It's not needed.
<jml> I'll try that.
<jml> wgrant, what's the test that's failing but not failing the suite?
<wgrant> jml: lp.buildmaster.tests.test_manager.TestBuilddManagerScan.testScanRescuesJobFromBrokenBuilder
<wgrant> jml: It fortunately shows up in ec2test success emails, so I know it's *only* that one.
<jml> removing python-xml does the trick
<wgrant> jml: It started failed in r9267 due to sampledata changes (bug #424797)
<mup> Bug #424797: lp.buildmaster.tests.test_manager.TestBuilddManagerScan.testScanRescuesJobFromBrokenBuilder failing silently <soyuz-build> <spurious-test-failure> <Soyuz:Triaged> <https://launchpad.net/bugs/424797>
<wgrant> Hmmmm.
<wgrant> That's odd.
<jml> oh good, there _is_ a bug.
<jml> that's something
<wgrant> I considered filing a -foundations bug about the lack of detection of the failure. Should I?
<lifeless> yes
<lifeless> if something is wrong, file a bug :)
<jml> wgrant, build-infrastructure tag please
<jml> hey look, it's a trial test case
<jml> I know this!
<wgrant> lifeless: Well, it might have just been a bug in the test.
<wgrant> It is a subclass of something I've not seen before.
<lifeless> wgrant: eitherway, is it not a bug ?
<wgrant> lifeless: Yes, but there's already a bug about the test being broken.
<lifeless> oh
<lifeless> I'd file one about the detection
<lifeless> if its a false positive bug, no harm done
<wgrant> Thanks.
<lifeless> jml: # tests: current=51, target=53.0
<lifeless> jml: from buildbot
<jml> lifeless, ECONTEXT
<lifeless> jml: subunit to buildbot progress
<wgrant> Bug #425113
<mup> Bug #425113: Some tests can fail without detection <build-infrastructure> <Launchpad Foundations:New> <https://launchpad.net/bugs/425113>
<lifeless> jml: "I have a patch to make buildbot show test progress if there is subunit in the output."
<jml> lifeless, ahh, awesome.
<wgrant> Very nice.
<jml> hmmmmmmmmmmm
<lifeless> Its so helpful that 'git diff' does not show what you will commit.
<lifeless> <sigh>
<wgrant> lifeless: Is there a way to work around that?
<lifeless> git add .; git diff --cached
<wgrant> Ah.
<lifeless> or use bzr-git and 'bzr diff'
<wgrant> Hahah.
<wgrant> That always hits me when I use git, but that's rare enough that I can never be bothered to look it up.
<lifeless> I had done a git add
<lifeless> so when git diff showed me nothing I was ... puzzled
<jml> ok
<jml> so what's happening is that our newer zope is demanding even more things from the traceback object than it used to
<jml> trial, in this case, is converting a twisted Failure object into an exc_info-alike tuple
<wgrant> jml: Is this related to there not being any output?
<jml> wgrant, yes.
<jml> wgrant, zope.testing is raising an AttributeError that somehow gets swallowed.
<lifeless> \o/
<wgrant> jml: Yay.
<lifeless> and failing open, lovely.
<wgrant> The best kind.
<jml> AttributeError: "'_Frame' object has no attribute 'f_locals'"
<wgrant> What's eating that?
<jml> not sure yet.
<lifeless> heh there isn't a BuildBot dev.lp.net wiki page
<jml> wgrant, probably something in trial.
<jml> lifeless, Trunk/Glue gets close
<lifeless> where is the buildbot status page?
<wgrant> jml: Ah, good.
<wgrant> lpbuildbot.c.c?
<jml> lifeless, https://lpbuildbot.canonical.com/
<lifeless> hmm, we already have test counts. \o/
<lifeless> jml: was my tribunal feedback useful ?
<jml> lifeless, yes, thanks.
<jml> lifeless, I'm having attention problems today, sadly.
<lifeless> fair enough
<lifeless> I had my allergy shot today
<lifeless> so I'm lethargic, sore, and hard of focus
<jml> untangling why trial is swallowing this error is going to be a challenge
<jml> and I don't think I care enough
<lifeless> is it trial swallowing it?
<jml> yeah.
<jml> http://twistedmatrix.com/trac/ticket/4003
<lifeless> hmm
<lifeless> SubunitTrialReporterTime
<lifeless> lp:twisted ?
<jml> lifeless, see the #twisted topic
<lifeless> thanks
<jml> (I think it's lp:twisted, but I can't remember)
<lifeless> yes
<jml> I've linked the Twisted bug & made a comment about the whole thing.
<lifeless> just quickly
<lifeless> is there anything in the ReporterProtocol that I should know about?
 * jml thinks
<jml> it does todos & skips differently from what you might expect
<jml> addError being called more than once per test is expected
<jml> instead of exc_info tuples, it expects t.p.failure.Failure objects
<jml> PyUnitTestResultAdapter should be a fairly good read in this respect.
<jml> oh, and the constructor is part of the contract, iirc.
<jml> sorry about tha.
<jml> t
<lifeless> class TestResult(pyunit.TestResult, object):
<lifeless>     def __init__(self):
<lifeless> doesn't seem that onerous?
<jml>     def __init__(self, stream=sys.stdout, tbformat='default', realtime=False):
<lifeless> oh
<jml> class Reporter(TestResult):
<lifeless> so this is a lie:
<lifeless>     implements(itrial.IReporter)
<jml> *sigh*
<jml> yes
<jml> I tried fixing that ages ago, and got blocked on something stupid
<lifeless> _observeWarnings seems to be on totally the wrong class, to me.
<lifeless> anyhow, lets see if this work
<lifeless> s
<lifeless> ok
<lifeless> what tests would you like? :)
<lifeless> jml: ^
<jml> uhhh
<lifeless> http://paste.ubuntu.com/266006/
<jml> lifeless, we really do aim for total coverage.
<lifeless> so, putting aside how to test the missing import block
<lifeless> which is thorny
<jml> lifeless, fwiw, there's a typo in the missing import block
<jml> 'Exceptioin'
<lifeless> do you have an interface conformance suite for reporters?
<jml> lifeless, sadly no.
<jml> there might be a test for the trial script
<lifeless> thanks for the typo note
<jml> to see which reporters are shown in the help
<jml> ... apparently not.
<lifeless> twisted.trial tests pass for me
<lifeless> (using --reporter=subunit too :P)
<jml> lifeless, I can't think of any appropriate tests.
<lifeless> ok
<lifeless> I'll write something plausible
<lifeless> but it works!
<jml> lifeless, cool.
<lifeless> http://twistedmatrix.com/trac/ticket/4004
<jml> yeah, I saw :)
<jml> thanks
<jml> there'll need to be some formatting tweaks too, but nothing that major.
<lifeless> there may be some friction
<lifeless> I don't know that trial is capable of downgrading properly to stock TestResult's
<lifeless> e.g. UnexpectedSuccess may go boom
<lifeless> jml: what is the reason that the result object hooks into log observing?
<jml> lifeless, kind of historical reasons
<jml> lifeless, log.err() calls in tests should fail tests
<lifeless> jml: sure; shouldn't each test hook in though?
<jml> particularly because that's how Twisted reports unhandled errors in deferreds
<jml> lifeless, I guess...
<jml> hmm
<jml> I wonder...
<lifeless> jml: so, I'm not saying 'hooking into log is odd', I'm saying 'Reporter hooking into log ...'
<jml> lifeless, it's been added since I last touched the code.
<jml> lifeless, so, I don't know the reason.
<jml> lifeless, http://paste.ubuntu.com/266016/
<lifeless> yes, that sounds nice.
<lifeless> I don't see that being at all incompatible with my question :)
<lifeless> its a separation of concerns issue, is all.
<lifeless> my reporter won't do this, because its totally outside the responsibilities of a reporter..
<lifeless> unless I multiple inherit
<lifeless> and I'm not sure what would happen if I do that :(
 * lifeless shruggifies
<lifeless> ooo, EINTR.
<lifeless> hmm, I suspect subunit doesn't loop no EINTR at the moment
<lifeless> jml: how important is this^ ?
<jml> lifeless, it's important.
 * lifeless files a subunit bug
<jml> lifeless, as for the location of responsibility, well, I haven't paged any of that code in, but what you say makes sense
<jml> fwiw, it looks like the only change needed to make ec2test work in karmic is to remove the version checking for boto :)
<wgrant> jml: That's what I suspected, but best to be safe.
<lifeless> jml: interestingly, I see this:
<lifeless> Ran 535 tests in 3.165s
<lifeless> PASSED (expectedFailures=1, successes=534)
<lifeless> Total tests:     516
<lifeless> Passed tests:    515
<lifeless> Failed tests:      1
<jml> lifeless, that is interesting.
<lifeless> 18 missing tests :)
<lifeless> -or-
<lifeless> 18 overreported tests
<lifeless> jml: further thoughts
<lifeless> jml: PyUnitResultAdapter would in some ways be better
<lifeless> for subunit; as subunit wants to be a real boy.
<lifeless> but 4005
<lifeless> its kindof late for yak shaving though
<jml> lifeless, yeah.
<lifeless> urgle
<lifeless> addExpectedFailure is sadly different to python 2.7
<jml> yeah, I kind of wish they at least investigated prior art before standardising
<lifeless> hell no
<lifeless> heresy!
<lifeless> http://twistedmatrix.com/trac/attachment/ticket/4004/subunit.patch
<lifeless> now what?
<jml> lifeless, you want to land it?
<lifeless> its ~ done
<lifeless> uhm, I think I need to add Failure support
<lifeless> which I'm about to do. then its done
<jml> yep
<lifeless> It may handle Failures already
<lifeless> as it just stringifies stuff it doesn't recognise
<jml> lifeless, you probably want to read over http://twistedmatrix.com/trac/browser/trunk/doc/core/development/policy/coding-standard.xhtml?format=raw
<lifeless> hmm, needs to be improved there
<lifeless> jml: thats a longish document
<lifeless> jml: what do I have wrong?
<jml> docstrings are """\nFoo.\n"""
<lifeless> shudder
<jml> "All unit test methods should have docstrings specifying at a high level the intent of the test."
<jml> two blank lines between methods
<jml> three blank lines between classes
<lifeless> I have to presume there are good reasons for this
<jml> lifeless, there aren't.
<jml> when it's ready to land, please add the tag 'review' and assign it to me.
<jml> I'll review it (and if appropriate, land it) tomorrow,
<jml> I'm going to step away from the computer for the evening.
<lifeless> ok
<lifeless> thanks
<jml> np
<lifeless> are you jml in matrix?
<lifeless> jml: enjoy, its in your paddywhack :)
<MTeck> jono: the whole irc thing is getting to me. I'm on the verge of not coming back for a year or to or ever - just in case I go off the deep end...
<MTeck> hrm... wrong channel
<MTeck> jml: I am excited to work on that project though
<mwhudson> good morning
<lifeless> hai
<thumper> morning
<thumper> mwhudson: call?
<mwhudson> thumper: yes
<thumper> mwhudson: how different is Launchpad's bzr-git from trunk bzr-git?
<mwhudson> thumper: not very, i thought
<mwhudson> thumper: easy to check cd ~/mumble/sourcecode/bzr-git; bzr missing lp:bzr-git
<thumper> mwhudson: we are using bzr-git out of sourcecode still?
<mwhudson> thumper: yes, aiui bzr doesn't work with plugins in eggs
<mwhudson> thumper: but admittedly i haven't really looked into the issue yet
<thumper> ok
 * thumper checks missing
<lifeless> eggs are evil
<lifeless> and no, we don't
<wgrant> lifeless: However, LP moving to eggs should make it easier to move to real packages later.
<lifeless> wgrant: uhm, that seems totally orthogonal
<lifeless> wgrant: just moving straight to real packages would be less likely to run into egg specific issues
<wgrant> lifeless: Not quite. It provides a greater incentive to use unhacked versions of the dependencies.
<lifeless> wgrant: which moving to real packages would do to
<wgrant> lifeless: Right, but I suspect it's easier to move to real packages from eggs than from sourcecode/.
<lifeless> wgrant: I have no reason to think that that is the case
<wgrant> OK.
<lifeless> eggs are a poor proxy for real packages
<wgrant> Certainly.
#launchpad-dev 2010-09-06
<thumper> bet that farmer is pissed
<thumper> how's he going to have a straight line of trees now?
<lifeless> wgrant: anyhow, I'mm not asking you to decide, I'm just going for as much input as I can get
<wgrant> lifeless: I know.
<wgrant> I don't really like the idea of rolling out with a known hole.
<wgrant> Plus it's easily CPable to the single host once the request path is fixed.
<wgrant> So I don't see the urgency, since the appserver code is fine.
<wgrant> (librarian updates can be done without downtime now, yeah?)
<lifeless> handwave
<lifeless> in principle yes
<lifeless> spm: do you have a few minutes?
<lifeless> spm: I want to talk RT 41202 and some related bits
<spm> ha! no. not atm. need abot 30+ mins. 11+ reds to deal with.
<lifeless> ok
<lifeless> I'll list out the bits here for when you get time
<lifeless>  - check the cert order can be done
<lifeless>  - check host headers getting to th ebackend librarian
<lifeless>  - check we can deploy librarian updates w/out downtime
<lifeless> and seperately
<lifeless>  update edge once we figure out whats up with bb again
<wgrant> So, I suggest the following:
<wgrant>  - Hook in the new view, but initially FF'd out.
<wgrant>  - Get the hack in r11506 controlled by FF>
<wgrant>  - Release.
<wgrant>  - Get request path sorted.
<wgrant>  - Get domain match fix CPed to librarian.
<wgrant>  - Test.
<wgrant>  - Flip the two FF flags
<lifeless> same flag, surely ?
<wgrant> Could be.
<lifeless> it hits the same view
<lifeless> so when the view is controlled by the ff, it will affect both, no ?
<lifeless> bbiab
<lifeless> spm: OOPS reporting on edge is naffed; just needs a redeploy. Add to the bottom of your reds list ?
 * lifeless awols
<spm> snort
<wgrant> lifeless: Yeah, but I'd like to have a way out.
<wgrant> Or Platform might kill someone :)
<wgrant> lifeless: Is IBuilder:+history featuring on the timeout reports?
<lifeless> wgrant: well I haven't filed a bug for it yet
<lifeless> wgrant: its gotta be close though
<lifeless> 17.68 99% completion time
<lifeless> wasn't in the top oops report in the weekend though
<wgrant> lifeless: Yeah, I've had a few reports of it today.
<lifeless> ok
<lifeless> did anyone give you an OOPS?
<lifeless> not that they are any use till edge is redeployed
<wgrant> OOPS-1709A1140 is one.
<lifeless> wgrant: FWIW https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=timeout is my master list
<lifeless> wgrant: no such oops, well not yet anyhow
<wgrant> Really?
<wgrant> It should exist by now, surely.
<wgrant> It was 8 hours ago.
<lifeless> got any from different server?
<wgrant> OOPS-1709F1169
<lifeless> that works
<lifeless> so, 'A' may not be rsyncing right
<lifeless> wgrant: https://bugs.edge.launchpad.net/soyuz/+bug/631206
<_mup_> Bug #631206: Builder:+history timeouts <timeout> <Soyuz:New> <https://launchpad.net/bugs/631206>
<wgrant> It's not fixed on edge.
<wgrant> OOPS-1709EB1925, for example.
<lifeless> wgrant: if you want to let me know what constants should be in there, or how to determine them, I'll do an explain analyze on staging
<lifeless> poolie: hi
<poolie> hi there lifeless
<lifeless> poolie: feature flags; whats the best place to look for a howto ?
<poolie> the docstring
<poolie> which i think is now up on the web
<lifeless> [and I hope you had a good weekend etc etc]
<poolie> you too
<lifeless> it was interesting
<lifeless> poolie: I'm trying to match up with sinzui did in commit 11470 and the docstring
<poolie> r11470 of devel?
<lifeless> poolie: lp.services.features.__init__ is the docstring I'm looking at
<lifeless> poolie: yes
<poolie> heh, that diff is interesting as an example of the kind of tests we should make it easier to write
<poolie> it's not all that bad, but itcould be smaller
<poolie> i'm really sorry it caused disruption
<lifeless> poolie: meh, don't worry about it
<lifeless> our process makes lp fragile like that
<poolie> +        def in_scope(value):
<poolie> +            return True
<poolie> +
<poolie> +        return FeatureController(in_scope).getFlag('gmap2') == 'on'
<poolie> this is a bit strange
<poolie> i would have just used the default controller, and relied on only trusting the 'default' scope for now
<poolie> it may be that api wasn't there in the initial landing to db-devel, which only had the sql and model code
<lifeless> http://itsnotthecoffin.blogspot.com/2010/09/christchurch-earthquake-new-zealand.html
<poolie> i mean, two orthogonal things:
<poolie> 1- that might have been the api skew that bit him
<poolie> 2- it's not wrong but it's not how i would have written it
<lifeless> how would you have written it? (So I can do the same)
<wgrant> lifeless: SELECT id FROM builder WHERE name='adare';
<wgrant> That's the first one.
<lifeless> 3
<poolie> lifeless, return getFeatureFlag('gmap2') == 'on'
<poolie> also perhaps we should add a thing for casting to bools, etc, so we don't have "== 'on'", "== 'yes'" etc all over
<wgrant> lifeless: The two for archive privacy are false.
<wgrant> lifeless: And use just about anyone for the person.
<poolie> lifeless, i'm going to ask spm to send the error output from process-mail.py to a log file synced to devpad
<wgrant> I forget my ID.
<poolie> rather than as at present being sent as mail and being ignored
<poolie> can i get a t-a rs for this?
<lifeless> poolie: theres an ongoing effort to rearrange things along those lines
<lifeless> I understand it to need code changes
<lifeless> rather than being a deployment issue
<poolie> code changes might help
<lifeless> my understanding from mthaddon is that the goal is:
<lifeless>  - must-see remain in email
<poolie> but here we just have a cron job and i want to put a '>file' into it
<lifeless>  - activity, details, etc go to a log
<lifeless> and we have the script-last-run mechanism to cope with as well
<lifeless> I haven't looked into how that works yet
<poolie> hm
<poolie> i wonder if anyone is reading them now
<lifeless> I was speaking with thumper about it this morning
<poolie> i suspect they get a mail for every mail received by launchpad
<poolie> so should i file this, or not?
<lifeless> poolie: anyhow, I'm ok in principle with any change to make the thing more diagnosable and useful; I have two concerns here: there are automated things that look for scripts running/not running and I don't know if changing mails will affect that
<lifeless> secondly I don't know where mthaddons change to make only errors be emailed is at either
<lifeless> oh, and thirdly disk space - in the current setup the logs are archived in the mail archive system, devpad isn't setup to scale to huge numbers of logs (e.g. we have to prune OOPS reports to conserve space)
<lifeless> poolie: I'll happily put my stamp to a change once those things are looked into
<poolie> maybe i'll just ask spm to pull things out of mail for me one-off then
<lifeless> I wish I knew this bit of the system better to be able to just say 'yeah, doit', but sadly I don't
<lifeless> I wonder if you can get a filtered mbox from mailman
<wgrant> lifeless: Hm, edge has 40x more non-SQL time?
<wgrant> It'd be handy if the timeline showed when we were GIL-blocked :(
<lifeless> wgrant: thats extremely har dto do
<wgrant> lifeless: Of course.
<lifeless> wheee that query is slow
<lifeless> 22 seconds
<wgrant> Ow.
<lifeless> plan in the bug
<wgrant> Yep, already reading.
<lifeless> 171MB disk merge.
<lifeless> \o/
<wgrant> I wonder why it decided to do that.
<wgrant> I can't see where the final 20s went, though.
<wgrant> Unless it was in that disk merge, and didn't show up in the Sort times.
<lifeless> qTime: 1811.852 ms
<lifeless> thats my rearranged one
<wgrant> What did you do?
<lifeless> put the condition in the join
<lifeless> rather than bringing back a metric tonne of unrelated data and filtering
<wgrant> Interesting.
<lifeless> no guarantee I got it right
<lifeless> but the first few rows certainly look the same
<lifeless> conceptualy we want:
<lifeless> oh and here
<wgrant> That is slightly confusing.
<lifeless> I had one bit I wasn't totally happy with
<lifeless> tweaking now
<lifeless> so the archive left join team-p thing brings back one row from team participation per archive, not *all the rows of the team*
<wgrant> Ah, good.
<lifeless> because (person, team) is unique in teamparticipation
<lifeless> archive.owner will be correlated against archive
<lifeless> but the planner has some choice there
<lifeless> we end up with an archive, teamparticipation table with one row per archive
<lifeless> and we then join that to packagebuild where packagebuild has an archive set
<lifeless> hmm, I think we can cut a left join here
<wgrant> Where?
 * wgrant looks.
<lifeless> packagebuild - archive
<lifeless> we always want to look at archive
<lifeless> if bfj.packagebuild is set
<wgrant> We do.
<wgrant> But how does that let us eliminate a left join?
<lifeless> (packagebuild inner join archive)
<wgrant> Oh, one of your compound joiny thingies which I haven't seen widely applied before.T
<wgrant> True.
<lifeless> 2.6 seconds. heh
<lifeless> fiddling at this level is risky
<lifeless> ahh
<lifeless> get rid of both, its much happier
<lifeless> 930ms
<lifeless> spm: how goes the meltdown ?
<lifeless> wgrant: left outer join means you have to iterate both sides rather than iterating one side and doing specific lookups on the other side
<spm> lifeless: more in a semi solid state atm; just kicked off a u1 staging DB whatsits; so should be able to spare you some attention shortly
<lifeless> getting rid of the packagebuild query may help a great deal
<lifeless> spm: ok, say in 40 ?
<spm> lifeless: that should be fine; hopefully sooner... but I was ever optimistic.
<lifeless> brb
<lifeless> wgrant: you may wish to read http://www.postgresql.org/docs/8.4/static/explicit-joins.html
<wgrant> lifeless: Ah.
<lifeless> basically the goal of the planner is to do the most effective work first
<lifeless> as soon as we explicitly constrain things it can't
<lifeless> and left outer joins explicitly constrain things simply by being used.
<lifeless> spm: hi
<lifeless> spm: thats 40 and a bit :)
<lifeless> wgrant: if you're tweaking soyuz queries
<lifeless> wgrant: bug 629921 may be of interest
<_mup_> Bug #629921: Archive:+packages with empty name search does like '%%' search. <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/629921>
<lifeless> spm: tap tap tap
<spm> :-)
<spm> lifeless: so aiui, 41202 is predominantly a GSA thing. Also; you've set the pri to 89, but haven't given us any indication of timing needs around this? do you need this for the rollout this week? or later this month? or Mid next. ??
<lifeless> spm:
<lifeless> blah
<lifeless> soon as we cna, before rollout if possible
<lifeless> spm: but first, can you please trigger an edge redeploy
<lifeless> spm: to fix OOPSes
<wgrant> lifeless: So, what do you think of the plan I outlined?
<wgrant> For the librarian stuff.
<wgrant> And has kees looked at the thing yet?
<lifeless> wgrant: sure, something like that
<lifeless> no response from him yet
<wgrant> :(
<spm> lifeless: does that mean we *need* it for this rollout? we're operating *really* short staffed this week, so "I want" vs "I *need*" is really necessarilly separated atm.
<lifeless> critical path is knowing that *we can get them*
<lifeless> can't land the code till thats acked.
<lifeless> The code is intended to be able to be turned on at will, so if we get the certs a few days later, thats fine.
<lifeless> When I say can't I mean 'it would be a bit odd to land something we're really not sure if we can do'
<lifeless> spm: the code proposal https://code.edge.launchpad.net/++oops++/~lifeless/launchpad/private-librarian/+merge/31020
<lifeless> spm: so, short story - this has had plenty of eyeballs.
<lifeless> spm: there's still hair and fine tuning, but less than we had live 3 months back for bug attachments
<lifeless> spm: requests for a file will allocate a token; the token will expire; folk can copy the token to e.g. wget if they want, content secured in this way is partitioned off from all other content by browser security rules
<lifeless> spm: to make this happen we need:
<lifeless>  - the certs
<lifeless>  - to check Host headers reach the librarian (we need to cross-check the domain)
<lifeless>  - various small code changes over and above the current patch
<lifeless> spm: the first two need your assistance
<lifeless> spm: so (ping)
<lifeless> spm: I get GSA on the certs; should I ask in is, or just wait :)
<spm> yup. been looking at the mp
<lifeless> spm: on the host headers side
<lifeless> we need to figure out if the requests squid is making to the librarian (for launchpadlibrarian.net requests) are preserving the Host header.
<lifeless> I'm not sure how to do that offhand ;)
<spm> I'll chase the vg and see if we can get some traction
<lifeless> thank you
<spm> hmm. except I'm not sure who the vg is today...
<lifeless> '-'
<lifeless> brb
<spm> gawd. I'm yak shaving again. need email to figure that; but home server (which holds email) is kaput. Need monitor on that server to WTF it; but desk is full of other crud and needs (mild) cleaning for room for a monitor. sigh.
<lifeless> :|
<wallyworld_> thumper: you having trouble with an unresolvable z3c lib? - "Getting distribution for 'z3c.recipe.scripts==1.0.1'"
<wallyworld_> thumper: this project doesn't seem to exist in launchpad? i tried to view the revision history of buildout.cfg but an getting bzr error:
<wallyworld_> bzr: ERROR: exceptions.TypeError: 'bzrlib._known_graph_pyx._MergeSortNode' object is not iterable
<lifeless> \o/
<wallyworld_> this was after running utilities/update-source-code
<spiv> Ooh, I haven't seen that one before.
<jtv> hi folks
<lifeless> hi
<spm> heya jtv
<jtv> hi lifeless, hi spm
<lifeless> \o/ oops are sane now
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1710ED237
<jtv> lifeless: thanks for fixing that oops problem in the weekend!
<lifeless> jtv: de nada
<lifeless> now we has sensible oopses
<lifeless> with librarian stats :)
<jtv> de algoâ¦ it was above and beyond.
<lifeless> thumper: ^ have a look at that one
<lifeless> thumper: items 98 and 99 in the 'sql' log
 * thumper has to run kids to art class
<lifeless> thumper: later will do, its user created
<lifeless> no idea why the analyser things the librarian stuff is repeated though
<jtv> Unfortunately I screwed up a little bit in my TranslationGroup fix.  I prefetched a lot of objects in queries that I moved to the slave store, when they're supposed to come from the default store.  So fetch the page from the master store and you've got lots of icon queries back.
<lifeless> something is naffed there
<lifeless> jtv: ahh
<jtv> That's why we still got a timeout on edge.  :-(
<lifeless> Store.of() might help too
<jtv> Well yes, whatever gets the default store.
<jtv> Or I use ISlaveStore(object).icon instead of object.icon
<jtv> lifeless: so librarian interaction is now also logged in the oops?  I took the page from 1050 db queries to 303 actions, and if those 303 actions actually count more than just queries, that's extra-great.
<lifeless> jtv: librarian download connects and reads are now logged yes
<lifeless> wgrant: I have an Idea
<lifeless> when you get back, tell me what you think:
<lifeless>  - i123.restricted... - must match domain and LFA
<lifeless>  - https?://launchpad-librarian.net/....  - also supports tokens
<lifeless> the tests will work
<lifeless> I can test once, directly, that when restricted is in the domain they must match
<lifeless> this will prevent injecting content into the security context of a restricted file
<jtv> lifeless: we've got a pretty annoying problemâthere's a db column that was supposed to become obsolete ages ago but was still in use.  We landed a branch that stops initializing it and stops using it.  Can you guess the problem?
<lifeless> other than db-devel going red because of conflicts?
<lifeless> or skew between two things modelling the same content
<jtv> edge vs production.
<lifeless> prod reads it back
<lifeless> with the old queries
<jtv> yup
<jtv> edge produces them without the data
<jtv> *I* *hate* so-called single-sign on for oopses!
<jtv> Open a dozen oopses in as many tabs: each and every one needs to go through the "single" sign-on page, and only the first one succeeds.  The rest just forwards to a different "single" sign-on page, which just loops back to itself.
<lifeless> please file a bug on that
<lifeless> and/or an RT
<jtv> Oh, and then there's a few that just fail with "invalid transaction"
<thumper> lifeless: what exactly about that oops should I be looking at?
<jtv> Yes, I will thanks.
<thumper> lifeless: apart from the general inefficiencies
<lifeless> thumper: search for librarian in it
<lifeless> or go to the row index in the sql statements that I pointed out to you
<thumper> lifeless: ah, nice
<thumper> 0ms  	librarian-read
<thumper> that's pretty fast
<lifeless> thumper: it will be in the socket buffer
<lifeless> thumper: so effective a noop
<wallyworld_> solved: TypeError: 'bzrlib._known_graph_pyx._MergeSortNode' object is not iterable
<wallyworld_> historycache plugin is bad
<wallyworld_> thumper: figured out the other problem - had to explicitly update the download cache. builds working again :-)
<thumper> wallyworld_: :)
<wallyworld_> thumper: bzr 101 question - can't recall, but i'm sure i did a bzr merge (and maybe also update) at the top level. is it sop to have to explicitly update the download cache?
<thumper> wallyworld_: the download cache is likely to be a heavyweight checkout
<thumper> wallyworld_: so it is the only thing you do bzr update one
<thumper> s/one/on/
<wallyworld_> thumper: thanks
<thumper> wallyworld_: *I* do an explicit update
<thumper> but I don't use the rocketfuel scripts
<wallyworld_> thumper: i don't use them either atm. is there anything in the workflow to indicate when one should update the d/l cache or should it be done say once per day?
<thumper> wallyworld_: I update the download cache when I pull devel/db-devel;
<wallyworld_> thumper: cool, i'll do the same. i was thinking one could also look for changes in the buildout.cfg file or other 3rd party depenency cfg file
<lifeless> wgrant: and its pushing
<lifeless> wgrant: I'm happy with this now, moving onto polish and integration
<jtv> spm: I'm landing an RC fix that's only needed on edge, and only until the rollout.  What can I do to ensure that it hits edge soon?
<thumper> jtv: cowboy hat?
<spm> jtv: get it landed in the normal manner asap. we're really unkeen on cowboying patches onto edge (aka prod-lite) without an incident report
<thumper> jtv: how do I find a product series that has a translation link for the branch?
<thumper> jtv: I want to test deletion (on staging ofcourse)
<jtv> thumper: Translations takes an interest in 2 branch links on a productseries:
<wgrant> lifeless: I'm still a little wary that we're allowing users to shoot themselves in the foot without noticing.
<thumper> lifeless: any idea how to record in oopses what other app server threads were doing?
<wgrant> lifeless: If launchpadlibrarian.net itself doesn't work, then people know not to use it.
<thumper> lifeless: I'm thinking of oopses caused by other long running requests
<jtv> thumper: 1: the development branch (it imports files that appear there, but it can also fire off build farm jobs based on changes there)
<thumper> lifeless: causing database locks
<jtv> thumper: 2: the translations_branch, which is where it can write snapshots of the series' translations.
<thumper> jtv: I guess I could just do a staging query now I have the power
<jtv> thumper: Finding cases of 2 is easy: WHERE ProductSeries.translations_branch IS NOT NULL
<thumper> mwa ha ha
 * thumper waits for staging to come back up
 * jtv looks up the enum for 1
<wgrant> lifeless: What's the benefit of allowing it?
<wgrant> lifeless: If you have to craft a URL to test, you might as well craft it to the restricted librarian.
<jtv> thumper: you find cases of 1 with WHERE ProductSeries.translations_autoimport_mode <> 1 AND branch IS NOT NULL
<jtv> spm: I'm landing in the normal manner asap.  It undoes one line of change from an earlier branch.
<thumper> how the hell am I supposed to submit someone else's work to pqm without a local copy of it?
<wallyworld_> lifeless: bug 631010? any eta on a fix to allow lp tests to run again? i've upgraded to maverick and can't run any tests
<_mup_> Bug #631010: ProgrammingError: operator does not exist: text = bytea <database> <maverick> <storm> <Launchpad Foundations:New> <https://launchpad.net/bugs/631010>
<thumper> phew
<thumper> finally
<wgrant> wallyworld_: You could try downgrading to Lucid's python-psycopg2
<wallyworld_> wgrant: thanks, i'll give that a try. been having a few issues with maverick and kde :-(
<wgrant> I should upgrade this week.
<wgrant> I normally upgrade around alpha 1...
<wallyworld_> you running kde or gnome?
<wgrant> GNOME.
<wallyworld_> they skipped the alpha this time i think
<wgrant> No, there were the usual alphas.
<wallyworld_> i mean they skipped the last one?
<wallyworld_> went to beta early
 * thumper off to get munchkins
<wallyworld_> i really hope kde gets fixed with maverick. me and gnome don't get on very well :-(
<lifeless> hi
<lifeless> wgrant: I do craft the right url
<wallyworld_> wgrant: thanks, downgrading to lucid's psycopg2 fixes it for now
<wgrant> wallyworld_: Great.
<lifeless> wgrant: the benefit is that we don't need wildcard dns on developers machines with https certs and -all that stack-
<wgrant> lifeless: We can't make the dev config use restricted librarian URLs?
<lifeless> wallyworld_: no info on the eta for it
<lifeless> wgrant: restricted librarian urls are different
<wgrant> Or at least only activate the tokens-on-launchpadlibrarian.net mode in the dev config?
<wallyworld_> lifeless: as per wgrant suggestion i downgraded to lucid's copy and it seems to be ok for now. thanks
<lifeless> wgrant: we do want to delete the proxy code
<lifeless> wgrant: so that isn't a tenable goal; as a stop gap maybe, but I don't see that its better or worse
<lifeless> wgrant: as for people foot-bulleting themselves; who are you thinking of ?
<wgrant> lifeless: What's not a tenable goal?
<lifeless> having the dev encironment run the current mode
<wgrant> Not the current mode.
<wgrant> Either linking directly to the restricted librarian (which is presumably accessible from localhost...), or having the local public librarian accept tokens on its primary name.
<lifeless> wgrant: the latter is what I've implemented
<wgrant> But only in dev mode.
<wgrant> Not on launchpadlibrarian.net.
<lifeless> I can add an if to turn it off, but I don't understand why
<wgrant> I can't think of a good reason to allow it.
<wgrant> And if there's not a good reason to allow access to private data through a second mechanism, can we please not do it?
<wgrant> Same-origin is a useful safetynet.
<lifeless> right ...
<wgrant> We probably want private files to be protected by it.
<lifeless> you're not joining the dots here; its the same mechanism
<wgrant> So let's not introduce a way to work around it.
<wgrant> Even if we can't immediately think of any attacks.
<lifeless> we'll want the apache front ends to be filtering tokens on http anyway
<lifeless> no harder to have them filter on subdomains too
<wgrant> Hm?
<lifeless> but adding another conditional in that code adds to the complexity there for no good reason
<lifeless> is anyone looking at the testfix ?
<wgrant> It's a workaround that's only require for dev installations. It is probably a single line of code to restrict it to that context, and it means that private content is forced to live in its own domain. That has to be a good thing.
<wgrant> Despite the slight increase in complexity.
<lifeless> so lets not add the workaround
<lifeless> that seems simpler to me
<wgrant> Then dev systems break.
<lifeless> The vector you are talking about is 'public content happens to know the url and token for some private content'...
<lifeless> they can just damn well load that directly
<wgrant> Or someone notices that omitting the 'i3532523.' works.
<wgrant> They proceed to do that.
<wgrant> There's nothing telling them that it's dangerous.
<lifeless> I just said above the frontends can enforce that if we want
<lifeless> trivially
<wgrant> Ah, I see.
<wgrant> I guess.
<lifeless> we have to have the front ends enforce httpS anyway
<lifeless> because if someone is going to disclose a private file it shouldn't be us. :)
<wgrant> True.
<wgrant> OK, well, as long as they're changed to do that, my objection is retracted.
<wgrant> And the plan seems good.
<lifeless> wgrant: folk can't omit the ix bit anyway
<wgrant> Why not?
<lifeless> wgrant: because the only way they get urls to use is via the appserver proxy service.
<wgrant> True.
<lifeless> they can, for the short period a token is active, manually edit and change the url
<lifeless> but honestly, really?
<wgrant> What is the limit?
<wgrant> An hour?
<wgrant> I forget.
<lifeless> current code says 1 day
<wgrant> I think paranoia is appropriate here.
<lifeless> thats arbitrary; no reasoning at all has gone into it.
<wgrant> But OK.
<lifeless> I wanted it to be longer than an ISO download to south africa
<wgrant> Right.
 * wallyworld_ off to pick up Martin Pool form the airport
* jtv changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: lp builds broken, db_lp buildbot offline | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<adeuring> good morning
<jtv> hi adeuring
<jtv> adeuring: to start the week off with a happy note, most buildbots are broken and even though you're probably innocent, you're on the blamelist.  :)
<jtv> Me, I suspect lifeless of landing python2.6 code before all buildbots support it
* jtv changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: non-lucid builds are broken, db_lp buildbot offline | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<adeuring> jtv: thanks for the heads uo ;)
<lifeless> adeuring: hi
<adeuring> hi lifeless
<lifeless> jtv: I'm -very- sure I haven't landed any 2.6 only code
<lifeless> jtv: for starters ec2 is still 2.5, and I run through ec2 religiously
<jtv> lifeless: ok, just trying to get your attention for this problem.  :)
<lifeless> adeuring: lp:~lifeless/launchpad/private-librarian is much closer to being done
<jtv> A lot of the failures are 2.6-isms afaict
<adeuring> lifeless: yes, I've seen your mp
<lifeless> adeuring: I mean 2 minutes ago :P I just pushed up more
<adeuring> lifeless: sounds great!
<lifeless> jtv: mmmm, not the best way to get my attention.
<lifeless> jtv: anyhow, I can se eyou and noodles775 looking into it; you're both good folk, I'm sure it will come good rapidly.
<jtv> lifeless: I wouldn't trust meâI merely spotted a 2.6-ism in propertycache breaking one  of the builds
<jtv> I wouldn't have any idea how to fix it
<lifeless> what line is it ?
<jtv> lifeless: it'll take me a moment to look that up, but it was a "next(counter)"
<lifeless> in devel ?
<jtv> lifeless: yes, it was in the "lp" buildbot log
<noodles775> jtv: I thought its only the lucid_db_lp buildbot that is critical (and it's a different error)
<noodles775> (by critical, I mean, stopping landings)
<jtv> noodles775: I just found I'm well out-of-date with what breaks what; we don't have hardy servers any more then?
<lifeless> noodles775: any of the main buildbots failing is textfix
<lifeless> we have hardy servers
<lifeless> if 'lp' breaks textfix is triggered
<noodles775> OK. I'm looking at the lucid_db_lp one then.
<wgrant> lifeless: The hostname restriction code is... um... But apart from that, your branch looks fine to me now.
<lifeless> oh, in the doctest
<lifeless> wgrant: buggy?
<wgrant> lifeless: Ugly.
<lifeless> wgrant: patches considered
<wgrant> I'm not sure there's a better way.
<wgrant> I just know that this way is ugly :)
<wgrant> lifeless: Why use a regex rather than just "hostname == 'i%d.restricted.%s' % (self.aliasID, netloc)"?
<lifeless> wgrant: good point, I should
<lifeless> I realised half way through that there was an attack
<lifeless> I was going to be fairly relaxed on the check
<lifeless> but
<wgrant> I also don't really get what you're doing with netloc and :. Shouldn't urlparse already do that?
<lifeless> i1234.restricted.iunderattack.restricted.launchpad-librarian.net would be bad
<wgrant> Oh, port.
<wgrant> Right.
<wgrant> lifeless: I wouldn't mind a comment saying that you're stripping off the port there.
<lifeless> jtv: so, if next(iterator) advances it
<wgrant> It's not blindingly obvious.
<wgrant> Or maybe I'm just tired.
<lifeless> wgrant: I'll add one
<wgrant> Thanks.
<jtv> lifeless: python2.5 doesn't seem to know about "next"
<wgrant> Now the only bad bit is the '.restricted.' check, but there's not much that can be done about that yet.
<lifeless> jtv: next(i) == i.next()
<lifeless> jtv: whats the next glitch
<lifeless> jtv: the difference is the ability to say "next(i, 42)"
<jtv> lifeless:     NameError: global name 'next' is not defined
<jtv> the line was         return next(counter)
<lifeless> right
<lifeless> replace it with counter.next()
<jtv> But it's only one of so many failures that I'm now trying to figure out what bigger picture I'm missing.  I'm told the failures on lp are not the cause of the testfix mode; lucid_lp_db is broken with a different failure.
<lifeless> its 8:30 herel; I need to go remind my wife what I look at.
<lifeless> s/at/like/
<lifeless> I'm sure there will be multiple failures
<lifeless> I'm also sure that lp failing will cause testfix, because bb is watching *both*
<lifeless> (or all 6 actually, anyhow)
<lifeless> jtv: I suggest you, or someone you get to agree to it, puts together a branch to fix all the known devel issues; send that to devel (labelled testfix). then look at db-devel.
<jtv> lifeless: noodles775 has been looking at db-devel already, and I've been trying to do the other thing for the past half day
<jtv> so yeah, I agree with the approach basically :)
<jtv> Go show your face to your family.  :)
<lifeless> jtv: doing so; at least to the extent of not staring balefully at the laptop
 * jtv would reply if it weren't for the expected consequence of a certain person staring at laptop even longer
<lifeless> adeuring: stub is reviewing the branch, but I don't know if all tests pass (the ones for code I've been directly changing do, of course)
<lifeless> adeuring: so perhaps you'd like to : throw it at ec2; start dealing with any fallout it has, and I'll finish up tomorrow with whatever you push (as long as you tell me what you've pushed ;P)
<adeuring> lifeless: ok, sounds good
<lifeless> adeuring: it is in principle feature complete though
<adeuring> ok
<lifeless> all cleanups etc deferred until we have successfully migrated
<lifeless> gmb: there is another release-critical thing up for you
<gmb> lifeless, Go ahead
<lifeless> gmb: its in your mail already
<lifeless> gmb: this is high risk high reward
 * gmb looks at his mail client
 * gmb sees a greyed-out window
<gmb> Hmm.
 * gmb switches to the web interface
<gmb> lifeless, Okay, I get - and like - the rewards. Spell out the risks for me since I'm under-caffeined this morning.
<lifeless> gmb: its a change to a fairly magical part of the system
<lifeless> anything could happen
<lifeless> we'll be able to QA that privat ebugs are no -worse- on staging
<lifeless> it will be hard to check the new stuff until we get the certs (but not impossible)
<gmb> lifeless, Do we have any alternative solutions that will fix the private attachment problem and will be ready before tomorrow evening (UK time)?
<lifeless> in principle the private attachment problem is fixed for a limited time via the firewall hole (though I may be out of date)
<gmb> adeuring, Can you confirm lifeless's statement ^^?
<lifeless> but we need to deliver a permanent fix fairly promptly
<adeuring> gmb: yes, the temporary fix for the retracers works
<adeuring> but we should get rid of it quite soon
<gmb> lifeless, I agree. Since adeuring's temporary fix is in place, go ahead and land yours - I'm less worried about backing it out if we've got a kludge for the problem already.
<gmb> I'll update the merge proposal.
<wgrant> And this solution is the first one that doesn't make me cry :)
<gmb> Well, naturally we wouldn't want that.
<wgrant> I mean, it is actually a good, effective long-term solution this time. Which is really nice.
<gmb> Agreed.
<lifeless> well, we can hope.
<wgrant> Well, there are no obvious fatal flaws in this one.
<wgrant> So I think it should be good.
<wgrant> bigjools: Is there a known issue with non-virt dispatches failing sometimes?
<wgrant> bigjools: I had some complaints about amd64 distro builders doing it this morning.
<wgrant> (the build restarted a few times)
<bigjools> have you got examples so I can check the log?
<wgrant> bigjools: kdeedu on amd64, reported a little under 10 hours ago.
<wgrant> Not sure when it actually happened, though.
<wgrant> Looks like it was not long before it was reported.
<wgrant> So look around 10 hours ago.
<bigjools> it looks like temporary network brownouts
<bigjools> or at least a lack of response from builders
<wgrant> Hmm.
<bigjools> ah I see what it is
<bigjools> slow reset
<wgrant> But it's non-virt.
<wgrant> There is no reset.
<bigjools> good point
<wgrant> Otherwise, yes, slow reset is the obvious thing.
<bigjools> well, it's still timing out anyway
<bigjools> the log has "User timeout caused connection failure."
<bigjools> followed by "reset failure"
<bigjools> which is odd
<bigjools> (for crested)
<wgrant> Er.
<wgrant> WTF?
<bigjools> I'd say that it's all working fine from a software PoV, it does what it's supposed to under the conditions
<bigjools> that log message is a little odd though
<wgrant> Except that it shouldn't actually be trying to reset a non-virt builder.
<wgrant> Or is the log lying?
<bigjools> I would not worry too much
<bigjools> I've completely re-written the failure handling for the next release
<wgrant> Yeah, I saw that.
<wgrant> Looks good.
<bigjools> it won't catch build failures, just dispatch failures
<bigjools> we need to add that feature at some point to stop bad jobs jumping around builders
<bigjools> my blood levels are dangerously high in my caffeine stream, I need to go fix that
<noodles775> voidspace: re. your oops - one of the LP registry team will be interested to look at fixing the bug, but for what its worth, the exception is being raised while it is trying to tell you that the email address is already in use.
<wgrant> For which we have ShipIt to blame :(
<wgrant> I suspect.
<noodles775> wgrant: Ah, does it create email addresses where email.person is None?
<wgrant> noodles775: ShipIt does Accounts, not Persons.
<wgrant> So, yes :(
<wgrant> And for no very good reason it still haunts the same DB as LP.
<lifeless> adeuring: bunch of incremental changes from stuarts review pushed now; I expected librarian.txt to fail, fixing that now.
<lifeless> adeuring: anything other than that failing is unexpected (but sadly probably predicatable :P)
<adeuring> lifeless: ok, I'll see what ec2 will tell us ;)
<lifeless> night y'all
<wgrant> Night.
* gmb changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.09 | PQM is CLOSED | firefighting: non-lucid builds are broken, db_lp buildbot offline | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<thumper> gmb: sha'ping
<thumper> gmb: I'm aware of the outstanding QA for code
<thumper> gmb: but staging has been futzed all day
<thumper> gmb: so I'll be looking at it tomorrow morning
<thumper> gmb: just letting you know
<gmb> thumper, Yep, I expected as much. No worries, and thanks for the update.
 * thumper -> cuppa
<thumper> gmb: np
<bigjools> wgrant: got a sec to help work out WTF is going on with bug 629835
<_mup_> Bug #629835: cannot delete 'linux' from the ubuntu-security-proposed ppa <oops> <Soyuz:New> <https://launchpad.net/bugs/629835>
<wgrant> bigjools: Sure.
<wgrant> Is that the delayed copy one?
<wgrant> Yes.
<wgrant> bigjools: You need to look for the initial error.
<wgrant> It's possible that that's it.
<wgrant> But I doubt it.
<bigjools> wgrant: it's allowed a 2nd copy of the same thing somewhere, somehow
<bigjools> which is scary
<wgrant> bigjools: Or it's just continuing to process the same copy, because it failed somehow the first time.
<wgrant> This has happened once before.
<wgrant> We couldn't work out what it was.
<bigjools> hmmm
<bigjools> he's done a pocket copy in the same archive
<wgrant> I wish we tracked where they were copied from.
<wgrant> Can you see when that delayed copy was initially processed?
<bigjools> I wonder if the delayed copy checks are broken
<bigjools> I can grep logs
<wgrant> There is a race in the delayed copy mechanism.
<wgrant> But it's pretty unlikely.
<bigjools> logs don't go back far enough :/
<bigjools> it's been doing this since 1st Sep
<wgrant> Er.
<wgrant> What?
<bigjools> at least
<wgrant> Yeah, it's been doing it since the 26th.
<bigjools> we only keep 5 days of publisher logs
<wgrant> ...
<wgrant> 26th, 23:01Z
<wgrant> Er, 25th, 23:01Z.
<wgrant> Why so short?
<bigjools> that's the default log rotation
<wgrant> Baaah.
<wgrant> Can you check for any other delayed copies of that source?
<bigjools> sigh, loads of PPA pub;isher OOPSes getting logged but not reported
<bigjools> PoolFileOverwriteError
<wgrant> Pool file overwrites, mostly, I guess?
<wgrant> Yeah.
<bigjools> the same file, over and over
 * bigjools wonders how that can happen
<wgrant> I used to know.
 * wgrant hunts.
<bigjools> so, the inconsistent state error first happened 2010-08-25T23:05:39.797695+00:00
<bigjools> which tallies
<wgrant> Ah, you have oopses?
<bigjools> yes
<wgrant> Awesome.
<bigjools> lot sof 'em
<wgrant> So, that's interesting.
<bigjools> all the same as the one I put in the bug
<wgrant> There's nothing four minutes earlier?
<wgrant> That's when it was processed first.
<wgrant> I expect a different error.
 * bigjools hunts
* lindbohm.freenode.net changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<wgrant> bigjools: Are any of the PoolFileOverwriteErrors for files published after May?
<wgrant> Before then the copy checker didn't actually check for contents conflicts.
<bigjools> hard to tell
<wgrant> (devel r10701)
<wgrant> We can query for that reasonably easily.
<wgrant> Find Pending PPA publications older than not very old.
<bigjools> there are two
<bigjools> 2010-07-27 and 2010-08-25
<wgrant> bigjools: Actually, can you tell (from the repetitve PFOE oopses?) if 23:01 was the same publisher run as 23:05?
<bigjools> and 9 more from 2010-09-03
<wgrant> Hm...
<wgrant> These are Pending, in undisabled PPAs?
<wgrant> Er, undisabled, and with the published flag set, too.
<bigjools> ACCEPTED, not sure about the status, hang on
<wgrant> Oh, these are the delayed copies of that source?
<bigjools> all enabled apart from the 07-27 one
<wgrant> Are these the delayed copy PUs?
<wgrant> If so, that's an awful lot.
<bigjools> 2 of them are delayed
<bigjools> one is the disabled PPA one is u-s-p
<bigjools> dated 2010-08-25
 * bigjools scratches head
<wgrant> Can you tell by the oopses if the 23:00 publisher finished in time, making the 23:05 OOPS the second run?
<bigjools> I can't tell
<wgrant> Not even by looking for the repeated PFOEs?
<bigjools> oh I am looking at staging which is out of date, which explains the load of 09-03
<wgrant> Ah.
<bigjools> I'm going to set that upload to rejected
<bigjools> to remove this OOPS
<wgrant> Sounds reasonable.
<wgrant> Can you also increase log retention, or make the OOPSes less easy to ignore?
<bigjools> I'm working on the latter
<bigjools> the former is a good idea
<bigjools> I'd love to know how this happens though :/
<wgrant> We'll find out next time :)
<bigjools> Jamie said he copied it from their private PPA in hard to maverick in the public one
<wgrant> I don't think I've tried a cross-series delayed copy.
<bigjools> I wonder if the change in series has anything to do with it
<bigjools> and then he deleted it
<bigjools> could have been before or after it was published in the new archive
<wgrant> You can't tell?
<bigjools> wgrant: http://pastebin.ubuntu.com/489219/
<bigjools> ummm interesting
<wgrant> The one that was never published is a little odd.
<wgrant> I noticed it before, but thought little of it.
<wgrant> However, there's something really odd there.
<bigjools> that's the most recent one
<wgrant> No, I mean the second one in that list.
<bigjools> oh right, missed that
<wgrant> Looks like he deleted it 25 seconds after it was published.
<wgrant> Which seems unlikely.
<wgrant> However, that's still well after everything initially broke.
<bigjools> deleting 25 seconds after sounds reasonable to me
<wgrant> After an out-of-band delayed copy?
<bigjools> it's not delayed at that point
<wgrant> It isn't?
<bigjools> the first one will make it instant for future copies
<wgrant> Wasn't that publication just created by the retry?
<wgrant> Yes.
<wgrant> But AIUI p-a is doing the copying.
<bigjools> retry?
<bigjools> only for delayed copies
<bigjools> aegh, I wonder if this is the "hit copy twice" bug
<bigjools> I bet it is
<wgrant> There'd be a DONE PU in that case.
<wgrant> There appears to not be.
<wgrant> Yeah.
<wgrant> That source has only ever been in Maverick in that PPA.
<wgrant> And there's only one delayed copy to Maverick.
<wgrant> And it's the Accepted one.
<wgrant> 2069696
<wgrant> So.
<wgrant> The bug suggests that the delayed copy keeps being reprocessed.
<wgrant> But the OOPS suggests that it's failing.
<wgrant> I wonder if it stops OOPSing when the source is deleted.
<wgrant> Oh look, yes it does.
<wgrant> So there will be some publisher runs where it doesn't OOPS.
<wgrant> That's immediately after jdstrand deleted the last publication.
<wgrant> So the next p-a reprocesses it successfully, creating new publications.
<wgrant> But somehow still fails to set the status.
<wgrant> Maybe custom uploads.
<wgrant> There must be more OOPSes there that we are missing.
<wgrant> If you can't see them, delete it again and watch the next p-a run.
<wgrant> Something has to show up.
<bigjools> yeah
<bigjools> I'll grab jamie later, I can't delete it
 * bigjools is desperate for food now
<wgrant> bigjools: The last publication is from the third.
<wgrant> 00:12Z
<wgrant> You should have logs for then.
<wgrant> The copy was processed, but not set to Done.
<bigjools> are you querying on the API or using my sql output?
<wgrant> API and web UI.
<bigjools> k
<bigjools> wgrant: so it started publishing it again at 2010-09-03 00:00:45
<bigjools> the publisher run at 00:10 has no errors
<bigjools> the next one goes bang
<wgrant> Did 00:10 actually run at all?
<bigjools> yes
<wgrant> Or was it skipped because 00:00 overran twice?
<wgrant> Hmm.
<bigjools> oh wait
<bigjools> no
<bigjools> damn this log file
<bigjools> the next run was at 00:15 and it failed on the same queue item
<bigjools> 2010-09-03 00:00:45 DEBUG   Publishing source linux/2.6.24-28.77 to ubuntu/maver
<bigjools> so it ignored the bad PU for that run only, it fails for the run before and after
<wgrant> bigjools: What's the extent of its logging of the 00:00 publication?
<wgrant> Any errors?
<wgrant> Any success?
<wgrant> I expect to see something there.
<wgrant> Preferably an error.
<bigjools> argh
<bigjools> I didn't scroll down enough
<wgrant> Heh.
<bigjools> so it published it for ubuntu/maverick/amd64
<bigjools> then failed for that queue item again
<wgrant> Hmmm.
<wgrant> I wonder.
<wgrant> I wonder.....
<wgrant> bigjools: There's nothing about lpia?
<wgrant> Publishing that build could blow everything up.
<wgrant> Badly.
<bigjools> nup
<wgrant> Or hppa?
<wgrant> I expect a NotFoundError when publishing either of those.
<wgrant> Ahem.
<wgrant> That would explain why amd64 is the only thing published there, if it's doing it alphabetically, which is plausible.
<bigjools> yep
<bigjools> then bails out with an exception
<wgrant> Can you see anything like that in the log?
<bigjools> like what?
<bigjools> NFE?
<bigjools> it does amd64, then bails out on the dodgy PU
<wgrant> Well, I want to see where it tries to accept the hppa build.
<bigjools> it doesn't get that far
<wgrant> Huh.
<wgrant> Is there only one PUB on the PU?
<wgrant> Hm, no.
<wgrant> It has all of them, AFAICT.
<wgrant> Including hppa, which is probably killing it.
<wgrant> But it should be logged!
<wgrant> DAmmit.
<bigjools> oO
<bigjools> huh?
<wgrant> So. The 00:00 publisher. Can you show me the lines between where it starts publishing the source, and where it starts publishing the next item?
<wgrant> I'm now reasonably sure that it's dying due to an NFE when attempting to realise the hppa build.
<wgrant> But it should be logging that somewhere.
<bigjools> this is your lot: http://pastebin.ubuntu.com/489241/
 * bigjools has to run for 30 mins
<wgrant> What's OOPS-1707PPA1?
<wgrant> I doubt it's the usual.
<wgrant> OK.
 * bigjools checks quickly
<bigjools> WTF
<bigjools> FatalUploadError: Signing key C51BC47D4C80ED00E96B4DC1839AEABCCF5C0A1F not registered in launchpad.
<wgrant> uH.
<wgrant> I doubt it, really.
<wgrant> Must be a different location.
<bigjools> that's can't be right
<wgrant> It's not.
<wgrant> They're throwing oopses in different directories.
<wgrant> So we have conflicts.
<wgrant> Yayyyyy.
<bigjools> the reporting tool is crack
<bigjools> the real error is:
<bigjools> NotFoundError: u'Unknown architecture hppa for ubuntu maverick'
<bigjools> :)
<wgrant> AHA
<wgrant> As expected.
<wgrant> Phew.
<bigjools> so, we've copied builds that can't be published
<wgrant> Exactly.
 * bigjools high-5s wgrant
<wgrant> So it was the cross-seriesness.
<wgrant> But with added complexity.
<bigjools> thanks for helping wgrant
<wgrant> I am glad we no longer have a copy corruption mystery.
<wgrant> bigjools: Just to be sure, can you confirm that the NotFoundError also occurs at the time of the initial publication?
<wgrant> Would be nice to be absolutely positive that it was the root.
<bigjools> ok
<benji> gmb: good afternoon; I noticed that one of my branches was reverted with the commit message of "Revert the merge of the check-in-WADL branch, which was causing the build to break."  I'm looking for more info on the breakage so I can fix it.  Any hints?
<stub> benji: Something about breaking the launchpad/stable -> launchpad/db-devel auto merge, because it would always conflict.
<gmb> benji, Sure, let me dig out the failure messages. There's also an ML thread about it between lifeless and I.
<benji> hmm
<gmb> benji, "WADL test will break db-devel merges regularly (was) Re: buildbot failure in Launchpad on lucid_db_lp" is the thread you're looking for.
<gmb> benji, And this is the build that failed: https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/176
<benji> gmb: thanks; what list was that thread on?  I don't think I saw it.
<gmb> benji, canonical-launchpad.
<gmb> benji, I'll find the thread in the archive for you.
<benji> much appreciated
<gmb> benji, https://lists.ubuntu.com/mailman/private/canonical-launchpad/2010-September/060017.html is the top message in the thread; everything else is under that.
<benji> thanks
<lifeless> adeuring1: hi
<lifeless> benji: hi
<benji> hello there
<adeuring1> hi lifeless
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance Tuesday! | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<lifeless> adeuring1: how did it go?
<lifeless> benji: see the thread, I think we covered all the salient points there
<adeuring1> there were basically two failures, one in test_db
<adeuring1> (fixed)
<lifeless> benji: there was one extra thing not really covered which is that as a way of prevented api incompatabilities, I think it will need to be ----way---- easier.
<lifeless> adeuring1: \o/
<lifeless> adeuring1: so its landed?
<adeuring1> lifeless: no, I could not fiugre out how to fix the other one, in lib/canonical/launchpad/ftests/../doc/librarian.txt
<lifeless> gmb: up still ?
<adeuring1> lifeless: AttributeError: 'thread._local' object has no attribute 'features'
<lifeless> adeuring1: ok, I'll fix the other. where is your branch with the db fix ?
<lifeless> adeuring1: that will be because a test has no request object live but is using a view.
<benji> lifeless: as soon as I get access to the list archives, I'm sure your comments will make total sense :)
<adeuring1> lifeless: lp:~adeuring/launchpad/private-librarian
<lifeless> benji: ah -
<lifeless> benji: ok so
<adeuring1> lifeless: I also moved some ZCML stuff forSafeStreamOrRedirectLibraryFileAliasView to l/c/l/zcmnl/librarian.zcml
<lifeless> adeuring1: great stuff
<gmb> lifeless: Yes, I'm still around.
<lifeless> rc stampy stampy needed
<lifeless> https://code.edge.launchpad.net/~gary/launchpad/bug627442/+merge/34701
<gmb> lifeless: Ah, yes. Looking at that now.
<lifeless> thanks
<gmb> gary_poster, lifeless : rc=me
<gary_poster> thank you gmb
<gmb> np
<lifeless> gary_poster: the new librarian stuff should land today
<gary_poster> awesome!
<lifeless> gary_poster: stub did a full review, there were two test failures abel found overnight, and he fixed one, I'm fixing the other now.
<gary_poster> fanstastic
<lifeless> I'll queue up the RT tickets later and get flumper to hi-prio them
<gary_poster> :-) k
<lifeless> gary_poster: also the memcache/email/librarian client stuff seems happy
<gary_poster> I didn't see that, but from the list I'm guessing that's timeout related?
<lifeless> logging those actions in OOPS
<lifeless> as part of the request timeline
<gary_poster> ah ok
<gary_poster> yeah I did see that actually
<gary_poster> cool
<lifeless> oopstools are getting a -little- confused on some of them
<lifeless> so we've got some fine tuning to do
<lifeless> but nothing major, probably just a stray \n ors something
<gary_poster> k
<lifeless> hah
<lifeless> and the 99999 thing
<lifeless> overflows oopstools :P
<gary_poster> :-(
<lifeless> ttps://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1710EC1430
<lifeless> I'll change it to 0
<lifeless> actually
<lifeless> I'll change it to consider it finished 'now' for that code path.
<lifeless> if you look in that oops, its all sensible
<lifeless> the last one is (6x9)ms as expected
<gary_poster> hee hee, I do like the overflow
 * gary_poster hasn't had lunch yet, and is starting to feel a bit...out-of-body
<lifeless> shoo
 * gary_poster should rectify that
<gary_poster> biab
 * gmb -> afk for a while; will check back periodically
<lifeless> man, this race condition on librarian startup is really annoying
<jelmer> lifeless: TacException: Unable to start /home/jelmer/lp/daemons/librarian.tac. Content of /var/tmp/librarian.log: ?
<lifeless> jelmer: ?
<jelmer> lifeless: Is that one of the symptoms of that race condition?
<lifeless> no, I don't think so
<lifeless> running just a librarian using test and having them deadlock is
<lifeless> the client has sent the reqest, the librarian is still reading it
<lifeless> or something
<jelmer> ok, that's different from what I've seen that
<jelmer> *then
<lifeless> it may be
<lifeless> I've seen that too, but very rarely
<jelmer> it's quite rare here as well, I'd say about once every two dozen test runs
<lifeless> \o/
<lifeless> now I just need an incremental review
<lifeless> hi poolie
<lifeless> I imagine you're running out how; will you be working today?
<thumper> gmb: ping when you are ready
<gmb> Hi thumper; Do you want to have a call or just a chat?
<thumper> gmb: call is fine
<thumper> my skype is running
<thumper> mumble doesn't like nz
<lifeless> those things used to be synonyms
<gmb> thumper: Okay, give me a minute to shift locations.
<thumper> lifeless: there is a different local isp here that offers SDSL
<thumper> lifeless: who doesn't get affected by telecom's shaping
<lifeless> thumper: ooo
<lifeless> thumper: who?
<thumper> lifeless: I'm wanting to try mumble with it
<thumper> wicked networks
<lifeless> I signed up for a year with telecom, just to get settled, its tolerable
<thumper> I know an office that has it all set up
<thumper> they offered for me to go and try it out
<lifeless> but yeah, I would totally pay as much as I was paying in .au for /actual internet/
<thumper> it is only 2 meg up/down
<thumper> as opposed to 8 down and .5 up that I get now
<lifeless> same average :P
<thumper> yeah, but I get a lot more down than up
<gmb> thumper: Okay, calling now.
<mwhudson> my up/down ratio at home is stupid
<mwhudson> 18 meg up, 0.5 down
<lifeless> nice
<lifeless> I'm getting 4MB here
<lifeless> but I suspect its the line quality as much as anything
<mwhudson> yeah, i think i'm very close to the nearest cabinet
<ajmitch> thumper: we use that isp at work, it seems OK most of the time. sometimes has some poor international connectivity, but generally not too bad
<lifeless> mwhudson: there are 5 phone points in the house
<lifeless> benji: did you get clarity?
<lifeless> benji: or would you like me to expound on the issue here?
<benji> not yet; I haven't gotten a response on my subscription request to the list
<benji> if you know who to pester, I'd appreciate to know
<lifeless> we really should move that to LP
<lifeless> uhm, its possibly out of date; #is would be a good place to ask - the gsas.
<lifeless> anyhow, let me recap
<lifeless> we have two branches
<lifeless> db-devel and devel both feeding into a single tree - the 'db-devel buildbot test tree'
<lifeless> if both branches have had API changes made directly on them, then every single time that devel change the API, the merge to db-devel will alter - correctly - the WADL, but the test will fail.
<lifeless> as gmb and I understand the goal of the test to be fixing the apidoc issue primarily, we rolled it back as the simplest way to unblock the release.
<lifeless> benji: but there are a couple of extra quirks to bear in mind when retackling it
<lifeless>  - as a way of checking for API regressions, examining a big bytestring is very human intensive. I fully expect the human response to be 'oh, I need to run X and commit it' - that is, it won't prevent incompatible changes at all.
<lifeless>  - its generally a bad idea to check in the output of build processes; VCS's can interact badly with that, and its wasteful to store the duplicate/derived data in the VCS.
<benji> I didn't intend it as a way of checking for API regressions; I had the goal of speeding up make.  The tests were a way of being sure the files checked in were current.  The tests also provide a guard against unintenional API changes (if you make a change and the WADL changes and you didn't know you were impacting the web service, then you've been warned).
<lifeless> benji: so, the pragmatic issue is: this must reliably, work when two branches with API changes are merged, with no human intervention.
<lifeless> benji: *I* strongly suspect that means that making wadl generation faster is an easier approach.
<lifeless> naively, I don't see why it would be more than a second or so's processing.
<thumper> lifeless: do you know if we test against the ubuntu-bug script?
<thumper> lifeless: or which version of the api it uses?
<lifeless> thumper: there is something done with apport yes.
<lifeless> 1.0 I believe, just because the distro did an audit-and-sweep for beta->1.0 a while back.
<gary_poster> hey poolie.  I'd like to briefly show off bzr in a talk I'm giving.  I was having trouble with bzr-git until I just upgraded to the new packages (https://launchpad.net/~bzr/+archive/ppa)--my example works now, yay!  why isn't bzr-hg in that ppa though?  In a perfect world, I'd show that too.
<thumper> lifeless: I'd like to talk to you at some stage to talk about how to understand some oopses I'm seeing
<mwhudson> gary_poster: bzr-hg is not nearly as polished as bzr-git
<mwhudson> i don't know if that's why
<lifeless> thumper: sure thing
<gary_poster> mwhudson: ok, thank you.
<lifeless> thumper: skype?
<thumper> lifeless: ack
<thumper> https://lp-oops.canonical.com/oops.py/?oopsid=1698XMLP108
<thumper> https://lp-oops.canonical.com/oops.py/?oopsid=1698XMLP110
<thumper> https://lp-oops.canonical.com/oops.py/?oopsid=1698XMLP115
<thumper> https://lp-oops.canonical.com/oops.py/?oopsid=1698XMLP118
<gary_poster> mwhudson, I've actually come to trust bzr-svn so much that I use it to commit to public svn repos.  I didn't always feel that way.  Do you happen to know if bzr-git is similarly polished?
<mwhudson> gary_poster: it's pretty close
<gary_poster> cool
<gary_poster> thanks again
<wgrant> Apart from the whole push vs dpush thing.
<mwhudson> it's less polished than bzr-svn probably, but it's a bit less of a model change so it's job is a bit easier
<mwhudson> oh right yeah, and it doesn't roundtrip
<jelmer> wgrant: roundtripping support is on the way and will be in the next non-bugfix release
<james_w`> \o/
<gary_poster> awesome :-)
<gary_poster> I see this page with caveats, and links to more http://doc.bazaar.canonical.com/migration/en/foreign/bzr-on-git-projects.html
<wgrant> jelmer: Ooh, nice.
<wgrant> jelmer: We're you storing the data?
<jelmer> wgrant: a special file in the tree and the commit message
<wgrant> jelmer: The file stores file IDs?
<jelmer> wgrant: yes, for file ids introduced by bzr
<lifeless> thumper: https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-pageids.html
<wgrant> jelmer: Isn't that going to be merge conflict fun?
<jelmer> wgrant: yes, there might be some issues in that regard
<wgrant> rockstar: FWIW, the lp-buildd chroots have nothing to do with security.
<jelmer> wgrant: fortunately it will only happen when people merge from multiple others who have used bzr to introduce new files
<wgrant> chroots are useless when you have root.
<wgrant> jelmer: Yep.
<wgrant> Still not ideal, but such is git...
<jelmer> wgrant: It's the best I could come up with that was scalable, reliable and practical.
<Snorlax> Anyone here, that has deployed a launchpad install on a private server?
<lifeless> jelmer: wgrant: merge is hookable. Hook and win.
<jelmer> lifeless: we were talking about git though
<lifeless> its hookable too :P
<jelmer> lifeless: fair enough :-)
<wgrant> So Git really doesn't have any custom metadata support?
<wgrant> *At all*?
<jelmer> wgrant: well, there is notes, which are basically branches with metadata about e.g. commits
<wgrant> But I hear they don't propagate.
<jelmer> they are used to make after-the-fact modifications to existing fields in git
<jelmer> so we can't use them - their contents would still show up in the git ui
<wgrant> Hah.
<lifeless> mars: around ?
<lifeless> OOPS-1710EC915, OOPS-1710ED880
<Ursinha> lifeless, I guess he's out for labor's day
<lifeless> ah yes
#launchpad-dev 2010-09-07
<lifeless> spm: hi
<lifeless> spm: I'm sure you have urgents, but I have some today
<lifeless> spm: firstly, gmb needs to know some stuff about the release.
<lifeless> gmb: ^
<spm> lifeless: heya
 * gmb is paying attention, may mumble further questions
<gmb> Have run out of Monday.
<lifeless> spm: specifically; how long did the staging db migration take (for downtime estimation); secondly, are the lucid upgrades tom is doing sequential or concurrent with the schema migration
<lifeless> also we seem to have a dead-but-accepting launchpad edge server - see #launchpad backlog with nightrose
<lifeless> and can I have a pony. please.
<spm> sorry. no ponies. we do have shiny ponies tho? does that count?
<lifeless> shiny will do
<spm> lifeless: <stub> spm: 25 mins for the db patches to apply anyway
<lifeless> gmb: ^
<gmb> spm: Ta.
<spm> tho. um. migration? I perhaps misunderstood the question?
<spm> did you mean the 8.3 -> 8.4 migration? or just the regular do the DB juju? cause the ans provided is for the latter; not former.
<spm> staging DB has not been migrated.
<lifeless> regular DB juju
<spm> coolio
<lifeless> oh and speaking of staging
<lifeless> I'd like to get a profile pretty please
 * thumper is finding :((
 * gmb exits stage right in search of sleep.
<gmb> 'Night folks.
<lifeless> shiny shiny - http://wiki.postgresql.org/wiki/Postgres-XC
<elmo> "Failover/High Availability: No"
<lifeless> yeah
<lifeless> but its the first multi master thing in the pg world that I've heard of
<lifeless> so things are starting to move
<thumper> ah no
<thumper> I was all about to go \o/
<thumper> but no failover is somewhat sucky
<lifeless> oh god no, we can't /move/ to this today
<lifeless> its simply 'if you need more writes than one machine can give you. I think it would be possibly to run slony on top if it if we wanted (arrghh :P)
<thumper> jelmer: ping?
<jelmer> thumper, pong
<thumper> jelmer: why am I not too surprised that you are up?
<thumper> jelmer: http://staging.launchpadlibrarian.net/54835281/chicken-chicken-git-mirror.log
<thumper> jelmer: looking at some staging import logs
<thumper> jelmer: Exception AttributeError: "'NoneType' object has no attribute 'close'" in <function terminate at 0x414d140> ignored
<thumper> jelmer: any ideas?
<thumper> jelmer: seems on bzr-git and bzr-svn imports
<jelmer> thumper: as far as I can tell they are to do with paramiko
<thumper> jelmer: ah
<jelmer> thumper: I've seen them from e.g. update-sourcecode as well
<jelmer> thumper: but I'm not clear as to what the problem is exactly
<thumper> ok
<thumper> jelmer: I've got a question about bug 599397
<_mup_> Bug #599397: code imports break when tdb is installed <code-import> <qa-needstesting> <Bazaar Hg Plugin:Triaged> <Launchpad Bazaar Integration:Fix Committed by jelmer> <https://launchpad.net/bugs/599397>
<thumper> jelmer: should we be running the import machines with tdb?
<jelmer> thumper: Yes-ish
<thumper> ?
<thumper> meaning?
<jelmer> thumper: There are some advantages to using tdb - it allows concurrent write access for example, so multiple clones from the same svn repository at the same. In my experience it's also faster than SQLite.
<thumper> I feel a but coming...
<jelmer> there is.. :-)
<jelmer> maxb has reported he's seen worse performance from tdb than from sqlite
<jelmer> there is also the fact that all existing caches are already in sqlite, moving to tdb would mean regenerating them or keeping the existing ones in sqlite and the new ones in tdb
<jelmer> concurrent access isn't really an issue except when people register svn imports for different branches in the same svn repository at the same time. after the initial import the incremental imports are quite quick so they hardly ever overlap, and even if they do they will be retried.
<thumper> jelmer: suggestion on an import to try to verify bug 617078?
<_mup_> Bug #617078: RedirectRequest errors trying to access .git/objects <qa-needstesting> <Bazaar Git Plugin:Fix Committed by jelmer> <Launchpad Bazaar Integration:Fix Committed by jelmer> <https://launchpad.net/bugs/617078>
<mwhudson> any github http import i think?
<jelmer> thumper: xwax, lp:~dholbach/xwax/trunk
<thumper> ok
<jelmer> mwhudson: I think most non-github failing git imports over http are failing because of that particular bug
<mwhudson> ah ok
<jelmer> mwhudson: github still won't work with the new rollout, they don't support "dumb" access to git repositories over http
<jelmer> mwhudson: s/github/github over http/
<lifeless> spm: can we do a profile on stagin gplease?
<spm> lifeless: sure; sorry was just stopping the librarian from running outa space and doing a major update on how docco on how to achieve that happy state :-)
<lifeless> yeah
<lifeless> I'd like to do the multi-disk-support thing properly.
<lifeless> if you have design constraints/ideas add em to the bug.
<spm> https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/IncrementTopLevelLibrarianDir for the bored
<lifeless> I wonder if we could just move to an S3 impl
<spm> written for the "AAAAA! Librarian! No Space! FIX NOW" I don't wanna have to work this out from first principles in a hurry losa
<spm> obvious jokes aside; it is something we've rambled about internally. I'm unsure if backups and/or IO would be sufficient tho.
<lifeless> spm: challenges challenges
<lifeless> anyhow, lets get our current system bug free
<lifeless> then we can transform what it does into a requirements spec for evaluating replacements.
<lifeless> spm: so profiling ? yes/no ?
<spm> getting there.... :-)
<thumper> lifeless: no
<thumper> lifeless: spm is doing something for me first :)
<thumper> lifeless: I'll take 3 minutes
<lifeless> ok
<lifeless> there, thats 6 :P
<thumper> lifeless: I'm all done
<lifeless> spm: oh hhhaaaaaiiiii
<spm> heh, you have not been forgotten; just another crit state I had to deal with...
<lifeless> I know
<spm> only 10 reds atm ; 2 since dealt with. progress... :-/
<lifeless> ><
<lifeless> any I can help with ?
<spm> kill the c o u c h d b developers?
<lifeless> I have a small moral objection there
<spm> oh. did that come out loud. oops.
<mwhudson> :)
<spm> lifeless: go for it!
<lifeless> grabbing the new docs
<lifeless> the new one only permits it rather than doing it on every request (unless you set another key as well)
<rockstar> wgrant, I disagree.  They provide a good clean system, but they also provide a system that can be safely compromised.
<lifeless> 30782            0     16.7424      1.1619   storm.store:669(_load_object)
<lifeless> that might be related to the assignments perf problem
<thumper> ââ¹
<thumper> that's how I'm feeling right now
<lifeless> where are enums again?
<lifeless> spm: can you kick the profiling rsync?
<thumper> lifeless: which enums?
<thumper> lazr.enum
<thumper> or for a particular lp part?
<spm> yarp
<lifeless> thumper: particular part
<lifeless> registry
<thumper> try lp.registry.enum
<thumper> I know they were moving them
<thumper> they may not have moved all yet though
<lifeless> thumper: and th eold place?
<thumper> lifeless: interface imports
<spm> lifeless: done
<lifeless> spm: thanks I'm done
<lifeless> turn off allow profiling please
<lifeless> so, storm's load-objects is 80% of Distribution:+assignments
 * lifeless again considers removing storms object cache.
<wgrant> rockstar: The VMs protect from compromise. Not the chroots.
<thumper> damn
<thumper> something is wrong with my groovey new code
 * thumper lunches and mulls it over
<spm> lifeless: reverted
 * spm goes to correct a ZOMG-ULTRA_VIOLENT Critical alert: -ENOCOFFEE
<lifeless> spm: zomg the librarian thing scares me shitless
<spm> lifeless: ahh. welcome to the club.
<lifeless> how often do you do this?
<nigelb> can I interest one of you folks to do a session during https://wiki.ubuntu.com/UbuntuAppDeveloperWeek
<lifeless> what sort of thing did you have in mind?
<nigelb> its about appp development, so something about how lp fits in, recipies, dailies, etc
<nigelb> how LP can be awesome to an app developer
<lifeless> recipes are a significant differentiator over e.g. sf
<lifeless> jamesh: hi
<jamesh> lifeless: hi
<lifeless> https://bugs.edge.launchpad.net/blueprint/+bug/618367
<_mup_> Bug #618367: Distribution:+assignments is taking a long time maybe 20% of the time <timeout> <Launchpad Blueprints:Triaged> <https://launchpad.net/bugs/618367>
<lifeless> jamesh: storm seems to spend about 1/2ms per object in load_objects
<lifeless> the last three comments are relevant
<nigelb> lifeless: sf?
<lifeless> source forge and others
<nigelb> yes, so that's why asked if somone could do a session about how LP could help :)
<lifeless> I htink its a good idea
<lifeless> ask on the lp-dev mailing list though
<lifeless> you'll capture more peoples attention
<nigelb> ok, I will :)
<nigelb> err, which team is that list attached to?
<nigelb> nm, got it
<lifeless> jamesh: any thoughts on that bug ?
<lifeless> jamesh: I'm going to reduce the columns and not return full objects for the eager loading
<jamesh> lifeless: I guess one obvious question is how many distinct objects are you getting from that query?
<lifeless> 3000 distinct -rows-
<lifeless> many repeated person/account/email triples
<jamesh> While it is 3000 rows x 10 tables, I assume it isn't 30K different objects, right?
<lifeless> thats right
<lifeless> if storm could skip deserialising when the primary key for a thing was already retrieved in the same query that would be useful
<jamesh> from the third last comment, is it set_values or _set_values you saw in the profile?
<lifeless> _set_values, attached the kcachegrind to the bug for you
<lifeless> to do seperate person + specs queries without code contortions at higher layers we need an even smarter resultset
<jamesh> so, from how _load_objec() is structured, the _set_values() calls occur when we're updating the values in an existing Python wrapper
<lifeless> I think investing in a sqlalchemy like mapping layer is probably well worth it
<jamesh> the get_object_info() calls are when _load_object() creates a new wrapper
<lifeless> jamesh: so having a way to skip that if its from the same query would be uesful ?
<jamesh> the _load_object() calls with neither are the cases where the row was all NULLs (I assume you're doing a left join here)
<lifeless> yes
<lifeless> the one I was pulling my hair out over the other week
<jamesh> I don't think I can give you a quick answer to the problem: we don't really keep track of whether an ObjectInfo has been fully filled out, so we'd need to track that to decide whether to attempt to fill it in
<jamesh> maintaining such a flag would need to track when the user assigns lazy values to properties too
<lifeless> hmmm
<lifeless> so the challenge is that we have apis which return resultsets, which shouldn't cost a lot to e.g. call and do count()
<lifeless> but /have/ to eager load this related data
<lifeless> its quite possible the db would be faster doing two queries as well (though in theory not; only if its misphrased)
<lifeless> I'd like to get rid of the use of count() in tales
<jamesh> creating a ResultSet and calling its count() method won't hit _load_object()
<jamesh> you should only hit that when calling an API that actually returns objects
<lifeless> let me rephrase
<lifeless> if we restructure this into two queries
<lifeless> there will still only be one attribute (specifications) that the browser/view/template iterate
<jamesh> first grab all specs, then grab all referenced (person, account, email) triples?
<lifeless> so it needs to eager load the people, for the rows being returned, before iteration, and only when iteration is hjappening
<lifeless> jamesh: right,thats what one would want to happen.
<lifeless> I doubt that we'll save much time doing that on the DB end: its 500ms to get the specs alone with no people
<lifeless> probably due to freakishly wide rows or something
<thumper> mwhudson: I need to talk through this problem if you have a minute
<mwhudson> thumper: ok, but a minute would be about right
<thumper> mwhudson: you don't have long?
<mwhudson> thumper: i just seem to have ever mounting distractions
<thumper> :(
<lifeless>  garh
<lifeless> what is
<lifeless> +class Array(NamedFunc):
<lifeless> +    """Implements the postgres "array" function in Storm."""
<lifeless> +    name = 'array'
<lifeless> +
<lifeless> doing in product.py?!
<thumper> lifeless: because that is where it is being used?
<lifeless> also lib/lp/services/database/prejoin.py should be deleted.
<lifeless> thumper: it should in storm itself or lp.services.database.storm
<lifeless> so other folk can find it
<thumper> lifeless: agreed
 * thumper just found a bug...
<lifeless> spm: hi
<spm> yo
<lifeless> spm: I have a small favour to ask
<spm> LIES!
<lifeless> I'd like you to pull a patch onto the staging appserver so I can see if it actually helps
<spm> sure. you have a patch file?
<lifeless> its one commit - rev 11516 in lp:~lifeless/launchpad/bug-618367
<lifeless> I'll make a patch, one sec
<lifeless> or you can just do a merge -c 11516 lp:~lifeless/launchpad/bug-618367
<lifeless> all it changes is specifications (hah what could go wrong) and it passes the tests I wrote for this the other day
<spm> "hah what could go wrong".... oh where do I start.... ;-)
<lifeless> http://paste.ubuntu.com/489589/
<lifeless> but the merge may be easier to do; you tell me :)
<spm> generally a patch is tbh
<lifeless> the pastebin should do then ?
<spm> yup. ta
<spm> lp is a little easier these days being open; but any locked branches - eg configs - are a pita. so patches are generally better.
<spm> lifeless: patched clean; restarting...
<lifeless> ok, its not enough
<lifeless> can you please enable profiling again
<spm> sure
<lifeless> and I'll get a profile and then be out of your hair for a while
 * spm restrains mightily against an outpouring of cynical laughter
<spm> lifeless: should be abck again
<lifeless> kick the profile rsync ?
<spm> oh yeah. sorry - should have anticipated that one.
<lifeless> spm: ok, I have it
<lifeless> profiling off, patch off, gtg.
<lifeless> thanks
<lifeless> spm: actually, hang a sec, i fyou haven't
<spm> I haven't.
<lifeless> trying to see if its DB IO
<lifeless> ok, there is clearly 10 seconds of storm parsing time in there
<lifeless> whats the status of 'mentoring'
<lifeless> should we be deleting stuff for it ?
<mwhudson> i think so
<thumper> k
<thumper> I've got the main fubar fixed
<spm> lifeless: can I reset that btw?
<lifeless> spm: sorry, yes please
<spm> np
<stub> lifeless: So I can't maintain the data structures for the page performance report in RAM now that devpad has less RAM
<stub> lifeless: So I need to spool parsed requests to disk
<stub> lifeless: sqlite3 or spooling to an xdr file (xdrlib) seem suitable
<stub> lifeless: xdrlib will be fastest to write, but generating 500 rows in the report will involve scanning it 500 times. sqlite3 is much slower to write as it needs to update indexes, but the whole file does not need to be scanned to generate a row as data can be pulled out with an index
<lifeless> stub: we'll be getting more ram agian
<stub> lifeless: So don't bother?
<lifeless> it seems to be generating OK at the moment
<stub> Whatever I do, it will make things slower.
<lifeless> so yeah, I wouldn't bother.
<stub> k. I'll dump the kanban card and see if IS keep wining.
<stub> whining even.
<lifeless> rt 41200
<lifeless> charlie mentions a different ticket to swap th ebits back
<lifeless> the monthlies may need a disk spool regardless
<lifeless> but dailies should be fine I think - no?
<spm> stub: we can whine no matter what you do.
<lifeless> rotfl
<lifeless> timing is everything
<spm> reckon. /me sulks
<nigelb> hrm, wgrant isn't around today?
<wgrant> nigelb: Am I not?
<wgrant> I'm reasonably busy.
<wgrant> But I'm around.
<wgrant> Although I'm about to be unbusy.
<nigelb> wgrant: I was asking around eariler if somone wanted to talk about LP awesomeness on app developer week... interested?
 * nigelb hugs spm 
<nigelb> someday you'll get the timing right ;)
<spm> :-)
<wgrant> nigelb: Not really, sorry.
<lifeless> wgrant: you bought a gun and found your project team?
<wgrant> lifeless: Sadly not.
<nigelb> lifeless: wait, what?
<mwhudson> wgrant: you just did all the work yourself?
<nigelb> wgrant: np, I'll look for more victims..err candidates
<wgrant> mwhudson: It's looking that way, although we're only half-way through.
<lifeless> nigelb: ?
<nigelb> lifeless: talk of guns and project teams.. nm, I figured it out
<lifeless> wgrant: did you get a chance to look at the +history thing?
<lifeless> ok, I'm off for a while, I'll be back later to nab jkakar / therve etc and talk about big picture refactorings of storm
<lifeless> jamesh: ping
<jamesh> lifeless: pong
<adeuring> good morning
<lifeless> jamesh: wanting to discuss storm perf with you more
<lifeless> jamesh: looks like 66% of the time was in 10% of the objects
<mrevell> Hello
<wgrant> bigjools: That fix hasn't landed yet, has it?
<bigjools> wgrant: I'll find out shortly :)
<lifeless> statik: if you want to catch up nowish I could do a relatively brief skype call
<statik> lifeless: i'm toggling between an in-person meeting on OpenERP and a security bug, i'll probably have to wait until next week. sorry man
<lifeless> thats ok
<lifeless> just bein gopportunistic
<lifeless> I have to wait around a bit to stalk jkakar about storm performance anyhow
<bigjools> lifeless: still around?
<allenap> Is there a staging codehosting environment?
<mwhudson> allenap: yes
<mwhudson> allenap: but it's a bit odd because it has hardly any other underlying branches
<allenap> mwhudson: It's not refreshed at the same time as the database?
<mwhudson> allenap: all 600 gigs of it?
<allenap> mwhudson: Yeah :) Okay, guess not. If I'm interested in manipulating some of those branches, I suppose I can push them first, and things will work?
<mwhudson> yeah
<mwhudson> the bzrtools branchess are synced
<mwhudson> iirc
<allenap> mwhudson: Excellent, thank you.
<deryck> Morning, all.
<lifeless> bigjools: not really; whats up?
<bigjools> lifeless: it can wait.  I want to talk about how zopless scripts interact with the webapp with regard to table locking and transactions
<lifeless> 'badly' :P
<bigjools> I figured
<bigjools> let me re-phrase
<bigjools> I want to make it work :)
<lifeless> I'm growing more and more of the opinion that we need the same protection the webapp has, in scripts
<bigjools> which protection?
<lifeless> there are a few ways to do that
<lifeless> timeouts
<lifeless> enforced
<bigjools> hmmm
<bigjools> in my case I want to add a build cancellation feature, but the webapp would need to change build states potentially at the same time as the buildd-manager
<lifeless> how so, and how is it different to two people changing a bug status at the same time ?
<bigjools> because the b-m is a script
<bigjools> and that is where my uncertainty lies
<bigjools> there's a race condition where the build is NEEDSBUILD and the webapp wants to cancel it and the b-m wants to dispatch it
<bigjools> I need to make sure that they behave.  In the old days you'd have mutexes or something.
<lifeless> ok
<bigjools> I can avoid it if I do something like make the webapp use another status that's a "request to cancel" and then make the b-m handle it
<lifeless> so you can model this with a table
<bigjools> but that's not ideal from a user PoV
<lifeless> you have NEEDSBUILD-><something>
<lifeless> and you have two mutators
<lifeless> webapp chooses cancelled
<lifeless> bm chooses dispatched
<lifeless> if the webapp ins the bm commit will fail
<lifeless> if the bm wins, the webapp request will be retried
<lifeless> (and won't see it as NEEDSBUILD the next time around)
<lifeless> if the bm commit fails, it shouldn't dispatch
<bigjools> what makes the commit fail?
<lifeless> conflict
<bigjools> what detects that?
<lifeless> pg
<lifeless> open up two pgsql launchpad_dev sessions
<bigjools> does it need a special txn mode?
<lifeless> in both do 'begin;'
<wgrant> The webapp uses READ COMMITTED.
<wgrant> So watch out.
<lifeless> then do an update in one like the bm will do, and in the other like the webapp - just the status field they both write to is enough
<wgrant> Really really watch out.
<bigjools> exactly my point wgrant :)
<bigjools> the b-m also uses the @write_transaction decorator which retries IIRC?
<lifeless> write locks are still txn wide in read committed
<wgrant> bigjools: It does retry, yes.
<StevenK> Does python have a sh -n equivlant?
 * StevenK is trying to remember
<lifeless> I think it will work; you will definitely ant to verify that it does - even if only by a manual test script
<bigjools> I'm going to run the b-m and the webapp in 2 debug sessions and see what happens
<bigjools> I don't trust it simply because I think this sort of thing has gone wrong before
<lifeless> plenty of ways it can
 * lifeless would be happiest if b-m didn't talk to the db
<lifeless> it would make this trivial
<bigjools> you mean talk through a webapp using the api don't you
 * bigjools shudders
<lifeless> or the internal xml server
 * bigjools shudders again
<bigjools> xmlsucks.org
<StevenK> Oh dear, this argument again?
<bigjools> :)
<bigjools> I'd be very happy to be persuaded that it would all work fine
<lifeless> oh xml is terrible, no argument.
<bigjools> but I'm unconvinced since it's pushing the problem to a different place
<lifeless> the LP API already has semantics for in flight collisions
<lifeless> it gives a patch error
<lifeless> FWIW
<bigjools> what about the webapp?
<bigjools> well, the UI I mean
<StevenK> lifeless: I think we need to make the API *much* quicker before we can argue about b-m using it with any credence.
<lifeless> when it retries -AIUI, I haven't checked this code myself- it will start over from scratch.
<lifeless> StevenK: yes, would you like a list of soyuz API calls that are shocking? Available on demand.
<bigjools> StevenK: I think that's already understood :)
<StevenK> lifeless: Not assigning blame
<lifeless> StevenK: the API is not slow because its the API - in general; its slow because LP is slow and the API is a thin skin over the LP model objects.
<lifeless> StevenK: its -trivial- to make a set of dedicated extremely fast API calls for services like the codehosting machines have
<bigjools> part of the problem with the API is that it serializes all of the object - even bits I don't care about
<lifeless> anyhow, yes thats a definite constraint
<bigjools> and often the developers don't remember this when adding new "properties"
<lifeless> yes
 * StevenK cries since making the wadl is dying with SyntaxError
<lifeless> I need to collate some of the lessons in this area for general discussion
<lifeless> but back to the question
<lifeless> the webapp starts over
<bigjools> lifeless: anyway, I shall try some debug sessions to see how the webapp interacts with a b-m under conflict conditions
<lifeless> it will read the *new* state, and as long as the form handler is sensible (that is that a 'cancel' returns a 'cannot cancel, its <something>' when the thing wasn't in NEEDSBUILD, then the web ui will be fine.
<StevenK> And the SyntaxError is *so* helpful :-(
<bigjools> lifeless: the problem is that the transaction boundaries in the b-m are shit
<bigjools> I need to review it and figure out WTF it has commit() inside @write_transaction
<lifeless> bigjools: I can believe that; thats part of the stuff making me speculate about APIs
<lifeless> bigjools: the merge proposal worker is currently causing timeouts in the webapp
<lifeless> similar story
<lifeless> long transaction
<bigjools> lifeless: I would be on board with the API idea iff it was *thin* and super super fast
<lifeless> webapp wants to update same thing, timeout.
<bigjools> blocked on a lock?
<lifeless> bigjools: While I can be very critical of some parts of it - and I have, on the list - it is pretty darn thin and quite lean; for the use case of b-m I think it would be fine.
<bigjools> lifeless: the restful aspect of it makes me cry
<lifeless> certainly you could try putting selected bits of b-m into the API and see whether they are good or bad, low risk, easy rollback etc.
<lifeless> bigjools: rest has some serious limitations that I dislike too.
<lifeless> and we're not even really doing rest all that faithfully.
<bigjools> honestly, I just can't see me ever making the b-m use the API as it is
<lifeless> I'd appreciate it if you made sure there are (reasonably) focused bugs for what you need changed.
<bigjools> lifeless: anyway, thanks, I'll let you know how I get on with the testing
<bigjools> I don't know what I would need changed :(
<lifeless> I definitely want the API suitable for implementing backend services on.
<lifeless> bigjools: well, 'I tried to put <x> in the API and it was 10 seconds slower' would be a great bug :)
<bigjools> heh
<lifeless> seriously, make sure that one update you do is in the API
<bigjools> before I take the time to do that I want to see clearly what the pros are
<lifeless> secured appropriately
<lifeless> the pros are:
<bigjools> the b-m would have to be an authenticated user
<bigjools> unless we give it a private webapp
<lifeless>  - clear transaction boundaries. Crystal clear, and enforced via the primary stack we use.
<lifeless>  - easier testing : the twisted stuff which doesn't play - that - well with storm. postrgresql & zope - will become more pure twisted, and less godzilla hybrid from the deeps
<lifeless>  - we can, if we go the full monty, get rid of zope from the b-m *entirely*
<lifeless> bigjools: yes, it would be a celebrity, like the software centre agent, I suspect.
<lifeless> or something approximately like that
<bigjools> well I am sprinting with jml in a couple of week's time and we're overhauling the remaining synchronous stuff in the b-m, I'll make sure we consider your concerns when we redesign bits of it
<lifeless> it is at this point just an idea; I think its worth exploring, but its an idea.
<lifeless> Its *one* implementation strategy for 'please ensure that ALL things connecting to the DB have *enforced* transaction timeouts. Thanks.'
<bigjools> the latency on the requests would have to be much much quicker (consider that versus a native storm query)
<lifeless> there are other implementation strategies, and we probably want to try them all in different places and see what works best.
<lifeless> bigjools: apples and oranges; you'd structure it a little differently
<bigjools> well, like I said, I'm sceptical but very happy to be proven wrong
<bigjools> wgrant: do you remember when the package copier was fixed to prevent file checksum conflicts?
 * bigjools is clearing up the 6 PPAs with unpublishable packages
<lifeless> bigjools: so I don't think its a matter of proving you wrong - there are plenty of services in the dc already running against the API
<bigjools> lifeless: I'm sure there are, but the b-m is an extremely sensitive component
<lifeless> bigjools: I'm totally sure there would be things to fix. I just can't predict them more accurately than saying - give it a shot, in a safe incremental way, and when we hit a problem, file a bug and stop there until its fixed.
<lifeless> anyhow, irssi tells me it just went midnight; have a great day!
<lifeless> gnight
<bigjools> lifeless: gnight!
<wgrant> bigjools: April some time.
<bigjools> wgrant: figures, the most recent corrupt publication is 2010-04-03
<wgrant> Excellent news.
<bigjools> I'll just delete them all
<wgrant> Delete or DELETE?
<bigjools> latter
<wgrant> Sounds reasonable.
<bigjools> whee storm is rolling in, I expect to go offline
<bigjools> the third world substation here trips as soon as I get thunder
<wgrant> Heh.
<wgrant> Lovely.
<thefish> sorry for silly question, but is it possible to run launchpad locally, similar to something like redmine? Ive installed as per the /Getting etc instructions, but the instance I have contains loads of other oss projects - would like to use this for internal-only dev if possible/allowed
<deryck> hi thefish.  That's probably more appropriate for #launchpad, but the short answer is that the code base is designed for running locally for development, not deployment.
<thefish> deryck: thank you for the answer, thats exactly what we want to do, just a replacement for redmine - sorry about wrong channel :) is there any docs about how to use it for local-only stuff (we dont want to see bugs for anything other than our own projects, and we dont want our own projects pushed to landscape.net)
<thefish> ^ shall i ask in #landscape instead?
<deryck> thefish, it's #launchpad not #landscape.  Too similar I know :-)  And sure, let's move it there.
<thefish> :)
<cr3> if I'm extending launchpad outside the core of the project, which naming would be preferable/more consistent: launchpad_foobar or launchpadfoobar?
<cr3> it seems that the former is more readable but that other projects prefer the latter, like launchpadintegration for example
<benji> cr3: if you mean a Python package name, many people have a strong dislike for underscores in package names
<cr3> benji: I recall something in the PEP8 that underscores are preferable for readability, but I guess many people prefer typability over readability :)
<benji> the pertinant bit of PEP-8 is "Python packages should also have short, all-lowercase names, although the use of underscores is discouraged."
<benji> (although I certainly don't treat PEP-8 as gospel)
<cr3> benji: ditto, thanks for the advice :)
<m4n1sh> I am preparing a presentation for giving a talk at PyCon India on Launchpad. I just wanted to make sure that Soyez has been made open source or it is still proprietary ?
<beuno> m4n1sh, it was open sourced with everything else
<beuno> nothing got held back
<m4n1sh> benji: I can't find the branches under it;s launchpad's project
<m4n1sh> https://code.launchpad.net/soyuz <-- can't find any source code
<benji> m4n1sh: me neither!
 * benji wonders what we're talking about.
<m4n1sh> benji: if it is open sourced, then where is the code?
<m4n1sh> even I heard it was getting open sourced. Didn't follow after the news came out
<beuno> m4n1sh, it's all part of the same source tree
<m4n1sh> under launchpad.net/launchpad ?
<beuno> yes
<m4n1sh> beuno: thanks.
<beuno> m4n1sh, the full bzr history got released
<m4n1sh> col
<m4n1sh> *cool
<cr3> when using openid to authenticate against login.launchpad.net, is there a way to get the email address of the user?
<cr3> I tried sreg.SRegRequest(required=["email"])) but this results in an empty sreg.SRegResponse.fromSuccessResponse(response)
<lifeless> moin
<james_w`> cr3: you have to get you app added to a trusted list
<cr3> james_w`: I'm trying to find a simpler way and I think I got it thanks to the openid identifier
<lifeless> jkakar: hi
<EdwinGrubbs> abentley: ping
<abentley> EdwinGrubbs, pong
<lifeless> abentley: btw got your mail the other day; am ruminating on it
<abentley> lifeless, cool.
<EdwinGrubbs> abentley: I was looking at the various BranchJob classes to understand how to create my own for adding members to teams. Why is there a branch foreign key on the BranchJob table, but there is also a branch id in the metadata sometimes? Why not just put all the data in the metadata?
<abentley> EdwinGrubbs, I'm not sure.  Which BranchJob has a branch id in its metadata?
<EdwinGrubbs> abentley: ReclaimBranchSpaceJob
<jkakar> lifeless: Hiyas
<lifeless> jkakar: hi
<lifeless> jkakar: want to talk storm performance ?
<abentley> EdwinGrubbs, probably because that one has to refer to the branch id of a branch that has been deleted.
<jkakar> lifeless: Uhm, sure.  Want to hop over to #storm?
<lifeless> sure
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<EdwinGrubbs> abentley: I'm wondering why we go to the trouble of creating new tables for new foreign keys when it is ok to put some foreign keys in the metadata.
<cr3> james_w`: easy question for you: if the same user authenticates using openid against login.launchpad.net and oauth against launchpad.net, what would be the simplest way to obtain a common handle about that user?
<abentley> EdwinGrubbs, at that point, the branch id is not a foreign key.  There is no id corresponding with that branch in the Branch table.  The purpose of the id in that case is to refer to the location of files on disk.
<cr3> james_w`: s/launchpad.net/api.launchpad.net/
<james_w`> cr3: I don't really understand the question. openid gives you an identity, oauth gives you an auth token. If your app cares about identity it would normally tie the latter to the former.
<EdwinGrubbs> abentley: ok, so for an AddMemberJob table, I should have a foreign key for super_team and for new_member.
<james_w`> cr3: or are you talking about from the LP point of view?
<jelmer> what's up with staging?
<abentley> EdwinGrubbs, that sounds like a good idea, because you might want to find that job by querying on super_team and/or new_member.
<cr3> james_w`: how could I tie the latter to the former, other than by making api calls with the handle obtained from oauth like handle.me.preferred_email_address for example
<EdwinGrubbs> thanks
<abentley> EdwinGrubbs, no problem.
<james_w`> cr3: depends on your app. I assume this is a webapp?
<cr3> james_w`: yes
<james_w`> cr3: and you want act on behalf of the user to do some things on LP?
<cr3> james_w`: but the webapp does not have a dedicated user for doing things on launchpad, it proxies request from the user interacting with the webapp using launchpadlib
<james_w`> cr3: don't use a single user!
<cr3> james_w`: on behalf of the user, but as the user
<cr3> james_w`: right, no single user
<cr3> james_w`: however, users will sometimes access the webapp using oauth and something openid. I'd like both to refer to the same person object somehow
<cr3> s/something/sometimes/
<james_w`> cr3: right, so to "impersonate" me, you would have me to the "oauth dance", which would give you an oauth token. You then make API requests signed by that token, which make the changes on Launchpad.
<james_w`> cr3: users /never/ access your webapp using /Launchpad's/ oauth
<cr3> james_w`: I've got that dance working already, it's pretty cool and I can use launchpadlib as is
<james_w`> cr3: you use oauth on their behalf to talk to Launchpad
<cr3> james_w`: exactly, the user does the oauth dance
<james_w`> cr3: great. Then how do you stop me from using your oauth token to do bad things that will be attributed to you?
<james_w`> or, how do I stop you? ;-)
<cr3> james_w`: you answer read-only access during the oauth dance or something
<cr3> james_w`: the concern you raise for the webapp is exactly the same as for any other application
<james_w`> cr3: I mean from the webapp's point of view
<james_w`> consider a really simple webapp with a text box, and a button that says "add comment to bug #1"
<cr3> james_w`: I'm not sure I follow, the "you" and "me" in your question confused me
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
<james_w`> and the webapp wants to add that comment as the user who is accessing it
<james_w`> it will do that using their oauth token.
<cr3> james_w`: I wouldn't do anything like that, changes to launchpad can only occur as a result of using the api, not the web interface
<james_w`> cr3: which api?
<cr3> james_w`: the webapp's api
<james_w`> cr3: ok, same thing applies though
<cr3> james_w`: conceptually, since I have the oauth information, I could very well make changes in launchpad as anyone
<james_w`> cr3: a webapp method, that adds a comment to that bug. How does the webapp know which user's Launchpad oauth token to use when making the Launchpad oauth request?
<james_w`> it may have thousands of oauth tokens
<cr3> james_w`: isn't that what /dev/srandom is for?
<james_w`> you are just going to pick a random user?
<cr3> james_w`: seriously though, I expect the session to tell me but, you're right, nothing prevents me from using an arbitrary user
<james_w`> cr3: right. I'm not trying to expose the fact that your webapp could be malicious, just trying to expose the real question underlying yours.
<james_w`> cr3: the session needs a user identity for which it can find the oauth token to use
<james_w`> therefore your webapp needs a user database, which contains at least a link to an oauth token to use
<james_w`> if there is no oauth token then you refuse the request and make the user do the oauth dance
<james_w`> but, you don't want to manage identity in the webapp, you want to delegate that to Launchpad
<cr3> james_w`: I'm trying hard to avoid having a user database. if I could get a common handle somehow, I was hoping to just be able to instantiate volatile users with the handle passed to the constructor
<james_w`> that's where openid comes in
<james_w`> you create a user when someone logs in using an openid that you have not seen before, and you tie the openid identity to the user, and hence the Launchpad oauth token
<cr3> james_w`: from the user, I could then get oauth_access_tokens, just like in launchpad, or identity_url for openid. all from the same volatile user object
<james_w`> then, assuming you are using oauth for the webapp's API, you also implement the server-side part of oauth to hand out your apps oauth tokens
<cr3> james_w`: so users need to use the web interface in order to create an identity in a persistent store, right?
<james_w`> cr3: you can call it something else than User, but that's what it is, it's an identity table, and there is no way to avoid it
<james_w`> cr3: indeed
<cr3> ok, so I need an identity table, whatever the name is. then, when the same user uses the api and authenticates using oauth, how do I relate that to the existing identity?
<james_w`> you need something to tie identity (delegated to Launchpad via openid), to the oauth token to use for authorization when acting on behalf of that user with Launchpad, and to the valid oauth tokens that they can use to talk to your webapp's API.
<james_w`> cr3: by implementing the server-side part of oauth, it should be fairly obvious how to do that part when you implement that
<james_w`> when you get an oauth signed request you can look up a user from that and retrieve the Launchpad oauth token to use
<cr3> james_w`: I started implementing the server-side part of oauth and I reverted that to proxy requests straight to launchpad
<james_w`> cr3: so oauth requests that your webapp receives would be forwarded straight to Launchpad, without re-signing or something?
<cr3> james_w`: one of the major roadblocks to having my own server-side part is that I'd really benefit from using launchpadlib as-is
<cr3> james_w`: that's where I'm at now, yes
<james_w`> cr3: ok, so two comments, firstly, I don't think that's possible under oauth, because the headers are signed, and the headers aren't valid to send to Launchpad, and two, why does your application exist at all if they can just talk directly to Launchpad?
<cr3> james_w`: firstly, my webapp seems to proxy the oauth dance to launchpad successfully, I just need to make sure the Referer points to the right place and voila
<cr3> james_w`: secondly, it exists because it extends the launchpad api with additional functionality. for example, using the same handle against my api, you could use launchpad like lp.projects['bzr'] and you can get results like lp.results[-1]
<james_w`> ok
<cr3> james_w`: sane, almost sane or completely insane?
<james_w`> if you can get it to work, then go for it, but I think you will find that you get requests that you can't proxy without re-signing
<cr3> james_w`: I'm not sure my understanding of oauth is good enough to validate the re-signing part and I suspect I'll find out at my own cost :(
<james_w`> cr3: you say you proxied the dance, did you proxy any signed requests?
<james_w`> because I would expect that it would just say 401 Unauthorized to any proxied requests
<cr3> james_w`: haven't tried yet, just the +request-token, +authorize-token and +access-token
<james_w`> +access-token I would expect to reject it
<cr3> james_w`: I'll give that a try next at least to validate I'm going in the right direction
<cr3> james_w`: the whole dance process worked, that I can confirm
<james_w`> plus, without your own oauth access tokens you have no way to validate requests like lp.addResult(...)
<cr3> james_w`: I was hoping that if launchpad returned an access-token, then requests are valid
<sinzui> thumper, ping
<cr3> james_w`: the only thing is that I don't necessarily have the access granularity of launchpad: read-only, read-write, etc.
<james_w`> cr3: but you can't do that on every request
<cr3> james_w`: when the information is successfully returned from +access-token, that's when I cache the oauth consumer and token in my own database and use that for subsequent requests instead of doing the dance every request
<cr3> james_w`: my understanding of oauth is very limited, so I'm not claiming this is necessarily correct, so your critique is very welcome :)
<james_w`> cr3: but how do you associate my access token that you just stored with me when I make my next request?
<cr3> james_w`: isn't that sent in the HTTP_AUTHORIZATION header?
<cr3> james_w`: however, note that I can only assess that the user has previously successfully gotten an access token from launchpad, I'm at the point of associating that with a user which is when the openid <-> oauth debacle all started
<james_w`> cr3: yes, but how do you have any idea that is is correctly signed?
<cr3> james_w`: crap, I need nonce information for that, right?
<james_w`> yes
<cr3> james_w`: and that's never sent over the wire, just server side, right?
<james_w`> yes
<cr3> james_w`: actually, I observed this was sent over the wire when doing the dance: janrain_nonce=2010-09-07T15%3A15%3A54ZoRpj7Y
<cr3> james_w`: however, that's for the request token rather than the access token, right?
<james_w`> cr3: in order to verify it you have to MITM the communication between LP and the user, which oauth is explicitly designed to prevent
<james_w`> cr3: basically, if you find a way to do it, it will be a security bug in the oauth protocol, and promptly closed
<thumper> sinzui: hi
<sinzui> thumper, We need to talk about the death of gmaps. Do you have time today for a skype call
<cr3> james_w`: dude, you totally broke my fun. now I have to redesign the whole oauth thing! but thanks, better to know early than late :)
<thumper> sinzui: sure, just let me get the kids off then we can chat
<james_w`> cr3: I suggest you add a User table (or whatever you want to call it), and implement oauth client and server
<sinzui> thansk
<sinzui> thanks
<james_w`> cr3: it's exactly how this stuff is supposed to be used, so you won't be fighting against it
<cr3> james_w`: I had most of that implemented already, so it's just a question of combining that with the launchpad dance
<mwhudson> janrain_nonce is to do with openid, not oauth, fwiw
<cr3> mwhudson: thanks
<cr3> james_w`: so, if my webapp handles the oauth server side, can the webapp turnaround and get a handle against launchpad or should it not do that?
<james_w`> cr3: it can indeed
<cr3> james_w`: I'm not sure what the user experience would be. lets say someone uses launchpadlib against my webapp's url with their email address, what would happen next in terms of the user experience?
<james_w`> cr3: they can't use their email address with launchpadlib
<james_w`> cr3: let me doodle in a pastebin
<cr3> james_w`: when I used launchpadlib, I've used something like: Launchpad.get_token_and_login(consumer_name...) where consumer_name is my email address
<james_w`> consumer_name is arbitrary
<thumper> sinzui: skype up and we can chat now
<thumper> sinzui: I have to take the little car for a warrent this morning
<cr3> james_w`: ah! that clarifies another question I had, thanks!
 * cr3 takes a little break to let all of this sink in
<cr3> james_w`: I totally understand how to link the openid to the oauth token now, that's why +request-token redirects to an openid page like https://launchpad.net/+authorize-token/+login?oauth_token=5V7gnDVP2MqTjw2TKPg2 which can then link that token to the identity, right?
<james_w`> cr3: if you are not logged in, then it will ask you to login, yes. It requires that so that it knows who is granting access.
<james_w`> http://paste.ubuntu.com/490014/
<cr3> james_w`: do you think I can conceptually achieve this user experience using launchpadlib as is, assuming I adhere to the urls expected by launchpad?
<james_w`> cr3: yes, if you use lazr.restful then you should be good to go
<cr3> james_w`: yep, I'll give that a try and let you know how it goes :)
<james_w`> great
<cr3> james_w`: by the way, do you happen to know of any code that does what I'm trying to do? I think that the software-center-agent uses a single user instead of going through the hoops we've discussed
<james_w`> cr3: I believe it does
<james_w`> cr3: the font testing tool did something like this IIRC
<cr3> james_w`: this code in sca seems to indicate they have a hard coded access token for all launchpad requests: oauth_token = OAuthToken(settings.LP_ACCESS_TOKEN, settings.LP_ACCESS_SECRET)
<cr3> james_w`: and I just got confirmation from achuni that actions are performed as a single user, hopefully my code could be reused by that project then :)
<lifeless> is staging meant to be down ?
<lifeless> hmmm, I guess it might be codeupdating
<lifeless> devpad, wherearthough ?
<lifeless> I hat emuscle memory
<EdwinGrubbs> abentley: does BranchMergeProposalJobDerived delegate instead of subclass from BranchMergeProposalJob just because storm doesn't like multiple classes linked to the same table?
<abentley> EdwinGrubbs, no, it does it because AFAIK, Storm doesn't support inheritance.  (Other stormified classes inherit from BranchMergeProposalJobDerived)
<EdwinGrubbs> abentley: according to the storm docs, you can inherit, but you have to override the __storm_table__ attribute in the subclasses.
<abentley> EdwinGrubbs, wouldn't that make it impossible to use the data provided by that table?
<EdwinGrubbs> abentley: right, it doesn't help in this situation. Storm usually expects subclasses to have a table with all the columns. Actually, now I see in storm's infoheritance.txt, it does have a pattern for subclasses that have an extra table with just the extra columns it needs. However, it doesn't seem to provide any benefits over using delegates, and it has a confusing registration process.
<wallyworld_> morning
<lifeless> is it possible to specify a host in the database config ?
<thumper> wallyworld_: morning
<wallyworld_> thumper: standup meeting happening today?
<thumper> wallyworld_: the other two had to head
<thumper> wallyworld_: but I'm around
<thumper> wallyworld_: but I need more coffee
<wallyworld_> thumper: np. i've got a couple of bugs left in my queue
<thumper> wallyworld_: let me make a coffee then we can have a call
<wallyworld_> thumper: ok, i'll have a flat white :-)
<thumper> wallyworld_: here you go -> â
<wallyworld_> :-D
<thumper> wallyworld_: what is the name of the IDE you use?  I've a friend asking about python IDEs
<wallyworld_> thumper: PyCharm (from the guys in do Intellij for java). It's still in beta though (beta2).
<wallyworld_> s/in/who
<bdmurray> If I want to use a css class but don't like one part of it how can I change it on just my page?
#launchpad-dev 2010-09-08
<beuno> bdmurray, you can override values
<beuno> do you have a pastebin or something more tangible?
<bdmurray> beuno: https://pastebin.canonical.com/36841/ unsub-icon uses an absolute position which doesn't really work for me
<beuno> bdmurray, so you can either create a new class and use that, or add an id in that td, and create a new css entry that does something like:  #new-id .unsub-icon {position:relative;}
<bdmurray> beuno: I believe there is some magic javascript that unsub-icon has for unsubscribing from a bug.  would that be lost?
<beuno> bdmurray, nope, the javascript would still find that class, should be fine
<beuno> well, fine if you use the second option, of course  :)
<beuno> overriding the specific option via CSS rather than creating a new one
<bdmurray> beuno: great, thanks!
<beuno> np
<lifeless> spm: here again ?
<spm> lifeless: yurs; but gettnig handover atm
<lifeless> when you have that done please ping me
<lifeless> tracking down this 10second overhead in staging.
<lifeless> of course, getting stating going again is a precursor
<wallyworld_> thumper: afaict, the only way to change a review type is to delete and re-create the merge proposal? resubmitting doesn't allow review type to be edited
<thumper> no
<thumper> wallyworld_: just request another with UI specified
<thumper> wallyworld_: there is a link 'request another review'
<wallyworld_> thumper: np, just wanted to check that i wasn't supposed to be able to just edit the current record
<thumper> wallyworld_: well... kinda
<thumper> wallyworld_: but not well
 * thumper heading afk
<lifeless> zomg Product:EntryResource:get_timeline - 2400 queries
<thumper> lifeless: eh?
<lifeless> lp net oops summary
<mwhudson> that's quite a lot of queries
<jtv> lifeless: too late for Performance Tuesday, but you might be interested in bug 632880
<jtv> wgrant: I think it's time I updated the translation templates build stuff
<wgrant> jtv: To the new schema?
<jtv> wgrant: yes
<wgrant> Yes please.
<jtv> One thing that seems needed is a TranslationTemplatesBuild.
<wgrant> Correct.
<jtv> Does that mean I can ditch TranslationTemplatesBuildJob?
<wgrant> You still need something like it.
<wgrant> For now.
<wgrant> Since we're still using the old queue mechanism, until you move across.
<wgrant> But eventually you can ditch it, yes.
<jtv> But first I'll make it refer to a BuildFarmJob instead of a BuildFarmJobOld?
<jtv> Or is that just a matter of  removing _set_build_farm_job?
<wgrant> jtv: TranslationTemplatesBuildJob continues to derive from BuildFarmJobOldDerived.
<wgrant> You then need a new TranslationTemplatesBuild(BuildFarmJobDerived)
<wgrant> Yes, this sucks, but it wasn't meant to last for more than a few weeks.
<jtv> sorry
<wgrant> Hm, Maverick's not bad.
<jtv> wgrant: how will I know that I've got a working new setup, as opposed to still leaning on the old one?
<wgrant> jtv: At this stage you pretty much just need a BuildFarmJob, I think.
<jtv> oh cool
<wgrant> Then we can migrate the queueing stuff to that, and dismantle the other infrastructure.... and work out what else needs to be moved before we can do that.
<wgrant> I'm not entirely sure of Soyuz plans for the next step. noodles did the work, so he might.
<wgrant> But the important thing now is that we get all jobs into BuildFarmJob, so we can kill off BuildQueue. Then we no longer need the BQ<->build link objects like TTBJ, SPRBJ and BPJ, and we can destroy them at our leisure.
<jtv> I guess Job also drops completely out of the picture
<jtv> Going to be fun cleaning up that forest of job/branchjob/buildqueue cross-references
<jtv> I suppose I also need a factory for my new class.
<wgrant> For the moment, yes, Job is pretty much out of it.
<wgrant> But once it's completely out of it, most of BuildFarmJob will be replaced with a new Job reference.
<jtv> whoopee
<wgrant> But that's easy to do without attacking BFJ users too much.
<wgrant> And probably better than having multiple Jobs for each BFJ, as we would if we made that change now.
<jtv> Yes
<jtv> I can see how it makes sense.
<mwhudson> wgrant: how did we end up in the position of needing to kill off BQ now, rather than when we changed everything else?
<wgrant> mwhudson: I don't recall exactly. But I think it partly stemmed from not wanting to have persistent objects for some job types (Translations, in particular).
<mwhudson> wgrant: oh, while still wanting to have them for packagebuilds?
<wgrant> And some were quite attached to BQ at the time.
<wgrant> mwhudson: Right.
<wgrant> So we couldn't have a persistent BuildFarmJob.
<wgrant> So it couldn't replace BuildQueue.
<wgrant> However, the objections to removing BuildQueue have now evaporated.
<mwhudson> heh
<wgrant> So we can make everything a BuildFarmJob, and remove BuildQueue.
<wgrant> And lots of other stuff.
<wgrant> Which shouldn't really have been there in the first place :)
<mwhudson> sounds a bit like taking the long way around
<mwhudson> but if it results in deleting code, i guess i'm happy
<wgrant> It certainly is the very, very long way around.
<wgrant> But opinions now seem different from what they were in Wellington, so the new, sane structure can prevail.
<mwhudson> heh
<jtv> Let the record show that I argued violently against sanity in any shape or form.
<wgrant> Heh.
<jtv> wgrant: where and when should I produce TranslationTemplateBuild objects?
<wgrant> jtv: Wherever you're producing TranslationTemplateBuildJobs now.
<jtv> Produce both for the time being?
<wgrant> jtv: The existing builds have a makeJob method which creates the BuildQueue and *BuildJob.
<jtv> Or stop producing TTBJs now?
<jtv> So I should mimick that?
<wgrant> Ah, in fact that's on the interface.
<wgrant> So, yes.
<jtv> OK, this sounds treacherously simple.
<spm> heya jtv, wgrant
<jtv> hi spm
<wgrant> Hi spm.
<wgrant> Hm, massive downtime for the rollout... does this mean the Lucid upgrade is happening?
<spm> yup
<spm> well. more of it.
<wgrant> DB?
<spm> no, that'll stay on 8.3
<wgrant> :(
<lifeless> jtv: thanks
<lifeless> wgrant: SPOF upgrades e.g. librarian to lucid
<lifeless> jtv: yes, I'd seen that; depending on caching is fragile and undesirable
<wgrant> lifeless: librarian really shouldn't be a SPOF. But OK.
<lifeless> wgrant: agreed. And yet, it is.
<lifeless> wgrant: there are two instances running, but one disk store and one machine.
<wgrant> Yay.
<jtv> lifeless, re caching: true, though I can't think of anything better we have at the momentâ¦  but would you agree that what I suggest is an improvement?
<jtv> There seems to be a pattern in the librarian of dealing with LibraryFileAlias ids where it should probably deal with LibraryFileAlias objects instead.
<jtv> Maybe in some cases that could let us avoid querying the objects, in which case it's not necessarily bad.  But again that sort of optimization will be fragile.
<lifeless> the main thing is to make sure you don't introduce the possibility of accidental DB access in the librarian server.
<lifeless> other than, sure
<wgrant> jelmer: In your b-m upload branch, are you deliberately not passing the buildid into the policy any more?
<wgrant> It should still work, but I'm a little doubtful that activating the guessing logic is a stunning idea.
<wgrant> I want to remove all that logic RSN, since mixed uploads are gone.
<wgrant> It is going to let people do some pretty damn confusing stuff if they want to.
<wgrant> But I don't think it actually opens up any security holes.
<wgrant> It also won't unset the upload log any more, but that might be done by the retry.
<wgrant> Well, no major security holes.
<gmb> noodles775, Morning. What's the story with your branch for https://bugs.edge.launchpad.net/soyuz/+bug/611568? Can you get it into PQM before 12:00 UTC do you think?
<_mup_> Bug #611568: Suppress email notification for SCA P3A subscriptions <software-center> <trivial> <Soyuz:In Progress by michael.nelson> <https://launchpad.net/bugs/611568>
<spm> ha. landed in pqm 2 minutes later. rofl.
<noodles775> gmb: yeah, I've got it on the pqm queue again now (it was rejected earlier).
<noodles775> Ah, great :)
<noodles775> Still on the queue afaics.
<gmb> noodles775, Cool.
<adeuring> good morning
<mrevell> Hello
<wgrant> bigjools: Have recipe builds been tested on DF with the build upload processor branch?
<wgrant> I don't see how they can work....
<bigjools> wgrant: jelmer says yes
<noodles775> gmb: landed with r9760
<wgrant> But the buildid isn't being passed in any more, AFAICS :/
 * wgrant tries.
<bigjools> hmmm
<bigjools> he made it part of the directory leaf name
<wgrant> Yep.
<wgrant> But it's not put in the upload policy.
<jelmer> wgrant, why would it need to be in the upload policy?
<wgrant> jelmer: Er, options, not policy, sorry.
<wgrant> jelmer: Binary and source uploads look in options.buildid to work out which build they're working on.
<jelmer> wgrant: If options.builds is set they extract the build id from the directory name, and ignore options.buildid
<wgrant> jelmer: Binary builds will guess which build to use, and will create one if it's not found (which users can abuse).
<wgrant> jelmer: Even BinaryUploadFile.findBuild?
<wgrant> And DSCFile.findBuild?
<wgrant> DSCFile.findBuild will just not do anything if it's not set.
<wgrant> Which should result in failedtoupload SPRBs.
<wgrant> (why yes, this is a completely revolting way of doing things, but that's how it is :/)
 * jelmer investigates
<wgrant> I'm trying to test locally.
<wgrant> But Maverick doesn't like me too much.
<jelmer> some of Aarons leftover sourcerecipebuilds ran through dogfood while this branch was there
<wgrant> jelmer: I can't see any... are they not under his recipe?
<wgrant> Ermm.
<wgrant> I think it may be broken in other ways as well.
<jelmer> wgrant: How?
<wgrant> jelmer: It doesn't unset BuildQueue.builder when it sets the status to UPLOADING.
<wgrant> SO when buildd-manager comes around again, it resets it and starts building it again.
<wgrant> (because it thinks the builder has forgotten it)
<bigjools> :(
<wgrant> So it retries.
<wgrant> And then the upload is skipped, since the status isn't UPLOADING.
<jelmer> wgrant: Argh, I see.
<wgrant> Once I"ve fixed that, process-upload then crashes.
<wgrant> zope.security.interfaces.ForbiddenAttribute: ('date_finished', <lp.code.model.sourcepackagerecipebuild.SourcePackageRecipeBuild object at 0xa2d720c>)
<wgrant> That occurs in the path that's executed if the build isn't FULLYBUILT at the end.
<wgrant> Which means that it is indeed finding the build properly.
<wgrant> So, 3 issues:
<jelmer> wgrant: I think I know what broke it :-/ I Made some changes friday after tests came back and moved the destroying of the buildqueue record from buildmaster to uploadprocessor.
<wgrant>  1) b-m needs to dequeue the build somehow, to prevent forgotten build handling from kicking in.
<wgrant>  2) processBuildUpload needs to set options.buildid, or something to that effect.
<wgrant>  3) processBuildUpload's date_finished setting falls afoul of security for SPRBs.
<wgrant> The first two are fatal.
<wgrant> The third is probably not absolutely critical.
<wgrant> Um, s/indeed finding/indeed not finding/ a few lines ago.
<gmb> noodles775, Thanks (Sorry, completely missed your ping)
<wgrant> Once I fix date_finished, it fails with a DB perm error when trying to read sourcepackagerecipe to send a failure notification.
<wgrant> And then it can't delete the SPRBJ.
<wgrant> Also... what about the upload log?
<wgrant> That doesn't exist any more.
<gmb> jelmer, What's the situation with your https://bugs.edge.launchpad.net/soyuz/+bug/506256 branch? I know I rc'd it yesterday, but that m-p has disappeared. Does that mean that you're not oging to try and land it before release?
<_mup_> Bug #506256: Remove Popen from buildstatus_OK <buildfarm> <Soyuz:Triaged by jelmer> <https://launchpad.net/bugs/506256>
<jelmer> gmb: It's gone through ec2 successfully last night but wgrant has pointed out some issues with it wrt sourcepackagerecipebuilds that I'm currently investigating
<wgrant> (not just SPRBs, but OK)
<gmb> jelmer, Okay. Realistically, then, is it going to make the PQM deadline (~13:00 UTC)?
<jelmer> gmb: I doubt it will. :-/
<gmb> jelmer, Hmm. How critical is it that it gets rolled out tomorrow morning? Can it wait to be CP'd or re-rolled?
<jelmer> gmb: The branch itself or the issues wgrant has found? The issues are definitely blockers for landing it; it should be possible to CP it.
<gmb> jelmer, So, just to be absolutely clear: The branch itself is not a rollout blocker per se and can be CP'd instead, yes?
<gmb> (Or rather the bug it fixes is not a blocker)
<jelmer> gmb: Yes, the bug it fixes is not a blocker though it should land sooner rather than later because it should significantly speed up the builddmanager.
<gmb> jelmer, Okay, thanks.
<gmb> jelmer, We'll hopefully open PQM up once we've determined which revision to roll out, so you should be able to land this later and then request a CP for it.
<gmb> jelmer, I'll let you know.
<jelmer> gmb: Thanks.
<wgrant> No re-roll freeze this time? Awesome.
<gmb> wgrant, Hopefully. We'll see.
<wgrant> Heh.
<bigjools> wgrant: spot the oddness: http://pastebin.ubuntu.com/490282/
<wgrant> bigjools: superseded by nothing?
<mwhudson> superseded by itself?
<bigjools> it's gcalctool
<bigjools> yes to both of those
<bigjools> and it build in maverick, yet appears in lucid
<bigjools> built*
<wgrant> superseded by itself is reasonable.
<wgrant> Happens on overrides.
<wgrant> or double-copies.
<bigjools> we have terrible auditing of these actions :(
<wgrant> we do.
<wgrant> But it's normally possible to work out exactly what happened.
<wgrant> What's the problem here?
<wgrant> It just looks like someone copied it from Maverick primary to a Lucid PPA, twice.
<bigjools> I am trying to delete the sparc/ia64 BPRs built in maverick, but they're in lucid too
<wgrant> a ha ha.
<bigjools> ah balls, I didn't spot the archive difference
<bigjools> I hate this data model sometimes
<wgrant> Howso?
<wgrant> Apart from making it hard to do evil things like deleting DASes.
<gmb> noodles775, Since https://bugs.edge.launchpad.net/launchpad-foundations/+bug/217644 is on edge and nothing appears to be broken, can it be marked qa-ok?
<_mup_> Bug #217644: ResultSet aggregates do not respect distinct option <qa-needstesting> <tech-debt> <Launchpad Foundations:Fix Committed by michael.nelson> <Storm:Fix Released by jamesh> <https://launchpad.net/bugs/217644>
<gmb> jtv, Can https://bugs.edge.launchpad.net/launchpad-foundations/+bug/629442 be marked qa-ok since it's a test-suite only thing?
<_mup_> Bug #629442: FakeLibrarian: create returns alias.id instead of alias <qa-needstesting> <Launchpad Foundations:Fix Committed by jtv> <https://launchpad.net/bugs/629442>
<jtv> gmb: didn't we go over this yesterday?
<gmb> jtv, Er. Don't think so.
<bigjools> wgrant: it's not been designed as much as evolved and it makes it too hard to do anything
<gmb> Maybe we did and I've forgotten.
<jtv> Ah, that was deryck perhaps.
<jtv> gmb: anyway, yes, it's either qa-ok or qa-untestable depending on your pov
<bigjools> wgrant: my main beef is the strife when writing apparently simple queries to get info
<bigjools> and the lack of auditing
<jtv> gmb: if the tests pass, that is the only QA that applies.
<gmb> jtv, Call it -ok then. The test suite didn't blow up.
<jtv> OK
<gmb> I call that a win
<jtv> :)
<jtv> done
<noodles775> gmb: Sure, I think its safe to mark qa-ok, but we could QA it on staging if/when its ready if necessary.
<noodles775> gmb: sorry, wrong bug.
<noodles775> gmb: yes, that's fine.
<gmb> noodles775, Okay. I'll change it. Thanks.
<jtv> noodles775, while I have you: I finally followed up on your review.  With apologies for the delay.
<noodles775> jtv: yes, I saw... I was hoping I could leave it for tomorrow when I'm OCR if it's not urgent... is that OK with you?
<jtv> noodles775: no worries
<gmb> adeuring, I'm looking at https://bugs.edge.launchpad.net/malone/+bug/618849 and whilst it's on edge, ISTR you talking with seb yesterday about timeouts when accessing attachments. Is that anything to do with this bug, do you think?
<_mup_> Bug #618849: Timeout accessing bugs' attachments using the API <qa-needstesting> <timeout> <Launchpad Bugs:Fix Committed by lifeless> <https://launchpad.net/bugs/618849>
<noodles775> jtv: great, thanks.
<adeuring> gmb: no, seb's problem is about uploading attachments, while this bug is about accessing existing attachment
<gmb> adeuring, Okay, thanks. Do you think you might be able to take the time to QA that bug against edge (since staging's down)?
<gmb> If you don't have time, I'll find someone else for it, it's just that you're here :)
<adeuring> gmb: no i can do that
<gmb> adeuring, Excellent, thanks.
<wgrant> bigjools: don't you have queries from the hppa/lpia removals?
<bigjools> only the ones where we set the status to deleted
<wgrant> Huh.
<bigjools> but later we had to delete with extreme prejudice because of arch-any uploads
<wgrant> arch-all, you mean?
<wgrant> arch-any should be dealt with by the chroot removal.
<bigjools> maybe, I aways get them confused :/
<wgrant> Heh, everyone does.
<wgrant> So you Delete them, wait for p-dr to kick them out of the archive, DELETE them, then remove the dists tree etc.?
<bigjools> basically yes
 * bigjools books another flight to Dallas and sighs heavily
<adeuring> gmb: the fix for the bug seems to be good
<gmb> adeuring, Awesome, thanks
<gmb> bigjools, Another flight? How many times are you going?
<deryck> Morning, all.
<bigjools> gmb: once was enough, and now we have to go again
<gmb> bigjools, Ah, as in you're booking for t'Epic.
<bigjools> gmb: yarp
<bigjools> we need to stop calling it that
<gmb> Yes.
<bigjools>  the original 2 week one was *the* epic
<gmb> Indeed.
 * gmb wanders off to get some vittles
<gmb> \0/ We can has staging!
<gmb> danilos, henninge, I see 2 items still qa-needstesting on Translations. Do they both need LOSA attention? https://bugs.edge.launchpad.net/rosetta/+bugs?field.tag=qa-needstesting
<danilos> gmb, one is done, one needs both well working staging and perhaps a bit of LOSA help
<danilos> henninge, can you qa-ok 597539 please? :)
<henninge> ah, there is a working staging now ...
<gmb> danilos, henninge: But no working LOSA for another 30 minutes :)
<gmb> But thanks anyway; please let me know as soon as everything's qa'd
<gmb> LMAO
 * gmb just noticed the Rosetta 10.09 release name.
<henninge> ;-)
<henninge> It's only fitting seeing Launchpad's release name ...
<gmb> \0/ 30 minutes out from the PQM deadline, no branches in the queue and the builds are GREEN. Win.
<gmb> noodles775, What will it take to QA your branch now that it's landed? Push the new code to staging, bounce the appserver and then have a LOSA help you poke it?
<salgado> bac, is there a process to apply as a UI reviewer for LP?
<bac> salgado: rockstar is the defacto UI reviewer go-to guy.  so i'd just ask him and get a mentor
<salgado> bac, cool, thanks
<salgado> rockstar, so, how do I get a mentor? :)
<henninge> danilos: Here is the new code in action: https://translations.edge.launchpad.net/deluge/1.2/+pots/deluge/de/7/+translate
<henninge> danilos: compare that with the production version. The external suggestion comes from a different po file.
<danilos> henninge, right, I see that, but I am not sure what are you trying to tell me? :)
<henninge> ;-)
<henninge> Just to see that the new code is working (and not breaking) ....
<henninge> danilos: nm, continue what you were doing
<danilos> henninge, heh, I already trusted you on that, but thanks :)
<henninge> I just thought it was neat to see it at work.
<henninge> danilos: just saw this
<henninge> https://translations.launchpad.net/bitlord/trunk/+pots/bitlord/de/37/+translate
<henninge> not related to my branch as it is on production
<henninge> danilos: two "Used in" statements for the same template ...
<henninge> argh, nm
<henninge> different series
<henninge> danilos: but on edge ... :(
<henninge> danilos: two "Used in" statements for the same pofile...
<danilos> henninge, right, that could be due to is_current flags set and divergences or is_imported messages (they are all listed as just "used in")
<danilos> henninge, suggestions code has never been nicely updated for message sharing, so it's just a bunch of hacks
<henninge> that's reassuring
<henninge> danilos: ah yes, it is diverged:
<henninge> https://translations.edge.launchpad.net/deluge/1.1/+pots/deluge/de/+translate?batch=10&show=all&search=Invalid+label
<danilos> henninge, yeah, it's not reassuring, but that's why we really need to fix up all the crap that +translate page is behind the scenes :)
<henninge> danilos: also, we still have a sequence number problem ... the magnifiying glass takes you to messge 21 ...
<henninge> I thought that was fixed a year ago or so
<danilos> henninge, yes, we do, we've never fixed it (oh, I think I mentioned how I planned to fix that with the +translate page rework)
<henninge> :-)
<danilos> henninge, no, we couldn't fix it properly because of the navigation
<noodles775> gmb: yes, if the new code hits staging we can QA it ourselves.
<gmb> noodles775, Okay, I'll try and get it pushed out ASAP. Waiting for a losa now.
<mars> bac, I won't be attending the reviewer's meeting today.
<bac> mars: ok, we'll miss you
<bac> abentley, adeuring, bac, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jtv, bigjools, leonard, mars, noodles775
<bac> : reviewer meeting starts now
<gmb> noodles775, Staging is up-to-date and restarting now.
<noodles775> Thanks gmb
<noodles775> gmb: staging is still showing r9755?
<noodles775> gmb: ah, but from Chex's comment it is really 9760?
<Chex> noodles775: yes. I pulled down 9760, a cowboy, if you will, and restarted
<noodles775> Thanks Chex
<Chex> noodles775: sure
<gmb> We should add "But YMMV" after the revno on staging.
<noodles775> :)
<gmb> benji, pign
<gmb> *ping, even
<benji> gmb: 'sup?
<gmb> benji, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/597324 is assigned to you and is tagged qa-needstesting. Can you take care of QA'ing it please?
<_mup_> Bug #597324: Login fails when logging in while accessing URL with query parameters  <openid> <qa-needstesting> <Launchpad Foundations:Fix Committed by benji> <https://launchpad.net/bugs/597324>
<gmb> gary_poster, What's the situation with QA for https://bugs.edge.launchpad.net/launchpad-foundations/+bug/308198?
<_mup_> Bug #308198: Javascript library doesn't wrap entries within collections <qa-needstesting> <Launchpad Foundations:Fix Committed by foxxtrot> <https://launchpad.net/bugs/308198>
<gmb> _mup_, shush.
<benji> gmb: I'm working on it a the moment; unfortunately I'm working on it because it's broken (I suppose I should set it to qa-bad, doing that now)
<gmb> benji, Is that a blocker for the rollout, or is it something we can easily fix with a CP?
<benji> I don't see why it would block a rollout; nothing is any worse off than production is currently.
<gmb> benji, Okay, thanks.
<benji> np
<gary_poster> thank you benji, gmb
<gmb> benji, So yes, for now just mark it qa-bad and we'll proceed as-is.
<gmb> gary_poster, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/308198 still needs QA; I don't think benji was referring to that one.
<_mup_> Bug #308198: Javascript library doesn't wrap entries within collections <qa-needstesting> <Launchpad Foundations:Fix Committed by foxxtrot> <https://launchpad.net/bugs/308198>
<gary_poster> oh
<gary_poster> sorry for misreading
<gary_poster> mars, leonardr, do you all have any idea how to QA the community fix for bug 308198?  (I ask Leonard because he filed it, and mars because he Knows About These Things.)
<_mup_> Bug #308198: Javascript library doesn't wrap entries within collections <qa-needstesting> <Launchpad Foundations:Fix Committed by foxxtrot> <https://launchpad.net/bugs/308198>
<gmb> Sorry if you've answered this already jelmer, E_TOO_MANY_CHANNELS, but what's the situation with QA on https://bugs.edge.launchpad.net/launchpad-code/+bug/599397?
<_mup_> Bug #599397: code imports break when tdb is installed <code-import> <qa-needstesting> <Bazaar Hg Plugin:Triaged> <Launchpad Bazaar Integration:Fix Committed by jelmer> <https://launchpad.net/bugs/599397>
<leonardr> gary, i can do it, but i think mars would do it faster
<jelmer> gmb: Sorry, I haven't gotten to that yet - staging was down last evening; I can have look this evening instead.
<gary_poster> ack leonardr.  I'll revisit in a few to see what mars' reply is
<gmb> jelmer, The sooner the better; I'd like to be able to say for sure that we're good to go with the current db-stable revision before I EOD if possible.
<gmb> (Though the final decision is at 21:00 UTC)
<gmb> jelmer, If you don't have time to do it nowabouts it's no problem; I'll ask someone else to take care of it.
<jelmer> gmb: that's ok, it shouldn't take long
<gmb> jelmer, Cool, thanks.
<jelmer> I'm just trying to keep day job and community work separate a bit
 * jelmer puts on his community hat
<gmb> jelmer, Thanks. Your willingness to put to one side your millinery separatism is appreciated.
<jelmer> hah, there's my dictionary word of the day
<bigjools> ha
<bigjools> we also appreciate your perspicacity
<gmb> Technically it's a noun, not an adjective, but let's adjectify it for the purposes of this discussion.
<gmb> jelmer, Thanks for QAing that for me.
<gmb> gary_poster, leonardr: I think mars is unavailable; can one of you guys take care of that last outstanding needstesting item please?
<gary_poster> leonardr: please do it then, sorry.  EIther that or you can try to guide me in doing it.
<leonardr> gary, sure, np
<gary_poster> thank you.  gmb ^^^
<gmb> gary_poster, leonardr: Thanks.
<gmb> leonardr, Please ping me when you're done.
<leonardr> gmb, verified
<leonardr> gary, if you're curious, here's what i did, from firebug. (it took me a while to remember the javascript stuff)
<leonardr> client = new LP.client.Launchpad()
<gmb> leonardr, Fab, thanks
<gary_poster> thanks, leonardr, looking
<leonardr> var people = null;
<leonardr> set_people = function(arg) { people=arg; }
<leonardr> client.get("/people", {on: {success: set_people}})
<leonardr> then i inspected people.entries[0] and made sure it was a 'entry' type object with lp_save etc.
<gary_poster> gotcha, leonardr.  I'm inclined to put it up on the wiki, for myself and other LP devs in the future.  Is there something like that already to your knowledge?
<leonardr> gary: no, something like 'ajax tips'?
<gary_poster> yeah
<gary_poster> 'cause that is kind of like being able to mess around with launchpadlib, albeit in a kind of cramped style
<leonardr> maybe it could go on some existing ajax page? i'm starting to realize that wikis are more useful to me if they have fewer, larger pages
<gary_poster> I don't know where one is.  I'm putting it on the existing Foundations/Webservice page for now.
<gary_poster> feel free to adjust
<leonardr> ok, that's good too
<lifeless> gary_poster: hi
<lifeless> gary_poster: on templates & template engines
<gary_poster> yes, lifeless
<lifeless> gary_poster: I've heard (unmeasured) claims that our current engine is really poor :) - chameleon is better, but, is there a -really good- engine out there?
<lifeless> gary_poster: e.g. one where the template processing won't show on our profiles at all
<lifeless> as a for instance, rather than compiling to python, one could compile to CPython (hey, if we're going to compile, may as well do it all the way).
<lifeless> gary_poster: if there isn't, this might be a challenge to offer the zope community :)
<lifeless> losa ping - status on staging please
<gary_poster> lifeless, summary: chameleon is the current best there is, and it is good.  Details follow.
<gary_poster> I did measurements last year.  I no longer have them, but they were based on ab-style testing of representative pages, rather than actual semi-production usage (e.g., playing back requests). it gave 10 to 20% better overall improvement on the server in my tests (I summarize as "15%" usually).
<gary_poster> It uses current "state of the art" for this sort of thing (it compiles to Python byte code, FBOW), and the project claims that it "performs 10-15 times better than the reference implementation" (http://pypi.python.org/pypi/Chameleon).  I think the Zope community would say that a request for a fast template engine is answered.
<gary_poster> In my attempt to wrassle with the code, I'd say compilation is not optimized at all, but that's at build time; and rendering seems very nicely optimized.
<gary_poster> I spoke briefly to the author of Chameleon about a month ago and he was interested in another version of the engine, but I think that's mad scientist talk ATM.
<lifeless> ok
<lifeless> for clarity, here's my thoughts on templating:
<lifeless>  - tal is ok, but not madly tied to it; if tals definition makes execution slow...
<gary_poster> (ab-style: apache bench, not marketing/usability-ab.)
<lifeless>  - engine agnostic : transparent/fast/decent memory footprint/fixable when we need to  - these matter to me
<gary_poster> tal: it doesn't, especially with Chameleon.  Moreover, if we are interested in mucking about with our template engine, that's what Chameleon is built for.
<lifeless>  - sample use case : render a 7000 row table with 15 columns in subsecond time.
<gary_poster> (so, for instance, the default Chameleon template interpretation defaults to python syntax, rather than path syntax)
<lifeless>    [on our appserver hardware, when 4 threads are doing template rendering at the same time]
<lifeless>    [-> 250ms template time to do that]
<gary_poster> fast: the claims are that this is close to state of the art, as I said
<lifeless> sure; I'm not dissing chameleon
<gary_poster> nor did I think you were
<lifeless> :)
<gary_poster> just trying to clarify my position on your bullet points as you go
<gary_poster> maybe unnecessary
<lifeless> my points are done, I think.
<gary_poster> transparent/decent memory footprint: we'll see.  We are supposed to be able to switch back and forth between reference implementation and chameleon implementation very easily--an env var for make.
<lifeless> oh, and once we have data, if this is a speed driver, we should, I think, invest in driving an implementation to where we need
<gary_poster> fixable: I'm very interested in that one.  We'll see there too. sidnei has significant experience with this package
<gary_poster> byte code generation starts from a bad place for that :-)
<gary_poster> but there appear to be debugging hooks.  I don't understand them yet
<gary_poster> the code generating the byte code is pretty clean, AFAICT so far
<gary_poster> sample use case: dunno.  we can come up with an artificial test to try to get data.
<lifeless> there are benchmarks already
<lifeless> of course
<lifeless> gary_poster: oh, and did I file the bug asking for a repeat construct with a 'did not iterate' code path?
<gary_poster> benchmarks: sure.  I suspect if we asked the author about it, or maybe sidnei, he would be able to give us stats.  They are probably on the web somewhere in a blog in fact, but a quick google search did not show.
<lifeless> http://blog.penzilla.net/2010/02/python-template-language-performance.html
<lifeless> didn't say what the reference test looked like :P
<gary_poster> heh
<gary_poster> bug for repeat construct: not that I know of, but that's the kind of thing we should be able to do with Chameleon.
<lifeless> I'll file a bug on foundations now
<gary_poster> we should be able to change tal:content="cache: foo bar" to tal:cache="foo bar" also
<lifeless> this will remove some COUNT()s that we don't need
<gary_poster> ok
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/633429
<_mup_> Bug #633429: repeat/not-iterated construct <Launchpad Foundations:New> <https://launchpad.net/bugs/633429>
<gary_poster> ok thanks
<lifeless> this will make a huge difference to some pages
<lifeless> as well as lowering our DB usage more
<gary_poster> ok cool
<rockstar> salgado, did you end up getting a UI mentor?
<salgado> rockstar, yes, sinzui
<rockstar> salgado, hurrah!
<salgado> rockstar, now I guess I just have to wait for people to ask me for UI reviews?
<rockstar> salgado, yes.  henninge hasn't been too busy, but he's had some UI reviews recently.
<salgado> rockstar, cool.  are there any guidelines I should read before I start?  I've seen UI/Reviews already
<rockstar> salgado, I don't think so.  I feel like UI reviews are less about "review" and more about enforcing people to stop and think about UI, and have a discussion.
<cr3> leonardr: in lazr.restful, is there a list of everything that gets exported which is looked up with looking for /api/1.0/foo for example?
<leonardr> cr3: yes, everything exported is described in the wadl file. it's a kind of site map
<leonardr> for launchpad, you can also see it in human-readable form at /+apidoc
<cr3> leonardr: is there a way to hook into lazr.restful to extend the wadl file dynamically?
<leonardr> cr3: no, it onyl contains what lazr.restful knows about
<cr3> leonardr: for example, when I call an unknown method, I'd like to the webapp to do something depending on the method
<cr3> leonardr: I think I see what you mean, the decision is being made client side whether a method exists or not, right?
<cr3> that would certainly explain why I'm not seeing a log entry when calling an unknown method on the server side
<leonardr> cr3: right
<leonardr> unknown methods don't go through--they're not part of the published interface
<leonardr> if there are specific methods handled by something other than lazr.restful, i could see adding a hook into the wadl for them, but that's al ittle odd
<cr3> leonardr: gotcha, thanks!
<cr3> leonardr: I'll try to live with it for now :)
<leonardr> ok
<lifeless> morning y'all
<jelmer> hellow
<tyarusso> What on earth...rocketfuel-setup just installed firefox.
<tyarusso> Them's some strange dependencies ya got there.
<lifeless> used by the test suite
<beuno> windmill, etc
<tyarusso> lifeless: ah.
<lifeless> rocketfuel-setup sets up development environments
<dobey> +recipes has quotas now?
<jelmer> dobey: Do you mean the max 5 per distroseries per person per day one?
<dobey> i guess so, yes.
<dobey> You have exceeded your quota for recipe ubuntuone-hackers/client-dailies for distroseries ubuntu maverick
<wallyworld_> morning
<jelmer> dobey: I think that's always been there, you probably just hadn't hit it before :-)
<jelmer> 'morning wallyworld_
<dobey> jelmer: it's not documented on the +recipe page or the help page?
<jelmer> dobey: You're right, it probably should be.
<wgrant> What is the quota's purpose?
<wgrant> Nothing else on Launchpad has a quota like that.
<dobey> ppas have quotas
<dobey> for space anyway
<wgrant> Yes, but they're not arbitrary.
<dobey> well it certainly makes it harder to debug software that uses the feature :)
<dobey> anyway, time for me to go
<wallyworld_> rockstar: ok, i can get you one in 15 mins
#launchpad-dev 2010-09-09
<lifeless> wgrant: the PPA quotas are precisely as arbitrary as the recipe volume quota
<lifeless> the PPA quota is less of a surrogate
<wgrant> lifeless: Well, arbitrary is perhaps not the best word.
<wgrant> But the PPA quota does what it's designed to do.
<wgrant> Places a hard, known limit.
<wgrant> The recipe one does not.
<lifeless> I'm having some trouble following your detailed description of whats wrong; perhaps you could give me a summary instead?
<wgrant> The PPA disk quota prevents users from using more disk than they should.
<wgrant> The recipe build quota is presumably to stop users from using more buildfarm time than they should.
<wgrant> But it limits how many times they can use it.
<wgrant> Not how much of it they use.
<lifeless> right
<lifeless> as I said, its a surrogate.
<lifeless> I'd welcome a series of bugs that would allow measuring and allocating the actual resource.
<poolie> hi there
<poolie> jml wrote some code to suck lp bugs into a desktopcouch db
<poolie> i was thinking in the shower this could possibly even be offered as a standard facility
<poolie> which would be pretty cool: cheap async (at least readonly) access
<poolie> this is very blue sky of course
<lifeless> maybe
<lifeless> I'd want couch to have an OOM less sysadmin time first :)
<poolie> :)
<lifeless> (and sadly, I'm not joking)
 * poolie reinterprets that as "order of magnitude" not "out of memory"
<poolie> yeah
<poolie> perhaps i'll look at his client code someday
<poolie> lifeless: your shoes are now available to be filled <http://webapps.ubuntu.com/employment/canonical_BSE/> - help me find someone good?
<lifeless> poolie: certainly.
<lifeless> Have you tweeted yet ?
<poolie> i can do that
<lifeless> http://www.google.com/instant/#utm_campaign=launch&utm_medium=van&utm_source=instant
<spm> why the GA tracking codes? (utm_* are google analytics tracking codes. fwiw)
<lifeless> spm: it was in my browser bar
<lifeless> copy-paste, you think I read these things?
<spm> fair enough :-)
<lifeless> does LoginToken:+validategpg talk to the gpg servers?
 * lifeless is guessing it does
<lifeless> I wonder, should oauthnonces go in the sessiondb
<cr3> hi folks, I created a couple projects recently without anything in them yet. would it be simpler to ask to rename them, or create another couple projects and ask to remove them?
<james_w`> poolie: lp in desktopcouch> standard facility where?
<lifeless> cr3: rename should be easy enough
<lifeless> cr3: unless you have mailing lists or ppas
<poolie> james_w`: well, if i ever get around to it, i would look about writing an apis-couch daemon
<poolie> eventually, and this is utter pie-in-the-sky, it would be cool to just have something like bugs.launchpad.net/bzr/+couch
<poolie> and to through that means get a whole copy of them
<james_w`> apis-couch? what would that do?
<lifeless> james_w`: map lp into couch?
<cr3> lifeless: nothing yet, so can I ask in the channel or is there a preferable avenue?
<lifeless> CHR should be able to help in #launchpad
<lifeless> failing that follow the instructions in the topic there.
<lifeless> :)
<james_w`> lifeless: well, I have a project already that doesn't need new daemons etc.
<cr3> "CHR"? is that a nick or an acronym I'm not familiar with?
<james_w`> I'm interested in whether poolie has other ideas
<cr3> lifeless: by the way, might you have a moment to chat about the progress of the results tracker?
<poolie> james_w`: i have very few ideas ;) other than that having it in a local db that can sync smartly with the server would be nice
<poolie> what's your project?
<james_w`> txrestfulclient
<james_w`> poolie: it's almost transparent to the app whether it is talking to LP or a couchdb copy of the data. The non-transparent part is that if talking to couch it has extra capabilities to push stuff back to lp
<poolie> really, wow
<poolie> james_w`: i'll check it out
<james_w`> I'd love some help getting it up to standard
<james_w`> it's not got full coverage of the capabilities of the API yet
<james_w`> rolling it in to my attempt to write a twisted API library wasn't the best thing
<lifeless> james_w`: I'd love to see that split out
<lifeless> james_w`: there is a use case in LP for a txlaunchpadlib
<lifeless> james_w`: but we wouldn't want the couch stuff mixed in
<james_w`> lifeless: the code is independent at that level
<james_w`> lifeless: but the couch feature relies on the tx code as I have written it, so people can't play with that until they use twisted
<james_w`> (and couch is only one possible implementation of the required interface, it just so happens that a JSON-based document store is kind of handy for this)
<cr3> lifeless: sleep time, I'll catch up with you another time about the results tracker. cheerio
<lifeless> cr3: ciao
<poolie> the latency is so high doing it on twisted would be good
<poolie> james_w`: so your code can use couch as a cache?
<james_w`> poolie: "cache", yes
<james_w`> poolie: you talk to LP and it sticks the documents it gets back in to couch before returning them to you
<james_w`> poolie: at any time you like you can reconfigure the client to talk directly to couch, and you will get those documents back again.
<james_w`> that's the read-only part
<james_w`> then you can make changes, and it will store the modifications, and give you the updated information if asked for it again
<poolie> sweet
<james_w`> then you can ask it to iterate the modifications and send them back to LP, and the collision detection will naturally act to prevent problems there
<poolie> so this just all works over the existing restful protocol?
<james_w`> there are still a bunch of things that need work, and I'm not sure whether the approach taken will ever get us to 100%, but it does have an elegance
<james_w`> poolie: yep
<nigelb> isn't this what someone demo'd at last UDS?
<james_w`> poolie: with a way to replace that restful protocol with queries in to couch
<james_w`> nigelb: yeah, me, very shoddily
<nigelb> james_w`: lol, laptop not working et al ;)
<james_w`> exactly
 * nigelb hugs james_w` :)
<mwhudson> james_w`: now do this for notmuch pls
<james_w`> mwhudson: one day
<james_w`> though I think I should try and add more moving parts next time
<jtv> wgrant: something I don't getâ¦ I'm to keep a bfj fk in my new TranslationTemplatesBuild tableâbut where does it come from?  I see build_farm_job being set all over the place, but that all refers to BuildFarmJobOld stuff.
 * poolie tries txrestfulclient
<poolie> and whacks in to bug 461356
<_mup_> Bug #461356: desktopcouch-service crashed with ImportError in <module>() <apport-crash> <i386> <ubuntu-unr> <desktopcouch (Ubuntu):Incomplete by cmiller> <https://launchpad.net/bugs/461356>
<lifeless> spm: suppose we could zap the first 8 months of successful-updates.txt?
<spm> sure. one sec
<lifeless> spm: of this year!
<lifeless> [I think it goes back forever, so keep a copy
<lifeless> but its getting a tad large]
<lifeless> spm: also, whats it up to?
<spm> successful-updates-2008.txt (already existed) also now have successful-updates-2009.txt
<lifeless> heh
<lifeless> gmm
<lifeless> is there a system load average python module already, I wonder
<spm> import system.load.averages ?
<spm> which is spm doing the equivalent of import icanfly
<mwhudson> lifeless: os.loadavg()
<mwhudson> er no
<lifeless> getloadavg
<mwhudson> right
<lifeless> bug https://bugs.edge.launchpad.net/oops-tools/+bug/243554, freshly updated
<_mup_> Bug #243554: oops report should record information about the running environment <infrastructure> <oops-tools> <Launchpad Foundations:Triaged> <OOPS Tools:Triaged> <https://launchpad.net/bugs/243554>
<lifeless> I wonder if time.clock() is pid wide or pid wide :P
<mwhudson> lifeless: one of those was supposed to be thread?
<lifeless> mwhudson: being droll about linux clarity in this area
<mwhudson> heh
<lifeless> we can use clock_gettime(CLOCK_THREAD_CPUTIME_ID) though
<jtv> lifeless: would you mind if I just made that Librarian change now?
<lifeless> jtv: I don't mind when you do it :)
<stub> lifeless: I've just pulled in the information from /proc before
<jtv> lifeless: :)
<lifeless> jtv: if you mean ...'and sneak it in the release', that would be risky, wouldn't it?
<jtv> lifeless: that would be, and it's not what I had in mind.  Thinking more of avoiding being the subject of future "what flaming idiot made this horrible change!?" inquiries
<lifeless> jtv: I'd hope noone in the team would take that attitude following something up
<lifeless> jtv: and the only risk I know of, is the one I mentioned: the librarian db access is currently very tightly encapsulated; it needs to stay that way.
<jtv> Well, so to speak.  The thing is, I'm not 100% happy about making an API where you pass "either an object or its id."
<lifeless> jtv: so don't do that.
<lifeless> jtv: make a separate pass-the-object API (perhaps on the object :P)
<jtv> Ahh
<lifeless> and have the current id based function delegate
<jtv> Now, what is the reason for the tight encapsulation?
<lifeless> because its in twisted
<lifeless> so its called via deferToThread
 * jtv likes reasonsâeasier to remember than rules :)
<lifeless> it can do DB access in the thread
<lifeless> it cannot outside of it, or all other requests in progress will block.
<jtv> I thought it ran as a separate process?
<lifeless> jtv: if you aren't touching code used in the librarian /server/ this won't matter - but I don't know exactly what you're touching (and be sure to check for imports :))
<lifeless> jtv: the librarian is a twistd process, yes.
<jtv> lifeless: I'm touching stuff in canonical.launchpad.librarian and canonical.librarian.client, but nothing in server.
<lifeless> in the process it has a mainloop, and worker threads; the worker threads do DB access, the mainloop does all the business logic (except DB access)
<lifeless> jtv: the server is in canonical.launchpad.librarian
<jtv> So client.py is sort of exceptional in there?
<jtv> I mean, FileDownloadClient does run client-side, right?
<lifeless> it might be nice to have the twisted code more visually distinct (e.g. in a submodule, or move the client to lp.services.librarian.client, or something.
<lifeless> I don't want to make the scope bigger on you :)
<jtv> Yes.
<jtv> I'm not going to do anything along those lines, no.  :)
<lifeless> so, to answer your question, I presume so, but I'd need to check.
<lifeless> All I'd advising is a little caution and investigation in this area, as we have two dramatically different programming models in play here, and they mix poorly.
<lifeless> speaking of which.
<jtv> ?
<lifeless> spm: are were there yet ? its been an hour.
<lifeless> jtv: the speaking of which was a joke.
<jtv> :-|
<lifeless> jtv: the line before wasn't.
<jtv> That much was clear.  The joke I still don't get.
<lifeless> oh, it wasn't a very good one
<lifeless> it was leading into the spm: line
<jtv> ah
<jtv> lifeless: still, better to have the other shoe dropped :)
<lifeless> taking a breather; I'll be back to heckle spm later
<jtv> lifeless: I think there's a better solution for the librarian problem: it's a bad internal distribution of responsibilities.  In _getPathForAlias, the LFA is loaded _only_ to determine that it's visible.  The actual work doesn't involve the object at all.
<jtv> nm; take your breather
<EdwinGrubbs> poolie: ping
<poolie> james_w`: a teeny patch for you
<poolie> EdwinGrubbs: hi there
<EdwinGrubbs> poolie: I have some questions about the preferred way to use the apport format. The oops currently groups the request variables together, but it seems cleaner to use email.message.Message than to use another ProblemReport to make a hierarchy, so that I don't end up with multiple Date and ProblemType fields.
<EdwinGrubbs> poolie: I also wondered if I should use the Stacktrace field for python stacktraces, or if it would be better to only use that for stacktraces created by gdb.
<poolie> EdwinGrubbs: you can look at bzrlib.crash to see what we do
<poolie> we use Traceback for the python traceback
<poolie> which i think is consistent with what other python programs use
<poolie> i would probably have one thing RequestVariables containing all the variables
<poolie> either just as they are in the url, or decoded
<EdwinGrubbs> poolie: oh, request variables in the oops actually is all the cgi variables like HTTP_REFERER, REQUEST_METHOD, etc. and not just the query string. I saw in the apport file format pdf that some of the hierarchical variables are stored as "name=value", but it seems more consistent to me to use "name: value". I'm trying to decide whether to use email.message.Message.as_string(), or ProblemReport().write(StringIO()) to create a
<EdwinGrubbs>  hierarchy.
<poolie> so you agreed with gary to do it in apport now anyhow?
<poolie> i'd probably just pprint a python dict
<spm> lifeless: looks like it came back about 5 mins ago
<EdwinGrubbs> poolie: well, I spent today determining how easy it would be to do now. I just saw Gary's email that he preferred to do it in the long term. However, my original solution, was almost identical to ProblemReport.write_mime() except that I don't base64 encode things and that it handled the special case of the request variables hierarchy. I really think we should use apport now. oops-tools will still be able to process the old oops
<EdwinGrubbs>  format, so the differences in the apport format don't cause any implementation problems.
<poolie> EdwinGrubbs: so they're not really a ahierarchy, are they?
<poolie> i mean it's just a dict of strings
<EdwinGrubbs> poolie: right, I just meant that the whole problem report is a hierarchy, since the request variables contains multiple values.
<poolie> the simplest thing that could work is to pprint an array or put them in urlencoded
<poolie> istm that using email formatting would be complicated, might break, and wouldn't help
<poolie> ditto nested appotr
<jtv> wgrant: seems I needed to create the BuildFarmJob from the factory for my custom job type.  Passing tests again.
<lifeless> spm: ok cool
<lifeless> spm: so, can we enable profiling, and make the load be good ?
<SpamapS> lifeless: hey, what ever happened with SSL improvements? Was just reviewing past threads..
<lifeless> SpamapS: theres an RT ticket open to increase the cache length
<lifeless> (for idle keys)
<lifeless> and theres another open to get me access to the DC apache front end over a VPN + HTTP; I can then test a FE SSL here
<SpamapS> ah cool. I have used distcache for mod_ssl in the past to great effect before btw. ;)
<lifeless> I'm not sure if we have dual apache or not
<lifeless> I suspect not
<lifeless> jtv: uhm, doesn't the name from the the LFA too ?
<SpamapS> wow distcache's last release was in 2004 .. man its been so long since I setup an actual SSL server .. got BigIP's to do it a while back and have just been soft on SSL ever since. ;)
<jtv> lifeless: yes, as usual I saw my mistake right after I said itâbut no reason to keep you at the time.  ;)
<jtv> lifeless: can there be thread/process boundaries in this call chain that I would not see at all?
<poolie> well, the txrestfulclient hello world passes again
<poolie> that's something
<poolie> but also probably enough for now
<lifeless> jtv: deferToThread in the librarian is the call boundary
<lifeless> jtv: when it returns from the callable supplied to that function, it comes back across the thread.
<jtv> lifeless: I don't see that happening anywhere in the call chain from the first fetch of the LFA to the redundant second fetchâI guess that means that it's safe to re-use the same LFA object.
<spm> lifeless: is back in profling mode
<spm> pro-fling. hrm. maybe not quite. profiling tho....
<lifeless> heh
<lifeless> spm: and hows the load ?
<spm> dropping. 1 3 4 atm
<lifeless> spm: is it running with/without the patch ?
<spm> good question...
<lifeless> spm: and are background jobs still disabled ?
<spm> with-out
<spm> yup; just nowish.
<lifeless> ok thats cool
<lifeless> spm: have they been killed :P
<spm> oh yes. legit excuse to kill cronjobs off? opportunities like this are rare and to be enjoyed post haste!
<lifeless> ok, profiling things now
<lifeless> first off, bugtask:+index
<lifeless> and now ubuntu/++assignments
<lifeless> ok, 7 seconds rather than 16
<lifeless> spm: the patch I put up the other day
<spm> hm?
<lifeless> spm: can we put that on again?
<spm> do you have a paste handy?
<spm> said file has been overwritten since
<lifeless> one sec , finding/making
<lifeless> http://paste.ubuntu.com/489589/
<lifeless> spm: ^
<spm> ta
<spm> sorry - was horribly distracted doing the push code for the release; and made the shocking discovery that praseodymium does NOT have sl installed.
<spm> naturally this is a critical/urgent problem and moves to rectify needed to be made.
<lifeless> indeed
<lifeless> sysadmin porn comes first, of course.
<spm> hahaha
<lifeless> darn, I typed that :(
<lifeless> :P
 * SpamapS cannot disagree with that
 * SpamapS prepares a petition to have sl added to the server cd seed
<spm> there. nicely taken wildly out of context.
<lifeless> rotfl
 * spm bows at the appreciation
<spm> the fine art of context free quoting - choosing the title
<SpamapS> damn, seems somebody beat me to it. ;)
<spm> lifeless: restarting with the patch; give it a few
<lifeless> thanks
<SpamapS> spm: your title was better than mine. :)
<spm> heh
<SpamapS> well done
<spm> blink. something crash nicely on the restart
<spm> wow. something is really not right here...
<lifeless> details?
<spm> oh ffs. it's doing a staging rollout AGAIN! aARGH
<lifeless> <...>
 * spm grumps off and puts in the lock file on sourcherry.
<spm> i've killed the crontab entry as a savage "don't do that" for now. I'll see if I can manually get the app server on asuka back to right'n'goodness
<wgrant> jtv: Sorry, I'm not completely down with the latest implementation details.
<jtv> wgrant: I think I've done all I know I should doâ¦ question now is: what next?
<wgrant> jtv: You have BuildFarmJob rows now?
<jtv> Yes
<jtv> I had to create them myself, which from what I see elsewhere seems to be the way.
<wgrant> I suggest talking to noodles.
<jtv> Yeah
<wgrant> How much do you still depend on BranchJob?
<jtv> I haven't gone through the details, but I thought it depended mainly on how much the dispatch machinery still depends on TranslationTemplatesBuildJob.
<jtv> I mean, there's not much in there other than methods the build farm needs.
<wgrant> True.
<wgrant> OK, so you've probably done your bit for now, but talk to noodles.
<jtv> If the build farm can start dispatching TranslationTemplatesBuilds instead of TranslationTemplatesBuildJobs, I'm either there or very close.
<wgrant> That's the next step.
<jtv> Yes, I will definitely talk to him once he appearsâhe also promised me a review this morning.  :)
<wgrant> Ah, excellent.
<jtv> I'm sort of eager to start cleaning out the old stuff, and sort of not looking forward to it at the same time.  :-)
<lifeless> spm: and the conclufsion is?
<spm> ARGH
<spm> am working on getting that to argh <== atm
<lifeless> ij
<lifeless> ok
<lifeless> bbs
<lifeless> can you kick the profile rsync in the interim?
<spm> so kicked
<spm> try to start the patched and profiling "new" version...
<spm> trying.
<spm> it's still "starting"....
<spm> lifeless: wooo. it's started. have at it.
<lifeless> spm: load is still low ?
<lifeless> spm: and does it have both patches, or only the query changing one?
<lifeless> spm: please kick the profile rsync - thanks
<spm> kicked and very low, 0.22 0.35 0.71
<lifeless> thanks
<lifeless> spm: which patch(es) did it have?
<spm>  lib/lp/blueprints/model/specification.py and the profiling on
<lifeless> uhm, both patches change that file :P
<spm> haha
<lifeless> have  alook
<lifeless> does it change the column definitions
<spm> one sec. just trying to stop a db from faceplanting
<lifeless> or the query
<lifeless> kk
<spm> lifeless: appears to be this one at a cursory glance at the first few lines: http://paste.ubuntu.com/489589/
<lifeless> ok, could you appyly the other as well ?
<spm> heh sure, you have a paste handy?
<lifeless> yes
<lifeless> hang on while I check the backlog
<spm> ta
<spm> just doing about 17 bazzilions things at once atm.
<StevenK> spm: Like notmal
<StevenK> Er, normal
<spm> ....
<lifeless> http://pastebin.com/E7hMnL28
<lifeless> spm: ^
<spm> ta
<spm> gimme 5-10; just need to disable.notify a bunch of things in prep for the release in 45.
<lifeless> sure
<adeuring> good morning
* spm changed the topic of #launchpad-dev to: Launchpad down/read-only from 0800-1100 UTC for a code update | Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<lifeless> spm: I'll check back in regularly till you say its done... I can see you're busay
<spm> ta
 * lifeless starts coming up with ideas to make rollouts even shorter
<adeuring> lifeless: did you talk with deryrck about my theory of the cause of the still remaining problems of the apprt retracer? that the librarian is simply queueing request from the app server for too long?
<spiv> adeuring: it was discussed; IIRC it was diagnosed as missing firewall rules for some app servers
<adeuring> spiv: intersting.... do you have any details?
<spiv> adeuring: see the #is and/or #launchpad-code logs from about 7 hours ago
<adeuring> spiv: thanks!
<mrevell> Hi
<lifeless> adeuring: hi
<adeuring> hi lifeless
<lifeless> adeuring: tcp connect timeout default is 30 seconds IIRC (you need to wait for the MSS, again IIRC)
<lifeless> if the librarian was doing that with 5 concurrent requests we'd be sunk, also its careful to do lots of incremental bits of work
<lifeless> so I'd expect a-diskio * 4 peak slowness, - a second or two tops - not 30
<lifeless> adeuring: which is why I looked elsewhere
<lifeless> adeuring: now, 4 is the number of threads our appservers have
<lifeless> so 5 spilling you into a new appserver was a reasonable assumption :)
<adeuring> right
<elmo> I'm changing our firewall rules so that we REJECT rather than DROP cross-site requests to make diagnosis of this kind of thing easier, FWIW
<lifeless> elmo: thank you!
<adeuring> lifeless: just tried my scrpit with 5 concurrent requests -- looks a bit better
<lifeless> adeuring: a bit?
<adeuring> no errors
<lifeless> thats a lot better then :P
<adeuring> lifeless:i think we should do more logging of what happens on the librarian server,
<poolie> lifeless: i've been reading a book ian gave me 'prefactoring'
<lifeless> adeuring: We should strike a balance between non and too much... note that the librarian does OOPS as of this rollout (or was it last one)
<poolie> it's a bit basic but it has some nice suggestions along the lines of your guide there
<lifeless> I think QA haven't added it to the daily reportcard yet.
<lifeless> poolie: intereseting
<lifeless> poolie: can I borrow it @ UDS ?
<poolie> if you remind me several times closer to the date :)
<lifeless> poolie: is this close enough?
<lifeless> poolie: how about now?
<spm> lifeless: applied that 2nd patch as well; restarting now
<StevenK> Haha
<poolie> uh
<poolie> good night :)
<lifeless> poolie: :P
<lifeless> poolie: I'll remind you just before we go
<adeuring> lifeless: well, I think the issue is not necessarily a bug in code or anything -- just that the librarian can't handle requsts fast enough
<lifeless> adeuring: I'm not aware of issues like that
<lifeless> adeuring: or data suggesting we have them; certainly I agree that we *need to be able to diagnose such things*
<bigjools> morning all
<lifeless> and if the logs are insufficient, we should increase them till they are.
<adeuring> lifeless: right
<lifeless> adeuring: we're now logging librarian times in the appserver for downloads; we can add uploads easily
<lifeless> hmm
<lifeless> for uploads we should also add the size, perhaps in the closing bit of downloads too
<adeuring> lifeless: ok, that would help. but i suspect that these connection timeouts are caused by the librarian, not the app server
<lifeless> adeuring: which timeouts?
<lifeless> adeuring: if you mean the ones apport has been having, that show up as 500 errors from the API with timeout to mizuho in them...
<lifeless> adeuring: they were a firewall
<spiv> adeuring: the evidence I've seen suggests the librarian server is coping just fine
<spiv> adeuring: why do you think otherwise?
<adeuring> lifeless: well, my little script causes them just fine even now
<lifeless> adeuring: it does?
<adeuring> yes, if it starts 8 concurrent requests
<lifeless> can you pastebin the appserver trace?
<adeuring> 5 seems to be better
<lifeless> or does it seem to be identical?
<spiv> "just now", you mean during a rollout? ;)
<lifeless> adeuring: oh hang on. lollolllollololl
<lifeless> adeuring: launchpad is down. Readonly mode. Zer iz no upload possible because the librarian is switched off
<lifeless> or meant to be; if you're successfully uploading there is something really wrong.
<adeuring> lifeless: yeah, ok, that's another possible cause
<adeuring> but... when exactly was the rollout started?
<spiv> About 10 minutes ago.
<lifeless> 11 minutes ago
<adeuring> ok.. hard to be sure then when exactly i ran the script again....
<adeuring> ok, let's try again once the rollout is done
<lifeless> yes
<lifeless> if you can provoke a connection timeout error, its a definite bug.
<lifeless> My first reaction is to look elsewhere than the librarian
<spiv> adeuring: so, connection timeouts are really unlikely to be a problem in the librarian server in my opinion
<lifeless> for a bunch of reasons.
<lifeless> but we can't exclude it; lets just keep the net broad.
<adeuring> spiv: well, we _could_ see them
<lifeless> e.g. today we found a concrete problem with the firewall config
<lifeless> adeuring: no, you saw a firewall.
<lifeless> adeuring: we know for *some things* it was definitely *a firewall*
<adeuring> lifeless: what firewall issue was it?
<lifeless> adeuring: 2 appservers are in a different datacentre.
<spiv> adeuring: it's a Twisted server that ought that ought to always be accepting connections rapidly; if connections are not being processed by that daemon in a timely fashion the only likely cause is that the librarian's host is totally swamped for disk IO
<adeuring> ahhh, so they could not access the librarian?
<lifeless> adeuring: the firewall rules for them did not include the restricted upload port, which is what the appservers connect to to upload restricted files.
<lifeless> adeuring: the firewall rules dropped the packets as hostile, and so at the network layer it looks like the librarian /machine/ is missing.
<adeuring> lifeless: ah, ok, that looks like a real problem....
<spiv> adeuring: so a failure to connect() from another machine strongly suggests problems in something other than the librarian daemon.
<lifeless> adeuring: but there may be other problems.
<lifeless> adeuring: however, we *know* there was *a* problem, that would cause the symptoms seen before
<spiv> That doesn't make it impossible, of course, but it does mean that assuming the daemon is the most likely source of the problem is likely to be the wrong approach.
<adeuring> yes, i understand
<bigjools> lifeless: are you familiar with the distroeries +queue page?
<bigjools> distroseries, even
<lifeless> I seem to recall it showing up on slow-pages reports
<bigjools> indeed
<bigjools> it's the bane of my life
<lifeless> anyhow, I'm not intimately familiar with it
<lifeless> but lets pretend I am
<bigjools> well it's mainly intended to let archive admins move uploaded packages into the accepted state
<bigjools> if they got held for some reason
<lifeless> righto, its the aa review queue
<bigjools> this is normally done in zopeless scripts if it's auto-accepted
<bigjools> when accepting packages we also close lots of bugs and email people
<bigjools> (potentially)
<lifeless> meep
<bigjools> and this is where the trouble arises with that page if any of the objects are private
<bigjools> (not to mention the query load)
<lifeless> and email load
<lifeless> that really needs to be out of appserver anyway
<bigjools> yes
<lifeless> I'm so excited
<bigjools> anyway, I have a bug where it's OOPSing occasionally for someone when it tries to access private email addresses
<lifeless> should be able to point really clearly at email perf tomorrow
<lifeless> we'll have failed convertToQuestions, I'm sure.
<bigjools> I am wondering if it's acceptable to remove the security proxy in carefully defined situations
<lifeless> so, why does it try to access their email address?
<lifeless> [clearly its ok to do that in carefully defined istuations
<lifeless> code exists to serve us, not the other way around; but if we can avoid it its somewhat nicer.
<bigjools> it's trying to email potentially private addresses as part of a) upload notification, b) bug notification
<bigjools> all done under the permission of the webapp user
<lifeless> now, to avoid disclosure that has to be part of the BCC right ?
<lifeless> or a direct mail
<bigjools> long term we need to jobify it of course
<bigjools> did I just make up a word? :)
<wgrant> I don't think that's the private email address problem.
<wgrant> IIRC it dies (possibly correctly) when trying to include it in Changed-By or Signed-By in the announcement email.
<lifeless> I'd suggest having some method that you pass to the Person asking it to do the bit thats private
<wgrant> But I said that in the bug... let's see..
<lifeless> yeah, I'd say thats correct.
<bigjools> that would be a) as I said above
<wgrant> bigjools: There's no problem emailing to them, though.
<wgrant> It's including them in the email that's the problem.
<wgrant> Oh, no, other way around.
<wgrant> This is confusing.
<wgrant> So it must already rSP in places.
<bigjools> no
<bigjools> well,
<wgrant> I said in the bug:
<wgrant> I believe it only fails if it would send a notification to the private
<wgrant> email address; using the private email address in the email (eg. if the
<wgrant> person is deactivated) seems to work fine.
<bigjools> I would have thought any access of a private email would blow up
<wgrant> Yeah, it does.
<wgrant> So it's rSPing already.
<bigjools> hmmm
<lifeless> broad brush strokes
<lifeless> here is how I would tackle it.
<lifeless> I would ensure that Person is responsible for deciding when to/not to bypass the proxy
<bigjools> removeSecurityProxy does not appear in the queue.py file
<bigjools> so if it's used it's elsewhere
<bigjools> lifeless: I'm not sure it knows enough to do that  does it?
<lifeless> bigjools: add methods ;)
<bigjools> eugh :(
<bigjools> Person is already bloated
<lifeless> multiple places may want to be able to send an email
<lifeless> *to* someone, with only one recipient
<lifeless> thats reasonable to bypass the proxy -in that case-
<lifeless> grabbing a private email to put into a template for announcements isn't ok though.
<bigjools> that was the point I was going to make
<lifeless> as long as folk choosing to use the method won't be confused or guided into making the wrong choice, I think its ok to do it anywhich way..
<bigjools> maybe we should refuse uploads on people who have private email addresses?
<bigjools> s/on/by/
<lifeless> why are we looking up their email (vs using the one in the changes file itself) ?
<bigjools> it uses the preferred email
<lifeless> so if I upload as noddy@example.com
<lifeless> but my preferred mai is fred@demo.com
<lifeless> the .changes file is regenerated with fred@demo.cpom ?
<bigjools> nothing's regenerated
<lifeless> ok, so why are we looking up their email?
<bigjools> we put email addresses on the email template
<lifeless> whats the template file
<lifeless> it will be faster than 20 questions :)
<bigjools> to, y'know, send it :)
<bigjools> man it's been 3 years since I looked at this code, hang on
<lifeless> bigjools: no, I don't understand why we need their email
<bigjools> lib/canonical/launchpad/emailtemplates/upload-accepted.txt
<bigjools> it uses changed-by, maintainer and signer
<lifeless> the approver did the approving; we need their Name; the uploaded uploaded it, we need their Name. Neither sent the mail, so we don't need their emails, and we send separate copies per recipient, so thats fine too.
<bigjools> lifeless: the current format was arrived at after extensive discussion with the ubuntu guys
<lifeless> I think we need to involve them then.
<bigjools> we need to re-create the problem first
<lifeless> I think its incompatible to both have 'you can have your email address private in launchpad' and to be putting it in mails sent to other people.
<lifeless> certainly a test case will help
<bigjools> yes, exactly
<lifeless> o/~
<bigjools> I also don't understand why a preferred email address would be private
<lifeless> its their only email ?
<bigjools> actually it's all or none isn't it?
<lifeless> probably
<lifeless> I think so
<bigjools> so trying to hide your email address while doing public works seems...odd :)
<lifeless> folk are very worried about spam
<lifeless> changes files go to a public list.
<bigjools> we could put <private email> on the template
<lifeless> for instance, yes.
<lifeless> Or an LP account url, or the SSO persistent url.
<bigjools> but the To: can't be hidden
<lifeless> bigjools: the To: shouldn't be them anyway ?
<bigjools> err From:, sorry
<bigjools> actually I can't remember
<bigjools> I think the uploader is CCed from memory
<lifeless> when I read that template, it doesn't look like it would make sense to be 'from them', because its 'thanking them'
<lifeless> bigjools: I have a suggestion: register a UDS spec for this.
<lifeless> bigjools: it needs it.
<bigjools> well, there's quite a few people involved in a package
<lifeless> well, maybe it doesn't, but I think the driving forces are complex enough it will benefit from that scale of analysis and discussion.
<bigjools> I'm not comfortable with the analysis yet
<lifeless> ok
<bigjools> once we get a failing test case (and when I can see the oops that might help) then we can decide where to go
<lifeless> lets get a autoated test
<lifeless> agreed
<lifeless> if I can help further I'd be delighted to do so,
<bigjools> great, thank you
<lifeless> but I suspect that its going to run into a definitional problem very early on rather than a code problem; and for that the stakeholders... have to hold their stakes.
<bigjools> dipped in silver nitrate?
<lifeless> lol
<lifeless> spm: hah just saw you put th epatch on... trying
<lifeless> its gone already...
<lifeless> will try tomorrow
<lifeless> bigjools: I bet that  https://launchpad.net/ubuntu/+search (Distribution:+search) will be your top timeout this cycle.
<bigjools> yay
<lifeless> anyhow, I'm going to be looking at when I get up :)
<lifeless> https://lpstats.canonical.com/graphs/OopsLpnetHourly
<lifeless> adeuring: try now
<wgrant> Now, let's see how badly Soyuz breaks on Lucid...
<bigjools>  /o\
<bigjools> I love your optimism
<wgrant> Oh, code upgrade's done already too? Nice.
<lifeless> whee things feel sluggish
* elmo changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<wgrant> bigjools: Is optimism ever a good idea when Soyuz's fragility is involved? :)
<bigjools> hey it's a lot better than it used to nbe
<wgrant> It is.
<wgrant> But buildd-manager will still break if you think about it the wrong way.
<wgrant> Although I guess that's not technically Soyuz any more.
<bigjools> indeeed
<bigjools> I need to set up a new project for it
<bigjools> launchpad-buildmaster has a nice ring to it
<lifeless> another project?
<wgrant> bigjools: launchpad-buildfarm, surely?
<wgrant> bigjools: This reminds me...
<bigjools> no, buildmaster, it's specifically for the master sude
<bigjools> side
<wgrant> Have you ever seen production or DF b-m start failing to dispatch builds, complaining that the XML-RPC build command is being given too many arguments?
<bigjools> nope
<wgrant> I've seen it locally, and had similar reports from a couple of others' local setups.
<wgrant> And I partly tracked down why it happens.
<wgrant> But I don't know what triggered it... and I'd never heard about it happening anywhere prod-ish.
<spiv> Sweet, codebrowse OOPSes appear to work.
<wgrant> (the problem is that BuilderSlave is broken now, but we only use RecordingSlave... except for in some circumstances that appear sometimes locally, which are not entirely clear)
<lifeless> spiv: you got one?
<spiv> Be one of the first to generate your very own codebrowse OOPS: http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/annotate/head%3A/README2
<lifeless> spiv: but can you see it on lp-oops?
<wgrant> That's a little more friendly than the old page
<spiv> lifeless: I haven't tried yet
<lifeless> spiv: acid test man ;)
<adeuring> lifeless: one test run with 8 parallel uplods: one failed with a "connection timed out"
<spiv> lifeless: but even that page beats "Internal server error"
<lifeless> adeuring: whats the oops?
<adeuring> wait a second...
<spiv> lifeless: so far the OOPS isn't on lp-oops
<spiv> (OOPS-1713CB6)
<lifeless> spiv: it may need some follow up
<lifeless> spiv: with QA
<lifeless> specifically:
<spiv> What's the typical delay for syncing?
<lifeless>  - needs to be added to the lpnet summaries.
<lifeless>  - needs to be added to the list of dirs to scan for the oops db scanner
<adeuring> lifeless: problem is that we don't get an OOPS
<lifeless> spiv: 3m I think
<lifeless> adeuring: what do we get ?
<lifeless> adeuring: if its apis check the X-Launchpad-OOPS header
<adeuring> just the error message "connection time out", my script doesn't print it
<lifeless> adeuring: (I think that is where the id goes)
<adeuring> ok, I'll try to find it
<lifeless> adeuring: to debug this we need:
<lifeless>  the backtrace
<lifeless>  the appserver it happened on
<adeuring> i know
<lifeless> it may be a further firewall issue on the other appservers.
<lifeless> or something.
<adeuring> so, how can I figure out which app server is used?
<lifeless> anyhow... late here. If you can get the appserver + error, ask the GSAs if they can confirm that appserver has access to the restricted upload port
<lifeless> adeuring: its in the OOPS :)
<adeuring> ah, ok
<lifeless> I'm quite sure we generate one, just goes into a header from what gary was saying th eother week
<lifeless> adeuring: you might like to file a new private bug
<lifeless> adeuring: unsubscribe everyone but you
<lifeless> and then test on it.
<adeuring> lifeless: yeah...
<lifeless> gnight
<lifeless> if its not a firewall issue I will debug it with your script (if you supply it) and the data you gather overnight; getting the OOPS is critical path to solving it.
 * lifeless waves gnight
<wgrant> jelmer: Hey, how's the branch? Were you able to reproduce the issues I reported?
<bigjools> nn lifeless
<wgrant> Night lifeless.
<noodles775> Enjoy the rest of your evening lifeless
<jelmer> 'night lifeless
<jelmer> wgrant, yeah, fixing + qa'ing at the moment
<wgrant> jelmer: Great.
<bigjools> so, my failure-detecting b-m hasn't failed anything yet.  Is it wrong to want to see that happen? :)
<wgrant> Sorry for throwing them at you so late... I wasn't aware until yesterday that the branch was targetted at 10.09.
<wgrant> bigjools: Does it manage to distinguish between build and builder failures?
<bigjools> that's the plan, yes
<jelmer> wgrant: Thanks for bringing it up in the first place. You saved quite a few people the stress and extra time that would've come with a broken rollout.
<wgrant> jelmer: I have a few other issues with the branch from a more thorough review today, but I'm sure I've caused you enough trouble for now.
<wgrant> None are particularly major, I don't think.
* gmb changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.09 | PQM is open for business | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<deryck> Morning, all.
<wgrant> Was cocoplum upgraded to Lucid?
<wgrant> grep-dctrl now chokes on maverick's Sources files for reasons which I cannot entirely determine.
<wgrant> Ooh dear.
<wgrant> I think cocoplum's a-f cache might be pretty fucked.
<wgrant> bigjools: ^^
<wgrant> maverick's main Sources is now ~3.4MB. From an out-of-date mirror, it's ~4MB.
<wgrant> Some sections are truncated.
<bigjools> wgrant: known bug with lucid a-f
<bigjools> bug 633967
<_mup_> Bug #633967: apt-ftparchive generates corrupt Sources stanzas for .dsc files without Checksums-* fields <apt (Ubuntu):In Progress> <apt (Ubuntu Lucid):In Progress> <apt (Ubuntu Maverick):In Progress> <https://launchpad.net/bugs/633967>
<wgrant> Ah, great.
<bigjools> life is never too easy is it
<wgrant> (now, what were you saying about optimism a few hours ago? :P)
<wgrant> bigjools: I bet it's the case-sensitivity stuff in changes file parsing.
<wgrant> The code looks for Launchpad-bugs-fixed, but it's normally spelt Launchpad-Bugs-Fixed.
<bigjools> well remembered
<wgrant> The uploads referenced are fine.
<wgrant> A ha ha.
<wgrant> A test was changed to use the former capitalisation.
<bigjools> jelmer!
<bigjools> (that's one way of making the tests pass)
<wgrant> Heh.
<bigjools> wgrant: which test was changed?
<wgrant> bigjools: See the last hunk of the rev...
<wgrant> Let me find it.
<bigjools> which revno?
<wgrant> Ah, no.
<wgrant> The tests were already broken.
<wgrant> sync-source was changed to use the bad capitalisation.
<bigjools> it's because of the changed parser we're using now
<wgrant> db-devel r9741
<bigjools> FML
<wgrant> It is, yes.
 * bigjools wonders why that didn't break the test
<wgrant> The test packages probably use the bogus capitalisation.
<bigjools> oh, I see
 * wgrant greps.
<bigjools> yes it does
<wgrant> YAY.
 * bigjools files a bug
<bigjools> wgrant: btw, isn't it way past your bedtime? :)
<wgrant> bigjools: It's barely midnight.
<tyarusso> Um, how would I set up an environment of db-stable instead of devel with rocketfuel-setup?  I tried changing all of the lines I could find in the script and still got a "devel" directory under lp-branches.
<tyarusso> ...or I could try reading the help output.  *headdesk*
<tyarusso> Where is the "etc/zope.conf" for disabling developer mode supposed to go, and how would one do that?
<tyarusso> Also, I can log in with the test account, but how do I register new accounts?  I don't see any links for that yet...
<leonardr> asking a possibly stupid question rather than wasting more time. i've got a totally up-to-date system and 'make' is failing with a bzrlib import error, "cannot import name SAFE_INSTRUCTIONS". any help?
<leonardr> gary, lifeless -^
<gary_poster> don't know leonardr but will try to dupe.  maybe try a code team or bzr team member after that.
<leonardr> maybe rockstar can help?
<gary_poster> tyarusso: register new accounts: not exposed on dev system.  our openid server is responsible for that in the production/staging.
<rockstar> leonardr, update download-cache?
<tyarusso> gary_poster: Oh.  Well how is someone supposed to use it then?
<gary_poster> tyarusso: etc/zope.conf: see configs/development/launchpad.conf
<leonardr> rockstar, up to date
<gary_poster> tyarusso: sorry, don't understand question.  dev build is for developers, which approximates production just enough to do dev style testing.  We're not in charge of making new users, so we don't expose it
<rockstar> leonardr, oh, and sourcecode also needs to be updated.
<rockstar> SAFE_INSTRUCTIONS should come from bzr-builder, which can't be eggified.
<tyarusso> gary_poster: Okay, my goal here is to set up a Launchpad instance that we could actually use for our company.  What other pieces would I need to get to accomplish that?
<leonardr> rockstar: sourcecode/bzr-builder is up to date at revision 63
<leonardr> however, revision 63 is from january. could it have moved?
<rockstar> leonardr, hm, it should have.  abentley?
<gary_poster> tyarusso: oh, eek.  I'm sorry, that's a huge deal that we don't support.  We open source the code to improve our free service.  ...um, you could try asking the dev list and see if anyone there has advice?
<abentley> rockstar, I have just landed my permit-commands branch.
<leonardr> rockstar: i've got the pqm-managed branch at bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/bzr-builder/trunk/
<abentley> rockstar, in that branch, the revno is 66.
<rockstar> leonardr, ^^
<abentley> leonardr, the revno in that branch is 66, so you aren't up to date.
<tyarusso> gary_poster: Hmm, okay.  Meanwhile let's try a different direction:  1) Is the stuff you use for your openid server all open source too?  2)  Do you offer hosted services that could use our own domain names & branding?  If so, what's the fee schedule like for that?
<leonardr> abentley: ok, i've got the new stuff
<tyarusso> gary_poster: Oh, I should note - we'd be interested in hosting both open-source and proprietary projects.
<gary_poster> bac, are you around to address tyarusso's question # 2 above?
<leonardr> ok, i think i've got a bunch of out-of-date stuff
<bac> mrevell: can you help tyarusso?
<tyarusso> haha, support hot potato!
<bac> ah, tyarusso, i read through all of your question.  to answer #2, no, we don't offer branded hosting
<tyarusso> bac: Hmm, okay.
<tyarusso> The normal stuff might be better than what we have now, but wouldn't let us put everything all in one place, which would be ideal.
<gary_poster> tyarusso: for 1) the openid server is closed source.  ...I keep revising my answer to cut to the chase sufficiently...in sum, you'd have to branch the code to make it work, and it would be hairy; maybe you could get some community people interested, dunno.
<gary_poster> tyarusso: everything in one place: this isn't my part of story, and I have to run now, but (A) you could have a group that collects your projects and (B) I'm almost 100% sure it can contain both proprietary and open-source bits.
<gary_poster> bac, mrevell, you can fix my reply if necessary :-)
<m4n1sh> is there any way in the Launchpad API for search a bug?
<m4n1sh> bugs has only createBug and in bug there is nothing which matches searchBug or something like that
<deryck> rockstar, did you ever get that widget moving with help from dav?
<rockstar> deryck, nope.  Was hoping to get with you tomorrow about it.
<rockstar> deryck, also, I just landed yui 3.2 into lazr-js, and that's got some more debugging happiness in it, so I thought I'd merge.
<deryck> ok, let's plan on it.  I'll try to poke at your code in my evening for home work. :-)
<deryck> indeed!
<gmb> m4n1sh, Find the project you want to search on and then use project.searchTasks()
<m4n1sh> gmb: so this means global bug search is not possible
<gmb> m4n1sh, Yes, but it's not possible in the Launchpad UI, either.
<m4n1sh> gmb: I think you can https://bugs.launchpad.net/
<m4n1sh> you can choose "All projects"
<gmb> m4n1sh, You're right; my bad.
<gmb> I thought we used Google for that.
<m4n1sh> me too :)
<gmb> (Launchpad Bugs developer doesn't know about Launchpad Bugs)
<gmb> m4n1sh, But no, that's not available in the API at the moment, I'm afraid.
<gmb> Of that I'm sure.
<m4n1sh> I could not find it in the API
<m4n1sh> actually I am presenting a talk on PyCon India 2010 on launchpadlib
<m4n1sh> so needed to know abuot this
<m4n1sh> gmb: your help is appreciated :)
<gmb> m4n1sh, I think that's because the API is basically an export of our underlying object model.
<gmb> And the UI sits on top of that object model, so it can do things the API can't.
<gmb> m4n1sh, There's probably a bug for it (I'm on the phone now, otherwise I'd check for you) but feel free to file one if there isn't.
<m4n1sh> gmb: sure :) thanks
<gmb> np
<leonardr> gary, rockstar: (rockstar, i know this isn't your problem, but maybe you can help anyhow)
<gary_poster> :-)
<leonardr> actually, let me check the wiki really quick, since the problem is probably that my dev process has gotten out of sync with launchpad's
<leonardr> gary, rockstar: no, i'm fairly certain i've done everything right, and i'm getting an import error from shipit
<leonardr> from canonical.cachedproperty import cachedproperty -> "No module named cachedproperty"
<rockstar> leonardr, oh, shipit? Uh, I have no knowledge in that area.  :(
<leonardr> rockstar: yeah, i said it wasn't your problem
<rockstar> leonardr, I thought we moved all the shipit import stuff to somewhere under lp though.
<gary_poster> I saw that leonardr...I think I ran utilities/update-sourcecode.  You've done that too?
<tyarusso> All right, flailing e-mail sent to the mailing list - maybe someone out there has a clue :S
<leonardr> gary: does rocketfuel-get not run update-sourcecode? i didn't run it specifically
<gary_poster> :-) Good luck.  It won't be a core LP dev, I strongly suspect, and I'll be surprised if someone knows what to do, but who knows.  I'm sorry that the available options don't work for you. :-/
<gary_poster> leonardr: yeah I think so
<gary_poster> I mean, I think it does run it
<gary_poster> it certainly should
<gary_poster> I'll look...
<gary_poster> it does
<leonardr> when i run it manually i get a bzr repository conversion error!
<gary_poster> in dulwich I bet
<gary_poster> yeah?
<leonardr> no, in pygettextpo
<gary_poster> oh
<leonardr> maybe i should just remove sourcecode and get everything again? i don't think i've run this properly for a _long_ time
<gary_poster> yeah ,maybe so leonardr.  I did surgery instead, myself
<gary_poster> I found the broken ones and did a fresh branch in launchpad/lp-sourcedeps/sourcecode
<gary_poster> of the broken ones reported by update-sourcecode
<gary_poster> your call
<cody-somerville> Who takes care of the svn import stuff? Can someone take a look at http://launchpadlibrarian.net/54208924/vcs-imports-django-trunk.log ?
<rockstar> cody-somerville, are there branches from this branch?
<cody-somerville> rockstar, it looks like it but most are hundreds of weeks old
<rockstar> cody-somerville, I suggest creating a new import, and maybe deleting or Abandoning that branch.
<rockstar> It's still using cscvs, which we're basically not maintaining anymore.
<cody-somerville> rockstar, https://code.edge.launchpad.net/django <-- do you see how there are two series of the same name associated with lp:django? Is that intentionally possible?
<rockstar> cody-somerville, whoa, that's weird.  I don't know if that's intentionally possible.  Maybe sinzui knows?
<beuno> cody-somerville, one seems to be off the project group, and the other off of the prohect
<beuno> *project
<cody-somerville> beuno, both are just normal projects
<rockstar> beuno, yeah, djangoproject should probably be deleted.
<cody-somerville> yea
<rockstar> Er, disabled.
<rockstar> cody-somerville, can you file a bug about that, and I'll discuss it with the team this afternoon?
<beuno> that is something interesting and new then!
<cody-somerville> It seems possible to associate any branch in launchpad with a series
<cody-somerville> Or no... if I try to edit the series in djangoproject I get an error
<cody-somerville> so I wonder how the value it has now got set... maybe via web services API if it has different error checking?
<lifeless> moin
<cody-somerville> rockstar, bug #634280
<_mup_> Bug #634280: Branch of another project associated with series <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/634280>
<rockstar> cody-somerville, thanks
<cody-somerville> rockstar, Is it possible to edit the URL for svn imports yet or does that still require a LOSA?
<rockstar> cody-somerville, are you trying to edit the cscvs one?
<cody-somerville> no, new one
<rockstar> cody-somerville, link?
<cody-somerville> rockstar, https://code.edge.launchpad.net/~vcs-imports/django/trunk (I renamed the old import to old-trunk)
<cody-somerville> rockstar, It looks like maybe the trailing slash is problematic on the URL
<rockstar> cody-somerville, looks like the old branch needs to be deleted...
<cody-somerville> rockstar, why?
<rockstar> cody-somerville, the urls need to be unique.
<cody-somerville> You can't have two imports with the same URL?
<rockstar> cody-somerville, nope.
<rockstar> I just pointed the old-trunk to django/trunkDISABLED
<cr3> in launchpad, an account has a identity url and a person has a an account and a name, does that mean that multiple people can share the same account?
<lifeless> no
<cr3> lifeless: so why doesn't he person contain the identity url instead?
<cody-somerville> rockstar, it looks like it  failed again... :/
<lifeless> cr3: account is there for shipit
<lifeless> from a long time ago, nothing to do with openid
<lifeless> it has grown and changed since then, but we're trying to delete the account table
<rockstar> jelmer, see https://code.edge.launchpad.net/~vcs-imports/django/trunk - Does that make any sense?
<lifeless> deryck[lunch]: when you return
<lifeless> have a look at this:
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1713L718
<cr3> lifeless: awesome, makes sense. I was wondering when logging in with openid how it might demultiplex the various person objects in the event there was a one to many relationship
<lifeless> 'do not permit one' :P
<leonardr> gary, how many packages are in your sourcecode/?
<gary_poster> leonardr: 20
<leonardr> gary: i've got 17, but i used to have 56. the difference between you and me probalby reflects the continuing trend of moving things out of sourcecode
<leonardr> i ask because i was still getting an error, but i think 'make' might fix it
<lifeless> gary_poster: hi, I'd also like you to eyeball https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1713L718
<gary_poster> yes, yay.  on call leonardr, but http://pastebin.ubuntu.com/491137/ fwiw
<lifeless> gary_poster: don't interrupt your call; just look when you have a chance.
<gary_poster> ack thanks lifeless
<lifeless> we are making -huge- numbers of memcache calls - and they appear to be capable of blocking for 20ms
<jelmer> rockstar: it works locally; it's hard to say much about it without the error from bzr-svn on lp
<rockstar> jelmer, do we need better error logging?  Is that what you're saying?
<jelmer> rockstar: it would be nice to have the bzr log output as well
<rockstar> jelmer, okay.
<jelmer> rockstar: Actually
<jelmer> this looks like a launchpad-code bug
<jelmer> the exception is being raised from the part of the code that fetches the existing import branch
<rockstar> jelmer, I suspected it might be.
<jelmer> my initial thought was that it was failing to open the svn branch using bzr-svn
<deryck> hi lifeless.  Looking at the OOPS report now....
<lifeless> deryck: so basically we're spending - FWICT - about 2 seconds in memcache on that page
<lifeless> deryck: that combined with the low hit rate we're seeing makes me strongly suspect that turning off memcache in the template/tales will take 2 seconds off that page.
<lifeless> we need to do some improvements to the oops aggregation facilities now that we're putting more data in them
<lifeless> (I had to stare rather hard at the page to get a good sense of whats going on)
<deryck> yeah, taking me a bit to process it too...
<lifeless> so interestingly
<lifeless> comments=all
<lifeless> is in the memcache key
<deryck> yeah, I was noticing that.  Seems like stupid caching in the first place.
<lifeless> thats an issue : it means the short and long versions of the messages won't share cache keys.
<lifeless> but perhaps on some pages that matters. Filing a bug on -foundations now vis-a-vis that
<deryck> yeah, I'm trying to look at the template.  We didn't add this caching, so I need to remind myself what it's doing.
<lifeless> deryck: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/634326
<_mup_> Bug #634326: memcache cache keys interact poorly with query parameters <Launchpad Foundations:New> <https://launchpad.net/bugs/634326>
<lifeless> deryck: Please understand, when I talk about turning memcache off for things, its not because I don't like it : its because I want the best performance for any given page.
<deryck> lifeless, I do understand that.  I'm not against turning off bits of it or poor uses *at all* :-)  I was just against ripping it our wholesale.
<lifeless> k
<deryck> though it's more accurate to say I'm *for* learning to use it correctly and having the ability to use it correctly.
<lifeless> right
<lifeless> I have the sense though, that we haven't finished putting our house in order in the underlying layers yet :)
<deryck> I completely agree
<deryck> or have the same sense rather
<lifeless> :)
<deryck> which is okay if we rapidly iterate on it.
<deryck> but as is, it has some issues.
<deryck> lifeless, so I don't know this spelling, I guess:  "cache:authenticated,comment/rendered_cache_time,comment/index"  is that part of your logging work?
<lifeless> hmm, for clarity by underlying layers I meant - db use; storm; tal rendering; python scheduling
<lifeless> my logging work puts this in
<deryck> ah, ok.  agree on that, too, though I thought you meant something different
<lifeless> category=memcache-get
<lifeless> details=$url where $url is the memcached url we're requesting
<lifeless> so pt:lpnet:lp/bugs/templates/bugcomment-box.pt,9760:l:1,0:MjE=,https_//bugs.launchpad.net/ubuntu/+bug/1/+index?comments=all4N2mYcTmyKyEC6n0gBJBFG
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
<lifeless> is the url in memcache
<lifeless> the cache: stuff I didn't touch, thats the existing control-memcache-in-templates
<deryck> hmmm, so I thought it was just "cache:type,howlong" so I need to refer to updated docs.
<lifeless> anyhow
<lifeless> if we're caching per url in the webapp thats going to go a long way to explaining the terrible hit rate.
<lifeless> many objects appear in many urls.
<lifeless> e.g person, branding, bug comments (per distrotask) etc
<lifeless> hmmm
<lifeless> create-question appears to send email via a different codepath or something. darn.
<lifeless> no, thats not it. I wonder
<lifeless> is email spooling perhaps deferred to the end of the request?
<lifeless> gary_poster: ^
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1713N810
<deryck> lifeless, so about this memcache question, are you asking me about ripping this out and seeing if we save 2 seconds?
<lifeless> deryck: I'm speculating; I have a lot of data gathering to do now.
<lifeless> deryck: but bugtask is one of your pain-pages
<lifeless> and turning that off would be a very easy thing to do
<deryck> lifeless, yeah, that was going to be my concern is that we don't know the savings (if any) vs the 2 sec. cost.
<deryck> but I'm open to turn it off and see what oops appear.
<lifeless> I think it would be worth trying
<lifeless> even with the overhead of doing a CP to get it on prod, it will be a pretty cheap experiment
<deryck> sure, I'm open to that.  With a close eye on it.  I do however wish we could feature flag it, if that works now.  So if it was a bust, we could turn it on/off easy.
<deryck> I didn't put this in, so I don't know the problems it was trying to prevent.
<deryck> or the knowledge that underpinned the choice.
<lifeless> feature flags do work, I think you'd need to repeat the contained section though
<lifeless> which is a bit ugly
<deryck> yeah
<lifeless> if we had a pageid /scope/ we could disable memcache for a pageid via a feature flag
<lifeless> that should be reasonably easy to do, for someone familiar with the page id logic.
<lifeless> I'll file a bug requesting that, and note we'd like to CP it to prod
<deryck> sure, sounds good.
<deryck> leave it New please, so I'll triage it and add it to the board.
<deryck> forces me to add a card when I have to toggle it to triaged.
<deryck> unless this is a foundations bug ;)
<lifeless> its a foundations arena bug, but you might find it easy enough to JDI - sec while I file it
<dobey> hmm, is there a known issue with the code scanner in lp right now?
<lifeless> deryck: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/634342
<_mup_> Bug #634342: need a features 'scope' for page ids <Launchpad Foundations:New> <https://launchpad.net/bugs/634342>
<dobey> like it seems to not be scanning only some branches, and sending proposal e-mails without diffs for them
<deryck> lifeless, got it.  I'll decide if I can JFDI that later today or not.  which goes a bit against JFDI, I know.
<lifeless> deryck: :P
<lifeless> deryck: how did abel go last night
<deryck> lifeless, he was worn out from all this, so worked on an easy bug. :-)  But glad that we had it fixed for retracers.
<lifeless> deryck: I'm glad too
<lifeless> deryck: also on performance - https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1713S81
<lifeless> I grabbed that for bug 1 on staging yesterday, with profile.
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
<lifeless> I'm going to put it in the
<gary_poster> lifeless, today is call day, but wading through backlog.  yes, email spooling is deferred to the end of the request.
<lifeless> gary_poster: ahha
<lifeless> gary_poster: can you point me at the thing that does that, so I can instrument it ?
<gary_poster> end of transaction to be precise
<lifeless> gary_poster: really? after request finalisation ?
<gary_poster> yes
<lifeless> can we change that?
<gary_poster> bad idea IMO.  the idea is that if a transaction retries, we don't want to send multiple emails.
<lifeless> to be on-part-of-something like that
<lifeless> ok
<lifeless> uhm, let me describe some symptoms
<gary_poster> k
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1713N810
<lifeless> 32 second request
<gary_poster> need to go in a few; and then will be back on calls, btw
<gary_poster> looking
<lifeless> it sends 115 emails
<lifeless> 2pc would solve this but be a bit nasty.
<lifeless> doing the email spooling from a worker thread would solve it and not distort the request time
<gary_poster> we're suposed to have the infrastructure to be doing this already. standard zope bit...
<lifeless> so, anyhow, 260ms per email is one useful thing to know
<lifeless> that seems slow.
<gary_poster> it's supposed to put in worker thread (in* the 2pc
<gary_poster> *in*
<lifeless> could I perhaps move the bug to foundations ?
<gary_poster> I guess...it's either something for foundations or something that foundations would be happy to advise on
<lifeless> blue sky, does pgsql let us ask 'will this commit succeed' ?
<lifeless> its bug 438116, I'll update the details now we have some instrumentation.
<_mup_> Bug #438116: Timeout when converting bug into question (BugTask:+create-question) <timeout> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/438116>
<gary_poster> lifeless, I don't yet see 260ms per email on that OOPS.  give me a hint as to where I should be looking?
<lifeless> gary_poster: look for sending emails
<lifeless> or 'send' perhaps
<lifeless> anyhow, 115 rows (by subtraction)
<gary_poster> e.g. 352.	3617	0ms	sendmail	[Question #124730]: No vdpau-va-driver for amd64 in Maverick
<_mup_> Bug #124730: Feisty - Sound stops after a few seconds and various apps fail to work properly <Ubuntu:Invalid> <https://launchpad.net/bugs/124730>
<lifeless> and at the end
<lifeless> the request does its session update @ 32 seconds in
<lifeless> and it finished its sql work, which for this code is ~= to finishing completely, ~6 seconds in
<lifeless> 26000/115
<gary_poster> ah ok
<lifeless> approximations-R-us-ly-yrs
<gary_poster> sure. :-) from users perspective, the request should have been sent back by 6 seconds, but agreed anyway that sending email ought to be done elsewhere.  as I said, the expected pattern is that you would spool in the commit phase.  well, since that's not happening, then maybe user didn't get the reply till 6 seconds in either. :-/
<gary_poster> didn't get the reply 6 seconds in I meant
<gary_poster> I need to go get boys from school
<gary_poster> then I have calls
<gary_poster> I will look at how we are sending emails
<gary_poster> if you happen to know the code path that would be nice
<lifeless> thanks, no panic - this like other timeouts is just kanban backlog
<gary_poster> (that is, where in LP the email is sent)
<gary_poster> right, cool
<lifeless> I will dig up some of that for you
<gary_poster> thank you
<gary_poster> biab
<dobey> i guess this is the more active channel now?
<lifeless> this is the development channel
<EdwinGrubbs> matsubara: ping
<matsubara> EdwinGrubbs, on the phone
<matsubara> will ping you once I'm done
<EdwinGrubbs> ok
<wallyworld_> morning
<gary_poster> lifeless: need to go, bug re bug 633664, when you talk about double-buffering, you mean at the load balancer and in the app server, correct?
<_mup_> Bug #633664: API file attachments are done via the appservers <Launchpad Foundations:Invalid> <https://launchpad.net/bugs/633664>
<gary_poster> s/bug re bug/but re bug/
<lifeless> gary_poster: the load balancer isn't buffering
<gary_poster> apache?
<lifeless> gary_poster: the double buffering is in the appserver
<gary_poster> buffers in asyncore, then?
<lifeless> gary_poster: from what you're telling me, yes.
<gary_poster> right but, I mean, where else?
<lifeless> gary_poster: there are some explicit things I know, and some things I don't.
<lifeless> gary_poster: we're calling addFile in the appserver, and we have a facility to retry the request if the db conflicts -> that implies a buffer
<gary_poster> (this isn't to ignore your other concerns, but I understand them)
<gary_poster> but why a second buffer?
<lifeless> ah
<lifeless> I think I've added confusion
<lifeless> what I mean is 'we're not sending it directly to where it belongs, but the design is intended to support big blobs by doing that'
<gary_poster> ah!
<gary_poster> got it
<gary_poster> fair enough, understood
<gary_poster> thanks
<gary_poster> must run :-)
<thumper> wallyworld_: morning
<lifeless> happy to talk another day
<lifeless> ciao
<gary_poster> cool, but I understand now and agree that what we have is a problem
<wallyworld_> thumper: happy birthday
<gary_poster> thumper: happy birthday :-)
<lifeless> gary_poster: cool, thanks.
<thumper> thanks
<jelmer> thumper: hi
<jelmer> thumper: did you see the wiki page maxb added with failing bzr-svn imports summary?
<thumper> jelmer: hi
<thumper> no
<thumper> I'm busy chasing a critical branch scanner fubar
<thumper> due to bug heat
<jelmer> I won't bother you then :-)
<thumper> jelmer: good that chicken finally imported :)
<thumper> I'm sure peter is happy
<mwhudson> jelmer: i saw a lot of code imports marked failed overnight, do you know what that's all about?
<jelmer> mwhudson: I retried all of the ones I thought *might* be fixed and some still failed, but a lot are now actually working
<mwhudson> jelmer: oh ok
<mwhudson> that's good
<jelmer> mwhudson, we're down to less than 100 failures in the bzr-svn/bzr-git imports, 59 of which are caused by lack of support for nested trees
<mwhudson> jelmer: \o/
<wgrant> Are nested trees actually happening at some point?
<lifeless> yes
<wgrant> Well, more than they were two years ago? :P
#launchpad-dev 2010-09-10
<lifeless> yes
<thumper> can someone else try authorizing a new launchapdlib token?
<thumper> I got a client side and server side blowup
<lifeless> win
<lifeless> OOPS in the response?
<wgrant> thumper: Works for me.
<wgrant> On Maverick.
<thumper> lifeless: https://pastebin.canonical.com/36957/
<thumper> lifeless: OOPS-1713EB2534 on server
<lifeless> thats...surprising
<lifeless> its also surprising to get the backend failed message
<wgrant> What exploded this time?
<lifeless> orry, there was a problem connecting to the Launchpad server.
<wgrant> That's... not an oops, though.
<lifeless> I know
<lifeless>   Module canonical.launchpad.webapp.publisher, line 297, in __call__
<lifeless>     self.initialize()
<lifeless>   Module canonical.launchpad.browser.oauth, line 259, in initialize
<lifeless>     assert self.token.is_reviewed, (
<lifeless> AttributeError: 'NoneType' object has no attribute 'is_reviewed'
<wgrant> Special.
<lifeless> at least its a fast page
<lifeless> 207ms
<wgrant> Heh.
<lifeless> LLLLEEEEOOOOONNNNNAAAAAARRRRRRDDDDDD
<lifeless> thumper: I'll file a bug for this
 * maxb wonders what an earth has happened to the kdegames import, which was "Successful", but "This branch has not been imported yet"
<maxb> https://code.edge.launchpad.net/~neon/kdegames/trunk
<thumper> maxb: not scanned I guess
<wgrant> The scanner is broken at the moment, isn't it?
<thumper> not fully broken
<thumper> only to those that are linked to bugs with distribution source package tasks
<lifeless> thumper: the token authorized ok
<lifeless> thumper: I think
<thumper> lifeless: well the client went on its merry way
<lifeless> yeah
<lifeless> but the 'show it to the user' page went boom
<maxb> If the branch was scheduled for scanner attention, ought it not show they pale yellow "Updating" box?
<lifeless> maxb: we've had a scanner malf
<lifeless> maxb: may be implicated
 * mwhudson afk for a bit
<rockstar> thumper, shall we skype about some UI?
<thumper> rockstar: still chasing critical issue
<rockstar> thumper, okay.
<rockstar> thumper, maybe some annotated mockups?
<thumper> rockstar: lets talk anyway
<rockstar> thumper, okay.
<MTecknology> Why is it that Configuration Progress is never complete until you actually enable everything? You can't just say, I don't want to use this.
<thumper> MTecknology: you should be able to
<thumper> MTecknology: if it doesn't work that way it is a bug
<lifeless> thumper: you can't for code hosting yet, I don't think.
<MTecknology> oooooh! You can now!
<MTecknology> I lied and ignore me :P
<MTecknology> lifeless: it seems you can :)
<wgrant> MTecknology: Yeah, as of yesterday it should be OK.
<wgrant> +configure-translations isn't exactly... intuitive.
<wgrant> Is it finished?
<MTecknology> kiko: hey, how ya been?
<MTecknology> I hate it when I should know exactly where something in LP is... but I can't remember. This is gonna bug me. :P
<wgrant> Things do move occasionally.
<wgrant> What've you lost?
<MTecknology> add a download file
<wgrant> You need to do it from within a release.
<MTecknology> and that's the part I forgot..
<MTecknology> thanks :)
<wgrant> lifeless: This new request timeline stuff is pretty awesome.
<michaelh1> Hi there.  I have a web app that uses launchpadlib to create tickets.  How can I set the reporter to an arbitrary user?
<michaelh1> (on creation that is)
<wgrant> The API doesn't support impersonation.
<wgrant> It may eventually for some trusted clients.
<michaelh1> OK.  I want the reporter to be the same as the person logged into my web app.  The web app is authenticated using OpenID against LP.
<michaelh1> ...and there's only a few users.
<wgrant> Ah. In that case you should probably get an OAuth token on behalf of the user.
<michaelh1> What's a clean way of doing that?  Worst case is I give them a small app that runs on their machine and get them to email me the credentials file...
<wgrant> Well, I'm not sure if there's a great web-based workflow yet -- I haven't done it myself.
<wgrant> But when you tell launchpadlib to request a token, it gives you an authorisation URL. The user has to visit that.
<michaelh1> Does the 'change non-private data' permission allow me to create and modify tickets?
<wgrant> If they're public bugs, yes.
<wgrant> Private bugs require the Change Private Data permission.
<michaelh1> Yip, these would be new public bugs.  What is a private bug?
<wgrant> Bugs start private if their project is configured that way, or if they're marked as security related. You can toggle privacy with the 'This bug is public' box in top right of the bug page.
<wgrant> Only subscribers can see private bugs.
<michaelh1> Thanks for your help.  We only have a couple of bug reporters so the OAuth hack-around will be fine.
<MTecknology> wgrant: for default private bugs, it needs to be either security related or have the proprietary thing, right?
<wgrant> MTecknology: I believe so.
<lifeless> wgrant: thanks
<lifeless> wgrant: what prompted that ?
<wgrant> lifeless: Just looking at OOPSes locally and seeing all the memcache etc. calls accounted for.
<wgrant> Fewer unexplained holes in time is good.
<lifeless> yeah
<lifeless> also I think sinzui was able to debug a memcache issue more quickly, via it
<wgrant> It looks like it.
 * StevenK tries to find how one uses lplib via the test suite
<wgrant> It makes it a lot less opaque.
<MTecknology> I think you guys should switch to Nginx :D
 * MTecknology wonders how hard it would be to do that
<wgrant> StevenK: Grep around for launchpadlib_for.
<wgrant> test_archive_webservice looks reasonable.
<wgrant> I should check how that performs.
<wgrant> Since it would be nice to use it everywhere.
<mwhudson> it's pretty slow i think
<wgrant> :(
<mwhudson> but don't take my word for it
<wgrant> We should be able to cache the processed WADL or something.
<thumper> mwhudson: I'm confused
<thumper> mwhudson: take a look at https://code.edge.launchpad.net/~neon/kdegames/trunk
<mwhudson> i remember debugging some weird interaction with launchpadlib_for that made me very angry
<mwhudson> thumper: it *is* friday, maybe a beer will help?
<thumper> mwhudson: I'm stopping soon to drink a bottle of wine
<thumper> probably pinot noir
<thumper> maybe two
<thumper> it has been one of those days
<mwhudson> thumper: branch puller is stuck?
<MTecknology> thumper: all I have for wine - two cheap brands I mixed together
<lifeless> wgrant: have you checked prod & edge?
<lifeless> wgrant: for tracebacks?
<thumper> mwhudson: no it isn't stuck
<thumper> mwhudson: it hasn't been pulled
<thumper> mwhudson: I've checked the puller logs
<thumper> mwhudson: and... there have been no scan jobs
<mwhudson> thumper: maaaybe the requestMirror call does to the internal xmlrpc server oopsed?
<lifeless> MTecknology: probably not hard; very uninteresting.
<lifeless> MTecknology: what would we gain?
<mwhudson> thumper: you could try calling requestMirror over the api and seeing if that kicks it into life?
<thumper> mwhudson: yeah, I'll try that
<thumper> mwhudson: maybe it was a timeout to finishJob ?
<MTecknology> lifeless: nginx is super awesome... and for static content kicks apache's ass - I don't know how many of your requests are for static content though
<mwhudson> thumper: oh right, yeah
<mwhudson> thumper: grep in the arsenic oops dir?
<mwhudson> thumper: would be odd to happen to the same branch repeatedly though
<thumper> it would
<thumper> mirror requested
 * mwhudson afk for a sec
<lifeless> MTecknology: super awesome really doesn't mean anything to me.
<wgrant> lifeless: Yes.
<lifeless> we have very few static files - css, the iconset and shortly the wadl. Oh, and the librarian which is only kindof static (its static but with complex url mapping & dynamic mime types and access control
<lifeless> wgrant: thanks
<lifeless> MTecknology: I have no interest in change for changes sake
<wgrant> lifeless: I've only seem them on production for a few minutes a few years ago. They were on beta for a while (when it was called beta), and on dogfood until just a couple of years ago.
<mwhudson> thumper: the branch seems to have been pulled now
<MTecknology> lifeless: With Apache if you have say mod_php or mod_python enabled, every single request forks a new instance that can handle those requests. Nginx only passes the request to that if it's needed. Then you can have it try to server static content that needs no additional processing which makes it extremely fast, and light. Then you could keep using Apache if preferred for those heavier requests.
<mwhudson> MTecknology: the chance of a server that's not in ubuntu main being deployed is .... small
<wgrant> MTecknology: Launchpad just uses Apache to serve a few static files and proxy to the Zope appserver.
<wgrant> Hm, nginx really still isn't in main?
<wgrant> odd.
<lifeless> MTecknology: mod_wsgi++, and php -argh.
<lifeless> MTecknology: also the prefork worker doesn't fork on every request
<mwhudson> thumper: come to think of it, it can't have been finishJob oopsing, because that's what records the success
<lifeless> MTecknology: I know that nginx has a lot of cred going around, but someone needs to map it to a /problem/ or /benefit/ that is relevant to us, before anyone is going to put in the time (non trivial) to change.
<thumper> mwhudson: hmm...
<thumper> mwhudson: WTF could it be then?
<MTecknology> lifeless: I guess if you proxy most with wsgi then it would be an insanely easy deployment. It's probably come down to just the ability to serve static content much faster. and in this case maybe the benefit wouldn't outweigh the hassle
<MTecknology> Either way I think I should pull the branch and try out some benchmarking.. Could be fun..
<lifeless> MTecknology: I think we're using proxypass for everything in fact.
<mwhudson> thumper: don't know
<thumper> :(
<MTecknology> lifeless: even for static?
<thumper> and Friday afternoon isn't the time to go looking for it
<mwhudson> thumper: it's not in the puller logs at all?
<lifeless> MTecknology: benchmarking would be essential. As mwhudson says, being in main is very important too (otherwise the lp team has to maintain the package)
<lifeless> MTecknology: for -anything-
<thumper> mwhudson: nope
<thumper> requestMirror worked
<MTecknology> .. so I definitely should peek at that..
<thumper> now pulled and scanned
<MTecknology> I would like to know why nginx isn't in main yet though :P
<mwhudson> thumper: i guess you could try to find the relevant worker logs from pear
<MTecknology> I should go find that out too..
<thumper> mwhudson: http://launchpadlibrarian.net/55296521/neon-kdegames-trunk.log
<MTecknology> lifeless: btw- thanks for giving an honest discussion to it. :)
<thumper> mwhudson: earlier one http://launchpadlibrarian.net/55292524/neon-kdegames-trunk.log
<mwhudson> thumper: that's the log from the worker -- i meant the log from the worker monitor i guess
<thumper> mwhudson: ah...
<mwhudson> they don't end up in the librarian, but are kept on pear
<mwhudson> dunno if they're synced to devpad, likely not
<mwhudson> spm: grep kdegames  in /srv/importd.launchpad.net/production-logs ?
<spm> mwhudson: they're on sodium, or should be
<mwhudson> i guess i should find out why i can't log in to devpad any more
<spm> mwhudson: /x/launchpad.net-logs/production/neumayer
<spm> haha, one sec then
<mwhudson> thumper can do it...
<mwhudson> :)
<spm> *snort*
<thumper> do what?
<mwhudson> thumper: grep kdegames -r /x/launchpad.net-logs/production/pear
<mwhudson> on devpad
 * thumper is there
<thumper> holy crap there are a lot of files in there
<spm> that's the "cleaned up" version. should have seen what it was like before.
<StevenK> mwhudson: Probably due to your team move
<MTecknology> lifeless: so.. Nginx in main, Patch for configs to work perfect, Benchmarks, A good reason to switch, anything else?
<mwhudson> StevenK: yeah
<lifeless> MTecknology: I think thats it; elmo has previously discussed this in a more pithy form "hell no" :- but if there is a compelling story I don't see why we wouldn't.
<lifeless> However, I don't *expect* a compelling story, based on what I know of our problems (today)
<MTecknology> Are most of the issues DB side?
<thumper> mwhudson: I can't see anything, but I'm feeling like I'm not getting very far
<lifeless> MTecknology: I wouldn't like you to spend a lot of time and end up empty handed is all I'm saying
 * thumper halts
<mwhudson> thumper: :/
<lifeless> MTecknology: DB issues are the proximate cause of a good 50% of the perf issues I look at; another 40+ are most closely identified as python - single threadedness, time-to-do-stuff
<MTecknology> lifeless: ya um... I tried to add a patch to the php package which added a whole new feature and I fought with it for about a month... The hardest part about this would be performing accurate benchmarking :P
<lifeless> another 10% are things like sending mail (200ms per mail)
<lifeless> we serve 2.6Million hits a day from the backend servers
<MTecknology> really?.. I sould thought it'd be more
<lifeless> some larger count from the squid farm (anonymous content only) & apache handles /all/ of that with nearly no load and no maintenance or uptime issues.
<MTecknology> Not to say 2.6mil is low.. I just woulda thought it'd be higher
<lifeless> let me check
<lifeless> 2838269 from lpnet
<lifeless> thats excluding:
<lifeless>  - librarian
<lifeless>  - bazaar.*
<lifeless>  - edge
<MTecknology> oh.. that sounds more like what my guess would been then.. My guess was ~10mil
<lifeless> 99% of our requests, by count, complete in 3.2 seconds
<lifeless> edge was only 66K yesterday
<MTecknology> oh- with nginx you can upgrade it in place, so you update the package and never lose any request. I don't think apache can do that - but I suppose you have plenty of redundancy too
<lifeless> OTOH we had a big chunk of downtime in place
<lifeless> MTecknology: yeah, we have L4 proxies in place already
<MTecknology> Not many people use edge?
<MTecknology> Now I really feel edgy :P
<lifeless> on a non-release day : 1681716
<lifeless> 1.6M
<lifeless> anyhow, we're going to get rid of edge real-soon now
<MTecknology> keep everyone looking at the same thing?
<lifeless> simplify deployment
<lifeless> get fixes to all users more quickly.
<MTecknology> oh
<MTecknology> alrighty - so I think I got my curiousity satisfied for the day :)
<MTecknology> I have one other question.. what is ~buy-something for?
<wgrant> The Software Center purchasing feature.
<wgrant> Which is very nearly there, I believe.
<lifeless> no idea but I'd guess its the agent account for the SCA
<MTecknology> oh
<lifeless> which is how folk buying debs from ppas via software centre will have their access requests relayed into LP
<MTecknology> elachuni seems to be doing a crap load of work
<MTecknology> Been a while since I talked to him or metcalfe - they seem to be locked in closets over there..
<MTecknology> Aight, it's beddy bye time - I'll ttyal
<MTecknology> lifeless: thanks for that chat; wgrant: thanks for the help
<wgrant> Night.
<lifeless> gnight
<lifeless> anyone around can do a small review ?
<adeuring> good moring
<mrevell> Morning
<lifeless> hi mrevell
<lifeless> mrevell: btw, it occured to me you might want to do a team-joining interview with me - or not, I'm easy.
<mrevell> lifeless, Oh, I thought I'd sent you one already.
 * mrevell goes to rifle through the Sent folder.
<lifeless> mrevell: heh
<wgrant> lifeless: Did you get a response from -security about the restricted librarian thing?
<lifeless> yes
<lifeless> +1, all things considered
<lifeless> we have the SSL certs
<lifeless> its with losa to be configured up on staging and lpnet
<lifeless> once thats all done we'll arrange to set the feature flag on staging and QA it all
<stub> And me... I need to sort the session db!
<stub> When are we looking at switching it on?
<lifeless> stub: Monday?
<stub> k. I'll do it after a coffee.
<wgrant> Excellent.
<wgrant> Has the domain restriction landed?
<wgrant> I forget.
<lifeless> wgrant: no, its in the rt for apache to do.
<wgrant> Ah.
<kiko> MTecknology, pretty good, just fell off the net :)
<deryck> Morning, all
<jtv> Reason #311 for not using ReST in doctests: bzr conflicts are hard enough to read with all the >>> that I really don't need any more ======= in there
<nigelb> heh ;)
<MTecknology> kiko: You did... and it made me sad. :(
<kiko> MTecknology, it's all good now!
<MTecknology> Yay! :)
<cr3> what's the difference between the "lp" icon (see blueprint project) and the "pad" icon (see launchpad-project project)?
<deryck> rockstar, ping
<rockstar> deryck, pong
<deryck> rockstar, yo, dude, I have progress on the widget mess.
<rockstar> deryck, really?
<rockstar> deryck, skype then?
<deryck> rockstar, yup.  And it's dumb but simple.  http://pastebin.ubuntu.com/491643/
<deryck> yeah, we can skype
 * deryck looks for headphones
<rockstar> deryck, are you freakin' kidding me?
<deryck> nope
<deryck> rockstar, do you want to skype, or that un blocks you now?
<rockstar> deryck, that makes the events fire?
<deryck> rockstar, let's skype to do high bandwidth.
<rockstar> deryck, skype doesn't think you're around.
<deryck> cause I am only now :-)
<deryck> ring me when you want, rockstar
<deryck> rockstar, guess I lost you.  I'm done though, if you are.
<rockstar> deryck, I think I lost you.
<rockstar> Yup.
<rockstar> deryck, we were finishing up anyway.  I was just saying that I'm going to consider my wizard work done (for now) when I have two steps.
<deryck> rockstar, righ
<rockstar> deryck, okay, cool.
<deryck> rockstar, so feel free to call on me or gmb if you need help next week.  Or me later today even.
<deryck> gmb needs to wizard widget to move forward, obviously.
<EdwinGrubbs> matsubara-lunch: ping
<matsubara> hi Edwin-afk
<matsubara> hmm
<matsubara> hi EdwinGrubbs
<EdwinGrubbs> matsubara: I just wrote a big reply to robert's comments on oops-tools, but I wanted to see if you could answer what the status is on open sourcing it. gary had implied that you would know more.
<matsubara> EdwinGrubbs, I'll reply. We had a conversation about it during the foundations stand up call today.
<EdwinGrubbs> matsubara: also, what email lists should I forward the email to in order to get ISD's and U1's input? I'm not sure if I'm even subscribed to their lists, though.
<matsubara> EdwinGrubbs, good question. I'm not sure either as I'm not subscribed to their lists. I usually talk to specific people on those teams (U1: Roman Yepishev <rye>, ISD: Anthony Lenton <achuni>)
<lifeless> EdwinGrubbs: I've replied ;)
#launchpad-dev 2010-09-11
<wgrant> lifeless: Are the appserver -> restricted librarian firewall rules completely sorted?
<wgrant> We are having 502s which could be caused by them.
<lifeless> wgrant: I don't know
<lifeless> abel said he was still seeing a failure if he pushed past 5 concurrent uploads, so I assume that we haven't figured it all out.
<lifeless> wgrant: gather oopes!
<wgrant> lifeless: There are no OOPSes.
<lifeless> https://edge.launchpad.net/sprints/uds-karmic/+temp-meeting-export <- why is this being hit :<
<wgrant> They're proxy timeouts.
<lifeless> restricted librarian isn't proxied
<wgrant> Yay, c.l.security is finally being split.
<wgrant> lifeless: Appserver connection timeouts, these are.
<wgrant> "Sorry, we couldn't connect to the Launchpad server."
<wgrant> On an action that would be accessing the restricted librarian.
<wgrant> And it's intermittent.
<lifeless> AIUI that error, that can't be related.
<lifeless> however, I may not understand the error
<lifeless> What server group ?
<wgrant> Hm.
<lifeless> edge/lpnet ?
<lifeless> file a bug, lets gather data.
<lifeless> it may well be related, but no assumptions
<lifeless> wtf
<lifeless> BugTask LEFT JOIN Bug
<lifeless> makes no sense
<wgrant> Looks like prod.
<wgrant> lifeless: If there are no timeouts on librarian connections, and the connections are being dropped instead of rejected, why couldn't it be related?
<lifeless> well
<lifeless> what does the error actually mean?
<lifeless> does it mean 'got no SYN-ACK
<lifeless> or does it mean 'got no HTTP response in X time' ?
<wgrant> I understand that it means the proxy didn't get a response from the appserver in a timely manner.
<wgrant> Which probably means the appserver was waiting for something.
<wgrant> Which, given last week's happenings, and the fact that other stuff times out, is quite possibly the librarian.
<lifeless> if it means no HTTP response in X time, then yes, it can be related.
<lifeless> but it also means we should be seeing OOPSes
<lifeless> what pageids ?
<wgrant> Even if there was no SQL executed afterwards?
<wgrant> Um, it was on bug submission.
<wgrant> So possibly BugTarget:+filebug-guided or something like that.
<lifeless> wgrant: yes, soft oops are generated if the request is > $time
<wgrant> lifeless: Ah, I didn't know if that also depended on SQL statements.
<lifeless> so
<lifeless> there's lazr.restful.utils.timeout or whatever it is
<lifeless> which does a thread based timeout enforcer
<lifeless> and there is the check in the storm tracer
<lifeless> I plan to move all these checks to requesttimeline.
<lifeless> or possibly something separate but connected.
<lifeless> gandwana
<wgrant> It's having lots of +filebug timeouts ?
<lifeless> first one is sql
<lifeless> death-by-a-thousand-LFA lookups
<lifeless> potassium looks similar
<lifeless> its awful o'clock to be calling the escalation phone just now
<wgrant> What needs escalating?
<lifeless> this issue
<lifeless> if its not fixed
<lifeless> 771 queries for +filebug
<lifeless> with apport data
<wgrant> Just tried some other restricted download stuff.
<wgrant> Got a failure from one prod appserver -- not sure which.
<lifeless> download or upload
<wgrant> Download.
<lifeless> we only had upload enabled on the firewall
<lifeless> this might explain it
<lifeless> well
<lifeless> maybe not
<wgrant> Download has been used for ages, though.
<lifeless> we only *corrected a missing rule* for upload
<wgrant> Ah.
<wgrant> So, since StreamOrRedirectLibraryFileAlias failed at least once, the firewall is probably the problem.
<lifeless> have you seen that ?
<lifeless> was there an oops?
<wgrant> No OOPS. Just a plaintext "There was a problem fetching the contents of this file. Please try again in a few minutes."
<lifeless> oh, feng shui ?
<wgrant> No.
<wgrant> This is displayed by the appserver proxy view.
<wgrant> When LibrarianServerError is raised by getFileContents.
<lifeless> I have to go
<lifeless> please - file a bug
<lifeless> lets get all the data we can
<wgrant> OK.
<wgrant> Thanks.
<lifeless> also it sounds like LibrarianServerError should be filing OOPSes
<lifeless> if you wanted to fix that we could CP it to get more data.
<wgrant> It sounds like it might be better to just not catch it at all.
<lifeless> it should generate oops, if the best way to do that is to not catch it - fine.
 * lifeless is gone, back in a few hours.
<wgrant> sinzui: Is OOPS-1714K1846 another of the openid_identity_url LocationErrors?
 * sinzui looks
<wgrant> The user has OpenID issues.
<wgrant> But it may be unrelated.
<sinzui> Yes it is
<wgrant> It works fine on edge, oddly.
<wgrant> And I don't see what's changed on edge.
<sinzui> I see two views definitely provide the attr
<wgrant> (in this case, post-rollout the SSO account mapped to the wrong account)
<wgrant> s/wrong account/wrong person/
<sinzui> wgrant, that me be the case
<sinzui> wgrant, this is the TB: http://pastebin.ubuntu.com/491936/
<wgrant> Huh.
<sinzui> ah we hit the XRDS code
<wgrant> Oh, right.
<wgrant> That's why it's only on prod.
<wgrant> Of course.
<sinzui> This is something that the foundations team may need to explain
<wgrant> Now, there were some changes relating to OpenID on account merges last cycle.
<wgrant> And the diff is huge, so I didn't even skim it. /me reads.
<wgrant> Grrrrar.
<wgrant> Branch is private.
 * wgrant diffs manually.
<lifeless> back
<lifeless> wgrant: how goes it, any more data?
<wgrant> lifeless: Nothing.
<wgrant> And I didn't file a bug, since if all goes well that view will disappear soon.
<wgrant> (once your stuff is active)
<wgrant> Or do you want a bug about the probably-not-bug +filebug issue?
<lifeless> the upload and download ports to the appserver need to be open regardless
<lifeless> because; in-appserver stuff uses the restricted librarian to get at content sometimes
<wgrant> They do, yes.
<wgrant> But it's not a bug.
<wgrant> It's an operational issue.
<lifeless> and uploads of all sorts are proxied via the appserver
<lifeless> wgrant: 'meh'
<wgrant> OOPS-1715S302
<wgrant> lifeless: You're not still around?
<lifeless> sigh, context manager fail
<lifeless> yes
<wgrant> What's the OOPS?
<wgrant> I got that the first couple of times before the "Please try again" started appearing on staging.
<lifeless> LaunchpadTimeoutError: Statement: 'SELECT DISTINCT SourcePackagePublishingHistory.archive, SourcePackagePublishingHistory.component, SourcePackagePublishingHistory.datecreated,
<lifeless> QueryCanceledError('canceling statement due to statement timeout\\n',)
<lifeless> SQL time: 10494 ms
<lifeless> Non-sql time: 175 ms
<lifeless> Total time: 10669 ms
<lifeless> Statement Count: 43
<wgrant> Hm, so probably unrelated.
<lifeless> its on staging
<lifeless> different librarian
<wgrant> It is.
<wgrant> But I still got the same error later.
<wgrant> So it's not prod-specific.
<wgrant> Is the staging librarian also on asuka, or not?
<lifeless> I think so
<wgrant> Urgh.
<lifeless> let me check
<wgrant> So... not firewall, in that case.
<wgrant> I could try dogfood, which I know is the one machine.
<lifeless> yes, asuka
<wgrant> If the failed request caused an OOPS, it should have been just after OOPS-1715S304.
<wgrant> Is it obvious?
<lifeless>  LaunchpadTimeoutError: Statement: 'SELECT BinaryPackagePublishingHistory.archive, BinaryPackagePublishingHistory.binarypackagerelease, BinaryPackagePublishingHistory.component,
<lifeless> thats 5
<wgrant> I didn't think I caused a third, but maybe I did.
<lifeless>   LaunchpadTimeoutError: Statement: '(SELECT "_259ce".name, Person.displayname, EmailAddress.email FROM Person JOIN Account ON Account.id = Person.account JOIN EmailAddress ON EmailAddress.person = Person.id JOIN TeamParticipation ON
<lifeless> thats 6
<lifeless> anon
<wgrant> Probably not, then (but that looks like an auth query... how would that be timing out so early?)
<wgrant> lifeless: The proxy timeouts go away if I remove most of the attachments from the uploaded blob, or if I file it against a project with only a couple of subscribers.
<lifeless> heh
<wgrant> Next test: Specifying a biggish team as the initial assignee, to emulate the lots of subscribers that Ubuntu has.
<lifeless> thought so
<wgrant> But that should still be an SQL timeout :/
<lifeless> and they all have been that I've seen, so far.
<wgrant> Oh look.
<wgrant> Setting assignee=ubuntumembers when filing the bug also makes it die like that.
<wgrant> But that should still be an SQL timeout. So why does it not appear as one...
 * wgrant creates a few hundred people locally.
<wgrant> Uh.
<wgrant> Would you like some queries?
<wgrant> That request has plenty.
<lifeless> heh
<lifeless> james_w: https://edge.launchpad.net/python-fixtures/trunk/0.2https://edge.launchpad.net/python-fixtures/trunk/0.2
<james_w> thanks lifeless
<lifeless> james_w: please let me know how you like/dislike it.
<james_w> I'll give it a go now
<james_w> I assume testresources will become a layer on top of fixtures now?
<lifeless> yeah
<lifeless> going to look at jmls remaining testrepository patches
<lifeless> then package up fixtures
<lifeless> then start working back along the stack, harmonising things
<james_w> excellent
<lifeless> I was surprised, 0.1 had 49 downloads.
 * jelmer cheers on lifeless
<james_w> the existence of fixtures fixture and testfixtures is unfortunate
<lifeless> yes
<lifeless> I thought had before wedging in there
<lifeless> I also looked at their designs
<lifeless> probably want to subsume fixture functionality wise in a couple of releases
<lifeless> and testfixtures, ah yes
<lifeless> sugar but not AFAICT fundamentally solving it
<lifeless> actually, revisiting, testfixtures is pretty neat
<lifeless> but the API for compare isn't quite disconnected enough for little ol me
#launchpad-dev 2010-09-12
<lifeless> wgrant: since you're here
<lifeless> wgrant: I don't hold with a strict ops/code split for bugs
<lifeless> wgrant: it doesn't make sense unless the eyeballs thinking about stuff are also partitioned.
<lifeless> And they aren't.
<lifeless> And shouldn't be.
<lifeless> I may well in future start agitating for LP ops stuff to be *on LP*, but that wouldn't make sense unless LP is a /lot/ more robust than it is now.
<lifeless> also, I wish 'str' was PEP8
<wgrant> lifeless: I guess.
<wgrant> What's un-PEP8 about it?
<lifeless> startswith
<lifeless> starts_with <- pep8
<wgrant> Ah, rihgt.
<wgrant> Also, why isn't it 'string'? :(
<lifeless> characters were expensive in the bad ol days
<lifeless> sigh
<lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/testing/layers.py", line 507, in setUp
<lifeless> ><
<lifeless>     time.sleep(0.1)
<wgrant_> I would really like to know why that happens.
<lifeless> why folk put time.sleep calls in layer setups ?
<wgrant_> Well, and why bin/test is so hard to kill.
<wgrant_> Often resulting in that traceback, then a hang.
<lifeless> I suspect the zope test running reinvocation stuff is broken subtly
<lifeless> grah - fugly - <script>LP.client.cache['context'] = ...
<lifeless> ok, I hate chromium
<lifeless> show source should /show the source/ not re-request
<lifeless> I just hit the filebug issue myself
<lifeless> on launchpad-foundations
<beuno_> lifeless, firefox does that as well, AFAIK
<lifeless> w/no apport glue
<lifeless> beuno_: firefox used to DTRT
<lifeless> hmm, something is really wrong, pages are on a go-slow
<beuno_> well, it used to be fast as well  :)
<lifeless> I wonder if we have a request backlog or something causing high perceived time
<lifeless> effectively lowering the service time
<wgrant_> lifeless: I'm still getting those truncated responses during most runs.
<wgrant_> So something is up.
<wgrant> Not something I've seen before, though :/
<lifeless> wgrant: grah
<lifeless> hmm
<lifeless> 7am
<lifeless> I'm seeing trouble too, but nothing that I can obviously identify
<lifeless> oops counts are slightly high, but not freakishly so except for a spike ~ 24 hours ago
<lifeless> wgrant: I got the connecting error on +filebug
<lifeless> want to know the weird thing
<lifeless> the bug got filed
<wgrant> lifeless: That's a different issue, then. Sounds like yours was post-redirect?
<lifeless> yes
<lifeless> not that you could tell
<lifeless> wgrant: I'm saying that I think its the same issue
<wgrant> Hmm.
<lifeless> that we're seeing two things : OOPSes w/stuff, and something networkish which is borking responses and causing 'count not connect' errors
<wgrant> Do you know how long it took?
<lifeless> a while
<lifeless> here is an explanantion for a truncated page:
<lifeless> an HTTP/1.0 page with no Content-Length had its network socket disconnect
<lifeless> we have two datacentres
<wgrant> httplib.IncompleteRead: IncompleteRead(15646 bytes read, 1272188 more expected)
<wgrant> is what I'm getting, FWIW.
<lifeless> the front ends are in one place.
<lifeless> the appservers are in both.
<wgrant> Wait, LP is split over both?
<lifeless> the database is in the same one as the FE's, AIUI.
<wgrant> I didn't realise it was in the second at all.
<lifeless> wampee and the other are, AIUI
<lifeless> anyhow
<lifeless> my working theory is connectivity issues between the DC's
<lifeless> this would:
<lifeless>  - increase SQL time (packet retransmits)
<lifeless>  - truncate pages partly transmitted w/out error (HTTP/1.0 We Loves You)
<lifeless>  - truncate pages with content-length with error, signalled by a socket shutdown only (again, 1.0 we loves you)
<lifeless>  - cause random timeouts if enough packets on the same tcp link drop
<wgrant> Hmm.
<lifeless> particularly if the 'failure to connect' error has a time < 2 * the retry interval for TCP
<lifeless> which I don't remember offhand
<lifeless> elmo: are thou perchance around?
<wgrant> lifeless: Is it just me, or is Launchpad in general *really really* slow at the moment?
<lifeless> yes
<lifeless> I think its a real issue
<wgrant>  /people took 2.97s with 29 queries.
<wgrant> But took like 20s to make it to the browser.
<lifeless> yes
<lifeless> Like I said, I think its cross DC fuckage
<lifeless> will ring elmo soon
<elmo> you'll need a little more evidence than that
<lifeless> elmo: hi cool.
<elmo> replication lag is > 300s, so we're on wildcherry, but that's recent and hardly "cross DC fuckage"
<lifeless> elmo: uhm, ugh
<elmo> we're also down to two edge servers because edge rollout has failed twice
<lifeless> elmo: well, I was using an abbreviation for the discussion earlier.
<lifeless> elmo: here are the symptoms I'm aware of:
<lifeless>  - apis and web requests are getting truncated responses
<lifeless>  - things feel slow
<lifeless>  - a fair number of requests get the 'could not connect to launchpad' error page (btw: what is the trigger for showing that)
<elmo> lifeless: so nothing in nagios jumps out as being obviously wrong
<elmo> if anything the system looks lightly loaded - despite being all on wildcherry, the load there is small - appservers are busy but not extraordinarily so
<lifeless> elmo: I looked at the hourly oops rates for edge and nothing was obviously bong there
<lifeless> elmo: does the load balancer show any sort of deep queuing going on perhaps? or apache?
<lifeless> elmo: the
<lifeless> 'could not connect to launchpad' page seems like a particularly relevant clue
<lifeless> elmo: what triggers it being shown ?
<elmo> I don't know
<elmo> there's no particular queues on haproxy
<elmo> and apache is fine
<lifeless> I *think* its shown when <the thing that shows it> gets no HTTP response header in 30 seconds from a server
<wgrant> Except it also shows sometimes after almost exactly 10s.
<lifeless> wgrant: odd
<wgrant> (this was in the +filebug stuff yesterday)
<elmo> both apache and haproxy have custom 503 pages
<elmo> and squid talks to haproxy
<wgrant> Is there a way to distinguish between the two?
<lifeless> is squid in the loop for authenticated requests?
<elmo> nope
<elmo> wgrant: they point at different files at least
<lifeless> the error I saw has no branding
<lifeless> just black on white text
<wgrant> offline-unplanned.html and offline-unplanned-haproxy.html?
<elmo> wgrant: right
<wgrant> The haproxy one has a comment.
<wgrant> So it should be easy to tell which is which.
<elmo> lifeless: like I say, AFAICS neither apache nor haproxy should do that for any of the main websites
<elmo> what was the URL you had failing?
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+filebug
<elmo> I'm also not seeing many 5*'s in the apache log
<wgrant> Er.
<wgrant> mtr shows some latency
<wgrant> Fluctuating.
<wgrant> In the DC.
<wgrant> 50-200ms
<elmo> wgrant: is fine from here (and I'm a couple of hundred miles away atm)
<lifeless> elmo: what are the two edge servers we have left?
<elmo> lifeless: potassium and palladium
<wgrant> elmo: It's settled down now, but for a while there was 200ms between chenet and nutmeg.
<wgrant> And the rest of the route was pretty stable.
<elmo> wgrant: that's just chenet depriotizing pings
<wgrant> Bah.
<elmo> it's a busy firewall...
<lifeless> its happier now than it was 3 hours ago
<lifeless> elmo: to humour me, which two edge servers do we have still live? and which DC are they in?
<elmo> 11:00 < elmo> lifeless: potassium and palladium
<lifeless> thanks, its late :)
<elmo> lifeless: they're in Goswell Road
<lifeless> is that the same one as apache / haproxy?
<elmo> no, different one
<lifeless> ok, I *know* I don't have enough data here, but my instincts are jumping on the inter dc link
<lifeless> What can we do to rule it out
<elmo> there's absolutely no evidence of a problem with the link
<elmo> our London stuff is relatively well spread out, if there were problems, more than just launchpad would be showing up and it would be in nagios
<elmo> the link is up, there's no problems with automated ping testing or manual testing, it's nowhere near capacity
<lifeless> was there anything ~ 3 hours back ?
<elmo> checking
<elmo> nothing shows in router logs or bandwidth graphs
<elmo> 3-4 hours ago is the sunday apache logrotate
<elmo> if you want to gut-blame something, I'd say that's a much better target
<lifeless> interesting
<lifeless> how long does that take?
<lifeless> or does it have a tendency to go skewiff
<elmo> the logrotate itself?  not very long but it does an apache reload/restart which sometimes goes mental on busy webservers
<lifeless> ah
<lifeless> I got back a partial page on one of the bugs I just filed as test
<lifeless> it cuts off right on the target table
<lifeless> elmo: so I don't quite know where to go; there *was* a serious persistent problem, its a lot better now but still a good fraction fof requests go into lala land, don't oops, but take ages (like 15 seconds, but reported as    At least 98 queries/external actions issued in 0.99 seconds
<lifeless> )
<elmo> i don't know where to go either, and I really need to run - unless we have some actionable next steps, I'd like to defer this
<lifeless> I think we have to
<lifeless> defer it
<elmo> ok, sorry I haven't been of much help
<elmo> I'm still available on my mobile if things get worse again
<lifeless> thanks
<lifeless> elmo: https://xmlrpc.edge.launchpad.net/bazaar/: Unable to handle http code 503: Service Unavailable
<lifeless> elmo: I've seen this a couple of times, someone in #bzr is asking now.
<lifeless> wgrant: grep -r 'except:' .
<lifeless> wgrant: then weep.
<lifeless> night all
<wgrant> lib/lp/buildmaster/manager.py:        except:
<wgrant> Eep.
<jelmer> 'morning lifeless, wgrant
<wgrant> (do we not have a zero-tolerance policy for this sort of thing?)
<wgrant> Morning jelmer.
<wgrant> Just got another 502 from edge.
<wgrant> Definitely from Apache.
<wgrant> Er.
<wgrant> I do believe I just got a truncated copy of the WADL.
<wgrant> Yes.
<wgrant> That is a little odd, since that's served fairly dumbly.
<StevenK> wgrant: Don't you have uni work to do on a Sunday night? :-P
<wgrant> Yes :(
<wgrant> But that doesn't stop my inbox from filling up with cron errors.
 * jelmer waves to StevenK
<StevenK> jelmer: O hai!
<lifeless> morning
<beuno> good morning lifeless
<lifeless> hi beuno
<mwhudson> morning
<lifeless> mwhudson: hu
<lifeless> mwhudson: hi
<lifeless> mwhudson: a) we have a small firefight going on, and b) I can has reviews? All ones you'll like, I swear. topic of launchpad-reviews
<mwhudson> lifeless: boo, and uh ok
<mwhudson> lifeless: what's the firefight?  i saw identi.ca/launchpadstatus
<lifeless> url is in the internal channel
<mwhudson> k
<wgrant> Morning lifeless.
<wgrant> Morning mwhudson.
<wgrant> Any news?
<lifeless> wgrant: some
<lifeless> elmo has been helping, but hes on a m*f* train.
<lifeless> when he surfaces again, we'll continue
<lifeless> wgrant: the only current theory I have is that a failed shutdown of the appservers leaves opstats working but everything else bork bork borked
<wgrant> Data I have: Pretty consistently broken from 02:00Z yesterday to 08:00Z, possibly OK for 3-4 hours, broken from 13:00Z to 18:00Z, possibly OK for a couple of hours, then broken since.
<lifeless> there is a bug about the failed shutdowns, but we'll need one about this failure mode of opstats
<lifeless> we have 2 appservers which are out of rotation but the process is running
<thumper> morning
<lifeless> hi thumper
<lifeless> wgrant: this theory doesn't quite explain the partial pages
<lifeless> wgrant: but if the broken appservers sometimes work, but haproxy kills it all when it decides the servers bong, that would explain it.
<wallyworld> morning
<wgrant> lifeless: Possibly.
<wgrant> lifeless: Also of note is the partial WADL that I got last night. So even really simple requests get truncated.
<lifeless> wgrant: that comes from the appservers still
<wgrant> It does.
<wgrant> But it should be really fast.
<lifeless> wgrant: and has been known to take minutes to generate.
<wgrant> Oh hmmmmmmm.
<lifeless> not everytime, of course. Just Some Times.
<wgrant> It's almost always truncated after ~15700 bytes.
<wgrant> It fluctuates, but that may just be header size differences.
<wgrant> Regardless of the file, it's within a kilobyte of that number.
<lifeless> elmo: ^
<lifeless> thats interesting
<lifeless> have you tried a loop on the wadl? if so what url are you using?
<wgrant> I haven't.
<wgrant> It needs some Accept header... let's see.
<thumper> wallyworld: morning
<wallyworld> thumper: looks like you had an interesting time on friday :-(
<thumper> wallyworld: FSVO interesting
<thumper> wallyworld: we should have a call
<wallyworld> thumper: ok, anytime
<wallyworld> thumper: https://code.edge.launchpad.net/~wallyworld/launchpad/link-bugs-in-merge-proposal/+merge/34826
<thumper> wallyworld: http://yuiblog.com/assets/pdf/cheatsheets/css.pdf
#launchpad-dev 2011-09-05
 * wgrant fires off a full test suite run in an oneiric container.
<wgrant> Hmm.
<wgrant> mailman does not like the distutils installation :(
<wgrant> Ahh.
 * G looks to see who the OCR is today
<wgrant> G: I'm not, but what needs reviewing?
<G> oh, not online yet
<G> wgrant: I've got a branch for Bug #697157
<_mup_> Bug #697157: Streamline "You have assigned this bug to yourself" message  <bugs> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/697157 >
<G> existing message for all cases except "You have assigned this bug to yourself" if you assigned it to yourself
<G> I'm just going to propose it
<G> wgrant: https://code.launchpad.net/~dev-nigelj/launchpad/bug-697157/+merge/74035
<wgrant> G: That seems like it will miss the case where it's assigned then unassigned quickly, once we move to queuing these (as we should). But you probably don't have enough information to detect that, so I guess it's reasonable.
<G> wgrant: hmmm I hadn't thought of that case
<G> wgrant: that said, wouldn't the method that e-mail is generated with change slightly if/when that happens
<G> I didn't poke too much into how the scripted bugmail is generated, so I'd have to take another look at that but just thinking along that line
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 254 - 0:[#######=]:256
<StevenK>  /wrists
<wgrant> And since we probably won't deploy for 48 hours or so, we should break the graph shortly :)
<G> ouch
<StevenK> wgrant: Declaring destroy-openidrp ready for a review. I will poke stub when he surfaces.
<wgrant> StevenK: But it doesn't have DB changes, does it?
<StevenK> wgrant: It does not, but stub is OCR. :_P
<G> I'm heading out, can someone ask stub to have a look at my MR?
<StevenK> wgrant: Do you think it's safe or unsafe to destroy the bazaar-experts team?
<wgrant> StevenK: Safe. We've deployed everything except librarian1 since then, AFAIK.
<wgrant> And if librarian1 needs bazaar-experts, then it should blow up, because something is seriously screwed.
<wgrant> wallyworld_: Have you tried to run tests on oneiric yet? rabbitfixture doesn't support rabbitmq-server 2.5.0.
<wgrant> I think I have a fix.
<wgrant> Need to test it on lucid too, though.
<wallyworld_> wgrant: i've only run some yui tests and they went ok
<wallyworld_> but nothong with rabbit
<wallyworld_> wgrant: i'm pissed off ay qas. it keeps timing out for a simple operation :-(
<wgrant> :(
<wallyworld_> hard to know whether to qa-ok it. the sql is fine
<wgrant> wallyworld_: Is this the bug privacy thing?
<wgrant> That's not a simple operation :)
<wgrant> Aha, got a rabbitfixture regex that works on both lucid and oneiric.
<wgrant> But is a bit evil.
<wgrant> Test attempt #2
<wallyworld_> wgrant: yes, the bug privacy thing. the sql before and after the change is similar, except the fix has a couple of extra selects to update the subscription portal.
<wallyworld_> wgrant: it works fine if i edit a bug that's on a smaller project than lp
<wallyworld_> so i'm marking it as ok
<wallyworld_> but, in testing, i found a bug that i confirmed is also on lp.net
<wallyworld_> but it's not a result of any disclosure work so i'm not sure if we'll end up fixing it
<wgrant> wallyworld_: What's the bug?
<wallyworld_> wgrant: bug 841511
<_mup_> Bug #841511: javascript error when changing a bug's security status <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841511 >
<wallyworld_> wgrant: it's in the same area of code as i was qa'ing so i had to look at diffs and also try it out on lp.net and found that it's an unrelated problem
<wallyworld_> i should just assign myself to the bug and fix it
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 255 - 0:[#######=]:256
<StevenK> Someone promote a high to critical ...
<wgrant> I'm just looking for one.
<nigelb> Morning
<nigelb> 255? :(
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 257 - 0:[########** stack smashing detected : ./lp terminated
<nigelb> BAH!
 * ajmitch thought that number was meant to be going down
<nigelb> Actually, I'm not sure if 256 was right.
<nigelb> Because there were bugs fixed and newer bugs added.
<wgrant> http://webnumbr.com/launchpad-critical-bugs is what that number goes by. I just added two because I bumped two OOPSes that had been forgotten to critical.
<wgrant> The real number is significantly higher, as several are private.
<nigelb> No, I meant the 256 that acted as the target from which you startred
<wgrant> Ah/
<nigelb> Now, it seems as though there was no progress.
<nigelb> That's bad.
<wgrant> There was no progress; that is bad :)
<nigelb> Huh?
<nigelb> But Francis had this graph which said there were bugs closed as well as new ones opened.
<wgrant> Well, yes, lots were closed. But lots more were found, and not all of them were new.
<nigelb> I refuse to believe there was no progress. You guys aren't /that/ bad ;)
<poolie> there has been progress, things have improved, but they haven't progressed in the intended direction of reducing the number of criticals
<nigelb> Ah :(
<G> so what, focus urgently on Critical bugs eh?
<nigelb> I spent two hours on this bug yesterday, 66345
<nigelb> eventually I realized its probably a bug in that library.
<wgrant> Bug #66345
<_mup_> Bug #66345: alt="" in dependency charts makes them poorly accessible <easy> <lp-blueprints> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/66345 >
<wgrant> Hmm.
 * StevenK looks for a critical, since the DSP vocab work is still blocked, and his two other branches are waiting for others
 * G spent ages trying to understand the API to fix a Medium bug, must try again now
<StevenK> Isn't bug 669296 mostly fixed?
<_mup_> Bug #669296: App servers die and hang <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/669296 >
<wgrant> StevenK: three died over the weekend.
<wgrant> But it is much better now that logrotation is fixed.
<wgrant> First time it's happened in weeks, I think.
<wgrant> Rarrrgh buildbot is broken again.
<wgrant> Tempting to roll back the three bzr revisions from the last couple of weeks.
<wgrant> Hmm. Except that this looks like a dependency issue.
<nigelb> Is this okay to fix? bug 99593
<_mup_> Bug #99593: Package names in search results should be linked <bug-columns> <easy> <lp-bugs> <search> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/99593 >
<nigelb> I'm not sure if I want it fixed :D
<wgrant> nigelb: The next scheduled feature involves reworking bug listings.
<wgrant> Ah, and it's even tagged (bug-columns).
<wgrant> So I would avoid it.
<nigelb> okay :)
 * nigelb is /very/glad he asked
<nigelb> I can't reproduce bug 750384 in my setup.
<_mup_> Bug #750384: After muting with Bug:+mute, you get redirected to +subscribe <story-better-bug-notification> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750384 >
<nigelb> (devel)
<nigelb> Could someone else confirm so I can close it.
<G> yeah I looked at that one
<G> can't reproduce either
<nigelb> Excellent.
<G> I meant to note it, but couldn't help but think I was doing something majorly wrong :)
<StevenK> poolie: O hai -- you reviewed the Twisted patch for bug 703807, should we cherry-pick it, or wait for a upstream release?
<_mup_> Bug #703807: launchpad sometimes serves download files as content-type text/html <regression> <Launchpad itself:Triaged> <pyOpenSSL:Invalid> <Twisted:Fix Released> <twisted-web (Ubuntu):Confirmed> < https://launchpad.net/bugs/703807 >
<poolie> hi stevenk
<StevenK> I have this feeling that there are multiple bugs about that issue
<poolie> good question
<poolie> well, it has certainly been duped several times
<poolie> and perhaps caused some confusion or questions short of filing a bug
<poolie> so python at the moment comes from an egg?
<poolie> stevenk i guess the questions are
<poolie> - how soon will we get an upstream release
<StevenK> bug 174224 looks like a dupe
<_mup_> Bug #174224: launchpadlibrarian sends wrong content-type header <librarian> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/174224 >
<poolie> - how much do we care about getting it done soon vs later
<poolie> - how much more work is it to cherry-pick?
<StevenK> possibly bug 645924
<_mup_> Bug #645924: .tar.xz files are served with text/plain content-type by launchpadlibrarian <content-type> <launchpadlibrarian> <lp-foundations> <mime-type> <xz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/645924 >
<wgrant> StevenK: None of those are dupes.
<poolie> in the thing i helpde fix in twisted, it would specifically give you text/html i think
<wgrant> Right.
<poolie> i suspect the first of them is just obsolete
<StevenK> poolie: So, I don't think we can answer how soon we get an upstream release; I'm not sure about making a cherry-pick release of Twisted ...
<StevenK> It is easier to just wait
<nigelb> Can someone help me understand bug 294050?
<_mup_> Bug #294050: Please add links for entries in the revision feed for a person <feeds> <lp-code> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/294050 >
<nigelb> I think it means the feeds in branches.
<nigelb> But I already see links to loggerhead
<poolie> whoa, feeds
<nigelb> heh, that's not improving my sinking feeling :)
<poolie> so, i can help you understand it
<nigelb> Excellent.
<StevenK> OH! I know what I can remove
<poolie> which is that basically it's about inserting a hyperlink, i guess pointing to loggerhead, into the revision feed data
<StevenK> DELAYED COPIES
<poolie> however, i don't know if revision feeds are a very useful feature
<poolie> i never look at them any more
<nigelb> poolie: I already see it everywhere I look.
<poolie> oh, in the feed?
<nigelb> yeah
<poolie> then maybe you should just close this
<nigelb> so, is it something wrong in how I perceive the bug.
<nigelb> For example, in here I can see links http://feeds.launchpad.net/~launchpad-pqm/launchpad/devel/branch.atom
<nigelb> Aha, let me close the bug then :)
<poolie> go you
<poolie> StevenK, i can still reproduce bug 645924
<_mup_> Bug #645924: .tar.xz files are served with text/plain content-type by launchpadlibrarian <content-type> <launchpadlibrarian> <lp-foundations> <mime-type> <xz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/645924 >
<poolie> i suspect it actually does have c-t text/plain in the librarian
<StevenK> poolie: What's the LFA ID?
<poolie> i guess it's 56261240 from the url?
<StevenK> SELECT mimetype from libraryfilealias where id = 56261240; => text/plain
<poolie> yep
<wgrant> So the sniffer is (or was) bad.
<poolie> so
<poolie> if you add a new upload what does it get?
<wgrant> I don't recall if we trust the browser or not.
 * poolie looks at other .tar.xz files
<StevenK> Hold on, I can do it faster
<poolie> i'm looking through the staging db
<poolie> so, only if you can type faster
<poolie> or know the schema better
<poolie> either is possible
<StevenK> I have a query running, so let's see :-)
<StevenK> SELECT distinct(mimetype) from libraryfilealias where filename like '%.tar.xz';
<poolie> almost all are application/octet-stream, it seems.
<StevenK> application/octet-stream and text/plain
<G> I've been looking at https://bugs.launchpad.net/launchpad/+bug/674854 can't help notice that https://api.launchpad.dev/devel/ubuntu/hoary/architectures is a bit funny... total_size: 3, entries: []
<_mup_> Bug #674854: [API] Add a way to get the list of "supported" architectures for each release <api> <lp-soyuz> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/674854 >
<StevenK> xz doesn't exist in /etc/mime.types
<StevenK> application/octet-stream is debatable, but loads better than text/plain
<wgrant> G: DistroArchSeries doesn't have a launchpad.View permission defined.
<G> wgrant: ahhh thanks
<wgrant> G: Since they're all public, they need an adapter class deriving from AnonymousAuthorization (in canonical.launchpad.security)
<wgrant> G: There's possibly already a bug for this.
<wgrant> G: (they'll already appear if you're logged in, just not to anonymous users)
<poolie> anyhow, it's not a dupe
<poolie> if launchpad is sniffing the type, it seems to be mostly getting it right now
<G> wgrant: apparently not
<StevenK> poolie: So, we can probably correct that mimetype and close the bug, then
<G> wgrant: http://pastebin.ubuntu.com/682351/ - but I will look at existing DistroARchSeries bugs
<poolie> i guess
<poolie> just fix it in the db?
<poolie> ironically that really may interact with the caches
<poolie> because this is exactly the unusual cache of the ct changing without the content changing
<poolie> so it will probably stay cached for a while
<poolie> arguably there's a bug here if lp trusts the client's ct even when it shouldn't
<poolie> StevenK: so how would we handle a cherrypick to something that currently comes from an egg?
<G> wgrant: oh I get you now
<G> wgrant: thanks for the pointer :)
<StevenK> poolie: We make a new egg and update the version in versions.cfg
<poolie> called 0.12.0+0launchpad1 or something?
<poolie> ok
<wgrant> StevenK: Aha, I know how to fix the Jenkins failure, and I'm just going to redisable the failing buildbot test (it was only reenabled a weekish ago)
<wgrant> G: Great. If you run into any issues, ask :)
<G> wgrant: yeah, although I do like to have a big play w/ stuff before asking, because if it is a simple thing, I learn other stuffs/tidbits about the code by stuffing up :)
<StevenK> poolie: I hope not. :-)
<StevenK> poolie: Since the version of Twisted we have currently is 11.0.0 :-)
<poolie> you know what i mean
<StevenK> We have done cherry-picks of Twisted before, I just can't remember their format
<G> wgrant: so really, http://pastebin.ubuntu.com/682356/ is all that is needed if I understand it correctly
<wgrant> jelmer: Ah, worked out that bzrdir import error
<wgrant> jelmer: The daemon was old, from a previous test run.
<wgrant> jelmer: So its tree had been deleted.
<wgrant> G: You're missing a newline, but yup.
<G> missing, don't you mean included by accident?
<poolie> StevenK: so can i leave that with you?
<poolie> i'm going to try to fix bug 643223 this afternoon
<_mup_> Bug #643223: should accept dkim based on from address and signing address belonging to the same person <dkim> <lp-foundations> <mail> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/643223 >
<wgrant> G: Two empty lines between classes.
<G> wgrant: oh good point, thanks :)
<G> I am also missing something else...
<G> a test!
<wgrant> Mmm.
<wgrant> Hah. Old bug for test_import_bzrsvn's disablement was 541526. New one is 841556. A bit too similar...
<poolie> 4 out of 14 sampledata people still work at canonical
<poolie> not counting foo bar
<nigelb> is that good or bad? :)
<poolie> i don't know
<poolie> it's pretty old
<poolie> not too bad
<poolie> i know all but 2 of them
<nigelb> sampledata = stuff that rf installs onto my machine right?
<poolie> yes
<poolie> database/sampledata
<nigelb> I only saw mark as a familiar name
<StevenK> No, the sampledata is stuff that make schema stuff into the development database
<StevenK> rf installs the developer dependencies
<nigelb> Ah.
<poolie> well, strictly, make schema not rf, yes
<nigelb> But poolie got what I meant :)
<nigelb> Hrm, today's Google Doodle is pretty awesome.
<G> I don't know, I haven't been impressed by Google's Doodle's for a while
<poolie> i had no idea he was Parsi
<nigelb> Neither did I :)
<G> hmmm think I may be able to put another bug in the 'fixed' basket
<poolie> stevenk, wgrant, in a test, i need a user account with multiple email addresses in different domains
<poolie> what's the tasteful way to get this?
<wgrant> makePerson()
<wgrant> makeEmailAddress()
<wgrant> makeEmailAddress()
<wgrant> makeEmailAddress()
<G> one at canonical.com one at ubuntu.com in makePerson?
<poolie> on TestCaseWithFactory?
<G> poolie: yeah, there is another test like that, that I was reading
<StevenK> poolie: self.factory.makePerson() ...
<poolie> thanks
<poolie> am i right that in general you prefer tests to always make the person (or whatever) rather than relying on the sample data?
<wgrant> poolie: Yes.
<wgrant> poolie: It is somewhat slow, but means we're not relying on fragile, rotten sampledata.
<wgrant> Which is largely invalid in great parts.
<StevenK> I'd like to remove most sampledatsa
<StevenK> s/sa/a/
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 258 - 0:[########*** stack smashing detected ***: ./lp terminated
<wgrant> 258 :(
<StevenK> What is the most recent?
<wgrant> Bug #841556
<_mup_> Bug #841556: test_import_cvs, test_import_bzrsvn and test_import_subversion disabled <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841556 >
<G> wgrant: I got it! http://bazaar.launchpad.net/~dev-nigelj/launchpad/bug-674854/revision/13863 allows me to do: http://pastebin.ubuntu.com/682386/
<G> wgrant: thanks massively for your help
<StevenK> poolie: This cherrypick is not so easy, since the diff from trac won't apply at all to 11.0.0
<poolie> my patch for 721166 passed an ec2 test
<poolie> bug 721166
<_mup_> Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <spurious-test-failure> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/721166 >
<poolie> should i land it?
<poolie> i think the tests can now be reenabled since lp has bzr 2.4
<StevenK> Didn't wgrant just roll that back?
<poolie> rolled back bzr?
<wgrant> I didn't roll back bzr.
<wgrant> I redisabled the undisabled tests.
<poolie> the ones that gary changed?
<poolie> they might have been different
<wgrant> Nah, different ones.
<wgrant> Did he actually end up disabling any?
<wgrant> I think the bzrdir import issues were due to the three I just disabled.
<wgrant> Leaving ancient processes around.
<poolie> so my question is, really
<poolie> if i land this, is there an unreasonable chance that either bzr will be downgraded or spurious failures will disrupt things?
<wgrant> 15 minutes ago the chance of a spurious failure disruption was probably around 0.95.
<wgrant> Now it's hopefully lower.
<wgrant> But please avoid any even slightly disruptive changes until late in the week.
<wgrant> We've just had about 20 failures.
<poolie> ok, there's no rush
<poolie> it's only a tidying-up fix
<wgrant> What with the three invasive bzr infrastructure changes lately, it's a bit of a mess.
<poolie> oh, what were the three?
<wgrant> There was the 2.4 upgrade, the puller SafeBranchOpener unification, and some other codeimport/puller change, the details of which I forget.
<poolie> did they break a lot of stuff?
<wgrant> It's not clear.
<wgrant> Because also around that time three tests were undisabled.
<wgrant> They spuriously failed in spectacular ways occasionally.
<wgrant> But then lots of other tests blew up in even more spectacular ways.
<wgrant> but I speculate that that is because of debris left by the original three broken tests.
<wgrant> So I've disabled them, will get buildbot cleaned up, and then hope that stuff stops failing dammit.
<wgrant> Otherwise I'll just revert all the bzr changes in the last two weeks and work out wth is going on.
<poolie> StevenK: you could just take my patch from http://twistedmatrix.com/trac/ticket/4156#comment:7
<wgrant> StevenK: How do I restart a Jenkins build? Just kill the executor and force?
 * wgrant tries.
<LPCIBot> Project devel build #1,031: ABORTED in 21 min: https://lpci.wedontsleep.org/job/devel/1031/
<wgrant> We may have success in 4 hours.
<LPCIBot> Project devel build #1,032: STILL FAILING in 6 min 34 sec: https://lpci.wedontsleep.org/job/devel/1032/
<wgrant> Lies.
<wgrant> urllib2.URLError: <urlopen error [Errno 2] No such file or directory: '/mnt/test/launchpad/workspace/devel/download-cache/dist/setuptools-0.6c11-py2.6.egg'>
<wgrant> How unpleasant.
<wgrant> StevenK: You broke everything.
<wgrant> Seems I can't destroy the executor.
<wgrant> :(
<LPCIBot> Project devel build #1,033: STILL FAILING in 39 sec: https://lpci.wedontsleep.org/job/devel/1033/
<wgrant> :(
<G> ouch, that doesn't sound good
<wgrant> Well, the test suite has sped up by 360x. That can only be a good thing, right? :)
<rvba> hehe
<G> eek GPG tests just OOM'd in my VM
<wgrant> How much RAM is assigned?
<G> and that is 2GB RAM
<wgrant> amd64?
<wgrant> There may be left over librarians and stuff eating a couple of hundred megabytes each.
<wgrant> amd64 with 2GiB is doable, but pushing it.
<G> yeah, amd64
<G> I might download lucid-server i386 and put on that VM tbh
<G> laptop is Core2Duo w/ 4GB
<wgrant> My laptop with the same specs runs it OK outside a VM, but if I want to do it in a VM i386 is better.
<nigelb> Is bigjools going to be around today?
<nigelb> There's a bunch of bugs that I want to talk to him first before touching :)
<adeuring> good morning
<wgrant> nigelb: I think he should be.
<wgrant> let me check.
<G> wgrant: yeah, don't blame you for using i386
<wgrant> nigelb: Looks like he should be here today.
<rvba> nigelb: Yes, he will be here today.
<nigelb> wgrant / rvba: Thanks!
<LPCIBot> Project devel build #1,034: STILL FAILING in 44 sec: https://lpci.wedontsleep.org/job/devel/1034/
<G> wgrant: fast but still failing?
<wgrant> Yeah, the slave is broken.
<wgrant> Need StevenK to fix.
<StevenK> You do?
<wgrant> StevenK: I can't work out how to kill the executor.
<wgrant> Only the build.
<poolie> would it be tasteful to add a script to lp that just processes one mail message from stdin?
<poolie> at least for the sake of testing
<StevenK>  Oh. Delete the slave.
<poolie> it seems a lot easier than configuring a mailbox
<wgrant> poolie: It is distasteful to not have one.
<wgrant> StevenK: warthogs doesn't seem to have permission to do that.
<poolie> or, to add a parameter to process-mail to do this (though that seems a bit hard)
<wgrant> Or at least I can't find a relevant button.
<poolie> :)
<StevenK>  Navigate to the slave, hit the button on the left.
<wgrant> https://lpci.wedontsleep.org/computer/i-225ca042/ has only four links on the left.
<wgrant> Back to List
<wgrant> Status
<wgrant> Build History
<wgrant> Load Statistics
<poolie> wgrant, ok, will do
<poolie> we could potentially even get the mta/mda to feed messages directly in to this
<poolie> though startup time may make that suck
<wgrant> "may"
<poolie> well
<poolie> will it be worse than the current latency from running from cron?
<poolie> and, will the wasted cpu matter?
<poolie> i don't know how busy the machine that currently does this is, and how much mail it gets
<wgrant> It's about 10s of CPU time per invocation.
<wgrant> That's not insignificant.
<poolie> true
<poolie> especially if we get mail more often than every 10s during peak time
<wgrant> Yes.
<poolie> yes we do?
<wgrant> And if someone sends an email to a few bugs, the world ends.
<poolie> hmm
<poolie> if the mail is exploded apart before it gets to us
<wgrant> It used to be frequent to email hundreds of bugs at once, but that is probably reduced by the API.
<poolie> which it probably wouldn't be
<nigelb> Is it known that loggerhead fails on first try to see launchpad code?
<poolie> but could be
<wgrant> Wouldn't it be?
<wgrant> nigelb: Yes.
<poolie> at any rate it's not optimized
<poolie> i mean, it's not worth optimizing
<wgrant> poolie: Unless it's treating process-mail as a relay, it should be exploded.
<nigelb> wgrant: Loggerhead suckage?
<poolie> yes, just that
<poolie> it needs to warm the cache
<wgrant> nigelb: Yeah, plus LP is insanely huge.
<G> bigjools: hey, is https://code.launchpad.net/~dev-nigelj/launchpad/bug-674854 anything like what you were thinking of to fix bug 674854?
<_mup_> Bug #674854: [API] Add a way to get the list of "supported" architectures for each release <api> <lp-soyuz> <trivial> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/674854 >
<nigelb> And, looking the launchpad devel branch also timesout on first try.
<nigelb> Probably related I guess.
<bigjools> G: why did you need the security adapter change?
<bigjools> did anon access not work?
<G> bigjools: correct
<G> it would reply w/ count: 3, entries: []
<wgrant> bigjools: Remember that lazr.restful filters items out of collections unless launchpad.View is held.
<wgrant> bigjools: And DAS has no launchpad.View.
<bigjools> G: ah bugger.  Well the bug is essentially already fixed, you could note that and/or file a new one.
<wgrant> So it reverts to the default one, which is checkAuthenticated -> True, checkUnauthenticated -> False
<lifeless> wow, 2 non-staff contributors. Excellent :)
<G> wgrant: it didn't even work when authenticated
<bigjools> lazr.restful is teh awsum
<nigelb> lifeless: both of us even have the same first name ;)
<wgrant> Evening lifeless.
<lifeless> 'lo y'all :)
<nigelb> (For added confusion)
 * bigjools disappears for calls and mountains of email
<G> bigjools: http://pastebin.ubuntu.com/682351/
<G> lifeless: howdy
<lifeless> I saw a mountain of cron spam this morning
<wgrant> lifeless: Chameleon has that effect on people :(
<nigelb> lifeless: Glad to hear that everything is now fine :)
<wgrant> The broken eggs need fixing :(
<lifeless> wgrant: these are setuptools non-dpkg eggs again ? -how-
<wgrant> lifeless: Well, it changed from a directory to a symlink.
<wgrant> And dpkg doesn't seem to always handle that nicely.
<wgrant> But sometimes it does.
<wgrant> And blah.
<wgrant> rm -rf, apt-get install --reinstall, problem solved.
<wgrant> But it is way down the queue :(
<poolie> i have a process-one-mail.py
<poolie> that's so much better
<nigelb> This reminds me.
<StevenK> wgrant: Do I still need to delete that slave?
<nigelb> How does one do QA of something email related on qastaging?
<wgrant> StevenK: Please.
<StevenK> nigelb: One panics.
<wgrant> Although it seems to be gone.
<wgrant> If you haven't already killed it.
<nigelb> StevenK: Test in production? :P
<StevenK> wgrant: If the slave sits unused for 30 minutes, it gets killed
<nigelb> We tested summit in production /at/ UDS :D
<bigjools> nigelb: ask someone with access to the staging inbox to look at it for you
<wgrant> StevenK: Ah, that would do it. Thanks.
<nigelb> bigjools: Great, that means anyone from LP team right?
<bigjools> nigelb: not everyone IIRC, all squad leads and assorted others
<wgrant> I think it's meant to be squad leads. But most people seem to know the password.
<nigelb> Anyway, I need to get that branch merged.
<nigelb> Testfix mode was blocking it.
<nigelb> I don't know if that's done yet.
<wgrant> I think I've fixed it all. But the current build is missing the last two revisions. If we are lucky we will be green in 3 hours. If we're not, it will be 8 hours.
<wgrant> And I may of course be wrong, and we will still be red in 8 hours.
<nigelb> Oh good, so by the time I'm home.
<wgrant> In which case I will be sad.
<wgrant> And hope that gary fixes it before I awake.
<nigelb> BUt you won't sleep anyway.
<nigelb> :D
<poolie> woo, locally accepted second-address signed mail
<poolie> that feels like a much better test
<lifeless> bigjools: inbox is accessible to all canonical I think :)
<G> bigjools: yeah, just tested again, the <distro>.current_series.architectures method against devel definately returns no entries, when authenticated, but this change allows it to happen, but are you saying it should be a seperate bug?
<G> (Just a tad confused, because this seems to be the last bit a puzzle (even if the majority of the work was done prior) to get what the description etc lays out
<StevenK> stub: O hai!
<poolie> wgrant, well, that script is added in https://code.launchpad.net/~mbp/launchpad/643223-dkim-aliases/+merge/74061 plus some other stuff
<poolie> i'm happy to hold off landing it until things are safe enough
<StevenK> stub: Can you please look at the SQL in bug 834384 and comment in the bug about it?
<_mup_> Bug #834384: StaticDiff is unused and needs to be shot <Launchpad itself:Confirmed> < https://launchpad.net/bugs/834384 >
<stub> StevenK: commented
<bigjools> G: that is a separate bug because it seems the main one was fixed some time ago
<bigjools> not sure why it's not returning anything even when authenticated
<bigjools> I'd hope there was a test for this somewhere!
<G> bigjools: couldn't find any, which is why my branch includes a test for this logic
<G> bigjools: so what, I close the current bug, open a new one tagged regression?
<bigjools> G: yes, I think so.  I'd check to see if there's a passing test for DS.architectures as well.
<G> bigjools: yes, that is what the test, is testing
<bigjools> apart from you new one I meant :)
<bigjools> your*
<G> that the number of entries returned via the API is the same as in the DB
<G> oh, I couldn't find any other tests that seemed to be testing it
<G> bigjools: oh funny. I tried again, and it worked for an authenticated launchpadlib setup this time
<G> so obviously it was just never enabled for anonymous
<bigjools> hah
<G> I was going through everything again unauthed, then authed to make double sure, and for clean output for the bug
<G> so I guess less regression, more oversight/accident?
<bigjools> could be an out of date wadl
<bigjools> cache
<G> this dev environment has only existed for half a week
<bigjools> oh you were testing against a .dev instance?
<poolie> does anyone know how i could get a gpg key into a dev instance?
<G> bigjools: yeah
<poolie> maybe poking it into the db
<wgrant> poolie: utilities/start-dev-soyuz.sh, push key to keyserver.launchpad.dev, add key as normal.
<G> bigjools: maybe I originally took too long to validate the OAuth session or something the first time round :)
<poolie> thansk
<poolie> *thanks
<poolie> and apparently i'm going to need to do a mail roundtrip to confirm it?
<wgrant> Or just grab the token from logintoken;
<wgrant> Then go to /token/$WHATEVER, and confirm.
<G> bigjools: alright, so I'll close 674854, and the new bug will be that it doesn't allow anonymous access :)
<bigjools> G: great!#
<poolie> just manually testing i haven't broken that
<poolie> s// gpg signing
<G> bigjools: okay, done, I'll propose the branch for merging now under the new bug
<bigjools> today's Google Doodle is the best ever
<poolie> is there going to be an ocr today?
<poolie> o/ bigjools
<nigelb> bigjools: I agree :)
<bigjools> o/ poolie
<bigjools> nigelb: is your 'tache in honour of Fred? :)
<nigelb> haha, no
<nigelb> hrm, but I can claim so now :P
<G> bigjools: I guess I'm too young to know much about Freddie Mercury the doodle is fancy, but yeah...
<nigelb> Also, pep8 and pyflakes are awesome. I just started using it on all my python code after getting used to make lint :D
<bigjools> nigelb: use vim?
<G> any yeah, no OCR today?
<nigelb> yeah, I'm guessing there's a plugin as well?
<bigjools> G: let's just say he was the frontman for the biggest rock band of all time :)
<bigjools> nigelb: there is and it's teh awsum
<nigelb> Oh good. let me find and isntall.
<G> bigjools: whats a good song of his band's then, I might look at it on YouTube or something
<StevenK> G: Are you serious?!
<bigjools> !
<bigjools> G: you've heard of Queen, right?
<G> yeah... Queen
<nigelb> G has successfully made everyone feel old I believe.
<G> Another One Bites the Dust, We Will, We Will, Rock You...
<StevenK> Bohemian Rhapsody?
<bigjools> nigelb: you're a whippersnapper yourself
<G> yeah...
<StevenK> I don't feel old, just surprised that someone doesn't recognise Freddie
<G> ohhhh he was one of the main guys?
<G> haha, never knew
<nigelb> bigjools: Hey, but I've heard of Freddie.
<StevenK> G: He was the vocalist until his death
<G> I'd heard of Freddie, but never knew the Freddie<->Queen connection
<StevenK> G: For David Bowie's 50th birthday concert, they had to get a women to sing Freddie's part of Under Pressure
<G> hmmm according to Wikipedia, I was born before he died
<G> so I guess that is something
<nigelb> G: How old are you again?
<G> learn something new every day eh
<G> nigelb: > 21
<nigelb> Hrm, we may be around the same age :D
<G> heck, I'd never seen the Matrix films and the Back to the Future films until a few years ago
<nigelb> ....
<nigelb> G: Star War / Star Trek?
<nigelb> *Star Wars
<G> nigelb: I've seen a couple of Star Wars films, didn't really like them that much, and only really seen the latest Star Trek movie (which I thought was okay), but I'm not generally a big fan of the whole Sci-Fi genre
<StevenK> G: Heathan.
<StevenK> Sigh. Heathen, even.
<G> StevenK: it's just not my thing
<bigjools> if it was two from episodes 1-3 them I am not surprised, I didn't like them either :)
<G> The Phantom Menace was one of the ones I know I've seen
<bigjools> awful movie
<G> but anyway, don't hold it against me :)
<bigjools> go and dig out the original 1977 Star Wars, it's great.
<nigelb> Amen
<nigelb> I watched the entire Star Wars series in one day from 8 am to 8pm (I was sick and had nothing else to do)
<G> bigjools: is http://www.fatso.co.nz/Movies/Star_Wars_-_Episode_IV_A_New_Hope_1977/5552 the one you were talking about?
<bigjools> G: yes
<G> okay, it's in my queue, might even get it Wednesday
<bigjools> rock on
<StevenK> nigelb: More impressive if you do it with LOTR
<G> meh, Demand: High :(
<bigjools> "PG - Contains medium level violence" - heh
<nigelb> StevenK: I didn't like LOTR for some reason.
<StevenK> nigelb: *Out*
<G> bigjools: gotta love the NZ ratings :)
<nigelb> I tried multiple times to get myself to watch it
<nigelb> Or read the books
<G> I'm a Kiwi, I watched all 3 LOTR movies, and didn't really like it, much prefer the books :)
<bigjools> I faltered half way through the second LOTR book.  But then I was 13 when I tried to read it
<StevenK> I managed all 3 when I was 15
<G> yeah, thats around when I managed to get them all ready
<G> *read
<StevenK> G: Out of interest, North or South?
<G> StevenK: Auckland
<G> ahh I see, you are Sydney
<bigjools> jafa!
<G> yep, born & raised
<G> although I spent a bit of time in Brisbane
<nigelb> jafa?
<nigelb> bigjools: Wait, your Australian as well?
<G> nigelb: Just Anotherl F'ing Aucklander
<nigelb> haha
<bigjools> nigelb: no, Brit.
<G> Jaffa on the other hand, is a nice lolly
<nigelb> bigjools: That's what I thought as well :)
<gmb> bigjools: The comments on https://bugs.launchpad.net/launchpad-buildd/+bug/840055 seem to suggest it's not actually a bug but is user error of some sort. Can you cast your weather eye over it and tell me if that's the case please?
<_mup_> Bug #840055: strange build loop with binutils 2.21.53 <Launchpad Auto Build System:New for dns> < https://launchpad.net/bugs/840055 >
<bigjools> gmb: sure
<gmb> Ta
<bigjools> gmb: it's a buildd-manager bug
<bigjools> and a dupe
<gmb> bigjools: Ah, thanks.
 * bigjools plays hunt the dupe
<bigjools> gmb: I suspect it's bug 717969
<_mup_> Bug #717969: storeBuildInfo is sometimes ineffective <boobytrap> <buildd-manager> <Launchpad itself:Triaged> < https://launchpad.net/bugs/717969 >
<gmb> bigjools: Thanks, I'll mark it as such and include this commentary.
<bigjools> ta
<wgrant> bigjools: A generic "audit" sounds a lot too much like "karma"
<StevenK> gmb: O hai, we seem to be Henning-less today. Would you mind reviewing https://code.launchpad.net/~stevenk/launchpad/destroy-openidrp/+merge/74007 ?
<bigjools> completely different
<wgrant> bigjools: Similarly FK-less and prone to weak keys.
<gmb> StevenK: I can't look right now, but I'll try to take a look this pm.
<wgrant> Unless treated very carefully.
<StevenK> gmb: Just to sweeten the deal, the line count is +2 / -283
<G> oh can someone comment on my suggestions for Bug 306569?
<_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/306569 >
<stub> Somewhere buried in our wiki there will be a document documenting how to rollback revision and revert rollbacks. But I know not where.
<wgrant> stub: You've fixed pgbouncer?
<stub> That will be part of this branch, yes
<wgrant> stub: You're hacking PATH? :/
<stub> No, I'll install a new .egg
<wgrant> Oh.
<wgrant> Right.
<wgrant> So, just do a cherrypick of the original revision, or an inverse cherrypick of my rollback.
<wgrant> bzr merge -r1233..1234 .
<stub> Any magic when it needs to go to pqm?
<stub> Why did we decide a hard coded path was best btw? That is making the module Ubuntu/Debian specific.
<lifeless> just add /usr/sbin to PATH in the subprocess call
<lifeless> subprocess has glue for that
<lifeless> and it won't make it so specific
<stub> k
<nigelb> gmb: Thanks! (re:spamming user)
<gmb> nigelb: No worries. I enjoy taking the loving mallet of correction (as John Scalzi would call it) to spammers.
 * gmb -> lunch
<nigelb> hehe
<wgrant> jtv: Hm, that's upsetting.
<jtv> wgrant: what is?
<wgrant> jtv: That the rabbitfixture fix didn't fix/
<jtv> Ah that.  Yes.
<wgrant> Can you pastebin 'sudo rabbitmqctl status'?
<jtv> wgrant: http://paste.ubuntu.com/682518/
<wgrant> Baaaah.
<wgrant> It's like postgres.
<wgrant> It only quotes when necessary.
<wgrant> Rip the quotes out of the first line of the regex. Should work then.
<wgrant> Or ? them, as desired.
<jtv> wgrant: quotes?  I only see one quote on that line.  And a few more in other lines.
<jtv> Making what look to be the opening and closing quotes optionalâ¦
<jtv> âand that does the trick.  Thanks wgrant!
<wgrant> The first line of the original uncommented version of the regex that I never pastebinned, come on, read my mind.
<jtv> Not on Mondays, you know that.
<jtv> Review needed!  Any takers?  https://code.launchpad.net/~jtv/launchpad/pre-832647/+merge/74087
<G> good point, I've got two MPs for review when there is an OCR around :)
<StevenK> I have one as well
<StevenK> G: I bet my MP removes more code than yours. :-P
<G> StevenK: you can be sure of it :)
<G> hold on. is this a "My MP is bigger than yours" thing? :)
<jtv> StevenK: how about we trade reviews and then take one of G's each?
<nigelb> bigjools: I think there's something ajax that changes the links from blue to green.
<wgrant> nigelb: You can't actually see the links he's talking about.
<wgrant> nigelb: They are indeed blue and un-spinnered.
<nigelb> wgrant: Why can't I see those?
<nigelb>  :(
<wgrant> They were mostly hacked up during the thunderdome, and are enabled only for ~launchpad right now AFAIK.
<nigelb> Also GAH for these things.
<nigelb> Can I see it on my dev instance?
<wgrant> Sure, if you push up a branch, go to https://launchpad.net/+feature-rules, add 'code.ajax_revision_diffs.enabled default 0 on', and view the MP page.
<wgrant> s/net/dev/
<nigelb> okay :)
<wgrant> https://launchpad.dev/+feature-info describes all the current feature flags.
<nigelb> QAing this will be fun.
<jtv> StevenK: I reviewed yours.  Will you review mine?
<wgrant> StevenK: Looks like Jenkins will succeed in 10 minutes.
<wgrant> With a little luck buildbot will follow.
<cjwatson> RT#47040 fixed - does that mean I can have another go at landing multiarch-translations?
<_mup_> Bug #47040: while booting initial kernel, progress bar looks like Win98 <gfxboot-theme-ubuntu (Ubuntu):Confirmed> < https://launchpad.net/bugs/47040 >
<cjwatson> or do I still need to wait for bug 809123 to actually go fix-released?
<_mup_> Bug #809123: we cannot deploy DB schema changes live <fastdowntime> <qa-ok> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/809123 >
<wgrant> As of a few hours ago we are, roughly, fastdowntime-capable. But we haven't actually done one yet.
<cjwatson> I suppose that multiarch-translations-schema (landed on db-devel a while back) is the thing that needs to be fastdowntimed
<wgrant> There is an as-yet undeployed Storm reconnection fix that we probably want to wait for, but if all goes to plan (as it hasn't for the last week) we'll be deploying that tomorrow.
<wgrant> Right. Let's see what the fastdowntime queue looks like.
<wgrant> I guess we could do more frequent episodes to clear it out.
<wgrant> But the first run will hopefully be on Wednesday, and it will be a no-op.
<wgrant> stub: Anything to wait for but the Storm fix?
<cjwatson> just bounce all the connections to see if anything falls over?
<wgrant> Basically.
<wgrant> The full update script asks the DB cluster to sync, kills all connections, works out patches to apply, applies any patches, reconfigures security, syncs the cluster again, turns connections back on.
<wgrant> All the clients just see no DB for a while. Which could have amusing effects.
<jtv> G: I reviewed one of yours in hopes that StevenK would perform the other half of the deal I proposed, but he seems to have bugged out.
<G> jtv: hold on, let me load it up on LP itself
<G> jtv: I'm not too sure on transaction.commit() I'll test it now
<stub> wgrant: Nothing required but the storm fix. The branch-revision fix would be nice, but that is an understood problem we can work around by bouncing that service immediately after the update.
<stub> Same for the storm fix really, but I'd like to minimize the amount of bouncing that needs to happen.
<wgrant> stub: We should be able to deploy all the way tomorrow, unless buildbot still wants me dead.
<stub> Yes, agreed.
<stub> mthaddon: Can we do updates tomorrow?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #1,035: FIXED in 4 hr 36 min: https://lpci.wedontsleep.org/job/devel/1035/
<wgrant> !!!!
 * stub looks into which particular db patch cjwatson needs
<mthaddon> stub: what kind of updates?
<stub> mthaddon: fastdowntime db update
<mthaddon> stub: have we done a test run?
<stub> mthaddon: This is the test run. Staging has been doing it for a while.
<mthaddon> stub: okey doke
<wgrant> We're starting with a no-op, right? :)
<cjwatson> stub: https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations-schema
<stub> wgrant: I can argue it either way, but will go with lifeless' suggestion of a no-op first up.
<wgrant> stub: We've not had wondrous success syncing the cluster recently, so it is tempting to eliminate unnecessary risk.
<wgrant> Even though we're not dropping a slave, so it should all be fine.
<wgrant> I still don't trust SSO to not end the world.
<wgrant> Should we provisionally announce the 0800 window for tomorrow?
<stub> cjwatson: I'll make sure getting that patch up is a priority, even if we can't get all the patches applied
<G> oh is LP.net getting updated
<cjwatson> great, thanks very much
<wgrant> G: We normally update launchpad.net roughly daily.
<wgrant> G: But database patches were previously applied only monthly, as they required 60-90 minutes of downtime.
<wgrant> G: But the new process allows us to it in hopefully less than a minute.
<stub> Announcements are not necessary for 5 minute outages apparently, so we just need to announce to identi.ca for the window.
<wgrant> The preparations are finally complete.
<G> wgrant: ahhh right
<wgrant> stub: Once we have them down below a couple of minutes, a couple of hours is probably enough. But we might want to give a bit more this time, as it's the first and not exactly unrisky.
<stub> We can make a blog post if we want I guess since it is the trial by fire of the new process.
<wgrant> Hmm, no mrevell.
<wgrant> identi.ca two hours before, then :)
<wgrant> Heh. Only 1.5 months after the blog post announcing the start of fastdowntime.
<stub> Not really much risk apart from OOPSes and more extended outages for the bits that need to be manually bounced.
<wgrant> stub: So we say until the cluster falls apart.
<wgrant> And takes an hour to recover.
<stub> wgrant: No risk of that with a no-op update
<wgrant> Hahaha.
<wgrant> Although I guess the last few bad rollouts have been because we didn't want to abort.
<wgrant> Because the window was huge and expensive.
<wgrant> If stuff breaks with fastdowntime we can just abort.
<stub> yup. everything that has caused failures in the past is checked for up front and the process doesn't kick off.
<wgrant> And even if it does break we don't have to recover into a deployable state immediately.
<wgrant> But hopefully SSO won't desync any more.
<stub> And even if there is some internal slony failure, we can switch to master-only fairly quickly with pgbouncer. We have covered everything we can, and at some point we need to push the button.
<wgrant> Yup.
<wgrant> OK, where is the fastdowntime process page these days? The LEP is unhelpful.
<stub> mthaddon: I'm going to need a fresh db-stable tree on wildcherry
<mthaddon> stub: fresh means which revno?
<stub> mthaddon: Not sure if we thought about that with your tree pushing scripts
<mthaddon> stub: we pull from wildcherry, but yep, we have thought of it :)
<stub> mthaddon: current head of launchpad/db-stable
<mthaddon> er, db-stable?
<stub> mthaddon: I will need to trim out unwanted patches for the no-op run
<wgrant> stub: Can't we just merge the new scripts to devel?
<mthaddon> yeah, my thoughts exactly
<wgrant> Most of that should come across harmlessly.
<stub> If we want to test something other than the scripts we have been testing...
<wgrant> I was actually looking at that this morning, when I cherrypicked some other stuff across.
<wgrant> stub: Hm? Can't we just merge all non-DB-patch-depending changes into devel?
<stub> Yes, and hope we get it right and we didn't miss any dependencies
<wgrant> True.
<stub> Or we can just use db-stable
<stub> I just noticed as I was going to check what db patches need to be pruned for the no-op run and if we need to do multiple real runs or a single 'real' update.
<wgrant> Hm. Did we get timings from staging yesterday?
<wgrant> I'm guessing not.
<wgrant> Because it likes to break just a day or two before the restore.
<wgrant> It broke again.
<stub> I think we have all the timings we need from all the previous runs.
<wgrant> Lots of permission denied (publickey) :(
<stub> 2011-09-02 14:44:00 INFO    2208-78-1 applied 2011-08-28 in 9.5 seconds
<stub> 2011-09-02 14:44:00 INFO    2208-79-0 applied 2011-08-28 in 1.3 seconds
<stub> 2011-09-02 14:44:00 INFO    2208-79-1 applied 2011-08-28 in 0.5 seconds
<stub> 2011-09-02 14:44:00 INFO    2208-80-1 applied 2011-08-28 in 0.1 seconds
<stub> 2011-09-02 14:44:00 INFO    2208-81-1 applied 2011-08-28 in 0.0 seconds
<wgrant> Ah, it was hwdb stuff breaking the update. The SSH errors were unrelated, apparently :/
<wgrant> stub: Do a no-op run tomorrow, and if all goes well do another one also tomorrow with everything?
<stub> yup, or even later in the same day or same 5 minute window ;)
<stub> I guess same 5 minute window is a bit ambitious given we need to confirm everything survived :)
<wgrant> Right, I meant later in the same day, as in the same 5 minute window is a little adventurous.
<stub> But the db patches I'm looking at all look really, really boring.
<wgrant> They are pretty bad, yeah.
<wgrant> A good start.
<wgrant> Apart from the 10s one.
<stub> Its still boring.
<wgrant> It is.
<wgrant> But slow boring.
<stub> 10s is not slow. We have had patches that took hours in the past ;)
<wgrant> Oh yes, I recall.
<wgrant> The good old days of 12-hour downtime to open a new Ubuntu release, and then the final 10-hour downtime to migrate to the new translations schema.
<stub> yer, and get that down to 10s and people still are not happy. I wonder why I bother sometimes :-D
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 258 - 0:[########*** stack smashing detected ***: ./lp terminated
<stub> Looks like just those 5 patches.
<wgrant> stub: What about 76-3 and 76-4?
<wgrant> Oh. Those are bugsummaryrollupbreakage, aren't they.
<stub> They are both live
<wgrant> How could I forget.
<stub> http://paste.ubuntu.com/682567/ will have to do since there is no report yet
<wgrant> Indeed, just those 5.
<stub> And upgrade.py --dry-run if I'm feeling hungover tomorrow.
<wgrant> Do you happen to feel like checking BugMessage.owner's NULLness, just in case?
<wgrant> That's the main reason i wanted the restore.
<stub> Still 0
<wgrant> The appservers aren't going to choke on 79-0? I can't remember if that restriction was relaxed.
<stub> I can't see why the appservers would complain.
<wgrant> A month ago they would refuse to start if there was a -0 patch applied but not in the tree.
<stub> That code has been turned off completely.
<stub> Guess we should check that revision is live
<wgrant> It seems to have been deleted.
<wgrant> I can't find it anywhere nor remember the rev it was deleted in, so it's deployed everywhere.
<wgrant> revno: 13638 [merge]
<wgrant> timestamp: Tue 2011-08-09 15:28:37 +0000
<wgrant>   [r=gmb][bug=809636,
<wgrant>         815706] Remove runtime checks of LaunchpadDatabaseRevision
<stub> ta
<wgrant> librarian1 would have been the last holdout.
<wgrant> So we're clean now.
<stub> only the appservers checked anyway
<wgrant> Yeah.
<G> henninge: as OCR, can you at some point take a look at https://code.launchpad.net/~dev-nigelj/launchpad/bug-674854/+merge/74064 please
<henninge> G: sure but otp right now
<StevenK> OMG, devel built on Jenkins
<wgrant> StevenK: Yup. jelmer fixed the four main failures on db-devel a week ago. I cherrypicked it into devel tonight, and disabled the other failing tests, which cascaded into everything else.
<wgrant> So we should be green for a while now.
<stub> Long enough to take buildbot out the back and shoot it?
<wgrant> The new bzr seems to have made everything more reliable.
<bigjools> sigh, is the poppy bug STILL not fixed? :/
<wgrant> The dozens of varied failures over the last week or two can all be explained by fallout from the three tests.
<StevenK> stub: Is that a +1 for Jenkins from you, then? :-)
<wgrant> bigjools: Nope.
<wgrant> :)
<wgrant> See /topc.
<wgrant>  /topic
 * bigjools hunts in vain
<stub> StevenK: I'm for whatever has the least maintenance burden, and whatever I don't have to be involved with.
<StevenK> Heh
<jml> probably jenkins
<StevenK> Death to buildbot and buildbot-poller
<G> must say, as an observer, jenkins looks kinda neat
<stub> I would like a bot I can tell 'Go test lp:~stub/launchpad/foo and give me the results', but we need the basics first.
<StevenK> stub: bin/ec2 test? :-)
<jml> that's easy to do w/ jenkins, but breaks polling ime
<StevenK> jml: Do it as a seperate job
<StevenK> But yes, fairly easy
<wgrant> Ideally we do that with Jenkins in front of a couple of machines that run a couple of 4-way test suites or so.
<stub> StevenK: I hate the 5 minutes it takes and the risk of leaving hung instances chewing through cash.
<stub> StevenK: I also hate having too many environments that tests have to pass on. We currently need everything to pass on 'local dev environment', ec2 image, buildbot image, staging environment, production environment. With the wonders of modern VM tech, there is no reason we can't get that down to one environment that is identical everywhere.
<StevenK> stub: +1,000,000,000
<G> yeah, I was kind of surprised when I kept seeing EC2 been mentioned
<StevenK> G: Whyfor?
<stub> Because while the test suite is solid, there are so many dependencies involved the environment ends up fragile. Time consuming to put together an alternative that is as automated and reliable.
<G> StevenK: great for testing, but it was a "I thought they'd be using an inhouse cloud or something using UEC"
<soren> I'm curious.. How many instances are spawned to run the tests?
<G> StevenK: put it this was, I was surprised, but understood
<StevenK> soren: One per branch
<soren> StevenK: Oh, just one?
<soren> Hm.
<soren> Ok.
<StevenK> soren: Yup.
<soren> How long does a test run take?
<stub> People are working on parallel testing, but it is still science fixtion
<StevenK> soren: 4 and a bit hours.
<soren> Ok.
<wgrant> stub: LXC-based parallel testing is pretty much doable now.
<wgrant> It has been done.
<StevenK> soren: It's not okay, it's awful. :-P
<wgrant> It's not completely reliable yet.
<stub> Yer, just the second 95% to do now.
<soren> StevenK: Well, yes :)
<G> hmmm so 4 & a bit hours at $0.68/hr?
<stub> Yer, the coffee you drink while waiting costs more.
<G> I guess it could be much worse
<stub> Being able to rent out servers by the hour for < $1 is pretty awesome.
<stub> Someone is going to notice this Cloud thing one day.
<G> stub: you mean they haven't?
<stub> Dunno. I stopped watching TV when I moved into exile.
<StevenK> wgrant: Are you happy for me to toss destroy-openidrp at ec2?
<wgrant> StevenK: Let me glance over it first.
<wgrant> StevenK: Hm, you now always warn?
<wgrant> I guess that's OK.
<StevenK> It's still dangerous
<wgrant> StevenK: To ec2 with it!
<StevenK> wgrant: Happy with no-qa, or shall I file a bug just so we can try it out?
<wgrant> Bug would be nice.
<bac> n
<StevenK> wgrant: You and I name too many branches beginning with 'destroy-'
<jml> s/too many/not enough/
<jml> functionality is an asset. code is a liability.
<nigelb> StevenK: Start using nuke- then ;)
<StevenK> icbm-
<nigelb> +1
<G> diediedie-
<nigelb> G wins.
<G> oh actually, why not go all James Bond... 007- :)
<StevenK> I think both wgrant and I have used 'diediedie' in a branch name before
<nigelb> lol
<G> or from the movie... RED- (Retired & Extremely Dangerous) :)
<G> tbh, I like 007- better
<nigelb> I'll go with diediedie
<StevenK> I don't like the idea of 007-. If he likes the look of the code, he'll seduce it and sleep with it
<G> StevenK: in that case, there is far bigger issues to worry about than the bug you are trying to fix :)
<bigjools> StevenK: the code is already fucked though
<nigelb> bigjools++
<wgrant> jelmer: Hi. Can you please QA the bzr upgrade? It's getting reasonably urgent.
<wgrant> aaaergwejfwf
<wgrant> test_import_bzr failed this time.
 * wgrant kills them all.
<jelmer> wgrant: I've QA'd it as much as I can so happy to mark it qa-ok
<wgrant> jelmer: Great, thanks.
<jelmer> wgrant: that's test_import_bzr for the workermonitor?
<wgrant> jelmer: Yes.
<wgrant> jelmer: I've already disabled the three that you reenabled a couple of weeks back.
<wgrant> But now test_import_bzr is failing the same way.
<wgrant> I think script startup is just too slow now.
<wgrant> So it hits the 20s timeout.
<wgrant> Bug #841556
<_mup_> Bug #841556: test_import_bzr, test_import_cvs, test_import_bzrsvn and test_import_subversion disabled <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/841556 >
 * wgrant sleeps.
<jelmer> blergh
<jelmer> wgrant: thanks for handling that
<henninge> G: reviewed. Sorry for taking so long. Please have a look.
<G> henninge: yeah, I couldn't find any tests that related to the distroarchseries webservice
<G> henninge: oh, so you are saying I should replace the whole setUp w/ a _makeDistroandArch() type thing?
<henninge> G: yup
<G> in lib/tests/factory (or the correct path) or in the new file?
<henninge> G: Much more obvious when you read it
<henninge> G: no, directly in your test case.
<G> oh right
<G> I'll be back in a few then :)
<G> henninge: everything else good okay though?
<henninge> yes, looks fine
<G> henninge: that said, as multiple tests can benefit from the one-time setup for the data, isn't that what setUp is for?
<G> (just curious)
<G> (i.e. if I was to do an authenticated test as well, surely the one time setup, instead of multiple setups, would be faster) or am I on the wrong thought track?
<henninge> G: No, it is not faster
<henninge> G: setUp is executed before each test.
<G> oh it is, okay then
<G> Okay, giving this a whirl
<nigelb> G: Don't you ever sleep? :)
<nigelb> I see you *all* the time :D
<nigelb> g32
<G> nigelb: I have a weird sleep pattern
<bigjools> I might also say that about nigelb!
<nigelb> bigjools: :D
<nigelb> Kettle, pot, black. I know!
<nigelb> I need to pull an all-nighter. I'll throw that frustration away by writing some patches.
<G> nigelb: I do some of my best thinking late at night
<nigelb> Someone needs to write "The therapeutic effects of coding"
<G> (not just coding, but thinking in general)
<nigelb> G: Join the club.
<nigelb> I sleep at about 2 am local time.
<G> Tue Sep  6 03:38:30 NZST 2011
<G> although, most python applications would have be believe it's currently +1300 for some reason :)
<nigelb> Is your tzdata package up-to-date?
<G> yep
<G> but I don't know when the last time NZ has been +1300 this early, ever
<nigelb> Try using pytz to figure out the date. That should work correctly.
<nigelb> If it doesn't, you can always crib to its author right here ;)
<nigelb> s/date/date\/time/g
<nigelb> PyCharm people just need to hire bigjools as their community manager :P
<bigjools> they need to write it in Python first :)
<nigelb> PyCharm isn't Python/
<nigelb> ?
<bigjools> Java
 * bigjools throws up a little
<nigelb> Ewww.
<G> a Python IDE written in Java, heh
<bigjools> and it uses Swing
 * G vomits a little
<bigjools> but I forgive it so far
<nigelb> Is it as slow as Eclipse?
<bigjools> pretty snappy for the most part
<bigjools> but then I have 4 cores
<nigelb> jml: ping
<jml> nigelb: yes?
<nigelb> jml: Could you join #ubuntu-classroom, #ubuntu-classroom-backstage, and #ubuntu-classroom-chat?
<jml> nigelb: done
<nigelb> The bot's complaining that you aren't around :)
<G> henninge: I think I might want to take another look at this again tomorrow
<G> I think my anonymous test, may not be logging into the API so anonymously afterall
<nigelb> I'm liking jono's session :)
<henninge> G: np at all, get some sleep first ... ;)
<henninge> How do I update sourcedeps.cache?
<jelmer> henninge: run ./utilities/update-sourcecode
<henninge> jeblair: next question: when would I do that?
 * henninge never had to do it.
<henninge> jelmer: ^
<henninge> sorry jeblair
<nigelb> I think either rf-branch or make run already did that
<jelmer> henninge: After you change utilities/sourcedeps.conf
<henninge> aha
<jelmer> henninge: (or after a merge changes it)
<henninge> ok
<henninge> jelmer: thanks
<jelmer> henninge: it would indeed be neat to add some deterministic ordering to it to prevent spurious conflicts and the like
<henninge> jelmer: I already found the place where to do that, I just wanted to find out how to try/test it.
<jelmer> henninge: cool
<G> henninge: actually, I worked out the problem, should be an updated branch coming to you in a moment :)
<henninge> G: cool
<G> two last bin/test's then I might be able to sleep easier :)
<G> okay, it works :)
<G> henninge: okay, I pushed the updated fix to my branch, if it's okay please feel free to land it, if there are other issues, I'll tackle them tomorrow :)
<henninge> G: I'll have a look in a minute
<G> yep, no hurry, I'm off to bed :)
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 258 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> sigh, buildbot
 * wgrant crushes buildbot into the ground.
<wallyworld_> wgrant: i forced a buildbot build but the compile failed - looks like a simple timeout. you seen this sort of thing before?
<wgrant> Yes, we are fucked.
<wgrant> I'm trying to track down when this failure first showed up.
<wgrant> But buildbot doesn't actually keep logs for more than a week or so.
<wgrant> Although I guess they should be in emails...
<wgrant> Or not.
<wgrant> It just emails links.
<wgrant> Sigh.
<wallyworld_> wgrant: so, it's not an actual compile error right?
<wgrant> There is something real there.
<wgrant> Oh.
<wgrant> That last failure is not real, no. That was hloeung killing the slave.
<wgrant> Thought you were talking about the previous ne.
<wgrant> But you were hopefully too asleep to force that.
<wallyworld_> yes, just saw that one issue when i checked earlier and thought i'd try a force
<wallyworld_> since it appeared to be a timeout
<wgrant> Yeah, we've forced that failure about 20 times in the last two weeks.
<wallyworld_> hopefully one of the maintenace squads is looking
<wgrant> lol
<wgrant> US and Canada were away yesterday.
<wallyworld_> ah, labour day
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 259 - 0:[########*** stack smashing detected ***: ./lp terminated
<wallyworld_> wgrant: you tried to install nautilus elementary in oneiric? i really need to be able to customise the toolbars
<wallyworld_> natty packages don't work :-(
<wgrant> I've not. I hardly use nautilus, but when I do the new one is nice and clean.
<wallyworld_> clean yes, but you can't do simple things like add a "up to Parent" button to the toolbar. very annoying when you need to navigate up one directory
<wgrant> Ah.
<wallyworld_> nautilus elementary also has a bunch of other useful features. it really should be what we ship imho
<jelmer> has anybody seen this error on oneiric: "psycopg2.OperationalError: fe_sendauth: no password supplied" ?
<wgrant> jelmer: Running the test suite locally?
<jelmer> wgrant: yeah - it worked fine a couple of days ago
<wgrant> jelmer: We connect over TCP now (as of the weekend, then it was reverted, and now as of a couple of hours ago)... perhaps your postgres config doesn't permit ident/hba over TCP?
<jelmer> wgrant: that might be it, I'll have a look at that
<wgrant> host    all         all         127.0.0.1/32      trust
<wgrant> That line should be in your pg_hba.conf
<wgrant> Odd that it's not /8
<wgrant> Perhaps your localhost resolves to something else in 127/8 instead?
<jelmer> maybe it's ipv6
<wgrant> Ah, or that.
<wgrant> But that should be ip6-localhost, normally.
<wgrant> Most of my home network is IPv6, and it all still works for me.
<jelmer> I can't find it right now and it's getting quite late, I'll have a look tomorrow.
<wgrant> Night!
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 260 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> :\
<nigelb> Are the critical bugs breeding :(
<wgrant> Yup!
<wgrant> wallyworld_: Are you running LP on oneiric still, or back in LXC?
<wallyworld_> wgrant: still on oneiric
<wgrant> wallyworld_: You've just been dealing with JS, then?
<wgrant> Since you haven't wanted my rabbitfixture fix.
<wallyworld_> wgrant: yes. but i'm in the middle of a python branch so that will change
<wgrant> wallyworld_: See the list when you run into RabbitMQLayer startup trouble. I have a properish fix now.
<wallyworld_> wgrant: awesome. thanks
<wgrant> pygtk seems somewhat broken on oneiric, so the YUI testrunner doesn't work.
<wgrant> But there are only 8 other oneiric failures.
<poolie> wgrant, if you have a chance can you read my dkim patch?
<poolie> i see one nit in it at least ,though
<wallyworld_> wgrant: i can run yui tests ok
<wgrant> wallyworld_: Odd. Maybe you upgraded so things work better?
<wallyworld_> wgrant: i did upgrade from natty but it hosed my system. so i installed oneiric from scratch
<wallyworld_> just to clarify, bin/test --layer=YUITest works, as does running individual test cases from the browser by loading the html page. what error do you get?
<wgrant> wallyworld_: No module named gtk. Then I installed python-gtk2, and now it says it can't initialise Gtk or something.
<wgrant>   File "/usr/lib/python2.6/dist-packages/gi/overrides/Gtk.py", line 1406, in <module>
<wgrant>     raise RuntimeError("Gtk couldn't be initialized")
<wallyworld_> hmmm. i never saw anything like that
<wgrant> RuntimeError: Gtk couldn't be initialized
<wallyworld_> i don't think i installed everything gtk though
<wgrant> There's probably some dep I'm missing, since this is in a minimal LXC, while you're presumably running with a full desktop environment.
<wgrant> Ah, umask changes are breaking some of the other tests :(
<wgrant> 002 ftl :(
#launchpad-dev 2011-09-06
<StevenK> wgrant: The librarian GC is hideous
<wgrant> Surely not.
<wgrant> But WTF are you doing in there?
<wgrant> Ripping out staticdiff?
<StevenK> No, it errors since it uses introspection and no longer has access to openidrpconfig
<wgrant> OpenIDRPConfig has an LFA?
<wgrant> Oh, the logo?
<wgrant> So, just leave the permission.
<StevenK> Yeah, the logo
<wgrant> It must hold it until we drop the FK, which I will do with the other 30.
<StevenK> wgrant: http://pastebin.ubuntu.com/682968/
<wgrant> StevenK: Interesting, jelmer had the same thing, but it works for stub and I.
<wgrant> host localhost
<StevenK> .1 and ::1
<wgrant> Open up pg_hba.conf, duplicate the 127.0.0.1/32 trust line with ::1/128
<wgrant> Reload postgres, and see if that fixes it.
<StevenK> wgrant: It does, yes.
 * wgrant fixes launchpad-database-setup and mails the list.
<wgrant> StevenK: +host    all         all         ::1/128      trust
<wgrant> ?
<StevenK> "host    all         all         ::1/128           trust" is what I added, yes
<wgrant> Hmmm.
<wgrant> Interesting.
<wgrant> My postgres isn't listening on v6.
<wgrant> Possibly because the container I was testing in isn't properly bridged, so it only has a link-local address.
<wgrant> But detecting using that sounds pretty hideous.
<wgrant> I know glibc did that sort of check to prevent DNS hangs... but postgres?
<wgrant> StevenK: Is your listen_addresses setting the default? (commented out or 'localhost')
<StevenK> wgrant: It's commented out
<wgrant> StevenK: Can I have your /etc/hosts? And is this natty?
<StevenK> wgrant: Yes, and http://pastebin.ubuntu.com/682984/
<wgrant> That looks either upgraded or mangled.
<wgrant> Hah, mangled.
<wgrant> I suspect line 6 is the difference. localhost isn't there on my machines.
<wgrant> Anyway, I will fix launchpad-database-setup.
<wgrant> Thanks.
<StevenK> Now the last failure is in xx-person-edit
<StevenK> Which now fails since it now prints the OpenID warning everytime
<wgrant> StevenK: Jenkins is still happy!
<StevenK> Gasp!
<StevenK> wgrant: http://pastebin.ubuntu.com/683002/
<wgrant> StevenK: Comment please.
<StevenK> wgrant: http://pastebin.ubuntu.com/683003/
<wgrant> Better.
<G> hey btw, the ec2 test results e-mails that get sent, appear to be dateless
<G> (i.e. no Date: header)
<StevenK> No, it's just ... timeless
<G> ewwww very punny :)
<StevenK> wgrant: Tossing destroy-openidrp back at ec2
<wgrant> StevenK: Great.
<StevenK> ec2test@i-dee51dbe$ sudo shutdown -P +480 &
<StevenK> That makes me happy
<StevenK> Applying postgresql configuration changes...
<StevenK> patching file /etc/postgresql/8.4/main/pg_hba.conf
<StevenK> patch: **** malformed patch at line 14:
<StevenK> wgrant: ^^
<wgrant> Bah.
<wgrant> How did that work before :(
<wgrant> StevenK: 13877
<poolie> does python-coverage work with lp?
<poolie> one way to find out
<wgrant> That sounds fatal.
<poolie> oh, it's unlikely to work?
<wgrant> zope.testrunner is somewhat special. And LP takes 10s or so of CPU time to start even without instrumentation, so it's probably going to be sloooow.
<poolie> oh but there is a --coverage
<LPCIBot> Project devel build #1,038: FAILURE in 1 hr 21 min: https://lpci.wedontsleep.org/job/devel/1038/
<wgrant> :(
<wgrant> Ah. My fault.
<wgrant> wallyworld, StevenK: https://code.launchpad.net/~wgrant/launchpad/fix-dbuser-override is ugly, but needs to land within an hour. Could one of you please give it a review?
<StevenK> wgrant: Did you fix the patch issue?
<StevenK> Since my ec2 run just utterly broke with
<wgrant> StevenK: Yes, 13877
<StevenK> OperationalError: FATAL:  Ident authentication failed for user "launchpad"
<wgrant> It's what broke jenkins too.
<StevenK> Let me delete that slave, since it's probably infected
<wgrant> Heh. It should be OK, but maybe.
<LPCIBot> Project devel build #1,039: ABORTED in 19 min: https://lpci.wedontsleep.org/job/devel/1039/
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks.
<jtv> wgrant: my connection's insane again but hope I can get a review for https://code.launchpad.net/~jtv/launchpad/pre-832647/+merge/74087
<wgrant> jtv: Sure.
<jtv> thanks
<jtv> wgrant: you'll be happy to note that I made the unification of [SB]PPH a bit more sustainable.
<wgrant> I am glad.
<wgrant> jtv: The name "GeneralizedPublication" made me think it was a wrapper for ?PPH.
<jtv> Yes, better suggestions would be most welcome.
<wgrant> Well, couldn't it be a wrapper?
<wgrant> Ah, that would make the sort() call awkward.
<jtv> Exactly,
<jtv> but I did consider it.
<wgrant> jtv: Approved with comments.
<jtv> Thanks.
<nigelb> Morning.
<StevenK> O hai
<nigelb> <-- sleep deprived :(
<StevenK> nigelb: So situation normal, then?
<G> haha, sounds familar :)
<G> btw, not going to bug you folks much today, working on my own project today :)
<StevenK> G: That's a bug!
<G> StevenK: "Bug: G is not fully devoted to Launchpad development" ? ;)
<StevenK> Haha
<StevenK> Priority: Critical
<G> to tell the truth, I'm not really that much of a developer, but I do like to dive deep into code, which is kinda funny :)
<nigelb> StevenK: Ha
<nigelb> Stayed up for App Developer Week and ended up not being able to sleep.
<nigelb> wgrant: Your ISP gives you IPv6?
<StevenK> No, both wgrant and I have a tunnel
<wgrant> I believe there's roughly one residential ISP in Australia that provides native IPv6.
<StevenK> Internode only so far
<StevenK> Telstra will be turning on it "soon"
<G> StevenK: wow seriously, nice
<StevenK> Exetel have performed trials, but no news on when the unwashed get it.
<G> one retail ISP in NZ just went live w/ non Beta native IPv6
<G> No idea when Telecom will
<G> Tunnel to HE.net for me, but it works well
<StevenK> Both wgrant and I are using AARNet's
<G> ahhh via the Hexago setup
<StevenK> I have no complaints
<wgrant> Most of Linode's location's have native IPv6, and my v6 tunnel has better US routing than native v4, so it's handy.
<G> There is one crowd that has a tunnel server setup via Sixxs or whatever they are,
<G> wgrant: I dumped Linode ~6 months ago
<wgrant> :(
<StevenK> wgrant: Both Atlanta and London are in internal testing for v6
<G> moved my VPS to an LA provider that runs KVM + Native IPv6 etc
<G> it was back before Linode decided IPv6 was a good thing
<StevenK> So all 5 should be v6 in a month or so
<G> well that is something I guess, they dropped the silly policy of charging for anything more than a /128 right?
<wgrant> G: HE.net's broker is in the same Fremont DC as Linode, so tunnels worked pretty excellently even before it was native.
<wgrant> I believe so, yes.
<StevenK> G: Yes
<StevenK> G: http://www.linode.com/IPv6/
<G> wgrant: yeah, I used to use Linode + HE.net Fremont tunnel, but went to ARPnetworks w/ Native IPv6
<G> my home IPv6 tunnel terminates in LA, so it's not much of a hop in that direction either
<G> plus, as the Southern Cross Cable terminates at LA.... well you get the picture :)
<G> StevenK: wgrant: ahh yep, iirc from the original announcement, they were only offering a /128 and more was $$$
<wgrant> I guess enough people told them that was completely stupid.
<StevenK> G: I waited for the backlash to die down, and then a little while for the kinks and reverse v6
<G> yeah, I just laughed
<G> IPv6 is definately the way to go
<G> (oh, the other reason why I ditched Linode, was the instability after those crazy outages)
<StevenK> Which isn't Linode's fault at all
<StevenK> HE couldn't design a stable DC with help
<G> StevenK: yeah, but the communication afterwards, wasn't that great either
<G> (imo anyway)
<G> and yeah, you are right, HE can't really design datacentres, so it was another good excuse :)
<StevenK> Personally, I didn't mind either way
<StevenK> I'm not about to jump up and down and demand Linode fix something that is HE's fault.
<wgrant> stub: Evening.
<stub> hi
<wgrant> stub: Your --port changes broke staging updates; I removed an assertion and things seem to work again, but I thought you should probably check it.
<stub> ta
<wgrant> r13878
<wgrant> Bug #842284
<_mup_> Bug #842284: connect_string can't override the username if already specified <regression> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/842284 >
<wgrant> Also got a couple of other DB-related archaelogy questions when you have some time.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 254 - 0:[#######=]:256
<stub> How did that happen? Thought that that assert was already in there.
<wgrant> It was.
<stub> wgrant: shoot
<wgrant> But db_options() didn't originally put dbuser in the connection string.
<wgrant> It just put it in options.dbuser, which pretty much everything that connected to the DB respected.
<stub> oic
<wgrant> It makes sense, in a not at all and how could somebody do that kind of way.
<stub> its retrofit on retrofit
<wgrant> yeah.
<wgrant> I'm ripping out lots of it now.
<wgrant> All the non-DB scripts just call connect() without a custom username, and canonical.lp is almost dead.
<StevenK> Kill it!
<wgrant> Which was one of the other questions: the checks in canonical.lp.sql look reasonably pointless now, and aren't running any more.
<wgrant> I presume the ZCML inclusion was accidentally dropped somewhere along the line.
<wgrant> The automatic FROM thing has been off by default since postgres 8.1, so probably hardly worth checking for.
<stub> Those checks used to be useful. Probably not so much now, yeah.
<stub> Maybe the encoding one, but that would be more useful as locale one in any case to confirm the expected collation order. I think we have duplicated these checks elsewhere?
<stub> But yeah, kill 'em.
<wgrant> Good, I already killed them early in this branch :)
<wgrant> Thanks.
<StevenK> wgrant: Does it mean that c.l.sql dies completly?
<wgrant> No, it will still have initZopeless and isZopeless.
<wgrant> They need a new home until they die.
<wgrant> But all the other crap is no longer used except by canonical.launchpad.scripts.db_options, and I hope to eliminate that in a sec by removing the LP_DB* envvars.
<wgrant> stub: Those envvars are pretty dead, right? :)
<wgrant> Since we have universal script args now.
<stub> wgrant: Yes. They sometimes get used, but we should be able to use the PG* env vars or the db_options helper for those rare cases.
<wgrant> Yup.
<wgrant> db_options is pretty handy.
<stub> All the times they were used recently I think were for work arounds, stuff like accessing a DB on a non-standard port.
<wgrant> Which is now fixed.
<wgrant> And PGPORT worked for anyway, IIRC.
<stub> twice :-)
<wgrant> There's also lots of cruft in database/schema/pending.
<wgrant> And what is lib/canonical/database/ftests/portforward-to-postgres.tac? :/
<stub> yer, its pretty much a garbage can for people to stick templates, works in progress etc.
<StevenK> Hahaha
<wgrant> Ah.
<stub> That is the old disconnection test harness IIRC, before it moved to Storm.
<wgrant> It's used by an unused TacTestSetup.
 * wgrant schedules that for deletion.
<wgrant> database/schema/restoredump.sh looks to be 6.5 years out of date.
<wgrant> Aha.
<wgrant> The config is now the sole repository of the DB config.
<stub> \o/
<wgrant> Although there is still the extra dbuser in each DBConfig section.
<wgrant> That is more forgivable, but still needs refactoring.
<stub> Yup. And I suspect we could drop them if we just use the script name as the dbuser
<stub> Overriding the database username was a big WHUI
<wgrant> Yep.
<stub> Thanks for that. Point me at a MP if you want.
<wgrant> Still need to work out how the dbconfig.dbuser stuff is tied in. I think I may have missed a spot.
<wgrant> Looks like it's mostly respected in LaunchpadDatabase.raw_connect, which will probably blow up if a script using it is run with -U...
<wgrant> But that's not restricted to this branch, and hopefully doesn't affect production.
<stub> We run the database update scripts with -U sometimes
<wgrant> Yeah, but they don't use the CA, do they?
<wgrant> They'll use sqlbase.connect, which does a direct psycopg2 connection.
<stub> Not sure - security.py would be the main offender, as we do -U slony to run it directly on the slaves since that is a superuser and has auth all setup.
<wgrant> security.py is fixed by the assertion removal I pointed you at earlier.
<wgrant> It uses connect().
<wgrant> But the Storm backend has pretty much the same bug, as it turns out.
<wgrant> But I don't believe we run any normal scripts with -U... only DB maintenance ones.
<wgrant> Like full-update on staging.
<wgrant> And no DB maintenance script uses Storm.
<wgrant> So we are safe for now.
<nigelb> Hrm, need to do QA.
<stub> wgrant: lagmon might be a problem
<stub> oh no.... connect too
<wgrant> Yup.
<wgrant> Argh, ZTM keeps its own copy of connstring components. /me destroys.
<mrevell> Hi!
<nigelb> Hello mrevell :)
<stub> I still don't understand why bzr doesn't automatically break insane locks. Or to people really hold their transactions open for a week?
<wgrant> stub: Have we ever needed storm_cache to be configurable?
<wgrant> It seems to be the same across the board, except for the dev config's launchpad section, where it's 100 instead of 500.
<stub> Yes. Helped migration when we were switching cache implementations, and currently our scripts run with a smaller cache than the appservers
<wgrant> Ah, missed that 10000 in there.
<wgrant> Fair point.
<jelmer> grar
<wgrant> jelmer: Oh?
<wgrant> I was going to ask if we'd broken the importds.
<wgrant> But we didn't deploy to the importds.
<wgrant> So that's unlikely.
<adeuring> good morning
<jelmer> wgrant: Wrong channel, I was just going to express my annoyance with mumble :)
<jelmer> g'morning :)
<wgrant> Ah, heh.
<wgrant> Morning.
<wgrant> jelmer: buildbot has succeeded once since you disappeared, and looks like it may again.
<wgrant> So it may just have been those WorkerMonitorIntegrationScript tests.
<jelmer> ah, cool
<jelmer> I also see I wasn't the only one with that password error, thanks for mailing the list about that.
<wgrant> Yeah, StevenK ran into it. IPv6 as you suspected, so was easily fixed.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #1,040: FIXED in 4 hr 36 min: https://lpci.wedontsleep.org/job/devel/1040/
<StevenK> stub: Any news WRT live patch to migrate away from StaticDiff?
<stub> StevenK: First thing after this coffee :-)
<stub> Landed the patch to drop the dud index
<StevenK> stub: So the patch to drop the bad index is already in db-devel?
<stub> StevenK: https://pastebin.canonical.com/52285/
<stub> StevenK: Yes, it has landed already
<stub> StevenK: Can you check those statements over?
<StevenK> stub: Looks good to me
<stub> StevenK: Done
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 254 - 0:[#######=]:256
<StevenK> stub: As you have a patch hitting db-devel or the migration is done?
<stub> StevenK: Its done
<StevenK> stub: Excellent, thanks.
<StevenK> stub: Would you mind commenting on bug 834384 that you've done the migration live on prod?
<_mup_> Bug #834384: StaticDiff is unused and needs to be shot <Launchpad itself:Confirmed> < https://launchpad.net/bugs/834384 >
<StevenK> wgrant: no-more-staticdiff tossed at ec2 -- +53 / -615
<gmb> bigjools, wgrant, Laney is asking questions about (I think) distros, blacklisting and suchlike over in #launchpad... Can either of you answer them?
<gmb> bigjools: Thanks for - once again - being the Tree of Knowledge-I-haven't-acquired-yet.
<bigjools> gmb: nae prob, it's brand new functionality that we've done so I don't expect it to be that common yet
<gmb> Ah, right. That's why I was even more "Eh? Whut?" about it than normal then.
<bigjools> :)
<bigjools> it's all part of derived distros
<bigjools> you were paying attention in my talk at the Thunderdome right? :)
<nigelb> pop quiz after a few months is unfair :P
<gmb> bigjools: Attention and retention are two different things :)
<gmb> (Though I'd figured out that it was derived distro speak; just couldn't get the specifics)
<nigelb> I didn't work on Launchpad today at all :(
<nigelb> Oh, sinzui will be back today right? \o/
<wgrant> nigelb: I believe he's meant to be, yes.
<nigelb> wgrant: I have a review pending UI review plus a few UI things I wanted to ask him for help :D
<jelmer> nigelb: what is your day job, officially?
<jelmer> (is launchpad a part of it?)
<nigelb> jelmer: lolno :)
<nigelb> My official day job is systems engineer
<nigelb> which is a far cry from launchpad
<nigelb> My contributions to Launchpad are entirely voluntary and out of my own interest
<jelmer> ah, cool :)
<nigelb> :)
<bigjools> nigelb: IOW you are pimping to be hired? :)
<nigelb> bigjools: IOW?
<nigelb> Oh, hahahaha
<nigelb> bigjools: I'm running a record breaking run to "Number of rejections by Canonical" :)
<bigjools> :)
<patrickg> nigelb: they actually write rejections? aiui you simply get no response ;)
<nigelb> patrickg: Nah, I've gotten actual rejections. I think my current count is 16.
<patrickg> you're quite persistent
<nigelb> heh
<nigelb> Indeed.
 * gmb -> lunch
<jtv> G: your bug-self-assignment-email branch has landed.  I'll Q/A it once it hits staging.  All it takes is filing a bug and then assigning it to myself?
<jtv> I'll also want to test assigning to someone else; the better part of Q/A is seeing that you didn't break anything that always worked before.  :)
<G> jtv: no, because it only sends the e-mail if you are new to the bug (i.e. never been a subscriber to it effectively)
<G> jtv: I've QA'ed a few bugs, but PM me when it goes on QAStaging and I'll create the bugs if you want :)
<jtv> So I'll have to do it somewhere where I can assign bugs, but am not automatically subscribed to them.
<G> (a few bugs = my other contributions)
<jtv> It's tempting to let you do the work, and nice of you to offer, but I'm afraid the responsibility falls on me.  :)  I have a separate testing account, so I can play two people.  No need to coordinate across timezones etc.
<G> jtv: okay, no problem :)
<stub> jtv: Ping re https://code.launchpad.net/~jtv/launchpad/bug-834388/+merge/73483 and resultset.set()
<jtv> stub: pong re etc.
<stub> jtv: In your fix, is there a chance that the list of ids has been sliced? .set() currently fails when used on a resultset that has been sliced (the slice is ignored, and you attempt setting far more than you wanted to set)
<stub> jtv: nm. checked the code and the ids are listified so it is safe
<jtv> Thought so.  :)
<stub> jtv: Is there a script that we can run to qa-ok it? Its next up on blocking a rollout (followed shortly by nigelb's branch)
<jtv> stub: been working on it since morning.
<stub> ok. I think it just landed on qastaging.
<stub> Who has the qastaging mailbox all setup for a trivial check?
<nigelb> stub: I'll do my QA in ~2 hours, unless someone else gets to it before me.
<stub> nigelb: You have access to do it?
<nigelb> stub: No, I have to ask someone else to check once I trigger the email in qastaging
<stub> Which is what I would be doing since I haven't got any mail readers setup :)
<nigelb> heh
<nigelb> Hrm, what do I ask to check if I want someone to check the mails on qastaging?
<StevenK> The staging mailbox
<nigelb> No, but is there an ID/ something of the particular email that I want checked?
<nigelb> FU.
<nigelb> MPs broken?
<nigelb> OOPS-2075QASTAGING48
<nigelb> OOPS on visiting all MP pages. I can't QA.
<StevenK> nigelb: Lemme sync logs
<nigelb> Thanks
<wgrant> nigelb: Merge proposal diffs are stored in the restricted librarian, which isn't accessible to (qa)staging.
<wgrant> So you have to create a new one.
<wgrant> Not use one from prod.
<nigelb> Ah.
<nigelb> Can I use an existing branch which doesn't have an MP?
<nigelb> Apparently I can.
<nigelb> So, erm, could someone check the mailbox on qastaging for me.
<nigelb> I'm looking for a Disapproved mail for this branch https://code.qastaging.launchpad.net/~summit-hackers/summit/install-docs/+merge/65302
<flacoste> morning everyone
<bigjools> o/ flacoste
<nigelb> bigjools: Halp.
<nigelb> bigjools: Need the mailbox from qastaging for QA-ing my fix.
<bigjools> nigelb: under normal circumstances I would help immediately.  However, due to the fact that the inbox has currently got 31k messages in it from cronspam kio_imap4 is somewhat struggling :/
<nigelb> Ah. Heh.
<bigjools> I will need to bite the bullet and install t'bird at some point, it handles massive inboxes much better
<nigelb> What are you using Kmail?
<nigelb> AFAIK it has slightly better support for IMAP.
 * G personally recommends mutt ;)
<cjwatson> mutt works fine with 31k mailboxes, though it's worth having a header_cache set up.
<G> I just use mutt w/ Maildir & SSH, but yeah, no problems w/ a default-ish configuration and large maildir folders
<bigjools> nigelb: yeah kmail.  kio_imap4 is not very good, which is a shame because kmail itself is fantastic
<bigjools> All email clients suck in one way or another.  Pick your poison.
<nigelb> I just use gmail web UI.
<wgrant> bac: How does ForwardedAuthorization compare to AuthorizationBase.forwardCheckAuthenticated?
<bac> wgrant: complementary.  the new one does both checkAuthenticated and checkUnauthenticated.
<wgrant> Aha.
<bac> there were instances were forwardCheckAuthenticated was still needed so it was kept
<wgrant> Great, just checking you hadn't somehow missed it.
<wgrant> Thanks for fixing that.
<bac> wgrant: nope but thanks for asking
<wgrant> Three successful builds in a row. Quite a novelty.
 * wgrant sleeps.
<wgrant> stub: Thanks, didn't expect that!
<wgrant> stub: Part 2 coming up... I might shorten the description if you're going to do it :)
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/destroy-lots-of-db-cruft/+merge/74233, if you're still around.
<wgrant> jelmer: Hm, is bzr 2.4 meant to be much faster at generating MP diffs, or am I just being very lucky tonight?
<wgrant> Twice I've had LP diffs within 40 seconds.
<jelmer> wgrant: lots of things are quicker in 2.4, not sure about MP diffs specifically
<jelmer> wgrant: 40s is really appalling btw. Generating a diff for a lp tree locally (couple years old machine) takes 20s here locally
<wgrant> buildbot also seems well and truly fixed, so your bzr stuff is all good, and jenkins seems far more stable with 2.4. Thanks.
<wgrant> jelmer: 40s is actually not bad, considering the cron job only runs every minute.
<nigelb> sinzui: O hai. Do you have a minute for a quick UI review?
<nigelb> I'm assuming you're currently swimming in email.
<wgrant> jelmer: So it was <40s from MP creation to diff being there (that's just when I refreshed), which includes the time to the start of the next minute, startup time, processing anything that's earlier in the queue, then the actual diff.
<sinzui> nigelb, I can review. I read email while I am away to ensure my inbox is always zero
<jcsackett> sinzui: free to mumble in a bit?
<nigelb> sinzui: https://code.launchpad.net/~nigelbabu/launchpad/ubuntu-font-787798/+merge/73793
<nigelb> sinzui: Smart man :)
<jelmer> wgrant: Ah, that seem pretty quick in that case. Still, it should really be instantaneous.
<sinzui> jcsackett, starting mumble
<sinzui> nigelb, opening page
<wgrant> jelmer: Oh yes.
<wgrant> jelmer: And with rabbitmq it can be.
<wgrant> And may well be within the year.
<nigelb> wgrant: I thought you went to bed.
<jelmer> wgrant: actually, that was a diff between my working tree and lp:launchpad. Generating a diff between a *revision* and the submit branch is less than one second on bzr.dev.
<jelmer> wgrant: Yeah, rabbit should be great.
<wgrant> nigelb: But then stub sneakily reviewed one of my branches.
<nigelb> Haha
<nigelb> Oh, its not yet 1 am local time.
<jelmer> it's 10 minutes to EOD for me..
<jelmer> (if I would work 9 to 5)
<wgrant> Meh, call is only at 9am, but I will sleep soon :)
<nigelb> 9 to 5? HAHAHA
<nigelb> I haven't worked at 9 am in a long while.
<nigelb> I start late and end late.
<jelmer> I conveniently did 9-5 BST (with some effort) when I started in Soyuz and have stuck with that ever since.
<nigelb> You're bzr team or launchpad team?
<nigelb> I'm always confused. Until ercently I thought poolie was launchpad team :D
<bigjools> the rest of us do 9-6 ;)
<wgrant> Well, bzr was technically a subteam of LP until January.
<bigjools> still here wgrant?
<wgrant> Silence!
<wgrant> Just waiting for a couple of ec2 runs to start.
<wgrant> Then I may disappear.
<bigjools> uh huh :)
<wgrant> I don't have to babysit buildbot tomorrow :D
<wgrant> It shall be a productive day.
<jelmer> nigelb: poolie and I are both bzr team, though working on other projects occasionally is fun too
<nigelb> jelmer: :)
<jelmer> there have been some contributions to bzr from lp folk too recently
<nigelb> Oh, nice
<nigelb> I wish I got over my mental block of staying away from things that are not web apps
<jml> nigelb: bzr is *way* easier than LP to contribute to
<nigelb> jml: You rickrolled me into contributing for launchpad. Not falling for that again :P
<nigelb> (j/k)
<nigelb> I like web applications a lot better :)
<jml> :)
<nigelb> The other thing is, bzr doesn't irritate me. Launchpad does. I'm fixing as much of those bits as I can.
<jml> heh heh
<stub> bigjools: Did that mbox load?
<bigjools> stub: hahaha
<nigelb> I can totally picture the evil laugh there
<bigjools> muuuuahahahahaaaaaa
<stub> I'm thinking of just flagging the bug qa-ok as the actual patch is simple enough that it couldn't possibly go wrong...
<nigelb> stub: You're waiting for an inbox fix as well?
<nigelb> Or waiting for my QA to deploy.
<stub> Or qa-untestable might be better as it technically is untestable :-)
<stub> I'd like it qa'd tonight as there should be deployments tomorrow.
<nigelb> TEST IN PRODUCTION
<nigelb> :)
<nigelb> Is there a qa-test-in-production? ;)
<stub> qa-untestable :-)
<nigelb> Oh, its secret-coded :P
<adeuring> gmb: could you please review this MP: https://code.launchpad.net/~adeuring/lazr.batchnavigator/lazr.batchnavigator-no-start-param-for-first-last-url/+merge/74249 ?
<gmb> adeuring: Sure
<adeuring> thanks!
<rvba> gmb: Thanks for your review!
<gmb> rvba: No problem.
<nigelb> Ha, mrevell replacing himself.
 * nigelb just saw it pop up in rss feeds
<stub> nigelb, bigjools: I flagged it untestable. It is too simple a patch to let it bureaucratically block stuff.
<nigelb> stub: okay :)
<gmb> adeuring: r=me with comments
<adeuring> gmb: great, thanks!
<bigjools> nigelb: mrevell is irreplaceable
<nigelb> bigjools: Ha, I tend to agree ;)
<adeuring> gmb: good points!
<bigjools> nigelb: I vote he clones himselfe
<bigjools> himself, even
<nigelb> haha
<gmb> adeuring: Well, I took jtv's comments about not doing rubber stamp reviews any more to heart :)
<benji___> gmb: have a minute for a medium sized review?: https://code.launchpad.net/~benji/launchpad/bug-742662/+merge/74254
<gmb> benji: Sure.
<nigelb> sinzui: I moved that line into the macro, could you check if its done right?
<sinzui> okay
<sinzui> nigelb, +1
<sinzui> nigelb, I approved the whole review since you addressed bac's concerns
<nigelb> sinzui: thanks :)
<sinzui> nigelb, I will merge this for you
<nigelb> sinzui: Oh, could you land it as well?
<nigelb> \o/
<nigelb> Thanks
<bac> thanks sinzui
<jtv> gmb: hah!  you actually read that stuff?  I never thought anyone would.
<gmb> benji: Approved. I don't actually have any comments... were there any questions you expected?
<gmb> jtv: I generally pay attention to your emails on the flip of a coin, truth be told...
<benji> gmb: if you have no questions then I did my job
<jtv> gmb: you're being more interesting than usual yourself because this time money seems to be involved.
<gmb> :)
<jtv> :-P
<nigelb> jtv: That was good stuff :)
<jtv> nigelb: thanks, I tried English-tradition cocoa instead of Dutch this time.  It's easier to mix.
 * nigelb blinks
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 254 - 0:[#######=]:256
<mtaylor> sinzui: hey! is there a way to bump a bug import request? it's been a few weeks now...
<nigelb> mtaylor: He's taking an IRC classroom session atm
<mtaylor> nigelb: ah. cool
<mtaylor> well - if any losas are around, this: https://answers.launchpad.net/launchpad/+question/168463 as been open since 18 August
<thedac> mtaylor: I will take a look
<mtaylor> thedac: thanks!
<sinzui> mtaylor, I think we need to shout for a LOSA's attention. I expected it to be done with a day of when I set the question back to open
<benji> sinzui: is there anything left to do on https://answers.launchpad.net/launchpad/+question/168463 ?
<thedac> mtaylor: bug-import has been run on production
<mtaylor> thekorn: thank you!
<mtaylor> gah
<mtaylor> thedac: thank you!
<mtaylor> thekorn: (sorry, bad nick completion)
<thedac> np
<sinzui> benji, based on thedac and mtaylor's remark just after yours, I think the question is really answers and can be marked solved by mtaylor
<benji> sinzui: which remark... oh after refreshing the page I see "David Ames (thedac) said 6 minutes ago..."; thanks
<sinzui> benji, in irc. between your last two comments
<lifeless> abentley: still around ?
<abentley> lifeless: yes.
<lifeless> I've replied to your mail; if you want a voice chat about it, I could do that too.
<abentley> lifeless: I've replied to your reply, but I have one question: is the output of bin/test --list-tests a list of test-ids?
<lifeless> bin/test --subunit --list-tests
<lifeless> otherwise the zope test machinery inserts gunk in there.
<abentley> lifeless: AHHH!
<lifeless> indeed
<lifeless> thats what I said.
<lifeless> abentley: I think we're imagining basically the same thing
<lifeless> abentley: I hope I've added clarity, not confusion :)
<abentley> lifeless: So is the partitioning happening at a per-test level?  Or are you using information on past runs to estimate how long each test takes?
<lifeless> the latter
<lifeless> subunit streams are datestamped
<abentley> lifeless: makes sense.
<lifeless> we get per-test durations from this, and there is a small db in the .testrespository directory that holds all the seen test ids -> duration
<lifeless> so partitioning is make N buckets, allocate to the lowest accumulated time bucket the next largest test for all known test durations, then round robin for all unknown test durations.
<wallyworld_> sinzui: can you hear me?
<sinzui> no, but I am killing thunderbirf
<sinzui> I will restart mumble after the murder is complete
<sinzui> wgrant, lp:~sinzui/bzr-gtk/gtk3 works, though I am seeing gtk3 warnings that is why I delayed proposing it
<sinzui> StevenK, https://bugs.launchpad.net/launchpad/+bug/485126
<_mup_> Bug #485126: Accidentally unsubscribing myself from a private bug makes the bug inaccessible <disclosure> <lp-bugs> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/485126 >
#launchpad-dev 2011-09-07
<wgrant> wallyworld_, StevenK: Please don't force buildbot yet.
<wallyworld_> ok
<wgrant> wallyworld_: However, I do need an urgent review for https://code.launchpad.net/~wgrant/launchpad/connectionstring-hates-production/+merge/74321
<wgrant> If you have a sec.
<wallyworld_> sure
<wgrant> Copyright year fail.
<wgrant> Fixing.
<wallyworld_> wgrant: shoud the test for changes include the case where the key being changed is in the original connection string, as well as just adding a new key value?
<wgrant> wallyworld_: Probably, yes.
 * wgrant fixes.
<wallyworld_> thanks
 * wallyworld_ wants longpoll NOW. hitting refresh on a mp is a pita
<wgrant> Diff has updated.
<wallyworld_> wgrant: r-me. thanks
<wgrant> Thanks.
<StevenK> wgrant: r13862 is marked as bad, what rev is the relevant fix/rollback?
<StevenK> [r=henninge][no-qa] Sort utilities/sourcedeps.cache. Finally.
<StevenK> Oh, Henning!
<StevenK> G: You have two revs to QA -- bug 697157 and bug 841658
<_mup_> Bug #697157: Streamline "You have assigned this bug to yourself" message  <bugs> <qa-needstesting> <trivial> <ui> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/697157 >
<_mup_> Bug #841658: DistroSeries/DistroArchSeries API link does not return any entries <api> <qa-needstesting> <trivial> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/841658 >
<G> StevenK: jtv said he was going to QA 697157 because it's his responsibility ;)
<StevenK> wgrant: Any idea how to QA bug 841850?
<_mup_> Bug #841850: OpenIDRPSummary continues to exist <qa-needstesting> <Launchpad itself:Fix Committed by stevenk> < https://launchpad.net/bugs/841850 >
<G> but I'll test 841658 now :)
<G> StevenK: it's on qastaging now right?
<StevenK> G: Aye
<wgrant> StevenK: Acquire a user without PPAs (or ask me to use one of my test accounts), ensure that renaming displays the warning sensibly, and that acknowledging it works.
<StevenK> wgrant: I have a PPA, so I think it should be one of your test accounts
<StevenK> And I don't really want to launch into the mess of deleting it just to QA something :-/
<G> StevenK: yep, 658 looks correct, I'll mark it qa-ok
<wgrant> StevenK: You can't delete it, because qastaging doesn't have a publisher.
<StevenK> So not signing up for a new account on qas :-)
 * wgrant is doing it.
<wgrant> qa-ok
<StevenK> wgrant: So marked, thanks
<StevenK> No staticdiffs against Launchpad. Bah
<wgrant> Hm, sure?
<StevenK> Perhaps I should get the LOSAs to actually do the migration on qastaging ...
<wgrant> You didn't accidentally commit the migration, did you?
<StevenK> On DF? I don't think so
<wgrant> You can't use the presence of staticdiffs for QA.
<wgrant> Because you can't view prod diffs on qastaging.
<StevenK> launchpad_dogfood=# SELECT count(*) from branchmergeproposal where review_diff is not null and merge_diff is null; => 5251
<wgrant> Just check the a new preview diff works OK.
<wgrant> No need to QA the migration you did yesterday.
<StevenK> So create a new MP and check the diff gets generated?
<wgrant> yes.
<wgrant> Ideally check that revision mail still works too.
<wgrant> So...
<wgrant> Push up a new branch.
<wgrant> Subscribe to revision mail.
<wgrant> Push up some new revisions.
<wgrant> Scan the branch.
<wgrant> Push up some more new revisions.
<wgrant> Scan the branch again.
<wgrant> Uncommit and push.
<wgrant> Scan the branch again.
<wgrant> Create a merge proposal somewhere (doesn't need to be same branch), and check that the diff is generated.
 * StevenK works through that list slowly
<StevenK> According to https://code.qastaging.launchpad.net/~stevenk/+junk/foo I'm already subscribed?
<wgrant> Not to revision diffs.
<wgrant> Probably only to merge proposals.
 * StevenK changes his subscription
<StevenK> Do I need a LOSA to run the scanner?
<wgrant> Yes.
<wgrant>  45 files changed, 127 insertions(+), 801 deletions(-)
<wgrant> ow
<StevenK> wgrant: The scanner has been run, but the branch page still says Updating ...
<Lifelrss> Way too much cron spam
<wgrant> Yes.
<wgrant> It is mildly insane.
<wgrant> Just don't look at the staging mailbox.
<Lifelrss> It's disturbing cynthia. Can fix ?
<wgrant> BHeh
<wgrant> Well. Part of it needs an RT that is way down the queue fixed.
<wgrant> Other parts need hloeung.
<Lifelrss> The apt pyrge thingy?
<wgrant> Yes.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 256 - 0:[########]:256
<Lifelrss> Meep
<wgrant> We got to 260 on Monday :)
<Lifelrss> Uhm, perhaps that ticket should be higher. It's not in fhe losa queue
<Lifelrss> Afaik
<wgrant> LOSAs can't do it.
<wgrant> So perhaps it shouldn't be in the LOSA queue.
<wgrant> But it is in the LOSA queue.
<wgrant> #9
<Lifelrss> Right :)
<wgrant> You have priority powers, feel free to use them :)
<Lifelrss> Talking to a gsa will be fadter
<wgrant> Nobody else subscribed to the list has powers.
<Lifelrss> Sm kn phone
<wgrant> Heh.
<StevenK> What the heck are you typing on?
<StevenK> Or *with*
<wgrant> Im on phone
<wgrant> I assume.
<Lifelrss> Yes
<hloeung> lifeless may be teaching cynthia how to type already
<wgrant> Or perhaps Cynthia is proxying for him.
<wgrant> Heh
<StevenK> It could have been a dvorak keyboard via a pencil in the mouth given the typos
<Lifelrss> cinerama: ^
<Lifelrss> If you could move that ticket around for yts that would be awesome
<wgrant> https://rt.admin.canonical.com//Ticket/Display.html?id=47596 is the ticket.
<wgrant> But there is no vanguard today.
<StevenK> cocoplum has one too
<wgrant> Yeah, there are lots.
<StevenK> I saw it with the paste from the dapper destruction
<wgrant> I really should just have reverted Chameleon while it was easy :/
<nigelb> Woah, Cynthia is already on IRC?
<StevenK> She should post her first branch in a few days
<StevenK> First landing in trunk in what, a week?
<nigelb> I guess so
<nigelb> buildbot failed again :(
<wgrant> Yeah. Something leaked a forkingservice again.
<wgrant> 1:20 to go on the next run.
<wgrant> Jenkins remains stable.
<StevenK> It's hard to leak stuff between runs when the entire instance is shot in the head
<jtv> Is it okay for a package publication to be superseded but not have supersededby set?
<jtv> StevenK, wgrant: this may happen if we allow multiple versions of a package to survive domination: a publication may become superseded but without a clear dominant.  The schema allows it, but won't it break anything?
<wgrant> jtv: No, it's only used by the UI.
<jtv> So it's fine to say "superseded, but not by any particular release"?
<wgrant> jtv: And deletions from before ArchiveRemovalRedesign are already Superseded without a supersededby
<jtv> Very very few.
<wgrant> Try to avoid it, but it's legal and acceptable.
<wgrant> How is there not a clear dominant, though?
<wgrant> Can't you just use the oldest live one after the publication?
<jtv> Idiot.
<wgrant> Oh?
<jtv> Heh.  I'm asking about the case where there isn't one, of course.
<wgrant> If there is no newer live publication, isn't it probably Deleted?
<jtv> That's the kind of thing I'm asking about.  Think "latest version of a package is removed from Debian Sources list, leaving only the previous version."
<cinerama> lifeless, wgrant: ok, i see the RT, i'll just go and see what we are going to do
<jtv> I could mark that as superseded (but by which version?) or as deleted.
<wgrant> jtv: Seems likely it's Deleted if there's nothing to supersede it.
<wgrant> "likely"
<jtv> How would we know the difference?
<wgrant> cinerama: Thanks. I know the issue fairly well, so can hopefully clarify anything that needs it.
<wgrant> What difference?
<wgrant> If there something newer alive -> superseded
<wgrant> Otherwise -> deleted
<jtv> Between a deleted one and a superseded one in this situation?
<jtv> So you meant "likely" as in "I'm not sure"?
<wgrant> No, I'm sure that it's true in most cases.
<jtv> Then how do we tell the difference between cases where it is and cases where it isn't?
<wgrant> We probably can't.
<wgrant> The status doesn't make a huge difference. I think the method I described will get it right just about all the time.
<jtv> OK
<wgrant> G: Around?
<G> wgrant: yo
<wgrant> G: Do you have a moment now to QA bug #697157?
<_mup_> Bug #697157: Streamline "You have assigned this bug to yourself" message  <bugs> <qa-needstesting> <trivial> <ui> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/697157 >
<wgrant> G: I can grab the mail from qastaging's mailbox.
<wgrant> Or I can do the whole thing.
<G> wgrant: jtv, said he was going to QA it but if you want to
<G> wgrant: I'll create two bugs and you can do the rest if you want
<G> wgrant: your LP login is wgrant I take it?
<wgrant> G: Indeed it is.
<wgrant> Note that it can't be a bug on any of the Launchpad projects, because I'm already subscribed to all of them.
<G> wgrant: you should have got a correct message about me assigning you https://bugs.qastaging.launchpad.net/pymheg2xmltv/+bug/800256
<_mup_> Bug #800256: Blue Triangle, top left, next to BFB. <Ayatana Design:New> <unity:Invalid> <unity (Ubuntu):Invalid> < https://launchpad.net/bugs/800256 >
<wgrant> Nigel Jones (dev-nigelj) has assigned this bug to you for MHEG5 to XMLTV Converter:
<G> wgrant: and if you assign https://bugs.qastaging.launchpad.net/pymheg2xmltv/+bug/800257 to yourself...
<wgrant> You have assigned this bug to yourself for MHEG5 to XMLTV Converter:
<wgrant> Looks good. Thanks.
<cinerama> wgrant: ok, so i've removed the egg info and reinstalled where i found python-tz, didn't see python-setuptools as i was expecting
<wgrant> cinerama: The package often isn't installed any more.
<wgrant> In that cause just remove the directory.
<wgrant> s/cause/case/
<G> wgrant: np, happy to help
<G> is there another downtime-less update happening overnight or something?
<wgrant> With a bit of luck we'll do one in about an hour.
<wgrant> Need to QA incoming bugmail now, though :(
<G> back to getting a sane xmltv file for my mythtv setup :)
<wgrant> Good luck with that.
<wgrant> I never had much luck.
<G> wgrant: I'm processing the MHEG data directly :)
<wgrant> G: Doesn't MythTV normally do that itself?
<wgrant> Ah, no, that's not MHEG.
<G> I've been using an xmltv file that someone creates using the Satellite EIT data, but the DVB-T (OTA) service doesn't really have much EIT data
<cinerama> wgrant, lifeless: ok, looks like we should be rid of those bad eggs
<wgrant> cinerama: Thanks. Which hosts did you clean?
<G> problem is, the Satellite data doesn't always match the DVB-T data :)
<cinerama> wgrant: the ones mentioned in your ticket for hloeung
<wgrant> cinerama: Thanks. Logging changes over the last couple of days have revealed four more that need fixing :( I've just updated the ticket.
<wgrant> hloeung: ^^
<cinerama> wgrant: ok, i think we've tidied up the others now
<wgrant> cinerama: Great. There's probably another host or two that isn't spamming dozens of times an hour and has been drowned out until now, but I'll update the ticket later if so.
<wgrant> Thanks!
<cinerama> wgrant: sounds good.  i am going to leave the ticket with the losas since they can do most of it and can just poke a gsa for help
<wgrant> Oh, can they?
<wgrant> Didn't know that :(
<wgrant> The staging mailbox has gone three minutes without a dozen messages a minute. This is good progress!
<hloeung> we can poke GSAs for help - that's "most of it"
<wgrant> Heh
<cinerama> yay, let's open some champagne to celebrate
<wgrant> cinerama: Hm, tellurium seems to still have pytz.
<wgrant> ... and asuka just showed up with 11 pytz-whining emails too :(
<cinerama> ugh, need to clean up /usr/lib/python2.6 stuff as well as /usr/share/pyshared stuff
<cinerama> brb
<wgrant> germanium too.
<wgrant> nigelb: Your webfonts stuff looks good.
<wgrant> Hm, actually.
<wgrant> It's fine in Chromium, but Firefox 7 doesn't display the bold fonts properly.
<wgrant> It seems to scale up the normal weight.
<lifeless> cinerama: wgrant: thank you both very much
<cinerama> there still appear to be some broken links for eggs in that area for packages you haven't mentioned -- do i need to fix those up as well?
<wgrant> cinerama: They're not being noisy, but if you want to...
<cinerama> ok, i think i've properly dealt with pytz
<adeuring> good morning
<wgrant> cinerama: Thanks.
<nigelb> wgrant: Oh. Its in production? \o/
<StevenK> It is not
<nigelb> ah, qastaging.
 * StevenK digs for canonical_url
<StevenK> So tempted to move it to lp.app
<StevenK> But the diff would be LARGE
<wgrant> nigelb: I think it may need further work; see the bug for details.
<nigelb> wgrant: I saw. Need to get back home to test on my machine.
 * nigelb just got to work
<nigelb> I'm using Firefox 8 though.
<nigelb> wgrant: Do you want to qa-bad that or just want me land a better fix later today?
<wgrant> nigelb: I don't think it's worth marking it bad.
<wgrant> Particularly given we're already fairly blocked.
<nigelb> heh, ok :)
<wgrant> Some bold fonts will look bad on some browsers for a day. Meh.
<nigelb> Yeah, stub marked my email fix as qa-untestable for faster deployment :)
<wgrant> That deployment sadly failed.
<nigelb> Oh. Why?
<nigelb> :(
<wgrant> Changes to DB connection string handling meant that hostnames with hyphens didn't work. Staging and dev use underscores, production uses hyphens, boom.
<StevenK> Do we have a ... um, something for a "Are you sure?" action?
<wgrant> There is a new confirmation overlay that rvba or allenap has been working on.
<wgrant> And the bug assignee picker already has a confirmation page at the end.
<allenap> wgrant: rvba I think.
<rvba> StevenK: ConfirmationOverlay
<rvba> StevenK: It's meant to be used to 'protect' an existing submit button. Very much like the js 'confirm' method.
<mrevell> Howdy
<allenap> Morning mrevell :)
<allenap> Who wants to do a review?
<allenap> https://code.launchpad.net/~allenap/launchpad/pcj-more-often-bug-770721-pre/+merge/74293
<allenap> Please :)
<StevenK> allenap: r=me
<allenap> StevenK: Thanks :)
<lifeless> jamesh: am I right that the only missing thing to migrate to oops-wsgi is an answer for storm query capturing ?
<jamesh> lifeless: that's the main one I've identified.  That said, we've got code to handle this, so I could wire it up with python-oops
<jamesh> lifeless: I need to follow up with a few others on the U1 team though to see if there is anything else.
<lifeless> the thing blocking me extracting the tracer we use from LP is identifying 'current request'
<lifeless> because 'current transaction' doesn't seem like a good match
<jamesh> lifeless: https://wiki.canonical.com/UbuntuOne/Futures/Notes/PythonOops has my in-progress notes for moving over.
<lifeless> unless we inject 'this is the oops-in-progress' into the tracer from a wsgi decorator
<lifeless> in which case we need a storm-timeline tracer (which could go in storm itself), and a storm-wsgi which would have a thin wsgi app to set and unset the timeline that the tracer should use
<lifeless> does this sound sane ?
<jamesh> lifeless: one other case I'd like to eventually handle (as in "I don't think we currently do it but it would be nice") would be tracing statements in the API server
<lifeless> what sort of statements ?
<lifeless> do you mean like a custom debug hook that logs rather than profiling/debugging ?
<jamesh> it is a twisted daemon that performs DB transactions via deferToThread()
<lifeless> cause that would be the bomb.
<jamesh> still storm statement tracing, but not in a "one thread per request" model
<lifeless> oh right
<jamesh> I guess a "this is the current timeline" API would handle that though.
<lifeless> uhm, so that seems straight forward, have a wrapper - deferToThread(setTimeline, timeline, realthingtocall, realparams)
<jamesh> right.
<StevenK> rvba: Are you off the stand-up?
<jamesh> lifeless: I haven't looked the python-timeline package you're using in LP yet.  I guess it would be worth looking at together with the other OOPS stuff though.
<lifeless> jamesh: its tiny, but yeah
<lifeless> jamesh: we put memcached call-outs, google search API calls, librarian operations, storm queries all into the timeline
<lifeless> jamesh: lets us see slow backend API's
<wgrant> (and SSO!)
<rvba> StevenK: not yet.
<jamesh> lifeless: I guess that is a bit more structured than just capturing all logging messages to the oops report ...
<jamesh> [that seems to be the equivalent functionality from the wsgi-oops package we're using]
<lifeless> yeah
<lifeless> log messages are fine
<lifeless> but this is introspectable
<lifeless> so we can get duration, which log messages don't have
<lifeless> and that lets oops-tools tell us that '5400ms were taken up by repeated query Y'
<lifeless> and be confident that its that, and not slow python code post-query, for instance.
<jamesh> yep.
<lifeless> you could do that with log messages with a well-known-form
<rvba> StevenK: ?
<lifeless> but IMO it gets a bit ugly having a heuristic to ditch ones not matching it
<jamesh> I'm not suggesting you switch to the logging module.
<lifeless> no :)
<lifeless> I was just being verbose
<lifeless> jamesh: do you think the tracer belongs in core storm? will add the dependency on the timeline module for testing at least.
<StevenK> rvba: You're working on a confirmation overlay?
<jamesh> lifeless: we've got a number of optional modules in Storm already: storm.zope (which you're using) and storm.django (which we're using).
<jamesh> one more wouldn't be out of place
<rvba> StevenK: It's already in the code actually: lib/lp/app/javascript/confirmationoverlay/confirmationoverlay.js
<StevenK> rvba: Nice!
<rvba> StevenK: It's used on +localpackagediffs to add a confirmation overlay when you're about to sync stuff.
<StevenK> rvba: I was about to ask
<StevenK> I was going to use it to solve a disclosure bug
<rvba> StevenK: See lib/lp/registry/templates/distroseries-localdifferences.pt
<rvba> StevenK: I hope it can suit your needs without modification.
<StevenK> rvba: That would be nice, I'll dig tomorrow, thanks for the help.
<rvba> StevenK: np.
<rvba> StevenK: when it's not timing out, you can see it in action and play with it https://dogfood.launchpad.net/ubuntu/oneiric/+localpackagediffs
<jtv> Is it just me or are EC2 tests not working?
<bigjools> jtv: I dunno, are you not working?
<jtv> bigjools: I am, but ec2 isn't.
<jtv> I get timeouts and hanging instances.
<rvba> jtv: I lannched an instance 1 hour ago without problem.
<jtv> Thanks.  Blast.
<lifeless> stub: hey there
<stub> hiya!
<lifeless> stub: python-pgbouncer seems to have some skew in its version numbers - __init__ says 0.0.1, setup.py 0.0.2 and pypi 0.0.3 ;)
<lifeless> stub: I'm really glad you updated it :)
<lifeless> stub: I was worried I hadn't setup enough permissions
<stub> setup.py should have 0.0.3 too since that is what is used to build and upload the package to pypi
<stub> Didn't think of a number in __init__, but now I think of it I've had setup.py pull the version number from __init__ before.
<lifeless> yeah, me too in other projects
<lifeless> need to copy that glue across
<lifeless> hey, so fastdowntime seems to be awfully close
<lifeless> nice work
<stub> Just trying to get the know issues stuff landed.
<lifeless> yah
<lifeless> we always knew there would be rabbits in holes.
<lifeless> its pretty awesome getting it all flushed out
<stub> I'll hack around lag mon with PGPORT instead of --port so I don't have to worry about that crud for now. I think the wanted revs are otherwise in place now, which I'll check after I've updated these rollout scripts for ISD.
<lifeless> lynne is having some trouble, I -may- not be back next week. I'll let you know when I let francis know (which will be ~ 5 minutes after I know :P)
<stub> Yup. No worries. Reviews have been pretty boring.
<lifeless> they always are... until you get one that terrifies :)
<stub> need to ensure we don't lose google juice with async loading of comments, and privacy is making some of the branch queries go recursive (if your stacked on branch is private, you are private too even if you claim to be public). But the queries I've seen so far are fast.
<lifeless> I believe google have js execution in their crawlers now
<lifeless> to handle twitter et al
<stub> Good luck with Lynne and the spawn too!
<lifeless> thanks ;)
<lifeless> 'spawn' is smiling and gurgling cutely already.
<wgrant> I think we probably want a boring old batched comment listing, as well as the async loading.
<lifeless> async should batch as well
<stub> yer, thats what I said. I don't think we lose with what is behind the FF at the moment either.
<lifeless> faster for the only-reads-a-few case
<lifeless> and permits scroll-to-end etc
<lifeless> but thats efuture :)
<wgrant> stub: We should be OK to deploy the port fix in an hourish.
<wgrant> stub: Did you see the fireworks overnight?
<stub> Apart from the bug being flagged as rolled back, I don't know what happened.
<wgrant> Well, ConnectionString thought that connection string values were in \w+
<wgrant> \w is roughly [a-zA-Z0-9_]
<wgrant> Production DNS aliases have hyphens...
<wgrant> So host=lp, and boom.
<stub> blah
<wgrant> And because of our marvellously unified database connection mechanisms, appservers were not affected so it wasn't noticed for a while.
<wgrant> Still, Zopeless is now very nearly dead.
<stub> wgrant: So you have landed the fix and it is with ec2 or buildbot atm?
<wgrant> stub: It's been on qastaging for a few hours now.
<wgrant> Just waiting on three revs of QA.
<wgrant> bigjools, henninge: ^?
<stub> Thankyou very much. I should sleep in more often :-)
<henninge> wgrant: will be a little longer
<wgrant> The connection string syntax supports escaping and quoting, but I decided we needed a quick fix so just added an assertion to prevent unnoticed misparsing of those cases.
<wgrant> So it just uses [^ ]+ instead of \w+
<bigjools> patience
<bigjools> ffs
<stub> Yes, I think that is the better approach too
<wgrant> stub: If someone tries to use a space in a username, hostname or database name, they probably deserve a crash :)
<stub> Yes :-) On my list is to switch all the usernames using hyphen to using underscore too, and block hyphen in security.py. Quoting the names is annoying.
<wgrant> Yeah, although that's fairly well encapsulated now.
<wgrant> Before I rewrote half of it it was quoting and unquoting everywhere :(
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/more-zopeless-destruction/+merge/74367 may be of interest
<wgrant> Pretty simple continuation of the series.
<wgrant> But gets rid of most of the rest of ZTM's non-wrapper bits.
<henninge> wgrant: I am having a problem with that revision
<wgrant> Uhoh.
<henninge> wgrant: It is not fixing the bug, just movingn an exception being raised from the UI to a script run.
<wgrant> Heh.
<henninge> where it is more likely to cause problems
<henninge> or be ignored ...
<henninge> wgrant: Unfortunately there does not seem to have been a pre-implementation chat with translations people.
<wgrant> bad enough to need a revert?
<wgrant> It sounds pretty rare.
<henninge> I hope jtv is around for a second opion.
<jtv> what now?
<henninge> ;-)
<henninge> jtv: Hi!
<jtv> hi!
<henninge> jtv: can you please have a look at https://code.launchpad.net/~benji/launchpad/bug-742662/+merge/74254 ?
<jtv> OK
<henninge> jtv: I am not sure it is an improvement.
<jtv> Waitâ¦ we always did report mixed newlines, didn't we?
<henninge> jtv: yes, it currently happens when submitting translations.
<henninge> (see the OOPSes)
<henninge> in the UI.
<henninge> This branch moves the check into the importer.
<henninge> Which would be fine if the exception was caught and added to "errors".
<jelmer_> wgrant: hi
<henninge> jtv: I don't like uncaught exceptions in the importer ...
<wgrant> jelmer_: Morning.
<jelmer_> wgrant: I saw something in the logs about an import test leaving processes around?
<wgrant> jelmer_: Something is occasionally leaving a forking service around.
<wgrant> The buildbot logs aren't a timestamped subunit stream, so I can't tell what.
<wgrant> But you can see the sort of failure it happens around at https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1342/steps/shell_6/logs/summary
<wgrant> I don't know if that was the cause or effect.
<jtv> I guess my connection bounced.
<jtv> henninge: where does the exception currently go?  We meant to have separate sanitizers for web and import, didn't we?
<jelmer_> wgrant: ah, hmm
<jtv> henninge: I mean, where does the exception currently go in benji's code?  What catches it?
<henninge> jtv: nothing catches it AFAICT
<jtv> That would be bad.
<henninge> that's my problem with the code ;-)=
<henninge> jtv: line 302 of the diff is where it would get raised.
<jtv> In fact I'm not even sure this should be an error instead of a warning.
<gmb> bigjools: Can you give https://answers.launchpad.net/launchpad/+question/169630 a decent canonical answer? maxb has already answered it but there seems to be some confusion.
<henninge> jtv: well, at least the importer has self.errors to which it should be added instead of halting the whole import by raisingn an exception.
<bigjools> gmb: I can look
<gmb> bigjools: Thanks.
<bigjools> gmb: dear lord some people are thick
<maxb> Oh that one. I started ignoring it because the user seemed lacking in clue
<jtv> henninge: absolutely.  There are cases where a bad newline will confuse things, but I don't see how they could lead to accidental acceptance of the import or anything disastrous like that.  So the parsing at least can proceed.
<gmb> Ah. I figured that might be the case, but I'm sufficiently clueless myself that I couldn't actually tell for certain.
<wgrant> maxb: But they seem to maintain a package in Debian.
<henninge> jtv: for that reason, the introduction of an uncaught exception in the importer code, I'd like to not have this revision deployed.
<wgrant> maxb: Which is a worry.
<maxb> Yes. Quite.
<jtv> henninge: agreed.  How did it get through Q/A?
<wgrant> This *is* QA.
<henninge> jtv: We are doing it atm. ;-)
<jtv> Ah OK
<henninge> wgrant: rollback, sorry.
<jtv> This is pre-imp stuff.
<henninge> yup, and that was lacking here.
<jtv> But it's hard to know that at pre-imp time.
<jtv> Good thing you caught it.
<henninge> wgrant, jtv: can either of you please prepare the rollback branch? I cannot do that at this computer atm, I have to relocate to the office first which will still be a while.
<jtv> henninge: I'm not in a position to do that I'm afraid, and wgrant is even further past EOD than I am.  :/
<henninge> oh, right.
<bigjools> I finished my QA
<StevenK> henninge: Mark the bug as qa-bad bad-commit-13889
<StevenK> I think it's bad-commit-XXXXX
<henninge> I think so, too.
<henninge> StevenK: done.
<stub> StevenK: You are handling the rollback?
<henninge> StevenK: will you do the rollback?
<wgrant> I'm on my freshly reinstalled laptop which has no keys at the moment.
<henninge> ;)
<wgrant> So I'm afraid I can't help.
<wgrant> buildbot is ready and waiting for the rollback, however :)
<StevenK> Yeah, alright.
<henninge> StevenK: thank you!
<StevenK> wgrant: Give me the bzr command, since I'm lazy and waiting for rf?
<wgrant> bzr merge -r1234..1233 .?
<StevenK> Right
<StevenK> I tend to do bzr di | patch -p0 -R which is a little disgusting
<wgrant> That's very CVS.
<StevenK> Guess where I learnt revision control ...
<StevenK> henninge: One line description for the rollback?
<bigjools> I try not to remember that I learned revision control in CVS
 * gmb lunches
<henninge> "[rs=henninge][rollback=13889] Rollback r13889 because it introduces an uncaught exception in translation import code."
<henninge> StevenK: ^
<henninge> like that?
<StevenK> Looks good.
<cjwatson> I'm pretty sure I originally learned revision control in RCS, which I'm not sure is an improvement
 * henninge lunches
 * wgrant only seriously learned version control with svn :(
<StevenK> henninge-lunch, wgrant, stub: Revert landed.
<wgrant> Thanks.
<StevenK> Now to wait 11 minutes for buildbot and then 4h20, and then 60 minutes.
<StevenK> Then we can think about deploying
<bigjools> svn isn't much better
<nigelb> I learned bzr, then git, and then svn.
<nigelb> Backwards, yeah.
<StevenK> CVS, then how to fix broken CVS repos, svn, svk (twitch), tla, baz and then bzr
<StevenK> I am allergic to git
<nigelb> There are creams for that :P
<StevenK> I no longer have a reason to learn it
<bigjools> I tried to use git for another OSS project I play with, and found it unfathomable
<bigjools> I think it's the colo branches that annoy me the most
<wgrant> bigjools: bzr-git :)
<bigjools> yes :)
<bigjools> does it work properly now? it had some bug that stopped me importing some kinds of git repos
<wgrant> The only failures I know of now are lack of submodule support and lack of support for backslashes in paths.
<nigelb> bigjools: I kinda like colo branches :)
<bigjools> nigelb: in some cases yes, I use a colo checkout in bzr but I just found the git interface to them confusing.  Having said that, I found bzr rather confusing initially too.
<bigjools> The help text has a nasty habit of defining some of bzr commands using the same word in the definition.
<nigelb> bigjools: I use a zsh extension to lessen the confusion of git branches
<bigjools> you use *zsh* to *lessen* confusion?
<bigjools> IRONY KLAXON!
<nigelb> heh
<nigelb> zsh is nice when you use oh-my-zsh
<gary_poster> stub, hi you still around?
<gary_poster> nm stub, thanks
<stub> gary_poster: yo
<gary_poster> hey stub, I have a corrupted postgres that won't start after a hard drive incident (PANIC:  could not locate a valid checkpoint record) and was wondering if there were a quick way to wipe it away
<gary_poster> and start again
<gary_poster> I decided to go ahead with some overdue upgrade work and restart, but if there is a quick answer, that would be cool
<stub> gary_poster: Its /var/lib/postgresql/8.4/main
<wgrant> utilities/launchpad-database-setup should do it, too.
<wgrant> It drops the cluster and is OK at recovering from completely broken setups.
<stub> even better as then we don't need to recover the config
<gary_poster> ah-ha, cool thanks wgrant and stub, will give it a try
<gary_poster> yes, worked a charm.  thanks again
<sinzui> jcsackett, do you have time to mumble?
<wgrant> gary_poster: We should be able to deploy r13891 in about 4.5 hours. Will someone from your squad be able to arrange that?
<wgrant> gary_poster: QA is done; all that's left is one rollback revision that's currently in buildbot.
<gary_poster> yes, wgrant, we'll do it.
<wgrant> Thanks!
<wgrant> gary_poster: buildbot has been a little unreliable lately: if this test run fails, get a LOSA to check for and kill any orphaned processes on pigeonpea. bzr forking services and rabbitmqs have been leaking occasionally.
<wgrant> I sorted most of the flaky tests on Monday, but there still seems to one somewhere doing nasty stuff.
<gary_poster> wgrant, :-/ ok thanks for warning.  I saw the Monday cleanups, yes
<wgrant> So we just have to remember to clean the slaves every so often for now :/
<mthaddon> I'd rather we figured out why these processes are hanging around - seems like we've had to do this kind of clean up quite a lot recently
<gary_poster> yeah :-/
<jcsackett> sinzui: sure, firing up mumble in a moment.
<bigjools> wgrant: what's the situation with Rabbit versions and packages?  Your email left me a bit confused,
<wgrant> bigjools: I have 2.3.1 of the server and the required plugins packaged for lucid->natty. But 2.3 is getting pretty old now; 2.6 was released a couple of weeks ago. Oneiric has 2.5, so I will need to package plugins for at least 2.5 as well.
<wgrant> So I'm not sure if we want to go to 2.5 everywhere, or even 2.6, depending on when this work is going to happen.
<bigjools> wgrant: so we need to backport and put it in the lp PPA
<bigjools> might as well go for the latest release provided it works on lucid and oneiric/natty
<bigjools> wgrant: are you the only one making packages at this point?
<wgrant> I think latest is probably best. It's not as if we're upgrading a core part of the system to an non-battletested release.
<wgrant> Well, the server packages and plugin infrastructure are maintained in Ubuntu.
<bigjools> indeed
<wgrant> 2.6 isn't there yet, but I have 2.6 server packages already.
<bigjools> I want to avoid a custom package
<wgrant> Hopefully we can get the extra plugins into Ubuntu for oneiric+1.
<bigjools> yes - which reminds me to send an email
<bigjools> what is not there right now that we need?
<wgrant> Check https://launchpad.net/~wgrant/+archive/rabbitmq
<wgrant> rabbitmq-server and rabbitmq-erlang-client are straight backports of natty versions.
<bigjools> what about the management plugin?
<wgrant> rabbitmq-management depends on mochiweb, webmachine, rabbitmq-mochiweb and rabbitmq-management-agent.
<wgrant> None of those are in Ubuntu.
<bigjools> ok ta
<bigjools> are they in Debian?
<wgrant> mochiweb/webmachine are general Erlang libs, not rabbit-specific.
<bigjools> as in, packaged, or did you package?
<wgrant> They weren't two months ago when I packaged them initially, but that may have changed.
<wgrant> I packaged them.
<bigjools> arse
<wgrant> Not that there was much to it.
<bigjools> ok
<wgrant> But yes.
<wgrant> eg. for mochiweb: https://launchpadlibrarian.net/74782554/mochiweb_1.5.2-0ppa1.diff.gz
<bigjools> what is rabbitmq-erlang-client  for?
<wgrant> It's a rabbitmq client for erlang.
<bigjools> no shit sherlock :)
<wgrant> Some of the other plugins speak AMQP using it.
<wgrant> It *is* in Debian and Ubuntu, I believe.
<bigjools> I was wondering what needs it since it wasn't mentioned in your dep list above
<wgrant> I think management-agent depends on it.
<wgrant> It's in Ubuntu, but not Debian.
<bigjools> ok
<allenap> abentley: Hi. Line 45 in https://code.launchpad.net/~allenap/storm/json-property/+merge/74428 looks a bit like a manifestation of bug 510052, but I can't figure out what I did wrong. Could you help me out?
<_mup_> Bug #510052: Incorrect Merged proposal diff produced with pre-requisite option <code-review> <lp-code> <udd> <Launchpad itself:Invalid> <Ubuntu Distributed Development:Invalid> < https://launchpad.net/bugs/510052 >
<wgrant> rabbitmq-management and rabbitmq-management-agent both depend on rabbitmq-erlang-client
<bigjools> coolio, thanks
<bigjools> so we need the lot
<wgrant> And the packages are all pretty trivial.
<abentley> allenap: if it is a manifestation of bug 510052, then there's nothing you did wrong.
<_mup_> Bug #510052: Incorrect Merged proposal diff produced with pre-requisite option <code-review> <lp-code> <udd> <Launchpad itself:Invalid> <Ubuntu Distributed Development:Invalid> < https://launchpad.net/bugs/510052 >
<allenap> abentley: Bug 510052 is marked Invalid, so I wondered if I had actually done something wrong instead, Occam's Razor and all that.
<_mup_> Bug #510052: Incorrect Merged proposal diff produced with pre-requisite option <code-review> <lp-code> <udd> <Launchpad itself:Invalid> <Ubuntu Distributed Development:Invalid> < https://launchpad.net/bugs/510052 >
<abentley> allenap: There is a legitimate bug where incorrect diffs are produced, but it's fairly rare.
<wgrant> sinzui: Ah, I'm glad it was not just me who ran your suggested test case blindly!
<abentley> allenap: So, I merged jkakar's branch into lp:storm, and it produced the conflict you see.
<allenap> abentley: My branch merges trunk and resolves those conflicts. Should I have started from trunk and merged Jamu's branch instead?
<bac> abentley: i have made the change to combine DerivedAuthorization and ForwardedAuthorization.  i chose a new name DelegatedAuthorization, which i think is more descriptive.  you can see the diff at http://pastebin.ubuntu.com/684417/
<bigjools> Delegated makes me think of lazr.delegates
<bigjools> or whatever it is
<bac> i considered that but they are different beasts
<bigjools> I think your original name was the best!
<abentley> allenap: yes, you would have needed to fix the conflicts in Jamu's branch.  I think the conflict removal is shown because that's one of the differences between your branch and Jamu's.
<allenap> abentley: Okay, cheers.
<abentley> allenap: Looks good.  I expect we'll want to support different permissions per object at some point, but we can cross that bridge when we get to it.
<allenap> abentley: Was that meant for someone else?
<bac> allenap i think that was for me
<abentley> bac: Looks good.  I expect we'll want to support different permissions per object at some point, but we can cross that bridge when we get to it.
<abentley> allenap: yep, thanks.
<bac> thanks abentley
<jkakar> allenap, abentley: Hi! :)
<abentley> jkakar: hi.
<jkakar> allenap: Thanks for picking up my branch and finishing it off.
<jkakar> allenap: Should I just merge it into my branch and get therve to approve it, so I can land it?
<allenap> jkakar: No worries, it was nice to see that what I was thinking of was already there :) And sure, that would be great.
<jelmer_> has something changed in the handling of merge proposals recently? They seem to be timing out more often than previously, even on a project like bzr
<allenap> jkakar: Fwiw, the diff in my merge proposal is now free of conflict markers, so you can see what you're getting easily.
<allenap> jkakar: Garg, no it's not. Ignore me :)
<jkakar> allenap: Heh
<jkakar> allenap: Okay, I'll merge it into my branch and harass therve. :)
<allenap> Awesome, ta :)
<bigjools> wgrant: I need to specify which rabbit package versions we want in oneiric+1.  AFAIK we don't *need* the latest and you packaged 2.3.1 but I suspect they'll just take whatever you packaged.
<bigjools> any preferences?
<wgrant> bigjools: What I have already can't go into oneiric+1, since it's lower than what oneiric has.
<bigjools> ha
<wgrant> And releasing the LTS with a very old RabbitMQ would be foolish.
<bigjools> well, yes
<wgrant> I suspect oneiric+1 will end up going with whatever Debian has at the time, but maybe not.
<fjlacoste> bigjools: did you get a volunteer yet?
<bigjools> fjlacoste: OTP, but no
<jkakar> allenap: The branch is merged, thanks!
<allenap> jkakar: \o/
<allenap> stub: I have an update to lp:~launchpad-committers/storm/with-without-datetime. What do I do next? My guess: (1) push with with-without-datetime, (2) build egg (how?), (3) put egg tarball in lp:~launchpad/lp-source-dependencies/trunk, (4) update versions.cfg in Launchpad.
<stub> Yes, 1) update lp:~launchpad-committers/storm/with-without-datetime 2) python setup.py egg_info -b-lpwithnodatetime-rXXX sdist
<stub> Twiddle the -b option until the generated filename is correct
<stub> and the rest you know
<allenap> stub: Brilliant, thank you.
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 256 - 0:[########]:256
<adeuring> abentley: fancy a review of a not-too-big diff? https://code.launchpad.net/~adeuring/lazr.batchnavigator/lazr.batchnavigator-result-length-from-range-factory/+merge/74459
<abentley> adeuring: sure.
<adeuring> thanks!
<bac> abentley: can you do a review at your leisure?  https://code.launchpad.net/~bac/launchpad/bug-838957/+merge/74460
<abentley> bac: sure.
<abentley> adeuring: "should" misspelled on 29.
<adeuring> abentley: right. fixed locally.
<adeuring> ...and line 30 has a typo too...
<abentley> adeuring: What do you think about saying "rough_length" or "approx_length" or something instead of "result_length"?
<adeuring> abentley: good idea! let's use rough_length
<abentley> adeuring: r=me
<adeuring> abentley: cool, thanks!
<abentley> bac: I think we're recommending catching exceptions using the "as" syntax, e.g. "except ValueError as e"
<stub> adeuring: You found estimateRowCount in lib/canonical/database/postgresql.py ?
<adeuring> stub: not yet; thanks for the hint! will make work on my next branch easier :)
<bac> abentley: good catch.  i haven't gotten into that habit yet
<stub> adeuring: Feel free to drop that function and create something more useful with Storm. I don't think there are any callsites.
<abentley> bac: r=me
<bac> tx
<bac> er, thanks
<adeuring> stub: ok, I'll check it
<abentley> gmb: I think your code could be tested using MockIo.  Have you taken a look at that?
<gmb> abentley: Ah, ISWYM. No, I hadn't thought of that - been too wrapped up in getting the damn thing working. I'll do that now, thanks for the suggestion.
<abentley> gmb: no problem.
<abentley> gmb: Even when we get those integration testing facilities, we should avoid using them where we're not actually testing integration.
<gmb> Fair enough.
<nigelb> jelmer_: could you /nick to jelmer?
<nigelb> Classbot is complaining that you're around
<nigelb> :)
<nigelb> And you can /join #ubuntu-classroom-backstage in case you need to poke us for help :)
<nigelb> can I tag a bug that's  fixed again if I'm improving the fix?
<sinzui> nigelb, we do not have a tag for that. It is best to report a new bug and add a comment to the old bug that you are working on an improvement
<nigelb> sinzui: excellent, ok
<sinzui> nigelb, I believe we will have bug linking by the end of the year
<nigelb> its the fonts bug. Needs a bit more tweaking
<sinzui> I saw William's remarks. Your branch did fix the bug
<nigelb> ok,I'll open a new one and assign it to me
<sinzui> I was tempted to comment that most of the uses of italics in Launchpad are incorrect. They should be strong instead of em
<jelmer> hi sinzui
<sinzui> hi jelmer
<jelmer> sidnei: saw your gtk3 branch for bzr-gtk, thanks for that. I'll see if I can have time to do a review of it in the next couple of days.
<sinzui> thank you
<jelmer> grarr. lolcats are messin with mai englishz
<jelmer> s/can have/have/
<nigelb> How hard is it to fix bug 88545?
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/88545 >
<nigelb> abentley: Hi, could you do a quick review? (its a tiny diff) https://code.launchpad.net/~nigelbabu/launchpad/font-fixes-787798/+merge/74492
<abentley> nigelb: certainly.
<abentley> nigelb: r=me.  What's it doing exactly?  It looks like it's specifying normal, italic, bold, bold-italic.
<nigelb> Is including that woff from google web fonts
<nigelb> .woff
<abentley> lifeless: just ran the first concurrent test-farm run.
<nigelb> what does bug-columns tag mean?
<stub> abentley: \o/
<lifeless> abentley: cool!
 * abentley may need to change the name of this project, because Spinal Tap's "Sex Farm" keeps going around in his head.
<nigelb> haha
<stub> http://pgfoundry.org/pipermail/pgbouncer-general/2011-September/000851.html
<stub> Its hyphenated database name day
<nigelb> Is the team leads meeting notes new?
<nigelb> I've never seen it on the dev list before.
<sinzui> nigelb, They are not new. I believe they were sent to the wrong list :)
<nigelb> sinzui: aww, drat. I enjoyed reading them :)
<benji> abentley: do you have the time/inclination to do a lazr.restful review (it's really simple)? https://code.launchpad.net/~benji/lazr.restful/do-not-hide-original-tracebacks/+merge/74516
<abentley> benji: r=me, and this is a bug that I reported.  Thanks!
<benji> abentley: cool, I'll find the bug report so I can decorate it appropriately
<abentley> benji: I believe it's bug 832136
<_mup_> Bug #832136: tracebacks are obfuscated <lazr.restful:Triaged> < https://launchpad.net/bugs/832136 >
<benji> cool, thanks
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 256 - 0:[########]:256
<sinzui> wallyworld_, wgrant, stevenK: I think mumble does not want me to attend the standup
<wgrant> sinzui: We were able to hear you.
<wgrant> Both the first and second times.
<jelmer> sigh, I now seem to have traded in my password error for another error when trying to run the launchpad tests
<jelmer> 	psycopg2.OperationalError: terminating connection due to administrator command
<jelmer> 	SSL connection has been closed unexpectedly
<jelmer> does that look familiar to anybody?
#launchpad-dev 2011-09-08
<wallyworld_> jelmer: i've just noticed the same thing, don't know what causs it :-(
<jelmer> I can reproduce it on both of my machines
<wallyworld_> i've just upgraded to oneiric
<jelmer> my machines are both on oneiric
<jelmer> strangely enough the server logs indicate that the client disconnected suddenly
<mwhudson> ssl error?
<jelmer> mwhudson: yup
<mwhudson> why do we use ssl on a db connection to localhost? :)
<cjwatson> it isn't trying to use SSLv2, is it?
<wallyworld_> and why has it started failing now? what's changed?
<cjwatson> well, actually, that was turned off in natty
<cjwatson> but OpenSSL 1.0.0 in Oneiric turned off various other rarely-used bits of SSL
<cjwatson> things like MD2 hashing
<cjwatson> might be worth checking that
<wallyworld_> might explain why lucid hasn't any problems
<jelmer> cjwatson: that's an interesting thought
<jelmer> cjwatson: that's just libssl0.9.8?
<cjwatson> jelmer: well, it's libssl1.0.0 now
<cjwatson> there's only a handful of libssl0.9.8 reverse-deps left, and none that I think have anything to do with Launchpad
<jelmer> it seems unlikely that's the culprit
<jelmer> I have libssl1.0.0 1.0.0d-2ubuntu2, which was uploaded on aug 15; the breakage started at most two or three days ago
<cjwatson> well, it could have been some dependency switching over to it (they're different sonames), but yes, two or three days ago does make that a bit unlikely
<jelmer> hmm
<jelmer> I'll have a closer look tomorrow, thanks for the hints
<jelmer> wallyworld_: if you do stumble upon a solution.. please let me know :)
<StevenK> jelmer: Can you explain https://code.launchpad.net/~loggerhead-team/loggerhead/1.18/+merge/74532 ? It's full of conflict markers.
<wallyworld_> jelmer: will do. i'm currently looking at something else though
<jelmer> StevenK: argh, fail. I'll set it back to wip for now
<jelmer> the trunk *packaging* branch seemed to merge 1.18 just fine for some reason, looks like 1.18 and trunk don't :(
<wallyworld_> jelmer: i replaced host with hostnossl in pg_hba.conf and it works for now
<wgrant> Odd, it's working fine for me.
<wgrant> But it's in a container.
<wallyworld_> wgrant: i think lucid is ok
<wgrant> Container is up to date, host is not... maybe will upgrade the host too.
<wgrant> No, this is an oneiric container.
<wallyworld_> something seems to have changed wrt ssl that has broken things, so if you upgrade, be careful
<wallyworld_> gah. my "fix" didn't work a second time
 * wgrant upgrades the host.
 * StevenK stares at Thunderbird
<StevenK> "Canonical has 21524 new messages."
<StevenK> OMGWTFBBQ?!
<wallyworld_> StevenK: that's in the staging mailbox?
<StevenK> wallyworld_: No, my Canonical mailbox
<wallyworld_> oh. that's a shitload of messages then
<StevenK> It was telling lies, but it was still a little shocking
<G> 21.5k new messages, ouch
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[########]:256
<LPCIBot> Project db-devel build #849: FAILURE in 4 hr 12 min: https://lpci.wedontsleep.org/job/db-devel/849/
<wgrant> Erk
<wgrant> Ah.
<wgrant> legit failure.
<wgrant> Merge issue.
 * wgrant enfixorates.
<nigelb> Oh. I may have gotten a branch reviewed but forgotten to ask to land
<StevenK> nigelb: Which?
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/font-fixes-787798/+merge/74492
<StevenK> I wonder if that's totally pointless to run through ec2
<nigelb> It is.
<poolie> o/ nigel, stevenk
<nigelb> Morning poolie!
<StevenK> O hai
<nigelb> Oooh, the Ubuntu font thing landed. It looks good.
<jtv> And while we're at it: hi Nigel, hi antipodeans!
<nigelb> haha
<StevenK> Yeah, we deployed early AU morning
<nigelb> \o/
<jtv> If I may interrupt brieflyâ¦ is anyone willing to review this soyuzzy branch?  Julian said he would yesterday but evidently he didn't get around to it, and I've got other work building on it: https://code.launchpad.net/~jtv/launchpad/bug-832647/+merge/74380
<StevenK> % df -h /home | tail -n 1
<StevenK> /dev/mapper/sys-home   46G   42G  2.1G  96% /home
<StevenK> Hmmmm
<nigelb> what's /dev/mapper? lvm?
<StevenK> Yeah, LVM
<StevenK> Well, strictly, it's device-mapper, so could be LVM or something else than makes use it
<poolie> StevenK: you're almost done!
<StevenK> Hah
<StevenK> I still have 75G unallocated in the VG, so I could just grow /home ...
<nigelb> How hard is bug 88545. Is it even possible to do for someone without acces to the db?
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/88545 >
<StevenK> It's a DB patch
<poolie> nigelb: you need to make a patch that changes the db but you don't need access to the db to write it
<StevenK> But it may take postgres more than 2.5 seconds to perform ...
<poolie> probably the first thing is to just grep for statusexplanation and see if anything uses it at all
<poolie> eg to remove it from the activity log
<nigelb> Ah
<nigelb> do I need db-devel setup?
<nigelb> (that seems to be the branch where all db stuff happens)
<StevenK> So to remove it from the activity log, no, that can probably just land.
<StevenK> You want to make sure the model and interface no longer reference it
<StevenK> And then you write *just* a db patch to ALTER TABLE DROP COLUMN
<nigelb> Okay :)
<nigelb> Graduating to the bugs not marked easy or trivial :D
<StevenK> The drop column might perform poorly -- I suggest you talk to stub when he surfaces
<nigelb> Yeah, I think so as well.
<StevenK> But the first two steps should be easy enough
<poolie> i think you get a badge for fixing an mpt bug too :)
<poolie> if you haven't already
 * StevenK fixed a few at the Thunderdome
<StevenK> 'a moment ago' is my fault
<poolie> i think i have too
<nigelb> I've fixed way too many mpt bugs already
<wgrant> StevenK: column dropping is cheap.
<wgrant> StevenK: The rows don't get rewritten, the column just gets ignored.
<wgrant> The next time the row moves, it will be shrunk.
<StevenK> Does VACUUM clean it up?
<wgrant> Possibly not entirely.
<wgrant> But it's not important.
<wgrant> It only takes a bit of disk space, and it's not going to be a truly massive column.
<StevenK> It's still used, so that has to go.
<wgrant> Yes.
<jtv> Still want a reviewer for a soyuzzy branch!  Blocking me a little and reviewer didn't get around to me yesterday: https://code.launchpad.net/~jtv/launchpad/bug-832647/+merge/74380
<wgrant> jtv: Interesting tactic to teach the dominator about deletion...
<wgrant> (looking)
<jtv> wgrant: "interesting" as in "cleverly unexpected," or "interesting" as in "oh my God what were you thinking"?  :-)
<jtv> Thanks btw.
<wgrant> A combination.
<wgrant> In the Ubuntu case I'd prefer a crash over a deletion.
<jtv> Crashing shouldn't be hard.
<jtv> Anyway, that's for later: I left Ubuntu domination in a state where you don't get deletions.
<wgrant> I'd prefer it was for now. Domination has always been very fragile -- I've spent today diagnosing issues surrounding it, with a production incident caused in part by a bad refactoring of the dominator that I made and fixed last year.
<wgrant> With a big change like this, I'd really prefer you stick an assertion in there so breakage doesn't go unnoticed.
<jtv> What would I assert though?  distro_name != 'ubuntu' or pubs[0].sourcepackagerelease.version == pubs[0].sourcepackagerelease.version?
<wgrant> I don't know. But gina and archivepublisher currently use dominator to do different things, so they should probably run it in different modes.
<wgrant> The dominator should never delete anything in a Soyuz-native archive.
<wgrant> Only for gina.
<wgrant> It will cease to be Ubuntu-specific this year.
<jtv> Wait, now I'm completely confused.  What are you saying is currently Ubuntu-specific?
<jtv> Not gina, obviously,
<jtv> and not the dominator, obviously,
<jtv> so what will cease to be Ubuntu-specific?
<wgrant> When I said "In the Ubuntu case", I really meant "In the Soyuz-native case"
<wgrant> Currently Soyuz-native archives are Ubuntu-specific, but they will cease to be soon.
<jtv> Okay, to prevent further confusion, could you re-state with that mistake fixed?
<jtv> Gina will cease to be specific to Soyuz-native archives this year?
<wgrant> gina is by definition not used for Soyuz-native archives.
<jtv> So what will cease to be specific to Soyuz-native archives this year?
<wgrant> Nothing. Soyuz-native archives will cease to be specific *to Ubuntu*.
<wgrant> I said before that stuff was specific to Ubuntu.
<wgrant> But I really meant specific to Soyuz-native archives.
<wgrant> In particular, the dominator should never call requestDeletion on a publication in a Soyuz-native archive.
<wgrant> It should instead crash, as that case doesn't make sense.
<jtv> Well the algorithm I've got here (the part that gina bypasses, so that's the "different mode" you referred to above) is such that the deletion case just can't happen.  It's the what-do-we-do-now answerâthe one you wanted, I should addâfor the newly possible case in gina only.
<jtv> How we may extend it later to make that case possible for the publisher's domination is a later concern.
<wgrant> Sure, it shouldn't be able to happen.
<wgrant> But I want the code to ensure that it can't.
<wgrant> Because nature finds a way.
<wgrant> We've been bitten before by things like supersede() working on a deleted publication. Other code breaks, calls methods in bad ways, and they do things that nobody ever expected.
<jtv> Okay, how would the assertion check for a Soyuz-native archive?
<wgrant> dominatePackage will need to know in what mode it is being called.
<wgrant> By _dominatePublications or that new thing.
<wgrant> Because it should never call requestDeletion when it's called by _dominatePublications.
<wgrant> No, I don't know what to call the argument :(
<jtv> I don't think it makes any sense to pass a mode argument just to enable an assertion that, essentially, the caller also passed the expected value for another argument.
<wgrant> I guess.
<wgrant> anyway. dominate_imported_source_packages looks like it's only going to dominate sources that are still named in the source archive's indices.
<wgrant> So it won't actually catch deletions, will it?
<wgrant> Because they'll no longer be in the indices at all.
<jtv> That's for my follow-up branch.
<wgrant> Ah.
<jtv> I've already got most of the code here; it works out to be quite elegant.
<wgrant> Indeed, this has turned out pretty well, I think.
<jtv> Especially in the way it lets us handle migration.
<wgrant> Yes. So I guess the first run will sort out everything that's not deleted... I think that looks OK.
<jtv> The new code iterates over all package names that are published in the series/archive/pocket, and then get()s the list from gina's src_map.
<wgrant> Ah. I was thinking you'd have to union, but of course this is done after everything is imported.
<jtv> *In the transitional code*, if it's not in there, it finds the latest version (reusing otherwise useful bits of the dominator) and passes that as live_versions.
<wgrant> So the DB set is fine, indeed.
<wgrant> Yup.
<jtv> *Once transition is complete*, I'll remove that code and then the get() will have [] as a default value.
<jtv> The really great thing about it is that it requires no code of any real complexity or intimacy, except stuff from the dominator that we can reuse.  Some of it is private, but who cares for explicitly temporary code?
<wgrant> Yep.
<StevenK> Oh, RARGH
<StevenK> nigelb: Still here?
<wgrant> I would appreciate a comment in dominatePackage stating that deletion should never happen in native domination, as the live publication is always first. But you have me convinced that the assertion is not required.
<StevenK> Thank you for forgetting my session totally, Firefox
<jtv> wgrant: I agree with the idea of a comment, but this policy choice falls above dominatePackage's pay grade.  It should be in _dominatePublications, where the choice is made that leads to this.
<wgrant> I know it's not really appropriate for dominatePackage, but with a conventional understanding of domination the code makes no sense.
<jtv> wgrant: I'm in the process of writing docstrings, with a comment in _dominatePublications pointing out how the mechanism enforces the policy.
<wgrant> jtv: Thanks.
<wgrant> Approved.
<jtv> \o/
<jtv> Thanks
<jtv> Now if only ec2 would work for me.  Perhaps I should give it another try.
<wgrant> Worked fine for me this morning.
<StevenK> Still confused :-/
<StevenK> But Eventbug helped a little
<nigelb> StevenK: yeah
<StevenK> nigelb: Compounded by testfix!
<StevenK> But your branch has landed
<nigelb> haha
<nigelb> I have great luck!
<nigelb> \o/
 * StevenK ponders asking for buildbot to be set on fire.
<nigelb> StevenK: need help burning it? :D
<StevenK> Haha
<StevenK> I need help with JS, I'm so confused
<wallyworld_> StevenK: i need to go to soccer soon, but can i help with js?
<StevenK> wallyworld_: I'm trying to figure out what JS is fired when I hit a link
<wallyworld_> StevenK: normally you would register an onclick handler
<StevenK> <a class="sprite modify remove js-action" href="#" style="">unsubscribe from this bug</a>
<StevenK> No onclick
<wallyworld_> StevenK: there is one set up - but it's done in js whe nthe page loads
<wallyworld_> the href="#" means that an onclick wil be used
<nigelb> But doesn't that mean there's no fallback in case of disabled JS?
<wallyworld_> probably, but we require js for lp now
<StevenK> cewrapper.fire(Event.getEvent(e, el, compat || false === facade));
 * StevenK blinks
<wallyworld_> StevenK: try looking in bugtask_index.js or bug_subscription_portlet.js
<wallyworld_> i think one or both of those have methods which run in the onload callback to set up the links
<StevenK> lib/lp/bugs/javascript/subscription.js looks very related
<StevenK> But I have no idea how to find the onclick handler function for the link so I can trace it
<wallyworld_> StevenK: it's bug_subscription_portlet.js. see it's make_link() method to set the attributes on the link as you posted above
<wallyworld_> nigelb: the link does have a fallback - the js sets the href="#"
<wallyworld_> StevenK: let me have a quick look
<nigelb> wallyworld_: Ah! :)
<wallyworld_> StevenK: search for link.on('click'
<wallyworld_> there's a few of those for the mute link, the subscribe link etc
<StevenK> wallyworld_: The page I'm on is https://bugs.launchpad.dev/product-name-989062/+bug/16/+subscriptions
<_mup_> Bug #16: "Swedish" and "Swedish (Sweden)" should be the same language <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/16 >
 * wallyworld_ looks
<StevenK> I know that doesn't help you much, but it isn't the bug page
<wgrant> Oh, that's not the subscriptions portlet.
<StevenK> Exactly
<StevenK> And Firebug's script tab isn't being very handy about pointing out where the JS is hiding
<wallyworld_> StevenK: i don't think you would normally get to that page except by url hacking
<wallyworld_> but it is the subscriptions portlet i think, put not rendered in a portlet
<StevenK> Huh? I followed a link to get to that page!
<StevenK> The "Edit bug mail" link
<wallyworld_> StevenK: oh ok. anyway see the bug-subsctipion-list.pt
<wallyworld_> it loads the bug_subscription_list.js
<wallyworld_> which is the same js as used for the subscription portlet
<wallyworld_> StevenK: i have to run to take the kid to soccer but will check back in 30 minutes or so
<StevenK> Oh god, all of the JS is in launchpad.js, isn't it
<StevenK> All 100,000 lines of it
<lifeless> isn't that a build product?
<nigelb> StevenK: yup
<StevenK> Yes, but it's the JS the browser sees
<StevenK> Whereas the JS we edit is split out
<nigelb> Its fun using firebug.
<nigelb> StevenK: found it yet? :)
<nigelb> wgrant / StevenK: Do you both have HE tunnels?
<wgrant> We use AARNet's tunnel broker.
<wgrant> It's a little closer :)
<nigelb> heh
<StevenK> I'm quite happy with it
 * ajmitch wonders when lp & *.ubuntu.com can be accessible on v6
<wallyworld_> StevenK: how's the js stuff?
<StevenK> wallyworld_: Crappy
<StevenK> Giving up for today
<wallyworld_> need any more help?
<wallyworld_> ok
<StevenK> wallyworld_: I might grab you inappropiately after the stand-up
<wallyworld_> oooh. promises, promises
<nigelb> BWAHAHA
<nigelb> Should I or should I not screenshot.
<jtv> nigelb: the way I see it, this could be your retirement money.  And then again, it could be your reason for not reaching retirement age.
<nigelb> jtv: I would much rather prefer using it in a presentation. So, I'm leaning to the second option.
<jtv> âNot screenshotâ?
<nigelb> Screenshot and use in a presentation
<nigelb> wgrant: do you have a minute to clear up a confusion in the ipv6 wiki page?
<nigelb> It says "gksudo gedit /etc/networking/interfaces", in lucid its /etc/network/interfaces. On oneiric do you have networking instead?
<jtv> IIRC it's always been /etc/networking/interfaces
<jtv> (and by "always" I mean "going back to Debian a decade ago"âI don't know much more than that really)
<wgrant> It's always been /etc/network/interfaces
<wgrant> As long as I've been using Debian, which is like 2002.
<jtv> Ah.  Tab completion has spoiled me.
<jtv> And somehow admins who don't seem to mind this keep refusing to set up tab password completion.  It's not fair.
<wgrant> Heh
<nigelb> ok, so the wiki is lying.
<jtv> BTW my having been using Debian for longer than wgrant comes as a bit of a shock.
<jtv> My using Debian for longer than wgrant can walk, that I would have believed.  But thisâ¦
<nigelb> well, he should be old enough to read before he uses a computer...
<jtv> Good point.
<wgrant> I sadly used Windows until 2001 :(
<nigelb> No buffer space available?
<nigelb> GAH. No ipv6 yet for me.
<nigelb> wgrant: I used Windows until 2007. Windows Vista for a year too (yeah, *stab* *stab* *stab*)
<nigelb> Lunch time! Laters folks.
<wgrant> Hmm, no stub yet :(
<wgrant> Morning stub.
<stub> Morning
<wgrant> stub: We snuck a librarian deployment through without downtime earlier, so everywhere is 13891 except germanium's poppy and cocoplum.
<wgrant> Hopefully that is all the fixes we need.
<stub> It is all the fixes we know we need in any way :-)
<wgrant> Right.
<stub> So if mthaddon has time when he gets here or spm is free, we can kick the tires.
<wgrant> stub: My zopeless destruction branches get rejected by the devel database/schema check :(
<stub> With luck you can land it on devel later today. Tests pass?
<wgrant> Yup, all three went through ec2 overnight.
<wgrant> Are we going to lift the restriction?
<wgrant> Since otherwise we're going to be doing a lot of manual merging.
<jtv> wgrant: I just realized we never did make Gina create Published publication records.  Still all Pending.  So need to change the code for that as well, _and_ update the database.
<stub> I like having it as a safety net, but I don't see the point of having be so vigorously enforced.
<wgrant> jtv: We do, but it's not critical.
<wgrant> jtv: Since your new dominator method finds everything that's active.
<wgrant> Not just published.
<jtv> wgrant: errrr
<jtv> I stuck to the logic that was already there, which looked for published.
<wgrant> + self._composeActiveSourcePubsCondition(distroseries, pocket))
<wgrant> Is that active, or published?
<jtv> It's published.
<wgrant> Heh.
<jtv> The dominator tends to say "active" when it means "published."  :(
<jtv> But let me just double-check, because we had the confusion in gina before.
<wgrant> stub: So, I guess we should actually try it tonight...
<jtv> Yup, published.  Not active.
<stub> wgrant: now if I can sucker hloeung into doing it
 * hloeung logs off... right.... now.... 
<rvba> StevenK: Hi, I'd like to ask you a question about the creator field of spph.
<rvba> It was introduced by a branch you created to fix bug #684101.
<_mup_> Bug #684101: Add an ancestor column to SPPH <derivation> <lp-soyuz> <qa-untestable> <Launchpad itself:Fix Released by stevenk> < https://launchpad.net/bugs/684101 >
<wgrant> rvba: It and ancestor were never used.
<rvba> Indeed, Julian an I took a look at the db and it's None. (besides, copyTo sets it to None)
<rvba> wgrant: StevenK I'd like to use it for real to track copies of spph. Any objection?
<wgrant> That was the intention of creator, so do it!
<wgrant> It was added alongside ancestor mostly because it was convenient.
<wgrant> And ancestor then became redundant; we examine changelogs instead.
<rvba> Okay great, this will save me a db patch!
<rvba> wgrant: Thanks.
<wgrant> rvba: Not that that is a problem.
<lifeless> stub: wgrant: landing schema changes on devel? I'd suggest needing a [schema] tag in the commit, but perhaps that wouldn't actually help.
<wgrant> lifeless: Possibly, yes.
<wgrant> lifeless: If you're not watching #-ops, we're about to do the first test.
<lifeless> I'm not here :)
<lifeless> safety net off, free falling :)
<lifeless> wgrant: someone should tweet :)
<wgrant> Once we're confirmed that it's going ahead, yep.
<stub> lifeless: something like that, yer. Although I hate overloading the commit message with metadata.
<lifeless> will need a pqm patch if we want it, but htats not as scary as some folk think :)
<adeuring> good morning
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #850: FIXED in 4 hr 36 min: https://lpci.wedontsleep.org/job/db-devel/850/
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (jtv) | Critical bugs: 253 - 0:[########]:256
<rvba> First day as OCR, allenap is off today so jtv has kindly accepted to be my mentor today.
<jtv> âSoft target.  Fire away!â
<rvba> :)
<wgrant> jtv: That is a very long XXX comment!
<wgrant> jtv: Are you going to land this before we do the code+data fixes for Pending->Published?
<jtv> wgrant: probably, unless it turns out to be inconvenient w.r.t. how we update the legacy data.
<jtv> As long as Gina doesn't get to observe the PendingâPublished change in an unnatural order, it shouldn't make much difference AFAICS.
<wgrant> Indeed.
<jtv> About the worst I can imagine for that scenario is intermediate, but more-or-less-accurate, supersededby links.
<wgrant> jtv: Why are you using _sortPackages instead of sorted(..., cmp=generalization.compare), as you did in dominatePackage?
<wgrant> You should only have SPPHs for a single name, so _sortPackages isn't needed.
<jtv> Just because it gets a little less intimate with the data.
<jtv> Matter of taste, really, no more.
<wgrant> k
<wgrant> Approvalated.
<jtv> Great, thanks.  We're really going through them at a clip now.
<jtv> Which is good, because I once again gave Julian a "done by the end of this week" estimate.
<jtv> Hmm I just realized though: with these changes, the new source file I added to gina ought to be called dominate.py, not retire.py
<wgrant> Probably.
<jtv> And done.
 * mpt awards nigelb his "fixed an MPT bug" badge: (â¯Â°^Â°)â¯ï¸µ{FAMPTB}
<mrevell> heh
<nigelb> haha
<nigelb> I should get that on a t-shirt.
<nigelb> Next UDS I attend, I'll do that :D
<G> rvba: welcome to reviewing :)
<rvba> G: ;)
<G> here have a look at this patch that removes all the code.... ;)
<nigelb> G: That sounds like StevenK's branches :P
<nigelb> stub: Hi
<G> nigelb: haha sounds about right :)
<nigelb> stub: I was asked to run bug 88545 by you. Specifically the bit that requires a db patch
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/88545 >
<stub> nigelb: I can't look right now - deployment stuff underway. Should be able to check it out within the hour though.
<nigelb> stub: sounds good
<cjwatson> [6~[6~[6~[6~[5~/wg 62
<cjwatson> GAH
<nigelb> heh
<nigelb> GAH. Bad thing about working from home - telemarketeers knocking on the door.
<wgrant> Telemarketers... on the door?
<nigelb> Well, wrong word.
<G> wgrant: I was just going to make that point
<nigelb> The door-to-door salesmen
<G> but it was pretty obvious that we all knew what the other Nigel meant :)
<ajmitch> parasites?
<nigelb> I guess you could call them that.
<G> one advantage of living up a 100m driveway and all that, about the only door-to-door thing we get is various churches/religous ones, and the occasional one that does things like door-to-door sales of aerial photography etc
<bigjools> Same here!
<bigjools> last time I had some religious dudes show up, I squirted them with "holy water" and brandished garlic
<lifeless> ttp://hawkersnotice.com/
<nigelb> bigjools: heh
<nigelb> lifeless: Its not linked! thedoghousediaries.com/?p=3003
<nigelb> Oooh,that is good stuff.
<nigelb> Amazon has "11.6 seconds mean time between deployments"... what in the world.
<lifeless> lots of focused services
<nigelb> Is that where the service oriented architecture is leading Launchpad to?
<lifeless> yes
<nigelb> excellent!
<lifeless> The goal is that all commits to trunks are deployed on their own, without batching, and that unrelated things are separate services with narrow contracts
<stub> Or it just means serialized deployments to lots and lots of hosts...
<nigelb> unrelated things?
<lifeless> nigelb: librarian vs codehosting vs group management vs build farm vs ...
<nigelb> ahhh, right
<nigelb> How does lp deploy now? I know its batched etc. But is it a click to deploy?
<lifeless> not quite but close
<jtv> Any reviewers in the house?  https://code.launchpad.net/~jtv/launchpad/bug-844577/+merge/74568  â bigjools maybe, since you didn't get around to my bigger branch yesterday?
<nigelb> The reason I ask is, would making each commit deployed on their own cause more work than now?
<bigjools> jtv: sorry, gotta sort out a prod issue
<jtv> Ah of course, sorry,
<jtv> âkept that in a different context
<jtv> Anyone else?  https://code.launchpad.net/~jtv/launchpad/bug-844577/+merge/74568
<lifeless> nigelb: it always will
<lifeless> nigelb: the question is, does the improved accuracy in dealing with fallout and omgs gain more than the cost.
<nigelb> Okay, yeah. That's a good point.
<nigelb> Also, fail whale for Launchpad would be nice ;)
<ajmitch> LP shouldn't need it :)
<lifeless> nigelb: patch it up
<lifeless> nigelb: I'd love a nicer OOPS page
<ajmitch> a rocket exploding?
<lifeless> nigelb: and OOPS for ajax requests
<nigelb> lifeless: Oh, OOPS for javascript?
<nigelb> Have you seen this -> errorception.com
<lifeless> nigelb: ajax get OOPS ids in the header of the reply
<nigelb> http://errorception.com <-- clickable.
<lifeless> nigelb: but the error is shown nastily to the user, not tastefully
<nigelb> That would be nice to improve, yes.
<nigelb> ajmitch: Rocket exploding OOPS page would be nice.
<ajmitch> nigelb: that'd just be an artwork change
<lifeless> nigelb: u1 has a form that can be POSTed to by javascript that fails.
<lifeless> nigelb: we just need the same for LP, rather than a 3rd party service.
<nigelb> lifeless: I'm guessing the code for the u1 page isn't open source?
<lifeless> nigelb: we can't use a 3rd party service without some very careful legal stuff, because we have non-public data in js contexts.
<lifeless> nigelb: no, but the js side of it is visible :) its a django stack not zope
<nigelb> Every page has a form that javascript posts to?
<nigelb> In caes of errors..
<lifeless> no there is one form for all errors
<nigelb> Ah.
<nigelb> hrm, how do I force an error from u1.
<ajmitch> this openid thing is puzzling me now
<lifeless> ask rockstar about it :)
<nigelb> Does u1 web team have a different channel or all in one channel.
<nigelb> I shall poke rockstar at a more favorable time, when he's awake.
<lifeless> well, not on freenode :)
<nigelb> GAH.
<nigelb> oh. I haven't looked at Ubuntu one website in a while.
<nigelb> Its...wow
<nigelb> lifeless: Nope, my firebug skills don't show me anything. Need to ask rockstar.
<bigjools> wgrant: we're getting rabbitmq 2.6.0 in oneiric \o/
<bigjools> it's got HA clustering
<wgrant> bigjools: Oneiric? Nice.
<wgrant> Well, yes.
<wgrant> It has active-active.
<wgrant> It requires some restrictions, though.
<bigjools> I'd like to say it was my doing, but it's a requirement for OpenStack
<wgrant> bigjools: It's not quite a trivial update, as the PID handling has changed. But it's like 5 lines of initscript that need adjusting.
<wgrant> Ahh.
<wgrant> bigjools: I guess I should update the plugins for 2.6.0 at some point, then.
<bigjools> wgrant: yes please ;)
<nigelb> we don't yet use rabbit heavily, do we?
<nigelb> I thought the only place I saw it was the new sync stuff
<wgrant> nigelb: We don't use it at all yet.
<wgrant> The first use is MP diff updating.
<nigelb> OH YES
<nigelb> PLEASE
<wgrant> It is implemented.
<wgrant> Just not deployed.
<nigelb> Ah. That will make a lot of people very happy.
<bigjools> coming soon to a Launchpad near you
<wgrant> "soon"
<bigjools> we start work on it next week
<bigjools> 2 weeks will be enough
<nigelb> Oh, btw. What does the tag bug-columns mean?
<wgrant> nigelb: That's the next feature.
<wgrant> nigelb: Customisable bug listings.
<wgrant> nigelb: So you can eg. show assignee in search results.
<nigelb> \o/
<ajmitch> that sounds useful
<bigjools> once that's done I want to replace crontabs with rabbit queues
<wgrant> bigjools: Yes.
<wgrant> Job runner daemons.
<nigelb> bigjools: YES!
<wgrant> Consuming immediately from rabbit.
<wgrant> It will be glorious.
<wgrant> GLORIOUS
<bigjools> yes and yes
<ajmitch> rabbit allows for scheduling at a specific time, or will you just run tasks as available?
<bigjools> which is something I was doing 18 years ago in a former job, but meh
<stub> chaos! glorious CHAOS!
<nigelb> ajmitch: as available.
<bigjools> it's to reduce latency and load spiking
<ajmitch> so karma updates would happen during the day rather than all at once? :)
<bigjools> we need a way to specify cumulative updates though
<bigjools> ajmitch: we have many, many cron jobs. one at a time :)
<ajmitch> everyone knows that LP is all about the karma :)
<nigelb> kim0 tweeted this the other day "Ask a sysadmin about something and he'll tell you there's a cron for that"
<nigelb> I can totally believe Launchpad having many, many, many crons.
<lifeless> > 200
<nigelb> Achivement Unlocked - I just rocketfuel-branch'd a branch starting with destroy.
<nigelb> ajmitch: Personally, I think we should have badges as well.
<ajmitch> hah
<nigelb> Like what stackoverflow does.
<ajmitch> nigelb: certainly, I've collected a few on askubuntu
<bigjools> I think we need a penis icon that gets longer and longer
<nigelb> heh
<ajmitch> bigjools: where would you fit it in the UI?
<lifeless> ajmitch: tell users they need multiple monitors.
<ajmitch> heh
<bigjools> you don't need to, you just lie about its size
<nigelb> I should suggest the badges idea at some point. But there stuff I'd like to see before that.
<nigelb> Like instant gratification for buying commercial subscriptions
<lifeless> yes
<ajmitch> then you have people doing things for the sake of badges, like commenting on 50 bugs
<lifeless> fwiw I think gamification has to be done ----very----- carefully
<lifeless> pathological incentives are terrible.
<lifeless> we already have badges for team memberships after all
<lifeless> and have seen folk making farms of teams with badges
<nigelb> ajmitch: Right. It needs to be done carefully. There already are people joining teams for bade collection..
<nigelb> err, what lifeless said.
<nigelb> There are also people creating blueprints and answering questionst for karma :/
 * ajmitch has got no further with being able to log in to launchpad.dev, sadly :)
<bigjools> get any reviews yet rvba?
<rvba> bigjools: nothing yet.
<nigelb> Hrm, is there a reason why on my ~ page only bugs assigned to me *and* In Progress are shows?
<nigelb> *shown
<nigelb> Ideally, I'd like to see everything that's valid and not closed.
<wgrant> It's the "Working on" section.
<wgrant> /~/+assignedbugs shows everything assigned.
<nigelb> aha
<bigjools> wgrant: ok finished reading that page
<bigjools> finally
<bigjools> wgrant: interesting analysis, although I am not sure why you marked two of the rows as "bad" in the last query output.
<nigelb> bigjools: You'll love this - "BCCI refuse sponsorship offer from Branson-We can't have VIRGIN written on our shirts, when we're getting screwed in every match in England"
<bigjools> :D
<wgrant> bigjools: Those are the two that expired the 53 binaries.
<bigjools> how do you know?
<wgrant> Just above that I have a list of the LFAs with their expiry dates.
<wgrant> Let me open that page.
<bigjools> ok
<bigjools> there are some leaps of logic that are hard to follow
<bigjools> let's go through this
<bigjools> might be easier on a call
<bigjools> s/might/will be/
 * wgrant finds headset.
<bigjools> skype is better, mumble gets on my nerves
<bigjools> or voip for purists
<wgrant> I only have mumble set up... let's see if Skype likes oneiric.
<stub> nigelb: So what did you need to know about https://bugs.launchpad.net/launchpad/+bug/88545 ?
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/88545 >
<stub> nigelb: It needs to be done in two steps. First is purging that column from our code. Once that is landed and deployed everywhere, we then land a db patch to drop the actual column.
<nigelb> stub: StevenK asked me to check with you about the expensiveness of the db patch
<stub> Dropping the column is not expensive.
<nigelb> Excellent, I'll go ahead with a destroy branch
<stub> PG doesn't rewrite the table or anything - the column is just flagged 'removed' and new rows won't store the data but old rows keep the crap until we recompact the table.
<nigelb> that's the bit that'll be expensive?
<stub> its the bit we don't bother with, as it will happen eventually with PG upgrades and similar.
<nigelb> cool, ok :)
<nigelb> Oh, and how does one write a db patch?
<stub> We do it if we really need the space or if things are performing badly because of the waste
<wgrant> bigjools: Skype seems to work.
<wgrant> bigjools: Ready when you are.
<bigjools> ok
<bigjools> opne mo
<bigjools> one
<bigjools> just looking at jtv's patch now!!!
<stub> nigelb: There are guides on the wiki. You branch from lp:launchpad/db-devel and add a .sql script to database/schema similar to the others in there. The file name requires a number which someone will need to allocate for you.
<nigelb> stub: okay, I'll worry about it when I land the code changes :)
<stub> nigelb: Cool. I don't think we need to worry about waiting for deployment between landing the code changes on launchpad/devel and the db changes on launchpad/db-devel. We do need to wait for those code changes to have been merged into db-devel though.
<nigelb> ok!
<StevenK> nigelb: IE, it will hit db-devel pretty much after it hits stable
<nigelb> StevenK: yeah, I understand now :)
 * nigelb is waiting for EOD to start working on this.
<rvba> stub: Hi, when you get a chance, could you please have a look at this mp: https://code.launchpad.net/~rvb/launchpad/sync-bug-827608-spph-creator/+merge/74571
<rvba> It's only huge because of the changes in database/sampledata
<rvba> stub: I'm wondering if CONCURRENTLY should be used for the index creation.
<jtv> Still no reviewers in the house?  Need one for https://code.launchpad.net/~jtv/launchpad/bug-844577/+merge/74568
<jtv> rvba: I won't be sticking around much longer though.
<jtv> jcsackett: are you reviewing today?
<bac> jtv: i can look at it in an hour or so if jcsackett isn't around today
<jtv> bac: that'd be great, thanks.  I sort of promised this stuff would finally be done this week, and I'd be ecstatic to be able to land it tomorrow.
<rvba> jtv: I'll let jcsackett handle the reviews when you're gone.
<bac> jtv: well let's go for ecstasy then
<jtv> rvba: OK, better take me off the topic now though.  :)
<jtv> bac: an oft-heard phrase in Amsterdam, actually.
<jtv> Sort of like "let's go for a beer."
<bac> rvba: great that you're reviewing now!
<rvba> bac: Well, I must say this first reviewing session has been quiet ;)
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (?) | Critical bugs: 253 - 0:[########]:256
<bac> rvba: that happens.  much preferred to the days when we always had a huge backlog
<rvba> jtv: I'll find someone to mentor my review if anything comes up before jcsackett takes over.
<jtv> OK
<rvba> jtv: Thanks a lot for your support.
<jtv> rvba: well you haven't needed any!
<rvba> jtv: Indeed but I appreciate the gesture anyway ;)
<jtv> You and your Gallic chivalry.  :)
<rvba> hehe
<bigjools> \o\
<bigjools> we need a smiley for a Gallic shrug
<rvba> Ok, that was a Gallic shug then ;)
<bigjools> not intentionally :)
<jtv> I suppose a Gallic shrug would be something like " M "
<bigjools> that just looks like someone doing air quotes
<jtv> I mean M
<jtv> but I quoted it.
<bigjools> ^O^
<jtv> The shruggie was meant to be the bit between the quotes.
<jtv> The ^O^ does have a certain gastonic quality about it.
<bigjools> jtv: it could also be someone turning into a bat
<jtv> hmm yes, happens all the time so people would get confused
 * bac begins jtv's review
<jtv> thanks bac!
 * bac sees mention of Gina and has instant regret
<nigelb> haha
<rvba> bac: This review was a trap!
<nigelb> Nicely played ;)
<stub> rvba: No CONCURRENTLY in DB patches ('make schema' will fail in any case). We add that in manually if we apply the patch live.
<rvba> stub: Okay.
<jml> I'm starting to work on something that's going to need a db patch.
<jml> Should I still start from db-stable?
<stub> rvba: Its not 2010 either, so the copyright in the patch is incorrect.
<rvba> stub: Arg ... sorry about that.
<rvba> Fixed.
<stub> rvba: Do you want me to allocate a patch number or are you happy doing that yourself?
<rvba> stub: I'm happy to do it ... could you just point me out the right direction?
<stub> lp:~launchpad/+junk/dbpatches
<rvba> Okay.
<wgrant> jml: db-stable is probably. But unless something goes really wrong, we'll be merging db-stable into devel for the first time in 2 months tomorrow.
<wgrant> db-stable is probably best, that is.
<jml> wgrant: thanks. are there any differences to what I should do now to what I would have done two years ago?
<wgrant> jml: DB patches and code must be in separate branches.
<jml> I looked on the wiki, but couldn't find anything.
<bac> jtv: r=bac
<wgrant> jml: We must be able to deploy the DB patch without the code.
<jml> wgrant: and the code without the db patch?
<jtv> Thanks again bac
<wgrant> jml: That's not really defined yet.
<wgrant> jml: But at the moment, no, the code usually depends on the DB patch.
<wgrant> So you land DB patch to db-devel, then code to db-devel separately.
<wgrant> We apply the patches, then merge db-stable to devel, and deploy the code.
<wgrant> jml: It's maintained by buildout, so it should start out empty.
<jml> hmm.
<wgrant> mkdir ~/launchpad/lp-sourcedeps/yui and symlink it into devel, I guess.
<wgrant> I wonder if the upgrade code was dropped at some point.
<jml> OK. I think it's actually created by buildout. So my *real* question is, why does link-external-sourcecode link it
<wgrant> But it's only like 2 months old...
<nigelb> AH, rf-get is what got it for me.
<wgrant> jml: Ah, so it probably only ends up in lp-sourcedeps in new installations.
<wgrant> jml: Same reason we share eggs/: caching for speed.
<wgrant> Although yui isn't too expensive yet.
<jml> Oh, I see. And it can't live in eggs/ because it's not Python.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, rvba* (?) | Critical bugs: 253 - 0:[########]:256
<rvba> jcsackett: Hi, I think I'll leave you the floor since I'm mentor less atm ;)
<jcsackett> rvba: sounds fine. :-)
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 253 - 0:[########]:256
<nigelb> yay, EOD. Now to work on the destroy branch.
<jcsackett> rvba: how did your first day of review go?
<rvba> jcsackett: 0 reviews asked, 0 reviews done ;)
 * jcsackett laughs
<jcsackett> yeah, that happens from time to time.
<rvba> The only branches up for review are mine.
<jcsackett> rvba: soon, you will have a day where 17 reviews hit the queue at the same time. frequently as you're heading to lunch. :-)
<rvba> hehe
<nigelb> Like wating for a bus? ;)
<rvba> jcsackett: one of my branches seems huge but really this is only because of the mecanical modifications to database/sampledata
<jcsackett> rvba: looking at it now. thought it was already reviewed but i see that was just db approval.
<rvba> True.
<jcsackett> the other one looks fine, assuming of course its prereq is alright. :-)
<rvba> The big one is the prereq.
<wgrant> jml: is it behaving now?
<jml> wgrant: yep
<wgrant> Great.
<gary_poster> henninge, none afaik (sorry, was doing car stuff)
<jml> unittest2?
<jml> really?
<wgrant> jml: I think that's for Chameleon.
<wgrant> Chameleon seems to be a bit special in far too many ways.
<jml> OK.
<wgrant> Don't worry, we're not using it :)
<wgrant> it == unittest2
<jml> good good :)
<nigelb> what's Chameleon?
<wgrant> A templating engine.
<wgrant> Implements TAL and some other stuff, I believe.
<wgrant> We use it on a page or two.
<gary_poster> jml, why it would be bad if we were?  It's stdlib 2.7, yeah?  Don't know much beyond that
<jml> gary_poster: prejudice.
<gary_poster> jml :-P
<nigelb> wgrant: A page or two? Isn't tal everywhere?
<nigelb> Or was that irony I failed to catch :P
<gary_poster> nigelb, Chameleon is an optimized engine
<gary_poster> we use a legacy engine everywhere
<wgrant> We use another TAL interpreter for the rest of the site.
<wgrant> Right.
<gary_poster> there's a LEP to switch the rest
<nigelb> AHHHH.
<gary_poster> It is supposed to save 10-20% processing across the board, IIRC.  It's been awhile since I did the research and tests
<bigjools> what we lose in render time, we gain in cronspam!
<nigelb> I was just about to ask why we didn't do that.
<gary_poster> :-P
<gary_poster> bigjools, there's an RT for that.
<bigjools> We're just like an alternative Apple app store
<gary_poster> :-)
<jcsackett> rvba: code in the big branch looks fine. one curiousity though. why remove the comments on columns like potemplate? i don't see anything wrong with doing so, just curious if that was cleanup or somehow within the scope of your changes.
<gary_poster> nigelb, because it is non-trivial work.  See Gotchas and schedule at bottom of https://dev.launchpad.net/LEP/Chameleon That analysis itself has been overtaken by events since it was written.
<gary_poster> For instance, on the bright side, maybe some of the Gotchas are no longer a problem because the newer version of Chameleon has greater compatibility.  On the not-so-bright side, we would need to rewrite the memcache tal plugin again, or decide conclusively to rip it out as failed; and do a reanalysis of What Breaks Now.
<nigelb> fun.
<gary_poster> yeah
<henninge> gary_poster: ok, I'll mark the bug as qa-ok then. (Sorry, was otp)
<rvba> jcsackett: no, that's not within the scope of my branch, let me have a look at that.
<henninge> or rather qa-untestable?
<jcsackett> rvba: like i said, it's not a big deal. the comments all basically are "this is the $var" where $var is the name of the column *anyway*.
<gary_poster> henninge, np, I am about to go on phone, but I was going to try and put a branch in junk for the heck of it as a smoketest.  If you can do that it would be great; if not I'll do it after I get off call
<henninge> gary_poster: yeah, I'll do that.
<jcsackett> so removing them has not materially changed the documentation. :-P
<gary_poster> thank you very mucj henninge
<henninge> np ;)
<jml> Can someone please give me a patch number?
<jml> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Patch%20ids says I need one before continuing.
<rvba> jml: I'm just modifing the patch number file.  If you give me a comment I'll do it for you.
<jml> rvba: what sort of comment?
<rvba> jml: like what the patch does.
<jml> rvba: "Add tables for a package symbol database"
<rvba> jml: Okay sir, here is your patch number: 2208-83-1 ;)
<jml> rvba: thank you.
<rvba> np
<rvba> jcsackett: I've fixed the comment removal. That was due to an unfortunate manipulation.
<jcsackett> rvba: ah, cool stuff.
<rvba> jcsackett: BTW I also fixed the patch number, it was 2208-82-0 and I changed it to 2208-82-1
<jcsackett> rvba: ok.
<jcsackett> rvba: looks good, branch is r=me.
<rvba> (the reason I did that is explained in the last paragraph of http://bazaar.launchpad.net/~launchpad/+junk/dbpatches/view/head:/allocated.txt)
<rvba> jcsackett: cool, thanks.
<abentley> gary_poster: Do you know what buildout does with binaries?  I've added Celery to buildout, but I can't find celeryd anywhere.
<nigelb> To delete a db field, I need to remove all instances that the field is being used, right?
<gary_poster> abentley, on call, but look in egg in EGG-INFO.
<gary_poster> abentley, you can also put a binary in bin.  There are a couple of ways, but most standard is to use entry_points in setup.py
<gary_poster> you can see examples there now
<abentley> gary_poster: It seems to have entry points already: http://pastebin.ubuntu.com/685317/
<henninge> Should code hosting on qastaging be expected to behave differently from production?
<gary_poster> abentley, yes those are if you install it in a Python.  Use what I told you for those entrypoints
<gary_poster> on call again
<abentley> henninge: yes.
<abentley> henninge: codehosting on qastaging has only a small subset of branches physically present.
<henninge> abentley: i saw this when pushing http://paste.ubuntu.com/685307/
<henninge> ah, I see
<henninge> abentley: but what about new branches
<henninge> I had to push twice.
<henninge> also, the branchscanner is either very slow or not running.
<abentley> henninge: I believe it's very slow.
<abentley> henninge: I wouldn't expect the first push to error like that.
<wgrant> It runs on staging, but I don't believe it runs regularly on qastaging.
<wgrant> That first error was because it's trying to stack on a branch that isn't on qastaging.
<wgrant> The subsequent push will see that the bzrdir exists, so not try to stack again.
<henninge> that seems to make sens
<henninge> e
<henninge> Pushing to +junk went fine btw.
<abentley> henninge: I can't reproduce your issue: http://paste.ubuntu.com/685321/
<wgrant> +junk doesn't stack.
<wgrant> So that's unsurprising.
<abentley> henninge: was there a previous push attempt?
<abentley> wgrant, henninge: true.  It would be complaining about the missing development focus branch.
<wgrant> Exactly.
<henninge> abentley: pushing to +junk worked fine for me, too.
<henninge> abentley: my bzr complains about lp://staging/ urls. What might be missing/old?
<abentley> henninge: bzrtools development focus is always present on (qa)staging.  Can you push as a branch of that instead?
<abentley> henninge: I don't know.  What's the complaint?
<henninge> "not a valid url"
<henninge> I will try bzrtools
<abentley> henninge: And it's complaining about qastaging, not staging?
<abentley> henninge: I mean, I can understand if it doesn't grok lp://qastaging, because that's newish.
<abentley> henninge: But lp://staging has been supported for years.
<henninge> yes, that works.
<henninge> abentley: and bzrtools works flawlessly, too. ;)
<henninge> thanks abentley, wgrant
<abentley> henninge: qastaging support was introduced to trunk in January, and should be in bzr 2.4.
<henninge> abentley: ah, I am running 2.3.4
<henninge> that's probably stock natty.
<henninge> abentley: there are two ppas in my config (deactivated): /bzr/ubuntu and /bzr/ppa/ubuntu
<henninge> abentley: which one should I use?
<henninge> or both
<henninge> ?
<abentley> henninge: bzr/ppa
<henninge> abentley: thanks, now I am on 2.4, too
<abentley> henninge: np
<henninge> Hi bac! How far along are you with QA for bug 838957? Need any help?
<_mup_> Bug #838957: AssertionError: Search by component requires a context with a distribution or distroseries. <oops> <qa-needstesting> <Launchpad itself:Fix Committed by bac> < https://launchpad.net/bugs/838957 >
<bac> henninge: i'm doing it now, thanks
<henninge> bac: awesome, thank you ;)
<bac> should just take a sec
<bac> henninge: it's gary you need to harass!
<bac> :)
<henninge> bac: I am done with that ;)
 * bac hurries...
<henninge> "Don't rush the miracle man or you'll get the wrong miracle."
<bac> henninge: done!
<henninge> wgrant: is the buildd-manager in nodowntime?
<henninge> bac: great, thanks
<henninge> ;-)
<flacoste> mrevell: we should announce that we are doing fastdowntime to our users
<flacoste> since they might encounter a 5 minutes outage weekdays at around 8:30 UTC
<bac> n
<mrevell> flacoste, Cheers, I'll get a blog post etc together.
<mrevell> bac, Hey
<mrevell> bac, got a question for you
<bac> hi mrevell
<bigjools> cjwatson: how would you feel about automatically cleaning out the archive and librarian of all binaries as soon as a series is marked obsolete?
<flacoste> mrevell: thanks! can you take care of sending the news to the stakeholders list also?
<mrevell> flacoste, Sure!
<flacoste> thanks
<stub> flacoste, mrevell: I put an announcement via launchpadstatus already
<stub> oh... just for tomorrow.
<mrevell> Thanks stub.
<flacoste> mrevell: it's probably important to point out that until we fix bug 844631, this will manifest itself through a bunch of OOPSes during the outage
<_mup_> Bug #844631: Launchpad should return 503 error pages when database is unavailable <fastdowntime-later> <Launchpad itself:Triaged> < https://launchpad.net/bugs/844631 >
<mrevell> Good point, thanks.
<cjwatson> bigjools: it needs a grace period so that we can make sure that old-releases has something current
<bigjools> ok
<cjwatson> (IMO)
<bigjools> cjwatson: is that not something you'd check before obsoleting it?
<cjwatson> it doesn't get moved to old-releases until it's obsolete
<bigjools> ah, faire enough.
<cjwatson> AFAIK anyway
<bigjools> I'm just trying to work out how to phrase a bug which captures the requirements
<bigjools> I want to automate as much as possible - sounds like we need an extra Big Red Button
<bigjools> cjwatson: are old sources captured on old-releases? And do you still need all the old versions?
<cjwatson> (a) yes (b) which old versions?
<bigjools> the superseded ones
<bigjools> ie can we make this like binaries
<stub> bigjools: I sometimes wonder if the Librarian should have an 'external' field which we can use to point to stuff no longer stored in the Librarian.
<bigjools> stub: yes, we desperately need offline storage.  It's a bit nuts keeping *everything* in there.
<elmo> blink
<elmo> how is that not just "make it someone else's problem"?
<elmo> if you want it online but still available, someone has to store it and make it available
<bigjools> If you want to keep adding disk space in the librarian then I have no problem filling it up
<stub> It very much is making it someone elses problem. I'm just wondering if it is appropriate sometimes (eg. of we are going to store archives for ever anyway on disk in an accessible format, no real point duplicating that storage in the Libraraian).
<gary_poster> sinzui, in a fresh oneiric server, when I try to import html5browser I get a "RuntimeError: Gtk couldn't be initialized" error in gi/overrides/Gtk.py.  I can provide a traceback if that helps.  I verified that various seemingly obvious gtk packages were installed, and they were.  Is there something else relatively obvious I can try?
<sinzui> gary_poster, thank you for reminding me. wgrant reported this as well. I think there is an undocumented dep
 * sinzui investigates
<gary_poster> sinzui, oh ok.  thank you
<sinzui> gary_poster, can you paste the traceback?
<gary_poster> sinzui http://pastebin.ubuntu.com/685391/
<sinzui> gary_poster, fab, we did the same thing and it works for me. Maybe I can publish the correct libs in a few minutes
<gary_poster> ok thanks sinzui
<sinzui> gary_poster, I think the dep that marries python to gir is missing. Do you have python-gobject 2.90.3 installed
 * gary_poster looks
<gary_poster> sinzui, -gobject-2 @  2.28.6-6svn1
<gary_poster> sinzui, oh and -gobject 2.90.3-1svn1
<gary_poster> both installed
<sinzui> gary_poster, that would be the problem  -gobject-2 is for obsolete libs
<sinzui> of dear. -gobject 2.90.3-1svn1 is the one I thought was missing
<gary_poster> sinzui, so they would conflict?  Should I try to install -gobject-2?
<gary_poster> I mean uninstall
<gary_poster> sorry
<sinzui> gary_poster, they do not, and I happen to have it installed. give it a try
<gary_poster> sinzui, "python-gobject depends on python-gobject-2 "
<gary_poster> (and I aborted the attempt to uninstall)
<sinzui> ah. python-webkit is a gtk2 app so it does want that lib
<sinzui> gary_poster, can you try both of these command, are they the same TB: http://pastebin.ubuntu.com/685415/
 * gary_poster tries
<gary_poster> sinzui, yes, same TB
<gary_poster> (for both)
<sinzui> thank you that helps. This is a python-gi-gtk lib issue. Mine works by accident because I hack with that combination for other projects
<gary_poster> ah ok
<mtaylor> morning sinzui ... I got a weird one for you
<mtaylor> one of our devs (~annegentle) has launchpad openid returning a different localid than annegentle. apparently this happened with adamg a little while ago and the working hypothesis is that it is related to having had an account get renamed
<mtaylor> sinzui: any idea who the right person to ping about this would be?
<sinzui> mtaylor, This is often caused by a disconnect between the person's login.ubuntu.com emails and launchpad.net emails
<mtaylor> sinzui: is that something that can be sorted out by the person themselves?
<sinzui> This was caused several times in the past by profile merges, but I think that is fixed given how rare it happens. I think this commonly happens when a user add/removed emails from wither site without knowing they are working with two sites
<sinzui> mtaylor, I honestly do not know. I think the person should testing logging in to login.ubuntu.com to verify the registered emails, then do that with Lp and add the missing addresses to Lp
<sinzui> mtaylor, gary_poster had some experience with this in the past. I believe the a DBA was employed on several occasions to fix the ids in either of the sites
<mtaylor> sinzui: gotcha
<gary_poster> hello
<mtaylor> hey gary_poster !
<gary_poster> :-)
<jeblair> i've asked annegentle to verify the email addrs are in sync
<gary_poster> So, lemme see if I can find the bug.  If it's the same problem, the proper fix is for ISD to make some changes that have been planned for a long time, but there's a recipe for fixing it on the admin side.
<jeblair> if it helps:
<gary_poster> So first let's verify that we're talking about the same thing.  1 sec, looking
<jeblair> https://launchpad.net/~annegentle says the local_id is https://login.launchpad.net/+id/6sDWhGL
<jeblair> but when she logs in, we get passed https://login.launchpad.net/+id/xQHRTxt
<jeblair> and she did say that she has had some kind of account rename (i don't know if it was executed as a merge)
<gary_poster> OK, this is the root bug... https://bugs.launchpad.net/launchpad/+bug/644824
<_mup_> Bug #644824: User created a second Launchpad account; now gets mixed success/denied to various services <canonical-losa-lp> <defect> <lp-foundations> <Canonical SSO provider:Confirmed> <Launchpad itself:Fix Released by stub> < https://launchpad.net/bugs/644824 >
 * gary_poster is scanning to try and find remediation...
<gary_poster> mtaylor, comment #31 says that ISD can delete the bad openid account.  contacting them might be your best bet.  https://answers.launchpad.net/canonical-identity-provider/+question/150135
<gary_poster> Alternatively we can ask losas to make a deletion on the LP side, per comment 12 of that bug.
<mtaylor> gary_poster: so is the best way to do that to file a question against canonical-identity-provider?
<gary_poster> mtaylor, that would be the better long term soluton, assuming anne does not want to maintain two Canonical openids.  I will also ask the losas to delete the tokens on the LP side, which should alleviate the problem for now (she will have to log in again, using the openid she actually wants to maintain)
<gary_poster> ah, right, there's a process for this now...
<sinzui> gary_poster, do your have libgirepository-1.0-1? I wonder if the -dev version also need to be installed, if so python-gobject-2-dev may also need installation
<gary_poster> sinzui, I have it installed, but not the -dev
<gary_poster> should I try it?
<sinzui> gary_poster, I think I could be smarter about this investigation. paste or email the results of this so that I know exactly what is different about our packages:
<sinzui> dpkg --get-selections > installed-software
<bac> mrevell: still around?
<mtaylor> gary_poster: ok. I'm confused now ... which thing should I be doing now. filing the question?
<gary_poster> mtaylor, I'll do it for you, and ping you with it.
<mtaylor> gary_poster: awesome. you are amazing
<gary_poster> sinzui, sorry it took a sec, paste isn't set up yet and had some kind of problem
<gary_poster> but sent
<nigelb> wgrant: you around?
<nigelb> wgrant: Nevermind :)
<sinzui> gary_poster, can you install gir1.2-gtk-2.0
<sinzui> oh, maybe I am the one without it
<sinzui> okay I have it and I am sure webkit is a gtk2 app
<gary_poster> sinzui, ok trying
<gary_poster> sinzui "Setting up gir1.2-gtk-2.0 (2.24.5-0ubuntu7)" but same traceback when I try to import html5browser
<sinzui> I will continue looking a the diff
<sinzui> gary_poster, gir1.2-webkit-1.0
<gary_poster> k
 * sinzui doubts that is correct, but if it is, he will be enlighten
<gary_poster> sinzui, same problem (and that had three dependencies)
<sinzui> okay :( I have python-gtk2-dev  python-gobject-dev python-gobject-2-dev too
<sinzui> They have binaries in them to
<gary_poster> sinzui, k trying
<gary_poster> whoo!  That'll bring in the neighbors...
<sinzui> yuck
<gary_poster> 59 dependencies
<sinzui> I see only two other cases like in on the net and they were solved by installing pygobject, which is python-gobject :(
<gary_poster> mtaylor, losas have sql request on LP side, and I filed https://answers.launchpad.net/canonical-identity-provider/+question/170575 for you on CIP side.
<gary_poster> I will ping when sql request has been run
<mtaylor> gary_poster: thanks man!
<gary_poster> sinzui, !! still same error.  you want me to be paranoid and do something crazy like restart or something?  That seems crazy, but...
<gary_poster> welcome mtaylor
<sinzui> gary_poster, no restart. removed those dev packages.
<gary_poster> ok
<sinzui> I have started a upgrade to oneiric on another machine and I will look at the diff some more
<gary_poster> sinzui, should I also remove gir1.2-gtk-2.0 and gir1.2-webkit-1.0
<gary_poster> or leave 'em around till later?
<gary_poster> sinzui, /me going to take lunch in a sec
<gary_poster> oh...
 * gary_poster has call soon :-P
<sinzui> gary_poster, take your time. I will continue to work on this
<gary_poster> ok thanks sinzui
<sinzui> gary_poster, Gtk is a GUI lib, as webkit is a gui app. you do not have an x server up
<bac> sinzui: is it possible to get the openid identifier for a user now?  it used to be in the +rdf but isn't now
<gary_poster> sinzui, duh!  sorry, I should have thought of that.  xvfb it is
<sinzui> bac it is not, but it is a trivial bug fix to remove the constraint in the template
<sinzui> the bug is already reported
<bac> sinzui: ok.  i needs it now, though.  :(
<sinzui> gary_poster, Sorry for being so slow. I always test html5browser with
<sinzui> Xvfb :23 -ac -screen 0 1024x768x24 &
<sinzui> env -i DISPLAY=:23  ./bin/py
<sinzui> I should have read my own notes before asking you to tell me what is installed
<gary_poster> :-) thanks again
<nigelb> rockstar: Hi, around?
<rockstar> nigelb, I am.
<nigelb> rockstar: lifeless suggested I talk to you about how U1 web deals with javascript oops for potentially duplicating it in LP :)
<rockstar> nigelb, short answer: we don't.
<rockstar> :)
<nigelb> Oh.
<rockstar> We've disabled jsoops for the time being, because it got in the way more than it helped.
<rockstar> nigelb, however, we're going to refactor it as soon as I'm done with my current project, which should release soon.
<nigelb> Ouch, okay
<nigelb> rockstar: Any pointers on what's the "right way" to do it?
<rockstar> nigelb, so, the very basics of it are this:
<rockstar> window.onerrer = function() { new Image().src = '/jsoops/endpoint' };
<gary_poster> mtaylor, please ask anne to try logging in again
<rockstar> nigelb, of course, the onerror function has message, url, and line arguments.
<nigelb> okay
<nigelb> But Image?
<mtaylor> gary_poster: will do
<nigelb> rockstar: I'm curious, why the Image()? Does the jsoops load and image to show the user?
<rockstar> nigelb, it's just a neater way to make a GET request than XMLHttpRequest
<rockstar> You never add the image to the DOM.
<rockstar> Also, you don't have to worry about the domain there either.
<nigelb> Oooh. Interesting!
<rockstar> nigelb, so, the endpoint itself will has ?message=...&url=...&line=... so you just make sure the endpoint knows what to do.
<nigelb> Oh, so these gets logged by the endpoint for future usage?
<nigelb> usage/triager/debugging/fixing
<rockstar> nigelb, Be advised, however, that YUI will step all over that once it gets loaded, so you'll have to do something a bit more YUI-ish after that.
<rockstar> nigelb, exactly.
<rockstar> nigelb, so we do Y.one(Y.config.win).on('error', function(e) { new Image()... }); once YUI gets loaded.
 * rockstar feels another blogpost coming on.
<nigelb> rockstar: Nice. But we'd be doing a GET to write data in this case right?
<nigelb> rockstar: WRITE IT! :)
<rockstar> nigelb, not necessarily.  You could parse logs after the fact, since you usually don't want to write on GET (and Launchpad specifically forbids it).
<rockstar> You don't even have to have an actual endpoint.  You could just parse Apache's logs.
<nigelb> Oh, right.
<nigelb> Yeah.
<rockstar> nigelb, the Ubuntu One web team is a bit more fast and lose when it comes to errors though.  It's often enough to see a graph of errors, rather than the actual errors.
<rockstar> (at least in the client)
<nigelb> rockstar: Oh, then you debug based on their numbers? (rising, decreasing)
<rockstar> nigelb, yeah, and usually it's pretty simple to find, especially if you get the url right.
<rockstar> Most of the time it's an unsupported browser, and the error was thrown right before the user got the "non-javascript" experience.
<nigelb> rockstar: ooh, right. But this at least seems fairly trivial to do.
<nigelb> may be its my ignorance though :)
<rockstar> nigelb, yeah, it's not difficult, no.
<rockstar> nigelb, but you get lots of edge cases (also referred to as "what users do") that make things less trivial.
<nigelb> I'll run it by lifeless or stub and ask for more comments before I start writing anything :)
<rockstar> For instance, you don't want the error to break the rest of the page.
<bac> sinzui: is this the bug you referenced? https://bugs.launchpad.net/launchpad/+bug/501731
<_mup_> Bug #501731: XRDS for launchpad users missing local id <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/501731 >
<rockstar> nigelb, also, remember that in a few weeks, U1 is re-addressing this, so we might want to knowledge share a bit.
<lifeless> nigelb: we'd generate a custom oops on GET, its not modifying the resource in the usual rfc2616 sense, so would be OK.
<nigelb> lifeless: So, what rockstar proposed sounds good?
<lifeless> yes, thats why I pointed you at him :)
<sinzui> bac no. I misunderstood you question. I thought you wanted the openid url which is shown only to yourself on your profile page
 * rockstar writes down in his diary "Dear diary, today lifeless approved of something I was doing. It made me feel all warm inside!"
<sinzui> bac: there is no bug/request to leak the id
<nigelb> rockstar: haha
<nigelb> lifeless: so, having a stub URL with nothing, on javascript error, do a GET request to that endpoint, look at logs in apache?
<lifeless> nigelb: s/look at logs in apache/create a custom oops/
<nigelb> lifeless: hrm, so the stub URL should grab it and push it somewhere?
<nigelb> custom logs?
<nigelb> like what wsgi-oops does?
<lifeless> it will translate between whatever format the javascript is producing and an oops report
<lifeless> probably wanting a custom oops.Config() without the in-appserver-hooks installed, because we're not reporting a problem *in this appserver*.
<lifeless> but don't take this bit literally: thats detail to figure out when doing it
<nigelb> lifeless: I think I need to look at how LP does oops. I have not much clue :)
<lifeless> lib/canonical/launchpad/webapp/errorlog.py - allow a couple of hours to read it and the supporting libraries and generally play around getting a feel for whats under the hood
<nigelb> okay, doing that over the weekend
<nigelb> rockstar: when you modify the jsoops for u1, I'd be intersted in details :)
<lifeless> nigelb: Oh, one further tweak: I suggest a dedicated plain-ol-simplewsgi daemon for doing js oops collection
<lifeless> nigelb: it has no reason to be in either the LP or u1 services; we can map the submission url to the daemon via apache
<rockstar> lifeless, I would most certainly agree.
<nigelb> lifeless: ah, this is part of the service oriented architecture bit?
<lifeless> nigelb: no more than anything else
<lifeless> nigelb: also no less so than anything else
 * nigelb couldn't parse that :P
<lifeless> nigelb: everything we do gets an soa assessment now
<nigelb> AH!
<nigelb> lifeless: so I can just write something in paste? ;)
<lifeless> nigelb: yes, or even simpler :)
<nigelb> Or just a wsgi app, right.
<lifeless> nigelb: lp:~lifeless/+junk/gpgverify might have some ideas; I *think* I pushed my simpleserver version
<nigelb> I'm going through error.py
<lifeless> not that
<nigelb> I'll add it to my reading list after that :)
<lifeless> errorlog.py
<nigelb> gah
<lifeless> ah, I'm still using web.py there
<lifeless> ah! its the gpgfixtures test keyserver I used simpleserver
<lifeless> http://bazaar.launchpad.net/~canonical-launchpad-branches/python-gpgfixtures/trunk/view/head:/gpgfixtures/keyserver.py
<nigelb> *click*
<lifeless> thats a microservice I was about ready to get the LP test suite to use when I went on leave ;)
<lifeless> wgrant was mumbling about finishing the migration off
<lifeless> okies, I have to run; ciao!
<nigelb> Laters!
<nigelb> Thanks :)
<jeblair> gary_poster, mtaylor: annegentle is still showing up with the old id, though her launchpad page now lists _no_ openid info
<gary_poster> jeblair, she needs to log out and log back in.
<jeblair> that was after a browser restart...
<gary_poster> jeblair, IIRC, and I'm pretty sure I do, the session cookie will be retained after a browser restart.  If she didn't log out and in again, there will be no change.
<jeblair> gary_poster: ok. i'll let her know.
<gary_poster> jeblair, I've also subscribed her to the question I filed for her against canonical identity provider (https://answers.launchpad.net/canonical-identity-provider/+question/170575) so if there is any reply there, she should know.  You or others can subscribe to replies as well, if desired.
<jeblair> gary_poster: thanks!
<gary_poster> welcome
<benji> jcsackett: do you have a minute to review lp:~benji/lazr.restful/error-on-multiple-ws.op-fields ?
<jcsackett> benji: sure thing.
<jcsackett> benji: i don't actually see an MP for that branch.
<sinzui> jcsackett, I am reviewing your branch now. I am going to to run it to see it
<benji> jcsackett: heh, you are correct; one second
<jcsackett> sinzui: ok. sorry i forgot to post screenshots. :-)
<salgado> can somebody with the needed rights see the contents of a private branch on bazaar.launchpad.net/...?
<benji> jcsackett: here you go: https://code.launchpad.net/~benji/lazr.restful/error-on-multiple-ws.op-fields/+merge/74682
<jcsackett> benji: i'm not getting a diff generated, any chance you can throw the diff to me as a paste?
<benji> jcsackett: http://paste.ubuntu.com/685561/
<jcsackett> thanks.
<jcsackett> benji: looks good to me. r=me.
<benji> thanks!
<salgado> does launchpad the web app have access to the contents of bzr branches?
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[########]:256
<nigelb> Anyone awake? :)
<nigelb> I'm removing a field completely.
<nigelb> Is it okay to remove statusexplanation = StringCol(dbName='statusexplanation', default=None)
<nigelb> from the model?
<mwhudson> if nothing references it, sure
<nigelb> I'm removing everything else that references it as well.
<reed> hello folks
<reed> is there a way to download the mailing list archives from launchpad? (like pipermail does)
<nigelb> grar, postgres error. /me postpones for tomorrow.
<reed> argh https://bugs.edge.launchpad.net/launchpad/+bug/588411
<_mup_> Bug #588411: downloading of mailing list archives <feature> <lp-registry> <ml-archive-sucks> <Launchpad itself:Triaged> < https://launchpad.net/bugs/588411 >
<sinzui> reed, there is not
<reed> sinzui, very unfortunate
<reed> I'll update the bug: that's a very important feature to collect metrics on the mailing lists
<sinzui> reed,  you can ask a question at answers.launchpad.net/launchpad. An admin can get you a copy of the archive
<reed> I need them on an ongoing basis, like every week or month
<sinzui> reed, mailing lists are not a priority now. I can update the bug with detail about the scope of the effort for community members implement
<reed> I can add a comment on the bug too if you want
<reed> I understand it's not a priority for you at the moment though... I hope you'll reconsider :)
<reed> sinzui, if there is a way to get the past archives as a one off, I may setup a workaround to gather new messages locally and process the stats from the local archive
<sinzui> reed, I believe other people have done that. As I said above, and admin can get you a copy of the archive
<reed> thanks
<nigelb> Achivement unlocked.
<nigelb> Removed a field successfully. Or so I think.
<poolie> hi all
<poolie> way to go nigelb :)
<nigelb> Morning poolie!
<wgrant> sinzui: Sorry, can't be at the standup this morning. I can probably talk in an hour or so if you want.
<sinzui> okay. I will check back later
<nigelb> wgrant: Is psycopg2.OperationalError: fe_sendauth: no password supplied related tosomething you and jtv were talking on the list?
<nigelb> (for tests)
<StevenK> sinzui: 35. http://paste.ubuntu.com/674281/
<wgrant> nigelb: Yes, that same thing.
<wgrant> nigelb: Wait, no, not jtv.
<wgrant> nigelb: My other email to the list.
<wgrant> Not the reply to jtv.
<nigelb> wgrant: Okay :) Updating my setup
<nigelb> wgrant: Wait, but I'm not on ipv6 :/
<nigelb> StevenK: I join your ranks with a destroy- branch ;)
<nigelb> +8/-58  not much though
<wgrant> nigelb: You have loopback IPv6.
<wgrant> What does your /etc/hosts look like?
<nigelb> Ah
<nigelb> ::1     localhost ip6-localhost ip6-loopback
<wgrant> That is the problematic bit.
<wgrant> Apply the fix, and all should be good.
 * nigelb runs l-d-s again
<wgrant> Anyway, I need to wander off properly for a while.
<nigelb> :)
<nigelb> thanks!
<nigelb> (I should sleep)
<wgrant> wgrant@carob:~/logs$ find */2011-09-07 -type f | wc -l
<wgrant> 5226
<wgrant> wgrant@carob:~/logs$ find */2011-09-08 -type f | wc -l
<wgrant> 72430
<wgrant> fastdowntime appears to have worked...
<lifeless> thats more than expected ;)
<lifeless> 2minutes at 900rpm => 1800 failures
<wgrant> They were mostly appserver OOPSes.
<wgrant> I calculated there were about 60000 extra appserver OOPSes, from the OOPS IDs at the time.
<wgrant> And yes, it is slightly high.
<wgrant> Er.
<wgrant> 900rpm?
<wgrant> Considering the appservers serve up to 14M requests a day (normally around 9-10M), 900 per minute is a little low :)
<lifeless> EONLEAVE.
<lifeless> we may need to revisit capacity again shortly.
<lifeless> anyhow, 3600 <<< 60000
<wgrant> >>> (10000000 / 86400) * 132
<wgrant> 15180
<wgrant> And given that the responses were immediate, API scripts would have made lots of requests.
<lifeless> 132?
<lifeless> seconds down ?
<wgrant> Maybe it was 127, but yes.
<wgrant> Roughly.
<wgrant> 132
<lifeless> 1736 rpm per appserver machine :)
<lifeless> but yes, our rate is getting pretty high
<wgrant> We had very odd spikes on the 18th and 20th
<lifeless> are we starting to queue in haproxy again ?
<wgrant> But given that they apply to both web and API traffic, I'm inclined to think it was something else.
<wgrant> We've had 5 queue depth incidents in the last week.
<wgrant> So yes.
<lifeless> meep.
<wgrant> They all cleared quickly, and only once did it exceed 20.
<wgrant> But it is happening increasingly often.
<wgrant> So yeah, we may want to expand a bit soon.
<lifeless> we need to check the PPR and see if there has been a systematic increase (5% would be significant) in page render time
<wgrant> Let's see how appserver load is...
<lifeless> if not, cross check on capacity
<lifeless> and then look at adding a core + memory to one of the appservers.
<wgrant> Let's see..
<lifeless> we *should* be running at N+1 at all times: hitting a queue warning -ever- is a Big Deal.
<StevenK> lifeless: I was wondering how you were thinking of scaling -- via a 5th machine or making the 4 beefier
<wgrant> We can scale these machines up further.
<lifeless> StevenK: elmo wants to consolidate in the DC
<wgrant> They're not full AFAIK.
<lifeless> StevenK: we have 4 because 2*dc + redundancy within the DC
<lifeless> StevenK: but thats the old (february) strategy; now we're moving DC's we're looking at something different e.g. full N+1 deploys in both DC's and one DC running hot-idle
<lifeless> its been 2 weeks since I caught up with elmo on this, I don't know the current status.
<wgrant> lifeless: We've had two days in the last month where average render time is 30% higher than the norm.
<lifeless> wgrant: was the sql time likewise higher ?
<wgrant> But apart from that it's only marginally higher.
<lifeless> or load spikes on the appservers correlatimg with that ?
<wgrant> The graphs don't show SQL time.
<lifeless> wgrant: check the daily PPR report to see that.
<lifeless> (for the day in question)
<wgrant> Yeah.
<wgrant> The first spike is mainly API with a bit of the rest, the second translations with a bit of the rest.
<lifeless> bbiab
<wgrant> The SQL increase was only 10% of the total increase.
<wgrant> The first load spike was, interestingly, the day in between the two >13M request spikes.
#launchpad-dev 2011-09-09
<wgrant> Hm, we can toss another 10 appservers on chae without much trouble.
<lifeless> ???
<lifeless> I wouldn't expect that, unless our SQL/total ratio has changed dramatically
<wgrant> https://lpstats.canonical.com/graphs/ChaenomelesCPU/
<wgrant> It hit 50% once in the last week.
<lifeless> if its seeing less load, thats possible an haproxy config bug
<jelmer> it's surprising how much people try to "fork" a branch on launchpad by registering a new code import...
<wgrant> Yeah :/
<wgrant> People don't understand that in a DVCS you can just branch.
<wgrant> Because on GitHub you can't :/
<jelmer> wgrant: though we do have a button to allow people to create an *empty* branch, which seems just as pointless
<wgrant> Rather.
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 253 - 0:[########]:256
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 253 - 0:[#######=]:256
<StevenK> wallyworld_: I added an alert() to that function and it didn't pop up a message box?
<wallyworld_> StevenK: did you add it just before the named_post?
<StevenK> wallyworld_: Just before the if block
<StevenK> (That is just before the named post)
<wallyworld_> StevenK: did you make jsbuild and refresh the browser?
<StevenK> I just ran make
<StevenK> And the alert appears in launchpad.js
<wallyworld_> make by itself should be ok but make jsbuild is quicker
<wallyworld_> what ui element did you hit? the red sprite or the link?
<StevenK> wallyworld_: The link in +subscriptions
<wallyworld_> StevenK: we may be looking at the wrong file. i know about the subscription portlet.  there's a structural-subscription.js
<wallyworld_> which may be used for the page you are looking at
<wallyworld_> line 1205?
<wallyworld_> StevenK: lp.registry.javascript.structural-subscription.js
<StevenK> Yeah, just found that
<wallyworld_> that's the only other place i know of in js that unsubscribes someone
<StevenK> The do_io does the actual unsub?
<StevenK> What a horrible function name
<wallyworld_> yes, looks like it
<StevenK> No alert either
<wallyworld_> hmmm.
<wallyworld_> StevenK: i'll run up a system and have a look.can you paste the url you are looking at
<StevenK> wallyworld_: It's a local bug
<wallyworld_> StevenK: but i can got to a bug and append +subscriptions i think
<StevenK> wallyworld_: So what I did was run make-lp-user stevenk; then jumped into a harness and getUtility(IPersonSet).getByName() and then factory.makeBug(private=True, owner=user)
<StevenK> If that makes sense
<wallyworld_> ok, so you are editing a private bug. there's one of those in the sample data i can look at
 * wallyworld_ waits for make to finish
<wallyworld_> StevenK: i think it make be lp.bugs.lavascript.subscription.js line 219
<wallyworld_> so there's 3 bits of javascript that are used to unsubscribe
<StevenK> lavascript sounds good
<wallyworld_> hah
<wallyworld_> nope, that's not it either
<wallyworld_> StevenK: it's the make_action_lin method
<wallyworld_> make_action_link
<wallyworld_> line 1012
<wallyworld_> my breakpoint just got hit :-)
<StevenK> And the on click function is internal to make_action_link
<StevenK> Sigh
<wallyworld_> at least you know where to start hacking now
<StevenK> How do I get at the bug in JS?
<wallyworld_> StevenK: what do you mean? the link to the bug?
<wallyworld_> there's a json cache entry LP.cache.bug
<wallyworld_> from there you have LP.cache.bug.self_link or LP.cache.bug.web_link etc
<StevenK> Ah.
<StevenK> LP.cache.bug is undefined
<wgrant> 30 seconds...
<StevenK> wgrant: ?
<wgrant> Until the OOPS report of doom.
<wgrant> Although it seems to be late, unsurprisingly.
<StevenK> It's probably choking on 70,000 OOPS
<wallyworld_> StevenK: there's LP.cache.context and LP.cache.context.bug_link. you can view page source and scroll to the end to see what's in the cache
<wgrant> 40814 DisconnectionError: could not connect to server: Connection refused Is the server running on host "lp-prod-pgbouncer.canonical.com" and accepting TCP/IP connections on port 5433?
<wgrant> Yay
<wgrant> 29665 AssertionError: Bug #504291: Store left in a disconnected state.
<wgrant> Yay
<_mup_> Bug #504291: DisconnectionErrors (already disconnected) happening again <lp-foundations> <oops> <Launchpad itself:Fix Released by stub> <Storm:Invalid> < https://launchpad.net/bugs/504291 >
<wgrant>  270 DisconnectionError: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.
<wgrant> Yay
<wgrant>   83 Fault: <Fault -1: 'Unexpected Zope exception: DisconnectionError: could not connect to server: Connection refused\\n\\tIs the server running on host "lp-prod-pgbouncer.canonical.com" and accepting\\n\\tTCP/IP connections on port 5433?\\n'>
<wgrant> Yay
<wgrant> And that's all of them.
<StevenK> So we get to look forward to 60,000 OOPSes every time we do fastdowntime?
<lifeless> we need to teach oops-tools to know there was a deploy
<lifeless> and hide them
<wgrant> Those assertionerrors shouldn't happen, though.
<lifeless> indeed
<wgrant> There is a bug somewhere.
<wgrant> Everything else looks pretty sensible.
<lifeless> indeed
<wgrant> So we are good to go for tonight.
<StevenK> Tonight? Seriosuly?
<StevenK> *Seriously
<wgrant> 1830: upgrade germanium and cocoplum
<wgrant> 1831: fastdowntime all the patches
<StevenK> Because what could possibly go wrong?
<StevenK> wallyworld_: LP.cache.context doesn't have what I want, how is that populated?
<benji> StevenK: IJSONRequestCache(view.request).objects['my_thing'] = my_thing
<wallyworld_> StevenK: there's 2 ways - automatically with the attrs from the page context, plus one can also instantiate an IJSONRequestCache object
<StevenK> So I suspect the bug is LP.cache.context, but I can't see private on there
<wallyworld_> StevenK: see expose_user_subscriptions_to_js in lp.bugs.browser.structuralsubscription.py
<wallyworld_> StevenK: if context were bug, then all the attrs would be there
<wallyworld_> so it's probably something else
<StevenK> Lots of context in those views :-/
<wallyworld_> yep
<StevenK> Looks like ... part of a bug
<StevenK> It's a task
<StevenK> So LP.cache.context.bug_link is the link to the bug
 * wgrant stabs Melbourne.
<wgrant> It was nice and cloudless and not freezing yesterday, today it is hailing.
<wgrant> It was almost getting springish, now back to mid-winter for a while :(
<StevenK> It's raining here
<StevenK> JS sucks
<jtv> wgrant: it's turning out to be another day of long unexpected phone calls, but here's what we discussed earlier.  Wanna review?  https://code.launchpad.net/~jtv/launchpad/transitional-published/+merge/74720
<wgrant> jtv: There you go again, making perfectly good classic Soyuz tests *readable*.
<jtv> Sorry.
<jtv> I could add some dire but incomprehensible XXX comments in the names of people who have long left the company, if you like.
<wgrant> Stuff like kiko's "this is very funny" gina XXX is good, too.
<jtv> "XXX kibonzi 2004-03-31: ON NO ACCOUNT should this plearfy the.  If get this wrong, ENTIRE ARXHIVE become problem.  DO NOT CHANGE without asking permission from Celve Alexelo."
<wgrant> jtv: For dominate.py, do you know of the store.find(SourcePackagePublishingHistory, archive=series.main_archive, distroseries=series, [...]) Storm shorthand?
<jtv> Yes, just don't like it much.
<wgrant> Indeed, it makes extension of queries difficult.
<jtv> I like the wider spacing and avoidance of ambiguity in the "==" notation.
<wgrant> I am normally tempted to use the kwarg notation for stuff like SPPH, but you have taken the probably better alternative of aliasing it.
<jtv> Also, I like to think of our database as a relational one.  Which involves so much more than just equality comparisons!
<wgrant> Otherwise you end up with 5 line conditions.
<jtv> Quite.
<jtv> I don't like doing the aliasing very much, but I still prefer it over the kwargs shorthand somehow.
<wgrant> Indeed.
<jtv> Also, this is transitional code.  It will disappear soon, so I'm not too bothered about it anyway.
<wgrant> Yup.
<jtv> wgrant: extra points if you can spot everything that's wrong with that XXX I gave earlier.
<wgrant> I could be cruel and make you fix gina.txt's lint, but perhaps not.
<jtv> Grrrrr
 * jtv snaps at wgrant
<jtv> I know what island you live on.
<wgrant> Better still, I'll make you fix it, and then I will petition to have the standard changed to something else, requiring every line to be reindented again.
<jtv> You're getting disturbingly good at this.  I may have to call your mommy.
<wgrant> Hey, I'm not the one making scarily plausible XXXs.
<jtv> Plausible?  2004-03-31?  Nah, we have code reviews to catch mistakes like that.
<wgrant> lol
<jtv> (Damn, should have written 2004-04-31 obviously)
<wgrant> Heh
<jtv> *That* is what we need: an automated checker for XXX dates in PQM, which will fail our branches because PQM thinks the date is in the future.
 * wgrant returns to necromancy.
<wgrant> Sadly, many of these binaries are being rescued from the pool only so they can be immediately reaped :(
<jtv> wgrant: I take it this is not about my branch?
<wgrant> No.
<jtv> Phew.
<wgrant> It's recovery from some different mass publication update queries from years ago :)
<jtv> Ah.  Andâ¦ they're binaries that ought not be reaped?
<jtv> rope?
<wgrant> They ought to be reaped.
<jtv> Is reaped really a regular verb?
<jtv> reapt?
<wgrant> But they cannot be, because we don't have them in the librarian any more.
<wgrant> reapt is too close to crept!
<jtv> Ahhh that one.
<wgrant> So I must rescue them from the pool back into the librarian, so they can be reaped from the pool, and then the librarian.
<wgrant> Sort of.
<jtv> Getting the patients all healthy and rosy-cheeked for the executioner, eh?
<wgrant> Exactly!
<jtv> Thanks for the review BTW.
<wgrant> Ah, yes, sorry, got distracted by your XXX abomination.
<jtv> Abomination even?  Glad you liked it.
<jtv> wgrant: about the multiple-publications-of-same-version issueâ¦  domination orders by (SPR.version, SPPH.datecreated).  I wonder if it shouldn't be something like (SPR.version, SPR.id, SPPH.datecreated).  The issue affects conventional and new domination, really.
<jtv> Can there be 2 published SPPHs for the same SPR for the same (Archive, DistroSeries, Pocket, SourcePackageName)?
<jtv> (And similar for BPPHs etc.)
<jtv> Sorry, for two _different_ SPRs for the same (Archive etc.)
<jtv> So two SPRs for the same version of the same package, and each with their own Published SPPH in the same (archive, distroseries, pocket).
<jtv> Is that at all possible?  If so, what if the SPRs are ordered differently in time than the SPPHs are?
<wgrant> It's not legal to have two SPRs in one archive with the same name and version, because they don't fit on disk.
<wgrant> However.
<wgrant> That doesn't mean it hasn't happened.
<wgrant> It has.
<wgrant> But they should never both be published, because they can't both be on disk at once.
<jtv> So domination doesn't need to go out of its way to accommodate this situation.
<wgrant> Probably not. The publisher will crash before it gets into that situation.
<jtv> Why are so many words ending on -ation nowadays?  Used to be different when I was your age, I can tell you.
<wgrant> Because it will see the publication marked as Pending, try to write it to disk, then blow up when it sees a different file already there.
<wgrant> So the conflict never gets marked as Published.
<wgrant> It just OOPSes forever until we SQL it out.
<jtv> Good enough for me.
<wgrant> Ahem.
<jtv> So then, would it be fair to say "if a single live version of a package has multiple Published pub records (within the same archive yadda yadda) then the most recent of those pub records dominates the rest of them, and they should become superseded"?
<wgrant> Yup.
<jtv> And it specializes neatly to the classic domination algo, so that's all dandy.
<wgrant> So... I guess you just sort by SPPH.datecreated, and drop a version from the live set once you see it?
<wgrant> Mm, but that's moving the logic into dominatePackage, which is slightly upsetting.
<jtv> The details get a bit intimate with the implementation, so step back:
<wgrant> But maybe.
<wgrant> Yeah.
<jtv> dominatePackage goes through a list of SPPHs sorted by descending (SPR.version, SPPH.datecreated).
<jtv> It remembers the last liveâ¦ SPR IIRC that it saw.
<jtv> If it encounters a non-live version: supersede by the last-live whatever-it-remembers.
<jtv> If it encounters a live version: leave it as it is, but record it as the last live whatever-it-is.
<jtv> What I need to add then, I think, is "is this SPPH the same as for the last live version?  If so, supersede it as well."
<jtv> (Alternatively I could compare to the last-seen version; same results, assuming the choice of dominant doesn't change, but probably slightly more code)
<jtv> Anyway, tests first, then look at the implementation details.
<nigelb> wgrant: Is the font fix okay?
<wgrant> nigelb: It looks good, but it's not on production yet.
<nigelb> wgrant: cool, okay :)
<wgrant> Thanks for sorting that out.
<nigelb> When I look at https://bugs.launchpad.dev/debian/+source/mozilla-firefox/+bug/3/+activity
<_mup_> Bug #3: Custom information for each translation team <feature> <lp-translations> <Launchpad itself:Fix Released> <Ubuntu:Invalid> <mono (Ubuntu):Invalid> < https://launchpad.net/bugs/3 >
<nigelb> oh wait, .dev
<nigelb> https://bugs.launchpad.net/launchpad/+bug/88545/+activity
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/88545 >
<nigelb> I see statusexplanation
<nigelb> Even though I removed all references to that field in this branch
<nigelb> Is that normal?
<wgrant> Sounds like you didn't remove all of them.
<wgrant> nigelb: Oh.
<rvba> Morning.
<wgrant> nigelb: Nevermind, that's fine.
<wgrant> nigelb: I was misunderstanding.
<wgrant> nigelb: If you look at the content of the bugactivity table, you'll see that there are changes with whatchanged == 'statusexplanation'.
<wgrant> nigelb: We could delete them too, but there's no need and little benefit.
<wgrant> Erasing history is bad :)
<nigelb> Excellent
<nigelb> wgrant: ah, that's bugactivity table
<nigelb> I tried to find the table
<nigelb> my psql skills are still not 1337 enough :P
<wgrant> \d bug<TAB><TAB>
<wgrant> Or not.
<wgrant> launchpad_dev=# \d bug
<wgrant> Display all 228 possibilities? (y or n)
<wgrant> :(
<nigelb> I did a \dd
<nigelb> and looked in there
<nigelb> was that wrong?
<wgrant> \dd is more than just relations, so it's far bigger than what you want.
<wgrant> \d shows just tables/views/sequences, so is more interesting.
<nigelb> ah
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[#######=]:256
<nigelb> oh, just missed wallyworld_ :(
<nigelb> wgrant: Out of panic, I did the ++profile++show page and looked at the sql queries ;)
<wgrant> Heh.
<nigelb> hrm, abel isn't in yet.
<nigelb> I'll have to ask later for my destroy branch
<henninge> wgrant: Hi! ;)
<wgrant> This can't be good!
<wgrant> Hi.
<henninge> wgrant: The process that logs to buildd-manager.log is the buildmaster, i.w. running on build-master host?
<wgrant> henninge: The process is buildd-manager, running on cesium, also known as builddmaster or occasionally buildmaster.
<henninge> wgrant: and it gets reststarted during a nodowntime rollout, right?
<wgrant> Yes.
<henninge> ok, just wanted to be sure.
<henninge> I added extra logging output at the end of a ttbj (elevated it from debug to info) but I don't see anythting.
<wgrant> :(
<henninge> Just the "starting templates build j..."
<henninge> so these builds must die before they reach the WAITING state.
<henninge> I will have to try this out locally or on dogfood.
<henninge> thanks wgrant ;)
<adeuring> good morning
<mrevell> Hello
<bigjools> morgen
<jtv> hey there mrevell, bigjools
<nigelb> Morning mrevell / bigjools :)
<nigelb> adeuring: Hi, are you OCR-ing today?
<adeuring> nigelb: yes. do you need a reviewÃ
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 253 - 0:[#######=]:256
<nigelb> adeuring: Yep - https://code.launchpad.net/~nigelbabu/launchpad/destroy-statusexplanation-88545/+merge/74704
<adeuring> nigelb: ok, I'll look
<nigelb> thanks!
<StevenK> \o/
<StevenK> nigelb: One thing to note is to ask your reviewer to land it with the --incremental flag -- the branch doesn't fix the linked bug entirely.
<nigelb> StevenK: ah, ok
<StevenK> nigelb: The bug is about the existance of the column, you haven't removed it yet. :-)
<nigelb> StevenK: AHH, right. That gets completely removed in the next step :)
<StevenK> nigelb: Right!
<nigelb> StevenK: First time doing such a thing, will remember next time :)
<StevenK> nigelb: Which is why I'm helping you :-)
<nigelb> \o/
<nigelb> StevenK: Do I get a destroyer badge now? :P
<StevenK> The destruction isn't nearly wide-spread enough
<nigelb> heh
<StevenK> steven@liquified:~/launchpad/lp-branches/no-more-staticdiff% bzr di -r 13779..13783 | diffstat -s
<StevenK>  21 files changed, 53 insertions(+), 597 deletions(-)
<StevenK> Now that is wide-spread destruction.
<nigelb> woooooah
 * nigelb isn't that invasive
<nigelb> (yet)
<StevenK> That was a fun branch. Does that look like StaticDiff or something that supports it? Yes, remove it.
<nigelb> heh
<adeuring> nigelb: a very nice  branch. Just one question: Shouldn't lib/lp/bugs/stories/bugs/xx-bug-activity.txt be changed too? I get the error "psycopg2.OperationalError: fe_sendauth: no password supplied" when I try to run the test. An unrelated problem, but I can't answer this question myself...
<rvba> adeuring: see William's recent email to -dev about this.
<nigelb> adeuring: Is that the one with the table?
<rvba> adeuring: and Hi btw ;)
<nigelb> adeuring: No, it doesn't need to be fixed. I asked wgrant earlier about this and he said history is a good thing :)
<adeuring> nigelb, wgrant: ok, I'll look. SImply missed the mail
<nigelb> That's actually in bugactivity and not entirely related
<nigelb> I faced the test trouble too last night :)
<adeuring> nigelb: ah, sure, so, r=me.
<nigelb> adeuring: \o/ Can you land it for me as well. And as StevenK said, with the --incremental flag ;)
<adeuring> nigelb: sure. And your reminder about --incremental may already answer my question: Would you like to work on the remaining usage of statusexplanation too, especially the DB patch?
<nigelb> adeuring: Yes, I would
<nigelb> In fact, I'll probably be starting on it tonight
<adeuring> nigelb: cool, many many thanks1
<nigelb> \o/
<al-maisan> I am getting errors when uploading a source package to a ppa .. is LP down for maintenance..?
<jml> al-maisan: no. Otherwise it would say on identi.ca/launchpadstatus and in the topic here.
<al-maisan> these are the errors: http://pastebin.ubuntu.com/685841/ -- and they are persistent
<al-maisan> hmm ..
<bigjools> wgrant: looks like poppy didn't reconnect?
<cjwatson> jtv: are all new Debian SPPHs permanently PUBLISHED from here on out, or will new ones still be created as PENDING for a while?
<jtv> cjwatson: all Published.
<jtv> Except _maybe_ (I'm not sure about this yet) ones that are copied when creating a new release.
<cjwatson> jtv: OK.  The change broke a swathe of ubuntu-dev-tools API scripts that were assuming PENDING, so I'll have to get that changed quickly
<jtv> cjwatson: ouch!  I'm sorry, I thought this difference had to be inconsequential since PENDING seemed so pointless.
<jtv> In the Launchpad codebase we have a helper variable active_publishing_status that consists of PENDING and PUBLISHED.
<cjwatson> we had various getPublishedSources queries that had to be special-cased for Pending before
<wgrant> jtv: Aren't they still created as PENDING until the next deployment?
<wgrant> cjwatson: Hmm, getPublishedSources should default to PENDING and PUBLISHED.
<wgrant> cjwatson: So it should have DTRT.
<jtv> wgrant: true, but that's not far off.
<cjwatson> wgrant: we were overriding the default :-)
<wgrant> cjwatson: Right, but it shouldn't have been necessary :(
<cjwatson> but hey, that makes it easier, I'll just delete some code.
<jtv> \o/
<cjwatson> that will presumably have the side-effect that now we'll notice Ubuntu uploads before the publisher runs
<wgrant> bigjools: Argh.
<wgrant> bigjools: @read_transaction should have handled that...
<jtv> Meanwhile, Soyuz-capable reviewer needed for the next piece of the Gina-the-Dominatrix puzzle: https://code.launchpad.net/~jtv/launchpad/bug-845326/+merge/74741
<wgrant> bigjools: Can you arrange for it to be bounced?
<wgrant> bigjools: And probably a bug filed for maintenance people to look at.
<cjwatson> wgrant: I don't see the getPublishedSources default you mention - can you point me to it?
<cjwatson> Oh, and could somebody attempt to land https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations for me?  Yay for fastdowntime at last. :-)
<jtv> cjwatson: I'll take it
<cjwatson> Hm, that's marked as for merging into db-devel
<cjwatson> that should be devel, yes?
<stub> yer, superceed the proposal and it lets you retarget it.
<wgrant> cjwatson: Ah, distroseries.getPublishedSources has something like that default, but archive doesn't.
<wgrant> Wow, those are bad APIs.
<cjwatson> wgrant: I guess I can just pass status=('Pending', 'Published') then
<jtv> cjwatson: the multiarch-translations are not attached to a bug?
<wgrant> cjwatson: Are you using the one on DistroSeries or Archive?
<jtv> cjwatson: if you're building this on top of the Launchpad codebase, use active_publishing_status.
<wgrant> cjwatson: If Archive, status=['Pending', 'Published'] should work, yeah.
<wgrant> And if you are using DistroSeries.getPublishedSources, then stop because it shouldn't exist :)
<cjwatson> jtv: bug> no, I never got round to filing one.  Is that a problem?
<cjwatson> jtv: active_publishing_status> this is an API script
<jtv> cjwatson: a bitâour Q/A process revolves around the associated bug.
<cjwatson> I can file one now then, one moment
<jtv> (Oh thank you pidgin for telling me _now_ that Colin asked me a question.  Why didn't I just start using Windows so I could get the beta experience all the time?)
<jtv> Thanks.
<jtv> bigjools: realistically I expect I need either you or william for that review, simple though it beâ¦ https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations
<jtv> Ahem, not that one
<jtv> I mean https://code.launchpad.net/~jtv/launchpad/bug-845326/+merge/74741
<cjwatson> jtv: bug 845475
<_mup_> Bug #845475: Separate translations out of Packages to reduce download requirements for multiarch clients <Launchpad itself:New> < https://launchpad.net/bugs/845475 >
<jtv> thanks cjwatson â I'll attach it if necessary
<cjwatson> wgrant: I can't pass status=['Pending', 'Published'] through the API.  It only wants one.
<rvba> wgrant: as part of fixing bug #827608, I'm currently chaning _latestSeriesQuery to return SPPHs instead of SPRs. I know I'll have to fix quite a bit of code after that change but I really think this is doable ... do you see a reason why I should not do that?
<_mup_> Bug #827608: Sync requester isn't credited with upload <derivation> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/827608 >
<jtv> cjwatson: annoyingâ¦ have you tried both tuple and list?  We have some code that gets finicky about that.
<cjwatson> jtv: yep, tried both
<wgrant> cjwatson: Hah. The Python API accepts both, but the interface for lazr.restful is declared to only accept one.
<cjwatson> wgrant: oh, and this is Archive, yes.
<wgrant> cjwatson: That is unpleasant.
<jtv> cjwatson: did you just resubmit your merge proposal?
<wgrant> al-maisan: The poppy error is just cosmetic, and we're working on upgrading to a fixed version now. Your upload should still have gone through fine.
<jtv> and hi al-maisan :)
<al-maisan> ah, I see, thanks wgrant !
<al-maisan> .. and hello to my favourite fascist reviewer: jtv :)
<al-maisan> how are you my friend?
<jtv> After calling me a fascist, that sounds as if you're rhetorically asking in what way I am your friend.  :)
<jtv> Wonderful here though, thanks.  Hope it's the same for you.
<cjwatson> jtv: yes, it's https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/74743 now
<al-maisan> jtv: I called you a "fascist reviewer" .. please observe the context ;)
<jtv> al-maisan: True.  And I must have upset the wrong people because I've been convicted to heaps of Soyuz work.
<al-maisan> jtv: yup .. doing great, thanks!
<wgrant> al-maisan: (poppy fell victim to our first real test of our new 2-minute-downtime DB patch process -- we're still ironing out the kinks, as you can see)
<jtv> cjwatson: so it's for devel now, instead of db-devel?  It has no dependency whatsoever on the db patch?
<al-maisan> jtv: uh-oh .. but you are enjoying soyuz .. admit it :)
<wgrant> jtv: The DB patch is applied!
<al-maisan> wgrant: change is always difficult :P
<wgrant> But we can apply DB patches daily now, so all is wonderful.
<jtv> al-maisan: http://paste.ubuntu.com/685825/ http://paste.ubuntu.com/685826/
<wgrant> ... apart from poppy.
<cjwatson> jtv: Yep, I held off asking for a landing until the DB patch got applied.
<jtv> wgrant: ah, I wasn't sure about this because it was for db-devel.  Anyway, I'll check the minimum of required boxes & land.
<cjwatson> jtv: The new MP is for devel.
<wgrant> jtv: I think it's all going to be a bit confusing like that for a month or two while people get used to the new hotness :)
<cjwatson> jtv: When I initially prepared this branch, I got caught up in some merge target confusion because the changes to fastdowntime were just starting.
<jtv> Yes, spotted it, thanks.  Unfortunately the resubmit also seems to have re-requested reviews.
<wgrant> And while we sort out all the pending MPs.
<wgrant> cjwatson: I think the process has been redesigned twice since you started the branch.
<cjwatson> jtv: I don't think it should actually need those db reviews now
<wgrant> You did rather well.
<cjwatson> But yes, it would need a normal review
<jtv> It's the same review it went through before, no?
<jtv> I mean, the code changes had already been approved, right?
<jtv> It was an approved MP with pure code changes after all.
<cjwatson> Right.
<cjwatson> The only change since the code review was a merge from db-devel
<jtv> OK.  It's going into EC2.
<cjwatson> Thanks.  Let's see how that goes ...
<jtv> It's been working for me today.
<jtv> Unfortunately the oneiric beta version is causing me some very serious trouble, with minute-long seizures and disappearing emails.  It seems to be related to dbus and X.  The bug for the most immediate symptom seems to be getting attention so let's see what happens.
<cjwatson> Restarting compiz occasionally helps.
<jtv> (The disappearing emails seem to be from TB being too busy to move emails from one folder to another; they probably turn up somewhere in the end)
<cjwatson> (IME)
<jtv> How do I restart compiz?  Actually I'm not even sure I'm running it.
<jtv> I was on unity2d but in oneiric I can't tell the difference any more.  Which is probably a sign that they're being unified, which is good especially given the name.
<wgrant> jtv: When you hover over a launcher icon, is the tooltip black and transparent?
<wgrant> Argh, unity 2D's are now transparent too :(
<wgrant> But they're a much lighter grey.
<wgrant> That's how I distinguish between them.
<wgrant> Black = 3D, grey = 2D
<jtv> It looks dark-gray and dithered; I can't see anything from the background through it.  Pretty much how it used to look in unity 2d.
<cjwatson> pkill -9 compiz (note: will shuffle all your workspaces among windows), or log out and back in.  Although I don't think unity-2d uses compiz; check with 'pgrep compiz'
<al-maisan> jtv: ah, I see, you are having too much fun .. they should reduce your wages ;)
<wgrant> Also, 'unity' now defaults to restarting everything.
<wgrant> And it even works without DISPLAY set, from back in natty when you needed to restart several times a day.
<jtv> So I'm still on 2d.  Strange, because I think the new login screen (wonderful, apart perhaps from the delay before the asterisks for my password appear) was telling me I was logging into "ubuntu" as opposed to "ubuntu 2d"
<wgrant> So you can just switch to a TTY, log in, and 'unity'
<jtv> al-maisan: heck, they even let me train little fascists now!
<al-maisan> oh no!
<al-maisan> :)
<jtv> cjwatson: nope, pgrep compiz returns 1
<jml> jtv: https://bugs.launchpad.net/unity-greeter/+bug/838777
<_mup_> Bug #838777: Really slow shortly after booting <Unity Greeter:New> < https://launchpad.net/bugs/838777 >
<jtv> thanks jml, as always concise and devastatingly accurate!
<jtv> al-maisan: I just graduated one, and when the other day I got to do some temporary mentoring for another, nobody wanted a review all of a sudden.  :)
<wgrant> jtv: Oh, you're not rvba's permanent mentor? :(
<wgrant> I want more clones of you.
<jml> jtv: np. I've been spending a lot of time filing bugs against oneiric recently. I've even managed to get some of them fixed.
<jtv> *Entire channel stares at wgrant*
<wgrant> Heh
<jtv> yay for jml
<al-maisan> jtv: for some inexplicable reason I can relate to that :)
 * jtv laughs
<nigelb> wgrant: More clones of jtv? Are you sure?
<cjwatson> jtv,wgrant: bug 845487, FYI
<_mup_> Bug #845487: Debian source publication checks have broken <ubuntu-dev-tools (Ubuntu):New> < https://launchpad.net/bugs/845487 >
<jtv> thanks cjwatson
<wgrant> cjwatson: Note that publications will shortly in the next few days be marked Superseded/Deleted as appropriate.
<wgrant> s/shortly //
<jtv> nigelb: according to my IRC client it took you 5 minutes and 54 seconds to recover enough from wgrant's statement to formulate that question.
<nigelb> jtv: I was at lunch :)
<rvba> nigelb: quick lunch then ;)
<jtv> cjwatson: to be precise: for packages that gina no longer sees in Debian, the last publication of the highest version will become or remain Published and the rest will become superseded.
<cjwatson> That's fine
<jtv> For packages that it _does_ see in Debian,
<nigelb> jtv: But yeah, took a while to recover :D
<cjwatson> I'm pretty sure ubuntu-dev-tools only cares about the most current one in a series
<jtv> if it only sees a newer version in Debian than in the DB, the publication will become Superseded.
<jtv> If there are multiple publications of the same version in the same series/archive/pocket (e.g. because of a component change), the older publications will be superseded.
<jtv> The newest publications for remaining versions become/remain Published.
<jtv> Any publications of versions that are _newer_ than anything that's currently seen in Debian will become Deleted.
<jtv> That's the âtransitionalâ stage of Gina domination.
<jtv> Once that has run, we'll get more aggressive: if the package is no longer in Debian, any remaining Published publications become Deleted.
<jtv> Soyuz-capable reviewer needed!  https://code.launchpad.net/~jtv/launchpad/bug-845326/+merge/74741
<jtv> Coincidence I'm sure.
<jtv> cjwatson: I could have a go at bug 845486.  My API knowledge's a bit rusty though.
<_mup_> Bug #845486: cannot pass sequence to Archive.getPublishedSources through API <Launchpad itself:New> < https://launchpad.net/bugs/845486 >
<cjwatson> mine is more so :-)
<jtv> Our API infrastructure really has a very 1990s notion of polymorphism, doesn't it?
<jtv> cjwatson: maybe it should accept, as alternatives, a âstatusâ _or_ a sequence of âstatuses.â
<cjwatson> Oh, yes, good point, breaking existing code that passes a single status would be bad!
<jtv> We'll have some callsites in Launchpad that need to be converted, but that's unlikely to pose many problems.
<jtv> It's still ugly, of course, but I'm not sure we can do better without breaking API compatibility.
<cjwatson> jtv: How so?  The existing LP code already accepts either a status or a sequence of statuses; it's only the API declaration that's less flexible
<cjwatson> oh, unless you mean as different parameter names
<jtv> Yes, the only proper thing to do would be to convert status=[â¦] in argument lists to statuses=[â¦]
<jtv> Which is something we've been wanting to do.
<jtv> Even nicer, arguably, would be to convert all callsites to use statuses.  But given API compatibility I don't think we can retire the "status" parameter anyway.
<cjwatson> Indeed
<jml> jtv: I think the status-as-list-or-string pattern is used by searchTasks.
<jtv> adeuring: would you perhaps be available for a pre-implementation discussion as a change from regular OCR duty?
<jtv> jml: how does the API deal with it there?
<jtv> The web-service API I mean?
 * jml looks
<henninge> Do the builders still run on lucid?
<jml> jtv: it's declared as a list, but apparently passing a string goes through ok. There is then a helper that ensures it's returned sanely.
<jml> jtv: it's all a bit convoluted.
<jtv> jml: convoluted sounds bad.
<jml> jtv: but I think the convolution is mostly searchTasks being weird for other reasons.
<jml> jtv: rather than to handle this case
<jtv> henninge: yes, they run on LTS
<henninge> jtv: thanks
<jtv> jml: it'd certainly give us an easier fix if this works.  Still a bit misleading though to have a strict type system and then lie about the type.
<jml> jtv: agreed. although, as an API user, I'd be mightily irritated by having to spell multiple statuses for bugs differently to multiple statuses for source package publishings.
<jtv> jml: what I hear you say is that maybe the strict type system is more to blame here than the lying.  :-)
<jml> jtv: well, it's not a strict type system.
<jml> jtv: it's a lau(d|gh)able attempt to build one in Python.
<jtv> You had me there for a moment.
<jtv> Okay, I'll try to get the List trick to work then.
<jtv> Although in the past I've been bitten by the two ways to test the API: one was fast and followed very dynamic (and for me, convenient) type rules.  The other was slow but more realistic, and applied WADL's type rules.
<jml> IBugTaskSearch.importance (or status) has the 'type declaration'. I think BugTaskSearchParams.fromSearchForm does the string->list stuff, I *think*.
 * jtv looks
<jtv> jml: any idea where importance could be passed in through the web UI?  Beacuse IBugTaskSearch.importance is an attribute, but I'm looking for a parameter.
<jml> jtv: sorry, I don't understand.
<jml> jtv: I mean, I don't understand why you care. If I were to figure out the answer, it'd be by grepping for fromSearchForm, since that sounds an awful lot like something used in the UI.
<jml> jtv: If you want to see it used as an API parameter, then look for searchTasks.
<nigelb> Okay, so I landed a branch that removes the usage of the field that I want to kill.
<nigelb> Now how do I proceed?
<rvba> nigelb: Create a branch off db-devel with a db patch in it: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<nigelb> excellent, thanks!
<rvba> welcome
<nigelb> Wow, this process looks detailed.
<nigelb> I'm going to spend 1 hour trying to read and understand first
<bigjools> haha - I just started documenting Soyuz.
<jtv> He's gone mad.
<jtv> It had to come to this.
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugs: 253 - 0:[#######=]:256
<nigelb> Hrm, I wonder if the DatabaseSchemaChangesProcess wiki page needs update post the new faster db updates
<adeuring> jtv: sure. And sorry for the late answer, was in the other room for lunch
<jtv> adeuring: actually we already had some discussions here, sorry
<adeuring> np
<wgrant> nigelb: I believe it's already up to date.
<wgrant> nigelb: The old process was abolished almost two months ago, because fastdowntime was meant to be done 6 weeks ago :)
<StevenK> wgrant: Can I put together a patch to drop FKs, then? :-)
<wgrant> StevenK: I was going to look at that next week, once we've had a clean run with rvb's patch on Monday night.
<wgrant> Since tonight was not entirely smooth, I don't want to go doing anything quite so big just yet.
<StevenK> wgrant: I was thinking we do it piecemail, TBH
<nigelb> wgrant: Oh. Ok.
<wgrant> StevenK: No, it should be one hit.
<wgrant> Since we only apply one patch a day.
<wgrant> And there's like 32 tables or something.
<StevenK> 35
<StevenK> wgrant: Do you have a patch already, or shall I look at writing it?
<StevenK> I'll make a start on it, anyway
<wgrant> I have one that's not quite fastdowntime capable.
<wgrant> I will fix it weekend or Monday.
<StevenK> It deals with everything, including staticdiff?
<wgrant> No, but it's trivial to extend.
<StevenK> It's too slow for FDT, or what?
<jml> so, did LP actually have a low downtime db patch rollout?
<jml> this means fastdowntime from now on?
<StevenK> jml: We can only do fastdowntime, lifeless removed the monthly process from us.
<jml> StevenK: well yes, but two days ago you couldn't even do fastdowntime.
<bac> mrevell: fwiw, i just bought a batch of vouchers and they showed up promptly
<mrevell> Cheers bac. Do you have any idea what might have gone wrong for Vadim?
<wgrant> jml: Yep, we can do fastdowntime daily now.
<wgrant> jml: Not completely smooth, but we know how to avoid the problems that show up.
<bac> mrevell: nope.  all i can think of is the proxy was stuck.
<bac> or dead
<wgrant> jml: Got dozens of DB patches for us yet?
<mrevell> bac, Cheers, well, I'll keep an eye out for similar things in future ... if we end up with a couple it might be easier to work out what's going on. Or maybe not.
<jml> wgrant: nope :) you put me off, so now I'm doing the data generation stuff locally first and postponing the "Where does the data live?" question.
<wgrant> Sounds like a good plan.
<bac> adeuring: are you currently working on any reviews?
<adeuring> bac: no
<wgrant> jml: We have 270 tables... I would personally like to discourage adding more that aren't really too directly relevant.
<bac> adeuring: cool.  i'll claim one of rvba's
<adeuring> bac: ok
<StevenK> wgrant: So tempted to land populate-bprc
<jml> wgrant: I'm pretty sympathetic to that line of argument.
<wgrant> jml: I can certainly see the convenience.
<wgrant> jml: But even LP is discouraged from putting tables in LP's database now.
<jml> wgrant: anyway, if I do the scraping & data gathering and just dump to a sqlite db locally for now, that'll help me make progress.
<wgrant> Yeah.
<wgrant> Maybe we will not be in limbo when you are done :)
<jml> The last time I did limbo it was on rollerskates.
<wgrant> That sounds inevitably disastrous.
<jml> :)
<rvba> bac actually, I had to recreate branches from code that was already approved yesterday by jcsackett.
<rvba> bac: [...]-spph-creator-code is simply the extraction of the code that was previously in [...]-spph-creator
<bac> rvba: so jcsackett approved a combined branch and you pulled them apart?
<rvba> bac: exactly.
<bac> rvba: good to know.  i'll look at them with one eye tied behind my back
<rvba> bac: I had to refactor that due to the fact that we want to separate code change from db changes completely for FDT.
<bac> rvba: right
<benji> henninge: do you have any answers to my questions on https://bugs.launchpad.net/launchpad/+bug/742662 ?
<_mup_> Bug #742662: Mixed new line markers causing OOPS while importing translations <bad-commit-13889> <oops> <qa-bad> <rosetta-imports> <Launchpad itself:In Progress by benji> < https://launchpad.net/bugs/742662 >
<henninge> Hi benji!
<henninge> benji: There is no oops or similar, we came to that conclusion just by looking at the code.
<henninge> benji: https://code.launchpad.net/~benji/launchpad/bug-742662/+merge/74254
<henninge> benji: line 302 of the diff
 * benji looks.
<henninge> benji: you add a call to verifyNewlineConsitency to the importer.
<henninge> benji: that will throw an exception
<henninge> if the newlines don't match.
<henninge> let me go to the code
<benji> henninge: true, it will throw an exception, which is caught in POTemplate.importFromQueue and POFile.importFromQueue (it wasn't being thrown before, but the catch has been there for a while)
<henninge> benji: I am sorry, I am just realizing that I was wrong
<benji> heh
<henninge> benji: it *was* being thrown before
<benji> henninge: well, it was being thrown, but during imports (only during TTW updates of translations)
<benji> (TTW = "through the web")
<henninge> benji: well, there is storeTranslationsInDatabase
<henninge> benji: which calls sanitize_translations_from_import
<henninge> benji: so, that was already throwing the exception
<henninge> benji: I may be missing what we gain from your, code though?
<henninge> your code, though
<benji> henninge: I don't know that you're missing anything, because now I'm confused
<henninge> benji: but I am sorry, I did not look far enough to realize that these exceptions are indeed being caught.
<henninge> benji: the fact that we have such data in the database is old data that was imported before sanitization was working properly.
<benji> ah!  that answers my next question
<benji> in that case I'm inclined to add some tests to the already existing code and then figure out how to kill the pre-existing bad imports
<henninge> benji: what I find interesing as well is that the bug mentions "importing translations" in the title but refers to web submissions.
<henninge> but I guess that is just Diogo getting mixed up with the terms.
<benji> could be, or he (like I) assumed that the data was recently added
<henninge> but the OOPSes in the bug are not from the imports
<benji> true, but the reason we get OOPSes is because bad data got in to start with (or rather, we redefined what bad data was after the fact)
<henninge> yes, so we need to live with that fact.
<henninge> benji: so, the fix for those OOPSes would be to not check the English string again when submitting translations.
<matsubara> henninge, I didn't change the description to include "importing translations". the original bug report only mentioned the mixed newline oops raised.
<henninge> matsubara: hm, the bug does not say it's title or description were ever changed.
<henninge> benji: ah, I see: the newline style is derived from the English original (it all comes back to me)
<matsubara> henninge, isn't it bug 742662? see the change between comments #2 and #3
<_mup_> Bug #742662: Mixed new line markers causing OOPS while importing translations <bad-commit-13889> <oops> <qa-bad> <rosetta-imports> <Launchpad itself:In Progress by benji> < https://launchpad.net/bugs/742662 >
<henninge> benji: that is when the OOPS gets raised
<henninge> matsubara: sorry /me is blind
<henninge> matsubara: you are right
<benji> :)
<henninge> benji: so the fix is the following:
<henninge> When sanitizing translations from the web ui, the MixedNewlineMarkersError needs to be caught and cause newlines not to be normalized
<deryck> Morning, all.
<henninge> because we don't know what the English original uses.
<henninge> When sanitizing translations on imports, the exception should be treated as before.
<henninge> For bonus points it would be great to not abort the import of the file completely but just report an error for this one string, like is done with other errors. But that is really outside the scope of this bug.
<henninge> benji: is that understandable?
<benji> that sounds quite reasonable
<henninge> Hi deryck! ;)
<henninge> benji: cool ;)
<henninge> benji: I updated the bug.
<benji> henninge: great, thanks!
<StevenK> Can haz review https://code.launchpad.net/~stevenk/launchpad/kill-malone.bugmessage_owner/+merge/74775 ?
<bac> StevenK: i'll review it next
<henninge> bac: too slow... ;-)
<henninge> StevenK: r=me
<bac> thanks henninge.
<henninge> which reminds me ...
<StevenK> henninge: Thanks!
<henninge> Do we have provisions for community reviewers?
<henninge> Another week and I'd be one.
<jtv> I don't think that ever really got off the ground.
<jtv> Otherwise I would have gotten al-maisan to review my branch today.
<StevenK> henninge: You can, but I still think it requires sign-off by a Canonical person
<henninge> That's ok, I'll probably be too busy anyway ...
<al-maisan> henninge: what is going to happen in a week's time?
<henninge> al-maisan: I am leaving Canonical on the 16th.
<al-maisan> ah, I see -- good luck with your future endeavours !
<al-maisan> s/with/in/
<henninge> al-maisan: thank you!
<henninge> deryck: Audio problems, rebooting.
<henninge> deryck: I just had a great business idea  - "Cake Pieces Online World-wide"
<deryck> heh
<deryck> For Virtual Going Away Parties Everywhere! :)
<henninge> you pick your type of cake online and send a number of persons a piece each with a little note.
<StevenK> bigjools: Can you make bug 396097 a dupe of jtv's gina domination bug?
<_mup_> Bug #396097: Debian import doesn't remove packages <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/396097 >
<henninge> deryck: exactly ;-)
<nigelb> "gina domination" sounds like something that should probaly be not read outo of context
<nigelb> henninge: you can sell to Canonical and Mozilla  - two companies with lots of remote teams ;)
<bigjools> StevenK: jtv can even do it!
<jtv> StevenK: you want it, _you_ do it!
<jtv> nigelb: see my merge proposal entitled âGina the Dominatrixâ
<nigelb> jtv: srsly?
<jtv> srsly.
<nigelb> I just googled for it.
<StevenK> jtv: I don't know the bug number, sorry
<nigelb> I wonder what google now thinks of me.
<StevenK> And I thought jtv was EOD'd
<flacoste> nigelb: you'll get interesting ad link now ;-)
<jtv> StevenK: I guess you want bug 830890
<_mup_> Bug #830890: Gina should delete publications if removed from source archive <derivation> <qa-ok> <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/830890 >
<nigelb> flacoste: Surprisingly, no ads. Noone thought of it!
<jtv> StevenK: and I'm trying to be EOD because I'm OD'ed on Soyuz.  But I gave Julian one "end of this week" estimate too many and I want this done.
<jtv> My main blocker for today: getting a reviewer.
<StevenK> jtv: So marked
<flacoste> nigelb: there is a business opportunity there!
<nigelb> flacoste: Haha, I thought so too. We could rick roll them into contributing to gina ;)
<flacoste> lol
<jtv> flacoste: thanks for pointing out the Google angle.  These are after all the people smart enough to charge their customers for showing you "spam recipe" ads in your spam mailbox.
<rvba> bac: if you're interested in the follow up branch ;) https://code.launchpad.net/~rvb/launchpad/sync-bug-827608-credit-copy/+merge/74769
<bac> rvba: ok
<rvba> bac: thanks.
<bac> rvba: in your MPs you should list the db branch as a pre-req, no?
<rvba> bac: well, the way I understand the new fdt process it's not a pre-req technically is it?
<bac> rvba: good question.  you can't run your test without it...
<bac> rvba: but in terms of getting a good diff you may be right
<nigelb> GRAR. Test failure :(
<rvba> bac: True, I've been advised by lifeless to put the sql patch in my tree, unversioned.
<adeuring> nigelb: that's what the EC2 tests are for :) The fix shouldn't be difficult
<nigelb> No, but I won't be able to get it for another 1 hour at least.
<rvba> bac: we will see if I'll be able to put it up for review before I EOD but a final branch is coming up. Then it will be the end of the saga :)
<nigelb> Can I work on the followup branch before this one lands?
<StevenK> nigelb: You can
<nigelb> Yay, not as depressing then
<adeuring> nigelb: the follow up branch is the one with the DB patch? If so, sure, continue to work on it. But we should not land it before the other one lands
<nigelb> adeuring: AH, ok
<deryck> abentley, codehosting runs for me.  on a freshly up to date devel.
<deryck> abentley, didn't try that, sorry.  Thought you couldn't get it to run.  Will try now....
<abentley> deryck: It ran for me, but I couldn't push to it.
<deryck> abentley, what's a local push command?  Don't remember, or haven't done it before.
<abentley> deryck: bzr push lp://dev/~mark/+junk/foo
<bigjools> is the person picker supposed to time out?
<deryck> abentley, literally try that?  Or push to some user I create for myself locally?
<abentley> deryck: literally mark.  You should have .ssh/config configured for that user.
<jml> lifeless said that Launchpad was growing a table that had same/similar data to Contents.gz
<jml> true?
<wgrant> jml: Yes.
<wgrant> jml: It exists, but is not populated.
<wgrant> abentley: Not by default any more: use utilities/make-lp-user instead.
<jml> wgrant: ok. ta. Exposed over API?
<jml> make-lp-user is the best!
<abentley> jml: A table of Contents.gz ?
<wgrant> jml: Not at this stage, but that is part of the rationale for having it in LP.
<jml> wgrant: cool. thanks.
<jml> abentley: not *that* similar.
<deryck> abentley, yup, works fine for me.
<abentley> deryck: okay, thanks.
<jml> also, getPublishedSources doesn't seem to have a date_published filter
<deryck> abentley, np.  Sorry it took me so long.
<jml> is that by design or simply because no one has wanted one yet?
<jml> (or am I silly for wanting one?)
<wgrant> jml: created_since_date?
<jml> wgrant: Well, that filters on SPPH.datecreated, afaict
<wgrant> jml: Do you really care about datepublished?
<wgrant> That's usually a mistake.
<jml> wgrant: and I don't really know enough to know if I want that or SPPH.datepublished
<wgrant> What is your use case?
<jml> wgrant: well, I don't know :)
<jml> wgrant: things that have been published recently
<wgrant> created_since_date=whatever, status='Published'
<jml> since I last checked, so I can maintain a local mirror of some of the data in packages.
<jml> ok
<jml> thanks.
<jml> happily, that's what I'm doing now.
<wgrant> If you want to be really quick, do a separate lookup for Pending.
<jml> why so?
<wgrant> Then you can grab the files directly from LP, and have the data before they hit archive.u.c.
<jtv> cjwatson: I'm afraid I haven't been able to make much headway with the getPublishedSources status arg.
<jml> wgrant: cheers.
<cjwatson> jtv: we'll probably just end up using status='Published' across the board for a while.
<jtv> cjwatson: it shouldn't be long before Published is the universal answer.
<bac> rvba: in the changes for https://code.launchpad.net/~rvb/launchpad/sync-bug-827608-credit-copy/+merge/74769 you've changed the ordering of the results, which will be visible in the on the browser page.  have you verified that doesn't cause confusion?
<jml> You know, I think maybe REST isn't such a good idea.
<cjwatson> Some of the callers may want to do an extra query for status='Pending' for the same reason wgrant lists above.
<wgrant> jml: It's a great idea.
<wgrant> jml: It just doesn't really work.
<jml> wgrant: +1
<jtv> Isn't that what mark said oh, pretty much off the bat years ago?  Me, I still don't know.
<bigjools> anything involving xml sucks
<wgrant> I am not seeing XML's involvement here...
<jtv> I'll tell James next time I see him though.
<bigjools> it's a general observation
<nigelb> bigjools++
<jtv> Not very often these days, but second-to-last time he said "I certainly hope I won't still be using XML in 10 years' time.  That would be kind of depressing."
<jcsackett> i have just had to rebuild a system, and i'm getting gnomekeyring.IOError when i try to run ec2 test/land. i seem to recall this being a problem once upon a time -- anyone remember the issue?
<nigelb> Has anyone tried Android development? Its Java and XML.
<bigjools> xmlsucks.org
<jml> jcsackett: not I, sorry.
<jtv> nigelb: thanks for pointing that out.  Now I feel even better about my basic old nokia.
<nigelb> jtv: I have a basic sony errikson :)
<jtv> jcsackett: I've seen thatâ¦ IIRC it was some package that you're assumed to have installed.
<wgrant> jtv: Ah! I am glad it's not just me who doesn't have a smartphone.
<jcsackett> jtv: hm. so now i must muddle through to find said package. at least it's a direction to investigate, thanks. :-)
<jtv> wgrant: flexispy.com is one strong argument not to have a smartphone
<nigelb> wgrant: So, the advantage of smart of being connected all the time. I don't think you stay away from the computer (ZING!)
<wgrant> Heh
<nigelb> *smartphone
<nigelb> Too hasty to make a funny remark
<jtv> jcsackett: I can dig up some more probably-unhelpful clues from my memory: it was something gnome and kde had their own packages for, and somehow it was assumed that you'd have it pulled in in one of those two ways.
<jcsackett> jtv: so, i know that there's like the wallet and the manager, but i have the full gnome version of that.
<jtv> jcsackett: now to find the python packageâ¦
<jcsackett> aaah. it's not just paramiko, then.
<jtv> bigjools: there's not much at xmlsucks.org
<jtv> And by the way, nigelb: flexispy is also all Java and XML.
<jtv> dumb is fine by me
<nigelb> jtv: *stab*
<jtv> aiigh!
<bigjools> jtv: there used to be more
<henninge> So, the launchpad-buildd runs on lucid but it requires bzr-builder >=0.5 which is not package in lucid. How do I solve that?
<henninge> jtv: ^ do you have an idea about that?
<henninge> I mean it must have been solved for our current builders somehow.
<jtv> henninge: one of the bzr people may know.
<henninge> yeah, I tried in #bzr ;)
<jtv> or lamont may know.
<wgrant> jelmer_: ^^
<wgrant> Are the cat packages available in a PPA somewhere?
<wgrant> There may be a PPA owned by ~launchpad or ~bzr for them.
<bigjools> james has one IIRC
<henninge> wgrant: yes, that's what I was hoping ;-)
<wgrant> henninge: lp-buildd also works fine on maverick/natty, FWIW.
<wgrant> And dapper and karmic.
<henninge> ;)
<henninge> but production buildd's run on lucid, right?
<jtv> Oh God I can't take these soap series any more. I think I'll curl up with my kindle.  Have a good weekend, everyone!
<wgrant> Uh. It's complicated.
<bigjools> o/ jtv
<wgrant> I forget what ia64/sparc/powerpc/hppa are running these days.
<wgrant> henninge: But the virt builders should all be lucid, yes, and I guess that's all you care about.
<henninge> jtv: see you next week
<henninge> wgrant: I guess I do, yes. ;-)
<henninge> but none of james's ppas has that.
<wgrant> https://launchpad.net/~dailydebs-team/+archive/bzr-builder
<wgrant> Oh.
<wgrant> Actually, the virt builders might still be hardy...
<wgrant> But anyway, the series doesn't matter too much.
<henninge> wgrant: thanks
<wgrant> Because all that's run outside the chroot is bzr-builder, basically.
<jelmer_> wgrant: sorry, back now
<wgrant> And that is heavily backported
<jelmer_> henninge: the newer bzr-builder is in the cat archive
<henninge> what is the cat archive?
<jelmer_> henninge: and there is an even yet newer bzr-builder/bzr for the buildds which I have backported here and is due to land on the buildds at some point: ppa:canonical-bazaar/cat-proposed
<jelmer_> henninge: an internal apt repository used by IS
<jelmer_> henninge: ppa:bzr/daily should also have a newer bzr-builder for lucid
<cjwatson> henninge: "Canonical Admin Team"
<henninge> jelmer_: I feel stupid but where do I see these ppas in the web ui?
<henninge> Also, I don't really want the latest, I just want >=0.5 ;)
<jelmer_> the canonical-bazaar/cat-proposed PPA is private, the bzr daily PPA is here: http://launchpad.net/~bzr/+archive/daily
<henninge> oh, +archive
<henninge> jelmer_: I don't see bzr-builder in the ~bzr ppas :(
<jelmer_> henninge: you're right - sorry. We don't seem to have a recipe for that, I'll see if I can add one
<cjwatson> Can anyone explain http://paste.ubuntu.com/685964/ to me?  I don't understand it, and it doesn't seem to have much to do with my branch
<cjwatson> Shouldn't the "..." match "api.launchpad.dev/beta"?
<henninge> cjwatson: I think that is noise
<henninge> This is probably the real culprit:
<henninge>     - suite_names:
<henninge>     -     [u'Release', u'Security', u'Updates', u'Proposed', u'Backports']
<henninge>     ? ^^^
<henninge>     + suite_names: [u'Release', u'Security', u'Updates', u'Proposed', u'Backports']
<henninge>     ? ^^^^^^^^^^^^
<henninge> No ... involved here.
<henninge> Failing doctests suck.
<cjwatson> That's odd though, because my branch doesn't touch anything that should change wrapping there AFAICS
<cjwatson> I did add a column to distroseries, but why does that change wrapping ...
<cjwatson> oh
<cjwatson>     + include_long_descriptions: True
<cjwatson> I bet *that's* the real culprit
<henninge> cjwatson: yes, fooled again ...
<henninge> Did I mention that "Failing doctests suck?"
<henninge> So easy to miss stuff in all the noise.
<jml> Hmm.
<jml> I wonder why it does that.
<jml> I bet it's because diff reporting doesn't interact with the NORMALIZE_WHITESPACE and ELLIPSIS options.
<henninge> It always does that to me: One failure and it complains about all ellipsis in the file.
<cjwatson> I suppose I'd better try to test that locally before resending to EC2
<nigelb> adeuring: Hi, can you re-land this branch for me? Fixed the test https://code.launchpad.net/~nigelbabu/launchpad/destroy-statusexplanation-88545/+merge/74704
<adeuring> nigelb: sure
<jml> yeah.
<nigelb> thanks!
<jml> it just passes expected and observed to difflib
<jml> with no processing on expected
<jml> Hmm. Shouldn't be *too* hard to fix, actually.
<jml> And by 'fix', I mean, "come up with a workaround that doesn't require patching the stdlib"
<jelmer_> henninge: there is a daily build in progress now, should hopefully land in ppa:bzr-builder/daily within the hour
<adeuring> nigelb: tests are running again
<nigelb> adeuring: thanks, I'm branching db-devel :)
<adeuring> nigelb: cool!
<gary_poster> sinzui or flacoste, do either of you have a quick answer to this question from https://answers.launchpad.net/launchpad/+question/170633 : If someone is subscribed to a blueprint, should they "receive *all* notifications, including the blueprint information changes like the milestone target" or a limited subset?
<sinzui> gary_poster, that is the wrong question :(
<gary_poster> heh, I don't understand
<flacoste> gary_poster: they only receive notifications about changes to the linked page
<sinzui> gary_poster, blueprint subscriptions are neither
<flacoste> if the linked wiki supports our "notification protocol"
<flacoste> which is basically sending emails on update to notifications@blueprints.launchpad.net
<flacoste> or something like that
<flacoste> in other words, it probably only works for the Ubuntu wiki :-)
<salgado> allenap, around?  (you're one of the people asked to review https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408 that's probably awake now, so I thought I'd ask you).  the Linaro team would benefit from having a way to get the contents of a branch as a tarball, via the API.  we'd be using it for tiny branches (less than a hundred KB), but people might end up trying to use that on large branches and that may be a pro
<salgado> blem.  so, my question is whether or not you think this could be accepted in LP
<flacoste> salgado: i think ensemble wanted somthing similar
<flacoste> salgado: there is a bug for that
<gary_poster> flacoste, sinzui, got it.  Thank you
<sinzui> gary_poster, The plans to create structural subscriptions never came to fruition, as you recall, they were moved to bug to be only about bugs
<gary_poster> sinzui, ah, right got it
<salgado> flacoste, the one I found was not about exposing it over the API, but maybe there is.  /me searches
<allenap> salgado: I'll have a look.
<salgado> flacoste, allenap, bug 515128 is the one I found
<_mup_> Bug #515128: Launchpad should make tarballs of branches available for download <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/515128 >
<gary_poster> abentley, do you have any suggestions on how to field https://answers.launchpad.net/launchpad/+question/170639 ?
<gary_poster> abentley, fwiw, I verified and what he describes does seem to be what's in the logs
<flacoste> salgado: yes, that.s the one
<abentley> gary_poster: we recommend using http with sourceforge as it seems a bit more reliable.
<gary_poster> abentley, ah, I probably could have found that somewhere; that rings a bell.  sorry to bother & thanks
<abentley> gary_poster: no problem.
<salgado> thanks allenap!
<allenap> salgado: It looks like mbp and wallyworld_ are happy with it. wallyworld_, fancy following up on your threat in https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408? That is, the last comment, where you said you'd land it :)
<jelmer_> abentley: I thought https worked better actually
<salgado> allenap, do you think this could be made available on the API?
<allenap> salgado: I mean, I don't have any problem with it. If people start downloading huge branches it /should/ time out and we'll get an OOPS report about it. I think that's okay.
<allenap> salgado: Ah, okay, I missed that part of your request.
<allenap> s/request/question/
<salgado> allenap, would we really timeout? I thought we timed out only on long-running DB queries
<allenap> salgado: I'm not sure that's a good idea. Perhaps it could be streamed into a librarian file and a reference to that returned.
<abentley> jelmer_: I've heard the opposite, but I guess that could be wrong.
<salgado> allenap, that would have to happen asynchronously, then?
<allenap> salgado: It times out on requests too now, afaik. I'm not the right person to talk about that really; I don't know enough about where those time-outs are applied.
<flacoste> salgado, allenap: only DB queries time outs
<salgado> allenap, oh, ok, I'll bug flacoste then.  thanks anyway
<flacoste> i think we had a few other place where we check the timeout
<flacoste> but not through general python processing
<salgado> flacoste, do you remember if what ensemble wants is to get the branch content via the API as well?
<flacoste> salgado: yes
<salgado> for formulas, I assume
<flacoste> exactly
<flacoste> so that they don't have to use bzr
<flacoste> to retrieve the formula
<flacoste> but fetch it directly using the API
<allenap> flacoste: Ah, okay, so if there's a non-db process taking a long time - like exporting a branch tarball - it would never trip the time-out machinery?
<flacoste> allenap: correct
<flacoste> allenap: or worse, it would trip it after too long
<salgado> flacoste, right. in our case we'd like to avoid bzr for private branches, so that we don't have to send private keys around
<flacoste> so process for 2 minutes, then do a DB query (for session or whatever other housekeeping) and then timeout
<allenap> flacoste: Are there any time-outs on the ha-proxy or Apache? I.e. if the response is not complete in X seconds it drops the connection?
<flacoste> allenap: there is one on apache, but it's much higher
<flacoste> which is fine
<flacoste> since then the client can decide to take its time
<flacoste> but the expansive server thread as been freed up at that time
<flacoste> so we don't care
<flacoste> or much less
<allenap> flacoste: Yeah, but none between Apache/ha-proxy and the app servers?
<flacoste> allenap: i don't know off the top of my head, a losa would know, not that it changes anything
<flacoste> since zope will happily continue processing
<flacoste> we'll know the connetion was close only when we attempt to write to the connection
<salgado> flacoste, so, do you think we could make that available over the API now or would we need some way to prevent it from being used to download large branches?
<allenap> flacoste: The loggerhead branch that exports tarballs expects to stream, I think, so a time-out on the app server connection would put a bound on that.
<flacoste> salgado: we had tons of problem with streaming through the app server, we moved that to the librarian
<nigelb> Hi, can I have a patch ID for a db patch?
<flacoste> allenap: in that case, yes
<flacoste> is this a loggerhead or launchpad patch?
<salgado> flacoste, the existing one is loggerhead
<gary_poster> abentley, any ideas on https://answers.launchpad.net/launchpad/+question/170658 ?
<salgado> flacoste, that may work for ensemble, in fact
<salgado> but not for us as we need to get the contents of private branches
<allenap> flacoste: D'oh, that's not going to work for salgado. /me headdesks.
<flacoste> salgado: why? loggerhead can be used with private branches
<abentley> gary_poster: no, at first glance, I'd expect it to be linked.
<allenap> flacoste: I think it's the proliferation of credentials that's a problem.
<flacoste> how is it different than with API?
<salgado> flacoste, yeah, but then I have to give the SSO credentials to my client and make it do OpenID, no?
<salgado> with the API I can use OAuth
<gary_poster> abentley, ok thanks.
<salgado> which was better summarized by allenap. :)
<flacoste> salgado: we could teach loggerhead to accept oauth auth
<salgado> that could work
<flacoste> salgado: or a way to get an api script to get an 'auth token' that will work to make loggerhead reqeuest
<salgado> flacoste, loggerhead doesn't have access to LP's DB, right?  I was wondering if it'd be possible to use the LP OAuth credentials to authenticate on loggerhead
<salgado> that'd be the ideal way, I think
<flacoste> salgado: right, it doesn't have DB access, but the auth glue code might have
<flacoste> that's the launchpad_loggerhead thing
<salgado> but getting a separate loggerhead auth token via the LP API, although more cumbersome, would still be acceptable, I think
<flacoste> i'm not that familiar with it
<salgado> ok, I'll have a look at that
<salgado> thanks flacoste and allenap!
<allenap> salgado: I think there's some support for general one-time tokens in Launchpad already, so half a solution might already be there.
<salgado> oh, cool, that's good to know
<gary_poster> flacoste, response from that blueprints notifications discussion:
<gary_poster> """Could you point me to the people behind the Ubuntu wiki changes (notification protocol support). That's probably something that we would like to have on Linaro's wiki (using moinmoin)."""
<gary_poster> flacoste, should I ask them to file a bug, or is there something quic we can do for them?
<flacoste> gary_poster: https://help.launchpad.net/Blueprint#Subscribing_to_blueprints
<gary_poster> flacoste, fantastic thanks
<flacoste> i think that page needs update as it's  likely the email is now  notifications@blueprints.launchpad.net
<flacoste> although there is still a MX record for specs.launchpad.net
<gary_poster> flacoste, are you confident enough on that that I shuld change the help wiki?
<gary_poster> should
<flacoste> gary_poster: i'm checking the code :-)
<gary_poster> ah, ok :-)
<flacoste> gary_poster: no, it's still configured as specs.launchpad.net
<flacoste> config value: launchpad/specs_domain
<gary_poster> ok cool thanks flacoste
<allenap> salgado: What's going to happen next? Are you going to file a bug, or will someone else request this formally? If you're expecting something from me, tell me :)
<gary_poster> allenap, so unreasonable :-P
<allenap> :)
<salgado> allenap, oh, don't worry about that; I'll investigate and see if we're going to pursue this further or just pass SSH keys around
<allenap> salgado: Cool, awesome.
<gary_poster> abentley, I changed the sourceforge import I mentioned to you before to http, and got a different error: http://launchpadlibrarian.net/79436540/zorba-coders-zorba-sf-trunk.log
<gary_poster> does that mean http is not enabled on sourceforge for this branch, or something else?
<nigelb> ohai, I asked earlier but didn't get a response. Can someone give me a patch number for a db patch?
<gary_poster> nigelb, only stub can AFAIK.  Make a mp against db-devel and ask specifically for his review (Stuart Bishop)
<gary_poster> at least...that
<gary_poster> 's how it used to work...
<nigelb> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess tells that anyone in ~launchad can do the patch ID thing
<nigelb> I can't make an MP without having a file to make changes in.
<gary_poster> nigelb, ok, hadn't seen that.  I'll do it for you, gimme a sec
<gary_poster> nigelb, I need a comment to describe what the patch does.  Short, like "Add tables for a package symbol database" or "BugMessage.owner NOT NULL"
<nigelb> gary_poster: "Deleting the statusexplanation column
<nigelb> "
<gary_poster> ack nigelb, thx
<gary_poster> nigelb, FWIW, the doc I changed had info that seemed like it was geared toward the patch writer.  It is here, if you'd like to take a glance at some advice: http://pastebin.ubuntu.com/685999/ .  Meanwhile, your patch number is 2208-84-1
<jml> Hmm.
<gary_poster> (I assumed that your patch would be fastdowntime-compatible, and not require a full old-style system outage; if it did, the last digit would be 0, and you would be asked lots of pointy questions :-) )
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 253 - 0:[#######=]:256
<jml> Actually, it's more annoying than I thought, because the ellipsis can span multiple lines.
<jml> Still, you can get something useful.
<nigelb> gary_poster: it is fast downtime compatible. I spoke to stub the other day about it
<nigelb> gary_poster: Thanks btw :)
<gary_poster> welcome :-)
<nigelb> mtaylor: Congrats on the OpenStack Policy Board elections :)
<mtaylor> nigelb: thanks!
<abentley> mtaylor: I've actually been hacking around with canonical's private openstack recently, trying to our test suite faster with parallelization.
<mtaylor> abentley: awesome
<mtaylor> abentley: how's that been going?
<abentley> mtaylor: I've run some tests in parallel, but not gotten the whole suite to run yet.  And I just found out about cloud-init, which I'm hoping will make it easier to set up the instances.
<mtaylor> indeed. I haven't used cloud-init myself, but SpamapS tells me it's awesome
<abentley> mtaylor: if it does what it says on the tin, it'll allow me to parameterize the worker nodes with the current master's IP address.
<mtaylor> that's what I hear
<nigelb> Hrm, I don't know how db-devel is supposed to work.
<nigelb> make schma asks me to run link-external-sorucecode
<nigelb> When I run it, it says
<nigelb> link-external-sourcecode: error: Parent branch not specified, and could not be discovered.
<abentley> nigelb: you need to specify the branch you want to link the sourcecode from, typically your branch of stable or devel.
<nigelb> ah
<jml> OK.
<jml> So basically doctest is terrible.
<abentley> jml: yes.  Yes, it is.
<nigelb> kill it with fire? :)
<abentley> deryck: So my problem was a stale socket, "/var/run/launchpad_forking_service.sock".  Suggestions on how to deal with this better?
<jml> http://code.mumak.net/2011/09/doctest-really-isnt-very-good.html
<jml> thoughts and suggestions welcome
<jml> disagreement will be dealt with according to its desserts.
<deryck> abentley, the code hosting issue was a stale socket?
<abentley> deryck: yes.
<deryck> abentley, from a crash, I assume, and not left behind from a normal ctrl-c?
<nigelb> I need some postgres help :/
<nigelb> tsvectorupdate BEFORE INSERT OR UPDATE ON bugtask FOR EACH ROW EXECUTE PROCEDURE ftiupdate('targetnamecache', 'b', 'statusexplanation', 'c')
<nigelb> what does that do?
<nigelb> Or rather, how do I found out what it does.
<abentley> deryck: i assume so.  Possibly a kill -9
<cjwatson> Could somebody feed https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/74743 to EC2 again?  The doctest that failed last time passes now.
<deryck> abentley, if it's a kill -9, not sure.  otherwise, so some sort of exit handler to clean up itself?
<deryck> cjwatson, I'll send it off now for you.
<cjwatson> thanks!
<abentley> deryck: I'd like it to either fail noisily and clearly, or remove the stale socket and succeed.
<jml> cjwatson: fwiw, I just spent a chunk of time trying to make those errors more decipherable. It's harder than I thought.
<abentley> deryck: or perhaps use a random socket.
<deryck> that could work
<nigelb> hrm, when I do \d bugtask in psql, I get the table as public.bugtask
<nigelb> Does this mean in my db patch I should do alter table public.bugtask and not just bugtask?
<flacoste> nigelb: nah
<flacoste> public is the default namespace
<flacoste> in pgsql
<nigelb> aaaaah
<nigelb> I need to spend a weekend reading up postgres.
<nigelb> I wish there were more hours in a day.
<nigelb> flacoste: Is something wrong with Launchpad emails/notifications?
 * nigelb sees delays
<flacoste> nigelb: not that i know of, what makes you say this?
<nigelb> Maybe its PEBKAC, checking :)
<nigelb> flacoste: Yep, PEBKAC :D
<flacoste> glad that's the case :-)
<nigelb> Hrm, really need to be able to batch team additions and deletions.
<nigelb> Oh wait. No. I'll end up doing it.
 * deryck is switching work locations, back online in 30 minutes
<nigelb> gah.
<nigelb> I broke something.
<nigelb> make newsampledata failed.
<sinzui> jcsackett, do you have time to mumble?
<nigelb> \o/ Test bassed.
<abentley> bac: could you please review https://code.launchpad.net/~abentley/launchpad/simplify-twisted-runner-3/+merge/74868 ?
<bac> abentley: sure.  let me wrap up CHR and then i'll get to it.
<abentley> bac: thanks.
<bac> abentley: done.  thanks.
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[#######=]:256
<nigelb> Hrm, are we in testfix?
<jcsackett> sinzui: I'm actually on the road; had a half day today. If you want to ring my cell tho I can talk.
<sinzui> jcsackett, sorry. I forgot
<sinzui> have a nice day
<jcsackett> No worries, and thanks. Have a good weekend.
<wgrant> abentley: That TwistedJobRunner branch solves the massive NoneType response tracebacks we get sometimes?
<nigelb> wgrant: ohai
<wgrant> nigelb: Hi.
<nigelb> wgrant: I have a doubt for a db patch
<nigelb> I ran into trouble with the fti trigger
<wgrant> Ah, that searches statusexplanation?
<nigelb> tsvectorupdate BEFORE INSERT OR UPDATE ON bugtask FOR EACH ROW EXECUTE PROCEDURE ftiupdate('targetnamecache', 'b', 'statusexplanation', 'c')
<nigelb> ^^ that one
<wgrant> So, fti is the thing most in limbo right now.
<wgrant> The script that maintains it (database/schema/fti.py) is not run any more.
<wgrant> Because it has to be done during downtime, but can take too long.
<nigelb> oh.
<wgrant> So you should talk to stub about this.
<nigelb> okay :)
<wgrant> We may have to fix fti.py to just maintain the triggers, without actually rebuilding the data.
<nigelb> THe "psql launchpad_ftest_playground -f your-patch.sql" step ran into trouble because of this
<nigelb> so, not sure how far I'll get
<nigelb> But looks like I need to catch stub on Monday.
<nigelb> so this branches goes onto hold mode :)
<nigelb> wgrant: Also. Are we in testfix?
<wgrant> nigelb: We are.
<wgrant> But I just got the buildbot slaves cleaned up, and forced the builds.
<nigelb> Cool. That explains that.
<nigelb> Yay!
<wgrant> You've got something that needs landing?
<nigelb> Its finished tests
<nigelb> so, I guess your buildbot cleaning should fix it.
<wgrant> It won't automatically reland.
<nigelb> oh.
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/destroy-statusexplanation-88545/+merge/74704
<wgrant> Could you forward me the ec2 mail?
<nigelb> Done
#launchpad-dev 2011-09-10
<wgrant> nigelb: Landed.
<nigelb> wgrant: thanks!
<cjwatson> wgrant: could you do one for me in the same situation?
<wgrant> cjwatson: multiarch-translations?
<cjwatson> yep
<wgrant> cjwatson: It passed ec2?
<cjwatson> yep, forwarding you the test mail now
<wgrant> Thanks.
<wgrant> cjwatson: Landed as r13909
<wgrant> cjwatson: I guess this is reasonably urgent?
<wgrant> We should be able to deploy it Monday/Tuesday.
<cjwatson> Thanks.  Moderately; if at all possible I would like to switch it on before beta 2.
<cjwatson> That gives us time to switch it off again and/or fix it up if it goes horribly wrong.
<wgrant> Yup.
<wgrant> I'll see how QA goes this weekend, and might arrange a poppy downtime window for my Monday morning if things go well.
<wgrant> Since Monday's 1830 slot is occupied by a DB patch.
<wgrant> 0830 slot, that is.
<cjwatson> Mon/Tue gives us adequate time.
<wgrant> OK, might just do it Tuesday then.
<wgrant> Thanks.
<StevenK> Rargh, and buildbot is going to fail.
<nigelb> lawl, so there's now Nigel and nigelb in the channel.
<nigelb> FUN.
<jtv> pgbouncer fixture is breaking buildbot: https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1353/steps/shell_6/logs/summary
<nigelb> ohai jtv
<jtv> Hi
<nigelb> *yawn* 4 hours of sleep. Sigh.
<abentley> wgrant: sort of.  a None response is treated as a failure now.
<wallyworld_> nsi
<nigelb> stub: Hey!
<nigelb> Got a minutes? I need some help with a db patch (the one I talked to you the other day about)
<stub> nigelb: ok
<nigelb> I removed the statusexplanation field, but I'm having trouble with this
<nigelb> tsvectorupdate BEFORE INSERT OR UPDATE ON bugtask FOR EACH ROW EXECUTE PROCEDURE ftiupdate('targetnamecache', 'b', 'statusexplanation', 'c')
<nigelb> william asked me to talk to you since we don't run fti.py anymore during the fast downtimes
<wgrant> I suspect we want to fix fti.py to have a mode to just fix the triggres.
<wgrant> Without rebuilding columns...
<stub> nigelb: I see. I'll need to handle the DB patch while I work out the best way to proceed. I think I'll be manually rebuilding that trigger function.
<nigelb> I already have a patch number and a patch, do you want me to continue or just take over?
 * StevenK wills the tests to write out the unknown file he is chasing
<stub> If you want to give it a go, I think it will be a "CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE ON bugtask ..." like you see, except with 'stausexplanation','c' removed from the arguments
<nigelb> I'll give it a shot.
<stub> wgrant: I'm just dropping fti.py, as we out grew it long ago. We really need data from multiple sources in a single index etc.
<wgrant> Yeah.
<wgrant> Good plan.
<nigelb> hrm, syntax error.
<StevenK> Paste what you have?
<nigelb> http://pad.ubuntu.com/kCRLMrXkW3
<nigelb> gah
<nigelb> wait
<nigelb> excellent, that didn't work. because the trigger already exists.
<stub> Yay SQL the query language for mortals
<wgrant> CREATE OR REPLACE TRIGGER?
<wgrant> Does that work?
<stub> yer, DROP TRIGGER first since there is no CREATE OR REPLACE TRIGGER
<nigelb> heh, ok
 * nigelb googles how to drop trigger
<wgrant> DROP TRIGGER whatever ON sometable
<stub> I find file:///usr/share/doc/postgresql-doc-8.4/html/sql-commands.html invaluable.
<nigelb> worked
<nigelb> Do I add the trigger queries into my patch as well?
 * nigelb heads for lunch
<stub> nigelb: The DROP TRIGGER, CREATE TRIGGER go into your patch, yes. I think you also need to modify fti.py as well, as it will currently undo your changes. So the db patch will affect staging and production which no longer run fti.py, and dev boxes get the rebuild version from fti.py. Which sucks, so Bug #815717
<_mup_> Bug #815717: Kill fti.py <fastdowntime-later> <Launchpad itself:Triaged> < https://launchpad.net/bugs/815717 >
 * StevenK finally figures out where lp.code.model.tests.test_sourcep and lp.soyuz.tests.test_binarypackag are leaking from.
<nigelb> stub: heh, okay :)
#launchpad-dev 2011-09-11
<nigelb> Is lifeless back tomorrow or next monday?
<LPCIBot> Project db-devel build #855: FAILURE in 0.28 sec: https://lpci.wedontsleep.org/job/db-devel/855/
<Nigel> ouch, build failure in 0.28 sec, that is fast
<nigelb> its going to get opened anyway :)
<nigelb> (bah, wrong channel)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #856: FIXED in 4 hr 41 min: https://lpci.wedontsleep.org/job/db-devel/856/
<lifeless> morning
<lifeless> nigelb: I am back now
<nigelb> lifeless: GOOD Morning!
<nigelb> I just wrapped up a db-devel patch
<nigelb> I have to run the entire *tests* before I submit an MP?
<lifeless> no
<lifeless> they *have* to be run before landing ;)
<nigelb> Ah. Instructions said so.
<lifeless> if you don't run them all then ec2 may find a missing change and you'll need to get reviewed etc again.
<nigelb> I can start a test run, but I'm afraid it might take quite a while on my core 2 duo :)
<lifeless> how much memory do you have?
<nigelb> 2Gigs
<lifeless> the process is less likely to get restarted if you've run them all, but I consider that 'must' a 'should' :P
<lifeless> you might like looking into the lxc parallel stuff and run 2 test runners with 1/2 the tests each
<nigelb> ok, I'll start a test run and head to bed.
<nigelb> lifeless: Family doing okay now? :)
<lifeless> we're still learning, but the medical stuff seems to have settled down.
<nigelb> :)
<nigelb> That's good to hear
<nigelb> Oh.
<nigelb> db-devel test != devel tests.
<jelmer> nigelb: are you running oneiric?
<nigelb> Nope. Old School! Lucid
<jelmer> hah, ok
<nigelb> Hrm, one error so far. Looks unrelated. http://paste.ubuntu.com/687111/
<jelmer> nigelb: looks like you might be missing a copy of pgbouncer
<nigelb> Isn't that a dep?
<nigelb> Or rather, shouldn't it be a dep?
<nigelb> webapp-authorization.txt is failing as well.
<nigelb> woah.*lots* of fail :(
<nigelb> ARGH.
<nigelb> How often does devel get merged into db-devel?
<nigelb> I may have made a *big* mistake :|
<lifeless> devel->db-devel - every time devel passes buildbot
<nigelb> makeBugTasks() in tests are failing because that field doesn't exit.
<nigelb> *exist
 * nigelb looks at models
<nigelb> GRAR.
<nigelb> Yep. Its here.
<lifeless> that means the process worked :P - we've just shown that we haven't (yet) removed deps on the field completely.
<nigelb> Well, technically, no.
<nigelb> I removed these deps the other day.
<nigelb> I sense my db-devel is out of date.
<nigelb> \o/ Yes. Stabbing myself in the foot++
<lifeless> ftw, ftw, ftw
<nigelb> On a more positive note, now I know why I should land the changes to not use fields first and then start the db patch.
<lifeless> doing teaches well :)
<nigelb> Yeah. The DB process is neat. Taught a bit of postgres as well.
<nigelb> lifeless: Is there already a bug about JS OOPS or do you want me to file one and assign to myself?
<lifeless> I don't know
<nigelb> A quick google search doesn't give me anything. But I might be phrasing it wrong.
<lifeless> I wonder if we've fixed bug 294656 yet
<_mup_> Bug #294656: Every page requests two JavaScript libraries (remove MochiKit) <lp-bugs> <lp-translations> <lp-web> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/294656 >
<lifeless> bug 741991
<_mup_> Bug #741991: launchpad users have to manually report javascript failures <javascript> <Launchpad itself:Triaged> < https://launchpad.net/bugs/741991 >
<nigelb> lifeless: nope, not fixed yet. Still see it.
<nigelb> need to start figuring out where to start.
<lifeless> on mochikit or oopses ?
<nigelb> oopses
<lifeless> make sure you have read https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements
<lifeless> if you would like some ahead-of-time design, create a technical LEP (I don't think it needs one)
<nigelb> I think you handed me a technical LEP the other day :)
<lifeless> then, bring up a microservice for it - involves making a new project - https://dev.launchpad.net/CreatingNewProjects
<nigelb> "Don't be overly cute." HA.
<lifeless> you may need 2 projects - one for the js, one for the server.
<nigelb> So, just start a separate project on Launchpad?
<lifeless> that will depend on how you plan to get the js into lp pages
<nigelb> I might need some help there, because I'm not sure how to do that.
<lifeless> sure
<nigelb> I've only touched Launchpad-related JS once.
<lifeless> I wouldn't make the project until you've a handle on that then.
<lifeless> cause you won't know if you're making 1 or 2.
<lifeless> anyhow the microservice needs to be truely self contained, tested, etc.
<nigelb> I guess the sensible approach is to just start writing the server-side code in a +junk branch and finish that off.
<lifeless> finally to integrate with LP, need to have a test that js failures in LP do reach out to the url with a GET, and that the url is correct
<lifeless> nigelb: sure, thats what I did with gpgverifyd
<nigelb> cool, so once I get this db patch done, I'll start writing the wsgi app.
<lifeless> wallyworld: hi
<wallyworld> lifeless: hello. how's parenthood?
<lifeless> wallyworld: you seem to be the closest working js person :)
<lifeless> wallyworld: *intense* :)
<wallyworld> hah, you must be desperate :-)
<wallyworld> how can i help?
<lifeless> wallyworld: nigelb is planning on working on bug 741991.
<_mup_> Bug #741991: launchpad users have to manually report javascript failures <javascript> <Launchpad itself:Triaged> < https://launchpad.net/bugs/741991 >
 * wallyworld looks
<lifeless> wallyworld: architecturally we're fine - but there is a pragmatic question around the (small) js needed.
<lifeless> wallyworld: big picture this is a microservice that translates crafted GET requests into oops reports.
<lifeless> wallyworld: with a couple of small bits of JS to turn errors in the page into GET requests to the url that that service is running on.
<lifeless> wallyworld: some of that js is about encoding stuff to the service. The rest is about how to trap errors - baseline js, and then after yui loads a yui last resort handler.
<lifeless> wallyworld: so the question is, how should we make that js available to LP pages?. The microservice URL can be mapped into our namespace (e.g. at <root>/+jsoops)
<m4n1sh> is curtis hovey around?
<m4n1sh> dont know his nick
<nigelb> m4n1sh: its sinzui and he's around US timings I believe.
<lifeless> he is not. It is sinzui.
<m4n1sh> lifeless: nigelb  thanks
<nigelb> UTC-0400 to be more precise.
<wallyworld> lifeless: we could hook something in using base-layout-macros.pt. i'm not 100% familair with yui's error hooks though
<m4n1sh> wanted some help on lp-release-manager-tools
<lifeless> wallyworld: I kindof meant - do we copy the js in (to avoid another page load), or do we have a combo loader that can talk across projects, or do we inline the code (and reference it via an egg or something?)
<lifeless> wallyworld: this doesn't need to be sorted $now, just when nigelb is getting close to done on the service.
<nigelb> (I'm just planning on starting the service this week)
<wallyworld> lifeless: first thought is that if the code is small which i expect it will be it would be nice to copy it in to avoid another page load, hence the base-layout-macro suggestion
<lifeless> inlining it but not having it in the LP tree itself would also avoid the page load
<lifeless> and perhaps be easier to maintain (can't forget to do something you don't need to do)
<wallyworld> yes. i didn't account for the fact that you want to have this as generic code outside of the lp tree?
<lifeless> thats right
<lifeless> if we do this well, u1, landscape, sso and perhaps others can use it.
<nigelb> Hrm, I hadn't thought that far.
<wallyworld> we could do what we do for yui - pack it with our stuff into a launchpad.js
<wallyworld> yui is now a tarball which is versioned using buildout
<lifeless> yeah
<lifeless> ok
<lifeless> so this microservice can be released as a tarball
<lifeless> and we can unpack the tarball and yank out the js
<wallyworld> i added a buildout recipe for yui
<wallyworld> yes
<lifeless> great
<lifeless> nigelb: I think one project
<wallyworld> and pack the js into launchpad.js and include it inline where required
<wallyworld> it will be nice to have this client error reporting functionality
<nigelb> lifeless: excellent, okay.
<lifeless> nigelb: how does js-oopsd grab you as a project name ?
<nigelb> Not cute, fits the bill :)
<lifeless> would you like me to set it up ?
<nigelb> Yes, please. I'm guessing a lot of it need Canonical powers
<lifeless> it shouldn't
<lifeless> but I have it memorised :P
<nigelb> HA
<nigelb> You used http://pypi.python.org/pypi/web for gpgverifyd?
<lifeless> I did. I used even less for lp:gpgfixtures
<lifeless> erm python-gpgfixtures. Something.
<nigelb> Yeah, I saw.
<nigelb> But it seems to be alpha.
<lifeless> its not integrated to LP yet, but neither is gpgverifyd
<lifeless> I am planning on dropping web.py from gpgverifyd for just plain ol wsgi
<lifeless> or wsgi + paste
<nigelb> I'm looking at paste as wel.
<nigelb> It seems simple enough but too much work.
<lifeless> paste is now webob
<nigelb> *not too
<lifeless> but the bits I needed from paste for the gpg keyserver were -tiny-
<nigelb> Gah, I wish documentation was updated on the wsgi website.
<lifeless> its 99% plain wsgi
<lifeless> there is an updated PEP
<nigelb> 3333?
<lifeless> yes
<lifeless> ok
<lifeless> js-oopsd is all ready to go except for trunk having *content*
<lifeless> you can branch it though
<nigelb> \o/
<lifeless> What I've been doing for things like python-oops is copying everything but .bzr in from the project before and then tweaking
<lifeless> checking with 'bzr diff' that I've caught every occurence I need to worry about :)
<nigelb> lifeless: Ah, the wsgiref.simple_server is the plain ol wsgi way?
<lifeless> yes
<nigelb> Now, I'm less confused. :)
<lifeless> wsgiref is in the standard library
<nigelb> I was thinking you wanted me to first implement a library based on the PEP and then use it.
<nigelb> And I kept wondering what I got myself into :P
<lifeless> -lol-
<nigelb> lifeless: This will be a stateless server right? Acting only on the parameters in GET.
<lifeless> yes
<lifeless> and the options passed to the command line :)
<nigelb> Oh.
<lifeless> (like --oops-dir and --oops-prefix)
<nigelb> What kind of options? Oh wait. Let me write iteration 1 tomorrow.
<nigelb> ah, right. those.
<lifeless> the gpgverifyd __main__ should show you how to configure that up [ except you will need *two* prefixes ]
<lifeless> one prefix for failures in this wsgi app itself
<nigelb> failures?
<lifeless> one prefix to use for failures generated from js pages
<lifeless> unexpected exceptions :)
<nigelb> Ah, that prefix.
<nigelb> Woah, 3 am. *heads to sleep*
<ajmitch> so I finally got LP to run again on my laptop, it was of course a simple PEBKAC...
<ajmitch> the appserver can't talk to the openid provider when http_proxy/https_proxy is set
<lifeless> ajmitch: :P
<ajmitch> lifeless: the exception wasn't being displayed or apparantly logged somewhere, so it took a little while to find
<ajmitch> I didn't expect that it would be looking at proxy settings
<lifeless> ajmitch: not logged? thats surprising.
<lifeless> what were the symptoms you had ?
<ajmitch> an error page about OpenID provider discovery
<lifeless> did it have an oops id ?
<ajmitch> not that I recall, let me start it again & try
<ajmitch> no OOPS, just "OpenID Provider Is Unavailable at This Time"
<lifeless> ajmitch: so, in a multi provider world, thats fairly reasonable. However we're stil single provider, and if/when we go multiple we'll still want to take action if Ubuntu SSO is having issues.
<lifeless> ajmitch: -> file a bug ('failure to talk to do provider discovery on the Ubuntu SSO does not OOPS')
<ajmitch> from what I can see, it just uses lib/lp/app/templates/launchpad-discoveryfailure.pt
<ajmitch> at least I can start looking at other stuff now instead of just being blocked on login :)
<lifeless> indeed.
<lifeless> you'll filed that bug ?
<ajmitch> in the process of doing so
<ajmitch> bug 847823
<lifeless> thanks!
<ajmitch> s/823/423/
<lifeless> mwhudson: 120 lines. Really?
<lifeless> mwhudson: as in, can't hatchet it down to size ?
<mwhudson> lifeless: as in, this was presented to me as the way to do the query using our current data model
<lifeless> ><
<mwhudson> which we kinda knew was wrong, but this is a new illustration :)
<lifeless> pgsql will probably have a hernia optimising it
<lifeless> and degrade to to a genetic optimiser
<mwhudson> really we should be using a kv store for this (i suspect)
<lifeless> what thing?
<mwhudson> http://validation.linaro.org/lava-server/dashboard/reports/kernel-ci-data/
<lifeless> wow
<lifeless> I don't see how you can get to 120 lines for that :)
<mwhudson> well, using django's contenttypes thingy is part of it
<mwhudson> (and that wasn't my fault)
<lifeless> thats a horrible name in a web app
<lifeless> truely inspired
<lifeless> you'd have to search quite hard to find something more ambiguous with a different default association
<mwhudson> the horror does not stop with the name
<lifeless> did you mean 120 lines of python that generates sql, or 120 lines in the resulting sql ?
<mwhudson> i mean 120 lines of hand written sql
<lifeless> wow
<lifeless> how does that related to the contenttypes thingy?
<jelmer> hey lifeless, welcome back
<lifeless> thanks
<lifeless> flacoste: horrible name, but http://pypi.python.org/pypi/RunningCalcs/0.1 - online algorithms; might be able to simplify the PPR using this
#launchpad-dev 2012-09-03
<wgrant> wallyworld_: Right, but you need to query the APs to find out the in-use stuff, and the APs include the permitted stuff
<wgrant> So it can be simplified to just returning the APs
<wallyworld_> sure, i was speaking logically as to what is missing
<wgrant> wallyworld_: Do you think we can Won't Fix bug #1008538 now?
<_mup_> Bug #1008538: Bug Supervisors are not notified when the project is not shared <bugs> <disclosure> <sharing> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1008538 >
<wallyworld_> perhaps. we were waiting for everything to be migrated across i think
<wallyworld_> so can't we just leave it in open till then?
<wgrant> I guess
<wgrant> I just don't really think there's much we need to do in terms of it now
<wgrant> Or in the future
<wallyworld_> i think we were going to create structural subscriptions
<wgrant> Mmmm, possibly.
<cr3> when using launchpadlib, how can a bug_task.status == u'Triaged' and bug_task.date_triaged == None :(
<wgrant> cr3: Possibly if it was created Triaged
<cr3> wgrant: interesting, so I'll fallback to date_created when any of the status related dates is None. thanks!
<StevenK> wgrant: https://oops.canonical.com/?oopsid=8dffb8ed96a390893ecaef560b7af41a
<wgrant> StevenK: wallyworld's thing will fix that
<wgrant> Though there is an underlying bug that should also ideally be fixed
<wallyworld_> if the branch scanner ever completes :-(
<wgrant> wallyworld_: You linked the bugs before the scan completed
<wgrant> It's cursed
<StevenK> wgrant: What's the underlying bug?
<wallyworld_> linking bugs is ok
<wallyworld_> creating mp is not
<wgrant> wallyworld_: Evidently not :/
<wallyworld_> i've linked bugs without problems for ever
<wallyworld_> but creating mp is problematic
<StevenK> Doing *anything* with a LP branch before its scanned leads to cursing.
<StevenK> Even looking at it.
<wallyworld_> hmmm. not for me till now
<wallyworld_> i've tried the push -r-2 trick. what else can i try?
<StevenK> Renaming the branch on LP and re-pushing
<wallyworld_> besides that :-(
<wgrant> Nothing
<wallyworld_> balls
<wgrant> Besides SQLing the BRs away without a timeoput
<StevenK> Haha
<wgrant> In general, don't touch a large branch before the initial scan completes
<StevenK> Can we delete BranchRevision for being complete crap?
<wgrant> [2012-09-03 02:01:54,379: INFO/PoolWorker-2] Job resulted in OOPS: OOPS-5de0a53e4be5589c2a0309cbe7f925b9
<wgrant> anyway
<StevenK> PoolWorker-2 really is cursed
<StevenK> Bleh
<StevenK> Can't add a team to +sharing, so can't change the owner of the branch
<StevenK> Or maybe this thing has a BVP, which would be quite sad
<wgrant> StevenK: Oh, sorry, meant to say: you can work around that one by just setting Private and Private Security to All.
<wgrant> As well as Proprietary
<wgrant> the first two will do nothing
<wgrant> But it should prevent the crash
 * wgrant tries
<StevenK> Yeah, that worked out it
<StevenK> around it, that is
<StevenK> Hah
<StevenK> +sharing for me states the branch is Private, when the branch itself states it is Proprietary.
<wgrant> Have you refreshed?
<wgrant> Where does +sharing state that it's private?
<wgrant> I don't see that
<wgrant> Oh
<StevenK> +sharing/stevenk
<wgrant> That is interesting
<StevenK> Quite
<wgrant> 'cause the APA is correct
<StevenK> Maybe we have a bug in PersonPillarSharingView?
<StevenK> However it's named.
<wgrant> I assume so
<wgrant> Or possibly in the sharing service, probably
<StevenK>             # At the moment, all branches displayed on the sharing details
<StevenK>             # page are private.
<StevenK>             information_type = InformationType.USERDATA.title
<StevenK> Face. Palm.
<wgrant> heh
<wgrant> Sounds like an easy fix
<StevenK> Yeah, I'll look at it
<wallyworld_> i think that stuff was written for the sharing details page way before information types
<wgrant> Before they were implemented properly for branches, yes
 * StevenK tries to work out where the tests for PillarPersonSharingView are hiding.
<StevenK> wallyworld_: I don't think PillarPersonSharingView is tested. :-(
<wallyworld_> StevenK: not sure, i didn't write it. i can look
<StevenK> wallyworld_: Second set of eyes would be excellent
<wallyworld_> StevenK: PillarSharingDetailsMixin
<wallyworld_> test_pillar_sharing
<wallyworld_> so TestProductSharingDetailsView etc extend the mixin
<StevenK> wallyworld_: Ah ha! Thanks
<wallyworld_> np
<wallyworld_> the tests look like they could have some added
<StevenK> Bah.
<StevenK> I have it create a proprietary branch, and it dies creating an APA saying it already exists
<wgrant> You shouldn't need to manually create APAs any more
<StevenK> Heh, nice. Dropping the APA lines gives me +22, -23
 * StevenK will push the branch up and propose after lunch
<StevenK> Actually, no. Since my lunch has to cook, I'll put it on and then push the branch
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/branches-are-more-than-private/+merge/122436
<wallyworld_> StevenK: i'd add the test to the subclass
<wallyworld_> if it's only applicable for products
<wallyworld_> i'd also just call the test view_data_model_branch , remove the word proprietary
<StevenK> wallyworld_: branches are already tested by test_view_data_model, I was trying to be clear.
<wallyworld_> hmm. i'm not sure we need a separate test then. we dont' have a separate test for proprietary bugs
<StevenK> wallyworld_: So, I can rename it to proprietary, create a bug as well and check that both of their information_types are golden?
<wallyworld_> do we need to? so long as the data model tests check that the json info type matches that of the artifact, aren't we ok?
<wallyworld_> ie is the existing test sufficient?
<StevenK> wallyworld_: So the
<StevenK> Sigh
<wallyworld_> i guess i'm asking, why doa special test for one specific info type
<StevenK> wallyworld_: So the problem was that the existing tests create a bug and branch as USERDATA and then assert that is what is in the JSON.
<wallyworld_> maube should need to loop across all info types then
<StevenK> wallyworld_: The UI was overriding all branches information_type to USERDATA -- so I'm creating a branch that is a type that isn't USERDATA and checking what's in the JSON.
<StevenK> I could use PRIVATESECURITY just as well
<wallyworld_> yeah, understood
<wallyworld_> i guess i have a small issue with adding a new test just for one specific info type. we should have a general test that lops over all info types if we don't trust the test that's there
<wallyworld_> or change the info type to != USERDATA
<wallyworld_> as you suggest
<wallyworld_> that may be easiest
<wallyworld_> since it gives us a failing test before the code change, and a working test after the fix
<StevenK> wallyworld_: Sorry, was noming. http://pastebin.ubuntu.com/1182844/
<wallyworld_> np, let me look
<wallyworld_> StevenK: sure, but i think we don't even need a whole new test. just make the current data model test fail by changing the info type of the branch to PS or something other than UD, and the code fix makes everything ok again
<StevenK> wallyworld_: Oh, HAH
<StevenK> wallyworld_: I can't do it that way -- the branch is created as USERDATA and the JSON is populated on view creation so I can't transition it away, which would have been why I created a new test in the first place.
 * StevenK sees if he can grab the branch before the view and transition there.
<wallyworld_> can can change the makeArtifactGrantee method
<StevenK> I can change it to create a PRIVATESECURITY branch, sure
<wallyworld_> and that will make the test fail without the code fix
<wallyworld_> so i think that's all you need
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1182879/
<wallyworld_> StevenK: i think that's ok. minimal changes, but the tests still fail without the code fix being applied
<StevenK> And they do, yes.
<wallyworld_> cool. adding a new test in this case just didn't sit right weith me
<wallyworld_> StevenK: see, i even got you some LOC credit
<StevenK> wallyworld_: Mercy buckets
<StevenK> wallyworld_: It's not like I needed more.
<StevenK> % loc-contributions 'Steve Kowalik'
<StevenK> -3236
<wallyworld_> well give some of yours to me then
<wallyworld_> :-P
<StevenK> wallyworld_: I would if I could
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<StevenK> That's better.
<wallyworld_> lol
<wallyworld_> i'd love to revisit out bug triage policy based on these numbers
<wgrant> http://webnumbr.com/launchpad-critical-bugs :)
<wallyworld_> no, i meant i disagree with the triage policy
<wgrant> I know, I was just showing you some even prettier numbers
 * micahg stopped filing timeout bugs
<StevenK> wgrant: That user destroyed thier MP
<StevenK> Which is unfortunate, I wanted to grab sinzui's testcase off it
<wgrant> StevenK: I can forward the email if you wish
<StevenK> wgrant: From Curtis? Please do
<wgrant> This is why we shouldn't allow anybody to ever delete anything :)
<StevenK> No, we need to congratulate him -- he managed to delete a Launchpad branch
<wgrant> You have a good point...
<StevenK> He is nothing but persistant
<StevenK> He's put another branch up which actually fixes the issue, but doesn't include a test.
<StevenK> wallyworld_: Is PopulateProjectSharingPolicies done?
<wgrant> Yes, it can die
<StevenK> stub: O hai, https://code.launchpad.net/~stevenk/launchpad/drop-unused-branch-function/+merge/122431 would love a review
<wgrant> As of Friday all projects are created with either Public or Proprietary
<StevenK> I'll kill it
<wgrant> There's even a card for it
<stub> StevenK: Applied happily to qastaging. Guess I should run it on production.
<stub> staging is rather borked - empty db
<StevenK> Haha, ouch
<StevenK> stub: Let me lp-land it
<StevenK> stub: Aren't we in the middle of backups?
<stub> probably, but that doesn't matter for this
<stub> That is only a problem for live index builds
<StevenK> Right, okay.
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-populate-project-sharing-policies/+merge/122442
<wgrant> StevenK: +3? :(
<StevenK> wgrant: Oh, bleh.
<StevenK> And all of those 3 are imports that went from multi-line to single, so shush
<wgrant> It was worth a try.
<wallyworld_> wgrant: i just had a look at the accesspolicy code - are you stre the other delete methods just pass in ids? see for example def delete(cls, concrete_artifacts):
<wgrant> wallyworld_: Natural keys, not necessarily primary keys
<wallyworld_> hmm. ok. seems a bit superfulous if i have the objects
<wgrant> Yeah, but the inconsistency is a bit bad
<wallyworld_> i'll now have to unpack them just to apss them in
<wgrant> Note that it's delete(concrete_artifacts), not delete(abstract_artifacts)
<wgrant> One could argue that these shouldn't really be objects at all :)
<wgrant> But mumble mumble
<wallyworld_> i agree. i prefer to delete using just ids
<wgrant> Right, but it's a toss-up between the performance-based surrogate key and the natural key.
<adeuring> good morning
<ev> ah, staging is dead. That would explain my inability to do much of anything with the API.
<czajkowski> hmm
<czajkowski> getting it kicked
<czajkowski> ev: people are on it and it's being restored
<ev> yay
<lifeless> o/
<jml> hi
<lifeless> jml: oh hi
<lifeless> jml: did you review my fixtures branch ?
<jml> lifeless: maybe?
 * jml looks
<lifeless> jml: there is a class in there that wants to be free, pondering a new pypi project for it
<jml> lifeless: yes, I reviewed.
<lifeless> ahha so you did. Thanks
<jml> lifeless: I'd wait until there was a project that wanted CallMany & didn't want anything else of fixtures
<lifeless> jml: like testtools :P
<jml> lifeless: huh, I guess so.
<jml> lifeless: "Hard Facts" is a great read, btw.
<lifeless> jml: it is, isn't it!
<lifeless> any reviewers around ? https://code.launchpad.net/~lifeless/python-oops-datedir-repo/bug-960775/+merge/122580
<lifeless> ev: o/
<lifeless> ev: will reply to your mail today
<czajkowski> lifeless: we're down some due to USA being off today
<lifeless> yeah
<lifeless> I can still hope :)
<czajkowski> true :)
<czajkowski> lifeless: hows the little one?
<lifeless> just had a growth spurt
<lifeless> was eating massive amounts every couple of hours for 3 days
<czajkowski> heh keeping ye busy I'm sure
<lifeless> yah
<lifeless> I was at kiwipycon in the weekend
<lifeless> we could have done with more sleep :P
<czajkowski> I was at a 2 day Irish wedding , my liver isn't speaking to me
<lifeless> LOL
<lifeless> any reviewers around ? https://code.launchpad.net/~lifeless/python-oops-datedir-repo/bug-960775/+merge/122580 <- StevenK
<StevenK> wgrant is OCR today. :-)
<lifeless> StevenK: its small and you're doing work already :P
<StevenK> lifeless: I'm in the middle of a few things, I'll look at it after if someone else hasn't already grabbed it.
<lifeless> OTOH so is wgrant I see
<lifeless> StevenK: thanks!; I'll whinge at wgrant right now.
 * lifeless whinges
<StevenK> Default state, isn't it?
<lifeless> por moi?
<wgrant> lifeless: 44	+ except IOError, e:
<wgrant> 45	+ if e.args[0] == 'Empty OOPS Report':
<wgrant> Really?
 * cjwatson bangs the Python 3 compatibility drum a little
<cjwatson> (s/,/ as/)
<lifeless> cjwatson: I might care if we deployed on !lucid
<wgrant> neem is precise
<wgrant> We don't deploy on <lucid
<wgrant> so as is fine
<lifeless> lucid is python 2.6 isn't it ?
<lifeless> I thought as was 2.7
<lifeless> wgrant: this gets deployed on the various servers - e.g. everywhere.
<cjwatson> 2.6 has as
<cjwatson> LP uses 'except ... as'
<wgrant> Right, we use it in lots of places in LP
<wgrant> It's not new in 2.7
<cjwatson> Everywhere - I went through and fixed everything using the old style
<wgrant> Ah, indeed
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/new-style-except/+merge/112727
<lifeless> so sure, will change.
<lifeless> cjwatson: everywhere in the main code tree; I guarantee lots of untouched places :)
<lifeless> wgrant: as for the inspection, yes, really.
<cjwatson> Well, yes, but enough that I also guarantee it will work for you
<wgrant> lifeless: :(
<cjwatson> Life's too short to figure out how to submit patches to everything else
<lifeless> wgrant: lower layer signals that way
<lifeless> cjwatson: sure, theres about 20 projects that have the same submission process as lp:launchpad :>
<wgrant> lifeless: :(
<lifeless> cjwatson: but I understand the overhead
<cjwatson> And a bunch that don't
<cjwatson> Also about 50% of the time reviewers seem faintly confused when I submit a patch to one of the other projects
<lifeless> cjwatson: :( - thats sad, a little understandable, but sad.
<lifeless> I probably need to beat that drum some more
<cjwatson> Mostly about how to deal with landing
<lifeless> cjwatson: ah, so thats more tractable
<lifeless> wgrant: I can change the signalling, but honestly it doesn't seem worth a specific class
<wgrant> lifeless: OK
#launchpad-dev 2012-09-04
<wallyworld_> StevenK: wgrant: bug 900431. the get_branch_privacy_filter() method ignores stacked on branches. so we have a new version of the transitively_private issue. i'm not sure if we want to solve this or not. thoughts?
<_mup_> Bug #900431: branch visibility queries do not consider visibility of stacked on branches <403> <branches> <disclosure> <privacy> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/900431 >
<wgrant> wallyworld_: Right, that's related to the MP security check performance issue
<wgrant> wallyworld_: It's not a recurrence of the transitively_private issue; it existed even then.
<wallyworld_> it's a similar issue though
<wgrant> transitively_private ensured that the stacked branch was private, but it never ensured that the stacked-on branch was actually visible
<wallyworld_> sort of. it allowed a quick check to be made
<wgrant> Which is why I didn't really want to keep it -- it didn't solve enough of the problem that it was really worth it
<wallyworld_> without needing recursion each time
<wgrant> Right, but you still had to check the stacked-on visibility
<wgrant> It wasn't a sufficient check
<wallyworld_> it was sufficient before we introduced info types
<wgrant> No
<wgrant> I can very well be subscribed to the stacked branch, but not the stacked-on branch
<wgrant> Even back in the olden days
<wallyworld_> i was ignoring subscription/visibility conflation
<wallyworld_> is there a bug for the mp security check?
<wgrant> There's one or two timeout bugs about it
<wallyworld_> so do we think we can solve this without a denormalisation effort?
<wgrant> And the subscription/visibility conflation isn't really relevant here; visibility can differ regardless of the precise mechanism
<wgrant> I don't believe the denormalisation is practical even if we wanted to do it
<wallyworld_> so we need efficient sql then
<wgrant> In either the old or the new model
<wgrant> Right
<wallyworld_> denormalisation worked for private
<wgrant> It didn't.
<wallyworld_> why?
<wgrant> I have this private branch
<wgrant> It's stacked on another private branch
<wgrant> I'm subscribed to my branch, but not the one it's stacked on
<wgrant> transitively_private is true on the private branch, but it doesn't tell the code that I can't see the stacked-on branch
<wallyworld_> it wasn't supposed to
<wgrant> Right, so the situation is not significantly different from today
<wgrant> In the old days you had to check visibility of the stacked-on branches
<wallyworld_> it was just supposed to highlight that public branches stacked on private were private
<wgrant> Now you have to check visibility of the stacked-on branches.
<wgrant> Sure, but that's only vaguely related to this issue
<wallyworld_> right. so the reason it is no longer needed isn't because it didn't work as advertised, but because we no longer allow public stacked on private
<wgrant> The role that transitively_private served was of very limited use, due to exactly this issue
<wgrant> From the day transitively_private was created.
<wallyworld_>  it servered the purpose it was written for - to stop public branches stacked on private advertising themselves as public
<lifeless> yay svn
<wgrant> wallyworld_: Right, but that's not fundamentally hugely useful in itself
<wgrant> lifeless: Oh?
<wgrant> wallyworld_: transitively_private was floated as a partial solution to the issue you're looking at now
<wgrant> But it doesn't work for that at all unless you squint very hard
<wallyworld_> i recall it slightly differently - it was intended to stop people trying to access a so called public branch and being denied access
<wgrant> Right
<wgrant> But that doesn't help much
<wgrant> Since you can still access a so-called private branch to which you have access
<wallyworld_> i think it does, but anyway
<wgrant> But still be denied access
<wallyworld_> sure, different use case
<wgrant> It helps the public case, but not anything else
<wgrant> It's very very similar
<wgrant> I have access to a branch, except I don't.
<wallyworld_> but in the public case, yu expect access
<wgrant> transitively_private solved the public special-case of that
<wallyworld_> since there's no padlock etc
<wgrant> Sure, and in the private case where I have access I expect access.
<wallyworld_> 5 minutes is up, i haven't paid for the 10 minute argument
<lifeless> wgrant: stabbing log rotate, tired of the incident reports
<wgrant> lifeless: Ahh
<lifeless> https://code.launchpad.net/~fdrake/zconfig/trunk is the closest zconfig has to a trunk
<wgrant> lifeless: svn.zope.orgZ?
<wgrant> -Z
<wgrant> Oh, that's an import from there, I see
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/proper-error-on-bug-attachment-bad-filename/+merge/122608
<wallyworld_> StevenK: perhaps include the filename in the exception message?
<StevenK> wallyworld_: I think this particular exception is only triggered via the API, in which case the user is well aware which filename.
<wallyworld_> ok
<StevenK> I'm still not sure about raising BadRequest directly
<wallyworld_> we are losing the root cause info from the original exception
<wallyworld_> so the user doesn't get to know why it's a bad filename
<StevenK> wallyworld_: They didn't before either -- it tripped a constraint in the DB.
<wgrant> Raising a BadRequest in the model sounds like a Bad Ideaâ¢
<StevenK> Add more lines and create class InvalidFilename(Exception) ?
<wallyworld_> sounds more reasonable. but we need to try and propogate the cause of the error
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1184984/
<mwhudson> wgrant: postgres performance question
<mwhudson> i'm finding that
<mwhudson> select * from foo where bar in $SUB_QUERY1 and bar in $SUB_QUERY2;
<mwhudson> is waaay slower than
<mwhudson> select * from foo where bar in ($SUB_QUERY1 intersect $SUB_QUERY2);
<mwhudson> wgrant: have you run into this?
<wgrant> mwhudson: It's probably trying to be too smart in the first one. Do you have plans for both?
<wallyworld_> StevenK: but it doesn't say *why* it's a bad filename - can we extract and use that from the original exception?
<StevenK> wallyworld_: Nope.
<mwhudson> wgrant: yes, but they may cause eyeball stabbing
<wgrant> StevenK, wallyworld_: How does the webapp handle this?
<StevenK> wallyworld_: Because the constraint error doesn't say either.
<wgrant> StevenK, wallyworld_: This is a normal validation problem
<wgrant> I'd suggest looking at how eg. product name validation is done
<StevenK> wgrant: I think well-behaved browsers are immune
<wgrant> Oh right, cause the filename is embedded in the widget, blah
<wgrant> So difficult to use a traditional validator
<StevenK> Right, which is why I was saying it's really only API
<mwhudson> wgrant: slow: http://paste.ubuntu.com/1184985/ faster: http://paste.ubuntu.com/1184987/
<wgrant> mwhudson: EXPLAIN ANALYZE is probably helpful, to see where its estimates go wrong
<mwhudson> wgrant: heh, http://explain.depesz.com/s/5i9 <- fast http://explain.depesz.com/s/xUB <- slow
<wallyworld_> StevenK: the IntegrityError exception does have a useful message doesn't it? so can we not reuse that?
<mwhudson> wgrant: seems that the materialize is the problem?  i'm not super experienced at reading explains
<wgrant> mwhudson: You're missing an index
<wgrant> Not quite sure what your schema is, but I suspect a CREATE INDEX foo ON dashboard_app_namedattribute (content_type_id, name, value); will fix it
<mwhudson> oh right, django
<wgrant> Currently it finds everything with the matching content_type_id
<wgrant> Then filters by name and value
<mwhudson> yeah, that's not going to be helpful
<wgrant> Which seems like the wrong way to do things, but is also trivially indexable
<wgrant> And should take the query into the hundreds of microseconds.
<mwhudson> oh
<mwhudson> is it pulling the common part out of the two subqueries and so doing things in a boneheaded order?
<wgrant> Oh, except for the testrun join, but still, tens of milliseconds.
<wgrant> I don't know what the query looks like
<wgrant> But no, it seems to be doing them separately
<wgrant> It's just really slow to filter dashboard_app_namedattribute how you've asked
<mwhudson> django generic relations, yay
<wgrant> :)
<mwhudson> oh well, i've wasted enough of your time, thanks
<wgrant> Still, create that index and see what happens
<wgrant> Should work
<wgrant> No idea how you create custom indices in Django, though...
<wgrant> yay django
<wgrant> I love django, as you can tell
<mwhudson> well
<mwhudson> i'm not even slightly pretending to be portable in this project
<wgrant> Good :)
<mwhudson> so i can generate them with sql/$model.sql i guess
<wgrant> Do you understand how to read the relevant bit of the explain?
<mwhudson> the index doesn't help though :(
<wgrant> :(
<wgrant> Does it change the plan at all?
<mwhudson> CREATE INDEX
<mwhudson> Time: 36607.299 ms
<mwhudson> wee
<mwhudson> the index is used by the plan now
<mwhudson> i think, as i could probably have predicted, the problem is that our schema is a bit nuts
<mwhudson>                                              ->  Index Scan using foo on dashboard_app_namedattribute u1  (cost=0.00..1773.90 rows=527 width=4) (actual time=0.043..30.131 rows=28501 loops=1)
<mwhudson>                                                    Index Cond: ((content_type_id = 23) AND (name = 'target.device_type'::text) AND (value = 'panda'::text))
<wgrant> Django has that effect on people
<mwhudson> 30000 index scans ?
<mwhudson> not going to be good, i suspect
<wgrant> And that's going straight into the testrun index scan, I guess, yeah :/
<wgrant> Not ideal
<mwhudson> http://explain.depesz.com/s/xkDK <- plan with index
<wgrant> What's the actual query?
<mwhudson> ah sorry
<mwhudson> wgrant: http://paste.ubuntu.com/1185000/
<wgrant> my eyes
<mwhudson> brutally word-wrapped to avoid the one long line effect
<mwhudson> hee hee
<wgrant> Mmm
<wgrant> That's ugly
<mwhudson> yeah
<wgrant> But nothing blatantly obvious
 * wgrant runs off to lunch, will look more later
<mwhudson> i think the problem is that the two queries on attributes vary massively in selectivity
<mwhudson> there are a _lot_ of target.device_type = panda attributes
 * mwhudson looks at the intersect plan some more
<mwhudson> heh the index helps the fast plan a _lot_
<mwhudson> down to 100ms now
<StevenK> mwhudson: Have you seen http://sqlformat.appspot.com/ ?
<mwhudson> StevenK: no!
<mwhudson> i actually went looking for one of them a few days ago
<mwhudson> StevenK: it doesn't do a very good job on this query :)
<StevenK> Hahaha
<StevenK> Yeah, it's formatting rules are a little strange
<mwhudson> for which i think we can just blame django
<mwhudson> heh heh
<mwhudson> optimization is fun: i now have a version of this query that runs in 22ms
<mwhudson> about 200x faster than where i started
<StevenK> \o/
<StevenK> wallyworld_: So http://pastebin.ubuntu.com/1184984/ is still bad?
<wallyworld_> StevenK: i think you missed my last comment - can we extract the root cause message from the IntegrityError and include it with the new exception?
<wallyworld_> otherwise the use has no idea *why* the filename is invalid
<wallyworld_> so they don't know what to fix or correct
<StevenK> wallyworld_: I did answer -- no, because the constraint error doesn't say either.
<wallyworld_> you sure? eg from hwdb.txt "IntegrityError: ... violates unique constraint "hwsystemfingerprint..."
<wallyworld_> there's info in there somewhere
<StevenK> IntegrityError: new row for relation "libraryfilealias" violates check constraint "valid_filename"
<StevenK> And it's an OOPS, so the user gets a 500.
<wallyworld_> :-(
<StevenK> Seriously, *anything* we do is going to be better.
<wallyworld_> ok. we do seem to have an inherent layering issue though
<StevenK> wallyworld_: Mmmm, we could move the exception handling into ILibraryFileAliasSet.create
<wallyworld_> that would be a bit better
<StevenK> But still makes you unhappy?
<wallyworld_> there's still an issue extracting the reason for the error and presenting it to the user
<wallyworld_> but moving the exception handling to the correct place helps
<StevenK> "valid_filename" CHECK (filename !~~ '%/%'::text)
<StevenK> Ah ha
<StevenK> raise InvalidFilename("Filenames can not contain slashes.")
<wallyworld_> yay
<StevenK> wallyworld_: Let me move it into LFA and test, as well as change the message. Hopefully that will fix your little red wagon. :-)
<wallyworld_> ok, thanks.i'm just thinking of the users :-)
<StevenK> Suuuuuuure
<wallyworld_> it would be nice to be able to look at the constraint name that was violated and re-raise with a sensible message. is there more than one i wonder. or is there only the slashes one
<StevenK> That's the only CHECK CONSTRAINT that libraryfilealias has.
<mwhudson> wgrant: for when you get back, it turns out that punching through the genericrelation abstract gets me a query that runs in 22ms :)
<mwhudson> *abstraction
<mwhudson> (with the index, 200ms without)
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1185054/
<wallyworld_> StevenK: comment on line 48 needs fixing. are there more test changes? i think it looks ok though. the users thank you
<StevenK> wallyworld_: It does? But it is a BadRequest?
<StevenK> The API will never see the internal exception, but it raises BadRequest
<wallyworld_> oh, yes, sorry. misread
<StevenK> wallyworld_: So, push it up and prod you somewhere inappropriate when the diff updates?
<wallyworld_> why not. i'm waiting with anticipation
 * wallyworld_ grips the desk
<wgrant> And he drops out just as I return
<wgrant> StevenK: Want to extend that ndt by a rev?
<StevenK> wgrant: To r15901? Sure.
<wgrant> mwhudson: How'd you avoid that expensive join?
<wgrant> I guess eliminating the duplicate dashboard_app_testrun join might have helped
<StevenK> wgrant: https://oops.canonical.com/?oopsid=475a419919a34a618febf7b2ba9573e0
<StevenK> wgrant: Why are their 5 repeated Archive lookups? :-(
<lifeless> StevenK: 'there'
<StevenK> lifeless: And where is your Grammar Nazi cape?
<lifeless> in the kitchen
<StevenK> Maybe I should have used "you're" just to make you twitch.
 * wgrant twitches
<wgrant> StevenK: It's probably trying to find archives that are buildable into, maybe.
<wgrant> Checking the tracebacks will tell you for sure
<StevenK> wgrant: The tracebacks for the UNION'd queries are quite unreadable
<wgrant> Let me prove you wrong :)
<wgrant> By reading them
<StevenK> Oh wait, there it is
<StevenK> The stack is like 100 deep and the font is tiny :-(
<wgrant>   File "/srv/launchpad.net/production/launchpad-rev-15890/lib/lp/code/model/sourcepackagerecipe.py", line 89, in get_buildable_distroseries_set
<wgrant>     supported_distros = set([ppa.distribution for ppa in ppas])
<wgrant> lol
<StevenK> That has to win some kind of award
<StevenK> wgrant: Tempted to just have that use ILaunchpadCelebrities.ubuntu rather than that insane query.
<wgrant> :(
<lifeless> LOLWHAT
<StevenK> Oh, SERIOUSLY
<StevenK>                     |     # Now add in Ubuntu.
<StevenK>                     |     supported_distros.add(getUtility(ILaunchpadCelebrities).ubuntu)
<wgrant> heh
<StevenK> (Guess what I'm reading ... :-)
<wgrant> heh heh
<lifeless> https://code.launchpad.net/~lifeless/zconfig/bug-481512/+merge/122612 - fingers crossed.
<StevenK> lifeless: WTB CMPXCHG for Python, PST ?
<lifeless> yup
<adeuring> good morning
<jelmer> moin
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: â
<jelmer> mgz: there?
<mgz> where?
<jelmer> mgz: le mumbles
<mgz> yeah.
<ev> is there an estimate on staging coming back from the dead?
<wgrant> ev: No, could easily be another day or so. Can you use qastaging instead?
<ev> wgrant: absolutely can - didn't realize it existed
<ev> thanks a bunch!
<wgrant> It's been around for nearly two years now, I believe
<wgrant> The DB is reset less frequently and the code updated slightly more quickly
<ev> neat
<mwhudson> wgrant: it was eliminating the joins in the subqueries that flipped the performance to good
<wgrant> mwhudson: Ah, good
<jam> jelmer: python-all-dbg: Depends: python-all (= 2.6.6-3+squeeze7~lp1) but it is not going to be installed                   Depends: python2.5-dbg (>= 2.5.5-6) but it is not installable
<jam> it looks like your squeeze package wants to keep python-2.5
<jam> vs 2.7?
<czajkowski> mgz: jelmer jam does this make sense to any of ye ? https://bugs.launchpad.net/lpsetup/+bug/1045728
<_mup_> Bug #1045728: lxcip.py looks for liblxc.so.0 in the wrong directory on 12.10 <lpsetup:New> < https://launchpad.net/bugs/1045728 >
<jam> czajkowski: looks to be a yellow squad thing (since they introduced lpsetup).
<jam> I understand what the bug is reporting (a library is looked for in .../lxc/lxc.so, but it would be found as just .../lxc.so)
<jam> I don't know what a fix would be.
<wgrant> Right, it moved in quantal
<czajkowski> jam: ahh ok
<czajkowski> are yellow off that area now though ?
<jam> I think so. I seem to remember a "we need maintenance to take over our work" email.
<czajkowski> yup
<czajkowski> saw that
<czajkowski> jam: so it's yers now :)
<jam> unless I stick my head in the sand for wgrant next week :)
 * wgrant points at the SEP field
<jam> SEP ?
<wgrant> Somebody Else's Problem :)
<wgrant> It's not my problem (yet?), so I don't acknowledge its existence.
<jelmer> wgrant: as if you are ever reluctant about acknowledging problems.. :P
<wgrant> Heh.
<czajkowski> jelmer: few of your questions on LP answers need follow up, do you wantme to assign them to you so you'll get the mails ?
<jelmer> czajkowski: that'd be great, thanks
<czajkowski> blues, I've created a ticket to track the bugzilla imports not working issue like mrevell suggested last week, this will be handy come handing off time
<czajkowski> https://support.one.ubuntu.com//Ticket/Display.html?id=21832
<czajkowski> as we're trying this out for a month be nice to have some feedback on this new way of tracking items raised.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban, rick_h | Firefighting: - | Critical bugs: â
<jam> jelmer, mgz: I think gary_poster has officially handed lpsetup over to Maintenance, so can one of you look at: https://code.launchpad.net/~jameinel/lpsetup/missing_lxc_1045728/+merge/122663
<jelmer> mgz: Can you look at that one perhaps? I haven't dealt with lpsetup yet
<gary_poster> no-one has except yellow, jelmer :-) we can do a review-of-the-review if you like?
<jam> gary_poster: it is a pretty small change, (just probe in 2 locations for lxc.so)
<jam> feedback on how I refactored for testability would be nice
<gary_poster> looks nice on the face of it jam.  I'll look more carefully--or get someone else to--but why don't we try to get someone else looking at the code first so we can start the acclimatization process, and then we'll come around and confirm
<jam> gary_poster: fine with me
<gary_poster> please ask reviewer to ping me (or someone else on yellow) to review the review
<mgz> so, I can confirm that you know how to write code jam, no idea if it's the right code.
<gary_poster> lol
<gary_poster> oh come on, lxcip is itsy bitsy and self contained :-)
<jam> gary_poster: is there a tarmac or something protecting the trunk?
<jam> or are you just expected to run pre-commit.sh yourself?
<gary_poster> yes jam
<gary_poster> jam, tarmac.  pre-commit.sh if you want to be super careful. It takes awhile though
<mgz> well, I don't understand why _load_library exists, and ctypes.find_library doesn't work
<mgz> but I presume there's a reason
<jam> mgz: does find_library know how to handle the multiarch stuff?
<mgz> I'd expect debian to patch the logic they need for multiarch, but perhaps they've not.
<gary_poster> I don't think it does. frankban, question about lpsetup from mgz ^^
<mgz> ctypes hasn't been well maintained of later
<mgz> -r
<mgz> and it's always been on the nastier side of neat hax
<mgz> there's a comment in my ctypes.util about the logic being wrong for biarch and multiarch
<gary_poster> jam, frankban reports that our pre-commit.sh only tests lpsetup, not lxcip.  An oversight.  In order to make the tests work on the machine we need to make some sudoers changes on that machine, since the tests need to run as root because lxc-ip needs to run as root
<gary_poster> I'll file a bug jam, and ask bac to comment
<gary_poster> (since he knows the most about the tarmac setup)
<jam> gary_poster: thanks. Looking at the code, it would appear to just run 'nosetests' which should run all of them, but maybe they fail when not run as root.
<gary_poster> right, I believe that's the case
<gary_poster> well...
<gary_poster> if they fail then tarmac won't work...
<gary_poster> and it does
<mgz> <http://bugs.python.org/issue13508> filed by lool is of some interest
<gary_poster> agreed
<gary_poster> bac, do we run nosetests as root on lpsetup tarmac?
<frankban> mgz: that's exactly the problem we had with multiarch. and about the tests: nosetests run in the base directory will only run the tests for lpsetup. `cd lplxcip && sudo nosetests` must be used to run lxcip tests. This is because 1) lxcip requires root 2) test_lxcip creates an LXC container, used to exercise the main functions
<czajkowski> a
 * bac looks
<gary_poster> bac, frankban, jam, ok cool.  I filed https://bugs.launchpad.net/lpsetup/+bug/1045807 .  bac, could you link the tarmac config info so that people can know where to look and what to do?  Or is this something only yellow squad can do (in which case I should probably file another bug :-P ) ?
<_mup_> Bug #1045807: tarmac should run lplxcip tests <lpsetup:Triaged> < https://launchpad.net/bugs/1045807 >
<bac> gary_poster: pre-commit.sh is run by the tarmac user.  however that user has sudo privileges so tests for lp-lxc-ip could be run as root.
<gary_poster> cool bac
<gary_poster> so it could be a trivial fix
<bac> yes, just mod pre-commit.sh
<gary_poster> bac, do you want me to add that to the bug (happy to) or are you?
<gary_poster> IOW, are you doing it now?  If not, I will :-)
<bac> mgz, jam: if you're interested the tarmac+lp2kanban canonistack branch is at https://bazaar.launchpad.net/%7Eyellow/lp-tarmac-configs/tarmac-puppet/files
<bac> gary_poster: please do it
<gary_poster> bac, on it.  thx
<bac> gary_poster: i turned off lp2kanban for the yellow board.  let me know if you ever want it fired up again.
<gary_poster> ack thanks bac
<bac> gary_poster: we should probably add the maintenance squad to the list of people who can access the canonistack instance.  unfortunately it is currently a static list of people, not teams.
<gary_poster> bac, makes sense. Ideally we would add all of LP
<gary_poster> bac, can you do that as a misc thing today, or should I file a bug for "someday"?
<bac> gary_poster: it shouldn't be hard to write a lplib script that can take a list of teams and do it.  i'll add a misc card for it.
<gary_poster> thanks bac. slack might be more apt.  dunno
<sinzui> jcsackett: am I really here? My irc is borked after quantal upgrade
<sinzui> well the fonts are extra ugly too
<rick_h_> sinzui: howdy
<czajkowski> sinzui: welcome
<sinzui> So freenode is indeed on despite the off switch
<czajkowski> sinzui: how were the holidays and were you sick ?
<sinzui> um, Staying in a beach house with extended family was awkward. That is the most polite way to describe the on going civil wars in house. There were only a few places to hide
<mgz> see, you make that sound fun.
<sinzui> I travelled to a Renaissance Festival yesterday. That felt like a holiday.
<rick_h_> my wife was just saying that she wants to go to that when it's in town.
<rick_h_> now I've got to go hiding
<sinzui> I go several times a year for the bad food, shlocky celtic music, and fashions (and fashion tragedies)
<rick_h_> hah, now I'll have to rethink my lack of desire to go. Your 'bad food' really sold me
<sinzui> This years secret theme is Doctor Who. I have never seen so many Whovians in one place before.
<sinzui> rick_h_ steak-on-a-stick is just the start. It eventually leads to macaroni-and-cheese-on-a-stick
<sinzui> I am not kidding either
<rick_h_> lol, I went one year and just don't have the desire to go back, but the wife wants to take the 2yr old and 'experience' it
<rick_h_> yea, the giant muttons walking around with people snacking on them really set the scene
<cjwatson> An ex of mine does full-on week-long Tudor immersion stuff
<sinzui> My children demand going. Anne made the costumes toos
<cjwatson> Scary amount of work
<sinzui> I dress as myself because I look the same in all eras
<rick_h_> cjwatson: oh hey, you're around. I'm looking at this MP
<rick_h_> cjwatson: what do you think of tring to celery-ize this? It seems like a perfect fit though you'd be the first thing full time on celery
<cjwatson> Heh, that was sort of why I wasn't :-)
<rick_h_> cjwatson: in looking at the irc logs you link it seems wgrant and StevenK were going to check some perf stuff, did that all work out?
<rick_h_> cjwatson: oh come on :P be brave and daring. It's a new job that only needs to happen in a specific use case, celery/queue ftw
<rick_h_> a job per bug found would rock
<cjwatson> I tried to include the necessary bits for celerification but I don't really know what I'm doing.  Would it just be an extra feature flag?
<rick_h_> abentley: would probably be able to help tell you that. I think they're pretty close
 * sinzui restarts to see if he can be on more than one irc server at a time.
<cjwatson> The perf work is basically not likely to get to the point of being good enough to avoid the need for this job
<cjwatson> So it's IMO orthogonal
<cjwatson> After this branch it would just have the effect of making the jobs run faster
<rick_h_> cjwatson: ok, but there were no performance reasons against this job?
<cjwatson> Not that I know of
<rick_h_> cjwatson: gotcha
<cjwatson> So, I'm up for giving celery a try if it's at all likely to be workable.  How about I stick the job creation behind a feature flag and then we can try it on dogfood?
<cjwatson> Not that I know whether it has celery runners
<abentley> cjwatson: You need to run celeryRunOnCommit.
<rick_h_>  abentley can you peek at https://code.launchpad.net/~cjwatson/launchpad/process-accepted-bugs-job/+merge/122420 and advise on testing it?
<abentley> cjwatson: The feature flag is already there.
<cjwatson> abentley: Right, I have celeryRunOnCommit.
<abentley> cjwatson: jobs.celery.enabled_classes
<cjwatson> abentley: I mean a different feature flag specific to this job, so that if it's off the code can adopt the previous behaviour of closing the bugs directly
<cjwatson> So that we could land this and test somewhere without fear of breaking existing behaviour yet
<abentley> cjwatson: Well, in theory you could reuse jobs.celery.enabled_classes for that...
<cjwatson> I guess
<abentley> cjwatson: I thought you already had a cron script.
<cjwatson> You mean in stable or in my branch?
<abentley> cjwatson: in your branch.
<cjwatson> Not a specific one; it would involve cronscripts/process-job-source.py.
<cjwatson> That will work, although it won't be as responsive as celery presumably could be.
<cjwatson> I don't think responsiveness is hugely critical, but I'm wary because I'm going from synchronous behaviour to asynchronous so I don't want to create a significant perceived slowdown.
<abentley> cjwatson: So I think the decision about whether to test the job under a feature flag is orthagonal to celery, then.
<cjwatson> That's probably a fair point.
<abentley> rick_h_: I don't know a lot about what's running or not running on dogfood.  You'd need to ensure a celeryd is running with the 'job' queue (and maybe the 'job_slow' queue).  And then you'd do an upload.
<cjwatson> There's no celeryd running right now.
<cjwatson> Where's this done on production?  I don't see it in lp-production-crontabs - is there an init script?
<abentley> cjwatson: It runs on ackee.  I don't know the details of the deployment.  I think it's puppetized.
<rick_h_> abentley: ok yuck on the extra work then.
<cjwatson> That might not be too horrible; I'll need to talk to ops I suppose.
<rick_h_> cjwatson: ok, well I guess it's up to you. I don't want to hold up your stuff, but it seems like a good candidate for the celery work.
<cjwatson> I don't mind trying, and it isn't OMG urgent.  It might produce a better result all round.
<cjwatson> Or it might involve mountains of extra debugging.  BUT THAT'S THE FUN
<rick_h_> cjwatson: wooo more fun for you
<rick_h_> abentley: does this ring any bell while tring to run tests? ImportError: cannot import name BzrDirMetaFormat1Colo
<rick_h_> abentley: full trace: http://paste.mitechie.com/show/771/
<abentley> rick_h_: No.  It sounds like it's running against an old version of bzrlib.
<rick_h_> abentley: ok thanks...looks like something broke on me from thurs. Will peek further
<czajkowski> bac: bug 1045774 want me to set the importance on it ?
<_mup_> Bug #1045774: Attempting to authorize from cron script leads to repeated reauth attempts <launchpadlib :Triaged> < https://launchpad.net/bugs/1045774 >
<bac> czajkowski: yes please
<czajkowski> low or high?
<mgz> 5am is pretty low :)
<mgz> I'd be tempted to revert r93 on launchpadlib
<mgz> it's a pain to deal with regardless
<mgz> needing to hit enter isn't that much more pain.
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: â
<cody-somerville> On the sharing pages, does anybody else notice that 'First' and 'Previous' never become links? (though clicking the words still works as expected).
<al-maisan> q
<rick_h_> deryck: is alive!
<deryck> rick_h_, indeed! :)
<deryck> actually deryck's internet lives!
<deryck> hi abentley, too :)
<abentley> deryck: Hi!
<deryck> I'll do the webcam home tour at tomorrow's stand-up.
<rick_h_> hah
<rick_h_> how many sq ft was that going to be? /me blocks out the mansion tour on the calendar :P
<deryck> heh
<deryck> yeah, it could take awhile :)
 * sinzui rescues children from state institution
<abentley> rick_h_: Merged devel and got BzrDirMetaFormat1Colo
<rick_h_> abentley: yea, I got it working with an apt-get update, source deps update, make clean and rebuild
<rick_h_> abentley: not sure which one is the magic sauce tbh, the last thing I tried was make clean_buildout && amke clean so not sure if the other steps were required/not
<abentley> rick_h_: Yikes.  Given that bzr is an egg, make should be all that's needed.
<rick_h_> abentley: might have been, when you said bzrlib was out of date I went hitting the system first
<rick_h_> so I probably went all the way around the thing
<abentley> rick_h_: Sorry if I mislead you, but Launchpad doesn't use the system bzrlib.
<rick_h_> well, I did a bin/py and import bzrlib; help(bzrlib) and found it was 0.0.1 behind my system so thought my lxc needed to be updated
<rick_h_> anyway, all better, thanks
<abentley> rick_h_: plain "make" solved it for me.  (Then I had to do update-sourcecode to handle bzr-loom imports.)
<rick_h_> abentley: cool, I'll rework my depth chart of "crap's broke...try these commands"
<abentley> rick_h_: yeah, "make" should be at the top of your list.  That's why I patched the 2.7 JS stuff-- because I use make a lot.
<cody-somerville> sinzui, Does the sharingdetailsview use the API to get the information it shows or just to revoke grants to individual artifacts?
<sinzui> cody-somerville, 100% api
<cody-somerville> sinzui, I can't figure out how to get the data on the individual artifacts.
<sinzui> ah
<sinzui> that bloody cache again
 * sinzui looks
<cody-somerville> sinzui, getPillarGranteeData doesn't return details on the individual assets granted - just if there is any for the different information types
<sinzui> I think the cache is being loaded. then maintained by the script. I recall there is a reload to get data. I am looking at how the page first gets the data. I suspect we may not be using API if I do not see a reload to sync changes
<sinzui> but it is supposed to be 100% api
<sinzui> cody-somerville, getSharedArtifacts() is not exported. I will report the bug now
<cr3> I just noticed that blueprints now have a work items section in addition to the whiteboard. Should work items be defined like before, ie: Lorem ipsum dolor sit amet: TODO
<sinzui> cody-somerville, https://bugs.launchpad.net/launchpad/+bug/1046022 should be fixed in a few days
<_mup_> Bug #1046022: Cannot get the artifacts shared with a person over the API <api> <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1046022 >
<rick_h_> cr3: I believe so. If I recall it has a format validation that runs in case it's slightly off
<czajkowski> cr3: that was there at last uds http://blog.launchpad.net/cool-new-stuff/work-items-in-blueprints
<rick_h_> czajkowski: ftw!
<cr3> rick_h_: sweet, thanks!
<czajkowski> new features go on the blog
<czajkowski> always check there :)
<abentley> rick_h_: could you please review https://code.launchpad.net/~abentley/launchpad/blueprint-info-type-change/+merge/122753 ?
<rick_h_> abentley: rgr
<abentley> rick_h_: thanks.
<rick_h_> abentley: r=me
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<abentley> rick_h_: thanks.  Have a good one!
<cr3> czajkowski: weird, when I save my work items, it adds the string "Work items:" at the top of my work items, which looks redundant considering the label of the text area is also called "Work items" :)
<cr3> ... and then I get an email that says: work items set to: work items: ..
<sinzui> cr3, work items are not complete and have no commitment to ever be completed.
<cr3> sinzui: ah, that has happened to be too: working on a feature that almost got done but not quite :(
<sinzui> cr3 czajkowski: the feature was built by linaro to their needs and only they can use it fully. When the registry.upcoming_work_view.enabled is set to default or the guards removed from the code, everyone can use the full feature
<cr3> sinzui: I'm sure contribute are quite welcome :)
<sinzui> cr3, they are. At this time it is a maintenance burden since removing the flags break Lp :(
<cody-somerville> sinzui, I notice that the API still returns 'USERDATA' for 'private'. Is that a bug/going to be changed to reflect what's in the UI?
<sinzui> cody-somerville, that is intentional. "Private" is for the punters. USERDATA is for people who know better
<cody-somerville> sinzui, It's likely that the lp api will be used by the punters too ;)
<cody-somerville> sinzui, Since 'private' is really 'USERDATA', should I migrate all our bugs to proprietary?
<sinzui> cody-somerville, you can try. Bugs that are shared with another project will raise and exception
<sinzui> I think true private projects will require the bugs to be proprietary, but that is two or more months away.
<rick_h_> no, we're currently looking at providing a 3 way option of public, proprietary, and embargoed
<sinzui> how can a private project have public bugs?
<sinzui> Good luck getting users to see the bugs when they cannot traverse to the project
<rick_h_> sorry, didn't mean that. Meant the project gets those three choices
<sinzui> yes. I agree
<rick_h_> so then the bugs have to be one of the 'private' types
<rick_h_> but won't require proprietary, embargoed will also be an option from what I understand
<sinzui> rick_h_ private/USERDATA can be shared with other project, which leaks. Leak a lot in fact. We agreed not to support it. So the sharing policies will must be Proprietary only. Embargoed is not support by bugs but could be. any other choosen sharing policy will lead to constraint violations because the default cannot be enforced
<rick_h_> sinzui: ok, I think our goal is to support embargoed in bugs, but as you said we've just started to maybe that doesn't work out
<sinzui> rick_h_I think we should. They asymmetry will probably cause problems
<sinzui> eg, can a proprietary bug be linked to an embargoed branch? What happens if the user cannot see both?
<sinzui> It probably causes the same problems as public branches stacked on private branches.
<rick_h_> sinzui: right, I assume we'll have some restrictions to manage that for sure
<cody-somerville> I was told that the ability to affect /private/ bugs on a private project against public projects would be retained - via using the 'private' type for some bugs vs. 'proprietary', ie. you have to make a conscious decision to lower the privacy type.
<sinzui> cody-somerville, yes, for pubic projects with private defaults.
<sinzui> cody-somerville, the bug-linking feature was (and honestly alway will be) a requirement to address the needs of true private projects looking at bugs in many projects
<cody-somerville> but the bug linking feature isn't going to be implemented - correct?
<sinzui> cody-somerville, The other choice is to cut scope, accept the risk, and allow projects to share with non-canonical projects
<cody-somerville> sinzui, I'll need to check with the folks who make heavy use of the bug linking. It might be acceptable that there projects have private bugs by default but not be an entirely private project. If not, that's indeed a discussion we'll have to have.
<cody-somerville> *their
<sinzui> It is a valid decision, though you need to be really sure the other project knows what you are doing to honour your intentions. And hope that the project does not send bugs to upstream bug trackers or mailing lists
<cody-somerville> lifeless, would there be an easy way to add support to lazr.restful for prefetching certain relationships - sorta like how django lets you?
<lifeless> cody-somerville: possibly
<lifeless> cody-somerville: but I don't know what it is
<cody-somerville> lifeless, basically being able to say when I make the request, for example, a project that I can specify that I want it to also return in the same response the entity linked to via the maintainer field instead of just a link
<cody-somerville> same rationale as making one db request to get the info I know I'll need instead of a bunch of small ones
<lifeless> yes, I know what you mean
<lifeless> there was a plan put together by leonard + gary + benji + others
<cody-somerville> lifeless, the getPillarGranteeData method on the sharing_service returns a json response with the bits of info I need on both the grants and the user so my script to inspect sharing information on a project is super fast compared to if I had to make an api request for each user
<lifeless> yup
<lifeless> thats why that approach has been adopted
<sinzui> wallyworld_, https://launchpad.net/~launchpad/+related-software <- makes little sense
<wgrant> cjwatson: It's a daemon
<wgrant> blah
<wgrant> I was scrolled back a few hours
<wgrant> nevermind
<sinzui> wallyworld_, https://bugs.launchpad.net/launchpad/+bugs/?field.tag=related-projects-packages <- the madness
#launchpad-dev 2012-09-05
<lifeless> would someone like to review my zconfig patch, as the maintainer seems to not be doing so :(
<wgrant> lifeless: Hm?
<wgrant> lifeless: You proposed it into an import
<wgrant> I'm not surprised nobody reviewed it
<lifeless> abel did that previously; I commented on the bug to grab freds attention too :)
<StevenK> I guess foo = [bar if bar.active and bar.baz in supported for bar in all_bars] is malformed?
<lifeless> it may want brackets
<lifeless> because you have two ins
<StevenK> if (bar.active and bar.baz in supported) also gives syntax error
<lifeless> oh
<lifeless> lol
<lifeless> the if goes at the end
<lifeless> [bar for bar in all_bars if bar.active and bar.baz in supported]
<StevenK> lifeless: Ah! Indeed, thanks
<StevenK> Bleh, Launchpad.login_with('foo', 'dogfood') doesn't work :-(
<StevenK> wallyworld_: O hai -- do you know much about StormStatementRecorder? I'm invalidating my store, but still can't get it to actually make a query
<wallyworld_> you mean record a query?
<wallyworld_> it's normally used after flushing the store, eg
<wallyworld_>             Store.of(bug).flush()
<wallyworld_>             with StormStatementRecorder() as recorder:
<wallyworld_>                 previous_dup.markAsDuplicate(bug)
<wallyworld_>                 self.assertThat(recorder, HasQueryCount(LessThan(95)))
<StevenK> I'm suspecting that it isn't working for the webservice
<wallyworld_> i wonder why if that's the case. i can't see why the webservice would not play nicely with it
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1186788/
<wallyworld_> hmmm. how come you can't test this using the recipes() method directly as part of a unit test?
<StevenK> wallyworld_: If I use owner.recipes[0], I get one query. I'm wanting to use the API because lazr marshalls the vocab that the recipe includes and that involves at least two queries, per recipe.
<wallyworld_> the vocab marshalling is a constant count though right? and outside our control? so do we need to measure it for these tests?
<StevenK> So I've rewritten the vocab, so it shouldn't make any queries, which is exactly what I'm trying to test.
<wallyworld_> can you test that as a unit test on the vocab?
<StevenK> Personally, I'd like to see it on the API
<wallyworld_> i'm not sure why its not working sadly
<wallyworld_> i'd have to fire up the debugger and see what's going on
<StevenK> wallyworld_: Could I bug you to do so while I do it as well and we'll meet in the middle?
<wallyworld_> ok
<wgrant> StevenK: launchpadlib_for requires AppServerLayer, doesn't it?
<wgrant> Which means it connects to a real appserver
<wgrant> So a statement recorder in the test process isn't going to do you much good
<wgrant> Also, don't use launchpadlib in tests
<wgrant> It's even slower than Launchpad
<StevenK> wgrant: Then how I can test the marshalling?
<StevenK> Or is the easiest answer "Just don't" ?
<wgrant> StevenK: webservice_for_person
<wgrant> Gives you a rawer interface
<wgrant> Without all the wadllib etc. overhead
<wgrant> Just dealing with JSON
<wallyworld_> StevenK: QueryCollector is think is what you want to use
<wallyworld_> it seems to be the way to collect sql statements for ws interactions?
<wgrant> QueryCollector uses the stuff in the request, I think, but it probably still only works if the request is in-process
<wgrant> SSR should work fine with webservice_for
<StevenK> Now I'm stuck on how to get at the recipes
<wgrant> Oh?
<StevenK> As changing the ws_owner.recipes[0] call
<wgrant> I'd get the recipe from the DB and call canonical_url on it
<StevenK> wgrant: Hah, except that canonical_url includes code.launchpad.dev, and webservice_for_person wants no part of that.
 * StevenK sprinkles in force_local_path
<StevenK> Right, that test in devel is 34 queries. With my rewritten vocab, 28.
<wgrant> :)
<wgrant> How'd you do it?
<wgrant> No longer a SimpleVocabulary?
<StevenK> Yeah, it's now a IHugeVocabulary
<StevenK> wgrant: Do you want to see a diff or shall I just put it up for review?
<wgrant> StevenK: Review!
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/sane-buildable-distroseries/+merge/122791
<wgrant> wallyworld_: Your stuff made it through buildbot finally
<wallyworld_> yes indeed
<wallyworld_> it's even ready for qa
<wgrant> StevenK: You have a bit of QA too
<StevenK> wgrant: Done.
<StevenK> Do we want another deployment?
<wgrant> We've not ndted in more than 24 hours!
<wgrant> So that's not a question
<jam> wgrant: you sound like a junky trying to get his fix. "Maaannn, I haven't had an NDT in 24-hours maaannn. i gotta score some NDT"
<wgrant> Heh
<adeuring> good morning
<nigelb> bigjools: heh, we got a pycharm license at work since all our work is open source :)
<bigjools> :)
<ev> just to confirm, there's no way to go from a binary package to a source package in the API, right? https://bugs.launchpad.net/ubuntu/+source/python-launchpadlib/+bug/597041 leads me to believe this.
<_mup_> Bug #597041: No way to get from binary package to source package <amd64> <api> <apport-bug> <lp-soyuz> <maverick> <ppa> <Launchpad itself:Triaged> <python-launchpadlib (Ubuntu):Invalid> < https://launchpad.net/bugs/597041 >
<wgrant> ev: There are no binary or source *packages* exposed in the API, just publications of those packages  -- do you mean binary_package_publishing_history and source_package_publishing_history?
<wgrant> If so, that's correct, there's no present way to do it
<ev> yes, that
<ev> hmm okay
<ev> thanks
<cjwatson> Indeed.  In auto-sync I just worked around it by reading current Sources files.
<cjwatson> Which isn't great but in context it's workable enough.
<ev> hm, we actually send the source package up in the reports that apport generates
<ev> so I'll just grab the first instance from the db for now
<stub> No handlers for logger bzr. Does this mean I need to setup that VM now?
<stub> TacException: Error running ['/usr/bin/python2.7', '-Wignore::DeprecationWarning', '/home/stub/lp/replication/bin/twistd', '-o', '-y', '/home/stub/lp/replication/daemons/librarian.tac', '--pidfile', '/tmp/tmpeX98el/librarian.pid', '--logfile', '/tmp/tmpeX98el/librarian.log']: unclean stdout/err: No handlers could be found for logger "bzr"
<stub> lp_sitecustomize should be fixing that :-/
<mgz> stub: means you need to do whichever update launchpad deps you need today
<stub> I'll try that again, but I think I'm fully up to date.
<mgz> but yeah, if the branch includes my lp_sitecustomise change, I'm suprised there's still a test failure
<stub> Not my branch, just looking at the code that is supposed to make this problem go away.
<mgz> what's the last rev in the branch with the test failure?
<stub> And now I can't even build mailman... grrr...
<wgrant> stub: make clean_buildout compile
<stub> Yeah, just let me remove the pdb.debug() from lp_sitecustomize.py
<stub> hit it harder, it is ok now.
<bac> hi jam and mgz, since you've taken an interest in the canonistack instance we're running, i wonder if you'd like to review a branch i just did that opens up management of it to ~launchpad?  it's at https://code.launchpad.net/~bac/lp-tarmac-configs/add-launchpad-members/+merge/122877
 * mgz has a look
<mgz> Unauthorized: (<Branch u'~bac/lp-tarmac-configs/add-launchpad-members' (651325)>, 'landing_targets', 'launchpad.View')
<mgz> bac: maybe you could also open up the branch to launchpad members? :)
<bac> mgz: i just tried and thought i had
<bac> let me lok
<bac> look
<bac> mgz: try now
<mgz> 200
<mgz> is get_lp_people correct? only deals with a limited level of team nesting
<mgz> I guess we're not that deeply nested in launchpad anyway...
<bac> mgz: team.participants does multiple levels, i believe
<mgz> ah, it's a flattened list of everything?
<bac> mgz, yes, as compared to .members
<abentley> Anyone else getting "AssertionError: newInteraction called while another interaction is active." on ec2 runs?
<abentley> deryck: Any idea about this? ^^^
<deryck> abentley, isn't that from when you're using one of the page test browsers and don't logout before starting a new operation?
<abentley> deryck: Okay, I'll see if that could be it.
<abentley> adeuring: reconcile_access_for_artifact is not working for me.  Has all the necessary code been merged?  http://pastebin.ubuntu.com/1187456/
<adeuring> abentley: no, it's only in my branch....
<abentley> adeuring: Okay, I need a working IAccessArtifactSource.ensure([blueprint]), so I'll wait on your work for the grants.
<adeuring> ok
<abentley> adeuring: I'll focus on the subscriptions.
<adeuring> abentley: but the fix for the the error is really simple
<adeuring> abentley: one more "elif ISpeicification.porvidedBy(...)" in _constraintForConcrete()
<abentley> adeuring: Okay.
<abentley> sinzui: I'm working on Specification.transitionToInformationType, and looking at the bug version, but having trouble understanding who should be automatically subscribed to a Specification.
<sinzui> the bug version is being removed in a few weeks. No one needs to be subscribed to bugs when every project uses bugs
<sinzui> sharing I mean
<sinzui> So the transition only needs to unsubscribe the people who do not already have access
<sinzui> You might want to subscribe the person making the change so that they can undo it
<abentley> sinzui: Okay, and the idea is that anyone else who needs access will get it by an AccessPolicyGrant?  What about the owner-- it seems like they might not have access via the policy.
<sinzui> The owner of the blueprint?
<abentley> sinzui: Yes, the owner of the blueprint.
<sinzui> What do we do for branches? oh, we give the owner a subscription that the project owner can later revoke
<sinzui> abentley, I think this is an odd corner case. This is like a random person creating a blueprint on my project that just happens to relate to a real secret thing I am working on
<sinzui> This is unlikely to happen, and if it does, the non-contributor is probably in the wrong
<sinzui> abentley, I am sharing ~canonical with all its commercial projects so that our organisation has full access. We advise every commercial project to to do that, I don't think we need to subscribe the blueprint owner or anyone other that the person that changed the information type
<abentley> sinzui: Cool.
<abentley> sinzui: Thanks for the clarification.
<sinzui> np
<StevenK> wgrant: e3522ed011014e61b507a1d730726769
<StevenK> sinzui: http://www.timeanddate.com/worldclock/meetingtime.html?iso=20120905&p1=240&p2=136&p3=263
<StevenK> wgrant: I guess bug 1046024 is untestable and should be marked as such?
<_mup_> Bug #1046024: product release finder ignores .apk files <product-release-finder> <qa-needstesting> <trivial> <Launchpad itself:Fix Committed by sinzui> < https://launchpad.net/bugs/1046024 >
<wgrant> StevenK: Right
<StevenK> wgrant: I guess bug 892511 can be nailed shut since the reoccurance was due to the DC move and sorted out quickly afterward?
<_mup_> Bug #892511: recipe build: ERROR: no such option: --allow-fallback-to-native <recipe> <regression> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/892511 >
<wgrant> StevenK: Indeed.
<wgrant> I wish people wouldn't reopen old long-closed bugs :(
<StevenK> I think bug 940168 is fixed too
<_mup_> Bug #940168: Two merge requests created on double-click <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/940168 >
<wgrant> It's not
<StevenK> I've not been able to trigger it
<wgrant> StevenK: Just click it twice quickly
<wgrant> While the first one is still spinning
#launchpad-dev 2012-09-06
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/format-imports-ad-infinitum/+merge/122981
<wallyworld_> wgrant: there's another existing css in choiceedit.js (similar to the one fixed in pickers) which i'll deal with as well
 * StevenK stabs Unity
<wgrant> StevenK: The diff in lib/lp/codehosting/__init__.py is slightly suspicious, but possibly harmless
<StevenK> wgrant: Right, I wasn't really sure about that one. Probably best to revert and land, I guess?
<wgrant> You can keep it if you throw it through ec2
<wgrant> wallyworld_: Bug #1045135, bug #1044546, bug #1044370 are all fixed now, right?
<_mup_> Bug #1045135: Changing a sharing policy can leave unused access policies lying around <disclosure> <easy> <qa-ok> <sharing> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1045135 >
<_mup_> Bug #1044546: +sharing shows '{policy_name}' for information types granted but not permitted by current combination of branch and bug sharing policies <qa-ok> <sharing> <ui> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1044546 >
<_mup_> Bug #1044370: update sharing policies via ajax needs to refresh +sharing details <disclosure> <sharing> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1044370 >
<wallyworld_> wgrant: yes
<wgrant> Also bug #1008538
<_mup_> Bug #1008538: Bug Supervisors are not notified when the project is not shared <bugs> <disclosure> <sharing> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1008538 >
<wallyworld_> yes, that's the one just applied via the sql
<wgrant> Right, just wanted to check. All closed now
<wallyworld_> wgrant: https://code.launchpad.net/~wallyworld/launchpad/css-fix-empty-sharing-table/+merge/122983
 * StevenK tries to work out the cause of bug 1036775
<_mup_> Bug #1036775: Launchpad shows bugs in projects that do not use Launchpad bugs <bug-columns> <bugs> <lp-bugs> <regression> <Launchpad itself:Triaged by stevenk> < https://launchpad.net/bugs/1036775 >
<wgrant> wallyworld_: I wonder if it would be cleaner to create a <tr><td><span>Pillar Name</span>'s private information blah blah</td></tr> in one Y.Node.create, then set tr/td/span's text
<wallyworld_> wgrant: i tend to find that Y.Node.create doesn't like everything in one statement, nt sure why
<wallyworld_> i can try it again, but every other time it returns something not quite right
<wgrant> wallyworld_: Hm, it should work, and it tends to be a bit cleaner I find
<wgrant> This particular case isn't that bad, but once you get another layer of nesting it ends up hideous
<wallyworld_> wgrant: it worked for this case. in the past, it has not gone well if i then try and set text on the node later, when it is created with nested elements in the string
<wallyworld_> i've pushed the changes
<wgrant> Thanks
<wgrant> wallyworld_: Hm, didn't you just revert that basically?
<wgrant> Now the only diff from devel is moving the Y.Node.create into the function
<wallyworld_> ffs, yes
<wallyworld_> wgrant: so here's the problem - Y.Node.create('<tr><td></td></tr>') doesn't work, so that's why i did it as a create of the tr node, then an append child for the td
<StevenK> Ah ha, I think I get it
<wallyworld_> i guess i need to do a .one(td) to set the text
<wgrant> Huh, that's really strange
<wgrant> I wonder if it's something about <tr>s not wanting to exist without a surround <tbody>
<wallyworld_> wgrant: the set text kills any nested nodes, so i needed to do a .one() to get the innermost node. that worked. changes on their way up
<wgrant> wallyworld_: I was thinking a .one('tr td span').set('text', pillar_name) would work
<wallyworld_> i don't need the span
<wallyworld_> and if i doa .one(td) on the row node, that works
<wgrant> Well, if you don't use the span then you need to also set the rest of the text
<wgrant> Which is really part of the template
<wgrant> But it doesn't really matter
<wallyworld_> i can just set the text of the table cell can't i
<wgrant> Right, but then you have to include the "'s private information blah blah blah blah" in the set() call
<wgrant> Rather than just the pillar_name
<wallyworld_> oh, i see what you are saying. but it works as is
<wgrant> wallyworld_: Are your latest changes pushed?
<wgrant> 'cause they're still bad
<wgrant> Oh, no they're not
<wgrant> Your order is just slightly confusing :)
<wallyworld_> wgrant: ok, i used a span, looks better i think
<wgrant> wallyworld_: Indeed, thanks
<wgrant> r=me
<wallyworld_> thanks
<StevenK> wgrant, wallyworld_: https://code.launchpad.net/~stevenk/launchpad/hide-search-for-unofficial/+merge/122993
<StevenK> Somewhat misnamed, but oh well
<StevenK> wallyworld_, wgrant: No review for me? :-(
<StevenK> Clearly not.
<wallyworld_> StevenK: sorry, was at lunch with bigjools
<wallyworld_> looking now
<StevenK> wallyworld_: Ah, then maybe you can help me with bug 984871, which has JS written all over it.
<wallyworld_> ok
<StevenK> wallyworld_: In Chromium, you load the page and you see the number of results, and then some AJAX events happen and it disappears.
<wallyworld_> but it works in ff
<StevenK> Yes, but we have to support both.
<StevenK> Unless they do stupid things like encode ~'s
<wallyworld_> sure, just remarking
<wallyworld_> i'll have a look at the js
<wallyworld_> StevenK: in your mp, i can't see where in the diff the !ServiceUsage.UNKNOWN block mentioned in the covering letter used to be
<StevenK> wallyworld_: It's not in the diff.
<StevenK> Line 74 of buglisting-default.pt
<wallyworld_> the covering letter implies it was changed
<wallyworld_> oh i see
<wallyworld_> the outer conditional stays but we are adding a new conditional deeper inside
<StevenK> Right, exactly.
<StevenK> wallyworld_: There is other stuff handled in the block that I've left alone.
<wallyworld_> StevenK: why not use should_show_bug_information as on line 89?
<wallyworld_> that seems to fit with using an external tracker or not
<wallyworld_> especially the explanatory text
<StevenK> wallyworld_: Mainly because that TAL is a mess.
<wallyworld_> one issue for me is that we would be using two different ways of determining the use of an external tracker - view/should_show_bug_information and the enum check
<wallyworld_> can't the form stuff just be added to hat same construct?
<wallyworld_> with the not: removed?
<StevenK> Ah, I see.
<StevenK> wallyworld_: The construct further down is for portlets
<wallyworld_> StevenK: you mean because class="top-portlet" ?
<wallyworld_> the search form is inside this block
<wallyworld_> so i think the correct thing to do is use view/should_show_bug_information
<wallyworld_> just like the block above
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1188320/
<wallyworld_> i don't quite fully understand line 55 of the diff - "action search_url|string:" is ultimately replaced with just search_url without the "|string:". I guess this is a consequence of removing the f-flag and stopping the use of tal attributes?
<wallyworld_> StevenK: yes, i think that new diff makes sense to my untrained eye.
<StevenK> wallyworld_: I have no idea about that bit -- I'm just trying to rip out the class python: bit
<wallyworld_> StevenK: i think you need action search_url|string: since it will handle an empty search_url
<wallyworld_> and just render action="" instead maybe of action=None or something like that
<StevenK> steven@undermined:~/launchpad/lp-branches/hide-search-for-unofficial% bzr grep 'search_url\|string:'
<StevenK> lib/lp/bugs/templates/bugtarget-macros-search.pt:    <a tal:attributes="href advanced_search_url|string:?advanced=1"
<wallyworld_> the |string bit will only be in the tal. the search_url will be set elsewhere
<wallyworld_> so given that attribute is not part of the feature flag removal, why not just leave it there as it was?
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1188340/
<wallyworld_> StevenK: it needs to be a tal attribute
<wallyworld_> for the expression to be evaluated i think
<StevenK> wallyworld_: Ah, http://pastebin.ubuntu.com/1188345/
<wallyworld_> StevenK: yes, that looks right i think
<wallyworld_> tests will test us for sure :-)
<StevenK> Yeah, indeed
<StevenK> Let me commit and push
<wallyworld_> ok, i'll wait for the final diff to update
<wallyworld_> StevenK: r=me. that chrome issue doesn't happen for me locally sadly, just with lp.net
<StevenK> wallyworld_: Do you have buglisting turned on?
<StevenK> There's a feature flag involved.
<wallyworld_> ah, maybe not. i always forget that. i wish it had been added to sample data
<wallyworld_> right, fails now. good.
<wallyworld_> StevenK: it's a hack that was added in for an issue with Chromium's history handling
<wallyworld_> i don't understand what the issue was or what the fix does, but removing the hack stops the failure
<wallyworld_> because it no longer updates history with an 'incorrect' copy of the model missing 'total'
<wallyworld_> but i don't know what else will break removing this
<StevenK> Strange
<StevenK> wallyworld_: Where's the hack?
<wallyworld_> StevenK: listing_navigator.js, around line 166
<StevenK> So I wonder if we could fix it by injecting total into e.prevVal
<StevenK> But no idea how to do that
<wallyworld_> i'd like to fully understand it all before we make changes
<wallyworld_> why was the change made, what was the issue etc
<adeuring> good morning
<mgz> good morning adeuring
<jml> lifeless: hi
<jml> lifeless: re the LP SOA docs
<jml> lifeless: where should test implementations of services live?
<lifeless> for use when testing things that act as clients?
<lifeless> They should be provided by the concrete service, or at least interface tested by it to prevent skew,
<lifeless> .
<jml> lifeless: yeah, for that use
<jml> lifeless: so clients will probably have testing dependencies on the server code tree?
<lifeless> or a pypi package or deb package
<jml> right
<lifeless> they shouldn't need to import it however. better if they can't in general - use processes to use it
<jml> hmm
<jml> lifeless: you have told me before but I can't recall (sorry). Why is it better if they can't?
<lifeless> it makes it harder to tightly couple these loosely coupled things
<lifeless> e.g. language neutrality - if you want to redo a client into go, you don't need to touch the server.
<lifeless> test or otherwise
<lifeless> you can argue that this is silly, you can always do that work later... and if it was the only advantage, I'd say sure
<lifeless> but you also get enforced layer safety; you don't need to worry about conflicting library versions and so forth.
<jml> right. discipline in that regard rarely works
<jml> at least, not for me :)
<lifeless> indeed, you can't be a little bit (binary state)
<jml> probably not today or tomorrow, but maybe next week, I'd like to set this up for libdep-service.
<lifeless> can I do anything to help ?
<jml> hmm.
<jml> You mean, other than you already have?
<lifeless> :)
<lifeless> boom-tish
<jml> Answering questions is the only thing I can think of.
<lifeless> ok, well I stand ready.
<lifeless> is libdep-service django ?
<stub> wgrant: http://paste.ubuntu.com/1188697/ has some changes to dbpolicy.py and test_error.py. Had some fallout, and think I've ended up tracking a problem that was bugging gary and benji for ages
<stub> still haven't done a full test run though
<wgrant> stub: What'd you have to change?
<wgrant> And what sort of fallout? It looked relatively sensible.
<stub> getUtility(IZStorm).get('foo') could emit an OperationalError rather than a DisconnectionError if the db was down and the Store needed to be constructed
<stub> Opened a bug on that one
<wgrant> Huh
<wgrant> That's surprising.
<stub> Also, the test_error was triggering our 'store left in a disconnected state' checks.
<wgrant> They are the stuff of nightmares
<stub> Yeah, and I think I've fixed it now
<stub> The stores look like they were being created in that state, and I'm now ensuring they always get registered with the transaction manager so transaction abort puts them into a state we expect them to be in.
<bac> mgz: hi, were you going to review that MP i posted yesterday?  if not, i'll get someone else.
<adeuring> jelmer: could you please have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/specification-grants/+merge/123062 ?
<jelmer> adeuring: looking
<adeuring> jelmer: thanks!
<mgz> bac: I shall post something to it, did look over the code
<bac> mgz, thanks.  gary is also doing it but i'd like your comments since you've already looked
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: â
<bac> mgz: gary has posted his comments on my MP.  i'd like to land this shortly so if you have helpful comments could you add them soonish?  thanks.
<mgz> bac: posted a few comments
<bac> thanks mgz!
<mgz> I see gary asked about the participants thing as well.
<mgz> so it's not just me not knowing launchpad :)
<bac> mgz: yep.  well commented now.
<adeuring> jcsackett: thanks for your review!
<jcsackett> adeuring: you're welcome!
<rick_h_> deryck: ping, got a sec?
<deryck> rick_h_, on call with abentley right now
<rick_h_> deryck: rgr
<deryck> rick_h_, free now.  What's up?
<rick_h_> deryck: looking at the sql count test failure stuff and I think part of it is the storm clearing or something. http://paste.mitechie.com/show/774/ is the test failure, which shows the product insert, but the test as a "Store.of(product).invalidate()" which I would expect to reset the queries?
 * deryck looks....
<abentley> deryck: Should we be posting design decisions like "blueprints support only PUBLIC and the proprietary types" somewhere?
<deryck> abentley, hmmm, good question.
<deryck> abentley, yeah, let's follow up to the dev list with that.
<deryck> abentley, actually, wait, no I don't think we need to for that.
<deryck> abentley, we're really just matching what we had before, plus privacy, or what people think of when they say privacy.
<deryck> rick_h_, so the selects are the new queries?  And you think these are left over from a different test?
<rick_h_> deryck: so I'm investigating part of it. I think it might be my code's fault and I'm investigating, but the INSERT being listed made me wonder if the invalidate() wasn't working/what I was thinking
<deryck> rick_h_, invalidate just tells storm to through away its own cache of the objects, to force it to get new objects.
<rick_h_> deryck: ok, so this is in fact saying the insert/update are part of the recorded queries then
<deryck> rick_h_, right.  I'd run this in devel and see what you get and compare the two.
<rick_h_> deryck: ok, digging. Thanks for the quick sanity check.
<deryck> np
<abentley> deryck[lunch]: I noticed that the set of allowed information_types for blueprints will exactly match that for projects.
<deryck> abentley, nice
<rick_h_> deryck: ping again, halp!
<deryck> rick_h_, give me just a few minutes and I can help.
<rick_h_> deryck: ok thanks
<deryck> np
<deryck> rick_h_, ok, what's up?
<rick_h_> deryck: hangout?
<deryck> rick_h_, sure.
<rick_h_> sinzui: halp please! have a few min for a hangout?
<sinzui> I do
<rick_h_> sinzui: invite coming
<deryck> sinzui, and if you can spare 5 minutes after rick_h_, I'd like a quick chat about something, too.
<abentley> jcsackett: could you please review https://code.launchpad.net/~abentley/launchpad/limit-blueprint-info-types/+merge/123150 ?
<jcsackett> abentley: certainly.
<jcsackett> abentley: r=me.
<abentley> jcsackett: thanks.
<deryck> sinzui, are you still chatting with rick_h_?
<sinzui> no
<sinzui> did rick_h_fall off the face of the earth?
<rick_h_> he wants you sinzui, not me.
 * rick_h_ runs away NOT IT!
<deryck> I'm losing cable modem now anyway.  ignore me.
<deryck> cable guy is wiring up voip phone and needs to disable lines to test.
<rick_h_> deryck: yea, you've been very jittery today
<deryck> I'll return when he's done. :)
<deryck> sinzui, let's try to chat, he has cut me yet.  should be brief.
<deryck> hasn't cut me
<sinzui> okay
 * sinzui thinks he is talking to himself
<sinzui> rick_h_I am here?
<sinzui> Am I alive?
<rick_h_> sinzui: yes
<sinzui> thank you for the validation of my existence
<sinzui> next up, find a purpose
<rick_h_> sinzui: I find it best to tie them together. I exist to serve a purpose, my purpose is to exist. Next question.
<lifeless> hmmm
<lifeless> new oauthkeys now notify with a from of bounces@c.c
<lifeless> :/
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<sinzui> wallyworld_, https://bugs.launchpad.net/launchpad/+bugs?field.tag=escalated
<lifeless> jcsackett: btw, you didn't land https://code.launchpad.net/~coreygoldberg/lp-dev-utils/ppr-access-parser/+merge/119409 for some reason ?
<jcsackett> lifeless: Looks like I just forgot. If you can land, feel free--storm just took out Internet, and cell phone can't land. :-)
<lifeless> jcsackett: cool, I will do so
<jcsackett> lifeless: Cool, thanks, and sorry for the forgetfulness.
<lifeless> no matter
<jcsackett> sinzui: A storm has taken out Internet, and it appears my phone can't handle mumble on 3G alone.
<sinzui> okay
#launchpad-dev 2012-09-07
<StevenK> wgrant: http://pastebin.ubuntu.com/1189950/
<wgrant> StevenK: Right, I'd grep around for egistrant and upervisor and ecurity and delete lots of stuff :)
<StevenK> wgrant: egistrant is hampered by branch and product registrant tests
<wgrant> True
<wgrant> But still
 * StevenK greps for ecurity and drowns in rSP calls
<StevenK> lib/lp/blueprints/browser/specification.py:        text = 'Change privacy/security'
<wgrant> Huh
<StevenK> wgrant: I'm quite tempted to just leave bug-private-by-default.txt alone and it can die when the rest does.
<wgrant> StevenK: That's the plan
<wgrant> The thing it still tests is the only situation in which the bug supervisor should still be subscribed
<StevenK> Then I can't find anything else, or it has been drowned out by the noise.
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/remove-registrant-tests/+merge/123207
<wallyworld> StevenK: too late
<StevenK> wallyworld: Indeed, thanks.
<wallyworld> np
 * StevenK peers at the end test in lib/lp/bugs/stories/bugtask-searches/xx-advanced-upstream-pending-bugwatch.txt
<StevenK> Does anyone *actually* do that?
<wgrant> Right, that's why it is how it is now
<wgrant> Curtis argues not
<wgrant> It used to be the case that +bugs-index avoided showing the list, but you could hack your way through to +bugs to get the listing regardless of configurtaion
<wgrant> But those pages are merged now
<lifeless>  * 19317 Time Outs
<lifeless> twitch
<lifeless> when do you guys start on maintenance ?
<wgrant> lifeless: Monday, allegedly
<wgrant>     12938 /    0  Archive:EntryResource:getPublishedBinaries
<wgrant> wat
<wgrant> I blame ev
<wgrant> I think
<lifeless> could be
<wgrant> HTTP_USER_AGENT: lazr.restfulclient 0.12.0; oauth_consumer="pkgme-binary"
<wgrant> jml
<wgrant> Trigram index on BPN.name will probably fix that instantly
<wgrant> But I wonder if they're deliberately using exact_match=False
<wgrant> Ah, it's also still querying on BPR.bpn
<lifeless> see also ev's code
<lifeless> forwarded you mail
<wgrant> Um
<wgrant> Do we really have a second service named 'poppy'?
<wgrant> There we go, got that query down to 10ms
<wgrant> From 4900ms
<wgrant> Launchpad is optimised :)
<lifeless> \o/
<wgrant> Oh good
<wgrant> The backup failed today
<wgrant> So we can create the indices now :)
<lifeless> again?
<wgrant> Yeah
<wgrant> It's almost like the slave feedback thing isn't working
<lifeless> stub will be miffed :)
<wgrant> But I thought it was
<lifeless> it has an upper bound IIRC
<wgrant> Yeah
<stub> I suspect it happens when the replication connection drops. It reconnects, but the master has already moved on and started cleaning crap up
<stub> I'll check the logs when I'm back upstairs
<lifeless> stub: perhaps we should backup on the master again ?
<lifeless> stub: identical bloatwise
<wgrant> And we have the CPU
<wgrant> The indices won't immediately fix either consumer's performance issues, but they'll fix the fixed pkgme queries from 200ms to 10ms
<wgrant> and the 5000->200 optimisation will land shortly :)
<lifeless> lol
<wgrant> Hm?
<lifeless> 15:34 -!- stub [~stub@canonical/launchpad/stub] has quit [Ping timeout: 268 seconds]
<wgrant> Maybe he's going back upstairs :)
<lifeless> I suspect he walked of upstairs right after
<lifeless> 15:29 < stub> I'll check the logs when I'm back upstairs
<lifeless> and didn't see anything we said after that
<wgrant> Yep
<wgrant> 13:29:20 < stub> I'll check the logs when I'm back upstairs
<wgrant> 13:29:57 < lifeless> stub: perhaps we should backup on the master again ?
<wgrant> 13:30:02 < lifeless> stub: identical bloatwise
<wgrant> 13:30:22 < wgrant> And we have the CPU
<stub> Yeah, that is the backup plan, so to speak :)
<stub> Chokecherry is preferred because it has more disk
<wgrant> Ah, indeed
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/bpn-spn-trgm-index/+merge/123208 is a nice difficult one for you
<stub> Cool. I think I suggested that to someone else recently
<wgrant> I meant to do it months ago after the first trigram trial, but didn't get around to it until now
<stub> And for bikeshedding, should we use a different suffix for trigram indexes?
<stub> Down to one test fail on my db policy updates branch
<wgrant> stub: Possibly. name tends to be (part of) a _key usually, so there hasn't been a conflict until now
<wgrant> But __trgm or something would work too, I suppose
<stub> yeah, just thinking hypotheticals
<stub> Every naming convention I know for dbs breaks down somewhere, I'm not too fussed.
 * wgrant checks ALTER INDEX ... RENAME lock requirements
<StevenK> wgrant: So should I drop that test completly?
<wgrant> StevenK: Which?
<StevenK> wgrant: "[13:01] < wgrant> Right, that's why it is how it is now" and so on
<wgrant> StevenK: Ah
<wgrant> StevenK: Well
<wgrant> StevenK: Decide whether the use-case is worth it
<wgrant> StevenK: If it's not, delete the test that says it is
<wgrant> Bah, renaming indices appears to require an exclusive lock
 * wgrant just creates the new ones with good names
<StevenK> Well, there's a critical regression that says it isn't.
<StevenK> So I don't know.
<wgrant> Ah, now I think I remember
<wgrant> There's a link
<wgrant> In a portlet
<wgrant> To that list
<wgrant> "XX bugs need forwarding upstream" or something like that
<StevenK> So I can close the critical by removing functionalility or by replying and saying it is used for this feature.
<wgrant> BugsInfoMixin.pending_bugwatches_url
<wgrant> Right
<wgrant> This changed when buglistings merged +bugs-index and +bugs
<StevenK> Ahh
<StevenK> Which might be where the confusion is coming from.
<StevenK> I'm tempted to just close the bug saying it isn't a regression.
<lifeless> how many tests does LP have today ?
<StevenK> 17453 or thereabouts
<wgrant> stub: I've renamed the indices. Can you throw them at prod?
<stub> k
<stub> wgrant: done
<wgrant> stub: Thanks.
<wgrant> stub: Can you also do it to qastaging?
<wgrant> Since it won't happen automagically
<stub> done
<wgrant> Great
 * StevenK can't work out what is touching IBug.date_last_updated when someone subscribes or unsubscribes.
<StevenK> OH. Via ZCML.
 * StevenK goes to review his lunch.
<wgrant> Yeah, Bugs uses a fair few subscribers
 * nigelb wonders what would happen if StevenK gave r- for lunch.
<StevenK> Hmmm, that still doesn't explain it.
<StevenK> And I doubt I can get a useful calltrace out of the function itself :-(
<wgrant> The stacktrace should be fine
<StevenK> Hm, calling bug.subscribe() and bug.unsubscribe() don't trigger it.
<wgrant> StevenK: Where are you calling them?
<StevenK> Directly
<StevenK> wgrant: Based on the bug filed, I thought the model functions might do it. But they don't.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/faster-binary-search/+merge/123213 â +86/-228 query optimisation
<wgrant> If you have time today
<StevenK> wgrant: Your indenting in lib/lp/archivepublisher/publishing.py looks wrong
<wgrant> StevenK: It's arguable. PEP 8 suggests extra indentation like that when it would otherwise be visually ambiguous
<StevenK> wgrant: r=me
<wgrant> Thanks
<StevenK> And grepping for date_last_updated doesn't help either
<StevenK> Clearly, it's updated by magic.
<StevenK> Subscribing and unsubscribing does *not* trigger an update of
<StevenK> IBug.date_last_updated.
<wgrant> lib/lp/bugs/configure.zcml:        handler="lp.bugs.subscribers.buglastupdated.update_bug_date_last_updated"/>
<wgrant> is a bit suspicious
<StevenK> Oh, why?
<wgrant> Hm, but it's not on IBugSubscription creation
<StevenK> Yes
<wgrant> So possibly it's the ObjectModifiedEvent from updating the subscriber or affected count or something?
<StevenK> I thought they just backed onto BugSubscription and didn't hit Bug?
<StevenK> And I thought affects me didn't subscribe
<wgrant> Some things are denormed
<wgrant> +affectsmetoo doesn't subscribe, no
<wgrant> But the operations are performed together
<StevenK> I suspect +affectsmetoo will trigger date_last_updated
<StevenK> wgrant: But by what? If stuff was denormed, you'd think subscribe() would perform it?
<wgrant> Maybe
<wgrant> I don't think it's a trigger
<wgrant> I hope it's not a trigger
<StevenK> Not that I can see
<wgrant> StevenK: I wonder if it's just lazr.restful firing an ObjectModifiedEvent after subscribe() is called
<StevenK> wgrant: The existing test calls notify(ObjectCreatedEvent(subscription))
<wgrant> I mean an ObjectModifiedEvent(bug)
<wgrant> I don't know if it does it implicitly on named ops
<wgrant> But it might well do so
<StevenK> Hmm, API scripts really don't want to talk to dev
<wgrant> There's an environment variable for that
<StevenK> LP_DISABLE_SSL_CERTIFICATE_VALIDATION?
<wgrant> Yeah
<StevenK> wgrant: http://pastebin.ubuntu.com/1190284/
<wgrant> StevenK: Make sure you're using a modern launchpadlib
<wgrant> eg. run with bin/py
<StevenK> Oooh
<StevenK> RARGH
<StevenK> IT DOES UPDATE
<StevenK> LAZR.RESTFUL!
<StevenK> wgrant: http://pastebin.ubuntu.com/1190287/
<wgrant> Right
<wgrant> That makes sense
<wgrant> So you need to fix the subscriber
<wgrant> To not subscribe to subscribers
<wgrant> (currently it sets date_last_updated unconditionally; it may want to verify some attributes of the event first)
<StevenK> Right
<StevenK> I should have guessed, since lazr.restful calls ObjectModifiedEvent on the object you called the method on
<StevenK> Right, bug updated
<wgrant> StevenK: Heh
<wgrant> StevenK: Did you really run something that only deleted tests through ec2?
<StevenK> Hm, that was a bit silly
<StevenK> Oh well
<jml> wgrant, lifeless: if there's something you want changed about our code, do let us know. <https://bugs.launchpad.net/pkgme-devportal/+filebug> is a good place.
<wgrant> jml: Do you know if you're deliberately using exact_match=False in the getPublishedBinaries call? That does substring matching on the name, which is a little slower and may return excessive results.
<jml> wgrant: good question. oddly enough, james_w sent an email about that to me yesterday...
 * jml pages in
<wgrant> exact_match=False is currently very slow, but so is exact_match=True. Both will be fast on Monday, but exact_match=True will obviously still be a little faster and probably more correct.
<jml> ah
<jml> "I think this might have had something to do with using exact_match=False when looking for publications, so it was doing fuzzy package name matching. I don't know why it was doing that, so I removed it. It should now just iterate over all publications for configured architectures for the given binary package name."
 * jml rolls up sleeves and does code reviews.
<wgrant> https://bugs.launchpad.net/launchpad/+bug/1047176 is the slowness issue
<_mup_> Bug #1047176: Archive.getPublishedBinaries is terribly slow <timeout> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/1047176 >
<wgrant> You're currently getting somewhere around 10000 timeouts a day, so this will hopefully improve things
<jml> ah right.
<jml> it's a lp:udd instance
<jml> so it's hard to tell
<wgrant> Ah
<wallyworld> sinzui: hi, i have a question about the product licence widget
<sinzui> yes?
<wallyworld> i have gobs of javascript from the tal into it's own module, and replaced all the custom expander code with standard collapsible elements. but it appears to me that the code which handles source_package_release and allow_pending_license is obsolete since i can't see how or where these attributes on LicenseWidget are ever set
<wallyworld> i also have the hidden stuff working for bots and screen readers etc
<wallyworld> it turns out yui convenientsly adds a yu3-javascript class to the html element if javascript is enabled
<sinzui> Register the project from a (disto) source package
<sinzui> There are 4 places to register a project:
<sinzui> projects/+new
<sinzui> <projectgroup>/+newproject
<sinzui> <dsp>/+index and <sp>/index
<wallyworld> ok. i only knew about the first way
<sinzui> wallyworld, https://launchpad.net/ubuntu/quantal/+source/cryptkeeper
<sinzui> https://launchpad.net/ubuntu/+source/cryptkeeper is really ^. It chooses the current SP
<wallyworld> so you mean the "Register Upstream Project" link?
<sinzui> yes
<sinzui> It prepoulates the form and adds the copyright file to help with choosing licenses
<wallyworld> ok. i'll have a play around on qas and see how it all behaves. thanks
<wallyworld> i am pretty sure i have it all working fine, i just want to check these extra cases
<wallyworld> i had hard coded stuff into the projects/+new to initially test
<sinzui> Oh, I got one url wrong https://launchpad.net/launchpad-project/+newproduct is correct
<wallyworld> ok
<wallyworld> with all the standard collapsible / expander stuff uses, the licence javascript shrinks dramatically in size. it was all very old code
<sinzui> Yes, it was barry's first js written just a few months after we started using YUI
<sinzui> I think keyboard users hate the form because you can tab into the a hidden license
<wallyworld> yeah, it was all that was possible in the day but years later we have better infrastructure. i'll check the tabbing issue to see that it's gone
<sinzui> well
<sinzui> I reviewed a lot of this code in 2009. I decided then and there that I will never write a windmill test or repeat the mistake of inline js
<sinzui> I used yui-test...then had to argue that my tests were better then the windmill site
<sinzui> shite
<wallyworld> i didn't know yuitest at first - my first tests of this nature were in windmill. i am glad it's gone for sure
<wallyworld> sadly, people all over still tend to write inline js. especially in the world of jsp, php etc
<deryck> Morning, all.
<rick_h_> morning
<rick_h_> wallyworld: not enough shooting taking place for violaters :P
<wallyworld> rick_h_: too soon after all the recent happenings over there?
<rick_h_> they keep hitting the wrong people. Missing the JS violaters
<wallyworld> hah
 * wallyworld doesn't understand th US gun culture
<lifeless> s/gun //
<wallyworld> well...
<wallyworld> that's a beer conversation
<lifeless> :)
<lifeless> so, week after next then
<wallyworld> yes, you be in brisbane?
<lifeless> likely to be
<wallyworld> cool. we'll have to organise dinner or somwthing
<lifeless> we're getting an ex-weta dude in to talk maas use cases.
<lifeless> I'll be there to help extract maximum knowledge
<wallyworld> ok. is this a consultant?
<lifeless> yes
<wallyworld> well, when you know dates let's get something organised
<lifeless> will do
<lifeless> thats pending flacoste and the consultant making sweet sweet arrangements.
<wallyworld> ok
<abentley> rick_h_: Do you have time for a UI question?
<rick_h_> abentley: sure thing
<abentley> rick_h_: So, only products are going to allow blueprints to have proprietary information types.  All other kinds of blueprints are necessarily PUBLIC.
<abentley> rick_h_: We have to display the information_type selector when creating a bluprint that might be associated with a product.
<abentley> rick_h_: should we also display the information_type selector if it won't be associated with a product?
<rick_h_> I'd say no, I'd hide the field unless a product is selected and if the product is selected we'd show the field, default to the products information type and maybe do a flash green to show it's there
<rick_h_> no sense adding UI that the user can't effect and the banner indicated public/private already so showing it is duplicate information
<abentley> rick_h_: Okay.  I'm inclined not to show information_type dynamically-- just keep it on the "https://blueprints.launchpad.net/specs/+new" page and the blueprint-for-product page.
<rick_h_> abentley: ok, then I'd probably at least disable/enable the field based on the "For"?
<abentley> rick_h_: Yes, that's what I mean.
<rick_h_> ah ok cool
<abentley> rick_h_: would you hide the picker if you knew the project policy supported only one value?
<rick_h_> I think so since the large majority of use cases don't need to know about the data
<rick_h_> it's like showing the current default information types for a project in a portlet, only show that portlet if one of the apps are non-public.
<abentley> rick_h_: Cool.
<abentley> deryck: I find it confusing that getAllowedInformationTypes does not consider getBranchSharingPolicies or getBugSharingPolicies.
<deryck> abentley, I'm in all that space myself nowâ¦. let me look closely again now....
<deryck> abentley, yeah, I find it weird, too.  Now that I get my head around it.  Sorry to be so slow.
<deryck> abentley, seems like it would save a query, too, unless the method is doing more than it seems to me.
<abentley> deryck: So, silly me, I assumed that Branch.getAllowedInformationTypes used ISharingService.getAllowedInformationTypes.  Instead, it uses IBranchNamespacePolicy.policy.getAllowedInformationTypes()
<deryck> ah, I was just thinking of the SharingService method.
<abentley> deryck: For bugs, though, the BugSharingPolicy doesn't seem to be used, just Pillar.getAllowedInformationTypes().
<deryck> I wonder why they're different?
<sinzui> deryck, abentley Pillar.getAllowedInformationType() uses the policy and understand that the types can be in transition
<abentley> sinzui: Oh, my bad, that's Pillar.getAllowedBUGInformationTypes.
#launchpad-dev 2012-09-08
<czajkowski> aloha
<jelmer> ahola czajkowski
<czajkowski> jelmer: ello greetings from bluefin where we're having a global jam
<jelmer> czajkowski: bluefin?
<Laney> the london office
<czajkowski> yarp
<jelmer> ah, nice
 * jelmer hasn't been to the new office yet
<czajkowski> bar it's very warm yeah
<czajkowski> it's about 26 in london today
<jelmer> same here..
 * jelmer is at the local hackerspace watching a talk
<czajkowski> oh what on
<jelmer> evading some of the security measures in Ubuntu's sshd :)
<czajkowski> hah
<czajkowski> tried to get these webaps working
<czajkowski> butno luck
<czajkowski> and now some unity testing
<jelmer> what are you jamming on?
<czajkowski> 12.10
<czajkowski> upgraded and filing bugs
<czajkowski> gwibber and compiz keep crashing
<czajkowski> xnox qote of the day:  I fix things after I break things,
<czajkowski> reason for Laney to not remove him :)
<jelmer> haha
<jam> czajkowski: very warm? it was 44 today in Dubai... 26 would be a cold snap :)
<czajkowski> jam: true indeed, but in the office at the weekend the air con is off
<czajkowski> still we had some fun :)
#launchpad-dev 2012-09-09
<cjohnston> Has anyone tried throwing lpsetup against quantal yet?
<lifeless> yes
<lifeless> ajmitch did, filed a bug, and it got fixenated.
<cjohnston> So then it should work?
<wgrant> It works fine once the liblxc path is fixed, and I think that's in trunk now
<cjohnston> wgrant: any chance it could get thrown onto the ppa in the near future?
<wgrant> cjohnston: Ah, it hasn't actually been merged yet. https://code.launchpad.net/~jameinel/lpsetup/missing_lxc_1045728/+merge/122663
<cjohnston> lol.. looks like jam may get into an fight with the QA bot
<wgrant> Yeah
<wgrant> I just approved it again. We'll see what it thinks
<cjohnston> hee
<cjohnston> wgrant: if the setup (install-lxc) fails, trying to run it again fails since lptests already exists.. is there a way to kill it to start over?
<cjohnston> wgrant: http://paste.ubuntu.com/1193839/
<wgrant> cjohnston: sudo lxc-destroy -n lptests
<wgrant> Will stop and delete the container
<cjohnston> thanks..
<lifeless> anyone around for a small patch review ?
<lifeless> https://code.launchpad.net/~lifeless/js-oopsd/post/+merge/123439
<lifeless> StevenK: wgrant: ^ ?
<lifeless> jelmer: hi; missing you in #subunit ;)
<cjohnston> wgrant: looks like it failed again :-(
<czajkowski> cjohnston: wrong timeszone for him
<cjohnston> :-)
<lifeless>  /win 85
<lifeless> any reviewers around ?
<lifeless> https://code.launchpad.net/~lifeless/js-oopsd/post/+merge/123439 still on offer
<czajkowski> lifeless: you're eager!
<czajkowski> lifeless: good weekend ?
<lifeless> czajkowski: pretty good yeah; how was the jam?
<czajkowski> lifeless: yeah a good bit of fun, found a lovely add on for thunderbird for bug mail from one of the folks there. installed 12.10, I hate compiz even more than usual
<czajkowski> but we had a fun day
<lifeless> poor compiz
<czajkowski> lifeless: it's evil ok! I upgraded and low and behold bugs
<czajkowski> lifeless: most annoying is the upgrade and my power icon not working as intended again. Every time I upgrade it breaks, and this is a new machine :/
<lifeless> czajkowski: I agree, I was teasing you
<czajkowski> ah
<czajkowski> lifeless: your tone isn't so easy for me to get at times over irc as I don't know you :/
<czajkowski> one day I'll get it :)
<lifeless> czajkowski: indeed, not enough bandwidth on IRC :)
<czajkowski> It was a good weekend, although it kinda felt like working yesterday so not had much of a break always a downside, but it was nice to do google hangouts with folks from different teams worldwide
<czajkowski> lifeless: will you be at UDs ?
<lifeless> czajkowski: should be
<czajkowski> ah good we shall meet then
<lifeless> StevenK: ahhahah nice.
<StevenK> lifeless: Hmm?
<lifeless> StevenK: the API datestime thing
<StevenK> Oh, right.
<lifeless> StevenK: lovely interaction you tracked down.
<lifeless> StevenK: if you have a few minutes,https://code.launchpad.net/~lifeless/js-oopsd/post/+merge/123439
<StevenK> Now for my next trick, I have to fix the bloody thing. :-)
<StevenK> lifeless: r=me with one caveat you're free to ignore.
<lifeless> thanks
<lifeless> StevenK: is the niggle that the docs are duplicated ?
<lifeless> StevenK: (btw I think you mean js_oopsd/__init__.py)
<StevenK> lifeless: Yeah
<lifeless> StevenK: so, we don't have a good answer yet.
<StevenK> lifeless: Like I said, you're free to ignore it.
<lifeless> StevenK: most of our litle projects have this issue; we want pydoc to work, we want README to be useful.
#launchpad-dev 2013-09-02
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/cleanup-bfjo/+merge/183396
<StevenK> wgrant: You say, as I closed my browser
<StevenK> wgrant: Would be nice if it didn't kill the entire build farm when deployed, but it's an excellent cleanup
<wgrant> StevenK: I've wanted to change the slave cookie for 3 years :)
<wgrant> Now is as good a time as any.
<StevenK> wgrant: Because it's wrong, broken, just unlikeable?
<wgrant> StevenK: It has delusions of relevance to security, contains unnecessary information which complicates the implementation, and its opacity complicates debugging.
<cjwatson> wgrant,StevenK: Thoughts on posting http://blog.launchpad.net/?p=4146&preview=true ?
<cjwatson> (In particular, anything else that's worth mentioning there?)
<cjwatson> It's fairly heavily biased towards the things I've been involved with, I expect
<wgrant> cjwatson: Sounds sensible. I think you got most of the concrete improvements.
<cjwatson> Thanks.  Published.
<cjwatson> (http://blog.launchpad.net/general/launchpad-build-farm-improvements)
<wgrant> :)
<xnox> \o/ cool stuff =)
#launchpad-dev 2013-09-03
<bigjools> wgrant/StevenK: did the recipe builds change recently to append ~ubuntuNN instead of ~series?
<wgrant> bigjools: Yes
<wgrant> Ubuntu's used the version number for backports for a while, and we need to switch before u-series
<bigjools> wgrant: ok.  you broke juju-core folks. haha :)
<wgrant> Why are they depending on the version number?
<wgrant> s/version number/structure of the version number/
<wgrant> That's pretty broken
<bigjools> stupidity I think :)
<wgrant> Still
<wgrant> That recipe is already pretty broken
<wgrant> I'm going to close that tab and pretend I never saw it.
<bigjools> heh
<StevenK> wgrant: The next branches for buildd-manager are due RSN?
<wgrant> Indeed
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/buildd-manager-inlineCallbacks/+merge/183578
<StevenK> wgrant: I'm wary of the the commit in the manager code
<wgrant> StevenK: It's committing after it comes back from a Deferred
<wgrant> Which doesn't make much sense.
<wgrant> Though I'm surprised you noticed that in the merged diff.
<StevenK> It's five lines into a green diff
<wgrant> In fact, you couldn't have noticed. Which commit are you talking bout?
<StevenK> 455	+ # Commit the changes done while possibly rescuing jobs, to
<StevenK> 456	+ # avoid holding table locks.
<StevenK> 457	+ transaction.commit()
<wgrant> That's not new
<wgrant> And it shouldn't do anything, but it's there to ensure that builder.currentjob is as up to date as possible.
<StevenK> Oh, it's just moved from other twisted garbage
<wgrant> I didn't change any code except for removing two xmlrpc.Fault wrappers.
<StevenK> And I guess that is one of the commits that is not long for this world?
<wgrant> Correct.
<wgrant> We can't afford to calculate currentjob there.
<wgrant> It will be precalculated, and that method won't touch the DB until further down.
<StevenK> So thanks to a day of JavaScript and Twisted, my brain is now leaking out of my ears
<StevenK> wgrant: r=me
<StevenK> wgrant: Sorry, I got distracted
<wgrant> Thanks
<cjwatson> wgrant: Your buildd-manager-inlineCallbacks branch and my destroy-builder-aborted branch clash.  Which would you prefer to have landed first?
<wgrant> cjwatson: I was deliberately waiting for yours to land, sorry.
<cjwatson> OK, landing
<wgrant> During the refactoring I discvoered something that I thought was just a production glitch a few months ago.
<wgrant> A manualled, disabled builder won't get its current build terminated.
<wgrant> Probably missed originally because of the convoluted callbacks. It makes no sense otherwise.
<wgrant> cjwatson: test_binarypackagebuildbehavior still references AbortedSlave
<cjwatson> what.  I grepped.
<cjwatson> one moment :(
<cjwatson> Oh, I must have only grepped buildmaster, damn
<wgrant> Ah, yeah
<cjwatson> Very confused by what buildbot just did
<cjwatson> It successfully ran no tests
<wgrant> It doesn't check the exit code due to an old lxc-start-ephemeral bug
<wgrant> The testrunner probably died badly due to the import.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/testfix-destroy-builder-aborted/+merge/183598
<wgrant> cjwatson: r=me
<cjwatson> landing
<stub> Anyone up for a Swift branch review? More /srv and /srv2 juggling underway it seems.
<wgrant> I really should get to that eventually
<wgrant> But the /srv and /srv2 juggling can't stop for at least a few months after we start using Swift
#launchpad-dev 2013-09-04
<wgrant> cjwatson: I concur, now that we have lots of PPA builders.
<wgrant> And I wasn't looking forward to porting that bit of the query to the New World.
<wgrant> :)
<cjwatson> Which of our several new worlds?
<wgrant> buildd-manager's.
<wgrant> Having a separate scanner to keep a copy of the heads of the queues in memory, so we don't have to have three transactions per builder per cycle.
<cjwatson> Hadn't realised you were rearranging the data model too.
<wgrant> The data model isn't changing, but the per-builder scanners in buildd-manager will pull from an in-memory queue rather than directly from the database.
<wgrant> So we'll pull the current queue for each (arch, virt) once every 15 seconds, rather than once for every builder every 15 seconds.
<wgrant> So a no-op scan cycle should be quick and not hit the DB at all
<cjwatson> Ah, right, I can see how that would break that query then.
<wgrant> It might not break the query itself, but it makes it harder to reason about.
<wgrant> Because a build can be disqualified based on the other builds that might be dispatched during that round.
<wgrant> I really should also relax the delay for private builds.
<wgrant> That bit of the query is relatively slow, and avoidable now we have the public restricted librarian.
<lifeless> should really do the url mangling bugfix for that ;)
<wgrant> Indeed, eventually :)
<cjwatson> wgrant: Do you agree with my comment in the MP that it can probably be no-qa?
<wgrant> cjwatson: Certainly.
<cjwatson> OK, cool
<stub> Anyone up for the usual thingami ?
#launchpad-dev 2013-09-05
<wgrant> StevenK: à² _à² 
<StevenK> Unicode failure
<StevenK> But I can guess the intent
<wgrant> 1) Fix your IRC/terminal configuration to be less broken when receiving Unicode
<wgrant> 2) Fix your MP :)
<StevenK> [14:33] -!- Irssi: Uptime: 99d 4h 39m 41s
<StevenK> irssi/screen loses its mind after about 40 days WRT Unicode
<wgrant> A likely story.
<StevenK> wgrant: Fix my MP how?
<wgrant> à² _à² 
<StevenK> wgrant: I seem to be missing my mind reading device. Secondly, did you manage to actually break the new auditor stack or did you get fully distracted by buildd-manager?
<wgrant> I only à² _à²  without elaboration in a very restricted set of circumstances
<wgrant> I'd examine your diff for obvious security vulnerabilities.
<StevenK> wgrant: I can think of two. person may not be a JSON blob of a IPerson, and we should escape the comment
<wgrant> + suffix = '-' + person.name;
<wgrant> + '<a href="' + person.web_link + '">' + person.display_name +
<wgrant> + ' (' + person.name + ')</a>');
<wgrant> + header = "Comment by " + personlink + " on " + date + ".";
<wgrant> + '<tr id="ict-' + middle + '-header" class="ict-header">' +
<wgrant> + '<div id="inlinecomment-' + middle + '">' +
<wgrant> + '<tr id="ict-' + middle + '"><td></td><td>' +
<wgrant> + '<span>' + comment + '</span></td></tr>');
<wgrant> At a quick glance.
<StevenK> I wasn't aware of a person formatter available to JS
<wgrant> There probably isn't one.
<wgrant> But the lack of one does not excuse multiple trivial XSS vulnerabilities.
<wgrant> Though existing code such as +sharing and the bug subscription widget generate person links somehow.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1221002/+merge/184030 might make SlaveScanner.scan() a bit more understandable.
<wgrant> And even less inconsistent :)
<wgrant> checkCancellation is still sick and wrong, but that's for another branch
<wgrant> I suspect I'll need to add a cancellation flag to BuildQueue
<wgrant> It's currently all rather messy, as the top level buildd-manager code knows about build.status.
<StevenK> wgrant: Sorry, I'll look tomorrow morning.
<wgrant> StevenK: Sure, no rush
#launchpad-dev 2013-09-06
<stub> https://code.launchpad.net/~stub/launchpad/mock-swift/+merge/178047
<stub> New python-gnugpg release last weekend
#launchpad-dev 2013-09-07
<rharish> hello , i'm new to launchpad and wish to contribute through bug-fixing
<rharish> can anyone help ??
<rharish> my previous open source contribution was a debian package :)
#launchpad-dev 2014-09-03
<bigjools> thought it was too good to be true - LP session cookie finally invalidated after over a year
<wgrant> I changed the session storage format, so had to wipe them all.
<bigjools> how many api scripts will that break.... hahga
<wgrant> bigjools: None. OAuth tokens were unaffected.
#launchpad-dev 2015-08-31
<cjwatson> wgrant: Have you had a chance to figure out how to QA https://bugs.launchpad.net/launchpad/+bug/1489674 ?
<mup> Bug #1489674: BugNotificationBuilder.build crashes with Unauthorized on invisible private teams <403> <fallout> <qa-needstesting> <regression> <Launchpad itself:Fix Committed by wgrant> <https://launchpad.net/bugs/1489674>
<wgrant> cjwatson: Oh yeah, I should try that.
<wgrant> Let me see.
<cjwatson> wgrant: Thanks
<cjwatson> blr: Were you going to land https://code.launchpad.net/~blr/launchpad/diff-code-select-bug-1483925/+merge/268017 ?
<blr> cjwatson: oh, forgot about that. Yes I should do
<cjwatson> I figured that was probably buried in flu.
<blr> thanks cjwatson
<cjwatson> wgrant: A quick test suggests that there are various BaseMailer call sites that could trigger this too, for example modifying a branch in the web UI that has a subscription from an inaccessible private team, so I think my first job tomorrow is going to be figuring out a central fix for that.
<cjwatson> Or as central as possible.
<wgrant> cjwatson: Sounds reasonable. I don't know how central it can be, but it needs investigation.
<cjwatson> Maybe BaseMailer needs to grow a rule that all uses of it need to be zopeless-style.  Fixing this "centrally" would still involve sprinkling rSP over all the various constructors.
<wgrant> Zopeless-style?
<cjwatson> Jobs, scripts, or whatever.  Not with a user interaction.
<cjwatson> Whatever the correct term is.
<wgrant> Ah, right. The defining feature is PermissiveSecurityPolicy.
<wgrant> But that makes it rather difficult to use from the large number of email callsites that aren't presently asynchronous, unfortunately.
<cjwatson> That's the one, thanks.
<cjwatson> Making those async would not be too hard nowadays.
<cjwatson> I'll have to see whether it's worth it.
<cjwatson> I always thought sending mail directly from something like a page rendering operation was a bit odd, though.
<wgrant> Oh, it would certainly be a positive change.
<wgrant> The question is how much work it is to fix.
<cjwatson> I could stick an assert getSecurityPolicy() == PermissiveSecurityPolicy into BaseMailer and see how much of the test suite explodes.
<wgrant> All of +queue, for one.
<cjwatson> From a quick scan I think the currently affected mailers are Branch, BranchMergeProposal, GitRepository, and PackageUpload.  All the rest are only used from jobs or scripts.
<wgrant> oh right, answers is async nowadays, isn't it.
<cjwatson> Answers doesn't use BaseMailer (yet).
<wgrant> oh
<cjwatson> But I think its mail notifications are indeed async.
<cjwatson> Bugs notifications are mostly async, except for the send_bug_details_to_new_bug_subscribers case you ran into.
<wgrant> If by "mostly async" you mean "lies"
<wgrant> The footer is calculated in the webapp.
<wgrant> Very, very expensively.
<cjwatson> Right, but only in the new-bug case.
<wgrant> Most bug edit timeouts go away if we make notification sending properly async.
<cjwatson> Er, new-bug-subscriber.
<wgrant> Nope.
<wgrant> The webapp populates BugNotificationRecipient
<cjwatson> Oh, yikes.
<wgrant> So the POST request calculates the set of people who should be notified and inserts a row for each of them, including the rationale text and footer etc.
<wgrant> The correct solution is of course a job that says "this change happened to the bug when it was in this state, now please populate bugnotification"
<cjwatson> I'll have a look at the BaseMailer cases tomorrow at least.  Would like to spend finite time in this swamp, but draining a bit of it is a prereq for landing team-mail safely
<cjwatson> (I strongly suspect)
<cjwatson> Anyway, bed.  Night
<wgrant> Night.
<wgrant> You weren't even here today, were you?
<cjwatson> Nope.  And mostly actually not, just nipping in briefly
<wgrant> If you can make it all PermissiveSecurityPolicy then that indeed makes things substantially simpler.
<cjwatson> Code has a suitable job framework already, so probably easy.  Soyuz might need a bit of fiddling.
<wgrant> Yup
<wgrant> Though there's actually already a bug about the Soyuz one.
<wgrant> https://bugs.launchpad.net/launchpad/+bug/566339
<mup> Bug #566339: Cannot accept package which would notify private email addresses <boobytrap> <email> <lp-soyuz> <notifications> <Launchpad itself:Triaged> <https://launchpad.net/bugs/566339>
#launchpad-dev 2015-09-02
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/team-mail/+merge/269382 could use another look; as well as the obvious changes to fix your review comments, I converted it to use async jobs to make sure that there are no security proxy problems with this branch.  (Might be easiest to look at just r17707.)
<cjwatson> And now to bed.
<wgrant> cjwatson: Thanks, those X-L-N-Ts make a tonne more sense now.
<wgrant> Night.
<cjwatson> PersonTransferJob is kind of a weird home for this, but it already had membership status change notifications ...
<wgrant> Yep
<cjwatson> wgrant: Was there a particular reason you preferred ArchiveJob over DistributionJob for package upload mail?  AFAIK neither is really very relevant - the values saved for the job would be PackageUpload.id, PackageUploadStatus, and summary_text
<cjwatson> So it's maybe more modern-feeling but I'm not seeing how it would actually matter, either would be just for expediency
<cjwatson> I guess it sort of categorises them, and it's as good as anything.
<wgrant> cjwatson: I simply have a great distaste for anything Soyuz-related talking about distributions when the thing is in fact about an archive.
<wgrant> See silly things like Distribution.getPublishedSources
#launchpad-dev 2015-09-03
<marcoceppi> launchpadlib and lazr.restfulclient 0.13.4 seem broken in vivid
<marcoceppi> http://paste.ubuntu.com/12265175/ thsi command runs fine in trusty with 0.13.3-1 build of python-lazr.restfulclient
<marcoceppi> https://bugs.launchpad.net/lazr.restfulclient/+bug/1491971
<mup> Bug #1491971: charm promulgate fails with lazr.restfulclient (0.13.4) <Juju Charm Tools:Won't Fix> <lazr.restfulclient:Confirmed> <https://launchpad.net/bugs/1491971>
<cjwatson> marcoceppi: I use it all the time in vivid.  What does "ls /usr/lib/python2.7/dist-packages/lazr/" say?
<marcoceppi> cjwatson: it works on my fresh vivid, i'm working with two users on two systems who reported it to charm-tools, let me drag them in here
<cjwatson> note it's my evening, can't guarantee to be around synchronously
<marcoceppi> cjwatson: thanks, I'm talking to them now
<cjwatson> I suspect this is namespace packages ftl
<cjwatson> /usr/lib/python2.7/dist-packages/lazr/__init__.py is probably missing for some reason
<marcoceppi> cjwatson: http://paste.ubuntu.com/12265709/
<cjwatson> hm, no
<marcoceppi> had him do a --reinstall, no avail
<cjwatson> afraid I don't immediately know and I have to go.  I'd try getting an strace and see what it's actually trying to load
<marcoceppi> cjwatson: ack, ta
<marcoceppi> I may have them do a purge then re-install
<cjwatson> get an strace first, don't destroy the evidence
<marcoceppi> cjwatson: sure, ack, will do
#launchpad-dev 2015-09-05
<costello> Any gnu gettext gurus in here? "info gettext" very politely documents call "dpgettext(..)" (a variant with context) but headers in /usr/include have no declaration and linker refuses to link -> is there working way to include context string into string lookups if using gettext version 0.19.4-1 ?
<cjwatson> costello: They're only in gettext.h, not in <libintl.h>; as per 'info gettext lib/gettext.h', you're meant to copy that into your project.  If you're already using gnulib then it's in the "gettext-h" module there.
#launchpad-dev 2015-09-06
<costello> cjwatson thanks, yes, the fact that it is under /usr/share and not /usr/include was a bit misleading. .. should I really make copy of the file or say #include "/usr/share/gettext/gettext.h" .. say, now it is #defined to  npgettext_aux and if that detail of internal implementation is to change in the future, then..
<cjwatson> costello: You should follow the directions in the gettext documentation, which say to copy it
<cjwatson> costello: npgettext_aux is a static function declared in that same header file
<costello> Ok, getgext.h is gpl. my package is lgpl and can't be gpl due to openssl licensing stupidities -> does this pose a problem?
<cjwatson> costello: No idea, ask the gettext maintainers
<costello> If I'm supposed to copy that. And that way also distribute it.
<costello> yes.
<cjwatson> I imagine you could just do the same things with dcgettext/dcngettext that it does
<cjwatson> That's the actual important function, the rest is just syntactic sugar over the top
<costello> y. better idea.
<vila> hmm, something is weird with  https://code.launchpad.net/~vila/bzr/1451448-skip-racy-tests
<vila> launchpad doesn't show the diff even after way longer than usual (not a metric ;)
<vila> bzr push --overwrite after 1 hour says 'No new revisions or tags to push' so the data is there
<cjwatson> vila: run this in "lp-shell production devel": lp.branches.getByUrl(url="lp:~vila/bzr/1451448-skip-racy-tests").unscan(rescan=True)
<vila> cjwatson: lp-shell ? never heard about that before
<cjwatson> vila: (LP's bzr branch modelling is terrible and requires inserting a row for every revision in the branch's history when you push a new branch, so it often times out for large histories)
<cjwatson> vila: it's in lptools
<cjwatson> basically just a Launchpad.login_with wrapper, but convenient
<cjwatson> vila: if the history isn't there after six minutes (technically five but let's make sure not to overlap) after you attempt the rescan, try it again, repeat until it actually manages to scan the thing
<vila> cjwatson: ack, ran
<vila> cjwatson: but well, it's 2 or 3 commits on top of trunk, I doubt we were close the limit before that...
<cjwatson> vila: it depends how hot things are in the DB (or something)
<cjwatson> vila: and this is not the first bzr branch that has timed out this way, by any means
<vila> cjwatson: ack, thanks for the feedback anyway ;)
<cjwatson> vila: it may happen a bit less often for bzr than it does for LP itself, but we've had a number of other reports
<vila> cjwatson: but the row has to be inserted for any bzr branch right ? Err, bzr the tool, not the project. I.e. every lp project using bzr share the same db table right ?
<cjwatson> they all use BranchRevision, yes
<cjwatson> but the number of rows varies wildly
<vila> so bzr the project itself shouldn't be more affected than any other project no ?
<cjwatson> it is, because it has quite a deep history
<vila> cjwatson: oh, the cache...
<vila> cjwatson: anyway, it worked ! Thanks a lot
<cjwatson> no, not the cache
<vila> cjwatson: so little activity on bzr that the revs are not cached anymore ?
<cjwatson> <cjwatson@niejwein ~/src/bzr/bzr/bzr.dev>$ bzr log | grep --count '^ *revno:'
<cjwatson> 6609
<cjwatson> you don't understand how terrible the LP data model is here
<cjwatson> if you push a new branch of bzr based on the above, it will insert 6609 + (something small) rows
<cjwatson> instead of, say, (something small)
<cjwatson> and it turns out that trying to insert thousands of rows is slow
<vila> O_o even for already known revisions ?
<cjwatson> yes
 * vila heart stops
<cjwatson> it's revision per branch
<cjwatson> like I say: terrible
<vila> I've heard stories about that table since... barcelona 2009 ? But I never realize there was revnos per *branch* :-(
<cjwatson> unrolling the whole graph into the DB in an incredibly inefficient way
<vila> cjwatson: say no more ;-}
<cjwatson> we avoided repeating this mistake for the git data model, obviously
<vila> well, you don't revnos in git anyway ;)
<cjwatson> it's not revnos anyway, it's revisions
<cjwatson> i.e. not just the left-hand side
<cjwatson> LP itself is at revno 17709, but every new LP branch we push involves inserting well over a hundred thousand rows
<cjwatson> it is very annoying
<vila> bzr has revids, what you mention seems related to revnos, i.e. for the same revid, tracking all the revnos in the branches
<cjwatson> err
<cjwatson> no it's not
<cjwatson> it's branch * revision (revid if you like), it's not relevant if they share revnos in many cases
<cjwatson> and it's not an attempt to track revnos, it's more like a performance-terrible implementation of merge detection
 * vila rollbacks
<vila> ok, so you mean for each branch (irrelevant of whether revisions are shared via repositories), a list of *all* revisions is maintained and used to detect merges ?
<cjwatson> yes
<vila> and this doesn't try to use the bzr code that do the same but knows how to load only part of the graph ?
<cjwatson> possibly other reasons
<vila> yeah, ok, nm ;)
<cjwatson> I don't think there's any point analysing exactly why it got that way TBH
<cjwatson> I believe it's trying not to use the bzr code in question because that would require a callout to a system that actually has the branches in question
<cjwatson> merge detection is done just with the LP database
<cjwatson> that said, for the git implementation, we said screw that, we just call out to the git hosting system and ask it to do merge detection
<cjwatson> much easier
<cjwatson> William had part of a BranchRevision 2.0 implementation at one point, but then we did git instead
 * vila nods
<wgrant> To be fair, at the time the code was devised, bzr LCA detection wasn't exactly fast.
#launchpad-dev 2016-09-06
<xnox> do builds use http://wiki.qemu.org/Features-Done/VirtIORNG ?
<xnox> last time libgnupg-perl was built in quantal in 8 minutes, now it is stuck in gpg key generation for the test-suite. I wonder if i should pregenerate a test key for it.
<wgrant> xnox: virtio-rng is not enabled on the scalingstacks. The compute hosts don't have hardware RNGs and guaranteeing QoS would be complicated. Packages should use /dev/urandom for test keys (but gpg --gen-key cannot be convinced to do that directly, unfortunately).
<xnox> wgrant, are they new enough intel cpus with intel cpu rng available at all? I did fix rng-tools ages ago to enable that.
<xnox> if the build times out, i'll ship a pregenerated key i guess.
<wgrant> xnox: RDRAND is new in Ivy Bridge, (ie. Xeon E.* v2). Most of the Intel compute nodes are Westmere-EP, with a handful of Ivy Bridge-EP and a couple of Haswell-EP, plus a smattering of Opterons. So maybe 10% of the x86 compute nodes have a hardware RNG.
<wgrant> (ignoring TPM1.2, but, well, lol)
<xnox> i'm yet to manage to get TPM1.2 to work....
<xnox> i keep hearing that it's like all cool and dandy and available....
 * xnox ponders if entropy keys should be plugged into all compute nodes and how much that would cost.....
<wgrant> Cool, dandy, available and slow.
<xnox> wgrant, http://altusmetrum.org/ChaosKey/ http://shop.gag.com/random.html
<xnox> is $30 a pop expensive?
<wgrant> Not dreadfully.
<wgrant> But I'd really prefer that someone took a hammer to GnuPG upstream.
<xnox> i trust bdale more than GnuPG upstream.....
<xnox> wgrant, please kill https://launchpad.net/ubuntu/+source/libgnupg-perl/0.19-1ubuntu1/+build/10713491
<xnox> wait, these are not good old days, i can do that myself.
<wgrant> Yup :)
<xnox> man.... can't wait for the UDS
<cjwatson> Do we need to bring camouflage?
#launchpad-dev 2017-09-04
<cjwatson> wgrant: Could you either release python-oops-twisted, or give me PyPI upload access so that I can?
<wgrant> cjwatson: Added
<cjwatson> thanks
<sil2100> Hello! Does anyone know if LP: #1714960 is a known issue? If yes, is there some easy workaround?
<mup> Bug #1714960: getArchiveSubscriptionURL() crashes on python3 <launchpadlib :New> <https://launchpad.net/bugs/1714960>
<cjwatson> requires trunk of lazr.restfulclient / launchpadlib / both
<cjwatson> (not sure exactly)
<sil2100> Any ETA when those changes will be released to the distro? :)
<sil2100> Since I guess I could work on a local checkout (since it's just me and Steve using this kernel tool) but I'd prefer if it didn't crash for others too
<cjwatson> It's been on my to-do list forever.  Let me see if I can figure out how to run the necessary tests.
<cjwatson> Hm, that's actually still buggy in trunk.
<cjwatson> wgrant: Could I have lazr.restfulclient PyPI access too, please?
#launchpad-dev 2017-09-05
<wgrant> cjwatson: done
