#launchpad-dev 2009-09-07
<thumper> mwhudson: LP's bzr-git is missing 44 revisions from trunk :(
<mwhudson> thumper: !?
<mwhudson> thumper: time for an upgrade i guess?
<thumper> yes, I think so
<thumper> mwhudson: does that sound like a build engineer task?
<mwhudson> thumper: not especially...
<mwhudson> otoh, i'm not actually doing much else now
<thumper> :)
<thumper> mwhudson: don't we just have to push it through ec2 with a different bzr-git?
<thumper> mwhudson: something like that?
<mwhudson> thumper: i guess landing the branch will still run all the tests in pqm
<thumper> mwhudson: does it?
<thumper> well
<thumper> pqm isn't doing much else right now
<mwhudson> this is true
<thumper> although we will want to land a fix for the registry breakage
<thumper> mwhudson: I'm going to fix the failing test
<thumper> mwhudson: I may get you to review the expected one line fix
<mwhudson> thumper: okidoke
 * thumper makes
 * thumper taps fingers while make makes
<mwhudson> oh yay, jelmer rebased dulwich trunk again
<spm> mwhudson: didn't we register a bug against that particular problem? ;-)
<mwhudson> yeah
<mwhudson> (in Quotes)
<spm> "where" doesn't matter so long as "did"
<thumper> mwhudson: http://pastebin.ubuntu.com/266368/
<thumper> mwhudson: it fixes the failure
<mwhudson> thumper: +1, with a sigh on the side
<mwhudson> thumper: please submit it as a [testfix] now
 * thumper nods
<thumper> mwhudson: submitted
<mwhudson> thumper: thanks
<jelmer> mwhudson: hmm?
<jelmer> mwhudson: I haven't touched dulwich in a while
<mwhudson> jelmer: we haven't updated the version we used in even longer it seems :)
<jml> jelmer, hello
<jml> mwhudson, spm, thumper, et al: hi
<jelmer> hey jml
<spm> hey jml
<jml> all of my discipline in waking up on time has vanished with my 8am daily call :)
<mwhudson> jml: oops
<mwhudson> jml: adapting yourself to london time, one hour a day?
<jml> mwhudson, heh
<jml> mwhudson, how's buildbot looking?
<mwhudson> jml: reasonably terrible, no worse than usual
<mwhudson> jml: it's building, at least
<jml> mwhudson, that's something, I guess.
<mwhudson> we'll all be surprised if the build doesn't fail though
<jml> mwhudson, I did a test run last night on devel -- there are failing tests
<jml> mwhudson, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/425113 might interest you
<mup> Bug #425113: Some tests can fail without detection <build-infrastructure> <Launchpad Foundations:New> <Twisted:Unknown> <https://launchpad.net/bugs/425113>
<mwhudson> jml: which ones?
<jml>   lib/lp/registry/browser/tests/productseries-views.txt
<mwhudson> jml: oh god
<mwhudson> (the bug)
<mwhudson> jml: thumper fixed that one already
<jml> glad to hear it :)
<jml> mwhudson, yeah, the bug is going to require some patches to Twisted, I think.
<mwhudson> though i don't know where his fix went
<mwhudson> thumper: did your submission fail?
<thumper> mwhudson: :(
<thumper> yes
<thumper> forgot the ui tag
<mwhudson> hee
<thumper> as it isn't needed in normal testfixes
<thumper> but is for fake test fixes
<jml> thumper, btw, bug 425399 is yet another bug in the cluster of "tighter review integration"
<mup> Bug #425399: too many clicks to get to important review <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/425399>
 * thumper nods
<thumper> jml: I saw it
<jml> cool.
<jml> thumper, btw, something interesting came up on the weekend -- would be keen to chat about it with you
<thumper> jml: ok, heating pizza now, how about in 10 minutes?
<jml> thumper, sounds good :)
<thumper> jml: if you don't mind hearing me munching, we can talk now
<jml> thumper, cool
<MTeck> jml: looks like I lost a group member - but I think we can still do the project just fine
<MTeck> jml: I've been thinking about it more
<jml> MTeck, cool
<jml> MTeck, how many does that leave you with?
<MTeck> jml: 3 total
<jml> MTeck, ahh, that's plenty :)
<MTeck> jml: should be plenty for it
<MTeck> ya :P
<jml> MTeck, so you've been thinking about it?
<MTeck> I'm planning on it
<mwhudson> jml, spiv, thumper: https://pastebin.canonical.com/21867/ (for bug 424136)
<mup> Bug #424136: Cannot upgrade stacked branches from 1.9 to 2a on Launchpad <branch-puller> <Launchpad Bazaar Integration:Confirmed> <https://launchpad.net/bugs/424136>
<mwhudson> hm, why did i use canonical pastebin?
<mwhudson> habit i guess
<mwhudson> http://pastebin.ubuntu.com/266396/
<thumper> mwhudson: extra carriage return between tests, but other than that r=thumper
<mwhudson> oh yeah
<mwhudson> guess i should fix the lint too
<mwhudson> thumper: thanks
 * mwhudson lunches
<spiv> mwhudson: looks good at a glance
<jml> mwhudson, looks good to me.
<MTeck> jml: I was thinking - I could probably make my own specs easily enough - any chance you'd prefer do it? I know you mentioned something about talking to people this week.
<jml> MTeck, I'm not quite sure I follow.
<jml> MTeck, I reckon that any specs you'd do would have to be a collaborative effort between you & whoever on the team was looking after that area
<MTeck> jml: I just meant however you guys would expect it to work
<MTeck> I'm guessing a lot like SSH keys and another drop down in the bug report
<jml> MTeck, yeah, it'd probably be something like that.
<MTeck> jml: sorry - I just want to make sure you guys get what you want - but when I think about it more, it's pretty hard to not make it fit in sensibly
<jml> MTeck, heh
<jml> MTeck, nothing to apologize about. :)
<jml> MTeck, I guess it's probably a good idea to send through a rough spec & some notes on the project to the launchpad-dev list
<jml> MTeck, don't spend too much time on it though.
<MTeck> yup - I'll enjoy this project
<jml> since discussion will definitely follow :)
<MTeck> jml: any issues with me using the LP Wiki?
<jml> MTeck, none at all.
<jml> unless it's locked :)
<MTeck> ok - I'll do it all there then - then you can all take a peak
<jml> thumper-afk, lp:~cprov/launchpad/bug-424797-buildd-manager-test-failure  â lp:launchpad/devel  is appearing on my personal +activereviews page.
<jml> thumper-afk, under "Approved to land"
<jml> thumper-afk, I'm not sure it should
<MTeck> jml: so - https://dev.launchpad.net/Bugs/UpstreamForwarding sound like a good url for us to use?
<jml> MTeck,
<jml> yes.
<jml> mwhudson, when everything is settled, could you please review a branch for me?
<mwhudson> jml: request away
<jml> https://code.edge.launchpad.net/~jml/launchpad/ec2test-karmic-bug-424197/+merge/11265
<jml> yay ajax bugs
<mwhudson> that was easy
<jml> mwhudson, I'm full of easy patches these days :)
<MTeck> I tried to do 'bzr push bzr+ssh://bazaar.launchpad.net/~mteck-devs/launchpad/UpstreamBugs/' but it didn't like me :(
<jml> mwhudson, it turns out that there's a fairly big number of useful easy things to do.
<jml> MTeck, what was the error?
<jml> mwhudson, thanks.
<MTeck> http://pastebin.ubuntu.com/266417/
<jml> MTeck, upgrade your branch to 2a
<jml> MTeck, 'bzr upgrade --2a' should do it.
<jml> (bzr ought to give you a way to push without stacking if the remote host has a defaut stacking branch set, but it doesn't)
<MTeck> You guys just completely lost me
<MTeck> stacking and --2a... lost
<jml> MTeck, ahh, I see.
<jml> MTeck, they are both implementation details of bzr
<jml> MTeck, bazaar branches have formats. normally, this doesn't matter, since bzr is normally able to interoperate between different format branches.
<jml> MTeck, one such format is '2a'. It has a feature that makes it incompatible with the default format.
<jml> MTeck, this feature is called "rich root support"
<jml> MTeck, Launchpad branches are all in the 2a format, since it's going to be the new default format.
<jml> MTeck, 'stacking' is a technical term for a particular way that one branch can share data with another branch.
<jml> MTeck, Launchpad has settings that hint to Bazaar that branches pushed to a project should be stacked on the trunk branch of the project.
<jml> MTeck, We do this to save us space and you time on the network.
<jml> MTeck, Normally it's just an implementation detail that you don't need to care about.
<MTeck> oh
<jml> MTeck, but in this case, you are trying to stack your branch on a branch in an incompatible format.
 * MTeck keeps learning more and more about bzr
<wgrant> How was a non-2a Launchpad branch obtained?
<jml> MTeck, so Bazaar gives you an error.
<MTeck> --2a made it all better :)
<jml> MTeck, Bazaar could do better at this, but no one has had the combination of motivation, time and energy.
<jml> MTeck, I told you it would :)
<MTeck> so, the stacked piece is the part of the project part in ~team/project/branch ?
<jml> not quite.
<MTeck> am I close?
<jml> if a project has a development focus set, then branches will be stacked by default on that branch.
<MTeck> oh - over 24hr no sleep now :P
<jml> so, in Launchpad's case, lp:launchpad
<MTeck> oh!
<MTeck> jml: thanks for teaching me more yet :)
<MTeck> I'm curious - how do you have so many people developing LP but keep conflicts down?
<wgrant> 30 devs over Python 500KLOC isn't very large.
<MTeck> kloc?
<MTeck> oh...
<MTeck> k loc :P
<spiv> kLoC ;)
<MTeck> So - how should I start developing on this piece
<jml> MTeck, tbh, I would look for an easy, unrelated bug to fix and do that first.
<mwhudson> the main weapon in the fight against conflict is short lived branches
<jml> https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.tag=trivial&field.tags_combinator=ANY
<MTeck> that's a lot
<jml> MTeck, you don't have to do all of them :)
<jml> (although it'd be nice)
<MTeck> :P
<wgrant> I wish that would tell me what criteria it used, although it's easy enough to work out.
<jml> mwhudson, I'm not sure whether "short lived" or "single purpose" is more important.
<mwhudson> jml: "small, in some sense"
 * mwhudson displays his mathematical background
<jml> heh
<jml> wgrant, https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.tag=trivial&field.tags_combinator=ANY
<jml> sorry
<jml> wgrant, https://bugs.edge.launchpad.net/malone/+bug/325982
<lifeless> Watch out, LINKS INCOMING
<mup> Bug #325982: Search URL is long and hard to paste <trivial> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/325982>
<lifeless> oh the irony
<wgrant> jml: Yep.
<MTeck> It's confusing that there's not just one branch that has all the code needed to develop
<wgrant> It's much better than the alternative (edge not updating for half of the month)
<MTeck> Isn't bzr supposed to be coming up with sub-branches so you can have one big branch with minor branches below it?
<wgrant> Oh, you're talking about the deps?
<MTeck> I'm not sure - it was metcalfe that mentioned it to me
<jml> I'm going off IRC for a while to try to focus a bit better. ping me via skype or xmpp if you need my attention.
<MTeck> http://bazaar-vcs.org/NestedTreeSupport
<MTeck> jml: ^ This is what I was thinking of
<MTeck> and he goes away so I can't annoy him :P
 * mwhudson stabs the aws console through the lungs
<MTeck> that brings offtopic thoughts to mind (annoying people)
<thumper> jml: if you reviewed it, it is going to show (now)
<thumper> jml: as your personal active reviews page shows the reviews you need to do
<thumper> jml: and are doing
<thumper> jml: would you expect to see it drop off?
<mwhudson> jtv: do you know anything about this bug? https://bugs.edge.launchpad.net/launchpad-foundations/+bug/47511
<mup> Bug #47511: pagetests add ghost new lines to textareas <build-infrastructure> <Launchpad Foundations:Triaged by bjornt> <https://launchpad.net/bugs/47511>
<mwhudson> jtv: carlos reported it, which is why i'm asking you
<jtv> mwhudson: ohhh
<jtv> that sounds like it's related to ye aulde problem of some browsers adding newlines at the beginning/end of what's in a textarea.
<jtv> mwhudson: have you tried running the test he attached?
<mwhudson> jtv: it looks a bit old school, i don't know if it will still run
<mwhudson> i guess i should just tyr
<jtv> You _may_ also have to disable whitespace normalization...
<jtv> mwhudson: the test looks old-school, but prima facie I don't see anything that shouldn't work any more
<kfogel> mwhudson: on devpad:
<kfogel>  bzr branch lp:~kfogel/+junk/lpcc trunk
<kfogel> Permission denied (publickey).
<kfogel> bzr: ERROR: Connection closed: Unexpected end of message. Please check connecti\
<kfogel> vity and permissions, and report a bug if problems persist.
<kfogel> mwhudson: this is something I must have neglected to set up on devpad, but what?  a keyfile somewhere?
<mwhudson> kfogel: agent forwarding?
<kfogel> mwhudson: er, do I need that to check out a public branch?
<mwhudson> kfogel: over ssh, yes
<mwhudson> kfogel: and you must have run bzr launchpad-login at some stage on devpad
<kfogel> mwhudson: maybe.  but my command is "bzr branch lp:..."
<kfogel> why would that do ssh, or at least, why wouldn't it fall back to http??
<sinzui> How do I solve this build problem:
<sinzui>     Getting distribution for 'zc.buildout==1.5.0dev-gary-r103553'.
<wgrant> sinzui: Is your download-cache up to date?
<sinzui> I claims to be for the last 48 hours
<spiv> kfogel: it would do ssh if you've previously run bzr launchpad-login (because it then assumes that ssh will work, and ssh is almost certainly faster because of the smart server).
<wgrant> What's the actual error?
<mwhudson> kfogel: bzr doesn't know that the ssh connection isn't going to work
<sinzui>   Bootstrapping.
<sinzui>   Getting distribution for 'zc.buildout==1.5.0dev-gary-r103553'.
<sinzui> Error: Couldn't find a distribution for 'zc.buildout==1.5.0dev-gary-r103553'.
<sinzui> Getting distribution for 'zc.buildout==1.5.0dev-gary-r103553'.
<sinzui> make: *** [bin/buildout] Error 1
<sinzui> make: Leaving directory `/home/curtis/Work/launchpad/rocketfuel'
<spiv> kfogel: it doesn't fallback because bzr doesn't really make it easy for that to work :(
<mwhudson> kfogel: it could catch and retry i guess, but the logic is all a bit upside down for that to be easy
<kfogel> mwhudson: it certainly knows that it fails :-).  But anyway, I'm not sure what the fix is.  I just need to check out some bzr branches.
<mwhudson> kfogel: log out, ssh -A devpad
<kfogel> mwhudson: yeah, catch and retry is what I was expecting I guess.
<kfogel> spiv: ^^
<kfogel> mwhudson: thanks
<kfogel> spiv: too bad about the code arrangement :-(
<sinzui> download-cache/dist is empty again. bzr says nothing is wrong
<mwhudson> sinzui: bzr up download-cache fixed that for me
<mwhudson> sinzui, kfogel: what the heck are you guys doing onine
<jml> thumper, yes, I'd expect it to drop off, since I can't do anything about it, and I don't really have responsibility for it.
<sinzui> download-cache
<sinzui> Tree is up to date at revision 9354.
<wgrant> bzr st
<mwhudson> sinzui: that's wrong
<jml> thumper, otoh, I guess if it were a patch from someone who can't land, then it should really appear there
<kfogel> mwhudson, spiv: the "ssh -A" worked, thanks.
<kfogel> mwhudson: I just wanted to get a cron job going before I went to bed.
<jml> thumper, which is starting to sound maybe a little too intelligent
<mwhudson> there's no way download-cache is at rev 9354
<thumper> jml: :)
<jml> thumper, what do you think?
<jml> kfogel, hi
<kfogel> jml: hey!
<mwhudson> kfogel: use an http url?
<wgrant> mwhudson: A very good point.
<jml> MTeck, yeah, bzr should have nested trees
<kfogel> mwhudson: mmm, I could have constructed that, yeah
<jml> MTeck, atm, Launchpad uses three ways to manage dependencies
<kfogel> mwhudson: but I wanted my strawberries and cream and "lp:" goodness
<kfogel> jml: https://dev.launchpad.net/Contributions
<jml> kfogel, yeah, I saw :)
<kfogel> jml: (as per our discussion)
<kfogel> jml: ah good
<jml> kfogel, oh, my privmsgs got silently lost
<jml> :(
<wgrant> That's a pretty depressing page.
<kfogel> wgrant: it is a bit lopsided :-)
<jml> kfogel, I love the new ToC
<kfogel> jml: yay freenode
<jml> wgrant, depressing for whom exactly?
<kfogel> jml: cool.  we under-use the "<<Anchor(foo)>>" syntax in moin I think.
<wgrant> jml: Everyone. There are practically no contributions.
<jml> wgrant, it's way more than I was anticipating.
<kfogel> wgrant: a month or so after the first release of a difficult-to-build hosted service?  I wish for more contributions too, but this isn't bad.
<kfogel> wgrant: remember maxb is in the pipeline now
<wgrant> kfogel: Mm, I guess.
<wgrant> That's true.
<jml> wgrant, but I'm very keen to see more contributors.
<jml> wgrant, got any ideas for how we could get more?
<wgrant> jml: Not restrict everything to Ubuntu.
<wgrant> Debian would be easy, but PPAs don't do Debian.
<kfogel> wgrant: +1 +1 +1
<kfogel> I completely agree.
<mwhudson> a bug!
<kfogel> Debian, and then other distros too.
<kfogel> wgrant: our dependency setup is our Achilles' heel.
<wgrant> jml: I count four types of deps.
<wgrant>  - Debian packages
<wgrant>  - sourcecode
<wgrant>  - bzr checkouts
<wgrant>  - eggs
<wgrant> Hm. Wait. Maybe I split something there.
<wgrant> They confuse me, anyway.
<jml> wgrant, 'bzr checkouts'?
<jml> wgrant, sourcecode, debs & eggs -- that's it, I think.
<wgrant> jml: Mm, right, the bzr checkouts are sourcecode.
<jml> wgrant, it's confusing
<jml> and I don't see us dropping to one kind any time soon
<jtv> mwhudson: still working on patching up that test
<wgrant> Has anybody tried to run LP on Debian or other derivatives yet?
<kfogel> wgrant: I thought someone had...
 * kfogel looks
<mwhudson> jtv: ok, don't sweat it too much
<wgrant> kfogel: I know somebody was considering trying it some weeks ago.
<kfogel> wgrant: oh, no, no record of success yet
<wgrant> But I don't recall hearing any results.
<jtv> mwhudson: the holdup is setupBrowser not being defined.
<jtv> Ahhh, stupid.  Got the trouble
<jtv> fixed
<ajmitch> wgrant: I see you've managed to get quite a few branches landed, well done
<jml> ajmitch, your turn :)
<wgrant> ajmitch: I've about another five sitting around, too.
<ajmitch> jml: Once I replace my dying laptop, I'll be able to look at it again
<wgrant> kfogel: How do you determine the author? Looking at the MP, or the latest revision, or something even more inventive?
<wgrant> Ah, there is code.
 * wgrant looks.
<mwhudson> jtv: does the test pass?
<jtv> mwhudson: still patching it up...  having to set up layers for every run is the big bottleneck
<kfogel> wgrant: one thing that hurts is that I can't (yet) determine the person who made the landing request.  That's due to everything looking like ~launchpad-pqm; we should find someplace for PQM to hang that information, and have the script look there.
<wgrant> kfogel: Right.
<wgrant> (and we should stop sticking metadata in revision messages!)
<kfogel> jml: okay, cron job on devpad updates the Contributions page every 10 min now.
<jml> kfogel, sweet.
<jml> one day, it'll be just as easy to update it in response to events
<kfogel> wgrant: well, yes, ideally.  bzr has the right kind of properties to handle this already, I think, no?
<wgrant> kfogel: It does.
<kfogel> jml: "events" like (say) maxb updating deps, or did you mean something else?
<wgrant> And the emails that Launchpad sends out already show the [r=foo][ui=bar] in another way.
<ajmitch> kfogel: you expect new contributions every 10 minutes?
<kfogel> wgrant: hmmm.  the script could parse that too... but I kind of think it's better that it show what the msg actually looks like
<jml> kfogel, I mean, update in response to an actual commit, rather than polling
<kfogel> ajmitch: no, but I never know *which* 10 minutes one will come in.
<kfogel> jml: ah, yes, that would be ideal!
<jml> doing that now would mean listening for email, which kind of sucks.
<wgrant> So, why is all that metadata in the revision message?
<kfogel> jml: yah, I'd rather Not Go There.
<jml> wgrant, pqm checks it.
<kfogel> wgrant: because we're unimaginative in our use of bzr properties?
<jml> wgrant, it forces submissions to match a regex
<wgrant> jml: I know.
<wgrant> jml: i mean why is it *in the revision message*.
<wgrant> And not in properties.
<jml> wgrant, limitations in PQM.
 * wgrant headdesks.
<ajmitch> sounds like something just waiting to be improved
<jml> wgrant, I'm not trying to be difficult, sorry.
<kfogel> wgrant: nice verb, I think I'm going to have occasion to use that in the future.
<wgrant> jml: Oh, I know.
<mwhudson> jml: should https://code.edge.launchpad.net/~jml/lp-dev-utils/bzr-plugin/+merge/7427 be a launchpad branch now?
<jml> mwhudson, I'm not sure what to do with that now.
<jml> mwhudson, utilities/ is a pretty crappy place to keep a bzr plugin.
<mwhudson> jml: you could say it's a pretty crappy place to keep any software
<kfogel> jml: I'm not sure the fact of being a bzr plugin is any more relevant than, say, being written in sh instead of py.
<jtv> mwhudson: I don't think I can patch up that test and be sure it tests what it was meant to.  :(
<mwhudson> jtv: oh well
<kfogel> jml: bzr is just one of many tools the script uses to do its job.  I don't think that affects its appropriateness for being in the utilities/ dir.
<jml> mwhudson, kfogel: you can run a script from utilities without any installation step or env variable setting, you can't do that for a bzr plugin
<kfogel> jml: is that true of all the scripts in there?  I'd never realized it was a qualification for being in utilities.  It's about to be violated again when I move lpcc.py  (the thing that updates the Contributions page) over to utilities/, since that requires editmoin.py which we currently don't ship in utilities (nor should we, probably).
<jml> kfogel, well, it's not, I guess.
<jml> kfogel, but it's a nice property, I think.
<kfogel> jml: "meh", I guess.  If it is a req, we should document it, but I don't feel strongly it is a req.
<jml> ok.
<kfogel> jml: ah, maybe a more positive way to say it:
<kfogel> jml: "something can be in utilities if it's useful for developing or running launchpad"
<kfogel> er, "and has no other home" :-)
<jml> heh
<jml> well, I can try to redeem the bzr plugin branch.
<jml> but it's pretty low priority for me (it wouldn't have been six weeks ago)
<kfogel> bedtime here.  You can run wild, I won't be around to argue.  Go nuts :-).
<jml> kfogel, heh
<jml> mwhudson, https://code.edge.launchpad.net/~jml/lp-dev-utils/moved-dev-tools/+merge/11237 -- want to approve that?
<mwhudson> jml: ok
 * jml is experiencing doc writing fail
<thumper> spm: what's up with the buildbot?
<spm> thumper: I just bounced it - is it ded?
<spm> belh. yes.
<thumper> spm: I'm gettign 503
<thumper> spm: had it succeeded?
<thumper> spm: why was it bounced?
<spm> thumper: I thought it had, but clearly not; should be good now;
<spm> mwh was doing some testing
<spm> dur. try again. mwh was doing some buildbot changes and testing thereof.
<thumper> it's up now
<mwhudson> thumper: it succeede
<mwhudson> d
<thumper> need to kick db_lp
<thumper> \o/
<mwhudson> well, there should be a stable -> db-devel merge soon
<mwhudson> but yes, probably need to kick db_lp anyway
 * thumper forced db_lp
 * thumper wanders of for dinner time activities
<thumper> off
 * mwhudson eods
<thumper> d'uh
<adeuring> good morning
<bigjools> morning all
<mrevell> Good Monday :)
<bigjools> eyup mrevell
 * wgrant grumbles a bit at the lack of Debian support in PPAs.
<allenap> bigjools: fmt:link dunt work for a distroseries. Is there any reason I shouldn't add one?
<allenap> mrevell: Morning :)
<bigjools> allenap: weird, it should
<danilos> jtv: I guess IRC woes continue
<danilos> jtv: btw, what's the status on translations-person-3.0-mechanical branch?
<mrevell> hey there allenap
<bigjools> allenap: how are you doing it?
<allenap> bigjools: series/fmt:link
<jtv> danilos: that one got stuck in EC2 trouble before I left, so I'll be submitting.  (I reverted the changes to the main page, as you asked).
<bigjools> allenap: <a tal:replace="structure series/fmt:link"/> ?
<jtv> danilos: I did land the 3 bugfixes from Vientiane
 * jtv checks... it's a bit of a "cold boot"
<danilos> jtv: Vientiane?
<jtv> The capital of Laos.
<allenap> bigjools: Yeah, basically that.
<allenap> bigjools: I get: NotImplementedError: No link implementation for <canonical.launchpad.webapp.tales.ObjectFormatterAPI object at 0x83d1a10>, IPathAdapter implementation for <DistroSeries at 0x83d90d0>
<danilos> jtv: ok, cool
<allenap> bigjools: Bizarrely, fmt:url works fine.
<bigjools> allenap: yeah it uses canonical_url IIRC
<allenap> bigjools: Ah, okay.
<bigjools> stick wid dat unless you want more work :)
<allenap> bigjools: Okay, will do :)
<bigjools> although distroseries is registry now, Curtis might have something to say
<allenap> bigjools: I'll send him a quick email.
<bigjools> wgrant: around?
<wgrant> bigjools: I am.
<bigjools> do you know the use cases for the related-software page?
<bigjools> eg https://edge.launchpad.net/~wgrant/+related-software
<bigjools> I remember fixing bugs on it but I've never seen a canonical set of use cases
<wgrant> bigjools: There are no really important use-cases for that page in particular (apart from general interest), but its children are important.
<bigjools> okay
<bigjools> so in theory that page could be junked?
<wgrant> In theory, but it is useful.
<bigjools> ok :)
<bigjools> it's a grey area in terms of application ownership
<wgrant> Indeed.
<wgrant> Although I'd say it was Soyuz.
<bigjools> it's my last set of pages to convert to 3.0
<bigjools> and as curtis so eloquently put it: nightmare
<wgrant> Why? Not just mechanical changes?
<bigjools> the nav menu needs to be dismantled
<wgrant> Oh, right.
<wgrant> Forgot that bit.
<bigjools> and it's a two-tier one
<bigjools> and it's a mix of registry and soyuz templates :/
<wgrant> bigjools: Just link the section headings?
<wgrant> Or add a "View all maintained packages" link at the right like new main-content portlets have?
<bigjools> yeah - I could convert to an action menu
<wgrant> Don't those end up down the bottom, so are impossible to find?
<bigjools> top right
<bigjools> related-pages appear at the bottom
<wgrant> Hm. I'm not sure I've seen non-indices with action menus.
<bigjools> well, in fact they appear where I put them :)
<bigjools> hmmm yeah that would fall foul of the guidelines
<bigjools> I've seen related pages links under action menus though, so if there's no action menu it can appear at the top right
<wgrant> It seems odd to have that as the only thing on the right -- it's a bit of a waste.
<bigjools> indeed
<bigjools> can you see why it was labelled nightmare now?
<wgrant> Yes.
<wgrant> My current favourite is the convention of having "View all <whatever>" links in the top right. But that might not work for a page like that where the sections are the primary content.
<wgrant> Top right of the relevant portlet, that is.
<wgrant> bigjools: Any great ideas?
<bigjools> wgrant: my only idea is not great, and that's to have a side bar
<wgrant> bigjools: :(
<wgrant> And both the 3.0 UI gods are absent?
<bigjools> yep
<wgrant> That seems like suboptimal timing.
<bigjools> only 1 day for curtis
<bigjools> and knowing him I bet he's around anyway :)
<wgrant> Ah, right.
<wgrant> Well, that was disappointingly uneventful (getting LP running on Lenny).
<al-maisan> Hello there, I added this merge proposal earlier today but no email was sent out: https://code.edge.launchpad.net/~al-maisan/ubuntu/karmic/pristine-tar/gendelta-fix/+merge/11303
<al-maisan> Is that normal?
<al-maisan> abentley: Can you please take a look ^^ ?
<abentley> al-maisan: I'd be happy to look tomorrow, but I'm off today.
<al-maisan> abentley: sorry .. I did not know.
<abentley> al-maisan: no problem.
<bigjools> sinzui: are you lurking?
<sinzui> I am
<bigjools> I knew I could count on you
<bigjools> sinzui: being a nice guy, I thought I'd start on the +related-software 3.0 change
<bigjools> this is in no way me feeling guilty for moving two of my templates into registry today
<sinzui> bigjools: That is very generous.
<sinzui> Which templates?
<bigjools> productseries-packaging and ... /me looks
<bigjools> sourcepackage-packaging
<sinzui> Yes, I agree they belong
 * sinzui was wondering why there were so few sp templates to convert
<bigjools> sinzui: my question is, I thought I'd use related-pages to put the mess of menus in a sidebar portlet
<bigjools> I started, but the context menu is for IPerson and that's not what I want, how do I use a different one?
<sinzui> That wont pass beuno's inspection because the menu is not a top-level collection
<sinzui> bigjools: I had a similar problem...
<bigjools> argh
<bigjools> ok
 * sinzui tries to remember what he did.
<bigjools> how about an action menu?
<sinzui> bigjools: I remember adding a <ul class="horizontal"> of links below the narrative on a page
<sinzui> bigjools: It is the same way links are formatted on the product page...
<sinzui> and then I realised that I had reinterpreted the black bar as a white bar
<bigjools> that's fabulously low-tech
<sinzui> bigjools: I hard coded it too, no nav menu
<sinzui> bigjools: but you can keep the menu since I think it already has what you expect to see
 * sinzui chooses the fastest way to get the page landed
<bigjools> indeed, I just need to repeat the links in the tal then
<bigjools> ok, I'll attack this tomorrow
<sinzui> bigjools: I will welcome the distraction from CHR
<bigjools> there's no description on any of those pages though
<bigjools> so the links will appear right at the top
<bigjools> can tal iterate over those menus so I don't have to hard-code the link names?
<sinzui> bigjools: maybe the lack if description is why I do not understand what the pages are for or who uses them. I suspect teams have different uses case than users (seeing the bug reports by persia)
<bigjools> yeah - one of the uses I am aware of is reviewing someone's contributions when they want to become a MOTU
<mrevell> night all
<bigjools> sinzui: still around?
<sinzui> yes
<bigjools> sinzui: check this out lp:~julian-edwards/launchpad/related-software-30
<bigjools> browse to https://launchpad.dev/~mark/+related-software
<sinzui> bigjools: this looks good
<bigjools> sinzui: better than I'd thought it would
<bigjools> what's with the weird breadcrumbs lately?
<sinzui> I ponder the uses of 'See also' and 'Full listing' Maybe the page doesn't need them
<sinzui> yes, I saw the breadcrumbs on Friday
<sinzui> bigjools: happily they are not your problem
<bigjools> sinzui: right, the extra text was an afterthought
<sinzui> Full listings is helpful.
<sinzui> I am just surprised by the See also. I understand why you need to change the label
<sinzui> bigjools: I think the <h1> and page title should either both include 'Summary', or both not use it
<bigjools> yeah, we could do with barry's heading branch landing ;)
<sinzui> bigjools: Actually, I am surprised that the heading and title are not the same
<bigjools> sinzui: coz it's a hack job
<sinzui> bigjools: the constant "Realated project" vs. "Projects related" looks like two different people working on the page.
<bigjools> sinzui: yeah I didn't attempt to do much with the headings/titles, I just wanted to visualise the new layout
<sinzui> bigjools: I do not have any suggest about how to fix this
<bigjools> well won't barry's branch reverse the title anyway?
<sinzui> the layout is a nice improvement. The links are easy to see now. This is a nice imrovement for users
<sinzui> bigjools: I do not this it will do that in the first round. I am not sure how that can be done since the page title would have to be generated from the breadcrumbs. The crumbs are not good english
<bigjools> okay, I'll knock up a page_title to sort it out
<mwhudson> hmm
<mwhudson> has gary been around today?
<bigjools> sinzui: I made some changes, check it out
<sinzui> bigjools: this is very nice
<bigjools> sinzui: the nightmare has ended :)
<sinzui> bigjools: you can have my ui* r=me
<bigjools> sinzui: rock n roll
<mwhudson> oh, labour day, duh
<thumper> morning
<thumper> mwhudson: yeah, just you and me again for our team I think
<mwhudson> thumper: abentley should be here?
<mwhudson> or is labour day the same day as labor day?
<thumper> I think it is labour day in canada too
<mwhudson> oh yes
<mwhudson> it's thanksgiving that's a week apart?
<mwhudson> thumper: skype then?
<thumper> yep
<mwhudson> hm, is this the usual confusion about who's online
 * maxb wonders why the tarballs at http://people.canonical.com/~herb/ are ~230MB, when a freshly `bzr pack`-ed repository is only 140MB
<lifeless> maxb: we had fragmentation bugs in the 2a format, in bzr. lp was suffering those, and its likely that herb tarred up the public repo, which would be fragmented
<wgrant> maxb, lifeless: Those were created on release night, from a fresh checkout.
<wgrant> But bzr sucked at that point.
<lifeless> wgrant: see under fragmentation
<lifeless> wgrant: we also didn't recompress when receiving fragmented streams
<lifeless> all of which combine. These things are fixed in 2.0rc2
<wgrant> lifeless: Right, but it wasn't the public copy's fragmentation.
<wgrant> Yep.
<wgrant> Nightlies work great now.
<lifeless> wgrant: a fresh checkout made on release night would have inherited the public copies fragmentation
<lifeless> wgrant: because lp was set hot, and hot things go to the public copy
<wgrant> lifeless: Ah, right.
<lifeless> wgrant: [and further to that, the master is on pqm; both lp copies are 'public' in that sense]
<lifeless> spm: ping
<spm> lifeless: heyo
<lifeless> spm: this all reminds me; care to do a bzr pack in the codehosting copies of stable, devel, etc
<wgrant> +inf
<lifeless> df -h .bzr/repository/packs before and after
<wgrant> But 1.17 doesn't do a very good job.
<spm> of launchpad? not without flacoste being ok with doing so
<wgrant> And isn't that what's on codehosting?
<lifeless> if its < 1MB there
<lifeless> 's no need
<lifeless> wgrant: it will pack just fune
<lifeless> *fine*
<wgrant> lifeless: It didn't get devel down below 200MB for me.
<lifeless> wgrant: with the C extensions?
<lifeless> spm: well, gather the data anyhow.
<wgrant> lifeless: I think so, but it was a while ago.
<lifeless> spm: I don't see that this is a flacoste thing; its operational mgmt for a hosted branch :)
<lifeless> spm: if anyone its a thumper/mwhudson thing
<thumper> eh?
<lifeless> thumper: theres a good chance that the copies of lp on codehosting are badly fragmented
<thumper> and
<lifeless> thumper: due to about 4 interacting bugs in bzr
<wgrant> They are very, very badly fragmented.
<wgrant> Nearly 100% larger.
<lifeless> it will improve fetch performance, decrease server memory use, and disk space, if they get 'bzr pack' run on them.
<lifeless> and network overhead on fetching too
<thumper> lifeless: however bzr pack takes ages
<lifeless> thumper: it doesn't lock the repository
<thumper> lifeless: and IIRC you told me you should never need to run bzr pack
<thumper> lifeless: so, why now?
<lifeless> thumper: you shouldn't; OTOH I also said 'I wouldn't upgrade to 2a until its fully finished'
<thumper> :)
<thumper> lifeless: will the autopacks fix the issue?
<lifeless> thumper: in 5 years or so
<lifeless> auto packs only touch recent data most of the time
<lifeless> thumper: how many revisions does your lp repo have in  it ? [bzr info -v]
<thumper> 67122
<lifeless> thumper: as to why now, I just got reminded of it by maxb above; we'll want to do this once, to all 2a repos on codehosting, when codehosting deploys 2.0
<lifeless> ok autopack will fix this for you in another 32878 commits
<thumper> lifeless: so why is it badly packed now?
<lifeless> because of bzr bugs in 1.16,1.17 and 1.18 with fetching of 2a
<thumper> lifeless: looking at the packs there, they look as large as expected
<thumper> lifeless: to me it looks like natural growth
<thumper> lifeless: why do you think it is different?
<thumper> lifeless: I pushed completely packed repos there the first time
<lifeless> looking at http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/.bzr/repository/packs/?C=S;O=D ?
<thumper> yes, I was
 * lifeless tests
<maxb> for comparison purposes, a freshly packed shared repo of all 4 primary branches packs down to one 109MB pack
<lifeless> thumper: ^
<lifeless> thumper: in a fragmented repo the ratios of packs stay the same; the packs are just bigger.  Its their internal structure that is odd
<lifeless> too many compression groups
<thumper> lifeless: so internal changes to bzr in recent revisions have fixed this?
<lifeless> 2.0rc2 will both no longer cause fragementation when fetching, and partially compensate for existing fragmentation on the client
<lifeless> doing a full pack now won't stop some further fragmentation on lp until lp rolls out 2.0. *but* packed data won't get fragmented: its only *new data* that fragments.
<thumper> lifeless: so no point really until we are on a later version of bzr
<thumper> lifeless: because we only have 1.17 on LP now
<thumper> lifeless: I'm in the progress of landing 1.18 final
<thumper> lifeless: and I would add in 2.0rc2 if I could find the tar.gz on LP
<thumper> but I can't
<thumper> lifeless: but this won't be rolled out until 3.0
<lifeless> thumper: there is very much a point now
<thumper> lifeless: we don't have the fixed bzr on the machines to do the packs
<lifeless> thumper: 99% of the data that will be in the repository when 3.0 is rolled out is already in it now; you can shrink the repo by ~ 60% according to maxb's stats
<lifeless> thumper: you don't need a fixed bzr
<lifeless> we haven't changed pack
<lifeless> I wouldn't be suggesting this if I didn't think it would help you!
<thumper> lifeless: why then is the repo so large when I packed it before pushing?
<thumper> lifeless: I had a single pack file when we created the repo
<lifeless> because pushing was fragmenting!
<maxb> So, basically, a repack now saves bandwidth for LP and downloaders between now and 3.0 ?
<thumper> hmm..
<lifeless> thumper: and you pushed to the private copy, lp then copied it again to the public one fragmenting it further
<maxb> (Somewhat meta question) Why does LP keep two copies anyway?
<lifeless> thumper: at *every step* dictionary sort order was changing what bzr considered optimal @ merge points
<thumper> maxb: histerical raisons
<lifeless> thumper: each time this happens a gc group splits; gc groups contain full texts, so that text doubles its storage size
<maxb> ahhh....
<lifeless> maxb: theory at the time was scaling and security
<thumper> lifeless: so to be effective we'd have to pack both copies?
<lifeless> thumper: users pull from the public copy only, even on bzr+ssh
<thumper> lifeless: how does pack work without locking?
<lifeless> so most important one is the public copy
<thumper> lifeless: no they don't
<thumper> lifeless: bzr+ssh pulls from the hosted copy
<lifeless> thumper: just a few days ago we had this confirmed
<thumper> if it is a hosted branch
<wgrant> thumper: Only if you have write access, right?
<lifeless> thumper: and I'm sorry to say, you're wrong  :)
<thumper> wgrant: it wraps it in a readonly transport
<lifeless> thumper: or jml/mwhudson lied to us :)
<thumper> lifeless: confirmed by whom?
 * thumper looks at the code
<mwhudson> thumper: well you and i will read from the hosted copy, cause we're bzr experts
<mwhudson> everyone else reads from the mirrored area though
<lifeless> mwhudson: am I a bzr expert ?
<jml> hopefully not
<thumper> mwhudson: I thought that all hosted branches were read from the hosted area
<mwhudson> lifeless: you're an admin aren't you?
<lifeless> mwhudson: not anymore
 * jml dislikes the group, and wishes it as small as possible
<mwhudson> thumper: how to put this tactfully?
<mwhudson> thumper: i think you thought wrong
<thumper> :)
<thumper> mwhudson: where is the code?
<mwhudson> thumper: it seems to be a bit convoluted
<lifeless> >>> t = bzrlib.transport.get_transport('bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/.bzr/repository/packs')
<lifeless> >>> t.list_dir('.')
<mwhudson> thumper: but it's in lp.codehosting.vfs.branchfs
<wgrant>         if writable:
<wgrant>             dispatch = self._hosted_dispatch
<wgrant>         else:
<wgrant>             dispatch = self._mirrored_dispatch
<mwhudson> TransportDispatch._makeBranchTransport
<lifeless> 1d0252d21c3632af79515b5e8c497c15.pack
<lifeless> thats the same large pack as on the public side
<lifeless> spot check of a few others matches
<lifeless> now that thats out of the way
<lifeless> thumper: bzr repositories have fine grained locking, they were designed to be as concurrent as possible.
#launchpad-dev 2009-09-08
<thumper> mwhudson: where is the writable parameter coming from?
<mwhudson> thumper: the xml-rpc server
<mwhudson> thumper: which does check_permission(branch, 'launchpad.Edit')
<thumper> hmm..
<thumper> seems that something is wrong
<jml> see lp.code.xmlrpc.codehosting and lp.code.interfaces.codehosting
 * thumper tries to work it out in his mind
<wgrant> This code is rather difficult to navigate :(
<lifeless> thumper: I've spent as much time as I can afford on this. Sorry.
<thumper> lifeless: ok
<lifeless> thumper: its a safe operation, bzr does it automatically without blocking out other users, it will help your servers and your users.
<jml> wgrant, sorry. it's as easy as I could make it.
<lifeless> thumper: oh, and if questions like 'how much will it help' are in your mind, use wget to mirror the public repo bit-for-bit, run a 1.17 server against it and fetch using bzr nightly
<lifeless> then pack it using 1.17 (with C extensions, of course), and repeat.
<jml> thumper, mwhudson: has someone addressed the broken test in db-devel?
<mwhudson> jml: thumper is, i mailed the list
<jml> mwhudson, ahh cool
<jml> sorry for missing it.
<mwhudson> np
<jml> mwhudson, do you want to chat about how to fix bug 419691
<ubot3`> Malone bug 419691 in launchpad-foundations "Many tests use unnecessarily expensive layers" [Undecided,New] https://launchpad.net/bugs/419691
<mup> Bug #419691: Many tests use unnecessarily expensive layers <build-infrastructure> <Launchpad Foundations:New> <https://launchpad.net/bugs/419691>
<mup> Bug #419691: Many tests use unnecessarily expensive layers <build-infrastructure> <Launchpad Foundations:New> <https://launchpad.net/bugs/419691>
<mup> Bug #419691: Many tests use unnecessarily expensive layers <build-infrastructure> <Launchpad Foundations:New> <https://launchpad.net/bugs/419691>
<mwhudson> uh
<mwhudson> jml: skype?  one sec in that case
<jml> bot wars :)
<spm> thumper: lifeless: fyi. bzr pack initiated on db-devel, mirrors
<thumper> mwhudson: I'm still confused
<thumper> but unfortunately out of time
<mwhudson> thumper: i'm on the phone
<mwhudson> later i gues
<thumper> mwhudson: the fix for the db-lp builder is not as trivial as I hoped
<thumper> will fix after class
<mwhudson> thumper: bugger
<spm> thumper: lifeless: fyi. bzr pack completed on db-devel, mirrors - looks to have finsihed about... 25 mins ago. taking around... 15mins to do it.
<wgrant> spm: Looks much better. Still not perfect, but much better.
<spm> wgrant: the end result is ~ 120Mb - eyeball wag
<wgrant> Is devel going to get the treatment too? That's the one that is going to reinforce people's thoughts about how much bzr sucks when they try to get LP's code.
<spm> yeah - it will. juggling 3 things at once :-)
<wgrant> Great.
<spm> need to run jml's nifty get-branch-info script etc
<spm> devel is started
<wgrant> What does that do? Tell you the ID/path?
<spm> yarp. decodes the nice name into the 00/00/12/34 path
<spm> has a few other nifties that help verify you've got the right branch.
<wgrant> Yay, looks like Ohloh's LP import is still working.
<spm> mwh's hack of sending an invalid .bzr location to codebounce is another used method
<wgrant> Or even a valid one.
<wgrant> Sometimes it'll redirect you to the internal URL.
<wgrant> No idea why.
<spm> wgrant: ha. you'll chuckle - straight from our "howto" notes: http://bazaar.launchpad.net/~wgrant/ivle/trunk-mirror/.bzr/ergasdf
<spm> Nooooo! it's ont working any more!!! mwhudson ^^
<wgrant> Indeed, I believe I deleted that branch some time ago.
<spm> ahh. does still work - the branch needs to exist in the 1st place.
<spm> http://bazaar.launchpad.net/~drizzle-developers/drizzle/development/.bzr/wibble by way of example
<wgrant> It got renamed to trunk when we moved to bzr.
<spm> so the diff there; with jml's script; is the latter gives the full path on disk; vs just the 00/00/12/34 bit.
<wgrant> Ah.
<spm> tho tbh, I can do the pre-path in my sleep I expect :-)
 * mwhudson lunchifies
<spm> and devel is also el-packed. fnished about 4 mins ago. ~ 111Mb. So again, aruond 15mins to run.
<wgrant> Excellent.
<thumper> mwhudson: quick call?
<mwhudson> thumper: ok
<thumper> omg skypes sucks
<thumper> mwhudson: what is the revno of the puller fix?
<thumper> mwhudson: I'm going to request a CP
<mwhudson> thumper: looking
<mwhudson> thumper: r9355
<mwhudson> thumper: would be good to test it out on staging first i guess
<mwhudson> except that it's not in db-stable yet, of course
<thumper> hmm..
<mwhudson> oh fucking hell
<mwhudson> mischan :)
<thumper> mwhudson: buildbot?
<mwhudson> thumper: yeah
<thumper> mwhudson: ok, lets wait for staging testing then
<thumper> mwhudson: do I need to update the dev tools to get the latest ec2 image?
<jml> thumper, got a moment?
<mwhudson> thumper: no
<mwhudson> thumper: the current ec2 image is rather old though
<mwhudson> jml: so it turns out that the time 'initialize database' takes for buildbot slaves varies wildly
<mwhudson> jml: it's not usually 20 minutes, more often ~1
<jml> mwhudson, wow.
<jml> mwhudson, I'm surprised
<mwhudson> jml: yeah
<thumper> jml: yes(ish)
 * wgrant stabs Launchpad mailing lists in the face.
<wgrant> Stop. Breaking. Threading.
 * mwhudson eods
<jml> BjornT_, I assume all the bug mail you just generated was only unassigning yourself?
<BjornT_> jml: yeah. and marking some bug as fix released, but it's safe to ignore everything
<jml> BjornT_, thanks.
<adeuring> good morning
 * jml off
<mrevell> Morning!
<jtv> Are we in testfix?
* mrevell changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ |  Please use #launchpad for support. | Get it: https://dev.launchpad.net/Getting | http://paste.ubuntu.com/ | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html | Buildbot is down
<jtv> mrevell: thanks
<bigjools> https://lpbuildbot.canonical.com/waterfall
<jtv> bigjools: so it's the db_lp failure that put us in testfix mode?  All this usually happens overnight for me, so I haven't built up any real experience with it.
<saispo> hi
<bigjools> jtv: mwhudson was fixing something with it, it might be bustifuct still
<thumper> I don't know why pqm still thinks it is in testfix
<thumper> I'm hoping the jscheck builder failure didn't tweak it
 * thumper looks at saispo
<lifeless> jml: hai
<wgrant> bigjools: AFAICT archiveuploader still doesn't apply any checks that binaries do not conflict.
<bigjools> wgrant: I've seen failed-to-upload errors for that though?
<wgrant> bigjools: Right, sorry, I was ambiguous. There is a check that they are not older than the currently published version, but not that they have never existed before.
<bigjools> that's wrong then I guess
<thumper> my icy stare obviously scared him off :)
<bigjools> thumper: I still have nightmares about your Halloween face!
<thumper> bigjools: I need to replace my fluffy bunny picture with my halloween hackergochi
<wgrant> bigjools: It should probably check that the version hasn't existed before, yes. It's unclear, however, whether one should be able upload an older version of either. I'm really not sure.
<mwhudson> we shouldn't be in testfix per PQM
<bigjools> wgrant: yeah, it's not exactly consistent with the source processing
<mwhudson> if we are, something's not working right
<thumper> mwhudson: something isn't working right
<mwhudson> thumper: luckily i'm not an osa so i can't help diagnosis or help
<mwhudson> i guess the buildbot-poll script is barfing for some reason
<BjornT> jtv, thumper: in my experience, it's good to show the exact error message from pqm to others. it happened quite often that people thought that we were in testfix mode, just because pqm didn't accept their commit message
<thumper> BjornT: Commit message [[rs=thumper][ui=none] Upgrade to bzr 1.18.] does not match commit_re [(?is)^\\s*\\[testfix\\]\\s*\\[(?:release-critical|rs?=[^\\]]+)\\]]
<bigjools> regex hell
<thumper> BjornT: that's the testfix regex
<intellectronica> maybe we need a UI for commiting
<thumper> heh
<intellectronica> thumper: why not. one day, when you can commit straight from a merge proposal, we could have all the fields that are relevant, and that will build the commit message for you
<wgrant> intellectronica: I think that one day the commit message will be normal text.
<wgrant> intellectronica: And the silly review metadata will be stored in the MP.
<BjornT> thumper: indeed. did you submit it before or after stub's testfix commit?
<intellectronica> wgrant: nah, you want it in bzr, but maybe it can be stored in bzr metadata (as long as there's a simple way to read it back)
<wgrant> intellectronica: OK, bzr revision properties then.
<wgrant> intellectronica: But automatic.
<thumper> BjornT: stub submitted his testfix after I submitted my testfix
<intellectronica> yeah
<thumper> BjornT: and the last one was after stub's
<thumper> BjornT: it appears there is something screwy somewhere
<BjornT> thumper: how long after? it takes a while before we get out of testfix mode.
<thumper> BjornT: about 40 minutes
<BjornT> thumper: ok. i don't see a testfix for the current db-devel failure
<thumper> BjornT: 4:42 on the waterfall
<BjornT> thumper: there are build failures for db-devel after that commit
<BjornT> thumper: the latest one is at 06:30:05
<thumper> BjornT: fair enough
 * thumper chucks in the towel
<deryck> Morning, all.
<jtv> hi deryck
<mrevell> yo dez
<danilos> BjornT, thumper: heya, I've got a breadcrumbs issue I've been wondering about (I'll ultimately wait for salgado, but perhaps you guys know why this could be)
<danilos> basically, an object doesn't end up in request.traversed_objects, and I am not exactly sure why
<wgrant> bigjools: The maximum time remaining should probably be exposed in each package row of the PPA index, as builds are undiscoverable.
<bigjools> undiscoverable?
<noodles775> on the new +packages traversal, perhaps.
<wgrant> noodles775: Probably.
<wgrant> bigjools: I've seen people completely fail to understand that the build pages exist.
<bigjools> if there's room in the new +packages table, we could potentially show it for each arch
<bigjools> but it would be nice to emphasis the links if people are missing them
<noodles775> Maybe we could even link from the 'Latest updates' portlet that will be on the ppa index.
<wgrant> It would be useful anyway to show the remaining time, even if it's only the maximum of all pending builds.
<bigjools> noodles775: not sure that's the right place, it won't necessarily show all uploads with pending builds will it?
<noodles775> bigjools: just the latest 5 published spph's (with their current build status)
<bigjools> yeah, so it's probably best to fit something in the table
<Daviey> Hey, is there a technical reason why merge review email needs white space before the command?
<intellectronica> Daviey: no, that's the ui. with space in front of the command it's easier to distinguish from the text. it's a quoting mechanism
<Daviey> ah!
<Daviey> intellectronica: The annoying thing is, an email that purely states "merge approve" is obviously not a normal comment.. but is seen as that :(
<kfogel> salgado: [transfer ping from #launchpad] :-)
<danilos> salgado: heya, my breadcrumbs master
<salgado> danilos, otp
<danilos> salgado: hang up and talk to me! :)
<kfogel> danilos: take a number, man
<danilos> kfogel: heh :)
<rockstar> abentley, call?
<abentley> rockstar: sure.
<danilos> salgado: when do you expect to be available?
<salgado> danilos, in a few minutes
<kfogel> noodles775: hey, re your Hacking Soyuz UDW session: it looked like it went well to me, but did any questions come up (especially re community development) that stumped?
<bigjools> kfogel: I was there, and I didn't see any, in fact there was only one guy interacting
<kfogel> bigjools: well I think the questions arrive via backchannel, so it can look less active than it is.
<bigjools> kfogel: yeah I was on the chat channel as well
<kfogel> bigjools: looking at the transcript, there were some questions.  Were they all the same person?
<bigjools> kfogel: mostly noodles himself :)
<kfogel> bigjools: oh :-)
<salgado> danilos, pong
<salgado> kfogel, pong
<danilos> salgado: hey hey
<danilos> salgado: so, attempt at a quick question: DistroSeriesLanguage, traversed via stepthrough, doesn't show up in request.traversed_objects, so I can't introduce a breadcrumb for it; is there way around it?
<kfogel> salgado: you were working on making launchpad-reviews public, but with exception for security branches.  I'm wondering where that's at.
<kfogel> salgado: dang, danilos beat me to it, and he didn't even take a number :-).
<kfogel> salgado: seriously, finish with danilos first, since my question is likely to take longe.r
<danilos> kfogel: heh, let's see if salgado can multi-task :)
<salgado> danilos, yes, there's a bug open about that and I have a workaround in one of my branches
<salgado> danilos, basically, you have to define a Navigation class for DistroSeriesLanguage
<danilos> salgado: ok, I am happy to wait for it to land, but can you point me at your branch so I make sure it works
<danilos> salgado: ah, ok, any examples that might kick start me?
<danilos> salgado: eg. does +source have that?
<salgado> danilos, I thought I'd done that for DistroSeriesLanguage but I didn't
<danilos> salgado: there's also ProductSeriesLanguage, perhaps that's where you did
<salgado> +class AnnouncementNavigation(Navigation):
<salgado> +    """The navigation for `IAnnouncement`."""
<salgado> +    usedfor = IAnnouncement
<danilos> salgado: register it with browser:navigation, right?
<salgado> danilos, that's what you need, together with a browser:navigation zcml declaration
<danilos> salgado: ok, cool, thanks
<salgado> kfogel, I never got around to it, sorry
<salgado> danilos, bug 423898
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<ubot3`> Malone bug 423898 in launchpad-foundations "Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects" [Undecided,Triaged] https://launchpad.net/bugs/423898
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<kfogel> salgado: hey, we're all busy, I understand.  Can you tell me what's left on it?  Maybe I can help unblock.
<danilos> whoa, whoa, ubot3`, mup, I hear you
<danilos> salgado: thanks
<kfogel> danilos: :-)
<salgado> kfogel, I need to read the thread again to see what should be done
<salgado> kfogel, I'll do that today and get the ball rolling there
<kfogel> salgado: thank you, much appreciated.  If it's a matter of seeing which way the ball has to roll, but not having time to roll it, please don't hesitate to say so -- I'm happy to do what I can (& to help find other people to do what I can't) if needed.
<salgado> kfogel, will do, thanks
<salgado> out to lunch now, back soon
<barry> oowww!  did --headless suddenly get much faster?
<barry> salgado: ping
<salgado> hi barry
<barry> salgado: hi.  about the not found error page title
<rockstar> What is the url that gives me all the icons available in LP?
<barry> salgado: i think i should move pagetitles.py/launchpad_notfound into a page_title attribute on NotFoundView
<salgado> rockstar, https://edge.launchpad.net/+graphics
<barry> salgado: but i also think i should move the other error related pagetitle.py attributes onto their respective classes in error.py.  what do you think?
<rockstar> salgado, thanks
<barry> salgado: and also, should we put a page_title attribute on SystemErrorView?
<salgado> barry, good question; let me check
<salgado> barry, I think that's a good idea, together with moving titles into webapp/error.py
<barry> salgado: cool.  let me make that change and i'll send you the incremental
<kfogel-lunch> flacoste: voice of sanity on "offer mentorship" ui, thank you
<barry> salgado: ping
<salgado> barry, pong
<barry> salgado: i wanted to talk about @@+page-title/has_custom_title again
<barry> i'm not sure the check is quite right.  the problem is, if you check for the existence of breadcrumbs, then putting a page_title on your view isn't enough to override
<barry> salgado: ^^
<barry> salgado: so the question is whether a view should have to specifically say "i don't have breadcrumbs, and oh yeah here's my page title" or just the latter.
<barry> salgado: i think it should be just the latter.  what do you think?
<salgado> barry, I think it's the latter, but all views must specify their title; either in pagetitles.py or through an attribute
<salgado> I suggest something along these lines:
<salgado> All view classes either define a .title attribute or have an entry in pagetitles.py
<rockstar> abentley, I'm trying to use the factory to create some revisions for a branch, but they don't seem to be showing up.
<rockstar> abentley, do you have any insight as to what I might be missing?
<abentley> rockstar: You're looking for them on the database class, right?
<salgado> barry,  but some views may also define a .page_title, which takes precedence over the title generated by PageTitleView()
<rockstar> abentley, no, through the web ui.
<barry> salgado: what's the purpose of the view.title attribute then?
<abentley> rockstar: Do they show up in the console in make harness?
<rockstar> abentley, good question, lemme check.
<rockstar> abentley, I'm using makeRevisionsForBranch in this context, for YOUR context.
<salgado> barry, that's the text for the breadcrumb's last item
<barry> salgado: is that there now, or you're proposing that to be added?
<salgado> barry, my idea was to rename AnyView.page_title to title
<barry> salgado: and that would be added as the last component to the breadcrumb, if it exists?
<salgado> barry, yes, and if it doesn't exist it must be in pagetitles.py
<salgado> so, in summary, View.title or an entry in pagetitles.py would be required
<barry> salgado: i'm still not sure how best to override the <title> though, so that a view can say "don't use breadcrumbs even if they're available"
<salgado> barry, some views would also define a page_title for that
<barry> salgado: so the presence of a .page_title would be enough to override the reverse breadcrumbs
<salgado> yes, or html_title, or do_not_use_breadcrumbs_for_title
<salgado> I guess a boolean that specified not to use breadcrumbs in the title would be best
<barry> salgado: i see.  so as not to repeat .title
<salgado> yep
<barry> salgado: my only concern is that that's a fairly large change for a branch i'm already overdue to land ;)
<rockstar> abentley, so the revision_count attribute on the branch object is populated correctly.  I still don't have revisions in the UI though. :(
<barry> salgado: maybe we can do part of this now?  add .page_title to override breadcrumbs now, and then in a follow on branch do the .page_title -> .title change?
<rockstar> abentley, wait, cancel that.  I'm seeing them now.  Not sure what changed, but everything works now.
<abentley> rockstar: Odd.
<salgado> barry, actually, there's no need for the .page_title -> .title change
<salgado> if we add the boolean
<salgado> and adding this boolean should be a simple change, no?
<barry> salgado: i think it would be.  so: "override_breadcrumbs = True" in the view means to use .page_title or pagetitles.py entry instead of breadcrumbs even if they exist?  if breadcrumbs don't exist, the boolean is obviously not necessary
<salgado> barry, yes, that's how I envisioned it
<barry> salgado: agreed.  let me see if i can make that work.  thanks!
<salgado> barry, thank you for bringing this up
<barry> salgado: thanks for the time to discuss it; i'll have to remember to add this to the wiki rules
<barry> salgado: small correction: override_title_breadcrumbs = True is better i think
<salgado> barry, agreed
<barry> cool, thx
<barry> salgado: so the problem is; context/@@+page-title gives us a view, but override_title_breadcrumbs is on the original view and i don't know how to get from the PageTitleView back to the original view to check the attribute
<barry> salgado: i suppose i can write a more complicated condition in base-layout.pt, but that seems a bit ugly
<salgado> barry, yeah, that wouldn't be nice indeed
<barry> salgado: can you think of any zope magic to the rescue? ;)
<salgado> barry, a tales formatter
<salgado> view/fmt:pagetitle
<barry> hmm....
 * barry hacks
<barry> salgado: i think this is going to work very nicely.  i will have to file a bug so that when main-template.pt is removed or updated, we can get rid of CONTEXTS/fmt:pagetitle and PageTemplateContextsAPI;  that's to much work to do now though
<salgado> barry, that's great news
<barry> salgado-afk: i have to write some tests but here's the diff so far http://pastebin.ubuntu.com/267554/
<mwhudson> thumper: morning
<thumper> mwhudson: morning
<mwhudson> skype?
<wgrant> gary_poster: around?
<gary_poster> wgrant: now I am
<wgrant> gary_poster: You might remember that branch of mine that you reviewed nearly two weeks ago. ec2test kept vanishing, so I wasn't able to fix the test failures until recently. Can you please land it?
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/structural-subscription-security/+merge/10776
<gary_poster> wgrant.  I'll be happy to.  Do you happen to know the procedure we're using to do so, so that you get in the message properly?  If not I'll ask around.
<gary_poster> the pqm commit message I mean, sorry
<wgrant> gary_poster: There is no standard, but the one that I've stuck on the MP is a common one.
<gary_poster> OIC
<gary_poster> k
<gary_poster> wgrant: in pqm...
<thumper> rockstar: ping
<wgrant> gary_poster: Thanks.
<rockstar> thumper, pung
<rockstar> Er, pong
<gary_poster> wgrant: through pqm, and buildbot sees it.  I'm signing off
<thumper> rockstar: call?
<gary_poster> night all
<thumper> by gary_poster
<thumper> bye
<thumper> damn
<gary_poster> :-) bye
<rockstar> thumper, yes
<barry> bac: ping
<bac> hi barry
<barry> hi bac.  it's unlikely i'll be able to do that ui review for you.  i'm dashing like mad to get this branch landed and tomorrow i'm chr.  if you can't find anybody else to do it for you, i'll see what i can do tomorrow, depending on how heavy a day it is (usually, following sinzui is pretty light :)
<bac> np, barry.  didn't realize you were chr.
<barry> yeah.  kind of sucks.  with the short week it's gonna be a death march to get this branch landed
<wgrant> kfogel: I think you want your contributions script to sort the revisions chronologically, to make that page a bit more useful.
<wgrant> (ATM they don't seem to be sorted, so they are ordered differently each time the page updates)
<kfogel> wgrant: agreed.  really, just sorting them by revnum should work
<kfogel> wgrant: if you whip up a patch, I'm sure it'll be correct :-), but if not, no worries, I'll try to do it later tonight.
<wgrant> kfogel: I've got enough uni stuff on my plate, sorry.
<kfogel> wgrant: no problem at all.  Thanks for the suggestion; I'll take it from here.
<kfogel> should just be the addition of a sort() call, really.
<wgrant> Yep.
#launchpad-dev 2009-09-09
<rockstar> My internet seems to be eating itself.
<mwhudson> thumper: just threw https://code.edge.launchpad.net/~mwhudson/launchpad/other-puller-argh/+merge/11400 your way
<thumper> mwhudson: ok
<mwhudson> thumper: jml reviewed it
 * mwhudson ec2s
 * mwhudson draws breath, tries to work out which pile of sand to shore up next
<thumper> :)
 * thumper heads out for lunch
<spm> mwhudson: may I suggest codebrowse? >:)
<mwhudson> spm: not really supposed to be doing that sort of thing this month, but yeah, it's getting really bad
<mwhudson> jml: i have this vague memory you had a list of things we could do to try reduce the cpu cost of the test suite
<mwhudson> jml: do you have it to hand?
<jml> mwhudson, https://dev.launchpad.net/FasterTests ?
<jml> mwhudson, it's not fully up-to-date from the mailing list thread where I first raised it for discussion
<jml> but it's close
 * jml away, call my mobile if you need me
<mwhudson> jml: thanks
<lifeless> mwhudson: also, see bzr's thread on similar stuff recently.
 * mwhudson is writing an awk script, spm will be happy
<spm> mwhudson: thrilled
<mwhudson> weird, most of the time on buildbot 'make schema' takes ~1 min 20s
<mwhudson> apart from when it takes 20 minutes
<mwhudson> hooray xen and lots of IO?
<wgrant> It varies here greatly as well.
<wgrant> security.py sometimes takes less than a minute. Sometimes half an hour.
 * thumper wonders about SSH oopses: OOPS-1347SMPSSH1, OOPS-1347SMPSSH10, OOPS-1347SMPSSH11, OOPS-1347SMPSSH12, OOPS-1347SMPSSH13
 * thumper looks at mup
<thumper> OOPS-1347SMPSSH1
<lifeless> is the postgresql on the slaves configured to not fsync etc?
<thumper> lifeless: best to ask stub
<thumper> I guess
<lifeless> thumper: I'm not sure stub has had much to do with buildbot
<lifeless> mwhudson is the current be :P
<lifeless> but spm probably can find out most esily
<thumper> lifeless: d'uh
<thumper> lifeless: I was thinking our db slaves (read only ones)
<lifeless> thumper: :P
<thumper> lifeless: I just misunderstood
<lifeless> thumper: I wasn't all that clear.
<mwhudson> lifeless: a good question, i guess
<lifeless> later
<mwhudson> spm: can you check this?  log into a slave and check the postgres config?
<mwhudson> spm: (unless you know off the top of your head)
<spm> mwhudson: what's the question context here? the slaves fsyncing updates to disk?
<mwhudson> spm: buildslave
<spm> ahh. right. looking.
<spm> wrong db slaves...
<spm> mwhudson: fsync = off
<mwhudson> spm: ok thanks
<thumper> mwhudson: I emailed you yesterday for some testing advice, do you have any?
<mwhudson> thumper: oh right, haven't looked in depth yet
<thumper> mwhudson: it would be great if you have 5 minutes or so and need a break, but it isn't urgent
<mwhudson> thumper: i need lunch :)
 * mwhudson has a quick look
<mwhudson> thumper: two thoughts (1) i kinda hate defining __cmp__, is there no way you can compute a keys (probably a tuple) from the various kinds of linked branch and sort by that?
<mwhudson> (2) i think you probably need to write a lot of tests indeed, but with a bit of infrastructure you should be able to write really short tests
<mwhudson> thumper: more thoughts after lunch if you want
<thumper> mwhudson: ok, ta
<thumper> mwhudson: perhaps a call when I'm back
 * thumper is running away now
<mwhudson> thumper: ok
 * mwhudson lunches
<kfogel> wgrant: sorting-within-contributor implemented, as requested
<kfogel> jml: did you do https://edge.launchpad.net/~launchpad-users/+mailinglist-moderate ?
<kfogel> jml: there are suddenly no messages left
<jml> kfogel, I didn't
<jml> kfogel, maybe mrevell did
<jml> kfogel, sorry for the lag, xchat lacks proper notification powers.
<jml> mwhudson, I'm thinking of writing a script that pulls the branches in sourcedeps.conf.
<mwhudson> jml: that would probably be nice
<jml> mwhudson, rather than relying on rsync, which I dislike for this purpose.
<jml> mwhudson, if I wrote such a thing, it's natural home would be in utilities
<jml> but I think I would like to write tests for it also
<jml> and I don't know where to put them.
<mwhudson> jml: utilities/tests ?
<mwhudson> i guess it's not a package
<jml> mwhudson, yeah.
<mwhudson> jml: lib/devscripts ?
<mwhudson> jml: have utilities/foo be an entrypoint for devscripts.foo.main
<mwhudson> ?
<jml> mwhudson, I think I could live with that.
<jml> (way too many clicks to browse code for a project btw)
<jml> mwhudson, I'm not sure how to make bin/test actually find the tests though.
 * jml saves that problem for later.
<stub> Where are the tests?
<jml> stub, lib/devscripts/tests
<stub> IIRC you will need an __init__.py in both lib/devscripts and lib/devscripts/tests
<jml> yeah, done that.
<stub> Then they should be picked up
<jml> but still no joy
<jml> './bin/py ~/src/twisted/trunk/bin/trial devscripts' finds them.
<stub> The .py has a test_suite() ?
<wgrant> kfogel: Great. Thanks.
<kfogel> wgrant: I'm about to send in a merge proposal moving that script to launchpad's utilities/ subdir, where it really belongs.
<wgrant> kfogel: Even better.
<wgrant> Hmmm.
<kfogel> wgrant: ?
<wgrant> The last two db-devel merges were 1.5 hours apart. It's been 2.5 hours since the last one, and this one should have had one of my revs in it.
<wgrant> Although I haven't been emailed, so I guess I'm safe.
<kfogel> wgrant: are you not seeing your change on the branch, or is it just not on the Contributions page?
<wgrant> kfogel: It's on the contributions page. I'm just fearing that buildbot failed it.
<kfogel> wgrant: *nod*
<kfogel> hmmmm
<kfogel> wgrant: is it just me, or have pages like https://code.edge.launchpad.net/~launchpad-pqm/launchpad/devel lost their "Source Code" link, such that files are harder to reach now?
<kfogel> wgrant: I wasn't following all the UI discussion..
<wgrant> kfogel: They have. The page has been ported to 3.0, but not yet redesigned, so it's just a transitional issue. Bug filed.
<kfogel> wgrant: heh, you're way ahead of me.  ok, thanks
<jml> stub, devscripts/tests/test_sourcecode.py does
<jml> stub, but nothing else does.
<stub> Yup.
<stub> Thats the obvious stuff I can think of.
<stub> You can try making test_sourcecode unimportable and see if bin/test test_sourcecode even tries to load the module
<mwhudson> jml: it looks like we only search the 'canonical' and 'lp' packages
<mwhudson> jml: look at buildout-templates/bin/test.in:141
<jml> ahhh, thanks.
<jml> mwhudson, yay updates
<mwhudson> jml: the FasterTests ones?
<jml> mwhudson, yeah.
<jtv> So... PQM still stuck but in a different way now?
<mwhudson> jtv: i think spm must be doing a cherrypick
<jtv> mwhudson: your branch is stuck behind mine in PQM, and mine's been in there for maybe two hours
<spm> aye. for thumper and cprov i think
<thumper> spm: ta
<wgrant> Ahh.
<jtv> good morning btw :)
<wgrant> The ensurePerson cprov cherrypick?
<spm> heh. morning whinger... errr jtv
<jtv> sorry for being a little jumpy about it...  I just hate it when landing a branch drags out for days and days.\
<spm> jtv: :-) is cool.
<jtv> How is our build/landing machinery doing in general?
<spm> jtv: getting there. mwhudson is going great guns on making my life a misery rebuilding ami's .... errr... ^W^W^W^W^W^W^W improving buildbot for you guys
 * wgrant petitions for public AMIs.
<jtv> spm: both worthy goals
<spm> ROFL (jtv, not to wgrants request)
<jml> wgrant, remind me... did I file a bug about that?
<jml> wgrant, if not, did you?
<jml> wgrant, if not that, did someone else?
<spm> wgrant: hmmm. possibly. I know that we have some stuff in there that is not releasable, based on an update I saw go in yesterday.
<wgrant> jml: You filed a bug.
<wgrant> jml: You commented on there two days ago, IIRC.
<kfogel> jml: all you :-)  https://code.edge.launchpad.net/~kfogel/launchpad/add-community-contributions-script/+merge/11417
<mwhudson> i guess as well as a public image, we also need a replacement for that rsync from devpad before random people can run ec2test
<wgrant> mwhudson: That too.
<wgrant> But an image would allow me to hack it.
<mwhudson> it's time for a new image anyway i guess
<spm> mwhudson: apart from poking fun at your scripts, coding styles, and generally being rude; what have I ever done to you that I deserve this??? :-P
<jml> kfogel, cool :)
<jml> mwhudson, well, that's maybe what I'm trying to do here, actually.
<mwhudson> spm: i can make ec2 images myself
<mwhudson> jml: right
<mwhudson> spm: ec2test, i mean
<kfogel> mwhudson: what exactly do we rsync from devpad still?
<wgrant> Just sourcecode, AFAICT.
<jml> kfogel, there'll be a little bit of a delay on the review, since I can't really think straight right now.
<wgrant> It looks like it'll work (but be awfully slow) if that rsync is removed.
<kfogel> jml: np
<kfogel> jml: I just want to make sure you enjoy the naming conventions in the script.  "ExternalContributor" was a bit long for a class name, same with similar variables throughout the script, so I... uh, had to shorten it a bit.
<mwhudson> i wonder if the aws console will work today
<ajmitch> kfogel: so that's what you think of outside contributions... :)
<kfogel> ajmitch: JUST a coincidence.  Really.
<mwhudson> (the launch instance dialog of elasticfox is too tall for my screen :()
<wgrant> kfogel: I hand you a DistroArchSeriesBinaryPackageRelease and a SecureBinaryPackagePublishingHistory.
<wgrant> (yay, soyuz)
<kfogel> wgrant: these are rather heavy...
<thumper> is there a storm identity expression?
<thumper> one that I can add to a list of expressions
<thumper> that always is true?
<jml> thumper, don't know.
<jml> #storm?
 * thumper looks at his expression generation again
<stub> thumper: Just use True?
<stub> SQL("TRUE") would work I guess if you can't find anything nicer that works.
<jml> why don't dicts have set methods!
<jml> python sucks. let's rewrite Launchpad in Ada.
<spm> pqm open again <== jtv
<jtv> spm: thanks!
<spm> jml: you what!
<jml> heh heh
<spm> jml: I'm just dissappointed you didn't suggest php
<spm> you're a php coder from memory?
<jml> spm, I thought we'd agreed that you'd never mention that.
<spm> s/you're a php coder from memory\?//
<adeuring> good morning
<wgrant> Did I murder buildbot, or is it just unhappy about the cherrypick?
<noodles775> wgrant: 12121 tests so far - all passed?
<wgrant> noodles775: Oh, OK. It's just taking a very long time, then.
<bigjools> morning all
 * rockstar does the tests pass dance
<thumper> is someone handling the merge conflict for stable -> db_devel?
 * thumper heads to the supermarket
<thumper> I'll do it then
 * thumper puts off the supermarket for a few minutes
<thumper> only fair since my work caused the conflict
<thumper> fix in pqm
 * thumper really off to the supermarket now
<bigjools> jtv-eat: ping for when you're back
<bigjools> anyone else getting 502 errors on https://xmlrpc.edge.launchpad.net/bazaar/ ?
<mrevell> Morning
<gmb> Goooooood morning from London, Launchpadders!
<gmb> bigjools: Yep, I'm seeing those too now.
<jtv> bigjools: back
<bigjools> jtv: OTP, one sec
<bigjools> jtv: right, I had a conflict in pagetitles.py and noticed you've added something
<bigjools> "person_vouchers"
<jtv> bigjools: nope, not mine
<jtv> bigjools: but I had a conflict in pagetitles when I removed something...
<bigjools> jtv: it could be a mistake
<jtv> maybe I resolved the conflict in the wrong way
<bigjools> jtv: because it's in a landing you made
<bigjools> probably
<bigjools> I can remove it again, no problems
<jtv> Thanks
<bigjools> cool, cheers
<wgrant> Hmmm, with what did my commit conflict in db-devel?
<gmb> launchpadlib's giving me a 502 when I call login_with, too, now. That's vexing.
<bigjools> could be a bad appserver
<wgrant> Yes, an edge appserver is dead.
<gmb> balls.
<wgrant> It happens rather too frequently :(
<gmb> There goes my lazy file-seven-bugs-at-once strategy.
<deryck> Morning, all.
<flacoste> morning launchpad
<sinzui> salgado: are you availble for our call?
<salgado> sinzui, the stand up call?
<sinzui> salgado: yes
<salgado> sinzui, yep, just restarted skype
<barry> reviewers -> #launchpad-meeting in 1m
<gmb> Brb (reboot)
<maxb> oh yuck. edge calls itself 2.2.6 if you hit a 2.0-style page but 2.2.7 if you hit a 3.0-style page
<sinzui> I wish there was an easy way to include the milestone in the template
<noodles775> salgado: is there some documentation about the breadcrumbs? Should I add a breadcrumb for a collection view so that I don't see '+packages'?
<noodles775> (I can see lots of easy examples of how breadcrumbs are added and tested, but wasn't sure if that's what I should do for a collection view).
<salgado> noodles775, the correct behaviour would be for the last breadcrumb there to use the page tite (instead of name) as its text, but you shouldn't worry about that as this will be fixed when I close bug 423691
<mup> Bug #423691: Use a page's title instead of its name on breadcrumbs <Launchpad Foundations:In Progress by salgado> <https://launchpad.net/bugs/423691>
<ubot3> Malone bug 423691 in launchpad-foundations "Use a page's title instead of its name on breadcrumbs" [High,In progress] https://launchpad.net/bugs/423691
<mup> Bug #423691: Use a page's title instead of its name on breadcrumbs <Launchpad Foundations:In Progress by salgado> <https://launchpad.net/bugs/423691>
<mup> Bug #423691: Use a page's title instead of its name on breadcrumbs <Launchpad Foundations:In Progress by salgado> <https://launchpad.net/bugs/423691>
<noodles775> salgado: ah, great. Thanks!
<danilos> gary_poster: hi, do you know how do I force a rebuild of one egg?
<gary_poster> danilos: remove it from eggs and run bin/buildout
<gary_poster> and hi :-)
<danilos> gary_poster: I tried that, apparently I have to change .installed.cfg as well, right?
<danilos> :)
<gary_poster> danilos: mm, no you should not have to
<gary_poster> danilos: so...an egg built badly somehow?
<danilos> gary_poster: well, the egg built badly with a few warnings (bzr 1.18), perhaps because I had 'make lint' and 'make run' running at about the same time
<gary_poster> (the only time I ever touch .installed.cfg is to remove it when I'm doing a make clean or equivalent; it is pertinent to the application itself, not to the eggs)
<gary_poster> danilos: so what happened when you removed the egg from eggs and ran bin/buildout?
<danilos> gary_poster: well, it would not try to rebuild even if I just removed an egg from eggs/
<danilos> gary_poster: hum, it seems to work now, I guess it's "operator error" then :)
<gary_poster> danilos: :-) ok, I was trying to figure out what to do, so good. ;-)
<kfogel> Anyone have time to do a very easy review?  This MP just adds a script to Launchpad utilities/.  The script is something we are already using; this just moves it from ~kfogel/+junk into Launchpad: https://code.edge.launchpad.net/~kfogel/launchpad/add-community-contributions-script/+merge/11417
<barry> deryck_: png
<barry> er, ping
<barry> or perhaps intellectronica, gmb, allenap
<deryck_> barry, pong
<intellectronica> ?
<barry> deryck: hi.  i'm chr today and i'm looking at a question about a guy who's trying to file a bug against ubuntu.  it's timing out for him, and i've tried and it's timing out for me every time
<barry> intellectronica: thanks (was looking for a "bugs" guy :)
<barry> deryck: https://lp-oops.canonical.com/oops.py/?oopsid=1348C2086
<barry> deryck: i think it's pretty bad if you can't file a bug on ubuntu ;)
<intellectronica> barry, deryck: that's the dupe search timing out. we've seen that happen before
<intellectronica> the workaround is to use a more specific summary, i think
<deryck> yeah, like intellectronica says, barry.  It's a dupe search problem.
<intellectronica> although i must say that the query looks quite specific already
<barry> deryck, intellectronica i just tried "fnords are hinky" on staging.  timeout
<dhillon-v10> deryck: hi, I finally found you
<barry> deryck, intellectronica is there a bug open on this already?
<intellectronica> barry: but that's a conspiracy ;)
<barry> intellectronica: :)
<dhillon-v10> deryck: I am Vikram, I hope you recognize me.... :)
<deryck> maybe there are a lot of fnords bugs ;)
<deryck> dhillon-v10, hi
<dhillon-v10> deryck: alright so let's get started here
<barry> deryck: it's good at least that you see the fnords
<deryck> dhillon-v10, I should warn you... :)  I have a call in ten minutes or so
<dhillon-v10> deryck: oh sorry, I will talk to you later
<deryck> dhillon-v10, I can chat in about 1.5 hour.  if you can be around then
<dhillon-v10> deryck: alright sure
<dobey> anyone know what the best way to support multiple versions of the LP API is?
<rockstar> abentley, you broke Tarmac...
<abentley> rockstar: Really?  I was only expecting to break mad.
<rockstar> dobey, I think leonardr has put some thought into it, but we currently only have one version of the API -> beta
<rockstar> abentley, yeah, dobey added some isPersonValidReviewer stuff.
<abentley> rockstar: Sorry.  I don't really remember that change.  Was it just renaming?
<dobey> abentley: yeah
<rockstar> abentley, :)  I'm just teasing you.  Serves Tarmac right for using some beta crap.
<dobey> oh i think it moved too
<micahg> kfogel: about that bug, how about bookmark queries/
<micahg> kfogel: what do you think of that idea of bookmarked queries?
<micahg> then you can use post for everything
<micahg> in the app at least
<micahg> and leave $_GET for the people who want to pull data
<kfogel> micahg: can you explain in more detail?  I'm not sure I understand.
<dobey> cprov: hi! is the lpia target going away for PPA builds anytime soon?
<cprov> dobey: I don't know, why should it ?
<cprov> dobey: it's supported in hardy LTS, isn't it ?
<dobey> cprov: i thought ubuntu wasn't doing lpia any more?
<dobey> cprov: oh maybe. but do karmic builds need to happen on it?
<cprov> dobey: they will, as long as lpia still marked as 'supported'. Better ping someone from ubuntu-platform, they will know the details.
<dobey> cprov: ok
<kfogel> micahg: I mean, this is more about the URLs themselves -- making them more palatable when pasted into emails or other forums.
<micahg> kfogel: that would make the url smaller https://bugs.launchpad.net/?bookmarkq=1589654
<barry> losa ping
<mthaddon> barry: hi
<barry> mthaddon: hi.  can you check the mailman logs on forster to see if the creation of ~launchpad-reviewers is hosed?  this is a new creation request for a list that was deactivated and purged, so something could have gone wrong
<barry> salgado: ^^
<kfogel> micahg: AH, I see what you mean.
<Chex> barry: I can look, hold please
<salgado> thanks barry
<barry> Chex: thanks
<mthaddon> barry: which logs are you interested in?
<kfogel> micahg: I think implementation-wise that would have a greater impact on the rest of Launchpad (because now we have to have the DB remember all those bookmarks), and it would make the meaning of the URL more obscure (== less hackable).
<kfogel> micahg: IOW, I think jml's suggestion in https://bugs.edge.launchpad.net/malone/+bug/325982/comments/2 is a cleaner fix.
<ubot3> Malone bug 325982 in malone "Search URL is long and hard to paste" [Low,Triaged]
<mup> Bug #325982: Search URL is long and hard to paste <trivial> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/325982>
<mup> Bug #325982: Search URL is long and hard to paste <trivial> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/325982>
<barry> mthaddon, Chex let's start with xmlrpc
<kfogel> ubot3, mup: can't we just get along?
<ubot3> kfogel: Error: I am only a bot, please don't think I'm intelligent :)
<micahg> kfogel: but it wouldn't shorten it much
<kfogel> ubot3: don't worry, that was never a danger.
<ubot3> kfogel: Error: I am only a bot, please don't think I'm intelligent :)
<kfogel> micahg: well, the length would now be proportional to the complexity of the query, sure.
<kfogel> micahg: all the unspecified fields  would be left off.
<micahg> yes, but most queries do have a lot ofs option
<mthaddon> barry: https://pastebin.canonical.com/21971/
<barry> mthaddon: thanks
<kfogel> micahg: I think the idea behind the bug is that when the specified value is the same as the default, you can leave it off the URL.
<kfogel> micahg: but you do have a point -- what if the defaults change?  Should the meaning of the query then change?
<barry> salgado, losa: hmmm... that's odd.  looking
<micahg> because the url has the defaults right now
<kfogel> micahg: yeah, the state is carried in the URL, not in Launchpad -- even if Launchpad's search query defaults change, the URL will still mean the same thing.
<micahg> right, that's what I meant by not shortening it too much
<barry> mthaddon: can you take a look and see what, if any mailing list artifacts exist for launchpad-reviews?  especially: list and archive
<mthaddon> barry: there's a config.pck (and .last) but nothing in archives
<kfogel> micahg: this ties into another bug we had about making a dedicated tinyurl service, which got swamped by discussion of why it's a bad idea IIRC.  Let me see if I can dig it up.
<barry> mthaddon: what's the date on that config.pck
<micahg> kfogel: it seems like if people could "send" queries to other users, that would help also
<micahg> then the user can see a list of their queries and what they are
<mthaddon> barry: -rw-rw---- 1 launchpad launchpad 4.9K 2009-09-09 20:21 config.pck
<barry> mthaddon: if i'm doing my utc math right, that's 3 minutes ago
<kfogel> micahg: https://dev.launchpad.net/Wishes/PersonalDashboard  :-)
<mthaddon> barry: yep
<barry> mthaddon: ok, that's bad.  for some reason the status of that resync isn't getting transmitted to lp
<barry> mthaddon, salgado gotta think...
<micahg> kfogel: it would tie in great there :)
<mrevell> night all
<kfogel> micahg: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/317136
<mup> Bug #317136: Launchpad should offer a "Get tinyURL link" option for each page <Launchpad Foundations:New> <https://launchpad.net/bugs/317136>
<ubot3> Malone bug 317136 in launchpad-foundations "Launchpad should offer a "Get tinyURL link" option for each page" [Undecided,New]
<mup> Bug #317136: Launchpad should offer a "Get tinyURL link" option for each page <Launchpad Foundations:New> <https://launchpad.net/bugs/317136>
<barry> salgado: there's nothing in this newly recreated mailing list that we need to keep, right?
<salgado> barry, right
<barry> salgado: cool.  mthaddon i say we blow away lists/launchpad-reviewers and watch the logs.  the resync should fail gracefully.  then we'll tweak the db to re-purge that mailing list and try again
<kfogel> micahg: I'm going to add a comment there, with our discussion (paraphrased), and make sure the two bugs know about each other.
<mthaddon> barry: I'm just about to head out to lunch - can it wait til after, or can another losa maybe take over?
<micahg> sounds good
<barry> mthaddon: either way is fine.  i'm here for hours still
<micahg> I would suggest sticking with numeric urls though to speed up the query from the db
<mthaddon> barry: k, thx - if someone hasn't picked it up by the time I back from lunch we'll do it then - thx
<micahg> kfogel: ^^
<barry> mthaddon: +1
<kfogel> micahg: maybe; might be overoptimization, I don't know enough about the DB to say for sure.
<kfogel> barry: got time to Quickest Review Evah?  https://code.edge.launchpad.net/~kfogel/launchpad/add-community-contributions-script/+merge/11417
<kfogel> barry: it's just adding a script into utilities/, and the script is already in use (it's just moving over from my +junk).
<micahg> kfogel: numeric column ids are normally the fastest in most engines AFAIK
<kfogel> micahg: oh, I agree they're faster, the question is whether the cost matters -- i.e., if that's just not a bottleneck, then speeding it up doesn't gain us much.
<barry> kfogel: i'm supposed to be chr'ing today :)  if you can't get the ocr to look, ping me
<kfogel> barry: my bad, sorry!
<micahg> kfogel: well, think about how many users will want short urls and the use of LP, isn't the DB already a bottleneck for LP?
<micahg> especially if you have to store 41 trillion like you posted, wouldn't you want any speed increase possible at that point?
<kfogel> micahg: not necessarily; it all has to do with usage patterns.  note, for example, that TinyURL.com doesn't use numerics, although they could.  Possibly they're translating alphanumerics to a number and then using that, who knows.  Lots of strategies available.  I think the way to do it is to determine what we want users to see, first, and then work backwards from there.
<micahg> ok
<kfogel> micahg: (i.e., "digits" != "number")
<kfogel> and "alphanumeric" != "!number" :-)
<micahg> I guess I think more in the implementation that presentation :)
<micahg> *than
<kfogel> micahg: my feeling is, the user is only going to see the presentation, so... :-)
<micahg> right
<Chex> barry: I am looking at the config files for launchpad-reviewers, I can work on this for you if you like..
<barry> Chex: -> #launchpad-code
<micahg> kfogel: is there anything in progress to have bugs relationships like in bugzilla?
<kfogel> micahg: hmm, not sure.  Do we have anything filed on it?
<micahg> just foundd it bug 95419
<mup> Bug #95419: Record dependencies between bugs <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/95419>
<ubot3> Malone bug 95419 in malone "Record dependencies between bugs" [Medium,Triaged] https://launchpad.net/bugs/95419
<mup> Bug #95419: Record dependencies between bugs <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/95419>
<mup> Bug #95419: Record dependencies between bugs <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/95419>
<kfogel> micahg: yeah, just added my +1 there.  Now, that is not a bug we can mark as "trivial", unfortunately.
<micahg> yeah
* kfogel changed the topic of #launchpad-dev to: is Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call reviewers available, see https://dev.launchpad.net/ReviewerSchedule | Use http://paste.ubuntu.com/ for pastes | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html | Buildbot is down
<rockstar> mwhudson, abentley, who's hosting the standup today?
<mwhudson> rockstar: i guess you're closest to the geographical centre...
<abentley> mwhudson: Yeah, but his router is suspect, so I'll host.
<rockstar> mwhudson, I'm also closest to some sort of bandwidth blackhole recently.
<mwhudson> rockstar, abentley: oh ok
<mwhudson> good morning all, btw
<mwhudson> flacoste: didn't really get around to looking at the graphing stuff yesterday so no questions yet :)
<flacoste> morning mwhudson
<flacoste> mwhudson: no problem
<flacoste> mwhudson: one thing you should know, the DB i gave you as all the data from the previous buildbots, no stats from the new one in the data center yet
<mwhudson> flacoste: oh ok
 * mwhudson wrote a terrible bash/awk thing to get stats out of the data centre buildbot yesteday
<flacoste> depending on what you were looking for, it's possible that this one already has what you need
<rockstar> barry, we doing a meeting today?
<flacoste> mthaddon: jamesh landed the OpenID XRDS + cookie changes yesterday
<mthaddon> flacoste: ok, so I guess we can put that into "production" after the next rollout - do you know how it all works?
<flacoste> mthaddon: which part?
<mthaddon> flacoste: where the file is served from and/or if any change in apache setup is needed to get things working
<flacoste> mthaddon: there is apache changes needed for sure, similar to what we did in the testing environment, basically a rewrite url iirc
<mthaddon> flacoste: ok, but in testing we were using a static file, and aiui, this branch has the file managed in the tree somewhere, so we'd need to know where that is
<flacoste> mthaddon: hmm, actually, he didn't add a file to the tree
<mthaddon> flacoste: ok, I guess we need to check with him about the details - if you can have him add them to the LPS wiki page under "Unusual Rollout Requirements" that'd be best
<flacoste> mthaddon: there is only a dynamic template, but the result can be cached
<mthaddon> flacoste: ah, okay - so we'd just need to know the URL and confirm it's squid cache-able
<flacoste> mthaddon: probably better to generate the static file and serve it right from apache, no need to send it over to squid
<flacoste> mthaddon: i'll make sure to add appropriate instructions to LPS
<mthaddon> flacoste: cool, thx
 * jml kicks xchat
<mrevell> kfogel, jml: Skype?
<kfogel> mrevell: hey, yup, plugging in now
<jml> hey, yes.
<kfogel> jml, mrevell: ready
<jml> kfogel, I can't see you online.
<jml> ok...
<jml> Science Time!
<spm> mwhudson: I object. it wasn't so much awful as eye/mind rending. (* mwhudson wrote a terrible bash/awk thing...)
<barry> rockstar: yes.  we'll do a meeting today.  8m
<mwhudson> barry: ping
<barry> mwhudson: fsck
<barry> mwhudson, jml, thumper, rockstar -> #launchpad-meeting in 2m
<jml> barry, I might be a little late, sorry.
<mwhudson> barry: thumper isn't around today
<barry> jml: i'll be here for a while, do you want to push this back a bit?
<jml> barry, if possible
<wgrant> Looks like Ohloh is happy with RF... 900 revs to go.
<barry> rockstar, mwhudson is that cool with you?  wanna say 22m?
<barry> jml: will that work for you?
<mwhudson> barry: fine with me
<jml> barry, wfm
 * barry sets his alarm
#launchpad-dev 2009-09-10
<mwhudson> jml: did you land your lib/devtools branch yet?
<jml> mwhudson, not even finished yet, I'm afraid.
<mwhudson> jml: i find myself wanting to move bits of ec2test in there
<jml> mwhudson, heh
<jml> mwhudson, please feel free
<jml> I'll suck up the conflicts.
<mwhudson> jml: i have this feeling i'll almost certainly want to talk to you about libraryizing ec2test in a while, are you going to be mostly online today?
<jml> mwhudson, I am.
<mwhudson> great
 * mwhudson looks at the code again
<jml> heh heh
<rockstar> Note to self: Never, ever, ever, ever do `easy_install launchpadlib` ever again.
<jml> 'sagi python-launchpadlib' works in karmic
<jml> (finally!)
<jml> very happy today because of that.
<mwhudson> rockstar: s/launchpadlib//
<rockstar> mwhudson, virtualenv
<mwhudson> rockstar: ah, ok
<jml> mwhudson, you mentioned last night that we only look in 'lp' and 'canonical'
<jml> mwhudson, I guess I ought to add 'devscripts' to that, if I can figure out how.
<mwhudson> jml: i think it's going to be pretty easy if you find the relevant part of the code
<jml> buildout-templates/bin/test.in looks promising.
<jml> wuu.
<jml> compare and contrast: http://paste.ubuntu.com/268282/
<mwhudson> jml: ugh
<jml> mwhudson, that's right, 2 seconds of zopey waiting.
<jml> mwhudson, actually, when you call me re the ec2test stuff, I'd like to talk with you a little about this branch.
<mwhudson> jml: would in about an hour work for you?
<jml> mwhudson, yeah.
<mwhudson> cool
 * mwhudson lunches
 * mwhudson sings the "someone broke the buildbot" song
<jml> the blame list is surprisingly empty
* bac changed the topic of #launchpad-dev to: is Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call reviewers available, see https://dev.launchpad.net/ReviewerSchedule | Use http://paste.ubuntu.com/ for pastes | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html
<jml> mwhudson, yo
<mwhudson> jml: hi
<mwhudson> jml: life's throwing me a curve ball, biab
<jml> mwhudson, np
<jml> cprov, I've done a bunch of stuff with nascentupload
<jml> cprov, I'd like to hook branches up to it now
<jml> cprov, which involves inferring an archive
<cprov> jml: you sound scary sometimes.
<jml> I'm not scary.
<cprov> jml: most of the time.
<jml> cprov, not branches up to nascentupload itself, but rather up to verify_upload.
<cprov> jml: right, calculating branch push perms with verify_upload, that was the original plan, wasn't it ?
<jml> cprov, that's right
<jml> cprov, but I can't seem to recall how we figured out the archive based on the SuiteSourcePackage
<jml> there are two choices I can see. ssp.distribution.main_archive
<jml> or something handwavey that looks at the publishing history
<cprov> jml: defaulting Primary, IIRC
<jml> cprov, that's ssp.distribution.main_archive, right?
<cprov> jml: the current SPBranch url doesn't support PPAs
<cprov> jml: yes, that's it.
<jml> cool.
<jml> easy as.
<jml> cprov, thanks.
<cprov> jml: but it may change on partner uploads
<cprov> jml: 'partner' component == PARTNER archive
<jml> hmm.
<jml> cprov, how do we get the component from a SuiteSourcePackage?
<cprov> jml: the component of the latest publication
<jml> cprov, is there already a function or method that knows that parter component implies partner archive?
<cprov> jml: there is a hack in nascentupload
<cprov> jml: NU.overrideArchive()
<jml> cprov, you owe me sooo many caipirinhas
<cprov> jml: countless.
<wgrant> cprov: When is that cesium ensurePerson CP happening?
<cprov> wgrant: it won't be released until 3.0 rollout, AFAICT
<wgrant> cprov: Oh, damn.
<cprov> wgrant: iron.cc (debian-imports) is fixed.
<wgrant> cprov: But Karmic is out of luck?
<wgrant> (as well as the rebuild -- I guess I'll get my script to ignore upload failures for now)
<cprov> wgrant: sort of, maybe the 'me too' in the bug report can help to make a point.
<cprov> wgrant: it's actually very simple, since the revision was already CPed.
<wgrant> cprov: I suppose. I will me-too.
<cprov> wgrant: CP requested, easier than I thought.
<wgrant> cprov: Thanks!
<wgrant> What sort of process is involved in that?
<cprov> wgrant: rolling the current production branch to a new set of machines
<wgrant> cprov: Well, obviously, but I imagine there's a fair bit of preparation and approval paperwork.
<cprov> wgrant: in this case all soyuz upload frontends (ppa, builddmaster and ubuntu)
<cprov> wgrant: the paperwork was already done for the CP on the debian-imports machine
<wgrant> cprov: Ah, right.
<cprov> wgrant: from the developer PoV it is: 1) get it reviewed and landed on lp:devel 2) talk to the release-manager and convince him the CP is needed.
 * mwhudson is still a bit unconvinced of the benefit of targeted fixes vs the downside of having different services running different code
<mwhudson> not my field though i guess
<wgrant> That has caused confusion in the past.
<wgrant> cprov: Why are the stats on /builders down the side rather than along the top?
<wgrant> If they were along the top, the builder listing would be much more readable as status lines would wrap less.
<cprov> wgrant: but OTOH to access the builder list you would have to always scroll down
<cprov> wgrant: also the official summary is bigger than the ppa summary, so an ugly whitespace would appear if they were in the same grid row.
<wgrant> cprov: Some of it would fit, and the builder list is always multiple screenfuls now.
<wgrant> cprov: Hm, damn, true.
<wgrant> It seems like such a waste.
<wgrant> And it's hard to scan the table.
<wgrant> But I can't now see where else to put it.
<cprov> wgrant: I still have to submit a change to fix the 'status' column content
<wgrant> cprov: What's wrong with it?
<cprov> doesn't show 'MANUAL' mode properly neither private-builds
<wgrant> cprov: Works OK for private builds, AFAICT. (or at least as well as HiddenBuilder makes it...)
<cprov> wgrant: but uses the NOT-OK icon, which is NOT-GOOD :)
<wgrant> cprov: Well, I'd say HiddenBuilder is really ungood!
<cprov> wgrant: the concept itself or the implementation ?
<wgrant> cprov: Concept.
<cprov> wgrant: the other alternative is to user an IBuilder macro, which seems much clearer conceptually, since IBuilder.status belongs to the view domain.
<wgrant> cprov: IBuilder.status makes me sad, yes.
<wgrant> If that goes into the view, and builder.logtail gets properly secured, I don't see any reason for HiddenBuilder to exist.
<cprov> wgrant: being on the view means we can't use it in zopeless mode (easily) but that's not a problem anymore.
<wgrant> It had me very confused when I first looked around that area of code.
<wgrant> cprov: Why was it used in Zopeless stuff?
<cprov> wgrant: when we had scripts/ftpmaster/buildd-monitor.py (a shell-like debug tool for the buildfarm)
<cprov> for instance.
<wgrant> cprov: Ah, right.
<cprov> wgrant: but as I said, not a problem anymore, it's gone.
<cprov> wgrant: getting rid of HiddenBuilder and IBuilder.status looks like a nice weekend task.
<wgrant> cprov: I'd not object to killing it myself.
<cprov> wgrant: if you have time, that would be great!
<wgrant> cprov: I will have ample time after finishing exams tomorrow.
<wgrant> I guess then the archive and sometimes source can be linked to from the status, too.
<cprov> wgrant: yes, once it's done in the view domain it becomes much easier.
<cprov> okay, bed-time over here. G'night all!
<wgrant> cprov: Night. Thanks.
<cprov> wgrant: good luck with your tests and we can talk more about the IBuilder changes tomorrow.
<jml> mwhudson, https://code.edge.launchpad.net/~jml/launchpad/better-sourcecode-update
<mwhudson> argh
<mwhudson> hm
 * mwhudson fixes, but wonders how that ever worked
<jml> mwhudson, what are you fixing?
<mwhudson> jml: i think i figured it out
<mwhudson> jml: ec2test imports plugins without enabling plugins
<jml> mwhudson, oh yeah.
<jml> mwhudson, I saw that behaviour in the pasnt.
<jml> I'm not a huge fan of ec2test's plugin dependencies, tbh.
<mwhudson> one thing at a time :)
<jml> heh
 * jml does some scienc
<jml> e
<jml> mwhudson, interestingly, pulling each branch one after the other with no actual changes takes substantially longer than rsyncing with no changes.
<mwhudson> jml: multiple ssh negotiations?
<jml> mwhudson, quite possibly.
<jml> 5m4s vs 12s
<wgrant> I find that SSH negotations to bazaar.lp.net can take up to 10 seconds at bad times.
<mwhudson> jml: if you're using bzrlib you can probably get it to reuse the transport
<mwhudson> jml: is your branch on lp up to date?
<jml> mwhudson, not quite
<mwhudson> jml: give it a push and i'll see if i can give you a hint :)
<jml> mwhudson, now that I can commit safely (and thus be allowed to push again!), sure
<jml> done.
<mwhudson> jml: oh
<jml> mwhudson, what?
<mwhudson>     cmd_pull().run_argv_aliases(['-d', destination, branch_url])
<mwhudson> bit hard to jam transport sharing in there
<jml> mwhudson, yeah. it was only a stop-gap.
<jml> figuring out the actual API for pulling required deciphering cmd_pull, which I'm not really up for right now.
<mwhudson> branch.pull(otherbranch)
<jml> does that update the working tree?
<mwhudson> jml: basically have a list around for the duration of the script
<mwhudson> jml: pass it as possible_transports when you open the branch
<mwhudson> jml: not sure actually
<mwhudson> jml: so you're probably right to be cautious
<jml> :(
<jml> because the config uses lp: URLs, sharing transports isn't as easy as all that.
<mwhudson> not sure i get the point
<jml> oh never mind.
 * mwhudson EODs
<jml> it's because an edge server is having a hissy fit :)
<jml> mwhudson, ciao
<mwhudson> jml: i think bzr+ssh://bazaar.launchpad.net/~mwhudson/launchpad/libraryize-ec2test is probably good to land, ish at least btw
<mwhudson> jml: maybe you could take a look?
<mwhudson> jml: it's pretty boring so far
<jml> mwhudson, sure
<jml> mwhudson, fwiw, sharing the transports makes it substantially faster, but still slower than rsync. I think the extra time is lp: resolution.
* jml changed the topic of #launchpad-dev to: Launchpad is having issues | is Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call reviewers available, see https://dev.launchpad.net/ReviewerSchedule | Use http://paste.ubuntu.com/ for pastes | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html
<adeuring> good morning
<maxb> jml: Hi, how did we leave things w.r.t you fixing the broken stack-on location for the meta-lp-deps branches?
 * maxb wonders if maybe they should be owned by a team that lets me write to them :-)
* spm changed the topic of #launchpad-dev to: is Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call reviewers available, see https://dev.launchpad.net/ReviewerSchedule | Use http://paste.ubuntu.com/ for pastes | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html
<maxb> How do you trigger the default branch stacking? Is it necessary to have a development focus?
<maxb> Does anyone fancy fixing some broken stacked_on locations for me or shall I mail the list?
<maxb> Now we have mad, should I assume diff-output in merge cover letters is redundant?
<wgrant> maxb: It's not because of mad, but yes.
<wgrant> maxb: LP will generate the diff and email it out.
<maxb> Perhaps we should update the template cover letters on the wiki?
<wgrant> maxb: Certainly a good idea.
<deryck> Morning, all.
<jtv> hi deryck
<jtv> barry, ping
<gmb> Hmm. Pulling devel is taking a while. Maybe I've just got a shonky connection.
<sinzui> OMG! I just got autoresponder from feedback@...I do exist
 * sinzui wonders if his email will show up
<mrevell> heh
<gmb> I would like to take this opportunity to say AAAAAAAAAAAH.
<gmb> (Because primal screaming in Starbucks is generally frowned upon)
<deryck> no wonder everyone looked at me strange last time I worked in Starbucks
<deryck> flacoste, ping
* kfogel changed the topic of #launchpad-dev to: is is Launchpad Development Channel | Week 2 of 3.0 | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html
<flacoste> hi deryck
<barry> bac: no karmic goodness yet.  it took all night just to get windows updated.  this morning wubi wouldn't work for either karmic or jaunty, but i booted to the live cd and was very happy that unlike vista, jaunty had no problem with my kvm switching.  i'll hack on this more tonight
<gmb> deryck: WRT bugtask-index.pt, I think there's some mileage in converting it to 3.0 and landing that before working on the redesign. It (should) only be a couple of minor tweaks to get it up to spec, AFAICT. That okay with you?
<deryck> gmb, of course.
<gmb> Cool.
<gmb> deryck: Of course, if I'm wrong and it ends up being more painful than I expected, I'll just go straight for the redesign since we're not really losing anything in that case.
<deryck> gmb, right, makes sense.
<deryck> gmb, you might just consider the questions of what actions go in the portlet, etc. based on the conversion rules.  That is a large part of the redesign idea anyway.
<gmb> deryck: Right. I'll look at that. After I get the portlets to appear in the right place, that is...
<deryck> heh.  understood.
 * gmb -> afk for a bit to pick the car up; back later on
<gmb> Oh, wait. Might be an idea to make sure there's a train to catch before buggering off.
 * gmb stays put
<flacoste> barry: finally it seems zope will check-in your doctest fix!!!
<barry> flacoste: wow
<gmb> Right, *now* i'm going to go get my car. Back later, people.
<matsubara> stub, Chex, gary_poster, rockstar, bigjools, henninge_, sinzui, intellectronica, Ursinha: LP production meeting in 20 @ #launchpad-meeting
<intellectronica> matsubara: thanks for the reminder
<gary_poster> ty
<bigjools> I love all-day meetings and phone calls
<henninge_> matsubara: thanks, danilos will sit in for me today again.
<Ursinha> cprov, hi
<cprov> Ursinha: yo!
<Ursinha> :)
<Ursinha> cprov, have you seen this problem before? OOPS-1347F129
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1347F129
<Ursinha> hmm bad link?
<Ursinha> https://lp-oops.canonical.com/oops.py/?oopsid=1347F129
<cprov> Ursinha: let me check.
<Ursinha> sure
<cprov> Ursinha: I never seen it before, let me investigate, I will report back in a bit.
<Ursinha> sure cprov, thanks!
<rockstar> abentley, Could this issue be caused by mad and some preview diff changes: https://lp-oops.canonical.com/oops.py/?oopsid=1348EA292
<abentley> rockstar: Yes, it could.  We may need to blank existing diffstats.
<maxb> flacoste: Thanks! And that version of launchpad-dependencies is good to be copied to jaunty too
<flacoste> maxb: good point, doing that right now
<maxb> I'm assuming that at this point in time we're just ignoring intrepid completely, and hardy mostly unless something urgent comes up
<flacoste> yes
<cprov-lunch> Ursinha: re. https://lp-oops.canonical.com/oops.py/?oopsid=1347F129, that's an *insane* URL (probably create with a double copy & paste). I don't really know what to do to prevent it.
<cprov-lunch> Ursinha: please file a bug on soyuz, I will find out how to generate a proper 404 for it.
<Ursinha> sure cprov-lunch, thanks
<barry> bac: only 100 failures to go!
<bac> barry: i thought you only started with 70
<barry> bac: i merged rf :(
<flacoste> Soyuz is DONE!!!
<flacoste> Congratulations to cprov, noodles775, bigjools!
<cprov> flacoste: thanks, and AFAIK the prize is 'now, convert blueprints' :)
<flacoste> part of the prize :-)
<flacoste> but i won't bug anybody with that until Monday
<flacoste> And registry has converted their 100th template
<deryck> congratz to soyuz!
<cprov> flacoste: I'm not complain, it's fair enough we've pushed most of the complex templates (distribution, distroseries) to foundations.
<flacoste> err, registry
<flacoste> foundations are the 3.0 UI slacker
<bigjools> don't forget I did some of registry's templates already :)
<salgado> gary_poster, around?
<gary_poster> salgado: yes
<salgado> gary_poster, hi there.  quick question... is it possible to go up from a subview to the view that originally called it?
 * salgado realizes this may not be a very clear question, but knowing how smart gary_poster is, it might not be a problem. :)
<gary_poster> salgado: heh.  hey.  in depends on how the subview was instantiated.  If the view used itself as a context then the subview might have stashed it away.  If that's not the case (and it very well might not be) the effective answer is "no".
<gary_poster> You could actually do it by climbing the stack to find the frame of the parent view maybe, but I don't think anybody wants to see that code. :-/
<gary_poster> salgado: ...there might possibly be something stashed on the request you could use, though I doubt it.  Well, maybe.  Looking...
<salgado> gary_poster, in this case the subview was called from the template ('context/@@+foo')
<gary_poster> yeah, that would not include the parent view as context, just the usual (context, request).
<salgado> I was afraid climbing the stack was the only option, but if there's something in the request, it'd be great
<gary_poster> salgado: still looking, because I thought there was something more likely to help and without the underline, but request._last_obj_traversed might be the top view...
<salgado> that'd be just perfect
<salgado> (Pdb) p self.request._last_obj_traversed.__class__
<salgado> <class 'zope.app.publisher.browser.viewmeta.SimpleViewClass from /home/salgado/devel/launchpad/breadcrumbs-for-leafs/lib/lp/registry/browser/../templates/person-edit.pt'>
<salgado> it is, indeed, the top view, but wrapped in this SimpleViewClass thing which prevents me from doing anything useful with it, it seems
<flacoste> salgado: no
<flacoste> salgado: views are generated class that have the actual view class as parent
<gary_poster> salgado: if that object seems to be what you want, I think we can make it accessible without the underline with just a small adjustment.  We already override the publication object, and it gets notice of the last traversed object, and the publication object is available off the request.
<gary_poster> Therefore we could make publication.afterTraversal, or even publication.callObject, stash the terminal bit of the traversal in a more "official" way.
<gary_poster> and then you could get the object from wherever you stashed it (the publication or the request or whatever)
<flacoste> gary_poster, salgado: we already have an array saving all traversed objects
<flacoste> although the view wouldn't be there
<flacoste> since it's updated by Navigation and not the request/publication
<flacoste> but salgado wanted to fix that
<gary_poster> flacoste: ah, I thought I remembered that.  That must be Launchpad-specific, rather than Zope?
<salgado> so many things I wanted to fix
<flacoste> yes
<salgado> request.traversed_objects
<flacoste> yeah, but that's updated by Navigation
<flacoste> not the request itself
<flacoste> which is dumb
<flacoste> and should be fixed
<salgado> right, but these are different problems
<gary_poster> yeah, if that were updated by the request then you'd have what you want.  If you want a hack, then you can do what I suggested, and have one of those hooks stash the last traversed object in the same list.
<gary_poster> then when you fix it to do the right thing, the code you write now will still work
<gary_poster> salgado: as far as the SimpleViewClass not being what you want, flacoste is right of course--it is a generated base class, not a wrapper.  You should still be able to get what you expect off of the instance.
<salgado> yeah, /me gotta learn how to read the output of print statements.  it clearly says it is a class
<gary_poster> salgado: maybe already clear, but I'd say that fixing the traversed_objects generation is not quite a different problem.  If the request updated that list of traversed objects, the last one would be the view to which the URL had traversed (as opposed to any traversal done within a page template) so it would probably be what you want
<gary_poster> so yes different, but if it were fixed, there would be a clear relatively easy answer here
<salgado> oh, right, I see what you mean now
<salgado> for some reason I assumed request.traversed_objects should only have the content objects that we traverse and completely ignored the fact that we could append the top view there too
<salgado> flacoste, do you see a problem with that?
<flacoste> salgado: i don't mind either way
<gary_poster> right.  it is in fact always traversed, so it is theoretically very reasonable.  practically, it might throw a monkey wrench in code using that list already, but you'd be looking for that.
<flacoste> right, there will probably fallout from such a change
<flacoste> although i think the only thing that use it is the breadcrumbs code
<flacoste> so not a big fallout
<flacoste> since that's the primary reason to make this change
<gary_poster> because you are rewriting that now, essentially, salgado?
<salgado> gary_poster, I don't understand your question
<gary_poster> I was asking if this effort--the thing you are working on that made you ask me about how to get the top-level view--is for a project of rewriting the breadcrumb code
<gary_poster> salgado: if the fallout is big, the other approach I talked about would be less elegant but could be easily made to be less invasive.
<salgado> gary_poster, not really -- all I want now is to use the title of the current page in the breadcrumbs
<gary_poster> I see
<gary_poster> Well, sounds like your call to me. :-)
<salgado> I've already spent a lot more time than I should on this, so I'd rather go with whatever is quicker now
<salgado> gary_poster, what should I do to get the actual view from that SimpleViewClass?
<salgado> I've tried self.request._last_obj_traversed.__parent__, but that gives me the actual view's context
<gary_poster> salgado: the SimpleViewClass is the view
<gary_poster> salgado: if you look in __class__.__bases__ (maybe you will have to walk up a bit) you will see the mix in that you expect, or if you do an isinstance(view, ClassYouExpect) you should see what you expect
<salgado> gary_poster, could that be security proxied?
<gary_poster> salgado: could be yes
<gary_poster> salgado: type(obj) should tell you
<salgado> that's it
<salgado> gary_poster, would it be ok if I use request._last_obj_traversed for now and file a bug to fix it right after 3.0?  I really need to land this so that I can get back to work on page conversions
<gary_poster> salgado: I don't like it, but I understand.  I don't want it to get lost, if that's what you do.
<gary_poster> salgado: but if it is fixed right after 3.0, that would be cool
<allenap> cprov, salgado: Fyi, I've forced a build in devel.
<allenap> cprov, salgado: Ah, and it's just fallen over again for a different reason.
<allenap> cprov, salgado: It's because LP is fubar'ed.
<cprov> allenap: yup, bzr smartserver is affected too
<salgado> gary-away, I'll give bug 423898 a go, then, but if it's tricky to fix I'll resort to request._last_obj_traversed for now. thanks a lot for the help
<ubot3> Malone bug 423898 in launchpad-foundations "Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects" [Undecided,Triaged] https://launchpad.net/bugs/423898
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<mup> Bug #423898: Should not rely on Navigation._publishTraverse() to have objects appended to request.traversed_objects <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/423898>
<salgado> flacoste, do you have a couple minutes to talk about ^?
<flacoste> sure
<flacoste> skype or IRC?
<salgado> flacoste, whatever works best for you
<salgado> I just want some pointers so that I can try and fix it
<flacoste> ok
<salgado> I had a quick look before filing it and it was not clear to me where I'd do what is currently done by Navigation
<flacoste> salgado: i'd suggest afterTraversal in the publication, do you agree gary-away?
<mwhudson> good morning
<barry> bac: canonical.launchpad.helpers.backslashreplace()
<bac> barry: nice
<bac> barry: your way is shorter
<barry> yeah, i wish i'd known about it like 72 hours ago ;/  waaaayyy too late to go back now
<barry> yeah
<mwhudson> abentley, rockstar: i need to be away at the stand up time, would you be ok with having it earlier?
<rockstar> mwhudson, sure.
<salgado> flacoste, that didn't work -- now only the view is added to traversed_objects
<gary_poster> flacoste, salgado: I was thinking of afterTraversal as a hack solution.  It would not replace the current code, only add to it
<gary_poster> flacoste: therefore it might be an ok interim step
<gary_poster> that's one of the approaches I mentioned earlier
<gary_poster> you could leave the current behavior of the navigation
<gary_poster> but add this other behavior
<gary_poster> and then when you did it "right" (by overriding the publication's traverse methods, and unfortunately maybe having to copy-n-paste code, because you would want to inject something in a loop)
<gary_poster> yhe code that gets the object for titles would still be looking in the same place
<gary_poster> salgado: did that make some sense?
<salgado> I see
<flacoste> ???
<flacoste> ah, ok
<flacoste> afterTraversal is called only once
<flacoste> not on every step
<flacoste> salgado: callTraversalHooks would be the one to use then
<flacoste> or traverseName
<flacoste> salgado, gary_poster^^^
<gary_poster> flacoste, traversalHooks maybe.  traverseName would make me nervous, but maybe.  I'd look into it if desired, but want to get to some other tasks ATM.  Well, I'll look at the publication again...argh
<gary_poster> ok looking
<salgado> gary_poster, I don't understand why I'd want a loop?  I thought it'd be just a matter of moving the "request.traversed_objects.append(ob)" from one place to another
<salgado> from Navigation to traversalHooks(), that is
<gary_poster> salgado: traversalHooks might work yes.  Didn't have code open, so working from memory, and have other things going on simultaneously :-) .  Going to look at code now...
<salgado> gary_poster, it works. :)
<gary_poster> salgado: yes, cool.  The loop I was talking about was the one in zope/publisher/base.py, in the traverse method.  Since we have the hook we don't need to rewrite the loop, yes.
<salgado> cool, let me run some more tests now to see if this is really what we need
<salgado> thanks a lot, gary_poster and flacoste
<gary_poster> cool, np
<mwhudson> abentley: have you seen https://bugs.edge.launchpad.net/launchpad-code/+bug/427383 ?
<ubot3> Malone bug 427383 in launchpad-code "ValueError: No JSON object could be decoded in merge through the API" [Undecided,New]
<jelmer> has launchpad switcehd to python 2.5 yet?
<mwhudson> jelmer: no
<mwhudson> jelmer: talk to gary_poster about that :)
<jelmer> ah, pity
 * mwhudson afk for a bit
<mwhudson> rockstar: you talk to abentley and i'll talk to you when i get back?
<rockstar> mwhudson, oh crap, I forgot he's off today.
<rockstar> You and I will chat when you get back.
<barry> bac: good news: i'm down to 22 failures.  bad news: these are the ugliest ones :/
<bac> go go go
<gary_poster> jelmer: Ideally for LP 3.0 (this release), definitely for subsequent release.
<rockstar> barry, sounds like you and I are in a similar boat.
<barry> rockstar: i'm paddlin' hard :)  what branch are you working on?
<rockstar> barry, I'm working on the branch index page.
<rockstar> barry, so when I ran the tests to see which failed after my changes, the answer was "all"
<barry> rockstar: heh, sounds like me.  my branch changes page <titles> so every story broke
<mwhudson> rockstar: oh yeah!
 * mwhudson still not ere
<maxb> mwhudson: "changes are propagated to the mirrored side" - when?
 * barry fixes 2 more.  20 to go
<micahg> hi, I"m getting a badsig on the lp-dev ppa
<micahg> kfogel, I'm getting a badsig on the lP dev ppa
<micahg> LP
<wgrant> micahg: Which distroseries?
<wgrant> And are you sure there's no proxy?
<micahg> jaunty
<micahg> well, I imported the key already
<wgrant> BADSIG means you have the key, but the signature is bad, doesn't it?
<wgrant> And the signature is good, so it's probably a proxy between you and LP.
<kfogel> micahg: I'm no sig expert, but don't install till this is resolved, obviously
<micahg> ok
<micahg> well, there is a proxy
<micahg> but it doesn't happen for any other ppa
<wgrant> A bad, evil proxy.
<kfogel> wgrant: how could a proxy do this?  The proxy shouldn't interfere with data transfer like that, should it?
<kfogel> A sig is just a file.
<micahg> ah
<wgrant> kfogel: Happens all the time with bzr. It caches one version of one file, another version of another.
<micahg> I had the Packages.bz2 file excluded
<micahg> maybe I need the sig file excluded
<micahg> wgrant: can you tell me what I need to have excluded from the squid proxy?
<wgrant> The relevant files are Release, Release.gpg, Packages, Packages.gz, Packages.bz2. But you shouldn't need to do that.
 * barry fixes 3 more; 17 to go
<jml> kfogel: hellloooo
<kfogel> jml: hi, let's resume your review of my merge proposal https://code.edge.launchpad.net/~kfogel/launchpad/add-community-contributions-script/+merge/11417
<jml> kfogel: as you know, I am reviewing your patch @ https://code.edge.launchpad.net/~kfogel/launchpad/add-community-contributions-script/+merge/11417
<kfogel> FAIL
<kfogel> jml: ok, so first thing is -- 4 space indentation, not 2
 * kfogel goes to his branch to make the change
<jml> kfogel: I don't normally do reviews in real-time chat, so we'll see how this goes.
<jml> kfogel: also, PEP 8 says that docstrings should be start with a single summary line, then have a blank line, then have any other text as following paragraphs.
<kfogel> jml: don't worry, I won't interpret anything as snarkiness unless it actually is.
<kfogel> jml: thx, ok
<jml> kfogel: which the module docstring doesn't do.
<jml> (see PEP 257 for details)
<jml> http://www.python.org/dev/peps/pep-0257/
<jml> comments are good
<jml> little bit disappointed that "Right Way" and "XML" are in the same sentence...
<kfogel> jml: ironically, if you look at ~kfogel/+junk/lpcc/lpcc.py, that's how I originally had it.  Later I compactified it.
<kfogel> jml: HAH, re XML
<kfogel> jml: I'm not reading all of PEP 257 now, unless you think I should.  I'm just changing as you directed.
<jml> kfogel: you never ever need to end a line of Python with a semi-colon
<jml> kfogel: for 'RevisionError', consider replacing the 'pass;' line with a short docstring
<jml> kfogel: sure, don't read it all now, just pasting for reference
<kfogel> jml: old C habit, thx
<jml> kfogel: we tend to use XXX, rather than TODO.
<kfogel> jml: ok
<jml> kfogel: there's (ugh) a format for them too.
<kfogel> jml: I'm falling behind, but I'll catch up if you slow down :-)
<jml> kfogel: heh heh.
<kfogel> jml: is there something I can put in the file to tell it the python indent level?
 * kfogel researches emacs
<jml> kfogel: don't know. I have 4 set as my global default indent.
 * jml looks for the XXX policy
<jml> (too many wikis)
<barry> kfogel: in python-mode.el (the one true python editing mode) there's py-indent-offset, which defaults to 4
<jml> # XXX: SteveAlexander 2007-03-12 bug=12345: This code is going to
<jml> #  fail after next year.
<barry> kfogel: python docstrings are easy.  the rules essentially came from elisp, circa 1994 (defined at the first python conference at NIST -- i'm old, i was there :)
<barry> s/conference/workshop/ but still
<kfogel> jml, barry: dang it, what's the Easy Way to re-indent a bunch of python from level 2 to 4?
<kfogel> barry: that's why I instinctively followed it at first, only to undo it later (not realizing it was the rule in Python too)
<barry> kfogel: C-c TAB -- py-indent-region, though it's often just as easy to indent-rigidly
<barry> :)
<jml> kfogel: I used to know this...
<kfogel> barry: indent-rigidly wouldn't work because this code nests to more than one level
<jml> indent-region, I think
<jml> C-M-\
<kfogel> barry: py-indent-region also doesn't exist -- isn't that an old python mode?  python-modes got all confused at some time in the past; I'm using whatever is in the Emacs bleeding edge tree right now.
<kfogel> jml: indent-region also doesn't work, because I have nesting
<jml> meh.
<jml> search and replace '  ' with '    '?
<kfogel> jml, barry: the only thing I can think of is a query-replace-regexp,...
<kfogel> yeah
<barry> kfogel: do yourself a favor and grab lp:python-mode :)
<jml> except it breaks other things you might like
<kfogel> barry: oh ho, is that it then?
<kfogel> barry: ok, doing now
<barry> kfogel: you don't really want the whole history, but python.el is a total rewrite that rms ordered for no good reason.  python-mode.el's been around for much much longer (1995 or so), is well maintained by python core devs, and is full of awesomehood :)
<kfogel> barry: why on earth aren't we shipping it in GNU Emacs??
<jml> ... and missing some features from python.el
<kfogel> barry: the indent is working fine now, thx
<barry> jml: works both ways, and there is some activity in python-mode world to port those over (though it appears a merge is politically impossible)
<barry> kfogel: yeah, exactly. politics (imho)
<jml> kfogel: for strings like rev_url_base, prefer foo = (\n"blah blah blah "\n"blah blah.")
<barry> kfogel, jml anyway, this is fun, but dinner awaits :)   if you're really interested in the gory details (from my admittedly biased position), give me a call sometime
<barry> i'll be back online later.  gotta get karmic up on my new machine!
<kfogel> barry: have fun!
<kfogel> ...stormin' da castle
<jml> heh
<jml> barry, g'night.
<kfogel> jml: so for "# ### TODO", switch to "# XXX" with subsequent lines indented 2?
<jml> indented 2? uhh, no.
<jml> kfogel: I'll amend the policy doc.
<kfogel> jml: sorry, indented 1
<wgrant> Should the anti-Karmic notes on /Getting and /Running be removed, or just altered to say it now works?
<kfogel> wgrant: if it works, then yeah
<kfogel> wgrant: oh sorry
<kfogel> wgrant: I think it's worth mentioning that it now works on Karmic
<wgrant> kfogel: That's what I thought.
<kfogel> we can remove that after a while
<jml> kfogel: no indentation. nobody does it, the policy is wrong.
<kfogel> jml: ok
<jml> kfogel: you should also include your name and the date
<jml> (now it's correct)
 * jml harnesses the lightning
 * mwhudson returns
<kfogel> jml: what page are you editing?  is there an example there?
<jml> mwhudson, hi
<mwhudson> maxb: "eventually"
<deryck> Hi, all.
<mwhudson> maxb: or after the branch is unlocked
<jml> kfogel: https://dev.launchpad.net/XXXPolicy
<mwhudson> maxb: i think i fixed all the branches now
<mwhudson> mwh@grond:libraryize-ec2test$ bzr log lp:~launchpad/meta-lp-deps/gutsy
<mwhudson> ssh: Could not resolve hostname bazaar.launchpad.launchpad: Name or service not known
<mwhudson> what on earth?
<mwhudson> bazaar.launchpad.launchpad?
<jml> kfogel: at one point, you do a lot of 's += '
<jml> kfogel: that's fine, but mostly you shouldn't do string concatenation that way, because Python is terrible.
<wgrant> mwhudson: That lp: URL resolves fine here....
<kfogel> jml:         rev_url_base = (
<kfogel>             "http://bazaar.launchpad.net/~launchpad-pqm/"
<kfogel>             "launchpad/devel/revision/")
<kfogel> jml: IRC-messed-indentation aside, that's the way to do it?
<jml> kfogel: that's right.
<kfogel> jml: and it's not a tuple because there's no comma at the end?
<jml> kfogel: correct.
<jml> kfogel: Python treats adjacent string literals as one string
<kfogel> jml: yeah, I just didn't know (or forgot) about the grouping parens, and could never find a way to get adjacent strings to indent nicely.
 * jml likes kfogel, but wishes kfogel_ would fall in a well.
<kfogel> jml: so instead of "s += ...", should I do "s.append()" or something?
<kfogel> jml: is there a kfogel_ here?
<kfogel> kfogel_: ayt?
<kfogel> whoa
<kfogel> how'd he get in here?
<kfogel> I don't even know how to make him go away.
<jml> x = ['ouaeo', 'hateouahseo', 'gq;jmkw']; 's = '.join(x)
<jml> umm.
<jml> s = ''.join(x), rather.
<jml> kfogel: you know about abbrev-mode right?
<kfogel> jml: yes
<kfogel> why?
<jml> kfogel: you say, # "ExternalContributor" is too much to type, so I guess we'll just use this.
<kfogel> jml: that's so I could get the great abbreviation, man!
<jml> kfogel: But I say, Ex M-/ is easy to type :)
<jml> kfogel: heh
<kfogel> jml: (also, just because I use abbrev mode doesn't mean other people do)
<kfogel> jml: I usually just use dynamic abbrevs anyway
<jml> kfogel: you can leave it as is, but normally Launchpad code favours longer names.
<jml> and relies on people having decent editors or great patience.
<kfogel> jml: ah, ok
<kfogel> jml: in this case, I'd like to leave it, for obvious reasons, but noted for next time.
<jml> blank line after class docstrings
<jml> kfogel: ooh. time to python up :)
 * jml is looking at show_contributions
<kfogel> jml: whew, ok.  slow down, still correcting all the s += stuff.
<jml> kfogel: ok. sorry.
<mwhudson> wgrant: repeatable for me
<kfogel> jml: (don't really have to slow down, just explaining where I've got off to)
#launchpad-dev 2009-09-11
<mwhudson> oh, i mangled the stacked on url it seems
<jml> kfogel: so, from 'def prefer_recent_revs', you could instead write
<kfogel> jml: ok, string extension all fixed
<jml> s += ''.join(map(str, sorted(self._landings, key=lambda x: x.top_rev.revno)))
<kfogel> jml: (blank line after class doc strings done)
<kfogel> reading
<kfogel> jml: nice
<jml> kfogel: and that would replace the for loop & function.
<jml> kfogel: if you prefer named functions (and We generally do, but I don't care), you can split out the lambda
<jml> kfogel: using 'key' is almost always faster than a cmp-based sort.
<kfogel> jml: wait, that's not really from prefer_recent_revs, right?
<kfogel> jml: it's from the outer  func
<kfogel> jml: IOW, def prefer_recent_revs() would just go away
<jml> kfogel: I meant 'from' as in 'starting at'
<kfogel> jml: nod
<jml> kfogel: but yes, you are right.
<kfogel> jml: hmm, doing that resulted in a difference
<kfogel> jml: let me investigate
<kfogel> jml: oh, it reversed the order of the sort
<kfogel> jml: note the order of a and b in prefer_recent_revs
<jml> kfogel: ahh, easy enough to add a negate to the key function.
<kfogel> jml: beautiful
<jml> unless they are strings
<jml> in which case, umm...
 * jml learns Python again
<kfogel> jml: they're numbres
<kfogel> jml: they should be, anyway
<kfogel> wait
<kfogel> hmmm
<kfogel> er
<jml> sorted(list, key=whatever, reverse=True)
<jml> that's clearer anyway
<jml> http://docs.python.org/library/functions.html#sorted
<kfogel> jml: wow, never knew about reverse=True
<kfogel> great
<jml> kfogel: sometimes the stdlib is awesome :)
<kfogel> jml: now there's no difference
<jml> cool.
<kfogel> jml: there's another sort function elsewhere
<kfogel> I'll see if I can do the same thing.
<jml> kfogel: when you're done with that can you commit & push the branch so I can take another look.
<kfogel> jml: sure, one min
<jml> cheers.
<mwhudson> jml: in your branch that you sent to me, there are a bunch of XXXed questions in lib/devscripts/tests/test_sourcecode.py
<mwhudson> jml: i'm sure some of them have answers now
<mwhudson> (like "can we rely on twisted")
<mwhudson> jml: can you run through and update them?
<kfogel> jml: pushed.  UI doesn't show all revs yet, but presumably bzr has them.  r9384 is latest.
<kfogel> jml: good advice on trivial bugs, btw (on ml)
<jml> kfogel: thanks
<jml> mwhudson, will do.
<jml> ta :)
<mwhudson> jml: thanks
<mwhudson> gosh, i know bash too well
 * jml is a little bit closer to my second cup of coffee
<jml> kfogel: more stylistic stuff coming your way
<jml> kfogel: # where "{X}" means "LogRevision for X" -- needs two more spaces of indent to align with the comment above
<jml> and a blank line afterwards to sate the PEP 8 gods' thirst.
<jml> kfogel: don't align '=', but rather keep one space on either side.
<jml> kfogel: empty list is written like s = [], not s = [ ]
<jml> similarly empty dict is {} not { }
<kfogel> jml:  reading
<jml> kfogel: indentation in get_ex_cons is still 2-space, should be 4-space.
<sinzui> jml: I started writing a new linter. It uses pep9.py
<kfogel> jml: ok, correcting, and dang it, I set the py-indent-offset var to 4, what the heck is up with that persistent 2 space then?
<sinzui> jml: I started writing a new linter. It uses pep8.py
<jml> sinzui, I actually have pep8.py wired up to my emacs
<jml> sinzui, you don't think I _remember_ all this crap? :)
<jml> kfogel: blame barry's non-standard python mode :P
<sinzui> jml: mine is wired in to gedit. I am announcing the release in two days
<jml> sinzui, cool.
<sinzui> jml: I think we should abandon pylint. pyflakes + pep8 is good enough for me
<jml> kfogel: get_ex_cons could probably be a classmethod, rather than a top-level function
<kfogel> jml: when you say "don't align '=', you mean for example in ContainerRevision.__str__() ?
<jml> sinzui, +1
<kfogel> jml: (missing " above, sorry)
<jml> kfogel: exactly so.
<kfogel> jml: ok, fixing
<kfogel> jml: but I don't have to like it :-)
<jml> kfogel: heh heh.
<jml> kfogel: in LogExCons.__init__, you assign to a local variable to no obvious effect (current_top_level_rev = None). A bug, perhaps?
<jml> kfogel: page_intro is really wide. Is it possible to wrap it without breaking wiki formatting?
<jml> kfogel: I dropped out, what was the last thing I said?
<kfogel> jml: hunh, I made get_ex_cons() a class method, and now it produces an empty page
<kfogel> one sec
<kfogel> jml: last thing you said was:
<kfogel> kfogel: page_intro is really wide. Is it possible to wrap it without breaking wiki formatting?
<kfogel> jml: sure, I can use "\" at end of line in page_intro, right?
<jml> kfogel: thanks.
<jml> nothing got dropped the.
<kfogel> jml: if a class method is to be used privately only, should I prefix its name with "_" ?
<jml> kfogel: yes, in general.
<jml> kfogel: but maybe I was mistaken about the classmethod thing...
<kfogel> jml: ? no other class uses it, though it is sort of of general utility
<jml> kfogel: well, if it looks better, do it :)
<jml> kfogel: I just realized that I misinterpreted its purpose, and advised based on that bad interpretation
 * jml almost always uses classmethod to make factories for the class.
<kfogel> jml: the current_top_level_rev intialization question is a deep philosophical question, actually: do we believe in "init explicitly to an invalid value" or do we believe in "don't init it if the initial value shouldn't matter" ?
<kfogel> jml: I don't care which the answer is; if it's 2, I'll do 2.
<kfogel> jml: re the get_ex_cons() thing: I think on balance top-level method feels a bit more right, so I'm going to uncommit that.  Not that it's a big deal either way.
<jml> kfogel: cool.
<kfogel> jml: how about the init question?
<jml> kfogel: it's a post-2nd-coffee question.
<kfogel> jml: there I have consulted with dusty tomes in Greek and Latin, and still find no certain answer.  I rely on your wisdom.
<jml> kfogel: I tend to prefer instantiating actual honest-to-goodness objects.
<kfogel> jml: mmm, good point -- the initialization misleads as to the expected type
<kfogel> jml: but I don't have an object yet
<jml> sec.
 * kfogel waits for the caffeine to reach jml's capillaries
<jml> kfogel: this is how I would do the string building stuff: http://pastebin.ubuntu.com/268841/
 * jml now turns to current_top_level_rev...
<jml> kfogel: ahh, right.
<jml> kfogel: set it to None.
<kfogel> jml: thx, I didn't know about string.extend()
<jml> kfogel: it's list.extend
<jml> kfogel: what it's doing is building a list of lines.
<kfogel> d'oh, sorry
<jml> kfogel: and then joining those lines together with '\n' as a separator.
<kfogel> jml: I think I'm going to rename that var from "s" so as not to confuse anymore
<kfogel> jml: one sec while I make everyone do that
<kfogel> jml: (I think it's worth it; good habits now == easier reviews later)
<jml> kfogel: maybe even to something that actually means something about the problem domain? :)
<jml> kfogel: ok, while you do that, I'll make a coffee. :)
<kfogel> jml: fwiw, my natural emacs indentation does not put the first " four spaces after the column of the [ above it, in this new list/string universe.
<kfogel> sorry
<kfogel> I mean
<kfogel> it does not put it right under the [, but instead under the =
<kfogel> I wonder if that is due to some indentation var still set to 2
 * kfogel looks
<kfogel> yup
<kfogel> py-indent-offset got reset to 2
<kfogel> dunno what's up with that; I'm just going to fix it in the vars line for the file
<kfogel> (and deal with my .emacs later -- it has to be compatible with many projects, not just lp)
<jml> kfogel: emacs should be better at per-project configuration
<jml> kfogel: ok, so... where are we.
<kfogel> jml: I'm still transforming the list/string stuff
<kfogel> jml: interesting; when I have only one elt, I can still use .extend()
<jml> kfogel: yeah. it's just joining one list onto the end of another
<kfogel> jml: yeah, just interesting that it will also join an elt -- i.e., won't create a terminal cdr, as (some) lisps might.
<barry> bac: done to 10 failures
<jml> kfogel: can you give an example?
<mwhudson> kfogel: python lists are lisp vectors, not lisp lists
<kfogel> mwhudson: thx
<mwhudson> (gosh, brain fade, lisp does call 1-d arrays vectors, right?)
<kfogel> mwhudson: I think so
<jml> kfogel: btw, you should have your name and the date in XXX comments
<kfogel> jml: "XXX: Karl Fogel 2009-09-10: ..." ?
<kfogel> jml: (trying to make you a lisp example, but it's back-burner)
<jml> kfogel: yeah. thanks.
<jml> I guess you are supposed to use WikiName...
<kfogel> jml: really?
<kfogel> jml: ok
<jml> but I couldn't possibly force someone else to do that, given I dislike it and there's no actual reason.
<kfogel> jml: ok :-)
<kfogel> jml: anything else?
<kfogel> jml: branch pushed again, to r9388
<jml> kfogel: alpha-sort imports, blank line between stdlib imports and bzr imports. barry has an elisp thingy to do this for you on EmacsTips
<barry> kfogel: if you grab the latest python-mode.el, C-c C-f is bound to py-sort-imports
<kfogel> barry:  it only works on multiline imports
<jml> :(
<kfogel> barry: gets an "unbalanced parentheses" error for me
<kfogel> same from EmacsTips
<kfogel> jml: is this right:
<kfogel> import getopt
<kfogel> import re
<kfogel> import sys
<kfogel> from bzrlib.branch import Branch
<kfogel> from bzrlib import log
<kfogel> from bzrlib.osutils import format_date
<barry> kfogel: yeah, it looks for names inside parens
<kfogel> or should bzrlib be first?
 * mwhudson still needs to write something/figure out how to drive ropemacs to do the import statements automatically
<jml> kfogel: bzrlib first
<kfogel> and sorted within that?
<jml> mwhudson, please please please.
<jml> kfogel: I don't understand.
<kfogel> jml: I can think about about three different ways that might be Right.
<kfogel> want to just paste what the right way is?
<mwhudson> jml: after filtering those questions, can you commit and push?
<jml> yeah, ok.
<jml> mwhudson, yes, I will.
<mwhudson> thank you
<kfogel> jml: there are 3 bzrlib lines; what order should they be in?  (granted that they come before the other imports)
<jml> mwhudson, this review is consuming all of my mental bandwidth right now
<kfogel> mwhudson: he's mine, don't touch him
<mwhudson> jml: ok
<jml> the slowness and unreliability of my internet connection is not helping...
<jml> kfogel: http://paste.ubuntu.com/268852/
<kfogel> jml: thx
<jml> kfogel: don't use has_key
<kfogel> jml: committed and pushed (r9389)
<kfogel> dang it :-)
<kfogel> jml: ok
<jml> you want... umm...
<kfogel> jml: 'in' ?
<kfogel> jml: btw, why not use has_key()
<kfogel> ?
<jml> ec = all_ex_cons.get(a, None)
<kfogel> jml: ok, wait a sec
<kfogel> How is that better?
<kfogel> IMHO it's worse, in terms of communicating what we're doing there.
<jml> if ec is None:\n    ec = ExCon(a); all_ex_cons[a] = ec
<jml> well, you can do the if a in all_ex_cons
<jml> has_key is deprecated, and won't be in future versions of python
<kfogel> ah
<jml> either use 'get' or 'in'.
<jml> I tend to lean to leap-before-you-look (hence the get), but both are appropriate here.
<kfogel> jml: I'll wait for moer comments before pushing again
<kfogel> (or for EOT)
<jml> kfogel: last thing
<kfogel> ?
<jml> kfogel: dict colon alignment is kind of like = alignment
<kfogel> ah
<jml> kfogel: {a: b, c: d, ...}
<kfogel> jml: in main(), right?
<kfogel> jml: any other places?
<jml> kfogel: no, just main
<kfogel> jml: pushed to r9391
<jml> kfogel: thanks.
<jml> kfogel: please merge it.
<kfogel> jml: thank you for the thorough review
<jml> kfogel: np.
<jml> kfogel: are we going to have that phone call?
<kfogel> jml: bzr pqm-submit -m "[r=jml][ui=none] Add utilities/community-contributions.py script, for updating https://dev.launchpad.net/Contributions."
<kfogel> ?
<kfogel> jml: on second thought, leaving out the latter half, as the script already says that.
<kfogel> (and if it ever changes, the script will change)
<jml> heh
<jml> sure.
<kfogel> jml: submitted!
<kfogel> jml: are you seeing my priv msgs?
<jml> sorry
<jml> I have dodgy internet
<kfogel> np
<kfogel> intellectronica: hey, what's diff btw a welcome msg and a topic?
<intellectronica> kfogel: a welcome message is sent to you once and can have more than one line
<intellectronica> i don't remember seeing them on freenode, though, so maybe you can't use them
<kfogel> intellectronica: well, it's important that the msg be re-gettable anyway, which /topic is
<kfogel> intellectronica: so i think maybe topic is better anyway?
<kfogel> intellectronica: I type "/topic" all the time in here, and in other channels.
<kfogel> intellectronica: (I mean to read the topic, not to change it)
<intellectronica> kfogel: fair enough. i don't rely on it as much, but that's hardly statistically significant
<kfogel> intellectronica: my use is not be statistically significant either; so hard to get real data, unless Freenode chooses to share with us :-).
<intellectronica> indeed
<intellectronica> anyway...
 * intellectronica goes to dream in ascii art
 * jml more coffee & more toast, then mwh-related branches.
 * mwhudson lunches
<jml> fwiw, my internet connection today is really crappy, so please forgive my occasional lapses.
<jml> mwhudson, I've filtered those questions.
<mwhudson> jml: thanks
<jml> mwhudson, you aren't in #launchpad-reviews, I take it.
<jml> internet hates me
<jml> x hates me
<jtv> jml: we don't hate you though
<jtv> (Insert your favorite snide follow-up here)
<jml> it's entirely possible that firefox hates me most of all
<jtv> wouldn't put it past it
<jtv> jml: is it perhaps time to get to the root cause of all this hatred?
<jtv> face it, with all the variables, the only constant is you.
 * jml bravely launches firefox again
<jml> mwhudson, hi
<mwhudson> jml: hello
<jml> mwhudson, I've pushed up my changes to that branch
 * mwhudson tests his ec2test-hackery branch on itself
<jml> mwhudson, how have things gone with you?
<jml> (sorry, it's been a very distracty day)
<mwhudson> jml: i have moved some code around in ec2test
<mwhudson> jml: yeah, no worries
<jml> mwhudson, yay
<jml> I guess I'll resubmit the mp or something
<mwhudson> jml: specifically, the connect_as_user methods now return something rather than mutating instance state
<mwhudson> jml: and the code to make a unix user is on instance now
<jml> soooo happy
<mwhudson> yeah, it's very much an improvement
<mwhudson> oh heavens
<jml> mwhudson, I really wanted to do that, but couldn't quite see how, at the time.
<jml> mwhudson, https://code.edge.launchpad.net/~jml/launchpad/better-sourcecode-update/+merge/11580
 * jml also wants to review all of maxb's recent branches.
<jml> oh man.
<jml> and my computer crashed / hanged before I could submit that other one, I think.
<mwhudson> jml: boto seems to import hashlib
<jml> mwhudson, why is this a problem?
<mwhudson> jml: does ec2test work with python 2.4 at all?
<jml> mwhudson, yeah, I think so.
<jml> but maybe I'm wrong
 * jml files https://bugs.edge.launchpad.net/launchpad-foundations/+bug/427703
<ubot3> Malone bug 427703 in launchpad-foundations "Way to see which branches I'm testing with ec2test right now" [Undecided,New]
<jml> ok
<jml> let's see if my computer will stay up long enough for this to work.
<jml> the ohloh import is up to phase 3!
<jml> Step 3 of 3: Counting lines of source code (Running 4875/66451)
<mwhudson> jml: cool
<jml> mwhudson, were you planning on reviewing the better-sourcecode-update branch today?
<jml> didrocks, wuuu.
<didrocks> :)
<didrocks> ok, I'm trying to checkout it in intrepid
<jml> didrocks, the secret page is https://dev.launchpad.net/Getting
<jml> didrocks, getting a Launchpad dev environment up and running is not as easy as it should be.
<didrocks> yes, I already read this "secret" page :)
<didrocks> ok, trying it right now
<mwhudson> jml: yeah, i'll try
<jml> mwhudson, cool. thanks.
<didrocks> ok, bzr doesn't know the format, let me have a look at the bzr ppa for intrepid
<mwhudson> jml: done
<jml> mwhudson, hanks.
<didrocks> jml: Hum, it will definitevely broke my intrepid server if I accept a "Score is -9863" :)
<didrocks> jml: Well, I will be able to do it from home, can you drop me some instructions so that I can start on it?
<jml> didrocks, ok. that sucks.
<jml> didrocks, sure. have you filed a bug for it yet?
<didrocks> no, I'm doing it right now
<didrocks> jml: against launchpadlib, right?
<jml> https://bugs.edge.launchpad.net/launchpad-registry/+bug/389872
<ubot3> Malone bug 389872 in launchpad-registry "GPG keys not exposed in API" [Low,Triaged]
<didrocks> ok, it already exist, great :)
<jml> didrocks, I'll make some comments there.
<didrocks> jml: thanks, and I try to assign it to myself :)
<jml> :)
<jml> didrocks, done
<didrocks> jml: yes, I'm assigning the bug to myself. I hope it will be a little more clear when seeing the code. Thanks :)
<jml> didrocks, np.
<jml> didrocks, good luck!
<didrocks> jml: thanks! I will certainly ping you :)
<jml> didrocks, note that I'm in Australia for the next week and a bit, so you may find timezones to be a problem.
<jml> didrocks, people in this channel will gladly help you though, if I'm not around.
<didrocks> jml: ok, I'll just ask here. Enjoy Australia :)
 * mwhudson eods
<jml> mwhudson, have a good weekend.
<mwhudson> jml: well i'm working tomorrow...
<jml> mwhudson, oops
<jml> mwhudson, have a good evening then :)
<mwhudson> (swap day for later in the year, so i'm not that upset about it...)
<adeuring> good morning
<xaver> hi i have trouble with launchpad repo. the download start with 900-1200 kb/s and slow down to 1000-3000 B/s
<xaver> hi
<mrevell> Guten morgen
<xaver> i have trouble with launchpad repo. the download start with 900-1200 kb/s and slow down to 1000-3000 B/s (kann auch deutsch schreiben)
<xaver> ^^
<mrevell> xaver: Sorry, that's the limit of my German :) What are you downloading? Launchpad itself or something from a PPA?
<xaver> ppa
<xaver> i have no idear why
<xaver> at home everything run fine
<noodles775> xaver: kommisch - ist es genau so fÃ¼r anderer PPAs?
<xaver> is nur bei ppa -> andere sidn kein problem nur die paketquelle
<xaver> nervig ist es bei groÃen files, keline da starte ich neu und shcon sind die da aber 30 mb aufwÃ¤rts ist es mÃ¼ll
<jml> mrevell hello :)
<mrevell> hi jml
<xaver> hi
<noodles775> xaver: hmm.. Ich habe kein problem - Ich hab jetzt ein update gemacht mit 3 ppas.
<xaver> is immer also nicht nur heute
<xaver> zuhause lÃ¤uft es
<xaver> nur hier...
<noodles775> bigjools: any idea why ppa's specifically would always slow down for xaver?
<bigjools> local networking issues
<noodles775> xaver: ah, so it's probably a network issue specifically ...
<noodles775> Yes, sounds like it, if it's specific to one location.
<xaver> ok thx
<noodles775> np.
<xaver> wir hÃ¤ngen direkt am RZ ;)
<bigjools> is there a proxy in the way?
<bigjools> etc.
<xaver> no
<xaver> we a direct likned ion the data center
<xaver> on
<xaver> frist 20 mb run with 900-1200kbs
<xaver> next second down to 1000 b/s
<xaver> another question, i pack a lot of packages at home, is a good docu how i can help with packages for launchpad.... -> problem at home is my connetion very bad, but now i can upload it at work
<noodles775> xaver: help with packages for Ubuntu? http://www.ubuntu.com/community/processes/newdev or https://wiki.ubuntu.com/UbuntuDevelopment
<xaver> is in linux a programm for M$ exchange 2007 beside from evolution mail
<xaver> bye reboot ;)
<wgrant> noodles775: Nice new archive views.
<noodles775> wgrant: thanks! There are still a few changes to land - but it was in a landable state.
<noodles775> wgrant: cprov is getting ready to land a simplified "Adding this PPA to your system" portlet which will mean the sources.list details are folded by default.
<wgrant> noodles775: Great!
<wgrant> noodles775: The multiple <dd>s in sequence on +packages look, IMO, much better if the non-final ones have a 'margin-bottom: 0' added.
<wgrant> noodles775: Otherwise there are awkward gaps.
<noodles775> wgrant: hmm... I can't see the awkward gap... do you mean, for example, between the last dd under 'Number of packages' and the 'repository size' dt?
<wgrant> noodles775: The package filter on +index is set to the right distroseries by default, but the table contains packages for all seires. Is that known?
<wgrant> noodles775: Between the binary and source counts/sizes is the main one.
<wgrant> There's as much of a gap between those two and between the last and the following heading.
<maxb> woo. I just got my first ec2test email :-)
 * wgrant eats.
<noodles775> wgrant: nope, first bug - (actually for me the package filter is *not* being set to the correct default even though the sources.list is... :/)
<danilos> jtv: hey, around?
<wgrant> noodles775: You're right about it not being set by default. Firefox was just doing bad cachy things.
<noodles775> OK, I'll fix that today.
<wgrant> Who is behind the new DSPR index?
<noodles775> wgrant: bigjools initially, but I think cprov-afk ended up implementing it?
<wgrant> noodles775: I see.
<bigjools> mostly cprov in fact, the original mockup was discarded since we decided it was no good
<wgrant> https://edge.launchpad.net/ubuntu/+source/empathy/2.27.92-1ubuntu1
<wgrant> The '(New)' there looks very strange and it's somewhat unobvious what it means.
<wgrant> maxb: Aha! Your branch was merged. Very nice.
<jtv> Hmm... why is "bzr push" transferring a hundred MB again?
<maxb> wgrant: 1 of 6 :-)
<maxb> no, 2 of 6 :-)
 * wgrant prepares the second part of PPA download counts.
<noodles775> wgrant: great! I've left space in the PPA stats portlet for that exact reason :)
<bigjools> ha
<noodles775> danilos: did you have an issue where the last object of a traversal was being traversed but not being added to request.traversed_objects? If so, can you tell me more?
<danilos> noodles775: yes
<danilos> noodles775: you just need to provide an empty Navigation implementation for your interface; see the branch you reviewed (DistroSeriesLanguage adds navigation and registers it with browser:navigation)
<noodles775> danilos: great, thanks!
<danilos> noodles775: salgado has a bug about it (i.e. wants to fix it so we don't have to introduce empty, silly, navigation implementations), but I forgot the number :)
<noodles775> danilos: np - I'm unstuck now so I'm happy :)
<danilos> noodles775: heh, you should be, I spent entire day trying to figure that out (it was a public holiday in Brazil :)
<noodles775> danilos: yeah, I just spent 45mins looking at it, and mentioned it to bigjools who remembered you'd had that issue - yay (I didn't take in the comment in the MP).
<bigjools> for once I actually remembered something
<noodles775> ;)
<danilos> bigjools: good job, here's a candy! :P
 * bigjools licks danilo's lollipop
 * jtv tries to forget what he just read
<mrevell> Launchpad community meet-up in London on the 28th Sept: http://blog.launchpad.net/general/launchpad-meet-up-28th-september-the-warwick-london
<bigjools> mrevell: we should offer a pint to anyone who turns up!
<mrevell> bigjools: heh, maxb would get a pint in that case :)
<bigjools> mrevell: I think I'm taking my bike down on the train this time. Walking to and from county hall was a PITA
<bigjools> mrevell: btw, I really think the location should be in the pub opposite the Warwick
<intellectronica> pretty much anywhere will be better than the warwick
<intellectronica> sorry, didn't mean to be bitchy. it's about the company, not the location
<jtv> gah... why did stacking stop working in my rf!?
<jtv> Far as I can tell, everything is in 2a.  But:
<jtv> Source branch format does not support stacking, using format:
<jtv>   Branch format 7
<jtv> danilos: I don't think I can land that test fix for you.  For some reason some branches will push and some won't.  :(
<jtv> ah, and as soon as I said that, this one suddenly came through...  Let's see if PQM still thinks it's a "different VCSystem."
<intellectronica> barry: i've got a lovely patch for you to review. removals only
<mrevell> I'm off for the weekend, byeeeee
<jtv-afk> bye mrevell!
<intellectronica> have a loveleee weekend everyone
<salgado> rockstar, how can we get diffs for merge proposals on launchpadlib branches? (https://code.edge.launchpad.net/~gary/launchpadlib/lp-dev-utils/+merge/11607)
<gary_poster> salgado: I think edge updates are broken.  I asked losas about it.
<salgado> gary_poster, I don't understand what's the relation between these.  do you mean that the script which generates diffs on edge is not running?
<gary_poster> salgado: yeah, that's my guess
<rockstar> salgado, I'm not sure what you mean.
 * rockstar thinks gary_poster might be telling lies
<gary_poster> rockstar: unintentionally :-)
<rockstar> gary_poster, that's a given.  :)
<salgado> rockstar, I was thinking that the script were only configured to generate diffs for launchpad branches.  should it generate diffs for any branches?
<gary_poster> rockstar: :-) what salgado is asking about is that my MP has been sitting around for over two hours, and I still don't have a diff.  Similarly...
<rockstar> salgado, if you're talking about moving diffs, abentley and I are talking about an oops on edge that may be caused by trying to get these moving diffs generated.
<gary_poster> oh, no, my "similarly" example is now ok
<rockstar> gary_poster, ah, okay, that's unrelated to what we're talking about then.
<gary_poster> well, I should say, "oh, yes! my "similarly" example is ok now!"
<gary_poster> but anyway, rockstar, should that page have a diff?  If not, does it indicate that something is not running somewhere, maybe?  (That was my suspicion given the recent troubles)
<rockstar> gary_poster, are the branches actually different?
<gary_poster> rockstar: yeah, added a buncha files
<gary_poster> rockstar: maybe adds are not noticed?
 * gary_poster doublechecks he has not done something stupid...
<abentley> gary_poster: The diff is not being generated because the branches are not different.
<gary_poster> I also moved a directory
<gary_poster> I just tried repushing
<abentley> gary_poster: Sorry, typo on my part.
<gary_poster> oh?
<abentley> gary_poster: I mistyped the command to determine whether the  branches were different.
<gary_poster> gotcha
<rockstar> abentley, https://launchpad.net/bugs/427383
<mup> Bug #427383: ValueError: No JSON object could be decoded in merge through the API <api> <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/427383>
<ubot3> Malone bug 427383 in launchpad-code "ValueError: No JSON object could be decoded in merge through the API" [Medium,Triaged]
<mup> Bug #427383: ValueError: No JSON object could be decoded in merge through the API <api> <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/427383>
<rockstar> Can we get an admin to kick ubot3?
<maxb> ubot3 has been kicked before and has come back. To get rid of it permanently, ask #ubuntu-bots
<ubot3> maxb: Error: I am only a bot, please don't think I'm intelligent :)
<maxb> ubot3 owner
<ubot3> This bot is owned by jussi01 - Questions about ubottu should be asked in #ubuntu-bots
<abentley> rockstar: I updated bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/codeimport_api and it seems to be failing differently now: http://pastebin.ubuntu.com/269372/
<rockstar> abentley, contents of the test?
<rockstar> abentley, actually, I'll take a look at it, and send the email to the list.
<abentley> rockstar: You mean text of the test file?
<rockstar> abentley, I meant the text, but don't worry about it.  I'll do some sleuthing, and if I get stuck like last time, I'll email to the list.
<abentley> rockstar: No point sending mail to the list if this is a trivial problem.
<rockstar> abentley, yeah, I'm not sure what's going on.  If it isn't a trivial problem, than I'll deal with it.  You're running out of core hours.
<abentley> rockstar: Cool.
<rockstar> abentley, also, I'm having a hard time finding the diffstat field in the database.
<abentley> rockstar: Are you looking at the Diff table?
<rockstar> abentley, no, I didn't know there was a Diff table.  Looking now.
<rockstar> abentley, thanks.
<abentley> rockstar: np
<mwhudson> good morning
<abentley> mwhudson: good morning.  Forgot you were around today.
<abentley> mwhudson: Was about to invite rockstar to stand-up.
<Fly-Man-> And another one bites the dust ....
<Fly-Man-> What is happening with GIT imports these days ?
<didrocks> Hey guys, I'm trying to checkout LP on mmy updated karmic, but I have an issue: http://paste.ubuntu.com/269385/
<rockstar> abentley, let's stand then.
<abentley> mwhudson: skype?
<rockstar> abentley, I couldn't hear you.
<abentley> rockstar: So I heard.
<mwhudson> abentley, rockstar: few minutes, i'm still eating toast?
<rockstar> abentley, hm, try again.  We can chat before mwhudson can join.
<salgado> sinzui, thanks a lot for the help in converting person-index
<sinzui> salgado: send me the review if you do not want to break it up
<salgado> sinzui, ok
<salgado> sinzui, I think the email addresses in contact-details should have a whole row for itself
<salgado> quite often people have email addresses that are too long for the space we have there, and they will overlap with the time zone there
<sinzui> salgado: Maybe, it was until I tried putting timezone there
<sinzui> salgado: if the address is long, timezone should fall below it. The overlap was cause by the odd contact markup that I put on the side
 * sinzui now doubt what he said is true
<salgado> and I'm thinking it might be a good idea to move the openid right below the email addresses, as it's only shown when the user is looking at their own page
<salgado> let me test it
<sinzui> oh, yes. I prefer that too
<mwhudson> abentley, rockstar: invite me now?
<mwhudson> i am online, whatever skype says
<didrocks> well, reading the mailing-list, I tried to execute make schema && make, without success, unfortunately: http://paste.ubuntu.com/269395/ :/
<salgado> sinzui, indeed, the timezone doesn't fall below emails when they're too long
<sinzui> salgado: So I must have been sniffing glue for lunch. Oh well. Move it to where you like, or drop it since it is on the map already
<salgado> sinzui, I'll move it inside the two-column div, together with the rest of the stuff there
<salgado> the problem really is with the email address thing, that needs the whole row for itself
<sinzui> irc is just as long
<salgado> but irc, as rendered there, can be broken into multiple lines
<salgado> sinzui, should all of our "see-all" links not have the 'Show' bit in their text anymore?
<sinzui> No, I had to remove them from a lot of portlets last week per beuno and kiko
<sinzui> salgado: you might fix bug 386163 by accident
<ubot3> Malone bug 386163 in launchpad-registry "Inconsistencies in contact detail labels on person page" [Low,Triaged] https://launchpad.net/bugs/386163
<mup> Bug #386163: Inconsistencies in contact detail labels on person page <Launchpad Registry:Triaged> <https://launchpad.net/bugs/386163>
<mup> Bug #386163: Inconsistencies in contact detail labels on person page <Launchpad Registry:Triaged> <https://launchpad.net/bugs/386163>
<mup> Bug #386163: Inconsistencies in contact detail labels on person page <Launchpad Registry:Triaged> <https://launchpad.net/bugs/386163>
<salgado> sinzui, not really by accident, but I will
<sinzui> I see edwin fixed the irc form by accident. I'll assign it for him to close
<salgado> sinzui, jabberids will have the same problem, but this only happens when they don't have any character (e.g. a dash) that the browser considers when breaking words
<salgado> maybe what we need is to output our email addresses using user-part<wbr>@host.domain?
<sinzui> Maybe we should consider somethi....yes
 * sinzui cannot remember the name of the formatter
<salgado> oh, a formatter?
<barry> sinzui: ping
<sinzui> salgado: yes there is something in tales that makes long lines wrap
<salgado> break_long_words!
<sinzui> hi barry
<barry> sinzui: hi.  let's talk base-layout.txt
<sinzui> okay
<barry> is there anything we can do other than print huge gobs of html?  it's just so fragile and very difficult to strip down the output
<sinzui> how to you propose we verify each element that it is required to output?
<rockstar> abentley, upon further inspection, it looks like that's a launchpadlib error.
<rockstar> Like it's an error in marshalling an object to rest.
<barry> sinzui: well, let's think about it.  i guess we don't have id's for the really essential bits do we?  and if not, should we?
<barry> i have nothing against adding ids for testing purposes
<barry> (we do that a lot)
<sinzui> barry: in the case of base layout, we are relying on it to make sane pages. so ids are not relevant
<barry> sinzui: ids for testing purposes could make sense though
<sinzui> good luck putting an id in a comment
<barry> sinzui: so then. we does base-layout.txt give us that other page tests don't give us indirectly?
<sinzui> barry: ids, will get you most of the way. You need to put parse <html> to get the whole doc
<rockstar> mwhudson, could you look at https://lp-oops.canonical.com/oops.py/?oopsid=1348EA292 for me?
<rockstar> mwhudson, I just need a hypothesis sanity check.
<sinzui> barry: ? title, scripts, styles, meta, body with classes, headers, footers, diagnostics
<mwhudson> rockstar: ok
<sinzui> barry: No test other than base layout verifies those, nor should they
<mwhudson> rockstar: simplejson.loads('') explodes like this, btw...
<barry> sinzui: indirectly some of them are tested, e.g. titles
<barry> sinzui: i'm not saying that's good, i'm just trying to boil this test down to essentials
<barry> sinzui: as a doctest, you gotta admit, it's not readable :)
<rockstar> mwhudson, yea, it's obvious there's a JSON issue here.  I feel like this might actually be lazr.restful not handling the blank correctly.
<sinzui> barry: I direct is not good enough in this case. that test is painful because base-layout is that important. add ids if you wish, and use BS to parse <html> you still need to verify doctype and comments
<sinzui> barry: I do not agree, because diagnosing failures in the macros was not possible without the test. You can improve the test, but I think you made changes without that test, and that is a big no no
<barry> sinzui: well, one simple way to fix this would be to just blow away the view.render() output and paste in whatever the template outputs now.  but i wouldn't say that's a good solution.  otoh, it's not clear to me it's any worse that what it's doing now
<sinzui> barry: you proposed ids, I said fine, why are you backtrackin?
<barry> i'm not.  i'm looking for options
<mwhudson> rockstar: hmm
<sinzui> I think ids are best because we do not really care about the element, except for title. In all other cases, I think we care about the atts (class, src, href, style) so...
<barry> ok.  i agree, let's go with ids
<barry> thanks
<sinzui> barry:
<sinzui> >>> print find_tag_by_id(page, 'body-tag')['class']
<sinzui> budy overview ponies
<barry> yes, that's the idea
<sinzui> barry: I think the end comment is testing using a split, so that test will continue to work
<sinzui> barry: the doctype could be tested as a split on <html
<rockstar> mwhudson, got time for a quick pre-imp call?
<barry> sinzui: i'm not too worried about the doctype because ellipsis can ignore everything following.  ellipsis work less well the farther deep you go in the output
<mwhudson> rockstar: i will have after i've refilled my coffee cup, one sec
<barry> sinzui: is it really that important to test end comments?
<rockstar> mwhudson, that's fine.  I need you caffeinated anyway.
<sinzui> barry: yes, for the diagnostics at the end of the page
<barry> sinzui: oh yes, those, but the <!-- yui-d0 --> comments?  or maybe i misunderstood what you meant
<sinzui> barry: and as you might guess from the the use of
<sinzui>     >>> print content[content.index('</html>') + 7:]
<sinzui> I was desperate to isolate the one part I needed to fix at a late hour
<mwhudson> rockstar: ok
<barry> sinzui: yep
<sinzui> barry: If you are using  BS you wont care because you can test containership
<rockstar> mwhudson, "User Not Found"
<barry> sinzui: right, that's what i think we really care about
<mwhudson> stupid skype
<sinzui> barry: agreed
<barry> sinzui: cool, thanks.  will you be around for a while?  i may ping you to review this last change when it's ready
<sinzui> barry: I think I will be in an out for the next 8 hours
 * sinzui tries to change 38 pages before Monday
<barry> sinzui: okay, that's cool.  if you're here and want to look, that's fine.  if not, i'm jfdi.  i'm tired of this branch and it needs to LAND! :)
 * barry will not sleep until this is in the tree
<didrocks> well, my launchpad dev setup try on jaunty fails as well :/
<sinzui> barry: I agree. We can fix bugs next week, but we wont know about bugs if you don't land it
<barry> sinzui: so right you are
 * barry hacks
<mwhudson> rockstar: http://pastebin.ubuntu.com/269419/
<mwhudson> (gee thanks, postgres!)
<wgrant> buildbot is sad.
<rockstar> wgrant, yeah, mwhudson said he was going to fix it, but I have a read lock on him right now.
<wgrant> Ah.
<mwhudson> hooray for test runs with 9 landings :/
<wgrant> Yes. It wasn't mine?
<mwhudson> wgrant: no
<mwhudson> it's a rosetta thing
<mwhudson> lib/lp/translations/browser/tests/translationmessage-views.txt
<wgrant> mwhudson: Ah, great.
<mwhudson> seems a bit of the test that should have been deleted
<maxb> When I make minor followup changes to a branch proposed for merging after comments from reviewers, should I resubmit the MP ?
 * mwhudson finally defeats pqm-submit for his '[testfix]
<wgrant> maxb: No.
<mwhudson> maxb: if the changes are small, paste the diff between what was reviewed and the current state of the branch into a comment
<wgrant> maxb: Just reply saying that you've done it, and give the diff.
<mwhudson> maxb: (we'll be fixing the workflow around this sort of thing soonish)
#launchpad-dev 2009-09-12
 * mwhudson_ lunches
<jml> hello all
<jml> maxb, hi
<jml> maxb, I made changes to your timing branch & tried to land it.
<jml> maxb, but something must have swallowed it (my system has been behaving badly recently)
<jml> mwhudson, btw, could you please take a look at https://code.edge.launchpad.net/~stub/launchpad/kill-harder/+merge/11517 -- istr we do very similar stuff in codehosting and wondered if we should share the love
<jml> allenap, are you around?
<mwhudson> jml: hello
<jml> mwhudson, hi :)
<mwhudson> jml: you know we said that the --stacked stuff in rocketfuel-get was pointless wrt shipit & the other thing?
<mwhudson> jml: i'm not sure that's true in a lightweight checkout
<jml> hmm.
<mwhudson> (it's been getting a standalone branch of shipit for me since before i went to lunch :/)
<jml> mwhudson, that's unfortunate.
<mwhudson> jml: yes
<mwhudson> (it also shows how long it's been since i updated sourcecode -- shipit wasn't there!)
<jml> mwhudson, what should we do about it?
<mwhudson> jml: restore the stacking stuff, maybe?
<jml> mwhudson, hmm.
<jml> mwhudson, so for a light weight checkout, it'd stack on the remote branch?
<mwhudson> jml: hmm
<mwhudson> jml: using launchpad with a remote lightweight checkout sounds pretty masochistic
<mwhudson> jml: i don't think it's a use case we need to cater too *all* that much
<jml> mwhudson, ok, so maybe I don't understand what just happened to you
<mwhudson> jml: maybe i didn't understand what you mean
<mwhudson> <jml> mwhudson, so for a light weight checkout, it'd stack on the remote branch?
<mwhudson> ^ in that
<jml> mwhudson, no, before that.
<jml> mwhudson, are you sourcecode branches not kept in a repository that's shared with Launchpad?
<mwhudson> jml: no
<mwhudson> oh actually, that is the problem duh
<mwhudson> i have ~/canonical/lp-sourcecode/sourcecode/<foo>
 * jml wants to stab exetel in the face.
<jml> well, not really.
<jml> what I really want is my karmic updates.
<mwhudson> jml: just think in a few weeks you'll get to experience old world internet
<jml> mwhudson, heh
<jml> mwhudson, a few days, even.
<mwhudson> subsecond ssh connections !
<barry> jml, mwhudson you guys around?
<mwhudson> barry: yeah, i'm working today as a swap day
<mwhudson> barry: jml is just an addict
<barry> :)
<jml> :(
<barry> i am so close to (trying to) land my bug 417089 branch, i can taste it
<mup> Bug #417089: Conform heading and breadcrumb rules to UI 3.0 <story-ui-3> <Launchpad Foundations:In Progress by barry> <https://launchpad.net/bugs/417089>
<ubot3> Malone bug 417089 in launchpad-foundations "Conform heading and breadcrumb rules to UI 3.0" [High,In progress] https://launchpad.net/bugs/417089
<mup> Bug #417089: Conform heading and breadcrumb rules to UI 3.0 <story-ui-3> <Launchpad Foundations:In Progress by barry> <https://launchpad.net/bugs/417089>
<mup> Bug #417089: Conform heading and breadcrumb rules to UI 3.0 <story-ui-3> <Launchpad Foundations:In Progress by barry> <https://launchpad.net/bugs/417089>
<barry> this is going to break everyone's page titles <wink>
<mwhudson> i'd better make sure i break ec2test good and proper to help with that then
<barry> yeah, thanks.  i've only been working on this damn branch for 2 weeks
<barry> but i think i'm reaching steady state with devel ;)
<mwhudson> i can see how working on the weekend helps there :)
<jml> brb. networking issues.
<barry> i'm having fun installing karmic on my new machine while i wait for the tests to run
<barry> so it's all good.  i'll be up at 2am to see how ec2 did
<mwhudson> barry: do you want me to review something?
<barry> mwhudson: actually, the branch has been pretty well reviewed, well except for the 100 tests i've been fixing
<jml> ... and yet again,
<barry> mostly crap stuff like fixing the browser.titles
<jml> ec2test --headless taking so long makes my life less pleasant
<barry> but you can imagine it's a LOT of pages
<barry> jml: i hear you there
<mwhudson> jml: working towards fixing that!
<mwhudson> barry: i guess i'm saying, i'm sure there was a reason you asked if we were around
<mwhudson> barry: do you want help, or just emotional support? :)
<barry> just emotional support :)
<barry> just sayin' hi to my crazy saturday workin' antipodean friends :)
<barry> go ec2, go
<jml> back
<jml> so....
<mwhudson> the inherent bone-headedness of some of ec2test is wearing me down
<mwhudson> EC2TestRunner should take an instance as a parameter, not construct one, fer 'eavens sake
<mwhudson> also, vals
<jmux> Hi. How can I use an object name containing a / in an URL traversal? I thought of mapping the slash to tilde, but this results in a traversal NotFound error from the ParentSet, as the traversal now uses the mapped name for lookup, but the ParentSet doesn't contain an object with this name.
<wgrant> jmux: What object are you attempting to traverse to that has a / in its name?
<jmux> I want to add Debians security components - updates/main, updates/contrib, updates/non-free
<wgrant> Why? That's probably not the right way to go about things.
<jmux> Because I want to import the debian security archive into Launchpad as it is.
<wgrant> I see.
<wgrant> Where are you seeing the traversal problem? It should only manifest itself in the API, AFAIK.
<jmux> I added an uri_name property to the component interface, which does the mapping
<jmux> This works as expected, so I get the updates~main link to manage the component in the ComponentSet View, but the traversal doesn't work and I don't know. how to tell Zope / Launchpad to map updates~main back to updates main for the ComponentSet lookup
<jmux> updates/main
<wgrant> You'll have to adjust ComponentSet to do it itself. There's nothing really magical.
<jmux> I added the reverse mapping to __getitem__, and this works, but it seems some other function is used for traversal and I couldn't figure out which one
<wgrant> Odd.
<wgrant> Although I wonder why you want to do this -- that's not how Soyuz models updates/security.
<jmux> Current Soyus DB model doesn't allow this construct of sub-components
<wgrant> No. It uses pockets for that purpose.
<wgrant> What exactly are you trying to achieve?
<jmux> Just to keep the Debian structure for import. I know I can set the pocket and override the component for gina imports, but I would like to use the Debian component names.
<jmux> But more importantly I have series names, which also contain a slash, which will have the same problem
<wgrant> that will cause utter chaos.
<wgrant> Why do you have such things?
<jmux> It has been like this the last 4 years - names like halut/2.0.0 where halut is our codename for Debian Etch and theh rest is the Version string
<wgrant> Oooh dear.
<wgrant> That's not going to go well.
<jmux> I would just need a mapping that tells Launchpad to map series and component names from "x/y" to "x~y" for travsersal...
<wgrant> More stuff than traversals will break if you mangle series names like that.
<wgrant> In fact, PostgreSQL will probably tell you that you are being silly.
<jmux> Why? I don't think anything in PostgreSQL depends on not using '/'
<wgrant> No, but distroseries.name should have a valid_name constraint on it.
<wgrant> And that should forbid '/'.
<jmux> Ok, but that's not really hard to change.
<wgrant> No. But more stuff than traversals depends on that constraint.
<wgrant> It's probably a very bad idea to attempt to change it.
<jmux> So is there a known way to do this mapping for traversal?
<jmux> My solution: one can use the traverse function of the Navigation class of the parent set to traverse the uri part directly. I missed that when I read the Navigation class.
 * maxb is scared by the preceding conversation and wonders if mapping to pockets wouldn't be entirely saner in the long run
<maxb> There is a discussion in the reprepro manpage about how the debian slash-in-component stuff is so broken
<maxb> search for FakeComponent
<wgrant> maxb: It does seem like a much saner approach to import them as pockets and perhaps alter archivepublisher to do them as evil nested components.
<maxb> Do we need to publish them that way?
<maxb> Let that insanity stay with security.debian.org :-)
<wgrant> maxb: If they have users, yes.
<wgrant> This sort of thing will come into the picture if Debian PPAs end up happening.
<maxb> I don't understand.... given LP hosts no debian archives currently, can't it decide what components to allow
<maxb> Well sure, but the main debian archive doesn't have nested components, that insanity is limited to s.d.o
<wgrant> maxb: Does it not seem like jmux is planning to do exactly that?
<wgrant> (not Debian, but a derivative?)
<maxb> How's that going to work then? Isn't that going to require a second public instance of Launchpad, or the derivatives buildds in canonical's datacentre?
<maxb> Or something similarly tricky to arrange
<wgrant> Probably.
<maxb> Anyway, the solution is for the derivative not to copy debian's weird naming for security updates
<maxb> :-)
<wgrant> Right.
<wgrant> But if they need to for compatibility, it's to map pockets differently on the output.
 * maxb mutters something about symlinks :-)
<wgrant> Ah, you're right, that will work. I was thinking that Release would have the full name, but it just lives within updates/.
<wgrant> That makes everything muuuch easier.
<wgrant> Oh, it *does* actually use the full names, but only at the top.
<wgrant> How confusing.
<maxb> It's more than a bit weird, but it mainly seems to be that you generate the "Components:" line in the Release file including extra path components which shouldn't really be there
<wgrant> Yes.
<wgrant> Which is substantially easier than doing what I suspected might be necessary.
#launchpad-dev 2009-09-13
<dhillon-v10> quit
<dobey> for person.getMembersByStatus() do results of "Approved" include people whose status is "Administrator"?
<dobey> no it does not
<sayakb> hello! I just read a UWN article and noticed that some work is being done in moving LP pages to AJAX. i dont have bzr setup, so before I checkout anything, can someone tell me about the other things I'll need to have it running locally? for example, is LP php based? ;)
 * sayakb notices py
<mwhudson> good morning
<allenap> mwhudson: Morning, and good night :)
<thumper> morning
<mwhudson> thumper: call?
<thumper> mwhudson: ok
<mwhudson> thumper: skype doesn't think you're online (even when i try to call)(
<jml> good morning
<mwhudson> jml: hello!
<jml> mwhudson, hi :)
<jml> having an espresso machine is nice and all
<jml> but very little beats a six cup plunger filled with eight cups worth of coffee.
<mwhudson> yeah, i have similar views
<mwhudson> (although i prefer filter to plunger coffee)
<jml> understandable
#launchpad-dev 2010-09-13
<thumper> lifeless: what have you done to edge?
<wgrant> It exploded yesterday.
<wgrant> Quite spectacularly.
<lifeless> wgrant: please try to get a broken wadl now
<wgrant> lifeless: I'm not sure this explains everything.
<lifeless> wgrant: I'm not sure it explains anything
<lifeless> wgrant: but, are the symptoms gone
<wgrant> the last run succeeded. Doing it again to confirm...
<poolie> lifeless: oh, did you land those flag scopes? or someone else?
<poolie> either way, that's great
<lifeless> I'm been extending it a little, yes.
<lifeless> the pageid one will land today
<lifeless> once we climb out of this downtime hole
<lifeless> wgrant: ?
<wgrant> lifeless: Seems happy now.
<lifeless> great
<wgrant> I can't say for sure, but three runs succeeded.
<lifeless> so running edge on 2 appservers with a backlog of 160 requests is a bad idea
<wgrant> Heh.
<wgrant> Remarkable.
<poolie> that's great
<poolie> i hope to lower my queue enough to do the edit gui
<poolie> it is in a branch though, if i don't get to it
<lifeless> poolie: I filed a bug to track it; if you're not actively on it I can unassign you
<lifeless> I needed a reference point for a code comment IIRC
<poolie> i'm not actively on it but i feel i ought to close a few bzr bugs first
<poolie> i'm hoping/aiming for quite a productive week here
<wgrant> lifeless: I threw some example error messages on bug #636713.
<lifeless> poolie: could you perhaps link your branch to the bug?
<poolie> perhaps i could
<lifeless> thanks:)
<poolie> nice summary :)
<poolie> i mean description
<poolie> bug 636192
<lifeless> poolie: ;)
<poolie> no bot?
<lifeless> _mup_: yo
<lifeless> tumbleweeds
<lifeless> wgrant: 41 /    9  Distribution:+search third top oops in prod
<wgrant> lifeless: Is that hard/soft?
<wgrant> So the third top OOPS only occurs 41 times a day? That's not bad.
<lifeless> on sunday
<wgrant> Ah.
<lifeless> still, edge is ~ 13 seconds, time to shrink lpnet a little
<lifeless> thumper: https://code.edge.launchpad.net/~lifeless/lp-production-configs/timeouts/+merge/35239
<lifeless> thumper: ^ when you have a sec ;)
<stub> thumper: So both the existing indexes *might* be useful - you can look up BranchRevision indexes by revision using the (branch, revision) index, but I think it is more efficient to look them up using the (revision, branch) index.
<stub> thumper: It would be nice if we can drop both unique indexes and use just the primary key index though, as inserts will be faster and we save disk space. And maybe it will even be faster as we are more likely to have the index in PG shared memory.
<stub> thumper: So if we drop both now, we can try adding one back later if there are read performance issues. Or we can leave things with two indexes for now, and try dropping one later.
<stub> Do we ever need to see which branches a particular revision is in?
<thumper> yeah...
<thumper> stub: but this is one of the things I'm wanting to either factor out
<thumper> stub: or find another way
<stub> I'm thinking we should keep the patch as you have it, and just be aware that if Revision -> Branch traversal is slow we might be able to fix it with a new index.
<thumper> stub: I hadn't thought of that reason for the constraint
<stub> I can't really guess which will be fastest for reads, and no index is faster for inserts and saves disk
<thumper> stub: *most* of the time we are going from the branch to the revisions
<lifeless> \o/ less bare excepts
<lifeless> stub: I just remembered I wanted to ask you about layer teardowns.
<lifeless> stub: looks like zope.app.testing.functional/FunctionalTestSetup.tearDownCompletely() should let use teardown some more layers?
<stub> Sounds like it would, yes. That didn't exist before.
<lifeless> ok
<lifeless> rs= you to do it ?
<stub> Maybe make the whole fork-a-new-process thing unnecessary too
<lifeless> yes
<lifeless> thats a good 2 or more minutes per test run
<lifeless> on its own
<stub> rs=stub. If the tests pass, great.
<wgrant> lifeless: Anything more to add to the bug?
<lifeless> stub: could you do me a small favour
<lifeless> ttps://devpad.canonical.com/~stub/dbr/last-3-hours.txt suggests to me that we've reduced load on the master DB post-rollout.
<lifeless> or is it within the variation you've seen anyway
<stub> lifeless: load seems less
 * stub looks for the graph
<lifeless> \o/
<lifeless> brb
<stub> lifeless: https://lpstats.canonical.com/graphs/DBCpuLoadAppServers/20100907/20100914/ -- doesn't seem much changed according to that
<thumper> lifeless: have you filed a bug about the code import xmlrpc timeouts?
<lifeless> yes
<lifeless> search on the page id
<lifeless> hmm, I thought I had
<lifeless> stub: you did the session db schema change?
<stub> lifeless: yes
<lifeless> cool
<lifeless> I doubt spm has the cycles
<lifeless> but I'll see if Tom does, to move the cert deployment forward.
<spm> me? doubt it. (cycles)
<lifeless> how hard is it to batch a page ?
<lifeless> is there a howto ?
<lifeless> so
<lifeless> batching
<lifeless> I guess I need to dig myself.
<stub> lifeless: Have you seen anything to suggest we should change our Storm cache sizes? Its available as a config parameter in launchpad-lazr.conf but we have never tuned it - just an initial guess.
<lifeless> stub: I haven't seen anything saying we're thrashing in swap
<lifeless> stub: and from a page layout perspective the storm cache is in principle unneeded anyway, once we finish structuring things right
<lifeless> stub: but even so, no, I haven't seen any [obvious] cache thrashing
<lifeless> once we get down to a sensible 10-20 queries per page that would be very obvious
<stub> I wonder if we would benefit from a larger cache then? I don't think flushes are instrumented, so I don't know if we are needlessly reloading objects that could have been pulled from cache
<lifeless> stub: I doubt we'd benefit - we flush the cache on transaction boundaries anyway
<lifeless> stub: flushes aren't recorded AFAIK; but we'd see the loads as a series of single object loads at the end of the apge.
<stub> I'm thinking about prejoin type stuff which by its design relies on objects remaining in the cache until they are needed.
<stub> (as nothing is holding references to them)
<stub> But I guess loading that many objects is the real problem, not how to ensure that we can access too_many objects efficiently.
<lifeless> yeah
<lifeless> I think theres a few angles to it
<lifeless> firstly prejoin is not a complete enough eager-loading for us
<lifeless> what we're starting to do in e.g. Person is sufficient
<lifeless> secondly, if we are hitting a cache threshold, I think we'll see it very clearly when we look at an OOPS.
<lifeless> stub: what is the cache size set to ?
<stub> Oh - for the appserver it is 10k so I guess no problem there. Most batch jobs have it set to 500.
<lifeless> ah
<stub> All in schema-lazr.conf - nobody has ever overridden the defaults in the instance configs.
<lifeless> batch jobs I'm not really scrutinising yet.
<lifeless> 500 is possibly too small hyes.
<lifeless> but OTOH
<lifeless> batch jobs should be pretty darn precise with what they do compared with appserver pages which are all over the shop
<lifeless> Intrinsically I mean; like 'show me all specs' is an appserver use case, for a batch job it would be 'email changes to specs' and thats time-bounded.
<StevenK> Okay, fine. SQLObject I hate you.
<lifeless> StevenK: you can return a storm ResultSet even while the class uses SQLBase, its easy and lets callers start depending on it.
<StevenK> lifeless: That sounds like a good plan
<lifeless> I did this to searchBugs, for instance.
<lifeless> oh
<lifeless> and for a relatively clean one - hahhaha -
<lifeless> see Distribution.specifications
<StevenK> lifeless: Right, which ends up using Store.of anyway
<wgrant> Store.of is Storm...
<lifeless> StevenK: thats the point, the guts of the function is still shitty hand-string-joined-sql
<StevenK> Then I may as well just use IStore(IDistroSeries) ...
<lifeless> StevenK: but it returns a perfectly cromulent storm result set
<lifeless> StevenK: Store.of if you have an object is *important*
<lifeless> StevenK: You want to match the store that the object came from
<StevenK> lifeless: Or can I just call methods on the SQLObjectResultSet I get back?
<lifeless> of course you can
<lifeless> converting that to a storm ResultSet would be a good idea though.
<lifeless> StevenK: perhaps you should explain whats up
<StevenK> lifeless: My hidden question is "Will that involve more database roundtripping" ?
<lifeless> StevenK: will whatr
<StevenK> lifeless: Will calling methods on the SQLObjectResultSet result in more database roundtripping?
<lifeless> if you call methods that do round trips.
<lifeless> (sorry that thats the answer, but thats the situation)
<lifeless> resultsets will do roundtrips when you call methods on them
<lifeless> which methods cause roundtrips depends on the resultset type
<StevenK> lifeless: To explain what's up: I'm writing a new method on IDistroSeries called deriveDistroSeries, and one of the arguments is the name of the distroseries. The method will look up if the distroseries exists, and if it does, use it, and if it doesn't, use it.
<StevenK> I'm having trouble looking up if the distroseries exists.
<mwhudson_> StevenK: i get the feeling that you are worrying about something that essentially shouldn't be worried about, but slightly over specific questions are masking the real issue
<lifeless> StevenK: so
<lifeless> StevenK: firstly, write the function taking a distroseries object not a name.
<StevenK> mwhudson_: TBH, I suspect my problem is that getUtility(IDistroSeriesSet).findByName(name) doesn't return what I think it should
<lifeless> StevenK: if you intend it to work in that way.
<lifeless> StevenK: secondly, if you want a name-based version, add that as a trivial wrapper.
<lifeless> don't conflate the two.
<lifeless> you'll find it clearer and easier to test if you avoid magic like that.
<wgrant> I can't see a reason to have it take a name.
<lifeless> thirdly, if you are concerned about db round trips, *test it* - write tests, like those I've mentioned in perf tuesday emails, that check the number of db calls that go on.
<wgrant> That's only good for things like components and sections.
<lifeless> there is /nothing/ like tests for catching gotchas; I can help you ensure that the test is valid, once you have a sketch of it up.
<lifeless> tests and bastard users AKA 'QA'
<lifeless> wgrant: mwhudson_: any tips on how to 'batch a page'
<lifeless> is there a howto
<lifeless> or docs
<wgrant> lifeless: I don't know of any.
<stub> I tend to avoid the old crufty SQLObject era APIs on the FooSet utilities and use native Storm syntax. IStore(DistroSeries).find(DistroSeries, name='myname').one()
<wgrant> I just grep.
<lifeless> wgrant: do you know how to batch a page? if so, please be writing it up!
<mwhudson_> lifeless: don't know of anything either
<stub> But I'm not usually in browser code, which is supposed to go via the *Set for security issues.
<StevenK> stub: This is API code
<lifeless> the store objects we have are security proxied
<lifeless> so they will wrap any objects returned
<lifeless> the Sets don't add anything security wise.
<StevenK> Just pain, misery and SQLObject
<lifeless> unless I'm deeply confused.
<stub> I didn't think our Store objects were security wrapped. Under the hood, they come from the IZStorm utility which is not registered as a secureutility IIRC.
<lifeless> hmm
<lifeless> then we should security wrap them
<lifeless> because API's also need the security enforcement, and model code what APIs expose.
<lifeless> I'll file a bug
<stub> Just include 'may' in there - I'm not 100% on this.
<wgrant> The security stuff here is a bit complicated.
<wgrant> Even if those stores are proxied, Store.of isn't.
<wgrant> I don't think IStoreSelector is secured.
<lifeless> all in the big
<lifeless> *bug*
<lifeless> ok
<lifeless> so whats a simple batched page
<lifeless> branches?
<wgrant> Bug listings, or possibly /people
<wgrant>  /people used to be non-standard, but I think Curtis fixed that recently, so it might be a nice example.
<StevenK> steven@liquified:~% du -ch /tmp/launchpadlib-cache-* | tail -n 1
<StevenK> 225M	total
<StevenK> For want of a test suite that doesn't do evil things to /tmp
<lifeless> we need to fix up zope.app.testing first
<lifeless> here, a blindness pull - zope.app.testing.functional.FunctionalTestSetup.__init__
<lifeless> the first line.
<StevenK> lifeless: TBH, I'm happy to accept it's a bug in Zope
<lifeless> its not per se
<lifeless> its a stack that needs cleaning up is all
<lifeless> poolie: those scopes have now landed.
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.10 | PQM is open for business | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<wgrant> Oh, nice.
<lifeless> wgrant: ?
<wgrant> pageid-based FF.
<lifeless> yes
<lifeless> in e2 atm is a FF memcache control
<wgrant> So we can turn caching on and off with a flag?
<lifeless> yes
<lifeless> per pae
<lifeless> per page
<lifeless> next up after that is per pageid timeout control, the prerequisite for that should land overnight too
<poolie> oh, great
<adeuring> good morning
<henninge> Is LP production running on python 2.6 now?
<lifeless> partly
<lifeless> AIUI rolling upgrades are still going on - check the wiki page if you care
<mrevell> Hello
<lifeless> hi
 * mwhudson_ wonders how crazy running launchpad on arm would be
<mwhudson_> i guess the lack of ram would be the killer
<lifeless> 1TB should be fine
<StevenK> Our current servers don't have that ...
<mwhudson_> 256 megs might be enough to have an appserver limp along, i guess
<elmo> 1000     21319 50.2 10.9 1022248 669056 ?      Sl   Sep09 2859:33 /usr/bin/python -S bin/run -i lpnet7
<elmo> mwhudson: ^--
<wgrant> You can just run a dev appserver and librarian in 512.
<wgrant> But it's, er, painful.
<lifeless> we should fix that
<wgrant> There's also the issue of the PPA not building on armel.
<wgrant> But apart from that it should work.
<mwhudson_> elmo: i wasn't suggesting it for prod :)
<lifeless> jml: welcome back
<jml> lifeless, thank you.
<mwhudson_> hello jml
<jml> mwhudson_: hi
<jtv> noodles775: hi there!  I'm working on displaying TranslationTemplatesBuild.  Ironically, it involves adding a view for TranslationTemplatesBuildJob so the builder displays can link to the TranslationTemplatesBuild.
<noodles775> Huh, a view for the TTBuildJob? Do you mean for the TTBuild?
<jtv> noodles775: no, that's the thing
<wgrant> Along the lines of BuildFarmBuildJob?
<jtv> I need the *-current.pt template to link to the build.
<jtv> wgrant: exactly
<jtv> and hi ;)
<wgrant> hi.
<jtv> But to stop me from getting it all wrong: how do I find the right BuildFarmJob for a given BuildQueue?
<jtv> (I realize it's all going to change)
<wgrant> You shouldn't need to know that.
<wgrant> (and I forget)
 * noodles775 would need to look at the code and see what soyuz/code does too.
<jtv> Okay, then can I just replace the display of the current BuildQueue for a builder with a display of the ongoing build?
<jtv> Or BuildFarmJob?
<jtv> I guess I'll just steal from there then.  :)
<wgrant> Stealing ideas from elsewhere is almost always the best idea.
<wgrant> Since that code probably works...
 * bigjools asks jtv to consider performance when writing this code
 * jtv is happy consider performance, but getting the smegging thing working at all does come into play as well
<jtv> Now, the soyuz builds seem to put all this into a link formatter.  Which one?
<jtv> PackageBuildFormatterAPI?
<wgrant> Watch out -- there are unobvious formatter specialisations which have caught me before.
<wgrant> I don't remember details. But search for subclasses first.
<wgrant> Or it gets very confusing.
<jtv> Hmm what _is_ the context for soyuz's view there?  The build or the job?
<wgrant> The +current view?
<jtv> Yes, on that one?
<jtv> Looks like it's still an IBuildFarmJob as before.
<wgrant> BuildFarmBuildJob wraps the Build.
<wgrant> Which is itself an IBuildFarmJob.
<wgrant> But not an IBuildFarmJobOld.
<jtv> So chances are that we'll want the same on BuildFarmBranchJob.
<jtv> And perhaps that'll allow us to get rid of BuildFarmBuildJob and BuildFarmBranchJob, which after all were really only there to cover for the absence of a Build.
<wgrant> Once Translations complies, we can do all sorts of mass deletions.
<jtv> Trueâno particular need to fix that up now.
<thumper> jml: https://dev.launchpad.net/Code/BranchRevisions
<lifeless> \o/ memcache branch passed ec2
<jtv> bigjools, noodles775: lib/lp/soyuz/browser/configure.zcml defines a browser:page (with name="+current") for lp.buildmaster.interfaces.buildqueue.IBuildFarmJob
<jtv> I'm surprised that doesn't blow up, given that I don't see IBuildFarmJob in lp.buildmaster.interfaces.build*queue*
<noodles775> jtv: it seems to have been added (or at least moved there) by you? I'm not sure why it was added (or perhaps just moved there from elsewhere).
<jtv> noodles775: I guess it's just a dead page and I can delete it.  But WTF doesn't anything complain about it?
<noodles775> Removing it and running the tests might show why... otherwise, not sure.
<jtv> I think I know why it's there and it should indeed be a dead page.
<jtv> wgrant: afaics I do need to know how to get from a BuildQueue to a BuildFarmJob because the UI seems to find my build as buildqueue.specific_job.build.  :/
<jtv> The specific_job is TTBJ.
<jtv> So the TTBJ needs to find the BuildFarmJob
<jtv> (I get here because Builder.currentjob is the starting point for the rendering of builds and buildjobs, and that's a BuildQueue.)
<jtv> noodles775: do you have any idea how I can find my BuildFarmJob given a BuildQueue?
<noodles775> jtv: Do you mean other than how the soyuz/code parts do it?
<jtv> (I also need to look at build histories, of course)
<jtv> noodles775: they keep references to build objects in the db, don't they?
<noodles775> Yep, they have linking tables (ie. BuildPackageJob)
<noodles775> I don't know what you do and don't have in the DB... can you remind me what's different for translations?
<jtv> That's hardly something I can do.  I guess I could move TTBJ over as some kind of view on TTB, butâ¦ yuck.
<jtv> noodles775: there was never any build object.
<jtv> And now the new infrastructure is at a point where it seems to require a build object in the obsolescent old-model classes.
<noodles775> jtv: what is build_queue.job_type in your case? TRANSLATIONTEMPLATESBUILD?
<jtv> yes
<noodles775> jtv: and can you get a hold of your TTBJ based on the queue (ie. based on the job.id I assume)?
<jtv> noodles775: yes, that's all linked together using the old modelâBuildQueue.job == TTBJ.job == Job.id
<jtv> The problem is getting from anywhere in the old model (BuildQueue, TTBJ, Job) to the new model (BuildFarmJob, TTB)
<noodles775> jtv: I was going to ask why buildqueue.specific_job doesn't just work (or at least, why it can't just work if you hook up the utility), but then saw that TTBJ.getByJob() is returning a BranchJob.
<jtv> Well TTBJ lives in the BranchJob table.
<jtv> buildqueue.specific_job does just work, but it returns a TTBJ not a TTB.
<jtv> If it needs to return a TTB, same problem: I have a BuildQueue and I need to find the corresponding TTB.
<jtv> Or BJF.
<jtv> *BFJ
<wgrant> jtv: isn't the TTB accessible from the TTBJ?
<jtv> wgrant: noâthat's _exactly_ the problem I've been trying to solve.  :)
<wgrant> In the other cases, the job exists to link the build and the buildqueue.
<jtv> Yes, and the big problem we were going to solve with this new model was that TTBJ was not related to any build object.
<wgrant> But it exists now.
<wgrant> So TTBJ could link to it.
<jtv> Exactly.  It's great, but I can't find it.
<wgrant> Oh, right, it's a BranchJob arrgh.
<noodles775> wgrant: TTBJ isn't a DB model...
<noodles775> yep.
<jtv> TTBJ could link to it _if_ I move the entire class to a new table, which will be lots and lots of work for an obsolescent class.
<noodles775> jtv, wgrant: I wonder if this would be a good opportunity to start the transition. ie. jtv's branch could add the job attribute to BFJ. Then his TTBJ class could have a build attribute that just looks it up based on the job?
<wgrant> Depending on how large the branch already is, that could be reasonable.
<jtv> Will they be the same Job though?
<wgrant> They probably could be.
<noodles775> jtv: you'll need to make sure they are - no one else is using it yet (so it will be NULL for code/soyuz).
<wgrant> I see no reason why the BuildQueue Job can't be made persistent.
<wgrant> It shouldn't break anything.
<jtv> flw :)
<wgrant> You don't even need to start using its attributes yet, I guess.
<jtv> I'm not going to be able to make this kind of change in my ongoing branch.  How about this:
<wgrant> Adding and populating a simple BFJ.job FK shouldn't be troublesome.
<wgrant> You don't have to do anything with it.
<jtv> I make sure build histories can link to and render TTBs (they already render).
<jtv> I get that change reviewed.
<wgrant> Linking isn't that important. Not crashing is.
<jtv> My branch can already render TTBs.
<jtv> So I make sure build histories can link to and render TTBs without crashing,
<jtv> get that change in together with a bunch of other ones I've already accumulated
<jtv> (they're blocked on this branch),
<noodles775> Sounds good.
<jtv> land the lot of 'em,
<jtv> then we twiddle the model further so that Builder.currentjob.specific_job.build works,
<deryck> Morning, all.
<jtv> and then (afaic in yet another branch) retire BuildFarmBranchJob and basically anoint BuildFarmBuildJob as the buildmaster-standard implementation for BuildFarmJobs.
<wgrant> And then delete it.
<noodles775> Yay.
<jtv> Yes.
<jtv> Kill.
<jtv> Kill.
<jtv> Kill.
<jtv> This means that for now, the "Building [...]" display for a builder will _not_ link to the TTB yet, only to the branch.
<jtv> That change comes in part (3).
<jtv> Which leaves me free right now to worry about functional display of TTBs in builder _histories_
<jtv> As an Orwell character might have put it, to fix the present we must first fix history.
<jtv> Now, how _does_ the builder history get rendered?  Retrieve all BuildFarmJobs and for each, obtain & render the specific job?
<jtv> noodles775: does BuildFarmJob play any role in rendering build histories yet?
<jtv> danilos: my plan is this:
<jtv>  * Finish & land my UI branch.
 * danilos is all ears (well, /me is all ears anyway, but now especially so)
 * jtv holds back nasty remark about huge nose
<danilos> :)
<jtv>  * Adjust the model so that we can find the new buildfarmjob class (the historic record, basically) where we need to from the old TTBJ class in the same way that the other old-model classes can.
<jtv>  * Clean out the classes, interfaces & templates that we currently have to paper over this problem, using the new tie between the two models instead.
<jtv> That last step eliminates our "special" position in the build-farm model afaics.
<danilos> jtv, can we not have a simple adapter for TTBJ to IBuild if that's what we need? what exactly is missing?
<jtv> What's missing is any kind of link between the objects in the old model and the objects in the new model.
<jtv> Because the two are tied together only through the build objects that we never had.
<danilos> jtv, right, so is it not simpler to add a link to TTBJ on the Build object then? (I assume Build object is part of the "new" model)
<jtv> You assume right (although it's confusingly called BuildFarmJob)
<danilos> jtv, ok, so that's what you mean with "adjust the model" step?
<jtv> Doing it that way would still be very ugly.  But more to the point, that's all for the next stepâwhich we should hammer out in detail with the Wellington crew.  There is a plan.
<jtv> For the more distant future, the plan was to re-introduce Job which is so far completely missing from the new model.
<jtv> In the old model, Job was what tied all of it together.
<danilos> jtv, hum, so what kind of "adjusting" is needed in the first step? uglifying and then cleaning up later is ok by me :)
<jtv> The first step is just making sure nothing blows up with what I already implemented, and landing it all.
<danilos> jtv, right, then I mean second step
<danilos> jtv, also, would what you implemented allow us to eg. generate stats based on the data in the DB and interrogate DB directly if we want to know more about any particular build?
<jtv> The second step _as it looks now_ involves moving the re-introduction of Job forward.  We add a FK to Job to BuildFarmJob, and then I can use that to find our new build class ("derived" from BuildFarmJob, though in a separate table) from our old buildjob class.
<jtv> danilos: it should yes, since it introduces creation of BuildFarmJobs for our job type.
<danilos> jtv, right, so step 1 is golden for us, step 2 and 3 are basically interdependent as well?
<danilos> jtv, also, what happens with TTBJ, where do we move the functionality it provides to?
<jtv> danilos: step 1 should be a small one from where I'm standing (cross fingers), which would be great for reporting on our end.  And again as you say, 2 and 3 are interdependent in that 2 moves forward design and implementation work that step 3 is currently stuck on.
<danilos> jtv, right, so let's do that, and then we can try to figure out the minimal stuff we need to do to unblock everybody else
<jtv> danilos: TTBJ provides very little.  It gets replaced with TranslationTemplatesBuild, which fits neatly into the new model.
<jtv> I think step 1 would unblock things for the others.  Which is one reason among many why I want to phase things this way.
<danilos> jtv, right, if that's the case, let's get it done, and not worry about other things yet :)
<danilos> jtv, though, I am a bit confused: wouldn't we need to introduce the link between the TTBJ and BFJ before we'd really unblock others?
<danilos> jtv, also, why can't we simply introduce a new field on TTBJ that links directly to BFJ (TTBJ is going away anyway)
<jtv> TTBJ lives in the BranchJob table, so we can't do that.
<danilos> jtv, uhm, why not? (yes, it might be ugly, but wouldn't it be dirt cheap?)
<jtv> I don't think we need to introduce a link in order to unblock others; what matters is that each of our jobs will have a representation in the new model.  Where the UI starts out in the old model and tries to navigate to the new model, our jobs simply don't exist just as things stand now.
<danilos> jtv, ok, fair enough
<danilos> jtv, but, how will the jobs be executed? will there be an interim phase where such link is needed for execution as well?
<jtv> Exactly _how_ we link the two is part of step 2, and that's something to be discussed with the Wellington team.  How the jobs will be executed won't matter much, since our jobs will be present in both models simultaneously.
<jtv> AIUI as soon as we start executing based on the new model, we also stop generating the old-model objects.
<danilos> jtv, it does matter because I can easily think of a migration approach that would leave our jobs not running
<danilos> jtv, but, since you seem confident, I'll trust you on that ;)
<jtv> We're generating the same jobs in both the old and the new model.  They both contain the same information.
<jml> I'm off to grab some lunch. Back soon.
<danilos> jtv, ok, imagine a migration path like this: "we want to start using the new code, but from within the old model, just like UI: 1. get old model jobs, 2. get a link to new model jobs [just like UI, right], 3. if found, run the job"
<jtv> Yes, we could do it wrong if we _really_ wanted to, but I don't really see the need.
<danilos> jtv, it's not about a need, I actually find the above approach quite sane, especially considering that that's how UI migration seems to be getting done
<jtv> Well what would happen if you tried it that way, things would blow up.
<danilos> jtv, how come the UI doesn't blow up, yet it uses new model for all the other builds?
<danilos> (or does it?)
<jtv> Because that's builds, not jobs.
<jtv> We execute jobs, not builds.
 * danilos is now thoroughly confused
<jtv> danilos: if you're thoroughly confused then at last you fully understand the model.
<jtv> congratulations!
<danilos> jtv, thank you :)
<danilos> jtv, ok, so let's watch out for any weird migrations, I'll go watch out for some food, and not worry about much else right now other than landing the code you've got
<jtv> I'm also past eod now, not just temporally but especially mentally.  :)
<jtv> But I did want to appraise you of this before the heart attacks started.
<danilos> jtv, sure, tty tomorrow
<gmb> rockstar, Is it fair to assume you're not around for the next hour or so?
<gmb> Heh. s/hour/3 hours/
<gmb> deryck, Do you know if rockstar got anywhere with his wigety wizard wonderousness? ISTR seeing something fly across my screen about it late last week, but I didn't really pay it much heed at the time due to being RM-frazzled.
<deryck> gmb, yeah, I worked with him on Friday to get his event bug fixed and unblocked him.
<deryck> gmb, I suspect he may still be today or tomorrow wrapping that up still.
<gmb> deryck, Okay. I'll find something smallish to do in the meantime.
<gmb> Oh, I've got that bug import to do first. Win.
<deryck> gmb, and there's really no bug in the story left that doesn't require the config widget, is there?
<gmb> deryck, One or two, but no, not many.
<gmb> It is something of a blocker.
<deryck> yeah, I told rockstar to please ping you or I if he needed help, since we really need this.
<gmb> deryck, Cool.
<deryck> gmb, he did mention he was stopping at a two-step widget, since that's all he needed.  Do you need more steps?
<deryck> gmb, maybe you should take a peak at lp:~~rockstar/lazr-js/wizard-widget to get an idea of what's coming to.
<gmb> deryck, Well, yes and no. There are potentially up to 4 forms that could be show in one overlay in a given sequence, but they could be logically split across two widgets. I'll take a look at rockstar's branch this afternoon and see what the deal is.
<gmb> (Also, there's always the chance that I could hack it to work the way I want, but that might not be TRTTD)
 * jml back
<jml> and my inbox is empty!
<gmb> jml, Select all, delete?
<jml> no. inbox zero style, instead of 1100 unread conversations in my inbox I know have about 100 flagged as @READ or @ANSWER
<jml> s/know/now/
<bac> welcome back jml
<jml> bac, thanks.
<sinzui> mrevell, ping
<mrevell> hello sinzui
<sinzui> mrevell, you may be interested in this question: https://answers.edge.launchpad.net/launchpad/+question/125165
 * mrevell looks
<mrevell> Thanks sinzui. Yes, that's right up my street. I shall deal with it.
<sinzui> thanks
<mrevell> In that I'll answer it and also write a blog post inviting nominations
<matsubara> hey bigjools
<bigjools> matsubara: hey dude
<matsubara> bigjools, could you help me with a upload failure from a recipe? I created a recipe to create packages for bzr-pqm and put them in the ~launchpad PPA. It seems the source build step worked correctly but the upload failed
<bigjools> matsubara: build URL?
<matsubara> bigjools, this is the recipe URL: https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa/+build/2465
<matsubara> I'll paste the failure email I got, just a sec
<bigjools> matsubara: did you read the log? :)
<bigjools> all will become clear when you do
<matsubara> bigjools, yep. it says the versions already existed there but this was my first try AFAIK
<matsubara> bigjools, normally I'd have to bump the version, right?
<bigjools> yes
<bigjools> it's the source that already exists, it was probably uploaded as a regular package already
<matsubara> bigjools, and when I look here: https://edge.launchpad.net/~launchpad/+archive/ppa/+packages, looks like the package is there and published and everything seems to be alright
<bigjools> there you go then
<bigjools> matsubara: oh it's never been uploaded at all before?
<matsubara> bigjools, not with that version number. there was a older bzr-pqm package in the ~launchpad PPA  uploaded by Gary some time ago
<bigjools> yeah bzr72
<matsubara> bigjools, btw, when I apt-get update && apt-get upgrade I get the new package (bzr 73) that I uploaded.
<bigjools> matsubara: hmmm I'd check with one of the Code guys then, seems like it tried to upload itself twice
<matsubara> so I'm confused by the email saying it failed to upload
<matsubara> hmm
<matsubara> bigjools, don't know if it helps, but here is the failure email: https://pastebin.canonical.com/37090/
<bigjools> ok
<bigjools> matsubara: no it doesn't help :(
<matsubara> and the recipe page says (https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa/+build/2465) still says it failed to upload
<matsubara> hmmm
<matsubara> looking here: https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa I see there are two build records there
<bigjools> matsubara: so 2 builds got created, which explains the error
<bigjools> matsubara: I don't know what could cause that to happen
<bigjools> but abentley might
<matsubara> bigjools, maybe I set the daily build option and requested a build manually through the "request builds" link
<matsubara> not sure, but looks like PEBKAC heh
<bigjools> yeah that could be it
<bigjools> :)
<matsubara> thanks for your help!
<bigjools> matsubara: it's a bug that it allows 2 builds for the same version though
<matsubara> yep, I'll search/file a bug
<bigjools> that should be caught earlier so that it doesn't waste build farm time
<matsubara> bigjools, launchpad-code or soyuz?
<bigjools> code
<matsubara> okie. ta!
<cr3> hi folks, I'm mapping some launchpad concepts externally for referential integrity purposes, so that my web app can refer to launchpad projects for example. my question is: should I name that particular concept as a "product" to remain true to launchpad or "project" because that's how it's exposed externally?
<matsubara> bigjools, aha! https://bugs.edge.launchpad.net/launchpad-code/+bug/620248 the description and tim's comment describes exactly my situation
 * cr3 is leaning towards remaining as close as possible to the launchpad core for consistency purposes
<gmb> cr3, Project. The "product" moniker is archaic and should be taken out and shot, but it's got its tentacles into a lot of code.
<gmb> (That and "Project Groups" are called "projects" in code, which is just weird)
<cr3> gmb: that's the feeling I got from the launchpad code, but it doesn't seem like it's ever going to happen
<gmb> cr3, I think it's a JFDI-when-we-have-breathing-room thing.
<gmb> Because it has the potential to break tonnes of stuff.
<cr3> gmb: breathing what? not in my vocabulary :)
<bigjools> matsubara: yep!
<gmb> Maybe we'll do it in the week before Xmas, when we have nothing major scheduled, just lots of code cleanup.
<abentley> matsubara, bigjools,  it is expected behaviour that when you enable daily builds, it will always attempt a build, even if you have previously done a build.  One has no way of knowing whether the result will fail to upload without doing a build.
<bigjools> abentley: I have deja vu about this fact :)
<gmb> cr3, Breathing room... er... space to make the change and clean up all the breakages without making everything far to stressful.
<cr3> gmb: that's much sooner than I had expected, I'll code for the future then. thanks for the advice!
<gmb> cr3, np. But note: "Maybe" from me != "We will" from flacoste or thumper :)
<cr3> gmb: as long as the intention is there, at least I can justify my naming decision. I will document that my concept of project maps to launchpad's concept of product though
<matsubara> gmb, cr3: https://bugs.edge.launchpad.net/launchpad-code/+bug/109153
<abentley> matsubara, bigjools: upload failures usually come from debversion, and some substituted values (e.g. {time} will always be different.
<gmb> matsubara, Hurrah, saved me a job; thanks :)
<matsubara> abentley, right. my case is that I think I set the daily build option and requested a manual build at the same time
<matsubara> then I was confused by the failure email
<matsubara> while the first request was successfully built
<abentley> matsubara, I understand that.
<abentley> matsubara, if you had requested the same build twice, that would have been rejected.  But the daily build would have been scheduled after your manual build completed.
<matsubara> abentley, even if it completed successfully, hence bug 620248?
<abentley> matsubara, even if it completed successfully, because there's no way to know that it wouldn't complete successfully again without attempting a build.  See above.
<matsubara> right, makes sense
<rockstar> gmb, skype?
<gmb> rockstar, Sure. Let me get mah Skype rolling.
<rockstar> gmb, I don't have your skype id already here, and apparently your name is not original enough.
<gmb> rockstar, binnsgm. :)
<rockstar> gmb, you'll want to bzr pull.
<gmb> rockstar, Okay, doing so now ,thanks.
<rockstar> gmb, you should have revno 187 (a bad omen)
<gmb> rockstar, yep :)
<jml> sinzui, we have a call now.
<jml> sinzui, but I need a cup of tea first.
<sinzui> jml, I am ready on mumble
<jml> sinzui, me too!
<bigjools> sinzui: I'm not sure, but I think this bug is registry?  https://bugs.edge.launchpad.net/launchpad/+bug/636151
<sinzui> bigjools, yeah. That has no home. give it to registry. I think most of the objects are registry related.
<bigjools> that's what I figured, thanks
<bigjools> that question about featured projects probably belongs there too
<bigjools> mrevell: did I see earlier that you were dealing with this? https://answers.edge.launchpad.net/launchpad/+question/125165
<mrevell> bigjools, Yeah, I'll be on it.
<bigjools> mrevell: ok mind if I assign to you?
<mrevell> go for it
<bigjools> ahhh questions needs some ajax love
<sinzui> jml, UbuntuBeta, Ubuntu, 'Bitstream Vera Sans', 'DejaVu Sans', Tahoma, sans-serif;
<gmb> bigjools, Questions needs some love from the Loving Hammer of Rectification, never mind AJAX.
<gmb> Oh, no, wait, I'm thinking of blueprint.
<james_w> some help with diagnosing bug 637323 would be welcome, as it is causing problems for the apport retracers in particular
<EdwinGrubbs> Is there something I need to do besides put /++oops++ in the url to trigger an oops on launchpad.dev?
 * bigjools EoDs, good evening all
<mrevell> night all
 * rockstar lunches
<jelmer> rockstar: hi
<lifeless> moin
<lifeless> james_w: so
<lifeless> james_w: bug:
<lifeless>  - appserver revno differences cause PATCH error
<lifeless>  - launchpadlib in stable Ubuntu points to edge
<james_w> that's intended design in the current code
<james_w> well, not exactly, but the revno changing the etag is
<lifeless> so, we have to fix that.
<lifeless> what else
<lifeless> (if we don't fix it we cannot have a smooth upgrade of the server farm). Its a designed in error.
<lifeless> oh, and the appserver 'name' should be present in responses so we can debug things remotely.
<bdmurray> I've running into an error when running make run - "cannot import name SAFE_INSTRUCTIONS" from bzrlib.plugins.builder.recipe
<lifeless> update your sourcecode
<lifeless> utitilies/update-sourcecode
<bdmurray> Updating dulwich to revision 424 (No change)2kB/s | Fetching revisions:Finishing stream
<bdmurray> Exception AttributeError: "'NoneType' object has no attribute 'close'" in <bound method SmartSSHClientMedium.__del__ of SmartSSHClientMedium(bzr+ssh://brian-murray@bazaar.launchpad.net/)> ignored
<lifeless> and cd to the download-cache and run bzr update
<bdmurray> it is up to date at revision 183
<lifeless> james_w: can we move the retracer to prod please?
<james_w> lifeless: that's not what I was playing with, and I have no control over it
<lifeless> ah ok
 * maxb raises two eyebrows at finding an executing code import for a deactivated project
<maxb> (lp:libs)
<jelmer> maxb: There is more than one..
<maxb> yes, my point is there shouldn't be :-)
<jml> it's not that surprising when you think about it.
<maxb> LP does at least does not forbid visibility of these
<bdmurray> lifeless: did you have anymore ideas?
<lifeless> sorry otp
<jml> why do we need to compile templates to run tests?
<jml> lifeless, are we having our scheduled call?
<cr3> jml: welcome back! fyi, I've spent the last week or so implementing the results tracker: https://launchpad.net/launchpad-results
<jml> cr3, thanks. I'm glad to hear it.
<cr3> jml: following discussions with lifeless, we decided to impelement this new feature as a standalone web app which could eventually be folded into the core once it becomes stable
<jml> a sound plan.
<cr3> jml: that way, we can reach stability more quickly than if I implemented directly in the core, and we can make mistakes much more easily too
 * cr3 usually gets things right on the third try on average :(
<lifeless> jml: yes
<cr3> jml: I already have a pretty neat web app pushed into a branch which integrates django, zope, storm and lazr. seems to work well so far
<lifeless> jml: ran slightly over with statik
<jml> g'night all
<mwhudson> jml: g'night
<lifeless> night
<lifeless> sinzui: hi
<lifeless> sinzui: I would like to learn aboot batching!
<sinzui> hi lifeless
<lifeless> sinzui: is now a good time for you?
<sinzui> it is
<lifeless> great
<lifeless> so I was thinking mailing list moderation pages would be a good one for me to learn on
<sinzui> Good choice
 * sinzui looks at browser code
<wallyworld_> morning
<sinzui> lifeless I think we want to start by making held_messages a cachedproperty because we may reference it 3 times
<lifeless> sinzui: ok. I'm just pushing current devel up to lp:~lifeless/launchpad/registry which is where I'll put this work
<sinzui> okay
<lifeless> so, I can do the cachedproperty change easily; can you tell me a little about how batching hangs together while I do that?
<sinzui> lifeless: lets talk about this: http://pastebin.ubuntu.com/493322/
<lifeless> sinzui: so held_messages won't benefit much by being a cachedproperty. we can come back to that
<lifeless> ah, interesting
<lifeless> so we wrap the result set
<sinzui> lifeless, the BN holds the record set and does not realise it until we iterate over it. BEware of indexing it though. I recently discovered that that does not realise the data, we just call the DB 75 times
<sinzui> lifeless our BN sets the default batch size to what we have in config.launchapd.default_batch_size (75?) you can pass size=20 if you believe we want a different size
<lifeless> to the constructor ?
<sinzui> lifeless correct. Subclasses of the BN may set different size rules or singular/plural headings so that we do not need to set the info in __init__
<lifeless> http://paste.ubuntu.com/493324/
<lifeless> thats obviously mostly your code
<sinzui> lifeless, the request is very important. The BN will look for start and batch (size) params to control the slice. We will raise an BatchSizeError (?) if the user tries to exceed our hard limit
<sinzui> ouch we were calling getReviewableMessages() twice for every page load?
<lifeless> yes
<lifeless> which is old school sqlobject in fact
<lifeless> note that creating a resultset is pretty cheap in general because they are lazy: they don't do sql immediately
<lifeless> brb guy at door
<sinzui> lifeless: We often have navigation links before and after the batch we are iterating over: This is template code: http://pastebin.ubuntu.com/493327/
 * sinzui looks for real template
<sinzui> ah, that is nice, we are adapting the message in the template. This might be an easy addition
<rockstar> mwhudson, is this better?
<mwhudson> rockstar: i think so
<mwhudson> rockstar: obviously there's no sql that can find the affected recipes
<bdmurray> sinzui: did you have an idea about how to fix bug 636412?
<rockstar> mwhudson, on the contrary.
<mwhudson> rockstar: tbh, i'm tempted to suggest a bzr-builder change
<sinzui> lifeless: in team-mailing-list-moderate.pt, I think we only require the navigation headers because the table is doing the right thing. This might be the final code for the template: http://pastebin.ubuntu.com/493328/
<rockstar> http://pastebin.ubuntu.com/493314/
<rockstar> mwhudson, ^^
<mwhudson> rockstar: how do you know that there are no recipes with the same problem?
<rockstar> mwhudson, a better query would be to find out where "." is the value of sourcepackagerecipedatainstruction.directory
<mwhudson> yeah, that would work here i guess
<rockstar> mwhudson, what would you bzr-builder change be?
<mwhudson> rockstar: i'm tempted to say that bzr-builder should change so that it can return an invalid recipe from stringification, but tell you about it
<mwhudson> so you can display a warning on the recipe's page
<lifeless> sinzui: sorry for the interrupt
<sinzui> bdmurray, since I cannot see how the user got to that page, I think we should consider that the view should handle the case of URL hacking...
<bdmurray> sinzui: right, I haven't seen something like that before
<rockstar> mwhudson, that would probably work longterm, yes.
<rockstar> mwhudson, for the short term, to restore access to the recipe, does the SQL route sound fine?
<mwhudson> rockstar: this would mean changing bzr-builder to not use __str__ to stringify recipes i guess, something i always hate a bit :)
<mwhudson> rockstar: yeah
<sinzui> bdmurray, the template could guard the form with a new view attr. Maybe view/can_subscribe. That view property does something like:
<sinzui> return self.context is not getUtility(ILaunchpadCelebrities).ubuntu
<mwhudson> rockstar: is it clear what the intent of the recipe is?
<rockstar> mwhudson, maybe we could just modify __str__ to be a bit smarter?
<mwhudson> that'd be possible too
<rockstar> mwhudson, the recipe has two lines: the base branch and the nest command
<sinzui> bdmurray, I suppose the form should also have a sentence for the ubuntu to explain we do not want the user to shoot himself in the foot
<lifeless> sinzui: that seems very easy
<lifeless> sinzui: whats the key thing in the tal:repeat that you looked for to make sure it wrks ?
<sinzui> lifeless yes indeed
<sinzui> That widget hack is rather convenient I think
<lifeless> sinzui: I don't undertand whats going on there
<lifeless> sinzui: what tests are normally added in a case like this?
<sinzui> lifeless: I like to see a test to verify the view is returning a BN. Since the headers are set, we should test the bn.default_singular_heading and default_plural_heading
<sinzui> lifeless: The template will have upper and lower batch ids to confirm that both navs are rendered. We can test by calling render() on the view or with a TestBrowser
 * sinzui looks for existing tests
<lifeless> sinzui: can you expand on what the 'widget hack' does?
<sinzui> lifeless the adapted message is converted to a HTML fragment in lp/registry/templates/message-moderation.pt
<lifeless> sinzui: is that the message/@@+moderation bit ?
<sinzui> lifeless yes, but I now think there is one more change needed to the table...
<sinzui> lifeless: I think we need to change the line before @@_moderation to
<sinzui> tal:repeat="message view/held_messages/currentBatch"
 * sinzui is still looking for existing tests for held_messages
<sinzui> lifeless: I do not see any tests for the view that is generating the form. We really want one and I really think we have once because I recall reading a test to explain how to put messages into the expected state
<lifeless> I'm adding one to test_team
<lifeless> can consolidate later
<lifeless> browser/tests/test_team, I mean
<sinzui> lifeless look at stories/mailing-lists/moderation.txt which puts messages into the state
<lifeless> thanks
<lifeless> most of these tests don't need that, just the batch ids do, right?
<sinzui> lifeless, yes. and we could check the batch ids in the moderation test since we have the template rendered. Lifeless I am a bit pedantic about verifying the batch ids because I discovered we had two upper navs before, or that one never rendered at the bottom of a very long list
 * sinzui also added the singular/plural message support
<lifeless> ok
<lifeless> looking in moderation.txt for the batch ids
<lifeless> Would this be a good point to check ?
<lifeless> Foo Bar sees that there is one message waiting for approval.
<sinzui> yes
<lifeless> is there an example of checking this elsewhere?
<sinzui> me thinks
<thumper> sinzui: are you free for a call?
<lifeless> thumper: sinzui is training me just now, should be finished soon.
<thumper> lifeless: training you to do what?
<lifeless> batch things
<lifeless> just bringing all the bits together quickly
<lifeless> I couldn't find a clear guide to batching-pages when I asked around yesterday.
<sinzui> lifeless print find_tag_by_id(admin_browser.content, 'batchnav_first')['class'] should print first
<sinzui> that does not look right, the upper class is missing from this template
<lifeless> will it error if someone has doubled things up or something?
<sinzui> yes
<sinzui> lifeless, I think my days of checking upper and lower batches are over. The markup is rendered with html classes now: upper-batch-nav and lower-batch-nav
<sinzui> lifeless, and the nav is only rendered if there is a batch! we need to exceed 5 messages (5 is the default batch size in test)
<mwhudson> jelmer: around? http://launchpadlibrarian.net/55539932/vcs-imports-r-project-trunk.log is very odd
<sinzui> I have just confirmed that the top and bottom navigators are rendering duplicate ids for the links :(
<sinzui> Well That is why i wanted a test ;)
<jelmer> mwhudson: It's happened on at least one other cscvs import as well
<mwhudson> jelmer: but not all?
<mwhudson> jelmer: could it be per-importd perhaps?
<jelmer> mwhudson, it doesn't appear to occur that often for that
<lifeless> sinzui:     self.assertEqual('message', navigator.default_singular_heading)
<lifeless> AssertionError: not equal:
<lifeless> a = 'message'
<lifeless> b = 'result'
<jelmer> mwhudson: actually, that reminds me
<lifeless> current diff
<lifeless> http://paste.ubuntu.com/493338/
 * sinzui rechecks code
<mwhudson> jelmer: well it will only happen when there is a new rev to import i guess?
<sinzui> lifeless ._singular_heading and ._plural_heading.
<jelmer> mwhudson: that's true I guess, but we've only seen 4 or so failures since the rollout. Are there really that few working and active CSCVS imports left?
<sinzui> lifeless ^ did you see my discovery that we are generating duplicate ids for links in the upper and lower nav?
<mwhudson> jelmer: don't know
<lifeless> sinzui: hah!
<lifeless> sinzui: fun.
<mwhudson> jelmer: but if it was just pear, it would have to run on pear 5 times in a row
<mwhudson> jelmer: that's fairly unusual
<jelmer> mwhudson: ah, I hadn't considered that. it seems likely that's the cause then
<mwhudson> losa ping time i guess
<sinzui> The fix looks trivial...concatenate the view class for the nav to the like id in the template. The view/css_class will make them distinct by upper and lower.
 * sinzui reports bug to explain
<mbarnett> mwhudson: heya.
<mwhudson> mbarnett: can you log into pear as importd and run "bzr whoami" pls?
<mbarnett> importd@pear:~$ bzr whoami
<mbarnett> Import Daemon <importd@pear>
<mbarnett> importd@pear:~$
<mwhudson> mbarnett: bzr --version?
<mbarnett> http://pastebin.ubuntu.com/493339/
<lifeless> BatchNavigator.count() is wrong
<lifeless> need to find the right way to do that
<jelmer> mwhudson: Perhaps not pear but one of the other ones?
<mwhudson> mbarnett: can you try python /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami ?
<mwhudson> mbarnett: pls modify path until it exists :)
<mwhudson> jelmer: https://code.edge.launchpad.net/~vcs-imports/r-project/trunk <- looks like pear is the bad apple
<jelmer> mwhudson: Ah, right
<jelmer> mwhudson: also, I hadn't considered the bzr path
<mwhudson> me neither until i saw that :)
 * thumper upgrades to maverick
<maxb> jelmer: I think I've figured out what's up with those slow KDE imports.... the bzr-svn logwalker doesn't incrementally commit the cachedb
<mbarnett> mwhudson: it is fighting with me: http://pastebin.ubuntu.com/493348/
<mbarnett> chmoding
<mwhudson> mbarnett: that's why i put python first :)
<lifeless> sinzui: so should we not test that here, for now ?
<mwhudson> sorry for not using quotes, that would have been easier
<mbarnett> mwhudson: ah, i missed the python
<mwhudson> mbarnett: "/srv/importd.launchpad.net/production/launchpad/bin/py /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami" would be even more correct i guess
<mbarnett> importd@pear:~$ /srv/importd.launchpad.net/production/launchpad/bin/py /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami
<mbarnett> bzr: ERROR: Unable to determine your name.
<mbarnett> Please, set your name with the 'whoami' command.
<mbarnett> E.g. bzr whoami "Your Name <name@example.com>"
<lifeless> hmm, getting a LocationError from hold_count now
 * lifeless puts it back as 'future'
<sinzui> lifeless the fix is pretty easy, and I am sure there is no test for th default BN since I can see any attempt will fail.
<sinzui> .me makes quick change
<mwhudson> mbarnett: ok, that's good (ish, it confirms my expectation)
<mwhudson> mbarnett: can you try the same on the other importds?
<mbarnett> heh
<lifeless> sinzui: yeah the test you gave blows up with NoneType in unsibscritable
<lifeless> there are also some later failures
<lifeless> like the Moderate button is missing?
<mbarnett> mwhudson: same
#launchpad-dev 2010-09-14
<mwhudson> mbarnett: ok, that's not what i expected
<mwhudson> mbarnett: which machines did you try it on?
<lifeless> sinzui: browser/team.py has an intersting thing
<lifeless> it looks up the action from the form
<mbarnett> mwhudson: galapagos, pear, russkaya
<lifeless> but it doesn't seem to check that the action is *for* that row
<mwhudson> !?
<mwhudson> mbarnett: sorry if this is tedious, but can you pastebin the ~importd/.bazaar/bazaar.conf files from pear and russkaya?
<spm> mwhudson: I'll sort that mbarnett needs to go and en-cake-enate
<sinzui> Lifeless yeah. I am staring at the template. I think the message is adapted to a widget and it builds ids from the message id...we automatically discard duplicate message ids
<maxb> jelmer: Did the bzr-svn on launchpad somehow just not ever try "discovering revprop revisions" before the last rollout?
<jelmer> maxb: it's always done that
<maxb> huh
<mbarnett> yes yes, cake levels dangerously low!
<jelmer> maxb: But we didn't do KDE imports until recently
<maxb> Yes, but the KDE imports which ran before the rollout didn't "discover revprop revisions"
<maxb> oh, wait, I'm getting my dates wrong.
<lifeless> sinzui: anyhow all tests are passing, except for
<lifeless> >>> find_tag_by_id(admin_browser.contents, 'batchnav_first')['class']
<sinzui> Yes that will be an issue for a few more moments. I have a patch
<lifeless> sinzui: and I don't think the bug with POST there is any better or worse due to batching; if its buggy its always been buggy.
<sinzui> I agree
<mwhudson> spm: cool, and good morning
<sinzui> lifeless: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/637654 has a proposed patch. It fixes the most common case of upper and lower. I image a page with two sets of BNs will continue to be broken
<spm> mwhudson: hmm. you may have hit the nail. pear doesn't have that file
<lifeless> sinzui: is this likely to cause other tests to fail ?
<spm> mwhudson: https://pastebin.canonical.com/37119/ <== russkaya
<lifeless> (I mean, will there be other fixups to do with that patch)
<lifeless> sinzui: could you commit that patch somewhere and push it? I'll pull it in
<sinzui> I bet not since there are no tests reporting that we have rubbish ids. I reported it as a separate bug because I think this issue is a separate concern from your branch now
 * sinzui does
<lifeless> sinzui: it is a seperate concern, but either I leave the test out or I include your branch
<mwhudson> spm: gar
<spm> mwhudson: I assume C&P from one t'other?
<mwhudson> spm: does /home/importd/.bazaar/sign-vcs-import exist on pear?
<spm> nope
<mwhudson> did pear get reinstalled at some point?
<mwhudson> spm: can you look at  /home/importd/.bazaar/sign-vcs-import on russkaya?
<spm> mwhudson: I'd assume it being a new machine; some things may have been missed :-(
<mwhudson> it probably references something ridiculous like ~importd/hoover/keys/key.gpg
<spm> holy dooly
<mwhudson> https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/SetUpCodeImportSlave -> "# Make sure ~importd/.bazaar/ and ~importd/botslave look like they do on a working slave. "
<mwhudson> :(
<spm> mwhudson: exec /usr/bin/gpg --no-default-keyring --keyring /home/importd/botslave/gpg/vcs-imports@canonical.com.pub --secret-keyring /home/importd/botslave/gpg/vcs-imports@canonical.com.secret --default-key A60FA0E1 "$@"
<mwhudson> spm: i was sure this was working at some point on pear :(
<mwhudson> maybe not
<mwhudson> it only affects cscvs imports i guess....
<spm> I'm surprised it's working on russkaya....
<sinzui> lifeless: lp:~sinzui/launchpad/batch-ids-0 There are no tests for these links. Nor can I find any tests for the existing of the upper or lower navs. All the tests for the BN are getting the Next link using TestBrowser
<lifeless> oulling
<sinzui> I checked for uses the the default BN. Subclasses like root search and bug do their own layouts
<lifeless> sinzui: so 'batchnav_first' is not what to look for
<sinzui> lifeless try 'upper-batch-nav-batchnav-first' but keep in mind the nav will not rendered if there is only one batch. We need 6 messages in the testrunner env or we add ?batch=1 to the url we are tesing to set the size to 1 message
<lifeless> and for the bottom lower-batch-nav-batchnav-first ?
<sinzui> yep
<lifeless> ?batch=1 doesn't do it
<lifeless> http://paste.ubuntu.com/493359/
<lifeless> hmm
<lifeless> 1 is too small
<lifeless> moving it lower down
<lifeless> sinzui: doesn't appear to be rendering the nav links to me
<lifeless> Continue to hold the message, deferring\n          your decision until later.</li>\n        </ul>\n      </div>\n\n        <table class="listing">\n
<sinzui> lifeless sorry, my screen was locked for a moment
<lifeless> sinzui: ^ thats with batch=1 and 2 messages
<wgrant> So, we have an issue with the OpenID identifier migration last week, causing incorrect accounts to be linked together... can someone poke around on staging to work out WTF is going on?
<lifeless>  2\n        \n        <span>\n        messages have</span>\n        been posted to
<lifeless> wgrant: sure, once I'm finished here.
<wgrant> lifeless: Thanks.
<lifeless> sinzui: only one message is shown
<lifeless> so the batch param worked
<lifeless> its the naviation bit that isn't
<sinzui> I am a bad advisor. you are on the first batch. We should be checking for upper--batch-nav-batchnav-next
<lifeless> no, you're fine
<lifeless> its no there
<sinzui> :(
<lifeless> after the advice
<lifeless> Discard</strong> - Throw the message aw
<lifeless> etc
<lifeless> your decision until later.</li>\n        </ul>\n      </div>\n\n        <table class="listing">\n        <thead><tr>\n          <th>Message detail
<lifeless> is whats in the browser.contents
<lifeless> the navigation bit is just awol
<lifeless> it should be after that </div>
<sinzui> We suppress rendering of the lower if there is no additional batches, so maybe that template fragment is wrong
<lifeless> well there is an additional batch - batch=1 and two messages to moderate
<lifeless> the count on the page shows '2' so we know there are two there
<lifeless> and there is only one "approve" in the output, so we know only one got shown
<sinzui> lifeless, the upper template must rendered since there is clearly a batch. the view guards the rendering with this: ``if self.context.currentBatch():``
 * sinzui is looking at canonical/launchpad/webapp/batching.py
<lifeless> what is the context object going to be - held_messages ?
<spm> mwhudson: pear now has those dirs/files setup per russkaya
<mwhudson> spm: great, thanks
<sinzui> lifeless: yes held_messages. we are adapting a BN
<lifeless> so I did this
<lifeless> +++ lib/canonical/launchpad/webapp/batching.py  2010-09-13 23:50:20 +0000
<lifeless> @@ -40,7 +40,7 @@
<lifeless>      def render(self):
<lifeless>          if self.context.currentBatch():
<lifeless>              return LaunchpadView.render(self)
<lifeless> -        return u""
<lifeless> +        return u"not rendered"
<lifeless> not rendered was not included in the output
<sinzui> ?
 * sinzui checks zcml 
<lifeless> I wanted to see if that code path was shortcircuiting or something
 * sinzui checks other batches with the hacked template
<lifeless> and this:
<lifeless> +++ lib/canonical/launchpad/webapp/batching.py  2010-09-13 23:52:07 +0000
<lifeless> @@ -38,6 +38,7 @@
<lifeless>      css_class = "upper-batch-nav"
<lifeless>  
<lifeless>      def render(self):
<lifeless> +        return u" fooo "
<lifeless>          if self.context.currentBatch():
<lifeless>              return LaunchpadView.render(self)
<lifeless>          return u""
<lifeless> also doesn't show up in the output
<sinzui> right. I am not seeing my template change when testing https://blueprints.launchpad.dev/firefox?batch=2
<sinzui> Or I could run the instance that I made the change in instead
<lifeless> and to cap it off,when I make that raise an Exception I don't get an error
<sinzui> oh.. I wonder. I see >>> admin_browser.reload() which has a history if being buggy
<lifeless> have a look at lib/lp/blueprints/templates/person-specworkload.pt
<lifeless> sinzui: I'm sure its not that, I made the view crash and the page rendered
<sinzui> I see my hack in specs now that I am running the right branch
<lifeless> I'm thoroughly confused
<lifeless> is there a sample data team w/list ?
<sinzui> lifeless me too, this always just works. Can you humour me by adding this link before we do the call the find_tag_by_id
<sinzui>     >>> admin_browser.open(
<sinzui>     ...     'http://launchpad.dev/~guadamen/+mailinglist-moderate')
<lifeless> of course
<sinzui> lifeless there are no mls in data. I have a make harness note about making them after a request in made in the UI
<lifeless>     >>> admin_browser.open(
<lifeless>     ...     'http://launchpad.dev/~guadamen/+mailinglist-moderate?batch=1')
<lifeless>     >>> find_tag_by_id(admin_browser.contents, 'upper-batch-nav-batchnav-first')['class']
<lifeless>     first
<lifeless>     >>> admin_browser.contents
<lifeless> thats what the story does
<sinzui> :(
<lifeless> whats weirded
<lifeless> I added a string literal and I can't see it
<lifeless> its almost like those divs are eaten
<lifeless> when I put a literal above it works
<lifeless> in the list of action descriptions
<lifeless> but when I add another div at the place we have the navigation ones it disappears
<sinzui> lifeless as a desperate act to to verify this we could add size=1 to the BN instantiation in the view to be certain that the URL is not being ignore
<sinzui> d
<lifeless> I'm certain its not
<lifeless> because only one "approve" action is in the contents
<sinzui> Ah, yes, that is what I did to be certain something showed up in my env
<lifeless> I suspect the metal:form stuf
<lifeless> I'm positive its simply not evaluation things without metal:fill-slot in that container
<lifeless> I think if we add a div around it it wil work, moving the widgets slot up
<sinzui> we are adapting the message in the same manner that we want to adapt the BN
<lifeless> yes, but the metal interpreter isn't evaluating things without slots
<sinzui> We can certainly move the navs out of the form to be sire it works
<lifeless> bet you that that is is
<lifeless> is it
<lifeless> yes
<lifeless> it was
<lifeless> I have this now
<sinzui> \o/
<lifeless> <a class="next" rel="next"\n           href="http://launchpad.dev/%7Eguadamen/+mailinglist-moderate?start=1&amp;batch=1"\n           id="upper-batch-nav-batchnav-next">
<lifeless> -      <table class="listing" metal:fill-slot="widgets">
<lifeless> +      <div metal:fill-slot="widgets">
<lifeless> +      <tal:navigation
<lifeless> +        replace="structure view/held_messages/@@+navigation-links-upper" />
<lifeless> +
<lifeless> +      <table class="listing">
<lifeless> thats the key
<sinzui> 1.5h of confusion and 1 minute to fix with insight.
<lifeless> we must be programming
<lifeless> Thank you for this; I'll push up and propose for merge
<sinzui> Thanks
<lifeless> and I'll write a mail to the list with a) a howto and b) asking for where it should go
<lifeless> does anyone remember the wiki page for the bug sprinty thing at the end of the year?
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance Tuesday | Week 1 of 10.10 | PQM is open for business | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<lifeless> sinzui: https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/35354
<wgrant> lifeless: So, I worked out what was up with the broken accounts.
<wgrant> Sadly more will likely break soon.
<lifeless> wgrant: ok cool
<lifeless> I learnt how to batch stuff
<lifeless> and to hate metal:form
<wgrant> Heh.
<wgrant> Who owns our OpenID consumer these days?
<lifeless> consumer? foundations
<lifeless> lnchtime
<lifeless> thumper: does transaction time == scan time ?
<thumper> lifeless: luckily, no
<thumper> lifeless: 5.5 minutes to get the ancestry from bzrlib :(
<lifeless> wheee
<lifeless> I think your idea of decoupling the tip change may not be enough
<lifeless> I'd start with autocommit
<lifeless> IMBW
<lifeless> but it seems like low effort for big return
<lifeless> thumper: can I borrow your eyeballs
<thumper> lifeless: no, they're mine
<thumper> lifeless: what do you need?
 * mwhudson is reminded of the end of hotshots
<lifeless> thumper: a review
<lifeless> its small, it will fix mailing list moderation (or make it fixable by further tuning)
<MTecknology> Just on the wild off chance... Is there anyone that knows very basic accounting principles in here?   I know there are very smart people in here and hoping one might be able to help me out..
<lifeless> MTecknology: I do, enough to say 'run run away'
<lifeless> spm: hey, don't suppose in the losa wiki you have a sql fragment to report on locks in the db ?
<lifeless> spm: I know I wrote one up years ago ...
<spm> yup sure do
<lifeless> thumper needs its.
<spm> it's a tad obscure to find tho. lp howto, troubleshooting from ememory
<MTecknology> lifeless: How about enough to help me figure out this problem that's driving me absolutely bonkers? I have the book - but the book doesn't cover the material.
<thumper> lifeless: did you mute?
<spm> thumper: https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/BlockedProcessesDBLocks as a general
<spm> https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/PostgresOldQueries is also vaguely relevant
<spm> MTecknology: google isn't helping find it? if they're basic accuonting principles, there should be heap of online references that explain them??
<MTecknology> spm: My issue is understanding the basics of what I'm even reading online
<spm> MTecknology: being quite serious here (I've got a few ni teh series): Perhaps "Accounting for Dummies"? serious suggestion, the dummies series are excellent for explaining the basic concepts. ??
<MTecknology> spm: might be worth buying from ya - any chance you could try to help me in a query with this one?
<MTecknology> or else there's a barnes & nobel here if you weren't offering to sell
<spm> MTecknology: accounting? hell no. I never studied it at school or uni. wouldn't have a clue. I just have a few Dummies books that I've found excellent for explaining early concepts in the topics in question. :-)
<MTecknology> oh
<MTecknology> BTW - This is what I'm fighting.  Pearson Brothers recently reported an EBITDA of $13.5 million and net income of $2.6 million. It had $2.0 million of interest expense, and its corporate tax rate was 35%. What was its charge for depreciation and amortization?
<spm> http://www.amazon.com/Accounting-Dummies-John-Tracy/dp/0764550144 fwiw
<MTecknology> cheap, and probably much more useful than this $150 unbound see through sheets of paper book I have
<spm> probably :-)
<cr3> lifeless: hi there, sorry I couldn't answer you earlier. still around?
<lifeless> yes
<cr3> lifeless: so, regarding test runs, do you also feel that's the best way to describe a group of test results run at a point in time in a given context?
<cr3> lifeless: typically, I prefer to name things with one word, like submission instead of test run, but I think the latter might be clearer
<lifeless> I commentted on thta in #testrepository
<lifeless> sorry otp now
<cr3> lifeless: heh, you seem to have been on the phone all day :)
<cr3> on an unrelated topic, I have a question about defining interfaces: if a class implements IBugTarget which inherits from IHasBugs, using bugs as an example, then that class typically defines a createBug method.
<cr3> however, why not have the class have a bugs attribute which returns a IBugSet which, in turn, implements a create method
<cr3> in other words, the difference is like product.createBug compared to product.bugs.create, does this make sense to anyone?
<lifeless> thumper: https://dev.launchpad.net/LEP/FeatureFlags#preview
<lifeless> thumper: if features.getFeature('code.incrementaldif') == 'on':
<lifeless> in templates, you do view/features/code.incrementaldiff
<lifeless> or something like that
<lifeless> spm: how many cpus on the master db?
<spm> lifeless: 16
<lifeless> cr3: hi
<lifeless> uhm
<lifeless> in reverse order, I don't know, IBugSet really isn't the specific code I'd use if sketching it that way, and don't forget that all calls to SQL are ~ 1000 times slower than python.
<lifeless> I odn't have a brilliant name for the result of running many tests other than 'test run'
<wgrant> lifeless: Any idea how to debug the +filebug issue?
<lifeless> hmm
<lifeless> spm: we really do need a hand
<lifeless> spm: when you can, its approx the top timeout on prod
<spm> lifeless: sure, was on a call, earlier hence the terse reply. sup?
<lifeless> spm: +filebug gives an apache/haproxy error, reliably, on staging and prod
<lifeless> wgrant has been looking at it
<lifeless> we need to know a bit more about whats actually going on.
<wgrant> Bug 636801
<spm> um. since when? I happily filed a bug earlier?
<lifeless> or for someone to make the request to a naked appserver
<wgrant> I guess we need someone to watch staging Apache and see why it errors.
<lifeless> or something
<wgrant> spm: Only when filing with lots of apport attachments.
<lifeless> spm: with apport on a package with 20+ subscribers?
<spm> no, just a soyuz one. qed. ;-)
 * wgrant kicks mup.
<lifeless> mup has mastered the fine art of silence.
<spm> mup appears to have left the channel
<wgrant> spm: WRT that Soyuz one, it seems to be a general problem. I've received complaints that lots of builds are dispatching repeatedly.
<lifeless> spm: _mup_
<cr3> lifeless: the interface question was mostly related to something containing other objects. put another way, I could have projects['bzr'].create_test_run() or projects['bzr'].test_runs.create()
<spm> ahh. it hides under a new name.
<lifeless> cr3: is this python or LP API's ?
<lifeless> cr3: if its LP API's you probably want to design to the wire protocol, given how round-trip-happy it is.
<cr3> lifeless: ok, so every dot is a roundtrip potentially
<wgrant> Not just potentially :(
<lifeless> if by potentially you mean 'almost guaranteed'
<lifeless> and by dot you mean 'python method invocation'
<lifeless> (which includes __getattr__ aka '.')
<cr3> I thought that perhaps launchpadlib could potentially cache information on the client side, sometimes avoiding a roundtrip
<lifeless> cr3: optimise for cold cache :)
<lifeless> (it can, under very limited circumstances)
<lifeless> which I suspect we'll be limiting to about 2.5 hours in the near future
<cr3> ok, that answers my question and provides good guidelines for the future. thanks!
<cr3> lesson learned, now time for bed. cheerio folks!
<wgrant> Bah, no staging.
<spm> lifeless: so, been doing some log snarfing and head scratching. not finding any errors in apache - but if a POST, and timing out; tbh I wouldn't expect to. :-( If this can be reliably repeated, I'd suggest 'a' way forward, would be to sniff the traffic at the client end when doing such a thing. even tho the connection'd be ssl'd, I'd betcha we'd get useful info out of the flow.
<lifeless> stub: https://code.edge.launchpad.net/~stub/launchpad/cronscripts/+merge/35279 reviewedish
<spm> wgrant: ref soyuz; yeah, I'm sure I'd seen comments around this bug before; but didn't have enough "knowledge" to find 'em. So figured a new with some detailed timeing info may help Julian. Being a private build I had to be a tad circumspect in what I put in unf. :-/
<lifeless> spm: we don't see the response on the client,thats the point.
<lifeless> spm: client -> server, pause, 'could not connect to launchpad'
<spm> lifeless: the tcp conenction stays open forever until it gets client killed?
<lifeless> spm: so we don't get an oops, don't get zip
<lifeless> spm: no we get the haproxy/apache lalala page
<spm> after what time period, repeatedly?
<lifeless> wgrant: please tell spm how to make it happen, then he'll see
<spm> same time period? longer? shortly? varies by moon phase and tides?
<lifeless> 10seconds sometimes apparently, though I think that was during the overload
<lifeless> 30 normallyish, I thinks.
<spm> different browsers to make a diff?
<lifeless> spm: don't think we've tested, because the browser is working fine.
<spm> just wondering if it's an internal browser timeout that's then kicking the server error
<lifeless> I don't even know how to parse that
<spm> ie. are packets actually flowing and then dying.
<spm> or no packets flowing at all
<lifeless> spm: its http - request/response model
<lifeless> spm: and apport does preuploading of the bugs, so its not a big post.
<spm> for sure; I'm looking at the tcp layer to get clues for wtf is happening at the http layer.
<lifeless> wgrant: whats a package that this has happened to ?
<lifeless> spm: I don't think its an http problem myself, I think its appserver lalalalala land time genuinely, but we don't see the oops clearly
<spm> fwiw, it should be pretty simple in staging: intranettertubers -> apache -> appserver. no squid, no haproxy.
<lifeless>  ok
<wgrant> spm, lifeless: I was trying to prepare a case, but staging is borked.
<lifeless> so we're seeing the apache 'server fail' message
<spm> actually there's a point. wonder if the oops are being generated; we're just not seeing 'em. looks...
<lifeless> OOPS-1717E1745, OOPS-1717G1716, OOPS-1717H1810, OOPS-1717K1882, OOPS-1717L1760
<lifeless> OOPS-1717E1218, OOPS-1717E1837, OOPS-1717K1949, OOPS-1717M1234, OOPS-1717N1211
<lifeless> OOPS-1717D703, OOPS-1717G778, OOPS-1717K884, OOPS-1717K885
<lifeless> they are listed as soft timeouts on +filebug
<lifeless> we also have some 'OffsiteFormPostError'
<lifeless> OOPS-1717M14
<spm> process-apport-blobs.log is remarkably unhelpful
<wgrant> process-apport-blobs is fine.
<lifeless> that happens async
<lifeless> its all in the appserver at this point
 * spm is doing the Sherlock Holmes method of debug - eliminate the working, to discover the not ;-)
<lifeless> :)
<wgrant> Can we expect staging to return at some point?
<wgrant> It's been down a lot lately...
<lifeless> wgrant: theres about 6 queries per attachment
<wgrant> lifeless: Really?
<spm> launchpad-trace.log has zip with 'filebug' in it. orsum
<lifeless> INSERT INTO BugAttachment (message, bug, libraryfile, type, title) VALUES (%s, %s, %s, %s, %s) RETURNING BugAttachment.id
<wgrant> Message, BugAttachment, BugNotification, FUCKLOADS * BugNotificationRecipient
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname
<lifeless> SELECT BugTask.assignee, BugTask.bug, BugTask.bugwatch, BugTask.date_assigned, BugTask.date_closed, BugTask.date_confirmed, BugTask.date_fix_committed, BugTask.date_fix_released
<spm> wgrant: for some reason, staging is being updated 'continuously' regardless of need. haven't had a chance to chase. yet.
<lifeless> SELECT StructuralSubscription.blueprint_notification_level, StructuralSubscription.bug_notification_level, StructuralSubscription.date_created, StructuralSubscription.date_last_updated
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname,
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname
<wgrant> lifeless: 'ugh' comes to mind.
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname,
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname,
<lifeless> SELECT BugTask.assignee, BugTask.bug, BugTask.bugwatch, BugTask.date_assigned, BugTask.date_closed, BugTask.date_confirmed, BugTask.date_fix_committed, BugTask.date_fix_released,
<lifeless> SELECT StructuralSubscription.blueprint_notification_level, StructuralSubscription.bug_notification_level, StructuralSubscription.date_created, StructuralSubscription.date_last_updated,
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname,
<lifeless> SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname,
<lifeless> SELECT LibraryFileContent.datecreated, LibraryFileContent.filesize, LibraryFileContent.id, LibraryFileContent.md5, LibraryFileContent.sha1 FROM LibraryFileContent WHERE LibraryFileContent.id = %s LIMIT 1
<lifeless> INSERT INTO BugActivity (oldvalue, datechanged, whatchanged, message, newvalue, bug, person) VALUES (%s, CURRENT_TIMESTAMP AT TIME ZONE 'UTC', %s, %s, %s, %s, %s) RETURNING
<lifeless> INSERT INTO Message (datecreated, owner, subject, rfc822msgid) VALUES (CURRENT_TIMESTAMP AT TIME ZONE 'UTC', %s, %s, %s) RETURNING Message.id
<lifeless> I'm going to stop there
<lifeless> 'lots'
<lifeless> wgrant: mailed you, its an open package, normal person
<wgrant> lifeless: Is this from a hidden OOPS?
<wgrant> Hm, that's only 500 queries.
<wgrant> Is this a soft timeout?
<lifeless> yes
<wgrant> Ah.
<lifeless> but we've no reason to assume that this is unrelated ;)
<wgrant> I'm more concerned with the bad error than the fact that there's an error.
<wgrant> We know why it's timing out.
<wgrant> We don't know why it's timing out like this.
<lifeless> oh it concerns me too
<lifeless> wgrant: are you doing some perf stuff today? It is tuesday...
<wgrant> Are we going to be able to work this out on staging soon, or should we do it on prod now?
<lifeless> prod it up
<wgrant> OK.
<wgrant> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+filebug/5ca89d78-bfa3-11df-905e-0025b3df357a breaks pretty repeatedly.
<lifeless> wgrant: whats your ip for spm to look in apache logs
<wgrant> Isn't the token in the URL sufficient?
<spm> "trust me, I'm a sysadmin"
<wgrant> Otherwise 122.108.38.217
 * lifeless looks in shock at spm
<jtv> wgrant: heya
<wgrant> jtv: Morning.
<jtv> wgrant: may or may not be related but last night at least, we had some edge breakage where one of the edge instances reported the wrong revision.
<jtv> So it'd say it was at r11532 but actually seemed to be stuck at r11522 like the rest.
<wgrant> jtv: Unrelated -- this has been going on since ~10.09.
<jtv> Oh ok
<jtv> nm that then :)
<wgrant> Heh.
<spm> bleh. aapche say '502'
<wgrant> Nothing useful in the error log?
<wgrant> Or does that mean the appserver said 502?
<lifeless> spm: now the question is, did an oops get generated
<spm> [14/Sep/2010:04:50:42 +0100]
 * wgrant stabs BST in the face.
<spm> indeed
<spm> [Tue Sep 14 04:50:55 2010] [error] [client 122.108.38.217] (70014)End of file found: proxy: error reading status line from remote server localhost, referer: https://bugs.edge.launchpad.net/ubuntu/+source/linux/+filebug/5ca89d78-bfa3-11df-905e-0025b3df357a
<spm> [Tue Sep 14 04:50:55 2010] [error] [client 122.108.38.217] proxy: Error reading from remote server returned by /ubuntu/+source/linux/+filebug/5ca89d78-bfa3-11df-905e-0025b3df357a, referer: https://bugs.edge.launchpad.net/ubuntu/+source/linux/+filebug/5ca89d78-bfa3-11df-905e-0025b3df357a
<spm> ^^ errorlog
<wgrant> Aha.
<wgrant> So, the appserver died.
<wgrant> Or otherwise closed the connection.
<spm> hmm. haproxy/squid are in there somewhere. there may be in ter est ing comp li ca tions
<wgrant> I thought they were on the other side.
<wgrant> But I could well be wrong.
<wgrant> So I guess you need to go through all the layers :(
<lifeless> apache -> ha -> appserver
<wgrant> With Squid in front of Apache?
<spm> apache -> (squid)? -> ha -> app; POsts don't go via squiddly
<wgrant> Ahh
<wgrant> Handy.
<lifeless> nor do authenticated requests IIRC
<spm> correct
<lifeless> thumper: https://bugs.edge.launchpad.net/launchpad-code/+bug/637758 please put the code walkthrough we did in there, for gary's info when he sees the other bug I'm filing :)
<thumper> ok
<lifeless> spm: how many appservers for lpnet ?
<spm> 15
<poolie> how's it going, wallyworld?
<wgrant> I was a bit surprised to see O oopses over the weekend.
<lifeless> so 60 threads
<wgrant> I didn't realise there were quite that many.
<lifeless> thumper: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/637761
<poolie> wgrant: because the counter was broke, or because we actually had 0?
<poolie> remarkably good if os
<poolie> *so
<mwhudson> does that 15 include the login and shipit servers?
<wgrant> poolie: O != 0
<poolie> haha
<poolie> O meaning some particular category?
<lifeless> poolie: server ID in the oops code
<lifeless> A, B, C, ...
<spm> mwhudson: no, theose are extras
<mwhudson> wow
<mwhudson> lots of hardware
<lifeless> some machines have multiple instances
<lifeless> but yes.
<poolie> aren't some higher letters used for something other than a machine id?
<poolie> or maybe that's a different field
<lifeless> its an arbitrary string
<lifeless> e.g. XML
<lifeless> date before, serial after
<lifeless> thumper: rt 41361 if you want to high-pri it
<thumper> lifeless: ack, dealing with a user on #launchpad right now
<wgrant> The appservers are single letters. Others are longer strings (eg. CW, FTPMASTER, PPA)
<spm> woo. progress. Sep 13 23:26:16 localhost haproxy[15039]: 127.0.0.1:39282 [13/Sep/2010:23:26:00.844] lpnet-app lpnet-app/potassium_lpnet_5 0/0/0/-1/15230 502 1184 - - SH-- 67/38/38/2/0 0/0 "POST /ubuntu/+source/linux/+filebug/8dc224d8-bf85-11df-806b-0025b3df357a HTTP/1.1"
<lifeless> I wonder how hard it would be to port storm & zope to stackless
<lifeless> or jython
<wgrant> spm: But what does it mean?
<lifeless> wgrant: it means SH--
<wgrant> Heh.
<lifeless> wgrant: one thing it tells us
<spm> lpnet 5 on potassium did the "work"
<lifeless> potassium should have the oops
<wgrant> If there was an OOPS.
<wgrant> I was hoping it would tell us on what terms the response ended.
<thumper> lifeless: I'm thinking that we are seeing other xmlrpc problems from the bzr client
<thumper> lifeless: as it does lp name resolution lookups
<lifeless> wgrant: divorced
<spm> lifeless: wgrant: it also tells us, this timed out after 15 seconds; some other logs around there have 300secs, so ... funky.
<lifeless> thumper: sorry, can you expand on that please.
<spm> or succeeded after 270 seconds; so I'd suggest this is *unlikely* (but not ruled out) to be a timeout issue directly.
<wgrant> spm: Does potassium have an opinion?
<lifeless> the 270 seconds will be a file attachment
<lifeless> wgrant: it likes water
<wgrant> spm: Also, it didn't time out after 15 seconds.
<wgrant> I don't think.
<wgrant> Because I get that error in less than 14 seconds.
<spm> 15230 <== ms, ~ 15 secs
<lifeless> :23:26:00 -> 23:26:16
<wgrant> (now, at least -- not sure about that request)
<spm> :-)
<wgrant> So it's not a pure timeout.
<spm> I'd be inclined to rule out an apache/haproxy/squid timeout, not exlcude, but look elsewhere.
<lifeless> spm: is that url in the zop elogs on potassium
<spm> huh. not the *same* url, but related: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1718F124
<lifeless> spm: its odd for the request to dispatch and not be in the access log
<lifeless> spm: wouldn't you say?>
<spm> which log? the lp one?
<lifeless> lpnet_5
<spm> oh yes.
<lifeless> isn't there an access log for it?
<spm> I've founjd it in the trace log
<lifeless> \o/
<spm> but not in launchpad-access5.log-20100914
<lifeless> ok that OOPS is also a soft timeout
<lifeless> I wonder if there is something special going on
<wgrant> There is clearly something special going on.
<lifeless> wgrant: I have an experiment you could do
<wgrant> Sure.
<spm> lifeless: https://pastebin.canonical.com/37129/
<lifeless> configure launchpad.dev to have a 1 second soft timeout and 1.2 second hard timeout
<lifeless> point apport at it and have fun
<lifeless> wgrant: theory: hard timeouts are breaking in this case
<lifeless> and - boom - I think I know why
<thumper> lifeless: could have been something else, don't worry about it
<wgrant> Let's see...
<lifeless> requesttimeline stuff we saw yesterday.
<wgrant> Yeah, I wondered if that was realted.
 * wgrant finds the timeouts.
<lifeless> spm: is there an OverlappingActionError in the lpnet5 appserver log ?
<lifeless> spm: (not the trace log)
<wgrant> lifeless: I only see soft_request_timeout
<wgrant> No hard_request_timeout.
<lifeless> db_statement_timeout
<wgrant> Oh.
<wgrant> The comment on that is misleadding.
<lifeless> really?
 * thumper looks at the daily timeout candidates
<wgrant> # SQL statement timeout in milliseconds. If a statement
<wgrant> # takes longer than this to execute, then it will be aborted.
<wgrant> # A value of 0 turns off the timeout. If this value is not set,
<wgrant> # PostgreSQL's default setting is used.
<thumper> what is BranchSet:CollectionResource:#branches ?
<lifeless> thumper: an API call
<wgrant> thumper: API branch collection.
<thumper> yeah but which?
<lifeless> I've got a bug open on the clarity for that
<lifeless> thumper: you have to look at the oops
<wgrant>  /branches
<lifeless> which is why i have a bug open
<lifeless> wgrant: *any* collection.
<wgrant> lifeless: It says BranchSet
<lifeless> ah true, damn your eyes.
<wgrant> But yes, normally it's stupid.
<spm> lifeless: not that I can find. afaict, that "request" doesn't exist in the lpnet5 access log. even looking at the full 10 min period around
<lifeless> wgrant: http://pastebin.com/naNudsp3
<lifeless> spm: check nohup
<spm> hrm. point
<wgrant> Proxy Error
<wgrant> BOOM
<lifeless> for for OverlappingActionError
<wgrant> OverlappingActionError: (<lp.services.timeline.timedaction.TimedAction object at 0xe91ebac>, <lp.services.timeline.timedaction.TimedAction object at 0xe71ebec>)
<lifeless> wgrant: you reproduced ?
<wgrant> You win.
<lifeless> wgrant: \o/
<wgrant> Question is... which are they...
<spm> 1st problem. nohup doesn't log times.
<lifeless> apply my patch
<wgrant> Ah!
<lifeless> spm: never mind, my WAG was spot on
<spm>     raise OverlappingActionError(self.actions[-1], result)
<spm> OverlappingActionError: (<lp.services.timeline.timedaction.TimedAction object at 0x2b034a6f7990>, <lp.services.timeline.timedaction.TimedAction object at 0x1338fd90>)
<spm>     raise OverlappingActionError(self.actions[-1], result)
<spm> OverlappingActionError: (<lp.services.timeline.timedaction.TimedAction object at 0x13ff2f90>, <lp.services.timeline.timedaction.TimedAction object at 0x147b1f50>)
<spm> lifeless: cool, fwiw tho ^^
<lifeless> spm: thanks
<wgrant> Uh.
<wgrant> OverlappingActionError: (<TimedAction SQL-launchpad-main-master[UPDATE Bug SET heat_]>, <TimedAction SQL-session[ UPDATE ]>)
<wgrant> How?
<lifeless> grah
<wgrant> Oh.
<wgrant> Maybe when it times out it doesn't close the action?
<lifeless> see the comment in errorlog about this
<wgrant> Which?
<wgrant> Ah.
<wgrant> I see.
<lifeless> sorry
<lifeless> in logTuple
<lifeless> storm tracers are not a stack.
<lifeless> our having a timeout tracer and a log tracer doesn't work as well as it should in theory.
<lifeless> I think I'm going to create a stack-lock tracer that delegates to two other tracers and combine them.
<lifeless> long term.
<lifeless> for now, lets get fugly, lets get fugly.
 * spm decides now would be a good time to run away for lunch
<lifeless> spm: I have a cowboy
<lifeless> spm: when you return
<lifeless> wgrant: what was the bug for this?
<spm> lifeless: cool; I assum by then you'll also have an incident report to go with? ;-)
<wgrant> Bug 636801
<lifeless> spm: I can make one
<lifeless> wgrant: please confirm that http://pastebin.com/iPnkpPpF fixes it
<wgrant> lifeless: Great success.
<lifeless> spm: we have the thing to cowboy
<wgrant> stub: Hi.
<stub> yo
<wgrant> stub: The multiple OpenID identifiers stuff has had some interesting consequences.
<wgrant> stub: In particular the bit where it respects email address linkage more than identifier linkage.
<wgrant> Which results in people being logged in as the wrong person, and the real person OOPSing because they no longer have an identifier.
<wgrant> I guess the users can fix it by merging the accounts... but I'm not sure that respecting the email address in the first place is a good idea.
<lifeless> I don't get not respect.
<wgrant> Hm?
<lifeless> playing with words
<wgrant> Hah.
<lifeless> badly
<stub> I don't understand what the problem case is. If you are logging into the OP using an email address, you want to login as the Launchpad Person attached to that email address.
<stub> I suspect the cases that are broken where broken already, caused when LP accounts which were merged (the main bug this change was supposed to tackle)
<lifeless> spm: when you return : the thing to cowboy is https://code.launchpad.net/~lifeless/launchpad/cp/+merge/35364
<lifeless> spm: its going through the motions now to get into prod-devel, and I'll request a normal reroll tomorrow or so with it in it, but we should fix it now.
<lifeless> wgrant: so, care to work on filebug ?
<lifeless> wgrant: -huge- room for improvement.
<lifeless> spm: and for edge, we need https://code.launchpad.net/~lifeless/launchpad/oops/+merge/35363
<lifeless> again, its in the pipe to be done the normal way
<wgrant> stub: In the cases I know of, the user had changed their LP email address to blah+launchpad@some.domain
<wgrant> stub: A package or translations import then recreated blah@some.domain
<wgrant> So the next time they log in, they land in a different account.
<stub> I see.
<wgrant> What is the purpose of the email address match?
<stub> Because people can change their email details in the OpenID Provider.
<stub> Edit your emails, create a new account with the old email, be unable to log into Launchpad.
<wgrant> Huh?
<wgrant> If the LP person was tied to an identifier, then email addresses don't matter.
<wgrant> It could be a little confusing in some cases, until the OpenID associations are listed clearly... but it wouldn't do strange things like this.
<lifeless> -> shops. If issues, ring me
<lifeless> oh, this is needed primarily on appservers, so just them for now.
<stub> Create foo@example.com in the OP. Log into Launchpad. Edit the account to be bar@example.com. Create a new account for foo@example.com. Now if you log in as foo@example.com, you can't log into Launchpad as your email address in Launchpad is associated with a different identifier.
<stub> Although the way we really triggered this was account merging.
<wgrant> lifeless: ECHAN?
<wgrant> stub: Ah, I see.
<wgrant> Hmm.
<stub> People had multiple accounts in the OP, and a Launchpad person with multiple email addresses. They had to log into the OP using the email address that happens to be linked to the correct Person.
<wgrant> I wonder what should be done here.
<wgrant> I cannot see a good solution.
<stub> Because we can now link multiple identifiers to a Person, and because person merge does the right thing now, we might be able to drop some of the repair work login does now if the solution is causing worse problems.
<wgrant> Does the use case you provided above have any legitimate reason for occuring?
<stub> If it does, it is pretty obscure.
<wgrant> Yes...
<wgrant> So I wonder if the repair is useful.
<wgrant> Or if it should just tell you that you are doing bad things.
<stub> We are in a half way stage to becoming a real OpenID consumer. I think the problems go away if we stop trusting the Canonical SSO and instead implement the work flow for attaching OpenID identifiers to Launchpad accounts. But there is a fair bit of work that needs doing first (shipit and our test infrastructure makes this more complicated)
<wgrant> Right, this is what I was thinking.
<wgrant> Except for the test infrastructure.
<wgrant> What's the issue with that?
<wgrant> Does it do something stupid like using basic auth?
<stub> The OpenID our tests use is the old Launchpad OP code. It uses the same underlying database tables, so it is all tied up in knots.
<wgrant> Ah, right.
<wgrant> stub: So, what should be done? Advise the users to merge?
<stub> At the moment, yes.
<wgrant> Thanks.
<stub> That might be the preferred solution too, as we ensure the SSO database and Launchpad database remain in sync. I'm not sure.
 * mwhudson eods
<wgrant> The separation needs to eventually be far more obvious.
<lifeless> wgrant: not echan, info for spm
<spm> hm?
<lifeless> spm: the cowboy
<spm> ah
<lifeless> spm: see all the backlog
<lifeless> spm: we're missing out on many OOPS at the moment, its rather important
<spm> yarp; just getting it all together atm
<spm> hrm. complex patch that one
<lifeless> hah
<stub> lifeless: by 'fail closed' do you mean if we can't load or parse the config file we should default to enabled?
<stub> I can argue that either way. losas might have an opinion.
<lifeless> I mean we should not run unless permitted too
<spm> losas have lots of opinions, some of them are even relevant
<lifeless> if theres something wrong, running is likely to add to the problem
<lifeless> (as a default, for most cronscripts)
<stub> ok. My reasoning for the current behaviour is a stuffup in the config mechanism (Apache dead, syntax error) shouldn't bring the Launchpad systems down. And your reasoning is just as valid :-) It does get noisy if things fail, but that is about it.
<lifeless> its just cronscripts now isn't it ?
<stub> I could make it a config option and make it somebody elses problem ;)
<stub> Yes - this is just cronscripts.
<lifeless> so if all the cronscripts are down, when apache is down, I don't think its raeally a problem :)
<lifeless> I mean, at that point, apache is down :)
<stub> I think a typo or mistake is more likely - this config file is being edited by humans.
<lifeless> so there are two places that can occur
<lifeless> the lazr config providing the url
<lifeless> and the referenced ini file
<lifeless> for the former, it should change nearly never
<lifeless> for the latter, we could provide a small lint-and-update tool
<lifeless> spm: what do you think
<spm> I don't think. sysadmin.
<lifeless> and if someone paid you to ? :P
<spm> with chocolate? I'd pretend to think REAL hard.
<lifeless> hhha
<spm> not sure I fully follow the issue? Q&D summary? something about not running cronscripts if parts of LP are borked?
<lifeless> ok
<lifeless> so there is a new ini file coming in
<lifeless> it will disable all cronscripts in one hit, no need to touch cron
<lifeless> if there is an error obtaining it (over http) and parsing it, what should happen: should the scripts run, or not run
<spm> where "one hit" is ~ 26 easily accessible servers and 4 difficult ones?
<lifeless> yes
<spm> cool, just ensuring we appreciate what "one hit" means :-)
<lifeless> as long as they can access it over http inside the network.
<spm> Oh! I see. Right! thats.... funky.
<lifeless> hyou odn't like?
<spm> No I do like
<spm> the scripts should do whatever the last invocation was. which sucks, because now you're maintaining state as well.
<spm> ie. network hiccups *will* occour. we don't want to clobber LP by such a transient
<lifeless> spm: so tcp syn will retry three times anyway
<spm> as a thought: you'd have a 2 checks. the official "check http"; with a secondary, check local/state. We can script update the state if necessary - eg apache update
<spm> even so. soyuz used to barf badly all the time on funky network woes.
<spm> I'm think more resiliant than what just tcp et al give.
<spm> does that make sense? crackful?
<lifeless> thinking
<spm> lifeless: that should be done on PROD btw
<lifeless> so, for daily and hourly crons
<lifeless> wgrant: please break it
<lifeless> wgrant: prod
<wgrant> lifeless: OK.
<spm> not edge, haven't done edge yet
<lifeless> spm: for daily and hourly cronscripts, not running is fairly significant
<lifeless> spm: OTOH daily and hourly things are background tasks mainly, and oops reporting etc is separate and not driven by this
<wgrant> lifeless: Success.
<wgrant> OOPS-1718H415
<lifeless> spm: \o/
<spm> right, which is why I'd want them to fail as safely as possible - via a local state "what did I do last time?" <== but state is also likely to shared (maybe??) so likely updated more frequently.
<spm> \o/
<lifeless> so, personal opinion, if things are fucked royally I'd rather have the cronscripts not running to facilitate recovery.
<spm> I'd probably suggest shying away from per-job state; go for a global
<lifeless> as long as when they don't run they log it, if we find tha network transients are an issue, we can iterate.
<wgrant> As long as it keeps attempting to retrieve it frequently...
<spm> and if things are that bad; humans are involved. and we can set the state file; or 'script roll out' an updated state file
<stub> I understand the 'what I did last time' argument, but the extra complexity makes diagnosis complex. I'd say keep it as simple as we can.
<lifeless> stub: I agree.
<spm> if that case, fail quiet; don't run.
<stub> But if that means maintaining state, we can do that (the cronscript can remember its last invokation on the file system, in /var/run or somewhere.
<lifeless> spm: I'd fail closed, don't run, and log the failure.
<lifeless> spm: why do you say fail quiet ?
<stub> spm: At the moment, if the config file cannot be found (404) we emit a DEBUG and enable. Any other errors, including syntax errors, we emit a ERROR traceback and enable.
<spm> probably via some mechanisim that makes it easy to nagios alert
<spm> lifeless: don't cronspam bombard == quiet
<lifeless> spm: thats in tension with 'diagnosable'
<lifeless> spm: logging to a file would be ok?
<lifeless> spm: also remember that this is on 404s and syntax errors
<lifeless> spm: so things are messed up if its happening at all
<spm> lifeless: you're talking > 260 cron tasks. if we get a global fail, that's a LOT of spam to wade thru
<spm> logging to disk is ok
<lifeless> so, if we said:
<lifeless>  - on failure to get ini and parse, don't run.
<lifeless>  - log that to disk, not stderr
<lifeless>  - nagios should be monitoring those log files
<lifeless> would that make sense to you?
<spm> yup
<lifeless> stub: to you ?
<spm> * log that to disk, not stderr: like oops' perhaps in a vague handwavy way. known dir; date time stamped; we can nagios alert on files between 0-60 mins old type of thing
<spm> setup a red button "archive and zot the cron logs" so the "all fixed" is not as painful
<stub> Seems a little fragile relying on nagios like this. It is looking for an error rather than checking something is reacting correctly.
<spm> stub: how so?
<stub> We are relying on the cronscript to log things correctly and ...
<stub> oh.. hang on.
<spm> being ware of assumptions: I'd assume the apache/configs setup is monitored
<stub> We already alert if scripts stop running.
<spm> only via scriptactivity, but yes.
<spm> it's arguable if that should be nagios'd. atm I'd be vehmently against it.
<stub> So we could just disable silently (DEBUG or INFO - whatever) and rely on the existing checks to beep if things remain screwed up for too long.
<lifeless> works for me
<spm> that works
<lifeless> as long as someone coming along to look can look at a file on disk to see whats up.
<spm> I'm only aware of one script that really should have a nagios check against it - branch-merge-proposals
<lifeless> wgrant:
<lifeless> SQL time: 10701 ms
<lifeless> Non-sql time: 4505 ms
<lifeless> Total time: 15206 ms
<lifeless> Statement Count: 536
<spm> it's timely, and is also requiring human intervention on fail
<stub> I think the 'too much spam to wade though' indicates too many scripts are emitting their logs via email. Perhaps they should log to file instead of stdout/email and losas look on disk when the alerts ping.
<lifeless> stub: mthaddon wants that
<lifeless> stub: for all basically
 * StevenK kicks webservice.get()
<spm> I think pretty much every LP scripts logs to STDERR by default, which is known as Doing It Wrong
<StevenK> First call works, second call returns AttributeError: 'thread._local' object has no attribute 'interaction' :-(
<stub> spm: There is --logfile available right now to log elsewhere.
<lifeless> StevenK: login()
<spm> stub: I'm sure we've used that elsewhere and it doesn't quite work as described...
<StevenK> lifeless: But why does the first call to webservice.get() work fine?
<lifeless> you're already logged in
<StevenK> And how does that log me out?
<lifeless> end of the request
<lifeless> feel free to clean this up
<stub> spm: That would be a bug (not surprising as nobody ever used --logfile after it got implemented 5 years ago)
<spm> :-)
<stub>   --log-file=LOG_FILE  Send log to the given file, rather than stderr.
<spm> Ahh. That's not helpful as is. what you want is all "normal output" to go to stdout, or the above option. any *real* errors that *REQUIRE* manual intervention get sent to STDERR.
<spm> atm we get craploads of "INFO" messages or "CRITCIAL ERRORS" that are nothing of the sort, sent to STDERR
<lifeless> spm: icing on edge is borked
<poolie> stub: yeah we were just talking about this
<poolie> lifeless: me too
<lifeless> spm: we haven't got that part of the deployment right yet
<stub> spm: ok. That is a change, but we could do it globally for all scripts at once. The desired behaviour will need to be documented (bug report?)
<spm> lifeless: bleh. edge is autodeploying atm
<lifeless> yes
<lifeless> spm: need a new RT ?
<poolie> i'm also getting a 'The following errors were encountered:
<poolie> Server error, please contact an administrator.
<poolie> OK
<poolie> '
<spm> stub: i think this is a bug I logged about 2 years ago... :-)
<poolie> in an ajax thing
<spm> poolie: I can't do anything until you contact me per the above error. sorry.
<stub> spm: Yes - I was expecting production scripts to all be run with -q
<lifeless> poolie: thats possibly/probably the thing we're deploying to fix
<poolie> :)
<poolie> like a finely-oiled machine
 * spm watches poolie hop on his bike to drive down here and slap me upside the head....
<poolie> i would but it's a bit cold and wet
<poolie> and probably doubly so down there
<spm> "horrible" <== and not just saying to keep you away
<lifeless> spm: so now I have 11542 with no icing
<lifeless> spm: can you check the apache ?
<spm> yarp
<spm> hrm. supposedly we *are* 11542 everywhere
<lifeless> yes
<lifeless> but the icing ain't
<spm> isn't this the build farkup we saw the other week?
<lifeless> https://bugs.edge.launchpad.net/+icing/rev11542/combo.css
<lifeless> spm: thats meant to be static, from apache
<spm> le sigh
<StevenK> lifeless: With webservice.get(), login(), webservice.get() I get newInteraction called while another interaction is active. for the second .get
<lifeless> StevenK: odd
<lifeless> StevenK: perhaps the get() isn't what was throwing.
<lifeless> StevenK: perhaps you're actually tring to access something in  between the ( and )
<StevenK> lifeless: Of course I am
<lifeless> don't
<lifeless> calculate the url outside of the function call
<lifeless> because ...
<lifeless> accessing objects requires a participation
<spm> oh awesome. that file doesn't exist on edge.
<lifeless> say what ?
<stub> lifeless: I'll land the branch I have with the abspath and maybe the timeout if it is simple, as it is still an improvement, and open bugs and kanban tickets on the next set of changes.
<lifeless> stub: thanks
<spm> lifeless: exactly that. the folder is there, that particular file (and possible some small number of others) aren't.
<lifeless> spm: they are built by 'make compile'
<lifeless> or possibly make build
<spm> apparenetly not in this case....
<spm> oh ffs. the make build blewup again.
<lifeless> spm: does the deploy script abort when that happens ?
<wgrant> Not my fault, this time, though!
<spm> lifeless: https://pastebin.canonical.com/37132/
<spm> the script can't - the make is continuing, so the deploy scripts doesn't know it's aborted
<spm> (AIUI, IMBW)
<lifeless> spm: filing an RT - it has to.
<spm> No. I lie. It does see the error.
<lifeless> make: *** [compile] Error 1
<lifeless> Error 2 running ssh launchpad@banana make -C /srv/edge.launchpad.net/edge/launchpad build LPCONFIG=edge1
<lifeless> Running ssh launchpad@banana "rm -rf /srv/edge.launchpad.net/edge/launchpad && ln -s /srv/edge.launchpad.net/edge/launchpad-rev-11542 /srv/edge.launchpad.net/edge/launchpad"
<lifeless> its not halting !
<spm> yeah...
<spm> yeah x 3
<spm> ok. later problem. reverting.
<lifeless> spm: why was it 11542 that rolled out ?
<spm> no idea atm
<lifeless> ok, thats tip of stable
<lifeless> fair enough (but wtf with the error)
<spm> oki, apaches rolled back; doing the app servers
<adeuring> good morning
<spm> truely. it's supposed to abort on errors. we use this logic all over the place. And it works on other systems :-(
<lifeless> spm: can you do 11538 which I think was previous with the patch applied; and we may need to stop other edge rollouts till we fix.
<spm> heya adeuring
<spm> lifeless: that I have/am
<spm> lifeless: launchpad@banana:/srv/edge.launchpad.net/edge$ rm launchpad ; ln -s /srv/edge.launchpad.net/edge/launchpad-rev-11538 launchpad
<lifeless> spm: I bet its python2.5
<spm> lifeless: shrug, I just blame wgrant for everything. faster, easier, if less accurate
<lifeless> spm: can you do this on a machine thats still 2.5 ?
<wgrant> But it *was* me (and python2.5) last time this happened.
<wgrant> So it's quite accurate.
<spm> haha. lets not let FACTS get in the way here!!!!
<lifeless> spm: find . -name 'potemplate.py'
<spm> let me finish getting the apps restarted :-)
<lifeless> spm: then, for each reported file, cd to that dir and run 'python -c 'import potemplate'
<spm> edge3 done
<spm> 2010-09-14 07:48:01 WARNING SIGTERM failed to kill launchpad (7487). Trying SIGKILL <== yay. it's back! wooo!
<spm> edge4 coming back
<spm> edge1 coming back
<stub> Is network syslog loathed by IS?
<spm> edge2 on the way back
<spm> stub: i don't mind it; no idea about others tho
<spm> edge1 & 4 being difficult and not working
<spm> edge2 is fine
<spm> edge1 being really painful and needing to be manually killed.
<spm> edge1 stabbing successful; it lives
<spm> retrying edge4...
<spm> edge5 coming back
<spm> edge4 lives
<spm> edge5 lives; should be done. verifying.
<spm> lifeless: have you logged a bug on this essplosion?
<lifeless> spm: urls like this: https://bugs.edge.launchpad.net/+icing/rev11542/combo.css - how are they served.
<lifeless> spm: I RT'd it
<lifeless> spm: for the explosion part
<spm> that's the continue, but not the root cause?
<lifeless> spm: waiting on your python2.5 test for confirmation
<spm> ah k
<spm> lifeless: so to recap a bit back - CP'd to prod; not to edge.
<spm> *cowboyed* not CP'd.
<lifeless> spm: right, can we do edge 11538 cowboy, not 11542 cowboy ?
<spm> I'll throw that to Tom I suspect
<lifeless> ok
<lifeless> rev 11542 looks like the bust one
<lifeless> which is jtv's patch
<jtv> ?
<lifeless> jtv: you appear to have used python 2.6 in lib/lp/translations/browser/potemplate.py line 971
 * jtv looks
<lifeless> jtv: look at spm's pastebin
<jtv> Wonder what's wrong with itâ¦
<jtv> ah
<jtv> lifeless: want me to write up a quick patch?
<mrevell> Hello
<jtv> hi mrevell
<jtv> lifeless, spm: I'd fix it thusly: http://paste.ubuntu.com/493508/
<lifeless> spm: ping
<spm> lifeless: yo
<lifeless> spm: need you to try applying jtv's patch to a 11542 dir (one of the failed ones)
<lifeless> and see if 'make build' will then work.
<lifeless> jtv: could you please do a few things for me, its getting on here.
<lifeless>  - file a bug that this is broken,
<jtv> lifeless: speak
 * jtv files bug
<lifeless>  - put your branch up for review etc etc - r=me to apply it, if spm confirms it works.
<lifeless>  - arraange for someone to let the LOSAs know when it lands in stable, so that edge updates can be reenabled.
<lifeless>  - (e.g. yourself, or your delegate)
<jtv> On the way.
<lifeless> I will mail the list about the process issue
<spm> trying atm...
<spm> try2 wit hright config...
<jtv> bug 637868
<lifeless> jtv: did you ec2 your test ?
<spm> lifeless: I'd suggest that cowboy get's rolled in with jtv's fix and just a regular edge rollout rolled.
<jtv> lifeless: not yet, not yet
<lifeless> jtv: sorry, let me be more clear
<lifeless> jtv: the patch that broke; did you land it:
<lifeless>   - by running all tests locally + pqm
<jtv> ec2 land.
<lifeless>  - by ec2 land
<lifeless>  - ...
<spm> jtv: lifeless: https://pastebin.canonical.com/37134/ looks good
<jtv> spm: still looking good?
<jtv> The codebase, I mean, not you.  You'll always look good.
<lifeless> spm: that looks good; thanks. jtv your patch fixes it.
<jtv> BTW it's odd that this passed PQM, what with the pagetests exercising it.
<lifeless> jtv: pqm doesn't run tests.
<jtv> Sorry, buildbot.
<lifeless> jtv: ec2 runs them, but its probaby running lucid
<wgrant> buildbot is Lucid.
<lifeless> buildbot has two separate jobs.
<jtv> Well, we have lucid and hardy buildbot slaves.
<lifeless> jtv: see my mail
 * jtv will see mail
<lifeless> I don't think bb requires *both* to be ok.
<lifeless> but thats what we probably need
<jtv> BTW should I MP the fix for stable or for devel?
<lifeless> devel
<jtv> OK.  I branched off stable though just to be sure.
<lifeless> now that edge rollouts are blocked, theres no panic (no reason to delay) but no panic.
<jtv> lifeless: the MP is at https://code.launchpad.net/~jtv/launchpad/bug-637868/+merge/35373
<wgrant> lifeless: ec2 has been running Lucid for a long time.
<wgrant> I think this is about the third time things have broken.
<lifeless> wgrant: definitely second.
<lifeless> wgrant: bug 637854
<wgrant> _mup_, I am disappoint.
<wgrant> lifeless: At least it tries to prejoin.
<lifeless> wgrant: ugh!
<jtv> lifeless: so now I can land on devel as normal and just wait for the fix to percolate?
<lifeless> yes
<jtv> (I would appreciate a click on the button from you btw, to prove I didn't invent your approval :)
<lifeless> I did
<jtv> Oh!  The MP just timed out for me is all
<jtv> Thanks.
<lifeless> you need to let mthaddon know when its good to go on stable
<lifeless> jtv: interesting, what OOPS id ?
<jtv> I already ran the applicable pagetests through ec2â¦ guess a full EC2 run makes no sense here.
<jtv> lifeless: I don't know; focused on fixing my bug, so just reloaded
<bigjools> http://www.workswithu.com/2010/09/07/measuring-the-value-of-canonicals-launchpad/
<bigjools> there's a certain person posting comments on that one
<wgrant> Let me guess...
<wgrant> remarkable!
<spiv> wgrant: you're psychic, clearly.
<wgrant> He does have some good points, as usual.
<lifeless> and they are clearly unbiased
<lifeless> which is refreshing
<bigjools> shame it's the same sound of that grinding axe
<jtv> Speaking of grinding axesâ¦
<jtv> The builds-list.pt template is supposed to work for any BuildFarmJob but it tries to access build/dependencies.  :(
<jml> "No longer needed: Python 2.5"
<wgrant> jtv: I'm glad you're completing the generalisation for us :P
<jtv> wgrant: remember that axe I mentioned  just now?  Kindly insert it into one of your feet.
<wgrant> Ow.
<jtv> Good.
<jtv> And thank you.
 * wgrant limps away viciously.
<bigjools> jtv: then return None
<bigjools> you don't have any dependencies
<wgrant> It's not on the interface.
<wgrant> Assuming it is illegal.
<jtv> No, that's the nasty bit.
<jtv> It's in IPackageBuild.
<jtv> So _implementing_ it in BuildFarmJob or BuildFarmJobDerived or my own specific buildfarmjob class isn't enough.
<jtv> It needs to move into IBuildFarmJob, which is uglier than a Windows desktop.
<bigjools> at least Windows has sound that works
<jtv> To name some arbitrary example of extreme ugliness.
<jtv> Yes, Windows often has working sound, so you can _hear_ the "I don't know this codec" error instead of just seeing it.
<jtv> But we digress.
<jtv> Here we are chattering about operating systems when wgrant's foot is oozing virtual blood.
<jtv> Help the man, for God's sake!
<wgrant> bigjools: Your sound isn't working?
<wgrant> I think mine might be slightly crackly on Maverick.
<wgrant> But it works mostly.
<bigjools> maverick re-installed pulseaudio
<jpds> jtv: He's in AU, dude.
<bigjools> pulseaudio is a crock of shit
<jtv> jpds: pronounced "Ow!!!"
<wgrant> bigjools: WFM!
<wgrant> Although I could be distracted by my poor, poor foot.
<bigjools> it's insisting on using my laptop instead of my headset's mic
<jtv> wgrant: nice save
<bigjools> I've no idea how to make it do what *I* want instead of what *it* wants
<jtv> Meanwhile, I just managed to insinuate a TranslationTemplatesBuild into a builder history!
<wgrant> bigjools: Kubuntu doesn't have a nice control panel for it?
<wgrant> jtv: But does it crash?
<jtv> wgrant: no.  Not that anyone'd notice: I don't see anything in the build code that would put a BuildFarmJob into the right state to show up there.
<wgrant> Hm?
<bigjools> wgrant: I dunno, what is it in Gnome?
<jtv> bigjools: a crock of shit.  Trick question.
<wgrant> bigjools: The sound preferences thing (in the volume indicator's menu) has radio buttons for the input device.
 * bigjools hears 2 drums and a cymbal falling off a cliff
<allenap> Hi jml, thanks for looking into my Zope befuddlement. I think mwh's reply has now hit the nail on the head, so I'll wikify that and reply to the list.
<jtv> AFAICS the Builder still selects and dispatches a BuildQueue object, not a BuildFarmJob.  How's it ever going to update BuildFarmJob.{status,date_started,date_finished}?
<bigjools> finally it works
<jml> allenap, cool.
<wgrant> jtv: It's complicated.â¢
<wgrant> But it works.
<jtv> wgrant: well since there is absolutely currently nothing coupling my BuildFarmJobs to my BuildQueues, I don't see how it can.
<bigjools> so, when using pulse with skype, how do I make it ring the PC speakers and not in the headphones, which I might not be wearing? :/
<wgrant> jtv: I believe it's handled by the IBuildFarmJobBehavior.
<wgrant> jtv: IIRC you override updateBuild_WAITING.
<wgrant> The default calls handleStatus on the build.
<jtv> I think we already override that.
<wgrant> I mean you do currently.
<wgrant> But you should probably stop.
<jtv> Ah
<jtv> That could hurt.
<wgrant> It will.
<jtv> Hand me that axe, will you?
<jtv> Thankâeewwww, there's blood all over the blade
<jtv> Anyway, for now <puts axe aside> I guess it's enough to get display working and then next we can focus on tying the BuildQueue and the TranslationTemplatesBuild together.
<bigjools> wgrant: the discussion in https://bugs.launchpad.net/bugs/635103 is a little over my head at the moment, do you know why it's not working for him yet fine in Ubuntu?
<wgrant> bigjools: He wants to not have to download and upload the whole thing.
<wgrant> To do that we'd need an ia32-libs-specific hack, to support the conglomeration of horrid hacks that is ia32-libs.
<wgrant> I was going to tell him to go away. But you're probably a better person to do it :P
<bigjools> wgrant: possibly, but I don't understand what that package is doing
<wgrant> bigjools: Do you *really* want to know?
<bigjools> yep
<wgrant> Well.
<wgrant> There is a reason that the source packages is 700MB.
<wgrant> It contains approximately an awful lot of packed source packages.
<wgrant> It builds on amd64, but builds them for i386.
<wgrant> Er, wait, no, it includes the binaries too.
<wgrant> So it doesn't build them.
<bigjools> Oo
<wgrant> It extracts the i386 binaries, and produces a big amd64 ia32-libs binary containing all of them.
<lifeless> fooooooogly
<wgrant> So you have this huge source package contain dozens or hundreds of sources and binaries from the archive.
<wgrant> Er, yes.
<bigjools> dot com
<wgrant> It will be rendered obsolete by multiarch.
<wgrant> But multiarch hasn't happened yet.
<lifeless> multiarch was 'coming' when we -started-
<bigjools> dem's de magic words
<wgrant> lifeless: it's seriously in development now, though.
<wgrant> Some of the work has been done in the last year.
<wgrant> Hell, even NMSP is happening.
<lifeless> wgrant: I know.
<wgrant> And derivative distros.
<wgrant> This is incredible.
<bigjools> I can only lever so much of the planet with the team size I have
<jml> bigjools, well, to be true to the metaphor, you just need a better place to stand
<bigjools> jml: I'll jump higher :)
<bigjools> jml: when are you arriving here BTW?
<jml> bigjools, Sunday. Let me check my ticket.
<bigjools> you bought train ticket in advance?  gosh :)
<bigjools> insert preposition as required. Sigh.
<jml> I think you mean "article", and yes I did.
<jml> it's hard to get out of the habit of booking travel in advance
<wgrant> Is this for the buildd-manager attack session?
<jml> indeed it is.
<wgrant> Excellent.
<jml> bigjools, anyway, I'll be taking an afternoon train, probably the 1442
<bigjools> jml: ok well when you get ensconced in the pub, gimme a shout and I'll pop over for a pint
<jml> bigjools, will do.
<wgrant> bigjools: I've received lots of complaints in the last few days that builds keep getting redispatched.
<wgrant> Even on non-virt builders.
<bigjools> jml: there should be taxis at the station but if there are not let me know and I'll come and pick you up
<jml> bigjools, thanks.
<bigjools> wgrant: the UI is a lie
<bigjools> the early commit, is not :/
<wgrant> Hm?
<lifeless> jml: can you do me a favour?
<wgrant> Even if it is committing before it confirms successful dispatch, why is the dispatch not successful?
<jml> lifeless, quite possibly.
<bigjools> what's happening is that we mark the build as running before it's completely dispatched.  If there's a comms error then it looks like it gets re-dispatched after the next builder picks it up
<lifeless> jml: my pqm-landed (nonec2) branch has a test failure
<lifeless> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/139/steps/shell_7/logs/summary
<lifeless> jml: its -extremely- shallow.
<lifeless> jml: (add the missing tuple)
<wgrant> bigjools: But there shouldn't comms errors :(
<wgrant> +be
<lifeless> jml: however the fix needs to be done to production-devel too.
<bigjools> wgrant: we don't live in a perfect world
<lifeless> jml: before the oops fix can be uncowboyed.
<bigjools> routers drop out, DC engineers kick cables
<bigjools> etc
<lifeless> jml: yes/no ?
<wgrant> bigjools: Over 20 minutes?
<jml> lifeless, you'd like me to land the fix for you?
 * gmb hates at typos in sampledata
<gmb> 'testible' indeed.
<lifeless> jml: yes, on devel and production-devel
<jml> gmb, it's a pun!
<bigjools> wgrant: yes, that's the interval I see because of the bad scaling
<lifeless> jml: its 22:20 here, more or less
<gmb> jml, I noticed that about half a second after pressing enter :)
<wgrant> bigjools: Ahh, true, forgot that bit.
<jml> lifeless, ok, will do.
<lifeless> jml: thank you!
<jtv> Are we going into testfix?
<jtv> The lucid_lp buildbot just failed.
<jtv> Is failing, rather.
<jtv>    lib/canonical/launchpad/webapp/ftests/test_adapter.txt
<jtv> Line 305, in test_adapter.txt Failed example:
<jtv>      get_request_statements()
<jtv> Differences (ndiff with -expected +actual):
<jtv>      - []     + [(0, 0, 'SQL-launchpad-main-master', 'SELECT 2')]
<wgrant> Is that what lifeless was talking about above?
<jml> wgrant, yes.
<jml> I wonder why emacs is segfaulting for me.
<thumper> jml: because it hates you :)
<bigjools> it's telling you to use a real editor
<deryck> Morning, all
<bigjools> howdy deryck
<jml> bigjools, yeah, you're right. I've had to revert to "emacs -nw"
<jml> deryck, hello
<bigjools> :)
<thumper> jml: ?? whazzat?
<jml> thumper, try it!
<thumper> not right now
<jml> thumper, ok, I give up, it's emacs in a terminal
<thumper> I'm trying to right a talk
<thumper> ah.. no windows, it's so obvious
<bigjools> that's either a typo or a clever play on words
<thumper> bigjools: which one?
<jml> thumper, "righting" a talk.
<bigjools> "right a talk"
 * thumper hangs head
<bigjools> lol
<thumper> it is a typo
<thumper> I'd like to be more cleverer
<jml> I'm off for lunch & errands. Back later.
<cr3> leonardr: when you have a moment, I would have a question for you about routes when exposing a restful interface with lazr
<leonardr> cr3, sure
<leonardr> routes as in the url traversal code?
<cr3> leonardr: yes, how can a collection be contextual? for example, lets say LP had /me/bugs and /project/foo/bugs, where both person and project would implement IHasBugs, how should the "bugs" part of the url be defined?
<leonardr> cr3: that's called a scoped collection, and lazr.restful traverses from 'leonardr' to bugs or from 'mozilla' to bugs by attribute access on the person or project object
<leonardr> so leonardr.bugs is /~leonardr/bugs
<cr3> leonardr: ah, so it must be defined as an attribute, I thought it might be ProjectNavigation in the browser layer or perhaps even using the Bag
<leonardr> cr3: no, once you have identified a specific object all further traversal happens through attribute access
<cr3> leonardr: would it make sense to have the IHasBugs define a "bugs" attribute?
<leonardr> cr3: afaik, yes
<cr3> leonardr: but if IHasBugs has a searchBugs method already which should essentially behave like the bugs attribute, given no parameters, then wouldn't searchBugs and bugs look a lot the same?
<cr3> leonardr: my concern is that every class implementing IHasBugs would essentially have to do something like: @property; def bugs(self): return self.searchParams();
<leonardr> cr3: well, you don't _have_ to put 'bugs' in IHasBugs if different implementations get the bugs differently
<leonardr> but, i have two things to say on top of that
<leonardr> oh, neverm ind, you're saying that all the IHasBugs feature 'bugs'
<leonardr> be that as it may, /bugs is better for the end-user than ?ws.op=searchBugs
<leonardr> however, there's nothing to be done about that for now
<leonardr> my next project will include things like
<cr3> leonardr: I was mostly using IHasBugs as an example for a collection which might be implemented by more than one context
<leonardr> cr3: sure, i know you're not really talking about IHasBugs. but i'm trying to deal with the situation as you posed it
<leonardr> my next project will include features like the ability to designate a method as being "the method you call to generate a scoped collection"
<leonardr> so you could tag searchBugs as the generator for /bugs
<cr3> leonardr: I was grepping through lazr for the concept of an alias, like "bugs" is aliased to searchBugs or somesuch
<henninge> sinzui: ping
<sinzui> hi henninge
<henninge> Hi!
<henninge> I am a bit at loss at how to downgrade a package.
<henninge> psycopg2 in this case
<sinzui> henninge, do you have the deb?
<henninge> sinzui: I have done that once or twice before but I could use a pointer, please ;-) ?
<henninge> No, I was just searching for that.
 * sinzui checks lp history
<sinzui> henninge, download the deb from here: https://edge.launchpad.net/ubuntu/lucid/i386/python-psycopg2/2.0.13-2ubuntu2
<sinzui> henninge, sudo dpkg -i --force-downgrade python-psycopg2_2.0.13-2ubuntu2_i386.deb
<henninge> sinzui: thank you very much!
 * henninge actually forgot to look on LP for the package ...
<sinzui> henninge, i decided not to pin the version. I hold out some small hope that lp or psycho will resolve there differences. I downgrade after every update breaks lp
<henninge> sinzui: yes, otherwise one might forget about the pinning and hit strange errors later ...
<gmb> rockstar: So, deryck solved our JS wizard problem :)
<rockstar> gmb, that's because deryck is awesome.
<rockstar> gmb, what was the issue?
<gmb> rockstar: Two things: 1) YUI auto-generates the CSS class names based on WIDGET.name - so the hidden class was yui3-lazr-wizard-hidden, which wasn't defined anywhere.
<gmb> Also, widgets that aren't created as hidden can never *be* hidden.
<gmb> (At least, that's how it behaves; I suspect theres a bug there)
<rockstar> gmb, wait, how was it never created?
<rockstar> gmb, and it's Widget.NAME, isn't it?
<gmb> rockstar, Yeah, sorry, bad caps.
<gmb> rockstar: Let me just check the patch deryck gave me so that I know I'm not BSing you.
<gmb> rockstar: http://pastebin.ubuntu.com/493666/
<rockstar> gmb, okay, so I had it defined as Wizard.NAME = "wizard"; so I don't know where the lazr comes from either.
<gmb> rockstar: The wizard was created but by default visible was True.
<gmb> At least, that's how it looks based on deryck's patch.
<rockstar> gmb, okay.
<rockstar> gmb, I'm not sure I understand the bottom patch though, to wizard.js.
<gmb> rockstar: Yeah. lp:~gmb/lazr-js/wizard-widget/ contains deryck's fix and some further CSS fixes.
<rockstar> gmb, I guess he's just demonstrating that there's missing CSS somewhere?
<gmb> I don't know if you need to do more with it.
<rockstar> gmb, well, it needs to get finished now.  If it's firing events, it can probably start moving through steps now.
<gmb> Well, I don't know. That seems to be related to the way the widget behaves... deryck, can you clarify exactly why your fix fixes?
<deryck> rockstar, yeah, that's all.  The CSS in use currently assumes the widget name is "lazr-formoverlay" and it wasn't set to hide by default.
<rockstar> deryck, okay, so we can't reuse that name, so we need to just define yui3-lazr-wizard-hidden in the CSS then?
<deryck> rockstar, yui3-NAME-hidden, where NAME is what you define in the widget.  This is how all those CSS classes get built.
<rockstar> deryck, yeah, so it should have been yui3-wizard-hidden
<rockstar> deryck, and I thought I had defined that.
<deryck> rockstar, right.  There is a yui3-NAME for every class this widget descends from.  But only the current NAME gets the hidden class added.
<rockstar> deryck, yeah, okay.
<deryck> rockstar, you did, but the CSS was not using it.  And you couldn't tell because you weren't hidding by default with visible: false.  So changes to the name had no affect.
<rockstar> deryck, ah, that makes sense.
<rockstar> So if it's not hidden by default, it can't be hidden again.
<rockstar> That is, quite possibly, one of the silliest things I've ever heard.
<gmb> rockstar: Yeah, I needed to mop the tea off my monitor when deryck told me.
<rockstar> gmb, I'm digging in my junk drawer for a baby to punch as we speak.
<gmb> ...
<deryck> well, no, not quite accurate
<deryck> rockstar, wizard.render() shows the widget without the hidden class.  Nothing in your code calls wizard.hide().
<rockstar> deryck, the defaultCancel should have been doing it.
<deryck> rockstar, I don't think so.  That's only called after UI changes, AIUI.
<deryck> at least, my reading of the code.
<rockstar> deryck, no, it's called when the CANCEL event is fired.  I know it was being called, because that's where the Y.log("aoeu") was.
<rockstar> I think you and I both confirmed that the -hidden CSS class was getting set as well.
<deryck> right, but that's not called on load.
<rockstar> deryck, yeah, so I either have to hide by default and call .show() in the example, or call .hide() and then .show() on load.
<deryck> CANCEL event is only fired by ESC or clicking away from the widget, no?  I never saw "aoeu" until I did some action, not on load.
<deryck> right
<rockstar> deryck, I saw it when I clicked the cancel button.
<rockstar> Yeah, okay.  So what you're saying is what I'm understanding.
<rockstar> Stupidness prevails.
<rockstar> Thanks for sorting it out.
<deryck> np :-)
<rockstar> We need a page on the wiki of all the YUI gotchas.
<deryck> yup
<rockstar> We'll probably just forget the page exists and create it 4 more times, but whatever.
<salgado> jcsackett, I was going to have a second look at your unknown-blueprints-service-597738 branch but noticed there's some discussion still going on, and I'm wondering whether you're going to do any more changes to it or if it's ready for a second look
<jcsackett> salgado: i'm still working on it.
<jcsackett> i actually needed to add an attr to the view, so i need to write some tests as well.
<jcsackett> salgado: i'll ping you and sinzui when i've pushed changes for round 2.
<gmb> rockstar: So - for my own clarity - are you now going to do further work on the wizard to make it do wizardly things properly?
<rockstar> gmb, I _can_, but it sounds like it's blocking you, and I'd like to avoid that as much as possible.
<gmb> rockstar, Right. That sounds fair. In that case, I'll get cracking on getting it doing what what we need and ping you if there are any further issues.
<gmb> rockstar: Is there a specific bug or LEP that your work so far is tied to? I don't want to make something that doesn't do what you need it to do.
<rockstar> gmb, does the overall design make sense?
<rockstar> gmb, I had a kanban card with the work on it.
<rockstar> (because we like to track our work in many different places)
<gmb> rockstar: Yes. In fact, it's pretty much exactly what I had in mind for my hack, although that was less elegant :)
<gmb> rockstar: Ah, cool. I shall go and find it.
<salgado> jcsackett, ok
<deryck> rockstar, hey, I think all the python was added to lazr-js for the testsing story.  To hook up the yui-unittest stuff with zope test runner.
<rockstar> deryck, I'm not so sure.  Our testing story uses Java.  It can be fired up from the shell.
<deryck> ah, ok.  Maybe not then.
<deryck> I thought that was why.  Why all the Zope packages then?
<deryck> and storm and lazr.restful.  god good y'all. ;)
<rockstar> deryck, so, the testing story does use lazr.testing, but it doesn't need to.
<rockstar> deryck, also, it could be used for the lazr-js testing, but not have to be distributed to our projects as well.
<deryck> right
<deryck> rockstar, just thinking more....
<deryck> rockstar, some of what the egg provides us is the js lint stuff.... perhaps that could be broken out into it's own package.... separate the testing, python utils, and js file building stories a bit?  Smaller simpler packages?
<rockstar> deryck, yes.
<rockstar> deryck, the problem is that we've tied ourselves to a buoy with no anchor, so we experience pain anytime we want to change anything.
<deryck> right
<deryck> yeah, maybe it's not easy to do a neat and clean separation.
<rockstar> deryck, the launchpad build system is too closely tied to the lazr-js build system, which makes it exponentially more complicated.
<jml> clearly you should attach a sail to your buoy
<jml> or something.
<rockstar> jml, engineering fail.  :)
<deryck> rockstar, it also makes adoption of lazr-js by any other web project outside Canonical difficult or impossible.
<rockstar> deryck, exactly.
<jml> rockstar, you're the one who's in pain and stuck to a buoy!
<rockstar> jml, no, my boat is stuck to a buoy.
<rockstar> jml, also, while the launchpad team does blow a lot of hot air, I think we want engines, not sails.  :)
<jml> rockstar, ok, as long as it's an electric motor
<rockstar> deryck, I tried to use lazr-js this weekend.  After about an hour, we gave up and used jQuery.  :)
<jml> with batteries charged by a wind farm.
<rockstar> jml, yeah, because we also want to be environmentally responsible.
<rockstar> Unfortunately, Google owns the wind farms, so we have to display Google Ads on our boat the whole time.
<jml> brb.
<mrevell> night all
<deryck> Some say money is the root of all evil, but it's really notifying subscribers in a web app request.
<rockstar> deryck, :)
<rockstar> deryck, sending mail in general...
<rockstar> james_w, why does Recipe.__str__ need to call .parse() ?
<james_w> rockstar: it doesn't
<rockstar> james_w, so I could write a patch that removes it, and you'd be happy with it?
<james_w> rockstar: but it's a good way of ensuring that we are not generating malformed recipes in __str__
<rockstar> james_w, shouldn't tests be enough?
<james_w> clearly you have found a case where that breaks down, but I'm not convinced that it warrants getting rid of that
<james_w> I would remove it if I could be convinced that there are many more cases where it isn't going to be a good thing
<james_w> rockstar: yes, they /should/ be enough
<rockstar> james_w, I can't foresee any other issues, but if I could foresee bugs, I would fix them before they became bugs.
<james_w> yes
<rockstar> james_w, I think that, for reads, we should trust that it creates them properly.
<rockstar> james_w, if we have bugs, then we can deal with them.
<james_w> why trust when you can verify?
<rockstar> As it is, if __str__ ever creates a bad manifest, it'll explode on a user in Launchpad.
<rockstar> james_w, I guess the question is "Is Launchpad bzr-builder's most common use case."
<james_w> yes
<rockstar> james_w, I'm all for verifying if it didn't raise an exception the way it does.  I'd like it to warn and move on, but that would be difficult to look for in Launchpad.
<james_w> bad manifest> explode how?
<james_w> warnings> considered that, but who ever pays attention to warnings?
<rockstar> james_w, yes, and warnings would be difficult to get out of a webapp.
<james_w> yeah
<james_w> well we don't have to use warn(), but still...
<rockstar> james_w, you're verifying that functionality you wrote is working properly.  That's noble and all, but if there were a bug, RecipeParser.parse() would raise a rather exception that isn't really the user's fault.
<james_w> yes
<rockstar> james_w, I think the best course of action would be to remove the parse in __str__.  We could have different method be more strict, but having __str__ be that strict seems odd.
<james_w> having it be sure to generate a valid recipe is odd?
<rockstar> james_w, in __str__ I think it is.
<rockstar> james_w, how 'bout this: Recipe.get_manifest() will always return a valid manifest or raise an exception, while __str__ just returns the manifest, valid or not.
<james_w> why would you ever want to put an invalid manifest in the edit box?
<rockstar> In Launchpad, we could then call Recipe.get_manifest(), and if it raises an exception, get the raw string.
<rockstar> james_w, I think we have a valid reason to put an invalid manifest in the edit box.
<james_w> if there is a bug that causes a round-trip to fail, then you will force the user to correct the error, then hit save, which will corrupt it again when displaying it
<rockstar> We can catch the exception and add an error box that says "This recipe is totally broken.  Please fix it."
<james_w> but you just said yourself that this would only happen for bugs where it isn't the users fault
<james_w> so don't we want an OOPS if they are bugs?
<rockstar> james_w, in this case, it's *kind of* the user's fault, but we let them save the bad data, and now they can't fix it.
<james_w> yes
<rockstar> james_w, but we need a better migration path than oopsing on crap data.
<james_w> but I'm now starting to think we should call it a bug and go back and rethink the fix
<rockstar> james_w, I've been calling it a bug the whole time.
<rockstar> Because to Launchpad, it is a bug.
<rockstar> And I'm trying to solve it for both Launchpad AND bzr-builder.
<rockstar> james_w, in this case, the current bug is caused by a bzr-builder bug getting fixed.
<rockstar> The user used . as a directory, and that never worked in building.  Now it fails earlier.
<james_w> yes, but perhaps we should be saying that making the parser more strict without a format version bump is a bug, and should be fixed
<rockstar> So the user entered bad data, but we accepted it.
<rockstar> james_w, possibly.  I suggested that to abentley, and he had a point that this was always bad data.  It's just that it fails earlier now.
<rockstar> james_w, and it never really affected users of bzr-builder itself like it did with Launchpad.
<james_w> yes, it was always bad data
<rockstar> james_w, it's just that now, the user has no way of getting to that data and fixing it themselves.
<james_w> but there is a distinction between parse+build in the code, and perhaps trying to conflate them like that isn't the best idea
<rockstar> So we need to teach Launchpad how to cope with crap data and encourage the user to fix the crap data.
<abentley> james_w, I *like* validation.  I think we should do *more* of it.  For example, bogus revision specs.
<rockstar> I'm not saying we shouldn't validate, but we should provide a path for coping with invalid data that doesn't require futzing with the database.
<james_w> abentley: then please file bugs
<abentley> I just think we should distinguish between "well-formed" and "valid", and allow parsing of recipes that are merely "well-formed".
<abentley> james, there's already a bug about that.
<james_w> rockstar: if the code had always been like this then the bad data could never get in the db. If we have a rule that more strict parsing would always result in a format bump then there are two ways we could get this problem in future:
<james_w> 1. a bug that means that the parser doesn't detect the problem in the first place
<james_w> 2. a bug in __str__ which is why the check is there
<james_w> or I guess 3. that we want to make it stricter without a bump for some reason
<rockstar> james_w, this change I'm proposing is STRICTLY for Launchpad sanity cases.
<james_w> in either the first two cases then there is a bug that we want to know about
<rockstar> james_w, have you seen https://bugs.edge.launchpad.net/launchpad-code/+bug/620868
<rockstar> (this is the bug I'm addressing)
<james_w> yes
<james_w> abentley: I don't see a bug
<rockstar> james_w, do you understand why that bug exists?
<james_w> yes
<rockstar> james_w, okay, there was a bug, #1 in your case.  It was fixed.
<rockstar> It caused existing data (that never really worked anyway) to cause oopses, but not provide a way for the user to fix it.
<james_w> It's not #1 in my case
<abentley> james_w, https://bugs.edge.launchpad.net/launchpad-code/+bug/592821
<james_w> they were well-formed recipes before, just ones that were never going to work
<james_w> abentley: ah, so not on bzr-builder
<abentley> james_w, right.  It doesn't have to be done there.
<james_w> why not do it there?
<rockstar> abentley, I'm not sure I see how your issue and mine go together her.
<rockstar> s/her/here
<abentley> james_w, because I don't know whether such a check is useful to bzr-builder.  Because if it is useful, I don't know why it's not there.
<james_w> because I never thought of checking?
<abentley> james_w, because the set of valid revision specs can vary, and maybe you don't want to get into that.
<james_w> exactly
<james_w> and maybe Launchpad shouldn't either?
<rockstar> james_w, in the case of your #1, yes, the parser wasn't detecting the problem, and now it is (with a newer bzr-builder)
<abentley> james_w, we can guarantee that it won't vary between our appservers and our builders if we choose to.
<rockstar> Because it wasn't detecting the problem, the invalid recipe made it into the database.
<rockstar> Now, with a new bzr-builder, it finds the bad data and oopses.
<abentley> rockstar, they go together because they are both issues of validation, where an incorrect value was put into a recipe field.
<rockstar> Launchpad needs a way for users to deal with bad data that made it into the data (however that happens) and allow them to change it.
<rockstar> abentley, okay.
<rockstar> james_w, so I'm proposing a change that would help Launchpad by providing a better interface the bzr-builder's Recipe class.
<rockstar> Launchpad needs to be more robust.  We can validate 'til the cows come home, but if we don't give users a way to deal with invalid data, then we've only made things worse.
<james_w> rockstar: I understand that, but I am looking to explore the issue in a little more depth. There's little point in asking the user to correct the problem if the problem was caused by us.
<rockstar> james_w, the problem was caused by us, only in the fact that we let them put bad data into the database, but that would never actually succeed.
<rockstar> james_w, format bump or whatever needs to happen, I'm happy with.  My big concern, however, is that the user's don't suddenly find out their recipe is broken by finding an oops where their recipe used to be.
 * rockstar should probably go eat something, so he stops this egregious use of apostrophes.
<james_w> rockstar: I agree with you that they should be able to fix bad data that they put it
<rockstar> james_w, the patch I propose would do that.
<james_w> rockstar: I'm arguing that doing this across the board leads to us possibly asking users to "fix" perfectly valid recipes due to bugs in bzr-builder
<rockstar> james_w, maybe.  I'm less concerned with that at this point.
<james_w> so I'm looking for ways for us to separate the two things such that we can ask them to fix bad data, while apologising for bugs and fix them
<rockstar> james_w, I will always apologize to the user.  The fact that we're just now telling them they're wrong is a no-no on our part.
<james_w> all of the examples given so far are recipes where we can perfectly understand the intent, they just aren't going to work
<james_w> so as Aaron said, splitting well-formed and valid might make sense
<rockstar> james_w, which is what I'm proposing.
<james_w> rockstar: at one level, yes, but we can push it deeper than the patch you are suggesting
<rockstar> james_w, here's my patch: http://pastebin.ubuntu.com/493798/
<james_w> yes, I perfectly understand the change you are proposing
<rockstar> james_w, I don't see necessity for going any deeper than that.  If I can catch the exception and somehow say "Hm, this is what you had, but for some reason bzr-builder doesn't think it's valid anymore."  then I'm happy.
<james_w> sure you are
<rockstar> james_w, I'm not sure how much more apart "valid" and "well-formed" you want.
<james_w> a way of saying to the user "your recipe is well-formed, but these things are likely to be problems:"
<james_w> at any time
<james_w> then we can make the parser more "strict", without causing issues like this, provide better assistance to the user, and still have validation that what we create is at least well-formed
<rockstar> james_w, yeah, I have no opinions on the overall architecture of bzr-builder.  I would just like something that works today.  If there's a bigger picture, great.
<james_w> this has an impact on LP too though
<rockstar> james_w, I think it does.  I think it'd be great for user's experience.
<james_w> it's about user-experience, so I won't let you wash your hands of it as lying outside of LP ;-)
<rockstar> james_w, however, right now, the user's experience is "WTF? Why can't I get to my recipe?"  That's my big concern now.
<james_w> sure, but it's only two recipes?
<rockstar> james_w, yes, but that's two more oopses that we don't need.
<james_w> the change you propose is a "narrow" interface to likely problems (it can only report one), and it has a poor API to use it everywhere
<lifeless> morning
<james_w> sure, it's just not a stop-the-line issue IMO
<lifeless> so the OOPS comes from where?
<james_w> so if we can we should come up with an API that nicely gives us the better experience and implement that
<james_w> lifeless: https://bugs.edge.launchpad.net/launchpad-code/+bug/620868
<lifeless> so maybe I'm confused
<lifeless> we used a plain text field to store the recipe didn't we?
<abentley> lifeless, no.
<abentley> lifeless, we store the recipe in object form.
<lifeless> sourcepackagerecipedata ?
<abentley> lifeless, yes, and the SourcePackageRecipeDataInstructions that refer to it.
<lifeless> ok, I see
<lifeless> thanks
<lifeless> I was wondering if it would make sense, when the recipe is invalid to still permit it to be edited
<lifeless> until it becomes valid.
<abentley> lifeless, that is what we want to do.
<lifeless> then we can handle a wider range of unexpected things like this
<lifeless> abentley: awesome
<abentley> lifeless, The problem is that we can't stringify the invalid recipe.
<lifeless> what do we we use stringification for ?
<abentley> lifeless, because bzr-builder checks for validity when it stringifies a recipe.
<abentley> lifeless, we use stringification for displaying the field to the user so that they can edit it.
<lifeless> does this mean that we can't show the user the invalid recipe
<abentley> lifeless, yes.
<lifeless> I see, certainly not going to help things along ;)
<lifeless> and it helps me understand the chat you were having - thanks.
<rockstar> abentley, http://pastebin.ubuntu.com/493798/
<wallyworld_> morning
<abentley> thumper, http://pastebin.ubuntu.com/493848/
<lifeless> Ursinha: hi
<lifeless> https://bugs.edge.launchpad.net/launchpad-registry/+bug/615237
<lifeless> oh, I see whats up
<lifeless> Ursinha: the ec2land stuff
<lifeless> that gets bugs from an MP, does it get it from the MP, or the branch ?
<mwhudson> lifeless: the mp gets bugs from the branch, unless i'm missing context horribly
<lifeless> mwhudson: so I use the same branch for domain-fixues
<lifeless> mwhudson: mps' show bugs already fixed in earlier MP's
<lifeless> mwhudson: but I need to know which -precisely- the ec2 land code uses, or where to find that code, to make it stop including fix-committed and fix-released bugs in the bugs list in the pqm mail.
<mwhudson> ah right
<mwhudson> i expect the ec2 land code isn't that impentrable...
<lifeless> indeed
<lifeless> its blatting a growing number of bugs every time i land
<Ursinha> lifeless, the branch
<lifeless> Ursinha: hi
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/638468
 * Ursinha looks
<Ursinha> lifeless, don't know if that works
<lifeless> Ursinha: 'that' ?
<Ursinha> lifeless, sorry, let me elaborate
<Ursinha> lifeless, problem is many times people mention bugs that weren't properly fixed, but had code landed, so bugs that are fix committed or fix released
<Ursinha> so for that to work we'd need to ensure that all bugs that have fix to land are !fix committed/released
<Ursinha> otherwise we'll start missing things
<Ursinha> my thoughts would be: create another branch
<Ursinha> than that won't happen
<lifeless> Ursinha: hang on a sec.
<lifeless> Ursinha: ec2 land will error if there are no valid bugs right ?
<Ursinha> lifeless, if there are no bugs and it's not no-qa, yes
<lifeless> right
<lifeless> so, if someone has references only fix-committed and fix-released bugs
<lifeless> they will get an error
<lifeless> and in that errror you could say (bug X Y and Z are also linked on the branch but are fix committed/fix released)
<Ursinha> lifeless, what will you do if you're trying to land a branch which is already linked to a bug which is fix committed, but that's what you want to do? are you going to unlink the bug?
<Ursinha> I don't like this idea
<lifeless> Ursinha: if the code I'm landing is needed to fix that bug, its not really fix committed is it ?
<Ursinha> lifeless, it can be, but qa-bad
<Ursinha> fix committed == there's a fix for that bug (working or not)
<Ursinha> in progress == fix is in progress (it might have incremental fixes but the whole fix isn't committed yet)
<Ursinha> s/committed/landed
<lifeless> qa-bad should imply in-progress or triaged
<lifeless> fix committed isn't 'commit in the tree' its *FIX* in the tree.
<Ursinha> right
<lifeless> I think that when something is bust we should make the bug be in-progress again
<lifeless> just like --incr
<Ursinha> lifeless, manually?
<lifeless> we could automate it
<lifeless> but sure, manually.
<lifeless> qa-bad + fixreleased makes no sense.
<lifeless> qa-bad + fixcommitted also makes no sense.
<Ursinha> qa-bad is added by the devel; we could a) change it manually in the same time we're adding the qa-bad tag or b) make the bot to check if there are qa-bad and change it to in progress again
<lifeless> Designing other workflows around nonsensical states is not going to work well.
<lifeless> Ursinha: I like both a) and b)
<Ursinha> lifeless, ok. what to say about branches that have several bugs linked, and some of them are already released
<Ursinha> ?
<lifeless> just list the other bugs.
<Ursinha> why are people reusing branches?
<lifeless> Ursinha: convenience; clarity; organisation.
<Ursinha> lifeless, well, I think we're trying to make the scripts to workaround a situation that could be avoided by just not reusing branches
<Ursinha> and the problem is that the script already tries to workaround some behaviors to create consistency
<Ursinha> I think the scripts will get more and more confuse because of that, but if you think this is really worthy, I can do that
<Ursinha> adding a mechanism to tagger to set qa-bad bugs to in progress isn't hard
<lifeless> It makes my dev environment a lot easier to manage.
<lifeless> I have 'librarian' for the librarian, 'registry' for registry, 'oops' for oops etc
<lifeless> sometimes I have bug-X branches when I have multiple things in flight : but the whole kanban + RFWTAD workflow is about removing the need for parallel-tasking.
<Ursinha> lifeless, what are you going to do about the fix committed bugs linked to your branches, when you land a new fix?
<lifeless> Ursinha: I don't understand the question.
<lifeless> if its fix committed there is no more work to do on the bug.
<lifeless> landing a new fix suggests its not fix committed.
<Ursinha> lifeless, that's not what I see following bugmail
<lifeless> thats a contradiction
<lifeless> Ursinha: can you point me at some examples?
<Ursinha> lifeless, I'd have to do some gardening in my bugmail
<Ursinha> lifeless, we can try that. the idea is to make ec2 land ignore fix committed bugs?
<Ursinha> or error on them?
 * wallyworld_ off to doctor appointment
<lifeless> Ursinha: ignore fix(committed|released) bugs
<Ursinha> lifeless, one case that came to mind now is, bug fix released but not really fixed. not tagged qa-bad or -needstesting, but has new fix
<Ursinha> what to do in this case?
<Ursinha> I saw that happen a few times after releases
<lifeless> so, something has been made better
<lifeless> but not good enough
<lifeless> with the QA workflow using bugs to permit commits to trunk to be deployed.
<lifeless> we need a new bug for the QA workflow.
<lifeless> don't we?
<Ursinha> not sure what you mean
<lifeless> so in this scenario:
<Ursinha> I guess ec2 land should check all bugs, see only the !fix committed/released and if no bugs left, error
<lifeless>  - bug X
<lifeless>  - branch lands that 'fixes X'
<lifeless> we QA it - bugX [qa-ok]
<lifeless> we deploy
<lifeless>  - bugx [FixReleased]
<lifeless> then we realise its still timingout (for instance)
<lifeless> thats your scenario, right ?
<Ursinha> right
<lifeless> now, there are two places we might find this
<lifeless> firstly, we might notice in QA
<lifeless> and secondly we might notice after deploy
<lifeless> if we notice in QA, because its 'better' we don't need to stop the deploy.
<lifeless> so any solution to this needs to cater for noticing in QA.
<lifeless> if we notice in QA, we could se the bug to qa-incremental (is that right?)
<Ursinha> hm
<lifeless> so In-progress status, and 'branch is ok to land'
<Ursinha> if we notice in qa, so it's qa-untestable
<Ursinha> or qa-bad if devel thinks the fix is going to bork prod. if rolled out
<lifeless> https://dev.launchpad.net/QAProcessContinuousRollouts#We can QA the branch, and it is an incremental step towards the fix of one or more bugs
<Ursinha> lifeless, but only if you landed the first fix as --incr
<Ursinha> if you know previously that the fix might not be the last one, than land it as incremental and all of that will be done automatically
<lifeless> Ursinha: So I guess I'm saying 'what you describe is us realising that the fix *was* incr, even if we didn't *say it was*'
<Ursinha> bug in progress, qa-untestable
<Ursinha> yes, I see that
<lifeless> so the right thing to do, whether we notice in QA, is to set the bug status in the same way.
<Ursinha> I'm saying there's room for tweaking things :)
<lifeless> yeah
<lifeless> and so if we set th ebug status the same way
<Ursinha> qa-untestable, in that case
<Ursinha> otherwise we'll be blocked
<lifeless> we'll set it to in-progress, qa-untestable(or-qa-ok I guess for the testable-incremental-case)
<Ursinha> doesn't matter for the script, both are "go-for-it"
<Ursinha> I like qa-ok best
<lifeless> right
<lifeless> and ec2 land would correctly *include* this bug in the later landing
<lifeless> because its in-progress
<Ursinha> right
<lifeless> that seems to work to me
<Ursinha> lifeless, right. I'll update the wiki page and let you know
<lifeless> Ursinha: thanks!
<Ursinha> I don't like the way the theme in dev.lp.net separates the sections of the page
<Ursinha> it's kinda hard to read
<lifeless> yeah
<lifeless> its rather awkward
<lifeless> mbarnett: nevermind,m just BB flakiness
<mars> lifeless, reading backscroll, looking sadly upon the BB waterfall - did you already pass the BB restart work along?
<lifeless> mars: restart work ?
<mars> lifeless, re: your "BB flakiness" comment
<lifeless> mars: well I haven't debugged deeply, for clarity
<mbarnett> lifeless: hehe..  kk
<lifeless> mars: but the most recent build was for an older rev
<lifeless> mars: so it hadn't tried tip of prod-devel
<lifeless> (and it was nearly 12 hours old that the fix landed in prod-devel)
<mars> lifeless, looking at the waterfall BB is completely hosed right now :(
<mars> so I need to figure out what to tackle first
<lifeless> mars: the machine gun ?
<lifeless> mars: parhaps bring out the 'restart the world' card?
<mars> lifeless, we used that card earlier today - I am worried
<lifeless> oh
<lifeless> that is a concern
<mars> and lp and db_lp have been down for a week
<lifeless> we has several CPs pending
<mars> :(
<lifeless> cannot rename, ubuntu bug uploads, and OOPS generation
<mars> ok, so we need to get lucid_lp and prod up first
<mars> mbarnett, I forced the lucid_lp build.  If the build does not start in, say, 10 minutes, you will have to restart the build slave.
<mbarnett> mars: kk
<mars> ok, prod_lp is offline for some reason (is it an EC2 slave?)
<lifeless> the log shows 'substantiating'
<lifeless> so I'd say yes
<mars> checking master.cfg
<mars> yes, EC2Latent
<mars> lifeless, you said prod_lp pulled a stale tip revision for it's test run?
<lifeless> mars: no, I said the last run in 'recent builds' was ages ago and for what is now obsolete
<lifeless> mars: and that it hasn't has a more recent run which would get a better tip
<lifeless> I hypothesised that it hadn't detected it
<mars> ok, I'll force a build then
<lifeless> alternatively, if the slave doesn't come up, I bet bb doesn't report that as a failed run.
<lifeless> mars: I forced a build
<mars> it is still offline
<lifeless> see 23:18:22 in the waterfall
<mars> ugh
<mars> have to restart the build master then
<mars> mbarnett, could you please restart the build master?  That should get the prod_lp builder running again
<mars> lifeless, EC2 build slaves need a master restart in order to bring them back up (lp, db_lp, prod_lp).
<mars> I haven't seen this problem before, but "restart the world" sounds right
<mars> use the unstoppable super weapons first
<mars> always the unstoppable super weapons first
<mbarnett> build master has been restarted
<lifeless> where's the earth shattering kaboom
<lifeless> there's meant to be an earth shattering kaboom
<lifeless> </marvin>
<mars> heh
<mbarnett> does not appear to be starting back up happily
 * mars mashes F5 a few more times in desperation
<mars> j/k, that actually speeds server death in resource-exhausted environs :)
<mars> mbarnett, do you have a log I could look at please?
<mbarnett> mars: sure, give me a couple
<mars> k
#launchpad-dev 2010-09-15
<mars> mbarnett, any word on the buildbot stuff?  I'm way past EOD here, but I don't want to leave the whole system offline like this
<lifeless> perhaps spm has the torch now
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.10 | PQM is open for business | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
 * spm hates at buildbot, reiterates plea from the heart and declares success in getting the PoS back
<mars> thanks spm
<spm> mars: np; it will need any 'was happening' builds to be stabbed tho. requerued? whatever it's called. forced?
<mars> spm, could you please restart the lucid_lp and lucid_db_lp build slaves too?  I don't want to have to return to this tonight
<mars> after that I will force everything to rebuild
<spm> mars: you can do all of that yourself
<spm> the lucid slaves have to be killed as part of the recovery
<mars> oh, good
<spm> basically you *must* shutdown everything BB related. kill/stab/burn with fire. then bring up the lucid slaves; then the master. repeat on restarting the master till the *(%(%^*&%^&*$%&*$%ing thing works.
<mars> spm, so your procedures now include the steps to preven process trash from half-done builds lying around?
<mars> ok
<mars> so 'yes'
<spm> mars: that's the burn with fire part :-)
<mars> \o/
<lifeless> sinzui: still around?
<lifeless> sinzui: how does class="listing sortable" interact with batching?
<mars> all builds forced
<mwhudson> lifeless: poorly, because sortable just sorts the current batch
<mars> spm, lifeless, if the prod_lp slave dies again, we'll have to do a full restart and force all the builders again.  It's the only way.
<lifeless> mwhudson: is it tolerable? (for +assignments)
<lifeless> mars: thanks
<mars> aw FL*(#
<spm> mars: orsum
<mwhudson> lifeless: no idea really, it's not great
<mars> lifeless, spm, so, about that "if prod_lp" dies thing - surprise!  Guess what just happened...
<lifeless> mars: raarh
<mwhudson> substantiate failed!
<mwhudson> quickly too
<mars> ugh, all the EC2 builders died
<spm> mars: I'll need a few more clues? ;-)
<lifeless> hammer
<lifeless> tongs
<mwhudson> maybe aws is having a bad day
<lifeless> apply
<mars> spm https://lpbuildbot.canonical.com/waterfall
<spm> mars try a second force - I did terminate the (old) ec2 instances that were running. so the master may have belatedly got poor signals?
<mars> need to go afk, crying babes in arms, back as soon as possible
<mbarnett> mars: sorry for dropping you there
 * spm forces a build on prod_lp
<wgrant> After the last rollout, we have a few users who need their multiple accounts merged.
<wgrant> Except they now only have access to the new account -- the one they don't want to keep.
<wgrant> Should I ask them to file a question so an admin can merge them the right way?
<poolie> jam: 'from twisted.internet import defer; defer.setDebugging(True)'
<lifeless> mwhudson: are there tests of specification template rendering ?
<mwhudson> lifeless: mostly only in the page tests
<mwhudson> akaik
<lifeless> mwhudson: where would they be? the .txt files i found had no browser_ stuff
<lifeless> mwhudson: skip that, time I learnt something new
<lifeless> mwhudson: whats the preferred way to render a template
<lifeless> I want to check that it gets batched in unittest style code.
<mwhudson> lifeless: using testbrowser is probably easiest
<lifeless> self.getUserBrowser() ?
<mwhudson> otherwise, create_initialized_view() and call .render() i think?
<mwhudson> yeah
<lifeless> ah, .render is probably cleaner
<lifeless> less layers for what I'm interested in.
<mwhudson> yeah, probably
<lifeless> ahhh stories/standalone is where stuff was.
<lifeless> nvm I'm deep into doing it right :)
<wgrant> spm: Do you have a moment to do a quick account merge? https://answers.edge.launchpad.net/launchpad/+question/125444 -- he's been unable to log in since the rollout last week.,
<spm> not really, but will do.
<wgrant> Heh, thanks.
<spm> ahh. thanks! you've done the research. much appreciated!
<spm> is done
<wgrant> Yeah, that's why it was really quick.
<wgrant> Thanks.
<wgrant> abentley: Why do we not permit arbitrary code execution in the tree build step?
<wgrant> Some seem to think that it is because of security.
<wgrant> But that cannot be the case.
<poolie> rockstar: hi?
<lifeless> wgrant: why can't it be the case?
<wgrant> lifeless: Because the chroot doesn't provide security.
<wgrant> So doing something outside the chroot can't be a security vulnerability.
<lifeless>  Hard / Soft  Page ID
<lifeless>      229 / 1575  ScopedCollection:CollectionResource:#message-page-resource
<lifeless>       91 /  255  BugTask:+index
<lifeless>       67 /   34  DistributionSourcePackage:+filebug
<lifeless>       52 /    7  Distribution:+search
<lifeless>       48 /  231  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<wgrant> What is message-page-resource?
<wgrant> I haven't seen that before.
<lifeless> I'm looking ATM
<lifeless> https://api.launchpad.net/1.0/bugs/136469/messages
<_mup_> Bug #136469: toshiba p100 series dsdt acpi error no sound, works with acpi turned off. <linux (Ubuntu):Fix Released by ubuntu-audio> <linux-source-2.6.22 (Ubuntu):Won't Fix by ubuntu-audio> <https://launchpad.net/bugs/136469>
<lifeless> ws.start=75&ws.size=75
<lifeless> SQL time: 4344 ms
<lifeless> Non-sql time: 10882 ms
<lifeless> Total time: 15226 ms
<lifeless> Statement Count: 288
<wgrant> Somehow I doubt it.
<lifeless> ?
<rockstar> poolie, hi.
<wgrant> That much non-SQL time for so few objects?
<poolie> hi rockstar, i was just going to ask if you could pair with jam on getting his bzr-lp forking server finished
<lifeless> wgrant: welcome to bad-O, or thread-thrashing, or a few things
<poolie> and on reviewing what he's done so far
<wgrant> lifeless: I meant that it was probably not bad code, but thread-thrashing, yes.
<lifeless> wgrant: its -so- frequent its not going to be a passive victim of worse pages
<poolie> i think you may have been there for some discussion of it at the epic
<rockstar> poolie, I might not be the best guy to pair with him.  I'm mostly a frontend guy recently.
<rockstar> poolie, in fact, I don't know much about the forking stuff at all.
<jam> rockstar: the big advantage is that you are in my timezone :)
<rockstar> poolie, but if you think I might be of help to him, I'm happy to do it.
<rockstar> jam, :)
<rockstar> jam, abentley might be better.  He's paired with jml on parts of that code at least.
<jam> rockstar: I've got a lot of small questions
<poolie> abentley could also be great
<rockstar> jam, okay.  Shoot.  I might be able to answer a few of them at least.
<lifeless> who knows about how deferred mail works (for bugs, mp's etc etc)
<lifeless> questions need this done to them pretty soon
<lifeless> but I don't want to do it arbitrarily differently.
<wgrant> lifeless: Haven't we established that we really don't want to follow the Bugs method?
<lifeless> wgrant: policy of naming, no. Method of deferring - I haven't considered.
<lifeless> wgrant: tell me about it.
<wgrant> Bugs and Code do it differently.
<lifeless> great. Please be informing your humble TA
 * lifeless grovels and drools a little
<wgrant> Bugs changes create rows in BugNotification, with a BugNotificationRecipient for each (direct or indirect) subscriber.
<wgrant> send-bug-notifications.py then processes those every 5 minutes or so, I believe.
<wgrant> MPs only started deferred sending recently... and I think they use BranchMergeProposalJob for that, but I haven't looked closely yet.
<jam> rockstar: I'm mostly watching a movie with my son right now, but I'll chat with you a bit tomorrow if you're around
<rockstar> jam, okay.  Maybe abentley and I can combine our powers.
<lifeless> mtaylor: oh hai
<lifeless> spm: I can has profile?
<spm> damn. just 5 secs too slow on heading too lunch ;-)
<lifeless> rotfl
<lifeless> har
<lifeless> yes? no? maybe?
<spm> oh, the yes was iomplied, just logging in
<lifeless> if you like, once its enabled, I'll keep poking at it while you go to lunch
<spm> sure
<spm> is restarting....
<lifeless> wgrant: https://api.staging.launchpad.net/1.0/bugs/1/messages?ws.start=150&ws.size=75 - da boom -
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
<lifeless> spm: whats staging load like ?
<lifeless> (can you kick it low before going)
<spm> lifeless: 4ish, kickin'
<spm> lifeless: and dropping. should be good to go
<lifeless> thank thee kindly
<lifeless> grah
<lifeless> no
<lifeless> ++profile doesn't work on the API
<lifeless> bug filing time
<lifeless> spm: turn it back off thanks, and shoo fo rlunch
<lifeless> spm: (when you get back is fine, no rush from my POV)
<spm> lifeless: is done
<lifeless> wgrant: https://bugs.edge.launchpad.net/malone/+bug/497386 for your skepticsm
<_mup_> Bug #497386: ScopedCollection:CollectionResource:#message-page-resource timeouts for bugs with many messages <api> <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/497386>
<wgrant> lifeless: Am I missing something, or can (1) be done trivially with a DRS?
<lifeless> wgrant: with a rank() function probably, as long as someone unwinds the login in indexed_messages appropriately
<lifeless> wgrant: (thats yes)
<lifeless> possibly better to just store in the table though
<lifeless> wgrant: more data in it now
<lifeless> hah
<lifeless> indexed_messages is whats exported
<lifeless> I bet its calculating one of the links in it that is the *actual* bottleneck today
<lifeless> hmm, I wonder how to trigger jsonification of an object for a test
<lifeless> wgrant: have you perchance poked at that ?
<wgrant> lifeless: No, sorry.
<james_w> lifeless: jsonified by the lazr.restful machinery?
<james_w> There are methods to make webservice requests, but I've never seen a test that jsonifies something in Launchpad
<james_w> looking at the lazr.restful doctests might give a hint
<lifeless> james_w: yeah
<lifeless> wondering how close to the issue I can get my test was all
<james_w> lifeless: representation = simplejson.dumps(self, cls=ResourceJSONEncoder)
<james_w> something like that might be close, but isn't everything the lazr.restful does
<lifeless> yeah
<lifeless> I'm just going to test the larger case - the key one - adding messages shouldn't increase queries.
<lifeless> once I get out of this ubuntu-bugs quagmire with our latest enthusiastic spammer
<james_w> otherwise you need to get an EntryResource for the object, and call _representation on it apparently
<james_w> and you need a request to do that
<james_w> yeah, the simplejson.dumps should be close, but bypasses all the lazr.restful machinery, so it depends where you want to test
<james_w> well, not all, but lots
<lifeless> need the machinery :P
<james_w> lifeless: you need the cache and things, not just the serialisation?
<lifeless> I need to reproduce what the appserver does
<lifeless> otherwise the test isn't complete.
<lifeless> anyhow, I'm good.
<james_w> well, you don't run the test via apache :-P
<StevenK> I've added a new model to soyuz in db-devel, and now that I'm getting around to using it, the security proxy is hiding all of it. How can I check/debug/fix that?
<lifeless> james_w: yes, but thats outside the scope that can trigger more db calls
<lifeless> StevenK: interfaces
<StevenK> lifeless: I added that too
<lifeless> and enabled it via zcml ?
<StevenK> lifeless: I certainly added it to the zcml, I'm not sure if it was right.
<StevenK> lifeless: I can dig up the MP if you want to take a squiz
<lifeless> StevenK: perhaps you should describe the symptoms more ;)
<lifeless> like 'in a test xxytyzz'
<StevenK> lifeless: I've added a new method to lp.registry.interfaces.IDistroSeries called deriveDistroSeries(), and in the model code it's calling job = getUtility(IInitialiseDistroSeriesJobSource).create(child), and whenever I try and access any attribute on job, I get denied.
<lifeless> is job the right type?
<StevenK> If I remove the security proxy to check, yes
<lifeless> check the security you've permitted for the attributes is appropriate
<lifeless> also getUtility(IInitialiseDistroSeriesJobSource).create(child) seems awfully ugly
<StevenK> lifeless: How do I check the security? My zcml-foo is fairly non-existant
<poolie> oh lifeless i meant to say: what a nice tuesday mail
<poolie> here, have a peanut :)
<poolie> (in the sense of gallery, not monkey)
<StevenK> Bwaha
<StevenK> Is there any easy way to introspect the name of a variable?
<poolie> look at it
<poolie> where are you starting from?
<poolie> StevenK: if you just have an object reference in python you can't directly tell what variable holds it
<poolie> however, you can look at locals() and globals() and walk through the stack
<lifeless> poolie: thanks
<StevenK> poolie: I'm looping over a list of variables and croaking when they're not set, I was wondering if I could get at the variable name easily for a nice error message
<poolie> pastebin the code?
<lifeless> StevenK: paste the code
<lifeless> lol
<poolie> :)
<poolie> pics or gtfo
<lifeless> poolie: was there anything particular about the tuesday may that made it nice?
<poolie> i meant actually the mail about it
<poolie> and only indirectly the content
<poolie> but it sounds like it was fun
<poolie> i liked the "today i learned about $x"
<poolie> it suggests other people could possibly learn about things too :)
<StevenK> http://paste.ubuntu.com/493988/
<poolie> oh it's going to say "None can't be unset"? :)
<poolie> so you could do 'if not locals().get(param_name)' and list the things as quoted strings
<poolie> i don't know if that's exactly idiomatic
<poolie> alternatively just 'assert displayname;assert summary;...'
<StevenK> poolie: Exactly
<poolie> which will give a clear but less pretty error
<poolie> (and it wouldn't be allowed in bzr code, which bans the 'assert' statement because it can be turned off)
<lifeless> so
<lifeless> I'd define a tested helper in lp.services
<poolie> also i would do raise DerivationError(....
<poolie> not the old syntax
<lifeless> and do
<lifeless> something useful with it :)
<lifeless> the args are available on the stack too ;)
<poolie> one could have a decorator
<lifeless> I think we may already have one
<poolie> @require_nonnull_parameters(['breakfast', 'lunch'])
<lifeless> the apis stuff has some declaritive foo here
<StevenK> Yes, but the arguments are only required in one circumstance
<lifeless> which circumstance is that?
<spiv> argvalues = inspect.getargvalues(inspect.currentframe()); for arg in argvalues.args: if argvalues.locals.get(arg) is None: raise ...  # ;)
<lifeless> jtv: hey, so hows the template page in edge ?
<jtv> lifeless: works fine
<lifeless> time to render?
<jtv> found it surprisingly fast on staging earlier
<StevenK> spiv: Hold on, I'm dabbing the blood from my eyes :-)
<jtv> lifeless: pretty decent; still slow but probably from a part of the process that we don't watch as closely such as transfer to the client
<jtv> Haven't seen any timeouts at all on edge; I just managed to trigger one on staging though.
<jtv> I'm guessing on staging it depends a lot on load.
<spiv> StevenK: oh, if you want blood then I'm sure I can find uglier ways to do that... sys._getframe().f_code.co_* springs to mind.
<jtv> Unfortunately chromium's "view source" breaks a lot on this page
<jtv> spiv: doesn't fit as neatly into the meter as "you got it" though
<StevenK> spiv: Bah :-)
<jtv> If You Want Bloodâ¦ sys._getgrame().f_code.co_*
<jtv> lifeless: just got a render time of 7 seconds.  There's other things the new code will let us do if it's too much.
<jtv> lifeless: in its current basic structure we could bring the critical loop body down to a single "%" operation.
<lifeless> jtv: jtv check the render comments to evaluate time, for now :)
<lifeless> once we're down to a second of render time we can worry about transfer time
<lifeless> jtv: oh, I see chrome fail
<jtv> lifeless: just got one rendered in 2.33 seconds, so I think rendering time is close to losing its position as the dominant time sink.
<jtv> Funny: looks as if firefox is actually faster than chromium for this page
<lifeless> jtv: thats great
<jtv> A bit disappointing to see so much variability.  It may be because we've become entirely bottlenecked on appserver CPU.
<jtv> At least, I think we have; I'll know more when my oops shows up.
<jtv> But as this becomes the norm for "performance-interesting" pages, we may have to look more critically at the GIL.
<lifeless> I've filed a bug and RT to experiment on this
<jtv> Ah, there's my informational oops for +templates: 3885ms, of which 206ms are spent blocked.
<lifeless> how many rows?
<jtv> Just over 2K.
<jtv> All the queries are at the head and the bottom, with a big chunk of rendering time for the table in the middle.
<jtv> The Big Query takes 86ms.
<lifeless> so 2K - probably 666ms in storm
<lifeless> on an idle machine.
<lifeless> more or less depending on row width
<StevenK> lifeless: O hai, did you miss my question?
<lifeless> question?
<StevenK> < StevenK> lifeless: How do I check the security? My zcml-foo is fairly non-existant
<lifeless> about the same as mine I suspect
<StevenK> lifeless: Bugger :-)
<lifeless> stub: how long till 8.4 ?
<stub> The packages are on sourcherry so I want to do staging today.
<lifeless> awesome
<lifeless> I want to use rank() to get message indices
<lifeless> should shave a tonne of time off of bug 1.
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
<stub> lifeless: tsearch2 rank()? Should be in 8.3.
<lifeless> stub: window function rank()
<lifeless> rank() OVER (..)
<lifeless> spm: hi
<spm> yo
<lifeless> spm: production rollout - buildbot borkage.
<lifeless> https://lpbuildbot.canonical.com/builders/prod_lp/builds/98
<lifeless> it grabbed rev 9762
<lifeless> I see 9765 as tip of production-devel.
<lifeless> we want ot remove the cowboy and get stuff out there. Dunno whats going on.
<lifeless> However, I has to run - family time.
<spm> beyond bb being a pos as in?
<lifeless> yes exactly
<lifeless> why isn't it getting the tip of production-devel
<lifeless> https://lpbuildbot.canonical.com/builders/prod_lp/builds/98/steps/bzr/logs/stdio
<lifeless>  argv: ['/usr/bin/bzr', 'checkout', '--revision', '9762', 'bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-devel', 'build']
<lifeless> so its explicitly asking for an old rev
<spm> that... is seriously weird
<lifeless> which is the nuts.
<lifeless> spm: can you take it on, and hand it off to someone else when you EOD?
<lifeless> we need a build of 9765 to do all three CPs
<spm> probably not. Tom's not around. I'll do what I can, but no promises.
<lifeless> and remove the thing, cowboy
<lifeless> I'll check in in ~ 4 hours I guess.
<spm> Oh I think I may know what did that.
<stub> So does everything explode if people push --overwrite a branch that is linked to builds etc.?
<spm> I used the 'restart with last failed build' juju in an attempt to encourge the pos to work. that'd do it.
<spm> builds forced, see if that grabs the latest
<lifeless> bbiab
<adeuring> good morning
<mrevell> Hey up everyone.
<jml> hello all
<lifeless> hello
<jml> lifeless, you broke prod_lp again :(
<lifeless> jml: no, buildbot failed
<lifeless> and failed again
<lifeless> and failed afte that
<jml> :(
<lifeless> and after that
<lifeless> so the changes, in production-devel, are still in production-devel and not in production-stable
<jamesh> you'd get less buildbot failures if you had less tests
<lifeless> jamesh: thanks for that.
<lifeless> night all
<bigjools> lifeless: wait!
<bigjools> darn too late probably.  jml, prod_devel is failing still
<bigjools> can you help please?
<jml> bigjools, I can try. On the phone.
<bigjools> me too :)
<lifeless> /usr/bin/bzr checkout --revision 9762 bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-de
<lifeless> its grabbing the wrong rev
<lifeless> (you're lucky, I am looking up the phone number for tomorows meeting)
<lifeless> the branch is fine
<lifeless> BB is confused
<lifeless> dunno why
<lifeless> I see rev 9765 in prod-devel
<lifeless> ok, water, phone, phone # all set.
<lifeless> _> flip side
<jml> I'v queued another build
<bigjools> jml: what do you mean?
<jml> bigjools, I went to "Force build" and forced a build.
<jml> building
<jml> 1 pending
<bigjools> jml: ok - do you know if that's any different to what happened at 07:43?
<bigjools> the overnight build failed too, and then someone kicked it off at that time
<bigjools> I don't have much hope that it'll work this time :/
<jml> bigjools, no, I don't. but I'm going to try it in lieu of having a better idea.
<bigjools> ok
<bigjools> I've got a rather urgent CP to get out :(
<jml> bigjools, ok. I'm off the phone. You have my full attention.
<bigjools> jml: not much we can do until the current run finishes
<bigjools> we can see if the next one checks out the right revno
<bigjools> if not, then I'm getting on the batphone
<jml> I wonder where it gets the revno from
<bigjools> why does it even need to know
<jml> bigjools, I'm guessing something about race conditions.
<jml> bigjools, or perhaps some secondary approval step from the losas?
<jml> good grief. every thing is failing for different reasons.
<bigjools> ew, it takes 45 minutes from buildbot instantiation until it actually starts running tests
<jml> yes.
<deryck> Morning, all.
<bigjools> hi deryck
<bigjools> jml: it got the right revno this time
<bigjools> only 4 hours to wait now... :/
<jml> bigjools, yeah I see it :)
<wgrant> Launchpad's misrepresentation of partner makes me sad :(
<wgrant> And makes RMS FUD harder to debate.
<jml> wgrant, which aspect in particular
<wgrant> jml: It says that Skype and Flash are in Ubuntu.
<wgrant> Which makes it difficult to tell RMS that he's wrong when he says they are :P
<beuno> oh, everything is difficult with RMS anyway...   :)
<wgrant> It is...
<jml> wgrant, I can see why that would be frustrating.
<jml> wgrant, otoh, I'm increasingly of the opinion that we should focus our efforts toward helping the people who are using Launchpad to try to get things done.
<wgrant> Probably, yes.
 * gmb -> breaking to escape brain-stall. 
<deryck> gmb, when you're back, could you look at bug 638284?
<_mup_> Bug #638284: Launchpad's Remote Tracker does not honor Debian Bug Tracker status changes <Launchpad Bugs:New> <https://launchpad.net/bugs/638284>
<deryck> I believe this is the debbugs sync disabled due to disk space issue, but would like you to confirm.
<gmb> deryck, You're right.
<gmb> deryck, Perhaps we should use that bug as a way of keeping users up-to-date with the problem. Thoughts?
<deryck> gmb, yes, definitely.  If we don't already have a bug about it, then let's use that one.
<gmb> deryck, I just looked and don't see one. I think that's mostly because we've been fielding Questions, IRC pings and emails about it but no bugs until now.
<gmb> And I know that the LOSAs have an RT for the problem.
<deryck> gmb, ok, cool.  Do you mind updating the bug and doing this?
<gmb> deryck, Done.
<bac> abentley, adeuring, bac, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jtv, bigjools, leonard, mars, noodles775: reviewer meeting starting
<deryck> awesome!
<jml> sinzui, where's the list of bugs remaining for the bridging-the-gap work that registry is finishing off?
<sinzui> https://bugs.edge.launchpad.net/launchpad-registry/+bugs?field.tag=bridging-the-gap
<jml> sinzui, cunning. thanks.
<jml> sinzui, is there a bug filed for the bad tooltip text for the branch configuration link?
<sinzui> yes, your incomplete one
<sinzui> jml, I cannot reproduce it. The description of what is in play (rendering engine + os) make it unlikely to be an issue with our code
<jml> sinzui, I don't mean the corruption thing, I mean the fact that the text says "Set branch for this series"
<sinzui> oh, sorry
<sinzui> let me look.
 * jml is sorely tempted to move LEP mgmt to blueprints.
<bigjools> I think that's a good idea
<bigjools> it would give a needed thrust to improving blueprints
<sinzui> jml, I tried that earlier this year, there is still a lot of blueprint update by hand because the statuses are more than any project needs and notifications are broken
<sinzui> You cannot use feedback requests for example...you still need to send the feedback request yourself
<deryck> jml, and then we should merge blueprints and bugs. ;)
<jml> deryck, *yes*
<sinzui> jml: I do not see a bug for the tooltip. The bug is in an Involvement Menu subclass and is trivial
<sinzui> deryck, and answers.
<jml> sinzui, ok. I'll file it anyway, if you don't mind.
<deryck> sinzui, woah there.  Not sure I'm feeling that one. ;) :)
<sinzui> please do, my team are asking for some easy work since the app UI is very hard
<sinzui> deryck, sure you do, less timeouts converting a bug to question.
<deryck> heh
<deryck> well that is persuasive
<sinzui> No more bug-question links and more portlet to show it
<deryck> I just think we want to work at moving the support requests out of bugs, not making it easier to misunderstand the two.
<deryck> gmb or allenap, bug 5786 is no longer relevant, correct?  NEEDSINFO is not referring to upstream status, but to an old bugtask state I take it?
<_mup_> Bug #5786: NeedsInfo should be moved into a separate bug or bugtask flag <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/5786>
<gmb> deryck, I think so, yes. In which case it was superseded by Incomplete aeons ago.
<deryck> right
<jml> sinzui, we talked about https://dev.launchpad.net/Registry/RegistryKarma a while back. What did we decide?
<sinzui> We will do nothing but continue to hope for a replacement for karma
<jml> sinzui, excellent. I'll make a note.
 * sinzui has talked to 2 users this month to explain karma is not a measure of value or work
<jml> sinzui, what about this one? https://dev.launchpad.net/Registry/EssentialSourcePackageInformation
<sinzui> That is done!
<jml> sinzui, ooh. I'll review then.
<jml> sinzui, the LEP says that https://edge.launchpad.net/ubuntu/+source/wsjt should have a link to copyright information. I can't see a link like that.
<sinzui> jml, that page should not, the series source package does...they change by series
<sinzui> jml: the SP descriptions will be used in the derivative distros diff page. the copyright is being used in register an upstream project
<jml> sinzui, sweet!
<deryck> adeuring, hi.  I'm linking your current kanban board card against bug 633698.  This bug is good for this issue you're seeing?
<_mup_> Bug #633698: missing API oops reports? <Launchpad Bugs:New> <https://launchpad.net/bugs/633698>
 * sinzui wonders if it is appropriate to quote his mistypes: "The involvement
<sinzui> portlet is controlled by the pillar owners and its links are enabled to
<sinzui> attack contributors to the applications the pillar uses."
<adeuring> deryck: sure
<sinzui> I think I meant to type attract, but maybe subconsciously i did mean attack
<jml> heh heh
<deryck> A little more clearly described now:  bug 633698
<_mup_> Bug #633698: Librarian errors missing API oops reports <librarian> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/633698>
<deryck> It seems people are either ignoring results suggested by dupe finder or it's become less useful with the recent preformance work.
<jml> bigjools, are package dependencies available over the API?
<bigjools> jml: do you mean as properties of source packages?
<jml> bigjools, Yes.
<bigjools> jml: no, we don't export those
<jml> bigjools, I mean, if they're available in another way that would be interesting too.
<jml> bigjools, any reason or just round tuits?
<bigjools> jml: we don't export source packages at all, just their publications.
<jml> deryck, I think it has become less useful.
<bigjools> deryck: I hardly ever see people paying attention to it
<jml> deryck, but a third hypothesis is more bugs filed via email.
<jml> bigjools, is there a reason for that?
<bigjools> jml: yes, what is a source package's URL?
<jml> bigjools, https://launchpad.net/ubuntu/+source/packagename
<bigjools> jml: wrong! that's a distrosourcepackage
<deryck> yeah, I can tell the difference in bugs filed via email and against the project group.  I'm seeing it more commonly filed straight against malone, too.
<jml> bigjools, ubuntu/lucid/+source/packagename
<sinzui> bigjools, jml, I want to toss a small grenade into your conversation. I do not think the DSP/SP pages and API is very good. Our pages have poor rank in Google. packages.ubuntu.com always wins. When a user is searching the Google package information, they are going to the other site, then following that link to Lp :(
<jml> sinzui, interesting.
<bigjools> jml: unfortunately wrong again.  We need a way to put a sourcepackagerelease in its own URL.  But I don't think that's useful at all though.
<bigjools> bear in mind that dependencies can change between versions
<jml> bigjools, something is using the word incorrectly here.
<jml> bigjools, https://edge.launchpad.net/ubuntu/maverick/+source/wsjt perhaps
<bigjools> this is why I added a bunch of source-related properties to publications
<jml> bigjools, since it does say "source package"
<bigjools> yes, but it's not a source package release
<bigjools> it's a source package in ubuntu maverick
<jml> oh, you didn't ask for a URL for that :)
 * bigjools rolls eyes
<sinzui> jml: bigjoolsI do not have any immediate ideas, I think this conversation is registry related too. SPR is a specific detail that users often do not care about, they want the current info for their distros or series. I think we need to re-ask who uses this information and what pages do we provide for them
<bigjools> ok ok :)
<jml> bigjools, my use case here is wanting to get build dependencies so I can start working on a branch.
<bigjools> sinzui: I think probably both are valid, which is another reason why I prefer adding more properties to publishing records
<bigjools> because traversing to those from the SPR or the DSP is trivial
<jml> bigjools, so I need to be able to get from the branch's project to some kind of package record, and from there to the dependencies
<jml> bigjools, is this currently possible over the API?
<bigjools> jml: it's not :(
<sinzui> I think users arrive at an SPR too early. Searching a series takes me to the SPR. If I bookmark that page, It could be wrong in 24 hours
<bigjools> sinzui: we don't have a page for an SPR
<jml> bigjools, what would we need to do to make it possible?
<sinzui> bigjools, True
<bigjools> jml: are you looking for a package in a distroseries in this project?
<jml> (I don't really care about the web ui for this, fwiw, although I do acknowledge that it's interesting)
<bigjools> or a particular version? or something else?
<jml> bigjools, I guess I don't care & am looking for the latest in Ubuntu, since that's likeliest to be close to trunk
<bigjools> ok
<bigjools> jml: are you looking at build dependencies or installation dependencies?
<jml> bigjools, both. but for this use case, let's focus on build.
<bigjools> then I'd do this:
<bigjools> add the build deps as a method on sourcepackagepublishinghistory
<bigjools> then use distrosourcepackage.currentrelease or similar to get the publication
<bigjools> but you need to export all that on the API
<jml> bigjools, are DSP and SPPH already exported?
<bigjools> actually, scratch that latter, use distribution.getCurrentSourceReleases()
<bigjools> which takes a string package name
<bigjools> SPPH is exported
<bigjools> distro is exported
<jml> OK, so it's just a simple matter of programming then.
<bigjools> yep, hopefully
<jml> bigjools, thanks
<bigjools> jml: oh actually, feh, getCurrentSourceReleases() returns DSPs which are not exported.  You should use IArchive.getPublishedSources
<bigjools> which returns SPPHs
<jml> bigjools, ah. fair enough.
<bigjools> getting the archive is trivial
<bigjools> distro.main_archive for example
<jml> bigjools, from time to time the thought occurs to me, perhaps soyuz's apis could be simpler.
<bigjools> jml: hellyes.  however, we're constrained by the decision to expose the internal model
<bigjools> but if you have any ideas on how to make it better, I am all ears
<jml> by "api" I meant internal model, actually.
<jml> no ideas right now :)
<jml> I'll let you know if any come up :)
<bigjools> jml: I get frustrated with the model too.
<bigjools> do you remember my presentation at the original epic?
<bigjools> and the db diagram?
<jml> bigjools, yeah, I do.
<bigjools> jml: recipe builds made that twice as bad :)
<henninge> danilos: ping
<danilos> henninge, hi
<danilos> henninge, sorry, was otp
<henninge> np
<henninge> danilos:  what is the meaning of the "from_upstream" attribute of a queue entry nowadays?
<henninge> used to be "is_published"
<danilos> henninge, ah, I think it needs to change (I know we changed it already)... it's not really from_upstream, I think it should be "from_package"
<danilos> henninge, though, then again, we can keep it as from_upstream to indicate what side  it's coming on
<henninge> danilos: to determine if an import is from upstream, the potemplate can be queried for productseries/distroseries.
<henninge> that would make from_upstream a computed attribute
<danilos> henninge, except in sourcepackage uploads, which are from_upstream, basically
<henninge> ???
<danilos> henninge, well, translations we import on ubuntu from source packages are upstream translations
<danilos> henninge, they are just from tarballs instead of from a VCS
<henninge> danilos: I am not sure I understand
<henninge> danilos: will they be imported into upstream templates or ubuntu templates?
<danilos> henninge, uhm, with the new model, into both
<danilos> henninge, but they will be considered is_current_upstream
<henninge> danilos: ah!
<henninge> danilos: so this is another parameter to the "share with other side" policy
<danilos> henninge, well, kind of... the good end result will depend on that policy being right :)
<henninge> danilos: "If the import is for a sourcepacakge but is marked as 'from_upstream', share it with upstream"
<danilos> henninge, no, "if import is for a sourcepackage but marked as 'from_upstream', consider it an 'upstream' translation"
<henninge> but that is more difficult for the importer
<danilos> henninge, then, a policy of "if there's an upstream translation and it's better than ubuntu one, use that one for ubuntu as well" will give us the same result we get today
<danilos> henninge, how come? it just means go through the same code path as if it's an import for a productseries
<henninge> yes but it needs to determine the productseries first.
<henninge> from the sourcepackage
<henninge> code duplication
<danilos> henninge, it should be possible to avoid it by calling setCurrentTranslation(upstream=True) directly
<henninge> danilos: My take: when sharing translations with the other side it does not matter on which side the translations are imported
<henninge> hm, maybe this is about template creation.
<danilos> henninge, well, I think we just need to call setCurrentTranslation with appropriate parameters, we don't need to know the exact template
<danilos> henninge, because we've got potmsgset already
<henninge> on template import? not necessarily
<danilos> henninge, I mean, we've got it on "ubuntu" side, we don't need it on the upstream side as well
<danilos> henninge, by "got", I mean "just created or existing"
<danilos> henninge, we should not import templates from sourcepackages as "upstream"
<jml> bigjools, the stakeholder board says "generalized build histories" is done. is that accurate?
<danilos> henninge, basically, what will happen is that we'll internally have potmsgsets with upstream translations, but there may not even be any upstream templates in LP
<bigjools> jml: yup
<henninge> yes, I am beginning to see that
<jml> bigjools, cool! I'll review the LEP for sign-off & get back to you.
<bigjools> jml: we're just waiting on jtv to finish off his work on actually using it, and then we can go on a made delete-fest of all the old code
<bigjools> s/made/mad/
<bigjools> recipes already use it
<jml> bigjools, yay deleting code
<bigjools> it's a good feeling :)
<danilos> bigjools, most of what we could do so far was in ec2 this morning ;)
<bigjools> \o/
<jml> "could do"?
<danilos> jml, if I understood it correctly, some bits need to be moved to using the new model fully (we were blocking that), and we didn't do any of that
<jml> danilos, ahh ok.
<jml> I have a mild mistrust of anything that claims to be generalized and is only used for one type of thing.
<abentley> jml, +1.
<abentley> Does anyone know how to configure zope to treat the return value of itertools.groupby() the same way it treats generators?
<jml> abentley, I don't follow. Doesn't itertools.groupby() return an iterator?
<abentley> jml, yes it does, but the security proxy doesn't treat iterators the way it treats generators.
<jml> abentley, I'm surprised, and also desire to be crystal clear on terminology.
<jml> abentley, generator here means... ?
<abentley> jml, so if I return itertools.groupby(), I get a security proxy violation on __iter__.  If I return (x for x in itertools.groupby), everything is good.
<jml> buggeration.
 * jml checks a thing
<abentley> jml, generators are iterators produced by functions with yield statements or generator expressions.
<abentley> iterators is a superset including generators as well as hand-crafted iterator classes.  itertools.groupby appears to return the latter.
<jml> abentley, ok. I think I see what's going on.
<jml> abentley, groupby returns an object that implements the iterator protocol
<abentley> jml, correct.
<jml> abentley, I'm guessing that Zope's handling for __iter__ and what-not is done in a type-based fashion.
<jml> abentley, and that the highly specific type returned by itertools.groupby() is not included in the list of types for which zope makes this exception.
<abentley> jml, me too.  I suppose they may have associated interfaces with the built-in types, though.
<jml> abentley, that's my guess
<jml> except they've missed the built-in type called itertools.groupby
<jml> I guess now it's time to explore zope.security
<abentley> jml, well, I was hoping someone would know off the top of their head.  But I don't see gary here today.
<jml> abentley, something in zope.security.checkers._defaultCheckers
<jml> working up from there
<jml> abentley, z.s.checkers.defineChecker(itertools.groupby, z.s.checkers.NamesChecker(['next', '__iter__'])) might do the trick
<jml> not sure how to do that in zcml
<abentley> jml, thanks, I'll give that a shot.
<jml> abentley, I've just done a pass over the SPRB LEP and kicked it into "In progress"
<abentley> jml, does that reflect drafting or implementation?
<jml> abentley, implementation
<jml> abentley, I'm happy with the LEP if you are.
<jml> I haven't looked over the UI stuff.
<abentley> jml, I think it's okay.  What do you mean by Allow users to use James Westby's imports to provide the debian directory *without the user leaving the web UI*
<jml> abentley, you noticed. I mean that if the way to use the lp:ubuntu/foo imports involves switching to a terminal and making a new branch or something similar, then that's not really acceptable
<abentley> jml, I am surprised by the idea that using the lp:ubuntu/foo imports might involve switching to a terminal and making a new branch or something similar.
<abentley> jml, Wouldn't that defeat the purpose of using recipes?
<jml> abentley, pretty much :) perhaps the qualifier is redundant.
<abentley> jml, to me it suggests perhaps you want it to automatically recommend a packaging branch or something.
<abentley> jml, so that the user doesn't have to leave the recipe creation page to search for the correct branch.
<jml> oh, I see.
<jml> well that would be nice but certainly not critical.
<abentley> jml, what makes mark a stakeholder?
<jml> abentley, this is something he cares about a great deal. it's largely his vision.
<abentley> jml, does "Promptly provide full information on failed builds to someone capable of debugging the failure" mean it must spam me?  Or the packaging author?  Or the build requester?
<abentley> jml, "Builds are performed in a timely manner" is something that the build farm does not guarantee for any builds.  Why do we have this special burden?
<jml> abentley, re timely
<jml> abentley, it's something that is necessary for me to consider the feature complete. it doesn't mean at all that you will be obliged to do the work to make it so.
<jml> abentley, indeed, that work is already been done by team soyuz.
<abentley> jml, it seems like a big scope creep to me.
<jml> abentley, not to me.
<jml> abentley, it's always been _daily_ builds
<rockstar> james_w, how do you usually run the bzr-builder tests?
<james_w> rockstar: bzr selftest -s bp.builder
<abentley> jml, It has not been done.  In fact, I was supposed to work on a new build farm, and AIUI that task is now on hold.
<jml> abentley, sorry, I meant "being"
<rockstar> james_w, ah, didn't know about -s
<abentley> jml, I'm not aware of any Soyuz plans to change the basic scheduling algorithms, but until that's done, we'll have the potential for build starvation.
<bigjools> it's an extremely hard problem to solve, but I've not forgotten it
<jml> abentley, so we'll have to solve it then.
 * jml seizes this opportunity to order a book on queuing theory
<abentley> bigjools, but until someone solves it, jml won't consider the daily builds LEP complete.
<bigjools> that's crazy
<jml> bigjools, what can I say, I'm loco.
<bigjools> recipe builds are now scored the same as PPA jobs, they won't starve
<abentley> bigjools, if we lose some of our loaded builders and a bunch of higher-scored builds get created, they can.
<jml> (Â£80 is not the price of a book!)
<abentley> s/loaded/loaned
<bigjools> abentley: in theory yes, but there's only one thing that generates higher-scored builds and they are not frequent, so in practice it does not happen
<abentley> bigjools, aren't there a bunch of modifiers that can push up the score?  Even a bunch of builds with a 2010 score can starve the daily builds.
<abentley> Or was that 2510?
<jml> abentley, wrt "provide information on failed builds", out of those, I mean the build requester
<jml> abentley, errors in the system are already (I hope!) handle by OOPSes
<jml> handle*d*
<jml> abentley, or the recipe author, I guess...
<abentley> jml, I believe the build requester is required to be a member of the recipe author.
<jml> abentley, that's good. even so, maybe the broader group is more appropriate. I don't know right now.
<abentley> jml, we don't distinguish between errors in the build that are system errors vs user errors.
<abentley> jml, rather like code imports.
<jml> abentley, by design?
<abentley> jml, kinda.  Our builds are just like other builds in this regard.
<jml> it seems that, for example, any exception that is raised outside of slave is guaranteed to not be a user error.
<abentley> jml, sure.
 * jml is having a bad day with grammar.
<abentley> jml, that's why I said errors in the build.
<jml> abentley, what happens with errors in the build now?
<abentley> Once the build is completed and the slave is no longer responsible, errors can be handled as user errors or oopses.
<abentley> jml, we send an email to the user who requested the build.
<jml> abentley, so how do we learn of failures in the system?
<abentley> jml, like with code imports, we rely on users to escalate system errors as bugs.
<jml> I could have sworn we had oopses for code imports.
<abentley> jml, maybe we do.
<jml> abentley, ok.
<jml> abentley, in any case, I'll leave banging on about architectural transparency to lifeless. I mostly care that users who have a build that screws up find out about it and are given enough data to try to fix it.
<abentley> jml, anyhow, how can we in principle know whether an error is due to user error or system error?
<jml> abentley, with the puller we have some heuristics. but that approach is probably not relevant here.
<abentley> jml, can I substitute "the person who requested the build" for "someone capable of debugging the failure"?
<jml> abentley, sure.
<abentley> jml, I'm also adding "Renaming branches does not break recipes" under "nice to have".
<jml> abentley, good call.
<jml> abentley, Launchpad more broadly needs to get its act together wrt renaming stuff.
<abentley> jml, (we kind of assumed that was important, so branches are stored as DB ids)
<abentley> jml, the relevant iterator from my zope security question is itertools._grouper, and it comes from a C module and is not exposed.
<jml> you mean type(itertools.groupby([1, 2, 3]).__iter__()) == itertools._grouper?
<jml> abentley, any more questions about the LEP?
<abentley> jam, I mean type(list(itertools.groupby([1, 2, 3]).__iter__())[0][1]) == itertools._grouper
<jml> ahhh.
<jam> abentley: I think you mean jml :)
<abentley> jml, should there be any requirements about bzr branch format support?
<jml> Python sucks.
<abentley> jam, sorry.
<jam> np, I just showed up
<jml> abentley, I don't care about anything that's not 2a.
<jml> abentley, is that what you mean?
<abentley> jml, many of our supported distros don't support 2a.
<jml> abentley, why does that matter?
<jml> abentley, because the people using those distros won't be able to push up compliant branches? or because we run the bzr-builder recipe in an image based on the distro it's targeted for?
<abentley> jml, do we want to be able to build bzr recipes for old distros if said distros don't support 2a but the relevant branches are in 2a?
<jml> abentley, yes, I think so.
<abentley> jml, our initial implementation did not support that, so it's easy to imagine that an implementation would not support that unless we make it a requirement.
<jml> abentley, for sure.
<jml> "branch formats usable within recipes is independent of branch formats supported by target distro" or something.
<abentley> jml, didn't see you writing.  I put "Allow recipes to be built from bzr 2a-format branches for any supported distorseries (whether or not the distroseries itself supports 2a). "
<jml> fair enough
<jml> abentley, if bzr gets another format between now and the LEP being finished we can adjust accordingly :)
<abentley> jml, if you keep the timeliness requirement, it probably will :-P
<jml> hah
<abentley> jml, anyhow, aside from the timeliness requirement, I'm happy with the LEP as it stands.
<jml> abentley, cool.
<lifeless> moin
<abentley> jml, do we have a graph of how long it takes SRPBs to get dispatched?  I know we have one for build queue size.
<abentley> ah, lifeless is around.  Must be lunchtime.
<jml> abentley, I don't know.  I don't think so.
<abentley> jml, could you set that up, or should I ask thumperfrancis?
<jml> abentley, ask flumper.
<jml> abentley, I just recalled a use case that isn't very well documented
<jml> abentley, one big use case for recipes is having a daily build of trunk without debian/ubuntu patches.
<jml> or with as few as possible.
<jml> abentley, I'm not sure how this translates into requirements.
<abentley> jml, I don't know, either, because some patches may be necessary to build at all.
<abentley> One could perhaps fork the packaging branch, remove unwanted changes.
<jml> abentley, or make a completely new packaging branch
<abentley> Then either use that in the recipe, or use the recipe to merge the packaging branch.
<jml> abentley, and express the ubuntu one as a recipe on top of that.
<abentley> jml, you can also use nest-part to exclude changes outside the debian directory.
<rockstar> mars, ping
<jml> abentley, the difference between the "pure" packaging and the ubuntu packaging is primarily of interest for testing upstream code.
<abentley> jml, but of course, some patches will be provided within the debian directory.
<abentley> jml, it also provides a vector for upstreams who feel that ubuntu's changes are unwelcome to get a pristine version to their users.
<jml> abentley, hello.
<abentley> jml, hi.
<jml> abentley, sorry, mischan
<jml> abentley, yes. that's also a good case.
<jml> abentley, perhaps this means that we should have some kind of special marking for "pure" packaging...
<jml> something to chew on.
 * jml otp
<abentley> If you look at bzr, the ubuntu version removes configobj from the tree and depends on its package instead.
<abentley> a "pristine" bzr would have configobj in the tree, and would therefore have an unnecessary dependency.
<abentley> Unless you change the bzr packaging manually.
<lifeless> deryck: can we have a brief call after this ?
<deryck> lifeless, I have a call with thumper after this, but depending on how long it lasts, after that would be fine.
<jkakar> Does the LP schema allow a project to be in more than one project group... I know the UI doesn't allow it... but is it possible?
<lifeless> deryck: great
<lifeless> jkakar: iirc no there is a unique constraint
<jkakar> lifeless: Cool, thanks.  It wonder if enough people would benefit from projects being able to be in more than one group to make it worth changing.
<lifeless> jkakar: I want to eliminate pgs as a special case
<lifeless> use multiple tagging/categorising systems
<lifeless> e.g. tags to to bottom up associations
<jkakar> lifeless: That sounds cool.
<lifeless> and 'defined groups' to do top down defined structures
<jkakar> lifeless: What I want is a way to group projects such that I can have a +activereviews page that shows me reviews for the group.
<lifeless> and permit both to exist
<jkakar> lifeless: Also, to see bugs in the group.
<jkakar> lifeless: Also, to create milestones for the group.
<jkakar> So I guess I want the group to be identical to a project, but with less arbitrary limits on how I can form them. :)
<lifeless> yes
<lifeless> I would like the same too, in the fullness of time.
<jkakar> lifeless: Cool!
 * jml gone
<mars> sinzui, do you think a wiki page would be a better way than the ML to track Maverick upgrade issues?
<lifeless> sinzui: hi https://bugs.edge.launchpad.net/launchpad-registry/+bug/638924
<_mup_> Bug #638924: Milestone:+index timeouts with many announcements <timeout> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/638924>
<lifeless> sinzui: I've just dug a bit deeper into that timeout
<lifeless> sinzui: I drew some different conclusions about the issue, and thought perhaps you'd like to spend a little time later discussing it
<sinzui> lifeless, I would like to discuss it. I have two more meetings today, but I am free this evening and tomorrow
<lifeless> sinzui: ping me anytime
<sinzui> fab
<lifeless> sinzui: I'd like to get some hints out there for doing the analysis stuff
<lifeless> Ursinha: https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt seems to have not updated ?
<lifeless> Ursinha: also, as that rev *is* deployed, I think you're pointing it at the wrong branch ?
<Ursinha> lifeless, wait, I'm checking
<lifeless> (its meant to start at the deployed rev, not randomly far back)
<Ursinha> it seems it was updated few minutes ago
<mars> lifeless, Ursinha, the ctime on those reports says they are only 3 minutes old
<lifeless> so, there is a bug then
<lifeless> bug 11518
<_mup_> Bug #11518: Linux kernel-image-2.6.9-1-686 synaptic pass through problem hoary <Ubuntu:Invalid by fabbione> <https://launchpad.net/bugs/11518>
<lifeless> bah
<lifeless> rev 11518 has bug 627741
<_mup_> Bug #627741: oops-prune is aborting with a regex mismatch(?) <canonical-losa-lp> <qa-untestable> <Launchpad Foundations:Fix Committed by stub> <https://launchpad.net/bugs/627741>
<lifeless> thats qa-untestable
<lifeless> thats one issue
<lifeless>  the second issue is that that rev is *already deployed*
<lifeless> it should be starting much higher up
<lifeless> what branches are you giving it ?
<Ursinha> lifeless, it seems last deployed revision on lpnet was 11515
<Ursinha> script starts processing at 11516
<lifeless> hang on though
<Ursinha> which is correct
<lifeless> 'stable' is deployed to edge
<Ursinha> now for the other problem
<lifeless> db-stable is deployed to lpnet
<lifeless> actually thats a convenient lie but its good enough for now.
<lifeless> (because we need something to measure which db commits can be deployed)
<lifeless> Ursinha: what branches is it running with please ?
<Ursinha> lifeless, stable and db-stable
<lifeless> Ursinha: it takes branch pairs IIRC
<lifeless> Ursinha: source and target
<Ursinha> MergeDeployment is lpnet
<Ursinha> devel, stable and db-devel, db-stable
<Ursinha> lifeless, merge environment for both stable and db-stable is lpnet
<Ursinha> so it looks lpnet for the last merged branch
<lifeless> ok, that will make sense once we remove edge
<lifeless> for now though, the devel/stable one needs to check against 'stable'
<lifeless> Ursinha: ah yes, I see
<Ursinha> lifeless, let me see if I got the picture: people land code on devel/db-devel, and it goes do edge/staging, they QA it,  and it supposedly should go live on lpnet
<Ursinha> right?
<lifeless> and I didn't provide a knob for this.
<lifeless> ignore db- for this discussion
<lifeless> Ursinha: actually, what I'll do is ask for a CP of everything.
<lifeless> that will sort this out.
<Ursinha> not sure what you mean
<lifeless> but it would be nice to know that everything currently in stable has been QA'd, to pick a rev to CP
<lifeless> Ursinha: well a CP is a merge to production-devel
<Ursinha> lifeless, I thought this was exactly how the script is working now
<lifeless> ahh,, and I'll be able to do that if we figure out this other isue.
<Ursinha> what's on stable is what should be QAed
<lifeless> Ursinha: ok, yes, I'm being noddy.
<lifeless> Ursinha: so lets talk about the other thing :) [sorry!]
<Ursinha> hehe
<Ursinha> I'm investigating the bug thing
<lifeless> ok
<mars> rockstar, I have a reply on the way to you, but I need to answer a question about the JS build system first
<rockstar> mars, great, thanks.
<rockstar> mars, I'd like to deal with this today, as it means that I have a kanban card that's just sitting.
<Ursinha> lifeless, ok, so that was a stupid thing: qa-needstesting wasn't considered a deployable status, only qa-ok
 * Ursinha goes add tests
<lifeless> Ursinha: qa-untestable do you mean ?
<Ursinha> lifeless, argh, yes, sorry
<lifeless> qa-needstesting isn't deployable ;)
<Ursinha> lifeless, qa-untestable
<Ursinha> lifeless, https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt
<Ursinha> it didn't go too far though
<lifeless> thats ok
<lifeless> we can look into that in a minute
<rockstar> ARGH!  Stabby stabby moin!
<rockstar> thumper, when you're around, I'd like to chat about merge queues.
<deryck> Later on, everyone....
<lifeless> rockstar: we hsould just use em.
<lifeless> if bugs turn up, we can fix.
<rockstar> lifeless, :)
<rockstar> lifeless, there are some things about the current model that I think complicated things greatly.
<mars> rockstar, understook
<mars> understood even
<lifeless> rockstar: not disagreeing
<lifeless> rockstar: but its inventory today.
<rockstar> mars, wtf keyboard layout are you using that had d and k right to them?  :)
<rockstar> lifeless, yeah, inventory that I want to throw out on the streets!  :)
<rockstar> lifeless, you will see, though, that we have a whole lane in kanban for finishing up merge queues now.
<jam> rockstar: ping about bug #612754
<_mup_> Bug #612754: Submit Request Failure: Signature couldn't be verified: (7, 8, u'Bad signature') - with email signed and sent from sup-mail <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/612754>
<jam> It is blocking me pretty much on every submission
<lifeless> rockstar: a lane? wouldn't it just fit under features?
<lifeless> rockstar: anyhow, details aside...
<lifeless> rockstar: I saw your update; why one queue per project? isn't one queue per series easier conceptually?
 * benji goes afk for a bit.
<rockstar> lifeless, one queue per branch, but many branches can be in one queue.
<rockstar> lifeless, we can't just draw the line at series or anything.  Think about all the branches currently managed by ~launchpad-pqm
<rockstar> jam, EVERY submission?
<jam> rockstar: any time I send a "merge: approve"
<jam> and a lot of bug reports, too, I believe
<rockstar> jam, yeah, it would use the same mechanism.  I think might fall under "foundations" domain, but not sure.
<rockstar> jam, could you try with a different mail client?
<jam> I can obviously go to the website, but the email interface is broken for me
<thumper> jam: which email client?
<rockstar> jam, mathiaz was having intermittent problems, so I was suspecting that it was a mail client issue.
<jam> rockstar: I don't have another mail client integrated with gpg. But even so, I can check the raw texts pass, and I had Vincent do the same thing for me
<thumper> rockstar: I'm nomming
<thumper> rockstar: and I'd like to grab abentley first
<jam> thumper: thunderbird 3 w/ enigmail
<rockstar> jam, is Vincent using the same client?
<jam> rockstar: vincent uses gnus, or whatever the emacs client is
<rockstar> thumper, we have our 1-1 today, right?
<rockstar> jam, *sigh*
<abentley> thumper, certainly.
<thumper> rockstar: I guess
<jam> rockstar: you should be able to just try "gpg --verify submit_request_failure.txt" that I attached to the email
<rockstar> jam, okay.
<jam> rockstar: I just sent another email (which I expect to be blocked). I suppose I can CC/BCC you one of them?
<rockstar> jam, absolutely.
<jam> (I just BCC'd you on an artificial one, though I think that request is actually bogus, as you can't set WIP from email)
<jam> but it should still gpg verify
<rockstar> jam, so I've verified that your message was good.
<lifeless> rockstar: sure, s/series/target branch/
<lifeless> rockstar: what I mean is that we wouldn't want db-devel and devel to both use the same queue - they would starve each other
<jam> rockstar: well, at least you and I agree. Now if we can just get LP to feel the same :)
<jam> rockstar: I haven't see the messages show up on 'review' but I also haven't gotten the "Submit Request Failure" yet.
<rockstar> lifeless, I disagree that they'd starve eachother.  We're essentially doing the same thing now with pqm.
<lifeless> rockstar: pqm isn't running the tests
<lifeless> rockstar: and pqm did starve each other
<lifeless> rockstar: huge issues with that
<rockstar> lifeless, tarmac, in this case, wouldn't run the tests either.
<lifeless> rockstar: we want to make the lander run the tests again
<lifeless> rockstar: thats gary's experiment, landing in batches of 5
<rockstar> lifeless, and we can do something like that.  But at that point, we'd have to separate out what branches are on what queue.
<lifeless> is it just me
<lifeless> or is
<lifeless> FAILURE: lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling (subunit.RemotedTestCase)
<lifeless> failing for every branch ?
<lifeless> on ec2
<jelmer> I had an error like that as well
<jelmer> I figured since the error in the oops was soyuz related it was somehow my fault
<wallyworld_> lifeless: i got the same error too when trying to land a branch
<lifeless> ok
<lifeless> its probably not unrelated to my oops work
<lifeless> running just that test works
<wallyworld_> don't you hate it when that happens :-(
<lifeless> so either oops-writing is naffed
<lifeless> or get last oops is
<lifeless> mars: still here?
<wallyworld_> thumper: rockstar: abentley: skype?
<rockstar> wallyworld_, sure.
<thumper> rockstar, wallyworld_: abentley and I are chatting
<thumper> give us a few minutes to finish what we are talking about plz
<wallyworld_> thumper: np
<mwhudson> morning
 * jelmer waves
<lifeless> I'm looking in uniquefileallocator if folk are interested
<mars> lifeless, yep
<mars> lifeless, what's up?
<lifeless> mars: how clean is the oops dir in an ec2 test run
<lifeless> mars: see the list for the context
<mars> lifeless, no idea. someone could run with ec2 test with --postmortem and find out though.
<lifeless> we really should change the filenames on disk to be less magical
<lifeless> and for fun
<lifeless> running the two relevant tests together, here, works.
<lifeless> :!bin/test -vvt test_uploadprocessor.TestUploadProcessor.testOopsCreation -t lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling
<lifeless> mars: a premortem for ec2 would be nice
<lifeless> mars: e.g. set it all up, but run nothing
<james_w> you can do it with "-d" and some stepping
<james_w> I thought jml had implemented --dont-test or something
<thumper> jelmer: any idea why https://code.edge.launchpad.net/~vcs-imports/mesa/master is failing?
<lifeless> hah, I just noticed my lp vm is in sydney time.. that explains some confusion I had
<jelmer> thumper: looking
<mars> lifeless, jml was implementing that option.  Otherwise you can do it with "ec2 test -p -o '-t asdf'"
<lifeless> this really has me flummoxed
<jelmer> thumper: Looking at the history, it seems the branch was reimported from scratch recently?
<thumper> jelmer: yes, yesterday
<lifeless> i have tried deleting some oopses
<lifeless> and it correctly ignores the holes and writes to the end and finds them again
<jelmer> thumper: So I guess this can't really be a bzr-git bug that caused incorrect repo data.. I'll try locally & file a bzr-git bug.
<jelmer> thumper: From just looking at the logs I don't really have an idea what could be going wrong.
<lifeless> I've put a branch up that may help
<poolie> lifeless: so, about sending output from process-mail.py to a log file
<lifeless> theres an option to do that already apparently
<lifeless> --log-file or something
<poolie> mm, apparently it's not used
<lifeless> should just need an RT (and for someone to be monitoring the log)
<poolie> they just manually redirect it in the cron script
<lifeless> yeah
<poolie> i did look at it a few weeks ago and ISTR that --log-file may be broken or something
<poolie> imbw
<poolie> but i think i said "how does it even work?" and spm said they don't use it :/
<lifeless> its a thing that was built and not deployed
<lifeless> we should use it
<poolie> also, as i recall, someone said "we can't just send it to a file because we need to know about errors"
<poolie> maybe this was you?
<lifeless> we do need a story around that
<poolie> but i don't see how sending it to a file is much worse
<lifeless> ideally --log-file would not hide ERROR severity messages
<lifeless> if you wanted to pull on this yak I think that would be great.
<poolie> 'i am in yaks stepped in so far that should i wade no more, returning were as tedious as to go over'
<poolie> so i propose
<poolie> - for now, just sending an rt asking them to send it to a file
<poolie> if they object, we'll deal with that, but i don't see how it should make things any worse
<poolie> - filing a bug about splitting >=error to stderr and everything to a file
<poolie> (which will necessitate actually using --log-file not just >&file)
<lifeless> seems fine to me if you add in
<poolie> and i may work on that one
<lifeless>  - email lp-dev letting folk know that errors from it may not show in the -errors-report anymore
<poolie> then using the results of #1 to work out what's up with dkim
<poolie> so i can say the rt has your approval?
<lifeless> sure; cc me?
<poolie> for the sake of parallelizing, is the errors mailbox anywhere i could see it?
<lifeless> lp-error-reports list should have it, yes
<poolie> so i'd need to join that list, disable delivery, then look in the archives?
<lifeless> yeah, I think so.
<poolie> some devops people would say that sending mail even in the case of errors is not the best idea
<poolie> if things are falling over, mailservers may be swamped or unreliabel, etc
<lifeless> mmm
<lifeless> that applies to all automation
<lifeless> email is one of most robust systems we have : more robust that being able to ssh into the problem box.
<poolie> it's a cheap but inefficient way to broadcast things
<lifeless> sure
<lifeless> we don't have better yet.
<lifeless> particularly for errors.
<poolie> so is it reasonable to send everything, including errors, to a file for now?
<lifeless> can we do voice? I'm having deja vu and perhaps more bandwidth will address your questions quicker.
#launchpad-dev 2010-09-16
<poolie> sure, call me?
<lifeless> are you on skype?
<poolie> i can be
<lifeless> please
<poolie> mm it's trying to connect
<lifeless> ok, I'll ring ze landline
<jam> rockstar: just to confirm, the email I CC'd you on failed with Submit Request Failure as I expected it would
<rockstar> jam, hm.
<jam> rockstar: would forwarding the failure help you?
<rockstar> jam, no, I see that it's valid.
<jam> sure, I didn't know if the failure included any info you could use
<rockstar> jam, does the same thing happen on bugs as well?
<jam> same thing
<rockstar> jam, hm.  I think the bug probably goes with foundations.  I wonder if something has changed with python gpg bindings.
<bac> lifeless: meeting ping
<lifeless> hish
<poolie> jam by 'avatar' do you mean 'current user'?
<poolie> is that the actual name for it in that code?
 * poolie is not pedantic, just curious
<jam> right
<poolie> at least in this context
<jam> avatar.user_id is the database identifier used for the branch vfs
<poolie> ok
<poolie> jam, robert recently landed (or at least proposed) some new scopes, which may give you a template
<poolie> lifeless: where did they go? they don't seem to be in devel yet
<poolie> oh, it is landed, i just expected it would be in scopes.py
<thumper> rockstar: want to talk?
<poolie> i'll reply by mail
<lifeless> poolie: feel free to refactor; I was following what I saw (right or wrong)
<rockstar> thumper, 5 minutes.
<thumper> rockstar: ok, I'll go make coffee
<wallyworld_> thumper: white with 2 sugars please
<thumper> wallyworld_: pass me your cup
<wallyworld_> thumper:  i would if i knew how to get the cup icon to display in irc :-)
<poolie> ââ
<poolie> "find" in character map :)
<wallyworld_> poolie: show off!
 * thumper waits for rockstar
<rockstar> thumper, wait no more!
<wgrant> Can we get ~ubuntu-bugs removed from ~launchpad-beta-testers? It was added overnight to allow the ubuntu-bugs ML to receive comments from remote bug trackers, but has the side effect of redirecting lots of people to edge.
<wgrant> Where "lots" is around 400.
<lifeless> that seems like a bad idea
<lifeless> file a bug, malone, critical sev
<wgrant> Why does it need a bug?
<lifeless> well, won't the comments not go to ubuntu-bugs if we remove it ?
<wgrant> There's already a bug for sending the remote comments to everybody.
<lifeless> ok.
<wgrant> And they've only been going there since last night.
<wgrant> So it's not a significant loss.
<lifeless> make sense to me
<lifeless> bugsquad can do it
<wgrant> Bug 639736
<wgrant> lifeless: How?
<lifeless> exit the team?
<wgrant> Bug Control is a member, but bdmurray is the only admin.
<lifeless> oh
<lifeless> bdmurray: yo!
<wgrant> Otherwise an ~l-b-t admin can do it.
<lifeless> bdmurray: a) you may want to have someother admins
<lifeless> bdmurray: and b) ^ the discussion aobove
<bdmurray> wgrant: sure I'll do it now
<wgrant> bdmurray: Thanks.
<bdmurray> I was trying to work around a Launchpad bug
<wgrant> is there a reason for bugcontrol to be a member these days?
<wgrant> Yeah, I saw the mail go past overnight :)
<wgrant> ~ubuntu-bugcontrol could be the bug supervisor, and ~ubuntu-bugs just be subscribed.
<bdmurray> wgrant: well Launchpad is oops'ing on me now wrt ubuntu-bugs and lp-beta-testers
<wgrant> bdmurray: What's the traceback?
<bdmurray> wgrant: its a TimeoutError in the sql statement
<wgrant> Awesome.
<wgrant> Retry a couple of times, I suppose.
<wgrant> Maybe it will eventually work.
<StevenK> thumper: O hai; I seem to recall SQLObjectNotFound coming up in your review of my branch -- did you recommend to just bin it and I missed it?
<thumper> StevenK: I think I recommended getting it form lazr.lifecycle
<StevenK> Ah yes, that sounds right
<StevenK> thumper: I'm fixing up all of the lint :-)
<StevenK> thumper: I don't see SQLObjectNotFound in lazr.lifecycle?
<thumper> what do you see?
<StevenK> I just grepped for it, I got no results
<lifeless> thumper: is there a bug that
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/bugmessage/+merge/35620/reviews/45660/+reassign
<lifeless> doesn't let you change the review type requested ?
<thumper> lifeless: yes
<lifeless> ok cool.
<thumper> StevenK: you are right
<thumper> it isn't there
<thumper> from lp.app.errors import NotFoundError
<mwhudson> is this a way of teaching your mentee a lesson? :-)
<thumper> that'll do a 404
<thumper> mwhudson: heh
<mwhudson> "don't take everything on trust"
<thumper> mwhudson: and "everybody lies"
<mwhudson> :)
<StevenK> Thank you, House
 * StevenK passes thumper a cane, and some Vicodin
 * thumper hobbles off to do some shopping
<lifeless> man
<lifeless>     2026  OOPS-1719N1979  BugTask:+index
<lifeless> still need to get that down
<lifeless> Ursinha-afk: if you see this https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt - not sure why 'Revision 11522 can not be deployed: not-ok'
<bdmurray> wgrant: so I set the expiration date for the 16th we'll see what that does
<lifeless> also we probably want to report on more of the non-ok revisions
<poolie> lifeless: bug 640125 for logging stuff
<wallyworld__> anyone seen this error running bin/test -vvm lp.registry.browser.tests.test_milestone
<wallyworld__> Failure in test lp.registry.browser.tests.test_milestone.TestMilestoneMemcache.test_milestone_index_memcache_anonymous
<wallyworld__>   File "/home/ian/projects/lp-branches/current/lib/lp/registry/browser/tests/test_milestone.py", line 108, in test_milestone_index_memcache_anonymous
<wallyworld__>     'anonymous, view/expire_cache_minutes minute', content)
<wallyworld__>   File "/home/ian/projects/lp-branches/current/lib/lp/testing/memcache.py", line 33, in assertCacheHit
<wallyworld__>     self.assertTrue(cache_start in before)
<wallyworld__> AssertionError
<wallyworld__> i was testing a small change which touched test_milestone.py and get the same error even after reverting back to the original version
<thumper> wallyworld__: what touched test_milestone?
<jtv> Is ec2 broken for anyone else?
<jtv> The script, I mean.
<wallyworld__> thumper: line 86 change to series = self.factory.makeProductSeries(product=product) -- it was just self.factory.makeSeries(...)
<thumper> jtv: I'm just testing, but it is an older one
<thumper> wallyworld__: ah.
<jtv> I just updated everything, and regret it
<jtv> Actually it may just be "ec2 land"âI'm firing off an "ec2 test" that's really firing up an instance afaics
<wallyworld__> thumper: i re-ran the tests on a different branch without the changes and it still failed the same way :-(
<thumper> wallyworld__: I'll try here
<wallyworld__> ok. thanks. i'll have to get out my scuba gear and dive in to take a look
<thumper> wallyworld__: it runs here on your tales-linkify-broken-links branch
<thumper> wallyworld__: on lucid
<wallyworld__> thumper: hmmmm. that's the same branch i tried it on :-(
<thumper> wallyworld__: it could be a lucid vs maverick issue
<thumper> can we find other volunteers to try?
<wallyworld__> ok. I'll make sure all other tests etc run locally and then run it through ec2 test
<thumper> wallyworld__: one thing on that branch
<wallyworld__> thumper: yeeeees?
<thumper> wallyworld__: can I get you to prefix any variable without a proxy with naked_ ?
<thumper> so...
<thumper> naked_product = removeSecurityProxy(branch.product)
<thumper> wallyworld__: as this confused me looking through the tests
<wallyworld__> yup. good idea
<thumper> wallyworld__: as I expected it to fail not realising that the product didn't have a proxy
 * wallyworld__ nods
<thumper> thanks
<wallyworld__> maybe one day next year this branch will get landed :-)
<thumper> it is ' ' close
 * thumper mimics ' ' this close with fingers slightly apart
<lifeless> jtv: symptoms?
<wallyworld__> still, i'm getting to play with pipes going back and firth between the 2 branches. nice plugin :-)
<wallyworld__> s/firth/forth
<jtv> lifeless: http://paste.ubuntu.com/494554/
<jtv> "ec2 test" works
<thumper> wallyworld__: don't forget to pump :)
 * thumper steps away to cook, back later for more evening calls
<thumper> wallyworld__: I'll check back on the tales fix branch before I sit down to eat to see if we can get that into the landing queue
<wallyworld__> thumper: i won't forget :-)
<wallyworld__> thumper: i'm just about to push the "final" changes. had to answer the door. a box from Think Geek arrived :-)
<wallyworld__> lifeless: what's the status of your getLastOops fix? ie when is it safe to try and land a branch again?
<lifeless> it landed
<lifeless> so the symptoms are addressed, I'm landing another branch via ec2 land atm, if that works I'll mail the list. if it doesn't I'll mail the list.
<lifeless> stub: hai
<lifeless> stub: I has db patch up
<stub> yo
<stub> yup
<lifeless> I'd like to check its a sane approach with you
<stub> lifeless: That seems fine. I want to run a quick test on staging to see if we should populate it in the DB patch and make it NOT NULL.
<lifeless> stub: coolio
<lifeless> stub: I was figuring on a garbo task to populate it
<stub> We could just use the message id instead of the index, but that would break existing links.
<stub> lifeless: I don't see how it is useful if it information is only available sometimes.
<lifeless> stub: garbo task to populate over a cycle
<stub> ok
<lifeless> once the row exists land a patch to set it on all new rows
<stub> The table isn't wide - I don't think it will take long to rewrite it but I'll check first.
<lifeless> then, once the garbo has caught up, we know its all there
<lifeless> land a patch to use it
<lifeless> stub: its going to be millions of rows though
<thumper> wallyworld___: around?
<thumper> wallyworld_: around?
<wallyworld_> thumper: like a rissole
<thumper> wallyworld_: I've approved the tales branch with an optional but suggested change
<wallyworld_> ok. will check email
<wallyworld_> btw, a make clean helped that milestone test to pass
<wallyworld_> thumper: i'll make the suggested change and land it once ec2 is back in business. thx
<thumper> wallyworld_: ack
 * thumper done until call with mthaddon later
<stub> lifeless: its not too slow to update, but it is too slow to calculate all the indexes
<stub> well... both...
<lifeless> stub: times?
<stub> longer than I could be arsed waiting ;)
<lifeless> stub: so incremental is the way
<lifeless> stub: how long to add null and index ?
<lifeless> stub: or should we add null and index after the rollout ?
<stub> The index is added in this rollout, we populate, then we add the not null next rollout.
<lifeless> kk
<lifeless> what patch number ?
<lifeless> stub: how many rows do we need to calculate?
<lifeless> stub:?
<stub> 3.3 million rows to calculate
<stub> So 42 seconds to calculate on staging with this approach... I'll leave the update running for a bit.
<stub> patch will be 2208-14-0
<lifeless> how are you populating ?
<lifeless> stub: and can you approve so I cans lands it ?
<poolie> lifeless:  are your Epic slides on the web?
<lifeless> yes
<lifeless> linked from the architectureguide
<poolie> tx
<lifeless> and emails n stuff
<stub> lifeless: http://paste.ubuntu.com/494601/ is the best I can do, but the update is too slow (killed it again)
<stub> lifeless: So I'll approve the patch in the form you have and we will need datamigration and a follow up patch
<lifeless> thanks
<adeuring> good moring
<mrevell> Morning
<jelmer> lifeless, hi
<jelmer> lifeless, I saw your getLastOops branch landed. Did it help with your ec2 runs?
<jml> good morning.
<jelmer> hi jml
<jml> I thought someone had done some work recently on exposing blueprints over the webservice
<wgrant> There have been a few efforts there.
<wgrant> ajmitch had one.
<wgrant> I don't recall who tried it last.
<wgrant> Everyone seems to give up quickly...
<jml> I think james_w & cjwatson have both tried.
<jml> *sigh*
<wgrant> Maybe some things just weren't meant to be (I may be thinking of the app itself here, though...)
<nigelb> jml: ajmitch said he was working on one.
<nigelb> arg, I should read
<jml> nigelb, it happens :)
 * nigelb is waking up at 3 pm - confusion guaranteed
<ajmitch> yeah, james_w had done a bit more than me & submitted it for review - I think people recoiled in horror from the blueprints internals that got exposed
<nigelb> heh
<ajmitch> I don't know if anyone else has tried further since then
<nigelb> Now I know why wgrant always wants that part killed
<wgrant> I too tried back in the early days.
<wgrant> But there's a lot of refactoring required.
<wgrant> Then I decided that attempting to arrange the application's demise was more productive.
<ajmitch> this is why I've not really touched it since
<nigelb> oh, it is going to be killed?
<ajmitch> if wgrant has his way
<wgrant> Sadly, others do not seem convinced.
<ajmitch> someone broke the css again?
<henninge> ajmitch: yup, looks like it
<deryck> Morning, all.
<jml> I'd love to spend a couple of months improving our developer documentation & tools.
<jml> deryck, good morning
<jml> deryck, just quickly, what's the status of https://dev.launchpad.net/LEP/BugzillaComponents ?
<deryck> jml, we're in progress on it.  bryceh started coding a couple weeks ago.
<jml> deryck, ta.
<deryck> jml, I thought you approved this one.  Did we leap before we looked?
<jml> deryck, it's approved, but it was in the "Ready to code" section. Just tidying up the LEP page.
<deryck> ah, ok.
<jml> the page tells a slightly different story to the Kanban boards, I think :)
<lifeless> jelmer: well it, itself, got through ec2
<lifeless> deryck: for your delectation https://code.edge.launchpad.net/~lifeless/launchpad/bugmessage/+merge/35620
<lifeless> jelmer: two branches since have passed ec2, but not got through pqm
<deryck> lifeless, nice!
 * thumper halts
<lifeless> deryck: hopefully
<lifeless> deryck: we're 3 weeks out from starting the data migration ;)
<deryck> heh, well true.  But still it's step in the right direction.
<deryck> bigjools, hey.  I just now noticed you requested a review from me.  Sorry.  You still need this?
<bigjools> hey deryck, yeah it would not hurt if you had a look.  It's to do with closing private bugs from uploads.
<deryck> bigjools, ok, will look now.
<bigjools> cheers
<bigjools> deryck: we had the conversation ages ago where we said it would be a good idea to do the closing in a Job, but I've decided on the dirty hack instead :)
<deryck> bigjools, right,  I recall that now.  And you dirty hacker, you. ;)
<lifeless> deryck: bugtrackerset isn't pagination
<lifeless> deryck: though it only has 900 rows, so its not really dying for it.
<lifeless> jml: thanks for landing that fix the other day
<lifeless> bigjools: your cps are done; can you refreeze the db in prod though?
<jml> lifeless, np
<bigjools> lifeless: it was first on my list of things to do this morning, and the branch is playing in prod_lp right now
<lifeless> bigjools: sweet, thanks
<bigjools> lifeless: although the DAS enablement branch should have gone to appservers as well, BTW
<lifeless> bigjools: we deployed to all
<lifeless> bigjools: edge prod backends the works.
<bigjools> lifeless: your CP request just said "soyuz" ?
<lifeless> yes, but there were 4? 5? things, and we want same revs as much as possible.
<bigjools> if it went everywhere then great
<lifeless> so chex just said 'meh, the works'.
<bigjools> :)
<bigjools> thanks for getting it in
<deryck> lifeless, right.  I didn't think it was paginated.  Would still benefit from that, though the page isn't really doing much short of the listing I think.
<lifeless> bigjools: If you'd had it partially there I would have known in advance to ensure it got to the appservers :) Still it, did, so its cool.
<lifeless> deryck: I'm sure of it
<lifeless> 11 /    8  BugTrackerSet:+index
<lifeless> pity that the oops are not listed
<lifeless> SpamapS: hey
<lifeless> SpamapS: what was that 11 second api you found again?
<wgrant> bigjools: What do you think about bug #566339? I'm fairly sure it's not the one that you've fixed.
<bigjools> possibly yeah, there's more than one place that sends emails
<wgrant> Yay.
<deryck> bigjools, so I'm not really worried about this change as it stands now, but I do have concerns other additions to the script don't come along acting on a not security proxied bug and doing things we don't want.
<bigjools> well the bug mail and the upload notifications I mean
<bigjools> deryck: totally - this change is completely isolated to that one function though, so it should be fine. I hope :)
<bigjools> deryck: thanks for looking
<deryck> bigjools, well, I would rather have the comment above be a true XXX or a strongly worded "if you do more than fix release bugs here, this has to move to jobs."  What do you think of that?
<bigjools> deryck: possibly - I'm not sure the job is necessary but I agree with a strongly-worded warning about being careful with the unproxied object in that function/.
<bigjools> then the developer can make an informed choice about what to do
<deryck> bigjools, right, that works.  Thanks.  r=me.
<bigjools> deryck: great, thanks
<lifeless> danilos: if I'm still not getting it, perhaps we can talk my morning; tis midnight here.
<danilos> lifeless, sure
<lifeless> danilos: also I really want to figure out if it was canonical_url, or the template code, or 50/50 that contributed to the big +templates speedup
<lifeless> danilos: I'm already encouraging gary to get the new template engine widely deployed asap, but don't have enough data yet to say if we need to overhaul/replace/ditch canonical_url.
<lifeless> deryck: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1720G80
<lifeless> 938 queries!
<deryck> oh, wow.  At least one very tracker then.
<lifeless> 1 915 4354 4 4350 SQL-launchpad-main-slaveSELECT COUNT(*) FROM BugWatch WHERE BugWatch.bugtracker = %s
<lifeless> its counting the bugs per tracker.
<lifeless> group by time boys!
<deryck> ok, so that's a pretty cheap fix!
<deryck> right!  group by FTW! :-)
<lifeless> should be for a while, yes
<deryck> Today's my long day, so I'll see what I can do today.
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/618375
<stub> lifeless: I'm thinking of making scripts that call logger.exception(msg) emit an OOPS. Worthwhile making scripts that call logger.error(msg) emit an OOPS too? Or logger.warn(msg)?
<lifeless> deryck: Time: 90.955 ms
<wgrant> jelmer: Morning.
<lifeless> select BugWatch.bugtracker, count(bugwatch.*) from bugwatch group by bugwatch.bugtracker;
<jelmer> wgrant: hi
<lifeless> stub: \o/
<lifeless> stub: here are my thoughts
<lifeless> stub: things that go wrong in scripts should be oops
<stub> To all three? I was thinking of starting with exception and error.
<lifeless> stub: except for things that go wrong writing oops out.
<lifeless> stub: I'd do exception and error, and let everyone know that thats happening.
<stub> k
<lifeless> stub: just make sure there is a backdoor to say 'hey, oops writing failed'
<lifeless> (and include the traceback of *that* failure)
<lifeless> ok, really must go.
<lifeless> avoir.
<stub> lifeless: I'm not going to stop the log.error() output, just create the logs in addition.
<stub> night
<wgrant> jelmer: Your build uploading fixes seem to have alleviated my concerns. Looks pretty good now.
<jelmer> wgrant: Thanks!
<wgrant> jelmer: Are you deliberately storing the upload log in all cases now?
<wgrant> There's also a tiny race (the upload processor may catch a build after b-m moves it into incoming, but before it commits the UPLOADING status), but that may not be a concern.
<wgrant> This is all too complicated :(
<jelmer> wgrant: you're thinking we should have an explicit database commit before we do the move?
<wgrant> jelmer: That might work.
<wgrant> Otherwise, well, the extremely unlikely case will probably happen at some point :)
<wgrant> And then we'll have builds stuck in UPLOADING and the world will end.
<jelmer> wgrant: actually, it won't be an issue at the moment since the uploadprocessor will warn if it encounters a build that doesn't have the right build.status and skip that directory.
<wgrant> jelmer: Didn't you just change it to move it to failed?
<wgrant> It was OK when it just skipped.
<jelmer> wgrant: hmm, perhaps it should be changed back to only skip to cope with that situation
<wgrant> jelmer: It'd be nice if it would skip it once, but that's probably hard to arrange.
<wgrant> jelmer: Maybe skip if date_finished is in the last 5 minutes or something.
<gmb> Gaaaaah.
<gmb> Can anyone remember how to raise an exception (or do something similar anyway) in YUI3?
<james_w> jml: I'm pretty close I think. I need to move the code from security.py on to the model, and tweak some methods, then get the pre-requisite branch reviewed
<james_w> I haven't worked on it in a while though
<gmb> Or maybe I should just stick with the JS way of doing things.
<jelmer> wgrant: A commit/flush seems like the proper thing to do though.
<wgrant> jelmer: As long as nothing expects an UPLOADING build to have an entry in incoming.
<jelmer> wgrant: right
<wgrant> jelmer: Previously the upload log was recorded only on failure -- it's probably fine to record on success too, but I'd like to be sure that's a deliberate change.
<jelmer> wgrant: yeah, I saw your point earlier. I was going to wait for Julian to come back from lunch and then ask him about it.
 * bigjools is back
<wgrant> It might be a good idea to record it. But it's a little bit of extra librarian bloat.
<jelmer> wgrant: it's not recorded on successful uploads to save up on space, but lp-code has a triaged bug open about the fact that not storing it for successful builds is inconsistent
<wgrant> Yeah, I saw that.
<wgrant> Given the other large files we store for every build anyway, the space is trivial.
<bigjools> jelmer: the bug is filed on lp-code?
<jelmer> bigjools: yeah - it just talks about recipe builds
<bigjools> I'm inclined to stay with the existing behaviour
<jelmer> bigjools: https://bugs.edge.launchpad.net/launchpad-code/+bug/
<jelmer> 581892
<bigjools> I don't see the point of storing successful upload logs
<bigjools> nobody will look at them
<jelmer> abentley: hi
<abentley> jelmer, hi.
<jelmer> abentley: You appear to be the reporter of bug 581892. I'm touching the related code at the moment and we were just discussing how useful having the upload logs for successful builds would be.
<jelmer> abentley: The upload logs are currently not stored because they're not particulary useful for anything and just take up space on the librarian.
<abentley> jelmer, I think that knowing what a successful upload looks like can help with debugging unsuccessful ones.
<abentley> jelmer, it also seems asymmetrical.  Why store build logs and not upload logs?
<jelmer> abentley: build logs contain output from the package build system, they might be interesting to the uploader.
<bigjools> my suggestion is to make it optional depending on what the upload policy says
<bigjools> nobody wants or needs it in soyuz world
<bigjools> AFAIK anyway
<bigjools> but that should not block this landing
<bigjools> (with the existing behaviour)
<jml> james_w, that's great.
<abentley> jelmer, I don't understand why "contain output from the package build system" makes it more storable.
<wgrant> It's useful to see how the package was built. There's more than a binary situation there.
<jml> james_w, btw, do you think it would be useful to have a tool that gets the build dependency branches for a given project?
<jelmer> abentley: it makes it more interesting, thus more storable. as a package uploader I often look at succesful build logs, I have never even thought about looking at the upload logs.
<wgrant> With a successful upload... it just works.
<jelmer> bigjools: That seems reasonable
<gmb> mars, Around?
<james_w> jml: I think so
<abentley> jelmer, bigjools: when I was first looking at it and coding my own build page, it looked like something was wrong.
<abentley> Because I knew the upload had happened, and I knew we store logs, but I couldnt' see it.
<jml> james_w, I don't think it would be too hard to do with what we have now.
<james_w> jml: indeed not
<abentley> wgrant, could you give me an example of different ways the package could be built?
<jelmer> bigjools: I was hoping there was quick consensus on this and we could deal with that bug (whether it be wontfix or fixcommitted), but since there isn't I'll just keep things as is for the moment and leave that bug alone.
<jml> james_w, perhaps I'll knock up a prototype and then see if I can get someone else to take over work on it :)
<bigjools> abentley: the soyuz build pages seem to be fine without it
<jelmer> abentley: the compiler output would be part of the build output
<wgrant> abentley: configure might make a bad decision based on the installed build-dependencies.
<bigjools> jelmer: it should be policy based
<jml> (although more likely I'll get distracted fixing txrestfulclient
<jml> )
<wgrant> abentley: There could be build warnings.
<wgrant> We could have braindead packages detecting the installed kernel version and tailoring themselves to it grrrrrr.
<james_w> jml: good idea
<james_w> both of them
<jml> :)
<abentley> wgrant, thanks.
<wgrant> So, I agree that it would be nice to store successful upload logs, and it would make things more consistent.
<wgrant> But they're really not useful, and are a bit of bloat.
<wgrant> abentley: Why did Code want them?
<abentley> wgrant, when I was coding up the the build pages, I found it surprising.  I thought the upload logs were "missing".
<wgrant> Ah.
<gmb> mars, Unping; I'm an idiot.
<bigjools> I don't think that's a compelling reason to add them, personally.
<wgrant> Well, it does simplify archiveuploader if they are added.
<abentley> bigjools, we probably have different thresholds for breaking symmetry.  Their slight bloat doesn't seem like a compelling reason to make the system more complicated to me.
<bigjools> I think you have that back to front, but that's just my opinion
<wgrant> Code does have branchrevision... :P
<abentley> bigjools, that opinon would be consistent with us having different thresholds :-)
<bigjools> heh, once you're used to branchrevision, nothing seems like bloat :)
<abentley> wgrant, thumper and I are looking at ways to get rid of it (or at least remove the scaling problems).
<wgrant> abentley: Excellent.
<bigjools> wgrant: I have a challenge for you
<wgrant> bigjools: Oh?
<bigjools> wgrant: yeah, we need a test case for bug 556839
<wgrant> Is that the +queue 403 that I was arguing about earlier?
<bigjools> it's the double copy thing.  Crap that's a private bug, sorry.
<wgrant> Oh, that one.
<bigjools> noodles775 may have pinpointed the source of the problem in the packagecopier code
<wgrant> yeah, can't see it.
<Ursinha> lifeless, rev 11522 was blocked on bug 86185, that was also linked to the landed branch and qa-needstesting when it was in fact qa-untestable, because unrelated to the fix
<bigjools> where if you're copying the same SPR with binaries, it lets it through
<Ursinha> lifeless, the problem we discussed before (bug 638468)
<wgrant> What's the actual problem?
<wgrant> A double copy should be fine.
<bigjools> wgrant: it was nightmarishly hard to reproduce, but I eventually found out that if you double tap the copy button you end up with the same package trying to get published twice, and the publisher spazzes out with a QueueInconsistentStateError
<wgrant> Oh, delayed copies?
<bigjools> nope, regular
<wgrant> Er.
<wgrant> Why would it whinge about that?
<wgrant> That doesn't make sense.
<bigjools> they were both private archives
<wgrant> What's the QISE?
<bigjools> "blah" is already published in archive for <series>
<wgrant> That sounds delayed to me.
<wgrant> In fact, yes.
<wgrant> The publisher only deals with the queue in the case of a delayed copy.
 * bigjools shrugs.  It happens.
<bigjools> but yeah it's in process-accepted
<wgrant> You're sure it was direct?
<bigjools> so something went very wrong
<wgrant> Ah, so no, delayed.
<wgrant> It's not easy to reproduce?
<bigjools> with the double tap I can do it
<wgrant> Hm, I guess you might have to get another request in before the PackageUpload is there.
 * wgrant checks.
<wgrant> Yeah, so you need to have two concurrent transactions.
<wgrant> Or the second will be rejected.
<wgrant> That is hard to test :(
<bigjools> exactly :)
<wgrant> It's the same as the upload conflicts we get sometimes.
<wgrant> Since simultaneous uploads on different machines can violate the file conflict checks.
<wgrant> These should probably be solved similarly.
<bigjools> yup
<bigjools> and I've no real idea how
<wgrant> Everywhere else solves them with UNIQUE constraints. But I don't think we can.
<wgrant> We need some way to lock an archive.
<bigjools> maybe a partial commit
<bigjools> as soon as we get into the copy code
<bigjools> so it's a race as to which appserver thread gets there first and only one wins
<wgrant> Partial commit in webapp == nauseating.
 * bigjools shrugs
<bigjools> how else?
<bigjools> we need a lock
<wgrant> We do.
<wgrant> But I've no idea how to do that here.
<wgrant> Hopefully somebody else does.
<bigjools> balls, stub just left
<bigjools> hmmm
<bigjools> maybe we can use optimistic locking
<bigjools> not sure that's the right term actually
<bigjools> since the appservers run with isolation level serializable, we can make the txn write to the same thing
<wgrant> No.
<bigjools> one will fail
<wgrant> The appservers are READ COMMITTED :(
<bigjools> nope
<bigjools> stub told me not 10 minutes ago they're serializable
<wgrant> Er.
<bigjools> which in PG is actually snapshot isolation
<wgrant> Did that change recently?
<wgrant> I see you're right now.
<wgrant> But I see a lot of READ COMMITTED in old logs.
<bigjools> however I have a horrible feeling that the transaction just gets retried if it fails
<wgrant> Yep.
<bigjools> we could work around that easy I think
<bigjools> treat a lock flag on an archive as a copy mutex
<bigjools> we just need a way of resetting it later
<wgrant> Well, we could probably just set it and unset it.
<wgrant> And that would cause a conflict.
<bigjools> it won;t work if the transaction is retried
<wgrant> And while the request would be retried in the case of a conflict, that would mean that the check would be executed, and fail this time.
<wgrant> So I think it would work.
<bigjools> you'd need to 1) check it's not already set, b) then set it or fai;
 * bigjools chuckling at jml's tweet
<wgrant> If the start of the copy just sets and unsets it, it should cause a conflict in other simultaneous copies, right?
<wgrant> In READ COMMITTED that wouldn't work, since they'd have the same final value.
<bigjools> wgrant: in that scenario, the 2nd copy would fail once then work the second time
<bigjools> I think
<wgrant> bigjools: It wouldn't work the second time: the whole request would be retried, which would mean the duplicate check would fail the second time.
<bigjools> ok I'm thinking of it re-running the code for the request, maybe that does't happen :)
<wgrant> It doesn't just rerun the SQL.
<wgrant> That would be insane.
<bigjools> eparse.  what?
<wgrant> When retrying because of a conflict, the request is restarted.
<wgrant> So all the checks are run again.
<wgrant> In a fresh transaction.
<bigjools> ok
<bigjools> so it would work the 2nd time
<wgrant> In that the duplicate check would reject the copy.
<bigjools> since there would no longer be a conflict
<wgrant> Why not?
<wgrant> Oh.
<wgrant> Conflict as in transaction conflict, not archive file conflict.
<bigjools> yes
<bigjools> the current code does not prevent archive file conflicts
<wgrant> Right.
<bigjools> so we need to check the flag before embarking on the copy, then try and reset it later.
<bigjools> that's the nasty bit, only the publisher comes to mind, and that's a long delay to prevent a copy :/
<bigjools> hmmm if you don't double tap the copy button, the 2nd request after the current one finishes does get rejects as I recall
<wgrant> Why check it? Nothing should leave it set.
<wgrant> It does, yes. It will confirm that there's no existing delayed copy.
<bigjools> if it's not left checked then what's to stop another transaction crossing?
<wgrant> bigjools: If each copy transaction sets and unsets it, they should all conflict.
<bigjools> yes and the retry will work, as we said
<bigjools> so we need the isolation guard and also a code guard
<wgrant> The retry has two possible outcomes:
<wgrant>  1) It occurs during the same conflicting transaction. It too will conflict, and be retried again; or
<wgrant>  2) It occurs after the conflicting transaction. It will perform the copy checks, notice the file or PackageUpload conflict, and reject the copy.
<bigjools> there is another way
<bigjools> we make a copy a "copy request"
<bigjools> they need to go offline anyway, they are too complicated to happen in a request
<bigjools> then we serialize the copies through a script
<wgrant> I don't think they're actually that complicated. The checker can be optimised much further.
<bigjools> it's not just the checker
<wgrant> And inline feedback is a whole lot better than forcing clients to poll..
<bigjools> copying one package with loads of binaries also goes titsup
<wgrant> Is that the checker, or the actual copy?
<bigjools> I *think* the latter
<wgrant> Hmm.
<wgrant> Odd.
<Ursinha> sinzui, hi, I'm sorry about your qa-ok bugs retagged by mistake -- I've already fixed them back to qa-ok. We're experiencing preconditional errors in the api with the script so it believes last run was unsuccessful and tries to tag again
<bigjools> but I am unsure
<wgrant> I guess for a kernel it could get fairly large.
<Ursinha> sinzui, I'm working on a fix
<bigjools> that's exactly the scenario I am thinking of :)
<bigjools> we can't currently copy a kernel reliably
<bigjools> unless it's delayed
<sinzui> Ursinha, its okay
<wgrant> But there's only 8 archs and probably only a few dozen binaries for each.
<bigjools> actually y'know what, delayed copies are requests-to-copy thinking about it
<wgrant> So it's only a few hundred inserts, even if we do it naively.
<bigjools> but we pre-checked 'em
<wgrant> We should try to optimise before we externalise.
<bigjools> agreed
<wgrant> Particularly given that the primary motive for delayed copies has evaporated.
<bigjools> yeah :/
<wgrant> Still, they'll be useful for distro copies.
<wgrant> So it wasn't all wasted effort.
<SpamapS> lifeless: the 11 second api call was for basically all bugs.
<jcsackett> is there a way to do conditional define statements in a div in TAL? sort of "condition:view/attr then define:var attr else define:var some_other_attr" ?
<benji> jcsackett: tal:define="var python:view.attr and one_value or another_value"
<jcsackett> benji: thanks. you are my new hero.
<benji> :)
<statik> sinzui, your mail about the text = bytea error on launchpad/maverick, there is a branch attached to bug#585704. Should I apply that patch to the version of storm shipping in maverick today?
<statik> the patch is here: https://code.edge.launchpad.net/~jamesh/storm/bug-585704/+merge/26472/+preview-diff/+files/preview.diff
<sinzui> statik: I believe henninge tried it without success.
<statik> odd. it's one-line patch to storm/variables.py, excluding tests. you would probably also have to fix some of the launchpad tests to make it all work though (U1 had to do the same)
<statik> sinzui: i ask because you were "hoping it would be fixed", and I can patch and upload storm to ubuntu if there is a patch required to enable launchpad.
<sinzui> statik. yes. excluding tests will not fix Lp
<sinzui> statik, Lp dies on the first db request for a lot of page, such as a person page
<statik> sinzui, my point is simply that downgrading psycopg is not a solution, and there is a rapidly closing window where fixes could be uploaded to maverick, and i am ready and willing to upload such a patch to storm if it helps the LP team.
<sinzui> statik. I am out of my depth. gary, stub, or benji-lunch may understand the issue better. I do not know for certain there is a fix I do know that the foundations team controls the versions of storm and psychopg2 and apply patches as needed
<statik> ah, ok.
<mars> deryck, ping, may have found a nasty regression on the +filebug page JavaScript.  I need some help verifying the problem.
<deryck> mars, sure.  What's up?
<mars> deryck, was visiting https://bugs.edge.launchpad.net/launchpad-foundations/+filebug, used the "Extra options" to assign a person.  Pressing the 'Find' link does not throw up a JavaScript dialog - instead it takes you to the search page, trashing all form input.
<deryck> yup
<deryck> nasty and known ;)
<mars> :(
<mars> ok, thanks deryck
 * deryck is looking for bug
<deryck> https://bugs.edge.launchpad.net/malone/+bug/513591
<deryck> mars, it's on my top 15 bugs list, but I just haven't gotten someone on it yet.
<mars> thanks, dupe search didn't turn up that bug
<mars> deryck, thanks for letting me know.  I'll stop bugging you now :)
<deryck> np at all
<Ursinha> rockstar, hi :) could you triage bug 639785, please? I've seen those on lpnet for a few days now
<rockstar> Ursinha, since you asked nicely.
<rockstar> Also, wtf is mup?
<Ursinha> it's dead
<Ursinha> _mup_, hi
<Ursinha> rockstar, thanks :)
 * gmb EoDs. Night folks.
<rockstar> jml, can you point me to an example LEP that you find to be the best example of how you envision a LEP?
<lifeless> Ursinha: ah cool
<lifeless> jcsackett: can you QA https://bugs.edge.launchpad.net/launchpad-registry/+bug/623408 please?
<lifeless> deryck: hiya
<lifeless> abentley: ping
<abentley> lifeless, pong
<jcsackett> lifeless: shoot, yes. sorry. qa'ed them in fact a few days ago off the kanban and forgot to update LP.
<deryck> hi lifeless
<jcsackett> lifeless: done.
<lifeless> abentley: just seeking an answer on the (probably unclear) mail I sent about the builder patch in stable & CPing it
<lifeless> jcsackett: thanks!
<jcsackett> lifeless: you're welcome. :-)
<lifeless> deryck: so, I'm thinking to do bugtrackerset, unless you have it in-flight at the moment
<deryck> lifeless, oh, no.  Was actually just about to turn to it in 30 minutes or so.  You're timing is good.
<lifeless> deryck: nono, don't let me stop you :)
<lifeless> deryck: just when you sign off commit the inprogress work and push it; then tag me.
<deryck> ah, sure.  I can do that.
<deryck> thanks lifeless!
<abentley> lifeless, I suppose it probably is.
<lifeless> abentley: thanks
<lifeless> abentley: we've got a bunch of OOPS and timeout fixes we can pull across https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt
<abentley> lifeless, it does require a new bzr-builder, and it requires it in sourcedeps.conf
<lifeless> its easier to merge rather than CP when possible
<lifeless> abentley: ah cool.
<abentley> lifeless, I'm not sure what you hope to gain from it.
<lifeless> if 'it' is your patch, bring it across lets me do a branch merge
<abentley> lifeless, it's just pushing responsibility for safety checks into a better place.
<lifeless> rather than a partial merge, when preparing the new revision for production-stable
<lifeless> abentley: cool, it was tagged 'orphaned' was all, so I wanted to be sure it was ok :)
<lifeless> categorised as orphaned, I should say, not tagged.
<abentley> lifeless, It's "orphaned" because it doesn't fix any bugs.
<lifeless> yeah, which is cool
<lifeless> as I say, I wanted to do a proper merge and was just cross t's and dotting is.
<abentley> I did not bother creating a bug that the code was not it the most optimal place.
<lifeless> exactly, I wouldn't either.
<abentley> lifeless, it sounds like a rather aggressive approach to CPing to me.
<lifeless> abentley: its precisely what we'll be doing once we have the production schema qa environment anyway
<lifeless> abentley: it can be done now but it needs a little more effort, which I'm delighted to put in, given the rewards
<abentley> lifeless, that also worries me.
<lifeless> abentley: can you enlarge on that?
<abentley> lifeless, merely sticking UI stuff behind feature  flags doesn't seem like a very good way to avoid introducing regressions.
<abentley> lifeless, I'm working on a lot of changes to put incremental diffs in merge proposals, and there's been a lot of refactoring.
<lifeless> abentley: we're also QAing it before deploying it.
<lifeless> abentley: I'm not CPing anything that hasn't been QAd.
<abentley> lifeless, One can only QA so much.  I would rather give the changes time to cook, too.
<lifeless> abentley: we're giving it to users uncooked at the moment
<lifeless> and unqaed
<abentley> lifeless, not really.  It stays on staging for half a cycle.
<lifeless> abentley: if its in db-devel, yes.
<lifeless> abentley: anything in devel, which is all thats in scope for now, hits users the next day after buildbot blesses it. (At the moment, this will change with RFWTAD)
<abentley> lifeless, some users, not all users.
<abentley> lifeless, I'd rather break edge than break both edge and production.
<lifeless> we appear to break both edge and production once a cycle or two at the moment due to the skew between edge and production.
<abentley> lifeless, I wonder how that compares to the number of times we break edge alone?
<lifeless> abentley: so do I
<lifeless> abentley: I also wonder how many of the things that break edge we would pickup if we qa'd
<abentley> lifeless, we often do qa on edge.
<abentley> lifeless, feature flags have been presented as equivalently safe to the edge/production split.  I'm just becoming accutely aware that they are not.
<abentley> lifeless, you can argue that on balance the risks are reduced or increased, but it's definitely a big change.
<lifeless> abentley: Its a huge change
<lifeless> abentley: I think it has massive benefits and that the risks are approximately the same.
<abentley> lifeless, I think the risks of breaking production are increased.
<lifeless> abentley: thats true; one of the things being used to mitigate that is smaller rollouts (more granular rollbacks) and manually initiated deployments (human paying attention after the deploy for issues) and no-downtime rollouts (lets us rollback without scheduling stuff if there is an issue)
<abentley> lifeless, anyhow the reason I didn't reply to your email is because I haven't landed anything worth CPing, so I thought it was for someone else.
<lifeless> abentley: no worries
<lifeless> abentley: I figured I was unclear or confusing :)
<abentley> lifeless, definitely some context on the change would have helped.  I don't keep revnos in my head.
<lifeless> hah! yes.
<lifeless> sinzui: hi, bug https://bugs.edge.launchpad.net/launchpad-registry/+bug/640700 - seeking your thoughts
<sinzui> lifeless, I think half the oopese were me after doko made me an admin of openjdk. I was just using the form normally. The so-called problem message changes twice over the 90 minutes I was using the feature
<lifeless> sinzui: and you were seeing it batched ?
<sinzui> yes
<sinzui> Skipping the the last batch, after doing a first batch seems to introduce the problem.
<sinzui> skipping TO the last btch
<sinzui> batc
<sinzui> lifeless,  the batches are essentially sequential by date. I realised that the problem message (shown as a message id/timestamp) in the oops was from the start of the batches
<lifeless> moderate uses currentBatch
<lifeless> sinzui: so i'm thinking there are two thinkgs we need to do.
<lifeless> we need to use message id as the selector in the form.
<lifeless> I'm not sure how one does that with the way forms are wired up in lp
<sinzui> lifeless, I think to loop in the action is brittle
<lifeless> we also need to handle receiving a form telling us to do something nonsensical and show that (1, N) messages as being unmoderatable, not cancel the entire thing.
<lifeless> sinzui: I agree
<lifeless> sinzui: in the sense that we bail out if anything is wrong, rather than doing what we can.
<sinzui> we loop over the messages assuming the batch has not mutates since the form as rendered. Maybe we should iterate over what is submitted and look for the messages another way
<lifeless> oh
<lifeless> also
<lifeless> the batch may not match the backing data
<lifeless> yes
<lifeless> so we need to quewry for the messages the form refers to only
<sinzui> oh, in the 90 minutes I was testing, surely we got spam in the queue
<sinzui> I agree
<lifeless> and handle those
<lifeless> sinzui: can you give me some tips on getting the message id (or some other primary key) in the form data?
<sinzui> well I have a few hours more experience then yourself...
<deryck> bryceh, hey.  Let's wait another day before we do another Bugzilla tracker.  I see a backlog for gnome-bugs.  And I want to make sure checkwatches isn't overloaded.
<lifeless> sinzui: hah!
<lifeless> sinzui: ok, I will look at it during my day
<lifeless> thumper: is https://bugs.edge.launchpad.net/launchpad-code/+bug/633758 qa'd?
<bryceh>  deryck, okie doke
<sinzui> MessageApproval (the message in holds) as a foreign key to the Message and the MailingList. We can query the table for ids
<sinzui> ^ lifeless
<deryck> lifeless, I'm being too optimistic.  I'm not going to get to this bugtracker set stuff today.  Unless later tonight for me.
<bryceh> deryck, might let some of the grousing about getting emails die down a bit
<deryck> yeah :-)
<lifeless> deryck: de nada
<lifeless> deryck: Being fancy free as I am, I may.
<lifeless> deryck: or I may not, I'm trying to organise a CP too
<deryck> right, np
<lifeless> hmm
<lifeless> actually, friday is a silly day for that
<lifeless> I'll agitate folk to QA stuff
<lifeless> and CP on monday
<bryceh> deryck, btw, I assume there's no way we can temporarily suppress the emails from the checkwatches results, or is there?
<deryck> bryceh, not really.
<deryck> hmmm, well unless there is a flag to the script.  Let me see....
<deryck> bryceh, no, sorry
<deryck> bryceh, maybe after Mozilla we should hold off unless some asks about a tracker?
<bryceh> ok cool, I'd be embarrassed if there had been an easy way to prevent the emails :-)
<deryck> heh
<lifeless> bbiab, doctor appt
<wallyworld_> morning
<jelmer> 'morning wallyworld_
<wallyworld_> hi jelmer
<abentley> thumper, are we standupping at 5:15 or 6:00?
<wallyworld_> abentley: i've logged in now so we can do it early if everyone is here
<mwhudson> morning
<mwhudson> Launchpad encountered an error during the following operation: generating the diff for a merge proposal.  The source branch has pending writes.
<mwhudson> it doesn't say which branch :)
<thumper> :(
<thumper> mwhudson: file a bug :)
<mwhudson> thumper: ok
<mwhudson> hm, https://bugs.edge.launchpad.net/launchpad-code/+bug/612171 is somewhat related
#launchpad-dev 2010-09-17
 * thumper attempts to suspend again
<lifeless> sinzui: I'm going to work on the moderate thing
<thumper> yay, it worked
<lifeless> thumper: ping
<thumper> lifeless: hey
<lifeless> https://bugs.edge.launchpad.net/launchpad-code/+bug/633758 - can has qa?
 * thumper gets on it
<lifeless> (oh, and goo dmorning)
<thumper> morning
<thumper> my talk went well I think
<lifeless> thumper: cool
<lifeless> what was it ?
<thumper> qa done
<thumper> I was giving a talk to second year software engineering students about our development process
<lifeless> one of the errors last night on BB : bzrlib.errors.InvalidHttpResponse: Invalid http response for https://xmlrpc.edge.launchpad.net/bazaar/: Unable to handle http code 502: Bad Gateway
<thumper> a one hour lecture
<thumper> ?!?
<lifeless> thumper: 'we throw stuff at the wall and hope it works'
<lifeless> thumper: the 502 is probably an edge deploy.
<lifeless> thumper: I wonder what bzr version it is, that its using edge
 * lifeless waits for https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt to update
<lifeless> sinzui: is there a list with lots of messags that I can look at the moderation page with ?
<lifeless> Edwin-afk: / jcsackett: https://bugs.edge.launchpad.net/launchpad-registry/+bug/597738 needs QA for the latest landing (rev 11547)
<lifeless> thumper: https://code.edge.launchpad.net/~lifeless/launchpad/cp/+merge/35764
<lifeless> thumper: No new code, and I've marked up all the user facing / risk mitigating/correcting changes.
<lifeless> thumper: I'd like to get this through ec2 today to ask for a CP on monday.
<lifeless> thumper: ping?
<thumper> lifeless: hey
<lifeless> could you tag your review release-critical please?
<lifeless> (thanks for doing it)
<wgrant> When's the next release?
<thumper> lifeless: so this is part of the "try to release more often?"
<thumper> just trying to work out your rationale
<thumper> lifeless: my thinking is based around marking it "release-critical" where it clearly isn't just to cross some 'T's and dot some 'I's seems stupid
<thumper> if we are changing the process, we should just do it
<thumper> and not require this fake stamp just for the hell of it
<lifeless> thumper: yes
<lifeless> thumper: I suggest; we do this under the current process, and simultaneously point out that the current process is more onerous than the proposed, approved RFWTAD process
<lifeless> thumper: and so qa-ok'd stuff gets a free pass for CP's.
<lifeless> thumper: we'll need to tweak some toolchain stuff though - at least for the moment we *need* release-critical stamps to get past PQM.
<lifeless> and PQM isn't really setup to check 'has been qa-oked'.
<lifeless> wgrant: three weeks
 * thumper sighs
<thumper> lifeless: so you need the rc to get pqm to land it?
<lifeless> wgrant: yes
<lifeless> bah
<lifeless> thumper: yes
<wgrant> Hm.
<thumper> ok, I'll stamp it
<wgrant> There are revs in devel that shouldn't go out onto prod, AFAICT...
<lifeless> wgrant: yes, my branch isn't all of devel
<lifeless> wgrant: its all the ones passed by the qatagger
<wgrant> eg. r11552 enabled some unfinished UI on prod.
<wgrant> The bug doesn't have a tag, so I don't know its QA status.
<lifeless> wgrant: I'm doing up to and including 11546
<lifeless> thats all that has been fully QA'd.
<wgrant> Aha.
<wgrant> Everything up to there is fairly sane, IIRC. Sounds good.
<wallyworld> thumper: ping?
<thumper> wallyworld: aye
<wallyworld> just want to check something....
<wallyworld> anyone can submit a merge proposal, right?
<thumper> wallyworld: as long as they can see the source and target branches, yes
<wallyworld> ok. there's some ambiguity with some of the comments in a test
<wallyworld> just wanted to confirm, thanks
<wallyworld> i might skype you later if i need to discuss
<jtv> StevenK: did you ever figure out that broken test?
<StevenK> jtv: Which one?
<jtv> StevenK: the one you suspected fakelibrarian of
<StevenK> jtv: I didn't investigate it, sorry
<jtv> StevenK: Oh, I thought it was what put db-devel into testfix so I thought it'd be a big thing.
<jtv> lifeless: are you familiar with LaunchpadScriptLayer?
<lifeless> No, but i can be
<lifeless> why?
<jtv> It combines ZopelessLayer and LaunchpadLayer, and adds one thing:
<jtv>         provideUtility(TestMailBox(), IMailBox)
<jtv> There's a 4-year-old XXX from Francis saying it should be registered via ZCML in LaunchpadFunctionalLayer.
<jtv> Sorry; the comment says it _is_ registered via ZCML in the LaunchpadFunctionalLayer.
<jtv> Anyway, this smells of installFixture to me.
<jtv> I was thinking tests that use this could use ZopelessDatabaseLayer instead, and install fixtures for the test mailbox and if needed, the fake librarian.
<jtv> (Or if they need the real librarian, they could use LaunchpadLayerâthough that provides memcached which I'm not sure scripts are currently supposed to access)
<lifeless> so
<lifeless> I'd like to move all our 'layers' to use fixtures, decoupling stuff
<lifeless> specifically
<lifeless> lp:python-fixtures
<lifeless> I need to tweak the API a little more though
<lifeless> jtv: 16:41 < lifeless> so
<lifeless> 16:41 < lifeless> I'd like to move all our 'layers' to use fixtures, decoupling stuff
<lifeless> 16:41 < lifeless> specifically
<lifeless> 16:41 < lifeless> lp:python-fixtures
<jtv> lifeless: thanks for repeatingâmy IRC connections are extra-weird today.  Yes, fixtures instead of layers would be good.  I've gotten used to reserving setUp for that sort of thingâas opposed to the creation of repetitive test data.
<poolie> hello jtv!
<jtv> hi poolie!
<jtv> How's things?
<poolie> good; i thought my drive had died but it was just a loose cable
<poolie> though it's a bit annoying to lose time to carefully working this out
<jtv> Life's little rollercoasters
<jtv> I'm _hoping_ I've got a similar problem with my backup drive. :)
<poolie> how do i provoke an oops again?
<lifeless> poolie: do something wrong
<lifeless> poolie: :P
<lifeless> poolie: ++oops++
<poolie> ah, and it appears in the comment
<poolie> thanks!
<poolie> and i see the memcache feature was used! yay
<lifeless> poolie: it was ?
<lifeless> oh yes, it will get evaluated
<poolie> according to the comment in view-source:https://edge.launchpad.net/bzr/ it was queried
<poolie> i don't know if it had any effect
<lifeless> yes, any page that uses memcache will query it
<lifeless> back later
<henninge> jtv: I have converted the import code but still some tests are failing.
<jtv> henninge: what problems are you hitting?
<henninge> jtv: let me have a closer look, 'll get back to you in a few minutes.
<henninge> jtv: http://paste.ubuntu.com/495141/
<henninge> jtv: I have not yet worked with Karma.
<henninge> jtv: Is that something updateTranslation does, too, which I need to do manually now?
<jtv> henninge: shall I run you through the conceptual model of karma?
<jtv> What's happening here is that the karma we assign has changed a bit
<henninge> jtv: that would be nice
<jtv> In a nutshell, karma can be assigned in various contexts (somewhat similar to our queue entries, really).
<henninge> (only 5 failing tests in lp.translations, btw)
<jtv> Additionally, each assignment of karma is in a particular category.
<jtv> In the database, you might thus have a karma record that says something like "henning earned 5 karma points today in category marking-a-bug-as-fix-released, for the gim."
<jtv> updateTranslation assigns karma.
<jtv> It does so in Mysterious Ways.
<henninge> ;-)
<jtv> I did not try to reconstruct and emulate exactly how, when & what.
<henninge> but it is still done by ... sumitSuggestion/approve/... ?
<jtv> Instead, we now have a model that's relatively easy to track (see POTMsgSet, though I added an implementation method to POTemplate)
<jtv> Yes.
<jtv> Well, not "still" since those messages are new.  :)
<jtv> *methods
<henninge> yes, but they are the replacements for updateTranslation
<jtv> Off the top of my head, we award 3 kinds of karma:
<jtv>  * Submitting a suggestion.
<jtv>  * Reviewing.
<jtv>  * Having your suggestion approved by someone else.
<henninge> jtv: ok, I see two of those in the traceback
<jtv> Those are now neatly separated, with rules like "don't assign review/approved karma for someone approving their own translations."
<henninge> where are they defined?
<jtv> Look for awardKarma in potmsgset.py
<jtv> One in submitSuggestion, and two in approveSuggestion.
<jtv> Come to think of it, I think I forgot to do that in approveAsDiverged!
<jtv> And clearCurrentTranslation.  That's review karma for the reviewer, I think (assuming that there really are suggestions).
<henninge> ok
<henninge> maybe I need to add that. I use approveAsDiverged in the import if the incumbent ;) message was diverged.
<jtv> Then that should probably award karma in the same way approveSuggestion does: only if the message was not previously current and the reviewer is not the translator.
<jtv> henninge: maybe updateTranslation would be a fit for what you're doing.  That also follows the "if it's diverged, and the new translation is not identical to the shared one, keep diverged" rule.
<henninge> jtv: oh, I have to add the "is not identical" ...
<henninge> jtv: and that is not something "approveAsDiverged" does?
<henninge> diverged = suggestion.is_diverged
<henninge> jtv: ^ Can a suggestion actually be diverged?
<mrevell> Hi
<adeuring> good morning
<jtv> henninge: a suggestion can be diverged _in another template_.
<jtv> henninge: as for approveAsDiverged: that method is for explicitly diverging a message.  It always diverges (except where you're actually trying to set the shared translation as a diverged translation).
<jtv> but setCurrentTranslation checks if the current message is diverged and if so, maintains divergence.  (Again of course, except if the new translation is identical to the current shared one)
<henninge> jtv: I don't like setCurrentTranslations because it expects strings ... ;(
<henninge> which is kind of dumb, considering that I already have a message, and msgstr objects and all that.
<henninge> that's why I'd prefer using approve
<henninge> approveSuggestion
<jtv> henninge: shouldn't be that hard to changeâit's only used by tests, and other methods that can look up the POTranslation objects.
<jtv> I'm not saying you have to use it, but have a look if it serves your needs otherwise.
<henninge> jtv: well, "approveSuggestion" is really only a wrapper around setCurrentTranslation
<henninge> plus karma assignment
<henninge> jtv: So, I guess I don't need to check for divergence - submit suggestion will DTRT?
<jtv> submitSuggestion can't
<jtv> But I see what you meanâapproveSuggestion maintains divergence.
<jtv> I do see one other thing that approveSuggestion probably should do: disable a diverged message in the same pofile that may be masking the suggestion.
<jtv> It shortcuts the case where the suggestion is already current.  It probably shouldn't, though of course not shortcutting does mean that the karma assignment needs a bit of extra care.
<jtv> But yes, this means that in principle approveSuggestion is good enough for imports.  No need for approveAsDiverged.
<jml> rockstar, https://dev.launchpad.net/LEP/BetterPrivacy, https://dev.launchpad.net/LEP/BugzillaComponents and https://dev.launchpad.net/LEP/BuildFarmScalability all spring to mind
<henninge> jtv: I think approveSuggestion may have gotten the Karma handling wrong.
<henninge> jtv: this is from the failing test: http://paste.ubuntu.com/495203/
<jtv> henninge: I can see how that's different from what approveSuggestion does, but how is what approveSuggestion does wrong?
<henninge> jtv: well, different from documentation == wrong
<henninge> ;-)
<jtv> This pastebin leaves me with the impression that updateTranslation gets it wrong
<henninge> that is possible
<henninge> jtv: should a translator not get karma twice -- once for suggesting and once for suggestion being approved?
<jtv> a translator, yes
<jtv> but if it's the same person approving that same translation?
<jtv> That seemed wrong to me.
<henninge> jtv: so, what is karma trying to show?
<jtv> It is trying to reward people for doing useful things.
<jtv> I'm trying to find something wrong with my own earlier reasoning, but not finding anything so far.
<jtv> Yes, it does mean that a reviewer can hope to gain more karma from translating in translation modeâ_if_ they can get those suggestions reviewed by someone else.
<jtv> I don't think that's a bad thing.
<henninge> A translator that translates 10 strings that are all approved by a reviewer will get twice as much karma as a reviewer that enters the same translations directly.
<jtv> I think that's actually rewarding good behaviour.
<jtv> At first glance it seems to increase the risk of those translations staying unapproved.  But the author doesn't get the extra karma until they are.
<jtv> So a proper, well-motivated karma whore will try to get them reviewed.
<henninge> In total, a suggested-approved translation will deal out three times as much karma as a directly-approved will.
<jtv> To different people.
<henninge> jtv: I am just finding reasons, trying to understand. I am not against your reasoning.
<jtv> That's fine.
<henninge> So I should update the test.
<jtv> I think so, yes.
<jtv> BTW dogfood is being upgraded to lucid.  It'll take about an hour.
<henninge> So far, imported translation would give the uploader "translationsuggestionapproved" karma, now they will get "translationsuggestionadded" karma.
<henninge> Are they weighed differently?
<jtv> Don't know.
 * henninge consults launchpad_dev
<henninge> jtv: http://paste.ubuntu.com/495216/
<jtv> henninge: thanks for digging that up!
<henninge> looks like we have some more karma actions ...
<jtv> BTW another way of looking at the current karma design is, "karma shouldn't reward people for joining a translation team just to translate."
<deryck> Morning, all.
<jml> deryck, good morning
<jml> I'm going to head out to Foyles and see if I can pick up a book on queueing theory
<jml> hello
<bigjools> does the .dev instance run with multiple request threads?
<salgado> bigjools, yes
<bigjools> salgado: cool thanks
<bigjools> now I have a fighting chance of re-creating my concurrency issue
<bigjools> salgado: do you have any clue how I could write a test case that simulates simultaneous requests in 2 threads?
<salgado> bigjools, I think the test suite runs single threaded
<bigjools> yeah, that's why I asked :)
<jml> bigjools, I guess it all depends on how the test browser works
<bigjools> jml: I've only ever seen it manage single-threaded stuff
<bigjools> jml: I've re-created an issue locally where I need to bash the copy button on the PPA copy page as fast as I can
<jml> bigjools, well, I don't know how it works. It might just be that all you need to do is spawn two threads each of which has its own test browser
<jml> bigjools, hmm.
<jml> bigjools, in that case, you might not even need to worry about the test browser
<bigjools> jml: that's one way but it seems as though it might not be deterministic
<jml> bigjools, you are using threads, kiss determinism goodbye
<bigjools> that's why I said "simulate" in my original question :)
<jml> bigjools, then what's the problem with spawning two threads and using the test browser?
<jml> bigjools, or, actually
<jml> bigjools, you don't need the test browser, you just need to have instantiated view objects pointing to the same store
<bigjools> I'm starting to think that a test case is not worth it
<jml> bigjools, hang on
<jml> bigjools, is this for debugging the problem or writing a test?
<bigjools> a test
<bigjools> I know what the problem is, I think
<bigjools> the two transactions don't clash at all, so don't trigger a serialization issue
<bigjools> but the checks that are made work once one of them is committed
<jml> bigjools, otp
<bigjools> so I have an evil plan to make them clash some how
<bigjools> ok
<rockstar> jml, did you see my LEP?
<jml> rockstar, I saw that it exists. still otp.
<rockstar> jml, ack.
<jml> bigjools, do you know off the top of your head what times the Walter de Cantelupe is closed on Sundays?
<bigjools> jml: late
<bigjools> usually ~11 I think
<bigjools> but if you're going to be late I'd call and let him know
<jml> bigjools, istr Martin saying that it's closed for some time during the afternoon
<jml> bigjools, I'll be arriving at ~6pm
<bigjools> jml: I'd call him then, yes it is closed in the afternoons, I can't remember what time it opens
<jml> bigjools, will do.
<jml> bigjools, thanks
<bigjools> np
<jml> grr.
<jml> Do you know what would make most technology better?
<jml> Actually working.
<jml> rockstar, I just scribbled all over https://dev.launchpad.net/Code/MergeQueues/LEP
<jml> rockstar, basically adding more details in the requirements.
<rockstar> jml, great, I'll take a look.
<rockstar> jml, ah, yeah, most of those details we've already addressed in my notes at https://dev.launchpad.net/Code/MergeQueues but I'm happy to LEPify them.
<jml> rockstar, yeah. I see the notes cover similar ground in story format.
<jml> rockstar,
<jml> rockstar, but I want to get it down in a requirements format
<rockstar> jml, absolutely.  abentley and I were just going over the recipe LEP and reviewing what else needs to be done, so I'm identifying what is important about a LEP.
<jml> rockstar, cool.
<jml> rockstar, also, I'm really keen to see thoughts on the full cycle of any given feature: how do people find out about it; how does one use it for the very first time; how does one know it's safe to try out; what do the inner cycles look like; how does one  stop using it? etc
<rockstar> jml, yeah, and I think we're going to identify much of that with our UI mockups.
<jml> rockstar, agreed
<rockstar> (I have a list of them I need to do)
<jml> rockstar, it's easy to let UI mockups focus on the inner bits of the workflow rather than a more broad context, so just flagging now
<rockstar> jml, yeah, I actually like to address the more broad context in UI mockups.  They should be a general idea, not the final UI.
<jml> rockstar, also, I noticed on your notes page that you're thinking of having something on the branch page that says the branch isn't managed by a queue
<rockstar> The final UI gets sorted as you actually implement the feature and find out what works.
<jml> rockstar, for a project that uses feature branches, that's going to be irrelevant for most branches.
 * gmb EoWs. Night folks.
<lifeless> bac: hi, around?
<bac> hi lifeless
<lifeless> rev 11552 on stable
<lifeless> I'm not clear if that is ok to deploy to lpnet now, or not.
<bac> lifeless: b/c it is qa-untestable or b/c of pre-requisite branches
<lifeless> because of the description
<lifeless> 'show xx on lpnet'
<lifeless> I don't know if you intend that to wait for the next release
<lifeless> or if showing it on Monday would be ok.
<bac> lifeless: Monday is great
<lifeless> ok, cool
<lifeless> we're not qa'd all the way up to it, but as things move forward...
<bac> lifeless: we should have rolled it in 10.09 but it slipped my mind
<lifeless> Ursinha-lunch: hiya
<lifeless> bac: feature flags will help :)
<bac> lifeless: yes, they just weren't ready at the time...
<bac> lifeless: why do you do the endwith test at line 27 of https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/35774
<lifeless> completeness
<bac> lifeless: but when would it not pass?
<lifeless> oh ahh
<lifeless> thats a bug, I meant to put %3f or whatever in there.
<lifeless> I'll come back to it, its already in ec2 land - thanks for noticing.
<bac> ok, i wasn't sure if it was a bug or you were doing something clever
<lifeless> very very clever
<lifeless> so clever you put a tail on it and call it a ferret
<Ursinha> lifeless, hi
<lifeless> Ursinha: hi there
<lifeless> uhm, /me pages in
<lifeless> Ursinha: oh right, tagger stuff - how goes it? Anything blocking you?
<Ursinha> lifeless, I'm not blocked on anything right now
<Ursinha> lifeless, more trying to figure out how to solve the mess unique branches for several bugs introduced to the scenario :)
<lifeless> heh, sorry ;)
<lifeless> grah
<lifeless> ec2 fail
<lifeless>  File "./test_on_merge.py", line 50, in setup_test_database
<lifeless>    con = psycopg2.connect('dbname=template1')
<lifeless> psycopg2.OperationalError: could not connect to server: No such file or directory
<lifeless>        Is the server running locally and accepting
<lifeless>        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
<lifeless> ok, is the AMI bust or something?
<lifeless> I'm getting pg not connected errors consistently on ec2land
<lifeless> mars: ^
<lifeless> # Run all tests. test_on_merge.py takes care of setting up the
<lifeless> # database.
<lifeless> /var/launchpad/test/bin/py -t ./test_on_merge.py -vv "--subunit -vvv"
<lifeless> Traceback (most recent call last):
<lifeless> ...
<lifeless>  File "./test_on_merge.py", line 50, in setup_test_database
<lifeless>    con = psycopg2.connect('dbname=template1')
<lifeless> psycopg2.OperationalError: could not connect to server: No such file or directory
<lifeless>        Is the server running locally and accepting
<lifeless>        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
<maxb> What is the status of Python 2.6 / datacentre lucid upgrades these days?
<Ursinha> lifeless, help me with something here
<Ursinha> lifeless, latest deployed revision from stable on lpnet was 11515, now the script states it's 11546
<Ursinha> while True: head -> desk
<lifeless> Ursinha: heh
<Ursinha> lifeless, no, not funny
<lifeless> Ursinha: looking at stable branch ?
<Ursinha> deployments.check_status() is returning that revision as latest deployed
<lifeless> thats right
<Ursinha> production-stable
<lifeless> its looking at the branch
<lifeless> its correct; it goes live Monday
<Ursinha> wait, what goes live Monday
<lifeless> check LaunchpadProductionStatus
 * Ursinha looks
<lifeless> jelmer: you need to update that page now you've landed your branch on production-devel
<lifeless> Ursinha: under 'requested cherrypicks'
<Ursinha> lifeless, oh
<Ursinha> so production-stable already has that
<Ursinha> ?
<lifeless> yes
<Ursinha> ah
<Ursinha> *whew*
<lifeless> working as designed
<Ursinha> I was about to kill someone
<lifeless> its not live on the appserver because doing it friday isn't the smartest thing for a big merge like that
<lifeless> not when we were low-losa to boot
<lifeless> jcsackett: ping
<lifeless> https://bugs.edge.launchpad.net/launchpad-registry/+bug/597738
<lifeless> thats the next blocking unqa'd commit
<jcsackett> the one for bugs?
<lifeless> jcsackett: rev 11547
<lifeless> jcsackett: but the process looks at a whole bug, not one branch-for-it.
<lifeless> jcsackett: its qa-needstesting at the moment, if the bugs bits are doing what they should on staging/edge, please qa-ok it
<Ursinha> do that and I will run the script
<jcsackett> i can ping EdwinGrubbs about the branch he landed to QA; the remaining hasn't reached fix-committed yet. Blueprints should be through buildbot shortly.
<lifeless> jcsackett: 11547 is what I care about right now.
<Ursinha> jcsackett, so it's part of a larger fix?
<lifeless> in general its a bit odd to do what this bug is doing
<sinzui> I QAed bugs yesterday and reported bugs.
<lifeless> sinzui: so should it be qa-bad ?
<sinzui> no
<sinzui> it is still an improvement
<lifeless> so its qa-ok?
<EdwinGrubbs> jcsackett: my branch was qa'ed fine. I didn't update the bug since it is is just going back to qa-needstesting when the next bugtask is completed.
<sinzui> yes
<Ursinha> cool
<lifeless> EdwinGrubbs: it needs to go to qa-ok to deploy
<jcsackett> EdwinGrubbs: okay, i thought you already had done that.
<lifeless> otherwise it blocks all commits after it
<Ursinha> EdwinGrubbs, you can tag the bug when your branch lands
 * sinzui found to pre-existing issues that must be fixed to say bridging-the-gap is done
 * jcsackett is confused about the QA process.
<EdwinGrubbs> oh, I forgot about that.
<jcsackett> are we not supposed to use needstesting anymore?
<lifeless> jcsackett: yes, you should. You should also test asap.
<Ursinha> jcsackett, yes, of course :) but you're saying qa-needstesting for a branch
<sinzui> Wake up, QA
<lifeless> jcsackett: https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt
<Ursinha> you landed a branch that fixes your bugtask in that bug, that lands, it's marked qa-needstesting for your landing
<sinzui> notice staging updates QA...QA is the fastest way to unblock the team
<lifeless> or on edge
<jcsackett> okay, that all jives with what i thought. have caught bits and pieces of conversations today that have left me off kilter. :-P
<EdwinGrubbs> jcsackett: I just treated this bug differently since it has multiple bugtasks. I didn't think about how it needs to go to qa-ok temporarily even though it will go to qa-needstesting when the next branch lands.
<lifeless> because edge runs stable
<sinzui> We were 3 hours from exceeding our limits 3 days ago
<lifeless> EdwinGrubbs: I would have done multiple bugs for this thing
<lifeless> because the work going on in each one is different
<EdwinGrubbs> yes, hindsight is 20/20
<lifeless> never too late to change ;)
<lifeless> anyhow yes, this isn't a big issue
<lifeless> we're changing process
<lifeless> so we're going to have muscle memory to unlearn and so on
<jcsackett> i'll keep this in mind tomorrow--the blueprints task should be ready to qa then.
<sinzui> I think the registry team can take ownership of EmailAddress if the foundations team have given up using the table for SSO
<lifeless> wgrant: please qa https://bugs.edge.launchpad.net/soyuz/+bug/564491
<lifeless> wgrant: and https://bugs.edge.launchpad.net/soyuz/+bug/566339
<Ursinha> lifeless, so I cannot mark that bug as qa-ok?
<lifeless> Ursinha: I don't know
<lifeless> Ursinha: hang on, which bug :)
<Ursinha> hehe
<Ursinha> https://bugs.edge.launchpad.net/launchpad-registry/+bug/597738
<Ursinha> the one holding the script
<lifeless> jcsackett: so
<lifeless> nvm
<lifeless> Ursinha: yes, sinzui said its ok
<Ursinha> hehe, cool
 * sinzui just changed is since his team are too frightened to say it is okay
<Ursinha> oh, I've done that too
<lifeless> Ursinha: I expect it to stop at 11556
<Ursinha> let's see
<sinzui> Ursinha, while I promised flacoste that I would let my team do QA, I need to schedule work. I qa every morning, even weekends, and I mark something bad if it really is bad
<Ursinha> even weekends
<sinzui> Ursinha, The delay is waiting for a registry developer to do his part
<sinzui> I have to review projects on the weekend too
<Ursinha> lifeless, that's correct, it stopped at 11556
<lifeless> Ursinha: still, thats another good batch of revs to land
<Ursinha> yeag
<Ursinha> yeah
<lifeless> sinzui: all teams are going to have to tune their processes, I wouldn't stress about it
<lifeless> sinzui: it'll become very clear the shorter the pipeline to production gets
<lifeless> sinzui: and once we stop edge autodeployments in favour of QA on stagingqa
<sinzui> lifeless, I think there is resistance to interrupting a task. When you are coding the a branch waiting for to QA your work, you must be will to stop coding when that work come to QA. Our team hit the QA limit twice last release Bac had to remind me that staging updated a few days ago so that he cold move a card
<lifeless> sinzui: that makes sense to me; but the price for avoiding that interrupt is a massive team wide interrupt every month, which everyone agrees they like less
<sinzui> As you can image, I have a very set pattern in the morning, lunch, and evening. I look like I am on top of things, but I just schedule repetitive tasks like QA and question every few hours
<lifeless> sinzui: I think if everyone looked for QAable stuff at the start of work, after lunch, and before signing off, we'd be in good shape
<lifeless> https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=qa-needstesting,qa-bad
<lifeless> thats one thing for every 2 people: not terrible, but not brilliant either
<lifeless> jcsackett: https://bugs.edge.launchpad.net/launchpad-registry/+bug/576757
<jcsackett> lifeless: qaing it as we speak, actually. :-P
<sinzui> lifeless, we are doing that one now
<lifeless> \o/
<sinzui> jinx
<lifeless> is there some other channel where you coordinate this?
<sinzui> lifeless, @lp-registry explaining how I pick a victim to test with can be a delicate subject
<jcsackett> lifeless: when i have no earthly idea how to set up the conditions to test i bug sinzui in another channel. :-)
<sinzui> #lp-registry
<lifeless> public or private?
<sinzui> lifeless, private
<Ursinha> I'm heading for the weekend
<Ursinha> will return later
<lifeless> sinzui: I'll hang out there if its ok; I may every now and then point stuff to this channel (I think we do way to much on the quiet). But you can tell me if I'm wrong :)
<lifeless> Ursinha-afk: ciao
<lifeless> Question:+huge-vocabulary 50.67 <-99% under
<lifeless> wheee
<jcsackett> alright, that's both of the bugs i can hit qa'ed. off to the weekend.
<lifeless> ciao
<jcsackett> g'night.
#launchpad-dev 2010-09-18
<wgrant> lifeless: I'm pretty sure that 566339 is not fixed by the branch in question.
<wgrant> And 564491 needs somebody to drive DF.
<lifeless> wgrant: is it made worse?
<wgrant> No.
<wgrant> Just not fixed.
<lifeless> so remove qa-needstesting from it
<lifeless> for the other can you arrange that with StevenK ?
<wgrant> True.
<wgrant> Well, can somebody else remove qa-needstesting? It's not my bug, and I'm not really Soyuz, so I don't really want to screw with their QA :)
<lifeless> revno: 11556 [merge]
<lifeless>   [r=stevenk, thumper][ui=none][bug=564491,
<lifeless>         566339] Fix an OOPS when an archive admin uses the +queue page to
<lifeless>         accept uploads that close private bugs.
<lifeless> its qa-needs testing because of that landing
<wgrant> It is, yes.
<lifeless> oh, bigj
<wgrant> bigjools thinks it fixes it, but I'm pretty sure it's irrelevant.
<lifeless> sorry
<lifeless> you only filed it ;)
<wgrant> Yep.
<wgrant> sinzui: What's the UI for pages like +configure-translations going to look like?
<sinzui> a lot like bugs and answer
<sinzui> oh
<wgrant> Because the current one that almost made it out to production is very confusing :(
<lifeless> wgrant: what rev?
<wgrant> lifeless: The one I mentioned yesterday. r11552
<wgrant> So not in your initial set.
<lifeless> wgrant: it is in the second set
<lifeless> Revision 11547 can be deployed: qa-ok
<lifeless> Revision 11548 can be deployed: qa-ok
<lifeless> Revision 11549 can be deployed: qa-ok
<lifeless> Revision 11550 can be deployed: qa-ok
<lifeless> Revision 11551 can be deployed: qa-ok
<lifeless> Revision 11552 can be deployed: qa-ok
<lifeless> Revision 11553 can be deployed: qa-ok
<lifeless> Revision 11554 can be deployed: qa-ok
<lifeless> Revision 11555 can be deployed: qa-ok
<lifeless> Revision 11556 can not be deployed: not-ok
<wgrant> This is what I was worried about.
 * wgrant wonders why it isn't FF'd out.
<lifeless> wgrant: I explicitly asked about it
<lifeless> wgrant: and sinzui believes its an improvement even though not perfect
<sinzui> wgrant, the registry team is only worked on the unknown, not-applicable, launchpad enum. The translations team can consider doing a unified page that does everything in one view like bugs
<wgrant> sinzui: IMO the current translations/answers UI is a regression.
<wgrant> What does it even mean?
<wgrant> I don't know -- how will users?
<lifeless> wgrant: wgrant huh
<lifeless> wgrant: sample url? I don't recall it being confusing
<wgrant> lifeless: https://edge.launchpad.net/launchpad/+configure-translations
<sinzui> wgrant,  I do not think so. consistent UI  is better, being clear about what is being used or where something can be found is better.
<wgrant> sinzui: Why not FF it out for a couple more weeks, until the new pages are built?
<wgrant> And not roll out a half-finished awesome feature?
<wgrant> Awesome features are much better when people don't see broken implementations first.
<sinzui> wgrant, I do not understand. My entire team is working on this and we need edge feed back there are only a few things left to be done
<sinzui> I think there is plenty of time to complete this
<wgrant> sinzui: edge is fine.
<wgrant> I don't think lpnet is the best idea yet.
<sinzui> god knows spending a year on a so-called 6 month vision has burned me out
<wgrant> Heh.
<sinzui> My only regret is that I still think the DSP page is near useless and I want to be able to state where external questions and FAQs can be found
<wgrant> We need to work out a strategy for linking them better to projects.
<wgrant> Particularly since there are a lot of Canonical projects now that only want to use one bug tracker.
<lifeless> s/Canonical/Ubuntu ?
<wgrant> lifeless: Sometimes.
<wgrant> But if you categorise Ubuntu One as Ubuntu, I will not be amused :)
<lifeless> u1's ubuntu-only components are open source
<mars> lifeless, mind if I bug you with a question about bug 598289?
<lifeless> I think *if* a healthy set of packages in say fedora nd suse were to spring up, that they would see the need for 2 trackers for those components.
<lifeless> mars: shoot
<mars> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/598289
<wgrant> lifeless: They're not Ubuntu-only, but OK.
<lifeless> wgrant: the developers are only targeting ubuntu today;
<lifeless> unless something radical has changed
<wgrant> Yes, but there's no reason they can't run elsewhere. Like Software Center, and other stuff like that.
<mars> lifeless, So the question was: is it safe to use feature flags to protect access to profile generation.  The answer was... yes, I think?  zope-funcertainty
<mars> lack of zope-fu leads to zope-funcertainty
<lifeless> wgrant: thats true; the question is whether upstream get bug reports that aren't relevant to Ubuntu
<lifeless> wgrant: and until they are geting patches and so forth routinely, the broad answer is going to be 'no'.
<wgrant> True.
<lifeless> wgrant: which is why I talked about healthy packages in other distros
<lifeless> mars: certainly
<lifeless> mars: you'll need two patches.
<lifeless> mars: one patch to add a team membership scope
<lifeless> see rev 11536 for a very similar scope definition that does team membership scoping
<lifeless> bah
<lifeless> that does pageid scoping
<lifeless> and please add it to https://dev.launchpad.net/LEP/FeatureFlags at the bottom
<lifeless> mars: or fix the self-documenting bit
<lifeless> because team lookup is a db thing, we probably want a note that using that scope will trigger team membership checks, so we don't go overboard on it.
<lifeless> mars: secondly, you need a patch to force 'permit_profiling' or whatver it is on via a feature flag
<lifeless> mars: see e.g. the memcache feature flag patch for an example of what that might look like
<lifeless> these two patches are entirely unrelated to each other
<lifeless> but together much power can be had
<wgrant> If you don't want to hit the DB, you could use the pre-populated developer flag in ILaunchBag.
<wgrant> That's populated for every request.
<lifeless> thats true
<lifeless> we could have team:<name> and special case well known names.
<wgrant> You could.
<lifeless> but
<lifeless> that shouldn't be needed if its populated sanely we'll get cache hits..
<lifeless> you could also have a flag called user_is_developer
<lifeless> which is less general but fits the actual intent here a little more closely.
<lifeless> I'm easy.
<lifeless> bah
<lifeless> not flag
<lifeless> *scope* called user_is_developer
<lifeless> mars: I'd add the flag ('permit_profiling') or whatever name you choose, first.
<lifeless> mars: because onces thats added sysadmins can turn profiling on and off on staging without restarting it
<mars> cool
<mars> this is why I like pre-implementation calls :)
<lifeless> I think flag are still misunderstood
<mars> how so?
<mars> and what is the 'fix the self-documenting' bit?
<lifeless> (third discussion I've had today showing the difference between scopes/flags and how it hangs together.
<wgrant> Does the infrastructure support scope conjunction?
<lifeless> mars: no
<lifeless> for a given flag, the rules are evaluated in priority order
<lifeless> mars: self documenting: we should be able to query the code to find out what scopes(including pageid: magic ones) and flags (like memcache, gmaps2 etc) exist
<lifeless> this would let us:
<lifeless>  - display that as help in the fictional editing UI
<lifeless>  - not have a redundant list in the wiki page
<mars> find -iname '*py' -print0 | xargs -n0 grep getFeatureFlag | sort -u
<lifeless> mars: you'll miss all the page template ones
<mars> TAL flag isn't mentioned in the LEP
<mars> only in the listmail
<lifeless> I'm not sure there are ones deployed in tal yet, but there probably are and we should have some way of documenting them
<lifeless> mars: find might be the answer
<mars> find -iname '*py' -o -iname '*pt' -print0 | xargs -n0 grep -e 'getFeatureFlag' -e 'features/' | sort -u
<mars> yuck
<mars> do two find calls
<mars> add a 'make doc' target
<mars> and do the work in there
<lifeless> mars: what might be nice is a makefile rule to do it
<mars> along with the API docs
<lifeless> I'd like something out that the admin UI can include and show
<lifeless> mars: in templates getFeatureFlag isn't it
<lifeless> its view/flags/flagname
<lifeless> I think
<mars> <div tal:condition="features/show_message">hello world!"</div>
<lifeless> ah yes
<lifeless> thats it
<mars> lifeless, so, the difference between flags and scopes: a flag only applies in-scope I assume?  Sounds like a scope is a flag to turn on and of a set of flags
<mars> or override a flag
<wgrant> lifeless: Can I see OOPS-1721K2125?
<mars> so you could have feature:X scope:default :off
<mars> and feature:X scope:server.edge :on
<mars> Which saves you from having to write:
<mars> if feature:X and feature:server.lpnet then :on
<mars> err
<mars> if feature:X and feature:server.lpnet then ...
<mars> which is messy.  You have to change the code if either of those two assumptions needs to change.
<lifeless> mars: code is controlled by flags
<lifeless> the value of a flag is determined by the highest priority rule with a matching scope
<lifeless> rules are a tuple (scope, flag, value) sorted by priority
<lifeless> mars: so yes, you have the point
<lifeless> we define a name for condition we want to control
<lifeless> and use rules to dynamically implement the particular flow control we want
<mars> \o/ I get yet another opportunity to use the awesome bzr pipeline plugin
<lifeless> mars: for this?
<mars> lifeless, three patches, so yes
<lifeless> mars: personally I wouldn't, they are totally separate patches
<mars> flag, scope
<lifeless> mars: unrelated
<lifeless> mars: thats the point really ;)
<mars> lifeless, well, you could add the flag in one patch, and not use it
<mars> then add the scope, and use it as well
<lifeless> mars: huh? once the flags added we can use it.
<mars> lifeless, I must be missing something: when I write getFeatureFlag(), I have to pass in a scope, no?
<lifeless> NO
<lifeless> you pass in a FLAG
<mars> so where did I get the wrong idea from...
<mars> reading over what I read again
<mars> Ok, so I thought I had read getFeatureFlag(flag, scope=foo) somewhere as the API.  That is wrong, and I can not find the reference.  Maybe I was confusing the "getFlag(scopes => None)" line from the LEP with the actual implementation.
<lifeless> heh
<mars> ok, this stuff needs a cheat sheet
<lifeless> wgrant: that oops
<wgrant> lifeless: That oops?
<lifeless> ywa
<lifeless> looks the same as the one in https://bugs.edge.launchpad.net/soyuz/+bug/276950
<wgrant> Ah, yes.
<wgrant> In fact it's the same OOPS.
<wgrant> I was interested in the content.
<wgrant> Because I don't see what is so slow.
<wgrant> Unless it's the bad copy checker stuff.
<lifeless> its a different oops #
<lifeless> wgrant: the huge numbers of repeats
<wgrant> lifeless: It's the one in comment 3.
<lifeless> are at least 2.5 seconds
<wgrant> Ew.
<wgrant> Oh, there are details in the description. I see.
<lifeless> 600 bug lookups
<lifeless> 200 das lookups
<lifeless> 200 archive lookups
<wgrant> Ew.
<lifeless> its just horribly inefficient
<wgrant> I might be able to reproduce it locally, then.
<lifeless> if you can't, shout
<lifeless> the params aren't visible so I can't see what packages are being manipulated
<lifeless> I'd guess at something stupid like all-bugs-ever-on-the-package being manipulated
<lifeless> one at a time
<lifeless> with an added helping of 'cache miss due to complex query results in reading what we had already' or some such
<mars> komodo only found 5 occurrances of the string getFeatureFlag() in the devel source tree... odd.
<lifeless> looks a little few
<lifeless> but not far off
<mars> Martin's mail said to "use lp.services.features.per_thread.features to look things up"
<mars> so maybe people were doing that
<lifeless> no
<lifeless> because it has getFeatureFlag as the method
<mars> hmm
<lifeless> there are < 10 flags in use - I know of three in python code, and a couple in templates
<mars> lifeless, which three in Python?  I can look up the exact string/symbol
<lifeless> getFeatureFlag
<lifeless> I documented them on https://dev.launchpad.net/LEP/FeatureFlags
<lifeless> 11519 claims to be behind a flag...
<mars> ah, like I said, my editor only found 5 occurrances of getFeatureFlag in the code base.  Over to grep!
<lifeless> oh fugly
<lifeless> folk are using FeatureController and in_scope manually.
<lifeless> nooooo
<mars> see?  Quickstart guides needed :)
<mars> Looks like rf-get did the wrong thing
<mars> and didn't update my source tree
<mars> eh?  No - 'bzr switch -b' did something odd instead
<lifeless> mail sent to the list
<lifeless> ok, we need to fix up
<lifeless> distroseries.py
<mars> lifeless, did you work around that ec2 database issue you had earlier?
<lifeless> mars: no, I'm assuming ec2 AMI is fucked and won'y try yo land stuff till tuesday
<mars> ugh, sorry :(
<lifeless> and registry/browser/__init__.py
<lifeless> mars: possibly a postgresql dep change or something
<mars> lifeless, should feature flags return a boolean?  Because the memcached one does "features.getFeatureFlag('memcache') != 'disabled'"
<mars> which feels a bit strange - what other values for the flag could there be?  'maybe'?
<lifeless> mars: they return the value
<lifeless> mars: look up the bug for timeouts via flags
<lifeless> I pasted a sample interpreting flag there
<lifeless> mars: other values are useful if we wanted to (for instance) keep memcache on a couple of pages but disable it globally
<lifeless> we'd set a priorty (say) 50 rule for (default, memcache, disabled)
<lifeless> and add priority 0, 1, 2 etc for
<lifeless> (pageid:pagetousememcacheon, memcache, '')
<mars> :/
<lifeless> mars: __nonzero__ is perfectly cromulent python
<lifeless> as long as it doesn't have unusual consequences (like being terribly slow, or some such)
<mars> lifeless, comparison to the string 'disabled' for a flag is a poor API imho.  featureIsEnabled('foo') => returns Boolean True or False, sounds like it would cover 99% of the cases
<mars> with this code, bool(getFeatureFlag('memcache')) == True, even when the memcache flag's value is 'disabled'
<mars> that's just not right
<mars> anyway, I need to run, have to put the kids to bed. ttyl
<lifeless> mars: ttyl
<lifeless> mars: I think bool would be overly restrictive for no benefit
<lifeless> mars: this is schemaless. Its a *benefit*.
<lifeless>   Hard / Soft  Page ID
<lifeless>      164 /  335  BugTask:+index
<lifeless>      107 /  277  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<lifeless>       70 /   16  Distribution:+search
<lifeless>       39 /   18  DistributionSourcePackage:+filebug
<lifeless>       16 /    0  Product:+bugs
<lifeless>       14 /   69  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>       14 /   18  DistroSeries:+queue
<lifeless>       13 /   43  Archive:+packages
<lifeless>       13 /    4  Archive:EntryResource:syncSource
<lifeless>        8 /    4  Person:+rdf
<lifeless> mars: I've tweaked the LEP
<mars> lifeless, by schemaless, do you mean key/value pair storage?  Don't we get that already from lazr.config and from memcache?
<lifeless> mars: lazr.config is horrific
<lifeless> mars: and memcache is transient
<lifeless> mars: by schemaless I mean no defined schema
<lifeless> this is more than just keyvalue (but not a great deal more)
<mars> lifeless, ok, put another way: is there a problem with:
<mars> if getFeatureFlag('foo'):
<mars>   bar = config.feature.value
<lifeless> yes
<lifeless> it will take hours to change.
<lifeless> it will be static
<mars> right
<lifeless> we'd need a new config.feature section for every rule in the DB
<lifeless> and rules are dynamic, so I don't see how we'd do that
<mars> ok
<lifeless> lazr.config is a very powerful, very complex, schemad system which pushed all integrity checking out to the setting-of-it
<lifeless> this has the following consequence:
<lifeless>  - its hard to use
<lifeless>  - and complex to use.
<lifeless> we're now, judging from what people are doing, abandoning it rapidly.
<lifeless> I wouldn't be surprised if folk route around it completely in the next 12 months and we have a bug called 'remove the last dependencies on it'
<mars> lifeless, so lazr.config stores data - what are they using instead?
<lifeless> stub has added ini files
<lifeless> poolie added feature flags
<lifeless> stub is looking with interest at zookeepr
<lifeless> who knows what else will turn up.
<lifeless> lazr.config is also very intolerant, making it hard to evolve.
<lifeless> From what I've seen of it, I'd be entirely happy to just switch to configparser now
<lifeless> but perhaps I've missed something.
<lifeless> (but even doing that owuldn't do what feature flags do).
<lifeless> feature flags are: very powerful, very simple, very dynamic.
<mars> lifeless, ok, then I would suggest a convention: 'on' or None
<lifeless> we can do A/B testing; dynamic configuration and more
<lifeless> mars: Why?
<lifeless> mars: what do we gain by limiting it
<mars> lifeless, because then bool(getFeatureFlag()) still works everywhere
<lifeless> mars: what value does that *have*
<mars> lifeless, bool('on') == True, bool(None) == False.  That makes sense.  The memcached flag (and it is a Bool flag right now) doesn't follow that convention.
<lifeless> because that convention doesn't exist.
<lifeless> And I don't see what value we'd get from it.
<lifeless> We'd destroy the usefulness of flags.
<mars> how?
<lifeless> for a consistency we don't need.
<mars> I am not saying "only use 'on' or None"
<lifeless> mars: did you look at the bug I suggested you do, about pageid based timeout control?
<mars> I am saying "if it is a flag, try to stick to 'on' or None
<mars> if it is not a flag, but a keyvalue, then you're on your own :)
<lifeless> its not a flag, its not a keyvalue
<mars> lifeless, I saw the example you pasted in the channel, but not a bug #
<lifeless> mars: look at launchpad-foundations bug
<mars> found it, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/627701
<sinzui> I think my weekend plans just went tits up. Has anyone noticed that ec2 images use PG 8.3, but that our own scripts to make the environment are 8,4?
<mars> lifeless, ^ that might explain the error you saw
 * sinzui thinks
<mars> did someone land a fix to the scripts to get Maverick running?
<sinzui> I do not know I think there is just one real error in setup: /etc/init.d/postgresql: command not found
<sinzui> I can update my branch to try 8.4 then fallback to 8.3
<sinzui> ah, we have two script and the image does not know which one to use. This may be a one line fix
<mars> sinzui, yep: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/11570
<mars> sinzui, consider switching to the /usr/sbin/service utility
<mars> then it will always do the right thing
<sinzui> okay thanks for the advice.
<wgrant> How will that work with version selection, though?
<lifeless> wgrant: no idea
<sinzui> wgrant, I see I can pass the version to service, but I do not know if service supports version in Hardy. I assume there are still some machines on Hardy
<lifeless> we stil have dc machines on hardy
<lifeless> but I don't know that we run any developer pg controlling scripts on them
<sinzui> take the safe path by adding a switch between the old command and the new one
<lifeless> james_w: http://pypi.python.org/pypi/fixtures updated
<sinzui> wgrant, lifeless. My pg hack played on my maverick instance and in ec2. If my branch passes, we will have a fix in the tree
<lifeless> sinzui: thanks
<lifeless> we'll need that CP'd to production-devel
<lifeless> by hand
<lifeless> or is it the local content on disk when you submit that controls this?
<wgrant> Why does production need it?
<wgrant> Isn't production still 8.3, so EC2 should probably use 8.3 too?
<lifeless> wgrant: we need to keep running ec2 land for prod-devel branches
<lifeless> do you have reason to think that that isn't affected in the same way?
<wgrant> True.
<lifeless> one way to make sure its not affected is to run pg 8.3 and only 8.3 on it and make the dev scripts do stuff for 8.3; I don't know if thats the case.
<lifeless> its also of minimal use because production doesn't take db patches
<lifeless> it hatses them
<wgrant> Although there was a bit of specialness WRT that this cycle...
<lifeless> this is not the patch you are looking for
<wgrant> Hm.
<wgrant> The Django Debug Toolbar is handy.
<lifeless> wgrant: how are you going with the +queue perf thing
<wgrant> lifeless: Doing some uni stuff first. +queue is being attacked this evening.
<lifeless> wgrant: K, I'll probably be around if you want to bounce ideas
<wgrant> lifeless: I was just adding an "assuming that maverick continues to be well-behaved"... when I got my first X hang in a few months.
<wgrant> Ahem.
<lifeless> :)
<wgrant> lifeless: Do you know if there's a known issue with the revision karma allocator?
<wgrant> I've checked a few users, and nobody seems to have any in the last couple of days.
<wgrant> Hm, there was some two days ago.
<wgrant> So maybe it just didn't run yesterday.
<lifeless> not aware of any incident report about it
 * StevenK kicks Maven until pieces fall off
<lifeless> anyone know the list that zope testing patches and plans are designed on?
<lifeless> wgrant: https://bugs.edge.launchpad.net/soyuz/+bug/629921 -please-
<lifeless> I'm begging you.
<wgrant> Hmm. That was filed two weeks ago, and we are around 12000 bugs further along now... does that mean we are having 1000 filed a day!?
<wgrant> But anyway, let's see how difficult that is...
<wgrant> Ahh, I see.
<wgrant> lifeless: Am I allowed to violate the no-implicit-truth guideline? That would reduce the fix to deletion of a few characters...
<lifeless> wgrant: no implicit truth?
<wgrant> lifeless: It's suggested not to say 'if whatever' -- instead we are told to say 'if whatever is not None'
<wgrant> This particular case is currently 'if requested_name_filter is not None', but actually wants to be 'if requested_name_filter is not None and len(requested_name) > 0'. But 'if requested_name_filter' would work just as well.
<lifeless> if requested_name_filter is clearer IMO
<lifeless> doit.
<wgrant> Excellent.
<lifeless> given I just wrote email to the list suggesting *just that* for flags earlier today :)
 * wgrant fixes the Maverick psycopg2 compatibility breakage.
<lifeless> its compatible; honest.
<wgrant> It is, but our celebrity code sucks.
<wgrant> This does not inspire confidence:
<wgrant>     def source_fies_dict(self, package_upload_source_dict, source_files):
<wgrant>         """Return a dictionary of source files keyed on PackageUpload ID."""
<wgrant> WTF
<wgrant> WTF
<wgrant> WTF
<wgrant> lifeless: So... DistroSeries:+queue redirects after post.
<wgrant> lifeless: I was looking around to see how the action handler was called.
<wgrant> It turns out it's called by the template.
<wgrant> After it calculates all the data to display in the event of a GET.
<wgrant> Gar.
<lifeless> wgrant: hah
<lifeless> I think I speculated on that somewhere.
<lifeless> like in the bug or something
<wgrant> Heh, so you did.
<wgrant> I knew it redirected, so I didn't suspect this... but I went digging, and couldn't find anything calling the handler.
<wgrant> Sigh.
<lifeless> I'm glad you've confirmed it
<lifeless> it suggests an immediate obvious improvement ;)
<wgrant> Heh, yes.
<wgrant> Now to work out how to make it not do that.
<lifeless> is it a systemic thing
<lifeless> that is, do all our template render-the-GET< process-the-POST ?
<wgrant> No.
<wgrant> This one isn't a real form.
<lifeless> whew
<wgrant> So it's speical.
<lifeless> I would have cryed.
<wgrant> I wonder if I can stuff it into a LaunchpadFormView.
<lifeless>  3 files changed, 16 insertions(+), 57 deletions(-)
<wgrant> What's that?
<lifeless> ported the fake librarian to python-fixtures
<wgrant> Aha.
<lifeless> I might go on a code deletion wander next week
<lifeless> might be fun
<lifeless> then again, I might not
<wgrant> lifeless: What are the columns in the repeated statement report (looking at the +queue bug description)?
<lifeless> RepsTotal timeAverage timeSavingDatabase idStatement
<lifeless> thats
<lifeless> reps total avg saving db statement
<wgrant> Ah, thanks.
<lifeless> for some reason irssi + screen turns columns from chrom into '' rather than '\t'
<lifeless> sinzui: when you see this, I'd love to know how to show multiple batchednavigators on a single template. lib/lp/bugs/templates/bugtrackers-index.pt specifically.
<lifeless> night all
<sinzui> I have an example for you
<sinzui> person.TeamMembershipView uses two subclasses of BatchedNavigator, ActiveBatchNavigator and InactiveBatchNavigator
<JesperHansen> Seeing that launchpad is starting to modify bugreports on mozilla's bugzilla since yesterday with "See Also" links. What launchpad bug report is related to this change?
<lifeless> JesperHansen: I'm not sure it was a bug report that drove it, rather a long term plan
<lifeless> https://edge.launchpad.net/bugzilla-launchpad/bugzilla-3.0
<JesperHansen> lifeless: seems that is almost year old
<lifeless> https://edge.launchpad.net/bugzilla-launchpad/bugzilla-3.2/1.0-3.2.2 is a bit newer
<JesperHansen> lifeless: Is "Code" page any indication of if the project is active? Says many weeks. What we're seeing is that feedback@launchpad.net has started yesterday to modify bugs
<lifeless> what bug tracker?
<lifeless> ah mozillas
<lifeless> so my guess - and its only a guess - is that this was designed way back and implemented in the plugin
<JesperHansen> And it would seem to require some rework as its modifying bugs See Also' on which the bug is RESOLVED DUPLICATE.
<lifeless> JesperHansen: please file a bug about that
<lifeless> launchpad.net/malone
<JesperHansen> yea, Just trying to figure out why it has begun in the first place
<lifeless> JesperHansen: we started syncing *into* launchpad this last week
<lifeless> I think the plugin is doing the comments, not launchpad
<lifeless> that is, Launchpad is saying 'we've synced comments to <location>' and the bugzilla plugin is noting and recording that.
<lifeless> you'll have seen a lot have that happen because we've only just enabled the syncing into launchpad
<JesperHansen> k, just beginning to see this random behaviour is hanging feedback@launchpad.net on the edge of getting a ban
<lifeless> so
<lifeless> first step
<lifeless> please filea  bug
<lifeless> if it is the plugin causing the comment to be added, banning lp won't stop it :P
<lifeless> mkanat will know more than I about how its meant to hang together
<JesperHansen> mkanat. I will try to remember that name
<lifeless> max kat-alexander
<lifeless> bugzilla upstream :)
<lifeless> I think?
<lifeless> sorry, Kanat-Alexander
<lifeless> https://edge.launchpad.net/~mkanat
<JesperHansen> That name actually rings a bell. Saw some comments or bug reports actually related to launchpad with him on bugzilla
<JesperHansen> I am guessing that this bug https://bugs.launchpad.net/malone/+bug/596931 relates to the search for dupe'ness. But seems incomplete.
<mkanat> Hallo.
<mkanat> lifeless: I am around. :-)
<lifeless> JesperHansen: no, that bug is a little different
<lifeless> mkanat: hi; JesperHansen here is finding 'see also' links on resolved-duplicate bug reports in mozilla's bugzilla instance
<lifeless> mkanat: since this week (which is when we turned on sync-into-lp for buzilla trackers)
<lifeless> mkanat: I seem to recall you knowing a bit about this stuff :)
<mkanat> lifeless: Oh, I think that gmb maintains the launchpad side of that.
<mkanat> lifeless: I did write the Bugzilla side of it, though.
<JesperHansen> Noticed when I got the 4th email when feedback@launchpad.net modified the 4th bugreport
<lifeless> mkanat: yeah, I think so to; I don't know though that lp would trigger a see-also unconditionally, would it ?
<lifeless> gmb: ^
<mkanat> lifeless: I think that would mostly be a question for gmb.
<lifeless> JesperHansen: gmb is in a uk tz, so won't be around now; he may well pop in in 9-10 hours
<JesperHansen> lifeless: excellent. I am CET. Just 1 hour ahead of him
<lifeless> JesperHansen: please file a bug though
<lifeless> or say that you want me too
<mkanat> lifeless: My suspicion is that launchpad is linking unconditionally.
<JesperHansen> lifeless: I never found out how to use launchpad
<JesperHansen> mkanat: but wasn't the plugin denied to be set in use? https://bugzilla.mozilla.org/show_bug.cgi?id=481161
<mkanat> JesperHansen: Bugzilla 3.4 and above don't need any plugin to interact with Launchpad.
<JesperHansen> Though you do have https://bugzilla.mozilla.org/show_bug.cgi?id=474902 which was kinda allowed over a year ago
<mkanat> JesperHansen: Canonical is pretty much the entire reason that the See Also field exists--they funded the development of it (although it's something we wanted anyhow).
<JesperHansen> mkanat: which was what I talked to lifeless about that I remembered seeing a bugreport or two with you :)
<mkanat> JesperHansen: Ah, yeah. :-)
<lifeless> JesperHansen: do you have an LP account?
<lifeless> JesperHansen: or an Ubuntu one account?
<JesperHansen> lifeless: Its 1am here and its pirate day, so I hope you can do it ;)
<JesperHansen> Trying to restrict myself from doing some YARRRR
<lifeless> sure; you'll need to subscribe to get notified of comments though
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/642418
<JesperHansen> lifeless: I will however make a bugzilla report that you can "link" up against after making a sandwich
<lifeless> sudo make me a sandwich?
<JesperHansen> [sudo] password for JesperHansen:
<lifeless> *******
<JesperHansen> with the exception that you cant make users with capital letters
#launchpad-dev 2010-09-19
<JesperHansen> lifeless: and maybe I should give you a link to a bugreport that has RESDUP status
<JesperHansen> https://bugzilla.mozilla.org/show_bug.cgi?id=454993 - this is the most recent one. The See Also modification was done 12 hours ago
 * wgrant ponders adding "You are probably wrong." to the +nominate template.
<lifeless> wgrant: hah
<lifeless> wgrant: you saw that lovely little chat?
<lifeless> wgrant: got anything you need reviewed?
<lifeless> wgrant: also did you have any luck with https://bugs.edge.launchpad.net/malone/+bug/637854 ?
<lifeless> wgrant: also https://bugs.edge.launchpad.net/soyuz/+bug/276950
<lifeless> wgrant: you might like 'https://bugs.edge.launchpad.net/malone/+bug/366134' for an example of insane code.
<wgrant> lifeless: I could really use a statement log for 276950.
<lifeless> wgrant: you've got mail
<wgrant> lifeless: Thanks.
<wgrant> lifeless: That statement log is nauseatingly bad.
<lifeless> yes
<wgrant> I wonder if createMissingBuilds sucks in as-yet-unknown ways
<lifeless> I like to think of it as 'plenty of room to improve'
 * wgrant pokes.
<wgrant> It does suck, but not the several-hundred-DAS-queries level of suckage.
<lifeless> wgrant:     2397  OOPS-1721H1326  Archive:EntryResource:syncSource
<lifeless> thats bad
<lifeless> its a query count
<lifeless> of course, worst is
<lifeless>     2442  OOPS-1721L267   BugTask:+index
<wgrant> lifeless: Same issue.
<wgrant> Or similar.
<wgrant> syncSource has copy checker + build creation
<wgrant> == fail
<lifeless> wgrant: If you can, writing a query scaling test would be a great idea
<wgrant> Build creation is terribly naive, I knew that. But it seems to be far worse than I thought.
<wgrant> Waitwhat.
<wgrant> The P-a-s code is awful :(
<wgrant> Oh, no, not too bad.
<wgrant> Then what is it...
<lifeless> man, I so want to turn memcache off and try https://bugs.launchpad.net/ubuntu/+bug/1/+index
<lifeless> I just needs a losa
<wgrant> Uggggh.
<wgrant> Garrrr.
<wgrant> SPR.getBuildByArch is the source of all our problems.
<wgrant> It is shocking.
<wgrant> It walks all the way up the series graph.
<wgrant> Grabbing the corresponding DAS all the way up to Warty.
<wgrant> Then *for each ancestor series*, it finds the distro archives.
<wgrant> Despite the fact that the distro archives are for the entire distro.
<wgrant> So that easily cuts out 1/3 of the queries.
<wgrant> Ah, it attempts to rationalise that insanity.
<wgrant> But it's still insanity.
<persia> What's the value of P-a-s again?  Can it just be dropped?
<wgrant> P-a-s is still wanted. And it's bad, but not as terrible as I first thought, so it can remain as it is.
<wgrant> SPR.getBuildByArch is now the target of my fury... it has two callsites, one of which I was going to delete anyway, so it should be easy enough to fix.
<persia> Well, I dis-want it, but I understand if others do :)
<wgrant> It makes sense for some things...
<persia> Go kill SPR.getBuildByArch : When I have time to get annoyed enough to really kill it, I'll benefit from from a long explanation.
<wgrant> lifeless: <3 StormStatementTracer and HasQueryCount.
<wgrant> Er, StormStatementRecorder
<wgrant> lifeless: In the unlikely event that you're still around, could you see if there's any reason bug #65712 is still private?
<wgrant> It's referenced in tests for the code I'm replacing, so it would be nice to see what it was about.
<wgrant> I suspect I know its subject, but I would like to be sure.
<jml> wgrant, I'll take a look
<wgrant> jml: Thanks.
<jml> wgrant, I've made it public.
<wgrant> jml: Even better.
<wgrant> Thanks.
<jml> wgrant, np
<wgrant> Bogus sampledata really does make for incredibly awful doctests.
<lifeless> wgrant: I'm glad you like it
<lifeless> jml: thanks
<lifeless> jml: did you perhaps see my testtools branch?
<jml> lifeless, I did
<jml> lifeless, I have comments
<jml> lifeless, but have yet to have an opportunity to write them down
<lifeless> ah
<lifeless> jml: I had hoped it would be a no brainer after our phone conversation
<lifeless> I wonder if we need tags-per-task to make bugs like https://bugs.edge.launchpad.net/launchpad-registry/+bug/597738 work properly with QA processes
 * beuno 's head hurts when thinking about the UI for such a thing
<lifeless> beuno: indeed
<lifeless> beuno: right now our QA process resets with each landing
<lifeless> its ... bad
<beuno> hrm
<lifeless> we may need a separate thing to tags
<lifeless> its all up in ze air atm
<beuno> QA could be a thing on tis oqn on Launchpad
<beuno> although I cringe at the idea of adding antoher knob
<lifeless> beuno: EPARSE
<lifeless> 'tis' 'oqn'
<lifeless> your spanish words are confusing and infuriating me :P
<beuno> it's still Sunday for me  :)
<beuno> "on its own"
<lifeless> rotfl
<lifeless> ah I'd no chance of getting that did I.
<lifeless> anyhow, yes, it could be.
<lifeless> That would be an alternative to more bug statii
<lifeless> grah
<lifeless>     module = original_import(name, globals, locals, fromlist, level)
<lifeless> ImportError: No module named mailman.monkeypatches.defaults
<wallyworld> morning
<lifeless> thumper: around?
<thumper> lifeless: am now
<lifeless> see -reviews
<jml> lifeless, approved
<lifeless> jml: thanks
<jml> beuno, when are you visiting London next?
<lifeless> jml: do you have anything else in the pipeline? perhaps we should release?
<jml> lifeless, for testtools?
<lifeless> yes
<jml> lifeless, nothing in the pipeline. I'd like to fix the thing about matchers always including matchee in the stack trace
<jml> lifeless, but I don't think that should block a release
<lifeless> jml: I'm not sure what the right thing is there, though I too would like to meet james_w's needs
<jml> lifeless, agreed
<beuno> jml, not super soon, no. How about you buenos aires?  :)
<jml> beuno, nothing planned yet.
<lifeless> hmm
<lifeless> there is definitely something funny with oopses happening
<beuno> jml, had anything planned?
<jml> beuno, no, just saw you online & thought it would be good to catch up
<beuno> jml, yeah, this second part of the year I'm not getting on a plane  \o/
<beuno> well, except for kiko's wedding
<jml> beuno, zowie
<jml> beuno, I'm toward the end of a 96 day stint in the UK :)
<jml> beuno, my longest yet.
<beuno> jml, congrats!
<thumper> beuno: kiko's getting married?
<beuno> thumper, I hope so, otherwise I'm going to be stranded somewhere in Brazil soon
<thumper> beuno: when?
<beuno> thumper, Oct 2nd
<thumper> cool
 * thumper lands the removal of branchrevision.id
<jml> g'night folks.
<lifeless> night jml
<thumper> lifeless: is buildbot down?
<wgrant> thumper: Is that going to make the next rollout take like two weeks?
<thumper> wgrant: faster than making the id a bigserial
<thumper> wgrant: I'm working on other branchrevision tweaks too
<thumper> wgrant: so we store less
<wgrant> Aha.
<lifeless> thumper: fallout from the reboots
<lifeless> spm will fix
<thumper> lifeless: ok
#launchpad-dev 2011-09-12
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 262 - 0:[#######** stack stamashing detected: ./lp terminated
 * lifeless heads off to a hospital apptment. Be a few hours.
<lifeless> mobile is on if anyone needs me.
<wgrant> Bye lifeless.
<wgrant> StevenK: You're doing it all wrong.
<wgrant> Why would it overflow before writing the 8th character?
 * wgrant fixes.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 262 - 0:[########*** stack smashing detected ***: ./lp terminated
<wgrant> That's how it was last week :)
<StevenK> Bleh
<wgrant> cjwatson: It looks like the lucid backported apt-ftparchive hangs when writing out Translations.en
<StevenK> wgrant: We are now QA'd up to r13911
<wgrant> Indeed.
 * wgrant deploys.
<wgrant> Although ITYM 13910
<StevenK> I meant r13911 is the next revision to QA, but probably.
<jtv> StevenK, wgrant: time to Q/A my frightening gina/dominator changes.  Given the risks, would very much appreciate your help.
<wgrant> jtv: Indeed.
<jtv> wgrant: glad you're here.  :)
<wgrant> jtv: I guess we should do lots of overrides, some multiple overrides, some reverted overrides, some overrides with uploads, and then attack gina...
<jtv> Overrides are something I have very little experience of, so good thing you brought it up.  Maybe we should try publish-ftpmaster first to see that regular domination still works properly.
<wgrant> I ran it this morning and it didn't melt, but I don't think there was much to do.
<jtv> Okay, not melting is goodâI'm more concerned though with whether it still does the right thing.  :)
<wgrant> Of course, but it's a first step.. sort of.
<jtv> âGood news!  We put 20,000 Volts through this sensitive circuit and the fuse *still* works.â
<wgrant> Heh
<jtv> To be honest I lack the knowledge or imagination right now to come up with any way in which the new Ubuntu domination could behave differently from traditional domination.
<jtv> Which, admittedly, means I need more coffee.
<wgrant> Right, which is why we are going to try lots of things, particularly stuff which has been traditionally.. problematic.
<jtv> I suppose âit doesn't meltâ is about the best test for this part.
<jtv> We could set up a derived distro for these tests, and run publish-ftpmaster on that.
<wgrant> I was just going to molest oneiric.
<wgrant> Or whatever the dev DF series is at the moment.
<jtv> Well, if you run publish-ftpmaster then it's going to dominate the whole series, right?
<wgrant> Since it has a representative selection of sources and binaries.
<jtv> Ahem, *distro*
<wgrant> Yes.
<wgrant> Well.
<wgrant> Could use -s
<wgrant> To restrict it to a series.
<jtv> Oh, yes, we could.
<wgrant> s/series/suite/, even
<wgrant> I used that to test index generation this morning -- publish-ftpmaster passes it through fine.
<jtv> This is on dogfood?
<wgrant> Yes.
<wgrant> Hm.
<jtv> I updated it yesterday, so you should have gotten the new code.
<wgrant> Ahh.
<jtv> (Took some nursing over the weekend to get this landed)
<wgrant> I wondered why it had nothing to pull.
<jtv> The other reason was semi-permanent buildbot failure.
<wgrant> Ja.
<wgrant> But that seems sorted now.
<jtv> stub was kind enough to fix up an apparent race condition in the pgbouncer fixture.
<wgrant> Even the stuff that has plagued us for the last three weeks should be fixed now.
<jtv> After some annoying SMS and email from me.  :)
<wgrant> So it might be stable.
<wgrant> Hah.
<jtv> To be fair to me, I reviewed it.
<jtv> Great thing about weekends: you can really get stuff done when you're not busy being managed.  :-)
<jtv> (I'm joking.  Julian's actually helpful as a manager.)
<jtv> Anyway.
<wgrant> DF's Debian is all published. We might want to unpublish bits of it.
<jtv> So the smoke test passed, the smoke alarms remained quiet.
<wgrant> To test the transitional code.
<jtv> Good idea.
<jtv> Shall I just do that then?
<wgrant> Sounds like a plan.
<wgrant> You attack gina, I attack archivepublisher?
<wgrant> For now, at least.
<jtv> By the way, just to avoid any misunderstandings, we're talking about making Published SPPHs Pending again, right?
<wgrant> Yes.
<wgrant> I'm talking about published, not published. Duh.
<jtv> Oh yes of course,
<jtv> how could I miss that
<jtv> (I know I do this a lot, maybe more than seems necessary â but the cost of backtracking from a mistake of this kind can be so costly that I'd rather spend an appreciable percentage of my regular communication time on it)
<jtv> I'm not sure I can do much with gina: how to set it up for a run, how to run it, how to look at overrides etc.  But I'll patch up the DB first.
<wgrant> The easiest way to fake gina override changes is probably to mangle the DB.
<wgrant> Rather than mangle the source archive.
<jtv> Can we run gina such that it really imports from Debian?
<nigelb> And 7 hours later, launchpad tests are still running.
<StevenK> jtv: You can fake a new import, yes.
<StevenK> jtv: It's an ... interesting process
<wgrant> Hm, I guess it's going to be a little difficult now.
<wgrant> Since we can't do a partial import any more.
<StevenK> We can't? :-/
<wgrant> It will nuke anything that isn't present.
<StevenK> Sigh. NFS mount from iron
 * StevenK waits for the ICBM
<wgrant> Hah.
<jtv> Wellâ¦ it'll only nuke it a little bit.
<jtv> Call it a tactical warhead, just fission without the fusion part.
<wgrant> Perhaps we should grab a small PPA and import it into a new distro.
<nigelb> wgrant: Thanks for QA-ing that bug. I wasn't sure how I'd do it.
<wgrant> nigelb: I just made a few changes, linked a couple of questions, checked that it didn't crash.
<jtv> Meanwhile, the latest 23K or so Debian SPPHs just went back to Pending on dogfood.
<wgrant> Great.
<jtv> (Is it a bad sign that I think nothing of generating histograms in SQL?)
 * jtv â â
<nigelb> Coffee? Java?
<nigelb> Running the full tests can take /forever/
<wgrant> Hmm.
<wgrant> https://launchpad.net/debian/+source/dpkg/+publishinghistory
<wgrant> gina stopped setting datepublished a while ago.
<wgrant> That is unfortunate.
<wgrant> We should probably fix it.
<wgrant> And likely set it to datecreated everywhere it is unset.
<lifeless> mmm, rain.
<poolie> hi lifeless
<jtv> wgrant: don't tell me there's another loosely-related ancient glitch that I also ought to fix!
<wgrant> Welcome back lifeless.
<jtv> hi lifeless, hi poolie
<jtv> and congrats!
<wgrant> jtv: Well, I think you probably stopped it from setting datepublished somehow.
<jtv> *me*?
<poolie> oh, he's officially back?
<jtv> Probably not, but he wouldn't be lifeless if he had a life.
<wgrant> I think he might be.
<lifeless> hi poolie
<wgrant> jtv: Well, it broke some time since February
<lifeless> jtv: thanks.
<jtv> wgrant: and obviously I didn't actually *set* it in my SQL.
<lifeless> I am back 4 days a week
<wgrant> lifeless: Which day off, or will it vary?
<lifeless> wed
<wgrant> Aha
<StevenK> Well, that was inconvient.
<jtv> wgrant: nah, he's going to keep working for one day longer than he should, every week, and so the day off will shift until it's sunday and then it will stay there.
<wgrant> Heh
<jtv> wgrant: two cases of missing datepublished that I should probably take responsibility for:
<wgrant> GUILTY
<jtv> 1. The patch-up SQL doesn't set it.
<wgrant> The patch-up SQL shouldn't.
<jtv> 2. When gina creates a BPPH, it just passes a status directly.  I changed that, but did not add a datepublished.
<wgrant> gina always used to set datepublished, even when it was setting them to Pending.
<wgrant> Ah.
<StevenK> Gina does not create BPPHs
<jtv> Not in its current form of use, no.
<StevenK> Well, not how we run it.
<wgrant> ?PPH, then.
<jtv> (For SPPH I call what I hope is the appropriate method so it should get set)
<jtv> wgrant: could you explain why the patch-up SQL shouldn't?  Isn't it inappropriate to have Published SPPHs without datepublished?
<wgrant> jtv: It is. But the patch-up SQL shouldn't have had to, since gina was always meant to set datepublished even for Pending records.
<wgrant> But it turns out that's no longer the case, so the patch-up stuff possibly should.
<lifeless> setting it if its null should be safe
<jtv> So you're saying the patch-up SQL merely shouldn't _need_ to.
<StevenK> wgrant: Out of interest, do you have a branch up for your db patch of much deletion?
<jtv> wgrant: do you expect any practical interaction with the gina and domination changes?  Or can we treat this as a separate problem?
<wgrant> lifeless: Right, I was imagining something like UPDATE sourcepackagepublishinghistory SET datepublished=datecreated WHERE datepublished IS NULL AND archive = 3;
<wgrant> jtv: Shouldn't be, just noticed when I was checking +publishhistory to see what state some packages were in.
<jtv> Phew.
<nigelb> lifeless: Could you review my db patch? https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/74934
<lifeless> nigelb: wrong reviewers :P
<nigelb> gah
<lifeless> nigelb: you need to request a review from both stub and I, of type 'db', per the wiki page.
<StevenK> nigelb: You need to add two other reviews: lifeless and stub with a type of 'db'
<nigelb> that was default!
<lifeless> yes
<StevenK> db patches are speical
<lifeless> actually we could change that now.
<nigelb> Hey, its my first time!
<lifeless> as only db patches go to db-devel.
<nigelb> Hrm, I can't set the type for the first person?
<jtv> wgrant: soâ¦ how are we going to set up these overrides and how are we going to run gina?
<wgrant> jtv: We can't really get a full Debian archive onto mawson, so we will have to import a smaller archive into another distro.
<nigelb> Wha. What happened to the type that I set.
<wgrant> I'm waiting for publish-ftpmaster to finish with the 10 zillion new Ubuntu series.
<StevenK> nigelb: The picker is just daft
<nigelb> :/
<nigelb> I have lifeless and stub on review with empty type fields.
<nigelb> I wish I could edit type later.
 * nigelb gives up
<jtv> nigelb: I think you can, but by ârequesting another reviewâ of the same person.
<lifeless> nigelb: file a bug :)
<wgrant> StevenK: You misspelt "awesome and flawless"
<lifeless> jtv: that will create a separate review request - its a feature.
<StevenK> wgrant: Did I? Try it :-P
<lifeless> jtv: hmm, perhaps not for non-teams. Who knows :)
<nigelb> What a ...useful.. feature.
<StevenK> wgrant: Damn it, answer my question :-P
<nigelb> Why did I get the review type wrong?
<wgrant> StevenK: Oh, the DB patch thing? Not yet. Distractions, Soyuz, and distractions.
<wgrant> Currently hacking p-d=r :(
<StevenK> wgrant: There's at least one bug we can link to it.
<wgrant> Oh?
<StevenK> wgrant: If there was a branch, I was going to do the linkage
<nigelb> I selected the person, the field was filled out and it runed out empty.
<StevenK> bug 345810
<_mup_> Bug #345810: Remove old infestations database stuff <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/345810 >
<wgrant> Ah.
<wgrant> Indeed.
<wgrant> I will get to it today.
<wgrant> Just want to get p-d-r done so Julian can approve it and we can get the bastard run.
<StevenK> And it really is going to be a bastard run.
<wgrant> Heh]
<nigelb> lifeless: you and stub are on the review list, but not 'db' review. I'm guessing that's minor.
<lifeless> thats fine
<nigelb> Heh
<nigelb> # These 5 lines are untested. Deal with it.
<wgrant> StevenK: <https://code.launchpad.net/~wgrant/launchpad/careless-executioner/+merge/74935> if you want to glance at the wonderful p-d-r "fix"
<StevenK> I'm not sure that I do ...
<StevenK> wgrant: So the plan is this lands gets run once, or cowboyed in for one run?
<wgrant> Cowboyed.'
<StevenK> Does it work acceptably on DF?
<wgrant> No idea. I'm running publish-ftpmaster for other reasons at the moment, and I need to check if dryrun mode actually is a dry run.
<wgrant> But the tests pass (as long as I use hoary instead of dapper)
<StevenK> I'm not sure I trust the domination tests.
<wgrant> Not domination.
<wgrant> Not even judgement.
<wgrant> This is reaping.
<nigelb> heh
<wgrant> nigelb: Those are all technical Soyuz terms!
<StevenK> wgrant: Do eet, then.
<wgrant> Publications are created, published, dominated, judged, then reaped.
<nigelb> wgrant: Imagine someone who doesn't know what gina is reads this stuff.
<ajmitch> nigelb: I think it shows the mindset of those who wrote it
<nigelb> heh
<StevenK> JS still sucks.
<nigelb> heh
<nigelb> JS is <3
<StevenK> nigelb: Then you can explain our JS to me.
<ajmitch> dealing with the DOM is not <3
<nigelb> StevenK: YUI, however, is special
<nigelb> I don't mind JS most of the time.
<StevenK> And why we have 3 different JS functions that all deal with unsubscribing people from bugs.
<nigelb> Point me to the code?
<StevenK> I'm just stuck trying to get at the bug itself when LP.cache.context is a bugtask
<nigelb> I couldn't parse that :)
<StevenK> We have a cache inside JS called LP.cache
<StevenK> It has a context memeber which is JSON of an IBugTask
<nigelb> You wwant to access parts of the JSON?
<StevenK> I can not work out how to inject the private attribute into the cache, since the view code is incomprehsible and I also can not work out how to get a JSON representation of an IBug given the link from the bugtask.
<nigelb> Where does this code live so I can poke?
<StevenK> The JS I'm currently dealing with is lib/lp/bugs/javascript/subscription.js
<nigelb> 1351 lines of JS.
<nigelb> How pleasant.
<StevenK> And I have this sinking feeling that lp.app.javascript.confirmationoverlay only deals with forms, which this isn't.
<StevenK> nigelb: 100789 lib/canonical/launchpad/icing/build/launchpad.js
<nigelb> Which is "okay"
<nigelb> should minify that for more scary effect ;)
<StevenK> wgrant: Have you jumped the gun?
<StevenK> wallyworld: *prod*
<StevenK> Do we have a procedure for dropping something from the API?
<wgrant> If it's only in devel, drop it.
<StevenK> DSP.getBugTasks()
<wgrant> If it's in 1.0, pretend to consider whether it's worth breaking backwards compatibility, then probably drop it anyway.
<wgrant> If it's in beta, that's EOLed so anybody using it is stupid :)
<wgrant> Why would you drop that?
<StevenK> Because it's horrible
<StevenK> And because better codepaths exist
<StevenK> And there's a bug
<wgrant> Are there better ones>
<wgrant> It's not just DSP.
<StevenK> IHasBugs.searchTasks()
<wgrant> Huh, how it only exported on DSP...
<wgrant> Sure, but getBugTasks doesn't do the same thing.
<wgrant> It takes a list of bug numbers and returns the bug tasks from those bugs in this context.
<StevenK> getBugTasks does not, no
<wgrant>     def getBugTasks(self, bug_ids):
<StevenK> Wrong method
<StevenK> line 185 of lib/lp/registry/interfaces/distributionsourcepackage.py
<wgrant> That's the interface.
<wgrant> It was called 105 times last month.
<StevenK> getBugTasks does not appear in DSP's model, since it's called bugtasks
<StevenK> And the interface renames it and the parameter
<wgrant> Yes. There is only one getBugTasks model method, and it's the one I quoted.
<StevenK> Is that the one that is being called?
<wgrant> Oh, so you're not actually talking about DSP.getBugTasks()
<StevenK> It appears like that to the API
<StevenK> In the model code it's DSP.bugtasks
<wgrant> So, it's still called, but the PPR doesn't say in which version.
<wgrant> You could check the appserver logs.
<wgrant> And you should probably also delete the Python getBugTasks... it seems to be unused.
<lifeless> I hates distutils
<lifeless> or setuptools
<lifeless> one of em
<lifeless> /usr/lib/python2.6/dist-packages/pytz/__init__.py:32: UserWarning: Module tests was already imported from tests/__init__.pyc, but /usr/lib/python2.6/dist-packages is being added to sys.path
<lifeless>   from pkg_resources import resource_stream
<lifeless> ---fail---
<lifeless>  /endrant
<StevenK> wgrant: Where are the logs hiding?
<wgrant> StevenK: /srv/launchpad.net-logs/lpnet
<StevenK> % bzr di | diffstat -s
<StevenK>  6 files changed, 124 deletions(-)
<StevenK> That was therapeutic.
<lifeless> anyone know how our storm stores get _database.name set ?
<lifeless> storm doesn't -seem- to do this itself
<lifeless> argh
<lifeless> class LaunchpadDatabase(Postgres):
<lifeless> ok
<wgrant> Yes.
<wgrant> I was dealing with that a bit last week.
<wgrant> What are you doing?
<lifeless> moving code to storm
<wgrant> I have three branches in the wild which delete lots of our DB handling, so don't be too invasive.
<wgrant> Ah.
<lifeless> so that u1 can use TimelineTracer
<wgrant> Sounds good.
<lifeless> so that they can use python-oops-*
<lifeless> our Tracer is 3, maybe 4 separate tracers.
<lifeless> gary's work added substantially to that
<wgrant> lifeless: Do you feel like hacking PQM to expose the commit message as an envvar or something?
<lifeless> wgrant: no, I am on the critical path for SOA.
<lifeless> wgrant: but I can advise.
<StevenK> wgrant: Why?
<lifeless> StevenK: handling of accidental db->devel commits.
<lifeless> StevenK: at a quess
<StevenK> How do you typo guess as quess? q and g are nowhere near each other :-P
<lifeless> very carefully.
<lifeless> hmm, this is a nuisance.
<lifeless> we should have pushed this upstream years ago.
 * lifeless kludges
<wgrant> lifeless: :(
<lifeless> erm
<lifeless>   File "storm/tracer.py", line 166, in __init__
<lifeless>     super(TimelineTracer, self).__init__()
<lifeless> TypeError: super() argument 1 must be type, not classobj
<lifeless> wtf
<StevenK> No __metaclass__ = type ?
<lifeless> thanks
<lifeless> been a long time since I hit that particular wtf
<wgrant> jtv-eat: I have performed several great evils on the primary archive, and it seems to have the same set of bugs as before, so we are good there.
<wgrant> jtv-eat: As for gina... who knows.
<StevenK> And make lint runs buildout? WTF?
<wgrant> StevenK: bin/lint is built by buildout.
<StevenK> narrative uses a moin header. == take the equals out and replace with what?
<wgrant> StevenK: An underline.
<StevenK> wgrant: 97 calls in the last month
<StevenK> In fact, a bit less, due to my branch name
<wgrant> Which version, and who?
<StevenK> Most of them identify themselves as ubuntu-dev-tools
<StevenK> But getBugTasks doesn't appear in ubuntu-dev-tools
<wgrant> It'll be someone using ubuntu-dev-tools' wrappers.
<StevenK> 82 without my IP
<wgrant> All in one incident?
<StevenK> No, spread around the month and the DSP they call from
<wgrant> :(
<wgrant> Similar IPs?
<StevenK> 19 unique IPs
<wgrant> Same ISP, or not?
<StevenK> Highly doubtful
<StevenK> No, they are not
<wgrant> :(
<StevenK> All of the calls are against 1.0
<wgrant> :(
<StevenK> We have to leave it then?
<lifeless> jamesh: https://code.launchpad.net/~lifeless/storm/timelinetracer/+merge/74947
<wgrant> StevenK: Unexport it from devel, at least.
<StevenK> wgrant: That makes me unhappy
<lifeless> wouldn't it be cool if launchpadlib *could* warn on using a deprecated API
<wgrant> http://ubuntu-dev-tools.sourcearchive.com/documentation/0.86/lp_8py-source.html
 * lifeless goes to cook dinner
<wgrant> StevenK: That's the only callsite that Google seems to know about.
<StevenK> Ignoring lp-shell and test gives 43
<StevenK> 'emesene bug maintenance' turns up 12 times
<StevenK> So 31 calls from something in u-d-t
<wgrant> StevenK: Did you see my link?
<StevenK> I did
<jtv> wgrant: thanks, it's good to hear that our beloved bugs are safe.  I suppose it's on to gina then.
<jtv> What would happen if we just ran it on dogfood?
<wgrant> It probably wouldn't work, because DF doesn't have a Debian archive.
<wgrant> And doesn't have space for one.
<wgrant> Unless StevenK's partial one is still there, in which case it would set most of the LP archive to Deleted.
<wgrant> Which would possibly be bad.
<jtv> I don't suppose staging would do it?
<wgrant> Well, we could hack it into staging's config, but I'm not sure that staging has a few hundred gigabytes of free space either.
<mrevell> Morning!
<jtv> hi mrevell
<jtv> wgrant: no smaller part of Debian that we could import?  If not, maybe we can run a sabotaged version of gina that doesn't mess with the filesystem at all?
<wgrant> jtv: Well, given that gina's whole purpose is to take an archive from the filesystem and put it in the DB, that sounds possibly difficult or not a representative test.
<wgrant> ie. removing FS stuff from gina basically becomes a no-op.
<jtv> Well the part that changed really only involves the Sources lists and the database.
<wgrant> I guess we could disable the non-domination stuff and then give it a fake set of (package, version) pairs.
<jtv> Why not feed it a real listing to parse?
<wgrant> Mmm, I guess we could do that and deal with the fallout.
<wgrant> It's just going to be a little odd, as the mirror is 4 months out of date, so won't have lots of the current versions.
<wgrant> And since we'd have to disable the import stage, they would not be imported.
<wgrant> So lots of stuff would be deleted instead of superseded.
<jtv> We could feed it a copy of an old Sources file, with manual changes.
<wgrant> StevenK: Erm.
<wgrant> StevenK: bzr grep [^p]review_diff
<wgrant> StevenK: There is some code that still talks about review_diff, and a test that checks it on BMP.
<wgrant> How...
<poolie> wgrant, is lp devel in a safe enough state that i could land my dkim branch this week?
<wgrant> poolie: It is.
<wgrant> The spurious buildbot failures appear to have disappeared.
<poolie> ok
<wgrant> Which may have had something to do with disabling the codeimport integration tests.
<poolie> i'm going to do a little more testing first
<poolie> i had an idea for my process-one-mail script, which is that before it exits it should look at the outgoing mail queue and print it to stdout
<poolie> either before or after or during finishing the transaction, whatever is easier
<lifeless> wgrant: before I went on leave you were muttering about picking up my gpg work
<lifeless> wgrant: did you?
<wgrant> lifeless: Sorry, didn't get time. Too much Soyuz and buildbot breakage.
<lifeless> thats fine
<lifeless> needed to know if I had sync-up to do or not
<wgrant> Nope.
<wgrant> So, get that sorted out this week, then we can be fully nodowntime :)
<wgrant> (by removing poppy from LP)
<wgrant> Cheating, I know.
<adeuring> good morning
<jtv> hi adeuring
<jtv> Hmm
<adeuring> hi jtv!
<jtv> I wonder if this comment is ready to be retired:
<jtv>        # XXX kiko 2005-10-23: <stub> Until I or someone else completes
<jtv>         # LibrarianGarbageCollection (the first half of which is
<jtv>         # awaiting review)
<jtv> Oh God this is so depressing.  That's in a function called check_not_in_librarian.  The part of it that interacts with the librarian has been commented out.
<wgrant> Yep.
<jtv> There's another comment saying it's untested.
<wgrant> Depressing? This. Is. Gina.
<jtv> âHi, Gina.â
<jtv> (Said in the tired tone of 12-step program groups in films)
<bigjools> But Gina is bug-free.
<jtv> There's an even older XXX from debonzi saying âcheck it laterâ where the computation of an unused variable is commented out.  The variable looks important.
<jtv> Also, I found a trilobite.
<StevenK> wgrant: I see that.
<rvba> jtv: I also found a trilobite: "XXX 2008-06-16 mpt bug=241298: [...] dangerous and should be renamed (or removed)"
<wgrant> StevenK: I have a branch in ec2 to remove those references.
<StevenK> wgrant: Pity, I was just about to.
<jtv> rvba: 2008?  That's old but definitely post-Cambian.
<jtv> rvba: it sounds like that may be a fossilized Critical bug.
<rvba> Hehe. Not critical I think... but this code is strange now that we have DD.  Using spr.upload_distroseries is likely to trigger bugs.
<wgrant> rvba: Yeah, bigjools fixed one of those last week.
<wgrant> rvba: Changelog bug closing was using that :(
<jtv> Oh that must have been fun.
<bigjools> not so much
<StevenK> wgrant: So I can't just break DSP.getBugTasks() for 18 users? :-(
<wgrant> StevenK: Ideally track them down and kill them or the script.
<poolie> wgrant, so i might retry the landing for bug 721166 if you're pretty sure lp will stay on bzr 2.4
<_mup_> Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <spurious-test-failure> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/721166 >
<wgrant> poolie: We are 2.4+abit
<wgrant> The abit increased late last week.
<poolie> >=2.4 is all i need
<wgrant> Yeah, we're staying.
<cjwatson> wgrant: apt-ftparchive> Oh hell.  Is there a bug for this yet?
<wgrant> cjwatson: No, sorry.
<wgrant> cjwatson: But it's happened on mawson and locally.
<wgrant> (locally on lucid)
<cjwatson> mvo seems to be back from holiday, so attempting to grab him
<cjwatson> (which is better than me fumbling with apt)
<wgrant> Thanks.
<wgrant> Yeah.
<cjwatson> wgrant: Was this with include_long_descriptions at the default of True (i.e. old behaviour) or set to False (new behaviour)?
<lifeless> bug 241298
<wgrant> cjwatson: I manually set it to false to a series to test.
<wgrant> cjwatson: With true it's fine.
<cjwatson> Right.
<mvo> wgrant: just ping me when you are ready, I will continue my mail catchup in the meantime
<wgrant> mvo: All looks good.
<StevenK> wgrant: DB patch? :-P
<wgrant> StevenK: I have the combined patch in ec2.
<wgrant> StevenK: To check that it all works with the tables remvoed.
<wgrant> StevenK: and am about to send off the three pieces separately.
<wgrant> (remove FKs, remove person merge code, remove tables)
<poolie> nigelb, hi?
<StevenK> wgrant: 3 seperate patches?
<wgrant> StevenK: Yes.
<adeuring> nigelb: you should request a DB review from stub for your latest MP
<jtv> wgrant: Julian has no suggestions for how to run gina.  But actually, if it could just run with the existing sources files I think it'd do just about what we need.
<wgrant> jtv: If we have exactly the version that was last imported from, sure.
<jtv> If notâ¦ is it likely to do anything bad that the next db restore won't fix?
<wgrant> No.
<wgrant> But we'll have to be careful what we look at.
<wgrant> Because, as I said, a lot of packages will have no live versions.
<jtv> Does that matter though, as long as some do?
<wgrant> If we can track them down :)
<wgrant> And track down intended deletion/superseded cases.
<jtv> Well we could approach it from the other side: see which ones become Superseded, Published, and Deleted; and take samples from each.
<jtv> That way the database does the searching.
<wgrant> Shh
<wgrant> bigjools: So, what do you think about cowboying careless-executioner?
<wgrant> bigjools: It is sort-of-tested.
<wgrant> bigjools: In that I made deathrow.txt test the missing LFC case, and it passes when restricted to hoary but fails when restricted to dapper...
<jtv> "careless executioner"..?  Wouldn't "The Sloppy Chopper" be catchier?
<bigjools> our sample data is not exactly a bastion of perfectly related data
<wgrant> Heh
<wgrant> bigjools: Lies.
<bigjools> Reminds me of the song about George Michael - Careless Wrister
<jtv> And that's how he got nicked?
<bigjools> allegedly
<jtv> Meanwhile, in the real world: if I run gina, will it try to download anything from the debian archive?  Or has that already been done by some other component?
<wgrant> jtv: It needs a local archive on disk.
<jtv> Yeah but we've got that, sort of, partially.
<wgrant> Well, at least somewhere in the local FS.
<wgrant> It won't try to grab it from the network.
<jtv> Soâ¦ just run the thing?
<wgrant> If you have enough of an archive to be a useful test case.
<bigjools> wgrant: the carless-executioner branch looks ok
 * wgrant dines.
<bigjools> wgrant: let's do it
<lifeless> jamesh: hi ? :)
<bigjools> StevenK: thank you for fixing bug 421705
<_mup_> Bug #421705: archiveuploader tests leave files behind <lp-soyuz> <qa-untestable> <soyuz-upload> <tech-debt> <trivial> <Launchpad itself:Fix Released by stevenk> < https://launchpad.net/bugs/421705 >
<wgrant> bigjools: Indeed. What could go wrong, and all that.
<bigjools> wgrant: run it on DF first
<wgrant> Yeah.
<wgrant> jtv: You're not still molesting mawson?
<jtv> wgrant: I am, just in a low-intensity way ATM.  OTP.
<jtv> wgrant: running gina on sid now.
<wgrant> Ooh.
<jtv> I backed up all SPPH statuses.
<wgrant> Ah!
<wgrant> Even better.
<wgrant> Thanks.
<jtv> Into a table, with supersededby.
<jtv> So it'll be easy to tabulate things like âall SPPH status changes that weren't for Debianâ
<wgrant> Yup.
<jtv> (Which would hopefully be somewhere in the vicinity of "None")
<wgrant> We can hope.
<wgrant> jtv: Do you value your gina -l diff?
<jtv> wgrant: no
 * wgrant demolishes.
<wgrant> wtf, puppet using 50% CPU
<wgrant> 80%...
<wgrant> Hm, gina using a bit too. I might wait.
<nigelb> poolie: hi!
<nigelb> adeuring: Yeah, I've marked him on the review itself. Need to find time to poke him :)
 * nigelb is having a busy day at $DAYJOB
<nigelb> Hrm, I guess I missed poolie :(
<nigelb> stub: Hi! Could you have a glance at https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/74934
<stub> k
<wgrant> jtv: qIt seems done.
<wgrant> Let's see the damage...
<jtv> Yes.  I'm checking results.
<jtv> The only change seems to be:
<jtv> 113 new Published SPPHs for sid.
<wgrant> Hmm
<jtv> (I ran it on sid alone)
<wgrant> https://dogfood.launchpad.net/debian/+source/dpkg/+publishinghistory
<wgrant> It didn't set everything to Published in the fixup?
<wgrant> But did set all the right stuff to superseded, it seems.
<jtv> I don't think so.  The only change I see in the database is those 113 new Published ones.
<wgrant> Oh. Those are all superseded from 1.5 weeks ago.
<wgrant> Bah.
<wgrant> Crap DB is crap.
<jtv> Don't tell me it's a missing commit.
<stub> nigelb: That branch can land
<wgrant> Ahaha
<wgrant> stub: Did you see my two pretty trivial DB reviews?
<wgrant> Just in the last few minutes.
<stub> wgrant: not yet
<nigelb> stub: Could you land it for me please?
<nigelb> stub: (Also, Thanks!)
<rvba> wgrant: I see you're OCR tomorrow ... may I take the liberty to assign the review of a branch I just finished to you? It's not big but I'd be happy it you were the one to review it.
<wgrant> rvba: Sure.
<wgrant> jtv: Hm, should that really all be one transaction?
<rvba> wgrant: Thanks!
<wgrant> jtv: That could possibly take a while, even on !DF
<stub> We are dropping webservice ban? Is this a WHUI, or are we using some other mechanism for throttling evil people?
<wgrant> WHUI
<wgrant> openidrpconfig, openidrpsummary, staticdiff are the only tables that code still used (apart from for person merges).
<wgrant> I don't think webserviceban was ever used.
<wgrant> And if someone happens to need it, they can spend the extra 10 lines to recreate it as they wish.
<wgrant> But it's now like 4 years old, so I think it can die.
<stub> fairy nuff
<wgrant> mailinglistban is the same.
<wgrant> Two months ago I would have said to leave them, as adding tables took a month.
<wgrant> Now? Not so much.
<stub> wgrant: In the -3 branch I don't see a db patch. Think it needs a bzr add
<wgrant> Shh.
<wgrant> Hmm.
<wgrant> It is there AFAICT.
<wgrant> 2588- in the diff
<stub> yer - ic
<stub> email truncated
<wgrant> Ah, heh.
<wgrant> The sampledata diff is large, yeah.
<stub> hmm... wonder if upgrade.py should drop tables in a consistent order? Not sure if it helps much
<wgrant> I considered that.
<wgrant> But it's normally pretty simple to drop the interdependencies.
<wgrant> Just this case is a bit of a mess.
<wgrant> Because there are 35 tables.
<wgrant> Which leaves only 270 tables!
<stub> Should probably make a 'drop_table' stored procedure that strips all foreign key references and moves the table to the todrop schema
<wgrant> Yeah.
<wgrant> I was going to use a procedure to automatically drop everything here, but decided I was too lazy.
<wgrant> Hmm.
<wgrant> Damn.
<wgrant> I made the "Unmanaged roles on managed objects" security.py warning DEBUG, so I can't see what cruft there is on prod.
<wgrant> (full-update seems to only run it at INFO)
<stub> I think it accepts -v too
<wgrant> Ah.
<wgrant> But security.py is pretty spammy at -v, so we probably don't want that regularly anyway.
<wgrant> gmb, adeuring: Which squad is on interrupts this week?
<wgrant> stub: Thanks.
<gmb> wgrant: Whichever one isn't the Yellow Squad.
<gmb> (I forget colours)
<wgrant> Ah, and we are henningeless today.
<StevenK> Orange
<stub> wgrant: It would also accept --log-file=DEBUG:/... I think. Not sure if I got it all wired up correctly.
<wgrant> So that's why there's no CHR person.
<wgrant> adeuring: Are you handling #launchpad from 1100?
<wgrant> stub: Probably. I might get a LOSA to run --no-revoke -v tomorrow.
<stub> wgrant: We do lose the slony spam though in the log files
<wgrant> We may want to use CommandSpawner or so to invoke slonik.
<stub> Haven't seen that
<wgrant> Since it pushes output through the logging infrastructure.
<wgrant> jtv wrote it a few months ago.
<stub> Sounds like an awesome fit
<wgrant> It was originally written to do parallelisation.
<stub> slonik spam bugs me a lot, but sometimes I need it.
<wgrant> But it's also convenient for making logging not suck.
<StevenK> bigjools: So the extra files bugged you to? :-)
<StevenK> s/\(to\)/\1o/
<bigjools> sometimes, I think Steve's typos are just a way to consequently willy-wave his prowess with regexes
<bigjools> StevenK: yes, they did bug me
<jtv> wgrant: this is more like itâ¦ http://paste.ubuntu.com/687535/
<jtv> Note though that nothing was superseded.
<StevenK> bigjools: 1) Like backreferences are hard. 2) I don't use regexes all the time to correct typos ...
<nigelb> bigjools: I suspect there's some http://xkcd.com/208 involved.
<StevenK> bigjools: If they bugged you, why didn't you fix it? :-P
<bigjools> StevenK: like I don't have enough to do already
<StevenK> :-P
<bigjools> nigelb: perfect! it even has the Perl reference
<jtv> wgrant: at this point it would be helpful to see those Sources files.
<wgrant> jtv: Erm, you don't have them?
<wgrant> Then what have you been running with? :/
<jtv> It plucks them out of the librarian, right?
<wgrant> No.
<wgrant> It gets them from whereever the archive is configured to live in the FS.
<nigelb> bigjools: :D
<wgrant> root: /srv/launchpad.net/gina-mirror
<wgrant> For dogfood
<jtv> Thanks.
<jtv> wgrant: I was just struck in the head by an irony
<jtv> *ow*
<wgrant> Oh?
<jtv> AFAICS this bug makes the branch qa-ok.
<jtv> Before we go to the trouble of tracing all the data.
<wgrant> Not necessarily.
<wgrant> We should check the deletions, I suspect.
<wgrant> I see some 113 of them.
<wgrant> Interesting.
<wgrant> As there are also 113 new Published publications.
<wgrant> Suspicious!
<jtv> The deletions only happen with the commit inserted that I forgot before.
<jtv> Hence my comment: as long as the commit is missing, they're not a concern from a deployment standpoint.
<wgrant> Ah, yes, because we are source-only.
<wgrant> So it skips the only other commit.
<wgrant> Agreed, looks like this is "ok"
<jtv> The 113 new Published records are there even without the commit, i.e. before domination.  I wonder what happened there.
<jtv> As you say, the equal numbers are suspicious.
<jtv> It'll be easy to figure out if they're for the same packages, as one would expect.
<wgrant> Yep.
<jtv> Might be component changes or something.
<wgrant> It's not unexpected that there would be new ones.
<wgrant> As the source archive could well have new ones.
<wgrant> Oh.
<wgrant> I know what it'll be.
<wgrant> They're new, but older versions.
<wgrant> Because the Debian mirror is older than the last one that was imported before the DB was dumped.
<wgrant> So the live version is older than the latest version in the DB.
<jtv> That makes sense.
<wgrant> So the latest version gets Deleted.
<jtv> Let's verify it.
<wgrant> That's not the sense of adventure I expect from a Soyuz engineer.
<jtv> I AM NOT A SOYUZ ENGINEER!
<jtv> Wanted to get that straight.
<jtv> One thing bothers me about the theory:
<lifeless> one word.
<lifeless> 'good'
<jtv> Thank you lifeless.
<jtv> One thing bothers me about the theory: wouldn't we be seeing the same SPPHs being created as Pending, immediately upgraded to Published, and then marked Deleted?
<jpds> IANSE.
<jpds> That could catch on.
<wgrant> jtv: Won't they be directly Published now?
<wgrant> jtv: And why would they be marked Deleted? They are the live version, so dominatePackage won't touch them.
<jtv> We're seeing 113 going Deleted, and 113 being created and going Published.
<wgrant> Yes.
<jtv> They are not the same SPPHs.
<wgrant> So, the DB has foo 1.1 live
<wgrant> The mirror is older than the DB.
<wgrant> It has foo 1.0
<jtv> jpds: IANASE maybe?
<wgrant> gina import foo 1.0 as Published.
<wgrant> It then calls dominatePackage on [foo 1.1, foo 1.0], with 1.0 as a live version.
<wgrant> foo 1.1 is not live, and there is no dominant, so it gets Deleted.
<wgrant> foo 1.0 is live, so it is untouched, and remains Published.
<jtv> Gar.  Watching publication in reverse.
<jtv> âYou get this ebony bathtub, right?  But the thing is, it's conical.  And you fill it up with fine, white sandâ¦â
<wgrant> Heh.
<jtv> I'm about to dig up an example.
<jtv> But first, checking for idempotency.
<wgrant> This theory is supported by the fact that gina didn't catch on fire.
<wgrant> The mirror must therefore be older than the DB: all the SPRs had already been imported, so no attempt was made to grab files from the disk, where they don't actually exist.
<jtv> Well you can prove _anything_ from a contradiction.
<wgrant> Heh
<wgrant> rvba: Are you also going to make normal uploads set SPPH.creator?
<jtv> Furthermore, I suppose it makes sense that the difference was small enough that none of these packages had more than one release "supraseded" by an older one.
<jtv> subseded?  infraseded?
<wgrant> fucked?
<lifeless> whats the recipe to make custom storm eggs again
<rvba> wgrant: AFAIK that's not part of the plan just now.
<bigjools> lifeless: one custom storm, add eggs to taste.  Beat until fluffy.
<jtv> wgrant: supporting fact â the deleted spphs were all for different sprs
<wgrant> jtv: SPRs or SPNs?
<jtv> sprs
<jtv> haven't tried spns yet; working on it
<wgrant> k
<wgrant> rvba: Oh, setting ancestor too. Thanks, this will make copies *much* more auditable.
<lifeless> wgrant: do you remember ?
<elmo> you guys need to find a way to backronym SRS and BSNS into soyuz
<wgrant> BinarySourceNameSpace
<wgrant> We actually need something just like that for disclosure :P
<wgrant> Mapping binary names back to source names.
<bigjools> Soyuz Really Sucks
<wgrant> lifeless: Ummm.
<jtv> wgrant: the deleted SPPHs â http://paste.ubuntu.com/687555/
<wgrant> lifeless: I may have manually edited setup.py and tarred. I can't remember...
<wgrant> lifeless: mtimes may tell you how badly I did it.
<nigelb> bigjools: You know this now? I've not touched it and I've heard that multiple times.
<nigelb> Ah, right. You're documenting soyuz.
<wgrant> jtv: Can you get (spn, deleted version, published version) for the relevant publications?
<bigjools> nigelb:  it's a meme. It's not really true. Soyuz is Really Sweet.
<wgrant> Nah, Soyuz Are Really Sweet.
<jtv> wgrant: working on it
<wgrant> jtv: FASTER!
 * wgrant fetches the whip.
 * jtv was once kicked out of a university course for fear of SARS
<nigelb> haha., I was about say "someone hand wgrant a whip"
<wgrant> jtv: https://dogfood.launchpad.net/debian/+source/simple-scan/+publishinghistory seems to reinforce my grand unified theory of gina borkedness.
<bigjools> orange or strawberry?
<wgrant> bigjools: Cool hwhip.
<elmo> http://cryptome.org/info/soyuz-tma18/pict2.jpg <-- soyuz
<elmo> (that never gets old)
<wgrant> elmo: I was displeased that bigjools replaced that as the #soyuz topic :(
<nigelb> wait, there's a #soyuz?
<wgrant> Internally.
<wgrant> And deprecated.
<bigjools> Fire the retro rockets!
<wgrant> It tends to nowadays be where people complain when Soyuz is more broken than normal.
<wgrant> Or when their builds have melted a few buildds.
<wgrant> Nothing good happens there.
<bigjools> only one soyuz capsule ever landed doing 150mph more than it should have been
<wgrant> Er, weren't there three?
<wgrant> Oh, no, one of those just depressurised.
<wgrant> But there were at least two with parachute issues.
<elmo> bigjools: when the criteria is '150mph more than it should have', once is usually enough
<elmo> especially for those in the vehicle in question at the time
<wgrant> At least our Soyuz never has the problem that it goes too fast.
<wgrant> And Dapper only partially depressurised...
<jtv> wgrant: visual animation also supports your GToGB.  The newly-created versions are all slightly older than the deleted ones.
<wgrant> Sounds like it is not fatal, then.
<bigjools> elmo: apparently the guy in that capsule was swearing at mission control all the way down to his death. It seems somewhat appropriate...
<jtv> And since the bird is cruel, zc.buildout just had to be among them.
<wgrant> Yes, there is a recording somewhere.
<StevenK> rvba: O Hai. I had some questions about confirmationoverlay, if you have a second.
<wgrant> jtv: I wonder, though, why there are none superseded...
<wgrant> jtv: Ah.
<bigjools> wgrant: only one had a parachute failure
<wgrant> jtv: I guess you cleaned them all up on the 1st.
<rvba> StevenK: sure, shoot.
<wgrant> jtv: We may want to set everything back to Published and rerun?
<StevenK> rvba: Does it show a Yes and No button?
<rvba> (StevenK: btw, love your spph denormlization work)
<wgrant> bigjools: Ah, right: a depressurisation, a parachute failure, and two ballistic reentries.
<jtv> wgrant: I don't see what you mean with the "cleaned them all up on the 1st."  Isn't it simply because of the version reversion we're seeing here?  The only superseding that happened was in reverse, leading to deletion.
<StevenK> rvba: If so, I don't want it to submit a form on Yes, I would like it to call a JS function, is that possible?
<bigjools> wgrant: yup
<wgrant> Soyuz 5 and TMA-11 were the amusing ballistic reentries.
<rvba> StevenK: it shows a formoverlay (that you are freel to fill with content) with two buttons.
<wgrant> That's what I was thinking of as the other two.
<rvba> free*
<wgrant> jtv: I was expecting to see 150000 superseded publications.
<jtv> ahhh
<rvba> StevenK: Ah, that's not possible yet.
<wgrant> jtv: But looking at https://dogfood.launchpad.net/debian/+source/simple-scan/+publishinghistory, that was all done on 2011-09-01
<jtv> the 1st of the month, now I see.
<wgrant> jtv: Should we set everything back to Published and rerun?
<wgrant> Given that's how it'll be on prod.'
<jtv> Yeah sure, it's getting to be fun.
<wgrant> (once we fix the commit)
<StevenK> rvba: I am also getting defeated by my utter lack of JS knowledge.
<rvba> StevenK: For now it only protects a form, and it submits the form in question if you click 'yes' on the confirmation overlay.
<StevenK> rvba: Okay, thanks. I shall chat to Curtis about it in the morning.
<rvba> StevenK: This is so easy to do that I'm ok to do it if you want.
<wgrant> jtv: If we wanted to be really accurate...
<wgrant> jtv: we could delete all Debian publications since the mirror date.
<jtv> We can pick some obscure release that nobody will miss.
<wgrant> jtv: That might yield a more analysable result.
<wgrant> Hm?
<jtv> (Which is usually anything except "sid," in my experience)
<jtv> Or is everything supposed to be restored anyway?
<jtv> Update done.  All is Published again in sid.  Here we go.
<wgrant> If we just work out the timestamp of the Sources file we have, and delete every DF publication since like a day after that, we will have a pretty simulation of what we're about to experience on production.
<wgrant> Rather than what we're doing now, which is regressing the mirror.
<wgrant> Still, this should be mostly sensible.
<wgrant> We just have to ignore the bits where it goes backwards.
<rvba> StevenK: I confess I wanted to do it because then the confirmation overlay would be usable to protect pure js calls.
<wgrant> jtv, bigjools: Any reason not to kill DF's buildd-manager?
<wgrant> It's eating most of a core.
<bigjools> nope
<wgrant> and can't be helping postgres.
 * wgrant murders.
<bigjools> I can't wait to get Rabbit working with it
<wgrant> Oooh.
<wgrant> Although it's not so useful there.
<wgrant> Since the slaves still have to be polled.
<bigjools> then those crazy periodic queries can die
<wgrant> Sort of.
<bigjools> you're kidding, right?
<wgrant> We'll have to invert the dispatch algorithm.
<wgrant> The slaves can't talk to rabbit.
<bigjools> no no no
<rvba> StevenK: and in return you might give me a hand with the branch that William kindly accepted to review tomorrow... deal? ;)
<bigjools> we just have the existing query replaced with a blocking fetch of an item from Rabbit
<wgrant> bigjools: At present dispatching is handled by the normal builder polling loop. For each idle builder, we find suitable builds.
<wgrant> Mm.
<bigjools> in that thread, anyway
<wgrant> I guess we could work that out, somehow.
<bigjools> polling - BAD
<wgrant> But it's not going to be as awesom as the job system.
<wgrant> Which can be fully rabbit and instaneous.
<bigjools> some good jokes on here today
<wgrant> Like it should have been 7 years ago, but I digress...
<jtv> bigjools: polling is bad for some things.  It can actually scale well in some circumstances.
<bigjools> jtv: I know, but I am talking about the insidious nature of it in LP generally
<StevenK> rvba: I may regret this ... but what branch? :-)
<rvba> StevenK: I see you're cautious ... https://code.launchpad.net/~rvb/launchpad/sync-bug-827608-credit-copy-2/+merge/74958.
<wgrant> mvo: Any luck with apt-ftparchive? I'm probably only around for another hourish if you need help to reproduce it.
<wgrant> StevenK: What's blocking Jenkins from moving into the DC now?
<wgrant> StevenK: I think we should kill off buildbot soon.
<wgrant> StevenK: No reason not to now.
<wgrant> Jenkins now has the good combination of attributes of a) not sucking and b) being reliable
<wgrant> Neither of which buildbot is good at.
<mvo> wgrant: could you mail me your config? just for completness? I did not get far, too many interruptions, but I attack it now
<wgrant> mvo: Let me attempt to obtain one.
<mvo> thx
<wgrant> There we go, hung...
 * wgrant grabs the config.
<wgrant> The hang makes it nice and easy to grab it from the tmp dir...
<jtv> wgrant: further checking in Sources.gz confirms the GToGB.  Another victory for theoretical soyuz physics!
<wgrant> jtv: Excellent. How is the next run going?
<jtv> It is, that's all I know.
<wgrant> Heh
<wgrant> mvo: <http://pastebin.ubuntu.com/687565/>, invoked as 'apt-ftparchive --no-contents generate /var/tmp/archive/ubuntu-misc/apt.conf'
<wgrant> Let me just try it in a clean dir.
<mvo> great, thanks
<wgrant> Hmm.
<wgrant> I may have to give you a tarball with other state.
<wgrant> Ah, there we go.
<wgrant> mvo: mkdir -p /var/tmp/archive/ubuntu/dists/warty/{main,universe}/i18n is sufficient to make it hang.
<nigelb> bigjools: When does the Rabbit stuff come up?
<bigjools> nigelb: in what context?
<wgrant> mvo: With just one of those it's fine. With both it hangs.
<nigelb> bigjools: In the context of MPs
<bigjools> nigelb: when it's done
<nigelb> HA. This sounds like the answer to "When does Ubuntu 10.10 come out?"
<wgrant> nigelb: Interesting that you should pick 10.10
<nigelb> ;)
<wgrant> nigelb: As it's the one release where the time was precisely known months before :)
<nigelb> I meant to type 11.10, but typo'd to the one release I'd be wrong :)
<wgrant> mvo: It spawns to child apt-ftparchives, each with a gzip, and they block on a pipe...
<wgrant> s/to/two/
<wgrant> On dogfood is spawned four, so I presume it's one per component, as would make sense.
<mvo> wgrant: great, thanks, that is really useful information, the vm is still building but that sounds like reproducing should be really straightforward
<wgrant> wgrant@lucid-test-lp:~/launchpad/lp-branches/devel$ apt-cache policy apt-utils
<wgrant> apt-utils: Installed: 0.7.25.3ubuntu9.6+mvo1
<wgrant> launchpad@mawson:/srv/launchpad.net/codelines/current$ apt-cache policy apt-utils
<wgrant> apt-utils: Installed: 0.7.25.3ubuntu9.6+mvo1+0.IS.10.04
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 262 - 0:[########*** stack smashing detected ***: ./lp terminated
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<jtv> wgrant: new gina results are in.
<jtv> Very similar to the old ones, I'm afraid.
<jtv> New: 69 went from Superseded to Deleted, and 3 from Deleted to Published.
<wgrant> O_o deleted to published?
<wgrant> Oh/
<wgrant> This is diffing from the old statuses, though.
<wgrant> So that makes sense.
<wgrant> old = before we set everything back to published
<wgrant> https://dogfood.launchpad.net/debian/+source/dpkg/+publishinghistory
<wgrant> Some stuff still from 2011-09-01...
<wgrant> Oh, but that's not sid.
<wgrant> All the sid stuff is freshly and correctly superseded.
<wgrant> I declare this a provisional success.
<jtv> Ah yes, ISWYM
<wgrant> Since I imagine you're EODing soon, want to set the whole archive back to published and let it rerun?
<wgrant> We can check it in the morning.
<wgrant> It could take a little while.
<wgrant> Do you have names for the Deleted->Published three?
<wgrant> That is a little bit odd.
<bigjools> wgrant: do you have any idea how we can manage chroots remotely?
<bigjools> I thought of doing some rain-type dance, myself
<jtv> wgrant: from the all-Published starting point, changes were: 18,668 stayed Published, 60,106 became Superseded, 182 became Deleted.
<jtv> wgrant: the three that now stayed Published even though they were previously Deleted:
<jtv>  bzr               | 1.5-1.1
<jtv>  a2ps              | 1:4.14-1
<jtv>  python-virtualenv | 1.3.3-1
<jtv> Mind you, those were Deleted before we started, and then we set them back to Published & re-ran domination, so maybe the previous deletions weren't even domination-based.
<wgrant> Hah
<wgrant> Since there was no way to delete them...
<wgrant> bigjools: Easy to do through the API.
<wgrant> bigjools: Permissions are the hardest bit...
<bigjools> wgrant: !!!!
<wgrant> ?
<bigjools> uploading hundred meg files?
<wgrant> Well, there is that, but it's not without precedent.
<wgrant> eg +storeblob does it sometimes.
<bigjools> no way jose, not in the API
<wgrant> jtv: Erm.
<wgrant> jtv: Want to check the archive on those?
<wgrant> jtv: I suspect they're not archive=3 :)
<jtv> No idea what to look for.
<jtv> Oh, archive in the DB
<bigjools> I suspect we need something like poppy and process-upload
<wgrant> bigjools: aaaaaaaaaaa
<jtv> wgrant-induced archive pollution
<lifeless> why?
<wgrant> HTTP uploads should work fine.
<jtv> wgrant: you're right, not in archive 3.  71, 2919, 7823 respectively.
<wgrant> We need to be able to do large HTTP uploads.
<wgrant> jtv: That would do it.
<lifeless> exactly. May I introduce you to the librarian.
<bigjools> as long as it's not through an appserver, yes.  I didn't specify which protocol.
<wgrant> jtv: Now, what have I told you about queries that mention a series or distribution without an archive ;)
<lifeless> appservers can do several hundred MB uploads fine actually; though it is undesirable I would advise not blocking on that aspect.
<lifeless> as we already have that through storeblob
<wgrant> Given how rare chroot uploads are.
<bigjools> lifeless: you want to tie up an appserver thread for an hour?
<jtv> wgrant: â¦that all I had to fear was the mess you left in the database?
<wgrant> I don't think it's going to be much of a problem.
<wgrant> jtv: Silence!
<jtv> Well you asked.
<bigjools> rare *now* yes
<lifeless> bigjools: well, its more complex that than... it doesn't get dispatched to a thread until the upload completes.
<bigjools> we probably want to do some checks on the upload too, which would mean unpacking it
<wgrant> bigjools: I don't think so.
<bigjools> I really, really don't want to do this in the appserver
<bigjools> wgrant: well, I do
<wgrant> bigjools: If a distro owner wants to fuck their own distro, they can feel free.
<lifeless> bigjools: that should be done post upload, same as the apport blobs that storeblob does
<wgrant> What checks can we perform?
<bigjools> wgrant: they'd also be fucking the builder
<wgrant> bigjools: Oh?
<wgrant> GNU tar shouldn't allow builder fucking.
<jtv> wgrant: for my next experiment, I'd love to delete a package from Sources.gz and re-run.  Or would that be very bad of me?
<wgrant> Not even LP does now.
<wgrant> jtv: That would be very correct and thorough of you.
 * jtv looks for the catch
<lifeless> bigjools: I guess for me - I'd like to see it be uploaded via +storeblob, then we can move *all* +storeblob to the librarian - less special cases, more win.
<wgrant> jtv: I think gina is crap and doesn't verify Release, so you should just be able to change the file.
<bigjools> lifeless: that sounds ok, as long as we have a way of processing it later
<wgrant> lifeless: That's roughly what I was thinking.
<wgrant> bigjools: The uploader gets a UUID back.
<lifeless> bigjools: yes, +storeblob goes into a  queue for post-processing
<bigjools> cool
<wgrant> bigjools: They can then give that to a chroot management function.
<lifeless> bigjools: you do an upload and then an API call saying 'hey, use this for $that'
<lifeless> bigjools: and stuff gc's after a day or something
<wgrant> Pretty much.
<bigjools> splendid
<bigjools> we don't need to worry about this yet anyway
<wgrant> But I'm pretty sure we can't do much sensible post-processing here, so it may just end up creating a PocketChroot with the LFA.
<bigjools> *shrug*
<wgrant> jtv: Could you set up the full archive run just before you EOD? It would be handy to have that test.
<wgrant> Although it is looking very promising so far.
<jtv> wgrant: as in, all of Debian?
<wgrant> Yes.
<wgrant> UPDATE sourcepackagepublishinghistory SET status=2, supersededby=NULL, datesuperseded=NULL WHERE archive=3
<jtv> And by the way, with the work I've been doing, the correct acronym for my transition from work to not working is OD, not EOD.
<wgrant> Heh
<wgrant> lifeless: We can still land DB patches with only one approval, right?
<lifeless> wgrant: yes, long as they complete in < 10 seconds etc.
<wgrant> I'm just dropping 35 tables... should be pretty quick.
<nigelb> wow
<lifeless> cool
<lifeless> that might shave 10% off our slony overhead
<wgrant> That was precisely my reasoning for doing it now.
<wgrant> 305 -> 270
<jtv> Is POComment among them?
<wgrant> No.
<wgrant> Should it be?
<wgrant> I noticed it, but didn't know it to be obsolete.
<jtv> I _think_ so.
<wgrant> I thought it was still used or something.
<wgrant> POSubscription is dying, however.
<jtv> I'm not sure we ever used it.
<jtv> Good.
<wgrant> I collected this 35 on one pass through \dt
<wgrant> I probably missed a couple of HWDB/translations.
<wgrant> Since I don't know those areas wonderfully.
<wgrant> http://paste.ubuntu.com/674281/ is the list I have patches for.
<jtv> We also have plenty unused columns.
<wgrant> Yeah.
<wgrant> Those are less of a win, though.
<wgrant> (deleting tables is a win because disabling/enabling triggers is slow, and slony has to do that to every table around every slonik script)
<jtv> Well we do have some tables that are painfully slow to scan or retrieve from.
<wgrant> Indeed.
 * jtv checks a few columns
 * wgrant goes through the table list again.
<lifeless> grr
<wgrant> Uhoh.
<lifeless> setuptools you really annoyme
<lifeless> I remember a --egg-info parameter
<lifeless> but not what command
<wgrant> setup.py egg_info -b-somesuffix sdist?
<lifeless> looks like
<lifeless> ./setup.py egg_info -b -lpwithnodatetime-r397 sdist
<lifeless> but now I need to wait for jamesh :)
<wgrant> :(
<lifeless> he had some feedback on the patch, I don't want to roll two eggs for LP to migrate across
<jtv> wgrant: Some potentially unnecessary ones in TranslationMessage: is_fuzzy, was_obsolete_in_last_import, was_fuzzy_in_last_import.  In POFile: description, fuzzyheader, from_sourcepackagename.  In POTemplate: copyright, sourcepackageversion, binarypackagename.  In POTMsgSet: potemplate, sequence.
<lifeless> I'll switch back to the gpg migration stuff
<jtv> wgrant: Not seeing any change after removing a package from Sources.gz and re-dominating.  :(
<allenap> benji: Got time for a shortish but possibly wartish review? https://code.launchpad.net/~allenap/launchpad/series-init-failure-explanations-bug-835024-ui/+merge/74974
<benji> allenap: sure
<wgrant> jtv: Sure it was the right series?
<jtv> sid.
<wgrant> :(
<wgrant> jtv: What did you delete?
<allenap> benji: Thanks.
<jtv> wgrant: adonthell
<wgrant> Hmm, indeed.
 * wgrant checks the code.
<jtv> Still seeing a neat Superseded, Superseded, Superseded, Published run for that one.
<wgrant> jtv: Oh.
<wgrant> You know, we're still in transitional mode...
<wgrant> It's not meant to be Deleted in transitional mode, is it?
<jtv> *bash*
 * jtv thinks
<wgrant>  The latest version
<wgrant>             # will then, finally, be marked appropriately Deleted once
<wgrant>             # we remove this transitional hack.
<jtv> Yup.
<jtv> I already have the follow-up branch waiting in the wings, and that messed up my view of transitional domination.  Thanks for spotting that.
<wgrant> I'd forgotten about it too, don't worry.
<wgrant> Until I checked the code.
<wgrant> And saw the 10 line XXX
<wgrant> ooooh.
<wgrant> jtv: I think pocomment is the only remaining unused table.
<wgrant> Well, unused except by the POTranslationPruner.
<StevenK> wgrant: Can haz link to your branches?
<wgrant> :( it references person
<wgrant> So I might add it to this lot.
<wgrant> Since it means it's a three-branch effort to remove it.
<wgrant> jtv: Can I kill it?
<StevenK> wgrant: Did you want me to clean up POTranslationPruner?
<StevenK> Since that should be quickish
<wgrant> No, I'll delete that in my second branch.
<wgrant> I already delete a bit of other stuff there.
<wgrant> https://code.launchpad.net/~wgrant/launchpad/demolish-unused-tables-1-db/+merge/74960, https://code.launchpad.net/~wgrant/launchpad/demolish-unused-tables-2/+merge/74961, https://code.launchpad.net/~wgrant/launchpad/demolish-unused-tables-3-db/+merge/74963
<StevenK> Love the branch names
<StevenK> wgrant, behind the wheel of a wrecking ball.
<wgrant> ï¼â¯Â°â¡Â°ï¼â¯ï¸µ â»ââ»
<jtv> wgrant: yes, please kill POComment.  If we ever used it, I think it has to be more than 4 years ago.
<StevenK> +ALTER TABLE pushmirroraccess DROP CONSTRAINT "$1";
<StevenK> Das twitch
<jtv> wgrant: Next experimentâ¦ change component on a package.  Should supersede the existing SPPH and publish a new one.
<wgrant> (pushmirroraccess is not related to distribution mirrors, but arch mirrors, FWIW... I thought it was a WHUI for managing Ubuntu push mirrors, but no, far worse)
<wgrant> jtv: Indeed. That works in archivepublisher, but I haven't tried gina.
<jtv> Errâ¦ I say Component, but what does that look like in a Debian sources list?
<nigelb> Ah, demolish.
<nigelb> Need to add to that list of names I should use in a branch.
<wgrant> jtv: Sources is divided by component.
<jtv> nigelb: it's been done now.  But âannihilateâ may still be available.
<wgrant> jtv: You'll note that you're looking at main/source/Sources... use contrib/source/Sources instead.
<jtv> Oh, OK
<StevenK> wgrant: Going to link bug 345810 to -3
<_mup_> Bug #345810: Remove old infestations database stuff <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/345810 >
<jtv> wgrant: I guess in this case I should move a package from one to the other.
<wgrant> StevenK: I'm going to search that out on Wednesday when I land -3, yep.
<wgrant> jtv: Indeed.
<jtv> Or copy.
<wgrant> That might get confusing.
<wgrant> It's not really a situation that the new gina logic supports.
<wgrant> But it should behave the same way.
<StevenK> wgrant: So, is that "Yes, please link" or "No, I will look for that and other bugs on Wednesday?"
<wgrant> StevenK: You might as well link.
<jtv> wgrant: I can't help but notice that the contrib Sources is empty.
<jtv> Not any more, of course, with a package moved in there.
<wgrant> Heh.
<StevenK> wgrant: So now it's the Comdemned 36?
<wgrant> StevenK might be a bad person, I suspect.
<wgrant> wgrant@lucid-test-lp:~/launchpad/lp-branches/demolish-unused-tables-1-db$ bzr ci -m "pocomment becomes the last member of the Condemned 36."
<wgrant> Way ahead of you, StevenK.
<StevenK> Heh
<StevenK> wgrant: Yes, I'm a bad person
<wgrant> StevenK: How did you create the gina-mirror on mawson?
<wgrant> StevenK: It seems to only have main.
<StevenK> By Perl and by hand ... :-)
<wgrant> As I said.
<wgrant> Bad person.
<wgrant> Hmm.
<wgrant> I wonder what breaks if I land -3-db with a commit message of "ï¼â¯Â°â¡Â°ï¼â¯ï¸µ â»ââ»"
<jtv> Please can I land mine first before you ruin us?
<StevenK> Ruin?!
<StevenK> We're deleting 36 tables!
<wgrant> s/Ruin/Make/
<jtv> By breaking PQM and/or buildbot?  Yes, ruin.
<StevenK> You should be PRAISING us!
<jtv> For breaking PQM and/or buildbot?  Alright: praise you.  Praise you all to hell.
<wgrant> But gina'
<wgrant> s still in transitional mode.
<wgrant> We won't actually be removed.
<StevenK> buildbot needs to die anyway
<wgrant> So you can send us to hell, but we'll still be here :(
<StevenK> And PQM sucks
<jtv> StevenK: yes but _after_ I've landed my next few branches please, is what I'm saying.
<jtv> wgrant: you'll be superseded here and published in hell.
<wgrant> fuuuu
<benji> allenap: your branch looks fine, I had one idea for a small simplification to the template
<allenap> benji: Ah, cool. If tal:content evaluates to nothing, does the paragraph get dropped?
<benji> allenap: no :)  I was figuring that an empty paragraph wouldn't hurt too much.
<allenap> benji: Okay. I'll check.
<jtv> wgrant: still no change.  Not sure what I've done to deserve this.  Have I been editing in the wrong place maybe?
<jtv> wgrant: I really have to go.  I'll kick off that Debian run first.
<wgrant> jtv: It didn't even crash?
<wgrant> jtv: I would expect a crash, given that the files aren't in the contrib pool.
<wgrant> jtv: But maybe it doesn't look there if there's something already in the archive.
<wgrant> We shall investigate tomorrow.
<nigelb> jtv: Need to borrow Mark's dictionary for some new words ;)
<jtv> wgrant: oh, then maybe it just ignores it.
<wgrant> nigelb: Heh
<jtv> benji: I chucked two very brief branches onto the review queue, but am now OD.
<jelmer> does anybody have experience working on Launchpad in lxc?
<benji> jtv: I hope that was a dropped E on EOD and not really OD ("overdosed") ;)
<jtv> benji: I know what I said.
<benji> heh
<wgrant> jelmer: lifeless and I do it.
<jtv> wgrant: gina didn't create a second SPPH for adonthell.  I could just make its next-to-last SPPH point to the same SPR as the previous one, and see how it fares in the Great Overnight Debian Domination of 2011.
<jelmer> wgrant: I just found /Running/LXC on the wiki
<wgrant> jelmer: I've been doing it for about two weeks now.
<wgrant> jtv: Let's not push it.
<jelmer> wgrant: I guess that means it works ?
<wgrant> jelmer: Yeah. I run mostly lucid-on-oneiric, but also sometimes oneiric-on-oneiric.
 * jelmer is a bit tired of Oneiric breaking his Launchpad setup every couple of days, so I'd like to run Lucid in LXC
<wgrant> jelmer: lucid-on-natty also mostly works, with a bit more effort as described on the page.
<jtv> wgrant: OK, I'll just kick off the GODD2011 then.
<wgrant> jtv: GODD2011A, just to be safe.
<wgrant> I doubt this will be the last :(
<jtv> OK.
<jtv> BTW you just had me update 172737 records.
<jtv> wgrant: I guess I should run gina with --all then?
<wgrant> jtv: A good question.
<wgrant> Let me check what targets we have...
<jtv> (BTW I have a branch on the review queue that adds a --list-targets option)
<wgrant> Hm. DF may only have sid configured...
<jtv> Brillant.
<wgrant> Bah, and we only have sid indices.
<wgrant> Upsetting.
<wgrant> I guess we'll have to deal.
<jtv> Soâ¦ still a sid-only run then?
<wgrant> Seems so :(
<jtv> Actually, that's a no-op now.
<jtv> No, it's not.  We just reset everything to Published.  I'll re-run.
<allenap> benji: Looks good. Also, the |nothing wasn't needed either; None is rendered as the empty string it seems.
<jtv> wgrant: it's on the way.  I suggest we both stop working now.
<wgrant> Probably.
<wgrant> Just finishing off the pocomment abolition.
<wgrant> I'm going to try and get the two DB patches deployed tomorrow and Wednesday.
<wgrant> As a test of fastdowntime efficiency.
<jtv> Then we'll have to finish Q/A for this first.
<nigelb> How does the DB patches deployment work?
<wgrant> jtv: Yeah, QA for this has to be done by Wed morning.
<wgrant> But we have all of tomorrow.
<jtv> And we've got a lot out of the way already.  Good thing they were separate tickets.
<wgrant> jtv: yeah.
<wgrant> jtv: I will check it all out in the morning.
<wgrant> And hopefully declare qa-meh
<jtv> Thanks.  And now, offness.
<wgrant> Night!
<jtv> nn
<nigelb> later jtv!
<wgrant> nigelb: The actual deployment? We turn off pgbouncer, which has the effect of disconnecting everyone from the DB and making appservers crash terribly when they try to talk to it.
<wgrant> Then we apply the patches through slony.
<wgrant> Recreate all stored procedures.
<wgrant> Reapply all security settings.
<wgrant> Then turn pgbouncer back on.
<rvba> StevenK: I've improved the ConfirmationOverlay so that it can take a callback method to call instead of submitting the form.
<wgrant> Roughly.
<nigelb> wgrant: And how often does it happen?
<wgrant> nigelb: Up to once a day.
<rvba> StevenK: The branch is up for review.
<wgrant> At 0830Z
<wgrant> If required.
<nigelb> wgrant: and sequential?
<nigelb> (in terms of db-devel revisions)
<wgrant> nigelb: Yes. And lifeless says we should apply one patch at a time, but I'm going to hope he's not around if we ever get more than one in the queue.
<rvba> benji: Hi, could you have a look at this tiny js branch please? https://code.launchpad.net/~rvb/launchpad/confirmationoverlay-fn/+merge/74997
<nigelb> wgrant: Well, I may have landed one today, and I thought you were landing a few as well?
<wgrant> nigelb: Ah, true, forgot there was yours in front of mine.
<wgrant> Sad.
<wgrant> I have two, but one can only be landed once the other is deployed.
<benji> rvba: sure
<nigelb> wgrant: Ah, ouch.
<wgrant> I may have to convince Lynne to drag lifeless away for a few hours tomorrow night.
<nigelb> wgrant: You do know lifeless is working 4 days a week right?
<nigelb> :)
<nigelb> Oh, that might work as well.
<rvba> benji: Thanks.
<benji> allenap: I should have realized you'd get a "None" out.  Is that attribute always a string?  If so, you should be fine doing it that way.
<allenap> benji: error_description can be None, but my point was that the template DTRT and renders it as "" instead of "None".
<benji> cool
<benji> rvba: done, I had one thought about a test addition, but the branch looks good
<rvba> benji: Okay, thanks.
<StevenK> rvba: You have a typo in your branch "Ann (optional)" it should be "An (optional)"
<rvba> StevenK: Thanks for spotting this, I'll add the test as benji suggests and land this.
<StevenK> Sounds great to me.
<rvba> benji: If you're up for that I've got another branch up for review ;) https://code.launchpad.net/~rvb/launchpad/sync-greyedout-resolved-841934/+merge/74975
<benji> rvba: sure thing
<rvba> Thanks.
<cr3> hi folks, is there a reason why bin/lint.sh in launchpad uses pocketlint instead of pyflakes? what's the difference between the two?
<benji> rvba: done
<nigelb> cr3: pocketlint uses pyflakes and pep8
<cr3> nigelb: thanks!
<rvba> benji: Thanks.
<benji> my pleasure
<nigelb> cr3: and I'm fairly sure it can lint more than just python.
<nigelb> There was more to it, but I didn't dig deep enough the last I checked.
<cr3> nigelb: correct me if I'm wrong but I don't think pocketlint considers disable-msg comments in source files, which is kinda annoying
<nigelb> I don't know about that, sorry.
<jcsackett> sinzui: care to mumble?
<sinzui> yes
<sinzui> jcsackett, sorry, I pressed screencap and too 2000 pictures
<jcsackett> wow. bet that ate some resources.
<sinzui> I heard you, but I need another minute to fix my oops
<jcsackett> ok.
<jcsackett> sinzui: i can hear you, seems you cannot hear me?
 * jcsackett restarts his mumble.
<abentley> deryck: chat?
<deryck> abentley, sure, give me 5 minutes to wrap what I'm doing.
<abentley> deryck: sure.
<sinzui> jcsackett, ssh-askpass-gnome
<deryck> abentley, I'm good now.  Firing up mumble.
<jcsackett> sinzui: just in case it helps, this is the error http://paste.ubuntu.com/687676/
<sinzui> jcsackett, bug 796873
<_mup_> Bug #796873: ec2 land generates gnomekeyring.IOError if run over an ssh session <launchpadlib :Triaged> <Python Keyring:New> < https://launchpad.net/bugs/796873 >
<nigelb> sinzui: around?
<sinzui> nigelb, I am
<nigelb> sinzui: Looking at #launchpad, I just noticed a weird picker issue.
<nigelb> The picker doesn't find me when trying to assign a bug to myself. Works on subscribing though.
<nigelb> (Well, the seach asactually.
<nigelb> Gah
<nigelb> The search can't find 'nigelbabu'
<bigjools> we need a picker for nosetests, then it would be a nose picker
<nigelb> Or anyone else.
<nigelb> bigjools: bwahaha.
<bigjools> I'm here all week
<sinzui> nigelb, yuck
<sinzui> I will look into this
<nigelb> sinzui: pfefferz just noticed this, you might want to help him out :)
<nigelb> bigjools: Also, you guys didn't get to win yesterday.
<nigelb> Well, we didn't win either. But meh.
<bigjools> nigelb: outrageous umpiring keeping them on in the rain
<bigjools> the 2 late wickets swung you a tie
<nigelb> bigjools: Hey the English tried to delay things in the last over :)
<bigjools> nigelb: it was pretty hilarious - when India was ahead in D/L they rushed off and left England hanging around. When England were ahead in D/L, the exact opposite happened.
<nigelb> :D
<nigelb> sinzui: I'd recommend a blog post about this. I'm sure more people will trip over this fairly soonish :)
<sinzui> nigelb, I was planning to. I am landing a branch that will require more attention from testers.
<nigelb> \o/
<nigelb> I like the new picker, except for when I have to click twice even though I know the person's LP ID.
<nigelb> (I think I saw that bug already)
<nigelb> Ok, so js-oopsd.
 * nigelb gets to coding it.
<nigelb> bigjools: semicolon slash slash?
<nigelb> you mean :~/
<nigelb> ?
<sinzui> nigelb, I am landing the branch to address that very issue. The expander will move the the next line so that you can use the first line to select the user
<bigjools> nigelb: yes, there was some C++ or Java in your tweet
<nigelb> sinzui: \o/
<nigelb> bigjools: lol
<nigelb> That's the zsh theme :P
<bigjools> I used to use zsh actually, back when the alternative was csh or sh
<bigjools> 22 years ago, yegads
 * nigelb was probably born around that time-ish :P
<nigelb> bigjools: I only use zsh, because oh-my-zsh pretty much configues it out of the box.
<nigelb> I also typo a lot and most of the time zsh catches it.
<nigelb> Most of the scripts I write are bash, not zsh though :)
<nigelb> Are there rules around how to create a new file?
<nigelb> (this is for a service, js-oopsd - https://launchpad.net/js-oopsd)
<Ursinha> I'd like to get the list of ubuntu packages available in main, how would one do that through the API?
<jml> hmm.
<jml> Ursinha: source packages or binary packages?
<jelmer> Ursinha: IIRC something like lp.distributions['ubuntu'].getSeries(name_or_version="oneiric").getPublishedSources()
<jml> jelmer: distroseries don't have published sources
<Ursinha> jml, packages that can be bugs files against
<jelmer> Ursinha: actually,  lp.distributions['ubuntu'].getSeries(name_or_version="oneiric").main_archive.getPublishedSources()
<Ursinha> main_archive is not just main
<Ursinha> it includes universe as well, I was told
<jelmer> ah
<jelmer> I guess you could filter out some packages based on section, though that would be quite slow
<jml> jelmer: not quite! lp.distributions['ubuntu'].main_archive.getPublishedSources(component="main", distro_series=lp.distributions['ubuntu'].getSeries(name_or_version="oneiric"), status="Published")
<Ursinha> In [41]: ubuntu.main_archive
<Ursinha> Out[41]: <archive at https://api.launchpad.net/1.0/ubuntu/+archive/primary>
<jml> oh crap
<Ursinha> ah
<jml> no "component" there
<jml> sorry
<james_w> yeah
 * Ursinha looks for a bug
<james_w> you you have to iterate over the return of getPublishedSources
<james_w> and do if publishing_record.component_name != "main": continue
<jelmer> Ursinha: I'm not sure in what context you're trying to do this, but another option could be to use the local apt cache
<james_w> I don't know of a way to have LP do that step for you
<jml> hmm
<Ursinha> that's bad isn't it
<Ursinha> james_w, do you know if there's a bug for that?
<james_w> I don't, sorry
<jml> Ursinha: I'm in mega-distraction mode anyway, let me try knocking up a patch.
<Ursinha> jelmer, I'm trying to get this information from launchpad, to check which teams are subscribed to them
<jml> (warning: very low timeout on this, I might give up)
<Ursinha> jml, awesome. I owe you a pack of the finest brazilian coffee.
<Ursinha> bug 848097
<_mup_> Bug #848097: Not possible to get a list of packages in a component through the API <api> <Launchpad itself:New> < https://launchpad.net/bugs/848097 >
<jml> Ursinha: ta
<jml> grunk.
<nigelb> grunk?
<Ursinha> querying the published sources timesout consistently :/
<jml> Ursinha: james_w has some code in lp:udd that could help with that
<jml> Ursinha: (I'm using the API for my pkgme work)
 * Ursinha looks
<jml> nigelb: the bug in the testing-cabal version of testtools that I just fixed is preventing me from running lp tests
<nigelb> jml: Ouch
<jml> (I'm waiting for LP to build & publish the new package)
<nigelb> use virtualenv more often? :)
<jml> nigelb: I had to use a lot of willpower then.
<jml> nigelb: I'd rather not.
<nigelb> :)
<nigelb> Hrm. Need more help on this js-oopd. Gah, should find lifeless tomorrow morning.
 * jml works around
<nigelb> jml: I'm curious, what's the work around?
<jml> nigelb: comment out the import of subunit in lp.testing
<nigelb> ah.
<cjwatson> jml: that udd backoff code really ought to move into launchpadlib ...
<Ursinha> +1
<jml> cjwatson: agreed.
<jml> cjwatson: I guess now that there's less internal resistance to adding stuff to lplib, I could probably start proposing patches.
<cjwatson> particularly since we're now doing fastdowntime which is likely to kick off API scripts once per business day
<cjwatson> even if it doesn't happen at other times ...
<jml> cjwatson: good point.
<Ursinha> bug 848120
<_mup_> Bug #848120: add the udd backoff code to lplib <launchpadlib :New> < https://launchpad.net/bugs/848120 >
<jml> Ursinha: :)
<Ursinha> :)
<jml> bugger.
 * jml installs rabbitmq-server, breaks "Restart"
<jml> rabbitmq being necessary to run archive pagetests, of course
<jml> Ursinha: https://code.launchpad.net/~jml/launchpad/get-published-sources-component-name/+merge/75054
<jml> ~43 mins?
<jml> LP needs to be easier to hack on.
<jml> or I need to get faster. or both, probably.
<Ursinha> jml, \o/
<jml> Ursinha: while you are in the flush of gratitude, might I ask you to update https://wiki.ubuntu.com/JonathanLange with a recommendation for membership?
<Ursinha> sure :)
<jml> Ursinha: thanks.
<Ursinha> benji, hello :) what's the criteria to have a branch reviewed? jml just wrote a patch that will make me cry rainbows and unicorns
<benji> Ursinha: crying rainbows and unicorns is, in fact, the criteria
<Ursinha> awesome
<Ursinha> :)
<benji> Ursinha: this one? https://code.launchpad.net/~jml/launchpad/get-published-sources-component-name/+merge/75054
<jml> benji: yep!
<Ursinha> benji, yes :)
<benji> I'll take a look now.
<Ursinha> benji, thanks!
<benji> jml, Ursinha: the branch looks good, approved
<jml> benji: thanks. Can you please land it? I don't have commit access any more.
<jml> (a matter of implementation, rather than policy)
<benji> jml: sure
<jml> benji: thanks.
<nigelb> crying rainbows and unicorns? heh.
<jelmer> jml: huh, why did you lose commit access?
<jml> jelmer: because I left the canonical-launchpad team.
<nigelb> I thought thre was an emeritus team.
<nigelb> *there
<jelmer> jml: I think I was kicked out of that too, but I still have commit access.
<jml> nigelb: there's a *plan* for one
<nigelb> Ah.
<jelmer> ah, ~canonical-bazaar is a member of ~launchpad for some reason
<nigelb> access probably.
<nigelb> jml: JFDI it ;)
<jml> nigelb: I don't have the necessary permissions. (or inclination, tbh)
<nigelb> heh
<jml> publishing takes for*ever*
<Ursinha> thanks jml and benji :)
<nigelb> lifeless: ping, for whenever you get on :)
 * deryck goes offline for lunch, back soon
<lifeless> nigelb: hi
<nigelb> lifeless: Hey, I had a few questions from python-gpgfixtures
<lifeless> flacoste: morning
<lifeless> nigelb: cool
<flacoste> hi lifeless
<nigelb> One is, what does "__metatype__ = object" do, and the other is, is threading necessary?
<lifeless> nigelb: __metatype__ = object makes
<lifeless> class Foo:
<lifeless> equivalent to
<lifeless> class Foo(object):
<nigelb> Ahh.
<lifeless> nigelb: as for threading, perhaps a little context will help me answer :)
<timrc> Why can't we re-use ppa names once the ppa has been deleted?
<lifeless> because apt.
<lifeless> https://answers.launchpad.net/launchpad/+faq/661
<nigelb> In the keyserver.py, there's "from threading import Lock" and "self.gpg_lock = Lock()"
<nigelb> I'm guessing that's threading, but I may be wrong :)
<lifeless> so, wsgi servers are threaded generally
<nigelb> ah
<lifeless> an app 'foo' will be called within each thread
<nigelb> I'm new to writing bare wsgi. So, my inexperience comes into play slightly :)
<lifeless> unless you take special steps to ensure non-threaded
<lifeless> simpleserver for instance has one thread listening
<lifeless> then on a new request hands that off to a worker
<lifeless> which does
<lifeless> app(environ, start_response)
<nigelb> AHH.
<lifeless> the gpg fixtures keyserver wants to avoid race conditions
<lifeless> so it has a little lock in it
<lifeless> that doesn't mean other wsgi apps need, or do not need, such a lock.
<nigelb> ok :)
<nigelb> lifeless: Oh and finally, could you review https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/74934 :)
<nigelb> I have stub's greenlight
<lifeless> no need
<lifeless> the review for me is so I am in the loop
<nigelb> Oh.
<lifeless> read the page again :)
<nigelb> In that case, I'm guessing stub just forgot to land it.
<nigelb> I thought it might be because you needed to ack :)
<lifeless> no, that would be a pita for folk
<stub> No, its because I'm too used to people landing their own branches that I didn't think :-)
<nigelb> Wha.
<nigelb> Why are you awake.
<lifeless> nigelb: you're like the third non-canonical person to land db patches
<nigelb> Oh.
<stub> Cause I woke up at 3pm of course. What a silly question!
<nigelb> Oh, nice.
<nigelb> lifeless: Who else besides William?
 * nigelb looks
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> Ah, I see.
<nigelb> stub: thanks again!
<stub> np
<nigelb> lifeless: The process was relatively easy. Except for the bit about fti.py which I needed stub's help.
<nigelb> <3 the detailed documentation.
<nigelb> stub: Your sleep cycle seems lovely ;)
<lifeless> its more of an orbit
<lifeless> cause it has precession
<nigelb> lol.
<nigelb> Yeah, I've seen him wake up at 3 as well
<nigelb> *3 pm
<Pendulum> lifeless: I think nigelb is attempting to get his own sleep orbit as well ;-)
<nigelb> I have a precise cycle. I sleep betwen 2 am and 3 am and wake up betwen 9 am and 10 am.
<Pendulum> nigelb: hah
<nigelb> Secretly, I've given up on seeing 3 am when I wake up.
<Pendulum> nigelb: I don't know that I believe you go to bed that early. Certainly not with that consistancy :P
<stub>    test_sigint_exits_nicely (bzrlib.plugins.lpserve.test_lpserve.TestLPServiceInSubprocess)
<stub> buildbot in testfix. Anyone in maintenance want to look before I rebuild? Got several branches with ec2 I'd rather not bounce.
<nigelb> what happened to the RT about buildbot access to community?
<nigelb> at least to see what failed.
 * stub shrugs
<stub> This one has been a regular failure in any case.
<stub> or maybe not. dunno.
<bac> benji: are you still reviewing?
<benji> bac: yep
<bac> benji: great.  https://code.launchpad.net/~bac/launchpad/bug-831991/+merge/75068
<benji> I'll take a look momentarily.
<benji> bac: done, it looks good
<bac> benji: thx
<benji> np
<benji> I'm getting this error when running the tests: http://paste.ubuntu.com/687893/; the punchline is "psycopg2.OperationalError: fe_sendauth: no password supplied"
<jelmer> benji: Ian and me hit that as well
<jelmer> benji: The fix seems to be to make sure you have lines for IPv6 in your pg_hba.conf
<benji> ok, let me look at that
<jelmer> benji: http://pastebin.ubuntu.com/687895/ is what mine looks like now
<benji> thanks
<benji> jelmer: that fixed that error, thanks!  (now to figure out the next one, maybe a good old-fashioned make schema will help)
<jelmer> benji: what's the next one?
<jelmer> I also hit another one shortly after that, and then decided to go back to lucid.
<benji> "ProgrammingError: column distribution.package_derivatives_email does not exist"
<jelmer> ah, heh, that does indeed sound like a schema issue :)
<sinzui> jcsackett, ping
<jcsackett> sinzui: pong.
<sinzui> did you get the email about the 404
<jcsackett> sinzui: yeah, looks like i missed a doctest that needed updating.
<sinzui> jcsackett, I can update the one broken test and submit to pqm if want
<sinzui> jcsackett, That is an evil doctest.
<jcsackett> sinzui: that's fine by me, if you have the time.
<sinzui> I do
<jcsackett> sinzui: all doctests are evil. :-P
<sinzui> That one take more than 10 minutes to run
<jcsackett> ok, that's exceptionally evil.
<jcsackett> sinzui: you need me to change the owner of the 404 branch, or are you good?
<sinzui> No, I just branched and will wait for the test to complete
<nigelb> jcsackett++
<jcsackett> sinzui: dig.
<jcsackett> nigelb: :-)
* benji changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<flacoste> jcsackett, sinzui: nice solution to to the "too clicky" problem!
<nigelb> Has that landed yet?
<jcsackett> it's in the buildbot queue.
<nigelb> \o/
<sinzui> flacoste, That was all jcsackett's work. I am his landing surrogate while he beats his computer into submission
<lifeless> flacoste: horrible name, but http://pypi.python.org/pypi/RunningCalcs/0.1 - online algorithms; might be able to simplify the PPR using this
<nigelb> Does LP by itself have an OOPS report page?
<lifeless> no
<lifeless> the reporting is done by a not-yet-opened tool 'oops-tools'
<nigelb> AH.
<lifeless> thats blocked on time more than anything else
<nigelb> Open it for help? :)
<lifeless> need to audit for passwords etc that need changing before releasing the code
<nigelb> AH.
<lifeless> yes, it had deployment details mingled with the source tree.
<nigelb> Ouch
<lifeless> so did LP :P
<nigelb> My memory of LP doesn't extend that far :P
<nigelb> I've been around for only 2 years. I'm guessing LP has been open for most of it.
<nigelb> "like the Forth Bridge in
<jelmer> nigelb: yep
<nigelb> Scotland - its a never-ending job to paint it from end to end
<nigelb> that sounds like critical bugs for LP :P
<nigelb> (its a quote from an email to ubuntu-devel@)
 * jelmer remembers being very curious about the LP source code around UDS Barcelona (Karmic?)
<nigelb> I'm sure there are some bits everyone's curious about :P
<nigelb> jelmer: Yep, Karmic (https://wiki.ubuntu.com/DeveloperSummit)
<flacoste> lifeless: indeed!
<flacoste> lifeless: we'd probably have to port my median implementation to it though
<lifeless> flacoste: that would give it a home :)
<wgrant> sinzui, wallyworld_: I can't hear anything any more.
<LPCIBot> Project devel build #1,061: FAILURE in 1 hr 3 min: https://lpci.wedontsleep.org/job/devel/1061/
<StevenK> Hm
 * nigelb waves
 * jelmer hums back at StevenK
<nigelb> 200 != 503
<nigelb> that looks like HTTP status
<StevenK> There were 1 imports of names not appearing in the __all__.
<StevenK> You should not import GeneralizedPublication from lp.archivepublisher.domination:
<StevenK>     lp.soyuz.scripts.gina.dominate
<StevenK> RARGH
<nigelb> heh
<nigelb> This the "Gina, the dominatrix" branch
<StevenK> Part of it, yes.
<lifeless> can has review - alternative fix for the pgbouncer buildbot fail. https://code.launchpad.net/~lifeless/python-pgbouncer/bug-846236/+merge/75091
<wgrant> lifeless: You're not going to remove the other fix?
<lifeless> wgrant: the one I rolledback ?
<wgrant> Ah, that would do it.
<wgrant> We're going to assume that pgbouncer isn't insane, so it writes the whole pid atomically?
<wallyworld_> StevenK: i only have one _ today
<wgrant> wallyworld_: The day is still young.
<lifeless> if its writing one byte at a time and doing a flush, then we should fix it.
<wgrant> lifeless: r=me
<lifeless> put it this way, *all* the service management code in Ubuntu assumes that 'pid file is intact'
<wgrant> True
<lifeless> and I don't think, in general, that you can assume anything else and use pid files at all.
<StevenK> Oh, bleh. The next thing to do is land indicies
<wgrant> wallyworld_: Why is transitively_private becoming explicit rather than explicitly_private?
<StevenK> wgrant: Your merged branch uses -83-1, but according to allocated.txt, that is jml's number ...
<wgrant> wallyworld_: Having the default attribute be the right one to check sounds beneficial.
<wgrant> Erm, I pushed.
<nigelb> cheating! :P
<wgrant> .... but it failed and I didn't notice, because I was behind.
<wgrant> Fail.
<wallyworld_> wgrant: not sure i fully understand your ?. column "private" = explicitly private. column "transitively_private" = private because stacked on private branch
<wgrant> wallyworld_: Yes, but checking the explicitly private field is never the right thing to do.
<wgrant> But it is looks like a sensible default, because it's named 'private'.
<wallyworld_> checking in what context?
<wgrant> API or internal.
<wgrant> It's never OK to check the explicitly private column.
<wgrant> You have to check the transitively private column.
<wallyworld_> yes
<wgrant> And having the wrong one named 'private' sounds like an accident waiting to happen.
<wallyworld_> wgrant: the attribute is call explicitly_private. it's just that the db column has not been renamed
<wgrant> wallyworld_: Ah, k, as long as that's not exposed outside the DB.
<wallyworld_> the Branch.private attribute has already been renamed to Banch.explicitly_private
<wgrant> wallyworld_: And as long as not much queries directly...
<wallyworld_> only on place has hand coded sql
<wallyworld_> and this should be done using storm in any case
<jelmer> wgrant: thanks for those hints on lxc btw, seems to work well
<wallyworld_> i'm not sure how expensive it is to rename a column, but that would be nice to do once the dust has settled
<wgrant> jml: I'm sorry, I've stomped on your DB patch number. I failed to notice that my push failed, so I've landed 83-1 already. Could you please take another number? :/
<wgrant> jelmer: Yeah, there's only one tests that seems to fail in LXC.
<wgrant> wallyworld_: It's trivial.
<wgrant> wallyworld_: Except that it's hard to do with fastdowntime.
<lifeless> s/hard/impossible
<wallyworld_> wgrant: except that it would need to be done as a column copy, and then a delete
<wgrant> lifeless: Lies.
<lifeless> you can do a copy+delete
<lifeless> but not a rename
<wallyworld_> so that the code could be changed in between
<wgrant> lifeless: I think we can mangle the Storm classes with feature flags.
<jelmer> lifeless: wait, isn't that the same?
<lifeless> wgrant: as long as its not fragile
<lifeless> jelmer: one takes 0.0001 seconds. One takes a week.
<lifeless> jelmer: so no, not the same.
<jelmer> lifeless: You're to quick. I had a good joke lined up about renames in VCS. :P
<jelmer> *too
<lifeless> :P sorry
<lifeless> wgrant: bug 848400 is the rollback
<_mup_> Bug #848400: fixture shares fd's with testrunner process, will cause test hangs and worse. <Python PGBouncer:Fix Released by lifeless> < https://launchpad.net/bugs/848400 >
<StevenK> Is that was just caused the failure on Jenkins?
<lifeless> doubtful
<lifeless> possibly related
<lifeless> the rollback was in python-pgbouncer
<lifeless> next step is a version bump in lp's versions.cfg
<StevenK> It looked like a hang, no test output for 600 seconds?
<wgrant> Where?
<lifeless> https://lpci.wedontsleep.org/job/devel/1061/testReport/junit/canonical.librarian.tests.test_db_outage/TestLibrarianDBOutage/test_outage/
<lifeless> ?
<wgrant> Oh, a new failure that I hadn't seen yet.
<StevenK> That's just the failure, the console log has a little more.
<wgrant> That's a brand new tests.
<wgrant> So it may actually be bad.
 * StevenK prods wgrant until he pushes a new allocated.txt
<wgrant> jml: I've tentatively given you 86-1.
<nigelb> ha
<lifeless> wgrant: if his patch is ready to roll, you might like to fix-and-land it as a courtesy. its $am there.
<wgrant> lifeless: I discussed it with him last week, and it was on ice.
<wgrant> Otherwise I would :)
#launchpad-dev 2011-09-13
<lifeless> cool
<StevenK> Rargh, the lack of tab completion for bzr mv is really really annoying
<wgrant> StevenK: It's pushed, anyway.
<StevenK> wgrant: I've already commited and pushed my revision, so like you said to me last night: Way ahead of you.
<wgrant> Heh
<StevenK> I wish lp-open worked with piped branches
<nigelb> fuuuu
<nigelb> test failuer.
<nigelb> and I don't even know whats going wrong.
<lifeless> well, what happened ? :)
<nigelb> http://paste.ubuntu.com/687980/
<StevenK> nigelb: You missed a bit
<StevenK> Effectively
<StevenK> column "statusexplanation" does not exist\nLINE 11:         SELECT statusexplanation FROM BugTask\n ...
<nigelb> FFFFFFFFFFFFFFFFUUUUUUUUUUUUUUU.
<nigelb> My devel landing missed something?
<StevenK> Looks like
<nigelb> *headdesk*
<StevenK> nigelb: How did you search for it?
<nigelb> grep inside lib/lp I think.
<lifeless> I see it in oops.py and oops_prune.py
<lifeless> grep lib, not lib/lp
<nigelb> Oooooh.
<nigelb> Sigh.
<StevenK> And look into bzr grep
<nigelb> Now I have to land something to devel
<lifeless> or . in fact, for complete coverage.
<StevenK> It is AWESOME
<nigelb> and then re-land
<lifeless> bzr grep ++
<StevenK> nigelb: lib/canonical/launchpad/scripts/ftests/test_oops_prune.py, lib/canonical/launchpad/scripts/oops.py and lib/lp/bugs/stories/bugs/xx-bug-activity.txt
<nigelb> the last one I think is fine.
<StevenK> nigelb: Since your earlier work has landed, this must be a seperate branch
<nigelb> another incremental landing for the same bug?
<StevenK> Yup
<nigelb> k
<StevenK> nigelb: You're learning, it happens
<nigelb> That doesn't mean I'm happy about it :)
<lifeless> StevenK: https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75093 has db and model code mixed - is that intentional ?
<StevenK> I missed the pre-req branch, I've resubmitted it
<lifeless> ah cool
<StevenK> Damn pipes :-P
<nigelb> Hrm, what do I do for this query? http://paste.ubuntu.com/687983/
<nigelb> remove the select and where?
<StevenK> Drop lines 10-12
<StevenK> UNION ALL
<StevenK>         SELECT statusexplanation FROM BugTask
<StevenK>         WHERE statusexplanation %(posix_oops_match)s
<nigelb> aha,okay.
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> Oh, excellent. Reviewer on duty earlier.
<nigelb> *earlier than I expected.
<LPCIBot> Project devel build #1,062: STILL FAILING in 1 hr 3 min: https://lpci.wedontsleep.org/job/devel/1062/
<wgrant> That is suboptimal.
<wgrant> StevenK, lifeless: Do we want to revert stub's rev, or expedite the pgbouncer fix and see if it helps?
<nigelb> wgrant: Was there thoughts of removing opinion?
<nigelb> (I should have asked "Whats your opinion on removing opinion")
<wgrant> It should never have existed.
<poolie> +1
<wgrant> It was meant to be removed once the experiment failed, but that never happened.
<nigelb> Excellent. Shall I proceed to destory that as well?
<wgrant> Needs discussion.
<wgrant> And migration.
<wgrant> Which is why it shouldn't have been added.
<nigelb> A bug with status "Opinion" perhaps? :)
<wgrant> It was pointless, and fixing it is expensive.
<nigelb> Expensive in terms of downtime?
<wgrant> No.
<nigelb> Or in terms of plan effort.
<wgrant> That's easy.
<wgrant> Plan effort, mainly.
<wgrant> plain
<nigelb> *plain
<nigelb> Hrm. How hard?
<wgrant> The Bugs squad no longer exists.
<wgrant> So they aren't around to abort the experiment.
<wgrant> lifeless: Can we abort the experiment? :)
<nigelb> Lets declare Teal as bugs squad and declare it aborted? :P
<lifeless> last I heard there was a final signoff needed
<lifeless> which jml may or may not have done before he stepped down
<nigelb> \o/
<nigelb> That kind of effort.
<lifeless> I think its currently in limbo, but there isn't a lot of point rushing it when IssueTracker is just around the corner
<nigelb> IssueTracker?
<wgrant> ... he says as a feature is extended to 10 months
<lifeless> wgrant: whats this about pgbouncer?
<lifeless> http://ec2-204-236-255-184.compute-1.amazonaws.com/ is running my 0.0.5 upgrade atm
<wgrant> lifeless: With stub's librarian<->pgbouncer interaction tests, jenkins no longer works.
<wgrant> lifeless: It's unclear if buildbot works.
<wgrant> We'll find out soon,
<lifeless> wgrant: do they fail locally ?
<wgrant> But I now consider Jenkins failures to be fatal.
<wgrant> lifeless: https://lpci.wedontsleep.org/job/devel/1062/console
<wgrant> It passes locally.
<wallyworld_> lifeless: if you are able to +1 my db-devel mp in the next 1/2 hour, i can make the next bb run :-)
<wgrant> It needs to be sooner than that.
<wgrant> 15 minutes will be pushing it, even.
<wgrant> buildbot's estimates sort of suck.
<wallyworld_> ah
<wgrant> It would not surprise me if it finished in less than 5.
<StevenK> Haha
<wallyworld_> that's not nice :-(
<nigelb> Operation timed out.
<wgrant> wallyworld_: And it's done.
<wgrant> rofl
<wgrant> Finished 00:48:43
<wgrant> 10:43:43 < wgrant> It would not surprise me if it finished in less than 5.
<nigelb> How the..
<wallyworld_> wgrant = nostrodamous
<StevenK> buildbot's estimates suck.
<StevenK> Jenkins tend to work out the time much better.
<wallyworld_> i know that now :-(
<StevenK> And buildbot will "forget" the time if a build fails
<wallyworld_> i wish we could move to jenkins
<lifeless> bah. iwlagn --fail--
<lifeless> wgrant: do they fail locally ?
<wgrant> lifeless: No.
<wgrant> It'll be in buildbot in a couple of hours.
<lifeless> I can lp-land the 0.0.5 upgrade if you want
<StevenK> lifeless: Can haz db review for two branches so they can hopefully be deployed tonight as part of FDT?
<wgrant> lifeless: That might be handy.
<wgrant> lifeless: It looks safe to me...
<lifeless> StevenK: urls ? [note that we only do one change per fdt. So you can't get in anyhow]
<wgrant> lifeless: We have a week of patches queued already.
<StevenK> https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/+merge/75094 is mine
<wgrant> And we deployed 7 on Friday without trouble.
<lifeless> I know we did.
<flacoste> wallyworld_: RT #43352
<_mup_> Bug #43352: ipp jobs not purged; purging causes 100% cpu usage <cupsys (Ubuntu):Fix Released> <gnome-cups-manager (Ubuntu):Invalid> < https://launchpad.net/bugs/43352 >
<wgrant> StevenK: Anyway, you're too late for today.
<lifeless> ok, lp-land on its way
<wgrant> StevenK: buildbot has started already.
<wgrant> Oh!
<wgrant> No it hasn't.
<lifeless> wgrant: the point of one patch at a time is recoverability and diagnosis. Risk mitigation.
<StevenK> Indeed
<lifeless> wgrant: its nice that folk are keen.
<wgrant> Because lucid_lp was broken, there was no merge.
<wgrant> wallyworld_: You can still get your patch in if you want.
<lifeless> wgrant: but, doing batching won't make this -better-
<StevenK> lifeless: This is half joking -- but we want to go faster and you're trying to slow us down.
<lifeless> StevenK: those are not FDT safe.
<StevenK> They aren't?
<lifeless> StevenK: they are hot changes.
<lifeless> StevenK: I want to make change *safe* and *frequent*
<StevenK> So I can just ask stub to add the indexes for me?
<lifeless> yes
<StevenK> Right
<wallyworld_> flacoste: ah thanks
<lifeless> can't be done during a backup or other long transaction
<StevenK> I'll talk to stub when he surfaces
<StevenK> Now to see if I can reorder this pipeline
<nigelb> StevenK: He surfaced a few hours back.
<wgrant> He was probably just still up.
<lifeless> StevenK: anyhow, as the guy that got buyin and resourcing for fast downtime, I think I can can say I've made a huge improvement to 'going faster' :)
<wgrant> Given he appeared right after FDT last night.
<nigelb> He said "Cause I woke up at 3pm of course. What a silly question!"
<wgrant> Haha
<StevenK> Hm, can't reorder pipes
<wgrant> lifeless: Thanks, jenkins is building with your pgbouncer fix.
<wgrant> StevenK: You have to do it manually.
<wgrant> Edit the branch.conf of each.
<StevenK> I'll just delete the pipe
<nigelb> wgrant: Can haz review? https://code.launchpad.net/~nigelbabu/launchpad/the-return-of-destory-statusexplanation-88545/+merge/75099
<StevenK> Hahahaha
<StevenK> destory
<nigelb> :D
<wgrant> nigelb: Hm, how did it land with those references still there?
<wgrant> Oh, right.
<wgrant> I understand now.
<nigelb> Entirely different places.
<nigelb> wgrant: err needs the incremental flag
<wgrant> nigelb: I see you fixed some lint... do you feel like fixing the last three?
<wgrant> (two long lines, one unused variable)
<nigelb> one unused variable is false positive
<nigelb> its used in SQL query
<wgrant> Ah, and it uses local() to get it in?
<wgrant> *locals()
<nigelb> vars()
<wgrant> Ew
<nigelb> want me to change that?
<wgrant> Let me see.
<wgrant> Since that's the only var, yeah.
<wgrant> Might as well fix it.
<wgrant> Also fix the two long lines while you're there.
<wgrant> One of which is the regex.
<wgrant> This stuff doesn't get touched frequently, so it's nice to get rid of the lint when it is.
<nigelb> close the quotes, use \, and start quotes in new line?
<wgrant> I'd use parens instead of \
<wgrant> foo = (
<wgrant>     "barfewfwefwQ"
<wgrant>     "efwfwefwfw")
<nigelb> okay.
<StevenK> Perl! s///x
<StevenK>  /x allows comments and whitespace *inside* regexs
<nigelb> http://xkcd.com/208/
<StevenK> Yes, yes.
<StevenK> Go way, I know regular expressions.
<nigelb> s/way/away/g
<nigelb> But,yeah.
<StevenK> g not needed, you're only replacing once
<StevenK> :-P
<nigelb> Dammit.
<StevenK> wgrant: If you're free, https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75100
<wgrant> StevenK: Doesn't that depend on the hot index?
<wgrant> You say it can land anyway, but doesn't it include the patch?
<StevenK> It doesn't any more
<wgrant> Ah, no prereq any more.
<wgrant> icy.
<StevenK> And it doesn't need the index, no, it just adds the columns to the model
<wgrant> Yeah, but I thought that was the branch I saw earlier with the DB patch.
<wgrant> That you said needed a prereq.
<StevenK> Yes, I was fighting with pipes
<StevenK> And then lifeless said the patch is unsuitable and so I resubmitted the MP again
<StevenK> "This time, for reals"
<lifeless> StevenK: erm, its not a FDT patch; it was fine in all other regards.
<StevenK> What other kind of patch do we have!?
<nigelb> wgrant: suggestions on "description='https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1Foo666'"
<lifeless> StevenK: hot db patches and code patches.
<nigelb> its inside the sql query
<wgrant> nigelb:
<wgrant> description=
<wgrant>     'https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1Foo666'
<wgrant> Is that sufficient?
<wgrant> Otherwise use || to concat.
<lifeless> StevenK: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Hot_Patches
<nigelb> wgrant: Is this fine? http://paste.ubuntu.com/688008/
<G> StevenK: I'm not needed? ;)  (really wish awaylog gave context)
<wgrant> nigelb: If that's short enough, yep.
<lifeless> StevenK: and https://wiki.canonical.com/Launchpad/PolicyandProcess/ProductionChange#Hot_database_patches
<nigelb> wgrant: Updated MP.
<StevenK> lifeless: That's what I said -- I was going to talk to stub about the indexes when he surfaces
<lifeless> right
<lifeless> StevenK: you still need to land the patch though
<lifeless> StevenK: I guess all I'm saying is that your patch wasn't *bad*, it just *wasn't FDT relevant*
<wgrant> nigelb: You are now lint-clean?
<nigelb> Yes
<nigelb> Pasted on MP description as well.
<lifeless> StevenK: and I suspect I gave the wrong impression before
<lifeless> StevenK: so I want to fix that up
<nigelb> What the... every other lint I read has barry as author/co-author.
<nigelb> s/lint/PEP
<wgrant> lint, PEP, same thing :P
<nigelb> lol
<lifeless> he is the flufl
<wgrant> But yeah, barry is fairly Python.
<nigelb> heh
<StevenK> lifeless: I didn't say bad, I said unsuitable
<wgrant> nigelb: Looks good, thanks.
<lifeless> StevenK: ok, if we're copacetic, cool. Sorry for the sidebar :)
<wgrant> nigelb: Shall I land it?
<nigelb> wgrant: yes, please :)
<StevenK> wgrant: With --incremental
<wgrant> Indeed.
<nigelb> I only noticed last week that Barry co-authored PEP8. I've read that a zillion times and never noticed!
<nigelb> Ok, now I can go sleep without something left to do.
<wgrant> Night nigelb.
<nigelb> Its 7 am, but yeah. Laters.
<lifeless> nigelb: uhm. Sustainable patches please ;)
<nigelb> lifeless: Huh?
<lifeless> nigelb: You Need To Sleep.
<nigelb> What did I write unsustainable?
<wgrant> nigelb: If you're not gone yet, care to set a commit message?
<nigelb> Setting.
<wallyworld_> lifeless: sorry to nag - if you are able to +1 my db-devel mp i can land it before something else triggers bb :-)
<nigelb> lifeless: I was up till 4 for work, then I got test failure email, and then sat and fixed :)
<lifeless> wallyworld_: I prefer to let stub review them all as much as possible
<wallyworld_> ok. np
<nigelb> wgrant: What's the official word on that? Some reviewers like me setting the commit message, otheres do it themselves.
<lifeless> wallyworld_: your patch will be several days away anyway, due to folk dogpiling in so excitedly ;)
<lifeless> wallyworld_: which is nice.
<wgrant> nigelb: They're just being nice.
<wgrant> nigelb: Generally the author sets it.
<nigelb> Oh. /me notes for future.
<lifeless> wallyworld_: and may make the one-patch-per policy get revisited.
<nigelb> I'm in queue for a db patch as well :D
<wallyworld_> lifeless: you you are enforcing that atm
<wallyworld_> sounds a bit rigid
<lifeless> wallyworld_: FTR - https://dev.launchpad.net/Database/LivePatching
<nigelb> (they landed 7 patches the other day. wgrant was plotting 3 tomorrow :P)
<nigelb> Or was it 2)
<StevenK> They can't land in the same run
<lifeless> wallyworld_: bah, wrong link
<G> nigelb: from reading the ML wasn't the 7 an exception to try and catch up?
<wgrant> nigelb: Instance is starting.
<wgrant> nigelb: Now go away.
<wgrant> :)
<lifeless> wallyworld_: on https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Making_a_database_patch it has discussion about adding columns and performance
 * nigelb goes away.
<lifeless> wallyworld_: the reason for one at a time is to avoid multiple zomg's.
<lifeless> wallyworld_: db schema changes are -the- most risky routine thing we can do.
<wallyworld_> sure, but if there are a few totally orthogonal changes....
<wallyworld_> these could be done together
<lifeless> they could
<lifeless> if we want to risk multiple disparate fallouts all at once.
<wgrant> Otherwise people will just conspire to do more things in one patch.
<wgrant> eg. I'm dropping 36 tables in one patch instead of 36.
<lifeless> which is fine
<lifeless> its one conceptual thing.
<StevenK> wgrant: Otherwise we'd need to jump to 2209 :-P
<lifeless> folk aren't going to be silly.
<lifeless> there is precisely one DBA.
<wgrant> lifeless: Right, they're going to be sensible and work around a silly restriction :P
<lifeless> speaking bluntly, if we have a shitstorm, there is only one person that will be fixing it. I don't want us to -ever- have 2 shitstorms and one person working on them.
<wallyworld_> lifeless: btw, i did read that wiki page on adding columns but then saw previous patches adding not null columns and figured it must have been a "soft" policy
<lifeless> wallyworld_: it is soft 'Similarly, new columns must default NULL unless the data size is extraordinarily small. '
<lifeless> wallyworld_: Branch is not extraordinarily small :)
<lifeless> wgrant: if folk batch inappropriately, I will veto their changes.
<lifeless> wgrant: and stomp very hard.
<wgrant> Bah.
<lifeless> wgrant: there is a fundamental cultural change we *will* make.
<wallyworld_> fair point. i can't recall the example i saw, but i could have sworn it was for a lrgish table
<lifeless> wallyworld_: if it was more than 6 weeks ago it was before FDT.
<wallyworld_> ah that makes sense
<lifeless> wgrant: the payoff for this change is a more reliable process and that means we can execute more often.
<lifeless> wgrant: we have a -long- way to go to prove that we can be trusted to execute changes rapidly. Until we have things that make it more risky, are well, silly risks to take.
<lifeless> wgrant: before FDT we were doing *6* schema changes a month.
<lifeless> wgrant: even with 2-step changes, 12 a month is well below the capacity we could do if we had to and still be doing one-per.
<lifeless> wgrant: so arguing that batching is important right now seems specious and to be ignoring the very real risk tradeoffs that the new process is designed around.
<wgrant> Perhaps.
<lifeless> can you make an affirmative argument for why we should accept the increased risk that batching brings *right now*
<wgrant> I don't believe there is significant credible risk in batching some kinds of patches.
<lifeless> how good would you say our risk assessment for schema patches is?
<wgrant> In terms of performance? Terrible. In terms of blowing up? Pretty good.
<lifeless> like, if we bet 1000 dollars on the patch being sideeffect free, per patch, how often would we payout [have a patch with undesirable side effects] ?
<wgrant> When have we had problems recently, apart from bugsummary fun?
<lifeless> If we deploy something that causes grief, we need to roll it back at the next slot - or even sooner if its really bad. That means a pipeline stall on *all* db patches [so we're not compounding the error]
<lifeless> wgrant: we've blown up a number of collections with indices that caused sideeffects in other queries
<wgrant> True.
<lifeless> add to this that we have a number of 24 hour cycles
<lifeless> oops reports
<lifeless> api scripts in cron
<lifeless> its hard to tell if we've borked something without giving it some time to settle in.
<lifeless> we need to fix all of this stuff
<lifeless> drive the intervals down
<lifeless> get more responsive.
<lifeless> everything we batch gets in the way of driving the lead-time lower
<lifeless> so, we're not batching here. Unless a -really really really- compelling argument that we *want* to batch, in everything, is made.
<wgrant> How about that we're not going to be driving down intervals for at least 18 months?
<wgrant> So we're going to have few opportunities for optimisation until then.
<wgrant> (unless you do it all single-handedly)
<wgrant> We are at 263 criticals, and no crisis has been declared.
<wgrant> We are not going to be improving anything apart from criticals for a *very* *long* *time*.
<lifeless> francis is analysing that
<lifeless> fdt is one example of driving down intervals
<wgrant> Right, but I think it needs more of a declaration of crisis than an analysis.
<lifeless> So I wouldn't say single handed driving things down.
<wgrant> FDT wasn't single-handed? It was entirely Technical Architecture.
<lifeless> But all the SOA stuff - oops with rabbit etc - are all dovetailing into more responsiveness in our systems
<wgrant> Single-handed.
<lifeless> red are working on rabbit
<wgrant> lol
<lifeless> thats support for this
<lifeless> we have the test suite stuff enqueue for this year
<wgrant> hahaahhahha
<lifeless> 30m test runs or thereabouts
<wgrant> rofl
<lifeless> okok, 60m.
<wgrant> lol
<wgrant> I am impressed with your vision for these improvements.
<wgrant> But I present exhibit A.
<wgrant> http://webnumbr.com/launchpad-critical-bugs
<wgrant> Everyone seems to be ignoring the fact that we are probably never going to exhaust the queue.
<wgrant> All progress is predicated on that happening.
<wgrant> It's been clear for at least a month, and probably 3.5 months, that a crisis needed to be declared.
<poolie> wgrant, so do you think there should be less feature work and more critical bug work, or ...?
<wgrant> I don't know.
<wgrant> Probably.
<wgrant> But the current situation is dire.
<wgrant> And it is steadily getting even worse.
<wgrant> What we're doing now is not working.
<wgrant> At. All.
<lifeless> so
<lifeless> the trend is a problem
<wgrant> Even more amusingly, the squad that was unassigned at exactly the time the downward trend stopped will remain on feature work until probably April.
<lifeless> thats why francis is analysing to determine the source of the criticals.
<lifeless> Are we breaking things too often?
<lifeless> are we still finding debt that meets the policy rules?
<lifeless> etc.
<lifeless> until we have that breakdown, there isn't any change that is sensible to take - we need data.
<lifeless> this is a crucial thing for us.
<wgrant> Well, when are we likely to have results?
<lifeless> I don't know.
<lifeless> I forgot to ask him today.
<lifeless> as an example though - if the problem is that we're creating more criticals.
<wgrant> We have more than 25% more criticals than when the problem first becams eclear.
<lifeless> more folk working on criticals might just increase the rate at which we break things.
<G> most of the criticals seem to be timeout/db related right? (just my observation, don't know if I'm right or not)
<wgrant> All is lost, so I am going to have lunch.
<wgrant> lifeless: your pgbouncer fix worked, btw.
<lifeless> heh
<lifeless> thats good
<lifeless> wgrant: care to note that in one of the bugs? (so we learn from our mistakes ;P)
<wgrant> lifeless: Commented on bug #848400
<_mup_> Bug #848400: fixture shares fd's with testrunner process, will cause test hangs and worse. <Python PGBouncer:Fix Released by lifeless> < https://launchpad.net/bugs/848400 >
<lifeless> jamesh: around ?
<jamesh> hi
<lifeless> hi
<lifeless> got time to talk storm ?
<lifeless> jamu and I weren't sure if we needed to rollback my landing
<jamesh> sure.
<lifeless> or to tweak, etc
<jamesh> I don't think it is worth rolling back: it is new code that isn't being used anywhere, so we may as well build on top of it
<lifeless> I replied to your review
<lifeless> (and thanks for that)
<jamesh> so, paramstyle was a reference to the DB-API spec: "pyformat" is the style used by psycopg2 with "%s" as a marker
<lifeless> ah
<jamesh> and qmark is the style used by sqlite with "?" as a marker
<lifeless> I haven't read that spec
<lifeless> how is something like "column LIKE '%s' AND column2=%s" handled where the LIKE clause is actually a suffix search, not a substitution
<jamesh> the degenerate case that would cause problems would be something like connection.execute(u"SELECT ? = '%s'", (u"foo",))
<jamesh> on a backend using the qmark paramstyle, the interpolation would fail in the tracer.  Of course, it would fail earlier on a pyformat paramstyle backend though.
<lifeless> one way to handle it would be a try:except: around the interpolation with a fallback of outputting something more like debug does.
<lifeless> or just the plain statement.
<jamesh> fyi: you might find the startswith(), endswith() and contains_string() sql expression methods in Storm handle most of the LIKE clauses you have pretty well
<lifeless> jamesh: I think we use them in various places
<lifeless> do they compile to "column LIKE %s" , (u"%suffix",) ?
<lifeless> more-or-less
<jamesh> for the quoted '%' case for a LIKE expression, the if you're using a pyformat paramstyle backend, you'd usually have to escape the percent sign (one of the reasons some people dislike pyformat)
<lifeless> \%s ?
<jamesh> %s
<jamesh> gar.  "%s"
<jamesh> xchat seems to be eating the double percent sign :(
<lifeless> hahaha
<lifeless>  %%s ?
<jamesh> yes.
<lifeless> so, what should we test ?
<jamesh> Storm's startswith(), etc methods change the glob characters to ones that don't look like a pyformat interpolation sequence
<jamesh> If you have an unescaped "%" character, I don't think psycopg will be particularly happy anyway.
<lifeless> I'm not aware enough of storms needs to predict the missing tests / changes.
<lifeless> what I copied over is working live in LP with the pg backend.
<jamesh> >>> cursor.execute("SELECT %s = '%'", ('%',))
<jamesh> Traceback (most recent call last):
<jamesh>   File "<stdin>", line 1, in <module>
<jamesh> IndexError: tuple index out of range
<lifeless> if I understand correctly you're saying there isn't an issue
<lifeless> because something that would break the tracer will break pscyopg too
<jamesh> there probably isn't an issue for psycopg2 and other pyformat backends
<jamesh> people could pass queries through that will cause problems on qmark backends
<lifeless> because we don't escape existing %s's on qmark lookups ?
<jamesh> right.  At the point where the tracer is running, the query will be in the adapter's preferred format, so we wouldn't expect '%' interpolation at that point
<lifeless> jamesh: but the quoted_statement re sub should fix that
<jamesh> I guess the code should look something like: (a) if connection.param_mark == '?', double all percent signs and convert param marks to '%s'
<jamesh> (b) if param_mark is '%s', don't alter the query string at all
<jamesh> if param_mark is something else again, panic :)
<lifeless> ah, we preserve %s's
<lifeless> mmm
<lifeless> how about
<lifeless> if param_mark is not %s: double all percent signs, otherwise double all non-%s % signs.
<lifeless> then convert marks to %s
<lifeless> done
<StevenK> wgrant: Please take back your "Lies." comment: http://pastebin.ubuntu.com/688053/
<jamesh> I don't think you should need to do anything to the '%' signs if param_mark is '%s'
<lifeless> ian broke LP with that assumption.
<lifeless> it was a fun day of broken buildbot.
<jamesh> if the non-interpolated '%' signs haven't been escaped, then they will fail when passed to the database adapter anyway
<lifeless> apparently we have something somewhere that manages this
<lifeless> ENOIDEAWHERE.
<wgrant> StevenK: Oh?
 * StevenK waits for wgrant to read the pastebin
<wgrant> I read the pastebin.
<wgrant> I see a SourcePackageName declared as an integer.
<lifeless> jamesh: what about the other thing
<lifeless> jamesh: the control api
<jamesh> lifeless: the part I cringed at was exposing the threading.local() object as part of the public API to the tracer
<StevenK> wgrant: ALTER TABLE SourcePackagePublishingHistory ADD COLUMN sourcepackagename INTEGER REFERENCES SourcePackageName
<lifeless> jamesh: yes, I explained the things that led me to doing that.
<wgrant> StevenK: I'm not sure what the DB has to do with this.
<lifeless> jamesh: and a possible alternative, but it seems rather like yagni to me. I'm willing to do it if needed
<wgrant> StevenK: SourcePackagePublishingHistory.sourcepackagename is not an integer, so why is ISourcePackagePublishingHistory['sourcepackagename'] an Int?
<StevenK> wgrant: Oh, I've declared it wrong in the model?
<wgrant> In the interface.
<wgrant> The code I quoted!
<StevenK> OH
<StevenK> Attribute() and Int() are the wrong way around.
<wgrant> You know, the only two types in the code I quoted :)
<StevenK> Sorry, I read it as "You have sourcepackagename in BPPH and vice versa"
<lifeless> jamesh: http://paste.ubuntu.com/688059/ for the quoting thing.
<jamesh> lifeless: the timeline_factory argument or an abstract get_timeline() method method would be my preferred APIs.  Do you have example code of how you'd use the class in Launchpad as it is currently written though?
<lifeless> jamesh: adapter.set_request_started
<lifeless> jamesh: the version of this tracer in LP knows about the 'current request', so it calls 'get_request_timeline'
<lifeless> jamesh: which could drop straight into a timeline_factory parameter.
<jamesh> lifeless: re. the paste, I still think you should be able to get by without munging the statement in the connection.param_mark == '%s' case
<lifeless> jamesh: you can't because "%%" % () -> "%"
<lifeless> jamesh: but we want "%%" to be shown as  "%%"
<jamesh> lifeless: why?
<lifeless> jamesh: the munging is to *preserve* the %'s.
<jamesh> lifeless: they aren't preserved in the statement executed by the DB
<lifeless> they match what folk see when they look at the code though, which was a strong consideration.
<jamesh> hmm
<lifeless> I could go either way.
<lifeless> I'm convinced its a presentation thing.
<lifeless> it also happens to ensure its safe to format, but thats incidental.
<jamesh> my point earlier was that if it isn't safe to format, then it is going to fail when passed to the adapter anyway ...
<lifeless> jamesh: so I changed to the thread locals() because it seemed to me that that is the common need
<lifeless> jamesh: right, I got that, but wouldn't it be better fo rhte adapter to be the thing raising the error ?
<lifeless> jamesh: lets users avoid confusion about what is wrong
<jamesh> lifeless: if that is a concern, I would consider including a code path that handles errors from the interpolation step.
<lifeless> jamesh: if you prefer a factory, I'll change it to want a factory, that happens to fit closer to LP because it already *has* such a factory; it will mean more glue in the wsgi case though.
<lifeless> mm
<lifeless> actually it has a factory which needs a guard because of other bugs.
<lifeless> but meh, its shallow either way.
<jamesh> here's how psycopg2 handles the percent sign:
<jamesh> >>> cursor.mogrify("SELECT %s LIKE 'foo%'", [u'foobar'])
<jamesh> "SELECT E'foobar' LIKE 'foo%'"
<jamesh> gar.  imagine there is a double percent sign in the LIKE clause
<lifeless> on the >>> or the " starting lines ?
<jamesh> the first line
<lifeless> right
<jamesh> mogrify() is a psycopg2 specific API for invoking its interpolation code
<lifeless> jamesh: ok, so what now? I'm getting lost ;)
<jamesh> lifeless: okay.  I think your paste looks like a good improvement, but (1) I'd prefer not to munge the statement in the pyformat backend case and (2) I think the actual interpolation stage should check for TypeError and use some other place holder string in that case.
<jtv> wgrant: see anything new in debian domination?  Our challenge for today is to get it to re-publish an SPR.
<wgrant> jtv: It looks OK, and we know that it handles the republishing case OK from my archivepublisher testing.
<wgrant> We could retest that with gina if we really want to, though.
<lifeless> jamesh: do we need to defend against unreprable variables ?
<lifeless> jamesh: to-database shold be sane, no ?
<jamesh> lifeless: I think we can depend on to_database() being sane: it gets called outside of the tracer in this code path anyway.
<jtv> wgrant: hmmâ¦  intra-release domination is the same thing for gina domination and publisher domination, so yes, we can take that as validation.
<wgrant> jtv: So, any reason not to deploy it and watch the world burn?
<jtv> The world won't burn.  For that, I need to add that commit first.
<wgrant> True.
<jamesh> not sure about repr().  The existing DebugTracer grabs the repr() of the values and hasn't been a pain point so far (although it isn't using to_database())
<jamesh> so it might be worth leaving it as is, and see how it works
<wgrant> jtv: Shall we deploy up to 13913, then?
<jtv> Let me look that up.
<wgrant> Hmm, we should really get past 13917 today.
<wgrant> But 13914 is a feature-flagged blocker.
<poolie> lifeless, can we have another go at bug 660264 now haproxy is live?
<lifeless> jamesh: typeerror - http://paste.ubuntu.com/688065/
<_mup_> Bug #660264: bzr+ssh on launchpad should fork, not exec <lp-code> <Launchpad itself:In Progress by jameinel> < https://launchpad.net/bugs/660264 >
<lifeless> jamesh: I'd like to leave the %% handling on postgresql alone for now
<lifeless> jamesh: get everything migrated and consolidated, then revisit.
<jamesh> lifeless: fair enough.  The second paste looks good.
<lifeless> poolie: no, we need those two bugs I escalated with you addressed first.
<poolie> smooth reconnect etc
<poolie> ok
<jtv> wgrant: 13913 is doable for me, although obviously I'd like to get that commit in with the rest.  Can we treat cocoplum as if it were NDT now?
<lifeless> poolie: haproxy is only half-live, we're not actually in nodowntime mode yet.
<lifeless> poolie: yes, those.
<wgrant> jtv: gina isn't on cocoplum
<lifeless> jamesh: so, you're really against the threadinfo thing ? [/me asks with a vague hope :P] - it just seems like -all- the factories folk will write will be identical.
<jtv> wgrant: where then?  And is it NDT?
<wgrant> jtv: iron, and yes.
 * jtv shuffles cards
 * jtv eagerly waits for wgrant's message notification bubble to get out of the crucial spot where he needs to see.
<jtv> Bubble bursts.
<wgrant> jtv: what was that?
<wgrant> jtv: I didn't quite hear you
<wgrant> jtv: Could you please repeat?
<wgrant> jtv: Please?
<jtv> That's okay, I no longer need to see that precise spot.
<jtv> But it's gratifying to see you go to all that wasted effort.
<wgrant> Curses.
<jtv> wgrant: what's that?
<jtv> I didn't quite hear you
<poolie> what does 'half-live' mean?
<poolie> it's in place but it doesn't fail over?
<lifeless> the project isn't complete
<lifeless> (because we can't deploy to it regularly, which was the goal)
<lifeless> meep
<lifeless> thats a worry
<lifeless>     Hard / Soft  Page ID
<lifeless>      902 /  227  ScopedCollection:CollectionResource:#message-page-resource
<lifeless>      870 /   21  Product:+bugtarget-portlet-bugfilters-stats
<lifeless>      820 /  158  BugTask:+index
<lifeless>      766 /  240  Distribution:EntryResource:searchTasks
<lifeless>      291 /    0  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>      288 /    0  Person:EntryResource:searchTasks
<lifeless>      211 /   24  MaloneApplication:+bugs
<wgrant> lifeless: Indeed. I've wondered if the journal rollup isn't happening or something.
<wgrant> lifeless: I saw a new garbo job fly past this morning.
<lifeless> jamesh: http://paste.ubuntu.com/688073/
<lifeless>  select count(*) from bugsummaryjournal;
<lifeless>  count
<lifeless> -------
<lifeless>    875
<lifeless> wgrant: ^
<StevenK> wgrant: Have another look at https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-model/+merge/75100 ?
<wgrant> lifeless: Well, there goes that :/
<wgrant> Hmm.
<wgrant> What could it be, then.
<wgrant> StevenK: What about the other places that create [SB]PPH?
<wgrant> StevenK: I think there are some more.
<wgrant> StevenK: Although I tried to unify most of them, some slipped through.
<wgrant> Also, readonly=False?
<wgrant> While it's not exposed through the API now/yet, that seems like a bad start :)
<jamesh> lifeless: that does look like a better API to me.  If we do find everyone is passing the same timeline_factory implementation in later on, perhaps we can revisit it.
<lifeless> https://code.launchpad.net/~lifeless/storm/timelinetracer/+merge/75109
<StevenK> wgrant: I thought it was only new{Source,Binary}Publication :-(
<wgrant> StevenK: That's the aim.
<wgrant> StevenK: But it's not there yet.
<wgrant> StevenK: Also, at least publishBinaries has a batch interface that doesn't involve instantiating the class.
<lifeless> jamesh: ^
<jamesh> lifeless: saw it.  I'll review it in a sec.
<lifeless> thanks
<wallyworld_> lifeless: is there a way in storm to hook into an object lifecycle such that i can access any dirty properties' old and new values when a db update is done?
<lifeless> uhm, probably at least two. Why?
<wallyworld_> lifeless: i want to trigger a db update if someone changes a particular property and then saves the object
<lifeless> wouldn't a regular property be sufficient ?
<wallyworld_> i could hook into an IObjectModifiedEvent i suppose, but i'd rather do it with just storm
<lifeless> huh?
<lifeless> just foo = property(...)
<wallyworld_> that would work. i was wanting to not do any db work until the flush. but i could set a flag and hook into the storm flush event
<lifeless> perhaps some specifics would help the discussion
<lifeless> I'm struggling to understand the constraints
<StevenK> wallyworld_: Er?
<StevenK> wallyworld_: Isn't that what triggers are for?
<wallyworld_> when a branch has it's stacked_on branch changed or it's private property changed, i need to update the branch table to set transitively_private for other branches that may be affected
<lifeless> that sounds like a trigger case to me
<wallyworld_> ok. for some reason i thought we preferred to avoid trigegrs
<lifeless> running sql *just-before-a-flush* is going to be -fuuuun-
<lifeless> wallyworld_: I loath and fear triggers, but they are the right solution to this sort of thing while we use pgsql.
<wallyworld_> i share your view, hence trying to finf a non-trigger solution
<lifeless> wallyworld_: if we were using e.g. cassandra, the whole way the problem is approach would be different
<wallyworld_> any pointers to where we define and set up triggers for other things?
<lifeless> trusted.sql
<lifeless> though that really wants to DIAF
<lifeless> I would just do it as a regular patch I think, at least to start with.
<G> wgrant: in my efforts to pick off some of the trivial fixes so you guys can focus on the criticals... with reference to bug 763820, would the text "with the Ubuntu keyserver" be better than "with an Ubuntu keyserver" iirc there is only one right?
<_mup_> Bug #763820: double "with" on +editpgpkeys <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/763820 >
<wgrant> G: Indeed, that sounds better.
<wallyworld_> lifeless: the patch needs to be in db-devel? and have it's own patch number and all that?
<lifeless> wallyworld_: yes, most definitely.
<G> not to mention, it should've been a not an, but thats by the point since you agree with the :)
<wallyworld_> cool. thanks
<wgrant> G: "an Ubuntu" is correct, not "a Ubuntu"...
<G> wgrant: you don't say "an United Nations <something>" you say "a United Nations <something>" because it flows better (just looked it up)
<wgrant> G: Sure.
<wgrant> But the U is not the same.
<wgrant> oo-boon-too
<wgrant> Not yoo-boon-too
<poolie> i bet some people do say "an United"
<poolie> maybe not incorrectly
 * wgrant blames the US
<wgrant> It is incorrect, I'm pretty sure.
<G> yep, lets just blame the US for everything :)
<StevenK> RARGH!
<StevenK> CONFLICTS
<StevenK> HULK SMASH
<wgrant> You conflicted with stub's new garbo?
<StevenK> No, this is still the model changes
<G> wgrant: you may have a point (w/ pronounciation)
<StevenK> But I'll probably have to cope with that too
 * StevenK gets a rope.
<StevenK> for i in $(find lib/canonical/launchpad/mail | tac) ; do bzr resolve $i ; done
<StevenK> And that solves one problem
<G> wow `make run` is much faster now
 * StevenK waits for bzr pump
<StevenK> wgrant: And again?
<wgrant> Let's see.
<G> wgrant: there you go, MR for you https://code.launchpad.net/~dev-nigelj/launchpad/bug-763820/+merge/75111 has to be the most trivial in a while
<StevenK> G: *MP*
<StevenK> Merge *Proposal*
<G> StevenK: I realized that after I said it
<G> StevenK: my mind is about 10 miles away from my body atm :)
<StevenK> Up, down, north, south, west or east?
<wgrant> StevenK: You didn't get publishBinaries?
<StevenK> wgrant: I did a grep
<G> StevenK: somewhat South East
<StevenK> bzr grep "BinaryPackagePublishingHistory\(" | grep -v makeBinaryPackagePublishingHistory
<wgrant> StevenK: Like I said, it does a bulk inset.
<wgrant> insert
<StevenK> Rargh, annoying
<wgrant> Rather, but it's the only way to not have terrible performance.
 * StevenK waits for sync-pipeline some more
<StevenK> G: By the way, if you haven't used bzr grep, you should -- it is love.
<G> StevenK: what is bzr grep when you meet it in the street?
<G> (still new to bzr, but I like it)
<StevenK> G: It runs grep over versioned files
<StevenK> G: It's useful for the Launchpad tree, since we have that whole annoying lib/lp versus lib/canonical split
<G> ahhh I just normally do "grep -r <something> lib/lp" :)
<poolie> bzr grep is faster, ignores non-versioned files, can search history
<StevenK> G: If you search just lib/lp, you will miss stuff. As nigelb found out this morning.
<StevenK> G: Hence my service announcement. :-)
<StevenK> bzr grep doesn't support -c :-(
<G> poolie: you got me at faster ;)
<G> where do I get it from? ;)
<StevenK> lp:bzr-grep
<G> should've guessed :)
<wgrant> Or apt-get install bzr-grep
<wgrant> On >= maverick, IIRC
<StevenK> cd .bazaar/plugins && bzr branch lp:bzr-grep grep
<G> wgrant: thanks
<StevenK> Since bzr can't deal with plugin directories that contain a -
 * G prefers packaged stuff :)
<poolie> not packaged?
 * StevenK prods wgrant for more review love.
<G> poolie: it's packaged for Natty, because after wgrant's pointer I just aptitude install'ed it :)
<wgrant> StevenK: What about gina for SPPH? Or does it use newSourcePublication?
<StevenK> wgrant: It uses newSP
<wgrant> Approvalised.
<wgrant> G: Also approved. Want me to land it?
<G> wgrant: might as well, thanks :)
<wgrant> Could you set a commit message on the MP, please?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #1,063: FIXED in 4 hr 10 min: https://lpci.wedontsleep.org/job/devel/1063/
<StevenK> steven@liquified:~/launchpad/lp-branches/no-more-lazr-utils% for i in $(bzr grep -l canonical.lazr.utils) ; do sed 's/canonical.lazr.utils/lazr.restful.utils/' < $i | sponge $i ; done
<StevenK> steven@liquified:~/launchpad/lp-branches/no-more-lazr-utils% bzr di | wc -l
<StevenK> 745
 * StevenK hides
<G> wgrant: set
<wgrant> Thanks.
<G> wgrant: at some point can you take a look at https://bugs.launchpad.net/launchpad/+bug/306569/comments/2 as well and give me your thoughts?  (I'm also now wondering if an AJAXy +help/foo page would also do the job instead of links to the wiki etc
<_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:In Progress by dev-nigelj> < https://launchpad.net/bugs/306569 >
<StevenK> wgrant: lazr imports are the same level as zope and so on right?
<wgrant> StevenK: Yes.
<mwhudson> StevenK: you might be interested in sed -i :-)
<StevenK> mwhudson: But then I can't use the awesomeness that is sponge!
<mwhudson> StevenK: admittedly, i had to look up what sponge does
<StevenK> Bah, MPs needs a link to bug link
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/no-more-lazr-utils/+merge/75117
<wgrant> StevenK: r=me
<poolie> psycopg2.OperationalError: fe_sendauth: no password supplied - halp?
 * ajmitch saw some mention of needing ipv6 address in pg_hba.conf for that
<wgrant> Yeah, see one of my recent posts to launchpad-dev
<wgrant> Or rerun launchpad-database-setup
<ajmitch>  host    all         all         ::1/128           trust
<poolie> thanks
<poolie> wgrant, how can i actually get your rabbitfixture fix active?
<poolie> just put a checkout of that branch on my path?
<wgrant> poolie: It should be merged now.. let me check.
<wgrant> 13906 Launchpad Patch Queue Manager	2011-09-09 [merge] [r=gary][bug=845658] Upgrade rabbitfixture to be oneiric-friendly
<wgrant> Dapper has too many architectures.
<lifeless> jamesh: ok, so now we need to glue this to wski.
<nigelb> Morning.
<jamesh> lifeless: yeah.  I wonder what the best way to slice things would be?  A WSGI middleware that simply introduced a timeline object into the environment, and another one that configured the storm tracer based on that timeline and updated oops.context might work
<jamesh> could do a single one if you'd never want to use the timeline module without oops and storm
<lifeless> I think I'd start with a single one and refactor if we change our mind
<lifeless> I can't recall if I've pulled the timeline formatting stuff out of LP yet or not
<lifeless> (it is what generates the db_statements key in the oops
<lifeless> I'll look at that on thursday.
<jamesh> yep.
<lifeless> where should this wsgi module live
<lifeless> its storm + oops + wsgi specific
<jamesh> so, do you stick the timeline in the oops context and rely on an on_create hook to format db_statements for the report?
<lifeless> yes
<lifeless> because the oops itself has to be bson serializable
<jamesh> right.  Was just wondering if you did the formatting prior to generating the report
<lifeless> nah, its lazy
<lifeless> so just-in-time, and no formatting if no report
<lifeless> I think the formatter can live in timeline itself
<lifeless> as it doesn't need to import oops by default
<lifeless> for packaging it can 'enhances: python-oops'
<jamesh> I wouldn't be opposed to making it a sub-package of storm, but it could also be its own module
<jamesh> [I assume you don't want this in python-oops-wsgi]
<lifeless> right, I don't :)
<lifeless> (because storm is the extending thing here - there can be many orms that need oops-wsgi glue, but only one oops-wsgi for now)
<wgrant> zomg, p-d-r didn't crash on DF
<rvba> wgrant: Hi, could you please update DF for me? (And BTW, thanks for the review, I'll address your concerns shortly)
<wgrant> rvba: Did you see I QAed your resolved difference thing?
<wgrant> (on qastaging)
<rvba> wgrant: yes I did.  Thank you for that.
<wgrant> I've got p-d-r running on DF now, so it'd be nice if I didn't have to update.
<wgrant> But if you do need it, I can.
<rvba> wgrant: no worries, I'll do that later today.
<wgrant> it should be done in ~1.5 hours.
<rvba> Okay.
<adeuring> good morning
<mrevell> Hello
<jml> wgrant: sure
<wgrant> jml:
<wgrant> Er.
<wgrant> Thanks.
<StevenK> Now part 2 of demolishment can land?
<wgrant> StevenK: Yup.
<rvba> StevenK: Hi. The confirmation overlay stuff has landed. I'm here to help if you need me for some dirty Javascript stuff ;)
<LPCIBot> Project devel build #1,064: FAILURE in 4 hr 9 min: https://lpci.wedontsleep.org/job/devel/1064/
<lifeless> wgrant: oh hai
<lifeless> wgrant: still reviewing ?
<wgrant> lifeless: If necessary.
<lifeless> wgrant: if so, gpgfixtures review request is up
<wgrant> With pleasure.
<wgrant> Is it?
<lifeless> I think I have enough now that on thursday I'll be integrating it with LP, and with gpgverifyd
<wgrant> I don't see it.
<wgrant> Oh.
<wgrant> *For* gpgfixtures?
<lifeless> yes
<wgrant> Not quite so exciting, but it'll do.
<wgrant> Still no MP diff...
<wgrant> Worrying.
<wgrant> But no scriptactivity whinging yet.
<StevenK> rvba: I've pushed my branch up: lp:~stevenk/launchpad/private-bug-unsubscribe-confirm
<StevenK> rvba: If you bzr di -r submit: you can see what I've done, and where I've tried to add ConfirmationOverlay -- a helpful hint for that bit would be most welcome.
<rvba> StevenK: looking.
<jml> .Hi
<jml> I'm trying to patch lazr.restfulclient
<jml> In the absence of hacking instructions, I'm running bootstrap then buildout
<jml> But buildout fails
<jml> Error: Couldn't find a distribution for 'lazr.restful==0.16.0'.
<wgrant> lifeless: assertNotEqual(None, [...])?
<wgrant> lifeless: Shouldn't it be assertIsNot?
<stub> allenap: In what circumstances will _loads get a byte string?
<stub> allenap: re: your Storm MP
<StevenK> rvba: It's 7:35pm here, so I'll be in and out
<allenap> stub: Not sure :) Just being cautious because of weirdness in simplejson/json.
<poolie> jml, wow good luck
<stub> allenap: Right. I suspect you only need to force it it the _dumps(). The _loads() worries me as you are hardcoding UTF-8, which is incorrect.
<jml> poolie: thanks.
<jml> *someone* has to do it :)
<rvba> StevenK: okay. I'll send you an email with comments ...
<poolie> i don't know about that, but i did run in to confusion caused by its pth file
<poolie> it seems hard to run it from source if you also have the package installed
<rvba> StevenK: but I'm surprised that you don't even pass a submit_fn function to ConfirmationOverlay.
<allenap> stub: From what I can remember, that's what json.loads() assumes given a byte string containing non-ASCII characters.
<lifeless> wgrant: thats a shrug thing
<lifeless> wgrant: they are equivalent
<allenap> stub: I'm happy to remove the _loads() stuff though; I only need the _dumps() fix.
<stub> allenap: If you do find byte strings being passed into _loads(), we need to know where they are coming from to confirm they are indeed UTF-8 (or what encoding, eg. if they are coming from a DB storing stuff in cp1252
<lifeless> wgrant: (in this case)
<rvba> StevenK: can you just give me an idea about how I can test this?
<lifeless> stub: evening
<lifeless> stub: Sorry about the shenanigans around pgbouncer
<stub> allenap: If the tests pass, I think that would be best. I'll mention this with the rest of the review.
<allenap> stub: So, perhaps there should be an assertion in there, that value is unicode?
<lifeless> jml: you need a buildout cache
<lifeless> jml: or to specify to buildout that it can run online
<lifeless> jml: there is a shared cache like the lp one, for lazr libraries.
<jml> lifeless: it was downloading all of the other things slowly enough that I thought it was online
<stub> allenap: Fine with me.
<lifeless> jml: hmm, perhaps it is then.
<allenap> stub: Cool, thanks.
<lifeless> jml: if so, is 0.16.0 still on pypi
<wgrant> jelmer: Anything special about your import-colocated-branches branch?
<stub> lifeless: No worries. I'd be interested in what jenkins saw. I doubt the stdout/stderr issue would be at fault, but perhaps the socket open caused problems in that environment?
<wgrant> jelmer: It is not making the MP diff generator happy.
<wgrant> At all.
<jml> lifeless: umm, where exactly?
<lifeless> stub: StevenK can hook you up with the hanging that happened
<stub> StevenK: You have a pastebin of anything interesting?
<lifeless> jml: http://pypi.python.org/pypi/lazr.restful
<jml> lifeless: I don't see any download links there
<lifeless> there is a link to lp's +download page there
<jml> lifeless: and buildout can figure out to download the tarball from there?
<lifeless> who knows
<lifeless> I don't :)
<jml> ok
<stub> Can we switch to jenkins yet? Supporting both buildbot and jenkins envs seems a pain as they seem to flake out in different ways.
<lifeless> stub: parallel test project may be a good time to switch
<stub> Any blockers apart from time?
<lifeless> I'm told IS are now actually running jenkins instances for other departments, so that hurdle is past.
<jelmer> wgrant: Hmm, not that I'm aware of
<lifeless> no blockers that I'm aware of; just need tuits to migrate processes across
<lifeless> stub: hey, so great work on fastdowntime
<stub> Yup. The second 90%
<wgrant> lifeless: It needs to run in the DC, and not on an unsupported cloud.
<gmb> wgrant: I'm going to assume that you're no longer OCRing.
<wgrant> jelmer: Hm, and now it's recovered.
<stub> wgrant: We can cannibalise the buildbot env if we want. Its just time and losa/dev coordination.
<gmb> (For the purposes of the /topic)
<wgrant> gmb: I was just about to remove myself.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<wgrant> Thanks.
<gmb> np
<stub> lifeless: Ta. Seems to be working as planned.
<wgrant> stub: http://www.postgresql.org/about/featuredetail/feature.213 looks handy.
<wgrant> stub: Is that what we needed for branchrevision?
<stub> wgrant: yes.
<StevenK> rvba: The easiest way to test it is to create a local user by using make-lp-user, and then create a private bug in a harness with the owner as the user you created. Then browse to the bug, log in as yourself, hit Edit bug mail, and then the unsubscribe link.
<wgrant> stub: You may be interested to know that we had similar lag spikes both times this week.
<wgrant> From :25-29 or so.
<wgrant> I think it's karma finishing.
<StevenK> stub: You wanted to know about the indexes I needed?
<rvba> StevenK: okay, thanks.
<lifeless> is this week ubuntu beta or next?
<StevenK> lifeless: Beta 1 is already out, dude.
<stub> wgrant: Yup. I expect it somewhat. Did the last run have the lag high water mark set to 60 seconds?
<lifeless> stub: not 1.
<lifeless> bah
<lifeless> StevenK: not 1.
<StevenK> Fail :-P
<wgrant> Beta 2 is the 22nd
<wgrant> So next week.
<StevenK> lifeless: Look at the ReleaseSchedule? :-P
<wgrant> stub: I'm not sure; I didn't see numbers.
<StevenK> rvba: I'm surprised that made any sense :-P
<lifeless> actually https://dev.launchpad.net/DowntimeDeploymentSchedule is what I needed
<lifeless> we have a multi-day no-fdt period during ubuntu release weeks
<stub> So I'd love if there is a push button make-virtual-lp-dev-environment script for when I switch to Oneric, and it is an opertunity to get everyone into identical dev environments.
<rvba> StevenK: ;)
<lifeless> stub: the lxc stuff is pretty close to that.
<StevenK> stub: Oh, Jenkins -- https://lpci.wedontsleep.org/job/devel/lastFailedBuild/consoleText
<cjwatson> wgrant: care to give https://launchpad.net/~mvo/+archive/apt-ftparchive-lucid/+packages a spin?
<lifeless> stub: for FDT going forward - I would like us to be very strict one-patch-at-a-time for the next few weeks.
<stub> lifeless: agreed
<StevenK> stub: I wanted to get the indexes in http://bazaar.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/view/head:/database/schema/patch-2208-87-1.sql applied to prod, what do you think?
<lifeless> stub: prove our reliability and get some feeling for how things behave.
<lifeless> stub: cool, thanks!
<jelmer> wgrant: it has a prerequisite which was already merged, perhaps that's confusing it?
<stub> StevenK: Fine.
<jelmer> wgrant: I got two "Launchpad internal failure" emails, which I couldn't explain. Would those perhaps be related?
<lifeless> ok, gnight everyone - see you thursday.
<jml> lifeless: g'night
<stub> StevenK: Is there somewhere to log the live patch requests? I forget.
<wgrant> jelmer: Do they have OOPS IDs?
<wgrant> lifeless: Night!
 * jml downloads a likely looking tarball and stuffs it into a cache
<stub> lifeless: o7
<StevenK> stub: So I should craft a script and prod a friendly LOSA?
<wgrant> jml: I wonder if 0.16.0 has migrated off the first +download batch, so buildout can't see it any more.
<jml> wgrant: that occurred to me also
<StevenK> stub: Which worked out *so* well for me last time :-(
<stub> StevenK: Nah, me or the losas do that. I'll get to it later, but if there is somewhere to log them then there is less of a chance of it being lost :)
<jelmer> wgrant: yep, OOPS-2082MPJ3
<StevenK> stub: I'll give you a friendly prod in a few hours for an update?
<lifeless> StevenK: losa explicitly won't do these
<lifeless> StevenK: they want it automated before they touch it.
<jml> *sigh*
<stub> StevenK: This is all new processes so we expect issues.
<StevenK> Right.
<jml> so, ./bin/test is supposed to work after buildout has run successfully, right?
<wgrant> cjwatson: Looks good.
<wgrant> cjwatson: Doesn't hang, at least.
<StevenK> rvba: You've managed to get into a testable position?
<wgrant> Can't really give it a real test until it's on DF.
<stub> lifeless: So a minor screwup happened when a series of scripts were put together and losa run, but the first one failed (setting a statement timeout for concurrent index rebuilds is borked). I think I need to explicitly sign off scripts to be run by losas for a while until we work out the edge cases and everyone is comfortable.
<rvba> StevenK: I'm just reading the code now.
<StevenK> stub: That sounds like a good plan.
<rvba> StevenK: I reckon what you want to do is now exactly what ConfirmationOverlay was originally designed for.
<rvba> StevenK: the only required parameter right now is a 'button' and AFAIK you won't be providing one.
<lifeless> StevenK: StevenK makes sense to me
<lifeless> bahhh
<lifeless> stub: makes sense to me
<rvba> StevenK: IÂ think I can modify the ConfirmationOverlay a little bit more and then all you will have to do is something like this http://paste.ubuntu.com/688208/
<StevenK> rvba: Ah, you need to make button optional?
<rvba> Exactly.
<jml> OK.
<jml> So if lazr.uri is in versions.cfg but not importable after buildout, what does that mean?
<lifeless> its not in setup.py
<lifeless> or in the transitive dependencies setup.py's.
<cjwatson> wgrant: can you take care of that or do I need to ask for something else (e.g. copying into the Launchpad PPA)?
<wgrant> cjwatson: I can copy it into the Launchpad PPA, but someone needs to get it into CAT.
<wgrant> lifeless, jml: Or you are having namespace package issues with the lazr.* stuff in Ubuntu.
<wgrant> lifeless: But you aren't here, are you?
<jml> wgrant: I am here. I'm *guessing* I am.
<jml> lifeless: it's in versions.cfg. Isn't that enough?
<stub> StevenK: So your patch number is allocated to something different in the dbpatches branch
<wgrant> jml: It needs to be in buildout.cfg
<wgrant> Or setup.py.
<wgrant> Depending on where you want it.
<wgrant> versions.cfg just defines which version will be used if it's needed.
<cjwatson> wgrant: I can do that via RT.  Is that needed to get it onto dogfood?
<stub> StevenK: You or wallyworld_ need to change, and I suspect it is you since you're still around.
<wgrant> cjwatson: Probably.
<stub> (2208-87-2)
<wgrant> mthaddon: How can we test a new apt on mawson?
<lifeless> wgrant: I'm not.
<lifeless> jml: versions.cfg says 'when you need X, choose version Y'
<lifeless> jml: it does not say 'you need X'
<stub> StevenK: Also, your MP seems to have disappeared
<mthaddon> wgrant: you can ask for it to be installed there...
<StevenK> stub: It has, yes. The branch still there
<StevenK> stub: I can resurrect it
<wgrant> mthaddon: Before it's in CAT? :)
<stub> StevenK: It needs a MP, as it needs to land on db-devel.
<jml> Adding it to install_requires and running buildout doesn't seem to change anything
<mthaddon> wgrant: that would be part of what you're asking - for it to be in a -cat suite that's only on mawson
<StevenK> stub: Let me jump to -87-2
<wgrant> mthaddon: :(
<wgrant> mthaddon: OK.
<mthaddon> wgrant: why sad face?
<wgrant> mthaddon: I was hoping for a quick wget and dpkg -i to test the fix. Not unexpected that it would be denied, but inconvenient :)
<mthaddon> yeah...
<wgrant> cjwatson: Can you file the RT?
<wgrant> cjwatson: I'm trying to not be here.
<StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-indices/+merge/75149
<wgrant> Or I could just unpack the deb on mawson and hack LP to use a non-packaged version, but that might be evil.
<stub> StevenK: Want me to allocate the patch number or are you doing that?
<StevenK> stub: I'll do it now.
<cjwatson> wgrant: sure.
<StevenK> stub: Done.
<wgrant> cjwatson: Thanks.
 * StevenK prods ackee
<StevenK> Can haz diff
<wgrant> Argh.
<wgrant> Maintenance people: merge-proposal-jobs is being slow and crashing sometimes.
<wgrant> pls fix.
<wgrant> gmb: ^^
<gmb> wgrant: Is there a bug for it?
<wgrant> gmb: Not for the stuff that's shown up tonight, no.
<wgrant> gmb: But abentley has been working on related stuff.
<wgrant> I would grab him, except he won't be around for 3 hoursish.
<gmb> wgrant: Okay. I'll grab him when he appears.
<gmb> Thanks for the heads-up
<wgrant> gmb: It may be more urgent than that.
<gmb> Hrm.
<StevenK> stub: Requested a review of you on the MP, when you're ready
<wgrant> it needs to be watched. it may recover again in a few minutes.
<wgrant> But it took half an hour to work again when it died 2 hours ago.
<gmb> wgrant: Okay. I'll keep an eye on it.
<gmb> But my knowledge of it is close to 0, so I might be reduced to prodding and going "oh, it's dead"
<gmb> But I'll do my best.
<jml> Hmm. OK. I think this *is* a namespace issue
<jml> my guess is that it's finding lazr.uri of the correct version on my system and thus not installing it.
<wgrant> jml: Yeah, that's been a problem sometimes :/
<wgrant> jml: Python namespace package support is awesome, see.
<jml> wgrant: what's the solution?
<wgrant> And lazr does a spectacularly bad job at it.
<wgrant> Um.
<wgrant> I tended to remove python-lazr.uri.
<wgrant> But that's probably not practical any more.
<jml> wgrant: that removes a lot of stuff. apport, for example
<wgrant> Yeah, not very practical any more.
<wgrant> What if you run with python -S?
<wgrant> Like LP does.
<wgrant> Yes, i know, that's Zope-grade evil, but it might work.
<jml> ./bin/test already runs with python -S
<jml> I guess ./bin/buildout needs to be run with that...
<wgrant> Maybe,.
<jml> nope, it is too
<bigjools> jml is a lean mean bug filing machine
<jtv> gmb: review pretty please w/cherry on top?  https://code.launchpad.net/~jtv/launchpad/bug-844550/+merge/75150
<Riddell> dpm: hi
<Riddell> what's a good time before a release to do a call for translations (for bzr)?
<Riddell> two weeks?
<dpm> Riddell, yeah, 2 weeks is a good rule of thumb
<stub> StevenK: Can you please lp-land that branch? (no need to shove it through ec2) I'd do it, but lp-land is bitching about missing revisions
<gmb> jtv: sure
<jtv> thanksâfeel free to skip much of the cover letter if it doesn't interest you.
<jtv> Mostly just writing it for historical record, and as proof that this has been thought about and discussed.  :)
 * jml moves on to launchpadlib
<jtv> cjwatson: what ever happened to your multi-arch translations branch?
<jtv> Still got a ticket on my board for landing it once I get word that it's been updated.
<wgrant> It's landed and deployed.
<wgrant> But not active, because we're waiting on an apt-ftparchive fix.
<wgrant> jtv: ^^
<jtv> OK, then I'll retire the ticket.
<Riddell> dpm: and where's the place to announce it?
<cjwatson> jtv: right, you can drop that ticket; I'll drive the apt-ftparchive deployment bits (though I'll need wgrant's help to test on dogfood once RT#47856 is resolved) and I'll deal with flicking the API switch at an appropriate time
<_mup_> Bug #47856: Screenshot needs to be updated <Ubuntu Website:Fix Released> < https://launchpad.net/bugs/47856 >
<cjwatson> jtv: thanks
<jtv> Already done, thanks.
<jtv> cjwatson: also, we're very close to deploying the final commit that will make all Debian SPPHs Published instead of Pending.
 * cjwatson nods
<cjwatson> the ubuntu-dev-tools change to just do status=Published is already out
<rvba> gmb: Hi, could you please review this mp? https://code.launchpad.net/~rvb/launchpad/confirmationoverlay-button-optional/+merge/75154
<gmb> rvba: Sure
<rvba> Thank you.
<gmb> jtv: I'm not sure whether to be worried or not: You branch is one of the clearest, best explained Soyuz branches I've ever read (though bigjools's branches are also usually equally clear). I must congratulate you, on the assumption that Soyuz hasn't got simpler and I haven't got any smarter - you've done an excellent job.
<jtv> gmb: that's easy thenâyes, you should be worried.  Thanks.  :)
<gmb> :)
<dpm> Riddell, I'd suggest launchpad-translators(AT)lists(DOT)launchpad(DOT)net and ubuntu-translators(AT)lists(DOT)ubuntu(DOT)com
<gmb> jtv: approved, with one comment about making the test a bit easier to understand (at least for us lackwits)
<jtv> gmb: thanks.  Insert obrant about why it's a good thing that you asked for an improvement.  :-)
<gmb> :)
<gmb> rvba: r=me with one minor tweak requested.
<rvba> gmb: thanks.
 * gmb lunches
<nigelb> wgrant: hi
<nigelb> wgrant: Were you able to reproduce the KHTML issue on your machine?
<nigelb> I can't see it in Maverick.
<StevenK> stub: Absolutely
<stub> StevenK: I just landed that branch
<StevenK> stub: How did you fix the missing revisions thing?
<wgrant> nigelb: I haven't tested.
<nigelb> Oh ok. I fear I may have to download debian sid just to test.
<nigelb> Hrm, I wonder...
<stub> StevenK: By using lp-land properly, as it only designed to land the branch you are in (so branch your branch and run lp-propose in that directory)
<nigelb> I could install it into a chroot and ask it to use the right display.
<StevenK> stub: Right, right. The indexes on are prod live now too?
<stub> StevenK: All except the master, which is blocked until gina finishes up its current transaction
<wgrant> Ah
<StevenK> stub: Which could take hours
<stub> StevenK: The statement should complete fine when it commits.
<stub> StevenK: It won't take hours, as the transaction reapers are still enabled...
<StevenK> Heh
<wgrant> jelmer: Is anything blocking your two approved import branches from landing?
<wgrant> import-colocated-branches and http-git-support
<jelmer> wgrant: they're in ec2 at the moment
<wgrant> jelmer: Excellent, thanks.
<jelmer> wgrant: There was a test failure I had to fix and having a local lp setup was blocking me from that.
<jelmer> (yay for lxc)
<nigelb> Oh, you got it working?
 * nigelb is tempted to upgrade to Oneiric.
<jelmer> nigelb: oneiric is the reason I'm trying lxc in the first place :)
<jtv> Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
<jtv> Upcoming buildbot failure.  :(
<nigelb> jtv: I thought wgrant was the nostradamus here. :P
<wgrant> I actually noticed it a couple of minutes ago, and was searching through the old builds to find what leaked.
<jtv> Oh, there's a whole bunch more.  :( :(
<wgrant> But I can't find anything.
<wgrant> This is spurious, but it means our diagnosis was wrong.
<jtv> AttributeError: 'TestPullerMasterIntegration' object has no attribute '_lock_actions'
<wgrant> There's more broken than we thought.
<jtv> This is going to hold up the commit for Gina.
<stub> Is the lxc setup pretty well scripted or documented, or does it require a clue?
<wgrant> stub: It's well-documented.
<wgrant> stub: Works best on oneiric, but mostly works on natty.
<wgrant> https://dev.launchpad.net/Running/LXC
<stub> This will be Oneiric
<stub> ta
<jtv> wgrant: that AttributeError doesn't strike me as particularly spurious, especially given the number of errors.
<wgrant> jtv: You would think.
<wgrant> But it is.
<wgrant> jtv: This is one of the types of error that's been showing up for 3 weeks now.
<wgrant> We'll clean leaked processes from the buildbot slave, force the build, and it will be happy again.
<wgrant> And then we'll abandon buildbot in favour of Jenkins, and all will be good.
<StevenK> rvba: Any news re: button becoming optional ?
<rvba> StevenK: The branch is in ec2.
<StevenK> rvba: Nice, thanks!
<rvba> StevenK: https://code.launchpad.net/~rvb/launchpad/confirmationoverlay-button-optional/+merge/75154
<rvba> StevenK: I've added an example on the MP.
<rvba> I mean an example usage*
<nigelb> rvba: Love your example :)
<rvba> nigelb: ;)
<jtv> bigjools, rvba, StevenK, wgrant: mind a dogfood update right now?
<rvba> jtv: fine by me.
<wgrant> jtv: Go ahead, if you haven't already.
<jtv> thx
<nigelb> Did I break buildbot? :(
<nigelb> jtv: Is this the failure you predicted?
<jtv> nigelb: I don't think you broke it.
<jtv> You may have broken one of the pieces that you see on the floor, butâ¦
<nigelb> heh
<jtv> But yes, this is the failure I predicted.  Now bow before your oracle!
<wgrant> Ah, gary_poster!
<wgrant> gary_poster: buildbot is being friendly again, this time without an unnoticed failure in the previous run :(
<gary_poster> wgrant, joyful
<wgrant> Rather.
<allenap> Is there likely to be any fallout from running bzr upgrade lp:lazr.uri?>
<gary_poster> wgrant, did you look at it?  I have other things going on, but ISTM that lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another broke, for unknown fragility reasons, and everything else fell apart afterwards because of it.  I'll simply disable the test and file a bug unless you or someone else has a better idea.
<wgrant> gary_poster: I hope so.
<wgrant> gary_poster: Sounds like a good plan.
<gary_poster> cool
<stub> Any objections to me killing staging and qastaging for a few seconds to QA my Librarian outage behaviour revision?
<gary_poster> allenap, thinking...yes...
<gary_poster> allenap, oh we do that with eggs
<gary_poster> so it's probably fine, actually
<allenap> gary_poster: Cool. Let the mayhem begin...
<gary_poster> (the only thing I know of that we have to worry about with bzr upgrade scenarios is buildbot and ec2 images for stuff we get in sourcecode)
<allenap> That didn't last long; I can't write.
<gary_poster> heh
<gary_poster> LET THE MAYHEM BEG...oh.
<gary_poster> odd that you can't write.  Lemme look
<allenap> gary_poster: It's PQM managed. Didn't realise before.
<gary_poster> ah ok allenap
<gary_poster> hm, I didn't think we had our "extras" under PQM control
<allenap> Yeah, odd. Oh well. I think I'll gently put that can of worms back on the shelf (i.e. I don't want to break PQM).
<gary_poster> k
<jml> jkakar: hey, fwiw, I'm trying to resurrect https://code.launchpad.net/~jkakar/launchpadlib/fake-launchpad/+merge/26391
<jtv> wgrant: I wonder if I shouldn't just remove Gina's dry-run mode, if it's not even tested anyway.  Far too risky.
<wgrant> jtv: It's not tested?
<wgrant> It's clearly not tested very well.
<wgrant> But it may be tested.
<jtv> Wellâ¦ an unconditional commit did not break any tests.
<wgrant> It doesn't somehow abort earlier?
<jtv> Nope, nope, nope.
<wgrant> Awesome.
<jtv> Insert a commit somewhere in an obscure place to reduce transaction durations, and kaboom.
<jtv> May already be happening for all we know.
<wgrant> That's quite useful.
<jtv> bigjools: your guidance wanted.
<bigjools> whu what?
<jtv> bigjools: Gina has a dry-run mode.
<jtv> It seems to run the whole script in one huge transaction, and then not commit.
<jtv> Not tested.
<jtv> We don't know of it works as advertised.
<bigjools> ok
<jtv> Want me to test it (script run in test, and who knows, fixes) or remove it?
<bigjools> first question, do we need a dry run mode?
<bigjools> IOW, how does it help us?
<jtv> Well since young Will here had no idea it even existed, I was wondering whether you might know the answer.
<bigjools> I don't I'm afraid, the Gina code has been barely touched in years, well before I started working on LP
<bigjools> but I can help with some probing questions
<bigjools> which might be slighly leading
<jtv> I was nowhere near the cottage.
<jtv> Not that there was a cottage!
<bigjools> I hereby name you Geoff
<jtv> Geoff?
<nigelb> I fully expected bigjools to give a Yoda-ish answer. bigjools, I am disappoint.
<bigjools> it was too risque, so I PMed it :)
<nigelb> lol
<nigelb> Yeah, channel' logged. Good idea.
<nigelb> This reminds me. Is #launchpad-reviewers used at all?
<jtv> *I* was of course referring to the opening episode of The Black Adder.
<jtv> nigelb: no, that's obsolete.
<bigjools> nigelb: no it's not
<nigelb> Hrm, okay.
<jtv> bigjools: so about these probing questions you promised..?
<nigelb> I noticed it in the logs.
<bigjools> jtv: I asked the first above, no answer yet!
<jtv> Does it help us?  âNot me, not William, clearly not you.  Someone else out there maybe, but who would know but Julian?â
<bigjools> jtv: I cannot think of a use for it either
<jtv> Soâ¦ zap it?
<bigjools> yup, blow it
<bigjools> away
<nigelb> More desctruction \o/
<nigelb> *destruction
<jtv> Well, with the "away" added, yes.
<nigelb> BWAHAHA
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #1,065: FIXED in 4 hr 10 min: https://lpci.wedontsleep.org/job/devel/1065/
<rvba> abentley: Hi.  I'd like your help to make codehosting run locally. I'm failing to push branches locally and see them on my local instance.
<rvba> I'm reading  https://dev.launchpad.net/Code/HowToUseCodehostingLocally.
<abentley> rvba: on the phone
<rvba> abentley: Okay, please ping we when you're available then (http://paste.ubuntu.com/688322/ is what I've done).
<StevenK> rvba: Are you using make run_codehosting ?
<rvba> I also tried to create the missing directory manually but then nothing seems to be picked up by make sync_branches.
<rvba> StevenK: yes.
<StevenK> rvba: bzr push lp://dev/~foo/bar/baz should work with the following .ssh/config snippet: http://pastebin.ubuntu.com/688323/
<wgrant> Damn, sinzui is quick.
<wgrant> And he's not even here.
<nigelb> lol
<nigelb> wgrant: Did he reply to your email?
<wgrant> https://code.launchpad.net/~wgrant/launchpad/delete-more/+merge/75179
<nigelb> Neat.
<nigelb> Its all a hidden plot to not make you sleep :)
<wgrant> He reviewed it within 2 minutes :(
<bigjools> wgrant: there's a subtle bug in that MP
<bigjools> you need dedent("""\ (with the backslash)
<wgrant> I hate that.
<bigjools> yep
<wgrant> So I called strip() instead.
<bigjools> hmm
<bigjools> missed that
<bigjools> dunno which is more ugly tbh
<wgrant> I know. But one doesn't involve a backslash, so I tend to go with that.
<bigjools> it's once place where I prefer it
<wgrant> I don't know.
<bigjools> dedent sucks
<wgrant> Derivation is done.
<wgrant> Yay
<wgrant> Interesting...
<bigjools> stalker
<wgrant> Well, DSD creation seems to be sort of screwed.
<bigjools> ?
<wgrant> https://launchpad.net/bilimbitest/angry/+localpackagediffs
<wgrant> Ahh, maybe the jobs haven't run yet.
<wgrant> So they could all be resolved in a few minutes.
<rvba> StevenK: this is what I get now :( http://paste.ubuntu.com/688331/
<wgrant> In fact, they're vanishing as I watch.
<wgrant> Or appearing.
<rvba> And I can see the connection failing in the logs of make run_codehosting
<bigjools> it pulls from RELEASE remember
<wgrant> bigjools: Ah, and DSDs look at updates too.
<bigjools> yes
<wgrant> Yeah, the ones I saw first were the same version.
<wgrant> But the jobrunner killed those off quickly.
<wgrant> Now we are settling down into sensible -updates or -security diffs.
<bigjools> there's an argument to pull from -security as well I guess
<allenap> gmb: Got time for a ~200 liner? https://code.launchpad.net/~allenap/launchpad/series-init-failure-explanations-bug-835024-db/+merge/75183
<gmb> allenap: For you? Sure
<StevenK> rvba: I am about to head to bed, but you want to run that against a DB that has had make-lp-user ran across it, and then mkdir foo ; cd foo ; bzr init ; bzr push lp://dev/~rvba/+junk/foo
<allenap> gmb: You're very kind :)
<rvba> StevenK: that's precisely what I did.
<rvba> StevenK: good night :)
<gmb> Or misguided :)
<StevenK> rvba: Then that's terrible.
<rvba> StevenK: I'll ask abentley when he is available.
<abentley> rvba: I think you have a stale socket file for the codehosting forker process in /tmp
<wgrant> rvba: Check that you don't have a stale forking service pid in /var/tmp
<wgrant> yeah, socket, not pid
<wgrant> What he said.
 * rvba checks
<rvba> abentley: wgrant that was it. Branch pushed all right ... let see if make sync_branches works.
<rvba> abentley: StevenK wgrant make sync_branches worked! Thanks for your help guys.
<abentley> rvba: You're welcome.
<wgrant> rvba: Excellent.
<wgrant> abentley: merge-proposal-jobs blew up a bit this evening, from around 5 hours ago.
<wgrant> abentley: You may want to check the logs to see if it's anything new.
<wgrant> As I see you have yet more improvements to this area.
<gmb> allenap: Am I right in thinking that the reason for this branch is that it does most of the encode/decode work for us and we're left to just use loads() and dumps() as appropriate?
<nigelb> wgrant: Well done driving abently away :P
<gmb> Oh...
<gmb> allenap: Or does the Storm JSON type avoid us having to use loads() and dumps() at all?
 * gmb might be getting confused by the diff
<allenap> gmb: We don't have to do the loads() or dumps() either.
<gmb> Right
<allenap> gmb: There's a bulk job creation bit which still needs to dumps() because it composes SQL directly.
<gmb> Okay
<abentley> wgrant: none of my improvements have been deployed yet.  I had to roll them back when QA failed.
<wgrant> abentley: Ah :( that would explain the failures that I thought were fixed.
<gmb> allenap: All looks sane. r=me
<allenap> gmb: Thanks!
<gmb> np
<jkakar> jml: Awesome!  I've basically ditched it... but I'm happy to help if need be.
<jml> jkakar: thanks.
<jml> jkakar: actually, I do have one question about _create_resource
<jml> jkakar: it seems that if you get something back from a method, you have no good way of determining if it's a collection or an entry.
<jkakar> jml: Am on a call, but ask here now, I'll take a look at the code again, and answer when I'm off the phone.
<jml> jkakar: ok. thanks.
<nigelb> When does devel get merged into db-devel? By buildbot?
<wgrant> nigelb: buildbot-poller merges stable into db-devel once buildbot passes it.
<nigelb> so, I have to wait because of builtbot breakage.
<bigjools> I hate browsers
<wgrant> nigelb: You can't land your DB patch until we have deployed your stuff, anyway.
<nigelb> Argh, ok.
<nigelb> Need to just work on js-oopsd then.
<nigelb> wgrant: I call conspiracy on that ;P
<wgrant> Oh?
<wgrant> It is, I admit, rather convenient, since I want the window tomorrow for meeeee. But it's also true!
<nigelb> HA!
<nigelb> Also, I don't think I'll ever forget about grepping the whole tree next time.
<wgrant> bzr grep is your friend.
<nigelb> I've lost count of the number of times StevenK and bigjools have already told me that.
<jtv> gmb: perhaps you'd like to review this one as well?  Not much to see, really.  https://code.launchpad.net/~jtv/launchpad/bug-848954/+merge/75188
<gmb> jtv: Sure.
<stub> nigelb: Your db patch dropping BugTask.statusexplanation had test failures. Didn't look like anything taxing but I didn't have a chance to look today.
<gmb> jtv: Points for referring to wgrant as "Young Will"
<jtv> heheh
<gmb> jtv: Approved with one question about a somewhat opaque (to me, anyway) error message.
<jtv> :)
<jtv> Thanks
<jtv> gmb: I suspect nobody's going to know much more about that message, so I might as well try to do something about it.  Hang on.
<gmb> Ok
<jtv> gmb: if it's any consolation, I've already asked Julian for permission to be left alone in a locked room with Gina and Katie for a while, so I can give them a stern talking to.
<gmb> ...
<gmb> E_NO_CORRECT_RESPONSE
<gmb> jtv: I'm not hung up on the idea of that message changing before it lands, but it seems like an opportune moment.
<jtv> Yeah, this is stuff that can do with any drive-by improvement.
<jtv> Including the one that this branch is about: âOh, there may be a feature there.  I think I broke it.  Let's destroy it.â
<jtv> gmb: the diff has updated.
 * gmb looks
<jtv> For some reason the page won't display very legibly on my browser.
<jtv> But that's probably just a bad connection.
<gmb> jtv: It looks good from here. Nice change, thanks!
<jtv> Thank you.
<jml> jkakar: also, I think I'm going to kill the testresources stuff
<nigelb> Hrm, just missed stub
<jkakar> jml: That was a long call... am off now.
<jml> jkakar: yay
<jml> jkakar: my earlier question still stands...
<jkakar> jml: Uhm, if I remember correctly, the reason for not knowing whether something is a collection or not is that you can't (or couldn't, maybe you can now) tell whether something is a collection from the WSDL.
<jml> jkakar: right. so leonardr's comment about trying to return a FakeCollection from methods is ... invalid?
<jkakar> Maybe there's a convention that you can use to tell... or something?  I rememeber it being pretty confusing (WSDL in general, in addition to the way it's used by Launchpad).
 * jkakar looks again
<jml> jkakar: I'm ignoring lifeless's suggestion about including a timestamp and doing IMS. Also ignoring suggestions to move to lazr.restfulclient.
<jkakar> jml: I don't understand leonardr's comment/question/suggestion.
<jml> jkakar: I'm currently preparing a smaller wadl file for testing
<jkakar> jml: Awesome!
<jml> jkakar: and have replaced the wadl file that you included with the official 1.0 wadl file
<jkakar> jml: Why do you want to drop testresources support...?
<jml> jkakar: because it makes it harder to use alternate wadls
<jkakar> jml: Yeah, flacoste was quite keen to have it be downloaded when tests run, as I recall.
<jml> jkakar: not "drop" per se, just not export it
<jkakar> jml: You could pass one in, I guess... but anyway, the testresources stuff is sugar, not a big deal.
<jkakar> jml: I'm not very happy with a test suite that *needs* net access, but I could live with it.
<jml> jkakar: I've also added FakeEntry and FakeRoot
<jml> jkakar: I'm not making it need net access
<jml> jkakar: just bundling 1.0 and putting a suggestion in the docs that you might like to download your own to test against.
<jkakar> jml: Cool... I meant that in reference to the (old) request from flacoste that the WSDL be downloaded automatically.
<jml> jkakar: oh right. ugh.
<jml> jkakar: also, I've expanded the docstring to http://paste.ubuntu.com/688401/
<jkakar> jml: Anyway, I'm really happy you're picking up that branch.  Sorry the code is so crap... it was a bit of a discovery process, since I discovered the WSDL file as I went.
<jml> jkakar: I'm going to ignore poolie's suggestions about changing the API
<jml> jkakar: and about not doing XML parsing
<jml> jkakar: the code is fine. I still haven't discovered WSDL yet, so maybe I'm behind :)
<jkakar> jml: I remember being surprised that wadllib didn't actually do much with WADL files.  I expected it to be an API to the WADL file, but it wasn't (maybe that's changed, it's been a long time).
<jml> jkakar: oh yeah :)
<jkakar> jml: WSDL is, uhm, fantastic! :)
<jml> jkakar: I think I'll leave that particular box closed for now.
<jkakar> jml: Cool.  Great work on the docstring, thanks.
<jml> hmm
<jml> does having the docs in the wadl actually *help* the client?
<jkakar> Now I'm totally confused about the difference between WADL and WSDL... uhm, I think I've been using them interchangeably...
<jkakar> I guess we care about WADL, not WSDL, only because we have a wadllib and not a wsdllib. :)
<jml> OK. Done.
<jml> gmb: Can you please review: https://code.launchpad.net/~jml/launchpadlib/fake-launchpad/+merge/75211
<gmb> jml: Sure.
<jml> gmb: thanks.
<nigelb> Hi! I'd like to run this by someone for sensibility. Javascript related.
<nigelb> Currently, we harvest bug links and send them to an internal API for checking if they're valid/accessible.
<jml> gmb: I have to go in 10m, fwiw.
<nigelb> Can I add a feature to pull out bug title from that?
<gmb> jml: Okay, no woirries. I'll put any questions in the mp.
<nigelb> deryck: Hi, around?
<deryck> nigelb, hi, I am.  Just about to break for lunch though.  but what's up?
<nigelb> deryck: I saw you were marked as javascript person, so I just wanted to run something by you
<jkakar> "<jml> Not doing this. If you really care about it, perhaps you could do it." <-- Awesome. :)
<deryck> nigelb, ok, cool.  go for it.
<adeuring> gmb: could you please review this mp: https://code.launchpad.net/~adeuring/launchpad/cronscript-for-hwdb-report-fixes/+merge/75214 ?
<gmb> jml: E_5000_LINE_DIFF
<nigelb> deryck: I'd like to add to the javascript that checks if the linked bugs in pages are valid by updating the "title" attribute of the <a> with the bug's title
<gmb> adeuring: I shall add it to my queue; not sure I'll get to it by EOD.
<jml> gmb: most of that is boring WADL diff.
<adeuring> gmb: ok, np
<jml> gmb: there's instructions on the linked mp for how to get a sane diff.
<gmb> jml: Ah, good, I'm just waiting for that to load...
<deryck> nigelb, ok, sounds fine
 * gmb wonders if it's LP or his connection that's being odd
<nigelb> deryck: cool! I'll file a bug and start doing it :)
<deryck> cool
<nigelb> deryck: Thanks!
<deryck> nigelb, subscribe me to the bug please
<deryck> and np!
<nigelb> okay :)
<gmb> jml: The saner diff is still 979 lines. I'm afraid I won't be able to get through all of this before EOD, and I need to be gone at 6 on the dot tonight.
<gmb> You might be better off finding someone at the start of tomorrow.
<sinzui> compiz/unity is very unstable today. I am restarting as a last resort to get control of my windows.
<nigelb> Is there documenation somwhere of what's returned by IBugTask.search?
<nigelb> I see how to call it, parameters, etc.
<danilos> matsubara, hi, I can't see OOPSes from ackee-bzrsyncd in OOPS reports (eg OOPS-2075SMS19) that's in /srv/launchpad.net-logs/scripts/ackee-bzrsyncd/branch-scanner/2011-09-06 on devpad; are these directories perhaps ignored?
<matsubara> danilos, let me take a look
<danilos> matsubara, thanks
<danilos> sinzui, hi, do you think you'd have a minute to chat about bug 480123? basically, while it sounds the solution is "simple", I am sure we need to worry about what's going to break, and you probably have a few other ideas of what's to look out for?
<_mup_> Bug #480123: Milestone names/version should be unique to series <escalated> <linaro> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/480123 >
<sinzui> it is not simple
<sinzui> We tried to fix that in 2009 and failed,. we reverted the schema changes after 2 months on no progress
<sinzui> danilos, mumble, skype?
<danilos> sinzui, skype works better for me, "danilo.segan"
<gmb> adeuring: Approved, but you need to change the "# Disabled because of bug ..." comments to XXXs for clarity.
<adeuring> gmb: ah, right! thanks!
<gmb> np
<danilos> abentley, hi, we have an escalated bug which is a (non-DB) timeout in "ScanBranch" jobs, could you perhaps look at bug 808930 and comment if you have any ideas of where one would start (i.e. could we somehow look into speeding these operations up for gcc-linaro project, checking if they are using stacked repos and whatever gives us the best performance)
<_mup_> Bug #808930: Timeout running branch scanner job <escalated> <linaro> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/808930 >
<abentley> danilos: I think the way to close this bug is to optimize bzr.
<abentley> danilos: I would start by downloading the branch locally and profiling the script.
<danilos> abentley, right, thanks for the input
<abentley> danilos: it is already using the fastest bzrlib operations I know of.  I wrote the script, and I also wrote the Graph operations, several years before that.
<abentley> danilos: it's possible the algorithm could be improved.  it's also possible that we have to switch to C or Pyrex to get that performance.
<abentley> danilos: Or we may need to implement a greatest-distance-from-origin cache in order to get good performance.
<abentley> danilos: jam has also done a lot of work in this area, and may have advice.
<james_w> hi, can someone tell me what caching may be involved in producing/serving +temp-meeting-export?
<james_w> hitting that URL repeatedly gives back results of varying staleness
<nigelb> I was just about to ask :)
<james_w> :-)
<danilos> abentley, right, at the moment I was just looking for some more info other than "this is slow"; I hope you don't mind me pasting your input on the bug?
<abentley> danilos: feel free.
<danilos> abentley, thanks again
<abentley> danilos: np
<sinzui> danilos, lp:~edwin-grubbs/launchpad/bug-174468-milestone-release-overlap should contain chunk of code it its history that made milestones unique to the series. I do not see that in the history though. The branch lived for many weeks and it is very confusing.
<sinzui> danilos, I can see from my discussion that changes to uniqueness will has a few themes of work: product-release-finder, project groups and ProjectMilestone, sorting milestones (wants debversion)
<danilos> sinzui, right, thanks, I'll add that to my notes
<danilos> sinzui, the DB patch in r7278 does change the unique constraint, fwiw
<danilos> james_w, hi, do you think you'd have a few mins to talk about what you need out of bug 480123?
<_mup_> Bug #480123: Milestone names/version should be unique to series <escalated> <linaro> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/480123 >
<james_w> danilos, hi, yes
<james_w> but not now
<james_w> sorry
<james_w> I need to get some food
<james_w> danilos, I'll forward you what I sent to flacoste, and then you can let me know if it's sufficient
<danilos> james_w, ok, I am not going to be around for much longer today, so we can perhaps talk tomorrow
<danilos> james_w, excellent, that's even better
<james_w> danilos, sent
<james_w> I'll be happy to talk if it's not clear from that
<danilos> james_w, cool, thanks, this should give me a better idea of what you need, I'll have to wrap my head around it first :)
<nigelb> deryck: Hey, could you give me a quick hand with BugTask.search? How do I find out what it returns?
<deryck> nigelb, I believe it returns a result set of bugtasks.  Let me look to confirm....
<deryck> nigelb, how I'm checking is to go to the method in lp.bugs.model.bugtasks
<nigelb> I'm there as well :)
<nigelb> I'm looking at _search()
<nigelb> since that's what seems to be called.
<nigelb> I think storm is what's confusing me there.
<deryck> nigelb, yeah, you're in the right place.  and see the comment about what it returns.
<nigelb> hrm, yes
<abentley> gmb: I suspect you're no longer OCR.
<gmb> abentley: You suspect correctly. I shall remedy this falsehood presently.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<abentley> deryck: Since there's no OCR, could you review https://code.launchpad.net/~abentley/launchpad/daily-build-disabled-archive/+merge/75239 ?
<deryck> abentley, sure
<deryck> abentley, r=me.  nice, succinct, and good change.
<abentley> deryck: thanks.
<deryck> abentley, np
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> I thought you were supposed to be away.
<nigelb> deryck: Still around?
<nigelb> I er have an implementation confusion
<deryck> nigelb, I am.  just tending to a few things, but ask away.  I'll answer as I can.
<nigelb> Hrm. I may have sorted this. Verifying.
<nigelb> Hrm. No.
<nigelb> Currently, we post all the branch URLs and all the bug urls to +check-links
<nigelb> This returns a JSON with invalid_branch_links and invalid_bug_links
<nigelb> I'll how to now modify how this works a bit more non-trivially than I thought.
<m4n1sh> is there a way to be subscribed to all bugs related to a product, but exclude selected tags. Example I want to free myself of all the apport bugs
<m4n1sh> they have the tag "apport-crash"
<nigelb> instead of +, use - I think.
<cr3> in lazr.restful, is there a way to export a parameter, not an operation, as another name?
<m4n1sh> nigelb: where?
<nigelb> m4n1sh: URL
<nigelb> m4n1sh: what project is this?
<m4n1sh> nigelb: for this https://bugs.launchpad.net/ubuntu/+source/zeitgeist
<m4n1sh> want to make myself free of apport-crash bugs
<m4n1sh> hundreds of them are filed in a week
<m4n1sh> duplicates
<m4n1sh> due to zeitgeist being installed by default
<nigelb> m4n1sh: https://bugs.launchpad.net/ubuntu/+source/zeitgeist/+bugs?field.tag=-apport-crash
<m4n1sh> yes then?
<m4n1sh> that is just for filtering
<m4n1sh> but for subscription
<nigelb> AH.
<m4n1sh> looks like there is no such right now
<m4n1sh> need to file a bug - to be marked as wishlist
<m4n1sh> btw against which package should I file?
<lifeless> sure there is
<nigelb> I wonder if you can do -apport-crash in the subscription.
<m4n1sh> launchpad-foundations
<lifeless> use an advanced subscription
<m4n1sh> lifeless: used that
<m4n1sh> that is "include tags"
<m4n1sh> lifeless: I want all minus specific tags
<lifeless> ah, then perhaps file a bug yes.
<m4n1sh> against which module?
<lifeless> bugs.launchpad.net/launchpad
<m4n1sh> ah. I thought foundations
<nigelb> I don't think there are modules anymore.
<m4n1sh> I mean like lazr.restful
<m4n1sh> launchpad-foundations
<m4n1sh> etc etc
<nigelb> lifeless: Could you quickly review what I'm doing for the bug autolinkifying?
<lifeless> m4n1sh: lazr.restful is a piece of code, launchpad-foundations was a sub-team of the launchpad team, but it no longer exists post restructure
<m4n1sh> so right now how many pieces are there?
<m4n1sh> only launchpad?
<lifeless> m4n1sh: in what sense?
<m4n1sh> like lazr.restful, launchpad etc etc
<lifeless> organisationally or bits of software?
<nigelb> lazr.restful is not launchpad only.
<nigelb> things like malone, blueprints, are all launchpad.
<m4n1sh> bits of software on which launchpad is dependent on
<lifeless> m4n1sh: several hundred
<m4n1sh> which are part of launchpad, but developed as seperate project
<lifeless> m4n1sh: about 50 of which we are the primary developers of
<nigelb> I think I added one to that list :P
<m4n1sh> launchpadlib also comes under launchpad.net/launchpad ?
<lifeless> https://launchpad.net/launchpad-project
<lifeless> m4n1sh: no, only the launchpad source code is managed at /launchpad. Components etc all have their own bugtrackers. malone and so forth are not (currently) components, they are just part of the launchpad source code
<nigelb> Yep. JS OOPS Daemon.
<m4n1sh> ah yes
<m4n1sh> that is what I was asking
<m4n1sh> like loggerhead is developed seperate
<m4n1sh> lazr.restful
<m4n1sh> lazr.enum
<m4n1sh> wadllib etc
<m4n1sh> done
<m4n1sh> https://bugs.launchpad.net/launchpad/+bug/849429
<_mup_> Bug #849429: Subscription: Allow the option of selectively blacklisting certain bug tags. In short add tag blacklisting <Launchpad itself:New> < https://launchpad.net/bugs/849429 >
<jml> gmb: thanks anyway
<nigelb> I think I just murdered some javascript. Gah.
<bac> nigelb: as they say in Texas: it needed murdering
<nigelb> haha
<nigelb> I took some perfectly generic javascript and made it specific.
<nigelb> Just realized I can make it generic with my changes as well. So fixing that.
<nigelb> How evil am I?
<nigelb> We went from http://paste.ubuntu.com/688640 to http://paste.ubuntu.com/688639
<nigelb> With changes in Javascript to deal with that.
<flacoste> bac: you got my unprompted review :-)
<bac> flacoste: great, thanks
<nigelb> Now I understand why debugging launchpad js is nightmare.
<flacoste> bac: well, you might not thank me after you read it :-)
<bac> flacoste: i'd read it.  i was just being nice.
<bac> :)
<flacoste> bac: do these comments make sense? i might be dreaming
 * flacoste does reality cehck
<bac> flacoste: i think so.  i'll have to look at the db procedure that updates the date_last_message. shouldn't be a problem
<bac> flacoste: i'd assumed (incorrrectly) that the triggers for that were sensible and what we wanted...
<flacoste> bac: ok, thanks for working on that btw
<bac> flacoste: np.  i'm off tomorrow but will wrap it up thursday
<nigelb> Anyone around for some evil javascript review? I don't need a full review, but just help on whether it looks okay.
<nigelb> flacoste: Do you have a quick few minutes?
<nigelb> I guess not. I'll grab some other javascript-y person tomorrow.
<lifeless> benji: re semver: you might also like reading the libtool soname rules
<benji> sounds like my kind of reading material
<nigelb> benji: Hey, could check something for evilness? :)
<benji> nigelb: if it's pretty quick, I have two chilli-cheese-onion-dogs with my name on them
<lifeless> benji: http://www.gnu.org/s/libtool/manual/html_node/Updating-version-info.html
<lifeless> "try to set the interface numbers so that they correspond to the release number of your package. This is an abuse that only fosters misunderstanding of the purpose of library versions."
<nigelb> benji: https://code.launchpad.net/~nigelbabu/launchpad/bug-title-849121/+merge/75267
<nigelb> You don't need to actually do a full review
<lifeless> bah, 'Never ...'
<benji> thanks lifeless, I've put that in a tab in my reading-for-after-the-kids-are-in-bed window
<nigelb> benji: Its a WIP. There's some code to be deleted in that. And *lots* of tests to be fixed. And javascript needs testing.
<nigelb> But I need help with the sanity of my approach :)
<benji> nigelb: it looks good to me, the only questionable thing is hand-building the bug links (line 35 of the diff), there is a mechanism to build those links for you
<nigelb> benji: I don't think so. I can't use canonical_url for that.
<benji> nigelb: may be, it's not a big sin (but I worry about the day we want to change our URL structure and all of these one-off URL generation sites have to be changed)
<benji> I really have to go now.  Later.
<nigelb> benji: Thanks!
<nigelb> I shall make it better over the week :)
<nigelb> Javascript - 0; Nigel - 1.
<mwhudson> nigelb: is what you are doing going to make bug titles reappear when i mouse over "bug 1234" in text?
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<nigelb> mwhudson: Yes
<mwhudson> nigelb: hooray
<nigelb> heh
<mwhudson> nigelb: are you coming to orlando so i can buy you a beer?
<nigelb> No, I'm not :)
<mwhudson> ah well, i'm sure i'll get the chance eventually
<nigelb> But I'll try to come for the next european one :)
<nigelb> mwhudson: I took the generic XHR updating framework and extended it to deal with valid and invalid.
 * nigelb dreads broken tests
#launchpad-dev 2011-09-14
<StevenK> wallyworld: http://penny-arcade.com/comic/2011/09/12
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> ohai StevenK
<nigelb> Could you point me to something that will help with javascript unit tests?
<StevenK> No
<StevenK> Since I have no idea myself
<nigelb> excellent!
<nigelb> I'll wait for wallyworld then.
<nigelb> I made a whole lot of changes to untested code. Can't get away without writing tests anymore.
<wallyworld> hi nigelb
<nigelb> wallyworld: Hi! I did some major changes to +check-links and lp-links.js
<wallyworld> nigelb: excellent. what changes?
<nigelb> it pulls in the title and updates the dom with bug title
<nigelb> so I had to change how the API is used to make my changes generic.
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/bug-title-849121/+merge/75267
 * wallyworld looks
<nigelb> Actually, the example in the MP is slightly wrong. /me corrects.
<nigelb> wallyworld: How does it look? I hope it doesn't make your eyes bleed :)
 * wallyworld looks again
<wallyworld> nigelb: the branch links value should be "invalid"
<wallyworld> but it looks fine i think
<wallyworld> it's a nice addition
<nigelb> \o/
<wallyworld> did you have a question you wanted to ask?
<nigelb> err, is there documentation about javascript unit tests?
<wallyworld> i didn't look at the code in detail, just the new output
<wallyworld> https://dev.launchpad.net/JavascriptUnitTesting
<wallyworld> that plus looking at what others have already done
<nigelb> oooh. Awesomeness \o/
<wallyworld> plus asking on here if you are stuck :-)
<nigelb> :)
<wallyworld> nigelb: i have to go out for a bit to see a doctor. but ping me later if you have any questions etc
<nigelb> wallyworld: sure! :-)
<wallyworld> good luck with the tests :-) yui tests aren't too bad when you get used to them
<nigelb> Hah. The last time I tried I was fairly homicidal :P
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/missing-all-still/+merge/75286
<wgrant> StevenK: jtv may have a branch to remove that import.
<wgrant> StevenK: It was a temporary import of a private class, I believe.
<StevenK> "temporary" -- it's been bugging me for two days
<wgrant> lifeless: You have a 5 month old LP branch, changing the way INCOMPLETE is stored.
<wgrant> lifeless: 'tis the oldest active review.
<lifeless> wgrant: feel free to fix it and land it
<rsalveti> hey, I'm from linaro and would like to test the derived distro feature, I wonder if I have any special permission even to play with it on dogfood/staging?
<wgrant> rsalveti: What sort of thing do you want to test? Much of it still requires sysadmin handholding at this stage.
<rsalveti> wgrant: basically the idea is to start using this feature to guide the linaro-ubuntu development
<rsalveti> I'm kind of the main user for this feature from the linaro side
<rsalveti> was talking with flacoste about it, and he also pointed me bigjools to help, but don't think he's on-line now
<wgrant> Right, bigjools is the one to talk to, but he's in the UK.
<wgrant> I don't think (qa)staging have all the cron jobs set up.
<wgrant> And we don't know if derived distros actually works yet.
<wgrant> Ubuntu is using the native syncing features, but that's all that's actually known to work properly.
<rsalveti> initially I'd like to see if I'd be able to test it against staging or dogfood, but don't know yet if I can get the permission to play with it just at staging
<rsalveti> yeah, that's the main feature even for us
<Ursinha> when I was doing exploratory testing, afair bigjools was running the cronjobs manually
<rsalveti> and the idea is to start using it to see what is still missing
<wgrant> rsalveti: Well, critically we don't even know if you can create a new usable distribution.
<wgrant> I have my doubts over whether initialising it with packages actually yields a usable set of packages.
<rsalveti> ideally we would like to be able to fully use it for linaro 11.10, that's the cycle we're planning to rebase on top of oneiric
<Ursinha> rsalveti, I think that you'd need superpowers to create a distro
<wgrant> Right. We can try creating a test linaro distro on staging once spm is back, if we want.
<Ursinha> wgrant, is the feature that alpha still?
<rsalveti> cool, will move this conversation over email then
<rsalveti> wgrant: thanks for your help anyway
<Ursinha> rsalveti, spm should be back soon, I guess
<wgrant> rsalveti: That's probably best. We can get some stuff set up now by running cronjobs manually, but bigjools is the guy you want to coordinate with.
<rsalveti> Ursinha: but I'll be gone soon :-)
<wgrant> Ursinha: The syncing is tested.
<wgrant> Ursinha: Everything else is not.
<Ursinha> wgrant, hm, right
<rsalveti> wgrant: cool, he's my main poc, so good to know he's the right guy for it :-)
<Ursinha> so I believe rsalveti would do the testing :)
<rsalveti> that's the idea
<wgrant> The extent of the initialisation testing so far seems to have basically been "it doesn't crash, therefore it must be good"
<Ursinha> haha
<rsalveti> we can only test when there's someone really using it :-)
<wgrant> We created the first production test last night.
<wgrant> https://launchpad.net/bilimbitest/angry
<Ursinha> hahaha
<Ursinha> lol
<rsalveti> cool
<wgrant> We may be able to push the first builds through production tonight, if things go well.
<wgrant> rsalveti: Does Linaro rebase on the latest Ubuntu release each time we release?
<wgrant> rsalveti: Or does it merge like Ubuntu does from Debian?
<rsalveti> wgrant: both
<rsalveti> wgrant: we use the stable ubuntu release to do our main releases
<rsalveti> but at the same time we're also working against the dev release, testing our packages and then pushing directly to ubuntu
<rsalveti> but the main priority is to be on top of the latest stable release
<rsalveti> like, we expect 11.09 to be our last natty based release
<rsalveti> and do the first oneiric based one next month, together with the ubuntu release
<wgrant> I am disturbingly ignorant of how Linaro operates. Do you have a dak archive, or do you just use PPAs on top of Ubuntu?
<rsalveti> wgrant: currently we're just using a PPA
<rsalveti> and that's why we wanted the derived distro so much, working with a PPA is quite annoying
<wgrant> Right.
<rsalveti> as we can't track upstream updates
<rsalveti> and we can have bugs against packages
<rsalveti> and so on
<rsalveti> *can't
<wgrant> Yep.
<rsalveti> it's specially annoying when working against the dev release
<wgrant> So, hopefully you can talk to Julian in your morning some time.
<wgrant> Otherwise I guess there will be a lot of email and talking to people like me, who are around sometimes when you are.
<rsalveti> yup, will try to ping him directly tomorrow morning
<StevenK> wgrant: Can haz review? https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-garbo/+merge/75118
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 263 - 0:[########*** stack smashing detected ***: ./lp terminated
<jtv> StevenK: I guess I'll go to work on your branches then.  Sorry, old boy.
<StevenK> jtv: I have a missing import branch -- which you can ignore if you're going to fix it.
<jtv> StevenK: too late.
<jtv> StevenK: I'm afraid I haven't been very kind to your branches.  I was joking when I did the "sorry I'm going to review you" bit though!
<StevenK> jtv: I've done two of these migrations using garbo jobs before and have never been asked to file a bug to remember to remove them when they're done.
<jtv> StevenK: it's just a recommendationâbut I find it helps, e.g. if the code becomes a problem while you're on holiday.
<jtv> It also means that the task is explicit, and not hidden among the pile of things you know you have to do that others aren't aware of.
<StevenK> Oh, wgrant and sinzui are well aware of what I'm doing.
<wgrant> A bug would be nice.
<wgrant> Given this should be quick.
<wgrant> It can sit around assigned to you for a few days.
<jtv> It's easy to forget these things, and that's how you getâ¦ Soyuz.  :)
<wgrant> jtv: NOW you're learning.
<StevenK> Meh :-P
<jtv> wgrant: technically, I've been drinking to forget Soyuz for months now.
<kb9vqf> Quick question: the builddmaster/accepted and builddmaster/incoming directories seems to be full of files; can these be safely deleted?
<kb9vqf> also wondering about the fatsam directory--what does it do and can it be cleaned up?
<wgrant> kb9vqf: fatsam is the librarian's old name.
<wgrant> kb9vqf: That's all the librarian data.
<kb9vqf> ok
<wgrant> kb9vqf: And I thought I answered your accepted/incoming question a month or so back :)
<wgrant> kb9vqf: Incoming is the queue... stuff shouldn't be in there once process-upload.py runs.
<wgrant> If there is, you're probably not running it properly.
<wgrant> accepted you can dispose of.
<kb9vqf> wgrant: You answered my question about poppy/incoming :)
<kb9vqf> I was wondering about builddmaster/incoming
<wgrant> kb9vqf: Ah.
<wgrant> kb9vqf: Are you running process-upload.py over that dir?
<wgrant> A year ago it was run automatically by buildd-manager, but that's no longer the case.
<kb9vqf> the builddmaster directory?
<wgrant> Yes.
<kb9vqf> I don't think so
 * kb9vqf goes to check
<wgrant> Something like 'scripts/process-upload.py -C buildd --builds /srv/launchpad.net/builddmaster'
<wgrant> The '-C buildd --builds' bit is important.
<kb9vqf> As far as I can tell I am only running it over the poppy directory
<kb9vqf> unless it is in a cron script somewhere
<wgrant> It's a separate cron job now.
<kb9vqf> and it is supposed to clean up the builddmaster directory?
<wgrant> It will do that once it has processed the uploads.
<wgrant> If it's not running, your builds aren't going to do much.
<wgrant> They will just sit in the upload queue forever.
<kb9vqf> everything is working OK so I think it is running
<wgrant> What rev are you running?
 * wgrant checks.
<kb9vqf> an older rev
<kb9vqf> probably a year or so earlier
<wgrant> r10983
<kb9vqf> ok
<wgrant> June last year.
<wgrant> So.
<wgrant> That's before this change.
<kb9vqf> ok
<wgrant> That may explain why there's stuff left in incoming/. Is it mostly old?
<kb9vqf> so it is safe to delete the builddmaster/incoming directory contents then?
 * kb9vqf goes to check
<kb9vqf> yes
<wgrant> Kill them all.
<kb9vqf> delete all the files?
<kb9vqf> there are some new ones in there as well
<kb9vqf> well, newer
 * kb9vqf just wants to double check before hosing his system :)
<wgrant> With the old code, if they're not from builds that have finished in the last few minutes then they're probably never going to be removed automatically.
<wgrant> buildd-manager ran process-upload automatically once each build completing, telling it to look only at that build.
<kb9vqf> so everything in this directory already has a copy in librarian for the PPA?
<wgrant> So it won't ever look at stuff from old builds.
<wgrant> Well, I wouldn't really expect stuff to be in there. But if it made it into the PPA's archive, then yes, it's in the librarian.
<kb9vqf> I think I understand
<kb9vqf> and deleting these files would not delete any published binaries or sources
<wgrant> Right.
<wgrant> That's just the intermediate queue between the buildds and the DB/librarian.
<kb9vqf> I'm tempted to go on the safe side and only delete stuff that is a few days old or more
<kb9vqf> ok, now I understand :)
<wgrant> What sort of stuff is left that is new?
<kb9vqf> .deb files
<kb9vqf> .dsc files
<wgrant> Hmm. Odd.
<wgrant> https://dev.launchpad.net/Soyuz/TechnicalDetails is mostly brand new, and might be a helpful reference at times.
<wgrant> It's still being written.
<kb9vqf> basically the output of a builder
<kb9vqf> ok
<wgrant> lots of it from a developer's perspective, but may still be handy for your purposes.
<kb9vqf> so just to make sure I have this straight...there are two queues, one that takes uploaded source packages and one that takes built binaries from the builders?
<wgrant> Right.
<wgrant> They're both handled by process-upload.
<kb9vqf> and each queue's contents are eventually loaded into librarian?
<kb9vqf> ok
<kb9vqf> I feel much better about deleting those files :)
<wgrant> But for buildd uploads, buildd-manager runs it automatically (well, not any more, but in your code)
<wgrant> Once process-upload runs, those files are useless.
<kb9vqf> right
<kb9vqf> and since I've had the buildd manager die on me...
<kb9vqf> it might explain the leftovers
<wgrant> Yep.
<kb9vqf> ok
<kb9vqf> thanks for the help :)
<wgrant> np
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<wgrant> StevenK: Are devel/db-devel jenkins builds meant to be disabled?
<nigelb> Morning.
<wgrant> Hi nigelb.
<nigelb> Now I've fallen ill. Sigh.
<wgrant> More LP time :P
<nigelb> heh
<nigelb> Hrm, devel has not yet been merged into db-devel.
<nigelb> buildbot breakage still on-going?
<StevenK> wgrant: Yes
<StevenK> wgrant: Let me fix that
<nigelb> wgrant: How did I end up asking the exact same question about a few minutes after you.
<nigelb> Should. Get. Away. From Computer.
<StevenK> wgrant: Fix0rated
<wgrant> StevenK: Thanks.
<wgrant> nigelb: buildbot should pass in <10min.
<nigelb> \o/
<StevenK> wgrant: It was disabled in case I was dealing with config
<wgrant> Ahh.
<wgrant> buildbot passed!
<StevenK> OMG
<wgrant> Both of them.
<wgrant> Too late for fastdowntime tonight, though :(
<StevenK> wgrant: Really?
<nigelb> This is what happens when you conspire to steal FDT time :P
<wgrant> StevenK: We need staging to update.
<wgrant> nigelb: wallyworld stole my window :(
<nigelb> bwahaha.
<wgrant> An hour before he was meant to start for the day, too!
<nigelb> haha
<wgrant> So I start at 9am, thinking I'm going to get my patch in first.
<wgrant> But find that his is already landed.
<wgrant> Baaaah.
<nigelb> :D
<nigelb> I love it.
<nigelb> Hrm. db-devel branch page timing out. Known?
<wgrant> It will have just pushed.
<wgrant> And will be scanning.
<wgrant> Might time out for 30s or so.
<nigelb> AH
<wgrant> Yes, that sucks.
<nigelb> haha
<wgrant> But branchrevision is terrible.
<nigelb> "udd importer should make tea while launchpad is down"
<wgrant> Heh
<nigelb> wgrant: I wish that and loggerhead started working better.
<wgrant> Yeah.
<StevenK> s/started \(work\)ing better/\1\ed/
<StevenK> Doh, one extra \
<nigelb> "I forgot to escape something. Wheeee[taptaptap]eeeeee"
<wgrant> StevenK: You were dealing with rvba's confirmationoverlay?
<wgrant> Also, your DB patch has somehow conflicted with db-devel.
<StevenK> It has?
<wgrant> This is a bit screwed.
<StevenK> stub landed that
<wgrant> Contents conflict in database/schema/patch-2208-87-2.sql
<wgrant> Hmm.
<StevenK> My patch should win, it's *applied*
<wgrant> Yes.
<wgrant> And there's no 87-2 in stable to conflict with it.
<wgrant> But it still conflicted.
<StevenK> And it wouldn't have landed with a conflict.
<StevenK> So, WTF.
<nigelb> How can you conflict with no file?
<StevenK> Oh, database/schema/patch-2208-87-2.sql exists in db-devel
<wgrant> Contents conflict often happens when the same file is added in two different branches.
<wgrant> StevenK: Yes.
<wgrant> But it's not in stable.
<wgrant> AFAICT
<lifeless> added twice rather than added + merged ?
<StevenK> lifeless: Shoo
<wgrant> If he's here, he could lend his bzr expertise to work out WTF is going on here :P
<wgrant> 87-2 has never been in stable.
<wgrant> So how on earth is it conflicting when being merged into db-devel.
<lifeless> let me just suck down both branch tips
<poolie> hi
<poolie> i'l leave that with him
<nigelb> heh
<nigelb> WIN. Yes. 87 is not in stable.
<wgrant> Hmm.
<wgrant> I wonder if my three merges that were based on devel are problematic.
<wgrant> except that devel has been merged since then.
<wgrant> And those three are a week old anyway.
<StevenK> jtv: Can you have another look at https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-garbo/+merge/75118 ?
<wgrant> It's probably a criss-cross, but there's no warning about it, and shouldn't have been conflicts.
<wgrant> Oh.
<wgrant> There is a warning.
<wgrant> It's just drowned out by tonnes of other crap.
<wgrant> lifeless: Sorry, missed that. Fixing now.
<stub> I didn't think we could land db patches directly on devel atm?
<wgrant> stub: PQM only checks for -0 :)
<nigelb> What went wrong?
<wgrant> nigelb: A criss-cross merge.
<wgrant> We're going to be having a bit of that if buildbot keeps playing up.
<poolie> if a criss cross of addition of one file is conflicting that seems like a bzr bug
<poolie> possibly already filed
<wgrant> poolie: There was more than that, but the addition had me WTFing so I ignored the rest... and missed the criss-cross warning.
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<jtv> wgrant: mind if I throw Katie out of the house?  She's a complete mess and she doesn't help us with _anything_.  Plus, she's not tested that I can see.
<jtv> I'm talking about the class; we can deal with the person later.
<jtv> Or nowâI don't particularly care either way, as long as we clean things up.
<wgrant> jtv: Indeed, gina's katie import support hasn't been used since 2006, and probably won't be again.
<wgrant> dak's schema has changed hugely anyway.
<wgrant> Delete.
<jtv> It can't be again.  We have no proof that it will even run, let alone work.
<jtv> If we want her back, we'll have to rewrite her either way.
<wgrant> You still don't understand how Soyuz works.
<wgrant> We don't care about this proof or workingness business.
<wgrant> Run and see what blows up.
<wgrant> That's the Soyuz Wayâ¢
<jtv> I thought I was doing rather well on that with transitional domination, actually.
<wgrant> Heh
<mrevell> Hi
<jtv> hi mrevell
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
 * StevenK prods jtv more.
<jtv> StevenK: what?
<StevenK> [17:06] < StevenK> jtv: Can you have another look at https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-garbo/+merge/75118 ?
<jtv> OK
<jtv> StevenK: you kept the suspicious and apparently either unnecessary or incorrect handling of offset?
 * mwhudson is tempted to implement a branchaccesstoken thing that allows access to a specific set of branches
<mwhudson> (in my spare time)
<mwhudson> anyone want to argue me out of it?
<jtv> wgrant: ye commit hath landed.  Blocked by qa-needstesting for stub & nigelb, but I guess it's too late for today anyway..?
<jtv> mwhudson: this is an auth thing?
<wgrant> mwhudson: That would be handy.
<mwhudson> jtv: yes
<wgrant> mwhudson: Do you know about the librarian's TimeLimitedTokens?
<mwhudson> wgrant: i am aware of them, i guess would be the best thing to say
<wgrant> jtv: We can do a nodowntime today if you desire.
<wgrant> mwhudson: They provide a day of access to a specific restricted library file. Sort of similar to what you're suggesting.
<wgrant> mwhudson: Complications for branches are many, however.
<wgrant> mwhudson: We can't safely serve branches over HTTPS under their current domain, for one.
<jtv> stub: busily q/a'ing garbo-frequently?
<mwhudson> i have three use cases in mind (1) allowing the extermination of the branch puller (2) recipe builds for private branches (3) allowing other services (i.e. offspring) to access a particular private branch
<mwhudson> wgrant: i was thinking of going over ssh
<wgrant> mwhudson: Ah, that works too.
<stub> jtv: spm is setting it up everywhere now
<mwhudson> i guess i should write a spec, really
<wgrant> mwhudson: That is the approach I have always assumed we would take for private recipe builds.
<wgrant> mwhudson: And getting rid of the puller would also benefit from it.
<spm> stub: well. ish. I've started it; but can't complete today. have put it high in the XS queue
<wgrant> So, yes, good plan.
<wgrant> LEP it up!
<jtv> stub: is that still Q/A we're talking about?
<stub> spm: Production could fall over, or at least the lpapis due to oauth nonces grinding to a halt.
<stub> spm: What is the blockage?
<spm> I EOD'd about 20 mins ago? :-)
<spm> and am busy writing an incident report
<stub> spm: Right, so need to hand it over then.
<jml> jtv: can you please review https://code.launchpad.net/~jml/launchpadlib/fake-launchpad/+merge/75211
<bigjools> StevenK: http://tinyurl.com/6b2d6v6 - I'll try not to troll you any more :)
<StevenK> jtv: I will look at the offset thing tomorrow morning, then.
<jtv> jml: OKâwill go into standup soon, so may take a bit.
<mwhudson> wgrant: in general, i'm thinking explicit revocation rather than time limiting
<jtv> StevenK: any idea how it might affect performance?
<jml> jtv: no worries.
<StevenK> jtv: I'm not too concerned, to be honest. The query to find work to do is fairly lightweight, and the process is do the work is very lightweight as well.
<jtv> StevenK: then you might as well just drop the offset.  Nice & simple.
<stub> spm: I can also cobble something together for production if you push a fresh tree to wildcherry
<StevenK> jtv: Right, I'll switch to a boolean -- it will declare itself done when the find functions resultset .is_empty()
<jtv> StevenK: great.  Simpler means less to go wrong.
<StevenK> jtv: So I lied. The branch has pushed, and I'll ping you when the MP diff is updated.
<jtv> StevenK: OK, but call first.
<StevenK> jtv: Sure, then have the stand-up and then another look at the MP -- it should be done by then
<jtv> StevenK: you're done.
<StevenK> jtv: Thanks!
<nigelb> QA?
<nigelb> How do I QA that.
<jtv> nigelb: the main thing is to make sure that it's safe to roll out.
<nigelb> I don't think it will topple over. But I have no idea how to verify that.
<nigelb> bigjools: Woah. That's a fun article.
<mwhudson> wgrant: do you think https://dev.launchpad.net/LEP/BranchAccessToken misses anything vital?
<nigelb> mwhudson: What's Offspring?
<mwhudson> nigelb: image building software
<wgrant> mwhudson: I like the stakeholders section.
<mwhudson> nigelb: ah, my attempt at wiki syntax failed there
<nigelb> heh
<wgrant> mwhudson: The third story title seems wrong.
<nigelb> I did wonder if it was some sort of Launchpad codename :P
<wgrant> mwhudson: Looks pretty good, though!
<mwhudson> wgrant, nigelb: both fixed i hope
<wgrant> mwhudson: Would be nice to have similar things for OAuth tokens, though.
<poolie> Riddell, thanks for posting those build service comparisons
<mwhudson> wgrant: i thought about oauth once
<wgrant> We probably want to think about other places we may want this before we go ahead and do stuff.
<mwhudson> wgrant: i think i've recovered now
<poolie> i know jml had looked at it before but actually using it in anger (or not) obviously gives much more depth
<wgrant> ie. I don't want all my API scripts to be able to upload to Ubuntu.
<wgrant> Heh.
<jelmer_> mwhudson, being able to delete the puller, that'd be nice
<nigelb> mwhudson: Yep :)
<mwhudson> wgrant: well OAuthToken does have a "context" concept, right?
<wgrant> mwhudson: Hee hee.
<wgrant> mwhudson: It's there.
<wgrant> mwhudson: And partially implemented.
<mwhudson> i don't think anything sets or reads it though
<wgrant> But not working or enabled.
<nigelb> OAuth is PITA, but I'm not sure if there's something else that does its job.
<nigelb> Woah. There's a LEP for a Dashboard? Activity Walls?
<mwhudson> wgrant: what things do you have in mind for oauth-y stuff?  oauth is pretty http specific isn't it?
<Riddell> poolie: not to be taken as critisism of course :)
<mwhudson> and I guess we _could_ enable branch access over http
<mwhudson> but well
<wgrant> mwhudson: I don't meant doing this with OAuth.
<wgrant> mwhudson: I'd like restricted tokens for more than just branches.
<mwhudson> ah right
<wgrant> Not as part of this, but we should probably at least think about it a bit.
<wgrant> So we don't go the wrong way.
<mwhudson> i guess the another similar thing is p3as?
<wgrant> Indeed, that is not dissimilar.
<wgrant> Each token has access to exactly one archive.
<mwhudson> what else?  i think anything involving the browser is a bit different somehow
<mwhudson> (e.g. allowing someone to see a particular bug, or branch)
<wgrant> Not much at the moment.
<wgrant> But I want to be able to write an Ubuntu bug robot and give it a token that allows it to write to bugs, but not root everyone's systems.
<wgrant> For example.
<mwhudson> yeah
<mwhudson> however
<mwhudson> i don't want to invent a schema that can describe both "all bugs on a particular distribution" and "a finite set of branches"
<wgrant> Certainly.
<mwhudson> that's not so much a rabbit hole as an open cast mine
 * mwhudson looks at translatePath, dies a little inside
<wgrant> Yeah, I thought you'd know not go to look at it without protection.
<mwhudson> one forgets things
<mwhudson> trying to do this by creating a new kind of principal is insane, right?
<wgrant> I expect that would be somewhat insane.
<mwhudson> yeah
<mwhudson> jelmer_: yes indeed
<mwhudson> jelmer_: it's tests too :)
<mwhudson> wow 'make' seems to be taking a preposterously long time
<nigelb> "seems"?
<mwhudson> ok, is
<nigelb> make run?
<mwhudson> just 'make'
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<mwhudson> it's creating wadl now
<nigelb> Everytime I touch javascript, I cringe about the amount of time make is going to take.
<nigelb> wgrant had a nice hack for the that, which I keep forgetting :(
<wgrant> make jsbuild
<lifeless> mwhudson: how is a branch access token different to an ssh key?
<nigelb> Hrm. My YUI test HTML doens't have styling.
<mwhudson> lifeless: it doesn't correspond to a Perosn
<jml> jtv: I guess you didn't get around to that review?
<jtv> jml: you just tore me away from it.  :)
<jml> jtv: oh, ok, thanks.
<nigelb> The css for javascript test is not getting loaded. Debugged for 20 minutes and still no clue. Anyone has suggestions?
<nigelb> Well, it is loaded. But some of it is not getting used.
<nigelb> aaaaaah.
<nigelb> the wiki is wrong.
<nigelb> <-- *headdesk*
<lifeless> mwhudson: I think we should talk about what problem you are addressing - as it stands, it has me a little worried about complexity, protocols, compatibility, auditing and security.
<jtv> jml: it was too much for one go, I'm afraid.  Posted notes for all but the tests; more tomorrow.
<jml> :(
<jml> jtv-afk: thanks.
<jml> jkakar: jtv has asked some questions in his review that maybe you'd be better placed to answer: https://code.launchpad.net/~jml/launchpadlib/fake-launchpad/+merge/75211
<nigelb> jml: Wow. 5000 line diff.  o_O
<jml> nigelb: not really.
<nigelb> 40,425.
<nigelb> Oh.
 * nigelb bows.
<jml> nigelb: it's mostly changing data files
<nigelb> jml: Ahh.
<deryck> henninge, ping for standup
<henninge> deryck: oh, sorry
<deryck> henninge, we're just finishing up, if you don't want to join
* abentley changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<jml> gmb: can I convince you to take a look at that branch again? jtv has reviewed all but the tests.
<gmb> jml: I've already scheduled some time for it this afternoon.
<jml> gmb: thanks.
<jkakar> jml: I think you did a great job answering jtv-afk's awesome review comments.  Thanks! :)
<jml> jkakar: thanks.
<jml> jkakar: I've been learning about wadl on the way.
<adeuring> abentley: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/bug-739052-8/+merge/75372 ?
<abentley> adeuring: sure.
<adeuring> thanks!
<abentley> adeuring: I'd be inclined to move most of the body of rough_length into a generic function, but perhaps you feel it's too early for that?
<adeuring> abentley: I thought about that too. But to be honest: I simply want to fix the main bug. Working on it took already too long ;)
<abentley> adeuring: Fine with me.  r=me.
<adeuring> abentley: great, thanks!
<gmb> jml: r=me on the tests; I'll leave jtv-afk to reply to your response to his comments. A couple more copyright notices that need updating, too (is there a template that needs changing somewhere?)
<jml> gmb: I don't know re template. As I might not have said, I'm just updating an old branch. :)
<gmb> jml: You may have, and I've probably missed it. Fair dos.
<nigelb> abentley: Hi! Could you land a branch for me into db-devel?
<abentley> nigelb: sorry, give me a couple minutes.
<nigelb> Cool :)
<abentley> nigelb: which branch?
<nigelb> abentley: https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/74934
<nigelb> The last time I missed a few places the field was being used.  Fixed it in devel and merged it down.
<abentley> nigelb: Okay, let me just snag a copy here...
<abentley> nigelb: landed.
<nigelb> abentley: Thanks!
<dobey> hrmm
<dobey> https://lp-oops.canonical.com/?oopsid=OOPS-2083J68
<dobey> that does not look fun :(
<dobey> can someone tell me what that means ^^?
<dobey> ShortListTooBigError: Hard limit of 1000 exceeded.
<salgado> dobey, I think it means someone expected a DB query wouldn't return more than 1000 items, but it did
<dobey> hmm. how can i work around this and let my bot request builds of that recipe?
<cjwatson> wgrant: fixed apt is on mawson now, if you'd care to try it ouot?
<cjwatson> *out
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: sure.
<jcsackett> sinzui: http://people.canonical.com/~curtis/target-picker/target-picker-3.html
<nigelb> Hi, could someone help me with mockio?
<nigelb> var mock_io = Y.lp.testing.mockio.MockIo();
<nigelb> The mock_io variable is undefined for me.
<nigelb> Included the mockio.js, declared it in YUI.use as well.
<nigelb> Ah. Again, wrong example. *fixes*
<nigelb> I made changes to mockio page and javascript unit testing page.
<nigelb> https://dev.launchpad.net/JavascriptUnitTesting/MockIo?action=diff&rev2=7&rev1=6 https://dev.launchpad.net/JavascriptUnitTesting?action=diff
<nigelb> If anyone wants to check
<abentley> nigelb: thanks for the fix.  Darn JS.
* abentley changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<mwhudson> well this was unexpected
 * mwhudson has implemented anonymous access to public branches over ssh
<james_w> the question is...what were you aiming to do? :-)
<mwhudson> james_w: well, i was thinking about https://dev.launchpad.net/LEP/BranchAccessToken
<cody-somerville> mwhudson, how do you do anonymous access to public branches over ssh?
<mwhudson> cody-somerville: fiddling with how the ssh server does authentication
<cody-somerville> is the benefit the ability to use the smart server to make things faster vs. over http(s)?
<mwhudson> yeah
<maxb> Anonymous ssh sounds weird, but there's a fairly large precedent for it in the BSD world as an anoncvs transport, IIUC
<mwhudson> yeahhttps://code.launchpad.net/~mwhudson/launchpad/anon-ssh-hack/+merge/75442
<mwhudson> oops
<mwhudson> https://code.launchpad.net/~mwhudson/launchpad/anon-ssh-hack/+merge/75442
<StevenK> G: You have QA, r13938, bug 763820
<_mup_> Bug #763820: double "with" on +editpgpkeys <qa-needstesting> <trivial> <ui> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/763820 >
<StevenK> nigelb: You have QA as well! r13932, bug 88545
<_mup_> Bug #88545: Abolish the "statusexplanation" database field <lp-bugs> <qa-needstesting> <tech-debt> <Launchpad itself:In Progress by nigelbabu> < https://launchpad.net/bugs/88545 >
<G> StevenK: thanks for the reminder :)
<G> mwhudson: if you ask me, I think it's genius :)
<mwhudson> G: heh, thanks
<G> I see another use for it too...
<mwhudson> G: whereabouts in nz are you btw?
<G> If at first you don't succeed, try, try again (as anonymous)   i.e. if a "bzr branch" or pull fails because of missing public key, give it a go as anonymous because there is a good chance there may be read-only access
<G> errr missing private key that should be
<G> mwhudson: West Auckland (I'm a JAFA)
<mwhudson> ah right
 * mwhudson is in wellington
<mwhudson> (but a brit)
<G> StevenK: done
<StevenK> G: Thanks!
<G> my logic is... if I say login to my VM, and do a rocketfuel-get or something and forget to invoke the SSH Agent, because the launchpad repo is world-readable, it'd fall back on the anonymous user, and still work
<wgrant> nigelb: That shouldn't have been landed yet :(
<wgrant> nigelb: The prereq is not deployed to anywhere yet, and because it affects oops-prune it needs to be deployed absolutely everywhere.
<G> to me, something like would be a great idea as far as usability
<G> (I'll let others come up w/ reasons why I'm insane for thinking that ;))
#launchpad-dev 2011-09-15
 * StevenK nails _mup_ to the channel.
<wgrant> cjwatson: That looks good.
<wgrant> cjwatson: indices look as expected.
<cjwatson> great.  where should I look on mawson?
<cjwatson> ah, deribuntu/baleful by the looks of things
<wgrant> cjwatson: Oh, forgot you had access to mawson.
<cjwatson> agreed, that looks cromulent to me
<wgrant> cjwatson: Indeed, that's the one I tried. As we don't have any oneiric on there at the moment.
<cjwatson> admittedly only a single package but should be fine
<cjwatson> I think I whined a few years back until I got it
<cjwatson> I wonder why Translations-en is only uncompressed + gzip
<cjwatson> maybe should add bzip2 versions at some point, but we'll see how things look in production
<cjwatson> ah, need to set Translation::Compress for that
<cjwatson> OK.  I think this is good though.  Shall I tell lamont to roll that out to production?
<wgrant> I can throw in a few more packages if you want, but I think this seems to work.
<wgrant> Please do.
<wgrant> Even if it does happen to break on prod, it's going to be pretty obvious within two hours.
<cjwatson> I'm fine with that.  I'll be watching shortly after I flick the switch anyway
<wgrant> Yep.
<cjwatson> RTificated
<wgrant> Ah, you exposed it through the API?
<wgrant> Handy.
<cjwatson> Yup.  You suggested that. :-)
<lifeless> wgrant: you're rolling back the db-devel patch ?
<lifeless> wgrant: what revno is being reverted?
<wgrant> lifeless: Yes, just checking up on what qa-tagger is going to do.
<wgrant> 10978
<lifeless> mail sent about it
<wgrant> Thanks.
<wgrant> There seems to be a lot of inclarity over how things work now.
<lifeless> wgrant: feel free to update the docs everytime someone is confused ;)
<poolie> wgrant, is there any more of a traceback for https://bugs.launchpad.net/launchpad/+bug/847485 ?
<_mup_> Bug #847485: process-mail.py crashing with Unicode logging errors <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/847485 >
<poolie> it might have been my change that provoked it
<wgrant> poolie: There isn't.
<wgrant> poolie: I was hoping maintenance squads would intervene on Monday.
<wgrant> But nobody did :(
<poolie> that's strange
<poolie> that there is just a one entry traceback
<wgrant> Rather.
<StevenK> Hah, it has changed
<StevenK> PircBot 1.4.6 Java IRC Bot versus PircBotX 1.5, a fork of PircBot, the Java IRC bot
<lifeless> what has changed ?
<StevenK> I upgraded Jenkins and then updated all of the plugins
<wgrant> StevenK: Is canonistack letting you in yet?
<StevenK> No
<nigelb> wgrant: BAH! I didn't know :(
<wgrant> nigelb: 'tis rolled back and I stopped it getting onto staging, no harm done.
<nigelb> Sorry!
<nigelb> StevenK: As I said the other day, I don' know how to QA it.
<wgrant> nigelb: It's pretty difficult to QA it directly.
<wgrant> So I poked around a bit and said it was qa-ok.
<lifeless> nigelb: I've tweaked https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<wgrant> Thanks lifeless.
<lifeless> nigelb: can you tell me if step 6 is clear enough to you?
<nigelb> lifeless: Reading
<nigelb> It is :-)
<nigelb> wallyworld: I tweaked the javascript unit testing and mockio pages after a few hours of headdesk. There were small typos in the code for both.
<nigelb> wgrant: Ah, is that why I don't have a success mail? :)
<wallyworld> nigelb: hopefully your head doesn't hurt too much :-) is it ready for a formal review?
<nigelb> No, not yet :)
<nigelb> I want to write more testss
<wallyworld> more tests = good :-)
<StevenK> wallyworld!
<StevenK> wallyworld: Your presence is *required*
<nigelb> That is not good :P
<wallyworld> StevenK: hello!
<StevenK> wallyworld: Can you log into Jenkins using SSO -- check the box for team membership and see if you get buttons on the right hand side
<StevenK> nigelb: ^
<StevenK> nigelb: You shouldn't get the box and no buttons, but you should be able to log in
<nigelb> we dont sleep dot org?
<wallyworld> StevenK: i forgot the url
<nigelb> https://lpci.wedontsleep.org
<StevenK> That's it
<wallyworld> ok, i can see a jenkins console
<nigelb> What box am I looking for?
<StevenK> wallyworld: So it says "wallyworld | log out" on the top right?
<wallyworld> StevenK: yes
<StevenK> wallyworld: Are there two icons on the far right of devel and db-devel that say "Schedule a build" when you hover on them?
 * wallyworld looks
<wallyworld> yes
<StevenK> wallyworld: Excellent, thanks
<wallyworld> np
<StevenK> nigelb: Are you logged in?
<nigelb> StevenK: http://people.ubuntu.com/~nigelbabu/jenkins.png
<StevenK> Excellent
<StevenK> Thanks
<nigelb> Did you write java code for this?
<wgrant> StevenK: You might want to get it authorized to get launchpad membership by default.
<nigelb> If so, respect.
<StevenK> Doesn't that require an RT?
<StevenK> nigelb: I did not
<nigelb> It requires an RT
<nigelb> and probably grabbing stuart
<nigelb> g26
<wgrant> A LOSA may be able to JFDI for you. May not require ISD intervention these days.
<poolie> wgrant, lifeless, is it ok if i send up my bug 643223 branch to ec2?
<_mup_> Bug #643223: should accept dkim based on from address and signing address belonging to the same person <dkim> <lp-foundations> <mail> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/643223 >
<wgrant> poolie: Sure.
<jtv> Reviewer wanted: https://code.launchpad.net/~jtv/launchpad/katie-and-gina-are-bad-bad-girls/+merge/75472
<wgrant> jtv: I can't review right now, but FWIW the katie celebrity is unrelated to gina.
<jtv> But is it related to katie.py?
<wgrant> Well, related only in that the celebrity represents katie for historical reasons, and gina's katie module was used to talk to katie.
<wgrant> gina's katie.py is not katie.
<wgrant> katie is part of dak.
<wgrant> gina's katie.py reads dak's DB.
<jtv> *cry*
<wgrant> Yes
<jtv> Oh well, it's not a concern for my branch then.
<jtv> That's something: Katie will at least mean Katie, not frontend for Dak.
<poolie> where should a script for developer-only use go
<poolie> specifically process-one-mail
<poolie> in scripts? utilities?
<poolie> well, perhaps losas will find it useful, but it's not for production
<wgrant> utilities, probably.
<wgrant> But maybe scripts...
<poolie> well, i'll leave it, someone can move it
<poolie> no one complained in review
<poolie> hm, what would be a clean way to let this hook into sendmail() so that what's going to be sent is instead printed
<poolie> i can easily imagine how to do it by, eg, monkeypatching, or adding a new configuration option
<poolie> i guess config is done for the special during-testing code, so perhaps i should do the equivalent?
<wgrant> poolie: It's... not config.
<StevenK> wallyworld!
<wallyworld> yo
<wallyworld> you rang
<wgrant> poolie: lib/lp/services/mail/sendmail.py, search for isZopeless.
<wgrant> poolie: That's the right part of the code.
<wgrant> poolie: You can see how it handles testing there.
<StevenK> wallyworld: var co = new Y.lp.app.confirmoverlay.ConfirmationOverlay({
<wgrant> poolie: (you may need to consult a medical professional afterwards)
<poolie> i do
<poolie> the simplest thing that would possibly work is just to stick another variable on config
<StevenK> Sigh, never mind, I see the error
<StevenK> It's confirmationoverlay
<wallyworld> excellent! pleased to help
<poolie> the results are awesome though
<wallyworld> :-)
<StevenK> wallyworld: JS has the 77 char limit?
<wallyworld> StevenK: sadly yes
<wallyworld> imho, in 2011, 78 chars is mental
<wallyworld> with wide screen monitors etc
<wallyworld> should be at least 120
<StevenK> wallyworld: http://pastebin.ubuntu.com/689734/
<StevenK> wallyworld: Suggestions how to break that up?
 * wallyworld looks
 * StevenK would like to say "Where's my hammer?" in a Jeremy Clarkson voice.
<wallyworld> StevenK: var ns = Y.lp.app.confirmationoverlay;
<wallyworld> var overlay = new ns.ConfoimationOverlay()
<wallyworld> or something like that
<wallyworld> where ns is short for namespace
<StevenK> Obviously
<StevenK> :-)
<StevenK> wallyworld: Where is your cape? :-P
<wallyworld> in the wash, it;s dirty :-)
<StevenK> From overuse, I bet
<wallyworld> you don't want to know :-)
<wallyworld> the Wonder Woman costume is also dirty :-P
 * StevenK scratches his own eyes out
<StevenK> CAN NOT UNSEE
<StevenK> Can I tell Firebug to jump to a line number in a JS file?
<wallyworld> StevenK: not sure. i usually just search for the loc
<StevenK> Oh, look, it has search
<wallyworld> you can type pretty well with no eyes :-)
<StevenK> I installed a screen reader, duh
<StevenK> And I don't need to look at the keyboard
<StevenK> wallyworld: Can haz mumble?
<wallyworld> StevenK: ok. just a sec. gotta plug in mic
<nigelb> jtv: katie and gina are bad girls? BWAHAHA
<jtv> Bad, bad girls.
<nigelb> *now* I know why lifeless had included "Don't be cute" with names for services :P
<lifeless> not to mention e.g. roomba
<nigelb> There is a part of launchpad called roomba?
<nigelb> Does it suck? :P
<lifeless> there was
<wgrant> That's news to me.
<wgrant> What was it?
<wgrant> librarian-gc?
<lifeless> import related
<lifeless> see circa 2006
<lifeless> there was another, named after one of the other automatic vaccums
<lifeless> we've done some terrible things
<nigelb> what's the origin of "gina"?
<lifeless> dak
<lifeless> ish
<wgrant> Yeah, it's following the dak naming scheme.
<wgrant> I forget who exactly it is named after.
<wgrant> I think dak's source says...
<nigelb> What is dak again?
<nigelb> VCS?
<wgrant> dak is Debian's archive software.
<wgrant> Like Soyuz, except even more of a trainwreck :P
<nigelb> heh
<wgrant> Blah, elmo deleted README.names in 2006.
<wgrant> Bad elmo.
<nigelb> heh
<wgrant> "Gina (Gershon)" is listed as a future name. 18 months after gina was written :(
<StevenK> wgrant: Land demolish-unused-tables-3-db!
<wgrant> StevenK: Can't.
<wgrant> StevenK: -2 is not deployed to loganberry/ackee yet.
<wgrant> And can't be, because garbo-frequently isn't alive yet.
<wgrant> Because puppet.
<nigelb> What is the "correct" way to split long javascript strings?
<nigelb> long strings in javascript rather.
<lifeless> nigelb: 'don't' ?
<nigelb> oh, lines of length 268 in js is okay?
<lifeless> well depends what you mean
<lifeless> js source code? no, our normal rules apply.
<lifeless> js sent to the browser? see the various compressors out there.
<nigelb> https://code.launchpad.net/%7Enigelbabu/launchpad/bug-title-849121/+merge/75267 (line 189)
<nigelb> Its a test
<nigelb> (also, why I can't I href to those lines :/)
<mrevell> Good morning 'padders
<rvba> Morning all, morning mrevell.
<nigelb> Morning mrevell / rvba :)
<rvba> Hey nigelb.
<mrevell> Salut rvba! Qu'est-ce qu'i ce pass en France? (/me realises how much he's forgotten)
<nigelb> rvba: Reviewing today? I *may* be able to get you something
<mrevell> Namaste nigelb ... I might as well try and do a bad job of greeting everyone in something approaching their local manner :)
<nigelb> mrevell: Haha
<mrevell> heh
<rvba> mrevell: Not so bad actually ;). Well, the rugby cup is on. Oh, and the economy is collapsing.
<rvba> nigelb: Okay.
<poolie> hello rvba
<mrevell> rvba, The first one helps us forget the second.
<rvba> True ;)
<mrevell> heh
<rvba> poolie: Hi!
<poolie> could someone read https://code.launchpad.net/~mbp/launchpad/mail-script/+merge/75488 for me?
<poolie> it's pretty small
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (allenap) | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<rvba> poolie: I can do that but will have to wait a bit for Gavin to double check that my review is ok.
<poolie> that's fine, and it would be appreciated
<rvba> Okay then, I'm on it.
<rvba> poolie: review done.  I'll ping Gavin (my mentor) when he logs in to double check what is actually my first review on lp ;).
<poolie> awesome
<adeuring> good morning
<allenap> rvba, poolie: I'll do that now.
<jtv> henninge: while you're still here, are you free for a pre-imp call in an hour or so?  For old time's sake.  âº
<henninge> jtv: yes, please! ;)
<jtv> \o/
<henninge> jtv: just ping me
<jtv> Will do.
<jtv> Tu'ich!
<poolie> thanks allenap
<jtv> henninge: ping :)
<jtv> henninge: can I find you on mumble somewhere?
<henninge> jtv: In the orange room. Gimme 2 mins.
<jtv> OK
<jml> jtv: hello. I've made the fix you required. What happens next?
<jtv> jml: otp, but if that's fixed, I'll approve right now
<jml> jtv: thanks. could you also land it please? I don't have commit access.
<jtv> !
<jtv> OK
<nigelb> How do I get make lint to 'see' files?
<nigelb> I commited something before linting and now make lint doesn't see it.
<jtv> nigelb: I'll explain in a moment, hang on
<jtv> nigelb: moment has passed.  âmake lintâ looks for files to check in stages:
<jtv> If any files have uncommitted changes, it assumes you want to check those.
<jtv> If no files have uncommitted changes, it looks for changes compared to the parent branch, and if there are any, assumes you want to check those.
<jtv> If it doesn't find any changes at all, it checks everything.
<jtv> nigelb: so try running it right after a commit.
<nigelb> okay :)
<nigelb> I did the hack of making whitespace changes
<nigelb> so it found those :D
<rvba> nigelb: You probably also can shelve your current changes for a moment.
<nigelb> Lint seems to think i should use === instead of == in javascript.
<wgrant> It's right. == is almost never what you want. It does type coersion.
<nigelb> Dear god.
<nigelb> That's what its been doing all this while.
<nigelb> Also, <3 YUI tests now.
<nigelb> Easy to run 'em.
 * nigelb curses lint.
<nigelb> Erm, help. I have this huge string in a line of javascript (for a test), how do I split it?
<nigelb> 'foo' + 'bar' + 'baz'?
<jml> nigelb: think so.
<nigelb> jml: Thanks.
<nigelb> its going to end up looking ugly :P
<rvba> nigelb: if the string is big but not huge, you can use the 'array trick' (see lib/lp/soyuz/javascript/tests/archivesubscribers_index.js)
<rvba> nigelb: if the string is really huge and is html code, the right place for this is the .html file associated with the js test code.
<rvba> nigelb: for an example of that, bzr grep derivedtd-template.
<nigelb> let me check out the array trick :-)
<nigelb> Oh, the array trick is *neat*
<nigelb> rvba: Can I add https://code.launchpad.net/~nigelbabu/launchpad/bug-title-849121/+merge/75267 to your list? :)
<rvba> nigelb: Sure.
<nigelb> Yay for a branch where I knew most of what I wanted to do :-)
<jml> jtv: any idea when that branch is going to land? I don't actually know the landing process.
<jtv> jml: I'm having some trouble landing it myself.
<jtv> jml: it looks like âbin/ec2 landâ won't do it; wants a GPG key I don't have.  Do we have anyone who's more familiar with the project?
<jml> jtv: benji, I think.
<wgrant> which project?
<wgrant> launchpadlib?
<jml> wgrant: yes.
<jml> personally, I would like to have landing and release processes for launchpadlib & other lazr projects in the source trees
<jml> (because guess what I'm going to ask for after landing things?)
<wgrant> https://dev.launchpad.net/HackingLazrLibraries
<wgrant> It's not PQM-managed, let alone supported by ec2test :)
<jml> ahh, thanks.
<jml> https://code.launchpad.net/~jml/launchpadlib/hacking-documentation/+merge/75520
<jtv> jml: how do I run the tests?
<jml> jtv: buildout, then ./bin/test
<jtv> Thanks.
<jtv> How do I get buildout?
<jml> jtv: it's documented here: <https://dev.launchpad.net/HackingLazrLibraries>. Essentially, 'python bootstrap.py' and then './bin/buildout'.
<jml> I find the dev wiki hard to read. I think it's the grey text.
<jtv> It's also a lot of work just to run tests.
<jtv> âaÂ non-system Python, or a virtualenv executable that does not include site-packages,Â is highly recommended, and may be requiredâ
<jtv> âDownload error: [Errno -2] Name or service not known -- Some packages may not be found!â
<jtv> May be normal.
<wgrant> jml: There's a bug for that.
<jml> wgrant: yeah. I might have filed it.
<jml> jtv: that's the glory of buildout. If it's any consolation, it's even more work just to run the tests for Launchpad itself.
<jml> jtv: that's not normal.
<jtv> Well, the tests ran.
<wgrant> Did they pass too?
<wgrant> jml: Bug #666143 is for the blog, which is a similar theme.
<_mup_> Bug #666143: Launchpad blog uses grey text <css> <lp-web> <Launchpad itself:Won't Fix> <Launchpad Blog WordPress Theme:Triaged> < https://launchpad.net/bugs/666143 >
<wgrant> Hah, closed.
<jtv> wgrant: yes
<wgrant> Oops, still open for the theme.
<jtv> jml: landed.
<jml> jtv: thanks!
<jml> jtv: could I trouble you to review the hacking documentation branch I just put up? https://code.launchpad.net/~jml/launchpadlib/hacking-documentation/+merge/75520
<jml> jtv: it's waffer thin
<jtv> jml: not tonight, sorry.
<jml> jtv: no worries. thanks.
<jml> rvba: could I trouble you to review this branch? https://code.launchpad.net/~jml/launchpadlib/hacking-documentation/+merge/75520 I'd land it myself sans review if I had commit access.
<rvba> jml: Sure. I'll take care of it after lunch.
<jml> rvba: thanks.
<nigelb> wgrant: How do I turn on feature flags?
<nigelb> You told me a while back and now I forget :/
<wgrant> nigelb: https://launchpad.dev/+feature-rules
<nigelb> Thanks!
<nigelb> Now I need to figure out which was the feature Julian and I were confused about.
<nigelb> What's priority?
<nigelb> (in the context of feature flags)
<wgrant> nigelb: The highest priority matching rule wins.
<wgrant> So, say I have 'default' and 'pageid:Person:+index' rules for a particular flag.
<nigelb> what's wrong with this line?
<nigelb> (code.incremental_diffs.enabled, default$, 1, true)
<wgrant> default$?
<wgrant> What's that %?
<wgrant> $
<nigelb> from the https://launchpad.dev/+feature-info page.
<nigelb> default$ 	
<nigelb> The default scope.  Always active.
<wgrant> Ah, that's a bit confusing.
<wgrant> That describes the regular expression that is used to match the scope in the rule.
<wgrant> so the rule would be something like 'code.incremental_diffs.enabled default 1 true'
<nigelb> GAH
<nigelb> On each line: (flag, scope, priority, value),
<nigelb> How confusing.
<jelmer> rvba: hi, can I add a small merge proposal to your queue?
<rvba> jelmer: Yes you can ;). You'll be number 3. (I'm working on number 1)
<jelmer> rvba: thanks!
<rvba> np
<jkakar> jml: You merged it!  AWESOME! :)
<jml> jkakar: :D
<jkakar> jml: Man, that feels so good... that's the longest living branch I've ever had. :)
<jkakar> Thanks for picking it up and getting it landed, much appreciated.
<nigelb> what do I do to see a diff in an MP for local codehosting
<jml> jkakar: my pleasure
<jml> jkakar: I know the feeling. I personally resent every unlanded branch I have.
<jkakar> jml: Yep, me too. :)
<wgrant> nigelb: 'make sync_branches' should scan the branches and generate requested MP diffs.
<nigelb> wgrant: I just realized that branch didn't have contnet.
<nigelb> This means I have to setup local codehostign to fix a bug I can't even see! :P
<wgrant> Heh.
<wgrant> make run_codehosting
<wgrant> Add http://paste.ubuntu.com/689936/ to your ~/.ssh/config
<G> or the make run all option (it's run_all right?)
<wgrant> utilities/make-lp-user nigelb
<wgrant> bzr push lp://dev/~nigelb/+junk/something
<wgrant> G: Or that, yeah.
<G> https://dev.launchpad.net/Code/HowToUseCodehostingLocally describes it all
<nigelb> I did see that.
<nigelb> But I was trying to figure out the effort of fixing a bug I can't see :p
<nigelb> (I still can't see it, until I go through all the hoops!)
<nigelb> wgrant: heh, a bug you filled seems to be a dup of one I filed
 * nigelb marks
<nigelb> bug 847556 vs bug 847682
<_mup_> Bug #847556: Person picker in the MP ignored the fact that I filled out a review type <Launchpad itself:Triaged> < https://launchpad.net/bugs/847556 >
<_mup_> Bug #847682: Can no longer set review type in "Request another review" person picker <disclosure> <person-picker> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/847682 >
<wgrant> ni	Bah, sorry.
<wgrant> Why does irssi do that...
<nigelb> I'm duping mine to yours
<G> wgrant: too quick w/ the tab & typing and latency I find
<nigelb> wgrant: You're is marked higher than the one I field :D
<nigelb> *filed
<wgrant> G: Yeah, irssi is currently running on a slow ARMv5 machine which is also a file+print+mail server, so the load sometimes gets a bit high and it lags.
<lifeless> nigelb: generally we dup onto the lower bug #
<wgrant> G: And about 18 months ago it started interpreting tabs literally during high lag.
<wgrant> It's annoyinfuriating.
<wgrant> Haha
<wgrant> There we go again.
<nigelb> His is marked critical, mine is low.
<nigelb> and has the right tags :)
<wgrant> How did that get marked low :/
<G> lifeless: an unnamed bug tracker & project I used to work with had the policy, dupe to the 'best' bug :)
<nigelb> That's my policy most of the time :)
<lifeless> wgrant: irssi and tabs with lag... thats been like that for many many years
<wgrant> lifeless: Hm. Only noticed it in the last couple.
<wgrant> Odd.
<G> yeah, it used to happen a lot on my Linode
<lifeless> wgrant: its the paste detection heuristic
<wgrant> Perhaps load has just increased too much.
<wgrant> May need to offload some stuff to a pandaboard.
<nigelb> wgrant: ARM macine? Your phone? :P
<wgrant> But they are no good for routers and fileservers :(
<lifeless> wgrant: http://irssi.org/about under 'paste detection'
<G> wgrant: they aren't what a pity
<lifeless> wgrant: its retarded but upstream consider it a feature
<wgrant> lifeless: That is indeed reasonably retarded.
<lifeless> if you have less text than the multi-line threshold, and faster-text than the input buffer heuristic, what you see happening will happen
<nigelb> GAH. I have just wondered why patches have an edit sprite. Despite me adding them.
<G> lifeless: can't you turn off?  (I thought you could)
<lifeless> http://irssi.org/documentation/settings paste_detect_time = 5msecs
<lifeless> Irssi will detect pastes when your input has less than this much time between lines.
<lifeless> except its a lie
<lifeless> its not line based ;)
<G> ahh you can yeah
<wgrant> paste_detect_time = pleasedie
<nigelb> heh
<lifeless> and/or paste_detect_keycount = 5
<nigelb> wgrant: alternatively, set your irssi as a bouncer :)
<lifeless> I think its the latter for the tab breakage
<lifeless> note the special special special docs about it on that page
<G> wgrant: I guess the other solution is to write a script to detect it happening ;)
<lifeless> night all
<G> lifeless: have a good one
<nigelb> night lifeless
<wgrant> Night lifeless.
<nigelb> Is bug 845339 already?
<_mup_> Bug #845339: Incorrect sorting on "Last Modified" or "Last Commit" in Code page <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/845339 >
<nigelb> Not able to reproduce it in production
<wgrant> nigelb: It depends on the browser.
<nigelb> Firefox 8 works.
<nigelb> Ohyes. Chrome.
<wgrant> Still reproducible there?
<nigelb> yup
<nigelb> Hrm, everyone has taken jtv's email to heart! :-)
<nigelb> rvba: Hi, I didn't understand the first bit
<rvba> nigelb: I think you should keep the logic that was in place before your change and create the list of the invalid bugs like it was done before instead of building it like you do now.
<rvba> nigelb: invalid = set(bugs) - set(bug_ids)
<nigelb> I can't do that anymore
<nigelb> since I don't get bugids
<nigelb> I get bugtask objects instead
<nigelb> so, doing that means looping through the list again.
 * rvba checks again.
<nigelb> (interestingly, that logic was written by me as well)
<rvba> nigelb: why do you need to deal with bugtasks instead of bugs?
<nigelb> rvba: to get the bug title
<nigelb> earlier I was just getting bug ID
<rvba> I see.
<rvba> nigelb: I guess I'll let you get aways with this if you add a comment saying that we're left with the invalid bugs before the second loop ;)
<rvba> s/aways/away/
<nigelb> rvba: heh, okay :)
<nigelb> I hope I didn't make your eyes bleed with that JSON ;)
<nigelb> s/bleed/bleed too much/g
<rvba> my eysdqsd sqdd sf.
<rvba> (I can't see the keyboard anymore)
<nigelb> haha
<rvba> ;)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (allenap), jcsackett | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<rvba> Hi jcsackett!
<jcsackett> good morning, rvba. :-)
<jcsackett> or good afternoon, in your tz.
<rvba> Afternoon, right, good morning to you ;)
<jcsackett> timezones are too much to deal with on low levels of coffee. :-)
<rvba> hehe
<rvba> jcsackett: LP seems to have a problem generating diffs atm.
<StevenK> merge-proposal-jobs failed to run
<jcsackett> StevenK: someone on maintenance looking at that?
<soren> This has been a fascination of mine for a while.. It would seem natural that inhabitants of a country that spans 6 timezones were able to deal with timezones literally in their sleep :)
<jcsackett> or does it just appear to be one off death?
<wgrant> jcsackett, rvba: See #-ops
<jcsackett> wgrant: looking in it now.
<G> soren: it's my hope that we ditch TZs all together ;)
<jml> soren: they are funny timezones though
<jml> soren: I mean, they call their west coast timezone, "Pacific time", as if the great big blue wobbly thing outside their windows stretched only a mile out from the coastline
<jml> hey, I just learnt a thing about source package publishing
<jml> which is that SPPH.sourceFileUrls() will give you some interesting stuff that's not the same for every package.
<james_w> that's links to the .dsc .orig.tar.gz etc?
<james_w> or whatever makes up that package?
<jml> james_w: yeah. sometimes there's a .debian.tar.gz (e.g. w/ gtk+2.0-0), other times just a diff.
<james_w> yeah
<james_w> the easy way of reliably getting it is the slow one
<james_w> fetch them all and call dpkg-source -x passing the .dsc file with them all in the same dir
<james_w> then grab the debian/ dir from the resulting directory
<jml> james_w: *awesome*
<james_w> you could write something 90% that extracts what you want from the debian.tar.gz or the diff.gz
<james_w> but if you want 100% that's the easy way to do it
<jml> james_w: I want 100%. Have too many other 90% things in this. They multiply and get smaller pretty quickly.
<james_w> yeah
<james_w> actually, I'm not experienced with it, but just looking in debian.tar.gz if one is there may be 100% for that subset
<james_w> and if there is no diff.gz then there is only one tarball to look in, so that's 100%
<james_w> so it's only the diff.gz case that is a bit tricky
<james_w> but given that maybe just coding one thing is the easiest for now
<jml> +1
<jml> james_w: did you not have to solve this for the udd importer?
<james_w> jml, we need the whole thing for that
<james_w> so I never looked for optimizations
<james_w> but yeah, it has code to download the files and extract them
<james_w> maybe split across two different places
<jml> heh
<james_w> import_package.py:dget
<james_w> import_dsc.py:SourceExtractor and subclasses in bzr-builddeb
<james_w> it doesn't use that API you mention
<james_w> and I'm not sure if it's going to be directly applicable to what you want
<james_w> the extraction probably does more than you need
<nigelb> rvba: I'm fairly sure I updated that MP with the changes you requested. I suspect something broke with diff generation, so you can't review it for a bit anyway.
<rvba> nigelb: indeed, wgrant is working on it.
<wgrant> Well, I'd prefer to drop it on abentley or a maintenance person :)
<wgrant> s/a/another/
<rvba> nigelb: I'll grab the branch directly.
<nigelb> \o/
<jml> james_w: thanks.
<nigelb> wgrant: do you have a quick minute to point me in the right direction for bug 843415?
<_mup_> Bug #843415: Can't assign people on bugs that affect many projects <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/843415 >
<wgrant> nigelb: Do you feel like optiming all of lazr-js?
<wgrant> optimising
<nigelb> :/
<nigelb> I thought it was a simple bug.
<wgrant> Well, you could "fix" it by adding an assignee edit link.
<nigelb> Yeah, that's what I was looking to fix.
<cjwatson> wgrant: would you have time to review https://code.launchpad.net/~cjwatson/launchpad/bzip2-translations/+merge/75547 ?
<nigelb> I'm at bugtask-tasks-and-nominations-table-row.pt and can't find anything relevant.
<cjwatson> is indeed trivial.
<cjwatson> jml: 'man dpkg-source' for the various source package format
<cjwatson> s
<jml> cjwatson: thanks.
<jml> cjwatson: this learning thing is fun
<cjwatson> james_w: sadly I'm fairly sure there's a non-zereo number of packages where the tarball has some debian/ bits in it and the diff just mildly patches them.  You're probably right in the case of debian.tar.gz
<danilos> matsubara, hi, I've put up a MP for review for oops-tools, I hope you can take that
<wgrant> cjwatson: Hm, difficult.
<wgrant> Want me to land it?
<cjwatson> wgrant: difficult?
<nigelb> Probably because diff thing isn't working.
<wgrant> Yeah, diffs are broken at the moment.
<wgrant> But loggerhead worked.
<wgrant> To see the whole one line of non-test diff.
<cjwatson> Ah, my sarcasm detector was broken I think.
<cjwatson> wgrant: yes please, landing would be good
<cjwatson> james_w: indeed, on checking, dpkg-source removes any debian/ directory it finds in the upstream tarball before unpacking debian.tar.gz
<cjwatson> which is a much better design than we used to have
<jml> rvba, allenap: would one of you please land that branch? I don't have commit access. (https://code.launchpad.net/~jml/launchpadlib/hacking-documentation/+merge/75520)
<allenap> jml: Okay, I'll give it a go.
<jml> allenap: thanks.
<rvba> nigelb: re-reviewed. I've a few more nitpicks but I've asked Gavin to do that final check anyway. After that we'll be good to land.
<nigelb> rvba: cool! Thanks :)
<rvba> Welcome.
<matsubara> danilos, will do. thanks!
<matsubara> danilos, I didn't get any email about it yet.
<matsubara> oh, just read the backlog. diffs are broken...
<wgrant> cjwatson: That branch has been in ec2 for a little while now.
<cjwatson> wgrant: great, thanks
<jml> LP mail is slow today.
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> rvba: Anything else I need to do?
<nigelb> Just wait for allenap? :)
<allenap> nigelb: I'm looking at it now.
<nigelb> AH!
<jelmer> hi jcsackett, can I add a branch to your queue?
<nigelb> allenap: I just pushed the last set of changes.
<allenap> Okay.
<jcsackett> jelmer: you can indeed.
<jelmer> jcsackett: thanks :) the MP is at https://code.launchpad.net/~jelmer/launchpad/follow-http-redirects/+merge/75561
<jcsackett> i'll start reviewing it in just a moment. :-)
<matsubara> danilos, replied
<nigelb> diffs still broken?
<rvba> nigelb: apparently yes.
<nigelb> Ouch.
<nigelb> If I visit https://code.launchpad.net/lp-production-configs, for which I don't have permission, Launchpad tells me - "Launchpad Production Configs has 0 active branches."
<nigelb> But the very next line is "There were 5 commits by 3 people in the last month. "
<nigelb> Pretty much makes the first line a lie. Is this known?
<jcsackett> jelmer: quick question; i see this code apparently captures too many redirections, but i see no tests for that. is it tested elsewhere?
<jelmer> jcsackett: do_catching_redirects will raise TooManyRedirections in that case
<jcsackett> jelmer: that's what i see, but i don't see a test making sure everything works as expeced (e.g. i do see a test for forbidden conditions).
<jelmer> jcsackett: ah, yeah, that's true - it's indeed only tested in bzr itself
<jml> jkakar: hi
<jelmer> jcsackett: would you like me to add a test for that in lp as well?
<jcsackett> jelmer: i think it would probably be appropriate, since the lp buildbot doesn't check the bzr tests, to my knowledge.
<jcsackett> (and given that my understanding is those also take several hours, that's a very good things. :-P)
<jelmer> it's not that bad actually, I think it was just 10 minutes on an 8 core machine :)
<jelmer> anyway, I'll have a look at adding a test
<jcsackett> jelmer: cool. i've marked the MP as Needs Fixing so i'll remember to take another look. aside from the test, this looks good, so i'll be happy to approve once that's addressed. :-)
<jcsackett> nigelb: regarding the lying code page, i'm checking if there's a bug now. we have several related to the disclosure work that are about confusing messages like that.
<jml> jkakar: so now I'm trying to use the launchpadlib testing that just got in.
<jml> nigelb: known bug. It's filed somewhere.
<nigelb> jcsackett: I just filed one, feel free to dup it if there's another
<jcsackett> will do. :-)
<jml> jkakar: I'm confused as to how to assign collections.
<jml> jkakar: and by how the code is supposed to work in that case.
<nigelb> jml: Its fun to catch LP lying :)
<jml> jkakar: I want to have lp.distributions['ubuntu'] return something useful.
<jcsackett> nigelb: can't find any dupes of it, so i've triaged your bug and tagged it with disclosure. concievably it will be addressed as part of our work.
<nigelb> cool!
<rvba> nigelb: jcsackett seems related to 656941.
<rvba> Not the exact same location.
<jcsackett> rvba: no, but close enough one could be made a dupe of the other and the summary updated. but i do sort of wonder why that bug wasn't added to the disclosure tag ...
<nigelb> I wonder if its simple enough to check if the value of branches is 0 and based on that hide the next sentence.
<rvba> nigelb: This won't be sufficent to fix the bug completely IÂ think.
<rvba> sufficient*
<nigelb> Ah right. Public and private.
<nigelb> Tricky!
<rvba> nigelb: I'll be off soon.  I'm off tomorrow but I think allenap will be delighted to give you a hand to land your branch when it's fixed.
<nigelb> rvba: Ah, thanks again for all the nitpick ;-)
<rvba> You're welcome nigelb.
<nigelb> Like jtv said in his email, its probably sad when there isn't nitpick :)
<jcsackett> nigelb: that depends entirely on how long one reviewer's queue is. :-P
<nigelb> heh
<nigelb> allenap: Woah. I just learned a lot of Javascript! Thanks :-)
<allenap> nigelb: Hehe, cool. Btw, I like the change; it's a neat feature.
<nigelb> allenap: It was easier than all the changes I made too. I touched this code before, so I knew what I was doing most of the time :-)
<jelmer> jcsackett: I've added a test for the too many redirections issue
 * jcsackett goes to look
<sinzui> nigelb, you and I are thinking about the issue in the same way. The template already uses the convention of checking the count. The bad chunk violates the rule
<sinzui> nigelb, you are now subscribed to the master bug
<sinzui> I think you could fix this in a few minutes
<jcsackett> jelmer: r=me.
<jelmer> jcsackett: thanks!
<nigelb> sinzui: But that's only a partial fix.
<nigelb> It would fail in the event of a project with public and privuate branches (is that possible?)
<nigelb> sinzui: Oh wait. My concerns depends on how private branches work. I don't know :-)
<sinzui> nigelb, we show private bug stats in all bug stats because it is too expensive to separate them
<sinzui> nigelb, We know something is amiss on the page because of the contradictory statement, and we can see in the template that other parts avoid contradictory statements. only the block we see is misbehaving
<nigelb> sinzui: Ah. So, this should be a good fix because its expensive for the "better" fix?
<nigelb> I'll take it to go ;-)
<sinzui> yes. if there is one public branch, we do want to show data
<sinzui> also note that there are now three reported bugs about this issue now, and all are about the one chunk of text, so I believe the hard fix is overkill. Someone has to prove the data is exposed elsewhere to justify a deeper fix
<nigelb> heh :)
<nigelb> Because this is very apparent. The other situation less so.
<nigelb> allenap: Y.Lang.hasOwnProperty isn't working :(
<nigelb> (javascript tests++)
<nigelb> Any JavaScript / YUI person around for some debugging help?
<allenap> nigelb: var a = {fred: 123}; a.hasOwnProperty("fred") // true
<allenap> It's part of JavaScript rather than YUI.
<nigelb> Oh.
<nigelb> allenap: that helps! Thanks
<m4n1sh> sinzui: is lp-release-manager-tools mature enough as per your expectations?
<nigelb> sinzui: That bug sounds way too easy. Want me add tests as well? :-)
<sinzui> m4n1sh, thy are for me
<nigelb> allenap: Pushed. Do you have time for another review of that MP?
<allenap> nigelb: Sure :)
<allenap> nigelb: Looks good. One *tiny* thing:
<allenap> bugs_ids = set([int(link[len('/bugs/'):]) for link in links])
<nigelb> Sure
<allenap> The outermost [] braces are not needed.
<nigelb> ah
<nigelb> since its a set anyway.
<nigelb> Right!
<allenap> nigelb: I'll land it as it is; it's not worth changing.
<allenap> Unless you're very keen :)
<nigelb> allenap: wait, doing set(1,2,3) gives me a traceback in python shell.
<allenap> nigelb: That does, but try set(i for i in 1,2,3), which is equivalent to set((i for i in 1,2,3)), i.e. set() gets passed a generator.
<nigelb> ah
<nigelb> SyntaxError: Generator expression must be parenthesized if not sole argument
<allenap> Grr, sorry, let me try again!
<allenap> set(i for i in (1,2,3)), which is equivalent to set((i for i in (1,2,3)))
<nigelb> aha!
<nigelb> allenap: Ah, you landed it? :)
<sinzui> jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/private-bug-5/+merge/75614
<allenap> nigelb: Well, it's in ec2, but if you have another change I can restart it, no problem.
<nigelb> nah, let it go :-)
<jcsackett> sinzui: sure.
<sinzui> thank you
<jcsackett> sinzui: this appears to be a big diff. lemme finish poking at one thing first and then i'll get into it. :-)
<sinzui> jcsackett, yes. it is larger than anticipated since I updated some supporting classes. I think you will find the implementation is often trivial, though I wanted better tests than what I started with
<sinzui> jcsackett, oh yes, I neglected to mention I wrote a test for every class before I made my change...that double the change size
<nigelb> sinzui: I'm assigining that bug to myself.
<sinzui> nigelb, thanks you very much
<jcsackett> sinzui: dig, that'll help me sort the diff out, thanks. :-)
<nigelb> sinzui: Hey, do you have a minute for a bit of UI help?
<nigelb> I tried fixing bug 806660. It ended up looking very ugly with too much whitespace.
<_mup_> Bug #806660: "Add a new address" in e-mail settings does the wrong thing when pressing Enter <easy> <ui> <Launchpad itself:Triaged by nigelbabu> < https://launchpad.net/bugs/806660 >
<benji> has anyone seen something like this when trying to run the LP tests? OperationalError: FATAL:  role "benji" does not exist
<benji> This is on a brand new oneiric install
<sinzui> nigelb, are you splitting the forms or adding a keypress handler?
<nigelb> I tried splitting the forms.
<nigelb> Its not a pretty approach for the UI.
<sinzui> nigelb, I wonder if we can hack the add button to have accesskey="Enter"
<nigelb> Hrm, I didn't think of that. That should be easier for sure.
<nigelb> The last time I tried it in JS, there was an issue with IE.
<sinzui> nigelb, of when something is entered, the other two buttons are deactivated, so the browser will choose the only active button
<nigelb> Oh.
<nigelb> deactivate the otheres on focus?
<nigelb> and activate it back when focus is lost for the text box?
<sinzui> nigelb, We may not need hacks. I looked at the template
<nigelb> oh.
<sinzui> The template is doing the actual layout, explicitly stating which chunk of the for to render. I think we want to call the form twice, each time only rendering the blocks we want
<nigelb> Didn't parse that :-)
<sinzui> nigelb, something like this: http://pastebin.ubuntu.com/690315/
<sinzui> nigelb, I think the table elements need reconciliation in my paste. We only want to render the add field and button in the second call of the form.
<nigelb> sinzui: heh, that's very close to what I did
<nigelb> Let me see if yours renders better than the mess I made
<jcsackett> sinzui: took a bit to read through all the tests, but r=me.
<sinzui> jcsackett, thank you for your patience
<jcsackett> sinzui: thank you for yours, given the time. :-)
<nigelb> sinzui: that didn't work.
<nigelb> one form cannot be inside another form.
<nigelb> actually, ignore me.
<nigelb> I was running the wrong launchpad.
<bac> lifeless: ping
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<lifeless> bac: hi
<bac> hi lifeless ... i've been looking at your branch for bug 759467
<_mup_> Bug #759467: incomplete-with-response searches require complex searches <Launchpad itself:Triaged by lifeless> < https://launchpad.net/bugs/759467 >
<lifeless> bac: cool
<bac> lifeless: i'm unsure of your intentions, though.  do you want someone to pick it up or do you plan to fix and land it?
<bac> it has considerable bit rot due to the BugSummary work
<lifeless> bac: I don't have the cycles to pick it up myself for a few weeks
<lifeless> need to get the SOA stuff rolling or risk blocking red
<bac> right
<lifeless> I would -love- it if someone else picks it up
<lifeless> if noone does, it still needs doing and I have the most context on the problem so would indeed pick it up eventually
<lifeless> if you're in a position to pick it up, I would love that.
<bac> as you said in my MP, the escalated bug i worked on really needs this in place to properly proceed
<bac> lifeless: i may need to pick your brain a bit to get through all of the conflicts caused by BugSummary
<bac> for instance, would BugSummary.status be the real db status with INCOMPLETE_WITH*_RESPONSE or the visible one?
<lifeless> bac: it would transition
<lifeless> so 2 stages
<lifeless> stage one, we start querying for both the current semi-status + date, *and* the actual explicit status
<lifeless> on bugsummary and bug queries
<lifeless> we also start *writing* explicit status at this point
<lifeless> then we do a garbo job to rewrite every bugtask's status from the semi-status+date into an explicit status
<lifeless> lastly we drop the garbo job, and the querying of both the semi status +date
<lifeless> so bugsummary.status in stage one will be mixed, until the garbo job has migrated all the rows of bugtask across
<lifeless> one sec
<lifeless> back
<lifeless> the contents of the bugsummary table don't need to be explicitly updated: as the garbo updates bugtask rows, the triggers will auto-update bugsummary for us.
<bac> so for writing, the db procedures will simply need to start using BugTask._status rather than BugTask.status
 * lifeless refreshes his memory
<bac> _status is real, what is in the db
<bac> status is the model view showing INCOMPLETE
<lifeless> yes
<lifeless> ideally we would migrate all the model code to be explicit too, but thats got lots of complicated bits from what I could see
<lifeless> and the performance / clarity wins are all db-side.
<lifeless> righto, so my branch has all the stage one stuff *except* making queries on bugsummary also include the two new enums
<jcsackett> can anyone think of a reason why responses from answers.launchpad.dev is substantially slower than from launchpad.dev?
<lifeless> jcsackett: nope :)
<jcsackett> i'm running in a vm setup for remote access per the wiki; can't figure out why subdomains are *so* much slower.
<jcsackett> lifeless: well damn. :-P
<bac> lifeless: ok, i think i have a shove in the right direction
<bac> jcsackett: i run with that configurationa and haven't seen such a thing
<lifeless> bac: I've just re-read the branch and yeah, I think that that is all thats missing (modulo surprises)
<jcsackett> bac: yeah, i've run this way before (new vm, but everything is roughly the same) and haven't seen this either. very perplexed.
<lifeless> for clarity 'all that is missing is teaching bugsummary queries that want INCOMPLETE to want INCOMPLETE + INCOMPLETE_WITH_RESPONSE + INCOMPLETE_WITHOUT_RESPONSE'
<lifeless> (unless they really want to be narrower anyway, in which case INCOMPLETE + one of the others)
<bac> lifeless: right, plus resolving all of the conflicts towards using BugSummary not BugTask in queries
<lifeless> bac: yes, though they should be mainly just take trunk's side
<lifeless> bac: I think you'll find, if you diff .BASE.THIS that the local changes are all INCOMPLETE -> INCOMPLETE + ... ,or status -> _status
<lifeless> flacoste: I think you win on the 'find oldest regression to date'
<poolie> hi all
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
#launchpad-dev 2011-09-16
<poolie> wgrant, i kinda hoped for a +1 on my process-one-mail announcement
<poolie> not all that externally exciting but i found it much more comfortable
<poolie> maybe someone will use it in future
<poolie> and, traditional email is kind of lacking a cheap non-noisy +1
<wgrant> poolie: Sorry, meant to +1 that last night, but was very busy.
<poolie> np
<wgrant> And then had far too much mail this morning.
<poolie> not really fishing :)
<wgrant> It is a really handy helper.
<wgrant> Often I avoid testing mail stuff because it's just so awkward :/
<wgrant> But not any more!
<wgrant> Thanks for JFDI.
<poolie> yeah, for me it was a minor mental breakthrough
<poolie> "this is so hard to test.... oh, but it doesn't actually inherently have to be"
<poolie> hello losa? i'd like some help testing email on staging
<wgrant> poolie: What are you wanting to test? staging processes mail automatically.
<poolie> oh does it now?
<poolie> and qas does too?
<poolie> for some reason i thought it needed to be manually prodded
<wgrant> Ah, qas, true. It used to do it less frequently than staging, and may no longer do it at all..
<wgrant> That's a good point.
<poolie> hloeung, spm?
<hloeung> poolie: hi, how can I help you?
<poolie> also i'm a bit curious what software ends up delivering the mail to lp's mailbox
<poolie> hloeung, does qas automatically receive and process mail or do i need to get you to thump it
<poolie> wgrant, re bug 813349, are you reopening it because it's flaky?
<_mup_> Bug #813349: hard to get from mp to per-revision diffs <code-review> <javascript> <qa-ok> <ui> <Launchpad itself:In Progress by spiv> < https://launchpad.net/bugs/813349 >
<poolie> i see the point about it not being done done
<hloeung> poolie: I'm not too sure
<wgrant> poolie: It's not done-done, needs more work, and was one of two bugs that were cluttering up my queue.
<wgrant> poolie: I decided to finally get rid of it.
<poolie> it would be easier if i just try it
<wgrant> Now that the queue can be otherwise clear.
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2084H53
<wgrant> lifeless: timeouts now break lazr.restful :/
<wgrant> I guess we want to capture sys.exc_info and use that.
<wgrant> Rather than assuming __traceback__
<wgrant> Isn't that new in 2.7/3.0?
<lifeless> wallyworld_: http://paste.ubuntu.com/690450/
<lifeless> wallyworld_: is what I recommend
<poolie> did ec2 land change so that it no longer detaches from the vm?
<wgrant> poolie: No.
<poolie> *intentionally?
<poolie> because it does seem to be not detaching now
<wgrant> What's it said?
<poolie> it's running 'python .... ec2test-remote.py', just printed some bzr hpss counts and now it's just sitting there connected
<lifeless> wallyworld_: that query will do unstacked branches first
<poolie> i can see the tests are running
<wallyworld_> lifeless: so the change is you added "ORDER BY stacked_on ASC NULLS FIRST"  ?
<poolie> when this happened yesterday, it did eventually complete
<wgrant> poolie: Ah. It should detach right after ec2test-remote.py
<wgrant> poolie: I haven't seen that happen in months.
<wgrant> But when it does you can normally just ctrl+c
<poolie> that's two out of two so far
<wgrant> After it starts ec2test-remote.py
<lifeless> wallyworld_: and then in increasing branch id, which will (except where folk have restacked) keep the depth of recursion low
<lifeless> wallyworld_: I think thats all I did yes
<wallyworld_> lifeless: that change somehow helps the optimiser ?
<StevenK> sinzui: There is something you can help me with, in fact.
<wallyworld_> lifeless: how did you know that's what you needed to do?
<lifeless> wallyworld_: it doesn't help the optimiser
<sinzui> StevenK, yes?
<lifeless> wallyworld_: but it will stop us looping 6 times through
<wallyworld_> ah ok. i'll look at the explains in more detail
<wallyworld_> they're still a bit hard for me to read
<lifeless> so we have a branch A->B->C->D->E->F
<StevenK> sinzui: I'd like a header and prose for the pop-up when the user wants to unsubscribe themselves.
<wallyworld_> and grok
<lifeless> until we update F, anything stacked on any of B,C,D,E,F, will recurse all the way to F
<lifeless> this tweak makes it be more likely to calculate F before A, as F will be unstacked, E will have a lower branch id than D, in the usual course of htings.
<sinzui> StevenK, I writing job? I am on it. Header: Hey, I see you are shooting yourself in the foot. I can help
<lifeless> there is one more step we can take
<StevenK> Hahaha
<lifeless> which is to use transitively_private and short circuit when it is set.
<wallyworld_> lifeless: but we don't limit the recursive query to take that into account
<wallyworld_> yes, so we add the short circuit
<wallyworld_> that's a nice optimisation
<sinzui> stevek prose: By unsubscribing from this bug you are trading lots of exciting spam for 404's Click Oops to stop this.
<poolie> hloeung, wgrant, it's not obviously responding to my mail, so i think i do need process-mail run by hand
<poolie> i'm pretty sure this was needed last time too
<poolie> robert said something abotu it creating too much load to run it automatically
<wgrant> poolie: We should set up a regular cron job, and bump staging's frequency.
<wgrant> poolie: Since as of like 12 hours ago we have a new staging scripts server.
<hloeung> wgrant: file an RT to have that done. In the meantime, I'll run process-mail by hand
<poolie> so should you file an rt for that or something?
<poolie> thanks
 * wgrant RTs
<wgrant> #47933
<_mup_> Bug #47933: [Feisty] system requires acpi=off noapic nolapic <cft-2.6.27> <linux-source-2.6.20> <linux (Ubuntu):Won't Fix> <linux-source-2.6.20 (Ubuntu):Won't Fix> < https://launchpad.net/bugs/47933 >
<hloeung> poolie: ok, done
<hloeung> wgrant: ta
<poolie> woo hoo
<poolie> that's so awesome
<poolie> thanks
<wgrant> poolie: Oh, it worked? :/
<poolie> only a wry face?
<wgrant> poolie: gandwana doesn't seem to be able to talk to the mailserver.
<wgrant>   File "/srv/staging.launchpad.net/staging/launchpad/lib/lp/services/mail/mailbox.py", line 119, in open
<wgrant>     raise MailBoxError(str(e))
<wgrant> MailBoxError: [Errno 111] Connection refused
<poolie> it did work
<poolie> https://bugs.qastaging.launchpad.net/launchpad/+bug/800275
<_mup_> Bug #800275: boot splash not branded <ugr-ebuild:Confirmed for soaringsky> < https://launchpad.net/bugs/800275 >
<wgrant> hloeung: Did you run it on asuka?
<poolie> hloeung, can it ever send outgoing mail?
<hloeung> wgrant: yeah, I ran it on asuka
<poolie> also, can you tell me what software is involved in delivering the message to lp?
<wgrant> poolie: We can run send-bug-notifications.py (on asuka, not gandwana yet, or we will be in big trouble) and it will end up in the staging mailbox.
<wallyworld_> lifeless: btw, thanks for all the xecellent input on getting that query optimised. i'm really pleased with the result
<poolie> does it really (as that traceback suggests) read it over pop/imap?
<wgrant> Yup. POP3.
<wgrant> Really we should have a handler in the MTA which sticks it in rabbitmq or something.
<wgrant> Where it's picked up by a daemon.
<wgrant> Rather than being a */3 cronjob.
<lifeless> wallyworld_: http://paste.ubuntu.com/690457/
<lifeless> wallyworld_: this is more of a rewrite
<lifeless> wallyworld_: it implements the recursion short circuit when you encounter a calculated transitively private
<lifeless> wallyworld_: it can do 800 a loop
<lifeless> in < 2 seconds
<wallyworld_> lifeless: i was in the middle of writing something similar myself :-) but you beat me to it
<lifeless> can't predict behaviour once you're 30/40K in trivially.
<lifeless> wallyworld_: happy to help
<lifeless> wallyworld_: I'm satisfied this will do reasonably well; I'm going to leave it here until/unless you ping me again
<wallyworld_> lifeless: np. it's all good. i'll check the unit test passes and then do a db-devel branch with the index
<poolie> hloeung, process-mail again please?
<hloeung> poolie: sure
<poolie> and once more?
<hloeung> poolie: done
<poolie> super
<poolie> looks like it all works
<poolie> thanks very much
<hloeung> np
<sinzui> StevenK, http://pastebin.ubuntu.com/690460/
<wallyworld_> hloeung: at some stage, no hurry, there's a new feature flag to go on production documented on LaunchpadProductionStatus
<StevenK> sinzui: Thank you, that is excellent.
<hloeung> wallyworld_: added
<poolie> ok now some more real work
<wallyworld_> hloeung: legend! thanks!
<poolie> what does 'orphaned' mean in the deployment report? that there's another branch for that bug?
<wgrant> poolie: There is no open bug related to the landing.
<wgrant> poolie: This could be because it was landed as testfix or a rollback, so has no bug, or the associated bug(s) are closed.
<lifeless> mwhudson: branch tokens - I think we need to talk about them :)
<lifeless> mwhudson: I have some fears
<lifeless> mwhudson: if you can make some time next week, that would be great
<mwhudson> lifeless: yeah sure
<mwhudson> lifeless: can you articulate your fears at all?
<mwhudson> i.e. conceptual problems, security problems, performance...?
<lifeless> performance impact, user model impact,  bzr ui impact
<lifeless> auditability too
<lifeless> and long term interaction with oauth/openid for bzr
<lifeless> (I want to ditch ssh keys altogether in some ways)
<poolie> really?
<poolie> tell me more?
<lifeless> poolie: we've spoken about this before
<poolie> i have an awful memory
<poolie> can you prompt it
<lifeless> poolie: I haven't advanced any plans, but ssh key management is a real headache for new users
<lifeless> doing an API style login once on the client would be much nicer.
<poolie> i would certainly agree with making them a lot less manually maintained
<poolie> right
<mwhudson> lifeless: would you be less concerned if i just targeted removing the puller?
<lifeless> mwhudson: I think jelmer will have that done soon anyway :)
<lifeless> mwhudson: won't he ?
<mwhudson> lifeless: no
<wgrant> Still needed for imports.
<mwhudson> the import puller will still be required
<lifeless> mwhudson: so, we should talk :)
<mwhudson> ok
<mwhudson> lifeless: i suspect your diary is massively more cluttered than mine, feel free to do something involving calendaring applications and michael.hudson@linaro.org
<lifeless> mwhudson: i'll ping you monday :)
<mwhudson> lifeless: ok
<lifeless> lowtek ftw
<mwhudson> relying on memory << * in my case though :)
<poolie> kk lunch biab
<mwhudson> is there a private xml-rpc server method to check for a feature flag?
<wgrant> No.
<wgrant> It's not quite trivial, since you'd need some way to work out scopes.
<mwhudson> what are scopes again?  things like projects and teams?
<wgrant> https://launchpad.net/+feature-info
<wgrant> "
<wgrant> Scopes" section
<mwhudson> ah nice
<mwhudson> well, as nice as it would be to restrict anonymous ssh logins to members of a team ... :)
<nigelb> wallyworld_: \o/ landed that branch last evening :-)
<lifeless> wgrant: mwhudson: pass in a dict of literal scopes you're interested in
<lifeless> as a first-pass approximation
<poolie> mwhudson, most obviously you could limit by ip
<poolie> maybe also ssh client signature
<lifeless> that will let some obvious ones like page-id be passed in
<wgrant> lifeless: I considered that.
<wgrant> Maybe.
<lifeless> for a v1 lets get unblocked thing
<lifeless> its fine.
<lifeless> internal-only.
<poolie> o/ huwshimi
 * mwhudson contemplates the fun that is adding xmlrpc methods to lp
<huwshimi> poolie: Hey!
<nigelb> mwhudson: xmlrpc it self is "fun" :-)
<mwhudson> also true
<lifeless> wgrant: what are the '53'
<poolie> lifeless, sorry to re-ask something i asked a month or two ago, but
<lifeless> 42
<poolie> once revisions are qa'd on stable, they'll deploy within a few days?
<lifeless> ideally ;)
<wgrant> No.
<wgrant> Ideally within 24 hours.
<poolie> unless something else fails on production, or fails to qa?
<lifeless> it all depends on someone putting up a deployment request, other revisions not being in the way etc.
<wgrant> Unless stuff goes wrong, they're generally out in 24 hours.
<lifeless> the goal time is < 1 hour from qa
<lifeless> but we are far from that yet
<wgrant> lifeless: Ah, that email is mostly directed at Julian. https://devpad.canonical.com/~wgrant/2011/soyuz-expiration-blargh.html is not light reading.
<wgrant> lifeless: But, in summary, there are 53 BPRs published since hardy that are illegaly expired.
<wgrant> Because they're still published in the primary archive.
<lifeless> righto
<nigelb> Ugh, Tal.
<wgrant> We need to recover them from the archive.
<lifeless> I've read it now
<lifeless> thanks
<lifeless> jamesh: hi
<lifeless> jamesh: got 5 ?
<jamesh> sure.
<huwshimi> Does anyone here use Identi.ca? I have an issue identical to this post but I don't understand what they're talking about: http://forum.status.net/discussion/873/badge-identica-badge.js-not-working-with-my-statusnet-installation/p1 (I'm asking here because I'm trying to set up something for Launchpad).
<poolie> i use it but not the badge feature
<poolie> idnetical how?
<lifeless> jamesh: so, I think: timeline wsgi injector should be a separate timeline-wsgi project; storm can grow a timeline_factory that timeline-wsgi talks to. python-oops-timeline should exist and have the context -> report code in it.
<lifeless> jamesh: so 2 new trivial projects, a patch to storm, and we're done
<huwshimi> poolie: I have the same issue, as in, no posts show up on the badge. That post seems to say something about only friends posts showing up, but I don't understand the identi.ca language enough to know what they're saying
<poolie> oh you want launchpadstatus's badge to be somewhere?
<poolie> i thought you were trying to implement lp badges or something
<jamesh> lifeless: so what would python-oops-timeline hold exactly?  Just the code to add the timeline to environ['oops.context'] and some on_create handlers?
<jamesh> or something more?
<poolie> o/ jamesh
<huwshimi> poolie: yeah I'm trying to show it on a status page
<poolie> huwshimi, i suspect 'friends' is a hippy equivalent of twitter 'following'
<lifeless> jamesh: oops-timeline would  be on_create handlers
<nigelb> statik: LOVE the reply you got from DEVOPS_BORAT :-)
<poolie> and launchpadstatus has no friends :)
<poolie> let's see if that's true
<jamesh> or I suppose it could do without copying to oops.context, since you could go context['wsgi_environ']['timeline.whatever']
<lifeless> jamesh: thats timeline-esgi :)
<lifeless> *timeline-wsgi*
<poolie> it's true: no friends
<lifeless> jamesh: I'm trying to avoid having specific projects have code for sibling projects in them
<jamesh> lifeless: okay.  So the timeline-wsgi module knows about oops-wsgi.  That sounds fine.
<mwhudson> i wonder who DEVOPS_BORAT is
<jamesh> or alternatively the other way around
<lifeless> perhaps call it timeline-oops-wsgi; then it clearly can know about oops variables
<mwhudson> sometimes it feels like it must be someone i know
<wgrant> mwhudson: Heh, yes.
<poolie> huwshimi, i wonder what happens if lpstatus follows itself
<lifeless> jamesh: or as we touched on the other day, environ['timeline.timeline'] could be the dedicated variable
<huwshimi> poolie: Can we make that happen?
<jamesh> lifeless: if the only connection is that it does "if 'oops.context' in environ: environ['oops.context']['timeline'] = timeline", then I don't think an extra package is worth it
<poolie> apparently no
<lifeless> jamesh: So I've found, when I dig into some java stacks, that having *tiny* packages is actually very useful
<huwshimi> poolie: OK, I might give up and use the twitter feed instead.
<lifeless> jamesh: it keeps the dependency chain very lean, when you go to reuse.
<huwshimi> poolie: Thanks for your help :)
<poolie> i have an idea
<lifeless> jamesh: I know storm doesn't do this, but I wish more code did.
<lifeless> it would be -awesome- if the overhead of doing a single python module (not package) was lighter.
<lifeless> jamesh: it won't have a practical difference for you though, will it ?
<poolie> huwshimi, what happens if you just make a copy of the js, and change 'statuses/friends'  to 'statuses/user_timeline'?
<poolie> i reckon there's a fair chance that will work
<jamesh> lifeless: I guess not.  It is one more thing to package though
<huwshimi> poolie: I can do that (and it will probably work), but I'm trying to limit the number of working parts as this will appear when LP is down.
<jamesh> [we haven't drunk the buildout kool aid]
<lifeless> jamesh: time to give pkgme a test drive then :)
<lifeless> jamesh: these should be ideal cases for fully automated packaging
<lifeless> wgrant: welcome to the dark side
<lifeless> stub: yo
<stub> yo
<wgrant> lifeless: Heh.
<lifeless> stub: time for a catchup ?
<wgrant> I indeed finally gave in :(
<stub> lifeless: sure. Gimme a sec to boot the lappy
<stub> and steal my mike back
<lifeless> you want to be like mike/
<lifeless> ?
<huwshimi> When the database goes down at the moment do we currently display anything? What's involved to put a [new] page there?
<huwshimi> stub: Matthew said you might know something about this...^
<stub> huwshimi: Bug #840098
<_mup_> Bug #840098: Librarian should return 503 pages during database outages <qa-ok> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/840098 >
<wgrant> huwshimi: We just display an Oops page now.
<wgrant> huwshimi: We can put whatever you want there, basically... as long as it doesn't involve DB access, obviously.
<huwshimi> wgrant: OK, I have (most of) a static HTML page (a little js and some images too) that will go there. Do you know how I can make this work?
<nigelb> wgrant: dark side?
<wgrant> nigelb: I created a Twitter account.
<nigelb> OHMYGOD
<nigelb> @wgrant?
<StevenK> wgrant: We can no longer be friends.
<wgrant> nigelb: I need to try to claim that using the 9 month rule. @wgrant_ for now.
<wgrant> Nasty common names :(
<huwshimi> StevenK: No, but you can be followers
<StevenK> huwshimi: Boo, hiss.
<nigelb> wgrant: _ is the new style on twitter
<nigelb> huwshimi++
<wgrant> huwshimi: It should ideally use normal LP JS/image infrastructure.
<wgrant> huwshimi: As it's rendered by LP, like any other view, except without the DB.
<wgrant> huwshimi: Just like the current OOPS page.
<mwhudson> poolie: could you look at https://code.launchpad.net/~mwhudson/launchpad/feature-flag-xmlrpc/+merge/75673 briefly?
<poolie> +"""XXX: Module docstring goes here."""
<poolie> or not :)
<mwhudson> well yes, not that stuff:
<nigelb> heh
<mwhudson> """Not ready yet (docstrings need writing etc) but would like feature flag manipulation looked at please :)"""
<nigelb> wgrant: Can we have a tumblbeast? :)
<poolie> i know i'm just being snarky
<poolie> yes, please
<poolie> or, that's more of huwshimi's department, beasts
<wgrant> Heh
<wgrant> Was that the one TheOatmeal created?
<nigelb> yeah
<nigelb> its free to use now
 * mwhudson goes to the pub
<poolie> oops
<poolie> mwhudson, it's not obviously wrong but i don't really understand it
<mwhudson> poolie: ok, let's talk next week then :)
<poolie> ok
<nigelb> I'm having brunch and mwhudson is headed to the pub.
<poolie> the intention is good
<nigelb> timezones.sigh.
<poolie> cheerio mwh
<StevenK> nigelb: Then move to Australia?
<nigelb> haa.
<mwhudson> nigelb: what bothers me is when i'm going to the pub, and my colleague in POLAND hasn't stopped working yet
<nigelb> heh
<nigelb> mwhudson: It probably bothers him more :P
<nigelb> him/her
<poolie> what bothers me is when i come home from the pub and robert is still working
<nigelb> StevenK: Only if it were that easy.
<nigelb> poolie wins
<huwshimi> poolie: It's only when I'm in another timezone when I realise how many people here work *very* late
<nigelb> Oh wgrant doesnt have a concept of time.
<nigelb> Neither does StevenK.
<wgrant> Last night was particularly bad.
<wgrant> Since a script I expected to take 3 hours actually took 13 hours.
<nigelb> Could someone help me with this? http://paste.ubuntu.com/690527/
<wgrant> nigelb: You have a tag mismatch somewhere.
<nigelb> I've been trying to trace it without much luck :(
<nigelb> http://paste.ubuntu.com/690530/ <-- pt file
<wgrant> Which table are you trying to close on line 68?
<wallyworld_> wgrant: StevenK: do you see an issue with lowering the minimum picker search term length from 3 chars to 2? perhaps if term is < 3 chars do not do fti search, just exact match to lp name
<wgrant> That table seems to be opened in the previous metal:widgets.
<wallyworld_> for bug 768148
<_mup_> Bug #768148: person picker does not handle exact matches well - including not permitting exact matches for usernames < 3 characters long. <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/768148 >
<nigelb> the one in line 25
<wgrant> nigelb: I think 68 is trying to close the table on 25, but they are in separate macros.
<nigelb> Oh. I can't do that?
<wgrant> This is approximately XML.
<wgrant> It's not text-based like Django markup.
<nigelb> :D
<nigelb> You guessed where I'm coming from.
<nigelb> Argh. Need to ask sinzui for help.
<nigelb> I'm not sure how to get this workign without it being ugly.
<wallyworld_> nigelb: cani help at all?
<nigelb> wallyworld_: ohai, I'm trying to fix bug 80660 without killing kittens and without it being ugly.
<_mup_> Bug #80660: GAIM2.0b5 crashes on quit <gaim (Ubuntu):Invalid> < https://launchpad.net/bugs/80660 >
<nigelb> err
<nigelb> 806660
<nigelb> bug 806660
<_mup_> Bug #806660: "Add a new address" in e-mail settings does the wrong thing when pressing Enter <easy> <ui> <Launchpad itself:Triaged by nigelbabu> < https://launchpad.net/bugs/806660 >
<wallyworld_> but i hate cats. they all must die
 * wallyworld_ looks at bug
<nigelb> wallyworld_: heh, on that you should look at Amber's (Graner) facebook photos (if you are her friend on fb)
<StevenK> wallyworld_: http://cheezburger.com/kenneystudios/lolz/View/1802695936
<wallyworld_> nigelb: i hate fb more than i hate cats. and i hate twitter too. total waste of time
<nigelb> heh
<wallyworld_> StevenK: don't know where you find this stuff
<nigelb> StevenK probably has subscription to icanhazcheesburger
<nigelb> I do as well. BUt to one of their smaller sites.
<wallyworld_> nigelb: so did you try splitting the form into two like suggested in the bug report?
<nigelb> I did.
<nigelb> It was ugly
<wallyworld_> no luck?
<nigelb> What I did was split the table
<nigelb> Because semantically, I cannot have one form in another form.
<nigelb> And a form must either include the entire table or be only in one td.
<wallyworld_> yes
<wallyworld_> and that didn't work? having two tables?
<nigelb> It worked!
<nigelb> BUt it was ugly with lots of padding
<nigelb> Can we fix that?
<nigelb> s/padding/margin/
<wallyworld_> can i see what you did?
<nigelb> sure
<nigelb> sec, let me restore yesterday's changes
<nigelb> Oh gah.
<nigelb> Give me 5 minutes I'll have to do it from scratch. Shoulda commited at safe point.
<wallyworld_> ok, just ping me
<wgrant> Shelve is handy.
<wgrant> Except when I end up with 60 shelves in devel.
<wallyworld_> my ide has shelve built in :-)
<wgrant> Does PyCharm have juju integration yet?
<wallyworld_> nope :-)
<nigelb> juju?
<nigelb> ensemble?
<wallyworld_> s/ensemble/juju
<wallyworld_> they renamed it
<nigelb> I like ensemble better :P
<wgrant> Yeah.
<wallyworld_> so does everyone else
<wallyworld_> when people found out, they wore out the WTF keys
<wgrant> Ahem.
<nigelb> lol
<wgrant> Yay, git branch importing works.
<wgrant> jelmer: Do you want a bzr bug for http://launchpadlibrarian.net/80061780/wgrant-mochiweb-R12B.log?
<wgrant> jelmer: I tried to use the original ,somebranch syntax instead of ,branch=somebranch
<wgrant> jtv: Do you have branches to entirely abolish katie yet?
<nigelb> branch scanning broken?
<jtv> wgrant: yes, have a look at lp:launchpad.
<jtv> wgrant: that's katie gina's buddy, not the other katie.
<wgrant> jtv: Ah, hadn't checked that in a few hours.
<wgrant> Nice, thanks.
<jtv> Tsk.
<jtv> Way behind the times.
<jtv> Soooo 07:44 ITC.
<nigelb> Despite living in the future.
<wgrant> nigelb: Which branch isn't scanned?
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/registry-email-add-806660
<nigelb> Its been 8 minutes. Launchpad is faster than that!
<wgrant> Hmm, 13 minutes.
<wgrant> That's not good.
<nigelb> wallyworld_: So, it looks like this -> http://people.ubuntu.com/~nigelbabu/first-try.png
 * wallyworld_ looks
<wgrant> Is that Lucid?
<wgrant> Possibly Maverick.
<wgrant> Old, anyway.
<nigelb> Lucid
<nigelb> How did you know....
<wgrant> The buttons are too brown.
 * nigelb bows.
<nigelb> And here's my diff -> http://paste.ubuntu.com/690552/
<nigelb> wgrant: I still haven't come to terms with unity.
<wallyworld_> nigelb: there's too much vertical space isn't there
<nigelb> Yeah
<nigelb> That's because they're in two different divs.
<nigelb> I don't know if I can move them into one.
<nigelb> Since the div itself is constructed from the <metal..
<nigelb> (or so I think)
<wallyworld_> nigelb: did you check to see if any ccs is in use for any of the div/table/form elements that may be adding the vertical padding
<wgrant> nigelb: Your branch scanned a couple of minutes after you asked. Still waiting on logs to see what the delay was.
<huwshimi> So, upgraded to Oneiric, rocketfuel-get, rocketfuel-branch foo, in foo: make schema: 'psycopg2.ProgrammingError: type "debversion" does not exist'. What else do I need to do?
<huwshimi> ... oh and I'm just assuming this has something to do with the Oneiric upgrade, it may well not be
<wgrant> huwshimi: Sounds like the launchpad PPA was disabled during the upgrade.
<wallyworld_> huwshimi: did you install launchpad-devel-dependencies?
<huwshimi> wgrant: Oh yeah, I think you're right
<wgrant> huwshimi: Make sure it's enabled, pointing to oneiric instead of natty, and then reinstall launchpad-developer-dependencies.
<huwshimi> wgrant: Thanks
<wgrant> I fixed that debversion issue in the PPA a couple of weeks ago.
<wgrant> So it should work.
<nigelb> wallyworld_: So, what's adding the margin is table and form elements
<nigelb> (fun!)
<nigelb> wgrant: Excellent! I broke it? :D
<wallyworld_> nigelb: are they rendered with a css style?
<wallyworld_> a specific css style?
<nigelb> No.
<nigelb> table.form
<nigelb> and form
<nigelb> generic styles
<wallyworld_> so perhaps you could initially try using an inline style with no padding/margins etc and see if that works
<nigelb> Is that kosher? :-)
<wallyworld_> just to see if it works
<nigelb> ah
<wallyworld_> then we can either define a style in the template (like is sometimes done for a one off) or see if there's already one in styles-3.0.css
<nigelb> You mean a style="margin: 0;" right?
<wallyworld_> something liek that
<nigelb> Can't touch <form> because its created by a macro.
<wallyworld_> there is a way, i've seen it used but can't recall where exactly
<nigelb> let me check the macros.
<wallyworld_> nigelb: you may not need the initial form macro. just using a stock <form> may work
<wallyworld_> i'd have to experiment a bit
<nigelb> ok
<wallyworld_> nigelb: in the template you are editing is an example of using a standard <form> element rather than the macro
<nigelb> wallyworld_: worked :)
<nigelb> I just added the inline styles from firebug.
<wallyworld_> excellent!
<wallyworld_> nigelb: i have to go out and pick up my wife from downtown
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<wgrant> Brisbane has a downtown?
<wallyworld_> wgrant: i was using the venacular that nigelb would understand as an american :-)
<wallyworld_> s/downtown/cbd :-P
<nigelb> haha
<nigelb> wallyworld_: I'm not American
<nigelb> Think East :-)
<wallyworld_> east of where?
<nigelb> I'm from India
<wgrant> nigelb: East of us is the US :P
<wallyworld_> oh, ok. sorry!
<nigelb> wgrant: HA.
<nigelb> East of the US!
<wallyworld_> what part of india? i've been to Bangalore
<nigelb> But that was a fail attempt at referring to East India Company
<nigelb> wallyworld_: I live in Bangalore :-)
<wallyworld_> :-)
<wallyworld_> Cat Logistics CLSI have an office in Whitexxxxx rd or something like that
<wallyworld_> east of the city centre i think
<nigelb> Whitefield
<wallyworld_> yes, that's it
<wallyworld_> i was there in 2008
<wallyworld_> when i worked for Caterpillar
<nigelb> You should visit again!
<wallyworld_> i hope so. i really liked it :-)
<nigelb> Its got a really awesome weather.
<nigelb> I probably won't leave the city just for that.
<wallyworld_> i really liked the food
<wallyworld_> nigelb: but i do have to go, to get to the cbd :-)
<nigelb> :)
 * wallyworld_ runs away
<nigelb> wgrant: You should tweet too. Not just sign up for twitter and follow people ;-)
<wgrant> Maybe :)
<Peng> Where's the fun in *that*?
 * nigelb blinks.
<nigelb> Peng: Are we both on another network as well?
<nigelb> My brain just got horribly confused seeing you in freenode :)
<Peng> Possibly. And it's not an uncommon nick.
<stub> huwshimi: On a call before and cited the wrong bug (not that you need the right one as it is assigned to you!)
<huwshimi> stub: Yeah, all good, I saw that
<stub> huwshimi: So 1) wording and design of the page, or just wording. 2) Register it as a view on storm.exceptions.DisconnectionError 3) Tests to ensure that when the db is pulled out from under lp's feed, the expected 503 error page is returned
<stub> huwshimi: I guess do as much of that as you are comfortable then hand off
<nigelb> Is this the bug about 503 when database is down?
<stub> huwshimi: I think wgrant or myself were going to do the wiring.
<stub> nigelb: I hope so, or I'm confusing the wrong issue.
<huwshimi> stub, nigelb: yeah that's the one
<nigelb> \o/
<huwshimi> stub: Cool. I am working with matthew on 1. I'm just about to head off for the week, but I'll take a look on Monday and get what I can done.
<stub> huwshimi: I suspect you just want to do 1) and hand it off
<wgrant> stub: That's why I didn't give him past step 1 :P
<huwshimi> stub: If that's easier I don't mind.
<stub> wgrant: You expect me to pay attention to scrollback? Sheesh.
<nigelb> heh
<adeuring> good morning
<wgrant> Morning adeuring.
<adeuring> hi wgrant
<jtv> hi adeuring
<adeuring> hi jtv
<nigelb> hi adeuring, jtv
<mrevell> Hello!
<nigelb> hello!
<nigelb> Codehosting throws me twisted.internet.error.CannotListenError: Couldn't listen on any:8022: [Errno 98] Address already in use.
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring| Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<nigelb> Nevermind. fixed.
<nigelb> How do I find what port loggerhead wants?
<nigelb> It seems to be already in use by something else which I want to kill.
<stub> Dunno, but my money would be another loggerhead process left unterminated.
<nigelb> I got it.
<nigelb> it was 8080
<nigelb> I had nginx running there.
<stub> Is Bug #809118  a dupe of Bug #845397 or are these actually separate things?
<_mup_> Bug #809118: builddmaster handling of lp-prod/lp-slave db connection interrupts is untested <fastdowntime> <Launchpad itself:Triaged> < https://launchpad.net/bugs/809118 >
<_mup_> Bug #845397: buildd-manager needs to survive database outages <fastdowntime-later> <Launchpad itself:Triaged> < https://launchpad.net/bugs/845397 >
<nigelb> I'm horribly confused with codehosting running locally
<wgrant> nigelb: Oh?
<nigelb> http://paste.ubuntu.com/690624/
<nigelb> where do I see these OOPS details?
<wgrant> nigelb: Hmm, you've pushed both branches up? It sounds like one or both might not exist.
<nigelb> I couldn't parse that :)
<nigelb> I just pushed one branch
<nigelb> Something into https://code.launchpad.dev/~landscape-developers/landscape/trunk so I could see commit numbers
<wgrant> nigelb: It seemed to be trying to scan two branches.
<wgrant> And generate an MP diff from them.
<nigelb> AH.
<nigelb> Delete the MP?
<wgrant> Not necessary.
<wgrant> Try pushing another branch up, maybe.
<wgrant> And rerunning make sync_branches.
<wgrant> Perhaps it will settle down.
<nigelb> I just deleted.
<nigelb> No jobs to run.
<nigelb> wgrant: http://paste.ubuntu.com/690626/
<nigelb> There.
<wgrant> nigelb: The OOPSes should be in /var/tmp/codehosting.test
<nigelb> Found em. http://paste.ubuntu.com/690627/
<wgrant> nigelb: Hm, apache's notrunning?
<stub> Are 'fiera' users besides the buildd manager fragile for fastdowntime?
<wgrant> stub: No, buildd-manager is the only one.
<stub> ok, I'll remove fiera from the fragile list and just add the new buildd_manager user
<nigelb> wgrant: It is. I see the web page.
<wgrant> nigelb: Even on the bazaar.launchpad.dev IP address?
<nigelb> wgrant: Ah not.
<nigelb> how do I fix?
<wgrant> Something must be wrong with your Apache config... it works by default once rocketfuel-setup has runs.
<nigelb> Hrm.
<nigelb> Do I run rf-setup again?
<nigelb> ah
<nigelb> sudo make install
<nigelb> Still fail.
<nigelb> Warning: DocumentRoot [/var/tmp/bazaar.launchpad.dev/static/] does not exist
<nigelb> There's potential fail there.
<nigelb> wgrant: still around? :)
<wgrant> nigelb: That shouldn't be a failure.
<wgrant> nigelb: It sounds like Apache isn't even listening on 127.0.0.99:80. I'm not sure what could be going on there.
<nigelb> wgrant: It is listening.
<nigelb> I see entries in error.log
<wgrant> Hm, why connection refused, then? :/
<nigelb> bazaar.launchpad.dev sounds like loggerhead.
<nigelb> How can I check if its up?
<wgrant> bazaar.launchpad.dev:80 is Apache, which forwards some requests to loggerhead.
<nigelb> :/
<nigelb> wgrant: Wait a minute.
<nigelb> 80?
<nigelb> I have bazaar.launchpad.dev:443, not 80
<nigelb> Well, that's the one that gets logged.
<wgrant> The branch scanner will want :80
<nigelb> So, you're right.
<nigelb> Something's wrong :/
<bigjools> lifeless: are you around?
<stub> bigjools: https://code.launchpad.net/~stub/launchpad/trivial/+merge/75696
<bigjools> stub: why not just fix the config?
<bigjools> I have a feeling tests depend on it
<stub> bigjools: Cause then other things that use that config item will also move and continue to use buildd-manager's database user
<stub> bigjools: database usernames in configs has been a WHUI anyway
<bigjools> indeed
<bigjools> are you relying on tests failing then?
<stub> bigjools: --test=buildd passes, I'll wait on ec2 to inform me of further fallout
<bigjools> ok
<bigjools> I'll approve it now
<stub> bigjools: I don't think tests will fail though, as permissions of fiera and buildd_manager are identical. So if a test is connecting using the config username, it will still work. Not a good thing long run, but lets us solve the immediate issue easily.
<lifeless> WHUI ?
<lifeless> bigjools: no, but I can be for short bits
<bigjools> lifeless: I wanted to catch up about python-oops
<bigjools> if it's not convenient we can do it Monday
<wgrant> lifeless: We Haven't Used It, I assume.
<jml> it's the past tense of YAGNI
<nigelb> YAGNI?
<StevenK> You Ain't Gonna Need It
<nigelb> heh
<bigjools> GIYF
<StevenK> Give In, You're Fired?
<bigjools> Google Is Your Friend
<StevenK> Mine was better.
<nigelb> Indeed.
<bigjools> Google, In Your Face
<nigelb> Now, who's got a few minutes to help me figure out why bazaar.launchpad.dev:80 isn't working?
<nigelb> (Actually, its why sync branches isn't working, but wgrant and I've gottten it to that point)
<lifeless> bigjools: ah
<lifeless> bigjools: I have no idea if monday will be better, but tonight would be awkward
<bigjools> lifeless: np, it's Friday night.  And I have to go to lunch soon too.
<lifeless> kk, ciao
<bigjools> I'll muddle through for a bit
<bigjools> have a good weekend
<bigjools> wgrant: have you built the txlongpoll branch lately?
<wgrant> bigjools: Not for a while. What's the buildout error?
<bigjools> wgrant: no error, but can't run tests as testresources is not in the path
<wgrant> bigjools: How are you trying to run the tests?
<bigjools> bin/test
<wgrant> :(
<bigjools> as per the README ;)
<bigjools> if I run up bin/py, "import testresources" fails
<wgrant> Yeah, that's meant to work.
<bigjools> so there's a buildout problem
<wgrant> Let's see...
<bigjools> and my ken of buildout is not so good
<wgrant> I learnt quite enough about it during the Thunderdome...
 * wgrant waits three forevers for buildout to finish.
<bigjools> wgrant: ARGH, it's not in install_requires
 * bigjools fixes
<wgrant> bigjools: It shouldn't be.
<wgrant> It's a test dep.
<wgrant> buildout is almost done...
<wgrant> Generated script '/home/wgrant/Development/txlongpoll/trunk/bin/test'.
<wgrant> Finally.
<wgrant> Let's see what happens.
<bigjools> right, it's in extras_require
<bigjools> something else is missing
<wgrant>   File "/home/wgrant/Development/txlongpoll/trunk/txlongpoll/testing/client.py", line 4, in <module>
<wgrant>     from testresources import (
<wgrant> ImportError: cannot import name FixtureResource
<wgrant> That?
<bigjools> yes
<wgrant> Need a new testresources.
<wgrant> I think a bzr snapshot is required.
<bigjools> testresources is not in the pathj
<wgrant> Not for bin/py, no.
<wgrant> Check buildout.cfg
<wgrant> Here's a crash course in the critical bits of buildout for this :)
<wgrant> Each of those sections specifies a recipe. bin/py is built by the 'interpreter
<wgrant> ' section
<wgrant> bin/test by the 'test' section.
<wgrant> The eggs included in each bin/* script are dictated by the eggs directive in each section.
<wgrant> It follows deps.
<wgrant> And you can specify an extra_requires section to use by putting it in square brackets after the egg.
<wgrant> This way the package itself doesn't require rabbitfixture and all that stuff on production.
<nigelb> Can someone pastebin their /etc/apache2/ports.conf file?
<allenap> nigelb: http://paste.ubuntu.com/690715/
<bigjools> wgrant: ok thanks
<nigelb> allenap: thanks!
<bigjools> wgrant: this is far too much like black magic
<wgrant> bigjools: Nah, this is Zope :)
<bigjools> like I said ...
<nigelb> Magic was one of the problems of Zope wwasn't it?
<bigjools> how did we end up with  the tests needing something not in the testresources egg  :/
<bigjools> that's a rhetorical question
<wgrant> I thought that was documented, but maybe not :(
<nigelb> wgrant: is it "normal" for bazaar.launchpad.dev to be directed to launchpad homepage?
<bigjools> the testresources in the lp tree is 0.2.4_r58
<bigjools> as opposed to 0.2.4 in the txlongpoll tree
<bigjools> and it has FixtureResource
<wgrant> bigjools: That is sufficient, I believe.
<wgrant> Yep.
<bigjools> so I'll just copy that egg for now
<wgrant> nigelb: Yes.
<nigelb> \o/
<wgrant> bigjools: You'll probably have to delete the old unpacked egg to make it use the new one.
<bigjools> yes
<nigelb> wgrant: \o/ \o/ \o/
<nigelb> sync_branches worked!
<wgrant> nigelb: It works?
<wgrant> nigelb: What'd you change?
<bigjools> and on that note, FOOD
<nigelb> wgrant: apache ports.conf was listenign to 8009. Because I have 3 webservers on thsi box.
<wgrant> Ah.
<nigelb> I suspected that, thanks to allenap, confirmed, fixed.
 * bigjools fixes setup.py
<nigelb> However, now I know my code doesn't work :D
<nigelb> Yak shaving is so much fun.
<nigelb> (not)
<jtv> wgrant: I have to leave nowish.  Will be back in a few hours, but on a low-bandwidth connection.  But perhaps you could review, and if appropriate land, this?  https://code.launchpad.net/~jtv/launchpad/bug-851684/+merge/75721
<jtv> Anyone else?  I'm in a hurry!
<jtv> adeuring, could you review this one?  https://code.launchpad.net/~jtv/launchpad/bug-851684/+merge/75721
<jtv> I have to run _now_, but I'll be back in an hour or two.
<adeuring> jtv: sure
<jtv> thanks!
<allenap> wgrant, bigjools: Either of you fancy looking at https://code.launchpad.net/~allenap/rabbitfixture/details-breakage-bug-851813/+merge/75726?
<danilos> matsubara, hi, fwiw, I've replied to the oops-tools MP, thanks for the review :)
<matsubara> danilos, cool! I'm going to land and roll that out today. thanks for working on that! :-)
<danilos> matsubara, you are welcome, it was interesting to dive into it a bit :)
<danilos> matsubara, btw, I don't have to send it to PQM for landing? (I've noticed ~launchpad-pqm is the trunk branch owner)
<StevenK> launchpad-developer-dependencies *Depends* on postgresql-8.4-doc? *Seriously*?
<matsubara> danilos, nope, just set a commit message in the MP and tarmac will take care of landing it on trunk
<danilos> matsubara, cool, I will do that then
<allenap> adeuring: Got time for a rabbitfixture branch? https://code.launchpad.net/~allenap/rabbitfixture/details-breakage-bug-851813/+merge/75726
<adeuring> allenap: time isn't a problem. But I have no real clue about rabbit...
<allenap> adeuring: This branch doesn't touch the Rabbit parts, it relates to detail handling in testtools.
<allenap> Don't be alarmed :)
<adeuring> allenap: ok, I'll look ;)
<adeuring> but need a coffee first
<deryck> Morning, all.
<adeuring> morning deryck
<bigjools> allenap: approved
<allenap> bigjools: Ta.
<allenap> adeuring: bigjools just +1'ed it :-/ eek, sorry.
<adeuring> np
<bigjools> ah sorry didn't see you looking at it adeuring
<StevenK> mtaylor: Is it just me, or does the Ec2 plugin for Jenkins not love OpenStack?
<matsubara> danilos, branch failed in the tarmac chroot
<danilos> matsubara, in what way? how can I fix it?
<matsubara> danilos, the followlinks parameter for os.walk isn't available and due to the failure temp directories are not being properly cleaned up
<danilos> matsubara, argh, we are not using 2.6 in there?
<matsubara> danilos, nope, I filed an RT to get that chroot updated to 2.6
<matsubara> looking for the RT number, hang on
<matsubara> https://rt.admin.canonical.com/Ticket/Display.html?id=47591
<matsubara> danilos, ^
<deryck> adeuring, abentley, henninge -- let's Skype today please.  I'll start a conf call.
<danilos> matsubara, right, fwiw, I believe followlinks=False is the default value anyway, I just wanted to be explicit, so I'll remove it and try again
<matsubara> danilos, ok. I think it'll still fail now because of the temp directories that were not cleaned up in the previous run
<danilos> matsubara, right, how can we fix that then?
<matsubara> danilos, one way to fix it is to have a losa run bzr clean-tree in the chroot
<matsubara> danilos, or we could add a bzr clean-tree to the Makefile and it'd be run before make check is run
<abentley> deryck: audio sadness.  rebooting.
<deryck> abentley, ok, cool.
<danilos> matsubara, should I do that?
<matsubara> danilos, bzr clean-tree --unknown --force to clean target should be enough
<matsubara> danilos, please do
<danilos> matsubara, ack, https://pastebin.canonical.com/52911/
<danilos> matsubara, I can probably remove the rms in there then
<matsubara> danilos, nope, because those are in the .bzrignore file
<danilos> matsubara, ah, ok
<danilos> matsubara, yay, merged :)
<matsubara> danilos, great!
<mtaylor> StevenK: probably not just you ... it's on my todo list to finish the jclouds plugin
<allenap> adeuring: Could you review a little fairly simple branch? https://code.launchpad.net/~allenap/launchpad/config-dir/+merge/75737
<adeuring> allenap: sure
<allenap> adeuring: Thanks :)
<adeuring> allenap: r=me; minor comment
<allenap> adeuring: Thanks :)
<benji> I'm having what appear to be SSL problems on a new oneiric LP setup: http://paste.ubuntu.com/690852/ Anyone have any hints?
<jelmer> benji: I've seen that as well on Oneiric
<jelmer> but I haven't figured out what's going wrong exactly
<jelmer> benji: Ian said changing host to hostnossl in pg_hba.conf fixed the issue for him, but that doesn't work here.
<jelmer> benji: it's worth a shot though
<benji> jelmer: thanks, I'll give it a shot
<nigelb> Can I make a branch, & commit to it in a test?
<jelmer> nigelb: of course
<nigelb> \o/
<jelmer> nigelb: if you have a testcase class that derives from bzr's TestCaseWithTransport, it should be as easy as:
<jelmer> tree = self.make_branch_and_tree(path)
<jelmer> revid = tree.commit("My commit message")
<nigelb> AH, currently its factory, so create a new class for this I guess.
<nigelb> let me grep as well.
<nigelb> Thanks!
<jelmer> nigelb:If you're not in a bzr TestCaseWithTransport, you should be able to use bzrlib.bzrdir.BzrDir.create_branch_convenience(path)
<jelmer> but you might have to do some other setup as well (overriding the config, etc)
<nigelb> I'm now thinking if I want all that for my test.
<nigelb> I may not.
<jelmer> I would recommend using TestCaseWithTransport if you reasonably can; it will help with test isolation problems.
<jelmer> e.g. if somebody has setup their bzr to always gpg sign commits you don't want the Launchpad test suite prompting them.
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring, bac| Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<jelmer> g'morning Brad
<bac> hello adeuring.  how's the reviewing today?
<bac> hi jelmer
<adeuring> bac: quiet :)
<bac> adeuring: no backlog?
 * bac looks
<adeuring> bac: yes, there was an MP... got distracted...
<adeuring> jtv: I have 1.5 questions about your MP, see the email
<jtv> hi adeuring!  Yes, I expected as much.  By the way, this is all pretty intensely tested, and functionality doesn't changeâthat's why so few test changes.  Should've explained that.
<adeuring> jtv: so, you think the 65% memory usage are, let's say, tolerable?
<jtv> For just this once?  Absolutely, as long as the script makes enough progress that the problem gets a little smaller every time.
<jtv> It's not really a matter of catching up with changes; it's more of a data conversion.
<jtv> adeuring: answered your questions, I hope.
 * adeuring is looking
<jtv> (Kudos for checking the report.  I think I've mentioned this recently, but I appreciate a thorough review.  :)
<jtv> Since the hardest of these packages appears to be only a few hundred records, even if the memory problem stays, we can go through a lot of packages before it becomes a problem againâand those packages will be permanently âdone.â
<adeuring> jtv: ok, sounds good. But I am slightly worried about txn.commit() in conjunction with two script instances running
<adeuring> I have no clue though if bad things can happen at in this case...
<jtv> adeuring: I don't think the second script instance gets past the startup-and-lock stage, but if it does, transitional domination leaves the last publication record untouched.
<jtv> So in the case where there's no intervening Debian import, AFAICS they would just duplicate each other's work with no ill effects.
<jtv> If there is one, things get a little more complicated.  Let me try to figure it out.
<jtv> (Though as I said, the script does use LaunchpadScript locking so I don't think it would be an issue)
<adeuring> jtv: OK. The alternative would be that this one run that needs a looong time and tons of memory is started manually, while the crontab entry is disabled
<jtv> Yes, that's what I had in mind.
<adeuring> though the losas might not agree to such a fix...
<jtv> We already mentioned it on #-ops and had no objections.
<adeuring> jtv: ok, the r=me
<jtv> It's reassuring that your independent thought processes follow the same linesâmeans less chance that we missed anything.
<jtv> Thanks!
<jtv> hey henninge, what time do you get off work today?
<henninge> jtv: I don't plan to hang around too long ...
<nigelb> henninge: Where are you moving to?
<jtv> henninge: nigelb reminded me that it's almost EOW over there.  As discussed, we'll have things to talk about afterwards but I wanted to give you an official goodbye & thanks for working with us!
<nigelb> :)
<nigelb> HAHAHA
<nigelb> http://www.customink.com/lab/?cid=zab0-000k-p2qg
<henninge> nigelb: hang around *the office* of course ;-)
<nigelb> henninge: haha :)
<henninge> jtv: thank you very much! I enjoyed it greatly, too.
<jtv> henninge: and I'm sure I'm not the first to say it, but if you ever need a reference, no need to ask.  :)
<henninge> jtv: no you are not but I am glad I hear it so much. ;-)
<jtv> :)
<jtv> jelmer: if you're still here, sorry to bother you with it on a Friday afternoon but if you could still get bug 850502 q/a'ed today, that could remove the last blocker for a rather important fix.
<_mup_> Bug #850502: probing smart http servers doesn't work with pycurl <code-import> <http> <qa-ok> <smart-protocol> <Bazaar Git Plugin:Fix Committed by jelmer> <Launchpad itself:Fix Committed by jelmer> < https://launchpad.net/bugs/850502 >
<jelmer> jtv: you are reading my mind
<jtv> Well, it _is_ the same language.  :)
<jtv> Thanks.
<jelmer> jtv: I literally marked it as qa-ok 15 seconds before you asked.
<jelmer> :-)
<jtv> \o/
<jtv> I've got one to Q/A, but I can do it on dogfood together with this fix.
<nigelb> I wonder why my revision isn't ready for QA yet.
<bigjools> jtv: every time I clear my throat, is that some meaningful statement in Dutch?
<jtv> bigjools: yes, and probably about the mother of the enormous Hell's Angel you happen to be talking to.
<jtv> Why do you ask?
<jtv> nigelb: it takes a while for the qa-tagger script to notice.
<nigelb> You've heard of click languages right?
<nigelb> That's means if you don't "chew" carefully, you might be cursing in some language.
<nigelb> What's the difference between a product and a project?
<nigelb> Projects have products?
<nigelb> Is this the way how launchpad project works? with several products in its umbrella?
<nigelb> jtv: still around? :)
<jtv> nigelb: yes, just rather busy with a little crisis.
<nigelb> ah
<nigelb> nevermind, i'll try and figure it out :)
<jtv> Project == Project Group.  Product == Product.
<nigelb> ok, so my guess was right. Cool! (also, psql <3)
<jtv> Left-hand sides are class names, right-hand sides are UI terms.
<jtv> Yes, psql is great.
<sinzui> bac: do you have time to review https://code.launchpad.net/~sinzui/launchpad/person-picker-tab-key/+merge/75771
<nigelb> sinzui: Hi! I have a first cut branch for that bug.
<sinzui> fab
<nigelb> Instead of doing the tal:condition in span, I had to do it in a tal for it to work
<nigelb> Writing tests now :)
<sinzui> nigelb, thanks. I see your diff and It looks as I expect
<nigelb> \o/
<nigelb> If I wwant to do something as admin, do I do this? --> with celebrity_logged_in('admin'):
<nigelb> Not admin, just a user who can see private branches
<nigelb> Or do I create a user and add that user to this product so they can see the branches
<sinzui> nigelb, only an admin and the team with the subscribed team/person can see the branch
<nigelb> hrm, I think I'll use admin.
<sinzui> I see lp.code.browser.tests.test_product has both a revision count test and a private test
<sinzui> I think test_committers_count_private_branch needs one more check to verify that the omitted id is not in the page
<nigelb> argh.
<nigelb> I started writing in lp.code.tests.test_product.
<nigelb> Let me move it.
<sinzui> nigelb, no
<sinzui> there already is a model and a test
<nigelb> Oh.
<nigelb> Ah, I see!
<sinzui> we just need to add an assertion. we can render the view, get the id, and verify it is None
<sinzui> nigelb, I think i misread the last test. It shows the owner sees the info. We want to verify the id is in the page for that test. We want an extra test for a non subscriber using the same steps as the existing test
<nigelb> and not seeing the info.
<nigelb> right.
<nigelb> I've not broken anything in that test. which is not good.
<sinzui> nigelb, This is what I imagined, though have not run the actual test http://pastebin.ubuntu.com/690942/
 * nigelb looks
<sinzui> nigelb, I copied the test comment. I should have changed it
<nigelb> wit
<nigelb> wait
 * nigelb is cnfused
<nigelb> ah, nevermind
<nigelb> I read your diff wrongly :)
<nigelb> sinzui: Thanks for the tests! I was doing it in a roundabout way :)
<sinzui> np. I had these test in mind when I suggested the fix. I should have noted them in the bug
<nigelb> I got two scary tracebacks http://paste.ubuntu.com/690949/
<sinzui> nigelb, I think we need to add the principal arg to get the view
<sinzui> view = create_initialized_view(product, '+code-index', rootsite='code', principal=observer)
<sinzui> use fsm in the first test.
<nigelb> Trying again
<nigelb> sinzui: second test fails with the same error
<sinzui> but the first test now works?
<nigelb> yes
<sinzui> okay
<nigelb> bigjools: HA, 300 has been crossed. I'm a happy man ;)
<bigjools> ?
<bigjools> ah cricket
<nigelb> :)
<bigjools> they'll still lose
<nigelb> And another wide \o/
<bigjools> on the line :/
<nigelb> The bribes we paid the umpires are paying off
 * bigjools off, enjoy your weekend
<sinzui> nigelb, are you missing the + in +code-index in the second test?
<nigelb> sinzui: ah,yes!
<sinzui> The tb shows 'code-index'
<nigelb> YEah :-)
<bac> sinzui: which feature flags do i need to set to see picker work?
<nigelb> And, there was one mistake I think
<sinzui> bac: sorry. I forgot
<nigelb> self.assertEqual(view.committer_count, 0) --> shouldn't that be 1?
<bac> sinzui: np
<bac> sinzui: just disclosure.picker_expander.enabled	default 0	on
<bac> ?
<nigelb> wait. No.
<sinzui> bac, http://pastebin.ubuntu.com/690967/
<bac> sinzui: thanks!
<nigelb> sinzui: Got an interesting assertionError.
<nigelb> henninge: \o/ Have a fun-filled future ahead :-)
<sinzui> nigelb, I think I know what you saw. I should have changed the first assert to check branch_count
<henninge> nigelb: thank you very much!
<sinzui> henninge, It was great working with you
<henninge> sinzui: thanks, same here ;-)
<nigelb> sinzui: I don't understand. Why should it be checking branch count?
<sinzui> nigelb, because you changed the template:
<sinzui> <tal:commit-info condition="view/branch_count">
<sinzui> when that is zero, the commits section is not rendered
<nigelb> Right!
<nigelb> But the count is till 1.
<nigelb> Ah.
<sinzui> view.committer_count is 1 view.branch_count is 0
<sinzui> I misread it too
<nigelb> Should I just check branch count and leave it at that?
<sinzui> nigelb, I think we should check branch_count, revision_count, and the commits node to verify there is something, but we do not want to show it, and we did not show it
<nigelb> So, we verify branch_count = 0, revision_count =1, and then somehow check the view to make sure only branch_count sentence is there.
<nigelb> Is that the right thinking?
<sinzui> yep
<sinzui> nigelb, the latter is done via         commit_section = find_tag_by_id(view.render(), 'commits')
<sinzui>         self.assertIs(None, commit_section)
<nigelb> Aha
<nigelb> So just add branch_count check
<nigelb> and correct the commits count
<sinzui> nigelb, view() or view.render() generates the markup using the principal as the user
<sinzui> nigelb, I think so
<nigelb> \o/
<jtv> bac: if you're looking for something to review, I've got one on the queue: https://code.launchpad.net/~jtv/launchpad/bug-613823/+merge/75773
<bac> jtv: was just about to claim it
<jtv> sweet.  thanks.
<sinzui> jcsackett, Do you have time to mumble?
<jcsackett> sinzui, sure. lemme fire up mumble.
<bac> hi jtv
<jtv> hi
<bac> jtv: in the factory, the logic around setting iscurrent is a little confusing
<bac> jtv: does it default to true when the object is created?
<jtv> Oh, it's just because iscurrent is True when we create the template.
<jtv> Yes.
<bac> oh, ok
<jtv> I could remove the âifâ if you like.
<bac> might be less confusing to just set it.
<jtv> OK
<jtv> bac: pushed the change.
<bac> jtv: thanks
<bac> done jtv.  nice.
<jtv> Thanks!
<barry> flacoste, sinzui and anybody else.  i get an error using 'make schema' in my py27 experiment.  i don't think it's related to py27 though, since pypi only has ipython 0.11 i think:
<barry> While:
<barry>   Installing iharness.
<barry>   Getting distribution for 'ipython==0.9.1'.
<barry> Error: Couldn't install: ipython 0.9.1
<barry>  
<flacoste> barry: do you have an up-to-date download-cache?
<barry> flacoste: i *thought* so since i ran rocketfuel-setup about 2hrs ago :)
<flacoste> you should have: download-cache/dist/ipython-0.9.1.tar.gz
<flacoste> in your tree
<flacoste> barry: did you run link-external-sourcecode?
<barry> flacoste: hmm, yep, it's there.
<flacoste> barry: anything after the Error: Couldn't install: ipython 0.9.1?
<barry> flacoste: ah!  how quickly we forget. and i'm not sure the help wiki descript that
<flacoste> barry: well, if download-cache is in your tree, link-external-source would have done its job
<barry> flacoste: hmm, okay, let me see
<sinzui> link-external-sourcecode is now managed by the make file
<sinzui> make compile will do the necessary buildout
<barry> i got tripped up on `make PYTHON=python2.7 schema`
<barry> i'm trying to create a branch of devel now, but the vm's disk i/o kind of sucks
<nigelb> bac: around?
<sinzui> nigelb, I can review your branch once gthe diff updates
<nigelb> sinzui: Ah, thanks :-)
<nigelb> I thought you left!
<sinzui> nigelb, The branch is good to land. I will do that now
<nigelb> sinzui: Thanks! :-)
<barry> flacoste, sinzui okay, i guess it *is* an issue with py27.  a make schema w/o PYTHON=python2.7 seems to be working okay.  maybe ipython 0.9.1 isn't compatible with python 2.7?
<sinzui> huh
<sinzui> barry, http://ipython.scipy.org/dist/0.9.1/ <- could be true
<barry> sinzui: yeah
<barry> sinzui: i'll see if upgrading my local download-cache to 0.11 makes any difference
<sinzui> ll supports py3. It looks promising
<nigelb> Oh no.
<nigelb> Can someone give me the tracback/details for OOPS-2085QASTAGING54
<nigelb> sinzui: Could you help wwith ^
 * sinzui looks
<nigelb> How did that fail QA. I did all the testing :(
<sinzui> nigelb, the oops is not in the db yet
<sinzui> nigelb, what were you doing?
<nigelb> +check-links
<nigelb> I was QA-ing my fix
<nigelb> Its mostl likely a QA-bad. But I want to see the traceback to confirm
<sinzui> nigelb, https://qastaging.launchpad.net/+check-links ?
<nigelb> Yeah
<sinzui> I do not get an oops, but I only see {}
<nigelb> Well, javascript POSTs bug numbers and branches harvested to it
<barry> sinzui: please remind me.  do i need to do anything more than put ipython-0.11.tar.gz in download-cache/dist and update versions.cfg?
<nigelb> It returns json of stuff that's valid
<nigelb> I'm QA-ing bug 849121
<_mup_> Bug #849121: Autolinked bugs should also have title attribute with bug title <qa-needstesting> <Launchpad itself:Fix Committed by nigelbabu> < https://launchpad.net/bugs/849121 >
<sinzui> barry,  that is all that is required. deleting old version is optional
<barry> sinzui: okay thanks.  trying it
<nigelb> I'm trying to see if the title gets updated to the URL for comment #2 in https://bugs.qastaging.launchpad.net/ubuntu/oneiric/+source/phaseshift/+bug/755937
<_mup_> Bug #755937: phaseshift version 0.40-13.2 failed to build on i386 <ftbfs> <natty> <oneiric> <universe> <phaseshift (Ubuntu):Confirmed> <phaseshift (Ubuntu Oneiric):Confirmed> < https://launchpad.net/bugs/755937 >
<sinzui> nigelb, what url did you use?
<flacoste> sinzui, barry: actually, you need to update the version number in versions.cfg
<barry> flacoste: yep
<flacoste> since we pin version using buildout
<flacoste> ah
<nigelb> sinzui: https://bugs.qastaging.launchpad.net/ubuntu/oneiric/+source/phaseshift/+bug/755937
<_mup_> Bug #755937: phaseshift version 0.40-13.2 failed to build on i386 <ftbfs> <natty> <oneiric> <universe> <phaseshift (Ubuntu):Confirmed> <phaseshift (Ubuntu Oneiric):Confirmed> < https://launchpad.net/bugs/755937 >
<flacoste> barry: i missed the second part "and update versions.cfg'
<flacoste> sorry for being redundant
<barry> no worries!
<barry> flacoste, sinzui: so this should work, right: `make PYTHON=python2.7 schema`
<flacoste> barry: maybe only in a new branch
<barry> flacoste: yep, new branch
<sinzui> nigelb, so you are watching the js console for errors and you see the 500 error for +check-links
<barry> w/versions.cfg changed
<flacoste> barry: since i'm pretty sure the Makefile doesn't version on the value of PYTHON env var
<nigelb> sinzui: Yep
<nigelb> Looking at firebug
<flacoste> barry: what I mean is that bootstrap needs to be run with that env
<barry> flacoste: ah
<flacoste> barry: but i'm not sure it will do the right thing if it was invoked first with python2.6
<barry> right.  fresh branch
<flacoste> barry: you can remove bin/buildout
<flacoste> and then do a make
<flacoste> make PYTHON=python2.7
<flacoste> that should do the trick
<flacoste> since bootstrap will be run in that case
<barry> flacoste: yeah, it still bombs out on ipython 0.11 :/
<nigelb> sinzui: I can't reproduce any problems in my dev setup. So, not a clue what's going wrong.
<flacoste> barry: it fails to build ipython with python2.7?
<nigelb> I'm guessing there could be a timeout issue for the search.
<barry> Getting distribution for 'ipython==0.11'.
<barry> An error occurred when trying to install ipython 0.11. Look above this message for any errors that were output by easy_install.
<barry> While:
<barry>   Installing iharness.
<barry>   Getting distribution for 'ipython==0.11'.
<barry> Error: Couldn't install: ipython 0.11
<barry>  
<barry> % grep ipython versions.cfg
<barry> ipython = 0.11
<barry>  
<barry> % ls ../download-cache/dist/ipython*
<barry> ../download-cache/dist/ipython-0.11.tar.gz
<barry>  
<flacoste> barry: why ../download-cache ?
<sinzui> nigelb, could be
<flacoste> barry: i'd expect ./download-cache
<flacoste> barry: do you a symlink in place?
<flacoste> ln -s ../download-cache .
<barry> flacoste: yep, that symlink is there and ./download-cache also has 0.11
<nigelb> sinzui: The tests cach all the use cases that I'd expect it to catch.  What should I be doing since I can't do a qa-ok with any sort of surety
<nigelb> *catch
<flacoste> barry: are you on lucid or oneiric?
<sinzui> Does staging have the rev yet? It is faster
<barry> flacoste: this is a lucid vm
<nigelb> It just got on qastaging.
<nigelb> I'm not sure eif its not stage
 * nigelb checks
<flacoste> barry: ok, i don't have this setup, but i can try on oneiric to see how far i get
<barry> flacoste: cool, thanks.  i'll keep poking at it too :)
<nigelb> stage is *old*
<sinzui> nigelb, I see staging is about 12 hours behind. I just got my work that landed before I woke
<nigelb> the bottom revision numbers of stage say 10984. Is that wrong?
<nigelb> sinzui: stage  = staging.launchpad.net right?
<sinzui> nigelb, yes
<nigelb> or is it something I don't know of.
<sinzui> nigelb, I am being an idiot. I can paste you the error I see hidden in the browser response
<nigelb> hah
<lifeless> flacoste: did you ask henninge about being in the emeritus team?
<nigelb> Also. 2 days of falling ill. 2 launchpad branches landed. Productive sick time I reckon.
<flacoste> lifeless: i didn't think about it, but he's still subscribed to launchpad-dev
<flacoste> so would probably be interested
<flacoste> barry: it's compiling dependencies over here
<barry> flacoste: hmm
<flacoste> barry: it builds cleanly on Oneiric using python2.7
<flacoste> barry: no need for ipython 0.11
<flacoste> barry: so i suspect a Lucid issue :-/
<barry> flacoste: huh.  yeah.  okay thanks
<flacoste> barry: do you have anything py2.7 eggs built under eggs?
<flacoste> just checking that it's really ipython that doesn't work and not all 2.7 deps
<barry> flacoste: yes, i do have some py27 built eggs in there (e.g. the zope stuff)
<flacoste> barry: ok, so either a buildout issue on Lucid/2.7 or something in ipython specifically
<barry> flacoste: nod
<barry> flacoste: can i get more verbose output out of buildout?
<sinzui> nigelb, sorry, I was pulled into anther conversation: http://pastebin.ubuntu.com/691043/ shows that there can be a mismatch between the set of ids and what is removed
<flacoste> barry: you can try running it manually
<sinzui> nigelb, maybe it is a parsing error
<nigelb> sinzui: Oh. Checking.
<sinzui> nigelb, one option is to look before removing the item
<flacoste> barry:
<flacoste> PYTHONPATH= $(PYTHON) bootstrap.py\
<flacoste>                 --setup-source=ez_setup.py \
<flacoste>                 --download-base=download-cache/dist --eggs=eggs \
<flacoste>                 --version=1.5.1
<flacoste> PYTHONPATH= ./bin/buildout \
<flacoste>                 configuration:instance_name=${LPCONFIG} -c $(BUILDOUT_CFG)
<flacoste> LPCONFIG=development
<flacoste> BUILDOUT_CFG=buildout.cfg
<nigelb> sinzui: *confused* It should be there
<barry> flacoste: okay, manually as per above still bombs out on ipython.  time to pdb i guess ;)
<nigelb> Because I'm checking in the list that I just sent to search.
<flacoste> barry: maybe try buildout ipython?
<flacoste> sorry building
<flacoste> might be because of missing unrepored dependency
<barry> flacoste: good idea
<nigelb> sinzui: Well, it is qa-bad anyway. :(
<barry> flacoste: well, this seems like a bad sign:
<barry> @lessons[/tmp/xx/ipython-0.11:258]% python2.7 setup.py build
<barry> Aborted (core dumped)
<barry>  
<flacoste> lol
<flacoste> yeah
<flacoste> nice euphemism ;-)
<bac> thanks sinzui!
<lifeless> flacoste: I will mail him
<lifeless> flacoste: ah, I only have his canonical address
<barry> flacoste: okay, i'll shut up now
<nigelb> lifeless: its in his email to launchpad-dev
<nigelb> the one about translations
<flacoste> nigelb: i pm it to lifeless
<nigelb> :)
<nigelb> Gah. After python tests and javascript tests I end up with a QA fail.
<lifeless> mailed him
<nigelb> Oh. Its a weekend.
<nigelb> Hrm, does bugtask.search always return a list of unique values?
<nigelb> Oh. FU.
<nigelb> No.
 * nigelb facepalms.
<jcsackett> sinzui: took a bit longer to get the MP to work out than expected, but the branch is up for review with you requested as reviewer. :-)
<cr3> am I supposed to get a file not found error with launchpadlib for the credentials file when used with desktop integration permissions?
<cr3> in unity, is there a shortkey to run a command? I can't access the pannel
<cr3> crap, wrong channel :)
<sinzui> thank you jcsackett
<nigelb> sinzui: Found the problem. Doing as you suggested.
<nigelb> I assumed search would return only unique IDs
<nigelb> I was wrong.
<nigelb> There can be two bugtasks for a bug.
<nigelb> *two or more
<sinzui> nigelb, that is right. we often use a set() object when building a collection to ensure out code sees one one item
<cr3> hm, I like this line in launchpadlib: credential_store = credential_store :)
<nigelb> sinzui: I can't do the set() here because I get a list of bugtasks which are indeed unique. I should have caught this.
<sinzui> nigelb, correct, but this is case were you want bugs, so your could do bugs = set(bugtask.bug for bugtask in bugtasks)
<nigelb> ahhh.
<nigelb> YES
<nigelb> but that's adding another loop
<sinzui> nigelb, you may have fixed bug 262545, or you have made it very easy to fix
<_mup_> Bug #262545: Milestone listings should show the bugs' titles/descriptions in a tool tip <lp-registry> <milestones> <Launchpad itself:Triaged> < https://launchpad.net/bugs/262545 >
<nigelb> I'm thinking the if would be less iterations
<nigelb> oh. looking
<nigelb> sinzui: are you around long?
<nigelb> I have a fix for the qa-bad.
<sinzui> I can linger
<sinzui> I can review your fix and land it
<nigelb> excellent. Give me 5 minutes to push it out
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/bug-title-regression-849121/+merge/75822
<nigelb> Diff shows empty. I'm confused.
<sinzui> I see
<nigelb> Delete and submit again?
 * sinzui looks in loggerhead
<sinzui> nigelb, your change and test looks good in loggerhead
<nigelb> okay \o/
<sinzui> nigelb, I resubmitted: In linkchecker, check if the bug ID exists in the list before removing from the list
<nigelb> Oh, cool.
<sinzui> and there it is
<sinzui> I am approving this and landing it now
<nigelb> Thanks a bunch!
<nigelb> Could you help me figure out what the other bug is about?
<sinzui> jcsackett, I approved your branch
<sinzui> jcsackett, I am sending your branch off to land
<jcsackett> sinzui: thanks.
 * jcsackett really needs to sort out his landing issues.
<nigelb> sinzui: for the milestone bug, does it mean https://launchpad.net/unity/4.0/4.2.0 page
<sinzui> nigelb,  yes, or its alternate url https://launchpad.net/unity/+milestone/4.2.0
<sinzui> The former is a released milestone, the latter always exists
 * nigelb looks
<sinzui> The url maps to the same view
<nigelb> sinzui: I see that the title of the bug is already there
<sinzui> It was once truncated
<sinzui> I think the intent was to load more information on hover, such as full title, description, and bug tags
<sinzui> I think the tags issue was reported as a separate bug
<nigelb> Do we have a hover for more info kind of story anywhere else?
<nigelb> It looks like the +check-links stuff isn't needed here.
<sinzui> nigelb, I agree it is not in this case, but I think you have laid the foundation to solve a class of bugs
<nigelb> \o/
<sinzui> We want to load extra bug information in an asyc fashion, either after the page loads or on hover. Ideally the first, but hover is okay
<nigelb> The foundation was there. I slowly extended it
<nigelb> :)
<sinzui> Showing the title in a tooltip solves the bug id in test case. Showing part of the description or tags allow users looking a lists of bugs to gain information without traversal
<nigelb> I think it would be nice if on clicking the URL, it loaded an "overlay" that would display the summary + tags
<sinzui> nigelb, see bug 401243, bug 1924, bug 86461, bug 262545
<sinzui> I bet we can dupe some of those
<nigelb> I see mpt bugs!
<nigelb> Those usually cannot be duped ;-)
<sinzui> They can when code was merged
<nigelb> I remember seeing 1924. That is a bit of work by itself.
<nigelb> sinzui: Ooh. bug 262545 looks like fun.
<_mup_> Bug #262545: Milestone listings should show the bugs' titles/descriptions in a tool tip <lp-registry> <milestones> <Launchpad itself:Triaged> < https://launchpad.net/bugs/262545 >
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: -| Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated
<sinzui> nigelb, We had timeouts trying to show that data. The hope was to solve the issue after page load. We stopped truncating bug titles since then. Release managers still want to see the tags and the description.
<nigelb> sinzui: We could create a similar interna API to solve this
<nigelb> It should not be *too* hard
<sinzui> nigelb, I was thinking the same since you have demonstrated how to do it
<nigelb> I think I'll pick it up after my current assigned bugs are over
<nigelb> (2 more)
<sinzui> nigelb, fab. I agree it is best not to commit to to many bugs.
<nigelb> :)
#launchpad-dev 2011-09-17
<nigelb> Morning
<StevenK> O hai
<nigelb> \o/ 20 landings
<nigelb> StevenK: How do I know when my db patch is ready for landing again?
<wgrant> nigelb: You don't, really. I'll try to arrange the remaining deployments on Monday.
<nigelb> wgrant: cool, okay.
<StevenK> Well, it can be worked it out. But only if you work for us.
<nigelb> StevenK: I'm happy to. Except I seem to headed to set records for number of rejections :P
<wgrant> Need to arrange cocoplum/germanium/librarian/codehosting/mailman deployments. Shouldn't be too hard.
<lifeless> hmm
<lifeless> why was registry removed from ~launchpad-translators?
<wgrant> lifeless: It looks like it was removed from *everything* :/
<wgrant> Hm, it's actually not been a member of much for ages.
<wgrant> Ah, ~launchpad is in ~rosetta-admins, so we indirectly own ~launchpad-translators. Not sure we were a member, though.
<wgrant> And oh wow that is a lot of mail.
<StevenK> Oh?
<wgrant> Roughly 10 shitloads of cronspam and wikispam.
<StevenK> That much? :-P
<nigelb> You never have too much email.
<nigelb> Unless you have Nagios
<StevenK> Right, Chapter Master finished
<LPCIBot> Project devel build #1,082: FAILURE in 4 hr 16 min: https://lpci.wedontsleep.org/job/devel/1082/
<wgrant> :(
<StevenK> I think I need to disable Jenkins builds, anyway
<wgrant> Why?
<StevenK> $$$$
<wgrant> :(
<wgrant> Probably.
<StevenK> And I can't get OpenStack working with Jenkins.
<StevenK> And the jclouds plugin, which is the new hotness isn't finished and doesn't build.
<LPCIBot> Project db-devel build #875: FAILURE in 3 hr 36 min: https://lpci.wedontsleep.org/job/db-devel/875/
<lifeless> mtaylor: ^
<cjwatson> so how do I upload to dogfood for QA purposes?  it doesn't seem to have a poppy instance running - at any rate, nothing's listening on ftp
<wgrant> cjwatson: Try 2121
<wgrant> cjwatson: Should also have SFTP on 5022.
<cjwatson> wgrant: neither of those is working for me, not even from chinstrap
<StevenK> Lesse
<StevenK> cjwatson: poppy is now running on mawson.
<cjwatson> and on FTP port 21 - thanks, sorted
<wgrant> Ah, someone must have fixed the firewall.
<cjwatson> what's the rune for running the publisher?  LPCONFIG=dogfood /srv/launchpad.net/codelines/current/cronscripts/publish-ftpmaster.py -d deribuntu -q # ?
<wgrant> cjwatson: I normally run publish-distro directly, but yeah, that should work.
<wgrant> cjwatson: You're QAing the translations compression change?
<cjwatson> yep
<cjwatson> if you have the thing you normally run in shell history, that would be ideal, I don't want to be inventing things as I go :)
<StevenK> There's a distro-run script somewhere on mawson, too.
<cjwatson> not finding it ...
<cjwatson> https://dev.launchpad.net/Soyuz/QA was somewhat helpful but only up to a point
<cjwatson> (or do I want to use --all-derived?  https://dev.launchpad.net/Soyuz/TechnicalDetails/Publishing)
 * cjwatson decides to go ahead with publish-ftpmaster -d deribuntu and see what happens
<cjwatson> That seemed to work.  Lots of errors for missing files but they seem unrelated
<cjwatson> http://paste.ubuntu.com/691497/
<wgrant> cjwatson: Yeah, we don't have the whole pool, because it's slow and not small.
<wgrant> cjwatson: That looks fine, then. Thanks for QAing it.
<cjwatson> Marked qa-ok.  Thanks for the guidance
<StevenK> wgrant: Can you think of any reason why queue-builder shouldn't be removed from the tree and burnt at the stake?
<wgrant> StevenK: It's possible we might want to do a rescoring run at some point... maybe.
<wgrant> StevenK: add-missing-builds also only works for a particular DAS, I think.
<wgrant> If we generalise add-missing-builds to also work over a whole archive, or it does already, then we should rip cMB out of queue-builder, at least.
<StevenK> I thought we hadn't run it for years?
<wgrant> I think it was running until early last year.
<wgrant> In --score-only, at least.
<wgrant> I came across its logs a while back.
<wgrant> It was a terrible experience.
<StevenK> Haha
<StevenK> add-missing-builds already takes an archive argument
<wgrant> Yes, and I even made it work for the primary archive.
<wgrant> But it requires an explicit arch list, which is possibly silly.
<wgrant> OK for its original purpose, but that was short-sighted.
<StevenK> So I'm free to gleefully rip cMB out of q-b?
<wgrant>         sources_published = distroseries.getSourcesPublishedForAllArchives()
<wgrant> From q-b.
<wgrant> It works over all archives at once.
<wgrant> (wow, how did that ever not take a week...)
<wgrant> So, q-b still possibly has a purpose.
<wgrant> And it's only like 180 reasonably unoffensive lines.
 * StevenK is tempted to put the final nail in the coffin of canonical.lp
<wgrant> Please don't.
<wgrant> I have big plans for initZopeless.
<wgrant> But there's more that has to be untangled first.
<wgrant> I have a rough idea of how it is going to go down, but need to work out specifics.
<StevenK> wgrant: My plan was to move it away so lib/canonical/lp can be deleted.
<wgrant> No.
<wgrant> That's the easy way out.
<wgrant> But initZopeless is pointless now.
<wgrant> It should die entirely.
<wgrant> Moving it away just hides tech debt.
 * StevenK rewrites lib/canonical/lp/ftests/test_zopeless.py, since it's utterly disgusting.
<wgrant> I deleted most of that last week.
<wgrant> All that's left is that doctest.
<StevenK> But it isn't even a doctest!
<wgrant> It is a doctest in a unittest!
<wgrant> Things that remain:
<StevenK> Which is disgusting
<wgrant>  - untangling our dbuser change support.
<wgrant>  - renaming "Zopeless" mail to non-queued mail
<wgrant>    (and detaching it from initZopeless)
<wgrant>  - Letting FunctionalLayer users specify a security policy.
<wgrant>  - Porting *ZopelessLayer users to *FunctionalLayer, using PermissiveSecurityPolicy
<wgrant>  - Delete initZopeless, because it no longer has a purpose.
<wgrant> (Zopeless currently means three things: 1) non-queued mail, 2) DB user and isolation level reconfiguration, 3) PermissiveSecurityPolicy)
<wgrant> I've got rid of most of the terrible old code, but we still have this awful tangled wrapper.
<wgrant> Conflating these three largely unrelated things.
<StevenK> wgrant: http://pastebin.ubuntu.com/691519/
<wgrant> StevenK: You could do that, but the sole remaining callsite of isZopeless is Zopeless mail, which is next on the list to remove.
<wgrant> So I would be surprised if that test wasn't deleted within a fortnight.
<nigelb> Gah
<nigelb> My nightmare with that bug continues
<nigelb> wgrant / StevenK around?
<nigelb> Hrm,looking at the time, I guess not.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #876: FIXED in 4 hr 50 min: https://lpci.wedontsleep.org/job/db-devel/876/
<nigelb> If anyone has a few minutes to review an MP for a regression that'd be great! https://code.launchpad.net/~nigelbabu/launchpad/js-regression-849121/+merge/75858
<lifeless> and again, deactivated. wtf
<lifeless> should be ban untaintableangel ?
#launchpad-dev 2011-09-18
<wgrant> nigelb: Are you sure that branch fixes it completely?
<wgrant> nigelb: If you're not pretty sure that's enough to totally fix it, we need to revert your revision.
<nigelb> wgrant: as embarassing as it sounds, yes.
<nigelb> I added tests
<nigelb> *more* tests
<nigelb> wgrant: Is there more I could do to actually prove it works?
<nigelb> Is there a test server I could push to?
<nigelb> without getting it committed that is.
<wgrant> I'll try merging it on DF.
<nigelb> DF?
<wgrant> dogfood.launchpad.net
<wgrant> Normally used for Soyuz testing.
<nigelb> Can I visit it?
<nigelb> i.e., can people not in ~launchpad visit it.
<wgrant> yes.
<nigelb> Ok, if you can get it running there, that'd be great.
<wgrant> Although it's very slow, as everything including the database runs on a machine from 2004.
<nigelb> I just need to load a bug.
<nigelb> wow, OOOPs all over the place.
<wgrant> Ah, yeah, loooks like we might have issues with testing this on DF.
<wgrant> Because its database is old.
<wgrant> And doesn't have BugMessage.owner set.
<nigelb> ah
<nigelb> argh
<nigelb> let me try something
<wgrant> So, I think I need to roll this back, and perhaps you can ask a LOSA to merge it temporarily on (qa)staging next week.
<wgrant> So you can test it out.
<nigelb> Is it possible to do a merge on DF?
<nigelb> i.e. code merge
<wgrant> Yes, but its DF is crap so that won't be much use.
<wgrant> Er.
<nigelb> \o/
<wgrant> s/DF/DB/
<nigelb> Okay, so I can see a merge
<nigelb> If you can push the code, I can test it in an MP as well.
<wgrant> You'll have to create a new MP.
<nigelb> I created one
<nigelb> https://code.dogfood.launchpad.net/~summit-hackers/summit/i18n/+merge/60007
<wgrant> JS should be updated
<nigelb> Code's still old.
<wgrant> Which code?
<nigelb> linkchecker.py needs to be updated as well
<wgrant> Hmm, but it should already be running the latest devel revision...
<nigelb> Its r13954
<nigelb> While qastaging is r13978
<wgrant> Well, it says that, but it often lies.
<wgrant> Let's see.
<nigelb> I know that linkchecker.py doesn't contain my changes, because I'm getting what used to happen earlier.
<nigelb> I'm guessing you just brought it down
<wgrant> It's restarting.
<wgrant> Slowly.
<wgrant> And it's back.
<wgrant> Hopefully with the new Python code this time.
 * nigelb does hard refresh
<nigelb> wgrant: \o/
<nigelb> Seems to work!
<wgrant> Do you have a bug that would be interesting to test? I can fix particular bugs to render on DF>
<nigelb> one sec
<nigelb> let me see the one we were testing on qastaging
<nigelb> bug 755937
<_mup_> Bug #755937: phaseshift version 0.40-13.2 failed to build on i386 <ftbfs> <natty> <oneiric> <universe> <phaseshift (Ubuntu):Confirmed> <phaseshift (Ubuntu Oneiric):Confirmed> < https://launchpad.net/bugs/755937 >
<nigelb> could you fix that on DF?
<wgrant> It renders now.
<nigelb> Trying to duplicate
<nigelb> Gah.
<nigelb> I'm awesome
<wgrant> Oh?
<nigelb> I made the qastaging bug render :p
<wgrant> Heh
<nigelb> Now I can't use that to compare
<nigelb> Works as expected in dogfood.
<nigelb> let me violate another qastaging bug :P
<nigelb> ZOMG.
<nigelb> QAstating did *not* timeout on me.
<nigelb> This needs celebration.
<nigelb> (spoke to soon. Damn)
<nigelb> wgrant: Seems to work like it should
 * wgrant lands.
<nigelb> wgrant: Sorry about the mess :)
<nigelb> I think I've made a whole lot of revisions undeployable
<wgrant> Indeed, but it's by no means the worst we've had recently :)
<nigelb> HA
<nigelb> I'm really glad I invested in the time to write javascript tests
<nigelb> I would never have fixed this on time without those
<wgrant> Yes, tests are handy :)
<wgrant> nigelb: Could you set a commit message?
<nigelb> yeah, sec
<nigelb> wgrant: done
<nigelb> wgrant: Oh. No test run?
<wgrant> nigelb: No point. It's a weekend, and the deployment pipeline is blocked until this is fixed anyway.
<nigelb> \o/
<wgrant> Plus it's a safe change.
<nigelb> Right. You just ran it in dogfood and I didn't bring it down :-)
<LPCIBot> Project devel build #1,083: STILL FAILING in 36 sec: https://lpci.wedontsleep.org/job/devel/1083/
<nigelb> ^ Didn't even start
<LPCIBot> Project devel build #1,084: STILL FAILING in 1.7 sec: https://lpci.wedontsleep.org/job/devel/1084/
<LPCIBot> Project devel build #1,085: STILL FAILING in 1.8 sec: https://lpci.wedontsleep.org/job/devel/1085/
<LPCIBot> Project devel build #1,086: STILL FAILING in 1.7 sec: https://lpci.wedontsleep.org/job/devel/1086/
<LPCIBot> Project devel build #1,087: STILL FAILING in 1.7 sec: https://lpci.wedontsleep.org/job/devel/1087/
<LPCIBot> Project devel build #1,088: STILL FAILING in 1.8 sec: https://lpci.wedontsleep.org/job/devel/1088/
<LPCIBot> Project devel build #1,089: STILL FAILING in 1.7 sec: https://lpci.wedontsleep.org/job/devel/1089/
<LPCIBot> Project devel build #1,090: STILL FAILING in 1.7 sec: https://lpci.wedontsleep.org/job/devel/1090/
<wgrant> cjwatson: Is remove-package.py used at all these days? It was superseded by lp-remove-package.py years ago, but never deleted...
<StevenK> I can not recall remove-package being used while I've been an AA
<wgrant> I'm waiting for mawson to tell me if anyone has.
<StevenK> It imports dak_utils for crying out loud
<wgrant> The last source it was used on was in 2007... binaries I'm still waiting for.
<wgrant> Yep.
<wgrant> But then again so does sync-source...
<StevenK> steven@liquified:~/launchpad/lp-branches/devel% grep -c '^#' scripts/ftpmaster-tools/remove-package.py
<StevenK> 145
<wgrant> But that can hopefully die soon.
<wgrant> Yes.
<wgrant> There's a revdep check all commented out in there :/
<StevenK> Yes.
<StevenK> Because removing code is hard, or something.
<StevenK> An XXX from elmo. Neat.
<StevenK> That is disgusting, kill it.
<wgrant> It's already gone.
<wgrant> Just wanting confirmation.
 * StevenK stares at scripts/_ginalog.py
<wgrant> Yes, pretty much.
<wgrant> But it doesn't use initZopeless, so this branch won't delete it.
<StevenK> Oh, nice, you're killing it already?
<wgrant> Well, porting most scripts that use it directly to use LaunchpadScript instead.
<wgrant> And deleting lots of scripts that use it but don't work any more.
<StevenK> Does that mean canonical.lp dies?
<wgrant> Hopefully in the next couple of days.
<wgrant> Still a little bit to go.
<wgrant> Hmm, untested script written in 2006 and unchanged since then except for compatibility fixes.
<wgrant> I think it can die.
<StevenK> Which one?
<wgrant> scripts/rosetta/check-distroseries-translations-diffs.py
<StevenK> Nothing seems to reference scripts/_ginalog.py, I'm tempted to just delete it.
<wgrant> I believe that would be the correct course of action.
<wgrant> 540KB of accidental addition, I suspect.
<wgrant> It hasn't changed since it was added around r1149
<StevenK>  1 file changed, 15640 deletions(-)
<wgrant> Heh
<nigelb> Woah
<nigelb> With the amount of "die" and "kill" you both use, a casual observer would think you both are professional hitmen.
<StevenK> I used 'rm' this time!
<nigelb> ha
<StevenK> wgrant: Tossed at PQM
<wgrant> The code with the lowest maintenance cost is code that doesn't exist :)
<nigelb> +1 to that
<StevenK> Neat. r13980
<nigelb> 20 more revs to r14000. Neat.
<wgrant> StevenK: Have you read sync-source.py lately?
<wgrant> def init(): global Blacklisted, Library, Lock, Log, Options
<wgrant> That's a pretty good summary of its style.
<StevenK> I don't think I want to.
<wgrant>  31 files changed, 558 insertions(+), 2198 deletions(-)
<wgrant> And canonical.lp is dead.
<StevenK> \o/
<StevenK> Have you pushed it?
<wgrant> The three branches are in ec2.
<wgrant> Will see how much is broken.
<wgrant> Ah, this is handy.
<wgrant> I think our manual escaping stuff might do the wrong thing with postgres 9.1's escaping changes.
<wgrant> :(
<cjwatson> wgrant: yes, feel free to kill remove-package.py
<wgrant> cjwatson: Thanks.
<cjwatson> we still need sync-source.py for a while until a few more bugs in the new-style thing are worked out
<wgrant> Yep.
<cjwatson> particularly sponsorship
<cjwatson> *sigh* must find time to finish writing that autosync API script too
<wgrant> cjwatson: Are archive-{integrity,override,cruft}-check used?
<cjwatson> cruft is very heavily used.
<wgrant> Ah, so we still use it for NBS?
<cjwatson> I can't remember about the other two.
<cjwatson> Yes.
<cjwatson> I kind of feel like we ought to use integrity but I haven't done so for ages.
<wgrant> It also does ASBA, which is a bit odd.
<wgrant> Yeah, I want to run something like integrity, that basically compares the pool with the DB.
<wgrant> Because there is heaps of cruft there, and some stuff is probably missing.
<cjwatson> I think this is the first time I've heard of archive-override-check.  Exactly what "inconsistences" [sic] does it report on?
<wgrant> It's the one I don't know about, too.
<wgrant> Let me read it.
<wgrant> I think it might check consistency between architectures.
<cjwatson> So like http://people.canonical.com/~ubuntu-archive/architecture-mismatches.txt except unused? :-)
<cjwatson> That report is just done based on Packages and Sources files.
<wgrant> Indeed, probably. I assumed NBS there was similar -- didn't realise you actually used archive-cruft-check.
<cjwatson> So, OK, it'll miss things with inconsistent overrides that aren't built, but ...
<cjwatson> Yeah, we use it as the first stage of the input
<cjwatson> I wouldn't object to rewriting it at some point
<cjwatson> http://paste.ubuntu.com/692165/ - don't vomit all at once
<wgrant> I plan to write a new integrity checker in the short term, but have no plans for cruft, apart from the port to LaunchpadScript that I did this afternoon.
<wgrant> Not bad, not bad.
<wgrant> That explains why the output was unrecognisable.
<cjwatson> I particularly like the grep '^ *o '
<wgrant> Yeah.
<wgrant> AFAICT archive-cruft-check's ASBA support is unused and doesn't even make sense.
<wgrant> Perhaps it originated with dak.
<cjwatson> I don't believe I've used it for some time
<cjwatson> If you want to delete that part, that's fine by me; or you could just ignore it until I rewrite cron.NBS and propose a branch that removes the script entirely
<cjwatson> OK, WTF was _ginalog.py about?
<cjwatson> Oh.  Wow.  ArchiveCruftChecker doesn't even talk to the database except to get the current distroseries and such (and the removal code which we don't use).  I could just pull that out wholesale.
<cjwatson> (Perhaps not on a Sunday morning though.)
<wgrant> cjwatson: Yeah, most of archive-*-checker don't really use the DB for anything useful.
<wgrant> There's a *lot* of cruft in LP :)
<wgrant> nigelb: Thanks.
<nigelb> wgrant: \o/
<nigelb> Not my happiest moment. Really.
<wgrant> Heh :)
<StevenK> Hehe, even cjwatson comments on _gina log
<m4n1sh> where do all the translation strings like here come from? Do they come from po/messages.pot file present in branch associated with series which is focus of development?
<nigelb> Not the best of times. Oceania still hasn't woken up.
<dobey> m4n1sh: "like here" ?
<dobey> m4n1sh: also, Oceania is at war with East Asia, so that whole thing might get in the way. ;)
<m4n1sh> errrr? had too much of whisky?
<m4n1sh> :)
<dobey> no
<dobey> did you miss the 2 minutes hate, brother?
<mwhudson> morning
<dobey> m4n1sh: http://en.wikipedia.org/wiki/Nineteen_Eighty-Four#The_War
<m4n1sh> ahhh
<m4n1sh> got it
<m4n1sh> :)
<dobey> m4n1sh: anyway, i'm not sure what you mean by 'like here' given you provided no link :)
<m4n1sh> yes
<m4n1sh> I know
<m4n1sh> that was the mistake
<m4n1sh> I realized it just now
<m4n1sh> it was the http://translations.launchpad.net/pinta
<m4n1sh> page
<m4n1sh> was trying my hands on gettext and all those, not able to understand many things
<dobey> ah; i think translations on a project are imported from the .pot file in the series, if there is one
<dobey> ugh
<dobey> what the heck is pinta doing
<dobey> eww
<m4n1sh> dobey: where?
<dobey> with translations
<m4n1sh> it got a rebirth
<m4n1sh> I dont think the translations work properly
<m4n1sh> the translations should be named like es.pot, ro.pot etc
<dobey> it's doing weird stuff manually with gettext, i presume to try and support windows also
<dobey> no
<m4n1sh> like locale.pot
<dobey> no
<m4n1sh> right now it is messages-<locale>.pot
<dobey> there is only one pot file
<m4n1sh> sorry
<dobey> no, it's .po
<m4n1sh> I mean
<m4n1sh> right now it is messages-<locale>.po
<m4n1sh> it should be just locale.po
<dobey> well, a lot of things should be different
<m4n1sh> but all the files in pinta repo is named messages-<locale>.po
<m4n1sh> differnet like?
<dobey> it's not clear how to me pinta expects to support windows/osx though
<m4n1sh> it does
<dobey> well, "messages" is not a proper gettext package name
<dobey> m4n1sh: i know it does. i mean in the technical sense
<dobey> ie, a single .exe, or does it require cygwin to work, or what
<m4n1sh> dobey: it works on windows and osx too
<m4n1sh> over mono
<m4n1sh> runs over mono for windows
<m4n1sh> or probably even .NET
<dobey> yes but that doesn't tell me anything
<m4n1sh> https://github.com/PintaProject/Pinta/tree/master/po
<m4n1sh> check this
<dobey> i looked at it
<dobey> but it doesn't tell me anything about it, other than it's being done wrong
<m4n1sh> lol
<dobey> anyway, not an issue with launchpad itself
<m4n1sh> I dont think so
<m4n1sh> launchpad exports it as <locale>.po
<m4n1sh> AFAIK
<m4n1sh> this messages-foo thing breaks pure gettext based files
<m4n1sh> like pinta.desktop.in
<dobey> yes, like i said. the way pinta is doing translations is totally broken :)
<m4n1sh> I get entries in pinta.desktop as
<m4n1sh> Name[messages-ro] = blah blah
<m4n1sh> I expect to get
<m4n1sh> Name[ro] = blab blah
<dobey> eh?
<m4n1sh> yes
<m4n1sh> after running make I get "Name[messages-ro] = blah blah" in pinta.desktop file
<m4n1sh> not what I expect
<dobey> i don't see how
<dobey> are you building/installing on windows?
<m4n1sh> no
<m4n1sh> Ubuntu
<dobey> is bzr way behind what's in git? or do you have some patch to make it build?
<m4n1sh> it looks for all the files in po/ dir and checks for the string I Used in _Name
<m4n1sh> so the translation for that string is in messages-ro.po
<m4n1sh> so it substitues Name[messages-ro] = fo fo
<m4n1sh> dobey: this way is not related to git or bzr
<dobey> is there a pinta irc channel?
<m4n1sh> yes
<m4n1sh> dead
<m4n1sh> only me and Laney are there :)
<m4n1sh> this gettext thing is done by intltool and autotools
<m4n1sh> two highly confusing things
<m4n1sh> nothing git or bzr specific
<dobey> let's move discussion there
<m4n1sh> yes
<lifeless> :P
<nigelb> morning lifeless
<lifeless> hi nigelb
<nigelb> Sleepless nights are so not fun.
<nigelb> wallyworld_: \o/ I landed that bug title via XHR that I was working on! After qa-bad twice though :(  (wgrant may stab me anytime :P)
<wallyworld_> nigelb: excellent. well done. i've been fighting with unity this morning :-(
<nigelb> Ouch, that sounds like a bad way to start a Monday
<wallyworld_> yep. been bad for a few days. hopefully beta2 will be better
<wgrant> lifeless: Are we still forbidden from fastdowntime during beta week?
<lifeless> wgrant: on some days yes
<wgrant> lifeless: Wed/Thu I could understand, but it seems like two minutes early in the week shouldn't hurt.
<wgrant> I guess it depends when they aim to have images, too.
 * wgrant stabs Optus a bit.
<cjwatson> I wouldn't expect a Monday fastdowntime to be a problem.  After that I think we'd prefer you held off
<cjwatson> and Friday should be OK, given that the main bit that isn't fast yet is the publisher downtime
<wgrant> cjwatson: Thanks.
<lifeless> cjwatson: wgrant: I think kate asked for wed/thu
<lifeless> skaet: ^ was it those two days you wanted blacklisted, or tuesday as well ?
<wgrant> Does she realise it means missing a publisher run?
<wgrant> Although it may be on manual at that point anyway.
<lifeless> https://code.launchpad.net/~lifeless/launchpad/use-oops-timeline/+merge/75935
<lifeless> wgrant: does it ?
<wgrant> lifeless: It finishes at :26-33 or so, so we disable it an hour before.
<wgrant> Meaning that the 0802 run doesn't happen.
#launchpad-dev 2012-09-10
<StevenK> Bleh, webservice.patch is giving me 401 :-(
<StevenK> How am getting ValueError: (<Bug at 0x112af190>, 'subscribe', 'launchpad.Edit'), the webservice is for the owner of the bug :-(
<wgrant> StevenK: The owner doesn't always have access if it's private
<StevenK> Which it isn't.
<StevenK> wgrant: Diff is at http://pastebin.ubuntu.com/1195794/ , test output at http://pastebin.ubuntu.com/1195796/
<wgrant> StevenK: Have you checked that webservice_for is not defaulting to an RO token?
<lifeless> or anonymous ?
<wgrant> It's not anonymous, since a person is being passed in
<StevenK> Oh, bleh, I bet that's it.
<StevenK> permission=OAuthPermission.READ_PUBLIC
<StevenK> Yeah
<wgrant> :D
<StevenK> lifeless: You'll never guess what the fix is.
<wgrant> Is it just a matter of checking in the subscriber if the modification event actually has any modifications?
<lifeless> StevenK: don't use the API ?
<StevenK> if event.edited_fields is None: return
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/subscription-no-longer-updates/+merge/123467
<wallyworld> StevenK: busy, will look soon
<StevenK> wgrant: Do you think 1021773 and 1035338 could be dupes? I can not load the oops to compare them.
<wgrant> StevenK: Yes, they're probably dupes
<wgrant> And they're probably related to a lot of the scanner timeouts too
<wgrant> Some thing or things are holding locks on branch for many seconds
<wgrant> StevenK: What if edited_fields is an empty list or so?
<wgrant> We probably just care if it evaluates to boolean false
<StevenK> wgrant: I can change that bit
<StevenK> wgrant: http://pastebin.ubuntu.com/1195864/  Is that your only concern?
<wgrant> StevenK: I think so
<wgrant> StevenK: Are there similar subscribers on other objects?
<StevenK> wgrant: I have no idea -- I didn't think anything but bugs used Zope subscribers.
<StevenK> And I didn't know bugs used them until Friday afternoon.
<wgrant> It's largely localised to Bugs, but I'd look around just in case
<lifeless> bugs
<lifeless> answers
<lifeless> soyuz has some bits I think
<lifeless> code
<lifeless> localised, no, centre of gravity, yes.
<wgrant> Soyuz doesn't really use them
<wgrant> Certainly not for ObjectModifiedEvents
<StevenK> Soyuz has one.
<wgrant> Oh?
<StevenK>     <subscriber
<wgrant> In other news, where are my OOPS reports
<StevenK>         for="lp.soyuz.interfaces.archivesubscriber.IArchiveSubscriber
<StevenK>              lazr.lifecycle.interfaces.IObjectCreatedEvent"
<StevenK>         handler="lp.soyuz.mail.notifications.notify_new_ppa_subscription"/>
<wgrant> Ah
<StevenK> wgrant: I can't think of a similar pattern off the top of my head, anyway. We call a method on X, don't modify X and return Y
<wgrant> I'd look around for the other ObjectModifiedEvent subscribers and see what they do
<wgrant> 24460 0 Archive:EntryResource:getPublishedBinaries
<wgrant> ouch
<lifeless> not deployed yet ?
<wgrant> A couple of hours ago
<wgrant> So tomorrow's report should be clean
<wgrant> hopefully
<StevenK> wgrant: Everything looks fine -- I'm not sure if subscribing to a branch over the API causes the same issue.
<StevenK> wgrant: Which is immune, so everything is fine.
<wgrant> OK
<wgrant> r=me, thanks
<lifeless> review plox: https://code.launchpad.net/~lifeless/js-oopsd/post/+merge/123474
<lifeless> wallyworld: or StevenK: ^ wgrant, is obsessing on performance.
<wgrant> Correctly obsessing; this missing index is at the root of all but one known disclosure performance regression :)
<wallyworld> otp, just a sec
<wallyworld> wgrant: what missing index is that?
 * StevenK stabs bugactivity
<lifeless> wallyworld: the inverse index on a M:M join table
<wgrant> wallyworld: See #launchpad-ops. accesspolicygrant(grantee) isn't indexed, so finding a list of policies that a person can see uses accesspolicygrant(policy, grantee)
<wallyworld> which M:M table?
<wgrant> And takes 100ms rather than 1ms
<wallyworld> ok
<wgrant> This fixes all the branch access checks
<wallyworld> excellent
<wgrant> It possibly won't fix code.launchpad.net/ubuntu, as the MP query there is pretty obscene, but I have a simple query workaround for that
<StevenK> Sigh, it's been so long since I've had do preloading, I've forgotten what to look for. :-(
 * lifeless begs for a review
<jam> lifeless: in your overview, you say any browser to do "POST" or "GET", but in your "Allow-Methods" header, you have "PUT", "DELETE", am I missing something there?
<jam> and your test seems a little bit circular (rather than asserting what the actual content is, it asserts that the content matches the list that you say it should be)
<jam> so if I put whatever I want it cors_headers, the test still passes.
<jam> (vs saying "GET" is in the list of Access-Control-Allow-Methods)
<jam> so it depends what you want to assert
<lifeless> thats true
<jam> if you are asserting that passing the cors_headers is hooked up == good
<lifeless> I had a more specific test but perhaps over enthusiastically refactored
<jam> asserting that it conforms to the spec == better.
<lifeless> the it should be GET+POST
<lifeless> jam: I'll a) fix the static set of headers and b) convert the preflight check to be manual rather than circular.
<jam> lifeless: aside from that sanity check, I don't know that it is worth having *me* fully understand CORS, so I'll trust that we can iterate until everything works.
<jam> (If it doesn't work properly in browsers, then we update the test suite until it does)
<lifeless> I'm not sure anyone really understands CORS.
<lifeless> I've not read a more confusing spec.
<jam> lifeless: which is why I think iterating towards 'working' is good.
<jam> As browsers may not conform at all to the spec either.
<lifeless> that too
<jam> confusing specs tend not to be followed by anyone very well :)
<jam> lifeless: review posted
<lifeless> jam: I've pushed a revised version
<lifeless> jam: if there was nothing more in the review, it should be good
<jam> lifeless: the only other thing I thought about, do we want to have a wide-open access policy?
<lifeless> also need a review for https://code.launchpad.net/~lifeless/python-oops-tools/bug-1048470/+merge/123480 please :)
<lifeless> jam: yes.
<jam> lifeless: mostly just making sure that was true. It might be good to have a code-level comment about it.
<lifeless> jam: Folk that want to attack us can just ignore the policy entirely.
<jam> as it looks like a security hole
<jam> I thought it is probably ok
<lifeless> jam: we can't ever trust the data we get for this service at all.
<jam> lifeless: aiui, this is about the fact that launchpad.net may send out a javascript header, but we want to post the OOPS to oops.canonical.com, right?
<jam> and anyone can just post to oops.canonical.com
<jam> and this is just about telling javascript that it is ok that even though launchpad.net gave you the JS file, it can send data to oops.
<lifeless> yes, or u1, or ...
<jam> lifeless: so you've gotten an approve* from me
<lifeless> tight rules are used to prevent query methods that return sensitive data, and are secured by cookies or whatever, from being fooled via CRSF attacks and the like
<jam> not sure if you need someone official on it :)
<lifeless> jam: you shold be official
<lifeless> wallyworld: thanks! I totally don't know why I didn't thinkyou'd reviewed.
<lifeless> wallyworld: slow mail day or something.
<wallyworld> np
<jam> lifeless: I've never been officially given the 'launchpad reviewer' title, at least that I can remember.
<lifeless> jam: you're in the group; do you have a mentor reviewer to give you the LP magic sauce ?
<StevenK> TAL, please die in a large chemical fire. http://pastebin.ubuntu.com/1196061/
<lifeless> wallyworld: if I can trouble you again .. https://code.launchpad.net/~lifeless/python-oops-tools/bug-1048470/+merge/123480
<jam> lifeless: https://code.launchpad.net/~lifeless/python-oops-tools/bug-1048470/+merge/123480 just to validate assumptions, the decoding of 'data' already maps things to Unicode, so that we don't have to worry that the data might be a UTF8 string that we are then doing .encode('utf8') on (which will break with the magic decode step).
<lifeless> wallyworld: its about 4 lines of code
<wallyworld> sure
<lifeless> wallyworld: or... jam has it :P
<lifeless> wallyworld: so no worries
<lifeless> jam: right, its coming out of the ORM
<lifeless> jam: so its always unicode.
<lifeless> jam: its assigned slightly higher in the file
<jam> lifeless: so the thing you changed was 'pageid' related, but the test is testing 'topic'.
<lifeless> jam: topic in the wire protocol is mapped to pageid in the schema
<lifeless> the schema is old and rather more LP centric than the wire protocol.
<lifeless> creaky even
<jam> lifeless: I don't think I've ever had an official mentor for obtaining LP secret sauce.
<lifeless> jam: so, lets get you one. I know, gmb, he has nothing to do :P
<lifeless> jam: you clearly know python and how to do code review; the only thing you need is to convince a mentor reviewer than you can tell the difference between code you grok and code that will contain surprising gems.
<lifeless> lets see what tarmac thinks of that one
<wallyworld> lifeless: should there also be a test for the utf8 encoded oopsid?
<StevenK> wgrant: http://pastebin.ubuntu.com/1196061/
<lifeless> wallyworld: mmm, I only changed that because the test I wrote caught it :)
<lifeless> wallyworld: so, I don't think an explicit one is called for, no. This whole code could be radically improved by using a template engine.
<wallyworld> ah, ok. gotta love tests :-)
<wallyworld> +1 to the template engine
<lifeless> yeah, I added the test to validate my assumption about the cause of the problem, and then fixed everything it uncovered.
<lifeless> Stuff trying to understand this stuff :>
<wgrant> StevenK: It's probably raising an AttributeError because your code is terrible
<wgrant> Anywhere inside the attribute it complains about
<wgrant> An AttributeError will just be squashed into that useless LocationError
<StevenK> wgrant: I didn't think http://pastebin.ubuntu.com/1196075/ was terrible :-(
<wgrant> StevenK: Possibly personID isn't on IBugActivity
<wgrant> So it's not accessible
<StevenK> ForbiddenAttribute: ('personID', <BugActivity at 0xc852410>)
<StevenK> But it's there if you dropkick the security proxy
<lifeless> that would be the attribute not being on the interface
<StevenK> It's SQLBase, so I have no idea why it isn't.
<lifeless> because they aren't magic.
<lifeless> They have to be explicitly listed like all other attributes.
<wgrant> The interface is not SQLBase
<wgrant> SQLBase only affects the model
<StevenK> Yeah, I realized after lifeless' comment
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/bugtask-activity-preload/+merge/123484
<wgrant> StevenK: Do you really care about just ValidPersonCache?
<wgrant> You probably want the Person too
<wgrant> Which means you want PersonSet.getPrecachedPersonsFromIDs
<StevenK> wgrant: It's strange, if I did load_related(Person, ...) the query count went to 15.
<wgrant> wallyworld: I was able to reproduce the request a review bug on prod a couple of hours ago
<wgrant> wallyworld: It works fine until you click to submit
<wgrant> Then it just disappears and does nothing
<wallyworld> wgrant: worked for me locally
<wallyworld> i didn't hit submit on prod, i'll try on qas
<wgrant> wallyworld: With the yui 3.5 FF set?
<wallyworld> i think so, unless i've remade my schema, which i may have done
<wallyworld> the bug said the link wasn't even clickable
<wallyworld> which is not the case anymore
<wgrant> It doesn't say that the link was unclickable
<wgrant> It even says "I was trying to request lifeless review a db patch branch.", which suggests it died post-input
<wallyworld> it says the console shows an error when it is clicked
<wallyworld> which implies the link didn't work
<wgrant> It doesn't say anything about clicking, just that the green link was being used
<wallyworld> how do you ue a link without clicking it?
<wgrant> You don't, but it doesn't say it failed during the clicking of the link
<wgrant> Just that it failed some time while using the functionality behind that link
<StevenK> wgrant: http://pastebin.ubuntu.com/1196108/ when preloading Person only
<wallyworld> sure, but i seem to have a recollection i saw the same issue and the link itself didn;t work
<wgrant> wallyworld: Hm, I've never seen that
<wgrant> StevenK: But it does reduce the query count somewhat?
<StevenK> wgrant: It does not. It goes from 13 to 15.
<wgrant> Oh
<wgrant> I wonder if you're failing to invalidate the cache properly
<StevenK> 97+ Store.of(bug).flush()
<StevenK> ^ Not sufficent?
<wgrant> flush is not invalidate
<wgrant> flush flushes
<wgrant> invalidate invalidates :)
<wallyworld> wgrant: submitting a review request works fine on qas too
<wgrant> wallyworld: Ah, qastaging only has 3.5.1 enabled for rick
<wallyworld> :-( i assumed it was on for everyone in lp
<wgrant> Likewise, but I just checked and it's not :(
<wallyworld> bollocks
<wgrant> Heh, we just passed MP 123456 a few hours ago
<wgrant> wallyworld: Try rerequesting a review from lifeless on https://code.launchpad.net/~wgrant/launchpad/i-am-stupid/+merge/123478
<wallyworld> yep, so that failed
<StevenK> wgrant: So my numbers were off. Slightly
<StevenK> wgrant: Putting in list(getUtility(IPersonSet).getPrecachedPersonsFromIDs(
<StevenK> ids, need_validity=True))
<StevenK> Still has a bunch of queries against ValidPersonCache
<wgrant> That's very odd
<wgrant> The same number of queries?
<StevenK> So with no preloading, 50 bugactivity rows is 109 queries. With that getPrecachedPersonsFromIDs, it drops to 33.
<wgrant> Oh, 50?
<wgrant> You can't test with 50
<wgrant> The dev storm cache size is only like 100
<StevenK> Why not?
<wgrant> Also, that's going to be slow even if the cache is huge :)
<wgrant> No point going above, say, 5
<StevenK> My original test had 10
<StevenK> 10 bugactivity rows drops from 29 queries to 7.
<wgrant> That sounds roughly correct, right?
<StevenK> Yeah
<StevenK> wgrant: Well, 5 rows is 7 queries, as is 10. Which is good.
<wgrant> Yes :)
<StevenK> wgrant: http://pastebin.ubuntu.com/1196125/
<wgrant> StevenK: I'd inline the list comp, but otherwise good
<StevenK> Ah yes, that would work.
<StevenK> Haha, wallyworld approved it 32 minutes ago, I just didn't notice.
<wgrant> Heh, so he did
<wgrant> I was right, though :)
<StevenK> Not disputing that one bit.
<adeuring> good morning
<StevenK> wgrant: Heh, if not event.edited_fields is not quite the solution -- the 313 failures from ec2 attests to that.
<wgrant> stub: Morning
<wgrant> stub: Could you review https://code.launchpad.net/~wgrant/launchpad/i-am-stupid/+merge/123478?
<lifeless> wgrant: love the name
<wgrant> I try.
<mgz> such a let down, just missing indicies
<nigelb> lol
<nigelb> Shouldn't it go into db-devel or something?
<wgrant> Nah, nowadays indices go to devel and get applied live, without downtime
<nigelb> Ooooh. Exciting :D
<stub> k
<stub> wgrant: Not stupid, I was hoping the primary key indexes would make these unnecessary
<wgrant> The only really important one is apg(grantee, policy), and it was pretty obvious that that was necessary from the start, but I apparently forgot to create it
<stub> well... grantor is needed or person merge explodes
<stub> yeah, but the (policy, grantee) would often be usable instead of the inverse
<stub> These can go live now?
<wgrant> Please
<wgrant> I will be watching the DBR clear up immediately :)
<wgrant> Well
<wgrant> I would be if my Yubikey wasn't flashing angrily at me
<czajkowski> hah
<wgrant> This is inconvenient
<czajkowski> usually I have to go find my phone and hope it's charged
<stub> wgrant: Might be worth making the compound indexes UNIQUE?
<stub> wgrant: Don't think it makes any difference now, but a future planner might notice.
<wgrant> stub: Right, it doesn't make any difference now.
<wgrant> But I could...
<wgrant> Possibly a good idea
<stub> yeah
 * wgrant does so
<wgrant> stub: That's done
<wgrant> Diff not updated yet, though
<wgrant> Hm
<wgrant> Do we still call unique indices __key?
<wgrant> Rather than __idx?
<stub> yeah, __key
<wgrant> stub: Done
<wgrant> I think that's it...
<jelmer> mgz: your summon worked!
<mgz> magic :)
<stub> wgrant: All done on production
<stub> and qastaging
<wgrant> stub: Thanks
<wgrant> branch and APG will hopefully drop off the top of the DBR's Most Read Tables list...
<wgrant> Well, APG is dropping to nothing, but branch is still terrible :(
<wgrant> stub: So, I see 9.2 has been released... just as we are almost completely migrated to 9.1 :)
<rick_h_> chaining repliate ftw!
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: â
<jam> benji: poke about lpsetup. It appears that my patch cannot land because the tarmac process is broken?
<jam> (someone said that if a patch breaks, it doesn't teardown the lxc instance, so the next submission will fail because the lxc is still there)
<jam> I'm not entirely sure who to talk to, but I figured starting with someone on yellow might be reasonable.
<benji> jam: hmm, I hadn't heard of that; I'll see what I can do about it
<jam> benji: https://code.launchpad.net/~jameinel/lpsetup/missing_lxc_1045728/+merge/122663 is the merge request
<benji> bac: are you aware of such a problem? ^^^
<jam> the failure from tarmac was 'juju', 'bootstrap', '-e', 'lpsetup-testing-lxc' was failing.
<bac> benji, jam: the first thing to do is to ssh into the machine and have a look around.
<bac> you should be able to 'ssh tarmac@lptarmac.no-ip.org'.  you'll need your .ssh/config configured like https://pastebin.canonical.com/74130/
<benji> ok, I'll take a look
<jam> bac: as user 'ubuntu' or as user 'tarmac' ?
<bac> jam: tarmac is easiest.  either works and both have sudo.
<bac> jam: did you get access to the machine?
<jam> bac: otp now, will try later.
<benji> bac: I did.  No lxc containers are running, no juju environment was bootstrapped.  I bootstrapped the test env and it seemed to work fine. (and then I destroyed it)
<benji> is there some way I can re-run the merge request?
<bac> benji: yes.  the log file is in ~/.config/tarmac/tarmac.log.  grep for Merging and it'll show you the branch name
<bac> cd ~/repos/lpsetup/trunk
<bac> hey benji, wanna use tb?
<benji> bac: termbeamer is up and sending you my session
<benji> bac: juju appears to be hung waiting for the unit to come up
<benji> juju status times out
<czajkowski> bac: ello when do you guys have your stand up?
<bac> czajkowski: 1600Z
<bac> why?
<bac> benji: that's ungood.
<bac> benji: we could bounce the instance but that doesn't solve the problem.
<benji> bac: yeah; I was hoping youhad seen this before; let me investigate some more and see if I can get closer to a root cause
<czajkowski> bac: might be good to join a few in case we have over lap on stuff
<czajkowski> given you're now on maintenance no ?
<stub> wgrant: Yup, but still waiting for a point release or two.
<bac> benji: ok.  if you ever want to bounce it, just use 'poweroff' in the instance and then the startup script in ./bin on your local machine
<benji> bac: where is the startup script?
<bac> in the config branch in bin
<bac> benji: lp:lp-tarmac-configs
<benji> cool, thanks
<rick_h_> deryck: ping, when you get a sec then I want to sanity check the +newproduct ProjectGroup vs +new and Product
<rick_h_> deryck: http://paste.mitechie.com/show/782/ for the quick rundown on the bits fitting together
<deryck> rick_h_, looking
<deryck> rick_h_, so looking at that, it seems to me you only care about getting allowed types in the second step if the context is IProduct.
<deryck> rick_h_, or else you could have the same method on IProject that only returns Public.
<rick_h_> deryck: right, ok that sounds good
<benji> bac: how long should the inttance take to start?  it has been around 30 minutes of "[localhost] local: scp -r -o StrictHostKeyChecking=no * 10.55.60.129:"
 * benji gets some lunch.
<deryck> lifeless, ping. wondering if my 10.4 second db patch, according to dbupragde.log is okay for deployment.
<rick_h_> benji: mp for you if you get a sec: https://code.launchpad.net/~rharding/launchpad/pp_register/+merge/122666
<rick_h_> deryck: I left the fake attribute in there with an XXX for the db side since I figured it was more ugly to try to dynamically add in the @property conditionally
<deryck> rick_h_, it's all behind a feature flag, too, right?
<rick_h_> deryck: right
<deryck> ok
<benji> rick_h_: sure; it will be a while though
<rick_h_>  benji no hurry, thanks
<rick_h_> deryck: and since I'm using an @property and you'll have a db attribute there shouldn't be much to resolve no matter who gets in first for conflict stuff.
<deryck> rick_h_, right.  sounds fine to me, but I'll defer to benji for the real review.
<rick_h_> deryck: right, just fyi your way
<deryck> gotchas
<benji> bac: I can't get the tarmac container to start; do you mind trying for me?
<bac> benji: can you ssh to it?
<benji> bac: nope
<bac> benji: you did try by ip addr?
<benji> bac: nope; let me find it (I don't have it on hand)
<benji> found it
<benji> bac: ssh tarmac@10.55.60.123 fails with "Permission denied (publickey)."
<bac> benji: yep, don't use tarmac@.
<bac> he's probably not created yet.  if you don't specify you log in as ubuntu
<benji> bac: "ssh 10.55.60.123" still gives me "Permission denied (publickey)."
<bac> benji: uhg
<lifeless> deryck: definitely not
<lifeless> deryck: we're looking for 4-5 seconds max IIRC the math correctly.
<lifeless> deryck: which patch was it
<lifeless> ?
<abentley> lifeless: I believe deryck meant https://code.launchpad.net/~deryck/launchpad/product-db-changes-for-privacy/+merge/122941
<deryck> lifeless, yeah, it's 2209-33-1, the specification_sharing_policy column.
<deryck> lifeless, I have no idea how to get a single column add any faster. :)
<deryck> lifeless, sorry for the delay, but I've got pool people here to give and estimate on my pool servicing.  I need to step away again, but won't be too long.
<lifeless> deryck: you're fine, a close inspection shows we have a time reporting bug.
<lifeless> 2012-09-10 00:40:09 INFO    Applying /srv/staging.launchpad.net/staging/launchpad/database/schema/patch-2209-23-1.sql
<lifeless> 2012-09-10 00:40:20 INFO    Applying /srv/staging.launchpad.net/staging/launchpad/database/schema/patch-2209-31-1.sql
<lifeless> 23-1 took 11 seconds
<lifeless> I need to see what that is.
<lifeless> it might be wgrants index, in which case it went on concurrently on prod and we can ignore it.
<lifeless> deryck: which is is, so your patch is fine, it isn't 11 seconds long.
<lifeless> deryck: could you please file a bug about this? It will confuse others..
<lifeless> probably fallout from slony
<bac> benji: i was able to restart the c'stack instance with no problem.  did you update your env per the instructions on the internal wiki?
<benji> bac: by sourcing ~/.canonistack/novarc?  if so, yes
<bac> benji: in .canonistack do you have a benji_lcy01.key or just benji.key?
<benji> bac: benji_lcy01.key
<bac> benji: ok, then, i'm not sure what the problem is
<bac> maybe later we can use a termbeamer to see what is going on
<benji> bac: that'd be great; ping me when you have a moment
<abentley> deryck: Can we chat?
<deryck> lifeless, sure, I can file a bug.
<deryck> abentley, sure.
<abentley> deryck: Okay, g+ hangout.
<deryck> ok, coming now....
<lifeless> gary_poster: I've fixed (just pushed to trunk now) a bug in testrepository causing test times to be badly undercounted when the stream isn't timed. I think this won't affect paralleltest due to the subunit stream from zope being timed, but if it isn't timed, then I'm wrong, and you would want the fix.
<gary_poster> lifeless, cool!  it is timed for us because we pass -vvv (or something silly like that) to bin/test, but might be helpful for us even in other situations.
<lifeless> https://bugs.launchpad.net/testrepository/+bug/1048126 for reference
<_mup_> Bug #1048126: Test times of untimestamped streams are bogus <Testrepository:Triaged by lifeless> < https://launchpad.net/bugs/1048126 >
<lifeless> flacoste: hi ?
<flacoste> hi lifeless
<lifeless> we on ?
<flacoste> we are
<lifeless> hah
<lifeless> thats something skype handles better
<lifeless> when both sides call
<lifeless> I'll start a fresh fresh one
<flacoste> lifeless: hang on, i'm with google on the phone (search broken)
<lifeless> ok
<lifeless> grab me whenever
<flacoste> lifeless: can you send me a new invite?
<lifeless> sure
<lifeless> flacoste: you should have it...
<bac> benji: still around?
<benji> bac: just :)
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<bac> benji: https://plus.google.com/hangouts/_/ef48f894900cf80079fb2996e64e6b0ae17df36d?authuser=1&hl=en-US
<benji> bac: I'm afraid I'm away from my hangout setup; mind if we do it in the morning?
<bac> benji: nope.  have a good evening.
<benji> thanks; you too!
#launchpad-dev 2012-09-11
<StevenK> wgrant: I've finally found the method that generates the query for bug 1001838
<_mup_> Bug #1001838: Builder:+history timeouts due to terrible main query <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1001838 >
<StevenK> Would be nice if I could actually find a relevant OOPS that hasn't been pruned.
<bigjools> awesome,  after upgrading to quantal I now have two X servers running
<wgrant> bigjools: What's the other one doing?
<wgrant> StevenK: Should be pretty easily reproducible on prod
<wgrant> StevenK: eg. try palmer's +history
<wgrant> Any of the busier builders should work
<bigjools> wgrant: one is gdm one is kdm ...
<wgrant> gdm or lightdm?
<bigjools> also:
<wgrant> The former would be pretty strange :)
<bigjools> $ dpkg -l
<bigjools> dpkg-query: error: parsing file '/var/lib/dpkg/status' near line 56897 package 'libao4:i386':
<bigjools>  mixed non-coinstallable and coinstallable package instances present
<wgrant> Uhoh
<bigjools> yeah
<wgrant> Ah
<wgrant> Found the bugsummary bug
<StevenK> wgrant: Heh, except is palmer is gone
<StevenK> roseapple works
<wgrant> palmer's still accessible
<wgrant> Just probably not linked
<StevenK> hooker works too
<StevenK> Right, palmer is cursed
<StevenK> wgrant: Ah, so there are two real cases in the code -- your explain and index is for the case where user is None.
<wgrant> Indeed.
<StevenK> wgrant: And what's the bugsummary bug?
<wgrant> A typo in bug_summary_dec
<wgrant> -        AND access_policy IS NOT DISTINCT FROM access_policy;
<wgrant> +        AND access_policy IS NOT DISTINCT FROM $1.access_policy;
<wgrant> is the fix
<StevenK> Heh
<wgrant> See, this is why we should move to 9.2
<wgrant> SQL functions can now use named args
<wgrant> Which would have made that more obvious :)
<wgrant> lifeless: So I expect us to be on 9.2 by this evening
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-1046713/+merge/123668
<wgrant> Or leave it for stub if you so desire
<lifeless> wgrant: what you expect and what you get may differ
<lifeless> bigjools: around ?
<bigjools> lifeless: otp
<lifeless> bigjools: need info about where in BNE to stay
<lifeless> bigjools: so that I don't end up with an hour commute or something
<StevenK> wallyworld can probably help too
<bigjools> yeah he's better for that
<wallyworld> lifeless: where are you meeting?
<bigjools> we dunno yet
<wallyworld> there's a few good irish pubs inthe city
<lifeless> I need to book accom via btstravel
<lifeless> so this is going to be tight if I don't put the request in today.
<wallyworld> if you can tell me where the meetings are, i can help with somewhere close by
<bigjools> I have a spare room if all else fails :)
<wallyworld> (bring earplugs then)
<bigjools> haha
<wallyworld> and i don't mean for the babies
<bigjools> comedian
<wallyworld> i'm here till friday
<lifeless> boom tish
<lifeless> I thought it was Ng that was.
<StevenK> Hah
<wgrant> lifeless: :(
<lifeless> implicit type promotion sucks.
<lifeless> that is all
<lifeless> can has review plox : https://code.launchpad.net/~lifeless/python-oops-tools/bug-1048470/+merge/123672
<lifeless> will unbreak neem
<lifeless> wgrant: StevenK: ^
<wgrant> lifeless: python3 :)
<lifeless> wgrant: that would be nice
<wgrant> Anyway, r=me
<wgrant> Thanks
<lifeless> wgrant: but I'm missing 3 unicorns and a couple of fairy godmothers to make that happen
<wgrant> Well
<wgrant> Django's mostly ported now
<wallyworld> wgrant: StevenK: should a distribution source package have an +expirable-bugs view?
<lifeless> bigjools: travel booked, but now we need to sort out accom :)
<lifeless> bigjools: <nag>
<wgrant> wallyworld: Doesn't really matter.
<wgrant> wallyworld: If it's easy then fix it, otherwise remove it
<wallyworld> wgrant: well, _getTargetJoinAndClause only wants distro, product, distroseries, productseries. if i can convert a dsp to any of those, then it will work?
<wallyworld> BugTaskSet._getTargetJoinAndClause
<wgrant> Ah
<wgrant> So it's only used by findExpirableBugTasks
<wgrant> I'd just add support for IDistributionSourcePackage and ISourcePackage in there
<wgrant> shouldn't be too difficult
<wallyworld> ok, thanks. just wanted to check that it made sense to do that
<wgrant> It doesn't not make sense
<wgrant> But it might not be too useful
<wgrant> So if it looks difficult, don't bother
<wallyworld> ok. i have no idea how often that view might be used, or if you need to url hack to see it
<wgrant> Not very, and probably
<wallyworld> hmm. in that case i might leave it. no point fixing something not useful
<lifeless> bigjools: did we need to speak arch ?
<wgrant> StevenK: Have you tested COUNT(*) with that new query?
<wgrant> It will probably be dreadfully slow
<StevenK> ... You say after I close the gedit that had my notes in it
<StevenK> Oh, and after I cleaned up the index on DF
<StevenK> wgrant: Yeah, 8.5 seconds
<wgrant> StevenK: Right, this is why StormRangeFactory is important
<wgrant> I'm not sure how difficult the port will be
<wgrant> Probably not very
<lifeless> this is also something we could denorm
<lifeless> avoid the count(*) at all
<StevenK> wgrant: Based on my reading, it looks like BuilderHistoryView uses it already?
<wgrant> StevenK: Where?
<StevenK>         self.batchnav = BatchNavigator(
<StevenK>             builds, self.request, range_factory=self.range_factory(builds))
<wgrant> Right
<wgrant> It uses a range factory
<wgrant> Probably not StormRangeFactory
<wgrant>     range_factory = ListRangeFactory
<wgrant> It's likely that a subclass or two override it
<StevenK> Can we just tell BuilderHistoryView to override to SRF and move on?
<wgrant> Right, probably
<wgrant> Try :)
<StevenK> wgrant: TestBuilderHistoryView doesn't explode
<wgrant> See if it works in the UI :)
<StevenK> Cowboy my changes onto DF, or 'make run' see if it works?
<wgrant> Try it locally
<wgrant> Confirm that no COUNT(*) is executed
<StevenK> steven@undermined:~/launchpad/lp-branches/buildfarmjob-index% grep BuildFarmJob /var/log/postgresql/postgresql-9.1-main.log | grep -c COUNT
<StevenK> 0
<StevenK> And yes, I have query logging on
<StevenK> :-)
<StevenK> Oh drat, I didn't use cat.
 * StevenK peers at spm.
<StevenK> wgrant: ^ Does that look good to you, or shall I pastebin the first grep?
<wgrant> StevenK: I'd confirm in ++oops++
<wgrant> Or the query log at the bottom of the page
 * StevenK tries to rememeber where OOPSes go
<wgrant> Nowhere, if rabbitmq is running
<wgrant> So I'd use the thing at the bottom of the page
 * StevenK turns on the FF for it
<StevenK> wgrant: No match for 'count('
<StevenK> Lots of matches for count, but that's things like 'Account' and 'failure_count'
<wgrant> StevenK: And it shows up if you revert the range_factory change?
<StevenK> 1.099ms 	
<StevenK> SQL-main-slave: SELECT COUNT(*) FROM BuildFarmJob LEFT JOIN PackageBuild ON PackageBuild.build_farm_job = BuildFarmJob.id WHERE BuildFarmJob.builder = 1
<wgrant> Excellent
<wgrant> Now I'd cowboy this onto DF and watch the fireworks
<wgrant> But it's hopefully fine
 * StevenK recreates the index again.
<StevenK> wgrant: DF cowboyed
<StevenK> wgrant: "31 queries/external actions issues in 0.93 seconds"
<StevenK> Means we can probably kill the Builder:+history FF too
<wgrant> That's the plan
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/builder-history-better-query/+merge/123676
<wgrant> StevenK: Does anything else use the modified method?
<wgrant> eg. the API?
<StevenK> wgrant: There's one other callsite, and it isn't exported
<StevenK> And the method itself isn't either
<wgrant> What is the callsite?
<StevenK> IBuilder.getBuildRecords()
<wgrant> Isn't that the method that we're looking for callsites for?
<StevenK> No, the method I've been beating into submission is IBuildFormJobSet.getBuildsForBuilder
<StevenK> IBuildFarmJobSet, even
<wgrant> Right, but isn't that only used by getBuildRecords?
<wgrant> The view should call getBuildRecords
<wgrant> Nothing else should call getBuildsForBuilder directly, AFAIK
<StevenK> Right, doesn't look like anything else wants IBuilder.getBuildRecords()
<StevenK> Pretty much everything is interested in IDistroSeries.getBuildRecords()
<StevenK> wgrant: So, are you happy with everything, and I can get stuff rolling?
<wgrant> StevenK: Indeed, r=me with one comment
<StevenK> wgrant: I was going to rename extra_clauses to clauses to save a bit too
<StevenK> wgrant: http://pastebin.ubuntu.com/1198057/
<StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/buildfarmjob-index/+merge/123675 would love a review.
<stub> StevenK: Do we really need id in there?
<wgrant> Ooh
<StevenK> wgrant wrote that index, I've just been using it.
<wgrant> I think I just won my battle against the planner
<wgrant> That was months ago
<stub> It looks like a sacrifice to the test suite
<wgrant> It probably has id as a tiebreaker
<wgrant> I don't remember
<stub> Can a builder run multiple jobs in one build?
<stub> So we need the tie breaker in the real world?
<wgrant> probably not, but it won't really hurt
<StevenK> TBH, I'm inclined to leave it, I've been testing against this index all afternoon
<stub> StevenK: Do the queries have id in the order by at the moment?
<StevenK> And it makes Builder:+history performant for the first time in $A_LONG_TIME
<stub> it is an extra integer column in an index on 4million odd items, makes the index slower
<StevenK> They do and they will.
<stub> ok.
<StevenK> wgrant: Happy with the pastebin'd diff I posted?
<stub> Both builder and date_finished are nullable.
<stub> Should this index be partial?
<StevenK> stub: builder will be NULL when a job is waiting in the queue, and date_finished will be NULL until the job completes.
<StevenK> The grand majority of rows will have them both set.
<stub> Right. Just looking at prod, 300k rows have a NULL builder
<stub> So thinking a WHERE builder IS NOT NULL clause might be appropriate
<wgrant> Probably mostly the imported binaries
<wgrant> I would indeed go with WHERE builder IS NOT NULL
<wgrant> But I think date_finished IS NOT NULL would break things
<wgrant> Since we don't have that constraint in the query today
<stub> Yup
<wgrant> And it's conceivable that a builder has something without a date_finished, though that should only happen if something is terribly broken
<StevenK> If it's currently building
<stub> The reverse is the weird one
<stub> date_finished but no builder, but that would be if we removed builders from the db and wanted to keep historical records.
<wgrant> Possibly
<wgrant> Does it exist on prod today?
<StevenK> -CREATE INDEX buildfarmjob__builder__date_finished__id__idx ON BuildFarmJob(builder, date_finished DESC, id);
<StevenK> +CREATE INDEX buildfarmjob__builder__date_finished__id__idx ON BuildFarmJob(builder, date_finished DESC, id) WHERE builder IS NOT NULL;
<StevenK> stub: ^
<stub> StevenK: yer
<stub> wgrant: 5081
<StevenK> stub: The diff on the MP has updated if you want to rubber stamp
<wgrant> stub: I've also got a DB review for you. A nice complex DB function change: https://code.launchpad.net/~wgrant/launchpad/bug-1046713/+merge/123668
<StevenK> stub: No +1 for me? :-(
<wgrant> Heh
<stub> eh? oh
<stub> StevenK: +1
<StevenK> stub: Thanks, that's landed as r15931. You can apply it to qas/prod when you're able.
<stub> wgrant: Dare I ask where the typo was?
<wgrant> stub: Check the diff I linked in the description
<stub> oh, ta
<wgrant> Thanks
<wgrant> It's been through ec2 already
<stub> wgrant: Happy for me to apply that with StevenK's index?
<wgrant> stub: That would be lovely
<wgrant> Then I'll rebuild bugsummary once revisionkarma has been killed off and the backup is done.
<stub> wgrant, StevenK: Both applied to qastaging
<StevenK> stub: Prod is blocked due to backups?
<stub> I'm just checking
<stub> Yeah, another 4 hours or so until it can be applied
<StevenK> That's a little worrying, given the FDT window is in 3 hours.
<StevenK> Well, one of them. :-)
<stub> wgrant: Your patch is applied to prod
<StevenK> Ah, functions aren't blocked by the backups, but indicies are?
<wgrant> stub: Thanks
<stub> StevenK: Correct
<StevenK> Well, that's just terrible.
<StevenK> I shall have to deal.
<wgrant> But it makes sense :)
<wgrant> (the backup and revisionkarma should both finish in 2hÂ±10min
<stub> We are going to keep having 10.5 hour logical dumps until we get the diskspace to store binary backups, so consider this next window pointless.
<wgrant> Nah, they finish in time
<wgrant> It's the midday AEST window that is unusable unless the backup fails
 * StevenK goes to kill Geth
<StevenK> Well, some Geth
<stub> I've got a 6.5 hour backup transaction in process, and I could have sworn the last successful backup took 10.5 hours
<wgrant> stub: The backups run from 00:41 to somewhere between 09:15 and 09:20, usually
<wgrant> (then takes 20-30 minutes to copy to sourcherry, but by then it's out of the DB)
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: â
<lifeless> hmm
<lifeless> we need to do some releases
<czajkowski> moring
<czajkowski> *morning even
<lifeless> https://bugs.launchpad.net/launchpad-project/?field.status%3Alist=FIXCOMMITTED
<lifeless> hi czajkowski
<czajkowski> hiya
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban, rick_h | Firefighting: - | Critical bugs: â
<rick_h_> sinzui: beat you on this one today :P
<sinzui> okay
<rick_h_> last week you beat me through one for jcsackett so need to get my revenge in
<nigelb> rick_h_: Caffeine overload there? :P
<rick_h_> nigelb: working on it
<nigelb> Heh
<daker> hey jtv
<tumbleweed> I tried my hand at scratching an LP itch last night. bug 891862. any one care to have a quick look at it and discuss it with me?
<_mup_> Bug #891862: Changing the owner of a packageset requires SQL <api> <canonical-losa-lp> <packagesets> <soyuz-core> <Launchpad itself:Triaged by stefanor> < https://launchpad.net/bugs/891862 >
<cjwatson> tumbleweed: You don't seem to have any tests
<tumbleweed> I am aware of that
<tumbleweed> there aren't any tests for any of that code, as it stands
<tumbleweed> and writing some is going to put me into a big positive LoC problem :/
<cjwatson> lib/lp/soyuz/tests/test_packageset.py
<cjwatson> Yeah, I've tried that excuse in the past, it never works :)
<tumbleweed> oh, does that cover the API?
<cjwatson> Oh and lib/lp/soyuz/stories/webservice/xx-packageset.txt
<cjwatson> For the API
<tumbleweed> aah, doctests
<cjwatson> doctest unfortunately, but it's workable if you hold your nose
<tumbleweed> that's why I couldn't find them
<cjwatson> Also destroySelf needs a model test
<tumbleweed> yeah
<cjwatson> (not totally sure I like that name but I'll leave it to a reviewer ...)
<cjwatson> and I would say you should be exporting the new method on API version devel, not beta
<tumbleweed> destroySelf seems to be the name everywhere else
<tumbleweed> duh, re devel
<tumbleweed> do you know of anything that can some me some LoC points?
<rick_h_> tumbleweed: https://dev.launchpad.net/PolicyAndProcess/MaintenanceCosts/HitList is the running list, I've not checked it out lately
 * tumbleweed supposes converting those doctests to unittests may be a starting point
<cjwatson> that's usually a worthwhile place to start although it can take a while
<deryck> adeuring, ping for standup.
<adeuring> deryck: oopps, coming...
<deryck> adeuring, np.
<rick_h_> deryck: abentley think I found it. It had a new information type that wasn't there before so that failed the verbose prettyprint match
<rick_h_> thanks for peeking
<abentley> rick_h_: np.
<deryck> rick_h_, cool, glad to hear.
<abentley> rick_h_: See, that's the thing.  Doctests should show you where the match failed, not what the differences are.
<rick_h_> abentley: right, blind by too much info
<rick_h_> just glad it wasn't anything huge. Hate it when you change something and then things seemingly unrelated go boom
<sinzui> jam I appear to have enough network now to have a proper talk about maintenance
<sinzui> jam: I am free today, Thursday, and Friday
<czajkowski> sinzui: love the new chart!!
<sinzui> thank you
<abentley> deryck: I've added a card for "Add Add Product.getAllowedSpecificationInformationTypes and Product.getDefaultBugInformationType"
<deryck> abentley, thanks!
<adeuring> rick_h_: could you pelase review this MP: https://code.launchpad.net/~adeuring/launchpad/use-access-grants-for-specifications-auth-check/+merge/123774 ?
<rick_h_> sure thing adeuring
<adeuring> thanks!
<rick_h_> adeuring: should an XXX or something be setup for the follow up branch work?
<rick_h_> adeuring: I guess how are we tracking that there's follow up for the owner not getting a grant atm
<tumbleweed> cjwatson: added a few tests and started finding bugs... Do you think we should try and re-link packageset heirarchies across removal? or just abort the deletion if there are relationships?
<adeuring> rick_h_: I think I should add a card about the fact that several people with "special roles" do not have automatic access
<rick_h_> adeuring: ok, cool
<rick_h_> adeuring: r=me
<adeuring> rick_h_ thanks!
<cjwatson> tumbleweed: err not sure :-)
<tumbleweed> or even cascade the deletion...
<tumbleweed> and a more general question: if I expose an API method for deletion. How do I not return a 404 on success?
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: â
<rick_h_> jcsackett: have time for a JS review? https://code.launchpad.net/~rharding/launchpad/productjs/+merge/123803
<jcsackett> rick_h_: sure.
<rick_h_> jcsackett: thanks, more a start than a finish, but want to do it in stages
<jcsackett> dig.
<benji> jam: I'm afraid I have to punt the lpsetup tarmac issue back to you guys.  I wasn't able to figure much out yesterday.  When rebooting the canonistac instance it did not come back up.
<jam> benji: technically it is up to purple, since they are now on maintenance, but since I'm the one with the delayed patch, I'll give it a look tomorrow.
<jam> I was able to log into the machine
<jam> so that worked
<benji> jam: oh, sorry, I thought you were on maint.
<jam> we just switched on Monday
<jam> so not too surprising you would think so
<jcsackett> sinzui: free to chat?
<jcsackett> sinzui: ignore that, posted from irssi history.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<jcsackett> rick_h_: some comments on MP.
<rick_h_> jcsackett: ty much looking
<jcsackett> rick_h_: also, given the branch of mine just landed that you reviewed, lazr.* > lp.ui.*
<rick_h_> jcsackett: yea, expected that to come along, just waiting on the merge
<rick_h_> jcsackett: comments added
 * jcsackett looks
<jcsackett> rick_h_: thanks for answers, that all makes sense. r=me.
<nigelb> I've always wondered, how'd LP have a lazr namespace?
<nigelb> i.e. where'd it come from?
<rick_h_> nigelb: so I believe the history was that lazr was supposed to be a set of reusable UI components
<rick_h_> but it's really only used in LP
<nigelb> ah
<nigelb> the idea was that it could be used in other places in canonical?
<rick_h_> right
<abentley> nigelb: Actually, I think the hope was that it would be useful outside Canonical, too.
<rick_h_> which is again my pipe dream, but we're just subsuming/taking over lazr into LP
<nigelb> ah
<lifeless> rick_h_: I think its a good dream. I think we tried to run before we could walk. Lots of effort in planning for things that haven't eventuated, too little on solving the friction around consuming widgets from the store etc.
<rick_h_> lifeless: yea, definitely. Not saying it's more than a dream to keep in mind as we do things
<rick_h_> in my head at least :)
<rick_h_> and it'll be easier with later YUI with Views
<tumbleweed> how do I make this API delete() function not return a 404 on success? http://bazaar.launchpad.net/~stefanor/launchpad/edit-packagesets/view/head:/lib/lp/soyuz/interfaces/packageset.py#L351
<cody-somerville> Is there a way to get what the link url would be for a resource without actually fetching it
<cody-somerville> I want to get the resource URL for a person
<cody-somerville> I want to pass this to another API call manually instead of the person object
<cody-somerville> as I might not be able to fetch the person object myself
<cody-somerville> I could easily do a hack but am wondering if there is a clean way to do this
<cody-somerville> sinzui, ^^
<cody-somerville> Also, it doens't appear that the objects returned by getSharedArtifacts get transformed into the right launchpadlib objects
<sinzui> I manually construct them sometimes because I know the API is stable and cannot change
<sinzui> If you know the type, you can trust the URL will be the same given that you know the unique names that construct it
<sinzui> oh dear
<sinzui> cody-somerville, you are not getting IBranch and IBug?
<cody-somerville> Doens't appear so. In fact, for one example I have a person that has one artifact shared with them. What the API returns is a list that contains two lists. The first sub-list contain a dictionary for the single bug shared. The second sub-list is empty.
<cody-somerville> sinzui, The dictionary does contain a resource_type_link key. The value is https://api.launchpad.net/devel/#bug_task as expected.
<sinzui> ah
<sinzui> This probably relates to what stopped me from exporting getAffiliatedPillar()
<cody-somerville> python-lazr.restfulclient: 0.11.1-1build1; python-launchpadlib: 1.9.7-0ubuntu2.1
<cody-somerville> sinzui, ^^ if that matters
<sinzui> It is not easy to return a heterogeneous set of objects
<sinzui> versions do not matter. The issue is Lp trying to adapt two types into one return type
<sinzui> report a bug.
<sinzui> I will discuss the issue the the squad. I think we have two cases that demonstrate that we need a solution that can be reused.
<cody-somerville> Will do.
<cody-somerville> sinzui, so the two lists within the lists. Is the second list intended to have branches?
<sinzui> yes
<sinzui> ah, I wonder if the rule is real method returns a tuple and Lp restful knows how to expand the tuple to the real types. That would allow me to send back distros, project groups, and projects from a singe method, and informs me of how to rewrite the query
<cody-somerville> sinzui, FYI, I imagine one of the primary use cases will be to display same information that the web UI does on command line. Since the current implementation returns a bug_task instead of a bug, you have to fetch each individual bug object to get the information_type.
<sinzui> cody-somerville, agreed
<sinzui> The bug-bugtask division does not help anyone
<sinzui> Even Lp has to use both every time
<sinzui> Pillar and PillarName are examples of intermediary object that no callsite ever want tot get back
<cody-somerville> sinzui, I filed LP #1049374
<_mup_> Bug #1049374: Resources returned by getSharedArtifacts not adapted to resource object type <Launchpad itself:New> < https://launchpad.net/bugs/1049374 >
<sinzui> thanks
<lifeless> whats involved in doing a launchpad-buildd release ?
<StevenK> wgrant: OOPS-995da926c1f98feed6844957a3a94559
<StevenK> sinzui, wgrant, wallyworld: http://www.somethingofthatilk.com/index.php?id=135
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/illegaltarget-if-pillar-is-new/+merge/123862
<wgrant> StevenK: Can you rework the test to not use rSP?
<wgrant> StevenK: eg. set the bug to USERDATA, then transition the sharing policy afterwards
<StevenK> wgrant: I think it will still be needed for the validate_target call
<wgrant> Not if you log in
<wgrant> Wait a minute, that test is invalid
<wgrant> The sharing policy is PROPRIETARY_OR_PUBLIC
<wgrant> So it permits USERDATA
<StevenK> Bleh
<StevenK> This diff should fix that too
#launchpad-dev 2012-09-12
<StevenK> wgrant: http://pastebin.ubuntu.com/1199747/
<wgrant> StevenK: Have you confirmed that it now fails with the diff reverted?
<StevenK> wgrant: Just did.
<StevenK> IllegalTarget: Product-name-100217 doesn't allow Private bugs.
<wgrant> Sounds good, then
<wgrant> StevenK: Hm, why are you using multiple products?
<StevenK> wgrant: Sorry, was noming. Yeah, let me remove that bit.
<StevenK> wgrant: http://pastebin.ubuntu.com/1199798/
<wgrant> StevenK: You could also remove the makeCommercialSubscription bit if you use makeProduct(bug_sharing_policy=BugSharingPolicy.PUBLIC_OR_PROPRIETARY)
<StevenK> Does that also allow USERDATA?
<wgrant> Yes
<StevenK> Which invalidates the test, does it not?
<wgrant> You'd still have to transition it to BugSharingPolicy.PROPRIETARY after creating the bug
<wgrant> It just saves you from having to manually create the commercialsubscription
<StevenK> I think it's moot, it would be the same amount of lines
<StevenK> But I can if you want
<wgrant> Perhaps
<wgrant> Anyway, it looks fine now
<StevenK> wgrant: The diff is updated if you want to +1
<wgrant> You can actually kill of two more lines by not setting the product or bug owners, but I think you might kill me too
<StevenK> wgrant: And use login_celebrity, I guess
<wgrant> StevenK: person_logged_in(product.owner)
<wgrant> Due to the APG it'll even have access to the bug
<StevenK> wgrant: owner killed
<wgrant> That sounds painful
<StevenK> Haha
<StevenK> wgrant: Thanks, tossing at ec2.
<StevenK> Bloody hell, buildbot, finish already
<StevenK> wallyworld, wgrant: Some cards can probably move out of QA-Landing
<wallyworld> probably
<wgrant> buildbot only finished 30 seconds ago
<StevenK> wgrant: Heh, it was saying 56 minutes when I looked 10 minutes ago.
<wgrant> 6 seconds after you mentioned QA-Landing, in fact...
<lifeless> bigjools:  1920093641
<lifeless> bigjools: 1.9B
<bigjools> lifeless: 433453453458798
<lifeless> bigjools: rows.
<bigjools> sweet
<lifeless> can we scale? Yes we can!
<lifeless> of course, it is 68GB on disk.
<wgrant> + indices
<wgrant> If we're talking about branchrevision, there's like 100GB of indices
<wgrant> Ah, no, only about 70GB now. We pruned a few
<bigjools> lifeless: is it web scale?
<lifeless> bigjools: http://www.mongodb-is-web-scale.com/
<bigjools> haha
<bigjools> lifeless: "If there's a problem writing your data, you're fucked. Does that sound like a good design to you?"
<lifeless> bigjools: "If that's what they need to do to get those kickass benchmarks, then it's a great design."
<bigjools> lifeless: "Does /dev/null support sharding?"
<bigjools> I can't continue, laughing too hard
<lifeless> bigjools: you hadn't seen this ?
<bigjools> I have
<bigjools> but it remains hilarious
<lifeless> indeed!
<lifeless> bigjools: http://www.xtranormal.com/watch/11762072/native-html5
<wgrant> ORDER BY Branch.date_last_modified DESC, Branch.target_suffix ASC, Branch.lifecycle_status DESC, Branch.owner_name ASC, Branch.name ASC
<wgrant> would you like some tie-breakers?
<wgrant> lifeless: Have you thought about getting eg. bug search totals from bugsummary?
<lifeless> I think that would rock
<lifeless> only works for some cases though
<lifeless> could be used to set upper bounds on bad estimates.
<lifeless> (can't be used to set lower bounds with our search algebra)
<wgrant> Yeah
<wgrant> Most of the common searches should work
<lifeless> personally I just want to nuke all the counts
<wgrant> Obviously not fti
<lifeless> or make them round - 10's 100's 1000's
<lifeless> put that behind a FF for a bit
<lifeless> see what the reaction is
<lifeless> if its good, change the code base to estimate everywhere.
<lifeless> bigjools: this is hilarious - http://www.xtranormal.com/watch/7023615/watch_movie
<bigjools> must...resist....
<bigjools> too much to do
<mwhudson> lifeless: i didn't think the native-html5 one was very funny
<mwhudson> it was quite odd, though, so that's something i guess
<wgrant> lifeless: Well, EXPLAIN-based estimates will often be waaaaaaay off
<lifeless> mwhudson: the html5 one was not as good as I thought it would be
<lifeless> mwhudson: the one I just linked is a follow on from the mongodb one
<lifeless> wgrant: shrug :>
<mwhudson> ahh
<lifeless> wgrant: I guess we need some care
<wgrant> It should be better now that we have bugtaskflat, I guess
<wgrant> But before bugtaskflat it would have been hopeless
<mwhudson> lifeless: ok, watch_movie is pretty good
<mwhudson> 3 mins in
<wgrant> OOPS report is pretty today :)
<lifeless> much better
<lifeless> not sure I'd call it pretty
<wgrant> Well
<wgrant> Prettier than it's been for a while
<lifeless> yes
<lifeless> massive improvement
<lifeless> just there is more to do
<StevenK> wgrant: https://dogfood.launchpad.net/ubuntu/+source/unity/+changelog
<lifeless> grah
<lifeless> Non-sql time: 5595 ms
<lifeless> if you're fixing timeouts
<lifeless> https://bugs.launchpad.net/~lifeless/?advanced=1
<lifeless> bug 1049452 if anyone wants to have a poke
<_mup_> Bug #1049452: Person:+bugs timeout on advanced search form  <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1049452 >
<lifeless> wgrant: StevenK: How do we release launchpad-buildd ?
<wgrant> lifeless: dch -i
<wgrant> lifeless: Then wait for IS to get us builders back
<lifeless> also, annoying - why is https://bugs.launchpad.net/python/+bug/96878 not FR for python, upstream has it as such.
<wgrant> Then RT
<_mup_> Bug #96878: Launchpad session cookie is accessible from Javascript <bugjam2010> <infrastructure> <lp-foundations> <qa-ok> <Launchpad itself:Fix Released by wgrant> <Python:Fix Committed> < https://launchpad.net/bugs/96878 >
<wgrant> lifeless: Resolution: accepted
<wgrant> lifeless: tsk tsk, duping your own timeout bugs
<lifeless> well, blame my short memory
<wgrant> lifeless: I was hoping the timeout was on +reportedbugs..
<lifeless> wgrant: the search works fine...
<lifeless> wgrant:
<lifeless> 57 queries/external actions issued in 1.68 seconds
<lifeless> AJAX Log â
<lifeless> not the fastest thing under the sun, but working >> not working.
<lifeless> Can't /setup/ the query though.
<lifeless> The form DIAF.
<lifeless> I bet its listing milestones for every project on Launchpad, more or less.
<wgrant> I made that pretty fast in most cases
<wgrant> But maybe you're terrible :(
<lifeless> and doing so super inefficiently. Probably a list of storm objects
<lifeless> with in
<lifeless> milestone in milestones: <- trigger the sqlbase __eq__ which is dog slow.
<lifeless> + quadratic in a list.
<wgrant> Not that slow
<lifeless> -bet you-
<lifeless> yes that slow
<lifeless> we've had 20s timeouts from that before.
<lifeless> shortly after new bug subscriptions, if you recall
<wgrant> Perhaps
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/dsp-changelog-timeout/+merge/123880
<wgrant> StevenK: Have you recowboyed that to check that the query count is now sanE?
<StevenK> No, let me that do
<StevenK> My dyslexia seems strong today
<StevenK> At least I didn't run make start stop
<wgrant> Heh
<wgrant> Huh
<StevenK> wgrant: 198 -> 51
<wgrant>  Limit  (cost=10000003268.61..10000003269.44 rows=1 width=0) (actual time=122.637..122.637 rows=0 loops=1)
<wgrant> The main +specworkload query apparently doesn't please postgres very much
<wgrant> enable_seqscan=0 applies a 10 billion cost penalty
<wgrant> But that plan still wins
<StevenK> wgrant: No review?
<wgrant> StevenK: That wasn't the only extract_bug_numbers callsite?
<wgrant> (it was)
<wgrant> Ah
<wgrant> There's one in stringformatter.py itself
<StevenK> It's called in stringformatter itself
<StevenK> Yeah
<StevenK> I can inline it and stop exporting it if you wish
<wgrant> Nah
<StevenK> Probably not worth it
<wgrant> r=me, thanks
<wgrant> We still have a few VPC queries, but that's possibly just removedby so I don't really care
<StevenK> Yeah, we could make the pre-loading smarter by including some of the data from spph
<wgrant> Not as big an issue as the bugs :)
<StevenK> Indeed, so it's fine for now
<wgrant> The page is still slow on DF, but a fair bit of it is probably template time
<StevenK> And Unity has a TON of SPRs
<StevenK> Tell me when you're done and I'll revert DF
<wgrant> I'm done
<StevenK> Tempted to close 1000257
<wgrant> bug #1000257
<_mup_> Bug #1000257: OOPS sending email notification for bug watch <bugwatch> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1000257 >
<wgrant> Indeed
<StevenK> I can't see how it happened, but it did
<wgrant> StevenK: ndt
<wgrant> Probably
<StevenK> wgrant: http://pastebin.ubuntu.com/1199974/
<lifeless> StevenK: https://devpad.canonical.com/~lpqateam/ppr/lpnet/
<StevenK> wallyworld: Nice easy one for you: https://code.launchpad.net/~stevenk/launchpad/destroy-plus-daily-builds/+merge/123888
<StevenK> wallyworld seems to be missing.
<stub> Been butting my head against pgbouncer.
<lifeless> stub: oh?
<stub> Unfortunately, root cause of my problems seems to be that KILL and RESUME just affect if opening new outgoing connections work or not.
<lifeless> stub: btw your part line is ... fun
<lifeless> '18:52 -!- stub [~stub@canonical/launchpad/stub] has left #launchpad-dev ["PART #launchpad :PART #postgresql :PART #slony :PART #storm :PART #zope3-dev :ISON cr3 jkakar daniels intellectronica daf czajkowski static SteveA fabbione Ng
<lifeless>           thumper bigjools debonzi BradB bac sinzui_is_awake jdub allenap Znarl chrism andrewv elmo lifeless `a"]
<lifeless> '
<stub> So the timouts for how long until I check if a backend is up, how long I wait for a backend connection, how long a client connection keeps retrying for a backend, all kick in
<stub> I have no idea what an ISON is
<lifeless> me neither
<stub> Anyway, unless I or someone wade into the C code, we might need two pgbouncers to do the updated fdt and actually lower downtime rather than increase it
<lifeless> stub: sure; how would that work ?
<stub> I can reduce the timeouts during the window, but to a minimum of 1 second. And that makes a huge difference if we are aiming at <5s
<stub> lifeless: Instead of KILL and RESUME on individual databases, we have separate pgbouncer instances for master and slave connections and shut them down/start them up
<stub> But it means a lot more bespoke production crap
<stub> init scripts etc.
<stub> And certainly not generalizable for outside of canonical.
<wgrant> Oh
<wgrant> I thought we had an alternative to pause that rejected new connections
<wgrant> But indeed, such a thing does not exist
<stub> wgrant: We have KILL and PAUSE, and we need KILL in current launchpad because we are running in session mode, not transactional mode
<wgrant> Right, but we have nothing like PAUSE that doesn't hang
<wgrant> WHich is what we really need
<stub> If we were running in transactional mode, yeah.
<wgrant> Even in session mode
<stub> But for current LP, the existing connections would never die unless we signalled all the clients 'oi, go reconnect now'
<wgrant> Right, but KILL serves that purpose already
<wgrant> We can kill all existing connections easily
<stub> Yes, so for now KILL does what we need
<wgrant> We just can't reject new ones claenly
<stub> KILL rejects new connections
<wgrant> If we run it constantly, yes
<stub> The documentation is fuzzy there
<wgrant> But it's a momentary thing
<wgrant> It drops all current and pending clients at the moment it's run
<wgrant> If we PAUSEd and ran kill every 10ms, it would work
<stub> On my system, KILL keeps it dead until RESUMEd
<stub> which is why I say the docs are wrong
<wgrant> I can still connect, it just hangs
<wgrant> On 1.5.1
<wgrant> If I then KILL again, the hung connection drops, as expected
<stub> $ psql -p 6432 launchpad_dev_master
<stub> psql: ERROR:  pgbouncer cannot connect to server
<wgrant> That's very interesting
<wgrant> Do you have a tiny bouncer->server timeout or something?
<stub> 2012-09-12 14:43:01.020 5838 LOG C-0x1e0d518: launchpad_dev_master/stub@unix:6432 login failed: db=launchpad_dev_master user=stub
<stub> 2012-09-12 14:43:01.020 5838 LOG S-0x1e2aec8: launchpad_dev_master/stub@unix:5432 closing because: pause mode (age=0)
<stub> wgrant: oh, you are experiencing the same problem my test case picked up
<stub> wgrant: After a KILL, the pools don't know that the db is down so keep trying to open connections, and failing because the db is down
<stub> wgrant: So it keeps retrying for 15 seconds until the 'db is down' flag is set
<stub> wgrant: And then, after a RESUME, that pool is still stuffed as it only tries to reconnect to a dead db every N seconds.
<wgrant> Ah, there we go, it does eventually time out
<wgrant> And now it is dead properly
<wgrant> So what we really need is a way to force it to poll and time out quickly
<adeuring> good morning
<lifeless> transactional mode still woudn't be fast enough, would it ?
<wgrant> I don't think so
<wgrant> Ah
<wgrant> It's pretty easy to make it to do what we want
<wgrant> Like
<wgrant> Really easy
<lifeless> share
<wgrant> Will have patch in a sec
<wgrant> Particularly if I don't try to run an amd64 binary on i386
<stub> wgrant: I can reduce the timeouts, but it still can add about 2 seconds to the time according to my test case
<stub> wgrant: Patch welcome though, I hate relearning C everytime I need to do this.
<wgrant> It all works, except that changing the size of the PgDatabase struct makes it segfault
<wgrant> Just adding the new unused flag
<wgrant> Segfault
<wgrant> But if I reuse one of the existing flags it all works
<wgrant> Will look further after dinner
<wgrant> I can't see a hardcoded size anywhere obvious
<czajkowski> jelmer: when you get a chance can you follow on the Question you were on https://answers.launchpad.net/launchpad/+question/206501  so I can see if it can be closed.
<czajkowski> cheers
<wgrant> (it turns out their makefile deps suck, so a make clean fixed it)
<wgrant> stub: http://pastebin.ubuntu.com/1200173/
<wgrant> You currently need a DISABLE/KILL/RESUME then ENABLE sequence
<wgrant> As disable doesn't actually kill, and kill implicitly pauses
<lifeless> nice
<stub> Cool
<stub> That on github?
<wgrant> I hate tracking down the official repo on GitHub
<wgrant> Maybe there's a link somewhere...
 * wgrant googles
<stub> I could argue that KILL should imply DISABLE
<wgrant> Indeed
<wgrant> Alternatively KILL could just kill
<wgrant> and you're expected to PAUSE first
<stub> yeah, I can take the patch to the mailing list
<wgrant> https://github.com/markokr/pgbouncer-dev looks like it
<stub> Unless they use git://git.postgresql.org/git/pgbouncer.git directly
<wgrant> They do, but the github account is owned by the primary committer
<wgrant> So it seems relevant
 * wgrant has forked it
<wgrant> I always forget how obtuse git is
<wgrant> stub: I've revised it a bit, so it now rejects only at client connect time and is not so terrible. https://github.com/wgrant/pgbouncer-dev/compare/disable
<wgrant> I'll create a pull request if you don't object soon
<stub> wgrant: Cool. I was going to build a package here to test my test case
<wgrant> Great
<stub> (running through the full process before I go and hack up full-update.py)
<wgrant> You still need the DISABLE/KILL/RESUME, do stuff, ENABLE sequence, but I'll mention improving that on the MP
<wgrant> If you leave it paused too long it'll still lose the backend connections and hang once you reenable
<wgrant> But if you resume immediately after killing you can keep it disabled forever
<wgrant> Also note that DISABLE only forbids new client connections. An existing client connection can still grab a server connection if it hasn't already. I could forbid in find_server as well, but I'm not sure if it's worth it
<wgrant> Given you need to KILL anyway
<stub> So htf do I convert https://github.com/wgrant/pgbouncer-dev/compare/disable to a diff?
<wgrant> stub: You're beyond my GitHub knowledge at this point, I'm afraid
<wgrant> Let me poke around
<stub> I guess I'm locked in ;)
<wgrant> http://pastebin.ubuntu.com/1200362/
<Laney> stick ".patch" on the end of that URL
<wgrant> Ah
<wgrant> I tried .diff
<wgrant> But not .patch
<wgrant> Hm
<wgrant> .diff works now too
<wgrant> I must have been on a different URL
<stub> Because hacking urls is discoverable UI
<tumbleweed> on commits, .diff and .patch give you different things
<wgrant> Yeah, .patch gives headers and commit message and stuff
<stub> Still seems noise to strip
<stub> oh, guess diff ignores it
<stub> no, noise to strip.
<stub> oh no, I'm on 1.5.2, patch is trunk
<wgrant> There shouldn't be conflicts outside NEWS
<stub> yeah
<stub> I just need more coffee
<stub> built the package, manual tests look good
<stub> And my updated tests like it too.
<stub> wgrant: Looks good.
<wgrant> Great
<wgrant> My only significant concern is that an evil client could connect but not cause a server connection to be established.
<wgrant> Then you DISABLE, and wait for connections to disappear from the backend server
<wgrant> But if you don't KILL, there could still be someone lurking in pgbouncer
<wgrant> waiting to strike and ruin your day
<wgrant> (to reproduce, open up a psql session but don't do anything. Call DISABLE in another session, and see that you can still talk in your original session)
<wgrant> stub: Hopefully this will be a little less messy (and faster!) than sudoing around multiple instances
<stub> Oh, much.
<stub> We will need to KILL anyway after a timeout
<stub> To catch evil transactions.
<wgrant> Yeah
<wgrant> Definitely
<wgrant> But for the master we will just want to DISABLE and then KILL immediately
<stub> http://paste.ubuntu.com/1200439/ is the test case for the always-slave process, and it is passing with the new pgbouncer
<wgrant> Oh, an actual LP test case
<wgrant> Fancy
<StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/buildfarmjob-index-redux/+merge/123854 would love a review
<wgrant> stub: Looks good
<stub> got the failwhale
<stub> not even an oops
<wgrant> 502?
<wgrant> Huh
<wgrant> Haven't seen many of those since we switched back to the rightful DB servers
<stub> StevenK: Didn't we do this already?
<StevenK> stub: Almost. The previous index did not have status, this one does. The two of them will work in concert for Builder:+history
<stub> yer
<StevenK> stub: Builder:+history with the index we added to prod makes it really fast. Builder:+history also allows filtering by status, which is slow.
<wgrant> Well, slow for uncommon statuses
<StevenK> Right.
<StevenK> It's noticibly slower. Like 4 seconds versus 0.4
<stub> It it ready for prod or just qastaging?
<stub> Oh, I'm building this pgbouncer package for precise. I can just copy it to lucid and it will be fine, right?
<StevenK> I think it should be fine, but I admit to not actually testing it.
<StevenK> stub: I'm happy to play around on DF tomorrow if you're more comfortable with me having tested it.
<stub> Just removing it requires a FDT update. Apart from that no real worries applying it to prod.
<wgrant> stub: I'm not sure if the same binary will work
<wgrant> We have a ~lucid1 for 1.5.1 atm...
<stub> So reupload for lucid or make a recipe to work around the limitation
<wgrant> Exactly
<stub> StevenK: Its on qastaging
<StevenK> Let's have a look
<StevenK> 0.8 hot for main
<StevenK> 1.3 for FTBFS hot
<wgrant> SUPERSEDED is more interesting
<StevenK> 0.9 for depwait hot
<StevenK> superseded is 2.2 cold, 0.29 hot
<wgrant> Excellent.
<wgrant> To production we go.
<StevenK> How about I land it first?
<StevenK> :-)
<StevenK> stub: It's landed as r15944, you can add it to prod at your leisure
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: abentley | Critical bugs: â
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: â
<abentley> deryck: I'm the OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/spec-creation-info-type/+merge/123821 ?
<deryck> abentley, sure
<abentley> deryck: Thanks.
<deryck> np!
<abentley> rick_h_: btw, my enum is lp.registry.enums.PUBLIC_PROPRIETARY_INFORMATION_TYPES
<rick_h_> abentley: cool, thanks. Will look at updating that today.
<sinzui> adeuring, can you comment on bug 839779 and bug 450480 about how to fix them?
<_mup_> Bug #839779: HWDB doesn't create HWDeviceClass anymore <hwdb> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/839779 >
<_mup_> Bug #450480: Get PCI and USB vendor/product names from the "PCI ID Repository" project and from linux-usb.org respectively <hwdb> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/450480 >
 * adeuring is looking
<adeuring> sinzui: I am a bit lost ATM regarding
<adeuring> #839779
<_mup_> Bug #839779: HWDB doesn't create HWDeviceClass anymore <hwdb> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/839779 >
<adeuring> I guess that newer HWDB reports do not contain any data about devices classes anymore.
<adeuring> but I need to check
<adeuring> sinzui: regarding bug 450480: WHat do you miss in the bug report?
<_mup_> Bug #450480: Get PCI and USB vendor/product names from the "PCI ID Repository" project and from linux-usb.org respectively <hwdb> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/450480 >
<sinzui> adeuring, why a cronscript? Shouldn't the data be processed correctly when it is uploaded?
<sinzui> or put another way? I do not understand why hal has that data, but udev does not?
<adeuring> sinzui: ask the udev developers ;) The data is probably not important enough
<sinzui> okay. So want a process that gathers the secondary data to provide consistent, historic reporting?
<adeuring> sinzui: the script that processed the HWDB report can of course also read the two URLs directly
<adeuring> and use the data.
<adeuring> but we should also update existing "bad" records.
<sinzui> okay. and calling out to external site is blocked for most servers
<adeuring> sinzui: that's a problem indeed... But we might be able to read the data manually form time to time
<sinzui> understood. The data does not change often, which is why you want to pull it daily or weekly
<adeuring> sinzui: right.
<sinzui> okay. I think I know enough to fix this
<deryck> abentley, your code looks great.  well tested and solid.  r=me.
<abentley> deryck: Thanks.
<adeuring> sinzui: BTW, we wil always have the situation that a report contains a devcie where vendor/product names are yet knwon, even in the two databases mentioned in the bug report.
<adeuring> I don't know how often these DBs are updated, but we must always expect some lag
<adeuring> when for example some small vendor sells a new USB mouse
 * sinzui nods
<sinzui> adeuring, what do you need to know about HWDeviceClass in staging's db?
<sinzui> is there a query I can run?
<adeuring> sinzui: the existing data itself is not a problem. Just need to check why no nbewer data exists
<adeuring> sinzui: after looking a bit more at the output of "udevadm info --export-db" I still have no idea why HWDeviceClass is not updated. Could be a bug in the processing the reports. On the other hand, it would make sense to check if the data from recent submissions really has no links to HWDeviceClass records. After all, the set of device _classes_ is rather small compared with the number of different device models: A USB mouse is is a USB mou
<adeuring> so it might even be a "non-bug"
<sinzui> okay. I will look into this.
<sinzui> adeuring, who uses the the hwdb now?
<adeuring> sinzui: I have no idea... ask cr3
<sinzui> oh, I did
<adeuring> or flacoste
<sinzui> he uses the XML only
<cr3> adeuring: ogasawara still uses the hardware db
<adeuring> ah, right
<rick_h_> abentley: quick ok on the change over to your enum please? https://code.launchpad.net/~rharding/launchpad/updated_product_allowed_types/+merge/123987
<abentley> rick_h_: r=me.
<rick_h_> abentley: thanks
<sinzui> jcsackett, do you have a few moments to discuss Bug #810664
<_mup_> Bug #810664: Specification:+index asserts -  (from_node, to_node) not in self.edges <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810664 >
<jcsackett> sinzui: sure.
<sinzui> jcsackett, http://pastebin.ubuntu.com/1200924/
<sinzui> jcsackett, I just got my replacement router. I am going to drop off the net for a bit
<deryck> abentley, I have two branches related to specification_sharing_policy if you have the bandwidth to review them.
<abentley> deryck: Sure.
<deryck> abentley, thanks.  see....
<deryck> https://code.launchpad.net/~deryck/launchpad/specification-sharing-policy-unused/+merge/124021
<deryck> and
<deryck> https://code.launchpad.net/~deryck/launchpad/specification-sharing-policy-garbo/+merge/124026
<abentley> deryck: The allowed types for SpecificationSharingPolicy.PUBLIC, SpecificationSharingPolicy.PUBLIC_OR_PROPRIETARY, SpecificationSharingPolicy.PROPRIETARY_OR_PUBLIC look incorrect because they include InformationType.PRIVATESECURITY, InformationType.USERDATA, InformationType.PUBLICSECURITY
<abentley> deryck: In the definition of setSpecificationSharingPolicy, can we use IProductPublic.specification_sharing_policy directly instead of copying it?
<deryck>  abentley, sorry was on call with flacoste
<deryck> abentley, so to the first question, sorry about that.  copied from branch which I thought would be the same. I'll take another pass at that.
<abentley> deryck: Cool.
<deryck> abentley, as for the copy, I did this exactly like bug and branch sharing policy.  both of those copied.  So I didn't question it.
<deryck> I don't know if there's some reason it needs the copy honestly.  and I was just playing it safe.
<abentley> deryck: I had a grovel through copy_field recently, and it seems like its only purpose is to let you change attributes (like readonly).  So give it a try without the copy and I bet it will work.
<deryck> abentley, ok, will do.  I can change the other places too and try them.
<abentley> deryck: Should your branch also be defining default values for the policies?  I don't see that.
<abentley> deryck: Bugs has lp.bugs.interfaces.bugtarget.BUG_POLICY_DEFAULT_TYPES for example.
<deryck> abentley, right.
<deryck> abentley, so I didn't realize those were needed until the two methods coming in the final branch.
<deryck> abentley, I have them in that, but can bring them up to this branch if it makes more sense.
<abentley> deryck: No, that's fine as long as it's in the plan.
<deryck> abentley, ok, cool.
<abentley> deryck: Do you think there should be a policy that permits PUBLIC, PROPRIETARY and EMBARGOED?
<deryck> abentley, on TL call, just a minute and I'll consider that.
<jcsackett> quit
<deryck> don't do it jcsackett!  it will be better tomorrow, I'm sure. ;) :)
<deryck> abentley, ok, thinking on that now....
<jcsackett> deryck: :-P
<lifeless> sinzui: do you need a link to the zconfig bug ?
<sinzui> I saw it yesterday in my browser
<sinzui> lifeless, it affect lp and zope right, and I think fred was the reviewer?
<deryck> abentley, so what use case are you thinking of?  Starting public, but shifting to either of the other two later?  or something else?
<lifeless> sinzui: yes
<lifeless> sinzui: that sounds right
<sinzui> lifeless, send me the bug, as my history does not want to reveal it
<abentley> deryck: No, I guess I was just thinking that we wouldn't want to prevent people using Embargoed from doing everything else that people using Proprietary can do.
<lifeless> sinzui: abel took a stab at it, but sadly only ended up moving the issue around, we didn't understand quite what was going on.
<lifeless> bug 481512
<_mup_> Bug #481512: race condition when rotating logs <canonical-losa-lp> <lp-foundations> <qa-untestable> <Launchpad itself:Triaged> <ZConfig:Fix Released by fdrake> < https://launchpad.net/bugs/481512 >
<abentley> deryck: I don't have a use case really, unless people other than PES use Embargoed.
<sinzui> yes, I saw the branch apparently. thank you lifeless
<lifeless> sinzui: de nada
<lifeless> sinzui: thank *you*
<abentley> deryck: We are famous now: http://feedproxy.google.com/~r/d0od/~3/3xNeugsEvpU/ubuntu-12-10-login-screen-adds-remote-desktop-access
<deryck> abentley, nice!
<deryck> abentley, I hope they have that server beefed up more than we did. ;)
<abentley> :-)
<abentley> deryck: Could you please review https://code.launchpad.net/~abentley/launchpad/fix-banner/+merge/124052 ?
<deryck> abentley, absolutely.
<deryck> abentley, should you be a bit more specific in the test?  to make sure that the phrase is actually in the banner, and not somewhere else in browser.contents.
<abentley> deryck: I can do that.
<deryck> abentley, ok, cool.  other that that, looks great.  I like the IInformationType approach.
<abentley> deryck: Great.  I considered extending IPrivacy, but I saw that there are circumstances where we know whether it's private, but not what the information_type is.
<deryck> abentley, right.  and r=me officially now on the mp.
<abentley> deryck: Thanks.
<deryck> np!
<abentley> deryck: r=me on the first one.
<deryck> abentley, thanks!
<lifeless> sinzui: can I hand that branch off entirely ?
<sinzui> okay
<sinzui> It is now mine
<lifeless> sinzui: thanks, much appreciated.
<wgrant> StevenK: OOPS-2bf590b755c7e5cd89893f05cfb1eb91
#launchpad-dev 2012-09-13
<StevenK> wgrant: Hm, I wonder if we should error if authorized_size is null for ArchivePurpose.PPA
<wgrant> StevenK: Or just enforce the quota iff authorized_size is set
<wgrant> Rather than iff it's a PPA
<StevenK> wgrant: This isn'
<StevenK> Bleh
<StevenK> I'm not sure about that bug, I'm looking at bug 965317
<_mup_> Bug #965317: TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'  on +repository-size page <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/965317 >
<wgrant> Ah, so separate but related
<wgrant> If there's no quota then there's no quota, I guess
<wgrant> http://irclogs.ubuntu.com/2012/02/18/%23launchpad-dev.txt
<StevenK> Right
<lifeless> can has review ? - https://code.launchpad.net/~lifeless/lp-dev-utils/ppr/+merge/124084
<lifeless> small
<lifeless> wgrant: ^
<lifeless> StevenK: ^?
<StevenK> lifeless: r=me
<lifeless> thanks'
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<lifeless> wgrant: test failures ~ critical?
<wgrant> lifeless: Running the test suite outside X or xvfb-run is unsupported
<wgrant> So that one is not critical
<lifeless> oh, I didn't notice that nuance.
<lifeless> where was he running it ?
<wgrant> Assuming you're talking about bug #1020146
<_mup_> Bug #1020146: lp.testing.layers.YUITestLayer tests fail in various ways without X <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1020146 >
<wgrant> canonistack, I assume
<lifeless> I'd wontfix in fact, if its not a supported config
<wgrant> Nah
<wgrant> It should work
<lifeless> how?
<wgrant> There's no reason our test browser should need to use X
<StevenK> But it's not critical
<wgrant> That's why I made it not critical.
<rick_h_> all of the webkit enabled browsers require x though I think phantomjs was working around it. Not sure if they got it done
<rick_h_> it's how they're working, by using the webkit ui toolkits to load the pages
<wgrant> Right
<wgrant> But nothing about this fundamentally requires X
<wgrant> It's just the library we happen to use atm
<lifeless> alloftheones
<rick_h_> so yea, phantomjs would be a good way to go: http://code.google.com/p/phantomjs/wiki/XvfbSetup
<rick_h_> I've testsed trying to run with grover + phantomjs but it blows up horribly currently
<lifeless> wgrant: so what was bug 739042 ?
<_mup_> Bug #739042: DistroArchSeries:+index timeouts <timeout> <Launchpad itself:Fix Released by wgrant> < https://launchpad.net/bugs/739042 >
<wgrant> lifeless: Terrible FTI that I fixed a few months back
<lifeless> ah cool
<wallyworld> wgrant: looking at another web service exported collection - IBug.bug_tasks. It also just gives a list of dict values ie lp.bugs[5].bug_tasks.entries gives [{...}, {...}] so i'm yet to find a collection where one can get at objects
<wallyworld> so i still can't see what the bug is wanting different to what we currently do elsewhere
<wallyworld> or if it's possible/feasible
<wgrant> wallyworld: Oh, why are you calling .entries?
<wallyworld> to get the values in the Collection
<wgrant> Iterate :)
<wgrant> Asking for the entries will get you the raw entries
<wgrant> In [3]: list(lp.bugs[5].bug_tasks)
<wgrant> Out[3]: [<bug_task at https://api.launchpad.net/devel/launchpad/+bug/5>]
<_mup_> Bug #5: Plone Placeless Translation Service metadata missing from po files <feature> <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/5 >
<wallyworld> oh, ok. i didn't realise entries did that
<wallyworld> and that i needed to iterate
<wallyworld> thanks
<wgrant> The collection is an iterable of EntryResources
<wgrant> It also has a .entries attribute which is an iterable of the raw JSON representation of the entries
<wallyworld> ok. i didn't appreciate there was a difference, i thought .entries just have the values in the current batch
<lifeless> heh, nice trap
<wallyworld> yeah, i fell into it. i'm still no expert on our restful api
<lifeless> is anyone/
<wallyworld> StevenK: auditor client bb failure!
<StevenK> Strange
<wgrant> sinzui: Can we kill bug #450480?
<_mup_> Bug #450480: Get PCI and USB vendor/product names from the "PCI ID Repository" project and from linux-usb.org respectively <hwdb> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/450480 >
<sinzui> I have not decided...
<sinzui> maybe you cane
<sinzui> help
<sinzui> Leann does is not affected by the bug. She will not no if we fix it
<sinzui> It is tempting to say that the unused feature is not a regression
<wgrant> lifeless: Your last act on #579602 confuses me
<_mup_> Bug #579602: cannot call getBranches on a source package using the API <api> <lp-code> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/579602 >
<sinzui> wgrant, I think can say wont fix, but what of the oopses...I don't think we can close them until the code is removed
<wgrant> sinzui: Which OOPSes?
<sinzui> wgrant, https://bugs.launchpad.net/launchpad/+bugs?field.importance%3Alist=CRITICAL&field.tag=hwdb
<lifeless> wgrant: me too
<wgrant> sinzui: The other three must remain by ZOP
<wgrant> sinzui: But the 450480 is a regression, not an oops
<sinzui> I agree
<sinzui> wgrant, as I suggested in the meeting, we can ignore them until we get the word that HWC has their replacement operational. There are about 31 bugs that can be closed with we remove the hwdb
<wgrant> sinzui: Right, but I'm going to demote the regression since nobody cares
<sinzui> +1
<StevenK> sinzui: I'm going to remove the Builder:+history timeout FF on prod with your blessing
<sinzui> Sure. We can put it back if we are wrong
<lifeless> wallyworld: are you still working on 681767 ?
<lifeless> bug 681767 that is
<_mup_> Bug #681767: Can't use relative URLs with launchpadlib.Launchpad - generates URI object that breaks wadllib <lp-foundations> <lazr.restfulclient:In Progress by wallyworld> < https://launchpad.net/bugs/681767 >
<wallyworld> lifeless: i think that one was put on hold
<wallyworld> since my solution was rejected
<lifeless> kk
<lifeless> thanks
<lifeless> sadface at the fix committed etc being images
<wgrant> lifeless: Where?
<lifeless> bug listings
<wgrant> They're not images
<lifeless> oh, wtf then
<lifeless> ctrl-f won't find them
<wgrant> Does so :)
<lifeless> mmm, I must have had a typo.
<lifeless> weird.
<wgrant> Works for Triaged in Firefox at least
<lifeless> ah
<lifeless> the fix is funky
<lifeless> try this
<lifeless> https://bugs.launchpad.net/~lifeless/?field.status%3Alist=FIXCOMMITTED
<lifeless> ctrl-F
<lifeless> fix
<lifeless> in firefox
<wgrant> Works
<lifeless> and nope, commit doesn't either.
<lifeless> unless I click in one of them
<lifeless> then it works
<wgrant> Works either way for me
<lifeless> so, doesn't for me :)
<lifeless> until I click in the page
<wgrant> Huh
<lifeless> do you have match case enabled ?
<lifeless> cause I did :(
<lifeless> <- muppet
<wgrant> haha
<lifeless> UI could tell you
<wgrant> Indeed
<lifeless> '15 more hits if match case is turned off'
<lifeless> wgrant: so - http://bugs.python.org/issue1638033 - what should the thing be set to to close in LP?
<lifeless> its 'accepted closed' atm,
<lifeless> surely closed should mean LP puts it in a terminal state ?
<wgrant> Probably
<wgrant> But I believe 'fixed closed' is what LP expects
<wgrant> Perhaps they've changed
<lifeless> what /is/ the python tracker ? RT ?
<wgrant> Roundup
<lifeless> ah
<wgrant> We have a custom set of mapping tables for the Python Roundup in our codebase
<lifeless> yay
<lifeless> well if you care - https://bugs.launchpad.net/launchpad/+bug/1050173
<_mup_> Bug #1050173: bugwatches set to accepted closed in roundup maps to 'fix committed' not 'fix released' <bugs> <bugtrackers> <bugwatch> <roundup> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1050173 >
<wgrant> I'd say Python roundup
<wgrant> I believe they're all different
<wgrant> We certainly have a custom status map for python
<lifeless> tweaked
<wgrant> Thanks
<wgrant> There's 3 or 4 checkwatches criticals, so someone's likely to get around to touching that area soon
<lifeless> \o/
<lifeless> on the make lifeless' bug list clean effort ;)
<wgrant> Heh
<wgrant> We should make more pages less terrible
<wgrant> Particularly since that tends to involve making code shorter
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/deal-with-no-authorized_size/+merge/124097
<wgrant> StevenK: Rejected due to invalid apostrophe use
<wallyworld> +100
<StevenK> What invalid apostrophe use ...
<wallyworld> sigh
<wallyworld> why don't people know grammar these days
<StevenK> I just fixed it, so I'm playing innocent.
<wallyworld> :-)
<wallyworld> it's one of my pet peeves
<wgrant> StevenK: Also, I'd prefer that you fixed policySpecificChecks  rather than checkArchiveSizeQuota, maybe
<wgrant> StevenK: At least de-dupe the check
<wgrant> It may be best to remove the PPA guard around the checkArchiveSizeQuota check, I guess.
<StevenK> wgrant: Why? There are other checks in checkArchiveSizeQuota.
<wgrant> After checking on production that it won't impact the primary archive
<wgrant> We can apply the quota check to any archive with a quota set
<StevenK> wgrant: Hmmmm
<StevenK> wgrant: http://pastebin.ubuntu.com/1201917/ ?
<wgrant> StevenK: You've changed the security rejection behaviour
<wgrant> Also, you might want to explicitly check if it's not None
<wgrant> Rather than if it's not 0
<StevenK> wgrant: is None, even
<wgrant> Well, yes.
<wgrant> that
<StevenK> wgrant: Changed? Sure, but I don't think it will impact anything.
<StevenK> wgrant: And "SELECT id, name FROM archive WHERE authorized_size IS NOT NULL AND purpose <> 2;" == 46 on DF
<wgrant> StevenK: It'll now reject security uploads to PPAs with a different message
<StevenK> wgrant: PPAs do not have a security pocket?
<wgrant> Right, they wouldn't have been accepted, but this might change the way they're rejected.
<StevenK> So I don't think it's a concern. It is certainly something to keep in mind when it's QA'd.
<StevenK> wgrant: ^ Agree or disagree?
<wgrant> StevenK: Have you checked the history of that method, to see why the guard was added?
<wgrant> It's possible it was incidental when the quota check was added
<wgrant> But it would be nice to check before we remove it
<StevenK> wgrant: For checkArchiveSizeQuota? Because it was written only for PPAs.
<wgrant> StevenK: But there's the opposite guard for the security pocket check
<StevenK> Because other archives have pockets, I guess
<wgrant> Right
<wgrant> It may have been there for a reason
<wgrant> Or it may not
<wgrant> But we should check before removing it
<StevenK> The else was added in r15663
<wgrant> Readded, perhaps
<StevenK> [r=wgrant][rollback=15658] Revert r15658, as uploads to -security will have their builds automatically failed. Update associated comments.
<wgrant> cjwatson removed it
<wgrant> Ah yeah
<wgrant> Then I readded it
<wgrant> annotate before that
<wgrant> Ah, no, cjwatson reverted it, but I reviewed it?
<wgrant> Odd
<wgrant> The else is a little over 5 years old
<wgrant> I would remove it
<wgrant> It was added with the ubuntero PPA check
<StevenK> You mean, like I did? :-)
<wgrant> Yes
<wgrant> But with verification
<StevenK> Which we just did
<wgrant> Indeed
<wgrant> Push and I can approve :)
<StevenK> wgrant: Diff updating. I attempted to destroy archive-views.txt, but I'm sad to say it defeated me.
<StevenK> wgrant: And diff updated
<ajmitch> https://code.launchpad.net/~ajmitch/launchpad/prerelease-backports/+merge/124100 for minimal change, if you want to comment
<StevenK> 8+ def test_insecure_does_not_approve_backports_before_release(self):
<StevenK> 10+ self.setHoaryStatus(SeriesStatus.CURRENT)
<StevenK> Uhhhh?
<StevenK> :-)
<ajmitch> lulwut
<StevenK> Your two test cases are indentical
<ajmitch> yeah, now I see that
<ajmitch> I wasn't sure whether to even keep those since I'm not touching the uploadpolicy now
<StevenK> Indeed
 * ajmitch was copying from the earlier tests in there, fwiw 
<ajmitch> just copied a bit too vigorously
<StevenK> I was about to say something like that
<StevenK> ajmitch: So, you don't touch the uploadpolicy at all, which means the comments for the two tests is technically correct, but only because the policy doesn't check
<StevenK> ajmitch: bzr revert -r submit: lib/lp/archiveuploader/tests/test_uploadpolicy.py && bzr ci ?
<wgrant> lifeless: Thoughts on my suggestion in bug #1050191?
<_mup_> Bug #1050191: allocate-revision-karma.py is too slow to complete <branches> <karma> <lp-code> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1050191 >
<ajmitch> right, I'd initially made the mistake of changing the approval policy & wrote the test
<StevenK> RARGH
<StevenK> Is the filename on neem pointless or something?
 * ajmitch commits & pushes
<StevenK> Bleh, I have a file with a current OOPS and the sha1 in the filename isn't the OOPS ID
<ajmitch> StevenK: ok, less LOC added now :)
<StevenK> ajmitch: test_checkUpload_backports_development could do with a comment
<StevenK> wgrant: Thanks for the +1, tossing at ec2
<ajmitch> ok to plagiarise from the test above, or do you like to have the bug # in there as well?
<StevenK> ajmitch: Just plagiarise, no need for the bug number
<StevenK> Right, so the oops id is 3607c5f44e382d14bd9ff678dbc51097, but the filename is OOPS-4f2364563d7a41ea4918845f2d86aec5.
 * ajmitch waits for the diff to update
<StevenK> Ah, so '?id%OOPS-' to search for. Sigh.
<wgrant> ?
<StevenK> Ah, so '?id%OOPS-' is the magic string to search for.
<wgrant> Well
<wgrant> I grep for the actual unique bit
<wgrant> Oh, you mean finding the web-accessible OOPS ID in a known file?
<StevenK> I had the filename, but not the ID
<StevenK> wgrant: Yeah
<wgrant> I just search for OOPS- :)
<StevenK> wgrant: I'm happy to approve ajmitch's MP, do you see any issues with it?
<wgrant> It's from New Zealand
 * ajmitch was expecting a comment like that
<StevenK> Oh, so, "This was written by a PHP developer." status => Rejected
<wgrant> That too
<ajmitch> the love I get in here
<wgrant> ajmitch: Have you discussed this with cjwatson?
<ajmitch> only in passing, not the submitted branch
<wgrant> Ah, I see TB discussion
<wgrant> Have you considered the issue of autoapproval?
<ajmitch> they should land in unapproved
<ajmitch> I'd considered it, made a change in uploadpolicy & then reverted once I clarified that they'll always be unapproved
<wgrant> Right, only RELEASE and PROPOSED (and only in unstable series) get autoapproved
<wgrant> I'd like to see an ack from Colin just in case, but otherwise that looks fine
<ajmitch> ok
<StevenK> Let me approve it, with that note
<ajmitch> I'll ping him later when he's hopefully around
<ajmitch> thanks for the review
<ajmitch> not that the diff was overwhelming
<wgrant> Heh
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/double-archivesub/+merge/124102
<wgrant> StevenK: It already deals with it
<wgrant> By raising an exception
<StevenK> wgrant: That isn't caught and ends up as an OOPS
<wgrant> StevenK: Right, but you could catch it
<wgrant> Or make it a 400 if it's an issue with the API
<StevenK> wgrant: I could, I decided to follow the pattern that we have for bug/branch subscriptions
<wgrant> It looks like that can only happen over the API
<wgrant> So I would just annotate it with BAD_REQUEST
<StevenK> https://oops.canonical.com/?oopsid=3607c5f44e382d14bd9ff678dbc51097 disagrees
<wgrant> Ah, indeed, missed the two browser callsites
<wgrant> In one case you want to crash
<wgrant> In the other you probably want to inform the user that they're insane
<wgrant> Alternatively, in the latter ('activate') case you could call getAuthToken first
<wgrant> Or catch the exception and call getAuthToken
<StevenK> We sort of
<StevenK>         if self.request.form.get('activate') and not self.active_token:
<wgrant> Hm, then do you know how the exception happened?
<wgrant> Perhaps the token was deactivated
<StevenK> Hm, perhaps
 * StevenK tries to remember where PersonArchiveSubscriptionView is hit from
<wgrant> You mean Person:+archivesubscriptions?
<wgrant> Or one of the archive-specific views?
<StevenK> I created an authtoken and then called deactivate on it and the UI says I have no tokens
<wgrant> You need a subscription
<wgrant> The UI shows subscriptions
<StevenK> Hm, Archive:+subscriptions is forbidden as an admin?
<wgrant> Yes
<wgrant> Admins don't hold launchpad.Append on IArchive
<wgrant> It's like launchpad.Special in that regard
<StevenK> Right, now I have a subscription
<StevenK> Then I set it to expired and it disappears from the UI again
<wgrant> Sure
<wgrant> Doesn't necessarily mean it's not accessible
<wgrant> It's also possible that the token is somehow deactivated, but the sub is not
<StevenK> Oh, there's a seperate table for the token?
<wgrant> (eg. a team sub was created, a token was issued, then the team sub was revoked, then a person sub was created, maybe?)
<wgrant> yes
<wgrant> Subscriptions can be for teams
<wgrant> Tokens can't be
<stub> wgrant: is this index I see before me to replace an existing index, or in addition?
<wgrant> stub: In addition
<stub> wgrant: Why have status included then, as the preamble states it is indended for use when not filtering by status?
<wgrant> stub: It's not necessary, but due to that order it also doesn't affect size/performance significantly.
<wgrant> I can remove it if you want
<wgrant> It's not really useful until we have index-only scans, I guess
<wgrant> cjwatson: What blockers remain to prevent the removal of delayed copies?
<adeuring> good morning
<stub> wgrant: So that new index, is that status ever going to be used? We already have an alternative index on (archive, das, status, bpn)
<stub> wgrant: Unless we need to use the four fields for ordering, but I don't think we ever want to do that.
<wgrant> Indeed
<wgrant> I might drop status
<wgrant> stub: That's done
<stub> ok
<stub> Is this the correct way for force the use of our patched pgbouncer in Depends: ? pgbouncer (>= 1.5.2-2+lp2, < 1.5.2-3)
<wgrant> stub: You need two separate dependencies
<stub> ok, so don't just believe that builddeb worked :)
<stub> 1.5.2-2+lp2~23~lucid1 and 1.5.2-2+lp2~23~precise1 are the two interesting versions it needs to work with
<wgrant> Also, << rather than <
<stub>   pgbouncer (>= 1.5.2-2+lp2), pgbouncer (<< 1.5.2-3),
<maxb> It might be more practical to make the patched pgbouncer "Provides: pgbouncer-lp" and depend on that, unless the patch is going to get upstreamed shortly
<wgrant> That won't work.
<wgrant> Due to ~
<wgrant> Yeah
<wgrant> I was about to suggest the Provides
<wgrant> But it will hopefully be upstream soonish
<wgrant> For the versioned Depends to work, you'd want pgbouncer (>= 1.5.2-2+lp2~), pgbouncer (<< 1.5.2-3~)
<wgrant> But the Provides is probably a better solution for now.
<stub> Sure about that twiddly after the 3?
<wgrant> Yes, otherwise eg. 1.5.2-3~lucid1 will satisfy your dep
<stub> oh, guess so
<wgrant> Which you probably don't want
 * cjwatson adds a kanban card to have a look at ajmitch's MP a bit later
<cjwatson> wgrant: I have branches leading up to it all locally.  The main blocker is that bigjools raised a concern about the PCJ queue for PPAs being starved if e.g. we'd just run an auto-sync.
<cjwatson> wgrant: Which is arguably a blocker for removing the feature flag guarding the fix for bug 575450 (releasing that from beta), which will allow us to remove the synchronous +copy-packages mode and thereafter remove delayed copies.
<_mup_> Bug #575450: Archive:+copy-packages nearly unusable due to timeouts <lp-soyuz> <ppa> <qa-ok> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/575450 >
<wgrant> cjwatson: Ah, right.
<wgrant> I'd forgotten about the starvation issue
<cjwatson> wgrant: I'm not sure whether the proper fix is to bring up multiple cron jobs for PCJ processing, or to convert it all to celery.
<cjwatson> The vague consensus seemed to be the latter, but also that celery was a bit untried.
<wgrant> That's one way of putting it
<cjwatson> I was due to try out celery for ProcessAcceptedBugsJob, when I get back to that ...
<wgrant> Indeed
<wgrant> adeuring: It looks like a recent private projects landing has broken buildbot. Can you investigate?
<rick_h_> adeuring: ping
<rick_h_> nvm, looks like wgrant already ping'd you on it
<adeuring> morning rick_h_
<rick_h_> adeuring: sorry, was just going to see if you'd peeked at the buidbot breakage. Looks like it's breaking on a query around specification stuff that landed
<adeuring> rick_h, wgrant: acutally, I did not notice...
<adeuring> looking now
<rick_h_> adeuring: cool, thanks. Sorry for the dupe :)
<dpm> hi launchpadders, could someone help me with a generic question on LP cronscripts? I'm trying to find out what it would take to move a script that currently talks to the DB to export translation data into the LP source tree. I'm told by jtv that the best way would be to turn the script's main code into a class derived from LaunchpadCronScript, but that I should double-check with other LP developers.
<dpm> For reference, the script is here: https://launchpad.net/lp-get-ul10nstats/trunk
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: â
<mpt> Are there any prizes given for marking really old bugs as duplicates?
<czajkowski> mpt: I'll give you cake
<czajkowski> how's that
<mpt> deal :-)
<czajkowski> win win :)
<czajkowski> mpt: but you're not allowed to create new ones :)
<mpt> What would be the point of that? If they're new ones they're not really old ones
<mpt> oh, you mean you're limiting me to marking duplicates, not reporting bugs at all
<dpm> > hi launchpadders, could someone help me with a generic question on LP cronscripts? I'm trying to find out what it would take to move a script that currently talks to the DB to export translation data into the LP source tree. I'm told by jtv that the best way would be to turn the script's main code into a class derived from LaunchpadCronScript, but that I should double-check with other LP developers.
<dpm>  For reference, the script is here: https://launchpad.net/lp-get-ul10nstats/trunk
<cjohnston> When trying to use lp-setup, I've tried many times now to run install-lxc.. I continue to get a Permission denied (publickey) error (http://paste.ubuntu.com/1202807/) each time I try.. any idea what is causing this?
<cjohnston> I've double checked my key and its correct
<cjohnston> hmm.. hmm.. appears as tho as somepoint I was logged out from bzr launchpad-login.. stand down all :-)
<cjohnston> ok.. more hmm.. any chance that lpsetup changes my launchpad-login?
<czajkowski> cjohnston: breaking things :)
<cjohnston> probably
<cjohnston> just not sure how :-(
<czajkowski> sinzui: might you give cjohnston some pointers please
<sinzui> I have't use it
<cjohnston> Not quite sure why I'm importing the Yellow Squad key: http://paste.ubuntu.com/1202922/
<deryck> jcsackett, hi.  I have a branch for review, but....
<deryck> jcsackett, I'm about to lunch.  So don't mind waiting until I'm back if you need to chat interactively.
<jcsackett> deryck: i don't mind looking now, i can put any questions in a comment.
<deryck> jcsackett, awesome, thanks!  https://code.launchpad.net/~deryck/launchpad/use-specification-sharing-policy/+merge/124240
<deryck> catch you after lunch then.
<cjohnston> sinzui: I found the issue through the code.. you can pass --lpuser, but if you don't it defaults to your system user name.. :-/  /me wonders how many people have their system user name and lp name the same and if that should be the default or a required flag
<sinzui> cjohnston, I agree. Lp chooses the user name from the first email address imported or sent from ubuntu SSO. There is little chance the system user will match the lp identity
<sinzui> cjohnston, this may relate to benji's bug Bug #1034605
<_mup_> Bug #1034605: handle_lpuser_as_username does validate the given LP username <lpsetup:Triaged> < https://launchpad.net/bugs/1034605 >
<cjohnston> I don't think so.. from what I can tell, you can provide whatever username you want, and that bug is referencing checking to see if you mistyped your username.. whereas the issue I'm having that because I didn't know that I needed to provide a username, its picking what is a valid lp username, so his test would pass.. it just wouldn't be me
<cjohnston> I guess related, possibly, the same, I don't think so
<cjohnston> sinzui: should I talk to frankban_ about it, or someone else?
<dpm> trying again...
<dpm> hi launchpadders, could someone help me with a generic question on LP cronscripts? I'm trying to find out what it would take to move a script that currently talks to the DB to export translation data into the LP source tree. I'm told by jtv that the best way would be to turn the script's main code into a class derived from LaunchpadCronScript, but that I should double-check with other LP developers.
<dpm>   For reference, the script is here: https://launchpad.net/lp-get-ul10nstats/trunk
<czajkowski> dpm: poked elsewhere as well
<dpm> thanks czajkowski
<mgz> fallback would be an email to list, which would add the australians as an audience
<jcsackett> dpm: assuming mgz's comment was at you, i agree with him. hit the launchpad-dev mailing list, and you'll get more attention.
<jcsackett> dpm: or rather, you'll get attention with more direct knowledge on what you're doing. :-P
<jcsackett> i would say though that we've done a lot of moving things *out* of the LP tree--what's the motivation to move this in?
<dpm> jcsackett, I've been told that such scripts cannot live outside the LP tree, so it was disabled
<jcsackett> dpm: that would be a good reason. :-P
<cjwatson> jcsackett: direct db access
<abentley> deryck[lunch]: Good morning.
<jcsackett> so, dpm, i think step one is def to derive from LaunchpadCronScript.
<jcsackett> i would still email the list though, so people who have done similar work can inform you of any potential gotchas.
<dpm> jcsackett, yeah, that was what jtv recommended (LaunchpadCronScript), but he suggested to check with other devs whether that was still the recommended way to run such jobs (or if using celery tasks or any other technology is the preferred way in LP nowadays)
<jcsackett> dpm: well, celery jobs and other job stuff is in my experience best meant for things that need to be jobs, but respond to real time events in the app.
<jcsackett> like merging accounts, for example.
<jcsackett> if this is just a nightly script or similar, i think cron is still the place for it.
<jcsackett> others may of course have different opinions.
<jcsackett> but i don't think we've abandoned cron as a way of doing things.
<abentley> jcsackett: We do have Celery running one periodic task, but we haven't really migrated to Celery for everything yet.
<abentley> jcsackett: So I don't know whether we'll prefer to use Celery for that in the future.
<dpm> jcsackett, it's currently a daily job, and it simply exports data automatically (i.e. it does not need to respond to anything), so I guess cron should be fine
<jcsackett> dpm: i would think so.
<rick_h__> jcsackett: ping
<jcsackett> pong.
<rick_h__> jcsackett: hey, do you know the details of the bug/stuff going aroud on the linking of the nav stuff in the table listing?
<rick_h__> I'm hitting a failing test out of no where with an error that linkify_navigation isn't defined and it's kind of confusing. I thought I saw some of your squad doing something with that in MP that flew by
<jcsackett> rick_h__: wallyworld__ addressed a broken link error i think, but i don't recall the details.
<jcsackett> i assume you're rebuilt js?
<rick_h__> jcsackett: yea, ah there's the branch.
<rick_h__> jcsackett: ok thanks
<jcsackett> rick_h__: yw.
<czajkowski> mpt: with those duplicates you get a cookie not a cake
<deryck> abentley, hi there.
<abentley> deryck: I see that abel was working on fixing my test failure.  I don't see it in devel, but I'm assuming it's in ec2.
<deryck> abentley, yeah, it's in ec2.  Should be in shortly, assuming it passes.
<abentley> deryck: I'm just digging into the privacy banner test failure, which appears to be from the same root cause.
<sinzui> jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/mailman-email-addresses/+merge/124273
<jcsackett> sinzui: sure. :-)
<jcsackett> sinzui: r=me. i love branches that delete doctests. :-)
<sinzui> thank you jcsackett
<rick_h__> jcsackett: branch your way if you have time: https://code.launchpad.net/~rharding/launchpad/lp_cache_update/+merge/124217
<rick_h__> diff still updating though
<jcsackett> rick_h__: one question: this dump to json utility only works for information types, and all the other info_type related stuff that's not tied to an artifact lives in registry. shouldn't it be there instead of app?
<rick_h__> jcsackett: so if you know of a place in there to put it I'm happy to move it.
<rick_h__> this is replacing the manual JSON stuff that was in each browser/xxxx.py
<rick_h__> so it was in browser/product browser/bugtarget and going into browser/specification
<rick_h__> since it was used in those places I defaulted to app/utilities
<jcsackett> rick_h__: hm.
<rick_h__> if no one minds it sitting in registry/enums I'm ok with it, or prefer a registry/utilities
<rick_h__> it kind of didn't seem to fit so open to anything
<jcsackett> i have no problem with enums, as its just a function to work with one of said enums.
<jcsackett> but registry utilities would be fine as well.
<jcsackett> i just don't want a developer months from now to waste time figuring out if information type lives in app or registry.
<rick_h__> jcsackett: right, understand. I've got no dog in the fight at all.
<jcsackett> so, i'll mark this approved with the understanding you'll move that to either reg/enum or reg/util?
<rick_h__> sounds like a plan to me, thanks
<jcsackett> r=me then. :-)
<jcsackett> other than that, nice cleanup. :-)
<sinzui> lifeless, do you have time to talk about escalated bugs
<czajkowski> sinzui: you playing whack a bug today on bugs, you're so making progress on them :D
<sinzui> czajkowski, yes, but it wont last. A lot are trivial or dupes. I expect the pace to slow in about 4 weeks
<czajkowski> that's still a lot of weeks and a lot of bugs sinzui
<lifeless> sinzui: I do. G+? Skype? IRC?
<sinzui> czajkowski, I will cheer when the bug critical bug count  drops to 254, 1 less than Purple left it in July 2011
<sinzui> lifeless, I want to try g+
<czajkowski> :/
<lifeless> sinzui: you or your canonical version should have an invite now
<wallyworld__> sinzui: do you have time for a quick pre-impl before the standup?
<sinzui> jcsackett, https://bugs.launchpad.net/bugs/bugtrackers/alsa-mantis lists fadly and it is only visible to ~registry
<sinzui> bug 210821 is still not fixes
<_mup_> Bug #210821: bug tracker list shows inactive projects <404> <bugwatch> <lp-bugs> <qa-ok> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/210821 >
<sinzui> fixed
<lifeless> czajkowski: why the sadface?
<czajkowski> lifeless: the number in stats
<czajkowski> lifeless: a year later and the number is down by 1
<lifeless> not yet
<lifeless> sinzui is looking forward to that point
<lifeless> czajkowski: http://webnumbr.com/launchpad-critical-bugs
<czajkowski> lifeless: aye he;s playing whack a mole on the bugs this week, it's interesting to follow.
<StevenK> czajkowski: Not just sinzui.
<StevenK> All five of us have been pouring over the critical bug list this week.
<lifeless> jam: around ?
 * lifeless doubts it
<czajkowski> StevenK: yes you have I just picked one and refere to the whole team, but yes it's a group thing
<czajkowski> lifeless: I think he's utc +5 no?
<jelmer> yeah, he should be asleep
<lifeless> should be :)
<StevenK> +4, I think
<lifeless> jelmer: have we published any bzr plugins to pypi ...
<jelmer> lifeless: I think so. I know I have.
<lifeless> I want to nuke sourcecode
<lifeless> means letting bzr import plugins from eggs
<lifeless> ahha
<lifeless> http://pypi.python.org/pypi?%3Aaction=search&term=bzr&submit=search
<StevenK> And mailman
<lifeless> jelmer: ^
<lifeless> StevenK: one crime against nature at a time
<jelmer> lifeless: I like the idea.
<jelmer> lifeless: so, bzr-git and bzr-svn are on pypi. we were going to get rid of bzr-hg. bzr-loom is missing, as is loggerhead.
<lifeless> and bzr-pqm and others
<jelmer> bzr-pqm is in sourcecode?
<lifeless> I was pinging jam to ask his view on pushing them up, and to get acls as we'll need to fix the dependencies, switch them to use distribute ('setuptools')
<lifeless> jelmer: its used for ec2test, so we need it somewhereish
<jelmer> lifeless: don't we have it in the Launchpad PPA? Or do you really want it as egg?
 * lifeless shrugs
<lifeless> jelmer: http://rbtcollins.wordpress.com/2012/08/27/why-platform-specific-package-systems-exist-and-wont-go-away/
<lifeless> jelmer: kindof like to setup lowest cost maintenance for the things we're maintaining
<jelmer> lifeless: I've read it earlier :-)
<jelmer> lifeless: I think that's the right approach, but bzr-pqm is also immutable enough that I wonder if it's worth worrying about at this point.
<lifeless> sure, I'm not pushing hard for it, but it was in the working set that popped into mind
<jelmer> Fair enough.
<lifeless> huh
<lifeless> looks like 2.6 does it already
<lifeless> at least under pip
<lifeless> ah, right, the virtualenv shoves it in the default search path
<lifeless> so that provides an alternative
<lifeless> gary_poster|away: if you come back, what would you say to a move to virtualenv+pip ?
<mwhudson> please no
<lifeless> mwhudson: details
<mwhudson> lifeless: well, one thing is "move to pip" from what?  buildout?
<lifeless> yes
<mwhudson> i talked about this a little in my canonical-tech thread a bit
<mwhudson> pip is not very good software
<lifeless> imagine I have no idea what you're talking about
<lifeless> desperately want to use pip
<mwhudson> sorry
<lifeless> and you have to dissuade me
<lifeless> This is a lie, but will get you to tell me what you need to :)
<mwhudson> lifeless: was trying to find a citation
<mwhudson> lifeless: basically if you depend on packages a and b and a and b depend on _different_ versions of package c
<mwhudson> lifeless: pip's response is "pick one"
<mwhudson> lifeless: pip has no equivalent of buildout's allow-picked-versions = false
<sinzui> wallyworld__, is this the bug?
<sinzui> Bug #867529
<_mup_> Bug #867529: Dynamic loading comments load on top of page content <firefox> <javascript> <qa-untestable> <regression> <story-batched-comment-loading> <Launchpad itself:Triaged> < https://launchpad.net/bugs/867529 >
<lifeless> ok, assuming you're using the implicit dependency graph vs flatten and resolving to a single requirements doc like LP does with buildout (and can be done with pip too)
<wallyworld__> sinzui: yes
<mwhudson> lifeless: and this is more anecdata than something i can be specific about is that upgrades are very difficult
<mwhudson> lifeless: so you want to make a new venv for each revno you deploy
<lifeless> mwhudson: isn't that what -I -r versions.txt is for ?
<mwhudson> lifeless: but there is no equivalent of the egg cache in the pip world, so making a new venv will take as long as a "make build" in a completely new launchad tree
<lifeless> right, because it doesn't use eggs. Some folk would call that an advantage ;)
<mwhudson> lifeless: i don't think there is any way to make pip check that versions.txt is complete
<lifeless> mwhudson: how would you solve this for pip ?
<mwhudson> lifeless: i wouldn't use it
<lifeless> Ok.
<mwhudson> oh you mean, if i was to try to fix pip?
<mwhudson> i don't know
<lifeless> How would you solve this in a world where buildout is going away, upstream are still discussing the colour of the bikeshed, and you are relentlessly pragmatic :)
<mwhudson> lifeless: is buildout going away?
<mwhudson> i don't know
<mwhudson> embrace juju aggressively and move to debs (maybe using pypi-install liberally)?
<lifeless> mwhudson: buildout: kindof, upstream have indicated that our LP specific tweaks are being removed in the next release.
<lifeless> unless we port them, gary_poster|away has the details.
<mwhudson> ah
<mwhudson> mmf
<mwhudson> this is the relative-path stuff i guess
<lifeless> So we have a choice of investing in buildout, investing in pip, or investing in the heat death of the universe.
<mwhudson> i guess we don't actually get a choice about the latter
<mwhudson> lifeless: well, i don't want to impose my views on you over strongly
<mwhudson> but we had a pip based deployment approached and it sucked
<mwhudson> and we now have a buildout one and it works well
<mwhudson> the underlying tech is surely not all of that
<mwhudson> but it is probably part of it
<lifeless> mwhudson: totally unscientific: http://www.googlefight.com/index.php?lang=en_GB&word1=pip&word2=buildout
<lifeless> It may mean, for instance, that pip is hard to use so folk write about it more.
<sinzui> wallyworld__, bug 402915 is what you may have a fix for
<_mup_> Bug #402915: Can no longer move a branch to another project using the UI <lp-code> <package-branches> <regression> <target-picker> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/402915 >
<mwhudson> lifeless: oh, pip is very popular no doubt
<mwhudson> lifeless: you can also play the game with python/php or jquery/yui
<lifeless> indeed.
<StevenK> wgrant: ComponentLookupError: ((<Person at 0xfda1810 person-name-100003 (Person-name-100003)>, <lp.services.webapp.servers.LaunchpadTestRequest instance URL=http://127.0.0.1>), <InterfaceClass zope.interface.Interface>, '+archivesubscriptions/16')
<StevenK> wgrant: I thought I'd constructed it to what the ZCML said, but obviously not.
<wgrant> That's no view name
<wgrant> That's a path
<wgrant> You'll need to create a request with ['16'] as stepstogo
<elmo> That's no moon!
<StevenK> Hmm, still no failure
<StevenK> wgrant: I need to grep through appserver logs?
<wgrant> StevenK: No, you can see the request URL in the OOPS
<bigjools> guten tag
<StevenK> wgrant: Right, and that gives me the archive id and the user.
<wgrant> StevenK: Right
<wgrant> StevenK: So what would appserver logs give you?
<wallyworld__> bigjools: your shift key broken?
<bigjools> wallyworld__: I don't believe in german capitalisation
<StevenK> wgrant: Handily, neither the archive or person exist on DF
<wallyworld__> bigjools: you don't have a choice if you want to write properly :-P
<bigjools> wallyworld__: I always have a choice
<wallyworld__> same goes for writing English with poor grammar/spelling I guess
<bigjools> wallyworld__: there's choice vs ignorance
<bigjools> I am mainly ignorant when it comes to German :)
<wgrant> StevenK: That's why we have ops :)
<wallyworld__> bigjools: i was just trolling
<bigjools> wallyworld__: I know, but I was chomping on the bait like a hooked fish
<wallyworld__> yep, i reeled in a big one alright
<bigjools> that begs an obvious response
<wallyworld__> i throw out the bait....
#launchpad-dev 2012-09-14
<cjohnston> Anyone know where the the text that was in http://goo.gl/O4CQ4 went to?
<cjohnston> (as in, where I might find it now)
<StevenK> wgrant: Hmmm, my pdb isn't firing.
<wgrant> StevenK: Where is it and why isn't it firing?
<StevenK> wgrant: http://pastebin.ubuntu.com/1203852/
<wgrant> StevenK: Oh, right
<wgrant> StevenK: That view isn't /+archivesubscriptions/$ID
<wgrant> It's ArchiveSubscription:+index or so
<StevenK> team_sub, '+index' doesn't fire it either
<StevenK> Oh, bah, ArchiveSubscription:+index is the edit view
<StevenK> wgrant: I think this is a race.
<StevenK> wgrant: They all back onto IArchiveAuthTokenSet.getActiveTokenForArchiveAndPerson()
<StevenK> So when the guard checks, self.active_token is an EmptyResultSet, then IArchive.newAuthToken() is called, and when it calls the same method, it returns a token and raises
<StevenK> s/an EmptyResultSet/None/
<wgrant> StevenK: But it's not a unique violation from the DB, is it?
<StevenK> wgrant: No, it's newAuthToken raising it
<wgrant> StevenK: Each appserver transaction sees an isolated snapshot of the DB
<wgrant> StevenK: If newAuthToken can see it in the DB (and not by a unique constraint violation error), then it's not a race
<wgrant> The appservers are not READ COMMITTED
<StevenK> wgrant: Both the view's self.active_token and IArchive.getAuthToken() back onto the same method
<StevenK> So I don't get how it could return None and then a token
<wallyworld__> wgrant: StevenK: sinzui: are we supporting embargoed specs for private projects?
<wgrant> wallyworld__: IMO no, but it seems like we are
<wallyworld__> i'm reviewing a branch which has them
<wgrant> There's already code landed that permits them
<wgrant> So don't block this on that
<wallyworld__> ok
<StevenK> wgrant: Can you think of a scenario? Because I can't, but it happens. :-(
<rick_h__> I think it's coming out of the need for Private Projects to have public, embargoed, proprietary options
<rick_h__> all of the apps will get default apps that match the project ootb I believe
<StevenK> Why? Embargoed is special
<StevenK> Public and Proprietary sure, sounds good
<wgrant> The debate over whether Embargoed makes sense for a project is still raging
<rick_h__> because it's strange to have a new project setup with one option and see something you didn't select somewhere else
<rick_h__> right
<rick_h__> agreed, so easier to take it into account and remove than to fit back in
<wgrant> If Embargoed makes sense for a project, then it makes sense for blueprints
<rick_h__> but if you stick to the idea that embargoed imples one day being public while proprietary does not, then it seems there's a niche for it
<StevenK> Other way around
<lifeless> StevenK: huh ?
<wgrant> StevenK: No, rick_h correctly portrays the other side of the argument
<wgrant> That Embargoed should also be used for Proprietary stuff that may become public
<wgrant> I tend to disagree
<wgrant> But there are reasonable arguments both ways
<StevenK> wgrant: So I can't think of how to write a test for this issue, and you rejected my current fix.
<wgrant> StevenK: can you link me back to an oops?
<StevenK> wgrant: https://oops.canonical.com/oops/?oopsid=OOPS-3607c5f44e382d14bd9ff678dbc51097
<wgrant> Suppressing an exception when we don't know why it happens is a good reason to reject a fix :)
<wgrant> StevenK: Do you have a recent one?
<wgrant> Or maybe that URL is just bad
<StevenK> That was the most recent one I could find, I can re-grep on neem if you wish
<wgrant> Because it's not there any more
<StevenK> Oh, sigh
<StevenK> I even mentioned it in a bug
<StevenK> Yeah, 2012-09-10 has been deleted
<lifeless> it shouldn't have pruned then !
<StevenK> Well, it did
<lifeless> ugh
<StevenK> I bloody hope it has turned up again, because that was the only OOPS we had
<wgrant> Wait
<wgrant> -10?
<wgrant> That's not yet 5 days old
<wgrant> Is the pruning threshold like 2 days now or something>?
<StevenK> drwxr-xr-x 2 oops_tools oops_tools 2.1M Sep 11 10:28 2012-09-06
<StevenK> drwxr-xr-x 2 oops_tools oops_tools 704K Sep 14 02:07 2012-09-11
<lifeless> didn't we lower it when we had that huge batched?
<lifeless> may not have been re-raised
<StevenK> In either case, I linked it to a bug and it pruned it
<wgrant> StevenK: You may have linked it while the prune was happening
<wgrant> It can take a while
<StevenK> It worked this morning while I was on the call
<wgrant> Mysterious
<wgrant> prune.log is modified at 11:41 today
<wgrant> The last line in the log:
<wgrant> INFO:root:Querying OOPS references on auditorclient from 2012-09-10 01:40:01.689000+00:00 to 2012-09-11 01:40:02.192021+00:00
<wgrant> I wonder if it's pruning to -3 days
<wgrant> And it also only looks at references up to -3 days
<wgrant> Hee hee
<wgrant> Yes
<wgrant> That explains a bit
 * StevenK can't convince his browser to show him the cached copy either
<wgrant>     prune_until = datetime.datetime.now(utc) - one_week
<wgrant>     references = finder.find_oops_references(
<wgrant>         prune_from, prune_until, options.project, options.projectgroup)
<wgrant> (I suspect one_week may be cowboyed)
<lifeless> right, it only needs to look at references made after the oops was created
<StevenK> Right, it pruned the only ArchiveSubscriptionError OOPS we had.
<StevenK> Which is just awesome.
<wgrant> lifeless: Right, but it also only looks at references made until the pruning threshold
<wgrant> lifeless: The start bound is correct
<wgrant> The end bound shouldn't exist
<lifeless> wgrant: oh, should be now() ?
<wgrant> Right
<lifeless> wgrant: probably a bad cowboy
<lifeless> hloeung: yo
<wgrant> lifeless: No, it's in trunk
<wgrant> My paste was from trunk
<lifeless> hloeung: unping
<lifeless> wgrant: dow; can has fix ?
<wgrant> Confirmed that neem has one_week cowboyed to be days=3
<wgrant> So this explains it
<StevenK> I wonder how many other OOPSes have been pruned that were referenced
<wgrant> Not too many
<wgrant> Anything referenced within 24h should be safe
<StevenK> So I'm guessing I should find something else to work on
<StevenK> Since we don't understand what caused it, and the only OOPS we had has been deleted.
<jtv> Hi StevenK, wgrant â can I bounce a problem & solution for format-imports off you?
<StevenK> format-imports doesn't have problems, only bad input
<wgrant> Sure
<jtv> StevenK: then try "from . import foo"  :)
<jtv> Blows up.
<StevenK> Use imp for that
<jtv> imp?
<StevenK> http://docs.python.org/library/imp.html#module-imp
<wgrant> Why use imp for that?
<jtv> What William said.
<StevenK> jtv: Django uses imp in it's manage.py
<jtv> Not a reason in itself.  It also doesn't believe in database transactions.  :)
<lifeless> well
<lifeless> so . is definitely local
<lifeless> format-imports just needs to be taught
<jtv> That's what I just did.
<jtv> In a nutshell: the first level of an import is either an identifier, or a series of dots.
<jtv> (Previously: just an identifier)
<jtv> And I list "." as one of the known local packages.
<jtv> Still leaves ".." wide open, but watch me not care.
<jtv> Any objections to that solution?  I'm implementing it in MAAS, but since the script was copied from LP, I'd like to share the fix.
<lifeless> seems fine, I'd like that.
<bigjools> StevenK: "its"
<lifeless> Would MAAS use the script if it was in lp-dev-utils ?
<lifeless> or would it still want a local copy ?
<jtv> Thank you bigjools :)
<bigjools> jtv: I knew you'd be itching after reading it too
<jtv> I was so proud of suppressing it.
<bigjools> haha
<jtv> lifeless: lp-dev-utils sounds like a better place.
<lifeless> jtv: is there any delta between the scripts ?
<jtv> But maybe that's a separate issue; this is really just a distraction for me so I'm very much looking for a quick fix.
 * jtv checks
<lifeless> jtv: if so, could it be made into data - a config file or something ?
<lifeless> jtv: yes, it is a separate issue.
<jtv> There are several isolated changes.  :(
<jtv> Changes the MAAS version makes compared to the LP version: http://paste.ubuntu.com/1203983/
<jtv> One of those is a typo fix in LP; the rest looks like modernizations in the maas version.
<mwhudson> hm, where has Y.lazr.activator.Activator gone
<lifeless> lazr is gone gone gone
<mwhudson> yeah i saw that commit
<mwhudson> it broke my greasemonkey work item editor :)
<mwhudson> lp.ui.activator
<StevenK> wgrant: Re: bug 966205, do you think it's sensible to delete all logintoken's on merge?
<wgrant> Bug #966205
<wgrant> Oh right
<wgrant> That one
<StevenK> Private
<wgrant> I'd just delete them all, yeah
<lifeless> purple, I have to run, C stuff; perhaps one of you could help #newbie in #bzr
 * StevenK tries to find the person merge tests
<StevenK> wgrant: https://pastebin.canonical.com/74532/
<wgrant> StevenK: There's no action there
<StevenK> I thought it POST'd to the same URL
<wgrant> Right
<wgrant> But I see no POST to the same URL
<StevenK> Odd, since there was POST, the OOPS had it.
<wgrant> What did you grep for?
<StevenK> ~thomas-guest/+archivesubscriptions/39008/+index
<wgrant> Why +index?
<wgrant> That wouldn't normally be there
<StevenK> It obviously was, there was three matches
<wgrant> None of which we were looking for
<wgrant> So I'd drop the trailing /+index
<StevenK> wallyworld__, wgrant: https://code.launchpad.net/~stevenk/launchpad/remove-logintokens-on-merge/+merge/124338
 * wallyworld__ sad, conflict ahead
<StevenK> wallyworld__: Oh?
<wallyworld__> person-merge.txt - it's being nuked for new unit tests
<wallyworld__> for a bug i'm ficing
<lifeless> yay django
<lifeless>   File "/home/robertc/source/launchpad/oops-tools/working/eggs/Django-1.4-py2.6.egg/django/contrib/auth/management/__init__.py", line 85, in get_system_username
<lifeless>     return getpass.getuser().decode(locale.getdefaultlocale()[1])
<lifeless> TypeError: decode() argument 1 must be string, not None
<StevenK> wallyworld__: That's awesome, and sorry
<wallyworld__> StevenK: no problem :-)
<wallyworld__> no need to apologies
<wallyworld__> bah, can't type
<StevenK> wallyworld__: You aren't fixing the linked bug?
<wallyworld__> no, i checked :-)
<wallyworld__> i'm fixing 1019975
<wallyworld__> bug 1019975
<_mup_> Bug #1019975: AttributeError: 'NoneType' object has no attribute 'displayname'  requesting people merge <merge-deactivate> <oops> <regression> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1019975 >
<wallyworld__> fix is easy, but adding a test is a pain until i convert the doctest
<StevenK> wgrant: 3 out of 4 appserver logs grepped, waiting for gac
<lifeless> grah
<lifeless> wgrant: in your lxc travels, did you see lxc choosing POSIX as the locale, no matter what /etc/environment and /etc/default/locale say ?
<wgrant> StevenK: Great
<wgrant> lifeless: I don't believe so
<wgrant> By POSIX you mean C?
<lifeless> no
<lifeless> robertc@lucid-test-lp:~$ locale
<lifeless> LANG=
<lifeless> LANGUAGE=
<lifeless> LC_CTYPE="POSIX"
<lifeless> ...
<lifeless> LC_IDENTIFICATION="POSIX"
<lifeless> LC_ALL=
<wgrant> Well that's odd
<lifeless> I have en_AU etc available
<lifeless> language pack en base installed
<wgrant> I could understand C
<wgrant> But POSIX is a bit odd
<StevenK> lifeless: Not en_NZ? :-)
<lifeless> of course, my environment outside of LXC is latin and klingon
<lifeless> StevenK: habits.
<StevenK> lifeless: tlh_LA ? :-)
<lifeless> robertc@lifeless-64:~$ locale
<lifeless> LANG=en_AU.UTF-8
<lifeless> LANGUAGE=la_AU:tlh_GB:tlh:en
<wgrant> Oh
<lifeless> $ LANGUAGE="en_US:en" LANG=en_US.UTF-8 ssh -A -X lucid-test-lp.local
<lifeless> Last login: Fri Sep 14 06:21:22 2012 from lifeless-64.local
<lifeless> robertc@lucid-test-lp:~$ locale
<lifeless> LANG=
<lifeless> LANGUAGE=
<lifeless> LC_CTYPE="POSIX"
<wgrant> lifeless: You haven't changed sshd's AcceptEnv default or something?
<lifeless> hah
<lifeless> AcceptEnv LANG LC_*
<lifeless> *I* did not set that.
<wgrant> That's normal
<wgrant> The default
<StevenK> ... Why is my LANG en_US.UTF-8 ? :-(
<wgrant> StevenK: Because you're terrible
<wgrant> Mine is correct :)
<StevenK> I'm trying to remember where it is set.
<wgrant> /etc/default/locale, usually
<StevenK> steven@undermined:~% cat /etc/default/locale
<StevenK> LANG="en_AU.UTF-8"
<StevenK> steven@undermined:~% echo $LANG
<StevenK> en_US.UTF-8
<lifeless> thats the same kind of FUNK I'm seeing
<lifeless> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=330500
<lifeless> is looking similar
<wgrant> Yeah, but that one's from 2005
<wgrant> If it's the one I just looked at
<wgrant> And SSH should be overriding that
<lifeless> ssh was present in that bug
<lifeless> and message 65 says it still happens
<wgrant> Hm, true
<StevenK> wgrant: https://pastebin.canonical.com/74533/
<wgrant> StevenK: Right, that makes a bit more sense
<wgrant> Though the +index bits are suspiciously missing now
<lifeless> oh charming
<lifeless> for some reason I've ended up with django 1.4 in my oops-tools setup
<lifeless> rather than 1.3 which I think it was running
<StevenK> wgrant: So it doesn't look like a race
<lifeless> ImproperlyConfigured at /oops/
<lifeless> Error importing template source loader django.template.loaders.filesystem.load_template_source: "'module' object has no attribute 'load_template_source'"
<wgrant> StevenK: Right, as expected
<wgrant> I knew it couldn't be, but evidence is good :)
<StevenK> wgrant: So then I should be able to trigger it with a test case. :-(
<wgrant> StevenK: Yes, if it's correctly set up
<StevenK> wgrant: But I still don't get it -- both methods back onto IArchiveAuthTokenSet.getActiveTokenForArchiveAndPerson()
<StevenK> Hmmm
<StevenK> I wonder if there is a value that date_deactivated can be set to that isn't None
<czajkowski> morning
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: â
<dpm> hi adeuring, would you have time to review https://code.launchpad.net/~dpm/launchpad/translations-exporter/+merge/124373 ? Thanks!
<adeuring> dpm: sure, I'll look
<dpm> excellent, thanks!
<StevenK> dpm: You can validate the distroseries by using getUtility(IDistributionSet).getByNameOrVersion()
<StevenK> I think.
<StevenK> I'm a little fuzzy on the details
<dpm> StevenK, ah, great, will look into that, thanks for the feedback!
<StevenK> Sorry, getUtility(IDistroSeriesSet).findByName()
<StevenK> The first one is for finding distributions
<dpm> ok
<mpt> hrm
<mpt> I know there's a bug report about "Can't search for the absence of a bug tag", but I can't find it
<mpt> It's not in <https://bugs.launchpad.net/launchpad/+bugs?field.tag=bugtag>
<mpt> oh, it's bug 81575, Fix Released
<_mup_> Bug #81575: no way to search for absence of a tag <bugtag> <lp-bugs> <oem-services> <ubuntu-qa> <Launchpad itself:Fix Released by allenap> < https://launchpad.net/bugs/81575 >
 * mpt scours the search form again
<mpt> aha, "To search for the absence of a particular bug tag, place a minus sign before its name: e.g. -test"
<adeuring> dpm: comment sent. Ping me if you have any questions
<adeuring> dpm: LP "censored" much of the replay. Use the "download full text" link
<dpm> thanks adeuring. I can't see any more text in the downloaded text file than the textbox in LP, though.
<adeuring> dpm: weird... But the mail you'll get sould be complete ;)
<dpm> adeuring, hm, the mail has exactly the same text as the textbox and the downloaded link, nothing additional
<adeuring> dpm: maybe I am too stupid to find all of my own comments in the web browser. WHat I don't see in the web browser is your Python code with some comments from me.
<dpm> adeuring, ah, wait, I was missing the inline comments in the diff, sorry, all good, now. Sorry...
<adeuring> np ;)
<dpm> adeuring, thanks a lot for the detailed review. Who should I best talk to to go past the "thou shalt not increase the LOC count for.." step?
<adeuring> dpm:  get some support first (the admins would probably appreciate a cron job, i assume), then talk with flacoste
<czajkowski> dpm: do you know who the team leads are?
<czajkowski> dpm: bigjools, sinzui gary_poster|away jam gmb
<cjwatson> czajkowski,dpm: https://dev.launchpad.net/PolicyAndProcess/MaintenanceCosts says "needs a waiver from the LP Project lead or CDO Technical architect "
<cjwatson> i.e. Francis or Robert
<czajkowski> cjwatson: nods
<lifeless> I'll grant a waiver
<lifeless> this code already exists in the wrong place
<lifeless> its not adding debt to move it into LP
<lifeless> dpm: ^
<lifeless> its reducing debt by getting it into the right place.
<czajkowski> :)
<dpm> thanks lifeless
<rick_h__> dammit, buildbot is starting to get on my last nerves.
<rick_h__> ok, so how did rev 15954 land when it's got a broken test in it I hit in both of my ec2 test runs? /me is so confused
<rick_h__> sinzui: it looks like the doctest cleanup in https://code.launchpad.net/~sinzui/launchpad/mailman-email-addresses/+merge/124273 breaks the doctest because it removes list_four but there's another test using it. Shold that bit just be reinstated ok?
<sinzui> damn
<sinzui> yes
<sinzui> I was going to rewrite that as unittests to day too
<sinzui> oh well
<rick_h__> how could the change have landed with a broken doctest in there though?
<rick_h__> sinzui: ok, added it back in and test passes, going to submit it as a testfix
<sinzui> You are a life saver
<cjohnston> It looks like LP has changed where it stores text for some pages now, any idea where the text for goo.gl/O4CQ4 might be located now
<cjohnston> ?
<czajkowski> cjohnston: rather than starting mid sentence, perhaps explainign what you are doing and what is wrong
<cjwatson> cjohnston: lib/lp/blueprints/interfaces/specificationsubscription.py
<cjohnston> Trying to change the text, it's not there.. Based on the conversation I had on the mailing list in the past
<cjwatson> (I grepped for 'Participation essential' and it was the first of a small number of hits)
<cjohnston> thanks cjwatson.. I guess grepping half the sentence was what got me. :-/
<rick_h__> sinzui: sorry, but I evidently fail. I've tried to submit this with pqm-submit and lp-land and not getting any email/pqm action I can see
<rick_h__> sinzui: can you see if you can push https://code.launchpad.net/~rharding/launchpad/testfix_message_holds/+merge/124405 through lp-land as testfix please?
<sinzui> okay
 * rick_h__ is about to start tossing things out windows gah!
<sinzui> rick_h_, I think I have a note about the obscure bzr pqm syntax to submit other peoples branches.
<rick_h__> I had thought that  bzr lp-land --testfix --no-qa https://code.launchpad.net/~rharding/launchpad/testfix_message_holds/+merge/124405
<rick_h__> would do the thing
<rick_h__> but no success or fail email so I'm not even sure I guess that it's getting there
<sinzui> rick_h_ try this
<sinzui> bzr pqm-submit -m "[testfix][r=sinzui] Revert doctest."
 * sinzui is watching https://pqm.launchpad.net/ for old-time sake
<sinzui> pqm saw it
<sinzui> That it. Today I am taking a torch and pitchfork and raising an angry mob on mailing list doc tests
<rick_h__> yay the pqm gods have accepted my offering
<nigelb> What sacrifice did you offer? A whale? ;)
<sinzui> :)
<rick_h__> 10 pqm-submits and a "@$#@$@ you"
<nigelb> lol
<abentley> adeuring: Here is an example where the web UI cannot protect you from entering disallowed values: https://blueprints.qastaging.launchpad.net/specs/+new
<adeuring> abentley: do you mean that "embargoed" might be undesired/invalid? Or that somebody crafts the POST request manually to send USERDATA or whatever else?
<abentley> adeuring: I mean that "embargoed" may be undesired/invalid.
<adeuring> abentley: anyway, transititonToInformationType() should check that the new value is valid
<adeuring> abentley: ok, if EMBAGOED is undesired, we should not present it to the user
<abentley> adeuring: We don't know that it's undesired at the time we present it to the user.  Look at the form.  One of the inputs is "target", which is the project or distro.
<abentley> adeuring: Where we know it's undesired, we hide it, of course.  This is an example where we cannot do that.
<abentley> adeuring: (target is listed as "For: The project for which this proposal is being made."
<adeuring> abentley: ah, sure. But we can (1) add a a warning to "Only shared with users permitted to see embargoed information. " that this might not work, and (b) if the value is selected but not usable, we should show an error.
<abentley> adeuring: My point is that we are currently showing an error-- an OOPS.
<adeuring> abentley: sure, that's bad. But getting undesired data into the DB is bad too
<abentley> adeuring: Right, but an assertion due to a missing AccessPolicy is the wrong way to defend against that.
<adeuring> abentley: the other artifacts raise a dedicated exception
<adeuring> we can do that too
<adeuring> for spec
<abentley> adeuring: When deryk's code lands, we'll have that.  Until then, I don't know if I can QA my code.
<adeuring> abentley: so what should we do?
<abentley> adeuring: There's no way to add an access policy on qastaging, right?
<adeuring> I don't know
<adeuring> My guess is: create a product which uses EMBAGOED for bugs too
<abentley> adeuring: Oh, yes, that might work.
<deryck> rick_h_96, rick_h__ rick who?  ping for standup.
<rick_h__> deryck: I'm in there?
<rick_h__> or not...
<abentley> deryck, adeuring: private & embargoed blueprints break the front page: https://blueprints.qastaging.launchpad.net/
<deryck> abentley, oh, ouch.  that's a bad one.
<abentley> deryck: Blocker for the beta?
<adeuring> seems that we need to filter properly...
<deryck> abentley, yes, we should fix that first.  but let's make that fix the priority.
<jcsackett> sinzui: i am looking at bug 808952 and wondering, is there any reason we haven't exposed all Message types on the API?
<_mup_> Bug #808952: NoCanonicalUrl using api to fetch bug comments <api> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/808952 >
<deryck> abentley, do you want that one next, or shall I?
<abentley> deryck: Could you?  I'm still on QA.
<deryck> abentley, I absolutely can.  I likewise am not free yet.  Still fixing a couple tests.  But when I get free I can take it next.
<sinzui> jcsackett, we only expose those that are needed. I think this case is differnt though
<sinzui> jcsackett, Our views can show objects without a canonical url, but the api requires that we define one, and that Lp provide the objects consistently to make the url
<jcsackett> so you think we should update the API to handle situations more like our views?
<sinzui> no
<sinzui> I am saying that comments do not match what we put in the zcml
<sinzui>         <browser:url
<sinzui>             for="lp.bugs.interfaces.bugmessage.IBugComment"
<sinzui>             path_expression="string:comments/${index}"
<sinzui>             attribute_to_parent="bugtask"
<sinzui>             rootsite="bugs"/>
<sinzui> jcsackett, I think bug 210821 might be easier to fix. We know the portlet that shows the active project. I think we can walk backwards to find the code that put there
<_mup_> Bug #210821: bug tracker list shows inactive projects <404> <bugwatch> <lp-bugs> <qa-ok> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/210821 >
<jcsackett> sinzui: fair.
<abentley> deryck: I'm done QA for now.  Shall I work on the front page query so you can work on the blog post?
<deryck> abentley, if you have the bandwidth, I would appreciate that!  I'm still fighting trying to figure out why a test is failing too.
<abentley> deryck: Okay.  I'm also filing a bug and adding a card.
<deryck> abentley, I added a red card, but didn't file a bug yet.
<abentley> deryck: ack
<abentley> deryck: What do you think about hiding Proprietary/Embargoed from the front page entirely, even if you have permission to see them?  Would be a simpler fix.
<deryck> abentley, I was going to take that approach.  simple and naive for now.
<abentley> deryck: Cool.
<deryck> abentley, other lists will need more care, but the top-level page can just be public stuff.
<deryck> abentley, also we're okay about other lists breaking, it's how we've been releasing this other stuff, and we can fix those later.....
<deryck> abentley, this one is a beta blocker because it would affect everyone.
<abentley> deryck: Agreed.
<deryck> cool
<deryck> abentley, while we're chatting... :)  I'd appreciate your second eyes on this test:  http://pastebin.ubuntu.com/1204931/
<deryck> abentley, it's breaking for me on line 5.  the match_it is in setUp and looking for field.information_type.....
<deryck> abentley, I have no idea what I changed to break it, and looking in the browser manually I see that field.
<deryck> abentley, am I understanding what it's trying to test?  Something else I should look at?
<abentley> deryck: I thought match_it was for the information type in the privacy banner?  And the privacy banner isn't displayed for PUBLIC.
<deryck> abentley, ah, ok.  So maybe a bad test then.
<deryck> abentley, the test class is TestNewSpecificationInformationType
<deryck> and then goes through each target in a small test like this.
<abentley> deryck: No, I was confused.  match_it matches the field, not the banner text.
<abentley> deryck: Are you sure the field is displayed?  For a project with only PUBLIC blueprints, it wouldn't be.
<deryck> abentley, ah, ok.  I looked wrong.  I had a test project which was proprietary.  Sorry.  I see now....
<deryck> I can fix it to test that it's Not match.
<abentley> deryck: No, I think the test needs to change so that the Specification policy defaults to public but permits proprietary.
<deryck> abentley, ah, gotchas.  ok.
<abentley> deryck: And in a later branch, we should add tests to ensure that the rest of the defaults work.  For the "Use sharing policy default for creating specs" card.
<deryck> abentley, agreed.  sounds good.
<deryck> abentley, thanks for the help!  I can move forward again, yay! :)
<abentley> deryck: np.
<abentley> deryck: I'm not sure I'll be able to QA this bugfix because the qastaging front page was already timing out.
<deryck> abentley, ok.  I guess test as good locally as possible.
<deryck> abentley, well, if it times out again though, you can consider it fixed. :)
<abentley> deryck: Yes.  Theoretically, I could have added unacceptable delay to the pageload, though.
<deryck> oh right
<abentley> rick_h__: Does this error look familiar? http://pastebin.ubuntu.com/1204999/
<rick_h__> abentley: looking
<rick_h__> abentley: yea, that's the testfix from this morning
<abentley> rick_h_: So merge and resubmit?
<rick_h__> abentley: rgr
<abentley> rick_h_: Actually, since that's the only failing test I'm gonna do a straight lp-land once I'm sure it's fixed.
<rick_h__> abentley: yea sounds good. It's running through buildout now so should be clear by EOD
 * nigelb wonders why gmail thinks mail from francis is spam :P
<rick_h__> jcsackett: ping, got a sec?
<jcsackett> rick_h__: i think you just pinged me, but could be stale message as my bouncer just reconnected.
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: â
<abentley> bac: are you OCR today?
<abentley> deryck: Could you please review https://code.launchpad.net/~abentley/launchpad/fix-blueprints-home-with-proprietary/+merge/124485 ?
<deryck> abentley, sure.
<deryck> abentley, looks good, thanks.  r=me.
<abentley> deryck: I've determined that with my fix, the search results and "completed" list also behave properly.
<abentley> deryck: However, "meetings" are broken by PROPRIETARY specs.  Bug 1051029
<_mup_> Bug #1051029: PROPRIETARY specifications break meeting listings <Launchpad itself:Triaged> < https://launchpad.net/bugs/1051029 >
<deryck> abentley, ok, thanks for checking all that
<deryck> abentley, I think we can live with that bug until we can get to it
<abentley> deryck: Agreed.
<deryck> cool
<abentley> deryck: I suspect we're going to have this issue with Product, ProductSeries, Distribution and DistroSeries, too.
<abentley> rick_h__: What's the status of lp:~rharding/launchpad/lp_cache_update into lp:launchpad?  I'd like to use json_dump_information_types.
<rick_h__> abentley: it should be coming. I got the ec2 success email just a bit ago
<abentley> rick_h_: excellent.
<rick_h__> deryck: ping
<deryck> hi rick_h__.  sup?
<rick_h__> deryck: just fyi, I can't find any reason for the js timeout error. jcsackett couldn't replicate it and I ended up tossing it at ec2 agin
<jcsackett> rick_h__: that's a weird one.
<rick_h__> if you get a chance to look you'd be another set of eyes, but fyi I did get a second look and nothing seems odd to praying to the ec2 gods
<rick_h__> just a heads up while I go run away for EOD
<deryck> rick_h__, sorry man.  I looked briefly and didn't see anything.  and spent most of the morning fixing my tests.  meant to come back, but switched to other stuff and forgot.
<rick_h_droid> understand, not a problem
<deryck> heh
<deryck> these are not the rick_h's you seek
<abentley> rick_h__: It hasn't landed yet, and the PQM queue is empty.  I don't think it's going to land without further effort.
<rick_h__> abentley: looking
<rick_h__> abentley: ok, wtf. email says sumitted to pqm, I've got nothing from pqm that says it was/wasn't happy
<rick_h__> so going to lp-land it and see what we get I guess
<rick_h__> is pqm having issues today? This morning it wasn't responding to my lp-land and looks like it's not again
<rick_h__> abentley: ok sorry but I've got to get the boy before the babysitter arrives in 30. I do see it in pqm, so will wait to see if it succeeds/fails for some reason
<rick_h__> abentley: but ec2 tests pass and dammit I'll get through pqm/buildbot if I've got to fly to london and throttle some hardware
<abentley> rick_h__: I've unblocked myself by making your branch a pipe in my pipeline.
<abentley> rick_h__: And of course, now it's merged.
<sinzui> rick_h_: use the force...
<sinzui> bzr pqm-submit -m "..."
#launchpad-dev 2012-09-15
<lifeless> yay
<lifeless>     10000  OOPS-4f236f15bf506c5061124374b4816633  Unknown
<cjohnston> When someone gets a chance, could I please get some opinons on the wording changes in https://code.launchpad.net/~chrisjohnston/launchpad/part-essential/+merge/124559  - the reasoning why the change is being made is listed in the MP
<czajkowski> cjohnston: please mail to the -dev list
<czajkowski> cjohnston: irc really at the weekend isn't a good idea for getting feedback.
<cjohnston> I know.. just posting it for if someone happens to be around and bored
<cjohnston> lifeless: I'm not against it being removed.. I am just not sure that I'm up to snuff with the LP code to be the one to remove it.. there is a 'middle' registration level between attending and required that participation essential is getting.. if you want to remove participation essential from LP, I'm good with it.. I would just rather it happen much sooner than later
<lifeless> the meetings system exists solely as a/the backend for summit
<lifeless> if there is a mismatch, I think you should fix it
<lifeless> I mean, you could have made participation essential a restricted field in LP
<lifeless> so that only drivers could set it.
<cjohnston> that would work for me as well... again, I am not sure how much success *I* would have with it.. :-)
<lifeless> LP is straight forward to develop on
<lifeless> there are occasional gotchyas, but that happens everywhere. The buggest hurdle I've seen folk have is just not trying at all.
<cjohnston>         or user is owner:
<cjohnston>         essential = BoolCol(notNull=True, default=False)
<cjohnston> err
<lifeless> cjohnston: moving the attribute from the Edit to Admin interfaces probably
<lifeless> cjohnston: and dealing with any manual forms that show it specifically and should instead not
<lifeless> cjohnston: probably a few hundred lines of diff once you track all the bits down.
<lifeless> cjohnston: if you're talking about changing who can set it
<lifeless> cjohnston: if you're talking about removing it, thats a little more complex, needs three patches:
<lifeless>  - one patch to make it a nullable column
<lifeless>  - one patch to drop all code references to it
<lifeless>  - one patch to drop the column
<cjohnston> by edit admin you mean https://blueprints.launchpad.net/summit/+spec/uds-q/+edit  ?
<lifeless> no, configure.zcml / interfaces, within blueprints
<lifeless>   <!-- SpecificationSubscription -->
<lifeless>   <class class="lp.blueprints.model.specificationsubscription.SpecificationSubscription">
<lifeless>     <allow
<lifeless>         interface="lp.blueprints.interfaces.specificationsubscription.ISpecificationSubscription"/>
<lifeless>     <require
<lifeless>         permission="launchpad.Edit"
<lifeless>         set_attributes="essential"/>
<lifeless> currently only requires Edit on the subscription to set it as essential
<lifeless> so you'd mov that to Admin, and write an adapter to grant launchpad.Admin to someone relevant to the spec (e.g. ubuntu driver, or spec owner, or whatevers).
<cjohnston> is there an example adapter I could look at?
<lifeless> dozens
<lifeless> IIRC they are in 'security.py' somewhere in the tree.
<cjohnston> so something like http://paste.ubuntu.com/1207923/ lifeless ?
<lifeless> something like
<lifeless> yes
<lifeless> there is zcml to register them to
<cjohnston> is that the ISpecificationSubscription?
<lifeless> no, grep for the name of one of the example security adapters
#launchpad-dev 2013-09-09
<stub> wgrant: ta
 * stub wonders how the .tac worked at all without os.path imported
<wgrant> stub: os.path is magical
<wgrant> stub: It's an alias for ${PLATFORM}path, which os/__init__.py determines and imports dynamically
<stub> wgrant: I've got the explicit wait-until-dead check, as I had a test confirming behavior when Swift is dead failing without it. I'm happy to move it to the base class if you want.
<wgrant> Well, except that I guess os is an extension, but same idea
<wgrant> stub: Hm, it seems odd that tearDown doesn't want until it's dead.
<wgrant> s/want/wait/
<stub> I prefer the 'magical' explanation
<stub> Shutting down is normally a layer tear down thing - you don't usually care that it takes a little while to actually die.
<wgrant> Except that some of our layers use the same port each time
<wgrant> (or at least used to)
<wgrant> So things clearly shut down in relatively short order.
<stub> I guess the shutdown time < startup time :)
<wgrant> Yeah
<wgrant> stub: Oh, the diff in the MP must have been out of date
<wgrant> setup.py showed like 12 new deps, then your new commit removed one and now it only shows three new ones.
<stub> yeah, it got stuck at some point.
<stub> Unclogged now (sourcecode suppository?)
<wgrant> That makes a lot more sense, because I remember bringing that up a few weeks ago and you fixed it
<stub> So I should land that now? And do we still shuffle things through ec2test nowadays?
 * stub hasn't inflicted reimbursement forms on his new management yet.
<wgrant> stub: We don't push stuff through ec2 very often at all nowadays, unless it has potential for widespread failure that might take a while to fix.
<wgrant> stub: mock-swift has no widespread impact, but I'd send the second one through ec2.
<wgrant> For small failures it's much quicker to get them from buildbot and then [testfix].
#launchpad-dev 2013-09-12
<StevenK> wgrant: So, JavaScript and microservices?
<wgrant> StevenK: Have you repushed the JS branch?
<StevenK> wgrant: Yeah, the .set() changes have been up since I started on destroying PF, I've just forgotten to ping
<wgrant> Ah
<wgrant> StevenK: Have you tested person names with special characters such as .?
<wgrant> And some of that code still uses appendChild excessively, which makes it pretty unreadable and slow when compared to code that creates the complex structure in a single Y.Node.create
<wgrant> (eg. the bit under draft === true)
<wgrant> I think it creates a <tr><td></td><td><td><div><span></span></div></div></td></tr>, but that would be much clearer if created as a single object.
<StevenK> wgrant: Converting that bit to using .set() almost forced my brain out of my ears to begin with
<wgrant> There were at least two mistakes in there.
<StevenK> wgrant: One two many <td>'s, and there's a missing <div>
<StevenK> One too, even
<wgrant> StevenK: You only need to set two IDs, which should be pretty trivial.
<StevenK> wgrant: Sure, but I want the outer div
<wgrant> node.one('tr > td > div') or use a class
<StevenK> Hm, yeah
<StevenK> wgrant: So newrow = Y.Node.create('{everything}'); and then two lines to call .one().set() ?
<wgrant> StevenK: I think that's a lot clearer, yeah
<wgrant> You can see the structure
<wgrant> And easily see what dynamic bits you're adding
<wgrant> And it's faster.
<StevenK> wgrant: Let me polish off dealing with this e-mail, and I'll change draft === true
<StevenK> wgrant: http://pastebin.ubuntu.com/6095769/
<wgrant> StevenK: That looks superior.
<StevenK> Hmmm, newone.one('tr') is null :-(
<wgrant> StevenK: Isn't that the node itself?
<wgrant> So finding a tr child is not going to be terribly effective.
<wgrant> s/child/descendant/
<StevenK> Yeah, I came to that realisation about 45 seconds ago
<StevenK> wgrant: Still not sure what to do about dots in names
<wgrant> StevenK: dots are fine in IDs, but then you're using CSS selector syntax to find them, which will fail.
<StevenK> So I should switch to person.id ?
<wgrant> We don't usually expose person IDs.
<wgrant> That's one possibility, but you could also just not use Node.one() to find them
<StevenK> And do what instead?
<wgrant> Fetch them directly by ID, rather than a CSS selector under a node
<wgrant> Recall that IDs are globally unique.
<stub> No quoting or escaping mechanism?
<wgrant> One could also quote them, but it's not necessary here.
<StevenK> Hmm
<Ursinha> wgrant, should I move all checks in publishRosettaTranslations to RosettaTranslationsUpload or moving away the part that creates the job is enough?
<wgrant> Ursinha: I think it makes sense to move the whole lot, for consistency.
<Ursinha> wgrant, right.
<wgrant> The custom upload types are fundamentally different in that rosetta translations don't get extracted, but we should still keep the code as similar as is practical.
<Ursinha> wgrant, so I could move it to a _publishRosettaCustom or something, should have the same effect and will keep publishRosettaTranslations consistent with the rest
<Ursinha> instead of moving the checks to RosettaTranslationsCustom, I mean
<Ursinha> in the end it's your call anyway
<Ursinha> :)
<wgrant> Ursinha: Sorry, let me see.
<wgrant> Ursinha: Ah, I see. We obviously can't use _publishCustom for this, as that assumes we're extracting a file. So I'd just move the contents of publishRosettaTranslations to new function in lp.archivepublisher.rosetta_translations and call that directly in publishRosettaTranslations.
<wgrant> Since we don't need the extra tempfile/cleanup behavior that _publishCustom provides
<Ursinha> wgrant, I thought we could avoid having to import a lot of things to do the checks in lp.archivepublisher.rosetta_translations when we already have it all in queue.py :) I was suggesting move the checks to a _publishRosettaCustom instead
<wgrant> Ursinha: I think it's probably still slightly nicer to have it encapsulated in the other module like the rest, rather than inline in queue.py.
<wgrant> But I don't care too much.
<Ursinha> because _publishCustom prepares and runs the process_* methods, so I thought it would make sense to have the checks before process_rosetta_translations as well
<wgrant> _publishCustom doesn't do sanity checks, though
<Ursinha> indeed
<wgrant> It just prepares the temporary file in a generic way, without doing anything type-specific.
<Ursinha> yeah, the sanity checks are all in CustomUpload.process...
<Ursinha> okay, I'll move it all :)
<wgrant> Sounds good.
<Ursinha> wgrant, should I also add tests for this change in lp.soyuz.tests.test_distroseriesqueue_rosetta_translations.py?
<cjwatson> IIRC the archivepublisher tests are unit tests and the soyuz.tests.test_distroseriesqueue tests are kind of integration tests.
<Ursinha> right
<wgrant> Ursinha: I believe cjwatson is correct. The tests for the other custom uploads aren't a completely terrible example, IIRC
<cjwatson> Yeah, I think I beat them into shape in the last big refactoring
<cjwatson> Looks like I converted them from doctests
<wgrant> Yeah
<wgrant> They used to be one or two doctests each
#launchpad-dev 2013-09-13
<mpt> StevenK, wgrant: I think bug 249504 is a duplicate of bug 239909 (especially since bug 289622 was already marked as such). The former is the problem, the latter is the solution.
<_mup_> Bug #249504: There are no ways to invite a person to join a team <lp-registry> <teams> <Launchpad itself:Triaged> <https://launchpad.net/bugs/249504>
<_mup_> Bug #239909: Team Administrator can arbitrarily add members without any action from the LP user <lp-registry> <teams> <Launchpad itself:Triaged> <https://launchpad.net/bugs/239909>
<_mup_> Bug #289622: User should be asked for confirmation when team admin adds him/her to a team <lp-foundations> <Launchpad itself:New> <https://launchpad.net/bugs/289622>
<mpt> ...Or vice versa.
<wgrant> mpt: not strictly, but they're probably close enough. feel free to dupe them.
<mpt> k
<Sylenthemp> #ubuntu-devel
#launchpad-dev 2014-09-08
<bigjools> wgrant: is launchpad still using ampoule?
<wgrant> bigjools: The non-celery MP and branch scan jobs still use it, in TwistedJobRunner.
<bigjools> wgrant: ok thanks, I am looking at using it in maas
<bigjools> it's not packaged though, which is a pain
<stub> https://code.launchpad.net/~stub/launchpad/trivial/+merge/233498 , prob should just trivial it
<jtv> stub: descent â descend
<jtv> (And this does look as if it can't do any harm â unless someone outside depends on the list not being modified in-place in this case)
<stub> fixed
<stub> Its in an os.walk stanza, so in-place is fine (and required)
#launchpad-dev 2014-09-09
<lifeless> wgrant: when you're around; sdague from openstack is reporting a 30% timeout rate setting bugs in nova (single task) to state triaged via the API.
<lifeless> wgrant: make that 66%
<cjwatson> Wow, yeah, timeout city, I haven't been able to add a comment to https://bugs.launchpad.net/ubuntu/+source/netcfg/+bug/1350302
<mup> Bug #1350302: Deploying node with di on armhf/keystone can't find BOOTIF <MAAS:Incomplete> <netcfg (Ubuntu):Fix Released by cjwatson> <netcfg (Ubuntu Trusty):New> <https://launchpad.net/bugs/1350302>
<cjwatson> Worked after like four or five tries
#launchpad-dev 2014-09-10
<wgrant> lifeless: It's a capital crime in most states to report a timeout without an OOPS ID.
<lifeless> wgrant: I know right
<lifeless> wgrant: users, what am I going to do.
<lifeless> wgrant: if the API library made it super clear, I might get them from other folk... I rather suspect it will be in the oops report though
<wgrant> Hmm, I don't see a significantly elevated timeout rate around three hours ago.
<wgrant> But there were some extremely anomalous peaks overnight.
<wgrant> lifeless: Something was very locked.
<wgrant> 1. 	62602.0 	2 	SQL-main-master 	
<wgrant> UPDATE Bug
<wgrant> SET date_last_updated=E$STRING
<wgrant> WHERE Bug.id = $INT
<wgrant> 62 seconds to update a Bug row...
<wgrant> Possibly BugSummaryJournal contention; we had an issue there overnight.
<wgrant> My previous calculations gave us at least 3 days before rollup delay became a noticable problem, but that was nearly two years ago.
<lifeless> 3 days 2 years ago?
<lifeless> wgrant: the rolluper died?
<wgrant> lifeless: The rolluper gets upset in a race condition between milestone and bugtask deletion.
<wgrant> Which occurred right before I went to bed last night.
<wgrant> Easy to fix (just progressively regenerate BugSummary using the script I wrote during BugSummary v2), but annoying and can apparently cause performance problems with rather less backlog than I had anticipated.
<lifeless> wgrant: ah
<lifeless> wgrant: perhaps stub can fix the race :)
<beaumanvienna> Hi there! I have an old package that only compiles on Lucid Lynx. Compiling it on Ubuntu 14.04 gives me the same message as here: #launchpad-dev. Is there anything I can do to get it into my PPA? Thanks!
<beaumanvienna> Sorry, broken link, I meant here: https://launchpadlibrarian.net/184493556/buildlog_ubuntu-trusty-i386.gens_1%3A2.16.7.0_FAILEDTOBUILD.txt.gz
<jelmer> beaumanvienna: that's not really a launchpad question, but rather a package build question
<jelmer> beaumanvienna: that's not really a launchpad question, but rather a package build question
<cjwatson> beaumanvienna: You should ask GTK+ people for advice on that kind of thing, since these are just ongoing deprecations in GTK+ rather than anything intrinsic to Launchpad or even particularly Ubuntu.  You could also try dropping some of the various disable-deprecated options I see there.
<cjwatson> Indeed, this channel is for the development of Launchpad itself.
<cjwatson> I think there's something like #ubuntu-packaging (?) for this kind of question in general ...
<beaumanvienna> Thanks jelmer and cjwatson! @jelmer: could you please recommend me a more suitable IRC channel then? Thanks! @cjwatson:
<cjwatson> Looks like GdkDragContext.targets is in gtkdndprivate.h, which is suggestive.
<beaumanvienna> There is no way to state a Ubuntu version for the build server it runs on?
<cjwatson> Maybe #ubuntu-packaging here, as I said, or I think #gtk+ on irc.gnome.org.
<beaumanvienna> I managed to run this on Lucid Lynx...
<cjwatson> Builds always run on the Ubuntu version that corresponds to the version of Ubuntu they're building for.  There's no other reliable way to make anything at all work.
<cjwatson> GTK+ often makes adjustments to its API.  You must follow along with those.
<beaumanvienna> ohh that's cool! My bad! I stated trusty! :-)
<beaumanvienna> let me try this :-) Thanks!
<cjwatson> You should still port to newer versions, since lucid is not all that far from being end-of-life.
<cjwatson> beaumanvienna: Ah, for now I think you just need to drop -DGSEAL_ENABLE.
<cjwatson> https://developer.gnome.org/gdk2/2.24/gdk2-Drag-and-Drop.html#GdkDragContext shows targets as being protected by GSEAL (which is basically "stuff that is not available at all in these structures in GTK+ 3, so avoiding it is a good start for porting").
<cjwatson> As of 2.22 there's gdk_drag_context_list_targets instead.
<beaumanvienna> Yes, you're right. I'll need to check out the option you mentioned. While trying to get this compile under 14.04 I understood from some forum posts that some gtk function are totally deprecated. If I remember right, a certain function is only available until 2.21.8 or so. But I definitely need to do something about this, I totally agree...
<cjwatson> The various deprecated options and -DGSEAL_ENABLE are for the purpose of porting programs to strict adherence with newer versions of the API, so they are exactly what you do not want when just trying to get older code building.
<cjwatson> *DISABLE_DEPRECATED, I mean
<beaumanvienna> cjwatson, that was very helpful! Thanks! My problem would be solved if I could just re-enable the deprecated code. Actually, this makes totally sense. How deprecates function from version 2.21 to 2.22 :-DD
<beaumanvienna> Ahh, still not: https://launchpadlibrarian.net/184494968/buildlog_ubuntu-lucid-amd64.gens_1%3A2.16.7.0.0_FAILEDTOBUILD.txt.gz. OK, need to go. Thanks for the great help! :-D
<beaumanvienna> Worked now, https://launchpad.net/~beauman/+archive/ubuntu/retrorig/+build/6348496 thanks again!
#launchpad-dev 2015-09-07
<cjwatson> The review queue is getting rather backed up ...
<wgrant> Hm, just the one from Thursday and five from Friday, or am I missing some?
<cjwatson> Those, I think.
<cjwatson> Oh, and ~cjwatson/turnip/merge-prerequisite, and a recheck of my recent changes to ~cjwatson/launchpad/team-mail would be good since those introduce async jobs.
<wgrant> Ah, I see, will do.
<cjwatson> Ta.  I'm about to tackle the immediate mail delivery thing in zopeless, at least moving things along a bit.
<wgrant> Hm, which thing is that?
<cjwatson> https://bugs.launchpad.net/launchpad/+bug/29744
<mup> Bug #29744: In Zopeless, mails should be sent only when the transaction is commited <email> <infrastructure> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> <https://launchpad.net/bugs/29744>
<wgrant> That's a huge amount of trouble.
<wgrant> Due to archiveuploader relying on them being sent on abort
<cjwatson> Not terrible to at least make it more opt-in.
<wgrant> IIRC
<wgrant> True.
<cjwatson> I've almost fixed archiveuploader.
<wgrant> Ooh
<cjwatson> But the first step is to make it easier for archiveuploader to opt-in selectively.
<cjwatson> Actually the first first step is to convert more stuff to pop_notifications, because otherwise there's a pattern that's rather too easy to fit where you do self.assertEqual([], stub.test_emails) without committing first, and moving away from immediate delivery makes that kind of test silently ineffective.
<cjwatson> I have a fully passing test suite (with the exception of some unrelated buildmaster cruft) with immediate delivery disabled in zopeless in general but enabled in BaseMailer and a few other places.
<cjwatson> The reason I care is that otherwise converting the world of mail notifications to zopeless (so that we don't have to care about having an actor who might not be allowed to know about some subscribers) might be a rather more exciting stealth change than it ought t obe.
<wgrant> I'm not too concerned about that.
<wgrant> It is a slight issue, but not very large.
<wgrant> As the main thing that can fail is operation itself, not the email transmission.
<wgrant> Good to fix, anyway.
<cjwatson> archiveuploader is relatively easy now that most of the code is in PackageUploadMailer.  The fix is to use the mailer directly rather than relying on going through PackageUpload.notify; then it doesn't have to abort in order to get rid of the temporary PU object, and can just do abort; send mail; commit
<cjwatson> Requires pushing a couple of extra bits of logic down into the mailer to avoid duplication, but not very much.
<wgrant> Yep
<cjwatson> And that self.queue_root.notify business in archiveuploader was something I always hated.
<cjwatson> Ha, I found something that was still depending on the "debian" alias for binarypackage!
<wgrant> I believe it works due to a very old launchpad-buildd on the buildbot slaves.
<cjwatson> Yep.  But they have at least 121 per my IRC logs, so we can use the new forms.
<cjwatson> I'll get a clean bin/test locally yet.
<wgrant> Oh, I didn't realise they were that new. Handy.
<wgrant> Hmm
<wgrant> We should really stop catching IntegrityError in places.
<cjwatson> aaaa
<wgrant> Only Packageset, Snap and LiveFS do it still.
<wgrant> What triggered the IPv6 alarm?
<cjwatson> Oh, yeah, I forget why I couldn't think of a better way to do that
<cjwatson> Just fear at what else that could hide
<wgrant> It just had me running in circles for a bit.
<cjwatson> Though it is indeed my fault in this case
<wgrant> Chasing down why a snap got created despite the feature flag being disabled and causing a 401.
<wgrant> Of course it was actually lying when it said one existed -- instead, git_path was null.
<wgrant> The correct thing to do in this situation is LBYL
<cjwatson> I was about to say exactly that!
<cjwatson> We're raising a specific error so should check for a specific condition.
<wgrant> Yep
<wgrant> I think you actually copied that originally from a place that I fixed reasonably recently...
<cjwatson> Almost certainly.
<cjwatson> Yay clean buildmaster tests
<wgrant> Marvellous.
<cjwatson> I believe that's an entirely clean bin/test for me locally now, with the codeimport fixes.  I ran the whole thing at the weekend to see what exploded without immediate delivery.
<cjwatson> 3.5 hours
<wgrant> Ah, that's right, LibraryFileAlias.create said a filename contained illegal characters whenever there was an IntegrityError.
<wgrant> Not quite the same, but close.
<wgrant> Yep, even Haswell-U is pretty snappy at running the test suite nowadays.
<lifeless> cjwatson: 3.5 *hours* ?
<cjwatson> lifeless: In which direction is that surprising? :)
<lifeless> cjwatson: seems slow - I recall running it locally in ~90m ?
<lifeless> cjwatson: (with parallel of course)
<cjwatson> lifeless: This is just serially
<cjwatson> No particular effort to make them go faster, I was doing housework for those 3.5 hours :-)  I very rarely run the whole thing locally, this is the first time it's been necessary for me in eight months full-time on LP
<cjwatson> If I did it more often I expect I'd bother setting up the parallel harness stuff
<lifeless> cjwatson: ah. Fair enough ... the parallel setup is pretty easy, and it helps even with running subsets
<cjwatson> It's probably worth an hour of effort at some point
<cjwatson> Effort in that direction would be more appealingly spent on being able to run tests on more than one branch at once, mind you (though I suppose it involves some of the same work)
<lifeless> huh, what stops you running on more than one branch?
<lifeless> presumably the only thing is the template db
<lifeless> and thats (IIRC) controllable via an env variable.
<cjwatson> Never having bothered to work out the details :-)
<cjwatson> There are various specific tests that are singleton in odd ways
<cjwatson> Like starting up servers on hardcoded port numbers
<lifeless> pretty sure all the fixtures report back the port number
<lifeless> if not, I think thats likely a regression
<cjwatson> Pretty sure not everything uses them right
<lifeless> entirely possible :)
<lifeless> anyhow, yes the work doth likely overlap
<lifeless> but - the container approach gives you entire isolation anyhow
<cjwatson> Like the buildmaster integration tests have AIUI never done dynamic port selection
<cjwatson> So a few tests fail obscurely if you happen to have a real buildd running in the same environment
<lifeless> so if you have parallel, you have multibranch with at most trivial glue
<cjwatson> Yeah.  One of these days ...
<lifeless> indeed http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/buildmaster/tests/harness.py
<lifeless> is missing port randomisation
<lifeless> there's glue for that elsewhere in the three. TUITs huh
<cjwatson> There's lib/lp/services/librarianserver/testing/server.py
<cjwatson> I think that's the only bit of .tac glue that does it
<cjwatson> log parsing, nasty
<lifeless> yes
<lifeless> well
<lifeless> its better than the race of the 'find a port in parent and say 'use port X' on the command line
<lifeless> an actual fd handover would arguably be better
<lifeless> log parsing is at least race free
<cjwatson> Yeah
<cjwatson> Would want to try to push that down a level if using that from more places
<wgrant> lifeless: There are heaps of things that were never isolated
<wgrant> That's why the container approach was used,.
<wgrant> eg. AppserverLayer even conflicts with the dev appserver.
#launchpad-dev 2015-09-08
<lifeless> wgrant: sure
<wgrant> cjwatson: So that branch isn't terrifying at all.
 * cjwatson attempts to apply sarcasm detector to wgrant
<StevenK> cjwatson: Surely that always returns true.
<wgrant> No it doesn't.
<StevenK> wgrant: Liar.
<wgrant> I don't know why I'd be terrified of a branch that changes the behaviour of lots of scripts that haven't been touched since I refactored this four years ago :)
<cjwatson> I've done a lot of grepping trying to think of patterns that might be problematic; direct use of test_emails, catching SMTPExceptions, aborted transactions, that kind of thing
<wgrant> Yeah
<wgrant> We'll need to do a lot of weird QA.
<wgrant> But it's manageable.
<cjwatson> Can you think of any other patterns I might have missed?
<wgrant> I don't think so.
<wgrant> The only other big risk I see is that some things might somehow break the email commit hook.
<wgrant> Or not commit at all.
<cjwatson> Well, if they don't commit at all then the operation being notified about doesn't happen either.  Unless they commit before sending mail
<wgrant> Right, but errors might do that.
<wgrant> Some errors abort because they don't LBYL
<wgrant> but others will.
<cjwatson> I'm definitely spooked by the oops/error handling in jobs, I don't really understand that yet
<cjwatson> (which is why I applied the context manager to those for now)
#launchpad-dev 2015-09-09
<cjwatson> wgrant: Is there a good way to handle the case where setting one field in a Storm class needs to potentially adjust that value depending on the value of another field?  Setting git_path needs to do git_repository.getRefByPath(); but if I just do the naÃ¯ve thing and write git_path.setter, I get an IntegrityError while submitting edit forms, because updateContextFromData sets git_repository and then attempts to set git_path, but the ...
<cjwatson> ... implicit flush at the start of the resulting Store.find causes an INSERT with git_path = null.
<cjwatson> I was looking for some kind of filtering property on the interface field but couldn't find one.
<cjwatson> (i.e. so that lazr.restful could do the mangling before it got as far as updateContextFromData)
<cjwatson> We have various other property setters but as far as I've found so far they aren't in situations where they need to care much about the implicit flush - the problem here is that git_repository and git_path need to be set consistently with each other, but setting the latter depends on the former
<wgrant> cjwatson: Is validating that really useful?
<wgrant> cjwatson: It is going to be able to drift anyway.
<cjwatson> wgrant: Canonicalising is useful, because it makes it a lot easier to implement GitRef:+snaps.
<cjwatson> And people are likely to enter master rather than refs/heads/master
<wgrant> cjwatson: Right, but that could just be a custom interface field.
<wgrant> It doesn't depend on the repository AFAICS
<cjwatson> I think it would be rather helpful to say that it doesn't exist if you make a typo when you try to enter it, even if it vanishes later
<wgrant> Doing that on the API is difficult.
<wgrant> For the web UI I'd just have something custom in validate()
<cjwatson> For form submissions I can do it in the view's validate method, and for Snap creation I can do it in SnapSet.new.  Maybe I just have to sacrifice the ability to do it on the webservice.
<wgrant> Correct.
<wgrant> The order of operations of a multi-field patch is not well-defined, and there's no way to take an action after everything is set (other than IObjectModifiedEvent)
<wgrant> And doing validation in the modified event is unprecedented AFAIK.
<cjwatson> If there's some way to have single-field webservice sets go through a different method than internal sets, that would be useful
<wgrant> @mutator_for
<cjwatson> Oho
<cjwatson> Of course.  That would cover more or less all the bases then
<wgrant> I don't think you can use it :)
<cjwatson> Why not?
<wgrant> If I'm changing the repo and path simultaneously, I may need both to change before either will validate.
<cjwatson> Well, only if there's a validator on the repository
<cjwatson> But yes, perhaps you're right
<wgrant> It would seem unusual for it to be illegal to set the ref to something that doesn't exist in the repo, but legal to change the repo to one that doesn't have that ref.
<cjwatson> I kind of want to say that this implies that only Snap.git_ref should be exposed
<cjwatson> But we decided not to do that for BMP
<wgrant> It was impossible for BMP as the repo references needed to remain intact even after the branch was deleted.
<wgrant> Oh
<wgrant> Could have done a virtual one, though, I suppose.
<wgrant> I don't recall why we didn't do that.
<cjwatson> I don't think I'd conceived of GitRefFrozen when we designed the BMP exports.
<cjwatson> Maybe.  Although I did eventually do that for BMP.
<wgrant> It is indeed possible not.
<wgrant> I would continue to expose the split fields directly anyway, if only read-only, for convenience.
<cjwatson> There's a good reason to continue to expose them: you might have a snap that was built from a ref that has been deleted.
<cjwatson> But I'm inclined to say it should only be possible to set by the combined field.
<wgrant> Or we could just have a named operation that sets both.
<cjwatson> GitRef is already exported, so I see no reason not to use it.
<cjwatson> BMP is partially consistent with this.  You can only create one using GitRef.createMergeProposal; but once you've created one, you can only modify it using the combined fields.
<cjwatson> Oh, but those fields are read-only on BMP anyway.
<cjwatson> You have to resubmit to change them, and resubmit isn't exposed on the webservice.
<wgrant> And resubmit is a named op anyway
<cjwatson> So BMP isn't as good a model as I thought it was.
<cjwatson> I guess the main remaining problem is that I'd need a GitRefVocabulary.
<cjwatson> Or maybe not, that field wouldn't be in the form.
<cjwatson> This feels a lot nicer already.
<wgrant> I dislike using object references when they are silly, but it may be best here.
<cjwatson> I like it when I can delete tests for whether things are canonicalised properly because there's no way to express passing in anything non-canonical.
<cjwatson> wgrant: Did you get a chance to look at team-mail again?
<wgrant> cjwatson: It should have a vote, unless I have too many MPs open.
<cjwatson> wgrant: So it does, buried in threaded mailer.  Thanks.
<lifeless> cjwatson: ping (https://github.com/testing-cabal/testtools/pull/149)
<cjwatson> ah, answered on #ubuntu-devel
<lifeless> :)
#launchpad-dev 2015-09-10
<wgrant> cjwatson: Can you look at my JS branch when you have a chance?
<cjwatson> wgrant: Sorry, right - will get to that before the meeting
<cjwatson> wgrant: If you can QA r17707, we can deploy
<cjwatson> The mail stuff seems to be behaving itself on DF
<wgrant> cjwatson: Whoops, forgot that one. Thanks.
<wgrant> Great.
<cjwatson> (qastaging isn't running quite the right jobs, which I should fix at some point)
<cjwatson> I'm going to mark the gmail filtering asana tasks done, since team-mail was the last major piece.  Anything else can be handled by more targeted bugs.
<wgrant> I did like the lack of support for @ bug.
<cjwatson> Did you have any thoughts on the bug mterry filed in which I suggested that bare "Rationale" should change to "Rationale @username" even for non-teams?
<wgrant> I'd throw rocks at Gmail rather than breaking sane people's filters.
<wgrant> But they haven't seemed to respond to rocks in the past, sadly.
<cjwatson> I don't know if it's actually lack of support for @ (it could be) or just lack of support for anchoring to end of line.
<cjwatson> Either would be a problem here, and the @ stuff doesn't actually need to change - I can check, but I expect gmail will just ignore that bit
<wgrant> Right, but some people no doubt filter lack-of-@ to inbox and blackhole everything from silly teams.
<cjwatson> "Rationale @username" would arguably be a simplification that makes things easier to understand, even if it breaks things for some people with anchored filters.
<cjwatson> Yeah.  It would have to be announced somehow ...
<cjwatson> Maybe a workaround would be a new header
<cjwatson> X-Launchpad-Message-Subscriber
<wgrant> It's not always a subscriber, but indeed.
<wgrant> Something along those lines could work.
<cjwatson> It's called a subscriber by BaseMailer :-)
<wgrant> X-Launchpad-Message-Get-Your-Shit-Together,-Google
<cjwatson> But nouns
<wgrant> Mm
<cjwatson> wgrant: Thanks, requesting a deployment.
<wgrant> cjwatson: Thank you.
#launchpad-dev 2015-09-11
<cjwatson> wgrant: Looks like you need some more preloading on SPRB
<wgrant> cjwatson: Already pushed and awaiting more failures :)
<cjwatson> Ah :)
<wgrant> Though, shockingly, it seems like that's the extent of them.
<cjwatson> wgrant: And now it's my turn, I see
<cjwatson> Also WTF, I ran those tests, I'm nearly sure
<cjwatson> I mean, I ran -t mail
<cjwatson> oh, I had a layer setup failure I didn't notice :(
<wgrant> Heh
<wgrant> Oh, that testfix wasn't nearly as bad as I expected.
<cjwatson> Amazing how much test failure verbiage can be caused by a small amount.
<cjwatson> Hm, snap-listings is harder than I thought.  Branch.recipes etc. can be fairly naÃ¯ve, but they get away with that because recipes don't support private branches.  I need something a bit more like Branch.getMergeProposals.
<cjwatson> Well, maybe Branch.snaps is OK, but at any rate Person.snaps (or equivalent) has to limit to snap sources that you can see.
<cjwatson> At least we don't have to deal with intersections between privacy of multiple branches.
<cjwatson> BranchCollection.getMergeProposals /o\
#launchpad-dev 2015-09-12
<wgrant> Yeah, MPs are always messy.
#launchpad-dev 2015-09-13
<rpadovani> Hey guys, sorry to bother you. I had a working lp build, but now when I try to connect to https://launchpad.dev I have an SSL error
<rpadovani> Error code: ssl_error_rx_record_too_long
<rpadovani> Any hint?
<cjwatson> Not one I've seen personally; perhaps the various recommendations in https://stackoverflow.com/questions/119336/ssl-error-rx-record-too-long-and-apache-ssl might be useful
<rpadovani> mhh, now it only says 'It works' seems my lxc went crazy
<wgrant> rpadovani: That usually means that you need to restart apache.
<wgrant> After enabling TLS for the first time without restarting it can sometimes respond with plaintext on the TLS port.
#launchpad-dev 2016-09-15
<mwhudson> what's up with the s390x builders taking forever to update themselves?
<mwhudson> Fetched 144 MB in 3min 26s (696 kB/s)
<brady> Is this the right place to discuss packaging/build issues with ppaâs?
<cjwatson> brady: #launchpad is probably a better place to start, though it would depend on the issue.
<brady> I have a package that requires golang-1.7, specifically the debian/control file contains âBuild-Depends: debhelper, golang (>= 1.7) | golang-1.7 (>= 1.7)â, except when the build system starts it seems to always install golang-1.6.. I am sure I am doing something wrong but for the life of me I canât figure out what
<cjwatson> brady: Would need to see the log really.
<brady> https://launchpadlibrarian.net/284492546/buildlog_ubuntu-xenial-amd64.logsaver_0.0.3-1_BUILDING.txt.gz
<cjwatson> brady: The golang package is at version 2:1.6-1ubuntu4, which is greater than 1.7.
<cjwatson> brady: You probably wanted (>= 2:1.7)
<cjwatson> brady: (But only for the golang dependency, not the golang-1.7 one.)
<cjwatson> brady: So golang (>= 2:1.7) | golang-1.7 (>= 1.7)
<brady> â¦. Son of aâ¦ I looked at that like 10 times and didnât see it =)
<brady> Woot! That got it! Thanks for the help @cjwatson! =)
<cjwatson> np
