#launchpad-dev 2009-10-19
 * mwhudson rebooting
<wgrant> mwhudson: I wonder if a distinct error page could be used for LH, given that it is somewhat failed most of the time, and gives Launchpad an even worse stability reputation than the bad one it deserves.
<mwhudson> wgrant: fixing the problems with loggerhead would be better, but yes, perhaps a good short term idea
<wgrant> mwhudson: At the moment, it's unobvious to the uneducated observer that it isn't LP which is broken most of the time.
<wgrant> mwhudson: You've been attempting to fix the LH problem for years. I don't think it will go away soon.
<mwhudson> make build is broken for me :(
<wgrant> mwhudson: What's it complaining about?
<wgrant> (you're Karmic now, right?)
<mwhudson> No local packages or download links found for PasteDeploy
<mwhudson> yeah
<wgrant> Up to date download-cache?
<mwhudson> yep
<wgrant> Hrmph.
<mwhudson> actually it's also saying
<mwhudson> Download error: (101, 'Network is unreachable') -- Some packages may not be found!
<wgrant> Ah.
<wgrant> That would do it.
<mwhudson> it's a lie though
<wgrant> It is not.
<mwhudson> hm?
<wgrant> That errno is rarely a lie.
<mwhudson> oh maybe pypi.python.org is down?
<wgrant> WFM
<mwhudson> interesting
<wgrant> That normally indicates broken local routing tables, doesn't it?
<mwhudson> sigh
<mwhudson> do i want to do battle with vodafone support, i wonder
<wgrant> Yay telcos.
<mwhudson> i guess step 0 is restarting my modem
<wgrant> At least something is working.
<mwhudson_> it does look like the isps routing is shafted
<wgrant> Uhoh.
<mwhudson_> !?
<mwhudson_> or, alternatively, the dns server on my modem is giving the wrong answer
<wgrant> What is it saying?
<mwhudson_> 32.1.8.136
<mwhudson_> googling suggests something wacky with ipv6 vs ipv4 might be going on
<mwhudson_> and now it works
<mwhudson_> ffs
<jkakar> I've spent a bit of time looking at how to write tests for launchpadlib-using programs and it's a non-trivial problem.
<jkakar> I think the right solution is to do something to launchpadlib (or maybe wadllib) to make testing easier, not to implement some special-case infrastructure in my application.
<mwhudson> thumper: here?
<mwhudson> wow, it's 18:45, i'll assume not
<lifeless> mwhudson: are you coming to brisbane?
<mwhudson> lifeless: yes
<lifeless> did you find a good flight from pycon?
 * mwhudson edits a wiki page
<mwhudson> lifeless: it's a bit of a mission, chc -> akl late ish on sunday, akl -> bne at gnat's fart on monday
<lifeless> mwhudson: grah
<mwhudson> lifeless: you thinking of coming to kiwipycon?
<lifeless> yes, I just signed up
<lifeless> if you can give me your flight details I can try to line up
<mwhudson> lifeless: cool
<mwhudson> lifeless: just fowarded my itinerary
<lifeless> If its too hard I may skip sun avo
<mwhudson> yeah fair enough
<mwhudson> i was signed up to talk at the conference before the sprint was organized so...
<lifeless> you're done by 10am :P
<jtv1> mwhudson, still here?  need some help setting up branches for a test
<mwhudson> jtv1: not really
 * jtv1 shrugs and plods on
<mwhudson> jtv1: jml will be here soon :)
<jtv1> ah good
<jtv1> he still owes me a pre-dawn call, heheh
<adeuring> good morning
<jtv1> hi adeuring
<adeuring> hi jtv!
<jtv> jml: need your help!
<mrevell> morning
* mrevell changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 2 of 3.1.10 | I am Zero OOPS and So Can You! http://is.gd/4fkLl | PQM is OPEN | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
<jml> mwhudson, jtv:  hell
<jtv> jml: hell?
<jtv> mrevell: morning
<jml> o
<jml> I really thought there was an 'o' there
<lifeless> jml: did you have a chance to peek at subunit/protocol ?
<jml> lifeless, no, sorry.
<jtv> jml: so one letter makes the difference between "yes" and "no."  I for one am glad for the change.
<jtv> jml: hope you can help me...  I'm trying to set up a hosted branch stacked on another hosted branch for my test.  A real branch that I can commit to.
<jml> jtv, ahh yes.
<jml> jtv, this is for some sort of integration test, I assume?
<jtv> jml: not really, it's for testing a workaround for a bzr bug
<jtv> I tried piggybacking on BzrSyncTestCase, but to no avail
<jml> jtv, it's a bit fiddly.
<jtv> that much I did manage to find out :)
<jml> jtv, do you know how to create a stacked branch independently of Launchpad, i.e. just using bzrlib?
<jtv> jml: set the stacking url... the problem seems to be either getting the right URL or making sure the branch is really there.
<jml> jtv, you need the db branch objects as well, right?
<jtv> jml: yup
<jtv> I'm particularly surprised that aping what happens in test_bzrsync.py doesn't seem to give me lp-hosted support.
<jml> jtv, not at all.
<jml> jtv, or rather, you shouldn't be
<jml> jtv, why does the scanner need access to the hosted area?
<jtv> jml: ah, it reads from the mirrored branches.
<jml> jtv, that's right.
<jtv> So back to the old useBzrBranches()-based approach.
<jml> probably
<jml> jtv, I have a few things I need to do now. I'll be back in about 40mins and can talk about this further then, if that's ok.
<jtv> jml: ok, thanks
<jml> jtv, back.
<jtv> jml: excellent.  Shall I run you through the code I'm trying now?
<jtv> (Not that I haven't tried other ways...)
<jml> jtv, please do.
<jtv> jml: I start with useBzrBranches().
<jtv> Then I create a base branch using create_branch_and_tree.
<jtv> I pass hosted=True there.
<jtv> Then I create another branch in the same way: stacked_branch, stacked_tree = create_branch_and_tree(hosted=True)
<jtv> Then I try to stack it using stacked_tree.branch.set_stacked_on_url(base_tree.branch.base)
<jml> jtv, what happens then?
<jtv> Then, several layers down into what I'm trying to test, I try to open a mirrorer for stacked_branch.getPullUrl()
<jtv> and that fails with BadUrlScheme
<jtv> BadUrlScheme: ('chroot-247936556', URI('chroot-247936556:///~person-name9/product-name4/branch11/'))
<jml> ahh, I see.
<jml> jtv, it'll probably help both of us if you give the branches explicit names like 'a' and 'b'
<jtv> jml: tbh I did, but left it out of this summary: "base" and "stacked."
<jml> jtv, I mean, the name of the branch on disk
<jml> jtv, not the variable names.
<jtv> How do I do that, if not by passing a first arg to create_branch_and_tree?
<jml> create_branch_and_tree(hosted=True, name='wooster')
<jtv> ahh
<jml> or you could create the db_branch manually and pass that in explicitly
<jml> create_branch_and_tree(hosted=True, db_branch=factory.makeAnyBranch(name='wooster'))
<jtv> Hmm... "file exists: u'/tmp/tmpyXcSbD/.bzr'"
<jml> oh
<jml> and the thing that will probably fix your problem
<jml> rather than set_stacked_on_url(branch.base)
<jml> do set_stacked_on_url('/' + db_branch.unique_name)
<jml> the tempfile thing disturbs me.
<jtv> Ah, I tried that too, but in a very different setup.  Too many variables...
<jtv> It's probably because those names I passed in previously went into the "location" parameter, which defaults to "."
<jml> jtv, yeah, the system itself is a bit too complex.
<jtv> Now that I was no longer passing those, create_branch_and_tree tried to set up multiple branches in . I guess
<jml> I hope that ditching the hosted / mirrored thing will be a good simplifying step.
<jtv> yesyesyes getting further
<jtv> I passed both that first argument _and_ the name parameter.
<jtv> Is there a way to ditch that!?
<jml> jtv, can you paste exactly what you are calling?
<jtv> jml: coming...
<jml> or are we talking across each other?
<jtv> jml: http://pastebin.ubuntu.com/296709/
<jtv> It's working now, except the code I'm trying to test is refusing to commit because the branch hasn't been scanned.
<jtv> which is a definite step forward
<jml> jtv, excellent
<jml> jtv, mwhudson & thumper have plans to get rid of the hosted area completely
<jtv> jml: shame about the data
<jtv> :-)
<jml> jtv, none of it is unique, that's why they want to ditch it :)
<jtv> jml: are we talking about not mirroring at all?
<jml> jtv, not mirroring hosted branches, yes.
<jtv> I guess they either came up with a performance improvement or found that mirroring wasn't saving us anything...
<lifeless> well we're not scaling out using it, which was one of the original intents
<lifeless> jml: no worries; I still desire your feedback on the api, so if/when you can
<jml> lifeless, btw, when did you send the email?
<lifeless> what mail?
 * jml rephrases
<jml> lifeless, where should I look to find the protocol stuff?
<lifeless> lp:~lifeless/subunit/protocol
<jml> lifeless, thanks.
<lifeless> you can ignore the wire protocol aspect if you like,though feedback is cool. Its the python API for TestResult outcomes that is interesting
<lifeless> I'm sure much of it will want to be in testtools, but I decided to get it done in situ, and divestify later
<lifeless> tia.gnight.
<jtv> lifeless: gnight!
<jml> lifeless, g'night-
<jtv> jml, a quickie: I recall there are different kinds of revision id... how do I get the one I'd like to store as last_scanned_id?
<jml> jtv, there's only one kind of revision id
<jml> jtv, you want branch.last_revision_id
<jml> jtv, revno is something completely different
<jtv> jml: I never doubted that it's different to someone in the know...  :-)
<jtv> To me it's highly confusing.
<jtv> 'BzrBranch7' object has no attribute 'last_revision_id'.  I guess tree.branch is not the branch I wanted.  :(
<jtv> ah, last_revision_info.
<jml> jtv, a revision ID is a globally unique identifier for revisions
<jtv> jml: works now, thanks for leading me out of this swamp!
<jtv> Ah, I guess that makes sense for the D part of DCVS
<jml> jtv, my pleasure.
<jml> yeah :)
<deryck> Morning, all.
<jtv> hi deryck
<jml> deryck, good morning
<maxb> barry: are you coming back in #launchpad-sprint today?
<maxb> salgado: Hi. Sorry for that 404ed merge proposal, I'd renamed the branch when its scope grew
<maxb> Are you #launchpad-sprint-ing again today?
<salgado> maxb, no worries.
<salgado> I should be.  let me join
 * maxb wishes for a "make run_all_except_mailman"
<jml> +1
<jml> maxb, really, what we need to do is change codehosting to not use the xmlrpc server.
<jml> at least optionally.
<maxb> I don't mind the xmlrpc server, I just mind mailman spamming up my dev access log
<jml> maxb, yes, that's what I mind.
<jml> maxb, but if codehosting went straight to the db, we wouldn't have to start up anything other than the ssh server.
<jml> maxb, so we could have a make run_dev_codehosting or something similar.
<maxb> Any reason it goes via xmlrpc at present?
<jml> a couple.
<jml> the first is that going via a zope webservice does automatic retries in the face of transactional conflicts.
<jml> the second is that the xmlrpc service adds a certain layer of protection between codehosting & the db.
<jml> the third is that having an XML-RPC server has forced us to think very hard about the interface between codehosting & the rest of Launchpad, and has left us with quite a narrow interface indeed.
<maxb> ...and going via the db would require zcml loading inside the bzr process?
<jml> oh. there's a thought.
<jml> hmm.
<jml> yes it would, at least in the most natural implementation.
<jml> which is obviously a blocker.
<jml> you could work around this by having sql queries in the codehosting server, and duplicating the db configuration
<jml> but... yuck.
<maxb> What is the policy for getting new things into download-cache? Who do I need to talk to?
<intellectronica> maxb: i think gary is the best person to talk to. at the very least he'll be able to dispatch you to someone else
<jml> there's a doc about it, actually.
 * beuno waves
<jml> hi
<jml> off to grab a bite & run some errands.
<gmb> allenap: On which product should I file bugs in EC2 test?
<allenap> gmb: launchpad-foundations
<gmb> allenap: Ta. I feel I may be landing a few itch-scratch branches today...
<allenap> gmb: Tag it with build-infrastructure.
<gmb> allenap: Okay.
<abentley> barry: When I run the people.txt tests, I get a lot of errors like this: http://pastebin.ubuntu.com/296842/ Any idea what's going on?
<beuno> intellectronica, flacoste, I have a doctors appt, so I will likely miss the ajax call today
<intellectronica> beuno: k, thanks for letting us know
<barry> abentley: yes.  we need to update launchpadlib and i believe gary_poster is landing or has a branch in dev that will address these
<abentley> barry: It's weird that we have test failures in stable.  Thanks for the info, though.
<maxb> abentley: I think it may be because you're inadvertently using different lazr.* from your system, rather than the buildout?
<maxb> The workaround is to erase the lazr.*-nspkg.pth symlinks from your python site-packages
<maxb> The proper fix is landing/ed?
<abentley> my lazr.enum is not coming from my system, don't know about the rest.
<maxb> What's PQM doing at the moment?
<allenap> gmb: There was another trac-launchpad-migrator bug that I didn't file (iirc): the migrator does not add the xmlns to the <launchpad-bugs> root tag.
<gmb> allenap: No, you filed it. I thought I just triaged it...
<allenap> gmb: Ah, missed that one. Cool.
<gmb> allenap: No idea why that happens though; according to the code it should work. Might be a bug in the XML generator.
 * gmb is trying to retrofit tests when he has time, which isn't often.
<allenap> gmb: Odd.
<gmb> yarp
<abentley> maxb: Thanks, that looks like it solved it.
<gary_poster> abentley: the reason launchpadlib tests failed on karmic and passed on buildbot/ec2 is that karmic uses a newer version of libxml that is more careful about validating XML (specifically, it does not allow ids to be reused).  The branch I landed fixes that problem, among many others. Unfortunately, while it passed locally and on ec2, there is a problem on buildbot.  I'm trying to dupe and address.
<abentley> gary_poster: I don't think that's the same problem as I saw.  My problem was that items expected to be unicode were actually strings.
<gary_poster> abentley: oh, right, yes.  that was the site-package problem.  After you fix that then you get the other problem I mentioned.
<gary_poster> or maybe at the same time.  don't remember any more
<gary_poster> BjornT: I'm wondering if my branch broke windmill layer.  This happens when you run the mailman error.  notice failure in line 27 of http://paste.ubuntu.com/296932/ .  Do you have quick access to a command for a quick smoke test for running a single windmill test?
<beuno> EdwinGrubbs, sorry for the long wait on your pre-review, looking now
<beuno> EdwinGrubbs, or did you already land that?
<barry> gary_poster, BjornT here is the fix http://paste.ubuntu.com/296944/
<gary_poster> barry: thank you.
<barry> gary_poster: i can land it if i can get review on my mm 2.1.12 branch too
<gary_poster> barry: cool.  we are in testfix because of the problem I mentioned, so trying to get that sorted first
<barry> gary_poster: and actually, i think i can make the other fix work for either our current mm 2.1.10 or 2.1.12 so i could land that first and then upgrade mailman
<barry> gary_poster: ack
<gary_poster> I mean, the one I mentioned to you elsewhere
<gary_poster> k
<danilos> sinzui: hi, is there any reason why default bug sorting order is not by importance and then status?
<sinzui> danilos: importance implies what should we worked on, with is more relevant to more users. status is import to a small group of people. In the milestone view, I always resort the bugs on status because I am watching progress
<danilos> sinzui: is there any reason why it can't be on both by default? (i.e. all high bugs, with fix released first, fix committed next, etc.)
<danilos> sinzui: that's what it used to be before
<sinzui> Yes there is a reason.
<danilos> sinzui: I am all ears :)
<sinzui> When I sort the milestone of importance, it is by time. It is a partial burndown chart. place you mouse over the the status and you will see how long the bug has been in that status
<danilos> sinzui: to me, it seems it's sorted by importance and then by bug ID
<danilos> sinzui: can you explain that to me on the example of https://edge.launchpad.net/rosetta/+milestone/3.1.10 ?
<sinzui> oh, yes I think that is true
<sinzui> We get that form the model
<danilos> sinzui: for example, look at #425651 and #446160
<danilos> sinzui: ok, so my question is: can the "sub-sorting" order be by status, instead of bug ID for the milestone page? is there a reason why it is not so today?
<sinzui> danilos: The issue is not important to me, nor were there bugs about this. so I did not rewrite the sorting code rules.
<danilos> sinzui: ok, thanks, I'll file a bug if you don't mind
<sinzui> danilos: I think bug 430702 is high. We are *required* to have this completed by the end of this release
<danilos> sinzui: I understand it might not be a trivial fix since you are relying on code from model/bugtarget
<mup> Bug #430702: Move Translations windmill tests to TranslationsWindmillLayer <story-windmill-layer> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/430702>
<danilos> sinzui: yes, thanks, I haven't gotten to fully planning out 3.1.10 (it has a long list of things that shouldn't be in anyway)
<sinzui> danilos: Patches excepted, but if you ignore it for two months, I might do it. I am building a list of change I personally want to see on the page
<danilos> sinzui: sure, and completely understood :)
<sinzui> danilos: have you ever seen intltool/make create Binary.gmo
<danilos> sinzui: not sure what you mean, but MO files should be constructed using regular msgfmt stuff
<EdwinGrubbs> beuno-lunch: no, I didn't land it. Thanks for the feedback.
<sinzui> danilos: I was helping a Russian user whose make dies on Binary.gmo. Since his LANG is ru_RU.utf-8, I could not image how Binary.gmo was create from an empty LINGUAS
 * sinzui created a package and suggest he install from PPA
<danilos> sinzui: nothing in there should make use of LANG variable either, but it might be a seriously broken setup on his system, or a bug in makefile scripts somewhere
<danilos> sinzui: i.e. even intltool/gettext makefile scripts
 * sinzui nods
<danilos> anyway, out now
<mrevell> Night all. See you tomorrow.
<jml> I'm off for a bit. Will be back later.
<rockstar> deryck, a user is getting this error when trying to assign himself a bug: https://pastebin.canonical.com/23481/
<deryck> rockstar, looking...
<rockstar> deryck, it looks like an issue with the API stuff, truthfully.
<deryck> rockstar, yeah, I think there's a bug on that, with recent activity even, but I'm behind on bugmail from being off.  maybe intellectronica knows more.
<deryck> maybe related to a fix I've been dealing with on subscriptions, too.
<deryck> oh, hmm, he's away.
<deryck> rockstar, this is bug #438802.
<mup> Bug #438802: UnicodeDecodeError changing 'Assigned to' field when summary contains non-ascii <post-3-ui-cleanup> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/438802>
<deryck> leonardr, ping
<rockstar> deryck, awesome, thanks.
<deryck> rockstar, I'll try to update with more info, if we have any, later today.
<leonardr> deryck, hi
<deryck> leonardr, hi.  Did we ever get that patch of Tom's in lazr.restful into production environment?
<leonardr> i'm not sure what you're referring to. is it the patch to the wadl that makes the id unique?
<deryck> leonardr, this was a patch Tom and I were chatting with you about week before last.  It saved us from errors with non-ascii characters in description editing, but it need forward porting to lazr.restful (because of committed to wrong branch.)
<deryck> leonardr, ring a bell?
<leonardr> ah
<sinzui> EdwinGrubbs: what have you done with bug 435260?
<mup> Bug #435260: IProduct's +packages pages has a ton of links to +distributions <post-3-ui-cleanup> <Launchpad Registry:In Progress by edwin-grubbs> <https://launchpad.net/bugs/435260>
<leonardr> yes, launchpad should be using lazr.restful 0.9.11, which has the fix
<deryck> leonardr, ok, cool, thanks.
<barry> flacoste: ping
<x-warrior> Guys, what do you think about create a filter "untranslated without suggestions" ?
<beuno> x-warrior, it would be super mega awesome
<x-warrior> can somebody told me, if i want to implement a feature like a new filter "untranslated without suggestions" I can start working on it? (I think it isn't hard... ) Or should I propose this idea in a specified place and wait the discussion?
<beuno> x-warrior, you can certainly jump in and start working on it
<beuno> you may want to talk to danilos-afk about it
<beuno> either on the mailing list or via IRC
<beuno> but I'd say, go ahead, get your hands dirty, and be sure to check in so we can help you as you go
<x-warrior> I will take a look but wait to starts coding it... I'm new in launchpad code, and don't want to do shit :X
<beuno> x-warrior, danilos-afk is usually around here earlier, if that time suits you, ping him tomorrow, otherwise, head for the mailing list
<x-warrior> earlier like what time?
<beuno> x-warrior, probably 3 hours earlier at least, he's in Serbia, so he's UTCish
<x-warrior> That's ok, I'm in Brazil
<x-warrior> :D
<beuno> super
<beuno> it would be a welcomed change
<x-warrior> hey beuno if i get the source and got it working, should I updated my source before start working? How can I update it?
<beuno> x-warrior, do you have the code yet?
<beuno> as a general rule, you should try and stay the closest to trunk's tip as you can
<x-warrior> yeah, already try it in my machine
<sinzui> bac: ping
<beuno> x-warrior, so run "rocketfuel-get", it will update trunk + deps
<x-warrior> but I downloaded it one week ago...
<x-warrior> oh thanks man
<x-warrior> :D
<beuno> x-warrior, after that, merge in trunk into your own branch
<beuno> do that every now and then to avoid conflicts as much as you can  :)
<bac> hi sinzui
<sinzui> bac: I just tried the captcha. I expected the field to be highlighted in red when it was wrong (setFieldError())
<sinzui> bac: Is that not possible?
<bac> sinzui: hmm, that may be an oversight on my part
<bac> sinzui: since the forms are not LPFormView i would've had to do that manually and i did not
<sinzui> bac: that is an excellent answer
<bac> forgetful/unobservant = excellent!
<sinzui> bac: This is definitely out of scope. I know ISD wants to change login technology to django. Those forms may never be converted to zope
<bac> ok
<sinzui> now if I could only find that bug mthaddon files about cracked bullets in the registration email confirmation screen
<EdwinGrubbs> sinzui: bug 435260 is the one that I sent screenshots to beuno.
<mup> Bug #435260: IProduct's +packages pages has a ton of links to +distributions <post-3-ui-cleanup> <Launchpad Registry:In Progress by edwin-grubbs> <https://launchpad.net/bugs/435260>
<sinzui> EdwinGrubbs: I looked at that page and thought we should rename the information link to make it clear that it is form the distro perspective. I will avoid touching that page while you work
<sinzui> EdwinGrubbs: The page is involved in the global duplicate sourcepackage links cockup
<EdwinGrubbs> sinzui: do you want me to change the link from "Packaging information" to "Distribution packaging information"?
<sinzui> EdwinGrubbs: yes, I think that is better
<sinzui> EdwinGrubbs: My other frustration with that page is that there were not links to the productseries, though they were listed by name. Once I got to the page, I had to back to the product index to get to the series :(
<sinzui> bac: I think you fixed Bug #118623
<mup> Bug #118623: Hinder spambot registration <Launchpad Foundations:New> <https://launchpad.net/bugs/118623>
<flacoste> hi barry!
<barry> flacoste: hi!  do you have a few minutes to talk about upgrading our sourcecode/mailman to 2.1.12?
<flacoste> barry: sure, now?
<barry> flacoste: sure.  i think we can do irc
<barry> flacoste: so, i have a branch in pqm that will make our MailmanLayer tests pass with either 2.1.10 (what we have) and 2.1.12.  it's really just one trivial test failure
<barry> flacoste: i have an uncommitted branch that merges lp:mailman/2.1 -rtag:2.1.12 to sourcecode/mailman
<barry> flacoste: that diff is almost 325k lines :)
<flacoste> wow
<barry> flacoste: the vast majority of that is translation updates
<barry> flacoste: but there are enough important updates in py files
<barry> flacoste: 2.1.12 is compatible with py 2.6
<barry> flacoste: i've run the MailmanLayer tests on devel + mm 2.1.12 and we look pretty good
<barry> flacoste: so how should we proceed?
<flacoste> barry: land the test compatibiilty change and then land the mailman update?
<flacoste> barry: you should then QA that change extensively on staging
<barry> flacoste: that's what i'm thinking.  does it make any sense to get the mailman change reviewed?
<barry> at 325k lines i think not :)
<flacoste> barry: you might want to delay that QA until next Monday to maximize your 2.6 sprint time, but not later
<flacoste> barry: rs=me
<barry> flacoste: excellent, thanks.  yes, we need this to land for the 2.6 stuff, but we can q/a it next week
<barry> flacoste: perfect, thanks
<flacoste> barry: you should probably took that opportunity to document a good mailing list QA on staging protocol
<barry> flacoste: good idea
<barry> ah.  pqm is still in test fix :/
<jml> g'night folks.
<Ursinha> gary_poster: hi
<deryck> later on, everyone.
<mwhudson> morning
<rockstar> mwhudson, morning.
<rockstar> mwhudson, did you have a chance to chase the pyrex issue yesterday?
<mwhudson> rockstar: argl, no
<mwhudson> well i had a chance to, but forgot
<rockstar> mwhudson, okay.  Is there anything I can do to help?
<mwhudson> rockstar: talk to a losa basically
<mwhudson> rockstar: they should know how to make a new image that has python-pyrex installed
<mwhudson> and if they don't, there's https://wiki.canonical.com/InformationInfrastructure/OSA/BuildbotAMIHowto
<rockstar> mwhudson, great.  I'll harass spm today then.
<mwhudson> rockstar: cool
<gary_poster> Ursinha: hey
<Ursinha> gary_poster: hi! do you know what can be possibly happening here:https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1385J519
<Ursinha> gary_poster: I couldn't find one bug, but if needed I'll be happy to file one
<Ursinha> :)
 * thumper is alive
<rockstar> thumper, that is good.
<thumper> rockstar: morning
<mwhudson> hello thumper
<thumper> hey hey
<rockstar> thumper, remember how we were talking about the branch picker?
<thumper> rockstar: yes
<rockstar> thumper, were you planning on working on that this cycle?
<thumper> rockstar: no, we are lacking the lazr.js infrastructure
<rockstar> thumper, okay.  Let's chat about it after the standup.  There are quite a few birds to kill with that one stone.
<gary_poster> Ursinha: reasonable guess is that someone is passing the same value twice in a request.  If I'm correct, it is a client error, and we should simply not generate an oops, and ideally generate a better error.  That is a generic problem for which there might be a bug, but I don't know the number and this is a specific instance of the problem that should be noted anyway.  Do we have a lot of these?
<gary_poster> yeah, if you look at the query string, that goes along with my guess
<Ursinha> gary_poster: nope, only a few
<Ursinha> but they're there
<gary_poster> Ursinha: sure, needs to be dealt with.  just curious.  Yes, if you would make a bug that would be appreciated.  thank you
<Ursinha> gary_poster: filing now :)
<thumper> rockstar, mwhudson, abentley: I have to go and collect the car from Jenna, shouldn't take too long
<mwhudson> thumper: ok
<gary_poster> Ursinha: thank you!
<Ursinha> gary_poster: bug 455778
<mup> Bug #455778: AttributeError when passing the same value twice in a request <oops> <openid> <Launchpad Foundations:New> <https://launchpad.net/bugs/455778>
<Ursinha> thanks gary_poster!
<gary_poster> :-) thanks again
<sinzui> bac: I award you a gold â for your sterling work on mugshots
<bac> sinzui: gold is better than beer.  of course it was your CSS
<rockstar> abentley, the inline commenting on merge proposals is teh sexy
<abentley> rockstar: Cool.
<Ursinha> gary_poster: am I crazy or similar thing is happening in this one: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1385D1430
<gary_poster> Ursinha: hard to tell.  You are right about the query string certainly.  Not obvious that it is the problem though.  Trying to find implementation of find_matches...
<thumper> ok, I'm back
<rockstar> thumper, does that mean standup?
<thumper> abentley, mwhudson: standup?
<abentley> thumper: check out your CPU usage
<rockstar> abentley, I'll host.
<abentley> thumper: Odd.  I was getting noise stuff that went away when killed some processes.
<thumper> that is what it was like for me
<gary_poster> Ursinha: my guess is that it is different.  The traceback says it has trouble finding has_matches.  Sometimes that's hiding a lower problem, but the implementation of ``has_matches`` is simply ``return self.matches > 0``.  Maybe self.matches is a property that is being confused by the query string, but that's as far down the rabbit hole as I'm going right now.
<gary_poster> :-)
<gary_poster> so, new bug for now is my opinion
<Ursinha> gary_poster: all right, I'll file another :)
<Ursinha> thanks gary_poster
<gary_poster> np
<gary_poster> mwhudson: hi.  have you used ec2 test to land someone else's branch without merging it yourself?
<mwhudson> gary_poster: yes
<gary_poster> mwhudson: OK that means it is possible.  :-) I would have thought that it would puke because it would find something local to doublecheck.  I'll review code or you can give me a hint. ;-)  (if you don't have time for hint that's ok: the confidence that it should be possible is enough ;-) )
<mwhudson> gary_poster: it doesn't do any local checks if you do ec2 test <url>
<mwhudson> (as opposed to ec2 test .)
<gary_poster> mwhudson: ah-hah!  yes, I vaguely remember it starting that way.  ok thank you
<mwhudson> (i haven't changed anything in this area fwiw)
<gary_poster> cool, yeah, I'm just forgetful :-/
<mwhudson> np
<thumper> flacoste: where is the cherry pick instructions?
<thumper> s/is/are/
<flacoste> thumper: https://wiki.canonical.com/Launchpad/PolicyandProcess/EChangeProcess
<flacoste> that's the canonical page
<thumper> flacoste:
<thumper> flacoste: ta
<flacoste> but it's out-of-date with the current way
<thumper> flacoste: wheren't you going to update it?
<flacoste> thumper: i failed!
<thumper> :)
<rockstar> Wow, I just got the most un-helpful PQM failure message evar.
<gary_poster> thumper: is this the secret to lightweight branches?  I want to give them a try.  I don't quite understand the instructions I've seen elsewhere (including the top of this page).  https://edge.launchpad.net/bzr-lightweight-branch
<thumper> gary_poster: secret to what?
<thumper> gary_poster: what is the question?
<gary_poster> thumper: how do I use them.  bzr help branch doesn't mention them
<gary_poster> how do I create them
<rockstar> gary_poster, bzr cbranch is your friend.
<rockstar> gary_poster, you'll need bzrtools
<thumper> gary_poster: I use cbranch from bzrtools
<gary_poster> thumper, rockstar: ah-ha!  reading that help, thank you
<rockstar> gary_poster, http://theironlion.net/blog/2009/01/23/more-advanced-bazaar-concepts/
<gary_poster> rockstar: :-) awesome, thank you
<jml> mwhudson, 9h vs 15m for real?
<mwhudson> jml: yes
<mwhudson> jml: i was a little surprised
<jml> mwhudson, bloody hell
<mwhudson> jml: hey, look at my branch: http://pastebin.ubuntu.com/297218/
<mwhudson> (or go to bed, probably more sensible)
<lifeless> mwhudson: what did you change to do that?
<mwhudson> jml: the staging appserver is often really slow, i would guess production would be less terrible
<jml> mwhudson, it looks very much along the right lines
<jml> mwhudson, the store selection seems a bit unfortunate
<mwhudson> lifeless: the slow version made an appserver request to find out the on disk path for the branch data
<jml> mwhudson, and it would be nice to identify places (mostly in the test suite?) where a direct-to-db transport would be more appropriate
<mwhudson> jml: the scanner could use it, for sure
<lifeless> mwhudson: and the fast one uses the db?
<mwhudson> lifeless: yes
<lifeless> conclusion: our appservers are slow
<mwhudson> the fast one actually reused the gadget that i wrote for the similar problem with http branches...
<jml> lifeless, the staging one is, at least
<mwhudson> they all are pretty slow
<lifeless> jml: they are  all slow
<mwhudson> 100ms/request is good
<jml> lifeless, sure, but you can't conclude that from an experiment on the staging server :)
<mwhudson> uh
<lifeless> jml: I think you can
<lifeless> jml: because the load on staging is lower per server than on prod; and the db was the same in the two experiments
<jml> lifeless, introducing new facts doesn't count.
<lifeless> hah!
<jml> lifeless, also, hardware.
#launchpad-dev 2009-10-20
<lifeless> makes a difference, but not that much... certainly not multiple orders of magnitude
<jml> :D
 * mwhudson afk for lunch and a few errands
<kfogel> jml: you're up?
<elmo> all the cool kids in london are
<thumper> rockstar: still around?
<rockstar> thumper, yes, kinda.
<thumper> rockstar: quick skype for a question?
<rockstar> thumper, um, can it wait 5 minutes?  I'm trying to finish off some pre-kickoff chores.
<thumper> sure
<thumper> Monday night football?
<rockstar> thumper, yesh.   Broncos tonight.
<thumper> rockstar: nm, I'll muddle through, taking a break now for lunch
<rockstar> thumper, okay, ping me when you're back.  I'm ready now.
 * mwhudson reappears
<thumper> rockstar: if you have a few minutes that'd be great, if you don't we can talk tomorrow
<lifeless> https://code.edge.launchpad.net/ubuntu
<lifeless> 1  â 100  of 139453 results
<lifeless> nice!
<thumper> mwhudson: skype?
<mwhudson> thumper: ok
<mwhudson> thumper: i could hear you
<thumper> mwhudson: I couldn't hear you
<mwhudson> i'm not muted
<thumper> I was getting no sound out of skype
<mwhudson> ah
<wgrant> Does Skype ever Just Work?
<lifeless> it does for me
<lifeless> jaunty, 64bit
<mwhudson> it does for me, but i cheat and use os x
<mwhudson> jtatum: hello
<mwhudson> oops
<mwhudson> jtv: hello
<jtv> mwhudson: hi
<jtv> did I break something?
<mwhudson> jtv: did you get your problem sorted yesterday?
<jtv> mwhudson: ah, so it's not about that buildbot failure.  :)  Yes, jml helped me out.  Thanks.
<mwhudson> cool
<jtv> mwhudson: it turns out that if you create two branches in the same test with create_branch_and_tree, passing neither name nor location, the two will try to create the same directory and things go boom.
<mwhudson> jtv: ah yes
<jtv> Actually, even passing names doesn't fix it.  You need to pass different locations.
<jtv> It looks confusing because the conflicting dirs are .bzr dirs in the same temp directory.
<mwhudson> yes, it creates the tree in the cwd doesn't it?
<mwhudson> that's a bit naughty
<lifeless> mwhudson: not if its bzr's helper; bzr's creates it on get_transport(self.get_url(relpath))
<mwhudson> lifeless: it's not bzr's helper
<jtv> mwhudson: right, the default argument for location is "."
<jtv> And creating those branches on a transport was getting a bit too involved for a non-Code[hosting] test.  :)
<mwhudson> jtv: good luck avoiding becoming a code hacker :-)
 * mwhudson EODs
<jtv> mwhudson: thanks, gnight
<adeuring> good morning
<jml> kfogel, no, I wasn't. sadly I'm not as cool as elmo.
<jml> and every day, I have to live with that.
<mwhudson> hello jml
<jml> mwhudson, hi
<mrevell> Morning!
<bigjools> morgen
<al-maisan> Good morning mrevell
<al-maisan> .. and bigjools :)
<mrevell> Hallo
<jtv> bigjools: thanks to you, bfb is tla #23478 in the gtf: http://xs4all.nl/~jtv/gtf/
<mup> Bug #23478: Breezy won't recover from hibernation on ThinkPad T42 <linux-source-2.6.15 (Ubuntu):Fix Released by ben-collins> <https://launchpad.net/bugs/23478>
<jtv> mup: shut up, this isn't about bugs.
<bigjools> jtv: !
<bigjools> \o/
<jtv> bigjools: so if you weren't already, you are now in the gcp
<jtv> it comes with a neat little button for your home page.
<bigjools> woo
<jtv> this puts you in an exclusive club with the likes of dek
<jtv> ...insofar as the great dek has likes.
<deryck> Morning, all.
<jml> deryck, good morning
<henninge> danilos: your suggested fix for bug 128324 sounds reasonable
<mup> Bug #128324: translator-credits remains "untranslated", so the percentage is < 100% <Launchpad Translations:Triaged by henninge> <Launchpad Translations 2.2:Won't Fix> <https://launchpad.net/bugs/128324>
<jml> al-maisan, assigning bug 347768 to you
<mup> Bug #347768: Allow anyone with upload rights to write to a package branch <package-branches> <Launchpad Bazaar Integration:In Progress by al-maisan> <https://launchpad.net/bugs/347768>
<danilos> henninge, glad you like it :) anyway, I'd be happy to pair with you on it, and we can split the work up into several branches
 * jml stepping out for a bit to get lunch & stationery.
<gary_poster> BjornT: call?
<sinzui> barry: EdwinGrubbs, bac: standup in 2 minutes
<EdwinGrubbs> sinzui: What do you think about a generalized css class like this? http://pastebin.ubuntu.com/297533/
<sinzui> EdwinGrubbs: does it have to be a UL, can I have a <div> or a <dl>
<EdwinGrubbs> sinzui: it doesn't have to be UL, but the only situation that I can think of that would benefits from DIVs would be two rows of action links, which seems unlikely.
<sinzui> EdwinGrubbs: We should not design CSS for the specific. The reason the old CSS is giant is that we had to add duplicate rules because we could not reuse rules.
<EdwinGrubbs> ok
<sinzui> EdwinGrubbs: We do not need to define an element until we know there is a conflict
<sinzui> EdwinGrubbs: I think this is a good rule.
<EdwinGrubbs> sinzui: I'll make the change.
<EdwinGrubbs> sinzui: There is one other problem that I keep ignoring. If the table doesn't exist because there are no entries, then I either have to remove the table-actions class or add style="float: right" on the element. Either way, it seems to break the generalized nature of the css class. It seems like I just have to choose one ugly solution or another.
<sinzui> EdwinGrubbs: That is a common problem without a solution in our design
<sinzui> EdwinGrubbs: witness the issue of a side bar without portlets
<sinzui> EdwinGrubbs: I think you need to look ahead to learn if there will be content.
<sinzui> EdwinGrubbs: I looked for a CSS3 selector that allowed me to ask if the element has children, I did not find one. maybe beuno knows
<beuno> sinzui, I don't recall anything like that in CSS3
<EdwinGrubbs> sinzui, beuno: here is a working css solution that seems a lot better. http://pastebin.ubuntu.com/297595/
<EdwinGrubbs> sinzui, beuno: I use text-align instead of float so that the UL will not end up outside the border of its container. This works since the LI is already set to display:inline.
<sinzui> EdwinGrubbs: : I award you a gold â for cleaverness
<jml> cleaverness?
<gary_poster> I'd make a joke, but I'm not sharp enough.  uh.  heh?
<sinzui> malapropism.  You can take a cleaver to a solution fit the problem. It can also be clever at the sametime
<jml> gary_poster, :D
<beuno> EdwinGrubbs, rockin solution
<flacoste> sidnei: your lazr-js branch has a huge diff, i think it builds on another unlanded branch?
<sidnei> flacoste: correct. here's the diff: https://pastebin.canonical.com/23563/
<gary_poster> sidnei: ah ok.  So that's Bjorn's branch, I assume?  Has his been reviewed?  If so, is it ready to land?  Would waiting for his branch to land before a review make sense?
<sidnei> gary_poster: you reveiwed bjorn's branch. it's marked as approved, supposedly ready to land. i can wait for his branch to land yes.
<gary_poster> sidnei: it's your call, and the reviewer's call I guess.
<flacoste> sidnei: add that pastebin link to the m+p, and I'd suggest also explaining what widgets.conf and src-js/yui-path.js are for
<sidnei> flacoste: ok
 * rockstar lunches
<jml> g'night folks.
<leonardr> allenap, deryck, yesterday you reminded me of a branch that was landed to lazr.restful and that launchpad should have by now. the branch fixed a unicode-related bug. do you remember the bug number for that?
<deryck> leonardr, bug #331990
<mup> Bug #331990: The inline editor widget reports a JSON error when saving non-ASCII characters <javascript> <Launchpad Foundations:Triaged> <lazr.restful:Fix Released by leonardr> <https://launchpad.net/bugs/331990>
<leonardr> deryck, thanks
<deryck> leonardr, np
<jml> thumper, ping
<beuno-lunch> deryck, ping
<beuno-lunch> gar
<beuno> deryck, intellectronica, jml, how controversial do you think moving bugtask tables beneath the description is?
<beuno> why am I experimenting with that, you say?
<beuno> well
<beuno> it reads better
<beuno> it's more consistent with the rest of the site
<beuno> makes people read descriptions
<intellectronica> beuno: the problem is, the height will always vary
<beuno> sure, if you're familiar with the bug, it sucks a bit
<deryck> hmmmm
<beuno> yes, it's less predictable, although we probably need to have a max-height, and let people expand
<jml> beuno, have you talked with mpt at all about the bug page design?
<beuno> jml, I have not
<intellectronica> beuno: even if you're not familiar with the bug, but familiar with LP, under the current scheme, the controls for changing the bug task are always in the same location. if the description pushes them to a different location every time it will be harder to hit them
<beuno> hm
<intellectronica> unless we make the description fixed height
<jml> beuno, there's a lot of history. You shouldn't be bound by it, but it might be good to know which bits of it you are ignoring :)
<deryck> beuno, I think this will be quite controversial, just because it's such a big change.  and heavy users are soooo used to looking top for the needed info.
<beuno> yes, some people will be pissed
<beuno> but lets think about that after we figure out what the best thing is
<deryck> but will *all* people be pissed, that is the question. ;)
 * deryck kids
<beuno> then we can think about how to make everyone as happy as we can
<jml> beuno, it's after hours for me and I have my happy Lean hat on
<intellectronica> b.t.w i think fixed height for the description is definitely an option (assuming it's quite short)
<salgado> sinzui, around?
<intellectronica> on most visits to the bug page you don't read the description
<jml> beuno, 1. Figure out the constraints.
<jml> beuno, 2. Come up with two or three solutions that meet those constraints.
<jml> beuno, 3. Profit.
<beuno> intellectronica, so leave white space for very short ones?
<sinzui> salgado: I am at this moment
<deryck> intellectronica, beuno -- so if you don't read the description often, why top post it?  uniformity?
<beuno> jml, well, I am loosely doing that, but maybe I should do it more formally. If only I had time to play with this more
<beuno> deryck, well, you don't interact with the same bug a lot very often either
<beuno> so, for new bugs, you do interact with it
<beuno> of course, if you use email, you will likely just go there to set statuses and such
<beuno> which is probably why intellectronica has that feeling
<jml> beuno, the slightly more formal approach might save you time.
<beuno> but if you live on the web ui, you read them most of the time
<salgado> sinzui, we're getting http://paste.ubuntu.com/297719/ when trying to run LP on python2.6, but I don't think we have a new twisted release with that fixed, so we're hoping to silence the fascist for twisted imports.  for now, at least
<intellectronica> actually, deryck playing back my argument at me reminds me that as a user i'd hate that change. i almost never work with the description and i always want to quickly look at the bug tasks
<salgado> sinzui, barry told me you know the fascist, so could help us figuring out how to silence it
<beuno> jml, I have 2 hours left, but maybe you're right, I don't know
 * sinzui thinks
<sinzui> salgado: I do, Let me try to remember this
<deryck> beuno, yeah, as intellectronica said, I don't read descriptions first often.  I look at the table first.  and I do use the web UI mostly.  but maybe that's function following form, too.
<intellectronica> beuno: and an expanding description is kinda' against what we're trying to achieve with the ajaxafication. you will now need an extra click when you do need to read the description
<jml> beuno, if you do the constraint stuff, then it helps others to come up with designs. I'm torn between not wanting to derail you, and thinking that it's a bad idea to do a rush job of the single most important page on Launchpad :)
<beuno> deryck, so what I'm going to do is, propose something crazy, write up the constraints I can think off, and hand it off to your team while I deal with ubuntu.com and the karmic release
<beuno> jml, I'm not thinking I can solve the problem, bue maybe I can kick-start it
<jml> beuno, cool.
<deryck> beuno, ok, but I'm a bit like jml and wonder what the value here is.  maybe we should just table a redesign, and let us do simple bug fix improvements until we can think this through more.
<beuno> jml, so I want to propose something more radical than polish
<jml> deryck, table means something different here :)
<deryck> heh
<intellectronica> beuno: also, not wanting to take the wind out of your sails, i must also add that there are many small things we need to fix on the bug page, and it's very likely that fixing them is much more bang for the buck than pretty much any new design we can possibly do right now
<deryck> jml, postpone?
<beuno> deryck, what's of most value to you for me to work on?
<beuno> intellectronica, great information. What would help you guys the most then?
 * deryck is thinking....
<deryck> beuno, how about a decision on open questions, rather than a mockup....
<jml> beuno, I want you to do something more radical than polish too :)
<intellectronica> beuno: i definitely need a bit of help deciding how to fix the top row of the bugs index page. that would really help me (that's a different matter, but since you asked)
<beuno> deryck, if you point me at them, I will spend these 2 hours replying to that
<deryck> beuno, mainly I'm think bugtask table stretch the top or not, and how to make the changing in statuses less easy?
<deryck> beuno, let me get bug numbers.  intellectronica, do you have bugs with open questios you can think of?
<beuno> intellectronica, the search box, etc?
<beuno> jml, want to schedule a call for that then? to start brainstorming?
<jml> deryck, http://www.economist.com/research/styleGuide/index.cfm?page=673901 (search for "table")
<jml> beuno, please do.
<intellectronica> beuno: yes
<intellectronica> beuno: https://bugs.edge.launchpad.net/malone/+bug/433854
<mup> Bug #433854: Bugs home page: search box and project details don't fit on the same line <ui> <Launchpad Bugs:In Progress by intellectronica> <https://launchpad.net/bugs/433854>
<beuno> intellectronica, ok, that's on my list now, I will comment on the bugf
<beuno> bug as well
<deryck> jml, interesting.  I was proposing postpone as an alternate to table? to suit the US usage.
<intellectronica> beuno: awesome, thanks
<beuno> deryck, so, as for the bugtask table, everytime I think about, I get the feeling we need to do something different with it
<jml> deryck, oh yeah, that'd be fine.
<beuno> deryck, which makes me need to change other parts
<sinzui> salgado: there I have two hacks in utilities/importfascist.py
<jml> deryck, I was just adding the link for interest.
<beuno> which makes me ask questions like whether to move it below the description  :)
<intellectronica> beuno: i think that we need to align the tasks table, description and comments, and that when we do that it will improve things considerably
<sinzui> salgado: I think you wan to use warnings.filterwarnings from line 16 as your guide to ignore
<sinzui>     DeprecationWarning: the md5 module is deprecated; use hashlib instead
<deryck> beuno, ok, fair enough. we're stuck. :)  but assuming we can't get a redesign this cycle, should the table stay where it is now, or move up above the two columns?
<beuno> deryck, intellectronica, it's interesting, because if I look at random bugs, the thing that itches the most is comments
<deryck> itches?
<beuno> espcially activity
<beuno> they use up too much horizontal space
<beuno> they don't collapse when doing multiple actions at the same time
<beuno> it's messy and hard to understand
<deryck> right
<intellectronica> yeah, that really sucks
<beuno> so, if it was up to me, I'd work on that and tweak the bugtask slightly to address the problem with long names in the objects
<beuno> to me, that's the biggest improvement I can see
<intellectronica> but the ui solution is (i think) obvious. we just need to get them to collapse like bug mail
<beuno> making conversations easier to understand and follow
<salgado> yeah, I think that's the only way.  even if I get the fascist out of the way we'd still get an error because our test runner tells python to convert warnings into errors
<salgado> sinzui, ^
<sinzui> yep
<sinzui> but there is a test that hacks around this.
<salgado> sinzui, thanks for the help, I'll continue the chasing in #launchpad-sprint with the others
<sinzui> Or ther ewas a test
<deryck> beuno, intellectronica -- that sounds a reasonable plan to me.
<beuno> intellectronica, yes, get them to collapse, especially with the comments, and, if that's out of scope for this cycle, make them use up less space
<beuno> but if you can make them collapse, I'm super happy
<beuno> and I can propsoe a few small tweaks to the bugtask table so it has a bit more space
<beuno> for example, the importance column does not need to be so wide
<deryck> beuno, intellectronica -- I would rather finish our work already assigned this week and next, not focusing on the bug page too much, then use week 4 to have everyone on bugs team on the bug page, and land something nice week 1 of next cycle.
<beuno> deryck, I'm cool with that, when would you need me then?
<sinzui> salgado: look in canonical/__init__.py. I puth something in there during the zope upgrades
<salgado> hmm
<deryck> beuno, how about we finish this dialog now or in email with your suggestions, then we do the work, and you take a look mid-to-late of week 4 at how we're doing?  see if something else springs to mind.
<deryck> intellectronica, does this type of plan seem reasonable to you as well?
<intellectronica> yes, absolutely
<deryck> ok, cool.
<beuno> I'm on board as well
<deryck> ok, cool. thanks guys!
<beuno> deryck, I will write up that email now
<deryck> beuno, awesome, thanks!
<beuno> jml, I want a bit overboard, but I sent an email of what was floating around in my head
<jml> beuno, yay
<beuno> left out the constraints because I felt people where going to derrail the email with their own crazy constraints
<thumper> morning peoples
<beuno> so I'd like that to be a separate thread when we have a better idea of when and what we'll be doing
<thumper> beuno: got time for a talk?
<beuno> hiya thumper
<beuno> thumper, yes, I haven't started doing what I'm supposed to be doing yet
<jml> thumper, quick question
<thumper> beuno: I have 15m before the standup
<thumper> jml: shoot quickly
<jml> thumper, any objection to me adding IBranchLookup.getByUrls and exposing it over the API?
<thumper> jml: none at all
<jml> thumper, yay.
<jml> thumper, I think I'm going to make a screencast of "how to patch Launchpad to expose an API"
<jml> thumper, and that would fit well, being simple and extremely useful to me personallye.
<beuno> thumper, I'm bringing up skype
<rockstar> jml, I had been thinking about that for some time.
<jml> rockstar, getByUrls or the screencast?
<rockstar> jml, the screencast/howto about exposing things through the API.
<jml> rockstar, cool. it's a good way to get people started hacking on Launchpad, I think.
<rockstar> jml, yea, it's pretty easy.  I think the testing infrastructure for the API could be a little better though.
<jml> rockstar, patches accepted :)
<rockstar> jml, really?  :)
<rockstar> jml, as I recall, it was more a philosophical debate that prevented patches from being made.
<jml> rockstar, oh well, philosophy accepted.
<beuno> thumper, https://code.edge.launchpad.net/~mwhudson/launchpad/in-memory-launchpad-server/+merge/13613
<mwhudson> hello
<mwhudson> that branch failed in ec2 test in the strangest way
<mwhudson> but i guess you're talking about merge proposal pages in general :-)
<beuno> mwhudson, hi
<beuno> yes
<beuno> the layout has something broken-ish
<beuno> the "commit message: label and the actual message are jumbled together
<thumper> abentley: skype?
<thumper> beuno: yes, just one of the things I want to fix
<rockstar> thumper, was just about to ask you the same thing.
<beuno> thumper, filing 2 bugs then
<abentley> thumper: Do you see me?
<abentley> thumper: I'll try hosting again.
<rockstar> I think thumper was the only one who couldn't hear any of us.
<rockstar> The benefit of Skype is that you don't need the T-Pain plugin.  Everyone already sounds like him.
<al-maisan> hello jml, I am considering tackling bug #401525, what do you think?
<mup> Bug #401525: Revision.getBranch is not source package aware <feeds> <package-branches> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/401525>
<jml> al-maisan, I think you should ask thumper
<al-maisan> ah, I see.
<beuno> thumper, it is pretty easy to get counts of files and dirs in bzr
<al-maisan> build https://lpbuildbot.canonical.com/builders/lp/builds/257 failed with:
<al-maisan>     OperationalError: could not write to file "base/289007/2691": No space left on device
<al-maisan> I understand Gavin forced it once already .. how can I find out more about the status of that build?
<allenap> al-maisan: That's a tricky one. The build slaves shut down shortly after a failure iirc, so it's difficult to know what's going on.
<al-maisan> allenap: OK .. so what is the next step?
<allenap> al-maisan: Something to check is if the db setup enables statement logging or not. I'll look into that in the morning.
<allenap> al-maisan: If the instance can be caught soon after the failure, a losa can connect to it and see what's filling it up. I think.
<al-maisan> allenap: I see.
<allenap> al-maisan: If you see it happen, ask a losa immediately.
<al-maisan> allenap: thanks -- will do.
<kfogel> +1 (+N, really) on jml's idea to make a screencast showing how to patch Launchpad to expose an API.
<kfogel> maybe I should say that to jml :-)
<jml> kfogel, glad to hear it.
<jml> kfogel, I was going to write an email to you & mrevell asking for help, but I got bored halfway through. :)
<jml> anyway, I'm going to rot my mind & teeth with video games and chocolate.
<mwhudson> ok this is screwy
<mwhudson> ./bin/test -vvct lp.code.model.tests.test_branchlookup.TestGetByLPPath fails
<mwhudson> ./bin/test -vvc lp.code.model.tests.test_branchlookup TestGetByLPPath
<mwhudson> passes
<mwhudson> i guess i need to look at test_suite() functions
<thumper> that is weird
<thumper> jml: how would you feel about a db patch that adds file and directory counts to the branch table?
<rockstar> mwhudson, what does -c do?
<mwhudson> rockstar: colorization
<kfogel> jml: I didn't think you were still here.  Go do something healthy like play video games, but yeah, a walk-through of exposing APIs would be a huge enabling thing for people -- e.g., lfaraone could use it, thinking back to our conversation last night.
<rockstar> mwhudson, ah.  I use -vvt to go all the way to a specific test by module all the time.  I'm not sure why that would be failing.
<mwhudson> rockstar: the difference is that if you say ./bin/test -vv module it doesn't load all the test modules
<rockstar> mwhudson, yeah, so you can specify the module, but not the test class.  At least, that's what I've seen.  Specifying -t means you can specify the class.
<mwhudson> rockstar: you can also say -vv module class
<rockstar> mwhudson, ah, okay.  So -t only loads SOME test modules then?
<mwhudson> rockstar: no, other way around
<mwhudson> -t loads all, then filters
<mwhudson> without -t, it loads some, then filters
<rockstar> mwhudson, ah, okay.
 * mwhudson has the feeling that he's going to be very annoyed when he gets to the bottom of this
<mwhudson> circular imports :(
<thumper> d'oh
<mwhudson> it's really fucking weird
<mwhudson> oh wow, found it
<mwhudson> basically, my branch accidentally makes lp.code.model.branchlookup imported at test-loading time
<mwhudson> and this module does adapter registration at import time
<mwhudson> and test discovery time is too early for this to work
<wgrant> Adapter registration at import time? Ewww.
<mwhudson> (in trunk i guess the module is only imported when the component architecture is set up and things work)
<mwhudson> wgrant: i'm going to blame jml's hatred of zcml i think
<wgrant> Grok!
<mwhudson> yes
<wgrant> Well, Martian.
<thumper> wgrant: go on then, submit a patch to base launchpad on grok!
<wgrant> ME GROK SMASH THUMPER
<thumper> mwhudson: has autocomplete on alt-/ stopped for you in emacs 23?
<mwhudson> thumper: no
<thumper> :(
<mwhudson> thumper: C-h c M-/ says what?
<thumper> M-/ runs the command pop-tag-mark
<mwhudson> well that explains that i guess
<mwhudson> it runs dabbrev-expand for me
<thumper> gah
<mwhudson> pop-tag-mark is on M-* for me
<thumper> found it in my .emacs :(
#launchpad-dev 2009-10-21
<mwhudson> time for lunch
<thumper> mwhudson: my work at the moment reduces code.launchpad.dev/mint from 350 queries to 150
<thumper> mwhudson: but that is because the development focus isn't worked out properly
<mwhudson> thumper: progress!
<mwhudson> thumper: or you mean, it's fast because it's wrong?
<thumper> mwhudson: it is just the listing class trying to determin dev focus
<thumper> mwhudson: right now it preserves the existing behaviour
<thumper> mwhudson: without the extra queries
<mwhudson> the existing behaviour being "lp:ubuntu/karmic/bob" rather than "lp:ubuntu/bob"
<mwhudson> ?
<thumper> yeah
<thumper> that's the next bit to fix
<mwhudson> i see
 * mwhudson afk for a few minutes
<thumper> mwhudson: got it from 350 down to 50
<thumper> mwhudson: now just to make it right :)
<mwhudson> thumper: cool
<thumper> I think I have it
<thumper> 59 queries
<mwhudson> thumper: for how many branches?
<thumper> 72 queries for 130 branches
<thumper> and it gets a nice star :)
<mwhudson> thumper: that sounds better than the product branch listing currently
<thumper> mwhudson: none of them have revisions :)
<mwhudson> thumper: ah
<thumper> mwhudson: I have a feeling that is where some of the extra queries are getting soaked up
<mwhudson> certainly could be
<thumper> mwhudson: but one thing at a time
<mwhudson> yep
<thumper> mwhudson: this is almost ready to submit \o/
 * thumper found missing test coverage
<thumper> mwhudson: has local launchpad sucked arse since you upgraded to karmic?
<thumper> mwhudson: I'm seeing massive cpu spikes
<mwhudson> thumper: haven't noticed anything odd
<thumper> :(
<thumper> when do we get memcached?
<mwhudson> !!??!?!?!
 * mwhudson finds that the sftp oopses are a twisted.conch bug that _i fixed_ over a year ago
<thumper> ?
<mwhudson> i guess we haven't updated our twisted in at least that long
<mwhudson> thumper: https://bugs.edge.launchpad.net/launchpad-code/+bug/376279
<mup> Bug #376279: AssertionError: still have data in FSETSTAT:  <codehosting-ssh> <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/376279>
<thumper> mwhudson: twisted time
<thumper> ?
<thumper> mwhudson: is twisted handled through buildout?
<mwhudson> thumper: no, it's still in sourcecode
<thumper> mwhudson: should it be buildout?
<mwhudson> thumper: probably
<mwhudson> i don't know if it would work
<thumper> hmm..
<mwhudson> huh, Twisted 8.2.0 is in the download cache
<mwhudson> but not in versions.cfg
<thumper> perhaps jml started
<thumper> mwhudson: the slowness was firebug
<thumper> mwhudson: having too much enabled
<mwhudson> thumper: ah
<thumper> *sigh*
<thumper> lazr.editor isn't working properly
 * thumper goes hunting for the actual source
<thumper> mwhudson: I have an amost working inline commit message editor
<mwhudson> thumper: yay
<thumper> using the same (ish) code as bug descriptions
<thumper> mwhudson: 2 issues though
<thumper> 1) accept_empty doesn't work
<thumper> 2) needs id="edit-description" for all the styling
 * thumper files bugs
 * thumper finds it isn't lazr-js
 * mwhudson eods
<adeuring> good moring
<al-maisan> moin adeuring
<adeuring> hi al-maisan!
<mrevell> Morning!
<jtv> mrevell: you may be interested to know that we have translation import from branches running on staging now.  At last people can do real testing there without admin help!
<mrevell> jtv: Ah, I am interested! Would you like to write a blog post about it?
<jtv> mrevell: yeah, I guess that's worth while
<jml> hello
<thumper> hi
<thumper> does anyone know how many lines of code there is in launchpad at the moment?
<jml> thumper, not off the top of my head.
<jml> thumper, http://paste.ubuntu.com/298097/
<thumper> jml: that includes the tests too right?
<jml> thumper, yes.
<thumper> jml: but is probably doesn't count the doc tests
<jml> thumper, it's just the output of sloccount
 * thumper nods
<deryck> Morning, all.
<jml> morning
<jml> anyone got balsamiq running on Linux?
 * jml is having some trouble.
<intellectronica> jml: yeah, i do
<intellectronica> jml: is it only balsamiq you've got a problem with. did you try other air apps?
<jml> intellectronica, no, I haven't tried other Air apps.
<jml> this is the first one :)
<jml> intellectronica, does Air have a command line I can use to try to run balsamiq?
<jml> as in, run the baslamiq installer (or whatever it is?)
<intellectronica> jml: not that i know. i don't remember how i got it installed
<intellectronica> jml: so, you can't even get it installed?
<intellectronica> jml: did you follow http://www.balsamiq.com/products/mockups/help#linux ?
<jml> intellectronica, I am following it now.
<jml> intellectronica, Step 3 never happens.
<intellectronica> jml: no idea. it "just worked" for me. you should try the balsamiq team - they're a pretty dedicated bunch
<jml> hmm.
<jml> intellectronica, are they on IRC?
<intellectronica> jml: no, but they are on jabber and skype
<jml> intellectronica, heh, ok. thanks.
<jml> ahh, here we go, found the air app installer
<jml> Error loading the runtime (libadobecertstore.so: cannot open shared object file: No such file or directory)
<jml> intellectronica, are you 32 bit or 64 bit?
<intellectronica> jml: 32
<jml> intellectronica, that'll be it :)
<intellectronica> orli? no air for linux 64-bit?
<jml> intellectronica, complicated air for 64 bit
<jml> intellectronica, I had to manually copy a library from /usr/lib to /usr/lib32
<intellectronica> eugh
<intellectronica> it's so hard to get excited about proprietary software, isn't it, even if you really try hard
<bigjools> just the games :)
<intellectronica> bigjools: games are not software
<intellectronica> as in "candy isn't food"
<bigjools> so games are what?
<jml> candy
<jml> executable polygonous candy
 * jml lunches
<sinzui> EdwinGrubbs: barry: bac: standup in 2 minutes
<barry> reviewers, beuno, lurkers, developers -> #launchpad-meeting in 3m
<EdwinGrubbs> james_w: ping
<james_w> hi EdwinGrubbs
<EdwinGrubbs> james_w: regarding bug 436507, would it be helpful to change the portlet name from "Releases in Ubuntu" to something else to indicate that you are in the distro context?
<mup> Bug #436507: Distribution series source package page has no links to distribution source package page <post-3-ui-cleanup> <Launchpad Registry:Triaged by edwin-grubbs> <https://launchpad.net/bugs/436507>
<james_w> EdwinGrubbs: I'm not sure I understand the question?
<EdwinGrubbs> james_w: well, in the bug description, you mentioned that you can change to the distro context with the "Releases in Ubuntu" links, but that it was not obvious. Are those links good enough, but just confusingly named, or should there be a different link to get you to the right place?
<james_w> well, I generally find myself dissatisfied with the distro/distroseries split in these pages. I'm not sure that just renaming that will help much.
<james_w> though I'm not sure how I would phrase a link to the distro context either
<james_w> do you have some suggestions for alternative wording?
<EdwinGrubbs> james_w: maybe, "Distribution source package for hotssh" or "Ubuntu source package for hotssh". What exactly is the difference between karmic's source package and ubuntu's source package? Are they shared? Do they have extra patches for each distro?
<james_w> well, ubuntu's source package is an overview of what is going on in each release
<james_w> the /karmic/ page shows you just the information for that release
<james_w> compare https://edge.launchpad.net/ubuntu/karmic/+source/hotssh and https://edge.launchpad.net/ubuntu/+source/hotssh
<james_w> I prefer to work in the latter, but LP likes to send me to the former
<james_w> also compare https://edge.launchpad.net/ubuntu/karmic/+source/hotssh/0.2.6-2 and https://edge.launchpad.net/ubuntu/+source/hotssh/0.2.6-2
<james_w> I can see why you might want the information for a single release, but LP linking to it tends to mean that I am on the page with less information
<james_w> and I think the distinction will be lost on many. They won't realise the different contexts, and so won't know to switch between them. That's why I think renaming the heading wouldn't help that much.
<EdwinGrubbs> james_w: it is interesting that /ubuntu/+source/hotssh has the heading "hotssh package in Ubuntu" instead of saying "source package"? Would it be better to combine the distro and the series pages, or am I missing something about their purpose?
<james_w> I believe bigjools has even tried to do that
<bigjools> I did
<bigjools> I was on a road to Fail
<bigjools> because it buggers up source package branches
<jml> \o/
<EdwinGrubbs> bigjools: is one of those pages a soyuz page and the other a registry page? The split confuses me.
<bigjools> EdwinGrubbs: they're both registry
<EdwinGrubbs> bigjools: oh, you have thwarted my attempt to make this someone else's problem.
<bigjools> :)
<bigjools> they used to be soyuz!
<EdwinGrubbs> bigjools: do you think it would be less likely to fail if I just moved a bunch of the /ubunt/karmic/+source/hotssh info into the /ubuntu/+source/hotssh page, and the other page is only used when it absolutely has to?
<bigjools> EdwinGrubbs: yeah that would work - be mindful of excessive queries and page load times though
<bigjools> EdwinGrubbs: but I think james_w is more concerned about other pages that link to the DSSP page, they might be more useful linking to the DSP page
<EdwinGrubbs> bigjools: the /ubuntu/+source/hotssh page already uses ajax to expand to show info, so the distroseries pages' info could just be dynamically loaded if they expand it.
<EdwinGrubbs> bigjools: are there pages that should continue to point to DSSP no matter what, i.e., should I just convert links on a page by page basis or try to convert all the links from DSSP to DSP?
<bigjools> EdwinGrubbs: we'd have to analyse each I think and see what's most useful
<bigjools> because if we lose all the links then how do you get to the page? :)
<EdwinGrubbs> bigjools: it all depends on whether that page has any info that isn't moved into the DSP page.
<bigjools> EdwinGrubbs: yes, I ajaxified that page when I redesigned it, but we need to be careful not to overdo that
<james_w> fwiw this would mostly satisfy me, I rarely go to the DSSP pages out of choice, and rarely need any information there. If LP stopped sending me to them I would stop getting frustrated that it apparently doesn't want to let me out of that context.
<bigjools> EdwinGrubbs: the reason it's there is so that you can click on a tab and change facet to Code
<bigjools> otherwise I'd be real happy to blow it away
<EdwinGrubbs> james_w: which pages have links that send you to the DSSP pages most often?
<james_w> let me think, there is one in particular
<james_w> +publishinghistory does, and going to "Overview" from branches does
<james_w> I understand why there
<james_w> and I'm sure that the latter can be blamed on me
<bigjools> ha, the latter is the reverse of my last justification to keep the page
<bigjools> it's because it's in the same context
<EdwinGrubbs> james_w: what is the full link for +publishinghistory?
<james_w> https://edge.launchpad.net/ubuntu/+source/kerneloops/+publishinghistory
<bigjools> that's a soyuz page
<james_w> it does it because it actually has lines for the publication of each version in each release it is in
<bigjools> indeed
<james_w> so going to the version+distroseries page is probably the right thing to do
<bigjools> there's a separate publishing history for a series
<EdwinGrubbs> james_w: am I missing something, or are all the links pointing to DSSP still need to do that? Would extra links help on those pages, or is the best thing just to put a very prominent link on the /ubuntu/karmic/+source/hotssh page pointing to /ubuntu/+source/hotssh as discussed before?
<james_w> possibly
<james_w> I imagine this is a symptom of a deeper issue, but I kept bumping in to it, and so wanted to report it
<james_w> I think a link would be a good start, I just don't know how to phrase it
<EdwinGrubbs> james_w: in some ways, it would be nice to put the link in the breadcrumbs below the <h1>, but I'm sure I would get complaints about that since it doesn't match the url hierarchy. The links could be called "All source packages in Ubuntu"
<james_w> you mean "9base, aalib, ..."?
<EdwinGrubbs> james_w: really, it should be "All source package in Ubuntu for hotssh"
<james_w> that could work
<EdwinGrubbs> james_w: do you think it would work to put that link to the right of the "View changelog" link?
<james_w> I think that link can appear multiple times?
<james_w> hmm, no, that seems odd
<james_w> but in that case, yeah, I think that would be appropriate
<james_w> something on the version pages would be good as well
<james_w> they are even harder to escape from
<james_w> oh, no, the breadcrumbs can take you to the non-version pages
<EdwinGrubbs> james_w: would you also want a link on the version page that would take you to the DSP page in addition to the DSSP link in the breadcrumbs?
<james_w> I would like it I think
<james_w> but given that the other link would give a way out, it's not critical
<bigjools> EdwinGrubbs, james_w: FWIW, earlier this year we tried to redesign the navigation for these pages, and we came up with something nice that then failed a bunch of simple use cases.  It's hard :/
<james_w> yeah, I can appreciate that
<bigjools> and there's a lot of resistance to change
<rockstar> jelmer, ping
<beuno-lunch> jml, can our call be after lunch?
<jml> beuno-lunch, before or after the TL call?
<beuno-lunch> jml, before
<jml> beuno-lunch, sure
<beuno-lunch> I'll be lunched in 30'
<beuno-lunch> great
<jml> james_w, can you do me a favour and pastebin the output of 'time ssh bazaar.launchpad.net'?
<james_w> jml: ssh bazaar.launchpad.net 0.03s user 0.02s system 0% cpu 6.135 total
<jml> james_w, thanks.
<beuno> jml, ready when you are
<jml> beuno, I don't have skype, what's the best way to do this?
<beuno> jml, I can call you, you can call me
<maxb> jml: Hi. You mentioned something about adding licensing to a sftp server a while back - wondering if you have time to say more?
<jml> maxb, yes! I do!
<jml> maxb, 'bzr branch lp:txsshserver'
<maxb> \o/
<maxb> thanks
<jml> maxb, and also, https://bugs.edge.launchpad.net/txsshserver/+filebug
<maxb> heh
<jml> maxb, there was a long time between me making the branch and me open sourcing the branch
<jml> maxb, so it's slightly abandonware, but I'm happy to hack more on it if someone finds it useful
<maxb> What is that Launchpad page which tells you the definitions of all the kinds of image sizes LP uses? badge, mugshot .... etc.
<salgado-sprint> +images, I think
<beuno> +graphics maxb
<salgado-sprint> or +graphics
<maxb> aha!
<maxb> thanks
 * jml is so gone.
<mwhudson> good morning
<thumper> morning
<Ursinha> barry: hi :)
<barry> Ursinha: hi!
<Ursinha> barry: here I am bugging you again about maillinglists oopses :) have you seen this one -> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1389XMLP50 ?
<barry> Ursinha: nope, that's a new one
<Ursinha> barry: do you mind to triage a bug when I create one?
<Ursinha> which will be pretty soon :)
<Ursinha> s/a bug/the bug/
<barry> Ursinha: sure
<Ursinha> barry: do you have a clue about that? I don't even know what to write in the bug summary :)
<barry> Ursinha: just say "OOPS in MailingListAPIView.holdMessage() and reference the oops page.  please add a 'mailing-lists' tag too
<Ursinha> sure, thanks a lot barry
<barry> np!
<Ursinha> barry: bug 457606
<mup> Bug #457606: UnknownSender OOPS in MailingListAPIView.holdMessage() <mailing-lists> <oops> <Launchpad Registry:New> <https://launchpad.net/bugs/457606>
<barry> Ursinha: got it, but i might not get to it until after our sprint ;)
<Ursinha> gary-sprint: when you have a minute :) can you take a look at bug 51843, please? thanks!
<gary-sprint> Ursinha: looking
<mwhudson> Ursinha: hi
<mwhudson> Ursinha: do you know why r9669 is on https://dev.launchpad.net/CodeTeamTestPlans/3.1.10 8 times?
<Ursinha> mwhudson: this is something weird that happens sometimes, it seems the script isn't able to record the last revision recorded in the file and so it adds again in the next run
<Ursinha> I wasn't able to reproduce the problem
<mwhudson> ok
<gary-sprint> Ursinha: reopened with explanation :-)
<Ursinha> gary-sprint: thank you very much :)
<gary-sprint> Ursinha: thank you!
<Ursinha> bug 331990
<mup> Bug #331990: The inline editor widget reports a JSON error when saving non-ASCII characters <javascript> <Launchpad Foundations:Fix Committed by leonardr> <lazr.restful:Fix Released by leonardr> <https://launchpad.net/bugs/331990>
<wgrant> (continuing the discussion from #launchpad) Why wasn't that CPed weeks ago?
<wgrant> The problem was rediscovered a couple of days after the release, and it is a really really bad bug.
<Ursinha> wgrant: I took a note about it and will discuss on tomorrow's production meeting
<jml> mwhudson, taking my product strategist hat off and elegantly throwing it onto the hatstand for a moment
<jml> mwhudson, a twisted web thing for serving branches would be very cool.
<mwhudson> jml: yeah
<jml> mwhudson, and done right, it might help other areas of Launchpad that have similar "store lots of stuff and serve it with cool names" needs.
<mwhudson> another thing would be something mod_python ish i guess
<jml> mwhudson, particularly combined with the sftp server.
<mwhudson> that might be less technologically exciting
<mwhudson> jml: i guess so
<mwhudson> abentley: your trick with pushing a 1.6 format branch worked, btw
<mwhudson> thumper: i just spammed a review at you, it's very simple
<thumper> ok
<lifeless> jml: isn't that 'all of lp'  ? (store stuff, serve with names)
<jml> lifeless, some more than others.
<wgrant> PPAs... branches.. what else?
<mwhudson> rockstar: https://bugs.edge.launchpad.net/launchpad-code/+bug/306290 is Fix Committed i think?
<mup> Bug #306290: OOPS in code-import page when missing the correct CVSROOT  <oops> <Launchpad Bazaar Integration:In Progress by rockstar> <https://launchpad.net/bugs/306290>
<rockstar> mwhudson, yes, thanksy.
<rockstar> Er, thanks.
#launchpad-dev 2009-10-22
<mwhudson> lunchtime
<rockstar> mwhudson, I'm going to request a review of you in just a bit.  You can get to it after lunch.
<mwhudson> rockstar: okay
<abentley> mwhudson: Cool.
<thumper> barry: hey, hows the 2.6 going?
<barry> thumper: sort of good ;)  i'm very confident we'll be on py2.5.  we have an intermediate branch that is zope toolkit + 2.5 and if that goes well, we'll concentrate on ztk + 2.6.
<thumper> cool
<thumper> what's the zope toolkit stuff focusing on?
<thumper> as in
<thumper> what is different
<thumper> ?
<barry> it gets us to a blessed combination of zope.* packages
<thumper> blessed by zope corp?
<barry> or the zope community
<thumper> what does it give us in reality?
<thumper> anything useful I would use?
<mwhudson> i guess it means things probably don't break in exciting and unexpected ways
<barry> i think that's right.  also, we had to do it if there was any hope of getting to py2.6
<thumper> mwhudson: I'm having a weird lazr-js issue
<thumper> barry: makes sense then I guess
<thumper> barry: how much work is the zope toolkit stuff?
<mwhudson> thumper: talking to anyone else is likely to be more use than talking to me :-)
<barry> thumper: a surprising amount.  we're making good progress, but api's have changed
<thumper> mwhudson: heh
<thumper> mwhudson: I was going to file bugs, and then chase flacoste
<mwhudson> thumper: good plan
<wgrant> How much ex-Z3 stuff does LP use from outside ZTK?
<thumper> fuckity fuck fuck
<thumper> mwhudson: I created a new branch and al
<thumper> mwhudson: but the remote side is packs
<thumper> mwhudson: and mine is 2a
<thumper> mwhudson: lazr-js that is
<mwhudson> thumper: doom, i think?
<thumper> gah
 * thumper makes a local packs repo
<mwhudson> well, maybe a diff and apply it to a local copy
<thumper> ah ffs
<thumper> I can't branch it into local....
<mwhudson> nope
<thumper> can I merge uncommitted from a different repo?
<thumper> I could uncommit
<thumper> then grab the changes
<mwhudson> worth a try
<wgrant> bzr merge --uncommitted?
<thumper> yep
<thumper> works
<wgrant> Although that might care about richrootiness.
<wgrant> I don't remember.
<thumper> ugly as sin
<thumper> mwhudson: I'd file a bug saying lazr-js is still in packs format, but it'd just be me fixing it
<mwhudson> thumper: most likely
<rockstar> mwhudson, thanks for the review.  How long are you going to be about?
<mwhudson> rockstar: probably quite a while yet
<mwhudson> (when i stop working i really need to write my kiwipycon pypy talk so i'll be by the computer...)
<rockstar> mwhudson, great.  I'll respond to your review in a bit.
 * mwhudson boggles
<mwhudson> does anyone recognise this error? http://pastebin.ubuntu.com/298762/
<mwhudson> oh nm
<mwhudson> (i still don't know what that's about, but i don't think it's my fault)
<rockstar> mwhudson, so, switching to get_transport(self.branch.getPullURL()) gives me: NoSuchFile: No such file: u'/tmp/tmpSIhIst/lp-hosted/~person-name9/product-name4/branch11': [Errno 2] No such file or directory: '/tmp/tmpSIhIst/lp-hosted/~person-name9/product-name4/branch11'
<mwhudson> rockstar: you need to pass hosted=True to create_branch_and_tree i think
<rockstar> mwhudson, ah.  So my test didn't ever really work.  :/
<mwhudson> well it did
<mwhudson> just the understanding about where the branch is was a bit off
<rockstar> mwhudson, so, when you lock_write, and then move the .bzr branch, you get: Exception exceptions.UserWarning: <exceptions.UserWarning instance at 0x284dd40> in <bound method _LockWarner.__del__ of <bzrlib.lockable_files._LockWarner object at 0x735f190>> ignored
<mwhudson> rockstar: yeah, i wondered if you might
<mwhudson> rockstar: i guess we should talk to a bzr developer about this
<rockstar> mwhudson, okay.  lifeless?
 * thumper EODs
<lifeless> ?
<lifeless> rockstar: you'll need to unlock the branch object
<lifeless> rockstar: and yes, there will be a race condition
<rockstar> lifeless, well, yeah, but we want to lock it so we can replace it.
<rockstar> lifeless, actually, I could probably get away with unlocking it right before rename('.bzr', 'backup.bzr')
<lifeless> yes
<lifeless> thats what I mean ;)
<rockstar> lifeless, gotcha.
<lifeless> it is a race condtion
<mwhudson> filesystems need to grow transactions already
<lifeless> but its tolerable small
<lifeless> upgrade has a similar one
<lifeless> you can't lock stock and barrel replace the locking mechanism
<lifeless> and have it keep locking
<lifeless> the bootstraps are not large enough
<rockstar> lifeless, yeah, it seems like the time between unlock and move is pretty trivial.
<rockstar> mwhudson, updated mp - https://code.edge.launchpad.net/~rockstar/launchpad/branch-upgrade-out-of-place/+merge/13747
<rockstar> I apparently also accidentally voted on my own proposal.
<spm> rockstar: the question then becomes - did you vote yay, or nay?
<rockstar> spm, obviously, I voted yay.  I'm my own best friend.
<spm> just checking. might have been .... interesting otherwise. ;-)
<lifeless> is the puller/mirror/scanner failing at the moment ?
<lifeless> https://code.edge.launchpad.net/~lifeless/bzr-builder/blocking has been pushed too but claims nothing
<mwhudson> lifeless: graph looks ok to me, spm could check more closely though
<spm> looking...
<spm> interesting. the puller* is working fine afaict; but I can't find any matches in it's log for 'bzr-builder/blocking'
<lifeless> well its not even showing 'updating this branch'
<spm> and only 1 match for 'bzr-builder'
<lifeless> :!bzr push lp:~lifeless/bzr-builder/blocking
<lifeless> No new revisions to push.
<lifeless> :!bzr info lp:~lifeless/bzr-builder/blocking
<lifeless> Repository branch (format: unnamed)
<lifeless> so there is data there
 * lifeless tries hitting it harder
<spm> a bigger hammer usually works - vs just harder. ??
<lifeless> no change
<lifeless> can you manually trigger a mirror?
<spm> not sure I understand your meaning? to manually trigger?
<lifeless> the scanner and puller
<lifeless> for this branch
<lifeless> mwhudson: any ideas about how to recover [short of rming the branch]
<mwhudson> lifeless: something that will lock and unlock it
<mwhudson> lifeless: (i don't think push with no revisions to push does that)
<lifeless> it does
<lifeless> we take a lock before finding out there is nothing to push
<lifeless> otherwise race conditions
<lifeless> still, let me do it manually
<lifeless> done
<lifeless> mwhudson: ^
<mwhudson> lifeless: no idea, sorry, have to plead nearly-19:00
<lifeless> fair 'nuff
<lifeless> oh I see
<lifeless> tickled the 'sometimes we push a checkout' bug
<al-maisan> Good morning
<al-maisan> what's the easiest/quickest way to get an empty storm result set?
<al-maisan> found it: from storm.store import EmptyResultSet
<adeuring> good morning
<al-maisan> Good morning adeuring
<adeuring> hi al-maisan!
<mrevell> Morning!
<jml> good morning
<deryck> Morning, all.
<jml> deryck, good morning.
<mrevell> yo deryck
<mrevell> Is "OOPS" an acronym?
<intellectronica> operational ongoing problem symptom
<jml> intellectronica, you're making that up.
<intellectronica> yes
<intellectronica> any better suggestions?
<jml> no, that one is pretty good.
<mrevell> I wonder why we capitalise it, then
<mrevell> If anyone knows, matsubara will know if OOPS is an acronym :)
<matsubara> hehe
<wgrant> The rest of the characters are full-height, so it would look strange if it wasn't allcaps.
<matsubara> that's a good one intellectronica
<intellectronica> i think it's capitalized because it's SERIOUS
<mrevell> :)
<mrevell> Perhaps it's Portugese
<mrevell> Obridgado Oh Problemo Superioro
<jml> mrevell, I think it's partly because 'OOPS' in the code itself is capitalized.
<jml> mrevell, so we talk about OOPS Reports, rather than Oops Reports.
<jml> mrevell, IIRC it's not capitalized in the code.
<mrevell> jml: I think you've contradicted yourself but maybe I've misunderstood. Either way, if it's not an acronym I don't need to explain it anywhere :)
<jml> mrevell, oh sorry.
<jml> mrevell, we have OOPS codes and we also have source code to handle OOPSes.
<mrevell> Oooh, right.
<jml> in the later, I think it's OopsHandler rather than OOPSHandler, for example.
<mrevell> I see
<jml> but then we also say getUrl
<jml> (blech)
<matsubara> mrevell, it's not an acronym
<mrevell> thanks matsubara
<sidnei> i guess it's "OOPS, something went wrong" :)
<bac> sinzui: what if the release notes on the download page was a collapsible field?
<bac> sinzui: i like that as it would give symetry with the changelog
<bac> and would address your concern
<sinzui> bac: maybe. How large is the bz/+downloads page now, and how large will it be. Are we making something too big to Au and China to load?
 * bac looks
<sinzui> bac: I am off to my appointment.
<bac> sinzui: ok.  stand up delayed until your return?
<bac> or i'm glad to host
<sinzui> bac: you may host. I have meetings at 10 and 11 today.
<bac> ok
<sinzui> bac: I landed my +ubuntupkg fixes. I am starting on bug 193469
<mup> Bug #193469: Unlink (Ubuntu) package from series <package-link> <Launchpad Registry:Triaged by sinzui> <https://launchpad.net/bugs/193469>
<al-maisan> I am trying to optimise a page and see quite a few of the following queries:
<al-maisan> SELECT ValidPersonCache.id FROM ValidPersonCache WHERE ValidPersonCache.id = %s LIMIT 1', (2116775,)
<al-maisan> Can somebody explain what these are?
<al-maisan> Probably a question for the LP foundations team? gary-sprint maybe ^^
<gary-sprint> al-maisan: I'm afraid I don't know.  Stuart is back tomorrow.  Asking salgado :-)
<salgado> al-maisan, most likely from IPerson.is_valid_person
<al-maisan> OK, thanks
<flacoste> jml, sinzui: coming to the TL meeting?
<flacoste> beuno: ^^^
<beuno> flacoste, yes, thank you
<jml> flacoste, yes.
<jml> oh crap
<leonardr> barry (or anyone), random launchpad question
<leonardr> i used to be able to do this but have forgotten how
<leonardr> i'm trying to test a new version of launchpadlib. right now it can only be tested within the context of launchpad
<leonardr> actually, nm. i bet i have to modify buildout.cfg now
<leonardr> i was trying to change the symlink in sourcecode
<jml> oh! that reminds me.
<jml> leonardr, I'll review your branch.
<leonardr> jml, thanks
<jml> leonardr, do you think it would make sense to have an exception in lazr.restfulclient that signals "permission error"
<jml> leonardr, the launchpad coding standards warn against raising ValueError, IIRC.
<leonardr> jml: a "permission error" would logically be a 401 response from the server
<leonardr> in theory these two problems could have the same exception but i question whether it's worth it
 * jml looks at the context
<jml> leonardr, a 401 response is raised as an HTTPError, right?
<leonardr> yes
<jml> ta
<leonardr> this error "you're trying to access something that's not a real url. but i happen to know what this non-url string is, and i can tell you what you're doing wrong"
<jml> *nod*
<jml> that probably comes under the aegis of ValueError then
<jml> I really don't like raising ValueError though. :)
<leonardr> i can make up an exception
<jml> that's ok. I can't enforce my aesthetics all the time :)
<leonardr> RedactedValueAccessError
<matsubara> Chex, bigjools, gary-sprint, danilos, rockstar, sinzui, allenap: LP production meeting in 15 min @ #launchpad-meeting
<bigjools> I hate Thursdays
<danilos> matsubara: thanks
<danilos> bigjools: why, my day consists of only 3 calls and one meeting :)
<bigjools> danilos: quite :)
<jml> leonardr, that'd be great, thanks.
<jml> leonardr, I've approved it with options for you to make that change.
<jml> leonardr, thanks again for doing this patch.
<leonardr> ok
<gmb> jml: Question: If I use ec2 land on a merge-proposal targeted at lp:launchpad, how do I ensure it lands on devel and not db-stable (or whatever lp:launchpad points at)?
<jml> gmb, you can't, I don't think. lp:launchpad points at db-devel iirc.
<jml> gmb, it'd be fairly easy to add an option to override the MP though.
<gmb> jml: Ah, so I'm better off just using ec2 test to submit the branch, then?
<leonardr> deryck, flacoste sent you an email a little while earlier about your <dd> parser
<jml> gmb, exactly
<gmb> jml: I'll file a bug.
<jml> gmb, use --dry-run to assemble the commit message, perhaps.
<leonardr> i can help you do things the right way. let me know when you want to talk it through
<gmb> jml: Ah, yes, good idea. Thanks.
<jml> gmb, np.
<deryck> leonardr, yes, indeed. let's talk.  chatting with flacoste about this now.  then we can talk.
<leonardr> ok
<jml> danilos, I was talking to someone about translating documents the other day
<jml> danilos, and they told me that some people break docs up into strings and then use Launchpad to translate them. Is this true?
<danilos> jml: cool, what kind of documents?
<danilos> jml: of course they do, I've even written one tool to do that that is used by GNOME and ubuntu-docs (xml2po)
<jml> danilos, wow.
<jml> danilos, I hope this isn't a silly question, but why not provide a way to translate documents in Launchpad without doing that?
<danilos> jml: heh, time and effort involved, that's the only reason :)
<jml> danilos, it's a good reason :)
<danilos> jml: fwiw, I am spending some of my time to make the mess that is xml2po today (it is python, but with very lousy high level acceptance tests) a much more nicely modularised, so we should be able to reuse it in Launchpad at some point in the future :)
<danilos> jml: but, I wouldn't hold my breath just now :)
<jml> danilos, cool :)
<jml> danilos, and don't worry, I won't.
<danilos> jml: there's other interesting stuff there, like handling images and such... many people are interested, and Kyle (from OEM) has started a doctemplate project to make a template translatable documentation project ("doctemplate" in LP) and we'll be discussing it during UDS (we already did some bits during our translations sprint)
<gmb> allenap: What's the deal with ec2 giving me import errors about PQM? (ISTR you mentioning this before; can't remember when or why).
<gmb> Ah, it's to do with python2.4, isn't it.
<gmb> Rats.
<EdwinGrubbs> bac: sinzui assigned me bug 458169, which is very similar to your bug 455812. I was wondering if you had any more information than what I commented on in my bug.
<mup> Bug #458169: Distroseries:+index page timing out <timeout> <Launchpad Registry:Triaged by edwin-grubbs> <https://launchpad.net/bugs/458169>
<mup> Bug #455812: distroseries milestone timeout <timeout> <Launchpad Registry:In Progress by bac> <https://launchpad.net/bugs/455812>
<adeuring1> can somebody give me a clue how to run the inline tests in lp/testing/__init__.py , and only those test?
<sinzui> EdwinGrubbs: Bac: I am very confused by the fact that this is a 3.0 problem given that we did not update the code for distroseries. Both pages are generally changes to the position of the portlet
<bac> EdwinGrubbs: i have not yet looked at it.  we can talk after i return from lunch
<sinzui> EdwinGrubbs: Bac: I am sure that when we make one query for the distroseries, all will be better
<EdwinGrubbs> bac: sure
<jml> adeuring1, what do you mean "inline tests"?
<adeuring1> jml: the tests in docstrings, e.g., in time_counter
<adeuring1> I want to add a little function there, but that one should be tested ;)
<jml> adeuring1, I don't think there's a way to run the tests in time_counter
<jml> adeuring1, personally, I think it would be a good idea to add lp/testing/tests/ and put in some unit tests
<sinzui> EdwinGrubbs: I was able to load older Ubuntu series and milestones. There may be some clue in that
 * jml regrets not writing the factory with unit tests to begin with.
<adeuring1> jml: you might be right... But I like the idea that you can at the same time doument and test
<jml> adeuring1, sure. in which case, I think you'll still need to add lp/testing/tests/ I think and then add a test_doc to that which scans lp/testing/__init__ and runs the tests there. Gimme a sec to confirm.
<jml> adeuring1, http://docs.python.org/library/doctest.html#unittest-api -- you need to have something like the code snippet there in a test_suite() method in a file with a name starting with 'test_'
<jml> adeuring1, you only want to build a suite object and return it.
<adeuring1> jml: thanks!
<jml> adeuring1, my pleasure.
<sinzui> EdwinGrubbs: flocoste pointed out that distroseries.currentseries can cause the problem. That is not a good method--it is taking a guess. We can either fix it or use a cachedproperty in the view. I favour a lower fix because this is a reoccurring issue
<barry> abentley, rockstar ping
<rockstar> barry, pong
<abentley> barry: pong
<barry> abentley, rockstar: hi.  i'm working on our python 2.5 + zope toolkit branch and we're seeing a ton of failures in the codehosting smoke test.  stepping through the code, i see this output in the bzr subprocess that gets called:
<barry> http://paste.ubuntu.com/299140/
<barry> abentley, rockstar any clues would be greatly appreciated! :)
<rockstar> barry, abentley would probably know more about that than I.
<barry> rockstar: cool
<abentley> barry: I think the warning on line 2 can be ignored.
<barry> abentley: k
<abentley> barry: I'm not sure what the one on line 7 would be caused by, but it's in our VFS implementation.
<barry> abentley: any ideas where i should start to look?
<abentley> barry: lib/lp/codehosting
<barry> abentley: does this call xmlrpc?
<abentley> barry: Yes.
<barry> abentley: hmmm
<abentley> barry: It looks like the message is due to raising InProcessTransport in lib/lp/codehosting/vfs/transport.py
<abentley> barry: This may be an error in bzr's error reporting, rather than the underlying bug.
<barry> abentley: otp, thanks, will respond shortly
<barry> abentley: okay.  can you think of any issues related to python 2.5 or 2.6 here?
<barry> abentley: i guess the question is: if this doesn't fail in py2.4, what would make it fail in 2.5?  and would it still fail in 2.6?
<barry> abentley: do you mean that there's some other failure going on that is being masked by this?  and if so, can you think of a strategy to debug it?
<jml> g'night all
<mrevell> night jml
<abentley> barry: I'm not aware of any issues with 2.5 or 2.6 on the bzr side, so I would expect it's a codehosting or twisted issue.
<abentley> barry: You can debug it by changing the sites that raise InProcessError to return '' instead.
<abentley> barry: There should also be logs / oopses for the twisted server, so it would be worth hunting those down.
<abentley> barry: You can test it using "make run_all".
<beuno> barry, quick question:  can we let people unsubscribe from the ML archives?  or is that getting too much into perl?
<barry> abentley: thanks for the tips.  i'm going to look into this more after lunch
<barry> beuno: you mean an "unsubscribe" link in the archives?  i think you're deep into perlworld
<beuno> barry, I suspected as much, thanks
 * sinzui wonders if the QA fairy will visit sinzui's landings
<sinzui> barry: you make need to make use of utilities/make-lp-user to get a working ssh key for bzr branch pushing
<beuno> bigjools, ping
<bigjools> beuno: I am about to leave for the day, is it quick?
<beuno> bigjools, it is
<bigjools> okidoki
<beuno> with private PPAs
<beuno> I see the deb lines ifI don't have permissions
<bigjools> ah yeah there's a bug about that
<beuno> and need to go to another page to see the lines with the tokens
<beuno> why don't we just hide them if you don't have permissions, and show you the debs with the tokens if you do?
<beuno> why do we have a separate page?
<bigjools> it's a page that shows all your tokens for all subscriptions IIRC
<bigjools> so has a slightly different use
<bigjools> but yes we can put the specific one in the PPA page
<beuno> perfect answer
<beuno> last one
 * bigjools knows all about beuno's "quick" things
<beuno> in what situation can I access a private PPA, but not have permissions?
<beuno> heh
<bigjools> beuno: there should be no situation like that
<beuno> bigjools, can I have permissions but not have a token yet?
<bigjools> yes
<bigjools> I think
<bigjools> yes
<beuno> bigjools, ok, good enough, thanks
<beuno> barry, ping again
<beuno> private mailing lists
<beuno> are they exposed on the team's page, but the archive blocked for users without permissions
<bigjools> beuno: actually I am talking tripe
<beuno> or completely hidden?
<beuno> bigjools, I will assume you mean nonesense
<bigjools> beuno: you can't see the web page at all, ever, with a subscription, and you need to get a token to see the repo
<bigjools> however we want to allow users with tokens to see the new user-focused page
<bigjools> s/tokens/subscriptions/
<bigjools> and now I really have to go
<bigjools> good night!
<beuno> bigjools, so if I don't have permissions, i never even see the link to the PPA?
<bigjools> you don't
<beuno> bigjools, thanks.
<beuno> go before I think of something else
<mrevell> night all
<rockstar> abentley, ping
<abentley> rockstar: pong
<rockstar> abentley, so, I'm writing this script to run branch upgrades.  How do I run the cronscript?  I assume there's some fancy helper here.
<abentley> rockstar: The current state of the art is in update_preview_diffs.py
<rockstar> abentley, yeah, I used that as my template, since it seemed to be the most up to date.  How does it actually get run though?
<abentley> rockstar: That is, subclass JobCronScript, specifying config_name and source_interface.
<abentley> rockstar: It is run by cron.
<rockstar> abentley, yeah, so how do I run it to test it?
<abentley> rockstar: bin/py cronscripts/my_script_name.py
<rockstar> abentley, ah, okay, cool.
<abentley> rockstar: You'll probably want to specify LPCONFIG=development as well.  That will force it to use the development config files instead of the production ones.
<rockstar> abentley, okay, thanks.
<abentley> rockstar: But of course, you then have to create development config files.
<rockstar> abentley, yup, chasing that now.
<barry> sinzui: thanks i wonder if that's it
<barry> beuno: correct.  the archives are blocked by our openid plugin
<sinzui> barry: I found that I could not connect to the ssh server because changed the user name of our leader to ~mark
<beuno> barry, but you can see the link to it?
<barry> beuno: well, no because if you're not a member of the team, you shouldn't even be able to get to the team page
<beuno> barry, so you can't have a private mailing list in a public team?
<beuno> if you can see the link, you can see the archive?
<barry> beuno: no, currently not though there's a bug open on that (a use case i agree with)
<beuno> barry, cool, thanks. I just wanted to know if you could ever get in a situation where you clicked ona link that you could not access
<barry> beuno: i don't think so
<barry> beuno: well, modulo openid bugs of course :)
<beuno> barry, that clears it up for me, thanks
<barry> cool
<sinzui> beuno: I am looking at the SP page for a packaging link that needs to be deleted: https://edge.launchpad.net/ubuntu/hoary/+source/jedit
<beuno> sinzui, I'm having a hard time understanding what that is
<beuno> or how you ended up there
<sinzui> beuno: 1) "Upstream release series" appears to be redundant with "Upstream associations", 2) I am surprised that in the https://edge.launchpad.net/ubuntu/+source/jedit, we do not ask the user to confirm the removal of the link. 3) I expect the unlink action to look like (-) Remove packaging link.
<sinzui> beuno: you can get the the SP page from the productseries index or +packages (https://edge.launchpad.net/jedit/+packages)
<sinzui> beuno: you may be familiar with this: https://edge.launchpad.net/bzr
<sinzui> ^ The extra edgy link is a data error. I want to allow users to remove the bad link to the SP.
<beuno> sinzui, so how do upstream associations happen?
<beuno> are they inferred from series linking?
<sinzui> https://edge.launchpad.net/bzr/trunk <-  (+) Link to Ubuntu package  and (+) Link package
<beuno> wow, I'm confused
<beuno> why do we have (+) Link to Ubuntu package *and* (+) Link package?
<sinzui> beuno: not inferred, explicitly set, though SP links cannot be deleted, only DSPs. Since there will never be a DSP because this is an error, it is not possible to correct the mistake. there are 81 broken SPs in production
<beuno> sinzui, so it sounds like you need to delete the link when the DSP are deleted
<sinzui> beuno: that is point 4) Should re rename (+) Link to Ubuntu package to link the the current Ubuntu series? It is takes most of the thinking out of the task. The other one can link to any distro and you can specify primary or include package relationships
<sinzui> beuno: the DSP is a binary. there is not any code, so the DSP will not be made
<beuno> sinzui, so one is for Ubuntu, and the other one is for any distro?   wouldn't it make sense to have one page?
<beuno> (or widget)
<sinzui> I tried and failed
<beuno> sinzui, code-wise or UI-wise?
<beuno> I can help you with one of thse
<beuno> *those
<sinzui> beuno: I think I need a lot of work to reconciile the history operation that is not present in the any packaging operation. we do not know Fedora's history
<beuno> sinzui, so one is to link 2.0 <> karmic
<beuno> and the other one is 2.0 <> bzr2.0~ubuntu1?
<sinzui> beuno: one is for the Ubuntu who cares about what they are doing right now, the other is really for projects who care about supporting any project
<beuno> sinzui, and what's the benefit (or use case even) of the latter?
<sinzui> making packages for debian or fedora, to indicate that your PPA provides a backport  for hoary
<beuno> sinzui, but we don't have any of those pacakges in Launchpad
<beuno> so it's more of a modeling thing, right?
<sinzui> beuno: Yes.
<beuno> sinzui, "external distribution links"?
<sinzui> beuno: And I should not that I am only working on this to kill the root cause of many kinds of oopes in production. Because we let users create invalid links, user create oops trying to delete, remove, transfer the problem
<beuno> ok, so the sane thing to do is not let userscreate invalid links
<beuno> but you already know that
<beuno> so I'm struggling here  :)
<sinzui> beuno: 1) I can add  [Delete link] to each item in this table https://edge.launchpad.net/jedit/+packages for the project side
<sinzui> beuno: 2) For the distro side I can add [Delete link] to the SP...then I saw the duplicate portlet in https://launchpad.net/ubuntu/hoary/+source/jedit
 * sinzui has already updated the packaging logic to not allow users to create duplicate links.
<beuno> sinzui, so why not just delete them on the DB and get it over with?
<sinzui> because the example I am showing is not a db error, it is a mistake that cannot be fixed
<sinzui> oh, and I need this feature to fix the 81 items in the db
<beuno> sinzui, remove links on the SP page seems fine
<sinzui> beuno: why do we not confirm the removal of a linked DSP?
<beuno> sinzui, oversight is my best guess
<sinzui> beuno: actually, can I try to change the button on https://staging.launchpad.net/ubuntu/+source/jedit to look like the remove icon (-) ?
<sinzui> beuno: I know the relink is easy if you know what the correct series is, but if you do it by accident, you need to know about staging to look up the answer right away
<beuno> sinzui, please do
<mzz> about how much ram does bzr 2.0.0 need to grab the launchpad trunk? I didn't give this vm all that much ram, if it needs more than about 500mb I should just ctrl+c it now...
<mzz> (it's "Inserting missing keys" while hitting the vm's swap heavily)
<mzz> bah, it's used about 2/3 of the vm swap now. I'll just abort it and give it some more ram
<mwhudson> good morning
<thumper> mwhudson, abentley, rockstar: I need to drop Rachel down in town today, so stand up will be delayed slightly, perhaps by 15m or so
<mwhudson> thumper: fine with me
<abentley> thumper: okay
<EdwinGrubbs> bac: ping
 * mwhudson takes delivery of his new phone, apologies if i don't get any work done today...
<rockstar> mwhudson, :)
<bac> EdwinGrubbs: ping
<EdwinGrubbs> bac: hi
 * thumper is back
<thumper> abentley, rockstar, mwhudson: conf call or skype?
<abentley> thumper: Skype please.
<rockstar> thumper, I'd prefer the conf
<thumper> mwhudson: tie breaker?
<mwhudson> i'd prefer skype if it works
<thumper> abentley: now?
<abentley> thumper: works for me.
 * rockstar stabs skype with a dull rusty spoon
<mzz> is it ok if I add "the file to edit is /etc/apache2/sites-available/local-launchpad unless mentioned otherwise" to https://dev.launchpad.net/Running/RemoteAccess or would that be incorrect?
<rockstar> mzz, that seems right to me, although I'm not the expert.
<mzz> it's definitely what I ended up editing here, and it seems to have worked, but I have no clue if this is different on different versions of ubuntu (I'm using karmic)
<rockstar> mzz, no, that's been the standard place for a long time.
<mzz> ok, thanks
<abentley> thumper: https://wiki.canonical.com/Launchpad/PolicyandProcess/EmergencyChange/Experiment
<mwhudson> thumper: have you read flacoste and deryck's mails on the list ?
<rockstar> mwhudson, is it policy that each script runs as it's own db user?
<mwhudson> rockstar: yes
<rockstar> mwhudson, where is the db user file?
<mwhudson> rockstar: you mean database/schema/security.py ?
<mwhudson> um
<mwhudson> rockstar: security.cfg i mean
<al-maisan> can somebody please help me figure out this test failure: http://pastebin.ubuntu.com/299311/ ?
<al-maisan> it's a page test using 'user_browser' and it fails to access EmailAddress.email for <test@canonical.com>:
<al-maisan>     Unauthorized: (<EmailAddress at 0x13aef490 <test@canonical.com> [Preferred Email Address]>, 'email', 'launchpad.View')
<mwhudson> al-maisan: well maybe that user has their email address hidden?
<al-maisan> mwhudson: aha, how would I find that out? Is that in the associated Person record?
<thumper> flacoste: ping
<mwhudson> al-maisan: i don't know, i guess i'd look at the person-index.pt template to find out
<thumper> or deryck: pign
<al-maisan> mwhudson: thanks for pointing that out, I believe this is the property: IPerson.hide_email_addresses
<mwhudson> al-maisan: sounds promising
<al-maisan> mwhudson: :)
<rockstar> jml, didn't you write a plugin that would tell me which of my branches had been merged already and could be deleted?  What is that plugin called?
<al-maisan> mwhudson: thanks again, your idea saved my evening :)
<mwhudson> al-maisan: np!
<thumper> rockstar: skype?
<thumper> sinzui: when are we talking?
<thumper> sinzui: are we talking today
<thumper> ?
<sinzui> We ca
<sinzui> n
<sinzui> It has been a while.
<sinzui> do you still have plague?
<thumper> rockstar: I'll talk with you after curtis
<rockstar> thumper, yes, if skype doesn't drive me mad with delay...
<rockstar> thumper, :(
<abentley> rockstar: next time we chat, we should try disabling pulsaudio first.
<rockstar> abentley, yeah, I suspect pulseaudio might be our culprit.  It may not like the USB headsets.
<abentley> rockstar: Considering there's ALSA between it and our headsets, it really shouldn't care.
<rockstar> abentley, I don't think so.  I think it just has an ALSA compatible API.
 * rockstar is not the expect.  Ask crimsun
<thumper> rockstar: call?
<rockstar> thumper, yessir
<deryck> thumper, hey
<deryck> thumper, I can just reply on list to your questions.
<thumper> deryck: I've found it now anywy
<thumper> but thanks
<thumper> it works
<thumper> not convinced yet that we are doing the right thing
<thumper> also
<thumper> not patching just the commit message
<deryck> thumper, ok.  I've got to fix the textarea widget to not need formatter, that was the just of my earlier mail.  but I'll do that fix.
<thumper> ok
<thumper> deryck: I've also been tweaking the lazrjs widget in
<deryck> thumper, you really should just need to define the python bit that spits back xhtml, like bug_description_xhtml_representation in lp.bug.browser.bug
<thumper> deryck: yes, exactly did something like that
<deryck> thumper, so if you're tweaking the formatter part, I wouldn't waste time there.  it won't matter with my changes coming.
<thumper> I'm not changing the formatter
<thumper> but allowing null to start with
<deryck> thumper, null what?
<thumper> deryck: http://pastebin.ubuntu.com/299348/
<deryck> thumper, ah, ok
<deryck> thumper, I'll be back in a couple hours to do my stuff on this.  kids to tend to right now. :)
<thumper> sinzui: can I get you to look at some pics for me>
<thumper> ?
<sinzui> with horns?
<thumper> sinzui: just realised that I have to go get Maia
<thumper> sinzui: lemmie paste you some pics
<sinzui> okay
<thumper> http://penhey.net/~tim/commit-msg-empty.png
<thumper> http://penhey.net/~tim/commit-msg-editing.png
<thumper> http://penhey.net/~tim/commit-msg-saved.png
<sinzui> thumper: the pics are nice editing implies trailing whitespace. Is that stripped?
#launchpad-dev 2009-10-23
<thumper> sinzui: there are foundational bugs about trailing whitespace
<thumper> sinzui: all I want is a ui=sinzui :)
<thumper> sinzui: I'll look into the whitespace issue
<thumper> sinzui: it appears that the multiline editor adds some carriage returns at the end for some reason
<thumper> yet to be determined
<thumper> mwhudson: got any comments on the pics above?
<sinzui> thumper: The UI is r=me. You may want to verify the Comments IField is strippedText* and then you want to verify the setter does the cleaning instead of the widget. I had to do something similar in other Ajaxed forms
<thumper> sinzui: ok, ta
<mwhudson> thumper: looks nice
 * mwhudson afk for lunch
<sinzui> thumper: the StrippedTextLine is flawed. StrippedTextWidget is doing the work. The comment's IField type must work like c.l.field.URIField
<thumper> anyone know our windmill xpath expressions?
<thumper> mwhudson: you around for a pre-impl call?
<mwhudson> thumper: sure
<wgrant> Gnargh @ people who don't check security declarations before they export objects.
<lifeless> going to ile a bug?
<wgrant> Just done so now.
<wgrant> bug #458833
<adeuring> good morning
<mrevell> Morning!
<wgrant> bigjools: Morning... Want to have a look at bug #458833?
<bigjools> wgrant: have you tried to rocketfuel-branch today?
<wgrant> bigjools: No, I don't normally use it.
 * wgrant tries
<bigjools> ok
<bigjools> I got an error when it builds the wadl - "ImportError: No module named apt_pkg"
<bigjools> awesome
<wgrant> Ah.
<wgrant> I know why that is.
<bigjools> missing __init_ ?
<wgrant> There's a new python-apt in primary.
<wgrant> Which supersedes the one in launchpad/ppa which supports python2.4.
<bigjools> GAH
<wgrant> Again.
<bigjools> ffs
<wgrant> maxb usually updates his, which then gets copied into launchpad/ppa, but it's easy enough for anybody to do it.
<wgrant> For now, you can just downgrade.
<bigjools> yarp
<jml> rockstar, bzr-removable
<gmb> allenap: I've a question about utilities/ec2 if you've the time.
<allenap> gmb: Sure.
<gmb> allenap: What Python should ec2 be using? It breaks with 2.4 because of the PQM issue and it breaks with the system python because of something to do with paramiko (don't have a traceback to hand at the moment).
<gmb> allenap: It does work with python 2.5 but I figured we were hoping to skip that and go straight to 2.6 with the rest of LP.
<allenap> gmb: It'll work with either 2.5 or 2.6.
<gmb> allenap: Oh, okay. My paramiko error must've been caused by something else. Cool, thanks.
<allenap> gmb: The utilities/ec2 script is a bit broken right now. See "Problems with utilities/ec2 test -s "..." on Jaunty (and possibly on Karmic too)?" on launchpad-dev@.
<gmb> allenap: Yeah, test -s is what broke for me. Using the version in an old branch of stable worked for me though.
<wgrant> Is a separate 2.6 branch going to be maintained for 6 months?
<wgrant> ISTM that neither trunk can be moved to 2.6 before Lucid.
<allenap> wgrant: Because 2.6 isn't on hardy?
<wgrant> allenap: Right.
<mwhudson> wgrant: there is A Plan
<mwhudson> which involves a ppa
<allenap> wgrant: I don't think a little thing like that will stop them :)
<wgrant> mwhudson: Urgh.
<deryck> Morning, all
<maxb> bigjools: hmm..... but how did you get that newer python-apt installed?!  the requirements of launchpad-developer-dependencies should have forced you to stick with the old version until the PPA was updated!
 * bigjools shrugs
<maxb> My updating just now has presented me it as a held-back package (which is what prompts me to go and update it
<bigjools> weird
<thumper> deryck: hey, you might want to look at my lazr-js branch too
<deryck> thumper, ok, will do.  do you do any work on this patching problem?  Or just fixes to get your stuff working?
<thumper> deryck: the lazr-js branch is all css styling to use a class rather than an id
<thumper> deryck: mwhudson has looked at it
<thumper> deryck: but i was wondering about replacing rather than adding css styles
<thumper> deryck: in my branch multiline edit branch I add the lazr-multiline-edit class to the bugs page too
<thumper> deryck: in preparation for the lazr-js fix
<deryck> thumper, ok, cool.  thanks for making this better. :)
<thumper> deryck: np, I'm off now, it is almost midnight
<deryck> cool, enjoy the weekend.
<bigjools> maxb: did you figure out why your PPA package wants to get upgraded to karmic's?
<maxb> It doesn't, for me :-)
<bigjools> maxb: I downgraded and it wants to upgrade again for me, is there any debugging I can do for you?
<maxb> What wants to upgrade it? apt-get ? aptitude? update-manager?
<bigjools> tried the first 2 and they both want to
<maxb> This makes no sense
<bigjools> aptitude says: python-apt [0.7.13.2ubuntu3launchpad~maxb1 -> 0.7.13.2ubuntu4]
<maxb> After upgrading to vanilla karmic, the dependencies of launchpad-developer-dependencies would be violated
<leonardr> allenap, when you are around would you check and make sure that bug 451158 is fixed in the version of launchpad i just released?
<mup> Bug #451158: AttributeError: 'Credentials' object has no attribute 'authorizeSession' <launchpadlib :Fix Released> <https://launchpad.net/bugs/451158>
<maxb> bigjools: you do have lp-dev-deps installed?
 * bigjools facepams
<bigjools> facepalms, too
<allenap> leonardr: Okay.
<bigjools> guess what!
<bigjools> the karmic upgrade must have removed it :(
 * maxb disappears for lunch
<bigjools> maxb: and now it works, thanks
<allenap> leonardr: I get this now: http://pastebin.ubuntu.com/299769/
<leonardr> aaargh
<allenap> leonardr: I got the Python console by: bzr branch trunk boot && cd boot && virtualenv --no-site-packages . && bin/python bootstrap.py && bin/buildout && bin/py
<leonardr> i tested get_token_and_login but not login_with
<leonardr> i think i need to create a fake launchpad so i can write tests for these methods
<allenap> leonardr: get_token_and_login() works fine.
<allenap> leonardr: Something for a quiet afternoon perhaps ;)
<leonardr> yeah, i think it's just called something other than 'credentials' now
<leonardr> allenap, it looks like the problem is that the credentials object is no longer set as 'credentials' on the service root
<allenap> leonardr: Is is somewhere else, or is this a pita to fix?
<leonardr> it's a bit of a pita
<leonardr> i'll probably end up doing another lazr.restfulclient release that sets it
<leonardr> i bet it would currently be available as ._browser.authorizer, but it makes more sense just to set it again
<leonardr> allenap: would you try your code in conjunction with https://code.edge.launchpad.net/~leonardr/lazr.restfulclient/set-credentials ?
<leonardr> and also review that branch? (i'm writing mp now)
<allenap> leonardr: Okay.
<gmb> Okay, can someone help my poor, tired brain: Why would bug 438985 occur in a web environment but not in the harness or in a test?
<mup> Bug #438985: Trying to make myself as bug supervisor of my project oopses <oops> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/438985>
<gmb> Oh, wait.
<gmb> I know why.
<gmb> It's a DB object thing isn't it?
<gmb> Okay, so, my next question: What's the best way for me to ensure that I've got two differnt objects that refer to the same row in the database?
<gmb> allenap: Do you know? ISTR you having similar problems a while back (I could, of course, be making this up).
<allenap> gmb: Why do you want to ensure that you have two different objects for the same row?
<gmb> allenap: So that I can write a regression test for the above bug.
<allenap> gmb: (a is not b and a.id == b.id)
<gmb> allenap: Right, that's what I need.
<gmb> Is there a simple way to do it that you know of?
<allenap> gmb: Oh, and check the table.
<allenap> gmb: I don't really understand. If you have two objects for the same row from the same store that's a bug in Storm isn't it?
<gmb> allenap: Okay, I'm confused... I don't need to check that they're different, I need them to actually *be* different. How do I create two different object referring to the same DB row?
<gmb> allenap: Well, it might be, but the problem is that we're comparing two Person objects using 'is' instead of '==', and it breaks in the web environment but not in tests or harness (because the object stays the same between queries)
<gmb> allenap: Fixing the bug is easy. Writing the test to make sure it doesn't happen again isn't.
<allenap> gmb: Ah, can't you just instantiate them and then set their properties? Or, if you don't need them to actually work, you could use a Snapshot.
<gmb> allenap: Ah, interesting idea. Yes, I could cheat like that, I guess :).
<gmb> \0/ for hacks
<gmb> allenap: Thanks, I'll try that.
<allenap> gmb: If the test is just to prevent someone from using "is" again, then it sounds like it's not worth it.
<gmb> allenap: Would you be happy for that to land, as a reviewer?
<sinzui> barry: stand-up
<gmb> It's a bit of a bugger of a thing to detect, so I'm inclined to at least try to get the test to fail.
<allenap> gmb: I guess I don't really know how this bug plays out in the code, so I'll keep quiet from here on :)
<gmb> allenap: Fair enough. Suffice it to say it's a PITA.
<barry> sinzui: just you and me? :)
<allenap> leonardr: That took me a while ("now, how do I actually use an unreleased library here?") but in the end, it works fine.
<leonardr> allenap: sorry, i could have helped oyu with that
<allenap> leonardr: I found HackingLazrLibraries which was a good thing to read anyway.
<leonardr> allenap: can i remind you to vote on the mp?
<allenap> leonardr: Just done.
<sinzui> barry: look at lib/canonical/__init__.py
<sinzui> barry: I think you can filter all Set, SHA, and MD5 from there.
<barry> sinzui: ah, very interesting.  thanks for the link.  one problem is that this won't filter anything for the twisted applications that run in a subprocess.  but i think i have hacked tachandler to do that.
<sinzui> barry: I think you can. There used to be hacks in the tac files for error loging
<sinzui> EdwinGrubbs: what is the status of bug 436507
<mup> Bug #436507: Distribution series source package page has no links to distribution source package page <post-3-ui-cleanup> <Launchpad Registry:In Progress by edwin-grubbs> <https://launchpad.net/bugs/436507>
<EdwinGrubbs> sinzui: I didn't work on it. It just needs a couple of minor changes before it can be put in review.
<sinzui> okay
<sinzui> EdwinGrubbs: the timeout bug is more import
<EdwinGrubbs> sinzui: if I don't work on the timeout, I might get karmic retribution
<sinzui> correct
<barry> jml: ping
<jml> barry, I'm on a call right now -- how can I help?
<barry> jml: just ping me when you're free, i'd like some suggestions on how to handle a tricky twisted problem
<jml> barry, ooh, fun.
<jml> barry, will do.
<barry> jml: thanks!  yes, "fun" :)
<bac> hi beuno
<beuno> hi bac
<jml> leonardr, lazr.restfulclient's setup.py says it uses zope.interface, but zope.interface isn't used in the code
<leonardr> jml: it's used by lazr.restful, which is a testing dependency of lazr.restfulclient
<leonardr> you might be able to remove it from setup.py but it will still be downloaded and used
<jml> leonardr, ahh ok.
<jml> leonardr, I'm just trying to figure out what extra dependencies bzr would need if it started to use launchpadlib
<adeuring> henninge: r=me for your branch from this morning
<henninge> adeuring: thank you
<adeuring> henninge: welcome :)
<barry> jml: reping (if you're free)
<jml> barry, like a bird on a wire
<jml> barry, what's up?
<barry> jml: maybe -> #launchpad-sprint?
<BjornT_> gmb: did you make any progress on bug 438985?
<mup> Bug #438985: Trying to make myself as bug supervisor of my project oopses <oops> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/438985>
<gmb> BjornT_: I've got a fix playing in ec2 now.
<BjornT_> gmb: i mainly wanted to know if you figured out what the problem was
<henninge> adeuring: when approving merge proposals, it would be great if you could set the review type to "code". The new "ec2 land" command looks for that to find the reviewer for the the branch. Danke.
<gmb> BjornT_: wgrant already did that: we were comparing two Person objects with "is" instead of "==" in the check to see whether one could subscribe the other. So even if you were subscribing yourself the comparison returned False, hence the OOPS.
<adeuring> henninge: thanks for the head-up. I wasn't aware to this
<gmb> BjornT_: The problem I had was writing a suitable regression test, but allenap helped me figure that out.
<henninge> adeuring: np
<henninge> adeuring: I wasn't either until it just refused to work for me ... ;-)
<BjornT_> gmb: do you know why you had different objects there? what does the test look like?
<gmb> BjornT_: No, I don't. I'll paste a diff of the test for you, hang on...
<BjornT_> gmb: ok. did you try passing in a security wrapped object to the method? that should do it; i'm quite sure that is want is happening.
<gmb> BjornT_: Actually, no I didn't. The test looks like this: http://pastebin.ubuntu.com/299839/.
<gmb> BjornT_: The security-wrapped object would probably be a better test, actually. What do you think?
<gmb> BjornT_: Actually, I wonder, looking at it now - is there a chance that that test could pass when it shouldn't? It failed for me every time without the fix, but perhaps I'm being overly optimistic about it.
<BjornT_> gmb: makePerson() is buggy. it strips the security proxy from the created person
<gmb> Ah.
<gmb> Eurgh.
<BjornT_> gmb: there is no rationale for stripping it (except for modifying some attributes), so i would try fixing it. if that fails, you should add an XXX, explaining why you get the created person from the db again.
<barry> abentley: ping
<gmb> BjornT_: Okay, will do. Thanks.
<abentley> barry: pong
<barry> abentley: hi.  i'm having a problem with 'bzr shelve somefile'.  it asks me whether i want to shelve it, i say yes, but then nothing gets put on the shelve and the change in the working tree doesn't get backed out.  any idea what's going on?
<barry> abentley: bzr shelve --all works fine though (but it's too coarse for what i need to do now)
<abentley> barry: And does "bzr shelve" work if you shelve only somefile?
<barry> abentley: no
<james_w> barry: do you press "y" twice in the case where it doesn't work?
<abentley> barry: I'm not sure why you would get that.
<james_w> al-maisan: we just tested your permission change on testing and it works, thanks
<barry> james_w: it only asks me once, so i only hit 'y' once
<james_w> barry: does it show a diff?
<abentley> barry: That's very odd.
<al-maisan> \o/
<barry> james_w: it does!
<barry> abentley: yeah
<abentley> barry: It should always confirm the overall shelve operation.
<barry> abentley: i never get that last confirmation
<barry> abentley: unless i use --all
<barry> Shelve? [yNfq?]y
<barry> Selected changes:
<barry>  M  lib/canonical/launchpad/daemons/tachandler.py
<barry>  
<barry> then it exits
<barry> % bzr shelve --list
<barry> No shelved changes.
<abentley> barry: Are you hitting lowercase y?
<barry> abentley: yep
<abentley> Does it say "no changes to shelve" before it quits?
<barry> abentley: nope
<barry> oh wow
<barry> abentley, james_w: um, if i run it in a gnome-terminal instead of an emacs shell buffer, i get the confirmation message!
<james_w> huh
<barry> and when i give the second 'y', it works
<barry> james_w: shall i file a bug on bzr?
<abentley> barry: Please do.
<barry> abentley, james_w thanks!
<abentley> barry: I'm not sure-- it could be caused by the way prompts are written/overwritten with \r.
<barry> abentley: very possibly so
<abentley> barry: Or it could be the way we retrieve a single character.
<abentley> barry: If you want to play with it, it's in bzrlib.shelf_ui.Shelver.prompt
<barry> abentley: thanks.  my stack is deep right now so i'll just report the bug and move on ;)
<abentley> barry: Understood.  Wouldn't want you to get a recursion traceback.
<barry> abentley: MemoryError
<barry> abentley, james_w: https://bugs.edge.launchpad.net/bzr/+bug/459185
<mup> Bug #459185: 'bzr shelve' with confirmation fails in an emacs shell buffer <Bazaar:New> <https://launchpad.net/bugs/459185>
<rockstar> abentley, could you give me hints on where to put pdb trace to find this hidden exception: http://pastebin.ubuntu.com/299933/
<abentley> rockstar: I could, but I've got a good guess at the cause: you haven't configured an oops directory.
<rockstar> abentley, you mean error_dir ?
<abentley> rockstar: possibly
<rockstar> abentley, looks like it, but it doesn't appear to be set in any of the other jobs in configs/schema-lazr.conf
<abentley> rockstar: True.
<rockstar> abentley, configs/testrunner/launchpad-lazr.conf ?
<rockstar> It looks like that's what I need.  Thanks.
<abentley> rockstar: The error isn't accessible to you because the test is running a script.  Run the script yourself in order for pdb to work.
<rockstar> abentley, yeah, but it didn't fail when I just ran the script.
<abentley> rockstar: Probably you didn't set the CONFIG environment.
<rockstar> abentley, maybe.  Adding it to the testrunner configs made my test work.
<abentley> rockstar: Great.
<rockstar> abentley, this job script stuff is quite nice once you get all the intricacies of it.
<abentley> Thanks.  It could probably do with some reworking, because I put it together gradually.
<abentley> rockstar: And the oops reporter should scream if you try to configure it with None as the error_dir.
<rockstar> abentley, yeah, probably.
<rockstar> abentley, chat?
<abentley> rockstar: sure.
<abentley> rockstar: Where?
<rockstar> abentley, GTalk
<mzz> is it normal for rocketfuel-get to complain about "Not a branch" twice (once for shipit, once for canonical-identity-provider)?
<wgrant> mzz: Yes. Those two components remain proprietary.
<mzz> ah, ok. I didn't notice anything breaking, just making sure I wasn't accidentally running a script normal people don't use
<wgrant> That script will die if one of the essential components fails to download
<mzz> so! I'm trying to fix bug 456764. Iiuc I need to teach lazr.restful how to convert a timedelta to json first. I figured I'd just do that separately, build an egg with the version bumped, put it in lp-sourcedeps, and tell launchpad to use it (versions.cfg?). But to do that I need a python 2.4 setuptools.
<mup> Bug #456764: please expose builder queue state through the api <api> <ppa> <Soyuz:Triaged> <https://launchpad.net/bugs/456764>
<mzz> questions: is this at all sane, and is there a python-setuptools for python 2.4 somewhere?
<mzz> or hmm, I guess I could be lazy and just let ez_setup grab setuptools
#launchpad-dev 2009-10-24
<al-maisan> hello leonardr, are you around and could you please help me with a LP API change?
<wgrant> Is this the SP.branches dict?
<wgrant> I think that will be difficult or impossible; I don't know of any existing exported dicts that contain other entries.
<al-maisan> wgrant: an attempt, yes.
<al-maisan> http://pastebin.ubuntu.com/300148/
<al-maisan> this is the error I get during testing: http://pastebin.ubuntu.com/300150/
<wgrant> That's what I expected.
<al-maisan> hrm
<wgrant> lazr.restful doesn't treat dicts specially.
<wgrant> Normally when it sees an enum in a field it will stringify it, and only then hand it off to be JSON-encoded.
<al-maisan> wgrant: aha
<wgrant> And I'm not sure if launchpadlib is set up to handle dict specifications in WADL.
<leonardr> al-maisan: i'm not really around but i can try to answer your question
<leonardr> unless wgrant already has
<al-maisan> leonardr: thanks -- please see http://pastebin.ubuntu.com/300148/
<leonardr> yes, i odn't think you'll have much luck exporting a dict
<leonardr> you can publish a named operation that returns a dict
<wgrant> There are existing named ops that return dicts.
<wgrant> Right.
<al-maisan> OK
<wgrant> But I guess launchpadlib won't interpret any entry links inside them.
<leonardr> no, you'll just have a dict of strings to strings
<al-maisan> I'll go down the named operation route then
<wgrant> So I guess just export a named op that takes a pocket.
<leonardr> al-maisan: you should be able to write a serializer
<leonardr> for dicts
<al-maisan> leonardr: aha
<wgrant> leonardr: But won't that require client changes?
<leonardr> look in lazr.restful for other examples
<al-maisan> OK
<leonardr> i don't think so. i think the client will see .foo as a dict
<leonardr> there might be problems changing the dict and sending it back
<wgrant> I mean, lazr.restful will serialise it and stop crashing, but the client will still just see strings.
<al-maisan> leonardr: changing the dict and sending it back is not an issue here..
<leonardr> wgrant: yes, the client will still see strings, but it's better than creating a named operation that will have the same problem
<leonardr> al-maisan: look in lazr/restful/marshallers.py
<al-maisan> leonardr: thanks for the pointer
<leonardr> i guess i should leave so people don't think i'm still around
<leonardr> good night
<al-maisan> Good night
<wgrant> I see a slight ordering issue here.
<al-maisan> aha
<al-maisan> wgrant: what do you mean?
<wgrant> al-maisan: Well, US people vanishing while you are still here. I make it early morning for you.
<al-maisan> wgrant: yeah ;)
<mzz> so still prodding my bug 456764: after teaching lazr.restful about timedeltas my trivial patch (adding two exported() calls) runs, I see the attributes show up in the apidoc (after I forced a regen of the wadl file), but I don't see them through launchpadlib.
<mup> Bug #456764: please expose builder queue state through the api <api> <ppa> <Soyuz:Triaged> <https://launchpad.net/bugs/456764>
 * al-maisan falls off the chair :P
<mzz> I'm not sure if this is because I need to patch more of launchpad, the objects I'm accessing through launchpadlib don't actually have that attribute, or I need to regen some other file than the wadl
<wgrant> mzz: You've restarted your launchpadlib app?
<mzz> my launchpadlib app is a script, so yes
<wgrant> (you need to re-create the Launchpad object)
<mzz> let me pastebin some stuff
<wgrant> mzz: what if you browse the object in a browser? (add '/api/beta' to the front of the path to the object)
<mzz> ah, estimated_build_duration does show up there but is None
<mzz> huh, but buildduration shows up and isn't None
<wgrant> That's what I would expect.
<wgrant> They should be mutually exclusive.
<wgrant> Shouldn't they?
 * wgrant checks the code.
<mzz> not what I meant
<mzz> I mean my launchpadlib script isn't seeing record.buildduration on https://api.launchpad.dev/beta/ubuntu/+source/pmount/0.1-1/+build/19 but my browser does see it on https://api.launchpad.dev/api/beta/ubuntu/+source/pmount/0.1-1/+build/19
<mzz> I think I'm missing a step or using a wrong url somewhere on the launchpadlib end then?
<wgrant> You removed and regenerated the *dev* WADL?
<wgrant> There are two different WADL documents.
<mzz> ah, let's see
<mzz> I removed wadl-development.xml, not wadl-testrunner.xml. After doing that and restarting launchpad my attribute showed up in launchpad.dev/+apidoc, which I figured was a good sign
<mzz> do I need to do anything on the launchpadlib end?
<mzz> I'm not running launchpadlib on the same host as launchpad
<wgrant> wadl-development.xml should be the right one.
<wgrant> Hmmm.
<wgrant> You don't have a launchpadlib cache set up, do you?
<wgrant> The etag of the object will not have changed, so a cache could break things.
<mzz> I do! that's probably it, killing...
<mzz> (and logging back in... is there a switch on launchpad to turn off access control if you're in a hurry? :)
<wgrant> Not without hacking launchpadlib and wadllib and lazr.restfulclient to use the browser webservice, and that makes leonardr angry.
<wgrant> But remember that you can save credentials.
<mzz> that did it! thanks
<wgrant> Excellent.
<mzz> (my script is using Launchpad.login_with(scriptname, service_root=...) to get a launchpad object, which apparently automatically caches not just credentials but also the wadl)
<mzz> (I'm not even sure that's the right thing to call, it just looked like the easiest thing to call)
<wgrant> I normally use Launchpad.get_token_and_login, but I am probably behind the times.
<mzz> well, that was pretty easy (after I'd found out I needed to remove the wadl by hand, but I guess that makes sense since detecting it's gone stale is painful)
<mzz> (I mean the wadl on the server end)
<mzz> now I just need to figure out how to test this stuff, and see if I can expose a bit of information about the queue too.
#launchpad-dev 2009-10-25
 * kfogel is away: zzz
#launchpad-dev 2010-10-25
<wallyworld> anyone around today?
<spm> ... maybe
<wallyworld> hey spm. dumb question. for some reason when i run up a local instance of lp, i can't connect to http://launchpad.dev but must use http://launchpad.dev:8085
<wallyworld> that gets me the home page but most everything else is broken
<wallyworld> i can't see what's changed
<wgrant> wallyworld: Sounds like your Apache config is broken.
<wgrant> Is Apache running at all?
<spm> wallyworld: " most everything else is broken" <== would be my guess at what's changed ;-)
<wallyworld> wgrant: i checked the configs but  they are uncganed since months
<wallyworld> spm: i meant that other lp urls don't work unless i hack in the 8085 port
<wgrant> wallyworld: Maybe Apache has stopped starting automatically.
<wallyworld> wgrant: i'll check
<wgrant> spm: Did you fix edge, or has it mysteriously rectified its own issues?
<spm> wgrant: not touched it
<spm> wasn't aware there was/is a problem?
<wgrant> Ah, it looks like it was fixed about 8 hours ago.
<wgrant> It spent all weekend returning truncated responses, 502s and 503s.
<spm> orsum
<spm> although.... that surprises me. I was under the belief that as of late last week edge == prod
<wallyworld> wgrant: yeah, apache not running, restart fails but error log is 0 bytes. something weird going on :-(
<spm> wallyworld: apache2ctl configtest ?
<wgrant> wallyworld: How are you restarting it?
<wallyworld> wgrant spm: i found it. i moved my checkout dir and the branch rewrite path needs to be changed in the apache conf
<wgrant> Aha.
<wgrant> I was going to suggest that, but didn't think it would be fatal.
<wallyworld> i had a working dir called sandbox but i now need to work with db-devel hence took the opportunity to rename
<wallyworld> oh well, a trap for noobs like me
<spm> wallyworld: now multiply that over ... lots of servers - welcome to my world :-)
<wallyworld> and it get the error print i had to /etc/init.d/apache2 stop and then start even though it wasn't running in the first place
<wallyworld> spm: you truly have a job i would not want to do and am very grateful that you do it :-)
 * thumper waves
<wallyworld> thumper: can you hear the sound of crickets? pretty quiet today
<spm> too quiet even for crickets.
<wallyworld> what's that? you'll have to speak up so i can hear you over all the noise :-)
<spm> heh
 * jpds listens to the airco.
<adeuring> good morning
<bac> hi adeuring
<mrevell> Hello
<mrevell> Hey, I'm doing a rocketfuel-setup on a fresh Maverick install. It's complaining that launchpad-soyuz-dependencies is broken. I don't see anything on the launchpad-dev list about this. Any ideas?
<wgrant> mrevell: Not launchpad-dependencies too?
<wgrant> mrevell: You need to install Lucid's python-psycopg2 first, then run rocketfuel-setup.
<mrevell> wgrant, Thanks. When I tried launchpad-depends it said that python-psycopg2 was broken. However, installing it manually works. Cheers.
<mrevell> Oh, it's complaining about the version ... launchpad-dependencies wants 2.0.14 but 2.2.1-1unbuntu1 is the installation candidate. Bah.
<mrevell> I have the PPAs in my sources.list.
<mrevell> trying a dist-upgrade
<wgrant> mrevell: You'll need to manually install 2.0.14 from Lucid. It won't install an older version automatically.
<mrevell> Ah okay
<mrevell> Oh, I see Maris' fix on the list
<maxb> Unfortunately there is no nice way to fix this (other than making LP work with 2.2)
<maxb> I am deliberately not suggesting uploading a 2.2.1really2.0.14 package, because I don't want to encourage delaying the fix
<wgrant> Don't give anyone any ideas...
<bigjools> hello from UDS
<lifeless> oh hai
<mwhudson> rockstar: why is there a wifi network named after you?
<StevenK> Because he's a rockstar
<jml> hi
<noodles775> cd 15
<noodles775> heh, and morning.
<allenap> noodles775: Are you able to do a review for a very short shipit branch?
<lifeless> mthaddon: is the pqm chroot on praÃ© lucid now ?
<mthaddon> lifeless: it is, yep
<lifeless> mthaddon: are you aware of any pre-lucid machines or chroots or vms in the lp arena now ?
<mthaddon> lifeless: nope, we should be all lucid now
<lifeless> awesome.
<lifeless> mars: ^
<mars> mthaddon, cool!  lifeless, so we can make the announcement?  (I'll give you some time to think about it when you are not in a keynote)
<noodles775> allenap: Where's the branch? I'm at UDS but if it's tiny, shouldn't be a problem.
<allenap> noodles775: https://code.edge.launchpad.net/~allenap/shipit/remove-propertycache-adapters-bug-628762-shipit/+merge/39251
<allenap> Thank you!
<lifeless> mars: if there are no pre-lucid environments, we're fine.
<lifeless> mars: if there are, we aren't
<lifeless> mars: I can't tell if there are other thank asking tom :)
<noodles775> allenap: I think you need to subscribe me to the branch.
<mars> mthaddon, so just to confirm, we are now running Lucid on all machines, virtual machines, chroots, and so forth?
<allenap> noodles775: Done.
<mthaddon> mars: yes
<mars> mthaddon, awesome, thank you.
<mars> gary_poster, lifeless, then we can make the announcement - Launchpad is officially running Python 2.6 and Lucid
<gary_poster> great mars, go for it.  thanks
<StevenK> mars, gary_poster, lifeless: Should we also add PostgreSQL 8.4?
<StevenK> mars: By the way, does ec2 use 8.4 now?
<gary_poster> StevenK: to the announcement?  yes
<gary_poster> yes it does
<mars> cool, I'll add that too
<bigjools> what's the launchpadlib hack where you can grab a resource from a URL directly?
<mars> thanks for the reminder StevenK
<gary_poster> is it just .open?
 * gary_poster forgets but that's what comes to mind
<bigjools> I'll try it, thanks
<mars> bigjools, check the launchpadlib recipes/hacking page?
<bigjools> didn't know we had one!
<mars> bigjools, https://help.launchpad.net/API/Examples
<mars> bigjools, I think "Fetching an object's raw JSON" is what you want?
<lifeless> bigjools: I don't remember, but I think I made hydrazine use it
<lifeless> though lplib has the relative open feature itself now
<bigjools> mars: that's not what I want - I want to navigate to an object's URL directly
<bigjools> open() is the thing I think
<lifeless> lp:hydrazine, or if not that then its in lp:lptools
<abentley> jml: one option that's available to you for testing testtools on various python versions is daily builds.
<mars> bigjools, maybe you could you add an Example for what you do want to do then?  It sounds like more than one person has needed it
<bigjools> yeah will do
<bigjools> trying to remember on which object open() exists :)
<lifeless> Launchpad
<bigjools>     AttributeError: 'Launchpad' object has no attribute 'open'
<lifeless> mmm
<lifeless> really, look in hydrazine/lptools
<lifeless> I think its lptools milestone stuff now that I think about it
<allenap> noodles775: Thanks for the review. Can you mark the proposal (as a whole) as approved; I do not have permission to do so.
<bigjools> if I have launchpad.things['thing'] what gets called to get 'thing' on the 'things' collection?
<bigjools> IPersonSet didn't give me much inspiration
<noodles775> allenap: done.
<allenap> noodles775: Cool, thanks.
<bigjools> leonardr: hi, around?
<flacoste> jml: what is this Launchpad roundtable session at UDS?
<leonardr> bigjools, yes
<bigjools> leonardr: I am exposing a new top-level collection in the API and sinzui says that you know how to make indexing of it work
<bigjools> we used to have to hack lplib but apparently there's something else I can do in LP now
<leonardr> bigjools: what do you mean by indexing?
<bigjools> so I am adding lp.builders
<bigjools> and I want to do lp.builders['bob']
<bigjools> right now that blows up :)
<leonardr> bigjools: ok, you need to change launchpadlib
<bigjools> :(
<bigjools> gah
<leonardr> look at launchpad.py, eg. BugSet
<bigjools> will do, thanks
* flacoste changed the topic of #launchpad-dev to: login problems on qastaging | Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<bigjools> leonardr: are there any instructions for monkeys on how to update launchpadlib, make an egg, release it etc etc ?
* flacoste changed the topic of #launchpad-dev to: login problems on qastaging, matsubara and LOSA are investigating | Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<bigjools> gary_poster: perhaps you know? :)
<gary_poster> bigjools: let's try benji before I wade in ;-)
<gary_poster> benji, do you know?
 * benji reads.
<benji> -
<benji> gary_poster/bigjools: the only ones I know about (and follow myself) are in https://dev.launchpad.net/HackingLazrLibraries
<gary_poster> thought as much
<gary_poster> thanks benji
 * bigjools reads, thanks
<bigjools> this is Too Hard just to make a simple API change :(
<StevenK> bigjools: I think it's just due to the fact that IBuilder is top-level
<bigjools> StevenK: IBuilderSet being a collection is the problem
<StevenK> Ahh
<bigjools> top-level collection
<LPCIBot> Project devel build (149): STILL FAILING in 4 hr 6 min: https://hudson.wedontsleep.org/job/devel/149/
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | login problems on qastaging, matsubara and LOSA are investigating | Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<LPCIBot> Project db-devel build (96): SUCCESS in 4 hr 1 min: https://hudson.wedontsleep.org/job/db-devel/96/
<leonardr> bigjools, sorry i disappeared. if you give me a launchpadlib branch i can do a release. i'd rather do the release myself atm
<mwhudson> rockstar: hey
 * mwhudson writes on the merge proposal instead
<flacoste> lifeless: it's not tuesday yet!
<flacoste> hmm, well, ok, maybe it is now down under
<StevenK> flacoste: It is in New Zealand :-)
<StevenK> (It's been Tuesday in Australia for 6 hours)
* flacoste changed the topic of #launchpad-dev to: Performance Tuesday | login problems on qastaging solved | Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<lifeless> flacoste: its tuesday
<lifeless> flacoste: 0809 in nz
<StevenK> Attending UDS while in a NZ timezone would be terrible
<lamont> wgrant: pls don't move louvi or lakoocha between architectures until further notice.
<lamont> wgrant: just in case the idea occurs to you
<maxb> You can learn so much interesting trivia by googling canonical machine names :-)
<mwhudson> the fruit theme is getting fairly stretched now
<bigjools> we should start using cuss words
<bigjools> there's an endless supply
<mwhudson> i bet lamont can supply quite a few just by himself :)
<lamont> ISTR that next up is 'star names'
<lamont> mwhudson: stretched is not the same as done
<mwhudson> lamont: indeed
<jml> I may have to start using tarmac & merge queues.
<lamont> though we have definitely moved out of the yummy ones and into the toxic ones
<lifeless> jml: for?
<bigjools> self flagellation
<jml> lifeless: testtools
<lifeless> jml: why?
<lifeless> actual, face time may be best
<jml> lifeless: because supporting multiple Pythons is really hard
<lifeless> I need to focus on this session
<jml> lifeless: ok.
<lifeless> jml: hmm, my hudson should be live again shortly after I get home; the machine is turned on again. But in the break find me
<jml> lifeless: I've got hudson on mumak.net too
<jml> lifeless: just had to get backports for 2.7 etc.
<LPCIBot> Project devel build (150): STILL FAILING in 3 hr 41 min: https://hudson.wedontsleep.org/job/devel/150/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless][ui=None][no-qa] Fix local failures.
<lifeless> jml: woo
<lifeless>  fixtures-0.3.tar.gz (md5, sig)Source tarball. 2,136
<lifeless> download count s;)
<jml> lifeless: nice :)
<jml> I wonder what testtools has
<jml> less than fixtures :)
<lifeless> wow
<lifeless> yes
<jml> lifeless: I'm patching testtools now to have a subunit runner and testr.conf. feel the need for a review?
<lifeless> delighted to if you want one
<lifeless> I think its time to add useFixture to testtools
<jml> lifeless: thanks. lemme just find a .testr.conf to cargo cult :)
<jml> lifeless: yes.
<jml> lifeless: 0.9.8 is going to be a big release
<lifeless> I've just added details to fixtures
<lifeless> in 0.3.4
<jml> actually... testr.conf is too hard. I'll just land the subunit thing without review
<lifeless> hmm
<lifeless> toss the branch up
<lifeless> I'll do testr.conf for you
<lifeless> jml: why do you want a subunit run thing though? Isn't it just
<lifeless> python -m subunit.run testtools.test_suite ?
<jml> hmm.
<jml> oh yeah, that works.
<jml> no point in patching then :)
<jml> I mean, we have testtools.run
<jml> but I don't really care about adding a --subunit option to that
<lifeless> right, in fact the subunit run extends the testtools one I think
<lifeless> so older us though that wasn't the best way forward
<lifeless> s/gh/ght/
<jml> lifeless: :)
<lifeless> jml: testtools is packaged though; whats its popcon count
<jml> lifeless: I don't really know how to find out
<StevenK> popcon.ubuntu.com
<lifeless> http://popcon.ubuntu.com/
<jml> StevenK, lifeless: thanks, I just went there. I meant I don't know how to navigate popcon.ubuntu.com
<StevenK> 21544 python-testtools                1153    40   352    73   688 (Unknown)
<lifeless> so 1153 installs
<lifeless> jml: click on the white page beside 'all' in the 'inst' column
<jam> just heard from poolie, he has now landed in Orlando
<StevenK> Crumbs
<lifeless> \o/
<lifeless> jam: did beuno get hold of you?
<jam> yep
<lifeless> cool
<flacoste> lifeless: i have fixed the ppr report!
<lifeless> flacoste: \o/
<flacoste> expect a merge proposal soon
<flacoste> trying to generate the failed reports on devpad for confirmation
<LPCIBot> Project db-devel build (97): SUCCESS in 3 hr 34 min: https://hudson.wedontsleep.org/job/db-devel/97/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=665301]
<LPCIBot> SourcePackageRecipeBuild.destroySelf clean up referenced objects.
<thumper> morning
<thumper> gary_poster: what is new in storm 0.18?
<gary_poster> thumper: hi.  have not sent out message because stuck with permissions on uploading docs, but message will come out.  But anyway: see release notes https://edge.launchpad.net/storm/+milestone/0.18
<lifeless> jml: ping
<gary_poster> thumper: we care about (https://code.edge.launchpad.net/bugs/659078 and https://code.edge.launchpad.net/bugs/620615), and one we had cherry-picked into our version of 0.17 (https://code.edge.launchpad.net/bugs/620508)
<wgrant> bigjools: I'm pretty sure I didn't have to change launchpadlib last time I added a top-level collection.
<lifeless> Ursinha-afk: how is updating the tagger going ?
<lifeless> wgrant: the ['foo'] stuff is special cased
<lifeless> wgrant: but he didn't need that, so is not doing it now,
<wgrant> Right, exactly.
<wgrant> Great.
<bigjools> wgrant: you do if you want to add a __getitem__ to it
<bigjools> good morning anyway
<bigjools> what's the future like? :)
<wgrant> bigjools: Well, yes, but that's an evil unnecessary hack.
<StevenK> wgrant: Please send cookies, lottery numbers, and stock tips to those of us stuck in the past.
<wgrant> It is futureful.
<wgrant> But busy, being the last week of classes, and all that.
<wgrant> How goes UDS?
<StevenK> I don't know, what day is it?
<mwhudson> i think it's monday
<wgrant> Heh.
<bigjools> grar, intel gpu lockup
<wgrant> Not X crash when typing?
<wgrant> That's all I've had lately.
<bigjools> wgrant: uds seems to be fine so far
<bigjools> there's a known bug I think
<bigjools> but ScottK assures me the developer is here so I can bug him :)
<StevenK> Hm, I don't get X crashes while typing
<mwhudson> i've had x hang on me a few times
<StevenK> Then again, I'm using evil-eats-kittens-nvidia drivers.
<bigjools> nvidia is far more reliable for me.. Go figure.
 * gary_poster looks about for someone to do a quick librarian diagnostic log review.  Potentially very easy review--I just add twisted.python.log.msg calls.  Anyone up for it? :-D  https://code.edge.launchpad.net/~gary/launchpad/bug662912/+merge/39315
<wgrant> StevenK: It's sort of like that old bug where X didn't completely steal the keyboard from the TTY, so pressing Ctrl+C killed X. But it's not that.
<bigjools> I can look
<gary_poster> thank you bigjools
<bigjools> np
<wgrant> bigjools: Has Soyuz had another five years of work requested yet?
<bigjools> not yet, but I feel the wind changing
<bigjools> I'm looking for a pile of sand so I can stick my head in it
<StevenK> I hear Orlando has things that are called 'beaches', but they don't look like one.
<lifeless> gary_poster: I've made a note on your review re: the log target.
<bigjools> gary_poster: did you look at addSystemEventTrigger()
<bigjools> just wondering why you're using callWhenRunning in particular, but no big deal
<gary_poster> bigjools: eh, it seemed like a good idea at the time ;-)
<bigjools> gary_poster: ok it's not a problem, just wondered if aSET would do something better in this case
<bigjools> gary_poster: why are you changing the dev log target?
<bigjools> is it for "make run" or test logs?
<lifeless> jml: ping
<bigjools> lifeless: given that he's put it in the development config not the testrunner config it won't affect tests?
<gary_poster> bigjools: aSET better: semantically/idiomatically, or...?  It does the trick practically, so this is a idiomatic/semantic thing.  My Twisted is a bit rusty, so if you think it's better I'm happy to swotch
<gary_poster> switch
<bigjools> gary_poster: no need to switch, I was just wondering if it would log at a more convenient timing for your needs
<gary_poster> bigjools: dev log target is just for make run
<gary_poster> and before I believe the default is "-"
<bigjools> so it won't affect anything that lifeless worries about I think?
<bigjools> right
<lifeless> gary_poster: whats the current behaviour ?
<lifeless> gary_poster: it should be logging to a sane place without that setting
<bigjools> - is stdout
<gary_poster> lifeless: currently I can't find the output.  stdout is lost when this is a daemon
<wallyworld> thumper: abentley rockstar: standup?
<gary_poster> lifeless: and as bigjools said, this affects make run, not tests
<lifeless> gary_poster: yes, I did misunderstand that
<lifeless> bear with me a sec
<gary_poster> np
<gary_poster> if you find the output, perhaps it is a documentation issue--but I'd argue a "librarian.log" next to our other devel logs is still a pretty good place for it.
<bigjools> +1 from m
<bigjools> e
<gary_poster> thanks bigjools
<bigjools> gary_poster: welcome!
 * wallyworld actually wishes logs when running under development were written outside the source tree
<Ursinha> lifeless, just put the reviewed version in prod
<gary_poster> wallyworld: interesting.  I typically have two or three branches going at once, so having the logs separate can be convenient for me.  It's also nice that when I delete a branch, I'm also automatically cleaning out the logs.  If you have a compelling argument for it, though, I'd be interested.
<gary_poster> For later :-)
<wallyworld> gary_poster: otp. will respond in a minute
<gary_poster> wallyworld: np.  need to step out myself.
<Ursinha> lifeless, https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<gary_poster> lifeless: need to step out.  Will be back for your approval or suggestion on the MP when I return.
<bigjools> lifeless: since edge is not updating now we should remove the redirect I think?
<lifeless> gary_poster: ok, I see that it is running with -i
<lifeless> bigjools: https://code.edge.launchpad.net/~lifeless/launchpad/edge/+merge/39233
<lifeless> gary_poster: doing that librarian.log thing is ok; I echo wallyworlds sentiments about out of tree, but lets not do it willy-nilly
<lifeless> Ursinha: thanks!
<lifeless> gary_poster: sorry for leaping to conclusions
<bigjools> lifeless: \o/
<bigjools> lifeless: this is a monumental occasin
<bigjools> occasion, even
<lifeless> yes
<lifeless> gary_poster: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> gary_poster: you're up!
<Ursinha> lifeless, he's blocked in another bug
<StevenK> Anyone around to review a small test-change-only branch?
<lifeless> Ursinha: how so?
<lifeless> StevenK: shoot
<StevenK> lifeless: https://code.edge.launchpad.net/~stevenk/launchpad/cronscript-idsjob-testfix/+merge/39321
<lifeless> Ursinha: can you bump qatagger if its not running already ?
<Ursinha> lifeless, just did it
<lifeless> \o/ nice long list.
<wallyworld> gary_poster: re: logs - one reason is that when grepping for stuff from the project root, you often get 1000s of false +ves since the log files are searched as well. plus they show up in my ide since they are in the root dir and not a sub folder that can be excluded easily. could we at least have a single sub dir for logging/trace and other similar stuff (eg the millions of thread-xxxx.request files or the (new) generated
<wallyworld> appserver config files)?
<flacoste> lifeless: https://code.edge.launchpad.net/~flacoste/launchpad/ppr-constant-memory/+merge/39324
<flacoste> the branch name is a lie
<jml> heh
<gary_poster> lifeless: librarian.log: ack, thanks.
<lifeless> flacoste: times_array[:,0] <- what?
<gary_poster> wallyworld: would be +1 on subdir; I'd like that
<flacoste> lifeless: that's numpy, that's getting the first column of the matrix
<flacoste> the matrix is app_time, sql_statements, sql_time
<lifeless> I would love to see more tests of this
<wallyworld> gary_poster: cool. it's nice to keep all that sort of messy stuff away "under the rug" so to speak. want me to do the code for it?
<flacoste> automated tests you mean?
<gary_poster> wallyworld: sure, thanks!  You can hit me for the review if you want (tomorrow; leaving soon :-) )
<gary_poster> (and don't feel like you need my review; just offering)
<lifeless> flacoste: yes
<wallyworld> gary_poster: i've got a few things on my plate today but will get to it in the next day or so
<lifeless> I'm glad you qa'd thoroughly
<gary_poster> cool
<lifeless> its sad that you needed to
<wallyworld> gary_poster: also, thanks for offering the review. i don't mind who does it, so long as it is done :-)
<gary_poster> :-)
<gary_poster> understood, wallyworld
<flacoste> lifeless: it's a lame excuse, but there were none
<flacoste> yeah, manual qa sucks
<lifeless> flacoste: yeah, I understand :)
<rockstar> lifeless, can you come find mtaylor and I in the Bonaire hallway?
<flacoste> lifeless: thanks
<lifeless> rockstar: omw
<rockstar> flacoste, lazr-js is not landed because I cannot get ec2 to actually complete a test run.  I've talked to deryck about it at length.
<thumper> flacoste: ping
<thumper> gary_poster: ping?
 * thumper sometimes wishes that there were a few less time zones between NZ and NY
<thumper> I'd just like to say that ORMs can really suck for efficiency
#launchpad-dev 2010-10-26
<LPCIBot> Project devel build (151): STILL FAILING in 3 hr 37 min: https://hudson.wedontsleep.org/job/devel/151/
<LPCIBot> Launchpad Patch Queue Manager: [r=abentley][ui=none][bug=620615,
<LPCIBot> 659078] change Launchpad to use Storm 0.18
<wallyworld> thumper: they can suck, you just have to know how to tune them :-) i think storm may not be as advanced as say hibernate in this area
<thumper> wallyworld: I'm attacking a place in the code where there is simplistic thinking
<wallyworld> hibernate has all sorts of goodies, including support for batch updates and mixing native sql with it's object query language (HQL)
<thumper> wallyworld: so does storm
 * wallyworld needs to learn more storm
 * wallyworld wonders, does storm support per query fetching strategies and pre-fetch sizes, or lazy instantiation of collections?
<thumper> wallyworld: kinda
<wallyworld> it's stuff like all that which is needed to properly tune one's app of course
<LPCIBot> Project db-devel build (98): SUCCESS in 3 hr 37 min: https://hudson.wedontsleep.org/job/db-devel/98/
<LPCIBot> Launchpad Patch Queue Manager: [rs=thumper][ui=none][no-qa] Manual merge of stable into db-devel.
<wallyworld> hey thumper, i'm likely the last to realise - i suppose you know about http://blogs.computerworld.com/17224/ubuntu_changes_its_desktop_from_gnome_to_unity
<thumper> no
<wallyworld> someone emailed it to me
 * wallyworld gets out the tissues and blubbers over how unloved KDE is :-(
<jcsackett> wallyworld: if it helps, i know in OS X circled KDE 4 got serious circulation. :-P
<jcsackett> s/circled/circles/
<wallyworld> it would help more if KDE had more support on Ubuntu :-)
<wallyworld> at least maybe there's a growing realisation that Qt is good. one step forward...
<jcsackett> it helps that Qts license situation got cleaned up.
<jcsackett> (or the FUD around it got cleaned up; whatever your preference might be)
<wallyworld> jcsackett: same thing really, given the impact it had...
<jcsackett> wallyworld: fair point.
<jcsackett> wallyworld: these days i'm a wmii man, if you want to talk no love. :-D
<wallyworld> :-)
 * thumper afk for walk to shops and lunch 
<maxb> Does anyone have any idea why ~vcs-imports is associated with https://code.edge.launchpad.net/~brian-rogers/gnome-power/report-percentage/+merge/39329 ?
<jelmer> maxb, it's probably the default reviewer for some reason
 * thumper needs coffee badly
 * thumper has coffee and is screaming at his screen
 * thumper dodges a tumbleweed
<thumper> why is that damn test being run twice?
<thumper> lifeless: ping
 * wgrant rolls some more tumbleweed in.
<lifeless> hi
<wgrant> It's not one of those tests which is confusingly run with the same name for several object types?
<thumper> lifeless: when running a doc test with "bin/test -vvt message-holds-xmlrpc.txt" it runs twice
<thumper> lifeless: any idea?
<thumper> lifeless: btw, this will fix the mailman xmlrpc call when it makes 5000 database calls
<lifeless> nice
<lifeless> what causes that ?
<thumper> simplistic iteration over a large result set
<lifeless> lack of eager loading ?
<thumper> and silly logic
 * thumper is enfixorating
<thumper> I'm changing it slightly
<thumper> so will just do 6
<thumper> I could attempt to make it 2, but I don't see the benefit for that
<thumper> the logic would be harder to follow if I did
<thumper> 5000 -> 6 is much better
<lifeless> I think 6 queries is fine ;)
<lifeless> sometimes less is worse too
<lifeless> depending on the related data density
<rockstar> deryck, lp:~rockstar/launchpad/javascript-refresh
<thumper> lifeless: this one: OOPS 1758XMLP341
<lifeless> nice
<lifeless> thumper: I see the double run in trunk
<lifeless> thumper: its being loaded twice by whatever module constructs it
<thumper> lifeless: is it all doc tests or just some?
<lifeless> lp/registry/tests/test_doc.py
<lifeless> will be a buglet in there
<lifeless> ignore for this branch IMO
<thumper> ok
<thumper> lifeless: are you going to file a bug for it?
<thumper> lifeless: if it is simple, I could fix in this branch
<lifeless> its late + jetlag
<lifeless> I haven't looked
<thumper> lifeless: ah, it seems to have two different setup methods
<thumper> lifeless: so it is testing things twice
<thumper> one with the real mailinglist api and another with a fake
<jcsackett> thumper: you mean it's doing two tests in one via diff setUps?
<thumper> kinda
 * jcsackett pulls up code.
<jcsackett> or not; i forgot my dev machine doesn't multitask well when updating.
<jcsackett> and here i thought i might be helpful. this is a test in registry?
<thumper> it is, and there isn't a problem
<thumper> just my misunderstanding
<jcsackett> thumper: ah, dig.
 * wgrant WTFs at launchpad-users.
<lifeless> oh?
<wgrant> The last email to it.
<lifeless> on oct 15th?
<wgrant> About 47 minutes ago.
<lifeless> swiss branded watches?
<jcsackett> chain letter about returning vets, looks like.
<wgrant> It looks like it.
<jcsackett> that does seem about as Off-topic as you're going to get...
<lifeless> \o/ edge bye bye
<wgrant> :( poor edge
<lifeless> hah
<jcsackett> i had only just met poor edge.
<bac> wgrant: the previous one about getting mugged in KL was mighty off topic too
 * wgrant remembers back when it was 'beta', restricted access, and had the shiny new UI.
<wgrant> bac: Ah yeah, I forgot about that one.
<jcsackett> bac: that one is however the result of a compromised account.
<jcsackett> my sister had some people get into her yahoo account a few months back and received (nearly) word for word that email. it's apparently a pretty popular scam.
<thumper> lifeless: has edge gone?
<wgrant> Heh. I was about to ask if someone had turned off the redirect.
 * thumper imagines some people getting a little miffed at no recipes ...
<wgrant> Then I noticed the last rev.
<EdwinGrubbs> thumper: ping
<thumper> EdwinGrubbs: hi
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (152): FIXED in 3 hr 37 min: https://hudson.wedontsleep.org/job/devel/152/
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Refactor page-performance-report to use
<LPCIBot> less memory by using a SQLite3 db to hold the requests and
<LPCIBot> generating statistics for only one key at a time.
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=none][bug=659085] Remove
<LPCIBot> getBugNotificationRecipients()
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=none][bug=628762] Ditch property cache adapters in
<LPCIBot> favour of plain old Python. The IPropertyCache adapter is
<LPCIBot> replaced by get_property_cache() and the IPropertyCacheManager
<LPCIBot> adapter is replaced by clear_property_cache().
<wgrant> !
<thumper> now that is a bit of IRC spam
<wgrant> Does this mean we are green?
<wgrant> Yes!
<wgrant> Both are green!
<wgrant> For the first time!
<EdwinGrubbs> thumper: according to bzr annotate, you worked on canonical.launchpad.webapp.sorting.sorted_version_numbers. I'm wondering why you the numbers are sorted in the reverse order as the letters. For example, [3, 2, 1, A, B, C], which looks wrong if a version combines letters and numbers like [5a, 4a, 3a, 3b, 3c].
<thumper> holy crap that is quite a while ago...
<EdwinGrubbs> thumper: I'm in the process of moving the sorting into the db so that the results can be batched. It looks like this is just used for the productseries.
<thumper> EdwinGrubbs: I think the reasoning was we wanted newer version numbers first
<EdwinGrubbs> thumper: ok, so there shouldn't be a problem in making the letters sort descending like the numbers?
<thumper> ahh...
<thumper> find out where it's used forst
<thumper> first
<thumper> I really don't remember
<EdwinGrubbs> thumper: so far, it looks like it is just used for https://qastaging.launchpad.net/launchpad/+series
<EdwinGrubbs> but I'll double check with sinzui
<thumper> EdwinGrubbs: in which case it is all yours :)
<thumper> who whatever you like
<thumper> s/who/do/
<EdwinGrubbs> thanks
<LPCIBot> Project db-devel build (99): FAILURE in 3 hr 39 min: https://hudson.wedontsleep.org/job/db-devel/99/
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11791
<LPCIBot> included.
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Fix the models and accompanying tests
<LPCIBot> for Branch Merge Queues to use the secured utilities, etc.
<rockstar> How does launchpad not depend on beautiful soup anymore?
<thumper> rockstar: eh?
<rockstar> thumper, launchpad used to use beautiful soup a lot, but apparently it doesn't anymore.
<thumper> ah ...
<thumper> really?
<rockstar> Also, I SHOULD go to bed, but deryck and I is watching teh football.
<rockstar> thumper, yeah, I just added soupmatchers to Launchpad and it needed BeautifulSoup.
<thumper> rockstar: still used by canonical.launchpad.testing.pages
<thumper> rockstar: it may be an egg
<rockstar> thumper, no, it wasn't in versions.cfg or setup.py
<rockstar> Weird.
<rockstar> At least, buildout couldn't find it, even when I just installed the package.
<thumper> rockstar: it is LP telling you to sleep
<thumper> :)
<rockstar> thumper, probably.
<bac> matsubara-afk: ping me when you arrive
<bac> rockstar: who is playing?
<LPCIBot> Project devel build (153): FAILURE in 3 hr 38 min: https://hudson.wedontsleep.org/job/devel/153/
<LPCIBot> * Launchpad Patch Queue Manager: [r=julian-edwards][ui=none][bug=662912][incr] Add diagnostics for bug
<LPCIBot> 662912
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Remove the beta redirect and is_edge facilities.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (100): FIXED in 3 hr 34 min: https://hudson.wedontsleep.org/job/db-devel/100/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11792
<LPCIBot> included.
<al-maisan> Hello! I cannot push branches to launchpad. Can somebody please look into this?
<al-maisan> Here's the error I get: $ bzr push
<al-maisan> Using saved push location: bzr+ssh://bazaar.launchpad.net/~al-maisan/landscape/edit-computer
<al-maisan> Traceback (most recent call last):
<al-maisan>   File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr", line 140, in <module>
<al-maisan>     exit_val = bzrlib.commands.main()
<al-maisan>   File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/bzrlib/commands.py", line 1191, in main
<al-maisan>     _register_builtin_commands()
<al-maisan>   File "/srv/bazaar.launchpad.net/production/launchpad-rev-9885/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/bzrlib/commands.py", line 182, in _register_builtin_commands
<al-maisan>     import bzrlib.builtins
<al-maisan> ValueError: bad marshal data
<al-maisan> bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist.
<al-maisan> sorry
<al-maisan> meant to paste pastebin url
<adeuring> good  morning
<wgrant> rockstar: BeautifulSoup is included via the Java Mentality.
<mrevell> Hi
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (154): FIXED in 3 hr 35 min: https://hudson.wedontsleep.org/job/devel/154/
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless,thumper][ui=none][no-qa] Slightly fix the IDSJob tests.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=664012] Fix the oopses raided by +branch
<LPCIBot> navigation when there is no referrer to be 404s.
<bigjools> good morning
<wgrant> Morning.
<mwhudson> jam: staging codehosting seems to be working
<mwhudson> this surprises me :-)
<jam> mwhudson: well, we haven't updated the config for the new code yet
<mwhudson> oh of course
<lifeless> poolie: bug 666765 is also on my nice-to-have for scopes
<StevenK> lifeless: Didn't you mention a fix for the "No module named mailman.monkeypatches.defaults" ? I seem to recall it involved wielding rm over a directory
<lifeless> rm -rf lib/mailman; make
<jml> why is stdout a text stream rather than a bytes stream in Python 3?
<lifeless> because python three
<lifeless> jml: hows your session
<jml> lifeless: not exactly riveting
<bigjools> lifeless: so, running tests locally I get a lot of errors about librarians either running when they should not be or vice versa
<bigjools> this Is Bad.
<rockstar> james_w, can we chat at some point about soupmatchers?
<bigjools> lifeless: looks like some layer issues
<rockstar> bigjools, yeah, I've had those problems for about three weeks.
<rockstar> bigjools, did you bin/kill-test-services ?
<bigjools> yes
<rockstar> bigjools, where is you?
<bigjools> rockstar: boner 6
<rockstar> bigjools, is there a session in there, or are you loitering?
<bigjools> rockstar: linaro session
<rockstar> bigjools, ah.
<bigjools> rockstar: but I may duck out RSN
<rockstar> bigjools, okay.  I'm at the end of the arterial hallway.
<lifeless> bigjools: last time I saw that I had a prod librarian running
<bigjools> I am getting motivated to fix test_on_merge
<bigjools> rockstar: wtf is the arterial hallway? :)
<lifeless> antigua
<rockstar> bigjools, the hallway that acts as artery to all the dead end hallways.
<bigjools> rockstar: which end?
<poolie> lifeless, replied
<poolie> lifeless, please tag them so i can find them later
<bigjools> lifeless: it was a stale pidfile
<lifeless> jml: :(
<james_w> rockstar, sure thing
<rockstar> james_w, are you going to be in a session after the one that just started?
<james_w> rockstar, yes. I think my afternoon is quieter though
<james_w> rockstar, or lunch?
<rockstar> james_w, lunch will work.
<StevenK> Is it just me or is codehosting down?
<fjlacoste> lifeless: around?
<lifeless> ys
<lifeless> fjlacoste: yes
<fjlacoste> lifeless: i need your advice, i'm working on a version of the ppr report that computes everything using SQL, that will operate in constant memory
<fjlacoste> lifeless: i have all the stats covered except the median
<fjlacoste> i know how to compute it, but it is very expensive
<lifeless> fjlacoste: btw, disk space on sodium - we had to kill your thing
<fjlacoste> i know
<fjlacoste> that's fine
<fjlacoste> we can sort that out later
<lifeless> it was using 5% of the disk :)
<fjlacoste> 2.9G
<lifeless> yeah
<lifeless> thats ~ 5%
<fjlacoste> it needs a bigger disk :-)
<fjlacoste> anyway
<fjlacoste> so the way i can compute the median is by sorting on the column and using LIMIT 1 OFFSET n/2 to get the value
<fjlacoste> but this seems like very expensive to do given that we are taking about millions of records
<fjlacoste> and we are computing 3 medians
<fjlacoste> and we can only do that by key
<lifeless> WHERE Rank = (SELECT (COUNT(*)+1) DIV 2 FROM
<fjlacoste> ah
<lifeless> also
<lifeless> http://wiki.postgresql.org/wiki/Aggregate_Median
<lifeless> though I haven't looked into that implementation
<fjlacoste> lifeless: i'm using sqlite
<lifeless> oh
<lifeless> :P
<fjlacoste> i was asking if you mind dropping it :-)
<fjlacoste> but your rank thing gave me an idea
<lifeless> fjlacoste: hmm, so if the column is indexed grabbing the median should be darn cheap
<fjlacoste> the question is how do I index the column relatively cheaply
<lifeless> insert the data
<lifeless> index the column
<lifeless> then start querying
<lifeless> oh
<lifeless> you'll want to increase the sqlite cache size too
<lifeless> because its like 2MB or something insane by default
<lifeless> give it 200MB or so
<flacoste> create an index on category, time for example?
<flacoste> ok
<lifeless> yeah
<lifeless> remember that the query prefix has to match the index
<flacoste> right
<lifeless> category,time won't help time only sorts
<lifeless> IIRC
<flacoste> well i need the rank of time within each category
<flacoste> (or url, or pageid)
<lifeless> yack
<jcsackett> lifeless: any chance you could help me think through bac's notes on bug 652156 and bug 652149?
<jcsackett> basically, it looks to me like the timeouts there aren't related to either bug/branch.
<lifeless> qastaging has a 10second timeout
<abentley> lifeless: do you mean yak (-shaving), yuck, or some combination?
<lifeless> I meant 'acl'
<lifeless> bah
<lifeless> 'ack'
<lifeless> jcsackett: so any page that has a timeout bug open (https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=timeout) is almost certain to timeout on [qa]staging too
<lifeless> jcsackett: I would not stress about incidental timeouts there (but I *would* fix the page to not timeout at all :P separately :) )
<mwhudson> mtaylor, jam: launchpad/bzr planning over lunch?
<jcsackett> lifeless: that makes sense. so bac and i can mark ours qa-ok if the functionality is working (aside from the branchbug timeout)?
<lifeless> jcsackett: as long as the page is already timing out with bug, production oops)
<flacoste> lifeless: actually, there is no Rank expression in sqlite
<flacoste> and generating the rank seems as costly as getting the median using iterative limit/Offset
<lifeless> flacoste: how slow is slow
<flacoste> several hours
<flacoste> actually, i don't know
<flacoste> just guessing here
<flacoste> i'll measure and see
<flacoste> assess
<jcsackett> lifeless: so, there isn't a bug filed against project/+branches, but person/+branches has one (bug 627945) and the query is the same in both. i'm thinking either add detail to that bug or file a new bug for project linked to it. thoughts?
<mwhudson> mtaylor, jam: which isn't right now, of course
 * mwhudson was just getting hungry and optimistic
<flacoste> lifeless: http://oreilly.com/catalog/transqlcook/chapter/ch08.html#Calculating%20a%20Median has an actual elegant solution :-)
<jam> mwhudson: sounds good
<jam> I'm pretty sure poolie is interested too
<lifeless> jcsackett: add the page id for the other case to the title, done.
<jcsackett> lifeless: fantastic. thanks.
<flacoste> hmm, that query does look expensive
<lifeless> flacoste: up the cache size as a start
<flacoste> lifeless: ack
<lifeless> you could denormalise the counts per category
<lifeless> given you're starting with a fresh db each run, right ?
<mtaylor> mwhudson: yes
<mwhudson> jam, mtaylor: i'll loiter by the registration desk
<mwhudson> poolie: ^^
<lifeless> jml: https://code.edge.launchpad.net/~lifeless/testtools/fixtures/+merge/39388
<LPCIBot> Project db-devel build (101): SUCCESS in 4 hr 4 min: https://hudson.wedontsleep.org/job/db-devel/101/
<jml> lifeless: reviewed
<lifeless> gary_poster: http://pypi.python.org/pypi/storm - needs 0.18 ?
<mars> the PyPi version is almost a year old
<gary_poster> lifeless, yes, remaining release of storm (per the release instructions) is stalled until Jamu gives me privs to upload docs or does it himself.  There are a number of small things remaining (also announcing it on ML, for instance).
<gary_poster> For PyPI, Storm didn't release 0.17 there and it's not on their release checklist, but I got privs to do it anyway, so I'll do it then.
<lifeless> gary_poster: Jamu here, go ahead and release, I'll deploy docs later.
<gary_poster> Cool Jamu, thanks
 * mars loves UDS :)
<mwhudson> i am inpressed by the fact that people are just registering kernel imports, and they basically work
<mwhudson> it would be much better if we supported stacking for import branches sensibly but well
<jelmer> mwhudson: +1
<jelmer> it shouldn't really be all that hard if I understand correctly
 * mwhudson quickly reaches for his "not a launchpad developer" hat
 * jelmer reaches for his "not a launchpad-code developer" hat :-)
<mwhudson> jelmer: i think the bug report on this describes what to do fairly clearly
<mwhudson> i hope so,  becuase i'm fairly sure i knew what to do at one point and certainly don't any more
<jml> lifeless: do you still thing that https://bugs.edge.launchpad.net/testtools/+bug/584824 is still worth doing?
<jml> think
<jml> lifeless: also, https://bugs.edge.launchpad.net/testtools/+bug/666923 wrt previous discussions
<jkakar> gary_poster: Hi!
<gary_poster> jkakar: hi :-)
<lifeless> deryck: hey
<deryck> hi lifeless
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<deryck> looking....
<deryck> slow openid dance
<lifeless> deryck: so https://bugs.qastaging.launchpad.net/ubuntu/+source/xorg/+bug/10000 looks ok
<lifeless> deryck: I'm not entirely sure how to validate it further
<deryck> lifeless, yeah, I'm looking at the changes now to see.
<deryck> lifeless, I smell a bad rev.  https://bugs.qastaging.launchpad.net/bugs/10000/+bug-portlet-subscribers-content
<deryck> hmmm, maybe that's not gmb's
<lifeless> deryck: whats wrong with it ?
<deryck> lifeless, that link OOPS for me.
<deryck> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1760QS46
<deryck> oh, that's security.cfg permissions again.  Like bug expiry script.  We missed something in pg 8.4 migration, I guess.
<deryck> lifeless, ok, so gmb's is qa-ok.  Visiting https://bugs.qastaging.launchpad.net/ubuntu/+source/xorg/+bug/10000/+subscribe confirms the new code is hit and works fine.
<lifeless> deryck: will you mark it as such? the two bugs ?
<deryck> lifeless, indeed.
<lifeless> thanks
<deryck> lifeless, do you not get an OOPS at the first URL I provided?  the portlet one.
<lifeless> yes, I do, i thought it might be transient but its very quick
<deryck> lifeless, transient?  You mean timeout?  I get ProgrammingError.
<lifeless> deryck: I meant I thought it might be a slow db or something
<deryck> lifeless, ah, ok
<deryck> lifeless, we should talk more about this after we get out of sessions, since I saw the same on expiry script test runs.
<bigjools> THE EAGLE HAS LANDED
<bigjools> where eagle == new buildd manager
<deryck> bigjools, awesome!
<bigjools> 7k branch with no mechanical changes!
<thumper> bigjools: what does it give us?
<bigjools> asynchronous comms
<bigjools> and far, far better failure detection and dealing with failures
<bigjools> lp:~julian-edwards/launchpad/builderslave-resume if you want gory details :)
<deryck> hi thumper.  Being at UDS makes us overlap for once. :-)
<rockstar> james_w, are you going to have some bandwidth this afternoon?
<thumper> deryck: hey
<james_w> rockstar, the next session looks pretty good to me
<rockstar> james_w, okay, I think I need to make some hard choices here.
<james_w> rockstar, 6pm would also work for me
<rockstar> james_w, yeah, that won't work at all.  Next session is good, I'll just chat with aq before the next session.
<james_w> rockstar, well, that was the session I was thinking going to as well :-)
<james_w> rockstar, are you here all week?
<rockstar> james_w, indeed I am, but I want to land some work before next week.  :)
<james_w> rockstar, sure :-)
<james_w> rockstar, where are you now?
<rockstar> james_w, I'm in the game development session, but it's a bust, so I can get out now if you can.
<james_w> rockstar, I can
<james_w> rockstar, outside the plenary room?
<rockstar> james_w, I'm in the hall.
<jml> bigjools: grats!
<rockstar> \o/ BrowserTestCase just got a whole lot better.
<bigjools> jml: and thanks to you!
<bigjools> jml: FWIW I hammered it on DF with a load of production builders that were borrowed and it didn't put a foot wrong.
<jml> bigjools: sweet.
<lifeless> rockstar: what got better about it ?
<bigjools> jml: of course, I just set myself up for a big fall :)
<jml> bigjools: sure. :)
<jml> bigjools: it's not going to work the first time. never does.
<rockstar> lifeless, I'm teaching it about james_w's soupmatchers.
<bigjools> kiko's 1st law of software?
<jml> bigjools: exactly
<jml> lifeless: https://code.launchpad.net/~jml/launchpad/atexit-warning/+merge/39403
<lifeless> jml: I saw, i don't understand it. It seems wrong
<jml> lifeless: that's why I wanted to talk about it :)
<lifeless> why not just remove the atexit call
<lifeless> and if things go screwy fix the code that isn't using a finally where needed.
<jml> lifeless: I don't know why the atexit call was there. it seems like a belt-and-braces thing
<lifeless> I think its because zope.testing stops teardown when a NotImplemented is enoucountered except in the last runner instance
<jml> lifeless: I guess I should do some archeology and find out why it was added.
<jml> lifeless: ok. I'm not sure what to do with that though.
<lifeless> jml: change zope.testing to not mess about
<lifeless> if a layer is setUp, it should tear it down
<lifeless> perhaps we could use Fixture to do that ;)
<jml> lifeless: I don't want to fix zope.testing to silence this warning
<lifeless> jml: the warning is a symptom of the root cause
<lifeless> the root cause is that teardown is inconsistently called
<lifeless> and because of that we have:
<jml> lifeless: the root cause being that layers is a pos
<lifeless>  - ordering problems
<lifeless> jml: well, what do you want to do?
<lifeless> jml: I don't want the fixture code to silently accept teardowns when torn down
<lifeless> that seems prone to hiding things
<jml> lifeless: it's not doing that, there's a special tear down for atexit, since there's no way of cancelling atexit calls
<lifeless> jml: you can use atexit._exithandlers.remove()
<jml> lifeless: you think that would be better than the code in the patch?
<lifeless> yes
<lifeless> because the ugliness would be outside the island of sanity I've created
<lifeless> better would be fixing the root cause: zope.testing
<jml> I'm not going to fix zope.testing to address this issue
<lifeless> well, I may :)
<jml> lifeless: you're welcome to it
<lifeless> jml: I'm not terribly concerned about pushing it upstream, because I think layers is so poor, I want a good platform for migrating from though.
<lifeless> jml: I want us to stop papering over things like this.
<jml> lifeless: sure, me too. but to me this is an incremental improvement. we're using atexit today to paper over an issue, and it's generating a warning that has little to do with the underlying issue....
<jml> lifeless: I'm patching it to not generate useless the warning.
<lifeless> StevenK: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html tag, you're it.
<lifeless> jml: but, you're:
<lifeless>  - duplicating code
<lifeless>  - adding a silent-wrong code path
<lifeless>  - leaving the underlying issue open
<lifeless> jml: I think its a much better use of your time to just fix the basic cause
<jml> lifeless: no, it's not. because that's a significantly larger amount of work, and I've got other things I want to do.
<lifeless> jml: I'd be happy with a change that didn't duplicate code and make the server.py module less clear
<jml> lifeless: ok. I'm moving to atexit._exithandlers now
<jcsackett> i may have missed an email on this topic, but does launchpadlib work on qastaging?
<lifeless> it should, but it may need a glue update
<jml> lifeless: patch updated.
<jcsackett> hm. question the second: is lplib supposed to work with regular staging?
<LPCIBot> Project db-devel build (102): SUCCESS in 3 hr 36 min: https://hudson.wedontsleep.org/job/db-devel/102/
<LPCIBot> Launchpad Patch Queue Manager: [rs=lamont][ui=none][no-qa] Update the changelog for launchpad-buildd.
<jml> lifeless: yes.
<jml> jcsackett: yes.
<jml> (sorry lifeless)
<jcsackett> jml: thanks. i must just have something wrong in my setup; staging is telling me that it can't connect b/c it can't forward a request.
<thumper> morning fellow hackers
<jcsackett> i'll continue futzing with it.
<jml> jcsackett: hmm.
<jml> thumper: hi
<thumper> jml: do you have a few minutes?
<jml> thumper: I will very soon.
<jml> thumper: in a session right now
<thumper> jml: well, I have the standup shortly, and a talk with flacoste after that
<jml> thumper: when are you next free?
<thumper> jml: probably in 1.5 hours
<jcsackett> jml: if that "hmm" indicates interest, the response is pasted here https://pastebin.canonical.com/39102/
<flacoste> thumper: i'm free whenever you are
<flacoste> i know that our call is only in 30 mins
<jml> jcsackett: that looks like a bug that a losa ought to know about
<jcsackett> jml: yeah? okay. i was assuming my own screw up, but i'll go tell one.
<jml> jcsackett: I don't know
<jcsackett> jml: dig. i'm getting a losa now.
<jml> thumper: I can be around in 1.5hrs time
<thumper> jml: ok
<wallyworld_> thumper: abentley: standup?
<lifeless> StevenK: ping
<StevenK> lifeless: O hai
<lifeless> I can has QA? https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> (Is it deployable)
<StevenK> lifeless: Yes, but let me finish what I'm in the middle of
<StevenK> I was unblocked to do so on Friday, and then there was a weekend and a plane trip
<LPCIBot> Project devel build (155): SUCCESS in 3 hr 56 min: https://hudson.wedontsleep.org/job/devel/155/
<lifeless> StevenK: bigjools I've requested a review of what was qa'd. Please unblock the queue as soon as possible.
<lifeless> or tell me enough to do it myself
<bigjools> lifeless: pardon?
<lifeless> s/review/deploy/
<bigjools> lifeless: need moar context
<lifeless> bigjools: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> bigjools: 9 day old unqa'd patch
<lifeless> bigjools: (which is to be expected, we're only just in the actually-use-it stage of RFWTAD
<bigjools> lifeless: Steve was in the middle of QAing that as far as I know
<lifeless> cool
<thumper> rockstar: ping
<jml> thumper: ping
<thumper> jml: just finishing off with flacoste
<jml> thumper: ok.
<wgrant> "The size of the diff (7194 lines) is larger than your specified limit of 1000 lines" D:
<jml> wgrant: it's a big patch.
<StevenK> lifeless: Are you saying "Please QA this right now", in leiu of say, dinner?
<StevenK> lifeless: And it would take me longer to explain how to QA it than to just do it myself. :-)
<flacoste> lifeless: what's the status of removing edge?
<flacoste> i actually need to go afk, let's talk about it tomorrow
#launchpad-dev 2010-10-27
<lifeless> flacoste: we're waiting for my patch disabling the edge redirect code to propogate through QA
<wgrant> lifeless: What about recipes?
<lifeless> wgrant: we're waiting for N things
<lifeless> wgrant: one at a time
<wgrant> lifeless: But if the redirect removal lands, then recipes vanish from Launchpad.
<wgrant> lifeless: Which may be slightly undesirable.
<wgrant> s/lands/deloys/
<wgrant> +p
<lifeless> huh, no
<lifeless> food, whilst I leave you to look closer.
<LPCIBot> Project devel build (156): FAILURE in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/156/
<LPCIBot> * Launchpad Patch Queue Manager: [r=bac][ui=none][bug=666580] Make getMessageDispositions much more
<LPCIBot> efficient in the number of DB queries.
<LPCIBot> * Launchpad Patch Queue Manager: [r=jml][ui=none][no-qa] Remove with_statement.
<LPCIBot> * Launchpad Patch Queue Manager: [r=jml][ui=none][bug=54946, 496574, 497282, 611258,
<LPCIBot> 618955] New almost-fully-asynchronous buildd-manager with lots of hot
<LPCIBot> sexy bug fixes.
<wgrant> lifeless: I see nothing in that MP which enables BFB on lpnet.
<wgrant> Unless you've also changed the prod configs, I guess.
<lifeless> wgrant: disabling the edge redirect doesn't remove the edge appservers
<lifeless> and yes, the prod configs have (naturally) changed
<wgrant> lifeless: It doesn't remove them, but it stops people from being redirected, so it effectively removes them.
<thumper> lifeless: did you change the prod config to enable recipes?
<thumper> lifeless: it isn't is_edge
<lifeless> no
<lifeless> we're adding a feature scope to support recipes
<lifeless> we're stalled on the edge removal on that
<wgrant> ... that is what I was asking.
<wgrant> Aha.
<lifeless> wgrant: see lp-foundations bugs
<thumper> lifeless: why stalled on edge removal?
<lifeless> jml doesn't want recipes on prod for all users
<lifeless> only beta
<lifeless> (and preferrably a dedicated smaller beta team)
<lifeless> so we need to add a scope that selects teams.
<thumper> ok
<lifeless> thumper: there's no panic on removing edge, we have time to do it methodically
<thumper> ok
<lifeless> all due haste, no panic.
<wgrant> lifeless: So you just CP'd up to r11738?
<lifeless> deployed, yes.
<wgrant> Hopefully the publisher won't explode.
<lifeless> CP's no longer exist.
<wgrant> Shh.
<lifeless> wgrant: why would it? We didn't deploy to soyuz machines
<wgrant> Ah.
<wgrant> So the Fix Released was a lie.
<lifeless> wgrant: which bug
<wgrant> Bug #655690
<lifeless> wgrant: well, its a nonfunctional change, right?
<lifeless> wgrant: or is it broken?
<wgrant> lifeless: No, it's fine. Shouldn't be a problem, except it's happened in the past that a CP has been partially deployed, marked as Fix Released, something has broken, and then confusion abounds when you're looking at the wrong code.
<lifeless> well the rev is on the bug
<wgrant> True.
<lifeless> I'm not sure what we should do here.
<lifeless> if it was a functional change, I wouldn't have toggled it.
<wgrant> Merge Fix Committed and Fix Released into Fixed, like mpt has wanted to do forever?
<lifeless> :)
<bac> thumper, did you see hudson failed on your branch?
<bac> thumper, well not just your branch...
<thumper> no
 * thumper afk to collect car
<thumper> bac: looks screwed up
<thumper> bac: as in hudson looks screwed up, not the branch
 * StevenK prods at hudson before bed
<LPCIBot> Project db-devel build (103): FAILURE in 3 hr 57 min: https://hudson.wedontsleep.org/job/db-devel/103/
<StevenK> That devel change is odd
<StevenK> Er, s/change/test failure/
<StevenK> The test suite is run under sudo, so I am tempted to blame a recent landing
<thumper> yes, yes it is
<thumper> wallyworld_: skype?
<wallyworld_> ok
<wallyworld_> thumper: now?
<thumper> wallyworld_: I can't hear you :)
<wallyworld_> i can't hear you either
<wallyworld_> !#@#@%@ pulse or whatever
<wallyworld_> i may have to reboot :-(
<thumper> ack
<thumper> contains(@class, "js-action")
<lifeless> thumper: you might like https://code.edge.launchpad.net/~lifeless/launchpad/zope.testing/+merge/39423
<lifeless> jml: or you may like it.
<lifeless> gnight
<LPCIBot> Project devel build (157): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/devel/157/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Prevent the atexit warning that occurs
<LPCIBot> when tests are run within the LibrarianLayer.
<adeuring> good morning
<mrevell> Hello
<bac> hello mrevell, adeuring
<wgrant> Someone is going to need to manually apply the permission changes from db-devel r9888, or we are going to have an awful lot of branch scan failures.
<wgrant> (that was merged into devel later on, and has security.cfg changes, which can't be deployed directly without downtime)
<wgrant> (but the revs that need the new perms were deployed earlier today)
<mthaddon> wgrant: interesting - this seems to be something of a failure in process... :/
<wgrant> mthaddon: One that people have been warned about.
<wgrant> I don't recall exactly where.
<wgrant> But it was brought up recently.
<mthaddon> yeah
<mthaddon> so allegedly qastaging should fix this, but it seems this wasn't exactly qa-ed there...
<wgrant> qastaging may have been set up too late.
<wgrant> So it may have had the right perms from the start.
<mthaddon> well, and we're not running scripts on it yet...
<wgrant> Ah, that would also do it.
<mthaddon> but my point is the revno was blessed for rollout without having really been QA-ed
<wgrant> It's almost impossible to completely QA something like that.
<wgrant> But more could have been done.
<mthaddon> anyway, thx for bringing it up
<wgrant> It's also possible that it was QA'd properly on db-devel, and then merged without properly being QA'd on devel.
<wgrant> But anyway.
<henninge> jtv: yup, the main facet feigns ignorance about a series branch to non-privileged users.
<henninge> if that branch is private.
<jtv> henninge: and ours does the same for the development branch but not for the export branch?  Or does it fail to feign and so feint when either is present?
 * jtv gets confused about spelling
<henninge> jtv: ;-) the latter.
<henninge> jtv: so, if translations would say somthing like "translations are imported from/exported to a private branch" it would already give away more information than the main page which claims "No revision control details recorded for ..."
<jtv> Right.
<jtv> Well, mystery solved I guess.  Good job.  :)
<henninge> So our page will have to look to non-priv users as if no branch sync had been set up.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (158): FIXED in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/158/
<LPCIBot> Launchpad Patch Queue Manager: [testfix][rs=bac][ui=none][bug=54946, 496574, 497282, 611258,
<LPCIBot> 618955][rollback=11801]
<lifeless> jml: oh hai
<jml> lifeless: hello
<lifeless> jml: I've fixed layers
<jml> lifeless: glad to hear it.
<lifeless> jml: it needs a review :)
<jml> lifeless: cool. I'll look at it once I'm done w/ email
<lifeless> -> food
<LPCIBot> Project devel build (159): SUCCESS in 3 hr 34 min: https://hudson.wedontsleep.org/job/devel/159/
<LPCIBot> Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=665407] Add invalid-link style to invalid lp
<LPCIBot> branch short links and prevent click through
<lifeless> jml: have you picked up a key?
<jml> lifeless: I don't think so.
<lifeless> I'll go get one
<flacoste> lifeless, jml: around?
<lifeless> jml: bony 3
<lifeless> flacoste: yes
<jml> lifeless: I don't understand...
<StevenK> It sounds like a room title to me
<flacoste> lifeless, jml: call me when you are ready
<lifeless> jml: all the meeting rooms are booked; marianna has put us in bonaire 3
<lifeless> flacoste: skype ?
<flacoste> lifeless: skype or POTS, whatever works best for you guys
<lifeless> flacoste: rephrasing; please join skype
<flacoste> lifeless: i should be there already
<flacoste> lifeless: are you hearing me?%
<allenap> jml: I have some grants for production that accompany a security.cfg change that has already landed in devel. Are you the person I should ask to review these?
<allenap> It's probably already in stable too, but I think all the grants are for script users so it hasn't caused problems so far.
<bigjools> allenap: he's in a session
<allenap> bigjools: Thanks. I'll email him (unless you happen to know who else I could/should contact?).
<bigjools> allenap: I can review/grant for you if you want
<allenap> bigjools: Awesome, thanks. http://paste.ubuntu.com/520757/
<bigjools> allenap: how do you know you have all the necessary changes?
<allenap> bigjools: Because you're reviewing it for me to check ;)
<bigjools> !
<bigjools> allenap: it's my job to ask probing questions :)
<allenap> bigjools: Do you know if it's possible to get Postgres to puke up it's permissions so I can compare before and after?
<bigjools> allenap: yeah I think I've seen something, can't remember what it is
<allenap> bigjools: Okay, I'll investigate. Thank you for your probe.
<bigjools> allenap: so assuming that you've already landed the changes with these then it looks ok - my only concern is that you've not missed anything and it would be nice to somehow verify that.
<bigjools> allenap: FWIW, I also bow to your grep skillz
<StevenK> If you could get a permissions dump from a running development instance, that would probably help for comparion
<StevenK> *comparsion
<allenap> StevenK: Yeah, that's what I have in mind.
<bigjools> jml: when you have a moment I could do with chatting about the new buildd-manager - the twistd process is hitting all of one core on mawson and strace is not helpful.
<bigjools> deryck: did you see bug 667215? :)
<bigjools> gary_poster: hi
<gary_poster> hey on call but off soon bigjools
<bigjools> gary_poster: np,  I am in a session soon though.  Just wanted to talk about bug 667183
<bigjools> I'll comment on the bug
<gary_poster> ack
<deryck> bigjools, yeah :-)  I haven't replied or un-security'ed it yet.  It's crying out for a clever response, and I don't want to disappoint. :-)
 * deryck kids obviously
<deryck> I just haven't replied yet.
<bigjools> deryck: seems ripe for "opinion" to me :)
<deryck> heh
<bigjools> abuse of the security tag is annoying though
<deryck> yeah
<deryck> launchpad session!
<jml> bigjools: will do.
<allenap> bigjools: You were very right to ask that question: http://paste.ubuntu.com/520782/
<bigjools> allenap: :)
<bac> mars ping
<mars> Hi bac
<mars> bac, regarding your earlier question, Ursinha would know
<bac> oh, ok
<mars> bac, and your EOD was 7:00am for me, I think? :)
<bac> Ursinha: ping...
<rockstar> sinzui, give me an example of your perfect pagetest story.
<bac> mars: yeah
<bac> mars: so you don't know which tools are supposed to set the bad-commit tags?
<bac> Ursinha: do you ?  ^^
<bac> if that could be documented on the the QAForContinuousRelease pages i think people would understand the process better
<mars> bac, not off the top of my head, no.  Ursinha and lifeless worked that out, and I remember something on the list about it having the tags be manual(?)
<bac> mars: really?  not what the wiki says at all.
<bac> well, i won't be getting my answer today, then.  :)
<jcsackett> hey, bac, as long as your online did you see my comment on the bug(s) we were talking about yesterday?
<mars> bac, that's why I did not reply.  I thoguht lp-land or ec2 land should have done everything for you
<bac> mars: me too.  this branch was full of mysteries.
<bac> bugs didn't get linked, diff didn't update, tags didn't update, MP didn't get set to 'merged'.  all very odd.
<bac> some of it may be to branch scanner weirdness today
<bac> jcsackett: i did.  thanks for following through.
<jcsackett> bac: no problem. were you able to qa your stuff?
<bac> jcsackett: in the future, could you send me email?  sometimes my IRC client tells me i got pinged but i cannot find it in the scrollback
<bac> jcsackett: i did
<jcsackett> bac: oh, sure.
<jcsackett> excellent.
<bigjools> bac: hello, would you mind testing a patch for me to fix that failing test of mine?
<lifeless> flacoste: can you invite me to the ISO call tomorrow/today ?
<flacoste> lifeless: sure
<flacoste> lifeless: not sure, it it's going to happen though given that elmo is at UDS, but we can probably have the call with tom
* flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<lifeless> flacoste: I'd like to talk with you, charlie & tom about  some ops stuff from chatting with statik
<lifeless> flacoste: that call seems ideally structured
<flacoste> lifeless: sure
<flacoste> lifeless: you should have the invite
<flacoste> lifeless: btw, you aware that converting daily builds to feature flag is a blocker for edge removal?
<lifeless> flacoste: yes, I said so on the prereq bug
<flacoste> what bug is it?
<lifeless> flacoste: bug 666538
<lifeless> (was just looking)
<lifeless> flacoste: we don't need to panic about removing edge - all due caution, and all due haste IMO
<lifeless> flacoste: we're already better off
<lifeless> matsubara: ping
<flacoste> lifeless: agreed, but the daily build conversion, did you intend to take it, or do we need to hand this off to the code team?
<matsubara> hi lifeless
<lifeless> matsubara: I'd like to interrupt you for a small but important thing.
<matsubara> shoot
<lifeless> matsubara: could you fold the 'edge' OOPS report summaries into lpnet.
<lifeless> matsubara: they are now running the exact same code.
<lifeless> matsubara: so we don't benefit from a separate report.
<flacoste> lifeless: ok, i see that the bug is assigned to poolie
<matsubara> lifeless, yes, are those instance not using the E* prefixes anymore?
<lifeless> matsubara: prefixes haven't changed
<flacoste> lifeless: so they are running the same code, but with a different config right?
<lifeless> flacoste: -very- slightly different config; only practical change I know of is the recipe stuff being enabled.
<flacoste> ok
<lifeless> flacoste: oh, and the timeout is 13000 not 15000 - but again, the oops /types/ and /causes/ should be identical.
<flacoste> right
<flacoste> should we harmonize the timeouts?
<lifeless> flacoste: on my todo :)
<lifeless> flacoste: waiting for the 600 a day to reduce
<flacoste> right, because of the 8.4 regressions
<lifeless> flacoste: which will happen as more things are qad and deployed - we have several timeout fixes blocked on qa
<lifeless> flacoste: right
<flacoste> awesome
<flacoste> this is really better
<lifeless> StevenK: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html - --please-- I know you're at UDS, but unqa'd code in trunk is a teamwide stall.
<StevenK> lifeless: I'm trying!
<lifeless> StevenK: what can I do to help?
<StevenK> lifeless: Tell me why the function appears in the WADL but not in the actual API code
<lifeless> flacoste: so the call is in 23.5 hours?
<flacoste> yep
<lifeless> StevenK: how do you mean? leonardr: we may need your help.
<lifeless> flacoste: thanks, wicked.
<leonardr> StevenK, give me a link?
<lifeless> matsubara: so, for clarity - you're folding the E prefix into the lpnet reports ?
<StevenK> lifeless: I'm in a session that I'm talking in, so I don't want to walk out
<leonardr> what do you mean by "the api code"? you can't access it from launchpadlib?
<matsubara> lifeless, yep
<StevenK> leonardr: Right. I'm looking at say dir(maverick)
<lifeless> matsubara: awesome, thank you very much. We can drop the edge-oops.html on lpqateam etc too, with this change.
<matsubara> lifeless, I seem to remember you saying that the weekly summary is not useful, so I'll get rid of that as well
<matsubara> Ursinha, what do you think ^?
<lifeless> matsubara: yes please
<Ursinha> reading
<Ursinha> matsubara, that's cool, want me to do that along with qastaging changes?
<matsubara> Ursinha, sure, getting you a diff, just a sec
<leonardr> StevenK: i assume you're busy in the presentation, but when you get a minute, i need a link to the problematic branch (or maybe lifeless can give it)
<StevenK> leonardr: rev 9303 or so in db-devel, I think
<StevenK> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/9903
<matsubara> Ursinha, merge from here: lp:~matsubara/oops-tools/edge-no-more
<Ursinha> thanks matsubara, will do that right now
<matsubara> Ursinha, will go for lunch and post office errands. please add a MP and I'll take a look after lunch and will deploy this so we'll get the new reports tomorrow. thanks a lot!
<Ursinha> sure, thanks matsubara-lunch!
<lifeless> mwhudson: https://bugs.edge.launchpad.net/launchpad-code/+bug/667322
<leonardr> StevenK: ok, so you have some tests that use launchpadlib, and those tests pass, so it should be working. i'd like to eliminate the possibility that your client-side launchpadlib is using a cached wadl file
<leonardr> can you run your test script but set "httplib2.debuglevel = 1" before using Launchpad.login_with or equivalent?
<StevenK> leonardr: Okay, doing
<mwhudson> lifeless: i think the port already comes from launchpad-lazr.conf
<mwhudson> lifeless: config.codebrowse.port
<StevenK> leonardr: What do you need from that?
<leonardr> StevenK: paste me the whole thing, but i especially need the first part
 * mwhudson comments on the bug
<StevenK> leonardr: http://paste.ubuntu.com/520833/
<lifeless> mwhudson: thanks, I'll make new configs
<leonardr> StevenK: you are using a cached wadl file. go into ~/.launchpadlib/api.dogfoot.launchpad.net/cache/ and look at the file that starts api.dogfood.launchpad.net,beta,-application,vnd.sun.wadl+xml
<leonardr> see if derivedistroseries is in there
<StevenK> leonardr: I don't have that directory, is: launchpad = Launchpad.login_with('qa-mawson', 'dogfood')
<StevenK> leonardr: Is that right to to talk to dogfood?
<leonardr> yeah. i misspelled dogfood in my earlier msg as "dogfoot"
<leonardr> do you have a ~/.launchpadlib/api.dogfood.launchpad.net?
<StevenK> Nope
<StevenK> Which is why I'm questioning if my login call is right
<leonardr> you're making calls to the web service
<leonardr> do you have .launchpadlib at all?
<lifeless> mwhudson: whats 'secret_path' for ?
<mwhudson> lifeless: signing cookies
<lifeless> is it sharable across processes ?
<StevenK> leonardr: Sorry, I have .launchpadlib/api.dogfood.launchpad.net/ but not .launchpadlib/cache/api.dogfood...
<mwhudson> yeah, it's a path to a file that contains the secret
<lifeless> like, if we're running two codebrowsers
<mwhudson> lifeless: different processes _need_ to share the same file content
<leonardr> do you have cache *inside* api.dogfood?
<leonardr> ie. when i do `ls /home/leonardr/.launchpadlib/api.dogfood.launchpad.net/cache/`, i see a file
<mwhudson> lifeless: afk
<StevenK> leonardr: Right
<StevenK> leonardr: I do have that directory
<leonardr> ok, is there a wadl file inside it, and if so, does the wadl file mention derivedDistroSeries?
<StevenK> leonardr: Let me delete everything in that directory
<leonardr> StevenK: ok, that's another effective tactic
<StevenK> leonardr: Before I deleted it, no, it doesn't
<StevenK> Let me do the effective tactic
<StevenK> leonardr: Ah ha! Now it shows up
<leonardr> StevenK: ok, that is your problem. wadl files are cached for one week
<leonardr> this is fine for casual users, but when you do web service development, you need to make sure you clear them out
<StevenK> leonardr: Why the two directories?
<mwhudson> lifeless: back again
<leonardr> StevenK: which two directories?
<StevenK> leonardr: .launchpadlib/<api...>/cache versus .launchpadlib/cache/<api...>
<leonardr> StevenL: you should not have .launchpadlib/cache/
<leonardr> it may have been created by a very old version
<leonardr> you can delete it
<StevenK> Okay
<lifeless> mwhudson: is cachepath safe to share ?
<mwhudson> lifeless: yes
<mwhudson> it's sqlite & there's no real difference between sharing between threads and sharing between processes
<bigjools> Can I get a vic^Wvolunteer to run a test on revno 11801 of devel for me please?
<lifeless> mwhudson: so log_folder, error_dir and oops_prefix are the only unique things ?
<StevenK> leonardr: http://paste.ubuntu.com/520844/
<lifeless> matsubara-lunch: when you return, can you add CB1 and CB2 oops prefixes?
<mwhudson> lifeless: well, and port
<mwhudson> lifeless: otherwise, yes
<leonardr> StevenK: would it be easy for you to try to reproduce that error locally?
<leonardr> just start up a local launchpad and run launchpadlib against 'dev'
<StevenK> leonardr: I can try, but I also have code access to dogfood directly
<lifeless> mwhudson: lp:~lifeless/lp-production-configs/codebrowse
<leonardr> StevenK: ok, the most likely problem is that your named operation takes an argument that is of a schema field type for which there is no unmarshaller
<leonardr> let me check your code
<leonardr> i don't know why the automated tests wouldn't have caught this
<lifeless> leonardr: if thats the case, will deploying it break other APIs ?
<lifeless> if not, then we can qa-ok this patch and move on with deployments; we can iterate on the feature
<leonardr> lifeless: no, it only means that this particular named operation is broken
<lifeless> leonardr: great, thanks.
<lifeless> bigjools: ok, so bad-commit explanation.
<leonardr> StevenK: yeah, i don't think you can have a named operation take a List
<StevenK> leonardr: Oh, drat
<lifeless> bigjools: bad-commit-XXXX will block deployments until either a) the tag is removed or b) a branch with [rollback=XXXX] is landed (even if its not a rollback, thats just the tag to use.
<StevenK> leonardr: Is there anything else I can do?
<leonardr> let me look at the code/think
<leonardr> bigjools or someone else who's been doing actual web service work might know
<lifeless> the qa-* tag doesn't override the bad-commit
<lifeless> once the rollback is present, then as long as the rollback rev will be deployed, it *unblocks* the bad-commit
<lifeless> and at *that point* the qa-tag gets evaluated.
<bigjools> StevenK, leonardr: nothing springs to mind but looking at IArchive might give inspiration.
<bigjools> lifeless: where is the link between the rollback rev and the bad rev so it knows to do that?
<leonardr> StevenK: the automated tests passed because you never passed in a value for any of the List arguments
<bigjools> and I'm not sure I am really following this
<jml> bigjools: StevenK: are you guys planning on going to https://blueprints.edge.launchpad.net/ubuntu/+spec/ubuntutheproject-foundations-n-releaseprocess ?
<lifeless> bigjools: in the commit message for the rollback rev
<bigjools> jml: yes
<lifeless> bigjools: we need to iterate on this process support stuff too, to make it more dynamic.
<jml> bigjools: cool. I really want to go but something just came up. glad you guys are.
<StevenK> bigjools: IArchive.getBuildSummariesForSourceIds takes a List
<bigjools> jml: when is it?
<jml> 5:10
<bigjools> StevenK: there's your inspiration then
<leonardr> bigjools, StevenK, do we know that getBuildSummaries.. will actually work when passed a list?
<bigjools> lifeless: I am still confused, let's go through it after this session
<mwhudson> lifeless: i don't think you need the launchpad.conf & mail-configure-normal.zcml crap
<bigjools> leonardr: yes. it's use in a UI page
<mwhudson> lifeless: otherwise +1
<bigjools> used
<leonardr> StevenK: in that case, i think your problem is that you need to specify a value_type for the List
<leonardr> otherwise, lazr.restful says "a list of what?" and can't validate the incoming json
<bigjools> sounds sane
<bigjools> hey jelmer, have you got time later today to talk about that task list?
<jml> bigjools: do you want me to run your patch through ec2 land?
<bigjools> jml: it would be nice to re-create it on someone's box first, but yes, that would help, thanks.
<bigjools> jml: how's this look? http://pastebin.ubuntu.com/520807/
<StevenK> leonardr: And then regenerate the wadl and remove the cached wadl?
<leonardr> StevenK: i don't think you even need to do that, but it wouldn't hurt. you should be able to write an automated test that fails pretty easily, and then make it stop failing
<jml> bigjools: you need to add a cleanup to revert the patch when the test is done
<jml> bigjools: otherwise, perfect.
<lifeless> mwhudson: everything has it :( don't want to experiment offhand
<StevenK> leonardr: Right
<bigjools> jml: why? it doesn't hurt to leave it
<lifeless> bigjools: ok
<bigjools> given that is going to be the "fix"
<jml> bigjools: because you should only break test isolation when there's a very, very, very good reason
<bigjools> hmm
<mwhudson> lifeless: well, i'm pretty sure you don't need it, but fair enough i guess
<mwhudson> bigjools, jml: the other option is to install the monkey patch at import time i guess
<allenap> deryck: I just trod on your toes wrt bug 667347.
<mwhudson> doing it in test setup and then not doing it in teardown is ick
<allenap> deryck: Can you look at my comment in there and let me know if I should revert my change?
<deryck> allenap, ah, no worries.  Thank thank thank you for triaging. ;) :)  It's never stepping on my toes. :-)
<deryck> allenap, the comment is fine and good.  I just wanted to have the bugs together about trac, since his bugs came out of a UDS session.
<matsubara> lifeless, the CB prefix is already known by oops-tools.
<allenap> deryck: Cool.
<lifeless> matsubara: so it will handle CB1 and CB2 automagically ?
<lifeless> matsubara: there are two new error dirs as well
<matsubara> well, CB1 and CB2 can't be prefixes
<matsubara> you can't use digits in the prefix
<lifeless> oh grah
<lifeless> so CBA abd CBB ? Why can't you use digits ?
<matsubara> because an oops id is formed by <day-since-epoch><prefix><unique-digit-for-the-given-oops-on-that-day>
<matsubara> if you add a digit in the prefix, the regex will think that's part of the unique-digit part of the oops id
<StevenK> lifeless: Ping
<lifeless> hi
<lifeless> matsubara: what regex
<lifeless> matsubara: the OOPS is in a header in the file
<StevenK> lifeless: Do you remember what you did on my laptop to make launchpad branches branch quicker?
<lifeless> StevenK: we edited bzr to not use accelerator branches, didn't we?
<StevenK> lifeless: It was somewhere in launchpad/.bzr ?
<lifeless> StevenK: or was it making the source branch a lightweight branch?
<StevenK> lifeless: It was lightweight
<lifeless> bzr remove-tree on your devel/db-devel branches
<StevenK> lifeless: I think it's interferring with pushing branches to lp and stacking
<lifeless> StevenK: no
<matsubara> lifeless, oops_re in models.py /me looking for a link
<lifeless> matsubara: can you file a bug on this, its an unnecessary constraint, as long as no prefix is a prefix of another prefix.
<bigjools> mwhudson: normally I'd 100% agree but I'm poking in the upstream's fix.  I guess I will do it anway since in retrospect it's kinda nasty.
<matsubara> lifeless, https://bazaar.launchpad.net/~launchpad-pqm/oops-tools/trunk/annotate/head%3A/src/oopstools/oops/models.py
<mwhudson> bigjools: either poke it in for the entire duration of the process or for an isolated controlled time, not "approximately half of the time"
<mwhudson> :-)
<mwhudson> in this case it won't matter, apart from wtfs/min
<bigjools> mwhudson: meh :)
<lifeless> I need someone to land a branch for me.
<lifeless> echo "star-merge bzr+ssh://bazaar.launchpad.net/~lifeless/lp-production-configs/codebrowse bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/lp-production-configs/trunk" | gpg -cl | mail launchpad@pqm.canonical.com -s "[r=mthaddon][ui=none] Reconfigure codebrowse to support HA: new log and error dirs needed."
<lifeless> preferrably someone not at UDS (mail queuing is odd here just now, for some reason)
<lifeless> flacoste: ^
<bigjools> jml: http://pastebin.ubuntu.com/520864/
<lifeless> flacoste: nvm, got it to work
<jml> bigjools: +1
<bigjools> jml: if that looks ok then I'll push up a new builderslave-resume and you can poke ec2land it
<bigjools> great
<bigjools> one sec
<matsubara> lifeless, I don't think it's unnecessary
<bigjools> jml: ok lp:~julian-edwards/launchpad/builderslave-resume has that change, thanks.  Maybe I should get an ec2 account.... then again maybe not.
<lifeless> matsubara: ok, why is it necessary?
<jml> bigjools: give a man fish... give a man a fishing license...
<bigjools> jml: I hate fish :)
<bigjools> come to think of it, I hate fishing
<jml> bigjools: what was your commit message
<bigjools> jml: [r=jml][ui=none][bug=54946, 496574, 497282, 611258, 618955] New almost-fully-asynchronous buildd-manager with lots of hot sexy bug fixes.
<matsubara> lifeless, because it's used to find the actual prefix in the oops-id. it's useful to know the prefix to find the files in the filesystem and to query the Prefix table. and this already works as is and changing it just because isn't a good reason either
<lifeless> matsubara: we have more servers than letters coming up
<lifeless> matsubara: LPNETAA is rather ugly
<lifeless> matsubara: would LPNET1- work as a prefix ?
<matsubara> and we'll have to have something that supports the old oops-id anyway for backwards compatibility
<lifeless> matsubara: well the old is a strict subset of the new, right ?
<matsubara> well, let's change the oops format then
<lifeless> matsubara: thats already happening, but I don't see why its related to this
<matsubara> oops-wsgi already generates oops-ids in a different format
<lifeless> what does it do?
<bigjools> sinzui: do you know if there is a way to disable custom_widgets based on a different widget's radio button selections?  Or do I need some custom JS?
<matsubara> lifeless, not sure exactly but they changed it to include the id of the thread the oops was generated or something like that.
<lifeless> ok
<lifeless> so I don't really care either way
<lifeless> here is what I would love to see us achieve:
<lifeless>  - use numbers for numbers.
<lifeless> or better yet
<lifeless>  - stop manually assigning and managing this stuff
<lifeless> its a terrible burden on deployment to have to care about this detail
<jml> bigjools: for your other issue, I'm free in the 16:00 session
<matsubara> lifeless, https://bugs.edge.launchpad.net/oops-tools/+bug/667373
<matsubara> lifeless, https://bugs.edge.launchpad.net/oops-tools/+bug/667375
<lifeless> matsubara: 667375 is a dupe
<lifeless> matsubara: 667373 is great, thanks.
<matsubara> lifeless, do you think it's a dupe of bug #271411?
<lifeless> absolutely
<matsubara> lifeless, that's fix release :-)
<matsubara> released
<lifeless> matsubara: then why are we having to add things?
<matsubara> so, yeah, I fixed #271411 but the next step is to make everything automagically
<lifeless> matsubara: ok cool
<lifeless> I see the difference, and the new bug does speak to the overall intent better
<matsubara> because that's the first step. adding support to oops-tools to read the lp-production-config file and have a web ui to trigger that
<matsubara> now we want to automate it further
<matsubara> ok, cool
<matsubara> :-)
<lifeless> also
<lifeless> 659752
<lifeless> matsubara: and bug 658863
<matsubara> lifeless, ah, thanks for the pointers, I'll cleanup  the mess
<lifeless> matsubara: thank you!
<lifeless> flacoste: ping
<flacoste> hi lifeless
<lifeless> flacoste: while you're doing PPR stuff, want to fold edge into lpnet ?
<flacoste> lifeless: sure
<lifeless> awesome!
<lifeless> jelmer: ping
<jelmer> lifeless, hi
<lifeless> bug 135610 -
<lifeless> it was qad very quickly after landing; we're just checking that it really did get vettted for ok-to-deploy
<lifeless> jelmer: ^
<lifeless> Ursinha: I've filed another couple of qatagger bugs, I put them on lp-foundations from lack of thought - sorry.
<lifeless> Ursinha: they are high return ones for folk doing deploys
<jelmer> lifeless: I probably tested that one on dogfood while it was ec2ing
<lifeless> ok
<lifeless> thanks
<lifeless> Ursinha: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/667390
<lifeless> and https://bugs.edge.launchpad.net/launchpad-foundations/+bug/667389
<Ursinha> lifeless, will check now
<leonardr> general python question: what will happen if you try to use a unix-specific function like platform.linux_distribution on a non-unix platform? will linux_distribution raise an exception or will it not even be present?
<lifeless> leonardr: case by case
<leonardr> does anyone have a windows machine they can try this code on for me?
<lifeless> sorry, no.
<lifeless> but you can use wine
<lifeless> matsubara: whats up with https://code.edge.launchpad.net/~ursinha/oops-tools/add-qastaging/+merge/39465 ?
<matsubara> lifeless, tests failed, looks like postgres is not running in the oops-tools chroot
<matsubara> losas: ping
<lifeless> matsubara: -> #launchpad-ops as per flacostes email.
<matsubara> oops
<matsubara> sorry
<lifeless> de nada
<lifeless> jml: are you free @ 3 or 4 ?
<lifeless> jml: and @ 9 tomorrow?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (104): FIXED in 4 hr 4 min: https://hudson.wedontsleep.org/job/db-devel/104/
<james_w> lifeless, jml says 4
<lifeless> james_w: thanks
<jcsackett> someday i'm going to count how many times in a month "make" is the solution to a launchpadder's problem.
<flacoste> jml: are we having a meeting?
<lifeless> flacoste: 4pm
<lifeless> flacoste: and 9 tomorrow, I hope
<flacoste> lifeless: nope, another one
<LPCIBot> Project devel build (160): SUCCESS in 4 hr 0 min: https://hudson.wedontsleep.org/job/devel/160/
<deryck> rockstar, I've disabled the windmill tests I believe to be failing and sent off to ec2 again.
<deryck> rockstar, just ec2 test to see if it works, and then you can land if so.
<rockstar> deryck, wtf are you?
<deryck> rockstar, in the A hallway.  Chair at the front.  Needed power and found it here :-)
<rockstar> deryck, I gots the power here.  Just required some looking.
<deryck> rockstar, where you at?
<rockstar> deryck, in the arterial hallway, where all the other launchpadders were.
<deryck> ah
<rockstar> lifeless and I found something nasty in feature flags and I are crying.
<deryck> rockstar, I should come there then and look into gmb's issues too
<rockstar> deryck, yeah, "with our powers combined" and all that.
<deryck> indeed
<deryck> headed that way now
 * rockstar WINS!
<rockstar> lifeless, are you around?
<rockstar> gary_poster or benji or other zope-y person: I has a questions.
<gary_poster> rockstar: i has ability to pretend not here, but will not use this ability
<rockstar> gary_poster, so, we have a new item on request.features, right?  And we use this in the base-template to assign it to features in tal.
<gary_poster> k
<rockstar> gary_poster, but this doesn't work if you use page fragments, since they are separate requests.
<gary_poster> you mean separate views, I think?
<gary_poster> should be same request
<rockstar> gary_poster, it doesn't look to be.
<gary_poster> ew
 * rockstar digs for an example
<rockstar> gary_poster, lib/lp/code/template/branch-index.pt:110
<rockstar> gary_poster, I've exposed a page fragment, so that we have a different template file, because otherwise the branch page can get rather large.
<gary_poster> For line 110 on that pt I get
<gary_poster>       <tal:branch-pending-merges
<gary_poster>            replace="structure context/@@++branch-pending-merges" />
<rockstar> gary_poster, yeah, that one.
<gary_poster> so, that's calling a separate view
<gary_poster> but it's the same request
<gary_poster> maybe this is a terminology thing
<rockstar> gary_poster, maybe.
<rockstar> gary_poster, the problem is that branch-pending-merges has none of the defined scope of the "parent" template.
<gary_poster> yes
<gary_poster> that I'd expect
<gary_poster> course, it should be able to get to it with request/features
<gary_poster> so shouldn't be hard to find
<rockstar> gary_poster, but the request object also seems to be different, since I can't do tal:define="features request/features" like base-layout.pt does.
<gary_poster> then that would support your statement that they are different request objects.  That's not...how I'd expect things to be done, though
<gary_poster> so IOW this is not zope-y
<gary_poster> but it may be launchpad-y
<gary_poster> I hope not :-/
<wgrant> gary_poster: With bug #662912, is it just that the librarian forwarder thing isn't handling restricted files?
<wgrant> I can reproduce the same behaviour with private bug attachments.
<gary_poster> wgrant: yes, that's what I just figured out
<gary_poster> wgrant: do you happen to know if we only recently stopped copying files over?
<gary_poster> in favor of the forwarder thing?
 * gary_poster was wondering if this has always been a problem and he just didn't notice before, or what
<wgrant> gary_poster: The forward is almost as old as staging, TTBMK.
<gary_poster> ah ok
<gary_poster> so MP and so on have been broken on staging forever?
<wgrant> But that's from before my time.
<gary_poster> probably before mine then too :-P
<rockstar> gary_poster, so, might you have any suggestions for me?
<wgrant> I don't know. I don't recall using MPs on staging, but it seems unlikely that they've been broken forever.
<gary_poster> that's what I thought too wgrant.
<gary_poster> rockstar: option 1) you give me super-easy steps to dupe and I'll try to pdb.  option 2) you do a pdb yourself.  option 3) we see if request/features is implemented in an odd way or something.  I'll look in on option 3 for a second...
<rockstar> gary_poster, lifeless and I just had a pretty epic pdb session, so I bet I have some answers for you already.
<gary_poster> ok
<rockstar> gary_poster, see http://pastebin.ubuntu.com/520979/ - It was in this debugging that I recognized that we're getting separate requests.
<rockstar> gary_poster, hit the branch page with that patch, and you'll see a few different requests being made.
<gary_poster> ok, will try, rockstar
<wgrant> gary_poster: Note that the restricted librarian is brand new compared to staging. So it's quite possible it was missed, as for years it was only used by the publisher, and that doesn't run on staging.
<gary_poster> ah, wgrant, maybe so
<gary_poster> rockstar: not having any luck with something accessing features.  Going to https://code.launchpad.dev/~name12 and https://code.launchpad.dev/~name12/landscape/feature-x for instance.  what do you mean by branch page?  or perhaps is there something I need to add to a template?
<rockstar> gary_poster, the latter url is what I mean by "branch page"
<rockstar> gary_poster, it seems to be starting a separate request (or at least providing a different request object) to ++branch-pending-merges
<gary_poster> rockstar: the pdb you gave me is not being hit, presumably because the associated templates do not yet refer to request/features?
<gary_poster> I can dig out the various templates involved, but I was hoping you'd guide me into the necessary changes to dupe
<gary_poster> as I suspect that will be faster and more efficient, and will enable me to be lazier
<rockstar> gary_poster, sorry, that pdb isn't going to help.  A type for if "features" in self.__dict__ was stopping at every get_features, and while dealing with every stop, we saw more requests popping through.
<rockstar> gary_poster, really I can't tell you exactly how we re-produced, other than to say "a new request is created after pdb gets to a certain point in rendering"
<gary_poster> ...
<rockstar> gary_poster, I guess I should say "this is the only explanation to deal with the various issues we're seeing, and so I'd like to investigate further."
<deryck> rockstar, https://code.edge.launchpad.net/~deryck/launchpad/populate-trac-bug-filing-form-667342/+merge/39479
<gary_poster> rockstar: ok.  My core response is "Zope is typically not supposed to have multiple request objects for the same web request.  You generally only do that when you are changing security interactions on the fly, which is an advanced use case, and likely to need special care.  I am not aware of LP needing to change security interactions on the fly, but it is conceivable."
<gary_poster> but pretty darn unlikely, especially with one view calling another from the page template
<gary_poster> rockstar: I'm happy to debug something if you can give me instructions to dupe, but I'm not keen on trying to come up with abstract ways to provoke what you describe
<rockstar> gary_poster, okay, so why would request be different between +index and ++branch-pending-merges
<gary_poster> they are just strings to me.
<rockstar> gary_poster, so the best way to dupe the error I'm seeing is to add a tal:condition="features/foo" to any div in lib/lp/code/templates/branch-pending-merges.pt, and see the oops.
<gary_poster> ok cool, I'll try
<rockstar> gary_poster, and then try to define features as being request/features - this time it'll oops with KeyError on features.
<rockstar> gary_poster, I assume it works with any page fragment.
<jcsackett> does anyone know some workarounds for ec2 test complaining about ssh agent? usually logout/login works for me (or failing that reboot) but nothing seems to do it now.
<jcsackett> error is: https://pastebin.canonical.com/39136/
<jcsackett> it's not an uncommon one, but as i said it's not usually this persistent. :-/
<gary_poster> rockstar: I have http://pastebin.ubuntu.com/521004/ .  When I go to https://code.launchpad.dev/~name12/landscape/feature-x I see HEEY GARYYY or whatever I wrote on the main template and no OOPS and no pdb
<thumper> rockstar: ping
<rockstar> gary_poster, I seem to have gotten SOMETHING sorted, making me look like a crazy person.  Sorry for the noise.  I'll consult with lifeless before stirring up more trouble concerning my suspicions.
<rockstar> thumper, pong
<gary_poster> cool rockstar
<thumper> rockstar: you have three cards for merge queues needing qa
<thumper> rockstar: I wasn't sure what was needed
<rockstar> thumper, yeah, duplication of effort.  Nothing is needed, or I would have addressed it in the bug tracker.  :/
<thumper> rockstar: want to move the cards then?
<wallyworld> thumper: abentle: standup?
<rockstar> thumper, moved them.
<thumper> rockstar: ta
<thumper> wallyworld: sure
<thumper> abentley: ping
<abentley> thumper: pong
<thumper> abentley: skype?
<thumper> wallyworld: https://bugs.qastaging.launchpad.net/launchpad-code/+bug/634149
<lifeless> rockstar: yes, can be now
<rockstar> lifeless, I sorted it a bit, but I bet you'd like to investigate it a little more.
<thumper> https://code.edge.launchpad.net/~weechat-devs/+recipe/weechat-daily WTH?
<thumper> Unauthorized: (<lp.code.model.sourcepackagerecipebuild.SourcePackageRecipeBuild object at 0x2b9aa96eb490>, 'status', 'launchpad.View')
<lifeless> hmm
<lifeless> no idea ;)
<lifeless> its still deployed via the edge config
<thumper> I can seem my wikkid one
<thumper> https://code.edge.launchpad.net/~thumper/+recipe/wikkid-daily works
<lifeless> that woorks for me too
<lifeless> jcsackett: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<jcsackett> lifeless: we switched it to inprogress.
<jcsackett> i can mark it as qa-ok to unblock deployment; it just needs further work.
<lifeless> I have done so
<jcsackett> one sec.
<jcsackett> lifeless: okay.
<lifeless> qa-ok means 'ok to deploy' really ;) - but we need to tune our tools more still.
<lifeless> jcsackett: (I grabbed curtis :))
<jcsackett> oh, right; you two are in proximity. :-P
<jcsackett> lifeless: how often does the deployment-stable report refresh?
<lifeless> 10 minutes or so
<lifeless> Once we catch up I'll ask urshina to meaure the duration and run high-frequency
<lifeless> jcsackett: its refreshed
<lifeless> leonardr: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> leonardr: please be QAing
<lifeless> gary_poster: in case leonardr has EOD'd, perhaps you could peek at this - its a lower commit # than the thing you're qaing.
<gary_poster> lifeless: I have to run now I'm afraid
<leonardr> lifeless, i'll start the qa process but it'll take about 2 hours for the request token to expire (since i'm testing that they now expire)
<lifeless> gary_poster: kk
<leonardr> if you want i can make sure that normal approval process works first
<lifeless> leonardr: please do - and remember for all your landings that we need to qa as soon as possible
<leonardr> ok, what am i testing against, dogfood?
<lifeless> leonardr: qastaging.l.n refreshes ~30 minutes after things get through buildbot to stable.
<lifeless> leonardr: qastaging.l.n
<leonardr> hm, that's a new one, we should add it to launchpadlib.uris
<leonardr> lifeless: please humor me and send me url to doc telling me what tag names to use, etc?
<leonardr> i've verified that normal token validation still works, so that should be enough to unblock it?
<lifeless> leonardr: qa-ok or qa-untestable if is ok to deploy
<lifeless> leonardr: bad-commit-XXXX where XXXX is the revno in stable if it is not ok to deploy
<lifeless> leonardr: reference material:
<lifeless> https://dev.launchpad.net/MergeWorkflow
<leonardr> lifeless: tagging it qa-ok. i'll come back in 2 hours and see if the tokens now expire
<leonardr> thanks
<lifeless> and
<lifeless> https://dev.launchpad.net/QAProcessContinuousRollouts
<lifeless> leonardr: ok, so its ok to deploy ?
<leonardr> lifeless: yes, it doesn't break the existing workflow
<lifeless> leonardr: even if it doesn't fix the bug
<leonardr> yeah
<lifeless> great, exactly what we want.
<leonardr> lifeless: one question, will my database entry still be *there* in two hours, or will qastaging have been wiped?
<lifeless> qastaging is wiped once a week
<bigjools> jml: so, the deal is that the new b-m just eats CPU.  I restarted it and it's no different.
<leonardr> ok, i'll come back in 2
<bigjools> I shall debug
<leonardr> bookmarking those pages now
<lifeless> leonardr: cool!
<lifeless> Ursinha-bbl: deployment report has crashed
<lifeless> Ursinha-bbl: its the unassigned bug thing, that we *fixed*
<thumper> lifeless: where is qastaging documented?
<thumper> lifeless: does it have an imap folder for outgoing email?
<thumper> lifeless: if so, where is it?
<thumper> lifeless: by the background I'm guessing it is a copy of the database, is this right?
#launchpad-dev 2010-10-28
<LPCIBot> Project db-devel build (105): SUCCESS in 4 hr 0 min: https://hudson.wedontsleep.org/job/db-devel/105/
<wallyworld> anyone running launchpad and storm 0.18, psycopg2 2.2 successfully? release notes for storm 0.18 implies this should work now?
<wallyworld> i still get the famous ProgrammingError: operator does not exist: text = bytea
<wallyworld> oh well, back to  psycopg 2.1.3
<wgrant> wallyworld: It's not a Storm bug.
<wgrant> It's a Launchpad one.
<wgrant> Storm has been 2.2-compatible for a while now, IIRC.
<wallyworld> yeah, but i thought that it could also be fixed in the storm layer as well? clearly i was wrong :-)
<LPCIBot> Project devel build (161): FAILURE in 3 hr 58 min: https://hudson.wedontsleep.org/job/devel/161/
<wallyworld> i just saw in the storm 0.18 release notes that "- Make storm compatible with psycopg2 2.2 " assumed :-)
 * mwhudson waves
 * thumper waves at mwhudson
 * thumper EODs
<LPCIBot> Project devel build (162): STILL FAILING in 3 hr 29 min: https://hudson.wedontsleep.org/job/devel/162/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jtv][ui=none][no-qa] Improved tests for
<LPCIBot> translations.ProductSeriesView and ProductSeriesLanguagesView.
<LPCIBot> LaunchpadObjectFactory does not require a language code for
<LPCIBot> POFiles and such anymore.
<LPCIBot> * Launchpad Patch Queue Manager: [r=edwin-grubbs][ui=none][bug=664060] Allow bug supervisors to
<LPCIBot> configure the bug tracker section of their project.
<jtv> bye thumper
<henninge> I wonder how to get check_permission working in a unit test (a view test).
<henninge> I thought "login_person" would do the trick but it seems not.
<henninge> It always returns "True"
<henninge> It may be a layer thing.
<adeuring> good morning
<mrevell> Hi
<thumper> mrevell: morning
<thumper> mrevell: thanks for the bug reports, I'll get on to it :)
<mrevell> thanks thumper :) I'll probably file a couple more later and also reply to poolie's comments
<thumper> mrevell: I've got some ideas, and jml suggested just getting something going so you can then suggest improvements
<mrevell> Cool, I'd be happy to do that, yeah
<lifeless> jml: around?
<jml> lifeless: sort of
<lifeless> jml: could I get your stamp on the layers patch
<lifeless> for ec2land
<jml> lifeless: sure
<lifeless> thank you
<lifeless> jml: https://code.edge.launchpad.net/~lifeless/launchpad/zope.testing/+merge/39423 for easy reference
<lifeless> jml: thank you
<lifeless> Ursinha: we might want to run the stuff that does 'fixed by a commit' at high frequency - or is it the tagger?
<matsubara> bac, around?
<LPCIBot> Project devel build (163): STILL FAILING in 3 hr 57 min: https://hudson.wedontsleep.org/job/devel/163/
<lifeless> flacoste: ping
<StevenK> leonardr: Hi, are you around?
<leonardr> StevenK: yes
<StevenK> leonardr: There seems to be a change made recently with launchpadlib that it wants to write to /root/.launchpadlib during the test suite
<leonardr> StevenK: this is when you're testing launchpad, or are you using launchpadlib trunk?
<StevenK> leonardr: So the slaves are connected to as root, and then I sudo to a hudon user.
<StevenK> leonardr: Sorry, this is when Hudson is running the full test suite on its build slaves
<StevenK> lifeless: Speaking of Hudson, you wanted a module enabled
<lifeless> no, I was saying we might need to
<lifeless> I need to talk with ops
<StevenK> lifeless: Sorry, this was like 3 weeks ago when you said you wanted the Chuck Norris plugin :-P
<lifeless> and it would be 'write a module' (either python or java) to do the devel->stable push.
<leonardr> ok. /root/.launchpadlib is the default location when you run as root. we ran into this problem before and changed the launchpadlib constructor used by launchpad so that it would use a temporary directory
<lifeless> StevenK: oh yes. JFDI
<StevenK> lifeless: Why!?
<StevenK> leonardr: Has this code recently changed, then?
<leonardr> StevenK: no, but if someone wrote a test that created a Launchpad object without going through lp.testing.launchpadlib_for, you'd get the old behavior back
<leonardr> can you pinpoint when the directory is created? what gets put in that directory?
<flacoste> lifeless: i'm back, sorry, family start-up got stalled this morning
<StevenK> leonardr: Certainly: https://hudson.wedontsleep.org/job/devel/163/consoleText
<StevenK> leonardr: If you search for '/root/.launchpadlib'
<Ursinha> lifeless, it's tagger
<jam> it looks like the build machine iridium is hung trying to build a webkit build, can anyone confirm?
<wgrant> jam: Indeed. lamont ^^
<leonardr> StevenK: so, these errors are in places that don't use lp.testing.launchpadlib_for -- the launchpadlib tests themselves, which are outside of launchpad, and a launchpad test that uses login_with
<leonardr> as to why this is a problem now, i couldn't speculate--the code on my end hasn't changed
<leonardr> i think the best solution would be to change all those calls to use a temporary directory, as launchpadlib_for
<leonardr> does
<StevenK> leonardr: I have to admit, I am curious as to why it's only occuring now
<lamont> meh
<LPCIBot> Project db-devel build (106): FAILURE in 4 hr 1 min: https://hudson.wedontsleep.org/job/db-devel/106/
<lamont> jam: and what I can tell you is what we all can see from the web ui in launchpad
<poolie> how cute: https://bugs.edge.launchpad.net/bzr-explorer/+bug/667782
<StevenK> Rargh, it just impacted db-devel too
<jam> lamont: I'm sitting here with shadeslayer (Rohan), who is trying to understand what is going on. Any chance we can just kick it?
<lamont> I just stabbed it in the face
<jam> wgrant: is abentley the right person to poke to see if we can understand what is failing here?
<jam> It looks like it is failing during the branch update
<jam> but it seems an odd place to get hung
<lifeless> Ursinha: ok cool.
<lifeless> Ursinha: and its running how often now ?
<jam> (I wonder if it just isn't reporting the 'next' thing.)
<Ursinha> lifeless, every 5 mins
<lifeless> awesome
<lifeless> abentley: hi; can I help you QA 638637 in some way?
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<leonardr> StevenK: is it possible that /root/.launchpadlib existed before and has been removed? the error is in trying to create the directory
<jam> abentley: I'm sure deployment-stable takes precedence over the webkit stuff :)
<wgrant> jam: It could have been bzr hanging there, or the builder could have had a seizure and exploded.
<StevenK> leonardr: Let me see if a slave is still up
<abentley> lifeless: You can push a branch, and see if it gets scanned and stuff, I guess.
<wgrant> lamont: You can't get a shell in a virt builder?
<jam> wgrant: sure, but both are pretty unlikely
<wgrant> jam: The latter isn't particularly unlikely.
<Ursinha> lifeless, you are not reusing branches anymore?
<StevenK> leonardr: There is no /root/.launchpadlib
<leonardr> StevenK: my speculation was that there might have been one earlier (preventing this problem) and that its removal caused this problem
<jam> lamont: it seems that stabbing it in the face leaves it in the "currently building" state?
<lifeless> Ursinha: Its always been adhoc :)
<lifeless> Ursinha: sometimes I will, sometimes I won't.
<Ursinha> :)
<lifeless> abentley: qa staging codehosting is not working yet; am going to get it working with losa
<jam> wgrant: any way to change the status from "it is currently building and should have finished 12hours ago" into some sort of "the build has failed" state?
<jam> It is a bit confusing to figure out what is going on.
<abentley> lifeless: I'd normally qa it on staging, anyhow.
<wgrant> jam: The builder facestab should have done that.
<wgrant> I suspect it needs harder stabbing.
<wgrant> Or buildd-manager is broken.
<jam> lifeless: what is the issue with codehosting on staging? (I'm concerned it might be my stuff, though comments from yesterday said that it wasn't)
<lifeless> jam: staging is working, qastaging isn't
<jam> lifeless: k, I guess i'm not fully aware of what the differences are
<noodles775> mars: Hi! Have you had a chance to look at gmb's issue (launchpad-dev mailing list, 'Problems with FeatureFlags and test isolation')?
<jam> wgrant: well, atm I suspect buildd-manager, because webkit has 2 jobs that were killed but are now listed as "currently building"
<mars> noodles775, no, I have not looked at it.  I think deryck said he would have a look
<noodles775> mars: ok, thanks. I'll check with deryck when he's available.
<mars> noodles775, I asked gary_poster about someone in our team picking that up, but then deryck offered to help
<gary_poster> y
<flacoste> bigjools: are you sure you wanted to land the new async build manager? it will be rolled-out soon
<lifeless> abentley: ok, so it scanned my new branch and an incremetnal push ok
<lifeless> abentley: is that sufficient, do you think ?
<abentley> lifeless: Well, merge detection also needed to be tested, but I've done that, and marked it qa-ok.
<lifeless> awesome
<lifeless> Edwin-afk: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html - whats up with 11762
<bigjools> flacoste: yes, it's QA-OK.
<bigjools> I need to mark the bugs.
<bigjools> jml: around?
<flacoste> bigjools: that's great!
<bigjools> flacoste: I need to land one more branch though, but it won't block rollout.  The new process seems to hammer the database harder than I want, so I need to back off the polling intervals.
<flacoste> ok
<lifeless> sinzui: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html - whats up with 11762
<lifeless> mwhudson: http://launchpad.net/bugs/660264 is ok right, in the absence of config changes?
 * mwhudson looks
<mwhudson> lifeless: it's ok in the sense of 'do no harm' indeed
<lifeless> StevenK: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html rev 11778 bug 664380
<StevenK> lifeless: Er? It says edwin for me
<lifeless> StevenK: yes, look at the bottom
<StevenK> Abh
<StevenK> s/b//
<StevenK> lifeless: I will look at doing that after I've emptied my plate
<lifeless> StevenK: is it lunchtime?
<StevenK> lifeless: Sigh
<StevenK> I'm in the middle of two things. Mental plate
<lifeless> mwhudson: jam: so forking server for bzr+ssh - can you describe somewhere what needs to be changed to test it? bazaar.qastaging is now working with the 'normal' service, so its ready to reconfigure to test.
<lifeless> StevenK: cool, great and thanks.
<mwhudson> lifeless: i believe jam has written at least one email containing the needed steps
<mwhudson> lifeless: Subject: Re: QAing the forking codehosting server
<lifeless> mwhudson: not in my mail
<mwhudson> bah
 * mwhudson forwards
<mwhudson> lifeless: it would be good to time bzr ping bzr+ssh://bazaar.qastaging.launchpad.net/ a few times before we test it i guess
<lifeless> bah, I don't have that plugged in
<lifeless> but yes
<mwhudson> i guess doing in from somewhere where ssh setup latency is not the dominant factor would be a good idea too...
<jam> echo hello | ssh bazaar.qastaging.launchpad.net bzr serve --inet --directory=/ --allow-writes
<jcsackett> leonardr: ping.
<leonardr> jcsackett, hi
<jam> a bit long, but always works, and doesn't need to have extra code installed :)
<mwhudson> it tells me permission denied in just 0.7s!
 * mwhudson fiddles
<jcsackett> leonardr: hi. i just read your review note. do you want me to add notice in the 1.7.0 block or add a new one?
<jcsackett> (in NEWS.txt)
<jam> mwhudson: it lets me in with a proper ssh-key
<mwhudson> yeah
<leonardr> jcsackett: add it to 1.7.0, it's not out yet
<jam> with my password cached, it is 3.3s from a machine in France
<jam> interestingly, bazaar.staging denies me
<EdwinGrubbs> lifeless: why does https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html list my bug as qa-bad, when it is tagged qa-ok? I didn't think I was supposed to remove the bad-commit-N tag.
<jam> though I have a different key on that machine
<jam> non-staging is 3-7s for me
<jam> (3, 7.1, 4.6, ...)
<lifeless> EdwinGrubbs: you're not
<jcsackett> leonardr: dig. and what's the landing policy for launchpadlib? looks like all the landings on trunk have been by you? or feel free to point me to docs if they're around for this.
<lifeless> EdwinGrubbs: you're meant to have landed a commit with [rollback=xxxxx] where xxxxx is the bad commit
<lifeless> EdwinGrubbs: until thats landed the qa tagger ignores the qa-* tags.
<leonardr> jcsackett: bzr co the trunk, then merge and commit
<lifeless> EdwinGrubbs: this is because we have to pair the bad commit with the fixed commit.
<EdwinGrubbs> lifeless: but what if I fix the bug instead of rolling it back. Should I still tag it with [rollback=xxxx]?
<jcsackett> leonardr: dig. thanks!
<lifeless> EdwinGrubbs: yes, sadly.
<lifeless> EdwinGrubbs: is that what happened here?
<EdwinGrubbs> yes
<lifeless> EdwinGrubbs: what commit fixed it ?
<jam> shadeslayer: ATM, it looks like we're disabling your daily build, until we can figure out why it is crashing.
<EdwinGrubbs> lifeless: if you look at revision 11763, it has the same info, since I had tried to get a losa to cancel the previous pqm run. I thought the cancelling had succeeded, which is why I landed it again with the same message.
<shadeslayer> jam: feel free to :)
<EdwinGrubbs> lifeless: I guess I should just land an empty branch with [rollback=xxxx] in it then.
<lifeless> EdwinGrubbs: no need
<lifeless> EdwinGrubbs: I'll explicitly call it out
<EdwinGrubbs> ok, thansk
<StevenK> lifeless: 11778 QA'd
<jml> bigjools: hi
<bigjools> jml: hi - just wanted to tell you that I tracked down why the b-m was using so much CPU
<bigjools> probably best to explain in person
<flacoste> jml: if you have time, i'd like your opinion on the new release schedule proposed by henninge, especially its impact on the bug jam
<lifeless> flacoste: I need to talk with you about a) storm and b) my leave (which is still darn vague)
<jml> flacoste: I'll try to look at it today.
<lifeless> mwhudson: I forwarded that mail to losas@
<jam> lifeless: you don't ever get to leave. I thought you knew that
<lifeless> jam: hah
<lifeless> ;p
<LPCIBot> Project db-devel build (107): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/db-devel/107/
<lifeless> flacoste: calling
<lifeless> is it really still week 1?
<LPCIBot> Project devel build (164): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/devel/164/
<rockstar> abentley, yo.
<abentley> rockstar: yo.
<rockstar> abentley, I'm having a hard time writing a test that will reproduce what Chex did today.
<abentley> rockstar: have you tried it out locally?
<rockstar> abentley, I wonder if you might have a better idea what's going on there.
<rockstar> abentley, I have.  I also wrote a test that will do exactly the same thing.
<rockstar> abentley, I just thought you might have a cursory "oh yeah, I think that problem might be in X"  If you don't, don't worry about it.  I'll sort it out.
<abentley> rockstar: Nothing strikes me.  I'd look at whether updateContextFromData is doing badness.
<rockstar> abentley, I suspect there's something admin specific here.
<jam> rockstar: you mean accidentally reassigning to himself?
<jam> mwhudson explained what happened in a different (but similar) page
<rockstar> jam, if I see him, I will ask.
<jam> he explained it to me
<jam> said that there is a dropdown which contains a list of your id + groups you are in
<jam> and tries to select the current group which matches the owner
<jam> but if it can't find that group, it just selects the first line
<jam> which is you
<jam> but for admins
<jam> they aren't in the existing group
<jam> so it always selects themselves
<jam> ideally, admins would just have a text-entry box
<jam> since they can set it to anyone
<jam> otherwise you have to play a game of "find the common group", set it to that, then have the user set it back to themselves
<lifeless> jam: so, I've forwarded your instructions to losas; but if you wanted to drive the testing of the forking service that wuld be best
<lifeless> jam: the code is deployed
<jam> lifeless: I'm happy to do the testing, but what about startup scripts, etc?
<lifeless> hop into launchpad-ops
<lifeless> ask losa there
<rockstar> jam, huh.  I have a test that would indicate that's not what's happening, but what you say makes sense.
<jam> rockstar: I don't know specifically what is happening. Just that mwh said it was happening on another page, and they fixed it somehow
<rockstar> jam, yeah, I just talked to sinzui about it.
<jam> ls
<jam> sorry, mt
<jam> rockstar, is poolie near you? vila would like to chat with him in #bzr
<rockstar> jam, poolie is not here.  Just deryck and I.
<jam> np
<rockstar> jam, actually, turns out that's EXACTLY what's wrong.
<rockstar> jam, also, I apparently already had a test that CLAIMED to be testing it, but it wasn't.
<jam> rockstar: I'm glad I could be proxy-help for mwh
<rockstar> jam, :)  Thanks.
<lifeless> deryck: hi
<deryck> hi lifeless
<lifeless> if you're in the main hall, I'm just qaing a bugs patch
<lifeless> would like your input
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<deryck> lifeless, sure, be right there.
<thumper> morning
<thumper> abentley: ping
<abentley> thumper: pong
<thumper> abentley: quick call?
<abentley> thumper: sure.
<lifeless> gary_poster: hi
<gary_poster> hey lifeless
<lifeless> gary_poster: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html rev 11792
<gary_poster> lifeless, just changed it
<gary_poster> like, seconds ago :-)
<lifeless> sweet
<lifeless> deryck: https://bugs.edge.launchpad.net/malone/+bug/659085
 * deryck looks
<deryck> lifeless, so I'm 85% sure that one is qa-ok.  But I cannot know for sure without pinging allenap about it.
<deryck> allenap, around?
<allenap> deryck: Whassup?
<deryck> allenap, see the bug lifeless pointed me at above... is it qa-ok?
<thumper> abentley: https://edge.launchpad.net/ubuntu/+source/gstreamer0.10
<allenap> deryck: I have tried a few times to test this in staging/qastaging but it keeps timing out, so qa-untestable I'd say.
<deryck> allenap, there's nothing changed in functionality?  Just moving code around?
<allenap> deryck: The implementation has changed quite a bit, but the tests didn't much.
<deryck> allenap, hmmmm
<deryck> allenap, what should I look at to qa this on staging?
 * allenap tries to remember
<abentley> thumper: https://code.edge.launchpad.net/~abentley/+archive/ppa/+builds
<allenap> deryck: Okay. Calculating structural subscribers has been refactored, but the code changed little. The difference is that the bug change notification code now uses the new structural subscriber code (getStructuralSubscribers) instead of getBugNotificationsRecipients.
<allenap> deryck: So we need to change some bugs and check that structural subscribers are notified.
<bigjools> abentley: I have bad news for you.  Backing out the buildd revision has cured our build farm ills.  There's not much chance of it going back in.
<allenap> deryck: I *think* that if it's wrong we will send too little bug mail.
<allenap> deryck: Well, or too much.
<deryck> allenap, ah, ok.  I can poke at that then and see what happens.
<bigjools> abentley: but I would like us to analyse the  changes and how they could have caused buildds to become slow to respond to probes
<abentley> bigjools: okay.  Do you have any logs?
<bigjools> abentley: sort of.  It looks like it's eating memory which causes the machine to slow down too much (swapping)
<bigjools> abentley: we have got strace logs, tcp dumps but no buildd logs since they are virtual
<bigjools> abentley: what changes did that buildd revision have?
<abentley> bigjools: It has an updated version of bzr-builder, it builds outside the chroot, and it uses xen-detect to ensure it doesn't build in an unvirtualized situation.
<bigjools> abentley: I wonder if bzr-builder regressed somehow then
<thumper> bigjools: ok, so where to from here?
<bigjools> thumper: we need to re-create this problem on dogfood
<thumper> bigjools: ok, do it! :)
<bigjools> then we can start to diagnose it outside of production
 * bigjools stares at thumper
 * thumper deflects the stare with a cunning mirror like shield
<allenap> deryck: It's still timing out all the time for me.
<bigjools> thumper: are you batfink in disguise!
<deryck> allenap, every bug is timing out?  Or trying to do something times out?
<allenap> deryck: Trying to create a structural subscription (I don't have any) is timing out.
<deryck> ah
<allenap> deryck: Ah. If I go to +subscribe directly it works. I can't get any project pages to work.
<deryck> allenap, do you think it's related to your work?  Or just qa/staging?
<deryck> ah right
<lifeless> allenap: what project did you try?
<allenap> lifeless: malone and scriptify (one of mine with no artifacts apart from a single trunk branch)
<lifeless> allenap: also we can raise the timout on qastaging instantly if needed (via a feature flag)
<allenap> Ah yes.
<lifeless> we should look at the timeout you got though, it may be a problem for prod too
<allenap> lifeless: Okay, I'll file a bug for it.
<lifeless> allenap: got an OOPS ID ?
<lifeless> allenap: so, do we think this change is ok ?
<allenap> lifeless: I think it's fairly safe, but I haven't been able to QA it so far. I also (or someone) needs to look in the shared catch-all mailbox.
<allenap> Chex: Did you find a shared mailbox for qastaging?
<lifeless> allenap: launchpad-ops, ping the losa :)
<allenap> lifeless: OOPS-1762QS22
<jcsackett> bzr uncommit and bzr shelve may be the two best commands ever.
<lifeless> :)
<StevenK> bzr needs a 'bzr moo'
<lifeless> garh
<jcsackett> 'bzr garh' might actually work. :-P
<jcsackett> "This version control tool doesn't have cow powers. It does however have a pained `lifeless`."
<StevenK> jcsackett: It's super cow powers :-P
<allenap> lifeless: I think it might be a cold cache kind of problem, because it's now snappy on staging, having timed out consistently 10-20 minutes ago.
<lifeless> allenap: or load, ok.
<StevenK> lifeless, jcsackett: http://paste.ubuntu.com/521648/
<jcsackett> StevenK: that's terrible. and hilarious.
<wgrant> bigjools: Why can't we get logs?
<wgrant> bigjools: Surely someone has access to the host...
<bigjools> securitai
<bigjools> it's a VM
<wgrant> Yes.
<wgrant> But there is a host.
<wgrant> Host can poke at VM FS.
<bigjools> which does not have any logs
<bigjools> no, it can't
<bigjools> securitai
<wgrant> !?
<lifeless> allenap: https://lp-oops.canonical.com/oops.py/?oopsid=1762QS22 - 11seconds of sql
<wgrant> bigjools: Have you checked the bzr versions on the VMs?
<bigjools> I've not checked anything
<mwhudson> build-dependcies: ... package-that-adds-lamonts-ssh-key-in-postinst ...
<wgrant> Unless you are running a very strangely mangled Xen, I don't see how the host can't poke around in the VM's filesystem and grab logs.
<allenap> lifeless: That query has some EXISTS smells in it too. I'll file a bug for it now.
<lifeless> allenap: exists isn't necessarily bad, but I agree that it would benefit from eyeballing.
<lifeless> allenap: - so +1 on that
<wallyworld_> abentley: thumper: standup?
<allenap> lifeless, deryck: I have to go now. There's still no word on the qastaging mailbox. I am also away tomorrow, but I can probably find an hour in the morning if necessary.
<lifeless> allenap: how can I help
<lifeless> allenap: e.g. if the mail is in the mailbox, its ok ?
<abentley> wallyworld_: thumper is having a call with flacoste.
<wallyworld_> abentley: ok. i'llbe here when he's done
<allenap> lifeless: Mail for bug changes should be sent to structural subscribers. If, say, a bug task is assigned to a milestone to which someone is subscribed, they should also get bug mail. Also distro subscribers should get source package bug mail.
<lifeless> ok and in this case we shouls expect ...?
<allenap> lifeless: The same bug mail as everyone else, but with a rationale that explains that they're subscribed to a milestone, or the distro.
<lifeless> kk
<allenap> lifeless: Just to check, r11794 is not-ok, but everything else can be pushed out?
<allenap> So it's not blocking lots of stuff yet.
<lifeless> allenap: its blocking 20 revs
<allenap> lifeless: Ah, okay, deployment-stable.html doesn't show the subsequent revs that are prevented from rolling out.
<thumper> abentley, wallyworld_: standup now?
<wallyworld_> yep
<thumper> http://www.shopwithmaverick.com/wp-content/uploads/wpsc/product_images/meerkats%20wall%20back%20-4.jpg :-)
<lifeless> allenap: yes
<mars> allenap, fwiw, we're working on that
<lifeless> Ursinha: hey
<lifeless> Ursinha: what did you think of my qatagger improvement requests
<flacoste> lifeless: https://code.edge.launchpad.net/~flacoste/launchpad/ppr-constant-memory/+merge/39578 if you have some time
<lifeless> how much memory is it taking now ?
<flacoste> lifeless: i'd say something like 800M, but that's because of the 400M cache size
<flacoste> lifeless: i didn't get that locally (since 300k are using 56M of storage)
<Ursinha> lifeless, I'm implementing the "codebrowse url on reports" now, which is fairly simple
<lifeless> \o/
<Ursinha> the other one I'll have to take a deeper look
<flacoste> lifeless: i'm it's using now 576M on sodium (still only inserting)
<Ursinha> I'm fixing the bug that shows all revisions instead of all until the not-ok one
<Ursinha> that flacoste reported
<flacoste> with over 5M requests inserted
<flacoste> i think it will think up to ~5h to generate a daily though
<flacoste> since it took 2h last time
<thumper> bzr missing --mine-only ../db-devel
 * wallyworld_ afk briefly to do school dropoff
<rockstar> thumper, do you know much about custom widgets in forms?
<rockstar> (sinzui was supposed to help me, but I think he forgot)
<thumper> rockstar: some
<thumper> whazzup?
<rockstar> thumper, I need to use a different widget in the form in the user has launchpad.Admin on the form.
<rockstar> thumper, we discovered a bug today where the select box for choosing on owner for a recipe re-assigns ownership of the recipe if an admin edits it.
<thumper> rockstar: take a look at the branch edit form
<thumper> rockstar: we do exactly that
<thumper> rockstar: for admins editing branches
<rockstar> I looked around, but couldn't see anything.
 * rockstar looks
<thumper> rockstar: around line 1000 of code.browser.branch.py
<rockstar> Oh crap.  I think I figured it out...
<rockstar> I was on the right track, just forgot to call the setUpFields on the superclass...  :/
<LPCIBot> Project devel build (165): STILL FAILING in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/165/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jml][ui=none][no-qa] [r=jml]Fix layers to teardown always not just
<LPCIBot> sometimes,
<LPCIBot> fixing teardown in our code base and freeing us from the tyranny of
<LPCIBot> atexit.
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=667687] Fix deactivateAccount to handle
<LPCIBot> drivers correctly.
<gary_poster> wallyworld_: back?
<wallyworld_> gary_poster: yes. thanks for the great email. i was going to respond this morning
<wallyworld_> i'll do the stuff you outlined
<gary_poster> wallyworld_: great.  happy to help
<gary_poster> (all I was going to do was see if there was anything we ought to talk about before I ran away)
<gary_poster> so I'm running away then :-)
<gary_poster> bye
<wallyworld_> gary_poster: np. the email spelt it all out extremely well. bye
<thumper> :)
<thumper> Done. 9470605 items in 949 iterations
<thumper> CodeImportEventPruner
<lifeless> \o/
<lifeless> is that 9.4M items? how long did it take?
<thumper> lifeless: a while, but not too long
<thumper> lifeless: and yes it is 9.4M items
<lifeless> thumper: how long, do you know?
<thumper> 9.4 mostly useless items
<thumper> lifeless: I can check
<lifeless> please, I'm very curious
<thumper> lifeless: it won't be as long with the next run time though
<thumper> lifeless: as it'll only be deleting around 30k items
<thumper> nah, 20k items
<thumper> lifeless: 3778.052 seconds,
<thumper> average size 9979.563215 (2506.74270824/s)
<thumper> so based on the time there, daily garbo for this should take around 4s
<lifeless> 1 hour?! wow. Was it multiple transactions ?
<thumper> lifeless: yes, 949 of them
<lifeless> whew :)
<thumper> lifeless: written the garbo way
<lifeless> yeah
<lifeless> I trusted
<lifeless> I'm just happy it panned out
<thumper> it did
<thumper> now to see if it makes a difference at all
<thumper> hopefully with that and the fix I've done for mailing lists we should see a difference
<thumper> on the private xmlrpc server
 * thumper closes up for lunch
#launchpad-dev 2010-10-29
<wallyworld_> rockstar: can you ping me if/when you might have a moment?
<lifeless> rockstar: so what was up with your template
<LPCIBot> Project devel build (166): ABORTED in 1 hr 39 min: https://hudson.wedontsleep.org/job/devel/166/
<LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][no-qa] New utility script,
<LPCIBot> massage-bug-import-xml,
<LPCIBot> that attempts to repair some commonly seen problems with bug import
<LPCIBot> XML.
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=655242][bug=505304][bug=504126][bug=606866]
<LPCIBot> Fix OpenGPG documentation.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=me][ui=None][no-qa] Slow the rate of polling the buildd-manager
<LPCIBot> does to reduce the load on the database and cesium (the
<LPCIBot> buildmaster host).
<StevenK> LPCIBot: Shhh, don't tell everyone.
<wallyworld_> thumper: quick question... ping?
<rockstar> wallyworld_, what's up gangster?
<wallyworld_> rockstar: can you skype?
<rockstar> wallyworld_, I can't, actually.  :(
<wallyworld_> ok. i just wanted to touch base about your merge queues email
<wallyworld_> one question - while i wait for your branch to merge in to db-devel, can i merge from your lp copy and later merge from db-devel with no problems?
<wallyworld_> that way i can start using your stuff now
<rockstar> wallyworld_, yeah, you certainly can.
<rockstar> wallyworld_, yeah.  deryck and I have learned some icky edge cases that don't play well with feature flags.
<wallyworld_> anything i need to know?
<wallyworld_> also, do turn on the merge queue feature flag, do i need to insert a record into my local db?
<rockstar> wallyworld_, just do it the way I did it, and you should be good.
<wallyworld_> ok. i'm also unsure what you meant by merge queue vocabulary and the comments around that
<wallyworld_> actuallt, to clarify my previous question,  i know how to wrap stuff inside a feature flag, but how do i enable that flag locally for my testing?
<wallyworld_> inprod, there would be a db record required, no?
<rockstar> wallyworld_, see my branch:  lp.code.browser.tests.test_branchmergequeue
<wallyworld_> rockstar: np
<LPCIBot> Project db-devel build (108): STILL FAILING in 3 hr 55 min: https://hudson.wedontsleep.org/job/db-devel/108/
<wallyworld_> that is fine for unit tests, but if i want to run up a system to try out, is there a preferred way for that?
<rockstar> wallyworld_, you don't need to worry about the vocabulary stuff.  I basically just need to be able to get a set of BranchMergeQueue items for a given criteria.  You're currently working on a solution for that.
<wallyworld_> rockstar: yes. the only methods currently supported are visibleBy() and ownedBy() but the infrastructure is there to extend as required
<rockstar> wallyworld_, yeah, I can extend it.
<wallyworld_> rockstar: ok, i'll grab your branch off lp. you can grab my working copy too i guess if you need to. i'll email you a status later today
<rockstar> wallyworld_, yeah.  It'd be great if you could get it landed though.  Pipes don't like to have merges from elsewhere.
<wallyworld_> rockstar: ok. the main work left is the visisbleBy() and the tests and merging in your latest once it lands into db-devel
<rockstar> wallyworld_, cool.  I've got LOTS of other things to work on, so don't feel too much pressure.
<rockstar> wallyworld_, I'm going to be busy all weekend.
<wallyworld_> ok. np. thanks.
<thumper> hey
<wallyworld_> straw
<thumper> I've made it to the shared work space
<thumper> very interesting
<thumper> and free for where I'm at :-)
<wallyworld_> so now everyone else there has to put up with you too and not just us :-)
<thumper> heh, yeah
<rockstar> thumper, cool.  I'm interested to see what your experience with it is.
<thumper> rockstar: sure
<thumper> rockstar: how's UDS going for you?
<rockstar> thumper, I'm REALLY tired.  :)
<thumper> heh, not surprised
<rockstar> deryck and I are watching football and spending a quiet evening in.
<thumper> :)
 * thumper needs to focus on actually doing something :)
<lifeless> rockstar: hey
<lifeless> rockstar: so what was up with the request.features thing the other day
<rockstar> lifeless, complications in using page fragments.
<rockstar> It seems that in some cases, page fragments result in entirely separate request objects than the original request, even though it's technically in the same request.
<rockstar> I think page fragments are a potential performance problem.
<rockstar> Maybe not the biggest performance problem, but something we may need to evaluate.
<rockstar> That's why we would get those separate "requests" halfway through a pdb of a page: it was getting another page fragment.
<lifeless> yeah, they've been on my radar already.
<lifeless> however why weren't the request *objects* getting a start_request event ?
<rockstar> lifeless, I think it's because it wasn't an actual request, but a page fragment.
<rockstar> They'd technically already had a start_request event.
<rockstar> When you left, we were checking for "features" in self.__dict__ when we should have been checking for "_features" in that pdb setup we made.
<rockstar> When I fixed that, I found that we weren't ever getting without setting.
<rockstar> Although the request object did some funky things.
<lifeless> so, you've a work around ?
<rockstar> lifeless, no, the actual issue was another type poolie found.
<rockstar> I was missing a semicolon in my tal:define.
<rockstar> lifeless, so I got it to work when I defined features correctly.
<rockstar> lifeless, the pdb quest was very eye opening though.
<lifeless> -ah
<lifeless> coolio
<rockstar> The branch index page traverses zcml at LEAST 8 times...
<lifeless> yeah
<lifeless> we have a lot of inefficiencies
<lifeless> what we need to do, once we have the coarse performance headaches out of the way
<lifeless> is to start streamlining the stack,
<thumper> rockstar: what do you mean traverses zcml at least 8 times?
<rockstar> thumper, every time we go get a page fragment, we have to look it up in zcml.
<thumper> rockstar: you mean humans? or machines?
<thumper> rockstar: because the machine doesn't touch the zcml
<thumper> rockstar: it is parsed once
<rockstar> thumper, for context, lifeless and I went on a pdb adventure yesterday trying to sort out a feature flag problem.
<rockstar> thumper, yeah, but it's stored in an object tree, which gets traversed.
<wgrant> Surely the adapter lookup doesn't suck too much, though...
<thumper> I thought that was pretty much optimized at the zope level
<lifeless> thumper: its about 40 function calls to do a notify()
<lifeless> make no assumptions
<thumper> heh
<thumper> 40?
<thumper> hmm..
<rockstar> thumper, easily 40.
<thumper> that's kinda shit
<lifeless> conservative
<lifeless> theres a profound degree of indirection in the zope core
<lifeless> some bits are highly tuned
<rockstar> lifeless, yeah, stepping through a whole request life was VERY eye opening for me.
<lifeless> but not all, not by any stretch
<lifeless> rockstar: ;)
<rockstar> lifeless, kapil also said he made a really good debug WSGI middleware that gets good SQL tracing and such.
<lifeless> sadly we really don't have a wsgi stack
<lifeless> we're glued on the end
<lifeless> OTOH we have damn awesome sql tracing already
 * wgrant likes the Django Debug Toolbar.
<rockstar> wgrant, I have tried to duplicate that since day 1 at Canonical.
<lifeless> web debugger?
<wgrant> (It hooks into WSGI in a couple of places, adding a toolbar on the right of every page which lets you easily see template contexts, seeing SQL statements and running EXPLAINs and all sorts of other awesome stuff.)
<lifeless> should be broadly straight forward.
<rockstar> lifeless, no, just adds some HTML with information about query times and templates used and performance times.
<lifeless> rockstar: oh, just make the comment at the bottom visible via a feature flag.
<rockstar> lifeless, we're not using a WSGI stack?  I thought gary said we were.
<lifeless> I've been meaning to do it, but after edge going
<wgrant> rockstar: FSVO stack.
<rockstar> lifeless, no, you'd never enable this on production.  It'd be a devserver thing.
<lifeless> rockstar: we're 95% custom publication code incompatible with wsgi
<rockstar> wgrant, FSVO?
<wgrant> For Some Values Of
<lifeless> rockstar: oh, we have a bunch of stuff that I'd be fine showing to you on prod.
<lifeless> you could expand it for lp.dev
<rockstar> lifeless, yeah, but the query middleware makes things pretty slow.
<wgrant> AIUI it's not so much a WSGI stack as some Zopeish Launchpaddery with a WSGI frontend glued on.
<lifeless> right
<rockstar> wgrant, yeah, that's what I was assuming, but I guess I assumed there was more WSGI-ish stuff than what's there.
<wgrant> Is crowberry still unwell?
<StevenK> This remains to be seen
<StevenK> wgrant: Why? Do you have an issue?
<lifeless>     Hard / Soft  Page ID
<lifeless>      283 /   17  Archive:EntryResource:getBuildSummariesForSourceIds
<lifeless>      180 /   30  Person:+commentedbugs
<lifeless>      140 /  203  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<lifeless>      100 /  251  BugTask:+index
<lifeless>       98 /    3  MailingListApplication:MailingListAPIView
<lifeless>       69 /  270  Distribution:+bugs
<lifeless>       16 /    0  Sprint:+temp-meeting-export
<lifeless>       13 / 1018  Archive:+index
<lifeless>       13 /   44  Archive:+packages
<wgrant> StevenK: Just wondering about the frequent downtime lately.
<lifeless>       13 /    1  ProjectGroup:+milestones
<wgrant> lifeless: That top one is a bit odd.
<StevenK> wgrant: We are clearly only trying to make you think there is an issue.
 * rockstar shuts laptop and concentrates on football
<lifeless> wgrant: whats odd aboot it ?
<wgrant> lifeless: I don't recall seeing that one high up before.
<wgrant> But I may be wrong.
<lifeless> https://bugs.edge.launchpad.net/soyuz/+bug/662523
<lifeless> wgrant: I've been nagging you to stab it for weeks
<wgrant> Hmm.
<lifeless> well
<lifeless> 11 days :P
<wgrant> StevenK: Has there been any discussion at UDS about deleting partner?
<lifeless> some
<StevenK> wgrant: Indeed. We have a plan. It will just take 4 years.
<wgrant> StevenK: ie. until Lucid EOL?
<StevenK> Yes
<wgrant> Can't we just repurpose commercial-compat to rebrand a PPA as partner?
 * StevenK shrugs
<wgrant> Then terminate ArchivePurpose 4 with extreme prejudice?
<StevenK> wgrant: Does this mean you have a branch that removes every bit of ArchivePurpose.PARTNER and you're looking for an excuse to land it?
<wgrant> StevenK: No.
<StevenK> Pity.
<wgrant> Unlike lucilleconfig.
<StevenK> Has that all trickled through?
 * thumper cries quietly while looking at blueprint code
<mwhudson> thumper: i wonder if you look at the code every week, say, it gets less surprising how awful it is
<thumper> mwhudson: I'm betting not
<mwhudson> (i doubt it, the brain doesn't remember pain)
<thumper> mwhudson: as part of my blueprint work I have a herd of yaks to shave
<StevenK> Which particular herd is it?
<mwhudson> i suppose if you spend an hour improving it each week, eventually it will be ok
<mwhudson> in like 150 weeks
<StevenK> Which is 3 years
<mwhudson> yeah, sounds optimistic
<StevenK> Optimistic at best. A nightmare at worst.
<StevenK> thumper: Put the rope down.
 * thumper might start soonish with the shaving
 * thumper shakes the shaving foam can
<thumper> is it worthwhile renaming Specification to Blueprint?
<wgrant> thumper: It's been through three names already... leave the poor creature alone.
<lifeless> that would confuse people
<thumper> what, renaming it?
<thumper> wgrant: three names?
<wallyworld_> thumper: what's the heritage of the blueprint code> why is it so bad?
<wallyworld_> s/>/?
<thumper> wallyworld_: that is a long story
<thumper> perhaps we can talk about it on Monday
<wgrant> thumper: Features, Blueprints, Specifications.
<wallyworld_> ok. sounds like something best discussed with a case of red at close hand
<wgrant> But it was renamed to Blueprints long before most people here appeared.
<thumper> the word Specification is only mentioned 2699 times in the code
<thumper> simple find and replace :)
<wgrant> Heh.
<StevenK> Was there a Feature to rename Features to Blueprints?
<thumper> StevenK: no, it would have been Feature to Specification
<thumper> and Specs to Blueprints
<wgrant> I think it was originally specification.
<thumper> the UI changed, but not the code
<wgrant> Then to feature, then specification, the blueprint.
<wgrant> But that first transition may be wrong.
<StevenK> We changed it *back*?
<thumper> I just find it a tad annoying in the code
<thumper> and 100 files would need to be renamed
<thumper> but that is a small price to pay I think for some form of consistency
<StevenK> And a 17,000 line diff
<thumper> sure, whatever
<thumper> as long as it doesn't conflict :-)
<StevenK> ... more than 3 times
<wgrant> I'd be against the rename. Most other objects in Launchpad now have sane names.
<wgrant> Specification -> Blueprint is the opposite of sane.
<StevenK> The URL is blueprints.l.n
<wgrant> It is.
<wallyworld_> i've never used the term Blueprint wrt software. for me, it's always been a Specification Document
<wgrant> But most people call them specs.
<wallyworld_> blueprint are something use use to build a !$%@$ house
<wgrant> Heh.
<StevenK> So now they're blues?
<mwhudson> i just try to alternate
<StevenK> As in, we have them?
<StevenK> wallyworld_: No, blueprints are used for a *bikeshed*
<wallyworld_> :-D
<wallyworld_> so my vote would be to use the industry accepted term, namely Specifications
<wallyworld_> why reinvent terms that are in common usage?
<wgrant> Because five years ago it was cool.
<wgrant> Or something.
<StevenK> wgrant is just bitter because five years ago he was eight.
<wallyworld_> how could calling something a non-standard term that no-one else uses in that context have been considered cool?
<wallyworld_> ever?
<wgrant> wallyworld_: Don't ever go back before r3000 or so.
 * thumper snorts
<wallyworld_> what's lurking there? anything interesting?
<thumper> FAAARRRRKKKK!
<thumper> buildbot is all read
<thumper> or red even
<wallyworld_> maybe it's feeling Blue :-)
<thumper>    lp.registry.windmill.tests.test_distroseriesdifference_expander.TestDistroSeriesDifferenceExtraJS.test_diff_extra_details_blacklisting
<thumper>    lp.registry.windmill.tests.test_product.TestProductIndexPage.test_title_inline_edit
<thumper> that's devel
<rockstar> thumper, yes, rs=me on deleting all windmill tests
<rockstar> :)
<wgrant> wallyworld_: Names were a fair bit less sane back then.
<thumper> db-devel: test_uploading_collection (lp.soyuz.tests.test_binarypackagebuildbehavior.TestBinaryBuildPackageBehaviorBuildCollection)
<lifeless> rockstar: beep, denied.
<wallyworld_> thumper: you saying those tests are failing?
<thumper> wallyworld_: failed on buildbot
<wallyworld_> what's the error?
<thumper> wallyworld_: did you want to take a look?
<lifeless> thumper: in terms of names, I think we should do serious user research before doing another rename.
<thumper> lifeless: ack
<rockstar> lifeless, I've said this before, but I think their cost outweighs their value.
<wgrant> *cough* Branches *cough*
<wallyworld_> thumper: tell me where to look - do i log into devpad to see the logs?
<thumper> lifeless: there is enough other yaks to shave in blueprints
<lifeless> thumper: exactly.
<thumper> wallyworld_: https://lpbuildbot.canonical.com/waterfall - click on summary
<rockstar> Blueprints mean something very valuable to the core users of blueprints: Ubuntu.
<rockstar> Also, of the three things I've sent to ec2 and the two that deryck sent to ec2 today, we have no emails to show for it.  :(
<StevenK> rockstar: I tossed two branches at ec2 and got 2 mails
<rockstar> StevenK, lucky you.
<wallyworld_> rockstar: i've had stuff go missing lately too :-(
<rockstar> StevenK, still, 2 for 7 isn't great.
<StevenK> rockstar: Better than 0 for 5 :-P
<rockstar> wallyworld_, yeah, it really affects my productivity, because now I have to go back and babysit.
<rockstar> StevenK, not for me it isn't!  :)
<StevenK> We should get out of testfix first, though
<wallyworld_> rockstar: i hear you. i've lost a lot of time as well, also chasing down "failures" which weren't
<wgrant> StevenK: Better than 0 for 9 :(
<StevenK> The last time it happened to me, there was a bug which I had introduced which was killing stuff
<rockstar> One of these branches has support for soupmatchers, which I want to use everywhere, so it'd be nice it that landed...
<rockstar> StevenK, so, there's some layer teardown stuff that's causing hangs in windmill.  deryck and I finally reproduced it locally with my lazr-js branch.
<wgrant> Ah, excellent news.
<wallyworld_> thumper: well, there's no dog's ball anywhere that i can see. and the code in question hasn't changed recently
<lifeless> rockstar: interesting
<lifeless> rockstar: tried with trunk? landed a layer teardown change today
<thumper> wallyworld_: does it pass locally for you?
<wallyworld_> trying now, updating as we speak
<wallyworld_> s/updating/building
<wallyworld_> have to stop my other db-devel system first and switch schemas @#@^^!$@
<lifeless> thumper: hey, could you perhaps https://code.edge.launchpad.net/~lifeless/launchpad/edge/+merge/39594 ?
<thumper> lifeless: there seems to be some waste there
 * thumper comments
<lifeless> oh?
<lifeless> thanks
<rockstar> lifeless, well, ec2 merges into trunk, right?
<lifeless> rockstar: yes, but it would depend on the revno at the time you started ec2
<thumper> lifeless: done
<lifeless> thumper: thanks
<rockstar> lifeless, yeah.  I'm sending them back to ec2.
<rockstar> lifeless, also, it seems that ec2 is still running postgres 8.3
<lifeless>  /o\
<lifeless> thumper: >< our coding standards make your change _longer_
<thumper> :)
 * lifeless ignores the nuts wrapping rules and just does the bzr way.
<lifeless> hmm, I've added another 80 lines of budget :)
<wallyworld_> thumper: apart from a LayerIsolationError, WFM
<thumper> hmm...
<thumper> wondering if we should just force a build to see if it works this time
<wallyworld_> that would be my plan of attach. at least the problem will be deffered for a little while :-)
<wallyworld_> i gotta learn to type better
<thumper> I've forced devel
<wallyworld_> let's hope it works
<thumper> wondering about the db-devel failure though
 * wallyworld_ looks
<wallyworld_> something just doesn't look right with that build server. those errors don't appear coding issues do they
<jtv> How can I have a webservice method that I can invoke in a stories test but not with launchpadlib against launchpad.dev?
<jtv> I thought the one implied the other.
<lifeless> night all
<wgrant> jtv: Are you using launchpadlib in your test?
<jtv> no, webservice
<wgrant> Your WADL is out of date.
<wgrant> make build
<jtv> !
 * jtv tries that
<jtv> wgrant: then again, I don't see the method on staging or qastaging either, and the revision's been on there for a while now.
<wgrant> jtv: You're not checking a frozen version of the webservice?
<jtv> wgrant: I tried version='devel'
<wgrant> Which method?
<jtv> IHasTranslationImports.getTranslationImportQueueEntries
<jtv> Tested in lib/lp/translations/stories/webservice/xx-translationimportqueue.txt
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (167): FIXED in 3 hr 36 min: https://hudson.wedontsleep.org/job/devel/167/
<LPCIBot> Launchpad Patch Queue Manager: [r=thumper][ui=none][bug=662552] Re-use POFile edit permission info
<LPCIBot> across message sub-views.
<jtv> wgrant: what I don't get is how that test can pass, yet I can't get at that method using launchpadlib interactively.  I haven't played with launchpadlib much so maybe I'm just doing something wrong there.
<wgrant> jtv: Old-style webservice tests do not use WADL.
<wgrant> So they don't care how broken it is.
<jtv> wgrant: so I neglected to update the wadl?
<wgrant> jtv: It's meant to be autogenerated.
<jtv> grrr
<jtv> But surely there is a way of getting our methods into the API, or else why do we have any
<wgrant> It is in the WADL, though.
<wgrant> Or in my WADL, at least.
<wgrant> And in my apidoc.
<jtv> So maybe I'm just doing something stupid.
<wgrant> When did it land?
<wgrant> Is it deployed?
<jtv> Days ago.  It should be on staging.
<jtv> Here's what I do locally:
<jtv> l=Launchpad.get_token_and_login('gaga', service_root='https://api.launchpad.dev/', version='devel')
<jtv> (Authorize in browser)
<wgrant> jtv: It's on qastaging's apidoc.
<jtv> l.me.getTranslationImportQueueEntries()
<jtv> AttributeError: 'Entry' object has no attribute 'getTranslationImportQueueEntries'
<wgrant> jtv:
<wgrant> grep -r getTranslationImportQueueEntries lib/canonical/launchpad/apidoc/
<jtv> Nothing.
<wgrant> Does 'make apidoc' take forever and say it's generating stuff for all three versions?
<jtv> I haven't tried that.
<jtv> (Didn't know it existed)
<wgrant> build depends
<wgrant> ... on it
<jtv> Well, "make build" does take forever now, after a "make clean."
<wgrant> But that's probably buildout automatically sacrificing various items of wildlife that if finds around you.
<wgrant> Not the WADL generation.
<jtv> wgrant: it's building that now
<wgrant> Aha.
<jtv> And now my wadl shows the method.  Wonder why "make build" didn't do that before.
<jtv> Are these files under revision control?
<wgrant> They're not.
<wgrant> But this may change soon.
<wgrant> They take forever to generate.
<jtv> Ah, I did this in an old branch.
<wgrant> So 'make build' probably won't regenerate them every time.
<jtv> I guess the dependencies aren't complete.
<wgrant> Or make run would take even longer than it does already.
<wgrant> s/aren't/can't be without taking 15 minutes to start an appserver/
<wgrant> Anyway, hopefully that will all work.
<wgrant> I need to run off and submit my final project.
<jtv> wgrant: thanks as always!
<jtv> Finalâ¦ university project?
<wgrant> That's the one.
 * thumper closes up.
<jtv> wgrant: good luckâace it like I did.  :-)
<jtv> bye thumper
<wgrant> Heh. Thanks.
<wgrant> Night thumper.
<wallyworld_> thumper: quick quest before you go
<wallyworld_> ======================================================================
<wallyworld_> ERROR: lp.codehosting.vfs.tests.test_transport.TestLaunchpadTransportImplementation.test_win32_abspath (subunit.RemotedTestCase)
<wallyworld_> ----------------------------------------------------------------------
<wallyworld_> _StringException: Text attachment: garbage
<wallyworld_> ------------
<wallyworld_> [<bzrlib.smart.medium.SmartTCPClientMedium object at 0xc109810>]
<wallyworld_> does that mean there's something screwed on ec2 wrt code hosting or something
<wallyworld_> an ec2 test run just failed with 1000 odd errors all like this
<thumper> ?
<thumper> wallyworld_: ah....
<thumper> not seen that before
<thumper> sounds like something screwy
<wallyworld_> thumper: no matter. looks like there's something environmental going on
 * thumper nods and leaves
<wallyworld_> i'll resubmit later. not having much luck with ec2 lately :-( have a good w/e
<spiv> losa fyi: I'm trying to reproduce a bzr/codehosting bug, and I think I may have caused a python process to spin in an infinite loop on codehosting.
<spm> impressive
<spiv> spm: if you're not too busy, I'd be quite curious to know if I really have done that...
<jtv> Is anyone here on first-name terms with the webservice API?
<wgrant> What has it done to you now?
<jtv> Well my method is documented now
<wgrant> !!
<jtv> I mean, it's in the wadl and apidoc now,
<jtv> _but_
<jtv> it doesn't appear on the objects that implement it and the interface it's in.
<jtv> Instead, it appears in a separate API class named after the interface.
<jtv> Which I suppose means that, instead of making objects implement the interface, I need to derive the objects' interfaces from my interface.
<wgrant> That's right.
<jtv> Pretty unfashionable strong static typing, that.
<wgrant> It is unfortunate, but fortunately not difficult.
<jtv> Unless it goes against the direction of all other dependencies and so is likely to introduce massive amounts of circular imports.
<wgrant> Well, true.
<jtv> And guess what?  That appears to be the case.  :)
<wgrant> :D
<shadeslayer> any ideas if my qtwebkit build was fixed?
<adeuring> good morning
<mrevell> Hello
<lifeless> mwhudson: does anything outside the dc know about guava ?
<jml> hello
<mwhudson> lifeless: no
<lifeless> mthaddon: ^
<mthaddon> I'm going to assume a whole lot of context there and merge that config change
<lifeless> mthaddon: there is an rt for getting a view on haproxy; should we add to that a note to get view on this new haproxy ?
<mthaddon> that'd probably make sense, yeah
<mwhudson> lifeless: although guava is in the dmz for historically daft reasons, so it can't connect to dc machines quite as one would expect
<mwhudson> (don't know if this is important here)
<lifeless> mwhudson: I don't think it is
<lifeless> mwhudson: story is - we're haproxying up codebrowse for no-downtime deploys.
<mwhudson> mtaylor: the launchpad/bazaar session has just been moved later, hope that's ok with you
<lifeless> mwhudson: so we just want to make sure nothing is left using the un-haproxy urls
<mwhudson> lifeless: yeah, all real traffic goes via crowberry
<mrevell> adeuring, Do you have a moment? I have a question about bug pages
<adeuring> mrevell: sure
<mrevell> thanks adeuring. I'm working on bug 117460 and I need to add a help link next to "Also affects project" on bug pages. Looking at bug.py I thought I should ask some advice before diving in. Do you have a suggestion for how to do it?
<_mup_> Bug #117460: "Also affects project" doesn't provide any instructions <better-forwarding> <trivial> <ui> <Launchpad Documentation:Invalid> <Launchpad Bugs:Triaged by matthew.revell> <https://launchpad.net/bugs/117460>
 * adeuring is reading the bug repoert
<mrevell> adeuring, It's quite an involved bug report and some of it has been split off into other reports and those solved separately. I'm going to deal with the "people don't understand what 'also affects project' means" aspect.
<adeuring> mrevell: right... difficult task ;) shall we talk on mumble about it?
<mrevell> adeuring, Sure!
<mrevell> adeuring, You don't hear me?
<adeuring> mrevell: no, i don't. also, mumble does seems to notice that you are speaking
<mrevell> Let me check the mic isn't muted
<adeuring> mrevell: so, the links (actually there are two: "add project" and "add distribution") are defined as objects of class Link in lp.bugs.browser.bug ; they are rendered in lp/bugs/templates/bugtasks-and-nominations-table.pt . That is the template where you can add the "help magic", I think.
<rockstar> Ooh! I think deryck and I have figured out why ec2 instances aren't sending out email.
<mrevell> Thanks adeuring, I shall give that a try!
<rockstar> mars, ^^
<mars> rockstar, eh?
<rockstar> mars, we shutdown automatically after 5 hours, right?  We're 20 minutes into that 5 hours before the tests even start running.  If our tests have any reason to take 4.5 hours (buildbot says 4.4 right now), it shuts down before the tests are done.
<rockstar> So any heavy I/O would slow the test suite down, etc.
<mars> rockstar, ugh, yes, great hypothesis.  "5 hours will be enough"  ("No one will ever need more that 640K of memory")
<rockstar> mars, if the layer tearDown stuff was really an issue, abentley would be saying something about it.
<rockstar> mars, but he doesn't have to worry about time limits when he runs his tests.  They're done when they're done.
<mars> rockstar, darn test run time - do we graph that I wonder?
<mars> rockstar, it used to be ~3 hours
<rockstar> mars, deryck and I are cumulatively 0 for 8 on ec2 runs.
<mars> that is why we set a 5 hour limit
<rockstar> mars, it USED to be ~2 hours, 3 years ago.  :)
<lifeless> up it ;)
<mars> 8(
<lifeless> we're working on test perf now
<rockstar> lifeless, yeah, I'm bumping the timeout on a single run.
<abentley> rockstar: I don't use test_on_merge because it times out.  Instead I run bin/test, which runs as long as it takes to finish.
<mars> rockstar, I think we should up the limit anyway
<mars> I'll land the branch. rs=lifeless?
<rockstar> abentley, I'm not sure if ec2 runs on test_on_merge specifically, but even if it didn't time out, ec2 shuts down after 300 minutes.
<rockstar> mars, rs=rockstar
<lifeless> mwhudson: yes
<lifeless> bah
<lifeless> mars: yes
<StevenK> mars: We're in testfix
 * rockstar facepalms
<abentley> rockstar: So, I know test_on_merge dies due to timeouts, but it's a different timeout.  It's a "minutes without output" timeout.
<rockstar> abentley, yeah, so in ec2, we could be getting output, but the test suite could be taking longer than the ~260 minutes we allot it.
<abentley> rockstar: indeed.  Now what's this layer teardown thing?
<rockstar> abentley, deryck and I reproduced an issue with windmill where it fails to teardown a layer and then just stops dead right then.
<abentley> rockstar: Ah.  Not a problem I recall.
<rockstar> abentley, yeah, I suspect it has a lot to do with windmill's asshattery.
<lifeless> rockstar: interesting
<lifeless> rockstar: how do you trigger that ?
<rockstar> lifeless, I thought deryck sent to output to the list.
<lifeless> I'm behind on mail just now
<rockstar> lifeless, he says he didn't send a mail, because he just disabled the test because it was failing spuriously anyway.
<rockstar> lifeless, bin/test --layer WindmillLayer
<lifeless> rockstar: did he file a bug? We need to treat our test environment as rigorously as we do our target code
<rockstar> lifeless, he didn't at the time, but I just asked him to.
<lifeless> \o/
<rockstar> lifeless, to be fair, we've been trying to get a new lazr-js landed in launchpad for more than a month now.  We're trying to get it done by hook or by crook.
<lifeless> rockstar: I'm fine with that
<lifeless> as long as we record interesting things like such problems as you've encountered.
<mars> rockstar, would that be the spurious failure Ian said he fixed?
<rockstar> mars, no idea.  We're getting all manner of spurious failures.
<mars> rockstar, I ask because I wonder if the test was re-enabled
<rockstar> mars, I don't think so, since this branch is pretty old.  Of course, we merged regularly, but there's lots of stuff that windmill does stupidl.
<StevenK> mars: bigjools is landing a testfix
<mars> StevenK, ok, thanks
 * bigjools is literally just landing it
<mars> lifeless, so this is an rs= fix, your new proceedure says I need an MP for it as well?
<lifeless> mars: my process experiment is for you if you do r=mars
<lifeless> mars: rs=otherperson is an existing defined process that I didn't touch
<mars> right
<mars> rs= is 'land this and skip the MP'
<lifeless> we could, if the experiment is successful, further iterate and combine these things
<StevenK> lifeless: r=trivial will continue to be verbotten?
<rockstar> mars, for instance, in a test, windmill has recently decided that even though a link is green and has a javascript event connected to it, it would rather just click through.
<lifeless> StevenK: yes
<lifeless> 'trivia' is not a very useful metric IMO
<StevenK> lifeless: Okay, good
<mars> rockstar, timing issue?
<rockstar> mars, no, not a timing issue, just a bug of some sort.
<rockstar> mars, something is quite obviously broken in windmill.
<mars> rockstar, if you can narrow it down, we can take it to #windmill directly
<mars> they would really want to know about that
<mars> rockstar, I assume it was the 1.4 upgrade
<rockstar> mars, probably, but I'd rather invest no more time in windmill at all.
<mars> rockstar, show me an alternative that takes less time than fixing what we have, and I'll go for it :)
<rockstar> mars, I keep meaning to write to the mailing list about this.  :)
<mars> oh, and that doesn't lose any of the benefits that windmill gives us
<rockstar> mars, right now, windmill isn't really giving us many benefits.
<mars> (notice that I did not say what the benefits /are/)
<lifeless> who knows how to get to the staging outbound mailbox ?
<mars> lifeless, deryck and team might?
<bigjools> my fix is in PQM
<mars> bigjools, ok, thanks for letting me know
<lifeless> I will hunt deryck
<rockstar> lifeless, deryck says bigjools has the credentials in an encrypted text file somewhere and can get to it easier than he can.
<bigjools> que?
<mars> bigjools, lifeless wants to access the staging outbound mailbox
<bigjools> mars: ok, I have it
<bigjools> kwallet FTW
<flacoste> lifeless: i found ftp://ftp.cs.wpi.edu/pub/techreports/pdf/06-17.pdf
<flacoste> it describes a space-efficient way to compute an approximation of the median
<flacoste> works on streaming data
<flacoste> should be fine for us
<flacoste> lifeless: another simpler approach would be to estimate the median using the frequency table
<lifeless> flacoste: both sound reasonable to me
<lifeless> and incremental
<lifeless> we should be able to do what we want without a DB
<lifeless> https://bugs.staging.launchpad.net/bzr/+bug/644855
<_mup_> Bug #644855: TestVersion.test_main_version fails run from installed package <easy> <selftest> <Bazaar:Fix Released by vila> <Bazaar 2.2:Fix Released by vila> <Bazaar 2.3:Fix Released by vila> <https://launchpad.net/bugs/644855>
<lifeless> https://staging.launchpad.net/bzr/+milestone/2.2.2/+subscribe
<lifeless> flacoste: ping
<flacoste> hi lifeless
<lifeless> calling you
<flacoste> lifeless: try again?
<lifeless> on skype... goes to voicemail
<flacoste> hmm
<flacoste> yeah, same thing if i try to call you
<flacoste> :-/
<flacoste> let me restart skype
<mars> bigjools, did your testfix go through?  Mine bounced on the testfix regex
<flacoste> lifeless: restarted
<flacoste> pfff
<flacoste> it does ring here
<lifeless> ring me ?
<bigjools> mars: it's in devel
<mars> bigjools, ok, testfix didn't clear - I thought it would have
<bigjools> me too :/
<StevenK> I think the poller needs to run to do so
<StevenK> And that will only do so after buildbot builds devel
<mars> StevenK, gary says the poller runs every 5-10 minutes?
<StevenK> Yes, but I think the last devel build in buildbot needs to be sucessful
 * mars opts for empirical evidence
<mars> pqm-submit - play it again!
<StevenK> lp-land FTW
<abentley> lifeless: will landing on stable update the cron scripts for qa-staging?
<lifeless> mm, ask mthaddon / chex
<abentley> mthaddon, chex: will landing on stable update the cron scripts for qa-staging?
<Chex> abentley: are you talking about the /cronscripts/ directory in the root of the LP branch?
<lifeless> bigjools: hw did you go?
<abentley> Chex: I'm asking about the behaviour of the whole system.  I haven't actually changed files in that directory, but I've changed files elsewhere that will change their behaviour.  Will I get the updated behaviour on qa-staging the way I would with staging?
<bigjools> lifeless: badly, still trying to get staging's inbox to load
<lifeless> bigjools: :(
<poolie_droid> jml: do we have any launchpad stickers here?
<jml> poolie_droid: no, we don't.
<poolie_droid> oh well
<poolie_droid> vikram was admiring mine
<shadeslayer> poolie_droid: bug 668407
<_mup_> Bug #668407: daily builds hang at pulling qtwebkit <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/668407>
<lifeless> I forgot my roll... sorry
<Chex> abentley: yes you should get the same behaviour that you would on staging with changes
<abentley> Chex: I know that asuka is the equivalent of loganberry for staging, but what's the equivalent for qa-staging?  Is there a diagram?
<abentley> rockstar: I just got a windmill hang.
<Chex> abentley: staging and qastaging are both like loganberry, the difference being qastaging is using a current copy of prod DB schema
<rockstar> abentley, ugh.
<rockstar> abentley, what kind of hang?
<abentley> Chex: what machine is running the cron scripts for qastaging that loganberry runs for production?
<Chex> abentley: asuka
<abentley> rockstar: The kind where it hangs.
<abentley> Chex: asuka is handling both staging and qastaging?
<rockstar> abentley, no traceback or anything?
<abentley> rockstar: no, that would be a crash.
<abentley> rockstar: It started running, and it never stopped, and it didn't appear to be making progress, and I waited about an hour.
<Chex> abentley: thats correct
<rockstar> abentley, yeah, so the one we saw was where it trackbacked in tearing down a layer, and then it just stopping tearing down the rest of the layers.
<abentley> rockstar: This was the whole output until I pressed CTRL-C.
<abentley> rockstar: http://pastebin.ubuntu.com/522177/
<abentley> rockstar: This is the output after I pressed CTRL-C: http://pastebin.ubuntu.com/522178/
<lifeless> abentley: we haven't enabled all cronscripts on qastaging because of load; if there's one you want to have execute work to do QA, ask a losa to run it (however many times you need)
<abentley> lifeless: it seems like landing on db-staging would be much simpler.
<abentley> lifeless: s/db-staging/db-stable/
<lifeless> abentley: we're going to have to do a similar thing for staging too - the load on asuka is just nuts.
<lifeless> abentley: or we may find a machine to do scripts on and move staging *and* qastaging scripts to that
<abentley> lifeless: up till now, we've relied on staging as a place we can test script changes.  It would be a significant loss if we were forced to use losas for our script testing.
<lifeless> abentley: yes, the current setup is undesirable, see bug ...
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/633713
<_mup_> Bug #633713: staging is overloaded <Launchpad Foundations:Triaged by canonical-losas> <https://launchpad.net/bugs/633713>
<abentley> lifeless: Please pursue that second option.  Could we use the internal UEC cloud?
<abentley> lifeless: Also, we could perhaps significantly reduce the load with a "New Task System" that was event-driven so it wasn't constantly polling.
<lifeless> the polling ones aren't the problems AIUI
<lifeless> anyhow, yes, am pursuing things
<abentley> lifeless: if the polling ones aren't the problems, you shouldn't need to disable them, right ? ;-)
<lifeless> abentley: it was simpler to just not turn anything on - we have /lots/ of cronscripts.
<lifeless> abentley: feel free to ask for specific ones to be enabled
<lifeless> gary_poster: are you here today?
<lifeless> gary_poster: I'd like to talk about that overload bug with isf/iso
<henninge> matsubara: hi!
<matsubara> henninge, hi
<henninge> matsubara: where does the code live that turns our script output into on oops?
<henninge> matsubara: or more specifically, can I avoid an oops by reducing a log message from "warn" to "info"?
<henninge> matsubara: or are you the wrong guy to ask? ;-)
<matsubara> henninge, lib/canonical/launchpad/webapp/errorlog.py and lib/lp/services/scripts/base.py
<henninge> matsubara: thanks, I'll look into those
<matsubara> henninge, yes, you can avoid an oops by reducing the log message to info only. See https://bugs.launchpad.net/malone/+bug/661441 where Robert suggest that approach to silence some checkwatches oopses.
<_mup_> Bug #661441: Duplicate oops reports logged for each checkwatches oops <oops> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/661441>
<henninge> matsubara: yup, it's in there. WARN and higher get turned into OOPSes.
<henninge> :-D
<matsubara> :-)
<henninge> matsubara: thanks
<matsubara> np henninge
<gary_poster> lifeless, here, was at lunch, 1 sec
<gary_poster> lifeless, dunno what isf/iso is, but back
<lifeless> jml: hi
<jml> lifeless: hello
<lifeless> where are you ?
<lifeless> bigjools: we're still blocked on that qa right
<lifeless> ?
<StevenK> jml: O hai, can haz mentor review for https://code.edge.launchpad.net/~henninge/launchpad/devel-bug-666660-poimport-oops/+merge/39653
<jml> lifeless: curucao 4
<jml> StevenK: granted
<henninge> jml: thanks!
<bigjools> lifeless: yes, I am unable to view the staging inbox
<jml> lifeless: do you not care?
<LPCIBot> Project devel build (168): SUCCESS in 3 hr 58 min: https://hudson.wedontsleep.org/job/devel/168/
<lifeless> jml: I care a lot
<lifeless> jml: I need to talk to you :)
<lifeless> bigjools: what were the details again ?
<lifeless> like
<lifeless> not the password
<lifeless> the other stuff
<bigjools> lifeless: ->ops
<gary_poster> lifeless: was @ lunch; appear to have replied when you were offline.  here now.  dunno what isf/iso is
<lifeless> gary_poster: is foundations/ops
<gary_poster> ah!
<gary_poster> ok
<lifeless> gary_poster: I cc'd you on mail instead
<gary_poster> oh
<lifeless> we'll get a machine for staging scripts
<gary_poster> cool, that will be great
<gary_poster> saw email
<lifeless> so, I'm trying to qa
<lifeless> but mail from bugs on staging doesn't seem to be going to the staging mailbox
<lifeless> mbarnett: we're not seeing the last few comments from https://bugs.staging.launchpad.net/bzr/+bug/644855 in the staging imap mbox
<_mup_> Bug #644855: TestVersion.test_main_version fails run from installed package <easy> <selftest> <Bazaar:Fix Released by vila> <Bazaar 2.2:Fix Released by vila> <Bazaar 2.3:Fix Released by vila> <https://launchpad.net/bugs/644855>
<mbarnett> lifeless: sorry, got a couple pings at the same time there.  give me a minute and i will take a look at that.
<lifeless> thanks
<mbarnett> lifeless: so, the issue is, new comments should be generating individual emails that are sent to the staging imap mbox, and this step isn't happening?
<lifeless> mbarnett: yeah
<lifeless> I think that that happens through a script
<mbarnett> lifeless: is this unique to that bug?
 * mbarnett goes to try and find the cronjob for generating bug-comment-emails
<lifeless> last mail I see there was 22 hours ago (for staging bug mail)
<lifeless> mbarnett: ^
<mbarnett> i ran the process-mail script by hand.  i saw some errors in the log file from the failed staging restore from a couple days ago
<mbarnett> but it seemed to return cleanly
<lifeless> I'll give it a minute and try again
<StevenK> losa ping
<mbarnett> heya StevenK
<lifeless> StevenK: <- -ops :P
<lifeless> StevenK: for the 'ping' itself.
<StevenK> lifeless: Yes, brain fail :-(
<lifeless> what does 'More than one process found' mean from cronscripts?
<lifeless> mbarnett: not seeing anything
<lifeless> mbarnett: what were the scripts errors ?
<mbarnett> lifeless: i saw some reference to shipit, which is what caused me to suspect it was from the failed staging update
<mbarnett> that and the fact that they were from 2 days ago
<lifeless> right; it might not be starting up properly then
<lifeless> ?
<mbarnett> lifeless: when you say "it" here, what exactly are you referring to?  /me is trying to figure out what *should* be happening with regard to bug-generated emails.
<lifeless> 'the bug mail sender' might not ...
<lifeless> mbarnett: I'd expect them to end up on m.c.c in the account 'staging'
<mbarnett> lifeless: right, but your question that "it" might not be starting up properly.. i am not sure what the "it" was.
<lifeless> mbarnett: the bug mail sending script - the one that gave you shipit errors
<lifeless> mbarnett: deleting lib/canonical/shipit should fix those errors
<mbarnett> lifeless: it is very possible i was looking at the wrong script... i am not sure if "process-mail" is what gets emails from bug comments to the staging ,mailbox
<lifeless> mbarnett: well the mail is sent to local smtp as usual
<lifeless> mbarnett: so you could look at that config
<mbarnett> lifeless: those errors were from a couple days ago, i haven't seen them since and the process-mail cronjob runs much more frequently, so i don't think we havea  problem there
<lifeless> mbarnett: ok; so its going somewhere else ;)
<lifeless> mbarnett: mail is being sent to localhost:25
<poolie_droid> Does anyone do their lp development on a cloud instance?
<lifeless> I use a vm, not a cloud instance though
<poolie_droid> As opposed to just running tests there
<lifeless> yes
<poolie_droid> Yes, I use a vm on my desktop
<poolie_droid> Am going to try using ec2 for a bit
<mtaylor> lifeless: that launchpad extension thing in the lunch session ... do you know if there's a project for that?
<lifeless> lunch session? I missed the lightning talks - was grabbed by two! people for 'must have before eow discussions'
<poolie_droid> Nice talk btw mtaylor
<mtaylor> lifeless: ah.
<mtaylor> poolie_droid: thanks!
<lifeless> mtaylor: what thing are you talking about
<mtaylor> poolie_droid: at this point, I have more-than-normal amount of experience talking to people who are switching to bzr :)
<mtaylor> lifeless: it was a thing someone was showing where he'd made a test-results web app thing, but used some amount of lpapi/something to make a) the app look launchpad-y and b) register with real launchpad the existence of his thing
<StevenK> mtaylor: That was cr3
<poolie_droid> Mtaylor the speaker was Marc Tardif, if you're talking about the thing with grease monkey
<lifeless> mtaylor: look at /launchpad-project
<mtaylor> poolie_droid: StevenK: sweet. thanks
<lifeless> mtaylor: its there
<mtaylor> lifeless: when you say "it's there" ... I am obviously unintelligent, cause I don't see nothing...
<mtaylor> lifeless: thanks
<poolie_droid> Got it now?
<mars> OOPS-1763CMP4
<StevenK> mars: You should be able to land your branch now
<mars> StevenK, thanks, I completely forgot about that :/
<StevenK> mars: Heh, I hadn't. I've been fighting to get us out of testfix for most of the day
<mars> StevenK, and most of your weekend, it appears.  That sucks.
<mars> Sent.  I'll know in (2*13)+(13/2) minutes
<mars> (approximately)
#launchpad-dev 2010-10-30
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (109): FIXED in 3 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/109/
<LPCIBot> Project devel build (169): FAILURE in 3 hr 56 min: https://hudson.wedontsleep.org/job/devel/169/
<LPCIBot> Project db-devel build (110): FAILURE in 3 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/110/
<LPCIBot> Project devel build (170): STILL FAILING in 3 hr 59 min: https://hudson.wedontsleep.org/job/devel/170/
<timoshin> package launchpad-dependencies have broken dependencies in ubuntu 10.10
<timoshin> i cant get sources
<timoshin> launchpad-dependencies : Depends: python-psycopg2 (< 2.0.14) but 2.2.1-1ubuntu1 is to be installed
<StevenK> lifeless: Rarh! Please don't, I have a testfix already in PQM
<lifeless> StevenK: it wasn't there when I prepped and sent it
 * wgrant wonders if the recent influx of bugs incorrectly targetted at 'launchpad' is due to the new footer link.
<lifeless> wgrant: possibly.
<lifeless> wgrant: lets try tweaking the words ?
<StevenK> lifeless: It should have been, I beat you by 3 minutes.
<lifeless> StevenK: mine will just conflict and bounce, so don't stress. Thanks for acting promptly to back your thing out
<StevenK> lifeless: I doubt it will bounce, mine is a proper testfix which fixes the tests.
 * StevenK ponders breakfast
<lifeless> StevenK: easy test
<lifeless> merge them together
<wgrant> StevenK: When are you flying out?
<lifeless> StevenK: you need to add rollback=xxxx to your fixes of this sort
<lifeless> StevenK: they *will* conflict
<StevenK> wgrant: I leave ORD at 530pm
<wgrant> Ah.
<StevenK> lifeless: Right, one conflict in lib/lp/soyuz/tests/test_archive.py
<lifeless> StevenK: did you ec2test your branch
<StevenK> lifeless: So PQM will mail you back saying "Too bad, so sad, it has conflicts" ?
<StevenK> lifeless: Originally, yes, but I stupidly did not after wgrant asked me to change some of the strings.
<wgrant> Sorry :/
<StevenK> wgrant: It's my fault, I changed the code, but not the tests.
<lifeless> StevenK: muscle memory needs to be 'via ec2 if *any* changes'
<StevenK> lifeless: Clearly. I think I was just so happy that I could finally land stuff I just didn't think.
<lifeless> kk
<LPCIBot> Project devel build (171): STILL FAILING in 3 hr 56 min: https://hudson.wedontsleep.org/job/devel/171/
#launchpad-dev 2010-10-31
<lifeless> zomg we're in testfix -again- ?
<lifeless>   Hard / Soft  Page ID
<lifeless>      273 /   12  Archive:EntryResource:getBuildSummariesForSourceIds
<lifeless>      216 /   48  Person:+commentedbugs
<lifeless>      137 /  161  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<lifeless>      101 /  214  BugTask:+index
<lifeless>       98 /    0  MailingListApplication:MailingListAPIView
<lifeless>       56 /  208  Distribution:+bugs
<lifeless>       15 /   31  Milestone:+index
<lifeless>       14 /    4  ProjectGroup:+milestones
<lifeless>       13 /   51  Archive:+packages
<lifeless>        9 /   34  Distribution:+addquestion
<wgrant> lifeless: Should there be an announcement about the beta redirect removal?
<wgrant> A few people have asked if it was broken.
<lifeless> wgrant: you mean like the blog post announcing it ?
<lifeless> wgrant: we can certainly do another announcement
<lifeless> http://blog.launchpad.net/general/continuous-deployment-in-launchpad
<lifeless> wgrant: good luck
<lifeless> someone needs to clear out a stale librarian in the db-devel test chroot
<lifeless> -> NZ
<LPCIBot> Project devel build (172): STILL FAILING in 3 hr 54 min: https://hudson.wedontsleep.org/job/devel/172/
<thumper> morning
<wallyworld> morning thumper. how about those magnificent wallabies?
<thumper> wallyworld: yeah, they're freaking awesome...
<wallyworld> glad you think so too
<wallyworld> i can already hear the all blecks choking next year for the world cup
<thumper> fcuk off
<wallyworld> :-D
<thumper> wallyworld: skype? so I can abuse you in private?
<wallyworld> ok
<thumper> https://bugs.edge.launchpad.net/launchpad-code/+bugs?field.tag=qa-needstesting
<lifeless> oh hai
<thumper> hi lifeless
<thumper> lifeless: back home?
<lifeless> yeah
<lifeless> walked in 10 minutes ago
<thumper> lifeless: buildbot isn't happy
<lifeless> needs losa intervention I think, if its what was up yesterday
<jml> hello again
<thumper> hi jml
<thumper> lifeless: there are spurious windmill failures on devel
<jml> thumper: hi
<thumper> I'm spinning up a test for devel to see if they are fine here
<thumper> I'm guessing it is a race condition and crappy timing
 * jml is in a Delta lounge, making lots of daily builds
<jml> (all of which fail to build from source, but there's only so much one can do).
<wgrant> Are many of the interesting ones importable?
<wgrant> Lots need submodules :(
<jml> wgrant: well, these are all for projects that use bzr in upstream
<thumper> jml: and none build locally?
<jml> thumper: pff, a secondary consideration
<wgrant> Ah
<jml> thumper: more seriously, it has taken roughly an hour to get these recipes created properly
<thumper> jml: what were the pain points?
<jml> thumper: slow internet, finding packaging source, lack of nest-part
<thumper> nest-part is coming...
<thumper> and had landed but was then reverted
<jml> thumper: yeah, I saw
<thumper> not sure how our changes caused the build farm to crap itself though
<jml> thumper: no idea either.
<wgrant> Do you know what bzr version is running outside the chroot?
<jml> thumper: we have quite a lot of information on the symptoms of the failure. if you think that will help, hassle bigjools
<jml> â¦ when he gets back
<thumper> wgrant: not exactly, 2.2 I think
<wgrant> :(
<thumper> jml: my solution is to offload the entire thing onto bigjools
<thumper> jml: it isn't something I can help with IMO
<thumper> perhaps getting aaron to work with him, but we don't have the access or knowledge in this area
<lifeless> https://code.launchpad.net/~lifeless/launchpad/prejoin/+merge/39720
<lifeless> https://code.launchpad.net/~lifeless/launchpad/mentoring/+merge/39721
<jml> lifeless: yeah, I saw the first. I'll review it now.
<jml> lifeless: is that an exact duplicate of DecoratedResultSet? the name implies that it does something prejoin-y
<lifeless> jml: different syntax for use and less flexible.
<lifeless> jml: no active users in the code base.
<jml> lifeless: +1
<lifeless> jml: I've been tweaking databasefixture but nothing substantial; you could look to see if there is anything unclear in my tweaks
<jml> lifeless: I won't have time before I board, sorry.
<lifeless> jml: no rush
<lifeless> jml: I'll toss it at ec2, and if we need to iterate, we can iterate.
<jml> lifeless: you'll no doubt be shocked to know that my testtools-experiment branch failed a test run.
<jml> thumper: that's fair enough. although, it would be good if we could spread the knowledge about these things to outside the soyuz team.
<thumper> jml: understood, but it is also access to dogfood
<jml> thumper: it would be good if we could spread access to effective dev environments beyond the soyuz team too :)
<thumper> jml: yeah... talk to bigjools about dogfood :)
<jml> thumper: I have in the past.
<jml> anyway, my flight's boarding.
<jml> I guess I won't get time to make a testresources daily build
<lifeless> thumper: we have to fix the things preventing qa on qastaging.
<lifeless> thumper: some of those need code changes e.g. to log more.
<jml> lifeless: https://launchpad.net/~testing-cabal â I've created a bunch of daily builds in a naive "I'm just an upstream developer" way
<lifeless> jml: I'm shicked I tell you, shocked.
<jml> lifeless: although I'd be happy if you fixed them, I would like to know exactly how, so I can learn.
<lifeless> jml: the builds ?
<lifeless> jml: I haven't looked specifically, but the use of nesting seems odd to me.
<jml> lifeless: odder still to me, since I thought I was merging :)
<lifeless> jml: well, you mentioned in tihs channel something about nest-part
<jml> lifeless: it would be better if I could use it.
<lifeless> thumper: what build has the spurious windmill failures
<jml> lifeless: anyway, I have to go.
<thumper> lifeless: devel
<jml> have a good day, antipodeans
<lifeless> jml: for testing sure, but for testing-cabal builds specifically I don't see why.
<lifeless> jml: ciao, fly well
<thumper> jml: have a good flight
<lifeless> man, I really want to know my overall diffstat for lp.
<lifeless> like
<lifeless> am I (since the TA job) a net code increase or decrease.
<lifeless> and how much simple 'churn'.
<thumper> lifeless: that info is there
<thumper> lifeless: we record lines added and removed for the diffs
<lifeless> per merge sure
<thumper> lifeless: well... what I'm saying is that the information is there so we could aggregate somehow
<lifeless> sure
<thumper> lifeless: when the rollout happened the other day, did it go to arsenic?
<thumper> lifeless: also, can we change the pageid for xmlrpc requests to include the method?
<thumper> lifeless: that would give us much better idea of which method calls are failing
<lifeless> whats arsenic
<lifeless> yes, there is a bug already
<lifeless> thumper: I'm ratshit today as I'm sure you can imagine
<lifeless> thumper: I'm not going to drill into the buildbot thing unless you or someone else hits a wall there : I have a backlog from UDS already and just pushing that forward will consume my all
<thumper> ack
#launchpad-dev 2011-10-24
<wallyworld_> huwshimi: do you have thoughts on how to display links that are not valid? hide or render as disabled?
<huwshimi> wallyworld_: Oh is this from the same as that email I got? Sorry I've been away sick the last few days
<wallyworld_> huwshimi: np. when you have some spare cycles, have a think and let me know
<huwshimi> wallyworld_: Thanks, I should get to it today
<wallyworld_> huwshimi: ok. you know the context etc i assume then? ie the bug task page and the "Also Affects" links. but i think we need a consistent policy for the product as a whole
<huwshimi> wallyworld_: Yeah, I think Francis's email has some info
<wallyworld_> cool
<poolie> wgrant, i wonder what you would think of https://code.launchpad.net/~mbp/launchpad/show-timeline/+merge/80166
<poolie> huwshimi, :-(
<huwshimi> poolie: Sorry :)
<poolie> is this actually less readable?
<poolie> some text is not wrapped at all and people cope
<huwshimi> poolie: Yes, and I don't want to make a regression and add to existing readability issues
<poolie> i anticipate markdown getting bikeshedded to death :/
<huwshimi> poolie: Why is that?
<poolie> oh, just previous similar changes, but i should be optimistic
<poolie> perhaps it can be done
<lifeless> poolie: in regards to the user commenting on a bug, or some other context?
<poolie> lifeless, what do you mean?
<lifeless> the markdown comment
<poolie> i think it would eventually go everywhere
<lifeless> the only reference to markdown I've read recently was a random popping into an old bug saying 'use markdown'
<poolie> huwshimi, i do respect your right to decide about it
<poolie> it is much better that someone is making the decisions
<poolie> lifeless, in the context of  https://code.launchpad.net/~mbp/launchpad/bug-width/+merge/80161
<poolie> i guess i'm just disappointed because for me, it was a big improvement and pretty easy to do
<poolie> i will try to be an adult
<wgrant> s/be an adult/use greasemonkey/?
<poolie> i suppose i feel there is also a general bias that "whatever crap currently exists" >> "something new but not perfect"
<poolie> but, if 45em really is the ideal width for text on a bug conversation, or close enough to ideal, that's ok
<lifeless> ah
<poolie> wgrant, yeah, or in fact probably a local stylesheet is enough
<lifeless> we don't have a good regression test suite around appearance
<poolie> but that too is disappointing
<lifeless> and huw is currently deep in an overhaul of appearance - the rebranding effort
<poolie> lifeless, i know
<lifeless> so its kindof hard right now to assess whether something is a win or not, because its so complex :(
<poolie> i think having humans assess regressions against a style guide would be nice
<poolie> perhaps the canonical style guide does cover this
<lifeless> I'm hoping one of the outcomes of the rebranding will be simpler presentation permitting easier understanding of changes like your proposed one
<lifeless> that said, I too am a little confused - I don't understand where we would lose usability with your change (but see under hard to asess because complex)
<poolie> so, very wide text is hard to read
<poolie> for example, mp comments don't wrap at all before the screen width, so they are slightly hard to read on wide monitors
<poolie> (unless, as fairly often happens, they are wrapped in the incoming email)
<poolie> i agree this is a problem, i just don't think using 80em is enough for the problem to bite seriously
<poolie> or at least it is a reasonable tradeoff
<poolie> lifeless, thanks for the review. what is ++profile++show
<poolie> ?
<lifeless> add this feature rule locally
<lifeless> profiling.enabled default 0 on
<lifeless> then visit /++profile++/
<lifeless> this generates an oops, then renders it in an overlay profile
<lifeless> there are two reasons we might not want to do that here: we don't want to send an oops on every developer request (but you can call create() and not publish() to achieve that)
<lifeless> and secondly there may be enough overhead to want to avoid it for regular developer requests (but thats unknown)
<lifeless> if you consolidated these code paths, it could be a pretty neat experience
<poolie> huh
<lifeless> the oops adds info like the pageid, which tells us the api or view invoked, host it ran on, execution time, and as we add things like thread rusage, that too
<poolie> yes that would be neat
<poolie> for this i wanted to make it something that would always be cheap enoguh to have available, at least to developers
<lifeless> I would be surprised if generating and not sending an oops were more than a few 10's of ms
<poolie> separately, i looked at https://bugs.launchpad.net/launchpad/+bug/878140 too but i'm stumped
<_mup_> Bug #878140: process-mail.py failed to resolve dns. Raised NXDOMAIN <dkim> <oops> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/878140 >
<poolie> StevenK, thanks for working on updating twisted
<lifeless> StevenK: wgrant: has sinzui set you loose on criticals or something ?
<wgrant> StevenK was for a while.
<StevenK> lifeless: I will tend to look at criticals when I'm booked -- sinzui has said that is okay.
<StevenK> s/booked/blocked/
<StevenK> And that twisted critical has been annoying me for a while.
<wgrant> We're nearly unblocked, finally.
<StevenK> I was pondering refactoring the switch team visibility to check for a PPA and private bug subscriptions
<StevenK> s/visibility/subscription policy/
<wgrant> Visibility, or membership policy?
<wgrant> Right.
 * StevenK de-prams his toys
<wgrant> There's several cards about extending the restrictions.
<wgrant> We need the restriction you are pondering, but it needs to be somewhat extensible.
<StevenK> Right
<StevenK> wgrant: Jump on mumble then?
<wgrant> Uhoh.
<StevenK> I can't have a few minute pre-impl? :-)
<wgrant> Critical bug balance over the last week is -16.
<wgrant> I need to file bugs :(
<lifeless> huh
<lifeless> I thought we showed a padlock for links to private things
<lifeless> have a look at bugs.l.n/python-oops-tools
<wgrant> lifeless: That table is special.
<wgrant> https://bugs.launchpad.net/python-oops-tools/+bugs
<lifeless> aieeee
<wgrant> Not quite sure why +bugs isn't the Bugs index now.
<wgrant> +bugs-index is pretty much a crippled, hideously useless version of +bugs.
<lifeless> by heat
<wgrant> Like Archive:+index vs Archive:+packages
<wgrant> Except that we have a vision for Archive:+index.
<elmo> wow
<wgrant> elmo: What have I done now?
<elmo> https://chinstrap.canonical.com/~james/nx/wtf.png
<wgrant> Indeed.
<wgrant> If you look down the bottom you can see the real error.
<elmo> yeah, I saw it
<wgrant> This wonderful error handling has been in place since early 2009 :)
<elmo> hmm, I don't think so
<elmo> or maybe I've been using refcontrol longer than I thought
<elmo> in any event, when I first started using refcontrol, it failed in less spectacular ways
<wgrant> It's a bit less ugly for non-AJAX operations.
<elmo> is there a bug about it and/or should I file one?
<elmo> ah, that could be it
<wgrant> It still uses ALLCAPS for no reason at all.
<wgrant> But at least doesn't spew JS at you.
<wgrant> I think there's a bug, but I can't find it.
<wgrant> Ah
<wgrant> Bug #521447
<_mup_> Bug #521447: ajax errors show 'following errors occured' or a big dump of the html source for the oops page <error-handling> <errors> <javascript> <lp-bugs> <ui> <Launchpad itself:Triaged> <LAZR Javascript Library:Triaged> < https://launchpad.net/bugs/521447 >
<poolie> huwshimi, lifeless, the text width thing would be a case where it would be interesting to have google labs style per-user feature flags
<wgrant> Also
<wgrant> bug #408491
<_mup_> Bug #408491: Javascript actions error handling needs work <javascript> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/408491 >
<poolie> to say "you can opt in to wide text", or markdown, or whatever
<poolie> and see what happens
<poolie> elmo, Note the consistent user interface and error reportage. Launchpad is generous enough to flag errors, yet prudent enough not to overwhelm the novice with verbosity.
<elmo> poolie: haha
<nigelb> Hrm, no jtv yet.
<elmo> (oh, sorry - I didn't mean to post that chinstrap link in here - non-canonical folks can infer the contents from the bug wgrant linked to)
<wgrant> Hah, I assumed this was -ops.
<wgrant> You don't often venture out here :)
<nigelb> elmo: I'm guessing you had a whole bunch of red text as the outcome of a failed ajax operation
<nigelb> wgrant: hehe
<lifeless> poolie: that was a little mean :(
<lifeless> still winning -
<lifeless>     2029  OOPS-2122AX39   Product:+activereviews
<wgrant> Product:+download is a pretty easy one, hopefully.
<wgrant> Just lots of LFADC queries.
<huwshimi> wallyworld: I've just replied to that email about hiding/disabling. Let me know if that doesn't help
<wallyworld> huwshimi: thanks. will look
<wallyworld> huwshimi: i also want to ask you about something else when you have a moment
<huwshimi> wallyworld: sure, anytime
<wallyworld> huwshimi: as an aside, there's a weird css glitch with the edit icons used for the yui activator - they are a few pixels higher than the text they go next to
 * StevenK melts from the heat.
<wallyworld> which leads me to this screenshot
<wallyworld> http://people.canonical.com/~ianb/bugtask-remove-icons.png
<wallyworld> huwshimi: i want to provide ui to delete bug tasks, so the screenshot puts the remove icons next to those tasks that are allowed to be deleted
<wallyworld> but it looks crap due to the css issue i just highlighted
 * StevenK kicks Unity.
<wallyworld> note that other edit icons in the same row are aligned
<wallyworld> i've had a quick look at the css but nothing jumps out at me as being the obvious cause
<wallyworld> huwshimi: so, i need input on the placement if the remove icons plus whether we want to fix the edit icon alignment
<huwshimi> wallyworld: the remove icons have the correct alignment. We need to fix the edit icons
<huwshimi> wallyworld: Is there a way for me to run that code locally?
<wgrant> Even better if you can remove that couple of em of padding between the expander and icon.
<wallyworld> huwshimi: to do that screenshot, i just run lp.dev and hacked the html with "<a class='sprite remove'></a>
<wgrant> I tried, but lazr-js is ew.
<wallyworld> we no longer use lazr-js :-)
<huwshimi> wallyworld: Ah great
<wallyworld> but we use its code
<wgrant> We use lazr-js, just it happens to have been merged into our tree.
<wallyworld> huwshimi: so use firebug i assume
<wallyworld> s/so/you
<huwshimi> wallyworld: yeah that's fine
<wallyworld> wgrant: i know, i was trying to be 'funny'
 * wallyworld needs coffee
<huwshimi> wallyworld: Oh was that a question?
<wallyworld> huwshimi: what was?
<huwshimi> wallyworld: Did your correction mean you were asking if used firebug?
<wallyworld> huwshimi: yeah, just making sure you were able to hack in the html like i did
<huwshimi> wallyworld: Ah yes that's fine
<huwshimi> wallyworld: Just waiting for my launchpad to update
<wallyworld> cool. i thought it would be, just making sure :-)
<wallyworld> np
<wallyworld> i'm waiting for your email to arrive
<wallyworld> so i'll go grab a caffinated beverage
<huwshimi> wallyworld: Oh strange
<huwshimi> wallyworld: please post caffeinated beverages
 * wgrant renames PillarObserverPolicies to AccessPolicies.
<wgrant> Considered VisibilityPolicy, but that's almost Soyuz-long...
<wallyworld> huwshimi: i agree with your thoughts. my preference was to hide the link(s).
<huwshimi> wallyworld: Right, I don't fully understand the situation, but it seems like they can never interact with those links if a bug is private so they should be hidden
<wallyworld> huwshimi: sort of. it's more that there are already bug tasks on the bug for other targets and so you can't go and associate the bug with something else as well
<wallyworld> since this will allow private info to leak
<huwshimi> wallyworld: oh
<wallyworld> does that change things?
<huwshimi> wallyworld: I'm not sure, it might be a situation for some explanation
<wgrant> (for now, but we are moving to removing those links for public bugs too eventually)
<wallyworld> yes, and it's not like the user can readily "fix" the situation to make those unallowed links valid
<wgrant> They can make the bug public :)
<wallyworld> sure
<wallyworld> but that goes against the intent
<wgrant> But we probably need to somehow make it clear that the links aren't just randomly missing from some bugs.
<huwshimi> ugh I can't get make schema to work
<wgrant> Oh?
<huwshimi> "psycopg2.ProgrammingError: function ftiupdate() does not exist"
<wgrant> Have you run launchpad-database-setup on this machine before?
<huwshimi> wgrant: No
<wgrant> utilities/launchpad-database-setup
<huwshimi> wgrant: unless it got run during the setup
<wgrant> It's not run automatically, no.
<wgrant> It's fairly destructive :)
<StevenK> utilities/launchpad-database-setup $USER
<huwshimi> thanks
<wallyworld> wgrant: the hard bit is to make it cleat that "links aren't randomly missing" in a concise way on the gui. maybe they should be disabled rather than hidden then
<wgrant> StevenK: Never recommend that directly.
<wgrant> StevenK: The username is an unobvious argument for a reason.
<StevenK> Aw
<StevenK> I just run it
<StevenK> But I'm pretty clear on what it does and why
<wgrant> Yes.
<wgrant> But new people are not.
<huwshimi> wallyworld: this is what I get: http://i.imgur.com/Nf7Bq.png
<wgrant> Hah.
<wgrant> Which browser?
<wallyworld> huwshimi: hmmm. that looks different to mine
<wgrant> It's the opposite.
<wgrant> The status icon is too high.
<wgrant> The target one is just right.
<wallyworld> i'm using ff 7.01
<huwshimi> wallyworld: I'm using chromium of some non-descript version number
<wallyworld> bollocks
<huwshimi> wallyworld: 14.0.835.202 (Developer Build 103287 Linux)
<wallyworld> so do we hack in css corrections for browser differences? surely not :-(
<huwshimi> wallyworld: Where in the html did you add the new <a>?
<poolie> lifeless, https://code.launchpad.net/~mbp/rabbitfixture/rabbit-startup/+merge/80174
<wallyworld> huwshimi: after the span i think. let me check
<huwshimi> wallyworld: This is what mine looked like: http://paste.ubuntu.com/717578/
<wallyworld> huwshimi: mine went after the div but before the span
<wallyworld> whereas yours in inside the div
<wallyworld> the div pertains to the edit widget hence i put it outside that
<huwshimi> wallyworld: oh right
<wallyworld> not sure if that will matter
<wallyworld> in terms of affecting layout
<huwshimi> wallyworld: wait, where?
<huwshimi> wallyworld: Do you want to paste it?
<wallyworld> huwshimi: here's the tales, which i've dome now that i have it working for real https://pastebin.canonical.com/5478
<huwshimi> wallyworld: Was that meant to go to me?
<wallyworld> yes, sorry, i can do html also
<wallyworld> just a sec
<huwshimi> wallyworld: What should I be looking at from that query?
<wallyworld> huwshimi: it's template which is turned into html, here's the html https://pastebin.canonical.com/54783/
<huwshimi> wallyworld: right, but that first paste was the results of a db query, I'm assuming that's not what you intended on pasting
<wallyworld> huwshimi: ah, right. cut and paste error. one digit missing https://pastebin.canonical.com/54782/
<wallyworld> sorry
<huwshimi> wallyworld: Ah :)
<huwshimi> wallyworld: I was very confused
<wallyworld> :-)
<huwshimi> wallyworld: What happens if you apply "vertical-align: middle;" to both of the icons?
 * wallyworld tries that
<wallyworld> huwshimi: trouble it, the edit icon is a <button>
<wallyworld> and addign that style makes it disappear
<wallyworld> at least it disappeared for me
<wallyworld> the html is <button class="lazr-btn yui3-activator-act"> Edit </button>
<huwshimi> wallyworld: that's strange, it worked for me
<wallyworld> hmmm. could be a firebug thing
<wallyworld> i'll add that to the lazr-js css and see
<wallyworld> huwshimi: seems to have helped a bit http://people.canonical.com/~ianb/bugtask-icons-1.png
<wallyworld> huwshimi: the css turns out as https://pastebin.canonical.com/54784/
<huwshimi> wallyworld: how do they line up with other icons on the page?
<wallyworld> huwshimi: as per the screenshot, they look a little higher than the other similar icons in that row (which have no vertical alignment style) but at least they appear consistent to each other
<wallyworld> perhaps we need to add that vertical alignment style to all such icons
<huwshimi> wallyworld: I believe the vertical-align: middle is on all the others
<wallyworld> huwshimi: ah, so it is
<wallyworld> so if i add that style to the lazr activator button css it seems to help at least
<wallyworld> huwshimi: thanks for the help. so you are happy for the delete icon to be rendered where it is (to the right of the change target icon)?
<huwshimi> wallyworld: yeah I guess that makes sense
<wallyworld> i can't think of a better place. the icon will only be there for those tasks which the user has permission to delete
<wallyworld> so some will have it, others won't
<wgrant> lifeless: Do we want an abstract product|distribution table as well?
<wgrant> (in addition to the abstract bug/branch/someotherartifact table)
<wgrant> I think you might have done some analysis around this.
<lifeless> wgrant: IBugContextPillars ?
<micahg> wow, Bug #878909 threw up a red flag, wallyworld, who can do this sort of thing?
<_mup_> Bug #878909: allow users to delete bugtasks using the web UI <bugs> <disclosure> <privacy> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/878909 >
<wgrant> micahg: At present nobody. We plan to open it up to pillar owners, and possibly bug supervisors if my objections are defeated.
<wallyworld> micahg: lp admins, project owners. i'll have to double check who else
<micahg> wallyworld: wgrant: excellent assuming wgrant isn't overruled, acceptable if he is, it might be good to clarify in the bug in case anyone else sees it though :)
<lifeless> micahg: why? whats the concern ?
<wallyworld> micahg: it's also behind a feature flag
<micahg> lifeless: regular users deleting bug tasks
<lifeless> micahg: how is that worse than regular users adding them ?
 * lifeless speculates
<wallyworld> micahg: permissions are: lp admins, pillar owners, pillar bug supervisors
<micahg> lifeless: irreversable?
<wallyworld> and only if the ff is on
<lifeless> micahg: add a new task set the old status
<lifeless> micahg: thats pretty reversable
<wgrant> Hah, it seems I have been defeated :)
<wgrant> Ubuntu bug supervisors possibly shouldn't have this power, but I guess we'll see.
<lifeless> perhaps we should let anyone delete invalid tasks
<wallyworld> wgrant: that's what the ff is for - to allow us to get that bit sorted
<micahg> lifeless: it's harder to notice a missing bug task
<wgrant> wallyworld: Yep, that's why I recommended it.
<lifeless> micahg: we'll send a notification out
<wgrant> But I didn't think we were doing bug supervisors in the initial pass.
<lifeless> however, implementation wise this is real deletes, not a status-that-hides-the-bug
<wallyworld> only if the ff is on
<micahg> lifeless: indeed, but a wrong status/bug task is visible in the bug, a missing one would only be visible in the history/e-mail
<lifeless> (because this is needed to let folk cleanup private bugs before the next stage, where we disable private bugs w/multiple pillars
<wallyworld> there are a lot of invalid bug tasks that have been assigned to the null project, no?
<wallyworld> only because we couldn't delete them
<lifeless> yes, but if we had a 'deleted' state they wouldn't be there either
<lifeless> and it would be more reversible
<lifeless> safer in a lot of ways
<wallyworld> lifeless: branch deletion is also permanent
<lifeless> wallyworld: the sky is blue
<micahg> lifeless: I don't have a problem if it's limited, we can warn Ubuntu bug control members more sternly about it, but open to anyone sounds like trouble, someone disagrees with won't fix, delete and create a new bug task
<wgrant> Only the branch owner can delete branches.
<wallyworld> although in general i agree with soft deletes for model artifacts
<wallyworld> so branch owners can still make mistakes
<lifeless> micahg: its not open to anyone, and if we did open to anyone I'm pretty sure we would status lock it (e.g. delete invalids only)
<wgrant> I think hard deletes are almost always a mistake.
<wgrant> Except this time we're trying to fix a broken model design from 2004.
<lifeless> wallyworld: anyhow, its moot: we need hard deletes for *some bugs* *now*
<wallyworld> yeah, hence it was done as a hard delete and not a soft delete
<lifeless> wallyworld: we may want soft deletes once the migration is over, note that soft deletes have further dev ramifications
<wallyworld> we can revisit later
<wgrant> Once we have bugs comfortably single-target in a couple of years, we no longer need deletes of any kind.
<wallyworld> soft deletes will have a *lot* impact on model queries etc
<lifeless> wallyworld: not really
<wgrant> It's just the same as a status search.
<wgrant> Which we always do anyway.
<lifeless> wallyworld: we already have sets for interesting-default status and custom-set-of-status
<wgrant> It is a rare query that doesn't exclude Invalid.
<wallyworld> why? *every* query will need an extra "is not deleted" condition
<wgrant> status NOT IN (invalid, fixreleased, deleted)
<nigelb> Hrm, is jtv not working today?
<lifeless> wallyworld: only if you implement deleted as a separate bit
<lifeless> wallyworld: which is an option, but not a requirement
<wallyworld> which one would normally o, no?
<wallyworld> status != deleted
<wallyworld> semantically sifferent
<lifeless> wallyworld: I think many folk would argue that deleted is a lifecycle state for a bug task, rather than separate to status
<wgrant> Deleted is a special case of Invalid.
<wgrant> Isn't it?
<wallyworld> then they are wrong :-P
<wallyworld> not in my view, but anyways, it's moot for now
<wgrant> "Deleted" == "So invalid that GTFO"
<wallyworld> even if it could be argued that way for bug tasks (which I'm not agreeing with), it doesn't generalise to other model objects
<wallyworld> since  not everything else necessarily has a lifecycle status attribute
<wgrant> A "deleted" flag is just a lifecycle status attribute that happens to be a boolean.
<wallyworld> perhaps. but i view it as subtlely different. it's just imo.
<poolie> allenap, review on https://code.launchpad.net/~mbp/rabbitfixture/rabbit-startup/+merge/80174  please?
<wallyworld> existence vs state
<wallyworld> wgrant: i can't quite see how "if len(bug.affected_pillars) > 1" is wrong. you say that it prevents adding *any* task but that's not true if the bug tasks are all for the same pillar
<wgrant> wallyworld: Sure, but say I have a private security bug affecting Ubuntu and Debian, and I want to fix it in Ubuntu.
<wgrant> Unless I have the new task deletion superpower, I can't fix it in Ubuntu, because I can't target it to the series.
<wgrant> We should forbid making the bug more broken.
<wgrant> But adding another task for an existing pillar is not making it more broken.
<wgrant> Because brokenness = number of pillars - 1.
<wallyworld> wgrant: so you want to remove that check entirely i think
<wgrant> Yes.
<wallyworld> ok
<wgrant> I think of those three checks, on the third is useful.
<wgrant> s/on/only/
<wallyworld> ok. so long as you are happy with just the third, i'll get rid of the others
<wgrant> I believe the first two are excessive, and the bits that aren't excessive are redundant.
<wallyworld> ok
<wallyworld> i'll redo and you can double check
<wgrant> Thanks.
<poolie> >> A lot of the feedback was very complimentary to the infrastructure (Launchpad, Wiki, IS
<poolie> facilities etc) that Canonical provides to help Ubuntu contributors to do their work.
<poolie> from the recent developer survey
<poolie> way to go robot
<nigelb> \o/
<nigelb> Launchpad bug tracking is pretty awesome, despite all its faults.
<nigelb> And the timeout work + less email work has everyone quite happy.
<wgrant> It's getting more awesome :)
<nigelb> Indeed.
<nigelb> I'm waiting to see the end of deryck's squad's work.
<wgrant> First major improvements in years, apart from the subscription work.
<nigelb> if you guys can pull of what mrevell mentioned in his email..
<poolie> full survey http://t.co/sdyJKAnO
<poolie> >> Here and elsewhere in the survey the patch pilot programme was specifically highlighted as
<poolie> a successful initiative.
<poolie> go me.
<poolie> and all the people who did the work
<poolie> :)
<nigelb> \o/
<nigelb> The patch pilot idea was inspired by bzr right?
<StevenK> Yes
<poolie> yep, after some lobbying
<nigelb> heh
<nigelb> Its a great idea!
<nigelb> ohai StevenK :)
<StevenK> Hai
<nigelb> heh
<nigelb> "do you feel you are able to influence the decision-making of Canonical"
<nigelb> Never.
<mrevell> hi
<nigelb> Morning mrevell!
<poolie> nigelb, yeah, that is
<poolie> not surprising, perhaps a bit worrying
<nigelb> Well, I wish people in the community could be part of the teams.
<nigelb> Like, say kernel or design.
<poolie> mm
<poolie> especially design perhaps
<nigelb> yeah
<poolie> i think a lot more of the kernel work is in the open
<nigelb> Like launchpad is intresting, because I can participate in the development along with paid employees.
<nigelb> Some teams are less open.
<nigelb> Espcially design.
<nigelb> If there's a UX trained person in the community who wants to help..
<nigelb> (I wish sladen wasn't working for Canonical :P)
<poolie> me too :-P
<poolie> jk
<nigelb> heh
<nigelb> He's the kind of person I mean, he knows what he's talking about and can give design feedback.
<nigelb> Its still mertiocracy.
<nigelb> Anyway, back to work :)
<nigelb> Monday Mourning and all that.
<adeuring> good morning
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: benji | Critical bugtasks: 266
<Ursinha> wgrant, hey, still there? :) do the daskeyboard beauties have backlight?
<Ursinha> since I lost control I'm pondering buying myself one
 * nigelb blinks and processes
<nigelb> ah, right. Lost control.
<wgrant> Ursinha: No backlight, no.
<Ursinha> nigelb, :D
<Ursinha> nigelb, http://www.facebook.com/photo.php?fbid=2360791707157&set=a.1738496950177.2097724.1471235315&type=1
<Ursinha> wgrant, ah. thanks anyway :)
<nigelb> Ursinha: I remember :)
<nigelb> I commented or Like'd it before.
<Ursinha> nigelb, yeah, I just wanted to make it easy for you :)
<allenap> jelmer: Hi, I have a question about stacking. Do you have a few minutes? Someone seems to have been able to get two of their branches to stack on one another, https://answers.launchpad.net/launchpad/+question/175926.
<allenap> abentley: Thank you for pursuing that storm fix. I haven't had time yet today - on maintenance - to do anything more, but I will as soon as I can.
<abentley> allenap: Thanks!
<allenap> abentley: jelmer ^ doesn't seem to be here right now, would you have time to help me with https://answers.launchpad.net/launchpad/+question/175926?
<abentley> allenap: I'm sorry, but I haven't had anything to do with the mirroring infrastructure since the bzr team rewrote it.
<allenap> abentley: Okay, no worries. Anyone in the bzr team in particular I should talk to, or shall I just badger all of them?
<abentley> allenap: I don't know of anyone in particular.
<sinzui> bac ping
<sinzui> benji, bac, gmb, allenap: Bug #879901 need immediate fixing. ISD sees no issues on their side
<_mup_> Bug #879901: Purchased commercial vouchers do not arrive at Launchpad <projects> <regression> <salesforce> <Canonical Salesforce:New> <Launchpad itself:Triaged> < https://launchpad.net/bugs/879901 >
<benji> sinzui: we will look at it now
<sinzui> benji, ISD sent me a list that confirm they have a record of my purchases on the 11, 12, 22 of this month.
<benji> sinzui: ok, so the purchases are correctly recorded in salesforce therefore we should start by looking at the lp<->salesforce interconnect
<sinzui> benji, exactly. The gateway port could be blocked or the query/protocol changed
<sinzui> maybe Lp is not polling salesforce anymore
<benji> sinzui: thanks for the verification, I'm reading voucher code to see where to get started
<sinzui> benji, bac diagnosed the last two outages. I think He had a losa switch a process to debug to log the underlying issue
<benji> sinzui: good to know, I'll see if he can give me a hand
<benji> bac: are you really not in #launchpad-yellow or is my IRC client acting up?
<sinzui> maybe bac is still travelling home
<cjwatson> Thank goodness Launchpad isn't like this: http://thedailywtf.com/Articles/The-Query-of-Despair.aspx
<sinzui> benji, Bug #557392 and Bug #597754 might be informative
<_mup_> Bug #597754: Launchpad voucher redemption fails with unactivated vouchers <lp-registry> <oops> <qa-ok> <Canonical Salesforce:Fix Released by jamesj> <Launchpad itself:Fix Released by bac> < https://launchpad.net/bugs/597754 >
<benji> thanks sinzui, taking a look
<sinzui> benji, James confirmed my vouchers are active
<benji> sinzui: that would seem to rule out a reoccurrance of bug 597754 then
<_mup_> Bug #597754: Launchpad voucher redemption fails with unactivated vouchers <lp-registry> <oops> <qa-ok> <Canonical Salesforce:Fix Released by jamesj> <Launchpad itself:Fix Released by bac> < https://launchpad.net/bugs/597754 >
<sinzui> I agree. the other bug looks promising
<jelmer> hi allenap
<allenap> jelmer: I've just spoken to mgz in #bzr, thanks for getting back to me though.
<jelmer> allenap: ah, cool
<benji> sinzui: do you have any hints as to where the proxy's logs are?  I gather from 557392 that the proxy runs on niobium, but I don't see that machine's logs in /srv/launchpad.net-logs/production/ on chinstrap
<sinzui> hmm
<nigelb> heh
<nigelb> http://thedailywtf.com/Articles/The-Query-of-Despair.aspx
<allenap> Are there any Soyuz Gods in here who can help with https://answers.launchpad.net/launchpad/+question/175403?
<nigelb> allenap: I thought all those gods were in australia? :)
<allenap> nigelb: I think you're right. I'm hoping that one of them has insomnia and an urge to check IRC.
<nigelb> :)
<nigelb> That does happen way too often.
<cjwatson> allenap: It's really a question about the platform, not a Soyuz question.  I've answered.
<allenap> cjwatson: Thank you :)
<cjwatson> (Although I suppose "no, we won't manually set up symlinks in /usr/bin for you" involves a degree of Soyuz knowledge, but. ;-) )
<mrevell> Night all
<nigelb> Ursinha: You might find this quote interesting "Chuck Norris's keyboard doesn't have a Ctrl key because nothing controls Chuck Norris." :)
<Ursinha> nigelb, ha!
<Ursinha> :P
<nigelb> :D
<mtaylor> sinzui: I bugged you the last time someone merged their account and wound up with two open ids, didn't I?
<mtaylor> well, in any case, we just had someone else merge accounts and wind up with SSO not working properly for him: https://answers.launchpad.net/launchpad/+question/176031
<sinzui> mtaylor, okay
<sinzui> I think we need to ensure a bug is reported and a maintenance is is working to fox the root cause.
<sinzui> I will do that no
<sinzui> now
<mtaylor> sinzui: cool. thanks!
<mtaylor> sinzui: I thought you might have had a thought on the root cause there
<mtaylor> :)
<lifeless> morning
<lifeless> benji: can has review?
<lifeless> https://code.launchpad.net/~lifeless/python-oops-tools/bug-879309/+merge/80175
<benji> lifeless: since you ask so nicely, sure
<lifeless> benji: :) thanks!
<lifeless> sinzui: on merge do we intend to keep all the identities active indefinitely ?
<lifeless> sinzui: what do we do with the alias that we used to offer ourselves (https://launchpad.net/~lifeless used to (still does?) work as an openid thingy)
<lifeless> sinzui: I ask because there is another bug elsewhere talking about persistent identifiers for users, and merge will affect that too
<benji> lifeless: I'm done with the review.  Only one small bikeshed.
<lifeless> :) thanks
<sinzui> lifeless, yes, we keep the ids forever to ensure no matter which identity SSO uses continues to work
<sinzui> lifeless, i think is is largely because we do not have a mechanism to sync with Ubuntu's SSO
<lifeless> sinzui: yah, thats in fact the other bug I'm thinking of :)
<lifeless> sinzui: well, kinda
<lifeless> sinzui: anyhow, can you check my logic here (not related to your bug):
<lifeless>  - we allow multiple openids to login to a single user
<lifeless>  - merging just unions them (in theory)
<lifeless>  - openids can not be deleted by the user?
<sinzui> lifeless, yes, These are issue I brought up in my email 14 months ago to the list. The foundations team was working in an adhoc fashion to solve the defects in the SSO split
<lifeless>  -> openids could be used as a static persistent identifier in the SOA?
<lifeless> step 3 I'm not sure of
<sinzui> stub was working with a lot of distractions and was asking for help to determine which bugs really needed to be addresses
<sinzui> addressed
<flacoste> benji: do you know why wgrant put bug #828572 back in progress?
<_mup_> Bug #828572: bugs are marked incomplete_with_response if users or scripts change the status / tags immediately after setting the status <escalated> <qa-ok> <ubuntu-qa> <Launchpad itself:In Progress by benji> < https://launchpad.net/bugs/828572 >
<benji> flacoste: I'm wondering the same thing; I figured that if I didn't see him by the time I EODed I'd email him to find out
<benji> he should be around in an hour or so
<lifeless> have we finished the migration and deleted the old query code entirely?
<benji> lifeless: yep, the migration is done (there were 5 unmigrated records last week that have now been fixed) and my branch switches to query on the new with/without response flavors of incomplete
<lifeless> benji: so your branch can close that bug :)
<benji> lifeless: I think so.  I'd like to know why wgrant reopened it though.  I don't want to get in a game of status tennis.
<lifeless> of course :)
<lifeless> wgrant: ^ :)
<flacoste> lifeless: https://bugs.launchpad.net/launchpad/+bug/314432
<_mup_> Bug #314432: It's impossible to see all the bugs that affect a BugTarget if some bugs are targeted to one or more series and the Master task is closed <api> <dhrb> <lp-bugs> <platform-blocker> <rls-mgr-p-tracking> <ubuntu-qa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/314432 >
<flacoste> lifeless: https://bugs.launchpad.net/launchpad/+bug/857109
<_mup_> Bug #857109: Cannot have a bug targetted only to an old series <Launchpad itself:Triaged> < https://launchpad.net/bugs/857109 >
<benji> wgrant: hi, you around?
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: - | Critical bugtasks: 266
<lifeless> sinzui: hey, if you have some time, I'd love to catch up
<sinzui> lifeless, I may have time after my standup that always goes longer than 15 minutes
<wgrant> benji: I didn't realise it was actually finished. I thought just the initial schema work had been done.
<benji> wgrant: ok, cool; I'll update the bug (and add a comment so the stakeholders can verify that the behavior is what they want)
<lifeless> sinzui: that would be cool
<wgrant> lifeless: lol
<wgrant> That's data corruption?
<wgrant> You'd better not look at Soyuz...
<lifeless> wgrant: binaries with no source records (and not deliberately mangled) yes
<lifeless> wgrant: look on the bright side, we have to start somewhere
<sinzui> StevenK, bug 784596 is now in scope
<_mup_> Bug #784596: UI implies open/delegated teams can have PPAs <confusing-ui> <disclosure> <ppa> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/784596 >
<StevenK> sinzui: Just thinking about it, I don't think open teams have the create PPA link any more
<sinzui> That would be excellect
 * wallyworld stabs unity and all the duplicate icons that have just appeared on the launcher
<StevenK> sinzui: I will investigate, in either case.
<sinzui> StevenK, lp.registry.interfaces.person.team_subscription_policy_can_transition does all sanity checking. It is used by the field to guarantee sanity. It must learn about subscriptions to private artefacts
<lifeless> wgrant: you asked yesterday about a new table
<lifeless> wgrant: i think we failed to discuss
<wgrant> lifeless: We did. I must have become distracted with something.
<wgrant> Possibly the factory being pathetically slow.
<lifeless> when you're off your call; and I've had a brief catchup with curtis, we can talk
<wgrant> The call is done. sinzui is setting dinner in motion, then talking to you.
<lifeless> ok
<lifeless> so until he arrives ..
<lifeless> you were asking about a denormed table the contents of which would be something like (product|distro) which I interpreted as ICanHasBugTaskAndArePillars
<wgrant> Right.
<lifeless> flacoste and I were talking about seriesonlybugtasks
<wgrant> I currently have an abstract artifact table, but not one for pillars.
<lifeless> it may get scheduled
<lifeless> via maintenance
<wgrant> Eww.
<lifeless> one of the impacts is that the fields in bugtask would be reduced - drop product and distro
<wgrant> I think it should be done as part of the larger IssueTracker rework.
<wgrant> At the same time as single-targeting bugs.
<lifeless> wgrant: single targeting bugs isn't [currently] a desired end-state
<wgrant> (one Product/Distribution/DistributionSourcePackage + its series)
<wgrant> I think it is.
<wgrant> You just don't know it yet.
<lifeless> wgrant: I know you hold a different opinion
<lifeless> wgrant: I think there are significant issues clustered around that, around work items, and around conversations.
<lifeless> I'm not willing to say you are wrong... but I'm also not willing to say you are right.
<wgrant> And I think pushing for series-only tasks before while we have NFI what we're doing around tasks is a big mistake.
<lifeless> well, the idea is to have product do some user research
<lifeless> validate or invalidate the idea as an incremental improvement
<wgrant> As long as they take the user input with roughly 10 megatonnes of salt, sure.
<lifeless> mmm, so for a quick update this has turned into a lot of hyperbole
<lifeless> Lets talk later
<sinzui> Hi lifeless. I am on skype
<lifeless> hi
<lifeless> wgrant: ok, want to talk about this now?
#launchpad-dev 2011-10-25
<wgrant> lifeless: Sure.
<lifeless> skype?
<wgrant> sec
<lifeless> wgrant: http://www.salmon-protocol.org/
<wgrant> OOPS reports MIA?
<wgrant> LOL
<wgrant> Date: Tue, 25 Oct 2011 02:26:19 +0000 (UTC)
<lifeless> ask and ye shall receive
<wgrant> Half an hour late :/
<wgrant> wallyworld: Ahh, I think I see now why canAdd*Task ignored the current privacy status.
<wgrant> So that AJAX privacy changes show/hide them?
<wallyworld> wgrant: yes
<wallyworld> but it works either way
<wgrant> Huh, does it?
<wgrant> If the bug is public and I make it private by AJAX, the links will not disappear.
<wallyworld> hmmm. maybe not, i've confused myself
<lifeless> wallyworld: quick question on bugtask deletion
<wallyworld> no, i think it works. if it is private, a css class gets added to the links
<lifeless> wallyworld: do you guard against deleting the last non-series bug (in a given context) ?
<wallyworld> wgrant:  and the css rule then hides those if body.private is true
<wgrant> A comment explaining this would be handy.
<wgrant> That the hiding is done through CSS based on body.private.
<wallyworld> lifeless: perhaps not. i guard against deleting the last task
<wgrant> wallyworld: And if it's not private, then the link class doesn't get added.
<wgrant> wallyworld: So if I make it private later, the links won't hide until I refresh the page.
<wallyworld> wgrant: ok, i'll add a comment somewhere, perhaps in the tales
<lifeless> wallyworld: hopefully we'll do seriesonlytasks (or decide we won't do that at all), but I think we should, until then, honour the data model
<wgrant> Before you changed it after my suggestion, this worked.
<wgrant> lifeless: Which data model?
<lifeless> wallyworld: you cannot create a bug with only series tasks today
<wallyworld> wgrant: ah, that's why i left out the public check
<lifeless> wgrant: ^
<wgrant> lifeless: That's true, but only because our API is terrible.
<wallyworld> wgrant: i'll revert that change
<lifeless> wgrant: no, and we went round this a fortnight back
<wgrant> wallyworld: Revert the change, but add comments :)
<wallyworld> wgrant: yes, sure. i knew how it worked but got myself confused
<wgrant> lifeless: It's true that you can't obtain a bug with only series tasks, but you can create orphaned series tasks.
<lifeless> wgrant: retargeting a non-conjoined non-series task ?
<wgrant> lifeless: Right.
<wgrant> Or through legacy.
<wgrant> eg. bug #43150
<_mup_> Bug #43150: [SRU] maxima frontends fail to connect <wxmaxima (Ubuntu):Fix Released> <gcl (Ubuntu Dapper):Fix Released by wgrant> <maxima (Ubuntu Dapper):Fix Released> < https://launchpad.net/bugs/43150 >
<lifeless> ouch
<lifeless> so, I'm merely saying that the current UI and docs and queries assume this isn't the case
<wgrant> We discussed this some months ago, remember? :)
<lifeless> we've fixed it a bit
<wgrant> And we agreed that it was reasonable to allow.
<lifeless> I recall agreeing that it exists and we need to handle it gracefully
<lifeless> it seems to go against the intent of the conjoined work to encourage it
<wallyworld> lifeless: wgrant: so, do i need to make a change to the deletion check? or leave it as is? ie i simply disallow deleting the last task
<wgrant> lifeless: Which conjoined work?
<lifeless> wgrant: the original :)
<wgrant> Disregard that.
<lifeless> wallyworld: I'm torn, on the one hand there is an escalated bug which would be solved by having non-series-less bugs
<lifeless> (SRU's)
<lifeless> wallyworld: on the other hand this will make those bugs unique and inconsistent with the vast majority of the rest of the system
<wgrant> That's true.
<wallyworld> i'm all for consistency :-)
<wallyworld> unless there's a really good reason
<wgrant> We're sitting just before a number of major model reworks.
<lifeless> wallyworld: tightening the check up a little would aid consistency in the short term. Like I say, I'm torn.
<lifeless> I don't *know* that we'll get fallout from having the check looser than the rest of the UI/model behaviour
<lifeless> I don't *know* that its safe either given how few examples of series-less bugs we have
<wallyworld> lifeless: so if i create a bug with target=launchpad project, isn't that a series-less bug?  product series = trunk may be implied but the BugTask  object simply references the project, no?
<lifeless> wallyworld: yes, I said non-series-less bugs
<lifeless> wallyworld: its not a double negative!
<lifeless> make a bug on launchpad
<lifeless> nominate to a series other than trunk
<lifeless> delete the non-series task leaving only the newly nominated task
<wallyworld> ah, ok. i did discuss this with wgrant and he should me how this is now handled in the ui (a recent change) so it was thought that it is ok to procede
<wallyworld> s/should/showed
<StevenK> lifeless: You made a comment on an MP of mine yesterday -- I couldn't find anything in my grep'ing, and Curtis couldn't remember anything like that -- happy to be pointed at something.
<lifeless> StevenK: which MP ?
<lifeless> could someone run bin/test -vvt BranchChangedErrorHandling
<StevenK> lifeless: https://code.launchpad.net/~stevenk/launchpad/extend-subscription-check/+merge/80173
<lifeless> tell me if it has any test stiplle (unusual output)
<lifeless> StevenK: ah no, I was noting the abstraction leaks in passing
<lifeless> StevenK: if there isn't something, there isn't (but perhaps you should create one)
<StevenK> lifeless: http://pastebin.ubuntu.com/718460/
<lifeless> ah good, I haven't messed up the branch
<lifeless> thanks
<lifeless> OTOH test noise fail
<StevenK> Agreed
<lifeless> grr!
<lifeless> tests fail in ec2
<lifeless> don't fail here.
 * lifeless hates 
<mwhudson> isolation fail?
<lifeless> mwhudson: presumably
<lifeless> mwhudson: its my use-oops-twisted branch FWIW
<mwhudson> bet it's a thread local :-P
<lifeless> mwhudson: if by that you mean a utility....
<lifeless> mwhudson: then I won't take the bet :)
<mwhudson> heh
<StevenK> wgrant: Mr OCR, do you feel like reviewing https://code.launchpad.net/~stevenk/launchpad/extend-subscription-check/+merge/80173 ?
<lifeless> \o/
<lifeless>     1173  OOPS-2123ED1193  BugTask:+nominate
<wallyworld> wgrant: i've updated the comments in that branch
<lifeless> mwhudson: fortunately the culprit is in the previous 100 or so
<lifeless> mwhudson: \o/ subunit and \o/ --load-list
<lifeless> mwhudson: testworkermonitorrunnoprocess.test_failure
<mwhudson> i guess the oops system doesn't even store state in thread locals, but on disk instead
 * mwhudson goes to collect the car after its wof
<lifeless> mwhudson: IErrorReportingUtility
<lifeless> mwhudson: utility & global, for the combined win of brain-fuckage
<mwhudson> oh yes
<lifeless> which isn't isolated by the default machinery
<lifeless> so reconfiguring that utility -> boom
<lifeless> was a foot gun by me in the branch, but still. FUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
<StevenK> How do I get the HTML out of a view?
<StevenK> view = create_initialized_view() ; html = view() ; I thought?
<lifeless> StevenK: test_request_daily_build_oops
<StevenK> What about it?
<lifeless> StevenK: would you happen to know why it generates two oopses, not one, when it only creates one test recipe ?
<StevenK> Nope
<lifeless> ah well, thanks
<StevenK> I didn't touch that bit of r-d-b
<lifeless> (another reason why getLastOops causes headaches)
<lifeless> grr
<lifeless> Launchpad encountered an internal error during the following operation: generating an incremental diff for a merge proposal.  It was logged with id OOPS-2124MPJ1.  Sorry for the inconvenience.
<lifeless> really must track that down. its getting tiresome
<StevenK> lifeless: *Please*
<lifeless> hah
<lifeless> it was logging at exception level
<lifeless> and manually raising
<lifeless> *fixes*
<poolie> lifeless: non-urgent review some time on https://code.launchpad.net/~mbp/testscenarios/module-scenarios/+merge/80287
<lifeless> hmm
<lifeless> I don't think job retries should generate oopses
<lifeless> still, ESCOPE
<StevenK> view() is returning "" :-(
<wgrant> StevenK: It probably relies on being executed inside a METAL template.
<StevenK> view = create_initialized_view(team, '+index')
<wgrant> Right, Person:+index can't be used that way.
<lifeless> ugh, and job running also generates 2 oopses per error
 * lifeless files an XXX
<StevenK> Just remove the job system
<StevenK> Please
<StevenK> wgrant: Thank you for the approve. If you want to help me collapse those two queries into one, that sounds excellent.
<wgrant> StevenK: Find any bugs that are private and there is a subscription or an assigned task.
<lifeless> beware over-combining
<lifeless> e.g. test the performance of separate vs together
<StevenK> wgrant: If I can't use Person:+index in create_initialized_view(), what should I be doing?
<poolie> is bug 871596 really still open? is there any workaround?
<_mup_> Bug #871596: Not handling administrative shutdown under Oneiric <build-infrastructure> <librarian> <Launchpad itself:Invalid by allenap> <Storm:In Progress by allenap> < https://launchpad.net/bugs/871596 >
<huwshimi> Do we have any code for generating thumbnails in Launchpad?
<lifeless> huwshimi: probably not, depending on what you are meaning
<huwshimi> lifeless: I'm investigating whether we can show small versions of images that are uploaded to Launchpad (e.g. for images attached to a bug). For that we need some code that resizes and probably caches the image.
<huwshimi> lifeless: I'm guessing such code probably doesn't exist
<poolie> !!
<poolie> that'd be nice
<lifeless> huwshimi: its been written many times :)
<lifeless> huwshimi: this is a perfect candidate for a separate service, it should be a fairly small amount of work to do it
<lifeless> huwshimi: librarian for storage, separate link in the bugattachment to the thumbnail, backend to process and upload
<lifeless> I don't think we're at the volume to need e.g. haystack yet
<lifeless> brb
<huwshimi> lifeless: (for when you get back) how long might this take (roughly)? A couple of days, a week?
<poolie> :) i like your estimation scale
<huwshimi> poolie: If it's outside that scale, it's not ever happening :)
<huwshimi> poolie: actually if it's outside of a couple of days it's probably never happening
<poolie> :)
<poolie> that was my concern about markdown :)
<poolie> i suspect you could write it in under a week but it would take a while to get deployed
<huwshimi> poolie: Wow, you think there's a week's worth of work for the first stage of markdown?
<poolie> i think for it to succeed it would have to be tightly scoped so that you could get something done in under a week
<poolie> have there been any LEP/feature type things ever done in less than a week?
<huwshimi> poolie: That's pretty depressing really
<lifeless> huwshimi: week(s)
<huwshimi> lifeless: Ok, sure
<huwshimi> lifeless: That also is very depressing
<lifeless> huwshimi: you need two schema changes (add field, index it).
<poolie> lifeless: re bug 878140, does logging a warning inherently generate an oops?
<_mup_> Bug #878140: process-mail.py failed to resolve dns. Raised NXDOMAIN <dkim> <oops> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/878140 >
<lifeless> huwshimi: then you need a job to walk things that need doing them and do it
<poolie> could https://lp-oops.canonical.com/oops.py/?oopsid=2116INBOUNDEMAIL2 have come from that?
<lifeless> huwshimi: lastly you need the actual image resizing code itself (and I would make it a microservice for reusability)
<huwshimi> lifeless: Ah, so writing the code wouldn't take a week or more, just the whole process?
<lifeless> huwshimi: yes
<poolie> huwshimi: another option is to just serve the whole image and rely on the browser to resize them
<poolie> for version 0
<huwshimi> lifeless: But maybe close to a week for the code?
<lifeless> poolie: see OopsHandler in the codebase. (short story -yes, warnings -> things we need to see -> OOPS)
<poolie> i see, and it logs the last exception?
<lifeless> yes
<huwshimi> poolie: We could, but that would mean loading some rather large images at times
<poolie> yep
<lifeless> huwshimi: the code duration could be a matter of hours, depending on soft things like 'minimum time to get the image resized' and 'do we need parallelisation'
<lifeless> huwshimi: there is lots of latency in the landing story today: 4 hours for the column through ec2, 4 hours for it through buildbot, then wait for a fastdowntime window.
<lifeless> huwshimi: then for the index on the column, 4 hours for ec2, 4 hours for buildbot, then apply live
<lifeless> huwshimi: then for the background job to grab and resize, 4 hours for ec2, 4 hours for buildbot, then wait for a nodowntimedeploy
<huwshimi> lifeless: I'm really trying to judge whether this is something that I want to do myself. If it was something that I could code in about 2 days that would be an ok use of my time, if it was much longer, probably not
<poolie> lifeless: ok so like https://code.launchpad.net/~mbp/launchpad/878140-dkim-nxdomain/+merge/80289 ?
<lifeless> huwshimi: and for the image resizing aspect, depending on whether the dev inlines it (simple but grows the LP codebase) or makes a service (more initial overhead, easier management and tuning thereafter) the same ec2 etc, or the delay with LOSAs to get a new service deployed.
<huwshimi> lifeless: So there's no window for actually getting it deployed
<poolie> huwshimi: i think it depends a bit how much you want to learn how to write microservices in lp
<poolie> etc
<lifeless> huwshimi: like most things in LP, its going to touch the whole vertical stack: db, deployment, middleware, object model, security rules, interfaces, and thats ignoring whatever template/js changes you want to support the UI
<huwshimi> poolie: Well I would want to do it in a useful way, I also don't want to over optimise
<lifeless> huwshimi: I would say, if you're fairly familiar with that vertical stack, then yes, the actual coding aspect could be <= 2 days. If you're not, you'll spend considerable time fumbling and learning (which is perhaps good in its own right)
<huwshimi> lifeless: Right, so I'm interested in getting to delve into the code base a bit more, I'm not sure building a whole service would be a good thing to start with
<lifeless> huwshimi: yes, its hard to find truely small-but-involves-learning things in LP
<lifeless> :(
<huwshimi> lifeless: haha, yes
<poolie> maybe we can quasi-pair on md?
<poolie> lifeless: thanks for the review
<poolie> would it be easy to add a test saying "this doesn't oops/warn"?
<lifeless> poolie: sure, you need to use one of the logging test helpers, simulate the situation, and check that nothing was logged at or above WARN
<lifeless> of course, this is vulnerable to skew, as we're assuming that OOPSHandler won't be given a level= parameter when its added in script setup
<lifeless> personally, I would hesitate to test this
<huwshimi> poolie: I'm keen to help out if there's a way we can do that
<poolie> are you going to uds (i'm not)
<lifeless> any test here is vulnerable to two forms of skew I think - the oopshandler config, and the actual exception raised.
<poolie> perhaps we could spike it for a day or something , maybe next week
<poolie> right
<lifeless> so you'd need to make sure you test deeply enough to insulate from that
<poolie> i found it confusing the oops is unclear about whether the exception traceback indicates the program stopped at that point
<poolie> to me it certainly does look like it stopped, but apparently it did not
<lifeless> a bug explaining the confusion would be good
<lifeless> I think its right to show the tb it did, but not to confuse you
<poolie> a bug against which project?
<huwshimi> poolie: No I'm not, I'd be keen for that
<lifeless> poolie: to start with LP, which is where OopsHandler is
<lifeless> poolie: and/or python-oops-tools which does the rendering
<lifeless> poolie: the emit() method on OopsHandler could usefully include the logging level in the message, and whether there was an exception attached
<lifeless> => cooking
<poolie> lifeless: ok https://bugs.launchpad.net/launchpad/+bug/881243
<_mup_> Bug #881243: oops display doesn't make it clear whether the exception stopped the process <Launchpad itself:Triaged> < https://launchpad.net/bugs/881243 >
<nigelb> Morning.
<nigelb> stub: You'll find this entertaining :)
<nigelb> http://thedailywtf.com/Articles/The-Query-of-Despair.aspx
<stub> nigelb: I just hope an ORM was involved or my remaining faith in humanity will dwindle.
<nigelb> "legacy" application.
<nigelb> I doubt if an ORM was involved.
<jtv> Who knows about LP's mailing lists?  Need help with this question: https://answers.launchpad.net/launchpad/+question/175718
<lifeless> wgrant: 2704 lines
<wgrant> lifeless: :(
<lifeless> jtv: the mailing list archiver gets rather behind
<jtv> lifeless: ?
<lifeless> jtv: I'm not sure if we've got a bug for that yet; moving it to a backend JSON server is the planned approach to fix it
<jtv> Ah, the question
<lifeless> yes :)
<jtv> Thanks for looking into it.  âRather behindâ doesn't really cover it in this case though.
<lifeless> jtv: days isn't uncommon. Its been weeks on occasion
<jtv> IIRC the last archived message there was a year old.
<lifeless> jtv: ah! that may need checking by a losa then on the list in question - check that mailman hasn't blerched itself
<lifeless> jtv: There are probably logs you can poke at on carob to start with
<jtv> Hmm
<lifeless> s/probably//
<jtv> I'll try that, thanks.
<lifeless> the key thing to know is that mailman handles the mail and forwards it etc
<lifeless> the list archive is really just another recipient - it queues it up and then rewrites the index page to link it in when it gets to it
<jtv> Any chance that it might even be a spam filter?
<lifeless> wgrant: it could be worse
<lifeless> wgrant: its mostly mechanical
<lifeless> jtv: I don't believe so; I would expect no-delivery to list members in that case
<wgrant> It's only known to be 4 days out of date.
<wgrant> I haven't seen any suggestion that an email was sent to the list between the last one in the archive and October 21st.
<wgrant> That's very much within mhonarc-is-slow-and-ubuntu-x-swat-is-huge-so-nevermind territory.
<wgrant> Indeed, October 21st is the first email since then.
<wgrant> http://www.mail-archive.com/nunit-dev@lists.launchpad.net/
<jtv> I wonder why the process-mail.log for the 21st seems to be binary garbage.
<wgrant> That sounds irrelevant.
<wgrant> mailman doesn't have anything named process-mail.log, AFAIK.
<jtv> What are the relevant logs?
<mrevell> Hallo
<wgrant> All the mailman logs on forster.
<adeuring> good morning
<jtv> hi mrevell, hi adeuring
<adeuring> hi jtv
<jtv> wgrant: hasn't forster retired?
<StevenK> What gave you that idea?
<jtv> I thought that was the server that loganberry replaced.
<wgrant> It was.
<wgrant> It is no longer the scripts server.
<wgrant> It continues to be the mailman server, however.
<jtv> So I'm looking for scripts/forster/mailman-xmlrpc/
<wgrant> No.
<wgrant> Well, probably not the xmlrpc bit.
<wgrant> xmlrpc is irrelevant here.
<jtv> Okay, what other mailman logs are there?
<wgrant> The logs from mailman, which are probably somewhere.
<wgrant> mailman/forster
<jtv> Ah, thanks.
<jtv> Some timeouts there.
<lifeless> wgrant: also, no more getLastOops
<lifeless> wgrant: more than makes up for size IMO :)
<nigelb> poolie: Hey, you guys had a circuit breaker thing for the "should make tea" bug right?
<nigelb> poolie: Can you link me to that bug if you have it handy?
<poolie> bug 795321
<_mup_> Bug #795321: udd importer should make tea while launchpad is down <fastdowntime-later> <Launchpad itself:Invalid> <Ubuntu Distributed Development:Fix Released by vila> < https://launchpad.net/bugs/795321 >
<nigelb> poolie: Thanks!
<nigelb> I ended up needed something like this for work :D
<poolie> are you going to use it somewhere else?
<poolie> oh ok
<nigelb> Implementing it in bash is going to be loads of fun.
<lifeless> \o/
<poolie> mrevell, danhg, https://code.launchpad.net/~mbp/launchpad/jobs-link/+merge/80307
<poolie> lifeless: ?
<lifeless> there is a config setting to put stuff in the footer
<lifeless> I'd just set that directly on prod
<wgrant> Can't do links, though.
<wgrant> I don't think.
<lifeless> wgrant: I thought it was html it took
<wgrant> I hope not.
<wgrant> Anything that I see that takes HTML gets added to my hitlist.
<poolie> are you referring to site_message?
<poolie> i thought that was meant for "end of the world imminent"
<poolie> s//eschaton
<poolie> that would be too heavy for this, i think
<lifeless> all hail our future light cone
<wgrant> site_message is used on staging/qastaging/dogfood
<wgrant> To say that changes will be lost.
<wgrant> It used to be used on edge to say it was running pre-release code.
<wgrant> It was originally along the top of the page, but was demoted to the bottom for 3.0
<lifeless> its tasteful but may not support a link
<lifeless> anyhow, I have no strong objection, it just seems like something better done in config than in code
<poolie> iswym
<poolie> otoh there's a fuzzy line between code and config
<poolie> lifeless: i was actually wondering about your \hooray emoticon
<poolie> was that for me?
<danhg> Hi poolie: what am I looking at on that link?
<poolie> ah welcome :)
<mrevell> danhg, We can talk about that on the phone now. It's your first merge proposal.
<poolie> you can do your first code review
<lifeless> poolie: implementing circuit breaker in bash.
<lifeless> poolie: it was a little ironic
<poolie> danhg: mostly if you talk it over with mr that's be good
<poolie> don't be too rough on me :)
<poolie> night all
<danhg> ok
<danhg> goodnight
<lifeless> zomg use-oops-twisted landed.
 * lifeless dances the happy dance
<bigjools> wgrant: thanks for QAing 747558 for me - only just saw my bug mail.
<lifeless> wgrant: btw, don't know if you saw but collapsing those private-asset queries into one using joins took it up to 5seconds hot (vs 4ms hot using a union all)
<wgrant> lifeless: The query was wrong. Not sure if that would make much of a difference, though.
<lifeless> wgrant: yah, table scans R us
<wgrant> bigjools: That was the ddeb thing?
<wgrant> Bug #747558
<_mup_> Bug #747558: PPAs should create backtracable packages <escalated> <ppa> <qa-ok> <Launchpad itself:In Progress by julian-edwards> < https://launchpad.net/bugs/747558 >
<wgrant> Right.
<bigjools> wgrant: did you test just the upload changes I made?
<bigjools> I need to add the copying safeguard and then it's probably ok to release
<bigjools> but I want to see it working on DF myself
<wgrant> bigjools: Yeah, I just built a package with some ddebs and checked that the overrides matched.
<bigjools> cool
<bigjools> where is it~?
<wgrant> Didn't try superseding, because I didn't want to sneak concordia away for too long.
<wgrant> https://dogfood.launchpad.net/~wgrant/+archive/dogfood/+packages <- aalib there
<bigjools> concordia is still on DF anyway
<wgrant> Needs process-accepted run, but I checked the BPR overrides in the DB.
<bigjools> running it
<nigelb> bigjools: lolcricket :D
 * nigelb ducks
<bigjools> (_._)
<nigelb> hrm, what's that stand for?
<nigelb> Or should I not be asking :D
<bigjools> I was mooning you :)
<nigelb> ha
<bigjools> nigelb: still laughing?
<bac> hi mrevell
<mrevell> hey bac
<thumper> morning
<deryck> Morning, all.
<allenap> Morning deryck.
<bac> yo mrevell
<abentley> allenap: how's that storm bug coming?
<allenap> abentley: I've resolved most of the issues; only one very stubborn one remains.
<abentley> allenap: oh dear.  Anything I can help with?
<allenap> abentley: I don't think so... but I'll ping if there is something. Thanks. As of about 10 seconds ago I think we might need to move to a newer psycopg2.
<abentley> allenap: Okay.  Good luck with it.
<allenap> And now I'm not so sure!
<abentley> allenap: I hate that.
<thumper> flacoste: ping
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: gmb | Critical bugtasks: 266
<danilos> flacoste, hi, I am Danilo from the Linaro team :)
<danilos> flacoste, asac mentions how there is a very important regression that hits us in https://bugs.launchpad.net/launchpad/+bug/881144, which is time critical; do you think it'd be possible to get someone to look at it today/tomorrow to at least get any idea if we can get some results by Thursday?
<_mup_> Bug #881144: field.tags_combinator=ALL gives same results as with ANY <bugs> <regression> <search> <Launchpad itself:Triaged> <Linaro Android:Triaged> <Linaro-Ubuntu:Triaged> < https://launchpad.net/bugs/881144 >
<danilos> flacoste, if not, perhaps we need to focus on a work-around on our side (we can load bug-by-bug through API and filter stuff out that way)
<nigelb> drat. no bigjools.
<nigelb> I wanted to poke fun at england's cricket team :D
 * sinzui sends all spam to himself before sending to the user so that he shares their pain
 * sinzui plants face in palm
<nigelb> sinzui: If you shoot your self in the foot...and then complain... :P
<sinzui> nigelb, My test went fine
<sinzui> But I left my name in one of the headers in the real send.
<nigelb> :)
<thumper> sinzui: hi
<sinzui> Hi thumper
<thumper> sinzui: how do I get project associations on the stakeholder list?
<thumper> sinzui: distro really want it now I told them what it is
<sinzui> thumper, you need to find stakeholder reps to bring it to the list.
<lifeless> thumper: Sounds like a handwave rather than a LEP :) [I mean, theres a lot of things it -might- mean]
<lifeless> thumper: its going to be feature level work I think, which means that the various stakeholders have to all agree its position on the list for it to get scheduled [more or less]
<thumper> I've been told there hasn't been a stakeholders meeting of 6 months
<lifeless> the queue already has items in it, and noone has proposed removing them
<lifeless> the stakeholder process is about prioritisation, not about getting things done more quickly
 * thumper nods
 * thumper goes back to the drawing board
<lifeless> [theory being that with consensus about what to do next, we can avoid chopping and changing mid-project, letting us deliver the projects more smoothly and with less defects]
<sinzui> thumper, I can brief people were we were in 2009. We had a proposal that Mark asked for UI changes to
<deryck> abentley_, hi.
<abentley_> deryck: hi.
<deryck> abentley_,  I see you got mail for the feature branch passing, but not clear on my end if you got the upgrade branch mail also.
<abentley_> deryck: No, I didn't get that one.
<deryck> abentley_, ok, I'll forward to you then.
<abentley_> deryck: did you know that {foo: 'bar', baz: 'qux'} is different from {baz: 'qux', foo: 'bar'} in Javascript?  Objects used as mappings apparently preserve ordering.
 * lifeless headdesks
<deryck> abentley, are you sure it's the ordering that matters, and not just that js sees two different objects?  i.e. how are you testing this?
<abentley> deryck: I'm producing query strings from a mapping, and the order of the items in the query string matches the order the items are added to the mapping.
<abentley> deryck: I think I saw this earlier with JSON stringification, too.
<deryck> abentley, so I'm having a little trouble following the specific example here of how ordering matters, sorry.  But....
<deryck> abentley, I did know directly comparing the same objects in js won't work.  it sees different objects.
<deryck> abentley, but this is to do with how js creates/sees objects.  and you usually just compare properties and values, not object to object.
<abentley> deryck: {foo: 'bar', baz: 'qux'} => "foo=bar&baz=qux",  {baz: 'qux', foo: 'bar'} => "baz=qux&foo=bar"
<abentley> deryck: So even when you serialize it, you get different values for equivalent mappings.
<deryck> abentley, you would expect {baz: 'qux', foo: 'bar'} to come out "foo=bar&baz=qux" ?
<abentley> deryck: I would like a consistent order.  I don't have a particular order in mind.
<deryck> abentley, is this something in the language causing the issue, though?  Or in how you're doing the serialization?
<abentley> deryck: Something in the language is causing the issue, because if mappings were unordered, as in Python, it would be impossible for a serialization to preserve ordering.
<deryck> abentley, are you doing the serialization yourself or relying on something in yui3?
<lifeless> abentley: have you seen ordereddict in python 3 ?
<abentley> deryck: I don't understand why that's relevant, but I'm using something in YUI3,
<deryck> abentley, I guess it's not relevant.  Just curious mostly.
<abentley> lifeless: No, but I have occasionally encountered situations where that would be useful.
<deryck> abentley, I guess it's not that shocking to me either because js objects aren't really the same as python dicts, even though they seem similar in ways.
<abentley> deryck: It was surprising to me, because it's the first language I've encountered that had ordered mappings by default.
<sinzui> wallyworld_, wgrant, StevenK: Gerald the Gorilla http://www.youtube.com/watch?v=beCYGm1vMJ0
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: StevenK | Critical bugtasks: 266
 * StevenK assumes gmb is no longer reviewing.
<StevenK> wgrant: When you have a tick, glance at the query in https://code.launchpad.net/~stevenk/launchpad/extend-subscription-check/+merge/80173 , lines 51 to 62
<wgrant> abentley: Any reason you didn't use IJSONRequestCache for storing the mustache template?
<wgrant> abentley: Using dumps directly is not safe.
<wgrant> StevenK:
<wgrant> tables=(
<wgrant>     Bug,
<wgrant>     Join(BugSubscription, BugSubscription.bug_id == Bug.id)),
<wgrant> Same with the multi-line where.
<wgrant> I also find the execute(Union syntax distasteful.
<wgrant> Particularly since you then have you use .rowcount afterwards.
<StevenK> wgrant: I've fixed the tables and where. I don't really want to reflow the entire block though. :-/
<poolie> jesus team roles are such a mess
<poolie> how can i find out what access an open team actually has?
<wgrant> You can't.
<poolie> yeah that was really a way of saying "i wish i could"
<wgrant> StevenK: What reflowing are you talking about?
<wgrant> StevenK: Also, when you raise the exception on line 68, what sort of string does the policy turn into?
<wgrant> Open? OPEN?
<StevenK> I think it's 'open'
<poolie> lifeless: re bug 881237, would it be too tasteless to just catch Exception around the call to dkim
<_mup_> Bug #881237: broken dkim key on qq.com causes mail to be dropped <dkim> <mail> <oops> <Launchpad itself:Triaged by mbp> <pydkim:In Progress by mbp> < https://launchpad.net/bugs/881237 >
<wgrant> Please try to check.
<poolie> so that random bad input, causing eg a KeyError or ValueError, doesn't result in the mail being dropped
<StevenK> wgrant: Hold on, I'll do it now.
<poolie> this is kinda papering over pydkim not handling those things cleanly but perhaps it's good to be defensive
<wgrant> poolie: Have you read pydkim's code? :)
<wgrant> s/good/necessary/
<poolie> there are no tests
<wgrant> I agree completely with catching Exception.
<poolie> presumably because the code has been formally proved to be correct
<poolie> ok thanks
<StevenK> Formal proof nothing. LP has taught me that you *NEED* tests.
<wgrant> Our fork is no longer a several-hundred-line function, and it's more correct and tested, but it's still awful awful.
<poolie> now that upstream seems to have appeared again perhaps we can push them up
<wgrant> Indeed, I was surprised to see it on github.
 * StevenK kills SSO
<wgrant> Since we had no response from him for several months.
<poolie> yeah, but isn't that always the way?
<poolie> actually doing something yourself prompts others into action
<StevenK> wgrant: What does it mean when icons in the dock have a green background?
<wgrant> StevenK: s/dock/Launcher/?
<StevenK> Right
<wgrant> It means either that it decided the primary colour of your icon was green, or possibly that your graphics drivers suck.
<StevenK> Sorry, one icon is green
<StevenK> And it changes
<StevenK> wgrant: The text is "The team subscription policy cannot be Open Team because it is subscribed to or assigned to one or more private bugs."
<wgrant> I guess that's OK.
<wgrant> We really should s/subscription/membership/ everywhere :(
<StevenK> If you're happy enough with it, I'll land it.
<wgrant> Sounds OK.
<wgrant> wallyworld_: But the action *can* be mostly undone.
<wallyworld_> sort of
<wallyworld_> i was adhering to the instructions in the bug report
<wgrant> As it stands, the statement is a lie.
<wgrant> It says that it will mark the bug as not affecting that target, and this action cannot be undone.
<wgrant> I can create a counterexample by clicking "Also affects project"
<wallyworld_> i think the intent is to make people really sure they want to delete something
<wallyworld_> in most all other cases, delete is permanent
<wallyworld_> perhaps good to be consistent with the warnings
<wgrant> In this case the task deletion is permanent.
<wgrant> The not-affectingness is not.
<wgrant> The only information that is deleted permanently is metadata that probably <10 people look at.
<wallyworld_> you may be best to take it up with the author of the bug report?
<wgrant> Perhaps.
<wgrant> sinzui: Still around?
<wallyworld_> i don't mind either way what is decided
<wallyworld_> so why can't i use the construct <p tal:define="foo context/bar"  tal:content="string:${foo}"/>  ? it only likes <p tal:content="string:${context/bar}"/>
#launchpad-dev 2011-10-26
<StevenK> wgrant: That branch is in ec2 now.
<wallyworld_> seems defines can't use used inside tal:context
<wgrant> wallyworld_: You're moving the confirmation message into the template?
<wallyworld_> yup
<wgrant> <p>Blah blah blah <tal:someproject replace="context/someproject">Some Project</tal:someproject></p>
<wgrant> Don't use string:
<wallyworld_> ok. other places do :-)
<wgrant> They're mostly wrong, and frequently holey.
<wgrant> I've seen it mostly used to construct JS.
<wgrant> Do you know of others that don't>?
<wallyworld_> there's a few places in tales but i've closed my search window
<wgrant> string: is OK for some things sometimes. But it's very rarely OK for content/replace.
<wallyworld_> wgrant: one place in lp form - tal:content="string:${widget/label}
<wgrant> wallyworld_: That's slightly more acceptable, since it just wants to append a colon.
<wallyworld_> also lots of places in <title> elements
<wallyworld_> more than just a colon
<wallyworld_> wgrant: so is  <p tal:content="string:${context/bar}"/> a security issue?
<wgrant> It's not a security issue.
<wallyworld_> it's far less verbose than your version
<wgrant> It's just pointless.
<wgrant> And obscure.
<wallyworld_> it was a contrived example
<wgrant> string: is useful when you're applying minimal formatting around a some variables.
<wallyworld_>  <p tal:content="Blah blah blah string:${context/bar} blah blah blah"/>
<wgrant> But in your case you're constructing paragraphs with a couple of substitutions.
<wallyworld_> compare what i just typed with
<wallyworld_> <p>Blah blah blah <tal:someproject replace="context/someproject">Some Project</tal:someproject></p>
<wgrant> TAL can be ugly, but putting everything in tal:content is not the solution.
<wallyworld_> i think mine is very readable
<wallyworld_> more so than yours
<wgrant> string: encourages people to do stupid things.
<wgrant> Like use structure.
<wallyworld_> but i'm not doing that
<wgrant> No.
<wallyworld_> that's a false argument
<wgrant> It's not false.
<wgrant> It's a perfectly valid argument.l
<wgrant> If we permit patterns that are not hard to use insecurely, people will use them insecurely.
<wallyworld_> my construct is readable and concise
<nigelb> Morning!
<wallyworld_> and not insecure
<wgrant> It's not insecure, but this sort of thing often leads to people putting structure in front of huge things.
<wgrant> Like you did in your first version of the branch.
<wallyworld_> code reviews are meant to pick up such things
<wgrant> They normally don't.
<wgrant> I normally pick them up by reading diffs after they land.
<wallyworld_> i thought structure(xxxx) is kosher?
<wgrant> No.
<wallyworld_> is it deprecated then?
<wallyworld_> and why do we have it?
<wgrant> Its use is discouraged, in deference to templates.
<wallyworld_> it's use for request notifications, no?
<wgrant> Because people have shown that they cannot use structured() safely.
<wgrant> Until we have a sane templating system, which is blocked on HTML being fucking terrible and IE not supporting XHTML, we have to discourage bad practices, because we cannot make things secure by default.
<wallyworld_> just about any programming construct can be misused, that's not a valid reason to reject valid uses
<wallyworld_> when the alternative is verbose and ugly
<wgrant> The alternatively is empirically less likely to be abused.
<wgrant> -ly
<wgrant> Because it keeps content of different security levels labelled appropriately.
<lifeless> 'The alternatively is empirically less like to be abused.' ?
<wallyworld_> i think you're creating an issue where one doesn't exist
<wgrant> lifeless: s/alternatively/alternative/
<lifeless> :>
<wgrant> wallyworld_: So, someone wants to extend this confirmation message.
<wgrant> They want to emphasise "permanently", because people are deleting tasks too readily.
<wgrant> They add markup to the template, but find it's being escaped. That region of the data is clearly literal, so they add the "structure" keyword.
<wgrant> They miss that three lines later, in the same string literal, there is a variable substitution.
<wgrant> Mixing data of different types in one literal is not a good idea.
<wgrant> This *does* happen.
<wallyworld_> i'll take your word for it
<lifeless> wallyworld_: we had a -raft- of security holes related to prior use of structured
<wallyworld_> that points to an epic fail in our code reviews then
<wgrant> Yes.
<wgrant> Lots of things do.
<wgrant> It also points to an epic fail of our infrastructure.
<wgrant> Because it should be *very difficult* to use it insecurely.
<StevenK> But yay TALES
<wgrant> I tried to fix it in March, but was foiled by inline JavaScript in HTML (not XHTML) being impossible.
<wgrant> TALES is only barely relevant here.
<StevenK> It's our own code that is at fault, then?
<wgrant> TAL
<wallyworld_> i see your point but i'm sad that i'm being forced to make something simple more complex just because someone *might* extend it insecurely in the future and the code review *might* not catch it
<wgrant> wallyworld_: Our code reviews are almost useless.
<wallyworld_> i've found that they've offered good feedback to my mp's
<wgrant> Sometimes they do.
<wgrant> They often miss things.
<nigelb> ..
<nigelb> *blink*
<wallyworld_> one would hope that between code reviews and tests we catch most things
<wgrant> One would, yes.
<wgrant> The fulfilment of that hope is... arguable.
<wallyworld_> i still believe that something should be done in the simplest way that makes sense now (YAGNI comes to mind) and anyone extending it is responsible for doing so the right way
<wgrant> They are responsible for doing it the right way.
<wgrant> But they don't.
<wgrant> Time and time again, people introduce holes through this sort of thing.
<wallyworld_> if that argument were to be taken a step or two further, no code would ever get written
<wallyworld_> because everything would be so complicated
<wallyworld_> because we would be guarding against every future possible issue
<wallyworld_> that may not even arise
<wallyworld_> and assming all lp devs are idiots
<wgrant> Just this morning, a limited XSS was introduced through the dumps issue that you argued about a few weeks ago.
<wallyworld_> ?
<wgrant> The "simplejson.dumps then put it into HTML unescaped" pattern.
<wgrant> That has been used in several places throughout the APP.
<wgrant> -caps
<wgrant> You argued when I told you in a review to avoid it.
<wgrant> Last night an abuse was landed which introduces unescaped data.
<wgrant> LP is complicated.
<wgrant> People will copy code.
<wallyworld_> so in that case the code could directly introduce unescaped data
<wallyworld_> i wasn't aware that dumps could
<wgrant> Right.
<wallyworld_> but in the case now with the tales, it doesn;t
<wallyworld_> it required modification to the code to do it
<wgrant> TALES is moronically designed.
<wallyworld_> requires
<wgrant> It is terribly easy to introduce unobvious holes.
<wallyworld_> i wouldn't call adding "structure" unobvious
<abentley> wgrant: Yes, I don't want it to be emitted every time I get data.
<wgrant> You'd think not.
<wgrant> But people throw it around carelessly.
<wgrant> abentley: dumps is not secure in its default mode, so its direct use is strongly discouraged.
<wgrant> In this case, you're making our HTML invalid by injecting unescaped tags from the template.
<wgrant> Fortunately the template is not user-controlled, so it will merely break parsers rather than being a terrible security hole.
<wgrant> (breaking correct parsers is what let us know about a security hole in February)
<abentley> wgrant: Yes, I didn't think this particular use was a security issue.  I'll look into the unescaped tags issue.  Do you have an example handy?
<wgrant> abentley: HTML is... special. Tags inside a <script> are sometimes parsed as tags, but you also can't escape them using &gt; etc. as you would in normal XML content.
<wgrant> We fix this for most of our JSON by encoding <>&"' as \uXXXX
<wgrant> It's the only way to get it through unscathed.
<wgrant> There's a special encoder for dumps around... I forget what it's called. It's used by the IJSONRequestCache.
<abentley> wgrant: ResourceJSONEncoder?  I thought that was just for rendering web service objects.
 * wgrant finds.
<wallyworld_> i've used ResourceJSONEncoder on a dict that had both web service objects in it and other stuff
<wallyworld_> so perhaps it does both things
<abentley> wallyworld_: Oh, it certainly works fine for plain types.  I just didn't know it had any advantage for plain types.
<wgrant> eggs/lazr.restful-0.19.4-py2.6.egg/lazr/restful/_resource.py:class ResourceJSONEncoder(simplejson.encoder.JSONEncoderForHTML):
<wgrant> You can use ResourceJSONEncoder, or JSONEncoderForHTML directly.
<wallyworld_> i wasn't sure whether it did, but was guessing that perhaps it did so
<abentley> wgrant: great, thanks.
<wgrant> abentley: I'm not sure if BugTarget:+bugs has the same strict parsers running over it, but if this leaked onto BugTak:+index it would certainly break things.
<wgrant> So probably best to fix before we turn the flag on widely.
<abentley> wgrant: Yeah, I can do that.
<wgrant> But I still don't understand why you can't conditionally inject it into the IJSONRequestCache. Is it because that is sometimes reloaded?
<wgrant> Perhaps we need a separate static version.
<abentley> wgrant: I don't want to get it when I access the cache via ++model++.
<wgrant> Right.
<lifeless> anyone around that knows yuixhr stuff ?
<poolie> wgrant: so should an unexpected error from dkim be a warning (and oops) or just info?
<wgrant> poolie: Warning, I think.
<lifeless> poolie: if you want an engineer to look at it without users complaining, an oops.
<lifeless> poolie: otherwise info
<lifeless> poolie: thats what warnings (and oopses) mean to us
<poolie> yeah
<poolie> we oughta fix pydkim to catch this internally
<poolie> my concern is just whether that actually counts as critical
<wgrant> lifeless: What's the yuixhr issue?
<lifeless> unexplained errror from the fixture tets with 'Unexpected error: Unknown error'
<lifeless> reproducable in https://code.launchpad.net/~lifeless/launchpad/useoops with bin/test -vvt test_yuixhr_fixture
<poolie> wgrant: ok then https://code.launchpad.net/~mbp/launchpad/881237-dkim-error/+merge/80413
<wgrant> poolie: Approved, thanks.
<wgrant> lifeless: blink
<wgrant>   Running:
<wgrant>  lp/testing/tests/test_yuixhr_fixtureSegmentation fault
<lifeless> wgrant: \o/
<poolie> woo, record low latency
<poolie> for me
<wgrant> poolie: I can rescind my approval if you're uncomfortable with such speed.
<wgrant> lifeless: Does it in devel too :(
<poolie> it's ok, there's still a good chance it will stall before deployment
<lifeless> wgrant: oh, phew.
<poolie> but if we're lucky it will be under 24h from a critical being filed to being fixed
<lifeless> I'll toss this at ec2 and see if its happy or not
<wgrant> poolie: Unless stuff goes wrong we should be able to deploy it before lunch tomorrow :)
<lifeless> wgrant: devel for me passed, so I thought it was my branch
<wgrant> lifeless: It probably is your branch, but it's difficult to say because it's so damn unreliable.
<lifeless> ec2-win
<lifeless> <Response><Errors><Error><Code>InvalidInstanceID.NotFound</Code><Message>The instance ID 'i-3bbc4e58' does not exist</Message></Error></Errors><RequestID>a71551e1-6488-407c-b4ae-8288ad4c8762</RequestID></Response>
<lifeless> alsy yay boto for not wrapping errors and instead spewing xml
<wgrant> Hello launchpadlib...
<nigelb> I was about say boto is WIN, but wgrant wins.
<wgrant> wallyworld_: Ah, good.
 * nigelb looks around for a bug to hack on for today's holiday
<wallyworld_> huwshimi: there seems to be split opinion as to whether to hide/disable links which the user cannot use
<nigelb> wgrant: hey can haz more details on bug 583392?
<_mup_> Bug #583392: IntegrityError raised setting a branch for a project series. <branches> <easy> <lp-code> <oops> <series> <Launchpad itself:Triaged> < https://launchpad.net/bugs/583392 >
<huwshimi> wallyworld_: Can these links *ever* be used on a private bug?
<wallyworld_> huwshimi: yes, but they are to be disabled when the bug has bugtasks targetted at certain pillar types
<huwshimi> wallyworld_: And in what circumstances can the links become re-enabled?
<wallyworld_> so sometimes they are valid but the user may not be able to change it (depending on their permission)
<wallyworld_> huwshimi: existing bug tasks need to be deleted
<wallyworld_> or retargetted
<wgrant> nigelb: I'm afraid it probably requires OOPS and DB access to diagnose :(
<nigelb> wgrant: :(
<nigelb> Moving on then.
<lifeless> nigelb: jsoops :_
<nigelb> AHA!
<StevenK> wgrant: So, paragraph on +activate-ppa, or missing link?
<nigelb> lifeless: Thanks for reminding. jsoops it is today :)
<huwshimi> wallyworld_: Is the state that makes the options unavailable transitory?
<wallyworld_> huwshimi: not really
<wallyworld_> huwshimi: if a bug has a bug task for a project, it doesn't make sense to add additional bug tasks for a distro
<wallyworld_> huwshimi: it's a new constraint due to the disclosure work
<wallyworld_> huwshimi: so instead the user would create a new bug affecting the distro/source package and link the bug to the other one
<wallyworld_> when bug linking is done
<mwhudson> https://lp-oops.canonical.com/oops.py/?oopsid=2107EE48 is really strange
<mwhudson> ah unless the form has changed since that oops
<huwshimi> wallyworld_: OK, so really we're in a situation where the bug is in a state where the control isn't temporarily unavailable and there's no direct way to enable the control, so this seems like a situation to hide the links. If it's ambiguous as to why they are hidden it may require an in-place explanation.
 * StevenK sighs at lp-oops forgetting the query string after login
<wallyworld_> huwshimi: i agree that they should be hidden. i was just making sure given that some folks dislike hidden links :-)
<wallyworld_> thanks
<lifeless> wallyworld_: some folk will dislike everything we do
<StevenK> mwhudson: Why would the form change since the 8th of October?
<lifeless> :)
<mwhudson> StevenK: the work to use the code import infrastructure with rcs_type == bzr instead of mirrored branches is pretty recent
<huwshimi> wallyworld_: right, but people don't like hidden links when they are hidden inappropriately or without explanation.
<wallyworld_> lifeless: yes, but one of those is a lp dev, so i was just being 100% sure :-)
<StevenK> mwhudson: Except we don't have a code team anymore ...
<wgrant> code team == jelmer
<mwhudson> StevenK: err what?
<lifeless> The connection to login.ubuntu.com was interrupted while the page was loading.
 * StevenK rattles wgrant for an answer to his question.
<mwhudson> hm
<StevenK> lifeless: Saw that this morning, annoying.
<wgrant> lifeless: I get that a couple of times a day.
<mwhudson> timestamp: Thu 2011-10-06 17:33:40 +0000
<mwhudson> ^ this was the change hitting devel that disabled the ability to create mirrored branches
<mwhudson> so it could well not have been deployed until the 8th or 9th
<wallyworld_> huwshimi: i'm not sure if any explanation is required on the page. the wording might turn out to be quite difficult to get concise
<StevenK> Indeed
<StevenK> And it hasn't OOPS'd since the 8th
<wallyworld_> huwshimi: and add to a already very busy page
<huwshimi> wallyworld_: Right, but it needs to be clear why things might be different on that page (i.e. missing links)
<wgrant> "Private bugs cannot affect multiple project."
<wgrant> +s
<wallyworld_> that is concise but my fear is that people won't appreciate what is means
<wallyworld_> but it's better than nothing
<wallyworld_> i'll add it to the template for display if a link is to be hidden
<poolie> StevenK: can you give me an opinion on https://code.launchpad.net/~mbp/launchpad/612171-diff-generation-spam/+merge/80288
<StevenK> I'm not sure why we do care about pending writes on a branch
<StevenK> But in general, yes, we should just retry the job.
<StevenK> MPJs make me very sad in many ways.
<wallyworld_> huwshimi: this ok? in this case, one link is missing. but sometimes also only the "Target to Series" link is visible http://people.canonical.com/~ianb/missing-link-text.png
<wgrant> wallyworld_: You're going to s/distribution/package/, right?
<wallyworld_> wgrant: i was going to but thought it may warrant some more feedback
<wallyworld_> but if you are sure....
<wgrant> I'm only mostly sure.
<huwshimi> wallyworld_: I think so, it might be nice in a slight grey. Let me see if there's an appropriate class
<wgrant> It could be considered separate.
<wallyworld_> wgrant: that's what i was thinking
<huwshimi> wallyworld_: Is that a <p>?
<wallyworld_> huwshimi: no, a div
<huwshimi> wallyworld_: No problems
<wallyworld_> huwshimi: also this is the css i used https://pastebin.canonical.com/54929/
<wallyworld_> ah bollocks. css is a bit screwed
<huwshimi> wallyworld_: Do you want to add color: #666; to that class?
<wallyworld_> huwshimi: will do. i think the css is not quite right though. perhaps is should be div.private
<huwshimi> wallyworld_: Ah ok, well whatever class it is for that message
<wallyworld_> yes
<wallyworld_> it works as is but is horrible. fixing
<wallyworld_> on second thoughts, i think it's ok. i got myself confused
<lifeless> wgrant: useoops diff is up to 2756 lines - but AFAIK there is the one mysterious failure left.
<lifeless> wgrant: its smaller than it sounds: +413, -875
<lifeless> wgrant: are you willing to review this ?
<wgrant> lifeless: This moves everything to AMQP?
<lifeless> wgrant: for tests, they are now all using self.oops or (doctests) CaptureOops
<lifeless> wgrant: tests generating external oopses now always get them from amqp (by calling sync() before inspecting self.oopses)
<lifeless> wgrant: production won't have rabbit configured yet so won't send over amqp
<lifeless> wgrant: but it will be ready to when we get a live rabbit
<lifeless> wgrant: I intend to get the oops-tools listener deployed before landing this (e.g. later this week)
<wgrant> Ah.
<wgrant> Good.
<lifeless> alternatively, when we make rabbit live we can disable the oops exchange which will disable it
<wgrant> lifeless: Note that txlongpoll is currently crap and exposes the entire vhost to the web.
<wgrant> So it's not safe for anything else to use rabbit yet.
<lifeless> wgrant: different vhosts ftw
<wgrant> (this regressed post-Dublin while I wasn't watching)
<wgrant> Also, I guess this introduces the controversial BSON change.
<lifeless> anyhow
<lifeless> if I click you in the ui for a review and move to needs-review, is that cool.
<wgrant> Sure.
<lifeless> alt - you can click on claim-it
<wgrant> 'tis claimed.
<lifeless> wgrant: https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/79516
<lifeless> ah, you found it, cool
<wallyworld_> huwshimi: i reused the formHelp class (color = #777 plus a tiny bit of spacing). looks nice. for completeness, can you Approve the mp now? :-)
<wgrant> lifeless: CaptureOops can't be used as a context manager?
<lifeless> wgrant: why not ?
<wgrant> Well, you use setUp/cleanUp over small chunks of code in eg. xx-request-expired.txt
<huwshimi> wallyworld_: Great. Do you mind quickly showing me a screenshot?
<lifeless> doctests
 * lifeless hates em
<wallyworld_> huwshimi: yep, just a sec
<huwshimi> wallyworld_: Thanks, I know it's a hassle
<wallyworld_> huwshimi: it's no problem at all. i'd rather get it right up front
<wgrant> lifeless: This looks generally good, but it'll be a while before I'm done with it :)
<lifeless> of course
<lifeless> no rush
<wallyworld_> huwshimi: http://people.canonical.com/~ianb/missing-link-text1.png
<lifeless> wgrant: a particular concern to look out for is a getLastOops (direct or indirect) looking for no-change replaced with self.oopses w/out a sync call.
<huwshimi> wallyworld_: Yep that looks great. I think it looks more like a notification now and less like content
<huwshimi> wallyworld_: I'll go deal with the mp
<wallyworld_> huwshimi: yes, agreed
<wallyworld_> thanks
<huwshimi> wallyworld_: No problems. Approved.
 * wallyworld_ fires up ec2
<StevenK> Hm, my branch should be almost done
<StevenK> Although ec2 seems to have grown out to 4h30 again
<StevenK> And there is no windmill to disable to gain 30 minutes :-(
<wallyworld_> StevenK: and there's nothing to blame when it goes wrong wither
<wallyworld_> either
<jtv> wgrant: any chance you could triage bug 881476 for me?  I'm out of my depth.
<_mup_> Bug #881476: Spurious Changed-By fields in native-sync -changes email. <derivation> <Launchpad itself:New> < https://launchpad.net/bugs/881476 >
<StevenK> wgrant: http://people.canonical.com/~stevenk/PPA-change.png
<wgrant> StevenK: Perhaps.
<StevenK> wgrant: You like it, you don't?
<StevenK> Sprinkle 'or' to taste
<wgrant> I don't, but I'm not sure what would be much better.
<wgrant> jtv: Haven't you been in that code lately?
<jtv> wgrant: it does seem somewhat related, but not that code, no.
<wgrant> "that code" == "Soyuz notification code"
<jtv> wgrant: this bug sounded more like it concerned _an input to_ the notification code to me.  Is that impression wrong?
<wgrant> jtv: I forget what you've been working on, but it's very related.
<wgrant> Ah, notification recipients, right.
<wgrant> Yes, somewhere between very and extremely related.
<wgrant> jtv: Do you want spoilers?
<jtv> wgrant: please be clear with me today.
<wgrant> So, lp.soyuz.adapter.notification will include Changed-By if it can find an email address.
<wgrant> It finds the email address using SPR.creator.preferredemail
<wgrant> Because gina is shit, SPR.creator for this SPR is actually the Maintainer field from the source package.
<wgrant> Looking in the DB, this is python-apps-team
<wgrant> Which no longer has the relevant email address, but must have when the SPR was created some time ago.
<wgrant> It no longer has any email address at all.
<wgrant> So there is no preferredemail.
<wgrant> So Changed-By doesn't show up in the email.
<wgrant> That explains the magical disappearing field.
<wgrant> And the rest is a matter for whoever is designing derivative distros.
<StevenK> "LOL"
<jtv> I was secretly hoping that this might also do something for the bug I was working on, but apparently not.
<wgrant> It's very very relevant to the bug you're working on.
<wgrant> It will be sorted out in the mid-late stages of the derivative distros project, after StevenK's refactoring of the notification code.
<wgrant> Because the notification stuff needs a big rework before we can seriously consider even going into beta.
<StevenK> wgrant: So I can't use create_initialized_view for Person:+index :-(
<wgrant> StevenK: Not currently, no. But you probably want to investigate why not and fix it.
<wgrant> It would make lots of tests much nicer.
<wgrant> I would dig, but I have a lot piled up at the moment :/
<jtv> thanks wgrant for looking into this.  Will have to process later.
<wgrant> lifeless: What was the difference between OopsHandler and OopsLoggingHandler?
<lifeless> wgrant: Do you really care ? ;)
<lifeless> wgrant: OH was warning and up and uses the global error utility and didn't honour record.exc_info
<lifeless> wgrant: OLH was error and up, used a supplied error utility and only oopsed if record.exc_inf was non-None
<lifeless> wgrant: OLH was unused.
<lifeless> I filed a bug, bah, haven't linked it
<lifeless> I want a search dialog on link-a-bug
<lifeless> linked now
<wgrant> Thanks.
<lifeless> OH will be moving out of tree soon anyhow
<wgrant> Good, good.
<lifeless> to python-oops
<wgrant> lifeless: I have a dev appserver running on a fresh rabbit instance, that's never had amqp2disk run to set up the exchange and queue.
<wgrant> OOPSes don't seem to be ending up in /var/tmp/lperr.
<lifeless> wgrant: interesting :)
<wgrant> Ah, looks like non-existent exchanges aren't an immediate error...
<lifeless> wgrant: first oops geerated should trigger a failed send, no ?
<wgrant> "The exception refers to the previous exchange, but that's as good as it gets with AMQP. When you send a Basic.Publish, there is no confirmation of whether it succeeded or not. If there was an error, you get a Channel.Close message with the reason, but this can happen after you already sent another Basic.Publish message."
<wgrant> I'm not sure if amqplib handles return messages like that at all.
<lifeless> amqplib looks for a reply on verbs it sends that generate acks, except when you tell it not to
<lifeless> that one sounds particularly evil though
<lifeless> poolie: qq.com being broken is just lovely-ironic
<poolie> mm?
<poolie> it's some kind of chinese yahoo-like thing, right?
<poolie> oops, i've got to go
<lifeless> poolie: qq in online games -> whinging or complaining
<spm> aiui, 'qq' used to be the key combo to 'exit game' in one of the RTS's (command & conquer??? doesn't matter). so in a player vs player match, if your getting upset/beaten, you'd "qq" and exit. In the vernacular it's come to mean "cry baby goes home" more or less.
<wgrant> WC2, IIRC.
<spm> I was curious one day. and looked it up. :-)
<spm> could be.
<rvba> Thanks for the review StevenK!
<lifeless> wgrant: thanks for the review
<lifeless> wgrant:
<lifeless> Could you please explain how this relates to OopsLoggingHandler?
<lifeless> wgrant: what do you mean by that
<wgrant> lifeless: Argh, you explained that here.
<wgrant> Nevermind :)
<lifeless> wgrant: ditto the oopshandler query I presume
<wgrant> lifeless: Indeed.
<lifeless> wgrant: also I know about the routing key; I want to permit us to change config in future without needing to patch the tree
<lifeless> wgrant: the thing with doctests is that the start and end have narrative in between them
<lifeless> wgrant: so using as a context manager is nontrivial
<wgrant> Ah, indeed.
<lifeless> two statements is due to line length
<wgrant> = (
<wgrant>    blah)?
<lifeless> less readable
<wgrant> More obvious and less namespace-polluting.
<wgrant> But OK.
<lifeless> This import is probably in the wrong place.
<lifeless> > +import oops_amqp
<lifeless> I don't understand
<wgrant> It refers to the line above.
<lifeless> the next line is import testtools
<wgrant> lp_customize, IIRC.
<wgrant> Which is an in-tree import, so belongs in the last import section.
<lifeless> ah, unchanged but yes wrong
<lifeless> Why not use self.oopses[-1]? If you want to handle the case where
<lifeless> there's more than one new OOPS, you'll need to compare the index and length.
<lifeless> -> because I want to capture the first oops generated in each loop
<lifeless> -1 would get the last
<lifeless> I'm not trying to enforce one and only one
<lifeless> I caught some places where we double-oops
<lifeless> but it wasn't the focus of the branch, and I'm trying to contain scope
<adeuring> good morning
<lifeless> wgrant: the runner logs adequately
<lifeless> wgrant: it was logging *twice*, in two layers of calls
<lifeless> wgrant: can you expand on  '> +    errorUtility._ignored_exceptions = set()
<lifeless> Can you feel my piercing glare of disapproval?
<lifeless> '
<lifeless> wgrant: the emails get = line extended but the doctest is looking at the wire representation
<allenap> StevenK: Up for a shortish review? https://code.launchpad.net/~allenap/python-pgbouncer/reliable-shutdown/+merge/80382. Unfortunately I have to go out nowish so I can't be around to field questions, so push back if that's a problem.
<wgrant> lifeless: Setting private attributes in production code upsets me.
<StevenK> allenap: I read it previously, but I don't feel confident enough in pgbouncer to review it.
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: - | Critical bugtasks: 266
<allenap> StevenK: Okay, thanks anyway :)
<allenap> Anyone else in a reviewy mood?
<lifeless> wgrant: it was previously a subclass setting a private attribute; swings and roundabouts
<lifeless> wgrant: I can make it public easily enough. Shrug.
<wgrant> lifeless: This is true.
<lifeless> wgrant: I'm inclined to ignore it, its private to the extent of don't fiddle without reason
<wgrant> lifeless: k
<lifeless> but python explicitly has this consenting adults approach
<lifeless> wgrant: I think you misunderstand notify_publisher
<wgrant> Yes, so I do.
<wgrant> I misread it.
<wgrant> But now I don't understand how OOPSes aren't added to self.oopses twice: once over AMQP, once through local event.
<lifeless> they are if you sync without reason
<lifeless> but sync() isn't free so its not done on __getitem__ in oopses
<wgrant> Or if you sync after generating both in-process and out-of-process OOPSes...
<lifeless> wgrant: which no tests do
<wgrant> Sure, I gathered that.
<wgrant> But then it's not so much syncing as pulling everything from AMQP.
<wgrant> Including stuff that may already be there.
<lifeless> everything since the fixture was created
<wgrant> This should be in some docstring somewhere, I think.
<lifeless> Your last statement suggests a misunderstanding
<lifeless> tests can only get their own oopses (assuming no rogue processes)
<wgrant> Sure.
<wgrant> So, I generate an OOPS in process, and also one in a subprocess.
<lifeless> and each oops from amqp is delivered only once
<wgrant> I call .sync()
<wgrant> len(self.oopses) == 3
<lifeless> yes
<wgrant> That is unobvious.
<lifeless> I can doc that
<wgrant> Needs documentation.
<lifeless> ok, I'll act on the review later this week.
<wgrant> Thanks.
<lifeless> there will be some more test fixes and I'll ask for incremental review on that
<bigjools> wgrant: so arch-all domination changes didn't break the world, it seems
<wgrant> bigjools: Not noticeable so, at least.
<wgrant> bigjools: Speaking of which, we should reobsolete all the obsoleted series.
<bigjools> some 30 seconds slower per suite at first
<bigjools> wgrant: for your deathrow fix?
<wgrant> bigjools: Right.
<bigjools> wgrant: sure, I'll approve if you request
<wgrant> Not quite sure how to request.
 * wgrant tries the easy way.
<cjwatson> Does anyone know why (a) https://launchpad.net/ubuntu/+source/adios/1.3-7/+build/2855760 is "Needs building" and (b) https://launchpad.net/builders shows the amd64 queue empty and all distro amd64 builders idle?
<cjwatson> Also I could have sworn I saw that adios build start and then flip back to needs-building.
<cjwatson> bigjools: Do you expect the slowness to reduce later, BTW?
<cjwatson> It's added about ten minutes to the distro publisher
<bigjools> cjwatson: it will not get better
<cjwatson> Actually maybe that's an exaggeration, but I notice it stretching out to :45 now
<cjwatson> maybe more like high thirties normally; before, it usually finished around :32
<bigjools> it depends on how many outstanding packages are waiting to be superseded
<bigjools> since that number has now increased
<bigjools> it re-considers them every time
<bigjools> we also have to do a 2-pass domination, which has added 30 seconds per suite
 * bigjools OTP
<cjwatson> It does unfortunately mean that the value to platform in speeding up cron.germinate is diluted, since it no longer looks as if that will be enough to get us to 30-minute cycles.
<cjwatson> But maybe I can argue it's a good idea anyway.
<bigjools> there are probably many optimisations
<cjwatson> Yeah, I was just naively hoping for a big bang :)
<cjwatson> https://launchpad.net/ubuntu/precise/+builds?build_text=&build_state=pending&arch_tag=amd64 -> 7 results, build queues empty.  Definitely something broken here
<bigjools> I'll look in a bit
<cjwatson> thanks
<wgrant> 10 minutes!?
<wgrant> You said it wasn't going to be much slower :(
<allenap> jtv: Are you in the mood for a review? Indeed, are you still "at work"? If you are, on both counts, would you mind looking at https://code.launchpad.net/~allenap/python-pgbouncer/reliable-shutdown/+merge/80382?
<jtv> allenap: can be with you in a few minutes.
<allenap> jtv: Thank you.
<cjwatson> The builders are doing something now; it's possible my recent batch of retries kicked something into action.  I'll see if they manage to flush completely.
<cjwatson> Hmm, the buildlogs just say "Waiting for slave process to be terminated" though
<cjwatson> And there https://launchpad.net/ubuntu/+source/cdargs/1.35-8/+build/2870409 flipped back to needs-building.  It's dead Jim.
<cjwatson> Same on i386.
<jtv> allenap: got some questions up on the MP.
<allenap> jtv: Thanks, just saw those.
<jtv> Code notes to follow.
<danhg> Is flacoste here? Am I spelling it right?
<wgrant> That's right.
<danhg> thanks wgrant
<wgrant> He's probably not here quite yet.
<danhg> fair enough
<wgrant> He was also unwell yesterday, I believe.
<danhg> Is jsackett the right irc name for Jon S?
<wgrant> jcsackett
<danhg> aha
<danhg> thanks!
<danhg> jcsackett - are you around?
<wgrant> He's not here yet.
<danhg> OK
<bigjools> heh
<wgrant> flacoste is in the channel, but idle.
<danhg> I'm just too eager today eh? :-D
<wgrant> jcsackett is not here at all.
<bigjools> can I help danhg?
<danhg> hey bigjools, na, it's all fine. Just waiting to send Jon something, thanks tho
<bigjools> ok
<rvba> danhg: jcsackett should be here in ~2 hours.
<danhg> thanks rvba - i'll email him, it's all good
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: abentley | Critical bugtasks: 266
<allenap> Is there anyone around who can give me upload rights on http://pypi.python.org/pypi/pgbouncer?
<flacoste> allenap: only lifeless and stub can do that
<flacoste> best to send an email to lifeless
<flacoste> as stub is flooded
<allenap> flacoste: Ta.
<danhg> jcsackett: Hey
<lifeless> allenap: flacoste: fixing; we should really have teams in pypi
<lifeless> allenap: whats your pypi account ?
<flacoste> yep, we should
<lifeless> flacoste: dunno if you've seen it, my useoops branch is now 2.7K LoD (+450, -850 LoChange)
<lifeless> flacoste: mega-branchzilla-incoming-this-week :)
<lifeless> ha!
<lifeless> http://limpesuatela.tumblr.com/
<flacoste> lol
<allenap> lifeless: Oops, it's gavinpanella.
<lifeless> on TL call now will add you post that
<allenap> lifeless: Thanks.
<lifeless> allenap: added
<mwhudson> [rollback=14192] back out r14191, <- something looks wrong here <wink>
<lifeless> lol
<mwhudson> i think 14192 is probably correct...
<abentley> lifeless: I have to go, but could you point out *where* I'm violating the principle that a memo is opaque?
<lifeless> abentley: int(memo)
<lifeless> line 87 of the diff
<lifeless> abentley: storm range factory memos are (IIRC) uuencoded blobs
<lifeless> abentley: I may have the encoding wrong, but they aren't ints.
<abentley> lifeless: that's a test helper.
<lifeless> abentley: ah!. I have a dentist appt, but I will re-review with that in mind afterwards
<lifeless> abentley: hopefully I just read things badly and its all copacetic
<bigjools> mwhudson, lifeless: mea culpa
<lifeless> abentley: looking briefly I suspect it will be
<mwhudson> bigjools: np, it just made me laugh
<bigjools> I blame bzr for making me think about two versions when backing out!
<sinzui> wallyworld, I had to visiting sound setting and toggle mute the other day
<sinzui> wallyworld_, The feature is still listed on the board to do: http://launchpad.leankitkanban.com/Boards/View/12720553
<poolie> hi all, thanks for the review abentley
<StevenK> sinzui, wallyworld_, jcsackett: http://people.canonical.com/~stevenk/PPA-change.png
<wgrant> mwhudson: 14192 is the number I gave him at 1:30am, so it's more likely to be wrong, but let me check.
<mwhudson> wgrant: i checked
<wgrant> Ah, good.
<wgrant> poolie: There may still be hope, then.
<wgrant> Aha, buildbot finished a couple of minutes ago.
<poolie> wgrant: oh for the fix to bug 878140 to get out in <24h?
<_mup_> Bug #878140: process-mail.py failed to resolve dns. Raised NXDOMAIN <dkim> <oops> <qa-untestable> <Launchpad itself:Fix Committed by mbp> < https://launchpad.net/bugs/878140 >
<poolie> wbn
<wgrant> Yes.
<poolie> no, wait, that one is older
<wgrant> The general Exception catcher that I reviewed yesterday, anyway.
<poolie> https://code.launchpad.net/~mbp/launchpad/881237-dkim-error/+merge/80413
<poolie> failed ec2 overnight because of some spurious timing related error :/
<wgrant> Ah
<wgrant> Fail, then.
<poolie> reminds me i need to resubmit and file a bug
<poolie>  File "/var/launchpad/test/lib/canonical/launchpad/utilities/ftests/test_gpghandler.py", line 173, in testHomeDirectoryJob
<poolie>    self.assertTrue(now <= floor(os.path.getmtime(fname)))
<wgrant> poolie: Ah, that lovely one.
<poolie> i haven't looked at it yet but i guess it's asserting something takes less than a second
<wgrant> Not quite. It's checking that the mtime of some files is current.
<wgrant> After they were set back 12 hours or so and a job run to bring them up to date.
<poolie> https://bugs.launchpad.net/launchpad/+bug/882324
<_mup_> Bug #882324: timing-dependent spurious test failure in testHomeDirectoryJob <Launchpad itself:Triaged> < https://launchpad.net/bugs/882324 >
<lifeless> poolie: why do you think its spurious ?
<poolie> oh, do you think my code broke it?
<poolie> maybe
<poolie> we'll see if it fails a second time
<poolie> anything comparing timestamps in a test looks pretty suspicious to me
<poolie> but of course some times they're valid
 * poolie looks
<wgrant> This one isn't valid.
<poolie> it obviously is
<poolie> it is essentially counting on all completing before the clock ticks
<poolie> s//obviously is dangerous
<wgrant> Yep.
<poolie> hm.
<poolie> actually i think it must be something more subtle than that
<poolie> - floating point rounding?
<poolie> - setting the directory mtime didn't do what we expected?
<poolie> -
<poolie> - different precision between time() and getmtime() causing different rounding behaviour?
<lifeless> poolie: well the now() is captured before the touching
<lifeless> poolie: and they are both floored
<poolie> mm
<poolie> so how can it fail?
<lifeless> I don't see any time sensitivity thing there, though floating point comparisons really do need a precision for safety
<poolie> - clock nomonotonicity on the test machine
<lifeless> poolie: as you say floating point precision may be the cause
<poolie> *non
<poolie> perhaps we should remove the 'floor' call on the rhs of the <=
<poolie> it does not seem to help anything and it may break something
<poolie> i might set up to run it repeatedly here and see if it ever fails
<lifeless> thats a good idea
<poolie> oh, could there be a symlink in that directory?
<poolie> i think you can't utime() them and that could account for it sometimes being old
#launchpad-dev 2011-10-27
<lifeless> there shouldn't be
<lifeless> its a gpg config dir
<poolie> gpg might make a symlink as a lock, but it looks like we're not actually running gpg in this test
<poolie> i've been running it for a while and it has not failed
<poolie> lifeless: how about https://code.launchpad.net/~mbp/launchpad/882324-mtime-test/+merge/80521 then
<lifeless> commented. Three times.
<poolie> you'd do it that way because it will automatically give a clear message?
<poolie> are you saying that you want me to keep the 'floor()' call, or do you want to have another attempt? ;-)
<lifeless> dropping the floor is fine
<lifeless> yes, it will automatically give a clearer message, and it eliminates another use of a poor primitive (assertTrue)
<poolie> great, thanks
<poolie> good idea in fact
<poolie> is there a link from the ppa to the recipe?
<poolie> i guess it's only off the branch
<StevenK> poolie: If an upload was created by a recipe, the package will link back to it
<StevenK> poolie: For example, https://launchpad.net/~launchpad/+archive/ppa/+packages and then expand any of the launchpad-dependancies
<poolie> ok thanks
<lifeless> it would be nice to link back recipes that are autobuilding separately from successful builds
<wgrant> lifeless: Huh?
<lifeless> auto builds build into specific ppas
<lifeless> if they haven't successfully built, there is no link from the ppa
<lifeless> the package has to build for the link to show up
<wgrant> Ah, daily builds?
<lifeless> yes
<lifeless> https://code.launchpad.net/~lifeless/python-oops-amqp/0.0.3/+merge/80524
<StevenK> lifeless: If the *recipe* built, but the packages resulting didn't, the link should still be there.
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
<lifeless> StevenK: indeed, I'm saying that waiting for that much success is still less clear than we could be
<lifeless> StevenK: can has review, as you are here?
<lifeless> or is it wgrants turn as wictum
 * lifeless consults the oracle
<lifeless> thursday. Its me!
<lifeless> -> bah
<lifeless> StevenK: wgrant: ^
<StevenK> Haha
 * wgrant blames wallyworld_.
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: me | Critical bugtasks: 266
<wallyworld_> huh?
<wgrant> poolie: https://staging.launchpad.net/~wgrant/+archive/experimental/+recipebuild/87804
<wgrant> bzr: ERROR: Cannot commit directly to a stacked branch in pre-2a formats. See https://bugs.launchpad.net/bzr/+bug/375013 for details.
<_mup_> Bug #375013: Cannot commit directly to a stacked branch <commit> <launchpad> <lp-translations> <qa-ok> <rodeo2011> <stacking> <Bazaar:Fix Released by jameinel> <bzr-builder:Fix Released by jelmer> <Launchpad itself:Fix Released by jtv> < https://launchpad.net/bugs/375013 >
<wgrant> Is that new?
<wgrant> I haven't used this recipe in more than a year, but it used to work...
<lifeless> looks like fallout from fixing commit to stacked branches in 2a; possibly due to a previously undocumented failure more
<lifeless> mode
<lifeless> bah https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2126AT10
<wgrant> lifeless: BranchRevision timeout...
<StevenK> wallyworld_: Can you update the kanban board? I think the cards in Review and Landing can move
 * wallyworld_ looks
<wallyworld_> StevenK: the one in landing, i'm fixing tests. makeBug, makeBugTask need a bit of work to make them compatible with the new disclosure rules
<lifeless> wgrant: looks like it wasn't safe for 1.9 either according to comments inline
<wgrant> :(
<lifeless> e.g. that the only reason things worked on 1.9 was it being a no-change commit
<lifeless> a change commit wouldn't be able to delta correctly in all cases in 1.9
<lifeless> I need a review - any volunteerS?
<lifeless> StevenK: can I beg a review off you?
<StevenK> lifeless: I suppose.
<lifeless> https://code.launchpad.net/~lifeless/python-oops-amqp/0.0.3/+merge/80524
<StevenK> lifeless: r=me
<lifeless> thanks
<poolie> (back)
<poolie> wgrant: pre-2a formats are kinda deprecated now
<poolie> i don't think that failure is very conclusive either way for qa
<jelmer> wgrant: is this about recipes using stacking and falling over when there is a pre-2a branch involved?
<wgrant> jelmer: Well, I was trying to QA the new bzr-builder, and my test recipe happened to fall into that category.
<jelmer> wgrant: that's not a new regression FWIW
<wgrant> k
<wgrant> Does someone have a less archaic recipe that they can reasonably test on staging?
<jelmer> wgrant: python-fastimport is one of my usual test recipes
<jelmer> I can have a look at adding that on staging, but not before dinner...
<wgrant> Ah, you're still in the US?
<wgrant> StevenK: Yay, p-d-r back down to 8 minutes.
<wgrant> Still sucks, but a bit better.
<jelmer> wgrant: yeah
<poolie> o/ jelmer
<poolie> wgrant:  i can try mine on bzr
<poolie> both should be modern formats
<wgrant> poolie: That works.
<poolie> ok doing it now
<wgrant> Thanks.
<poolie> ok https://code.staging.launchpad.net/~mbp/+recipe/bzr-daily exists
<poolie> queued for build
<poolie> building
<StevenK> wgrant: 8 minutes sounds pretty good to me.
<wgrant> StevenK: It is LP good.
<wgrant> It is not good.
<StevenK> wgrant: Pretty good compared to the 1.5 hour transactions we were having
<wgrant> It's considering less than 1500 items. There's no reason for it to take more than 30 seconds, and that's being generous.
<poolie> what's p-d-r?
<StevenK> process-death-row
<StevenK> Deletes superseded publications
<poolie> failed: https://staging.launchpadlibrarian.net/80692680/buildlog.txt.gz
<wgrant> It's the pool cleaner.
<poolie> i guess because of dpkg-deb: error: control directory has bad permissions 700 (must be >=0755 and <=0775)
<wgrant> poolie: sudo breakage
<wgrant> lamooooont
<wgrant> lamont: ^^
<poolie> which control directory is it talking about? the ./debian in the build area?
<wgrant> probably ./DEBIAN.
<lifeless> wgrant: that yuixhr thing is still booming
<wgrant> lifeless: :/
<lifeless> I found a think in my OopsHandler change
<lifeless> (is not None) s/not // to fix
<lifeless> thinko
 * StevenK kicks Person:+index for being stupid
<lifeless> anyhow, that *might* be it, if yuixhr is logging warnings or something
<lifeless> oh ugh
<poolie> has anyone seen a test fail with "error: pid file was not removed"?
<lifeless> so I just realised, if we want different vhosts for txlongpoll, I'll need separate config for oops-amqp
<lifeless> poolie: nope
<lifeless> wgrant: do you happen to know an ETA on txlongpoll being unbroken ?
<poolie> i see muharem hit it in 2009
<poolie> just bad luck i guess
<StevenK> wgrant: So, where should I start digging for this +index madness?
<lifeless> StevenK: queries ?
<StevenK> lifeless: Hm?
<lifeless> StevenK: what sort of madness
<StevenK> lifeless: create_initialized_view(<person>, '+index') doesn't work
<lifeless> StevenK: it does too
<lifeless> see TestPersonIndexView
<wgrant> Does that render the view?
<lifeless> oh, rendering doesn't work?
<wgrant> Doesn't seem to.
<lifeless> thats odd
<lifeless> uhm, there are additional parameters
<poolie> well https://bugs.launchpad.net/launchpad/+bug/882370
<_mup_> Bug #882370: spurious failure in pidfile.txt: "pid file was not removed" <Launchpad itself:Triaged> < https://launchpad.net/bugs/882370 >
<lifeless> poolie: again, what makes this spurious? Do you know that the thing actually stopped and didn't get lost?
<mwhudson> wgrant: didn't you work on ddebs for ppas a bit ages ago?
<poolie> what does "actually stopped and didn't get lost" mean?
<lifeless> poolie: we have a recurring issue with test helpers going awol and not shutting down
<wgrant> mwhudson: I did.
<mwhudson> wgrant: did that ever get anywhere?
<lifeless> a pid file not being deleted is often also a symptom
<wgrant> mwhudson: For PPAs? It's believe to be finished, with a small fix Julian made last week.
<mwhudson> wgrant: also if i say "ddebs" and "derived distros" in the same sentence, will the result be hollow laughter?
<mwhudson> wgrant: oh cool
<wgrant> mwhudson: Not fully tested, and can't be at the moment, because DF is screwed.
<wgrant> Ah.
<wgrant> Primary archives are another matter.
<wgrant> They have an implementation constraint that I consider to be false.
<poolie> perhaps we have different meanings of 'spurious'
<wgrant> A very onerous implementation constraint.
<mwhudson> o
<mwhudson> wgrant: i'm looking at https://blueprints.launchpad.net/linaro-ubuntu/+spec/linaro-platforms-lc4.11-ubuntu-leb-dbg-pkgs-support you see
<poolie> these tests failed when i didn't (afaik) touch anything related to that test
<wgrant> It is said that ddebs must live in a separate archive.
<poolie> and they're not consistently failing
<lifeless> poolie: that still fits the symptoms we've experienced.
<wgrant> mwhudson: Well, the Soyuz team can probably fix that oh wait.
<mwhudson> wgrant: ok
<StevenK> Now now
<poolie> are you saying this is not a bug, or are you saying it's a dupe of a known bug?
<lifeless> poolie: that said, the test helper has timing code all over it
<wgrant> mwhudson: So, PPA ddebs are probably a couple of months away.
<lifeless> poolie: I was investigating to assess that, yes.
<wgrant> mwhudson: Primary archive ddebs are an undefined period of time and convincement of IS and Julian away.
<mwhudson> wgrant: thanks
<mwhudson> wgrant: is there a LEP or anything for PPA ddebs?
<poolie> so on the face of it, it seems spurious that this test would fail based off this change
<poolie> imbw
<mwhudson> or anything i can link to, for that matter
<lifeless> poolie: oh, I don't think your change provoked it
<poolie> i can't see any known bug about this test
<poolie> are you saying you didn't want me to file a bug?
<wgrant> Bug #747558
<_mup_> Bug #747558: PPAs should create backtracable packages <escalated> <ppa> <qa-ok> <Launchpad itself:In Progress by julian-edwards> < https://launchpad.net/bugs/747558 >
<lifeless> poolie: the question was whether it was related to the test reliability issues we've been having on and off, or a different low-frequency issue all of its own.
<lifeless> poolie: I think its all of its own and I've commented in the bug
<lifeless> poolie: filing the bug was the right thing to do
<poolie> any test that has the line 'time.sleep(0.1)' is prima facie a critical bug ;)
<lifeless> poolie: tests that break in buildbot and cause 4 to 8 hour team stalls are, and for all that its low frequency this may well fit that
<poolie> yep
<poolie> this bug and the previous one caused ~5h stalls for me
<poolie> if it had happened in bb it could have been worse of course
<StevenK> lifeless: So, initializing the view gives the output as ""
<mwhudson> wgrant: i just read your explanation of how ddebs.ubuntu.com works
<poolie> lifeless: i wonder if AWS is having some kind of underlying issue that is provoking more timing bugs
<wgrant> StevenK: No, *rendering* the view does.
<StevenK> lifeless: wgrant thinks that there is METAL madness involved.
<wgrant> mwhudson: Sorry, I should have warned you.
<wgrant> StevenK: sinzui's theory is possibly more relevant for this view.
<poolie> eg something in the virtualization layer is making the machine clock irregular
<mwhudson> wgrant: yeah, i may not sleep properly tonight
<lifeless> mwhudson: :)
<poolie> two failures in a row seems suspicuous
<lifeless> poolie: that would be epic win wouldn't it ;)
<poolie> or, perhaps they're just bogging down sometimes
<poolie> eg this test waits only 2s for the thing to complete
<poolie> or possibly even less
<poolie> it's pretty plausible you could get >2s lag on a very loaded host
<lifeless> I need a rabbit sanity check
<lifeless> is it *possible* for rabbit to buffer messages in an exchange and send them to a queue that is subsequently bound to it ?
<StevenK> wgrant: My brain has melted, what was his theory?
<wgrant> lifeless: I imagine that such a race would be possible in rare circumstances.
<lifeless> wgrant: I have one nonreproducable failure.
<wgrant> StevenK: XRDS
<lifeless> wgrant: test_rosteta_branches_script_oops gets 7 oopses included some with storm DisconnectionError and similar in it
<lifeless> wgrant: my only theory is that having the exchange with -no- queue at all (e.g. tests in a too-low-layer) actually buffers
<StevenK> wgrant: eXtensible Resource Descriptor Sequence? Srsly?
<wgrant> StevenK: Correct.
<poolie> ew doctests
<StevenK> wgrant: So where do I start?
<wgrant> StevenK: Work out what's returning ""
<lifeless> http://www.rabbitmq.com/tutorials/tutorial-three-python.html says 'The messages will be lost if no queue is bound to the exchange yet'
<lifeless> so that rules out that scenario
<wgrant> It doesn't.
<lifeless> wgrant: how so ? stale appserver helper s?
<StevenK> wgrant: So create_initialized_view() ; view() should be enough in the usual case, right?
<wgrant> 1) It's a tutorial.
<lifeless> wgrant: on the rabbitmq site
<wgrant> 2) It probably just means "You can't rely on messages not being lost"
<wgrant> StevenK: Yes.
<lifeless> I find reading what is written usually works.
<poolie> lifeless:  i realize it's intermittent but it offends me
<wgrant> lifeless: Not in a tutorial, and not from rabbitmq.
<poolie> shall i just bump it up to waiting 20s?
<lifeless> poolie: I take it you also cannot see a cause
<poolie> i've looked for races and i don't see any
<lifeless> sounds like a (painful) workaround
<poolie> the fact that it's been there for >2 years but hit only intermittently suggests it timing related
<lifeless> agreed
<poolie> especially as i  had another perhaps timing related thing today
<poolie> i was wondering about making it also report if the process still exists
<lifeless> that might help with diagnostics
<StevenK> wgrant: I think lib/canonical/launchpad/webapp/publisher.py, line 296 is to blam.
<StevenK> *blame.
<StevenK> "Oh, Person:+index is a redirect, return u''"
<lifeless> StevenK: you may need to pass in a layer (not test layer)
<wgrant> StevenK: pdb will tell you...
<wgrant> But I doubt it.
<lifeless> StevenK: this is one of the extra params IIRC. Or something.
<StevenK> wgrant: I discovered it because of pdf
<StevenK> *pdb
<StevenK> I've been tracing for a while
<StevenK> Hm, or not.
<StevenK> return self.render()
<wgrant> It's probably feeds or XRDS interacting badly, but step through in pdb and find out.
<poolie> lifeless: https://code.launchpad.net/~mbp/launchpad/882370-pidfile/+merge/80536
<poolie> :/
<poolie> not the most satisfying series of patches
<poolie> i'm going to run some errands
<StevenK> wgrant: I'm in the middle of zope.tales, so it must be doing useful stuff
<wgrant> Indeed.
<wgrant> Which suggests that it may in fact be METAL.
<StevenK>         if current_url != expected_url:
<StevenK>             self.request.response.redirect(expected_url)
<StevenK>             return ''
<wgrant> lol
<wgrant> I found that within 5 seconds of you.
<StevenK> In lib/lp/services/openid/browser/openiddiscovery.py
<wgrant> Yes.
<wgrant> Was just switching over to paste that same thing.
<StevenK> Haha
<wgrant> Although I was 3 seconds later than you :(
<StevenK> I'm on my laptop due to my desktop being busy with this evil program called dd, so I was a little slower than pasting
<wgrant> Heh
<wgrant> So, that would do it.
<StevenK> Indeed.
<StevenK> Current URL: http://127.0.0.1/
<StevenK> Expected URL: http://launchpad.dev/~team-name-150898
<StevenK> Strangfe
<StevenK> s/f//
<wgrant> Right, the fake request (LaunchpadTestRequest, probably) used by create_initialized_view doesn't have a sensible URL.
<wgrant> Because most things are sensible enough to not require one.
<StevenK> Right, I was just thinking that LTR is probably implicated.
<StevenK> Can't we create a request and toss that into create_initialized_view()?
<StevenK> I have this feeling I've seen that done. *Where* ...
<wgrant> You could also change create_initialized_view to set the URL to canonical_url(obj, view_name)
<StevenK> create_initialized_view takes a server_url param
<poolie> lifeless: thanks
<lifeless> wgrant: I'm not going to move the lp_sitecustomise line; I worry its order dependent because its doing monkey patching.
<lifeless> wgrant: or am I wrong ?
<lifeless> wgrant: ah, I'm wrong.
<wgrant> lifeless: I thought that as well, but it's in the sort of file where that would surely not be well-placed.
<StevenK> RARGH
<StevenK> Current URL: http://launchpad.dev/~team-name-333852/
<StevenK> Expected URL: http://launchpad.dev/~team-name-333852
<StevenK> They both call canonical_url, why does one have a slash and the other doesn't ... :-(
<wgrant> The / will be because you asked for the +index view.
<wgrant> I suspect the other one just asks for the object.
<StevenK>     if not server_url:
<StevenK>         server_url = canonical_url(context)
<StevenK>         expected_url = canonical_url(self.context)
<StevenK> If both contexts are the same, the URLs should be equal
<lifeless> wgrant: I've just pushed rev 14150 if you want to do an incremental review. I've added autosync to the attachOopses() call, which will hopefully result in these naughty oopses being sucked up earlier.
<lifeless> wgrant: I've added de-duplication of oops ids to accomodate that
<wgrant> lifeless: I considered asking for that, but it sounds like it could be slow.
<wgrant> Although I guess it shouldn't be.
<lifeless> wgrant: if it passes ec2, I should have some timing data
<lifeless> wgrant: so we can assess, and we'll know
<wgrant> lifeless: To an extent.
<wgrant> But yes.
<wgrant> Wow.
<wgrant> The security declarations for Bug are... special.
<wgrant> Notably, there are some attributes listed that I've never heard of.
<wgrant> And they are in no order whatsoever.
<jtv> Any reviewers in the house?  https://code.launchpad.net/~jtv/launchpad/pre-876594/+merge/80537
<nigelb> "me" is new.
<nigelb> hah, ok, lifeless.
<jtv> Thanks for figuring that out, Nigel.  :)
<jtv> lifeless: could I trouble you for a review?  It's this one: https://code.launchpad.net/~jtv/launchpad/pre-876594/+merge/80537
<jtv> Not very exciting stuff, I'm afraid.
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
<lifeless> sure
<jtv> You had me there for a moment.
<nigelb> heh
<lifeless> well, its 8pm, well past EOD
<lifeless> even for extended-batteries-lifeless
<lifeless> I'd forgotten to update the topic
<lifeless> jtv: done
<jtv> thanks lifeless
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (allenap) | Critical bugtasks: 266
<jtv> morning rvba
<rvba> Hey jtv.
<jtv> rvba: I didn't make very much progress on your query, except to find a sort of rock-bottom reason why it's slow.
<jtv> See email.
<rvba> Saw your email, with many more details, your reach the same conclusion I did: rows=336922, that's a huge number.
<jtv> You may be wondering why I said it's not worth counting if the owner owns as little 1% of the branches.
<rvba> So I suppose the comment that lifeless put on the bug itself (that the is a way to use StormRangeFactory (tweaking involved)) is the only way to go.
<rvba> jtv: Yes I do.
<jtv> It's because with 1% of the branches, you could still end up touching a large portion of the table pages.
<jtv> Worst case, you need to touch 1 branch per page.
<jtv> If you can skip the count, great.
<rvba> Well, using StormRangeFactory would avoid the count, I suspect the perf of the page would be dramatically improved.
<jtv> Probably, yes.
<jtv> You'd also be rid of the OFFSET.
<jtv> Which could matter if we just grew a few hundred thousand branches recently.
<lifeless> private branches will be getting overhauled by disclosure
<lifeless> I suggest talking to them about the schema changes planned - the schema drives performance here in fairly significant ways
<jtv> The 300K branches were private on dogfood but public on staging.  They're only a problem when they're public, but I assumed they'd be public in production.
<jtv> The private-branches part isn't much of a performance problem, apart from a lot of repetitive querying.
<rvba> All the explains I've done are on production.
<jtv> So this isn't tied to branch privacy.
<lifeless> jtv: I did a previous optimisation pass in june or so, because its a problem :)
<lifeless> jtv: but if you've newer data than I, great.
<jtv> Not for what rvba is looking at.  In other words, you probably solved the problem as far as private branches are concerned and we're left with public branches as the bottleneck.
<rvba> The small improvement I did with counting private branch (simply caching the results) will help a little.
<rvba> But like jtv said, the main problem is counting the public branches.
<jtv> It got a bit faster when I omitted the "transitively_private is false" condition from the public-branch count,
<jtv> but still not as fast as counting public and private branches separately & adding up the numbers afterwards.
<lifeless> rvba: just a style thing - I'd hesitate to use a cached property on GenericBranchCollection
<lifeless> rvba: I suspect it could easily lead to incorrect behaviour
<lifeless> rvba: its really a fancy query builder; any caching should be done on the resulting query
<rvba> lifeless: my goal was to cache the results of a subquery that is repeated more than 10 times.
<lifeless> rvba: why is it repeated 10 times?
<lifeless> rvba: also are you aware that passing in long IN clauses can trigger inappropriate sequential scans ?
<lifeless> rvba: [so if you expect thousands of entries in an IN clause you may need to move to a temp table, insert, analyze, then query]
<lifeless> rvba: I certainly agree with jtv's suggestion that a union may be better here
<rvba> Right, I still need to make sure that the worse that can happen with the IN query is a few thousands.
<rvba> I've not used a with query because I wanted to improve lots of queries at once.
<lifeless> as a data point, the last time I touched this code I fixed one page and broke another
<lifeless> qa on this will be tricky
<lifeless> ah, ubuntu-branches, \o/
<rvba> And I confess I'm not sure why I have so many different queries using this subquery, I must say that this code is a little bit spaghetti like, but anyway, this was just a first attempt I made before jtv & I realised this was really not where the improvement should be done.
<lifeless> k
<lifeless> so the plan issue is the hashed SubPlan 1, I think
<rvba> I think the potential problems of this IN query optimisation are probably not worth the improvement it brings â¦ I shall see if stormrangefactory can be applied.
<lifeless> select count(*) from branch where owner=2866082;
<lifeless>  count
<lifeless> --------
<lifeless>  338016
<lifeless> (1 row)
<lifeless> Time: 284.695 ms
<lifeless> 343ms with the lifecycle status
<rvba> i've only touched the surface of if so far, but this stormrangefactory is a very, very nice piece of work I must say.
<lifeless> this is consistent with my previous memory
<jtv> That's a lot faster than on the test servers.
<lifeless> the public branch is not the issue
<rvba> That's a typical 1s query that the page is doing:
<lifeless> 710ms with the transitively_private check added, which is odd.
<rvba> https://pastebin.canonical.com/54885/
<rvba> The related explain: https://pastebin.canonical.com/54886/
<rvba> I suspect if you add the lifecycle_status check it will take 1sec.
<lifeless> 743ms with owner + lifecycle_status + transitively_priate
<jtv> lifeless: the slowdown from the privacy check may be something to do with access patterns within the page.  Do you know off the top of your head where a tuple's visibility-related info is stored?  Maybe it's a compact blob at the beginning of the page.
<rvba> I said typical because like I said earlier, many similar count queries are generated by this page (https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2124AO73)
<lifeless> jtv: you thinking a repack or something ?
<jtv> Wasn't, but always a nice thought.  :)
<jtv> Oh, I think I know.
<lifeless> jtv: I don't know that bit of the layout, no.
<lifeless> checking private (the old column) isn't visibly faster
<adeuring> good morning
<rvba> Morning adeuring!
<lifeless> 480k branches in total
<adeuring> hi rvba!
<lifeless> so this is getting 66% or so of pages
<jtv> lifeless: IIRC there's a shortcut for determining that all tuples in a page are visible.  We won't make full use of that without index-only scans, but it probably still means that the liveness check has better locality than actual tuple data check.
<jtv> hi adeuring
<adeuring> hi jtv
<jtv> Once you start looking at the actual data, you do need to look at the tuple's data.
<jtv> Until then, just visibility metadata.
<lifeless> anyhow, given we're getting 2/3rds of the data, the seq scan is appropriate
<jtv> Absolutely.
<jtv> Even for a few % of the data it would be appropriate.
<jtv> Why aren't we ever discussing clustering?
<jtv> Not that I know what to cluster _on_â¦
<lifeless> well, partly because it needs downtime :)
<lifeless> also its not maintained by vacuum
<jtv> Why does it need downtime?
<lifeless> and the online adding scares me ;)
<lifeless> jtv: 'When a table is being clustered, an ACCESS EXCLUSIVE lock is acquired on it. This prevents any other database operations (both reads and writes) from operating on the table until the CLUSTER is finished. '
<lifeless> http://www.postgresql.org/docs/current/static/sql-cluster.html
<jtv> That's no good.  :/
<jtv> I thought vacuum maintained clustering, if you assigned a clustering index.
<lifeless> nope
<lifeless> well, let me be precise
<lifeless> I know that autovacuum does not
<lifeless> I believe that vacuum also does not. IMBW on this one.
<lifeless> rvba: so looking at that oops
<lifeless> slowest query is the BMP one
<lifeless> second slowest is is the count of branches
<lifeless> third is the retreival of the branches batch
<rvba> Yes, again, a count(*), and you can see the repeated In subquery, hence my first idea to cache the result of this subquery.
<lifeless> rvba: yes, I get that... note though that that the fact you're querying in against so many public branches will reduce that impact ;)
<lifeless> rvba: which you already know
<lifeless> rvba: so I suggest a different tack on the analysis
<lifeless> rvba: lets ask what would be the fastest way to implement the slowest part of the page
<lifeless> rvba: and then fix that
<lifeless> rvba: (one fast way is to not do something at all)
<lifeless> rvba: the branch mergeproposal count isn't in a batch
<lifeless> rvba: and we'll cut 3.7 seconds out if we stop showing it; 1.8 if we get a 50% improvement there.
<rvba> Note that the total sql time is ~10s so we have a long way to go ;)
<lifeless> rvba: fixing 30% of that would be a pretty good start :)
<rvba> I suppose that there is one count for the batch, and the rest is to display the numbers in the right hand side portlet.
<rvba> That is a costly portlet :)
<lifeless> working page: https://code.launchpad.net/~rvb/+branches
<lifeless> rvba: yes, note that https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2124AO73#longstatements top-N 5
<rvba> Yes, that's precisely the one I'm looking at.
<lifeless> 5. 	948.0 	1 	SQL-main-master 	
<lifeless> SELECT BranchMergeProposal.commit_message,
<lifeless>        BranchMergeProposal.date_created,
<lifeless>        BranchMergeProposal.date_merged,
<lifeless>        BranchMergeProposal.date...
<lifeless> that seems total waste
<lifeless> as the reviews are not show.
<lifeless> so you can save 1 second by fixing htat bug.
<lifeless> work not done is cheap :)
<nigelb> I never knew there would be a day I'd be reviewing danilos' code :P
<rvba> lifeless: Very true :)
<lifeless> rvba: I have to wonder if showing a count of subscribed branches is needed
<lifeless> rvba: e.g. perhaps we shouldn't show random users how many branches someone is subscribed too, nor what they are subscribed to. Perhaps we should.
<lifeless> its consistent with what we do to bugs, so I wouldn't arbitrarily change it
<rvba> Also, maybe the numbers are not that important, but maybe the links are for some workflows, maybe we could remove the numbers and keep the links.
<rvba> Lots of 'maybe' :)
<lifeless> contrast with https://bugs.launchpad.net/~rvb/
<lifeless> no numbers there
<rvba> Right.
<lifeless> that is perhaps a sensible first step; get us some headroom
<lifeless> more consistent with the bugs application
<bigjools> morning all
<nigelb> Morning bigjools
<nigelb> *snicker* English cricket team.
<rvba> lifeless: That's a good idea, and an easy fix :). Should we run this past someone first?
<StevenK> English "cricket" team.
<bigjools> nigelb: you mean the #1 test team?
<lifeless> rvba: up to you :) - I'm one of the people it would be run past :P; secondly you'll need to convince a reviewer that its reasonable, and they can always suggest you seek UI discussion too
<nigelb> bigjools: Yeah, the one that didn't win a single match :P
<bigjools> nigelb: that'd be you guys over here then?
<nigelb> haha
<bigjools> ;)
<nigelb> Apparently our respective teams perform only in their home country.
<rvba> lifeless: ok, I think the argument will depend on the perf improvement it brings, I'll do it, have it reviewed, and the will we evaluate the benefit it brings on staging. As this time we could have the UI vs. performance arguement
<nigelb> (Not like I watched either set of matches)
<lifeless> rvba: you can feature flag it
<lifeless> rvba: makes the decision to flip the flag separate to the coding aspect
<rvba> lifeless: Good idea, this way we can test it on production.
<bigjools> nigelb: yes, for someone who doesn't like cricket, you seem to spend a lot of time talking about it :)
<nigelb> bigjools: I follow StevenK's principle about cricket :D
<lifeless> allenap: you are added now
<allenap> lifeless: Thanks, I saw that, and uploaded 0.0.6 last night.
<lifeless> cool
<nigelb> rvba: Congrats!
<rvba> nigelb: thanks ;).
<rvba> nigelb: brutal reviewsâ¦?
<nigelb> rvba: where you nitpick every line.
<rvba> rvba: There are a few "fascists reviewers" out there.  But I'm not one of them ;).
<nigelb> rvba: You should be.
<nigelb> :)
<rvba> I follow another way to do things, no brutality involved :)
<nigelb> hehehe
<rvba> It's called the Panella style.
<nigelb> haha
<rvba> :)
<nigelb> I'm too used to the jv/fascist style.
<bigjools> the best reviews are the ones that make me think about what I am actually doing
<nigelb> A lot of times I see my code evolve from "this works" to "this works and looks pretty neat as well" during code review.
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba | Critical bugtasks: 266
<bigjools> :)
<rvba> Yeah!
<nigelb> \o/
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba, jcsackett | Critical bugtasks: 266
<rvba> Morning jcsackett.
<jcsackett> morning, rvba. thought you were on monday shifts now?
<rvba> jcsackett: yeah, but I'm finishing today's shift since I'll be off on Monday.
<jcsackett> Ah, dig.
<deryck> Morning, all.
<jcsackett> Morning, deryck.
<deryck> rvba or jcsackett -- I have a js branch for review if either of you care to review it.
<rvba> deryck: sure.
<deryck> rvba, thanks!  https://code.launchpad.net/~deryck/launchpad/buglists-orderby-widget/+merge/80563
<flacoste> morning deryck!
<rvba> deryck: just checking: the code in the demo is in sync with the code in the MP?
<deryck> rvba, on call, sorry, just a sec.
<deryck> rvba, ok, off now, sorry.  by demo do you mean the html stuff at people.canonical.com?
<rvba> deryck: I'm asking because if I click on 'Importance' then on 'Bug heat' and then repeat the operation, I get this: http://people.canonical.com/~rvb/big_button.png. Looks a space somewhere is not removed.
<rvba> deryck: yes
<rvba> s/Looks/Looks like/
<deryck> rvba, yeah, so the demo version was just a proof to get the interaction down.  The widget doesn't copy it exactly, and that particular issue should be fixed in the widget.
<rvba> deryck: Ok.
<deryck> rvba, it's due to popping a span on the li for the arrows, and in the final version you're reviewing the span is always there.
<rvba> deryck: Right, I saw the code was not exactly the same but if you're aware of the issue in the demo that's fine.
<deryck> rvba, gotcha.
<flacoste> rvba: congratulations on achieving reviewerhood!
<rvba> Thanks flacoste!
<deryck> rvba, thanks for the review!  I've no issues with anything you mention and will take your suggestions up before landing.
<rvba> Cool.
<deryck> oh rvba, I did like the suggestion for the tearDown destroy, but the one place it's not used is not needed because that's an error test.  The render never completes.
<rvba> deryck: ah ok.
<rvba> deryck: I also suggested that because of http://bazaar.launchpad.net/~deryck/launchpad/buglists-orderby-widget/revision/14212
<deryck> rvba, yup, it's a good suggestion.
<cr3> hi folks, what's the difference between documentation under doc/ in the source tree and dev.launchpad.net?
<sinzui> jcsackett, pinf
<sinzui> jcsackett, ping
<jcsackett> sinzui: ponf/g.
<sinzui> jcsackett, I have a data fixing script that needs review. I want to talk to you about it since it is not a branch
<sinzui> mumble?
<jcsackett> sure, one moment.
<bigjools> cr3: hi
<bigjools> cr3: the stuff under doc/ is generated and shown at readthedocs. I'm currently looking into sorting the documentation mess out, so bear with me.
<cr3> bigjools: what's "readthedocs"?
<bigjools> cr3: http://launchpad.readthedocs.org/en/latest/
<sinzui> jcsackett, http://pastebin.ubuntu.com/720766/ and http://pastebin.ubuntu.com/720768/
<cr3> bigjools: I'm adding documentation to the launchpad-results project which tries to follow launchpad practices as closely as possible, any suggestions for me?
<bigjools> cr3: not really :(  I've not started sorting this out yet, it's a fledgling initiative. My main aim is to get low-level API info in the Sphinx stuff so that it lives with the code, and the wiki should contain more higher-level stuff.
<cr3> bigjools: no worries, since I won't have much documentation for a while, it shouldn't be too difficult to migrate later
<bigjools> cr3: I'd use reST markup as well
<bigjools> it's rather flexible
<cr3> bigjools: I thought about that but then I figured that using text under doc/ the same as the doctests throughout the code might be beneficial, rather than having two formats
<bigjools> cr3: doctests suck
<bigjools> cr3: the stuff under doc/ is reST
<cr3> bigjools: I know, I know, but some might be useful. I think the problem is that launchpad has been using them too much and for the wrong purpose, like unit testing in doctests, but I'm not convinced they're all bad
<bigjools> cr3: maybe - LP's use is teh suck for sure, but they can get unwieldy very fast IME
<cr3> bigjools: so, ultimately, the objective would be to have no doctests at all?
<bigjools> cr3: that's the majority consensus in the LP team, yes
<jcsackett> sinzui: r=me on the sql, looks good.
<sinzui> thank you
<jcsackett> sinzui: one note on the script. there are two exceptions in which you log, but since those are both errors don't you want to pass in error=true to the log function?
<sinzui> Oh
<sinzui> jcsackett,  no for unauthorized, yes for the WTF !!
<jcsackett> I figured a comment like "Something went very wrong" implied error logging should be enabled. :-P
<sinzui> jcsackett, Since I did not get any !! in the log, I did not catch that I needed to report the super error
<sinzui> fixed
<jcsackett> sinzui: otherwise, this looks good to me. r=jcsackett.
<sinzui> thank you
<flacoste> bigjools: well, there's a disagreement in the sense that some people (at least stub) argues that documentation should be tested
<flacoste> so a kind of doctest (even if doctest tool isn't used) could be used
<flacoste> i think manuel was suggested
<flacoste> as it integrates with sphinx
<bigjools> flacoste: well he was the lone dissenter, hence my phrase "majority consensus"
<bigjools> flacoste: right - I need to investigate all this manuel/sphinx etc
<bigjools> abentley: hey do you have a brief moment to talk about canonistack + LP?
<abentley> bigjools: sure.
<bigjools> abentley: I've got it such that I have the same 9 tests failing every time, and they are a bunch of them in test_lpserve
<bigjools> because bzr is outputting:
<bigjools> bzr: warning: unsupported locale setting\n'
<bigjools> If I can figure out why it does that, we've got a winner
<abentley> bigjools: I'm not optimistic about using openstack for running tests, because they still haven't fixed my RT about instances not starting properly.
<bigjools> right - I've kept the same one up for a week :)
<abentley> bigjools: But I would imagine the complaint about locale setting is related to LANG.
<bigjools> abentley: yes - I've tried setting it to en_US.UTF8 and I get the same error
<abentley> bigjools: And it fails when you run the test by itself?
<bigjools> abentley: tell a lie - it's working now
<bigjools> only one test failing now!
<abentley> bigjools: Let's hope it stays that way :-)
<bigjools> abentley: http://pastebin.ubuntu.com/720814/
<abentley> bigjools: Since it's in an assertRaises, I guess it's raising the wrong exception?
<rvba> bigjools: I remember having this one failing too!
<rvba> abentley: exactly
<bigjools> which is odd
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugtasks: 266
<flacoste> deryck: yesterday on the TL call, in the feature checkpoint when i mentioned that you guys were going to look at multi-sort next
<flacoste> lifeless pointed out that you might hit some performance problems
<flacoste> as all our bug queries are optimized including the sort order
<flacoste> you might encounter case that don't perform well
<flacoste> so it's something you'll have to be wary of
<deryck> flacoste, ah, ok.  Good to know.  adeuring, see ^^
<adeuring> deryck, flacoste: yeah, that's what I was thinking too...
<deryck> abentley, ok, I have my head around your branch now.  Can we dialog about this review now?
<elmo> is there a tool for caching Launchpad bugs for offline use?
<abentley> deryck: I'm free when you are.
<deryck> abentley, ok.  Let's mumble to make it quicker.
<abentley> deryck: sure.
<flacoste> elmo: i think rick spencer had a LP bugs GUI at some point that was doing something like that
<jelmer> flacoste, elmo: lp:bughugger
<flacoste> yep, that sounds like a rick_spencer project name!
<jelmer> that's Rick's LP bugs GUI, not sure how much offline support it has though
<flacoste> thanks jelmer
<jelmer> flacoste: :)
<flacoste> i know it was downloading the bugs api representation in a local cache and then was operating the gui from that
<nigelb> flacoste: brian murray had something like that. Maybe its the same project.
<elmo> flacoste, jelmer: thanks - that's trying to install desktopcouch and stuff and looks dead upstream
<elmo> I guess it'll be easy to do a dumb cache with launchpadlib
<flacoste> desktopcouch!
<flacoste> wow, yeah, that's dead
<flacoste> the hug of death
<nigelb> desktopcouch is dead?
<nigelb> Doesn't Ubuntu use it in a lot of places?
<nigelb> Or is it couchdb.
<nigelb> elmo: james_w had some API demo that did some offline stuff.
<nigelb> At Belgium.
<james_w> I assume that elmo wants something that works
<nigelb> lol.
<nigelb> james_w: A common feature of your lightning talks :P
<elmo> nigelb: desktopcouch may or may not be dead - I just don't want (any more of) it installed on my machine if I can avoid it.  my 'dead' comment referred to the fact there haven't been any code changes to bughugger in almost a year
<sinzui> jcsackett, ping
<nigelb> elmo: Ah, right. Its unloved.
<elmo>  The Launchpad API is currently in beta, and may well change in ways
<elmo>  incompatible with this library.
<elmo> is that still true?
<deryck> elmo, depends on which version of the api you're using.  there is a beta, a 1.0, and a devel.
<deryck> or maybe I just inserted myself in a conversation without context. :)
<elmo> well, that comes from the package description for python-launchpadlib
<deryck> ah
<deryck> I would guess launchpadlib uses the 1.0 api now.
<jcsackett> sinzui: sorry, didn't see your highlight an hour ago.
<sinzui> jcsackett, I reported https://bugs.launchpad.net/launchpad/+bug/882714 and attached the script that I have used.
<sinzui> jcsackett, please review the script, I want an admin to run it once to retire it
<jcsackett> sinzui: happily. :-)
<jcsackett> sinzui: big script, eh? may take a bit.
<sinzui> jcsackett, you do not need to know every project in the dict. that is what I collected in the evening reviewing projects one January
<jcsackett> yeah, i just realized that the dicts could be skipped.
<jcsackett> quite a bit shorter without that. :-P
<jcsackett> sinzui: you say you've been using this script before?
<flacoste> deryck: yes, it does, we should update the package description
<flacoste> or the Ubuntu maintainer should
<sinzui> jcsackett, I run it once a month
<jcsackett> dig.
<jcsackett> sinzui: i see nothing i can add about or too this script, looks good.
<jcsackett> r=me, sinzui.
<sinzui> it fixes about 1 project every other run because the user gave ~registry the project
<deryck> abentley, I have bug listings reordering now.  it all works with very few lines changed.
<deryck> abentley, http://pastebin.ubuntu.com/721063/
<abentley> deryck: Awesome!
<benji> hi jcsackett, do you have time to review this for me?  https://code.launchpad.net/~benji/launchpad/bug-877195-code/+merge/80617
<jcsackett> benji: sure thing.
<benji> cool
<deryck> abentley, I still need to setup the widget correctly, to match actual sort orders we use.  And then some css and we're good.
<abentley> deryck: Cool.  I've got the polish basically done.  It now updates the batch info "1 â 5 of 256", the actions are green if active, and the actions are greyed out if not applicable.
<deryck> abentley, very nice.
<jcsackett> benji: just checking, is there any reason you didn't set the pre-req branch as the pre-req in the MP?
<benji> jcsackett: heh, I just realized that I should do that
<jcsackett> :-)
<jcsackett> cool, if you could resubmit with that i'll pick up that MP, so i know what i should be looking at. :-)
<jcsackett> benji: i've got it now. thanks.
<benji> k
<mwhudson> so the launchpad thing that kills idle transactions... is that reusable at all?
<lifeless> pgkillidle
<lifeless> yes
<lifeless> isd and u1 use it already
<lifeless> should ust be a matter of asking losa to configure it up
<poolie> hi all
<benji> morning, lifeless; I left a present on your doorstep (or stubs, but I figure he's somewhat distracted right now): https://code.launchpad.net/~benji/launchpad/bug-877195-db/+merge/80618
<lifeless> benji: you're a good cat. Miaow.
<benji> :)
<lifeless> benji: what are lines 54 and 55 for ?
<lifeless> benji: or is it just a confusing diff ?
<benji> lifeless: they are to make something unrelated to my change stop barfing.  when I run "cronscripts/run_jobs.py pofile_stats" without those lines I get exceptions about them being missing
<lifeless> benji: if you copy lines 61 and 62 up, then bzr may anchor the diff better
<lifeless> its a confusing diff basically.
<mwhudson> lifeless: ta
<benji> lifeless: will do
<lifeless> benji: more importantly, you need to bzr add your schema change
<benji> lifeless: ooh, indeed
<lifeless> benji: also, this should be in two pastches
<lifeless> benji: a) schema only
<lifeless> benji: b) the lazr.conf and any other code things
<lifeless> benji: because we don't deploy from db-devel
<benji> lifeless: oh, so I should move the schema-lazr.conf changes to the code branch; ok
<lifeless> benji: yah, db-schema != config-schema
<jcsackett> benji: r=me on the code changes as is. i gather from the above you'll have another branch with the schema changes? :-P
<benji> jcsackett: yep; thanks!
<jcsackett> sinzui: the post-oneiric-install script you pointed me at last week is somewhat frightening in all that it does. :-P
<sinzui> yes, but you should consider that that script is very similar to what many deb packaging scripts to at build and install phases
<flacoste> benji: i think you code as tiny but (i think benigh) race condition
<flacoste> benji: it could still end up creating two POFile job
<flacoste> since there is no constraint preventing two from existing at the same time
<lifeless> flacoste: uhm hi
<flacoste> but i think we don't care about that, right?
<sinzui> I image that a a tuning script framework could be written to get the most out of several lines of computers. I have a sony that also needs special drivers configured to the the keyboard tor rock
<benji> flacoste: hmm, you may be right
<lifeless> flacoste: I've just replied to your new RT - I think we need to coordinate on that a bit more :)
<flacoste> it would simply waste some CPU cycle
<flacoste> lifeless: that last statement was adressed to benji, just in case, you got hit by paranoia overnight
<flacoste> :-)
<lifeless> flacoste: :)
<lifeless> no paranoia, I *know* they are out to get me!
<flacoste> that's why i tell myself whenever i see an (american) football scrum!
<flacoste> they are plotting against me!
<flacoste> lifeless: so basically, the DB load problem isn't for the normal situation, but for the fail-over one?
<lifeless> flacoste: theres a few interlocking things
<lifeless> let me just pull up some rports
<lifeless> flacoste: this might be easier in voice
<flacoste> lifeless: then let's skype, there are a couple of things to cathc-up on from the LP/IS call
<benji> flacoste: yeah, there is a race there (well, it's not technically a race, but the opportunity for the same POFile to be scheduled for an update multiple times); since updates take 20 or 30 seconds and they could be scheduled tens of times in a particular update window) I'm inclined to collapse them if you don't object
<flacoste> benji: sure
<flacoste> benji: you mean at processing time?
<jcsackett> sinzui: that seems reasonable. i'm just trying to sort out what differs between macbook5,1 and macbookair4,2.
<benji> flacoste: I figured I'd do it at job creation time, but in a simple, actually racy way that would eliminate the vast majority of duplicate jobs but still be very straightforward
<flacoste> benji: makes sense
<flacoste> soviet enginerring
<flacoste> the Launchapd way :-)
<benji> flacoste: makes me think of the US SR-71 spy plane, it got so hot during flight that it was built with gaps, so when they fueled it up on the ground it would leak until it got into the air and the gaps would close
<benji> lifeless: https://code.launchpad.net/~benji/launchpad/bug-877195-db/+merge/80618 looks better now
<lifeless> benji: are you intending to address bug 781274 as pat of your arc?
<lifeless> *part*
<jcsackett> sinzui: i'll be a few minutes late to standup.
<lifeless> benji: reviewed
<benji> lifeless: thanks; I'll take a look at that bug tomorrow and see what I can do with it
<poolie> benji: istr lots of people got cancer from dealing with the consequences of that design
<poolie> how naive would it be to want to move some build jobs onto ec2 instances (not necc AWS ec2)
<poolie> actually that might be too big of a question for irc
<benji> poolie: well, that's quite a symptom for trying to make a change to software, but I like a challenge
<lifeless> poolie: build jobs?
<poolie> sr-71 leaking fuel
<poolie> imbw
<lifeless> poolie: '11:10 < poolie> how naive would it be to want to move some build jobs onto ec2 instances (not necc AWS ec2)
<lifeless> '
<poolie> oh, yes, build jbos
<poolie> *jobs
<lifeless> what do you mean by build jobs, specifically.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
<poolie> the work done by buildds
<poolie> to accommodate people who want more or larger buildders
<poolie> i realize there are trust issues about running some jobs on hardware we don't specifically control
<lifeless> so, buildds are trusted machines, because their output runs as root on other peoples machines.
<poolie> but perhaps for other jobs it would be an ok tradeoff
<lifeless> ec2 is very expensive in lots of ways
<lifeless> so even if we handwave the trust aspect away
<lifeless> I doubt it would make financial sense
<lifeless> internal cloud, perhaps.
<poolie> well, at the moment the price is infinite :)
<lifeless> (but thats what the builders basically are anyhow, except they have a -very- tuned image-start process (we can bring up a slave in a second or so)
<poolie> anyhow, so the answer is basically:
<poolie> trust, and cost?
<lifeless> those are the nontechnical bits yes
<jml> poolie: people have asked for that though
<jml> poolie: openstack, notably
<lifeless> technically, we'd need some fairly deep changes to the idea of a builder (ephemeral vs static), and obviously the implementation of protocols to deal with ec2
<poolie> jml, right, it seems like trust and cost is something that a user can make their own decision about
<jml> who really dislike the fact that their development process can be disrupted by the whims of Ubuntu
<poolie> assuming it's technically feasible for them to pay the financial cost
<jml> poolie: and also financially feasible :D
<StevenK> jml: But it can be anyway -- if they are building against precise, library changes can impact them.
<lifeless> The way I would address trust and cost is to allow a ppa to choose to use our cloud (existing builders) or supply their own cloud credentials
<poolie> yep
<jml> StevenK: but again, that's something they can choose
<poolie> so my question was really how deep that is
<jml> lifeless: you might also want info about that choice on the PPA page for prospective downloaders.
<lifeless> poolie: a couple man-weeks + review deployment and friction
<lifeless> jml: yes, and/or even in the archive metadata
<poolie> thanks
<jml> *nod*
<lifeless> jml: I don't think the thinking about it is complete yet
<poolie> jml, info for prospective downloaders about where it was built?
<lifeless> poolie: about whether its trustable like things built on our cloud
<lifeless> poolie: e.g. was it actually built from the source we think it was.
<lifeless> poolie: for something out of our control the answer has to be 'unknown'
<poolie> i suppose there is a channel there whereby running on aws gives them a chance to inject things not visible in the source
<poolie> right
<poolie> though, this is a little academic if you're already trusting the author to provide all the source
<lifeless> perhaps :)
<lifeless> see under reflections on trust
<poolie> 'on trusting trust'
<poolie> sure
<lifeless> yes, that one :)
<poolie> i'm just saying that's a bit academic compared to the way people treat ppas today
<lifeless> e.g. if openstack point at a rackspace cloud because they trust it
<lifeless> etc
<lifeless> poolie: there are folk in the distro deeply concerned about the way people treat ppas today ;)
<poolie> for instance that every ppa can upgrade any package
<lifeless> -> because it installs as root
<lifeless> -> and can change anything
<poolie> sure, i sympathize with them, i just don't think the idea that AWS is malicious or subverted would be very high on the list
<lifeless> There is a 'any unnecessary risk is too much' mentality, which to a large extent I agree with.
<lifeless> in this area
<poolie> it's true this is opening up some new risks
<poolie> that would be a tradeoff against allowing these people to use the system at all, vs not building packages, or building them on entirely separate machinery
<poolie> or not getting a good quality of service
<lifeless> well
<lifeless> yes, but that doesn't mean we should do it :) - the tradeoff may not be worth it.
<poolie> yes :)
<poolie> can i navigate back to previous builds from https://code.launchpad.net/~neon/+recipe/project-neon-kdesdk
<wgrant> I don't believe so.
<wgrant> Grar
<wgrant> More string interpolation to generate HTML.
<wgrant> Without escaping.
<poolie> wgrant:  at least one build has passed of a package that was previously reported to be failing
<poolie> https://launchpad.net/~bzr/+archive/daily/+recipebuild/108303
<wgrant> That's an odd place to build it into :)
<poolie> gar, unix copy and paste again
<poolie> no, actually
<poolie> i was fooled by the 'request builds' having an interesting default
<wgrant> Yeah, the alphabetically first PPA by display name.
<poolie> obviously
<poolie> can i cancel it?
<wgrant> It depwaited.
<wgrant> Oh, the new build in ppa:mbp?
<wgrant> I can cancel it if you want.
<poolie> ah which does not actually mean "i'm waiting" but "i'm not waiting"
<poolie> no, the new build into my ppa is fine
<wgrant> For binary builds it means "i'm waiting".
<wgrant> But not for recipe builds.
<poolie> ah ok
<poolie> ok, so on a later attempt, the build into ~mbp failed, but i think after all the bzr work completed
<poolie> and just because of an actual build problem in the source
<poolie> so, it's a good sign
#launchpad-dev 2011-10-28
<wgrant> poolie: Indeed.
<wgrant> Looks fixed.
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugtasks: 266
<poolie> i'm going to update the bug then
<wgrant> poolie: Thanks.
<poolie> thank you very much
<wgrant> Huh.
<wgrant> We are now in the longest downward Critical bug trend since May.
<poolie> stop the line
<poolie> i am curious how many i have personally fixed, caused, and filed
<poolie> not quite enough to search right now though
<huwshimi> wgrant: wow, is our bug submission form broken or something?
<StevenK> Either that, or lifeless has stopped esclating for some reason.
<huwshimi> StevenK: Quick, someone land a branch with the critical status removed
<lifeless> wow
<lifeless> 53 cleanups in an LP test case.
<lifeless> anyone spot something a little wonky there?
<wgrant> Where?
<lifeless> self.case._cleanups
<lifeless> my call to sync barfed
<lifeless> I'm investigating why
<lifeless> there are 53 cleanups registered, but only 14 unique functions amongst them
<wgrant> "only"
<lifeless> 14 is a reasonable number
<lifeless> 53 less so
<lifeless> ahha
<lifeless> setUp is being called twice.
<lifeless>  https://bugs.launchpad.net/testtools/+bug/882884 about the lack of alerting
<_mup_> Bug #882884: setUp being called twice is not an error <testtools:Triaged> < https://launchpad.net/bugs/882884 >
<lifeless> test run time was 1912->2326 or 4h10, so the sync() isn't a perf issue
<wgrant> Great.
<lifeless> the reason I noticed this setUp twice thing was the sync() failing due to queue not existing
<lifeless> which should make you scratch your head :)
<StevenK> wgrant: So all I did was add 'with person_logged_in(team.teamowner):' before create_initialized_view(), and it blows up :-(
<lifeless> aieee
<lifeless> check out PullerBranchTestCase.setUp()
<lifeless> this is what super() is for
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2126BZR113161
<wgrant> Most useful OOPS ever :)
<wgrant> lifeless: Hah
<lifeless> that oops isn't rendering for me
<wgrant> It takes about 15 minutes.
<lifeless> wow
<wgrant> It's also empty.
<lifeless>  /o\
<lifeless> ugh, rosetta_branches_script_oopses still sees too many.
<lifeless> wtf is going on there
<lifeless> I think I found a wabbit bug
<wgrant> Uhoh.
<lifeless> CaptureOops uses an autodelete exchange
<lifeless> it uses Receiver() to suck messages out of a queue bound to that exchange, also autodelete.
<lifeless> the Receiver is given a brand spancing new connection
<lifeless> unless amqplib has die die die hidden connection pooling (I'm about to check this)
<lifeless> then when the Receiver closses its connection, the queue and exchange should be untouched
<wgrant> You're relying on autodelete to take effect immediately?
<wgrant> I'm not sure that immediate reaping is guaranteed by the spec.
<lifeless> no
<lifeless> I'm being fucked by it taking effect when a different connection is closed
<wgrant> Ah
<lifeless> so connection A:
<lifeless>  declare exchange
<lifeless>  declare queue
<lifeless>  bind
<lifeless> connection B
<lifeless>  send
<lifeless> connection C
<lifeless>  send sentinel
<lifeless> connection D
<lifeless>  consume from queue
<lifeless> connection E
<lifeless>  send sentinel - 404 no such exchange
<lifeless> connection A is still open at this point
<lifeless> sound like a bug to you ?
<wgrant> "auto-delete: the queue will get deleted as soon as no more subscriptions are active on it."
<lifeless> ugh
<lifeless> ok, fail-to-understand
<lifeless> fair enough
 * lifeless reworks that part, again.
<lifeless> wgrant: that was in the amqplib docstring I guess? a quick google hadn't found that for me
<wgrant> lifeless: Wikipedia.
<lifeless> *blink*
<wgrant> I know, the canonical protocol reference.
<wallyworld_> wgrant: StevenK: buildbot weirdness. you seen that failure mode before?
<wgrant> wallyworld_: Someone or something turned off postgres.
<wgrant> See #-ops
<wallyworld_> ah ok thanks
<wgrant> Hmm.
<wgrant> Librarian 500s locally :(
<StevenK> I guess create_initialized_view doesn't like with person_logged_in
<StevenK> bzr grep doesn't support -{A,B,C} :-(
<StevenK> Ah ha, I was right.
<StevenK> create_initialized_view() returns anonymous views
<wgrant> Doesn't it take a user kwarg?
<StevenK> create_initialized_view()'s docstring sucks
 * StevenK tries user
<StevenK> wgrant: It does not
<mwhudson> the whole area is confusing
<mwhudson> StevenK: you want the principal argument though
<StevenK> But that didn't work either
<lifeless> wgrant: oops still hasn't rendered
<StevenK> mwhudson: principal should be a IPerson or something else?
<mwhudson> StevenK: yes
<mwhudson> StevenK: sorry, have to run -- i think maybe IParticipation(person) works?
<wgrant> lifeless: Heh
 * lifeless throws useoops at ec2 again
<lifeless> and EOW
<StevenK> AH
<mwhudson> IPrincipal, rather
<StevenK> mwhudson: Thank you for the principal hint -- you need *both* with person_logged_in and principal specified in c_i_v()
<wgrant> wallyworld_: Around?
<wallyworld_> yes
<wgrant> You know how you added the feature-flagged pillar role private bug visibility rules?
<wallyworld_> yes
<wgrant> Those only apply to the task search stuff. Bug.userCanView has its separate implementation.
<wgrant> Not behind a flag.
<wgrant> And slightly different.
<wgrant> I'm reimplementing userCanView in terms of the task search query.
<wallyworld_> was the bug usercanview done separately
<wgrant> Which will drop the non-flagged owner visibility rule.
<wgrant> Yes, it was done a yearish ago.
<wgrant> Currently the bug pages are pretty useless, because all the tasks are invisible.
<wgrant> So I don't think removing this is going to be much of a loss.
<wgrant> But I wonder if you have any thoughts on the matter.
<StevenK> Oooh, if they're invisible, can we ignore them?
<wallyworld_> so you are looking to remove the current bug usercanview?
<wallyworld_> and replace with something that matches the bug task role visibility rules
<wallyworld_> ?
<wallyworld_> it would have to be a union of those task rules?
<wgrant> Yes.
<wgrant> Hm?
<wallyworld_> so if the user can see any task, they can see the bug
<wgrant> I'm just using get_bug_privacy_filter.
<wgrant> Right.
<wgrant> The privacy filter already just looks at Bug.
<wgrant> because tasks don't have separate visibility.
<wgrant> (ignoring the fact that we exclude tasks for inactive products normally)
<wallyworld_> i can't recall the details of get_bug_privacy_filter - let me just check the code
<wgrant> It takes a user and produces a condition that restricts Bug to those visible to the user.
<nigelb> I wish launchpad was less noisy about expiring memberships. Or I could go click "Hey, I understand. Stop warning me!"
<wallyworld_> you going to keep the feature flag protection?
<wgrant> wallyworld_: I'm not changing that bit of get_bug_privacy_filter. The first branch will not change it at all.
<wgrant> It will just reuse the task search rules for Bug.userCanView.
<wgrant> So yes, the flag protection will be preserved.
<wallyworld_> it all sounds good to me. it doesn't make sense if the use can see an empty bug which is what i think you are saying happens now
<wallyworld_> and it makes sense when you consider we are going to only single pillar private bugs
<wallyworld_> i guess i should have done this change within the original branch
<wgrant> Well, this should have been done years ago :)
<wallyworld_> better late than never
<wgrant> Indeed.
<wallyworld_> wgrant: so when you do this, the fflag can be turned on i assume
<wallyworld_> ah, it's on for !launchpad
<wgrant> It's on for ~launchpad.
<wgrant> And can only be on for trusted people.
<wgrant> wallyworld_: Several tests rely on it, unfortunately :(
 * wgrant hacks around.
<wallyworld_> wgrant: ah, yes. not surprising. i had similar issues yesterday. drove me crazy
<wallyworld_> at least there are tests :-)
<wgrant> Indeed.
<wgrant> Most of these tests aren't too ugly, either.
<wgrant> Just a bit awkward in the transitional period.
 * wgrant stabs LaunchBag.
<wallyworld_> StevenK: i'd prefer it if your tests iterated over the policies in OPEN/CLOSED_TEAM_POLICY and did the checks
<wallyworld_> btw, it only ended up being a few lines to fix c i v for Person+index ?
<StevenK> wallyworld_: So it would seem, yes. And I'll look into that.
<wallyworld_> StevenK: thanks. you should just add a for loop to you current tests
<StevenK>  Right.
<wallyworld_> StevenK: i think the tests should be beefed up to test that the edit link is a) there and b) has the expected url.
<wallyworld_> this can be easily done by giving the link an id and doing a find_tag_by_id() in the test
<wallyworld_> the same could be done for the message text. give the text node an id and check for that rather than the actual text wording
<wallyworld_> this allows the wording to be changed and the test still passes
<StevenK> wallyworld_: Do you want to put all of that on the MP and I'll sort it all out on Monday morning?
<wallyworld_> StevenK: will do. have a good weekend
<StevenK> wallyworld_: You too :-)
<wallyworld_> i would if we had a better ajax story around partial page refreshes :-(
<nigelb> Australia is EOD already? *jealous*
<wallyworld_> haha
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
<proyvind> hello, dear launchpad developers! long time launchpad user, first time "caller" here..
<proyvind> I'm currently wondering about plans for git support in bazaar
<proyvind> how are the actual plans for this, has any specific plans been made? has any work taken place? what's the state? will there be any priority given to focus on adding any of this upstream? :)
<proyvind> (and just to be clear, yes, I am asking about support for git in place of bazaar, not support for external git branches ;)
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugtasks: 266
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugtasks: 266
<bac> hi adeuring
<adeuring> hi bac
<bac> adeuring: i'll take wgrant's MP...confirming you aren't working it
<adeuring> bac: no , I did not look at it
<bac> adeuring: it is the only one outstanding.  cool.
<bigjools> I just added another :)
<bigjools> I am going for lunch though, so no rush, back later
<deryck> adeuring, ping for standup
<danhg> flacoste
<adeuring> deryck: oops, sorry
<deryck> np
<adeuring> abentley: I think "this.current_batch.mustache_model.bugtasks.length + 1" in line 113 is wrong. Shouldn't that be "... -1"?
<adeuring> (line 113 of the diff)
<adeuring> abentley: no, that should be "+0"...
<abentley> adeuring: So, the +1 is trying to compensate for start being 0-indexed.
<adeuring> abentley: right. But for 5 results and start = 0, we want to show "results 1-> 5"
<adeuring> while we get "1 to 6"
<adeuring> abentley: just click on any of the links ;)
<abentley> Hmm, so why does it look right in use?
<adeuring> and compare the number in "n->m of k" with the real number of results ;)
<abentley> adeuring: that's an estimate, though.
<adeuring> abentley: well, k is an estimate (at least for StormRangeFactory), but we known the number of results in the batch precisely
<abentley> adeuring: You're right, I'll fix it.
<adeuring> abentley: cool, thanks
<adeuring> abentley: another quirk: For the first batch, the "first" and "previous" links are gray, but still active. "unlinking" them is something for antother branch, I assume?
<adeuring> (same for the last batch and the last/next links)
<abentley> adeuring: I would prefer if the CSS was fixed so that inactive links don't show as links.
<adeuring> abentley: right, makes sense. So, r=me, and file a bug about the links?
<abentley> adeuring: Thanks.  I'll file a bug.
<abentley> deryck: could you please ec2land https://code.launchpad.net/~abentley/launchpad/display-cleanup/+merge/80621 ?
<deryck> abentley, sure
<abentley> deryck: thanks.
<deryck> abentley, np!
<flacoste> when running make (after a make clean) on latest devel, i get the following error:
<flacoste> awk: (FILENAME=lib/lp/contrib/javascript/yui3-gallery/gallery-accordion/gallery-accordion.js FNR=2916) fatal: cannot open file `lib/lp/contrib/javascript/mustache.js' for reading (No such file or directory)
<flacoste> ah
<flacoste> nm
<flacoste> it's a symlink
<flacoste> i need to update sourcecode...
<flacoste> though
<flacoste> doh
<flacoste> is it just me or make run-testapp is broken?
<abentley> deryck: sorry I've been invisible.  I went home because my ISP wasn't reporting an outage anymore, but it turns out the outage is now my problem.
<deryck> abentley, no worries.
<abentley> deryck: did you want to chat?
<deryck> flacoste, I don't get any errors doing make run-testapp, but the test in the browser hangs for me, seeming to never connect.
<flacoste> deryck: well, that's a step beyond me :-)
<flacoste> deryck: do you remember doing anything special for the pgbouncer change?
<deryck> flacoste, yeah. :)  I didn't do anything special at all.  Update devel.  make and then make run-testapp.
<flacoste> deryck: i mean more like a month ago when pgbouncer landed
<deryck> I usually do a make by itself first out of habit, just to see if I hit any errors.
<deryck> oh
<deryck> flacoste, hmmmm, no I don't think so.  I don't recall anything special.
<deryck> bac, hi.  I've got a pretty easy js branch if you have time to review it.
<bac> deryck: i'd be happy to
<deryck> bac, awsome, thanks!  https://code.launchpad.net/~deryck/launchpad/orderbybar-integration-fixes/+merge/80704
<bac> deryck: 3600+ lines?
<deryck> bac, hmmm, no.
<deryck> bac, wonder what I did.  let me see.
<bac> good, b/c 3500 is my absolute limit on JS or soyuz branches
<deryck> bac, hmmm, the MP is trying to merge into lp:~deryck/launchpad/devel.  I don't know what I did wrong.
<nigelb> bac: what happens for longer? stack overflow? :)
<deryck> I didn't even know I had my own devel. ;)
<deryck> bac, let me try to sort out the mp.
<bac> deryck: cool.  ping me
<danhg> flacoste
<deryck> bac, try this one:  https://code.launchpad.net/~deryck/launchpad/orderbybar-integration-fixes/+merge/80705
<bac> deryck: did you run the linter?
<deryck> bac, I thought I did.  Let me check.
<bac> deryck: i see some things that jtv just fixed up in a huge branch.  he'd cry if you introduce more.
<deryck> bac, yeah I see 4 or 5 CSS lint errors.  I can fix those, assuming I understand them.
<bac> thx
<bac> deryck: nothing uses this yet, correct?  so there is no way for me to see it working?
<deryck> bac, no, nothing uses it.  Just the test file to run, but it moves quick, so you never really see the widget.
<deryck> bac, ok, bzr push'ed the CSS fixes.
<bac> deryck: looks good
<deryck> bac, awesome, thanks!
<nigelb> flacoste: Hey, around?
<nigelb> flacoste: I've subscribed you and mrevell to https://blueprints.launchpad.net/ubuntu/+spec/community-p-summit.  We'd really like to have smoeone from lp in that session :)
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  | Critical bugtasks: 266
<flacoste> nigelb: great, we should be there
<nigelb> \o/
<nigelb> flacoste: Thanks :)
<flacoste> test_clashingPOFileTranslatorEntries: benji that sounds like something you touched recentely no?
<benji> flacoste: sounds like it, but I don't think I actually have; let me look to be sure
<benji> flacoste: right, I haven't been in that region of translations
<flacoste> benji: ok, was wondering about allenap latest email
<allenap> flacoste, benji: I'm about to disable it. Want me to hold off?
<flacoste> allenap: not from me
<benji> allenap: I don't have enough context to have an informed opinion, so I don't object
<allenap> benji: Cool, okay. I've just disabled it, and bug 883274 exists to track it. Fingers crossed that buildbot gets through this time.
<_mup_> Bug #883274: test_clashingPOFileTranslatorEntries failing in buildbot, okay locally <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/883274 >
<allenap> Cheerio everyone, have a good weekend.
#launchpad-dev 2011-10-29
<nigelb> Gah. lying launchpad.
<nigelb> I keep getting - If your membership does expire, we'll send you one more message to let
<nigelb> you know it's happened.
<nigelb> BUT I GOT THAT 3 TIMES NOW :)
<StevenK> That message continues to annoy me.
<StevenK> I think I have a branch somewhere to change it.
<nigelb> StevenK: I will forever be grateful if you fix it :)
#launchpad-dev 2011-10-30
<cjohnston> any devs around? Have an issue between LP and summit that we need some assistance figuring out please.
<lifeless> cjohnston: shoot
<nigelb> lifeless: ohai. We're having an issue wwith the https://blueprints.launchpad.net/sprints/uds-p/+temp-meeting-export thingy.
<cjohnston> yay!
<cjohnston> go nigelb!
<cjohnston> its like nigelb has my nick on highlight
<nigelb> The subscribers listed in https://blueprints.launchpad.net/linaro/+spec/linaro-summits-server-1 don't show up on the meeting export page :(
<lifeless> should really switch to using API's :)
<cjohnston> lifeless: feel free to join our session on wednesday ;-)
<nigelb> lifeless: I don't think there's an appropriate API for this.
<nigelb> If there is, I'm willing to make the switch.
<lifeless> cjohnston: if its in your afternoon, I may; I'm not at UDS though
<cjohnston> It's at noon EST IIRc
<lifeless> nigelb: just export sprints on the API; there doesn't need to be a tailored one
<nigelb> Hrm. How do I get subscribers to a BP on the API?
 * nigelb looks at docs
<lifeless> anyhow
<lifeless> uhm, are all bp's faulty, or just that linaro one ?
<cjohnston> I believe there are atleast a couple
<lifeless> worth checking what they have in common then, if its not all
<nigelb> linaro one is the one we have a complaint about.
<lifeless> e.g. unicode names (loÃ¯c)
<lifeless> or being in two different sprints (uds-p and lcq4.11)
<lifeless> obviously there is a bug somewhere; I presume you've checked the meeting export page to determine that the data is missing there, vs a summit bug ?
<nigelb> Yeah, we have :)
<nigelb> summit uses the meeting export page.
<nigelb> And the data isn't there.
<lifeless> ok; and what another faulty blueprint ?
<nigelb> cjohnston: Know of another faulty one?
<cjohnston> ya
<cjohnston> looking for it
<cjohnston> https://blueprints.launchpad.net/linaro-ubuntu/+spec/linaro-platforms-lc4.11-improving-ubuntu-upstream-relationship
<nigelb> cjohnston: that one is fine breakage.
<nigelb> Not accepted into uds-p
<nigelb> and hence won't show up in meeting export.
<cjohnston> its on the schedule nigelb
<nigelb> wha.
<lifeless> it shouldn't be :)
<nigelb> lifeless: sprints are exposed over the API? I don't see anything in API doc.
<cjohnston> tuesday at 11 in grand serra h
<cjohnston> its approved for lcq4.11
<lifeless> cjohnston: it may be manually locked in but the lp data is inconsistent
<lifeless> nigelb: may need to be added...
<cjohnston> so if its approved for lcq4.11 doesnt that make it valid?
<nigelb> lifeless: Agreed. I'll work on that next cycle and get it done.
<nigelb> cjohnston: meeting export is only run for uds-p, and not being approved there doesn't get it added into summit automatically.
<nigelb> I think someone added at manually.
<lifeless> easy fix - get a uds driver to ack it
<nigelb> Jorge!
<cjwatson> I can do it
<nigelb> cjwatson: Please do :)
<cjwatson> done
<nigelb> Thanks!
<cjohnston> there are more that are only approved for lcq
<lifeless> that might be a distraction then, lets get that one acked and refresh summit, see if that fixes it.
<cjohnston> how are they getting some of the participants but not all of them
<cjwatson> though, um, it's pretty nasty that we have to basically make uds-p a superset of lcq4.11
<lifeless> cjohnston: well, the next candidate is a unicode bug
<lifeless> e.g. the first has loÃ¯c the second has ÐÐ°Ð½Ð¸Ð»Ð¾ Ð¨ÐµÐ³Ð°Ð½
<lifeless> I'd hope we were well beyond such things, but ... you never know your luck in the big city
<nigelb> I'm going to bet on a unicode bug.
<nigelb> Because I've seen 2 BPs with danilos in it that broke updates :)
<cjohnston> https://code.launchpad.net/~james-w/summit/usability/+merge/80108
<lifeless> have you guys filed a bug yet ?
<cjohnston> lifeless: nigelb ^
<cjohnston> i wonder if thats what that one is for
<nigelb> Nope.
<cjohnston> the unicode part of it
<nigelb> Django's __unicode__ has nothing to do with what lifeless is talking about.
<cjohnston> k
<lifeless> cjwatson: does man-db hit all man page inodes ?
<cjwatson> probably yes, e.g. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620947
<cjwatson> (well, that's cat page directories)
<cjwatson> it'll skip input directories that weren't modified since the last run
<cjwatson> cjohnston: if the LP export is wrong then I doubt it's worth looking for relevant changes to summit ...
<cjohnston> that was a new change cjwatson and i didnt really ever look at it, so i was just wondering if that could be it
<lifeless> cjwatson: heh, so yah i was watching man db on my freshly booted desktop and going 'how can that be measurable time' :)
<cjwatson> cjohnston: but logically it can't have affected the content of the LP export
 * nigelb dons his LP community developer hat on mucks around in sprint code.
<lifeless> nigelb: cjohnston: is there a bug for this yet ?
<cjohnston> https://bugs.launchpad.net/summit/+bug/883407 is our bug
<_mup_> Bug #883407: Summit fails to show all my subscribed talks <Summit:Incomplete> < https://launchpad.net/bugs/883407 >
<cjwatson> lifeless: the thing that really kills man-db performance is forking subprocesses when it doesn't need to; I've been working (in my CFT) on adding better in-process support to libpipeline, sort of coroutine-like
<cjwatson> lifeless: microbenchmarks suggest that that accounts for the overwhelming majority of the runtime
<lifeless> cjwatson: interesting; it must create bazjillions of subprocesses - my desktop is a very recent performance cpu w/ tonnes of RAM
<cjwatson> several per page
<lifeless> cjwatson: that, or microbenchmarks are ignoring IO time (a common issue)
<nigelb> FUUUU.
<nigelb> I found the problem.
<cjwatson> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630799#20
<nigelb>             if subscription.personID not in attendee_set:
<nigelb>                 continue
<nigelb> IF someone is not registered on lp for uds-p sprint, their name doesn't show up in temp-meeting-export.
<lifeless> of course
<cjwatson> lifeless: I doubt that's it, because the "fork roughly the right number of processes" microbenchmark I ran had runtime very similar to that of man-db itself
<nigelb> That yes, the people missing don't seem to be registered for uds-p.
<nigelb> s/That/And
<lifeless> cjwatson: ah, thats good corroboration
<cjwatson> er, wait, misremembering slightly
<nigelb> Yay for mucking in launchpad source :)
<cjwatson> depends how you run the microbenchmark, but at any rate details in that bug message
<nigelb> *so* glad I started hacking on LP.
<lifeless> cjwatson: thanks, yes I see
<cjwatson> in /usr/share/man, 'find -type f | xargs cat | zcat >/dev/null' => 2.5s, 'find -type f | xargs -n1 zcat >/dev/null' => 98s, mandb => 100s
<lifeless> 20K processes \o/
<cjwatson> s/98/88/
<lifeless> cjwatson: the libpipeline thing would also be a good point to introduce concurrency
<cjwatson> now all I need is less work to do so that I can have time to finish this ;-)
<cjwatson> maybe, but my gut feel based on the 'find -type f | xargs cat | zcat >/dev/null' test is that it doesn't need it
<lifeless> cjwatson: won't help on cold cache situations much/at all, but would on hot
<lifeless> cjwatson: depends how fast you want it to be :)
<lifeless> cjwatson: on cold cache situations it may permit some extra disk concurrency if depth(disk tag queue) <= cpucount
<lifeless> if thats false, you might overwhelm the scheduler
<cjwatson> I think shaving off 80% is eminently possible simply by avoiding the process storm, and that would bring it down to an entirely acceptable runtime in my book; I would rather not introduce concurrency when I don't need to
<lifeless> :) shaving off 80% would be awesome, and your plan sounds eminently sensible
<lifeless> cjohnston: nigelb: so, long story short - folk that haven't registered are silently dropped.
<nigelb> lifeless: Yep.
<lifeless> cjohnston: nigelb: workaround is to get them to register if the are planning to attend remotely, I think there is a flag fo rthat
<nigelb> lifeless: No, no. The problem is the 2 sprints
<nigelb> The issue is because linaro has a sprint of its own and a lot of linaro folks are only registered on the linaro sprint.
<nigelb> We need to get them to register on both.  That's the least painful way.
<nigelb> Anything else involves a non-trival summit code change less than 24 hours before UDS :D
<cjohnston> lifeless: http://summit.ubuntu.com/uds-p/2011-11-01/  why would riku-voipio show up in linaro server summit (grand sierra I at 9) but not Linaro Ubuntu LEB (grand sierra H at 11)
<lifeless> cjohnston: summit has local overrides for stuff; check that hasn't been done first.
<nigelb> Is that the one cjwatson just approved. They may be some manual thingies there.
<nigelb> I don't think it was in +temp-meeting-export until about 10 minutes back.
<cjohnston> nigelb: he was already in that one.. so cjwatson just approving it wasn't relevent
<cjwatson> no, I approved https://blueprints.launchpad.net/linaro-ubuntu/+spec/linaro-platforms-lc4.11-improving-ubuntu-upstream-relationship
<nigelb> Yeah, that's the Linaro Ubuntu LEB
<cjwatson> oh right
<nigelb> cjohnston: It is. Someone manually added the BP and probably a few relevant people.
<nigelb> Fairly sure ricardo is a track lead.
<cjohnston> added releevant people where? in summit? because he wouldn't get imported on that one if he isnt an attendee for uds-p
<nigelb> he's a summit attendee.
<nigelb> But if the BP and its people were manually imported...
<lifeless> this is why having summit manually override is nuts ;)
<lifeless> either move the data store to summit
<lifeless> or enhance LP to directly do what you want
<lifeless> that will avoid all this skew
<lifeless> which completely confuses folk :)
<nigelb> I'll take option 2 :)
<nigelb> well, the problem is, people don't trust summit
<nigelb> and override stuff t work
<nigelb> Instead of *asking*
<lifeless> nigelb: no reason the overrides can't be stored in LP
<nigelb> I thought this stuff was not going to be maintained in the future?
<lifeless> huh?
<lifeless> issuetracker says we're consolidating concepts, not eliminating functionality.
<nigelb> ah.
<nigelb> In wwhich case, I will bug fracis at the summit session and figureout something
<nigelb> I'm definitely will to put the LP work in :)
<nigelb> (Sigh, I wish I was there in person)
<lifeless> it should be fairly straight forward: identify the places that summit is duplicating data from LP, and nuke with vengeance.
<nigelb> heh
<lifeless> the cost of changing LP has dropped a bit already
<lifeless> I'm confident the next 3-4 months will see another drop
<nigelb> \o/
<nigelb> and 2 of us have commited patches to LP.
<nigelb> So, we're good on that front mostly.
<lifeless> yah
<lifeless> the alternative is to move attendance totally out of LP
<lifeless> or things like that
<lifeless> now, I don't have a preconceived 'right' solution
<nigelb> Hrm, I'm open to that as well.
<lifeless> my criteria are:
<nigelb> Just import relevant people from BPs. Not from sprint attendance only.
<lifeless>  - no duplication once its done (mirroring is ok, but duplicate isn't)
<lifeless>  - LP isn't left with stub functionality: what remains should make sense on its own [or with summit as a blessed official addon that anyone can run]
<lifeless> My *suspicion* is that moving session times into LP, to permit LP storing pinned sessions, is the simplest change.
<nigelb> ..
<lifeless> Separately we need to address this two-sprint thing.
<nigelb> You'll let us do that? :)
<lifeless> nigelb: why wouldn't I ?
<nigelb> \o/
<lifeless> nigelb: LP was *born* to support Ubuntu
<nigelb> I will completely jump with joy if we can get that done :)
<lifeless> pics or it didn't happen
<lifeless> :)
<nigelb> heh
<nigelb> To be honest, I thought those bits were not changeable from launchpad end.
<lifeless> ok, well I'm going to put nose to the grindstone and put this amqp oops workaround in place
<nigelb> This opens up a whole new avenue of possiblities.
<lifeless> nigelb: Ok, so big picture and little picture.
<lifeless> big picture: we don't want to add random new features to LP without them tying in sensible. Just because it needs doing doesn
<lifeless> 't mean it needs doing the LP DB
<lifeless>  -> Look at the results tracker, separate DB, will become part of LP UI soon though.
<lifeless>  -> fixing an existing feature is not adding a random new feature.
<nigelb> Right.
<lifeless>  -> supporting Ubuntu is a primary role for LP [explicitly Ubuntu in addition to its broader role of supporting the free software community]
<lifeless> little picture:
<lifeless>  -> summit and lp sprints have overlapping models that leads to skew and confusion
<lifeless>  -> this is a problem
<lifeless>  -> lets fix :)
<nigelb> It took me half a day to track this down. I've worked on both summit *and* LP.
<nigelb> So, yes. Skew and confusion indeed.
<lifeless> one approach is to let LP store the entire model
<lifeless> another approach is to remove from LP the parts of the model that are maintained by summit
<nigelb> Most of wwhat summit does is mirroring.
<nigelb> But summit lets overriding of stuff. Like creating a meeting without a BP.
<lifeless> yes, but ELOCALOVERRIDES
<nigelb> Exactly.
<lifeless> so I think if we say 'LP sprints are not able to represent what happens at UDS', thats fairly clearly a defect in LP
<nigelb> Indeed.
<lifeless> 'What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projectsâ15 to 35 percent longer. '
<lifeless> *interesting*
<lifeless> wgrant: when you are around, I'd like that incremental review.
<lifeless> wgrant: other than this weird rosetta isolation thing, i think the branch is GTG
<wgrant> lifeless: Sure, will look shortly.
 * nigelb blinks.
<nigelb> Oh. Its a Monday.
<nigelb> Right.
<lifeless> I'm thinking of making TestCase.setUp create an oops-message with the testid
<lifeless> I think that might make debugging some of these things easier
<wgrant> What do you mean?
<lifeless> ErrorReportingUtility.oopsMessage('test %s' % (self.id(),))
<wgrant> lifeless: Ah!
<wgrant> Hmm, perhaps.
<wgrant> lifeless: Could you please explananlyze https://pastebin.canonical.com/55125/ on qastaging?
<wgrant> 1-2ms on DF, 400-600ms on qas.
<wgrant> At least that's what ++profile++sql says.
<lifeless> 4.5ms
<lifeless> 6.1ms 'total runtime'
<lifeless> ah
<wgrant> Perhaps ++profile++sql is buggy.
<lifeless> wgrant: when you say 1-2ms on DF
<lifeless> wgrant: are you executing the query, or the explain analyze ?
<wgrant> explain analyze
<lifeless>  ?column?
<lifeless> ----------
<lifeless>         1
<lifeless> (1 row)
<lifeless> Time: 745.439 ms
<lifeless> when I execute it
<lifeless> try executing it
<wgrant> Still lightning fast.
<lifeless> explaining is lightning fast, executing is slow
<wgrant> Well, 8ms, but still fast.
<lifeless> on qas
<wgrant> Odd.
<lifeless>                                                      Index Cond: ((public.teamparticipation.team = public.distribution.owner) AND (public.teamparticipation.person = 21997))
<wgrant> Any idea how that could be?
<lifeless>  Total runtime: 0.754 ms
<lifeless> (64 rows)
<lifeless> unreported time is usually either startup or wire-transmission overheads
<wgrant> But the returned value is [(1,)]...
<lifeless> I didn't say it fitted ;)
<lifeless> what does this intend to do ?
<wgrant> It's the legacy bug visibility query.
<lifeless> 0.9ms to profile on wild
<wgrant> profile == explanalyze?
<lifeless> 386ms to execute
<lifeless> yes
<wgrant> WTF
<lifeless> that was local machine so no networking etc possible
<wgrant> With \timing on, how long does the EXPLAIN ANALYZE take?
<lifeless> 6.1ms
<wgrant> Not the time it gives, but the time it takes?
<lifeless> 6.1ms!
<wgrant> Bah.
<lifeless> Repeating the question presumes I misunderstood
<lifeless> I have timing on always ;)
<wgrant> So it's not some pathological query parsing or anything. it's just insane.
<lifeless> please tell me you're not going to put this into every bug query ?
<wgrant> What about https://pastebin.canonical.com/55126/?
<wgrant> That's a query that's already there.
<wgrant> Very similar.
<lifeless> I'd factor out the TP stuff into a CTE
<wgrant> Is it still slow?
<wgrant> So would I, but this whole thing is being deleted shortly.
<lifeless> 580ms
<lifeless> 0.7ms to profile it
<wgrant> explanalyze gives a quick plan, though?
<wgrant> Right.
<wgrant> WTF?
<wgrant> This making our bug pages 600ms slower than they have any justification for being.
<lifeless> that should be an explainanalyze
<lifeless> bah
<lifeless> a union all
<lifeless> but it doesn't correct the issue
<wgrant> The first query I gave you is now on qastaging. I landed it because it performed really well on DF :)
<wgrant> Right, I didn't write those conditions.
<wgrant> The second query is an existing one which is on prod.
<wgrant> The first is the condition for the second one being reused to implement Bug.userCanView.
<lifeless> dropping the bug columns doesn't help
<wgrant> So, any leads on the slowness? I can't reproduce on DF.
<wgrant> Hmm.
<lifeless> fiddling
<wgrant> Thanks.
<lifeless> 43ms do ?
<lifeless> https://pastebin.canonical.com/55127/
<wgrant> That's still crap, but how?
<lifeless> CTE
<wgrant> Indeed, that is faster...
<lifeless> 12ms hot
<wgrant> Unsurprising.
<wgrant> But how?
<lifeless> CTE, you can ignore the inner limit 1, thats not needed
<wgrant> Well, not surprising that it's faster.
<wgrant> But I didn't bother optimising because it was sub-ms.
<wgrant> Except that it's not.
<wgrant> Because EXPLAIN ANALYSE is a lie.
<wgrant> lifeless: https://pastebin.canonical.com/55128/
<lifeless> so, there is probably an execution bug in pg
<lifeless> and its probably fixed in 9.3
<wgrant> 9.3 is out!?
<wgrant> You mean 9.1?
<lifeless> no
<lifeless> That was called humour
<wgrant> Hah
<lifeless> 181ms cold
<lifeless> and hot
<wgrant> This is the real one that will be executed for !~launchpad on prod.
<wgrant> Hmm.
<wgrant> Not sure if worth rolling back...
<lifeless> hah
<lifeless> I fluffed my edits
<lifeless> check the bugsub clause
<wgrant> Heh
<lifeless> still 12ms
<wgrant> Does that fix it back to the old speed?
<lifeless> returns 0 rows
<wgrant> That's wrong.
<wgrant> I have a subscription through a team.
<lifeless> on qas ?
<wgrant> ~ubuntuone, in particular.
<wgrant> Yes.
<lifeless> 13ms 1 row
<lifeless> fixed
<wgrant> Damn.
<lifeless> https://pastebin.canonical.com/55129/
<lifeless> 4ms for the new one CTEified
<wgrant> "new one" == only two things being unified?
<lifeless> yes
<wgrant> Thanks.
<lifeless> 4 with and without UNION ALL
<lifeless> https://pastebin.canonical.com/55130/
<wgrant> So, this is a bit of a worry :/
<lifeless> whats the current live query ?
<wgrant> Bug.userCanView on prod is currently an undefined number of queries, depending on caching.
<wgrant> Might get a ++oops++ to find out.
<lifeless> remember that we don't cache across requests
<wgrant> Yes.
<wgrant> But I mean it's not obvious from the method what will need to be done.
<lifeless> sure
<lifeless> ok, I'm going to go start doing limited test run bisects to find this rosetta issue
<wgrant> Good plan.
<lifeless> its reported test 12751. I think I'll power-of two working up, rather than shrinking
<lifeless> wow
<lifeless> pickledb is -not- what you'd think it is. Way to confuse.
<lifeless> it may still be terrible, but its a different sort of terrible
<wgrant> lifeless: How is https://pastebin.canonical.com/55131/ on prod? sub-ms on DF, 340ms in ++oops++.
<lifeless> 253ms
<lifeless> 160 repeated
<lifeless> 250ms third time
<wgrant> 1 row?
<lifeless> so very variable
<lifeless> yes
<wgrant> WTF?
<lifeless> I have drop lynne up the street
<wgrant> Let's run LP on mawson :)
<wgrant> k
<wgrant> Well.
<wgrant> Indeed, that bug renders in 0.81s on DF.
<wgrant> 2.5s on prod.
<wgrant> Ah, because I'm an admin on DF, so it skips the bits of the queries that make everything slow except not.
<wgrant> (but the queries I was timing before were from qastaging)
<wgrant> Hah.
<wgrant> 4 extra queries, but still 0.88s.
<wgrant> So, DF is three times faster than prod.
<wgrant> prod is pretty fucked.
<poolie> hi all
<wgrant> Morning poolie.
<poolie> so that was a bit of an anticlimax that the buildd upgrades did not fix all the memory issues
<poolie> we can investigate more
<poolie> locally
<poolie> i hope a second attempt won't take so long
<lifeless> wgrant: thats interesting
<lifeless> wgrant: unpleasant. But interesting.
<lifeless> wgrant: I suspect table/index bloat
<wgrant> That would be the leading theory, I expect, but the high variance suggests that it may be something else.
<lifeless> not really
<lifeless> I have a unified theory ;)
<lifeless> table bloat means more pages are consulted
<lifeless> consulting a *page* involves a lock
<wgrant> Ah.
<wgrant> Yeah.
<lifeless> pg lock scalability is known suboptimal (and under improvement)
<wgrant> We really should upgrade to 9.1 at some point.
<wgrant> I guess that can happen soon after slony1-2.
<lifeless> increasing the page count by (say) 100 fold would increase lock overhead by 1000 fold
<lifeless> bah 100 fold
<lifeless> anyhow, that would provide lots of room on a busy server (e.g. prod) for happening to not contend on those page locks
<lifeless> the CTE causes a more efficient scan of the persons rows after after that stops consulting that table at all
<wgrant> Yes.
<lifeless> thats my theory anyhow
<wgrant> Quite reasonable.
<wgrant> So, sounds like we should upgrade postgres and continue fixing LP to not be ORMish.
<lifeless> its a 72MB table
<wgrant> What is?
<wgrant> TP?
<lifeless> yes
<lifeless> conceivable packable during FDT
<wgrant> Surely that's all going to be hot.
<lifeless> oh it will be
<lifeless> no disk IO at all I expect
<poolie> huwshimi: hi, maybe we can have a talk about markdown together some time
<lifeless> bah, can't find the blog post
<lifeless> wgrant: anyhow, fseek() takes out a kernel lock
<lifeless> wgrant: and postgresql fseeks(-1) to get the size of files it 'reads'
<poolie> a per-file lock, or a larger scope than that?
<lifeless> poolie: I don't remember the analysis
<wgrant> lifeless: Can you check bloat on tables?
<lifeless> yes, if I poke around a bit
<wgrant> 00113-00338@SQL-main-slave SELECT DISTINCT ON (Person.name, BugSubscription.person) Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname, Person.hide_email_addresses, Person.homepage_content, Person.icon, Person.id, Person.logo, Person.mailing_list_auto_subscribe_policy, Person.merged, Person.mugshot, Person.name, ...
<wgrant> ... Person.personal_standing, Person.personal_standing_reason, Person.registrant, Person.renewal_policy, Person.subscriptionpolicy, Person.teamdescription, Person.teamowner, Person.verbose_bugnotifications, Person.visibility, BugSubscription.bug, BugSubscription.bug_notification_level, BugSubscription.date_created, BugSubscription.id, BugSubscription.person, BugSubscription.subscribed_by FROM Bug, BugSubscription, Person, TeamParticipation ...
<wgrant> ... WHERE (Bug.id = 597155 OR Bug.duplicateof = 597155) AND BugSubscription.bug = Bug.id AND TeamParticipation.person = 21997 AND (TeamParticipation.team = BugSubscription.person) AND (Person.id = TeamParticipation.team) ORDER BY Person.name
<wgrant> Fail.
<wgrant> Didn't meant to paste all that.
<wgrant> But anyway, that's 225ms on prod, 4ms on DF (from ++oops++)
<lifeless> oh cool
<lifeless> index-only scans is in tip
<wgrant> Nice.
<StevenK> wgrant: When I hung around on #linux/ircnet years ago -- we had a term for that; "slammermaus": a mouse that randomly selects great gobs of text and pastes it into the channel.
<wgrant> And then the big query I asked about second ("SELECT BugTask.blah ... WHERE (massive union)") is 420ms vs 5ms,
<wgrant> Those two queries make up half the render of time of the bug when prod is hot.
<lifeless> wgrant: http://pgsnaga.blogspot.com/2011/10/index-only-scans-and-heap-block-reads.html
<wgrant> And there's still another 200ms I need to track down.
<wgrant> Ahhh very nice.
<wgrant> Although this is going to make vacuums even more critical for performance.
<lifeless> open question is if autovaccuum manages
<wgrant> Exactly.
<wgrant> lifeless: I guess it's good that we can reproduce this on qastaging.
<lifeless> tes
<wgrant> Without those two bad queries, we could reasonably easily get such bugs below 0.5s.
<wgrant> Which is still terrible, but not as bad as it is now.
<wgrant> Anyhow.
<wgrant> I might land that CTE.
<wgrant> Which will fix one of the queries.
<wgrant> And add another instance of it.
 * lifeless headdesks
<lifeless> running test_rosetta_branches_script before $1_oops causes the too-many-oops situation
#launchpad-dev 2012-10-22
 * wgrant finishes another gardening round
<jelmer> \o/
<wgrant> One deployment and 7 further critical fixes until we get back to the first half of last year
<jelmer> in terms of critical bugs you mean?
<wgrant> Yeah
<wallyworld_> wgrant:  have you used storm's ClassAlias? I have an issue where it is spitting out the name of the original table, rather than the alias, when outputting a column. my usage seems to be in line with how it's used elsewhere
<wgrant> wallyworld_: There's a bug with ClassAliases and References
<wgrant> wallyworld_: use the _id column instead
<wallyworld_> oh, awesome
<wallyworld_> thanks
<StevenK> Storm has bugs? I'm shocked
<wgrant> eg. ClassAlias(Person).teamowner will compile to Person.teamowner, ClassAlias(Person).teamownerID will be like "_1".teamowner as you'd expect
<nigelb> lol
<wgrant> Bug #682989'
<wgrant> Bug #682989
<nigelb> StevenK: How was your vacation? :)
<StevenK> nigelb: Still on it
<nigelb> Hah.
<StevenK> Still in Adelaide, even
 * nigelb really needs to plan an Australian vacation
<wallyworld_> wgrant: right, that's the exact issue, thanks. i should have asked earlier. i thought it was me. oh well
<wgrant> wallyworld_: It's not you, it's me.
<wgrant> s/me/Storm/
<wallyworld_> yeah
<StevenK> Flight was 1pm, but has been delayed due to an engineering issue after they had us wait on the tarmac for an hour
<StevenK> Hopefully we'll board at ~5pm ADL time
<wgrant> Is that +9.5 or +10.5 atm?
<wallyworld_> wgrant: do you know a way to get store.find() to allow a "select 1 from where..." without a table. it's legal postgres sql but storm complains
<StevenK> +10.5, it's currently 4:40pm
<wallyworld_> bah, without the from of course
<wgrant> wallyworld_: What are you trying to do?
<wallyworld_> select 1 where exists(some condition)
<wgrant> Specifics :)
<wallyworld_> only returns one or zero rows
<wgrant> There's probably a better way
<wallyworld_> that's what the tableless select is for
<wallyworld_> in oracle it would be "select from dual..."
<wgrant> If it's literally a SELECT 1 WHERE EXISTS (some subexpression);, just remove the outer expression
<wallyworld_> i just want a true/false if some condition exists in the database
<wgrant> SELECT 1 FROM blah WHERE blah LIMIT 1;
<wgrant> An exists check without the EXISTS
<wallyworld_> i could add limit 1 i guess, but i would be adding a surperfulous table
<wgrant> No
<wgrant> I mean with the inner expression
<wgrant> You don't need the outer one
<wgrant> What's the statement that you're trying to Stormify?
<wallyworld_> this works now: https://pastebin.canonical.com/77001/
<wallyworld_> but i don't need the table in the outer select
<wgrant> I'm confused
<wallyworld_> i could put any table in the using() and it would work
<wgrant> Isn't that just "return not Store.of(self).find(SourcePackageRelease, SourcePackagePublishingHistory.sourcepackagereleaseID == SourcePackageRelease.id, Archive.id == SourcePackagePublishingHistory.archiveID, *clauses).is_empty()"?
<wallyworld_> probably, i didn't know about is_empty
<wallyworld_> is that efficient?
<wgrant> It's shorthand for SELECT 1 blah blah blah LIMIT 1
<StevenK> Not when you're linking SPPH and SPR
<wallyworld_> it will be a whole lot better than that is done now luckily
<wgrant> Even if it didn't exist, I'd just manually do a SELECT 1 FROM spr JOIN spph blah blah blah LIMIT 1
<wgrant> No need for a subquery
<wallyworld_> i wasn't sure how ggod postgres would be at that
<wallyworld_> i assume it will be smart about seeing the limit 1, and stop computing as soon as it finds a matching row
<wgrant> Right
<wgrant> Just as it doesn't compute the entire result of Ubuntu's bugs just to show the first 75 of them
<nigelb> So, who's the new product manager for LP?
<wgrant> Nobody yet
<nigelb> And no new lifeless either?
<wgrant> Also not yet
<nigelb> :)
<wgrant> wallyworld_: btw
<wgrant> In [14]: list(store.execute(Select(1, where=Exists(Select(SourcePackageRelease.id, tables=[SourcePackageRelease], where=True)))))
<wgrant> Out[14]: [(1,)]
<wgrant> In [15]: list(store.execute(Select(1, where=Exists(Select(SourcePackageRelease.id, tables=[SourcePackageRelease], where=False)))))
<wgrant> Out[15]: []
<wgrant> It works fine without tables
<wgrant> Not sure what you were running into
<adeuring> good morning
<czajkowski> morning
<czajkowski> anyone know which team worked on the hide a commetn feature on LP bugs ?
<czajkowski> *comment
<bigjools> bzr annotate is your friend
<czajkowski> bigjools: morning
<bigjools> g'day :)
<czajkowski> bigjools: you at the hotel ?
<bigjools> sprinting, yup
<wgrant> czajkowski: That was finished off by Purple near the end of one of the subfeatures
<wgrant> I'm not really happy with how it turned out, but such is life
<czajkowski> wgrant: was just curious as there is a bug/feature issue being reported on it
<czajkowski> just wondered if you knew more about it
<cjwatson> Could I have a DB patch number for work on bug 1068071, please?
<_mup_> Bug #1068071: Need facility to redirect Ubuntu uploads to non-release pocket <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1068071 >
<czajkowski> cjwatson: you need jtv or wgrant I think as stub is away this week
<cjwatson> czajkowski: Any LP developer can claim a patch number.
<cjwatson> That's why I just asked generally.
<cjwatson> You only have to be in ~launchpad.
<czajkowski> oh ok.
<wgrant> cjwatson: Sure
<wgrant> cjwatson: 2209-36-0 is yours
<cjwatson> Thanks
<adeuring> abentley: https://code.launchpad.net/~adeuring/launchpad/sec-adapter-projectgroup-milestone/+merge/130800
<abentley> adeuring: Did you consider using DelegatedAuthorization?
<adeuring> abentley: not, i was not aware of this
<abentley> adeuring: I think it simplifies things a bit.
<adeuring> right
<adeuring> abentley: changed
<abentley> adeuring: I think milestone.userCanView is dead code now.
<adeuring> abentley: gahh, forgot to save that file before commtting...
<adeuring> done now
<abentley> adeuring: Cool.
<abentley> adeuring: You're indirectly asserting what the contents of "IProjectGroupMilestone" should be.  I'm okay landing this way, but let's talk about this in Copenhagen.  I bet we can come up with an equivalent that takes IProjectGroupMilestone as its input.  Like verifyObject.
<adeuring> abentley: sure
<abentley> adeuring: r=me.
<adeuring> thanks
<abentley> adeuring: np.
<sinzui> jcsackett, do you have time to talk?
<jcsackett> sinzui: sure, one moment while i find my phone.
<sinzui> https://bugs.launchpad.net/launchpad/+bug/164530
<_mup_> Bug #164530: Translation import queue showing broken links <lp-translations> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/164530 >
<sinzui> https://bugs.launchpad.net/launchpad/+bug/227380
<_mup_> Bug #227380: SourceForge ExternalBugTracker doesn't handle non-existent bugs <bugwatch> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/227380 >
<sinzui> jcsackett, https://bugs.launchpad.net/launchpad/+bug/227380
<_mup_> Bug #227380: SourceForge ExternalBugTracker doesn't handle non-existent bugs <bugwatch> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/227380 >
<abentley> adeuring: Apparently, DelegatedAuthorization is also performance improvement, because it allows LaunchpadSecurityPolicy to cache Product security checks.
<adeuring> abentley: right
<sinzui> jcsackett, https://bugs.launchpad.net/launchpad/+bug/924292
<_mup_> Bug #924292: TypeError raised during email authentication when the email contains non-ascii characters <email> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/924292 >
<jml> for IBranchMergeProposal.createComment, 'subject' is mandatory. How do I specify "just use the default subject that would be used if I were commenting through the web UI"?
 * jml experiments
<cjwatson> jml: the guard for that in the model code is "if not subject:", so subject="" should work for that
<jml> cjwatson: thanks. subject=False seems to work too.
<deryck> abentley, adeuring, rick_h_ -- hi, guys.  I reviewed bugs for the product relationship, and as near as I can tell, there's only one issue.
<cjwatson> lib/lp/app/javascript/comment.js uses subject: ''
<nigelb> cjwatson: Interesting work on archive for raring.
<rick_h_> deryck: cool
<deryck> abentley, adeuring, rick_h_ -- and I filed a bug and added a card to the board, but it's not beta critical.
<nigelb> cjwatson: Is this the LP work you were doing this cycle?
<abentley> deryck: Great.
<deryck> abentley, adeuring, rick_h_ -- it's a bit of a weird situation to reproduce, so we can get it post beta.
<adeuring> deryck: ok
<cjwatson> nigelb: There's only quite a small LP component to this, since the copy system is now fairly flexible.  Just a few bug fixes and tweaks.
<nigelb> cjwatson: Ah.
<cjwatson> nigelb: I was intending to finish off the replace-archive-admin-shell-access work if I could.
<cjwatson> (Which is basically chroot management and copy archives.)
<cjwatson> Nothing hugely exciting.
<nigelb> That means AA work will not need shell access?
<jml> Hah, no.
<jml> False does not work
<jml> cjwatson: ta. will try that.
<cjwatson> nigelb: It mostly no longer does.
<cjwatson> Only a very small number of rare operations.
<nigelb> Aha
<nigelb> Oh, UDS is european this time. I should try to listen to a few sessions at least.
<czajkowski> sinzui: did you get the Rt I created to track the translation import issue that was logged via Answers ?
<sinzui> No,  Do I need once since I need to track it on my kanban board and I have that card
<sinzui> already assigned
<czajkowski> sinzui: well was trying out this new policy where I can track an issue that I raise with maintenance
<czajkowski> that way instead of poking for an update I can check the RT
<czajkowski> like all the other depts do in canonical
<sinzui> We will update it when we do something next
<czajkowski> thank you
<czajkowski> now to go take the 3 year old for a walk who has been sitting patiently with her new kite
<czajkowski> toodles
<cjwatson> DatabaseSchemaChangesProcess still has some stuff about slony and the todrop namespace, which seems to have been removed in devel r15757.  Should that be removed from the process?
<sinzui> wallyworld_, this is the list of related issues https://bugs.launchpad.net/launchpad/+bugs?field.tag=related-projects-packages
#launchpad-dev 2012-10-23
<StevenK> wgrant: Hmm, I thought we already had a script to clean up failed translations imports? Ref bug 576822
<_mup_> Bug #576822: Add an option to clean up after a failed run of copy-translations-from-parent.py <lp-translations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/576822 >
<wgrant> StevenK: I think there was a one-off script for precise
<StevenK> 450282.22s  OOPS-674539797ca55814a0c0894007b34498  Unknown
<StevenK> BLINK
<StevenK> 5 days, 5 hours
<lifeless> thats some request
<wgrant> oooooh
<wgrant> checkwatches unhung
<wgrant> That explains a lot of the other OOPSes
<StevenK> Right
<StevenK> I just loaded that OOPS
<wgrant> So
<wgrant> Next step
<wgrant> Work out what bugwatch 57103 is
<wgrant> bugzilla.filesystems.org #632
<wgrant> The host responds, but doesn't look much like a bugzilla at all really
<lifeless> honeypot?
<wgrant> Ah, on https there's a bugzilla with a bad cert
<StevenK> But 5 days 5 hours? Surely we should timeout much quicker
<wgrant> Certainly
<wgrant> But should and do are different
<wgrant> Particularly when it comes to checkwatches
<StevenK> Indeed
<wgrant> That's the only bugwatch from that tracker
<StevenK> wgrant: You've commented on bug 758258 about a migration
<_mup_> Bug #758258: buildfarmjob schema is inefficient for reporting <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758258 >
<wgrant> That's a reasonably big effort still, so it remains suspended until we run out of low-hanging fruit
 * StevenK has been looking to no avail for low-hanging fruit
<wgrant> There's a lot of fairly easy timeouts and OOPSes left
<wgrant> 2012-10-17 10:39:23 INFO    Updating 1 watches for 1 bugs on https://bugzilla.filesystems.org
<wgrant> 2012-10-22 15:44:03 INFO    An exception was raised when updating https://bugzilla.filesystems.org/ (OOPS-674539797ca55814a0c0894007b34498)
<wgrant> 2012-09-24 07:11:24 INFO    Updating 1 watches for 1 bugs on https://bugzilla.filesystems.org
<wgrant> That same bugtracker caused the earlier hang
<wgrant> 2012-10-23 04:03:58 INFO    Updating 1 watches for 1 bugs on https://bugzilla.filesystems.org
<wgrant> and now we wait..
<wgrant> :(
<wgrant> 2012-10-23 04:04:10 INFO    Error updating https://bugzilla.filesystems.org/: Failed to parse XML description for https://bugzilla.filesystems.org bugs [u'632']: mismatched tag: line 101, column 4
<wgrant> It worked
<StevenK> What did you changed?
<wgrant> This was locally
<StevenK> Sure, but did you change anything
<wgrant> No
<StevenK> Ah
<StevenK> Maybe bugzilla.filesystems.org is blocking loganberry?
<wgrant> It even works via squid.internal
<StevenK> Bleh, are there no webservice tests for DSDs?
<wgrant> Not even under registry?
<StevenK> Not that I can see
<StevenK> I suspect it's a lazr.restful bug anyway
<StevenK> wgrant: I am disappoint. You didn't rip out contrib.glock in disgust while I was on holidays.
<wgrant> Indeed.
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/lazr.restful/lazr.restful-choice-marshaller-none/+merge/130932
<wallyworld_> StevenK: looks ok, but i don't know enough about the semantics of the code to be able to offer a considered opinion
<wgrant> StevenK: Is this circumstance actually valid?
<StevenK> wgrant: Bug 924291
<_mup_> Bug #924291: AttributeError: 'NoneType' object has no attribute 'title' on package differences page <derivation> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/924291 >
<wgrant> StevenK: It can happen, but is it valid?
<StevenK> parent_package_diff_status is a Choice of PackageDiffStatus, but the property returns None if the DSD's parent_package_diff is None
<wgrant> I thought it would usually escape earlier if the value was None
<StevenK> wgrant: Like I say in the description, unmarshall_to_closeup will return the entire vocab in any case.
<StevenK> I can change LP to just return PackageDiffStatus.PENDING and never-mind this lazr.restful branch
<wgrant> Ah, I see now reading the code that your argument makes sense
<wgrant> r=me
<StevenK> wgrant: Thanks
<wgrant> lifeless: Was lazr.postgresql intended for just schema migration stuff? There's some long lock/transaction tools that I need to find somewhere to put
<lifeless> wgrant: it was intended to pull out everything postgresql specific that doesn't rightfully belong in storm.
<lifeless> wgrant: or in LP itself.
<wgrant> Right
<wgrant> Thanks
<lifeless> and isn't pgbouncer specific
<wgrant> That's what I thought, but wanted to check
<lifeless> :)
<cjwatson> wgrant: So, early this month you fixed a packagediff generation bug that happened when we deleted things from -proposed a bit too early and then tried to copy the orphaned binary resulting from a build
<cjwatson> wgrant: packagediff generation is pretty late in the copy process; does that mean that this kind of situation should now completely work without us having to do weird copy dances?
<cjwatson> Archive.copyPackage and PCJ.attemptCopy don't seem to care about the publication status of the origin SPPH
<cjwatson> So they ought to be able to copy from a deleted SPPH that's had some binaries added to it later ... I think
<cjwatson> wgrant: Also, did you forget to close bugs following the recent NDT, possibly?  I'd have expected e.g. bug 1068616 to be closed
<_mup_> Bug #1068616: Archive.copyPackages hardcodes RELEASE pocket <package-copies> <qa-ok> <releases> <Launchpad itself:Fix Committed by cjwatson> < https://launchpad.net/bugs/1068616 >
<wgrant> cjwatson: Um, I was probably accidentally on the critical list, so missed non-criticals
<wgrant> As for the orphaned binary copy, yes, I believe it should all work now
<wgrant> But there's a catch
<cjwatson> Also bug 1014641 (critical)
<wgrant> There's no way to specify the source pocket today
<_mup_> Bug #1014641: +vouchers times out <bad-commit-16172> <qa-ok> <salesforce> <timeout> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/1014641 >
<wgrant> cjwatson: That's not deployed yet
<cjwatson> Oh, sorry, that's still on the report
<cjwatson> Confusing, it was in the LPS dump
<wgrant> cjwatson: Ah, yeah, it was landed and then reverted, and then landed again
<cjwatson> Oh, the ambiguous pocket thing - right, that sounds pretty easy to fix
<wgrant> The first two bits are deployed
<wgrant> Indeed
<wgrant> But reviving orphaned binaries should work fine now, if you copy back to -proposed first
<wgrant> Or if you fix the API to allow an explicit pocket
<cjwatson> Right, so I'm conscious that there are some situations where britney is going to create this kind of thing
<cjwatson> e.g. 1.0-1 ftbfs on powerpc, 1.0-2 initially does as well, britney sees it's no worse than before and copies, then somebody retries 1.0-2/powerpc and it works
<wgrant> Note that we haven't tested it, but packagediff generation is so close to the end of the process that it surely has to
<wgrant> work, that is
<cjwatson> I was contemplating beefing up test_incremental_binary_copies a bit
<wgrant> That might be handy
<cjwatson> It tests about 80% of this
<wgrant> While you're there it may also be worth sorting out the random override issue
<wgrant> Since that's a matter of a .order_by(BPPH.id) and a test
<cjwatson> Oh god this is a horrible ancestry nightmare isn't it?
<cjwatson> No?
<wgrant> I don't think so
<wgrant> It's not a total nightmare
<cjwatson> Last night I found vague allusions to a lot of this stuff in IRC logs from earlier in the month, but there wasn't enough to track them down to things I could fix on the spot
<wgrant> I don't think it'll do the override dance for an intra-archive copy, will it?
<wgrant> It'll just use whatever getBuiltBinaries returns
<wgrant> The problem with that method is that it doesn't guarantee which publication will be returned if there are multiple BPPHs for a BPR
<wgrant> So, if you try to copy an upload that's had its overrides changed, you're going to get a random assortment of overrides
<cjwatson> .order_by doesn't sound like a complete fix, then
<cjwatson> There won't be a BPPH until the build happens; and we probably want to pick up the overrides from some other architecture that's already built
<wgrant> getBuiltBinaries iterates through the entire set of binary publications, and IIRC uses the first one it finds for each BPR
<wgrant> So an order_by should fix it
<wgrant> Hmm?
<wgrant> The build will upload and do normal ancestry calculation, which is correct.
<wgrant> It's only copies that use getBuiltBinaries' arbitrary chooser
<cjwatson> Otherwise you change 1.0-2 to component: main for all the architectures that already exist, and then when 1.0-2/powerpc appears it'll flip back to whatever 1.0-1/powerpc was?
<wgrant> Well, yes, but that's an ancient bug
<cjwatson> Oh, right, I understand now
<wgrant> From early 2006
<wgrant> Which we've lived with for nearly 7 years :)
<wgrant> https://bugs.launchpad.net/launchpad/+bug/180218
<_mup_> Bug #180218: override mismatch race needs to be fixed <boobytrap> <lp-soyuz> <soyuz-publish> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/180218 >
<wgrant> There was an older one, once upon a time
<cjwatson> Yeah, not fixing that this week
<wgrant> But that's the current one about the situation you describe
<wgrant> Mine is perhaps more insidious and easier to fix
<cjwatson> Wrong overrides are a relatively minor problem in the development series; we have a report for them
<cjwatson> (i.e. override mismatches between architectures)
<wgrant> Sure
<cjwatson> And it's empty most of the time
<cjwatson> But yeah, I'm a small handful of LP commits away from having britney allegedly fully working, so just trying to think of ways it might unintentionally explodify the archive in advance ...
<cjwatson> I've been able to run it over nearly-trivial cases in raring-proposed in production and have it copy stuff successfully
<wgrant> As long as we have robust binary revival copies it should all be recoverable
<cjwatson> wgrant: could you have a look at https://code.launchpad.net/~cjwatson/launchpad/db-redirect-release-uploads/+merge/130857 ?
<wallyworld_> sinzui: lines 236-241 of the diff add the extra Related Projects link to the person index page
<sinzui> i suck. sorry wallyworld_
<wallyworld_> np, merely an oversight
<wallyworld_> easy to miss
<wallyworld_> the rest we can discuss tomorrow, you raise good points, i wasn't sure what to remove and what to keep
 * wallyworld_ off to bed
<sinzui> wallyworld_, oversight? I knew exactly where the change had to be, and I read the diff, and I looked over the diff for the very file you did change, then told you you needed to make the change that is clearly in the diff
<wallyworld_> don't worry, i'm not :-)
<cjwatson> wgrant: Can you see why debootstrap/1.0.42ubuntu1 and apparmor/2.8.0-0ubuntu6 didn't get announcement mails when they were accepted into raring-proposed?  I didn't think PROPOSED was special here.  I think I used the API to accept the former; not sure how the latter was done.
<czajkowski> cr3: your ml request should get done today
<cr3> czajkowski: sweet, thanks so much!
<czajkowski> cr3: well we're still checking it but updates will be on the question
<cjwatson> wgrant: So, this random overrides thing - I'm having a hard time seeing how to trigger it, since getBuiltBinaries requires the BPPHs it returns to match the distroseries and pocket of the SPPH.  How could there ever be more than one per BPN/architecturetag?
<cjwatson> Oh, changeOverride would create a new BPPH, wouldn't it, duh
<wgrant> cjwatson: It doesn't require that they're published, and changeOverride creates a new BPPH, indeed
<wgrant> cjwatson: Did you find out what happened to the debootstrap and apparmor announcements?
<cjwatson> No
<wgrant> k, will investigate
<cjwatson> Couldn't figure out where useful logs would be
<wgrant> You tell funny jokes
<wallyworld_> wgrant: StevenK: sinzui: http://askubuntu.com/questions/202677/nvidia-driver-doesnt-work-in-12-10
<sinzui> wgrant, I forgot to ask you to take a look at Colin's db branch in reviews
<wgrant> sinzui: It's already on my list after he pinged me about it last night :)
#launchpad-dev 2012-10-24
<wallyworld_> sinzui: http://people.canonical.com/~ianb/new-related-projects.png
<wallyworld_> remove bugs, questions, blueprints; add the role info as shown
<wallyworld_> i think this then fixes bug 996599
<_mup_> Bug #996599: Related projects does not say how the person is related <confusing-ui> <lp-registry> <related-projects-packages> <Launchpad itself:Triaged> < https://launchpad.net/bugs/996599 >
<StevenK> Bleh, I can't copy stuff into the bzr ppa
<wgrant> cjwatson: There's a small potential issue with your source suite branch: the source series is looked up in the target archive's distribution, so it can't be used to do an explicit copy from another distro
<wgrant> Not an issue in practice today, though
<StevenK> lifeless, maxb: Can one of you please copy bzr-git 0.6.9-1~bazaar1~precise1 from bzr-proposed Quantal to bzr Quantal so that the bzr PPA has an index for Quantal?
<lifeless> StevenK: you could just fix that bug
<StevenK> lifeless: About needing a package published to produce indices? Yeah, no.
<lifeless> yeah, you could
<StevenK> lifeless: And you could just copy the package, too. :-)
<wgrant> The bug's not very easy to fix, and it's not quite clear what the fix is
<lifeless> StevenK: so why do you need a quantal index? doesn't apt deal ?
<lifeless> wgrant: as in, do you want empty indices ?
<wgrant> lifeless: Right
<wgrant> lifeless: As a PPA owner I don't want to support natty
<wgrant> Why does my PPA have natty indices?
<lifeless> wgrant: you want just in time indices!
<wgrant> It confuses my users :(
<lifeless> wgrant: thats clearly orthogonal
<lifeless> wgrant: as otherwise any mistake leads to the same sitation and no way out
<wgrant> It may lead to an undesirable situation, sure
<wgrant> Doesn't mean it's entirely orthogonal
<StevenK> lifeless: No, apt does not deal
<StevenK> lifeless: And rf-setup will add the bzr ppa for <release>
<lifeless> hah
<lifeless> uhm
<lifeless> I don't want to put the wrong thing in there is all
<StevenK> lifeless: The version of bzr-git as published in Quantal is higher anyway, we just need *something* copied in
<lifeless> done
<StevenK> lifeless: Thanks
<StevenK> wallyworld_: You can change your sources.list entry for the LP PPA to quantal, it will work fine
<wallyworld_> StevenK: awesome thanks
<StevenK> It will probably want to upgrade a few things, just due to q sorting higher than p
 * StevenK tries to find a bugwatch with comments
<wallyworld_> wgrant: StevenK: do you have any recollection of what causes timeouts in merging people based on any past experience looking at oopses? i have a bug report but the oops has been deleted. i'm working on another bug and could potentially put in some effort to fix a merge timeout root cause as a drive by, but need info
<StevenK> wallyworld_: Lots and lots and lots of FKs
<wgrant> wallyworld_: They've mostly gone away now, since it's asynchronous
<wgrant> The FK thing is only relatively minor now
<wallyworld_> the bit i'm looking at is 189 queries to collect the fk references
<StevenK> wallyworld_: Person merging is a job now
<wallyworld_> yes but isn;t a check done first?
<wallyworld_> to see if merging is possible?
<wgrant> It has no reason to check all the FKs
<wgrant> But perhaps it does
<wallyworld_> i'll look a little more, it could be the 189 queries is done in the job so it doesn't matter
<wallyworld_> so much
<wallyworld_> so it raises a RuntimeError in IPersonSet.merge if the FK stuff is not as it should be
<wallyworld_> and that's done in the job
 * StevenK tries to work out how to delete a bugwatch
<wgrant> edit, delete
<wgrant> iirc
<StevenK> From a bug page?
<wgrant> Yeas
<wgrant> wallyworld_: Why don't we need to cascade, and are we sure that union will not perform absolutely terribly?
<StevenK> wgrant: Is an unlinked bugwatch with a comment a plausible scenario?
<wgrant> StevenK: Not linked to a task, just a bug?
<StevenK> wgrant: Well, I suspect it's invalid -- I'm using factory.makeBugComment(bug_watch=) rather than IBugWatch.addComment()
<wgrant> It may be invalid, but I'm not sure if it's relevant
<wgrant> 16:32:54 < wgrant> wallyworld_: Why don't we need to cascade, and are we sure that union will not perform absolutely terribly?
<StevenK> wgrant: bug 301740
<_mup_> Bug #301740: It should be possible to delete an auto-generated bugwatch that has comments attached <bugwatch> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/301740 >
<wgrant> Yes, but I'm not sure how that's relevant to "an unlinked bugwatch with a comment"
<StevenK> wgrant: Apparently, it OOPSes. Based on my reading of the code, it will only do that if it has no linked bugtasks and it has a bugmessage that references the bugwatch and IBugWatch.getImportedBugMessages() is empty
<wgrant> StevenK: Well, I'd check the DB to see if there are any bugmessages with a bugwatch but no remote_comment_id
<StevenK> So, there are. So, awesome!
<wgrant> So I saw
<StevenK> https://bugs.launchpad.net/hplip/+bug/1031503 is the most recent
<_mup_> Bug #1031503: hplip-3.12.6 fails to scan on Photosmart_C309a <HPLIP:New> <Gentoo Linux:Fix Released> < https://launchpad.net/bugs/1031503 >
<wallyworld> wgrant: sorry, missed that question before. i'm not sure i understand what you are asking wrt cascading?
<wallyworld> the indirect references are only applicable for merge jobs
<wallyworld> since there is the need to check any and all indirect delete cascading
<wallyworld> but here for this case, we only care about whether there are any referencing columns
<wgrant> Ah, fair point
<StevenK> Ah ha
<StevenK> I wonder if it's the  Awaiting synchronization comments
<wallyworld> wgrant: and the union has got to be better than 189 individual, separate queries, each using a separate db call. the union does the same thing, but just in one db call
<StevenK> Hm, how do I check that
 * StevenK cheats and uses mawson
<StevenK> Right, it is indeed, the Awaiting synchronization comment
<wgrant> wallyworld: It's better in terms of roundtrips, but it's not necessarily better in terms of optimisation
<wgrant> In fact it's almost certainly much worse
<bigjools> hello hackers
<wgrant> Morning bigjools
<StevenK> bigjools: How is sunny, downtown Copenhagen?
<bigjools> evening
<bigjools> StevenK: haven't seen any sunlight since I got here - and we're not downtown, it's another "Brussels"
<wgrant> In more ways than one
<StevenK> Hahaha
<StevenK> bigjools: Oh, you're 20 minutes in the forest?
<wgrant> In its defense, it was a pretty nice forest
<bigjools> StevenK: worse - in a faceless, flat area with no trees near the airport
<wallyworld> bigjools: stop whinging
<wgrant> :/
<bigjools> wallyworld: blow me
<wallyworld> i'll let StevenK do that
<wallyworld> take one for the team
<bigjools> haha
<StevenK> But wallyworld will enjoy it much more
<wallyworld> only if i skipped breakfast
<wallyworld> dog barking, gotta investigate
<StevenK> wgrant: Isn't there already a bug for LocationError: (None, 'builder')
<StevenK> Found it, since it's Fix Released
<nigelb> quite a crowd at UDS ;)
<lifeless> nigelb: are you there ?
<nigelb> lifeless: No. I was commenting on the joins ^^
<rick_h_> party time!
<cjwatson> wgrant: The missing -proposed announcements were actually a list mail configuration bug.  Apparently raring-changes is misconfigured and is requiring manual moderation.
<cjwatson> I hadn't realised that and will chase it up at some point, since that's clearly ridiculous.
<wgrant> cjwatson: I was going to ask you to check that, since I could see nothing else wrong
<wgrant> Though I thought the settings were carried across from quantal-changes
<cjwatson> So did I.  But IS denied knowledge.
<cjwatson> I'll try a different sysadmin at some point :-)
<wgrant> Heh
<cjwatson> How do I 'bzr lp-land' onto db-devel?  I've already run the full test suite.
<wgrant> It'll detect from the MP
<wgrant> So 'bzr lp-land'
<cjwatson> Ah, OK
<StevenK> cjwatson: ec2 land and bzr lp-land will prod at the relevant MP to work out how to land it. bzr pqm-submit has to be told by hand.
<cjwatson> wgrant: Thanks for spotting that bug in my copy-explicit-pocket branch.  How does https://code.launchpad.net/~cjwatson/launchpad/copy-explicit-pocket-2/+merge/131145 look?
<rick_h_> anyone know what I'm missing in my locations.conf to get a "ERROR: No PQM submission email address specified" when lp-land'ing?
<rick_h_> it's copied from my working install from before, I've got email, pqm_email, and pqm_user_email all set in there
<cjwatson> pqm_email = Canonical PQM <launchpad@pqm.canonical.com>
<wgrant> cjwatson: r=me, thanks
<rick_h_> hmmpqm_email = Launchpad PQM <launchpad@pqm.canonical.com>
<rick_h_> grrrr
<StevenK> rick_h_: locations.conf is set by path, perhaps that's different?
<wgrant> rick_h_: Is your Launchpad tree in the same spot?
<rick_h_> wgrant: yea, it's the default setup from a rocketfuel
<StevenK> Hah, wgrant and I come to the same conclusion at roughly the same time.
<wgrant> rick_h_: does 'bzr config' in the branch show pqm_email?
<rick_h_> ah, I see part of my problems I think.
<rick_h_> wgrant: no, don't see it
<StevenK> Then check the headers in .bazaar/locations.conf versus your current path
<rick_h_> StevenK: ah, you're right gotcha. I forgot I had the ubuntu user on the vm
<cjwatson> So, if buildbot passes, would there be any chance of an FDT today?  I'm running out of time to get the -proposed migration stuff working before UDS :-)
<rick_h_> oh son of a ...
<wgrant> cjwatson: We're going to miss the 10UTC window, but we can probably just ignore that and do it at our leisure
<wgrant> The next window (in 8ish hours) is unstaffed
<wgrant> Apart from sinzui and jcackett
<wgrant> Otherwise we can do the window in 16 hours
<rick_h_> wgrant: StevenK thanks guys, think I got it in now.
<wgrant> rick_h_: Great
<wgrant> cjwatson: So, when do you expect to have the code changes ready/
<cjwatson> Heh, I'm just pushing a first cut as I type
<StevenK> Ah, so then it's 'RSN'
<cjwatson> (In the expectation that it will need some review fixes, mind)
<wgrant> cjwatson: You seem to log the warning twice
<cjwatson> Oh, because of the nascentupload / uploadprocessor split, I guess
<cjwatson> Maybe this is a case where a doctest would actually be useful
<cjwatson> But I'll see what I can pick out of test_uploadprocessor
<cjwatson> Hmm, I only see one warning ...
<cjwatson> Also, there seems to be a DB patch already ahead of mine in the queue - or was that applied live?
<wgrant> cjwatson: Well, two places in the code log it.
<cjwatson> Yeah, I was just trying to work out why I wasn't seeing it in practice, but I've got it now
<wgrant> Ah, and indeed there is still one in the queue, hmm
<cjwatson> wgrant: fixed
<wgrant> cjwatson: Indeed, thanks
<wgrant> cjwatson: I'm not quite seeing how queue admins manage to bypass it
<cjwatson> *blink* That's a good question, so how come that test passes ...
<wgrant> It looks like it'll just reject, which must mean the last test is wrong
<wgrant> Exactly
<cjwatson> Oh, blast, silly CannotCopy handling
<cjwatson> Yeah, right, fixed the test
<cjwatson> (locally - working on the model)
<wgrant> You had me quite worried there for a few minutes, as I struggled to work out how I had so fundamentally misunderstood the uploadprocessor checks :)
<StevenK> dpm: O Hai. We've updated translations for raring on staging. The pages might require a bunch of reloads to load, but can you check it out?
<dpm> heyt StevenK, thanks, looking...
<StevenK> dpm: I'm about to crash into bed, but if it looks good, we should be able to run the script on production tomorrow.
<StevenK> dpm: Drop me a note here, and I'll see it when I wake.
<dpm> StevenK, I'm looking at https://translations.staging.launchpad.net/ubuntu/raring and I don't see any translations yet. Does it need to take a while to update the stats?
<cjwatson> wgrant: OK, should be fixed
<StevenK> dpm: We haven't actually turned it all on, we've only run copy-translations-to-parent on staging, but the +translate URLs for raring source packages should work
<dpm> StevenK, I can see that the templates have been copied ok from quantal at https://translations.staging.launchpad.net/ubuntu/raring/+templates but I the translations themselves still don't appear on the stats page at https://translations.staging.launchpad.net/ubuntu/raring
<dpm> I'll have another look later on in case it takes sometime for the stats to be updated or translations to be imported
<StevenK> dpm: Ah, those are run by cron, which we don't run on staging
<StevenK> s/run/updated/
<wgrant> It would be worth running that manually.
<dpm> StevenK, gotcha. In any case, without the stats it's really difficult to see whether the translations look overall right. I'm trying to look at individual templates right now, but it's timing out
<wgrant> dpm: Templates should load after a refresh or two
<dpm> thanks wgrant, that did it.
<StevenK> wgrant: What does the stats? cronscripts/rosetta-pofile-stats.py ?
<wgrant> StevenK: Heh heh heh
<wgrant> You may recognise that script
<StevenK> Quite
<dpm> StevenK, so at a quick glance I can confirm that the translations are indeed in some individual templates such as https://translations.staging.launchpad.net/ubuntu/raring/+source/gfxboot-theme-ubuntu/+pots/bootloader/ca/+translate - but updating the overall stats would give us a much better picture
<StevenK> Ah ha, cronscripts/update-stats.pu
<StevenK> *py
<wgrant> No
<wgrant> That's the global stats
<StevenK> wgrant: Then it is the great evilness that is rosetta-pofile-stats ?
<wgrant> StevenK: I suspect so
<wgrant> I doubt the copier creates jobs
<StevenK> IDistroSeries.updateStatistics() did look relevant, though
<wgrant> That mostly just creates DistroSeriesLanguages
<wgrant> We care about POFile stats
<StevenK> wgrant: What about the last bit, which is updating the message counts for the series itself?
<wgrant> Not very interesting for this
<wgrant> Well, slightly
<wgrant> But it's not the main thing that matters
<StevenK> dpm: So with the stats updated it should be identical to precise?
<StevenK> Er, quantal, even
<dpm> StevenK, yes, identical to quantal, as the translations imports from package uploads are disabled for raring, so no new translations should have come in
<StevenK> dpm: Right, I'll sort that out -- I'll prod you tomorrow to have another look
<wgrant> StevenK: So, we could lower taxes to support POFileStatsJob creators, or we could be socialist and spend probably about 4 days running rosetta-pofile-stats
<dpm> ok cool, thanks StevenK :)
<StevenK> Ew, 4 days
<wgrant> It's probably faster to create jobs
<StevenK> wgrant: Let's talk about it on the call tomorrow
<wgrant> Indeedily
<rick_h_> gary_poster: ping, deryck wants to know where you are
<sinzui> jcsackett, r=me
<jcsackett> sinzui: awesome. off it goes.
<cjwatson> I'm having great difficulty figuring out the problem in bug 1070804.  Can anyone help me out?
<_mup_> Bug #1070804: Error notification by mails doesn't describe the error <email> <package-copies> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1070804 >
<cjwatson> I'm using http://paste.ubuntu.com/1302902/, which indeed causes the test to fail
<cjwatson> But CannotCopy is in PlainPackageCopyJob.user_error_types, which *should* suppress this
<cjwatson> If I use pdb to stop in JobRunner.runJobHandleError, the exception that's raised is an instance of CannotCopy, and job.user_error_types is indeed (<class 'lp.soyuz.interfaces.archive.CannotCopy'>,) - it's not the case that user_error_types is being defined in the wrong place, as far as I can tell
<cjwatson> But the except doesn't fire, and I get:
<cjwatson> (Pdb) p isinstance(info[1], job.user_error_types)
<cjwatson> *** TypeError: TypeError('isinstance() arg 2 must be a class, type, or tuple of classes and types',)
<cjwatson> which has me totally baffled because as far as I can tell it *is* a tuple of types
<cjwatson> ('isinstance(info[1], (CannotCopy,))' returns True)
<cjwatson> Is this a Zope weirdness, or a Python weirdness, or what?
<sinzui> cjwatson, I am still thinking about this. I think this is python, not zope, but I agree that (CannotCopy,) is tuple
 * cjwatson wonders if a debug build of Python will make life any clearer
<cjwatson> (But then I would have to rebuild all the extensions needed, argh)
<cjwatson> sinzui: Ah - job.user_error_types is security-proxied
<sinzui> buggery
<cjwatson> isinstance(info[1], removeSecurityProxy(job.user_error_types)) -> True
<cjwatson> So is the right answer here to use 'except removeSecurityProxy(job.user_error_types) as e:' ?
<cjwatson> Seems a little clunky ...
<sinzui> that's yuck
<sinzui> y
<sinzui> I don't think any error should be wrapped in a proxy
<cjwatson> Well, the error isn't, but the attribute retrieved from job is
<sinzui> ah, be I see we got the job via a zope utility, so all attrs are wrapped
<cjwatson> (rSP there does work)
<cjwatson> Is there a way to declare some attributes as always being unproxied?
<sinzui> doesn't this mean that lazr.jobrunner.jobrunner never gets the tuple it expects
<sinzui> no, but there is a helper that will unwrap everything. we use in webapp.authenticate I think
<sinzui> cjwatson, we can create a property on our job class that unwraps and returns the error classes...
<sinzui> maybe we want the property to be on out base class, so that all subclasses define a private attr with the attr classes
 * cjwatson tries that
<cjwatson> haha, there's already one of these for retry_error_types, called retryErrorTypes
<cjwatson> So it kind of seems to me that to avoid surprising future developers we should add a userErrorTypes property to LP, deploy that, then add userErrorTypes support to lazr.jobrunner
<cjwatson> Does that make sense?
<cjwatson> Otherwise we have two very similar things using different patterns
<sinzui> we could avoid changing lazr by instead defining our user_error_types property on lp.services.job... the four cases that have the property get a leading underscore to make them private
<cjwatson> Sure, but then we'd have two things right next to each other that should really look the same but are different
<cjwatson> Is that worth it to avoid changing lazr?
<cjwatson> It sort of seems like a "future developers will curse thy name" to me
<sinzui> cjwatson, http://paste.ubuntu.com/1303118/ looks right to me because lazr.job is not zope, so the zope implementations are violating the contract
<cjwatson> By which I mean, user_error_types and retry_error_types are literally right next to each other in BaseRunnableJob
<cjwatson> They should surely be handled similarly
<sinzui> ah, yes, they must be treated the same
<cjwatson> (BaseJob, I mean)
<cjwatson> Actually, with the way the code is structured, it could work either way round; doing lazr.jobrunner first would make QA easier
<sinzui> right, so we want the same solution, and lazr.job has chosen its method approach
<cjwatson> Also, lazr.jobrunner is only at 0.9 on PyPI for some reason; what happened to 0.10?
<cjwatson> adeuring: ^-
<jcsackett> sinzui: if you have time, short review for you https://code.launchpad.net/~jcsackett/launchpad/email-authentication-type-error/+merge/131282/
 * sinzui looks
<sinzui> jcsackett, I have a script I use to send "volleys" of emails at Lp. We might be able to use to to contrive the example address in the bug
<sinzui> jcsackett, did I ever send you testemail.py
<jcsackett> sinzui: you did not, but it sounds promising.
<jcsackett> (or if you did, i have since forgotten).
<sinzui> I will send it
<jcsackett> awesome.
<sinzui> jcsackett, you might want to just run all the tests in lp.services.mail and on success, lp-land the branch
<sinzui> StevenK, https://bugs.launchpad.net/launchpad/+bug/401633 , I found people removed the unused attrs, but left the schema behind
<_mup_> Bug #401633: Remove POTMsgSet.potemplate from db schema. <lp-translations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/401633 >
<sinzui> StevenK, https://bugs.launchpad.net/launchpad/+bug/373269 is the bug I was thinking of about message-sharing
<_mup_> Bug #373269: Message sharing and POFile statistics <lp-translations> <message-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/373269 >
<sinzui> wgrant, https://wiki.canonical.com/Launchpad/Translations/UbuntuOpenings
#launchpad-dev 2012-10-25
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/db-destroy-potmsgset-potemplate/+merge/131294
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/deal-with-unsynched-bugwatch-comments/+merge/131293
<wallyworld> StevenK: shouldn't test_can_delete_watch() test that the bug watch has been deleted?
<StevenK> How do you suggest I test that something is deleted? :-)
<wallyworld> ask the db if it is there
<wallyworld> the doc test did it
<wallyworld> also, what about that bug which requires the Not()
<wallyworld> it seems the Not() is removed now
<StevenK> That bug is marked as Fix Released
<wallyworld> ah ok
<wallyworld> that's a problem in general - bugs get marked as fixed but the xxxs stay in the code
<wallyworld> maybe the bugs should have the xxxs listed
<StevenK> wallyworld: http://pastebin.ubuntu.com/1303889/
<wallyworld> that works i think, thanks
 * StevenK fixes it so it actually works
<StevenK> wallyworld: I like http://pastebin.ubuntu.com/1303891/ much better
<wallyworld> StevenK: my brain is dumb today - how does the branch fix the issue? it seems the same query is run as before the branch?
<StevenK> wallyworld: Today? :-P
<wallyworld> well, more than usual
<wallyworld> the second attempt is better yes
<mwhudson> huh
<mwhudson> https://dev.launchpad.net/CleaningUpOurCode talks about make xxxreport
<mwhudson> which is a Makefile rule that exists
<mwhudson> but doesn't work
 * wallyworld didn't know about that report
<StevenK> wallyworld: Right, so before the view code was using IBugWatch.getImportedBugMessages() which wanted both BugMessage.bugwatch == self and BugMessage.remote_content_id != None. The new method is IBugWatch.getBugMessages() which only checks BugMessage.bugwatch == self.
<mwhudson> wallyworld: you reviewed the branch that deleted it :-)
<StevenK> wallyworld: As a bonus, I refactored IBugWatch.getImportedBugMessages() to call IBugWatch.getBugMessages() to save LoC.
<wallyworld> mwhudson: oh, right. i didn't look at the contents. the file was just moved to lp-dev-utils
<wallyworld> StevenK: ok, thanks. obvious now. r=me
<StevenK> wallyworld, mwhudson: I've done a drive-by to drop that Makefile target in the branch
<StevenK> wallyworld: In the one you just reviewed, if you don't mind much
<wallyworld> sure, np. i didn't realise the makefile target was there. curtis just said to move the files across to lp-dev-utils which i did
<wallyworld> i'll update the wiki
<mwhudson> There are 5244 XXX comments in revno: 16183.
<StevenK> You're a little out of date
<mwhudson> only a couple of days i thought?
<StevenK> Only 5 revs, so like yesterday
<StevenK> Sorry, 6 revs
<sinzui> mwhudson, I think the page is somewhat outdated. I was given instructions to make it obsolete by tying all the XXXs there were about bugs to real bugs in Lp, then stop using the script
<sinzui> That was 3 years ago I think
<mwhudson> sinzui: yes, it all looks a bit stale
<StevenK> wgrant: Can haz db review?
<wgrant> StevenK: Sure
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/db-destroy-potmsgset-potemplate/+merge/131294
<StevenK> wgrant: Should I hold off landing that until tonight?
<wgrant> StevenK: Please do
<wgrant> There's a bit of a queue
<StevenK> Where a bit is 2 :-P
<wgrant> And no rush on this
<StevenK> https://oops.canonical.com/oops/?oopsid=OOPS-d79b482989c5e4175a55160a8e1aee5c
<StevenK> 5 second translations SQL statement, and I can't work out which bit of the code is doing it
<lifeless> StevenK: you've got the backtrace
<lifeless> sure
<lifeless> ly
<wgrant> Not if it's from a resultset that's lazily evaluated later
<StevenK> I've found the method that is directly respsonsible, anyway
<StevenK> ITranslationsPerson.suggestReviewableTranslationFiles()
<lifeless> wgrant: ugh true. Another reason to dislike such.
<StevenK> Yay Storm
<StevenK> ... This query is disgusting
<StevenK> wgrant: http://pastebin.ubuntu.com/1303986/
<StevenK> That's with it hot, cold is something like 116 seconds
<mwhudson> is something in there a view?  that seems surprisingly horrible
 * mwhudson is just waiting for a backup to finish before disappearing, don'
 * mwhudson is just waiting for a backup to finish before disappearing, don't let me distract you
<StevenK> mwhudson: Yeah, pulled in via TranslationsPerson:+index
<wgrant> Not that sort of view :)
<mwhudson> StevenK: i mean database view :)
<StevenK> Oh, no.
<StevenK> Thankfully
<mwhudson> also what happened here?
<mwhudson> ORDER BY POFile.date_changed; > 0uched >= '2012-03-16 06:00:00+00:00'tor.transla
<StevenK> That would be readline
<StevenK> http://pastebin.ubuntu.com/1303993/ better
<mwhudson> ah
<mwhudson> the query in the oops looks more horrible :)
<mwhudson> or indeed, that one
<StevenK> wgrant: So how do I start to work out what is causing the slowness instead of 'the whole thing' ?
<wgrant> Well
<wgrant> The whole thing
<wgrant> Work out what the query is being used for
<wgrant> Make sure Launchpad never thinks of doing it again
<StevenK> It's a table of suggestions they could work on
<StevenK> 'alsa-utils needs 1 string review in Spanish
<wgrant> StevenK: First step is to get a version of that query that doesn't make one wish to be blind
<wgrant> ie. indent it nicely
<StevenK> wgrant: http://pastebin.ubuntu.com/1304003/
<wgrant> It burns
<wgrant> Oh god
<wgrant> It burns
<wgrant> That's not indented, that's deindented.
<StevenK> You can blame sqlformat for that
<mwhudson> http://www.dpriver.com/pp/sqlformat.htm appears to do a better job
<wgrant> http://pastebin.ubuntu.com/1304010/
<wgrant> Although mwhudson's link isn't bad
<StevenK> http://pastebin.ubuntu.com/1304014/
<wgrant> StevenK: So, what's the query trying to do?
<StevenK> wgrant: It is trying to return a list of up to 9 projects/packages that the person could review strings on.
<wgrant> Right
<StevenK> wgrant: So, now that I've had lunch
<wgrant> StevenK: Indeed
<StevenK> wgrant: So I'm not comfortable just ripping out the functionality, but it only caused 21 OOPSes yesterday
<wgrant> StevenK: I'd see if the query performs reasonably for any people at all
<StevenK> wgrant: Based on the bug, it is only used if you load up your own +translations page
<wgrant> Sure
<wgrant> But more than that one person loads up their own +translations pages
<StevenK> 3501ms for me, since I know my DB ID
<StevenK> So, "No." ?
<StevenK>               ->  Index Scan using pofiletranslator__person__pofile__key on pofiletranslator  (cost=0.00..0.97 rows=1 width=4) (never executed)
<StevenK> Is that postgres' way of saying that I have no pofiles and so it doesn't need to index scan?
<wgrant> Well, it never got to a point where it needed to do that scan
<StevenK>                                        ->  Materialize  (cost=8940.46..12479.23 rows=24747 width=16) (actual time=4.288..25.864 rows=24756 loops=42)
<wgrant> Possibly because all the pofiles were already filtered out by that point
<StevenK> Hm, haven't seen that one before
<wgrant> StevenK: It's used when a join can use a reasonable (ie. fits in memory) subset of a table more efficiently than the whole one
<wgrant> So it'll fetch that subset into an automatic temporary table in RAM
<wgrant> Once, usually
<wgrant> Then the join can loop over that subset more efficiently
<wgrant> So it's like a seqscan, except over a subset of the table
<StevenK> Ah
<StevenK> So, the whole thing is a mess, and looks like it performs horribly no matter who the person is
<wgrant> Right
<wgrant> I'm not sure how much better we can actually make it
<wgrant> Given that what it's doing is fairly hideous
<StevenK> Yeah, 3546 ms for lifeless, since his ID is so trivial
<StevenK> wgrant: I should /+daily-builds it?
<lifeless> it should be better than it was :)
<wgrant> Only marginally
<StevenK> lifeless: Yeah, I think this query is so terrible that it doesn't matter who we ask it about
<wgrant> StevenK: Are you sure you got the right method?
<wgrant> I don't think it's the one you mentioned
<StevenK> Looks right to me, based on the call chain and the query
<wgrant> StevenK: I think it's getReviewableTranslationFiles, not suggestReviewableTranslationFiles
<wgrant> Indeed, the traceback confirms it's _review_targets
<StevenK> Hmmmm
<StevenK> I wonder if I've deleted too much
<StevenK> wgrant: Is this where you say "Impossible!" ?
<wgrant> No
<wgrant> This is very possible
<wgrant> There's one big simplification in getReviewableTranslationFiles
<wgrant> Which makes things much easier
 * StevenK calls revert a lot
<StevenK> wgrant: Now, share?
<wgrant> It's trying to find suggestions that the user can review, as you say
<wgrant> But, more specifically, the relevance is based on whether they've translated in them recently
<wgrant> The last join is a big win
<wgrant> As it restricts the set to those pofiles for which there is a recent pofiletranslator for the person
<StevenK> The method doesn't do that already?
<wgrant> It does.
<lifeless> but if you did it earlier, it might work better
<StevenK> wgrant: Oh, you're saying we shouldn't do that join?
<StevenK> This code isn't particularly clear to me, and my blocked sinuses are not helping matters.
<wgrant> What lifeless said
<wgrant> People generally translate fewer POFiles than there are POFiles in all of Launchpad
<wgrant> s/People/A single person/
<StevenK> IE, a CTE?
<wgrant> So it's likely to be more efficient to work from the list of recently translated pofiles
<wgrant> That would be a good start
<wgrant> It'll probably require a bit more fiddling
<wgrant> As there's a lot of joins, so it's not necessarily going to be able to optimise very well
<wgrant> But I'd start by trying to coerce it into starting from POFileTranslator
<StevenK> Given the function brings together the query via calling functions, this is going to be ... fun
<wgrant> (this time I don't have a <20ms solution yet)
<StevenK> Haha
<StevenK> Oh ugh, trying to pull out TeamParticipation makes it all unravel, because that's JOINed via TranslationGroup which is pulled in via Project/Distribution and then POTemplate
<wgrant> StevenK: Yeah
<wgrant> It's a bit ugly
<StevenK> I can see how starting from POFileTranslator.person and Translator.translator is a win, but I can't quite work out how to do that
<wgrant> Pull it out bit by bit until it works
<wgrant> Then try to compress it back into something that's not hideous
<wgrant> But still works relatively quickly
<wgrant> At this point we know the sort of plan we want
<wgrant> We just need to convince postgres of it
<StevenK> wgrant: http://pastebin.ubuntu.com/1304155/
<wgrant> I have a 15ms query that mostly works
<wgrant> By "mostly works" I mean "returns some similar results but isn't actually the same thing and I haven't worked out why yet"
<wgrant> Also it's pretty fugly
<StevenK> wgrant: http://pastebin.ubuntu.com/1304156/ is as far as I've gotten, but it's still 3150ms
<StevenK> So better, but still terrible
<wgrant> Oh
<wgrant> It helps if I use productseries rather than distroseries twice
<wgrant> But still, 30ms and it seems correct
<wgrant> I have four CTEs: recent_pofiles, reviewable_groups, reviewable_distroseries, reviewable_productseries
<StevenK> wgrant: Are you going to share, or do I need to rewrite my query until it matches? :-)
<wgrant> Just checking some stuff
<wgrant> eg. performance for other people
<wgrant> StevenK: http://paste.ubuntu.com/1304165/
<StevenK> (POTemplate.productseries, POFile.language) IN (SELECT * FROM translatable_productseries)
<StevenK> That's pretty awesome
<StevenK> And sneaky
<wgrant> I try.
<StevenK> Heh, it's as slow as the old hot query when it's cold.
<StevenK> If that sentence makes sense.
<StevenK> Hm, it seq scans both distroseries and distribution
<StevenK> But they aren't large
<wgrant> It can be indexed if needed, but those tables are so small it probably wouldn't use them
<StevenK> The SubPlan 5 and 6 are the above awesomeness I pasted?
<wgrant> Yeah
<StevenK>  Total runtime: 0.543 ms
<StevenK> For me
<wgrant> Right, because pofiletranslator will have nothing for you
<StevenK> Yeah, and it just skips large parts of the query
<wgrant> Note that pofiletranslator isn't even properly indexed, but people rarely have more than 5000 rows so it's still fast
<StevenK>                  ->  Bitmap Index Scan on pofiletranslator__person__pofile__key  (cost=0.00..5.43 rows=128 width=0) (actual time=0.056..0.056 rows=14 loops=1)
<StevenK>                        Index Cond: (person = 6874)
<StevenK> Looks indexed?
<wgrant> It's indexed *enough*
<wgrant> But it's not indexed properly
<StevenK> Ah ha
<wgrant>            ->  Bitmap Heap Scan on pofiletranslator  (cost=5.43..480.97 rows=6 width=4) (actual time=0.422..2.305 rows=215 loops=1)
<wgrant>                  Recheck Cond: (person = 1780257)
<wgrant>                  Filter: (date_last_touched >= '2012-03-16 06:00:00'::timestamp without time zone)
<wgrant> It finds all the pofiletranslator rows for the person, then filters them by date_last_touched
<wgrant> When it would save a couple of ms to have an index on (person, date_last_touched)
<wgrant> And if we were on 9.2, it'd be even faster with (person, date_last_touched, pofile)
<StevenK> Right
<wgrant> The precise query formulation is going to become a *lot* more important when we upgrade to 9.2
<wgrant> There's a lot of opportunity for big big wins
<StevenK> Index only scans have you salivating?
<wgrant> Only salivating? You underestimate me.
<StevenK> Haha
<StevenK> I'll leave the dirty jokes to wallyworld, he's better at them.
<wallyworld> no i'm not
<wallyworld> what does 9.2 offer?
<StevenK> Mainly, index only scans
<StevenK> IE, don't even hit the table, just pull the data straight from the index
<wallyworld> oh very cool
<wgrant> Right
<wgrant> It'll make teamparticipation/APG/etc particularly fast
<wgrant> But also lots of other things, with a bit of query tweaking
<wallyworld> bring it on then
<StevenK> wgrant: Holy crap, I'm down to 2 failures.
<StevenK> AND POFileTranslator.date_last_touched >= None)
<StevenK> Hahahaha
<wgrant> StevenK: There were failures? :(
<StevenK>   Ran 23 tests with 0 failures and 0 errors in 20.290 seconds.
<StevenK> wgrant: Yeah, but they were all due to my Stormification of the query
<wgrant> Ah
 * StevenK tries to work out how to drag ITranslationsPerson.suggestReviewableTranslationFiles() into line
<StevenK> The queries are virtually identical, the ORDER BY is different, and it uses a LEFT JOIN rather than a JOIN on POFileTranslator with the added condition of POFileTranslator.id == None
<StevenK> Ah, no, it's a LEFT JOIN rather than a JOIN, and the TRUE that was in the guts of the old query is POFileTranslator.id == None
<wgrant> Right, it's an antijoin
<StevenK> Except that we've CTE'd the query within an inch of its life
<StevenK> wgrant: I think my CTE antijoin doesn't quite work
<wgrant> StevenK: Well, do you have a plan that should be fast?
<StevenK> wgrant: I'm using the four CTEs of Doom with the LEFT JOIN change etc, but two tests fail, so it isn't quite right
<wgrant> StevenK: Remember that my query is based around the assumption that there a user contributes to only a tiny subset of the full set of pofiles.
<StevenK> wgrant: http://pastebin.ubuntu.com/1304235/
<StevenK> My guess is the LEFT JOIN in recent_pofiles is what is wrong
<wgrant> Right, it's wrong, but even if it was right it would probably be *extremely* slow
<wgrant> Because you'd be asking it to build a list of all pofiles that you haven't contributed to
<wgrant> Which is not a tiny subset of the full set of pofiles
<StevenK> At the moment, it's incredibly fast :-)
<wgrant> So the premise of my original restructured query is broken
<StevenK> Right, so pulling ITranslationsPerson.suggestReviewableTranslationFiles() into line is harder than I thought
<wgrant> What's it used for?
<wgrant> Can you see an optimisation strategy similar to the one we used for getReviewableTranslationFiles?
<StevenK> It is used in a view to find random translation targets for review
<StevenK> That the user hasn't contributed to
<wgrant> Right
<wgrant> That last line is the key
<wgrant> We were able to heavily optimise the previous query because we could restrict the search early to just a few thousand pofiles.
<wgrant> With this query we don't have that luxury.
<StevenK> Indeed
<StevenK> Let me revert the suggest bit
<wgrant> So attempting to reuse the optimisations that worked for the last query is perhaps slightly unwise
<wgrant> Since the base assumption of the optimisation strategy does not hold here
<rick_h_> benji___: ping
<benji> rick_h_: hi (I didn't hear your ping, for some reason my laptop does not beep when on battery)
<rick_h_> benji: np, nvm though. Was going to go through and run the tests for lpsetup but wanted to check how much of my systemit would take over and should I setup a virtualenv
<rick_h_> but going through the readme see what's up
<StevenK> dpm: O hai, can you look at staging? The stats should be fine
<dpm> morning StevenK, cool. Give me a minute and I'll have a look
<benji> rick_h_: yeah, the unit tests are safe; there is a functional test that is more intrusive, but it isn't run by default
<rick_h_> yea, the unit tests all pass fine
<czajkowski> morning
<StevenK> dpm: No news is good news?
<dpm> StevenK, sorry, I was on the phone, looking at it now
<dpm> StevenK, the stats are not 100% identical, but that's fine, as I guess there has been some translation activity in quantal in the meantime. So all looks good to me except for one thing: it seems the Contributors column hasn't been updated along with the other statistics, and it's empty
<wgrant> The near-daily ScrubPOFileTranslator job should take care of that after a few days, I believe
<wgrant> But we can't sensibly run that in full on staging due to resource constraints
<dpm> in that case, it all looks good to me
<StevenK> dpm: Excellent, thank you. I'm not really fan of running the job on prod tomorrow, how does Monday sound to you?
<dpm> StevenK, sounds perfect to me, I wouldn't consider this to be urgent
<StevenK> dpm: Oh, sure, but I'd like to get it off my plate. :-)
<dpm> absolutely :)
<cjwatson> wgrant: I've fixed the problem you spotted in redirect-release-uploads now; do you need to re-review?
<wgrant> cjwatson: Looks good, thanks
<wgrant> adeuring: Hi, will you have time to QA bug #1069826 today?
<_mup_> Bug #1069826: privacy aware security adapter for IProjectGroupMilestone <qa-needstesting> <Launchpad itself:Fix Committed by adeuring> < https://launchpad.net/bugs/1069826 >
<adeuring> wgrant: doing it right now :)
<wgrant> Lovely, thanks
<cjwatson> wgrant: Thanks.
<abentley> deryck: Could you please review lp:~abentley/launchpad/beta-banner ?
<cjwatson> abentley: I think https://code.launchpad.net/~cjwatson/lazr.jobrunner/userErrorTypes/+merge/131248 probably needs either you or adeuring to review it, ideally, since you're the owners on PyPI?
<abentley> cjwatson: be with you in a few minutes.
<cjwatson> Thanks
<abentley> cjwatson: Wow, took me a while to understand why we did that.  r=me.
<cjwatson> Thanks.  I doubt I can land and release it. :-)
<cjwatson> (It'll presumably want to go into lp-sourcedeps after release, too.)
<abentley> cjwatson: You can certainly land it.  I or abel would have to release it, though.
<abentley> cjwatson: I'll go ahead and do it all, this time.
<cjwatson> Oh, do such things just go through PQM?
<cjwatson> The trunk commit history all seems to be direct commits.
<cjwatson> Ah, I'm in ~canonical-launchpad-branches.  Fair enough.
<cjwatson> Thanks.
<abentley> cjwatson: np
<abentley> cjwatson: But I'm having trouble getting tests to pass, so the release may take a while.  You can generate your own tarball for now, if you want to keep hacking.
<cjwatson> No rush, I have plenty of other stuff to do.  Just wanted to make sure it wasn't blocked on me.
<cjwatson> Eek, I broke buildbot.  (I.  Hate.  Doctests.)  Could I have a review of https://code.launchpad.net/~cjwatson/launchpad/redirect-release-uploads-2/+merge/131383 so that I can land a testfix?
<cjwatson> It's a one-liner.
 * cjwatson trolls for a review of one-liner testfix https://code.launchpad.net/~cjwatson/launchpad/redirect-release-uploads-2/+merge/131383 in case the vagaries of the UDS network mean that more people can see it now
<jml> cjwatson: approved.
<cjwatson> jml: thanks
<cjwatson> jcsackett: Are you likely to be able to manage your QA today?  I'm hoping that I might be able to get the branch currently working its way through buildbot deployed a bit later.
<abentley> cjwatson: new release at http://pypi.python.org/pypi/lazr.jobrunner
<jcsackett> sinzui: can you send me testemail.py?
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: ~240
<sinzui> jcsackett, I will when qastaging is running. the schema needs patching
<jcsackett> sinzui: aaah.
<jcsackett> sinzui: nm, i found it in my mountains of email.
<jcsackett> of course, still waiting on qa server.
<sinzui> jcsackett, qastaging is back
<jcsackett> oh sweet.
<sinzui> jcsackett, my provider wont let me send. If you get that message, mark it qa-untestible
<jcsackett> sinzui: yeah, i'm having the same issue.
<jcsackett> ok, qa-untestable it is.
<rick_h_> jcsackett: for your eyeballs if you get a sec. Kind of small https://code.launchpad.net/~rharding/launchpad/limit_product_types_1066904/+merge/131368
<cjwatson> Could somebody add lazr.jobrunner 0.11 to download-cache?
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~240
<wgrant> cjwatson: Did someone get around to that download-cache change?
<cjwatson> wgrant: No
<wgrant> cjwatson: I don't see 0.11 on LP
<wgrant> I guess I'll grab it from PyPI
<wallyworld_> wgrant: why weren't packaging components, ie universe, multiverse etc, done as a enumerated type? the code is littered with string literals 'multiverse' etc
<wgrant> wallyworld_: Because they're not an enum, they're distro-specific config data
<wgrant> Launchpad unfortunately retains lots of hardcoded rules for Ubuntu because it was never done properly
<wgrant> Hence the hardcoded strings.
<wallyworld_> ok, thanks
<wallyworld_> i'm looking at soyuz code for the first time
<cjwatson> one of these days ...
<wgrant> I'm sorry for your loss.
<wallyworld_> well, if i can understand enough to fix the bug, i'll be happy
<cjwatson> (aka UE is not desperately keen on it being this way either, but not upset enough to demand it be fixed :-) )
<wgrant> s/distro-specific/distroseries-specific/, even
<cjwatson> if we ever get much further down the archive reorg then it might matter
<wgrant> Indeed.
<wgrant> cjwatson: lazr.jobrunner 0.11 is in download-cache
<cjwatson> Thanks
#launchpad-dev 2012-10-26
<cjwatson> wgrant: bzr mv lazr.jobrunner-0.11.tar.gz dist/
<wgrant> cjwatson: blargh, oops
<wgrant> cjwatson: Fixed, sorry
<cjwatson> np
<cjwatson> Huh.  I set lp.distributions["ubuntu"].redirect_release_uploads to True, and yet https://launchpad.net/ubuntu/raring/queue?queue_state=1 shows anna having gone to the Release pocket :-(
 * cjwatson wonders what he got wrong
<cjwatson> I don't suppose poppy uses a different tree?
<cjwatson> Well, except this is process-upload, not poppy as such
<cjwatson> Unless I was in slightly too much of a rush and beat the rollout to pepo ...
<cjwatson> That was it.  Phew.
<wgrant> Yeah, appservers are done early on
<wgrant> pepo not so much
<spm> cjwatson: I can probably fill you in on a secret. we webops watch for activity from certain users - you're one of the lucky ones - and deliberately break things when you're on, just for our twisted amusement.
<nigelb> lol
<spm> Some of the devs have gotten in on this, and do similar, but release NDTs at those times.
<nigelb> Heh, wgrant "I'm sorry for your loss"
<spm> cjwatson: http://www.xfos.com.au/tears.jpg so yeah, drink up mon!
<cjwatson> spm: I already assume computers are malevolent; operators being malevolent is just a logical extension
<spm> c
<nigelb> clearly spm's *s* expands to Simon, of BOFH.
<spm> the p is derived from Rick.
<spm> applied to, perhaps
<wgrant> Hm
<wgrant> Something increased the registry 99% render time from about 1.7s to 2.4s between the 19th and the 21st
<wgrant> And it remains high
<wgrant> That coincides well with a large number of private projects query changes
<wallyworld_> wgrant: if you have a spare moment today, could you take a look at my preliminary experimentation with setting the correct binary package components? the extra joins in the binary overrides query are needed to get the associated source package release to get the spr component, but performance may be an issue. is what i've done on the right track?
<wallyworld_> https://code.launchpad.net/~wallyworld/launchpad/package-defaults-192076/+merge/131509
<wgrant> wallyworld_: Hm, doing it in the override queries is an interesting approach
 * wallyworld_ has no idea about soyuz
<wallyworld_> i just tried to gather the info that was needed
<wallyworld_> maybe i can do another query later in the workflow
<wallyworld_> i'm also unsure how to do the distro validity check
<wgrant> wallyworld_: So, the main place this is a problem is in NascentUpload.processUnknownFile
 * wallyworld_ looks at that method
<wgrant> It's a bit sad that you've chosen possibly the worst bit of Soyuz to work on
<wallyworld_> i have no idea what's good and what's bad
<wgrant> The override stuff is particularly hideous, as it follows a design which I intended to be followed up by a major refactoring cleanup pass
<wallyworld_> the calculateSourceOverrides() method seems to implement the rules in bug 120052, so i was doing a similar approach for binary packages
<_mup_> Bug #120052: Component mapping for new source packages <lp-soyuz> <Launchpad itself:Fix Released by julian-edwards> < https://launchpad.net/bugs/120052 >
<wallyworld_> or so it seems to me anyway
<wgrant> wallyworld_: Right, but you'll see that NascentUpload uses UnknownOverridePolicy.getComponentOverride directly
<wallyworld_> does NascentUpload process binary packages?
<wallyworld_> since that's what the bug is referring to
<wgrant> Yes.
<wgrant> It processes both source and binary uploads
<wallyworld_> hmmm. ok. so binary packages are processed there and also in the code i've touched
<wgrant> The code you've touched is mostly for copies
<wallyworld_> copyBinariesTo()
<wgrant> Which are somewhat interesting, but post-date the bug by some years
<wgrant> They're not the common case, even today
<wallyworld_> so, if NascentUpload is changed, the copy stuff should be changed too i guess
<wallyworld_> to be consistent
<wgrant> Sort of
<wgrant> The copy overrides stuff is pretty awful
<wgrant> I don't quite remember how that side of things works with respect to the queue
<wallyworld_> so, i assume that NascentUpload has enough info to find the spr associated with a binary, then i can go from there
<wgrant> Right, when uploading a binary we always have a target build, which has a related SPR
<wallyworld_> how do i check if a component is valid for a distroseries?
<wgrant> That's recorded in the ComponentSelection table
<wgrant> However
<wgrant> however...
<wgrant> Are you using SPR.component?
<wgrant> I haven't read the MP in detail
<wallyworld_> yes
<wgrant> That's incorrect, sadly.
<wgrant> You need to find SPPH.component, for some relevant SPPH
<wgrant> Which is probably BinaryPackageBuild.current_source_publication
<wallyworld_> my brain hurts
<wgrant> SPR.component is set only once, when the SPR is created in the DB for the first time
<wallyworld_> ok, i haven't seen any data, i was just guessing based on attribute names
<wgrant> For lots of Ubuntu packages, they're first imported from Debian
<wgrant> So they appear first in main in the Debian primary archive, so SPR.component == main
<wgrant> Then they're copied into Ubuntu, where the SPPH.component is overridden to universe
<wgrant> The binaries should default not to main, but to universe
<wgrant> Even though SPR.component still == main
<wallyworld_> ok. could i perhaps just add an override mapping from main->universe?
<wgrant> You may now be getting the rationale behind my campaign to unify uploads and copies
<wgrant> Nope
<wgrant> It won't always be universe in Ubuntu
<wgrant> It could be main
<wgrant> or restricted
<wgrant> or multiverse
<wallyworld_> sure, but i mean just for the default mapping
<wgrant> So, the general rule is that if a new binary has no existing ancestral component, it should inherit the component of its build's current SPPH
<wallyworld_> ok, so i look in ComponentSelection with spph.component to see if it is valid for the distroseries
<wallyworld_> if valid, use that, if not valid, what do i do then?
<wallyworld_> just use universe?
<wgrant> Well
<wgrant> I'd just always use SPPH.component, probably
<wgrant> The SPPH is in the relevant distroseries
<wgrant> If it's not, crashing is OK
<wallyworld_> ok, and if there isn't one?
<wallyworld_> raise an exception
<wgrant> That's a more interesting question
<wgrant> We already have an issue today with builds where there is no current source publication
<wgrant> Usually because the source was supereseded/deleted before the build finished
<wgrant> eg. http://launchpadlibrarian.net/120668537/aYDS2Qq6K4jADFvNDYo0RyGBld1.txt
<wgrant> And I think there's a bug for that
<wallyworld_> ok, that can be for some other branch to fix i guess
<wallyworld_> so above you said "sort of" when i asked about ensuring both places were consistent - copy and upload
<wallyworld_> if i change upload, then copy has to change too i would think
<wgrant> So, I'd use SPPH.component, assume it's valid, use the current default mechanism if it's not set
<wgrant> You need to preserve the current default mechanism for sources, anyway
<wgrant> (processUnknownFile is used for both sources and binaries)
<wallyworld_> yes, i wasn't going to touch sources
<wgrant> I have to hope that StevenK remembers how missing copy overrides work
<wallyworld_> i guess target_build = None for sources?
<wgrant> It's relatively modern and I have repressed that memory
<wgrant> Um
<wgrant> That's one way
<wgrant> You'll need to redesign the way processUnknownFile is called
<wallyworld_> missing copy overrides seems to use the archive component or defaults to "universe"
<wgrant> So it can be given some context
<wallyworld_> ok, i haven't looked at the code fully yet
<wallyworld_> so, is using the archive component correct then for mising copies?
<wgrant> Oh, as in archive.default_component?
<wallyworld_> yes
<wgrant> If it's set, sure
<wgrant> It overrides the traditional default logic
<wallyworld_> the code defaults to universe if it is not set
<wgrant> But only for partner/PPA
<wgrant> Well
<wgrant> IIRC it defaults to universe unless [SB]PR.component is contrib or non-free, in which case it defaults to multiverse
<wgrant> Right?
<wallyworld_> that's force sources i think
<wallyworld_> for
<wallyworld_> not binaries
<wgrant> Today it's meant to go: ancestry.component, archive.default_component, [SB]PR.component -> universe/multiverse
<wgrant> The bugfix will be: ancestry.component, (if binary) build.current_source_publication.component, archive.default_component, [SB]PR.component -> universe/multiverse
<wgrant> I think
<wallyworld_> i think it looks in spph first, and then uses a default mapping for unknown
<wgrant> Right, it will always look up the [SB]PPH ancestry first
<wallyworld_> default mapping is contrib/non-free -> multiverse
<wallyworld_> everything else -> universe
<wgrant> Right, that's the last resort
<wallyworld_> above, build.current_source_publication.component = spph.component, right?
<wgrant> current_source_publication is an SPPH, right
<wallyworld_> ok, i'll give it a go. i might delay doing the extra query when copying binaries so that only unknown ones are processed that way
<wallyworld_> rather than doing the extra joins up front
<wgrant> Yeah, definitely.
<wallyworld_> i wonder how many copied binaries are unknown
<wgrant> It happens most often with kernel SRUs and security updates
<wgrant> But it's relatively very rare
<wallyworld_> ok. thanks for all the help
<wgrant> The plan was to rewrite find_and_apply_overrides to basically use lp.soyuz.adapters.overrides for everything, but it never got done
<wallyworld_> i haven't come across find_and_apply_overrides yet
<wallyworld_> ah, it's in NascentUpload
<wgrant> Right
<wgrant> It's what calls processUnknownFile
<wgrant> It's the original implementation of the override logic
<wgrant> Copies didn't respect overrides until 18 months ago
<wallyworld_> so we have 2 implementations, do they do the same thing?
<wallyworld_> ie give the same result?
<wgrant> Mostly except sort of not
<wgrant> The idea was that lp.soyuz.adapters.overrides would be the One True Implementation
<wgrant> But the old one was never destroyed, in true LP style :(
<wallyworld_> so maybe i should at least make upload and copy do the same thing then
<wallyworld_> and delete the old code
<wgrant> That sounds slightly too adventurous
<wgrant> For a first adventure into Soyuz
<wgrant> Although it would be a good tour, I guess
<wallyworld_> isn't it just the find_and_apply_overrides method to be replaced?
<wallyworld_> with calls to the adaptor
<wgrant> mostly
<wgrant> It's unlikely to be as smooth as we hope
<wgrant> As we don't have publications here
<wallyworld_> is there adequate test coverage?
<wgrant> Not adequate enough that I'd deploy it without extensive manual QA, but probably enough for development.
<wallyworld_> ok, i'll have a play around and see what falls out. but if i do it, i'll have to beg for help with qa
<wgrant> With the history of this stuff, I would be QAing it personally regardless of who wrote it and had already QAed it, don't worry
<wgrant> The archiveuploader tests are some of the most archaic we have
 * StevenK can recall that about his overrides policy code
<wallyworld_> ok, thanks. it may be that i chicken out
<wallyworld_> but i'd like to get rid of inconsistent, "duplicate" code
<wallyworld_> where possible
<wgrant> Definitely
<wgrant> This is an example that really needs it, but it's not the easiest way to start off in Soyuz.
<wgrant> So we'll see
<wgrant> StevenK and I probably know both sides of this code reasonably well
<wgrant> So can unstick you as required
<wallyworld_> ok, thanks
<wallyworld_> it's hard because i have no domain knowledge and don't know where to find data to help me understand the code :-)
<wallyworld_> but, we'll see
<StevenK> wallyworld_: I think your first step is to print out http://behlerblog.files.wordpress.com/2012/06/frustrated-kit.jpg
<wallyworld_> hah, i did that when i first used zope
<StevenK> Tell me about it, so did I.
<StevenK> *And* I had to learn Soyuz at the same time
<wallyworld_> a good debugger is absolutely mandatory
<wgrant> But you had Soyuz experience!
<StevenK> wgrant: Sure, with using it!
<nigelb> lol
<StevenK> Tells me absolutely nothing about how the code hangs together.
<nigelb> That reminds me of my first lp patch.
<wallyworld_> StevenK: so, same with women
<nigelb> I was using grep to find things and I needed like 5 windows for grepping different things.
<wallyworld_> nigelb: you need an IDE with code navigation features
<wallyworld_> much easier
<nigelb> I'm guessing you use pycharm?
<ajmitch> nigelb: you obviously were changing a simple part of LP then
<wallyworld_> yup
<nigelb> ajmitch: Heh, yes.
<nigelb> I need to get back to LP dev. Been lazy.
<wallyworld_> s/lazy/smart
<nigelb> ^ I was trying to be diplomatic
<wallyworld_> i was joking :-)
<wallyworld_> lp is nice once you get to know and love it
<nigelb> s/love/not want to kill yourself/g
<nigelb> *kill yourself because of
<wallyworld_> it's just like any lrge software system with lots of legacy
<nigelb> That's true.
<ajmitch> wallyworld_: stockholm syndrome is real
<nigelb> ajmitch++
<wallyworld_> lol
<nigelb> hahah
<wallyworld_> sadly i have to pause the fun soyuz stuff to deal with some ec2 hate mail
<nigelb> ajmitch: I'll admit, fixing LP bugs was among the most satisfying bugfixes I've done.
<nigelb> When I did the mouseover for bug titles thing, a lot of people pinged me say they love it ;)
<nigelb> (granted it took like 3 attempts at landing to actually land)
<wallyworld_> wgrant: have you seen ec2 failure with "ClosedError: Connection is closed" but the tests pass locally?
<wgrant> wallyworld_: In what sort of test?
<wallyworld_> a couple of doc tests
<wgrant> Hm
<wallyworld_> eg xx-bug-also-affects.txt
<wgrant> If they're the only failures and not very relevant to your changes, just lp-land I guess
<wgrant> Hm
<wgrant> This is for the +distrotask change?
<wallyworld_> yes
<wallyworld_> could be relevant
<wallyworld_> but i would expect a local failure also
<wgrant> Yeah
<wgrant> Perhaps ec2 again to be safe
<wallyworld_> yeah :-(
<wgrant> Won't be a significant delay, as we are unlikely to deploy again today
<wallyworld_> well, here's hoping for 2nd time lucky
<nigelb> lifeless: Did you see that sentry has a js logger thing?
<nigelb> Hehe, I was going to setup sentry for python logging when I noticed it :)
<lifeless> nigelb: yah, did you see the branch for oops that adds that ?
<lifeless> imbrandon was working on a revised version
<nigelb> lifeless: hah. no.
<nigelb> awesome.
<lifeless> nigelb: https://code.launchpad.net/js-oopsd
<nigelb> \o/
<nigelb> Nice.
<deryck> StevenK, are you around?
<adeuring> deryck: https://code.launchpad.net/~adeuring/launchpad/bug-1067736/+merge/131527
<deryck> adeuring, got it.
<rick_h_> deryck: https://bugs.launchpad.net/launchpad/+bug/1068817
<_mup_> Bug #1068817: Person.assigned_specs is an attractive nuisance <private-projects> <specifications> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1068817 >
<StevenK> deryck: I am now, but not really as it were.
<deryck> StevenK, right.  I replied on abentley's MP.  We just need to get that landed, and I assumed you were done with it.
<StevenK> deryck: Link me, sorry?
<deryck> StevenK, https://code.launchpad.net/~abentley/launchpad/person-assigned-specs-in-progress/+merge/130888
<deryck> StevenK, last comment from me.
<StevenK> deryck: Well, obviously :-)
<deryck> Just trying to be clear :)
<StevenK> deryck: I am concerned about the NULL thing, since it may leak data or provide results that are not wanted, so then please do land that branch, but also please investigate to make sure that the NULL doesn't sneak in for the actual code.
<deryck> StevenK, there is no NULL in the code.  that was a manual substitution we did to test the query.
<deryck> StevenK, the actuall code-generated query uses two values from the enum.
<StevenK> deryck: Ah. What I do to test SQL like that is to run a single test with LP_DEBUG_SQL=1 to grab the actual SQL, change the id's for say person or what not and then run it
<abentley> StevenK: We were trying to be clever, and failing.
<deryck> StevenK, cool, thanks for the tip.
<deryck> StevenK, so we're cool then?
<StevenK> deryck: Yes. I'm happy to approve it and you guys can land it if you wish?
<deryck> StevenK, thanks!
<StevenK> deryck, abentley: Done.
<abentley> rick_h_: Could you please review this? https://code.launchpad.net/~abentley/launchpad/milestone-spec-privacy/+merge/131560
<adeuring> deryck: https://code.launchpad.net/~adeuring/launchpad/filter-private-product-RosettaApplication/+merge/131613
#launchpad-dev 2012-10-27
<wgrant> lifeless: Those translations timeouts are quite impressive
<wgrant> lifeless: 280 repetitions of various stats queries
<wgrant> Not a clever page :/
<czajkowski> wgrant: morning
<wgrant> Morning czajkowski
#launchpad-dev 2012-10-28
<lifeless> wgrant: nice
#launchpad-dev 2013-10-21
<ultimex> hi all, I send a bug for launchpad recipes: https://bugs.launchpad.net/launchpad/+bug/1242796 Someone can tell me if this bug can be fix? (contrary I have to find a bypass soon)
<_mup_> Bug #1242796: Fix destdir for recipe <Launchpad itself:New> <https://launchpad.net/bugs/1242796>
#launchpad-dev 2013-10-22
<StevenK> wgrant: So, something about transistionToStatus. I respect your current reasoning, but do you still have an issue if createManyTasks forgoes the permission checks and calls the setting logic?
<wgrant> StevenK: That might be more reasonable.
<StevenK> I'm not sure I can hack create() to set them based on the status
<StevenK> wgrant: http://pastebin.ubuntu.com/6280794/
<wgrant> StevenK: Ew.
<wgrant> StevenK: Can you factor that out?
<wgrant> Surely transitionToStatus can use that.
<StevenK> wgrant: transitionToStatus works on the changes from old status to new status
<wgrant> StevenK: Right, and what's the problem?
<StevenK> So transitionToStatus will only set date_triaged if old_status < TRIAGED and new_status >= TRIAGED
<wgrant> Sure
<wgrant> That logic works for createMultipleTasks too
<wgrant> Because old_status is NEW
<StevenK> wgrant: createManyTasks :-P and http://pastebin.ubuntu.com/6280868/
<wgrant> That's no method name
<StevenK> _set_date_properties ?
<wgrant> StevenK: Methods are mixedCase
<StevenK> wgrant: http://pastebin.ubuntu.com/6280909/
<wgrant> StevenK: That looks relatively sane.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/use-transitionToStatus-createManyTasks/+merge/191554 has updaed
<StevenK> *updated
<wgrant> r=me
<wgrant> Do QA lots of cases, though
<StevenK> Yeah, I'll go through NEW, INCOMPLETE, TRIAGED, CONFIRMED, FIX*
<wgrant> StevenK: And creating new tasks directly or by release-targeting, and check the conjoined master and slave cases.
 * StevenK stabs swift
<lifeless> StevenK: how is swift-for-librarian working out ?
<StevenK> lifeless: It is still waiting for review.
<lifeless> StevenK: by you and wgrant ?
<wgrant> Yes, I need to bring myself to look at it at some point.
<StevenK> lifeless: We have a subunit stream that imports in 0.0.4, and spins for ~30 seconds in 0.0.17 and gives no failing tests.
<lifeless> StevenK: v1 vs v2 probably; does | subunit-1to2 | <whatever processor you're using> do better?
<StevenK> lifeless: Ah ha!
<StevenK> lifeless: It can't tell the difference between v1 and v2 streams?
<lifeless> StevenK: not automagically
<StevenK> wgrant: The test_distroseries_context_with_no_series_task
<StevenK> Bleh
<wgrant> I agree.
<StevenK> wgrant: The test_distroseries_context_with_no_series_task failure is strange -- none of those tests are invalidating caches
<wgrant> It could be a test ordering issue.
<StevenK> My fingers are itching to refactor that file to a shadow of its former self.
<lifeless> StevenK: wgrant: so, having a review getting stale is just a bad idea; surely 'go live' is a separate problem to 'the code is good enough' ?
<harishnavnit> hello all
<harishnavnit> new to this place , need some help with bug fixes for launchpad
<harishnavnit> can anyone help ??
<harishnavnit> ping
<wgrant> harishnavnit: Hi
<harishnavnit> wgrant: hello
<harishnavnit> @wgrant : can you help me contributing to launchpad ?
<wgrant> harishnavnit: Have you found a bug that you want to fix?
<harishnavnit> yes
<wgrant> Which one?
<harishnavnit> #128642
<_mup_> Bug #128642: searching for a bug using #59348 fails <lp-bugs> <search> <trivial> <ui> <Launchpad itself:Triaged by harishnavnit> <https://launchpad.net/bugs/128642>
<harishnavnit> yes
<harishnavnit> what next ?
<wgrant> harishnavnit: How far have you gotten?
<harishnavnit> not very far . i'm yet to start coding
<harishnavnit> i have the build
<harishnavnit> lp-branches
<wgrant> Have you managed to get Launchpad running?
<wgrant> The code you probably want to alter is in BugTaskSearchListingView.buildSearchParams.
<harishnavnit> well this is my first attempt for a bug fix
<harishnavnit> can you elaborate please ?
<harishnavnit> i haven't managed to get launchpad running
<wgrant> https://dev.launchpad.net/Running describes how to get a local Launchpad instance running.
<harishnavnit> yea , i had gone through it
<harishnavnit> what is postgreSQL by the way ?
<wgrant> http://www.postgresql.org/about/
<harishnavnit> thank you
<cjwatson> wgrant: The Nagios change that was blocking https://code.launchpad.net/~cjwatson/launchpad-buildd/remove-old-status/+merge/191809 should be in now
<wgrant> cjwatson: Thanks, approved.
#launchpad-dev 2013-10-23
<StevenK> Bleh, I do not get why setting date_* on creation means two tests fail
<StevenK> wgrant: So the issue is the validator for the date_* properties
<StevenK> If they don't call validate_conjoined_attribute, the tests don't fail
<StevenK> wgrant: Is the @block_implicit_flushes decorator to blame?
<wgrant> StevenK: it could be.
<StevenK> Blah, using PassthroughValue(bugtask.datecreated) doesn't cause the tests to pass
<StevenK> Well, it does if I set the attributes directly, but not if I call _setStatusDateProperties
<StevenK> wgrant: http://pastebin.ubuntu.com/6286668/
<wgrant> StevenK: I guess that works.
<wgrant> Does it works?
<StevenK> wgrant: The test I added in the previous branch and all of the failures from buildbot pass.
<wgrant> Sounds sane
<StevenK> wgrant: Hmm, Also affects project doesn't allow you to set the status on creation
<wgrant> StevenK: Correct.
<StevenK> wgrant: But then I can't check that the properties are set correctly for createManyTasks?
<wgrant> StevenK: Probably only for filing a new one, correct.
<wgrant> But even that uses createManyTasks
<wgrant> s/one/bug/
<StevenK> Well, date_triaged works, at least.
<StevenK> wgrant: Hm, so postgres will expand * to every column, but not date* ?
<StevenK> Or is * actually special and not expanded?
<wgrant> StevenK: * is special.
<StevenK> wgrant: http://pastebin.ubuntu.com/6287016/ pleases you?
<wgrant> StevenK: And you've tested that all other ways to create tasks don't crash?
<StevenK> wgrant: Also affects product does not crash
<StevenK> I'm trying to recall others
<wgrant> Targetting, distro tasks
<StevenK> wgrant: Targetting 1206155 to Staging works fine
<StevenK> And added an eglibc task onto 1206154 fine
<StevenK> wgrant: I can't file a new bug that will have conjoined tasks on it?
<wgrant> StevenK: Not unless you can file it directly on a distroseries or productseries.
<StevenK> wgrant: Doesn't look like it
<StevenK> wgrant: Looking at bug 1234497, I've created a product and a series, created a proprietary bug on the product, created another bug targetted to the series, and then removed the product task. I can then set the product to proprietary, but it should fail, right?
<_mup_> Bug #1234497: Project series bug tasks on proprietary project can target non-proprietary bugs <oem-services> <oops> <privacy> <private-projects> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1234497>
<wgrant> StevenK: Right.
<wgrant> The check only looks directly for product tasks.
<StevenK> Right, and milestones are always going to be targetted at a series, so all bugs for the product and its series should be fine.
<StevenK> milestoned bugs, that is
<wgrant> A milestone isn't a bug target.
<harishnavnit> hi
<harishnavnit> i'm facing an error when facing an error when i run "make schema" in the devel directory
<harishnavnit> here's the output log http://pastebin.ubuntu.com/6289662/
<harishnavnit> and when i run the utilities/link-external-sourcecode , i get the error stating "Parent branch not specified and hence could not be discovered."
<harishnavnit> can anyone please help me resolving this ? Thanks
#launchpad-dev 2013-10-24
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/check-series-bug-changeitype/+merge/192449
<wgrant> chan gei type? chang eit ype?
<wgrant> No parsing sounds dreadfully plausible.
<wgrant> StevenK: I'd consider using bugtaskflat for that, but I guess other parts of the code will time out first on a large project.
<StevenK> wgrant: I'm happy to switch it to BTF if you wish
<wgrant> nah
<StevenK> It's like a three line change and involves no longer joining Bug and BugTask
<wgrant> I guess we already have other internal consistency checks based on the throwaway cache table
<wgrant> So OK
<StevenK> Hah, it ends up as +6/-6
<StevenK> wgrant: http://pastebin.ubuntu.com/6292383/
<wgrant> Right
<StevenK> I might switch that information_type for IS IN (PUBLIC), rather than IS NOT IN (PROPRIETARY)
<wgrant> That would be incorrect.
<wgrant> Private Security and Private are forbidden for proprietary projects
<StevenK> It would?
<wgrant> But there may be a non-proprietary list somewhere, I can't recall.
<StevenK> We're looking for public bugs, why would IS IN PUBLIC_INFORMATION_TYPES be wrong?
<StevenK> Oh
<StevenK> I get it
<StevenK> Right
<wgrant> We're looking for non-proprietary bugs
<wgrant> Not just public ones.
<StevenK> Right
<StevenK> FREE_INFORMATION_TYPES = (
<StevenK>     PUBLIC_INFORMATION_TYPES + FREE_PRIVATE_INFORMATION_TYPES)
<wgrant> That's the one.
<StevenK> So it looks like IS IN FREE_INFORMATION_TYPES would be fine too
<wgrant> That would be better, yes.
<wgrant> Indexable and stuff.
<StevenK> Right, hence why I wanted IS IN, rather than the inverse :-)
<StevenK> wgrant: I know you've already approved it, can you cast your eyes over it again given the BTF and FREE changes?
<StevenK> wgrant: And it's 'change itype'
<StevenK> check-series-bug-change-information-type seemed too long
<wgrant> StevenK: Looks good
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/refactor-test-bugtaskfilter/+merge/192456
#launchpad-dev 2013-10-25
<harishnavnit> hi
<harishnavnit> what is the proper way of running the utilities/link-external-sourcecode ?
<harishnavnit> I get an error stating that "Parent branch not specified and could not be discovered " , when i try running the utilities/link-external-soureceode
<cjwatson> harishnavnit: link-external-sourcecode is for when you've made a branch of a properly-set-up devel checkout, which doesn't sound like your case.
<cjwatson> harishnavnit: It doesn't sound like you used rocketfuel-setup (which you should run inside a container, see https://dev.launchpad.net/Running/LXC
<cjwatson> )
<cjwatson> Given that you've got the branch already, you should be able to run utilities/rocketfuel-setup (INSIDE A CONTAINER as it will do interesting things to your system)
<cjwatson> If you've run that already and it just went wrong somehow, do this instead:
<cjwatson> bzr branch lp:~launchpad/lp-source-dependencies/trunk/ download-cache
<cjwatson> utilities/update-sourcecode
<harishnavnit> i'm prompted to run /utilities/link-external-sourcecode when i run make schema
<cjwatson> ... actually ignore that, just use rocketfuel-setup :)
<cjwatson> Yes I know you are but that's assuming that you have a properly set up parent branch, which you don't
<cjwatson> Did you just fetch the code with bzr branch lp:launchpad or similar?
<harishnavnit> yes , i used bzr
<harishnavnit> how do u set up the parent branch ?
<cjwatson> Right, so set up a container and run utilities/rocketfuel-setup inside it.
<cjwatson> Then when you make YOUR OWN BRANCHES from this branch you'll be able to use link-external-sourcecode in those.
<cjwatson> But that doesn't apply to the first one.
<harishnavnit> uhm , what do u mean by set up a container ?
<cjwatson> I gave you a link above.
<cjwatson> https://dev.launchpad.net/Running/LXC
<harishnavnit> thanks , will go through it
<cjwatson> You'll need to do that anyway otherwise things like running tests will be a pain later.
<cjwatson> So you might as well do it now.
<harishnavnit> yeah
<harishnavnit> well i;m currently running updates , so sudo apt-get install lxc wont work right now
<harishnavnit> unable to unlock the administration directory
<cjwatson> Sure, just wait until you're finished with that.
<harishnavnit> it's a long wait :(
#launchpad-dev 2013-10-27
<harishnavnit> i need help in working with lxc's
<harishnavnit> https://dev.launchpad.net/Running/LXC
<harishnavnit> this is what i'm referring to
<harishnavnit> here's another question that I had , in the following command
<harishnavnit> $ ./utilities/launchpad-database-setup $USER
<harishnavnit> what is the $USER used for ?
<harishnavnit> is it my username ? or do I type in as it is , i.e $USER
<harishnavnit> is it necessary to run the rocketfuel setup inside an LXC ? that is something that I didn't
<harishnavnit> now i get an error when  i run "make schema"
<cjwatson> harishnavnit: It's an environment variable; you can type it as it is, or substitute your username
<cjwatson> harishnavnit: I *specifically* warned you that you must run rocketfuel-setup inside a container or it will modify your host system in ways you probably don't want.  Now you probably have to read the script and figure out how to undo it.
<harishnavnit> wouldn't it work if i simply removed all the lp directories
<harishnavnit> ?
<cjwatson> harishnavnit: No, it makes various changes under /etc/, fiddles about with your Apache configuration, may wipe any PostgreSQL databases you may have had ...
<harishnavnit> thankfully i didn't have any PostgreSQL databases
<cjwatson> But the other changes are still relevant
<cjwatson> Do this in a container.  Seriously.  I warned you the first time ...
<harishnavnit> so i'll now have to go through the entire shell script to remove it ?
<harishnavnit> :(
<cjwatson> Probably.
<cjwatson> 22:04 <cjwatson> Given that you've got the branch already, you should be able to run utilities/rocketfuel-setup (INSIDE A CONTAINER as it will do interesting things to your system)
<cjwatson> I'm not sure how that could have been much clearer
<harishnavnit> yea , that was clear alright . I'd run the rocketfuel setup days earlier
<harishnavnit> after I did the formalities with bzr
<cjwatson> Ah
<harishnavnit> i installed lxc and i'm unable to login aswell
<harishnavnit> what's wrong ?
<cjwatson> You might want to be more specific about exactly how it goes wrong
<harishnavnit> i'm sorry , what happens is that , I was given the default user and password as 'ubuntu'
<harishnavnit> but when i enter the details , it shows incorrect password
<cjwatson> That sounds like you forgot to use -b $USER when creating the container
<cjwatson> Or else the message was buggy
<cjwatson> Try logging in with your normal username and password
<harishnavnit> how do i close a running container ?
<cjwatson> sudo lxc-stop -n lpdev
<cjwatson> Although why?
<cjwatson> You don't need to stop it just to try logging in with a different user
<harishnavnit> bcoz , i couldn't get out of the lpdev container login prompt
<harishnavnit> so closed the terminal
<cjwatson> Ctrl-a q   perhaps
<harishnavnit> yea ,i'll try
<harishnavnit> well it's taking a long while to stop the running lxc lpdev
<cjwatson> You can force it with   sudo lxc-stop -k -n lpdev
<cjwatson> I suggest having a look through the documentation before asking - a lot of stuff is answered there
<cjwatson> (e.g. man lxc-stop)
<harishnavnit> okay
<harishnavnit> okay , leaving for now , i'll get back later
<harishnavnit> thanks a lot
<cjwatson> np
 * cjwatson pushes up his work from the plane
<cjwatson> I had a reasonably productive flight :)
<lifeless> cjwatson: all hail DVCS ?
<cjwatson> lifeless: Indeed!
#launchpad-dev 2014-10-23
<bigjools> have you changed something lately around transactions or concurrency? Over the last few days I've seen bug posts and MP comments appearing multiple times (and in on email)
#launchpad-dev 2015-10-19
<stub> cjwatson: There is also that 'fades' thing  Facundo put together that builds the virtualenv on demand
<replaceafill> hi wgrant, are you around?
<wgrant> rpadovani: I am.
<wgrant> Er
<wgrant> replaceafill: ^^
<replaceafill> :D
<replaceafill> may i take a few minutes of your time with a lazr.restful + lazr.batchnavigation question? :)
<replaceafill> i'm not sure if this is the right place to ask
<replaceafill> but i've seen lazr discussed here i think...
<lifeless> it was built for lp
<lifeless> so seems fine to me
<replaceafill> lifeless, thanks :)
<replaceafill> i'm using it with a SchoolTool experiment
<replaceafill> and i have a lazr.restful implementation running through paste
<replaceafill> i just noticed one thing in the URLs generated in my queries
<replaceafill> specifically the next_collection_link and prev_collection_link
<replaceafill> they return URLs like: "next_collection_link": "http://localhost:7080/1.0/persons/...
<replaceafill> when i get resource_type_link's and self_link's like:
<replaceafill> "self_link": "http://localhost:7080/api/1.0/persons/student051"
<replaceafill> which are correct, notice the /api/ part
<replaceafill> that's from my paste configuration
 * replaceafill is not sure he's clear enough explaining this... :(
<replaceafill> anyway i tracked the url generation in both cases and noticed that lazr.restful generates the urls using absolutURL calls,which seems fine
<replaceafill> but lazr.batchnavigator doesn't
<lifeless> sounds like a bug somewhere ;)
<lifeless> I don't recall the nuance of that code - its been ~4 years since I hacked on it
<replaceafill> lifeless, i could be it's here: http://bazaar.launchpad.net/~lazr-developers/lazr.batchnavigator/trunk/view/head:/src/lazr/batchnavigator/_batchnavigator.py#L272
<lifeless> wgrant: may have more recent memories
<replaceafill> :)
<replaceafill> i guess i just wanted to confirm it's a bug and i should just not trust those batch links
<lifeless> it may be that batchnavigation thinks its returning html urls not api urls
<replaceafill> i wonder if batchnavigation could use this same approach used in lazr.restful: http://bazaar.launchpad.net/~lazr-developers/lazr.restful/trunk/view/head:/src/lazr/restful/_resource.py#L2070
<wgrant> Whoops, sorry, got called away.
<wgrant> replaceafill: Hmm, so batchnavigator is meant to use the same request as lazr.restful.
<wgrant> replaceafill: But it's possible you have to instantiate it directly with the right request, let me see.
<wgrant> Hmm.
<wgrant> replaceafill: Have you worked out what the difference between the URL generation calls is between lazr.restful itself and lazr.batchnavigator?
<wgrant> I can't see why batchnav would be getting the wrong request.
<wgrant> Unless you have a property that directly returns one?
<wgrant> replaceafill: So the only way I can see that happening is if your code creates a BatchNavigator directly.
<wgrant> If you simply declare an attribute as a collection and let lazr.restful batch it for you, it should Just Work.
<cjwatson> https://code.launchpad.net/~cjwatson/turnip/virtualenv/+merge/274701 should be less flaky now
#launchpad-dev 2015-10-21
<mortenoh> anyone awake?
#launchpad-dev 2016-10-24
<kb9vqf> wgrant: After rebooting our production Launchpad instance to apply dirty COW fixes Launchpad won't start
<kb9vqf> stopping services on exception Exception('Timeout waiting for txlongpoll to start.',)
<kb9vqf> never seen this before, any ideas?
<kb9vqf> I forced the production instance back online by disabling txlongpoll entirely in the startup script.  From what I understand, it's some kind of Internet download interface so _maybe_ we won't miss it?  Still would like to know why it failed after a simple reboot
<wgrant> kb9vqf: I'm not sure why it would fail to start, but it's not used by any production features so it's fine to turn off.
<kb9vqf> wgrant: Great, thanks!  Had a bit of a panic moment when it wouldn't start and a lead developer on one of the projects needed to get release builds uploaded....
<kb9vqf> I think there are a couple other logs on the Internet showing the same failure but in those cases retrying it fixed the problem.  This was quite persistent across multiple machine reboots, etc.
<kb9vqf> the process itself was running, but the diagnostics were useless\
#launchpad-dev 2016-10-27
<Terry_> Anybody can help?When I running "make run_all", it output "Error: unable to connect to node tmpoPism1@localhost: nodedown", how to fix it?
<xnox> When create a project series; or when "configuring code" for a series (project/series/+setbranch) there is no option to use git, only bzr.
<wgrant> xnox: Series repositories don't make sense for git, so the option isn't provided. But we haven't worked out what to do instead.
<wgrant> (series branches are the mechanism by which lp:foo is defined for bzr, but for git it is done directly)
#launchpad-dev 2016-10-28
<xnox> wgrant, right... use git branches. ok.
#launchpad-dev 2017-10-23
<xnox> i wonder if the postgres timers/jobs that block me from commenting kick in at the worst possible time every monday morning =) instead of like running sometime between saturday & sunday.
 * xnox goes to grab a coffee
<cjwatson> xnox: empirically they seem to be closer to once a day than once a week
